In a significant leap for gaming technology, NVIDIA has launched a tool known as the Avatar Cloud Engine (ACE). This revolutionary technology allows gamers to engage in a lifelike conversation with non-playable characters (NPCs). Revealed at the generative AI keynote during Computex 2023, the tech represents a huge stride towards more immersive gaming experiences.
During a demonstration, NVIDIA showcased a scenario titled ‘Kairos’ which involved an interaction between a player-controlled character and an NPC named Jin, situated in a futuristic Ramen shop. The demonstration, which was the widest widescreen display seen yet, beautifully illustrated the potential of ACE.
In the scenario, the player initiates a conversation with Jin by asking, “Hey Jin, how are you?” To which Jin responds, “Unfortunately, not so good. I am worried about the crime around here. It’s gotten bad lately. My ramen shop got caught in the crossfire.” While the dialogue appeared a bit stiff, the real novelty here is the possibility of the NPC responding correctly in context to the player’s verbal cues, marking a significant shift from traditional NPC interactions.
The development of the demo was a collaborative effort between NVIDIA and Convai to display the capabilities of ACE. The software can operate both on local hardware and in cloud environments, provided that NVIDIA’s hardware is in use. The technology incorporates NVIDIA NeMo for constructing, tailoring, and deploying large language models, which can be further refined with lore and character backstories. NVIDIA also implemented guardrails to prevent any inappropriate conversations.
The new tool also makes use of Riva, a speech recognition and speech-to-text tool, as well as NVIDIA’s Omniverse Audio2Face, which provides on-the-spot creation of expressive facial animations for game characters that correspond to any speech input.
NVIDIA utilized Unreal Engine 5 to construct the demo, taking the opportunity to highlight their ray-tracing and other GPU features. While the AI dialogue needs some finetuning, the visual aspect of the demo was incredibly impressive, and the dialogue potential is evident. NVIDIA has yet to announce any games that will incorporate this new technology, but titles like Stalker 2: Heart of Chernobyl and Fort Solis have announced plans to use Omniverse Audio2Face.
