Convai takes AI-driven NPCs to next level with Nvidia Avatar Cloud Engine

08.01.2024 - 17:20 / venturebeat.com / Ai

Do you want to get the latest gaming industry news straight to your inbox? Sign up for our daily and weekly newsletters here .

Convai and Nvidia are planning to take AI-driven non-player characters (NPCs) to the next level with more generative AI advances.

They already created an amazing demo of the tech with a realistic ramen shop and its characters Jin and Kai. Now Convai is showing off the next evolution in AI-driven character technology, which gives characters the ability to hold realistic conversations with players in games.

Showcasing more advances in conversational, perceptual, and action generation capabilities, Convai unveiled a leap in immersive gaming experiences.

The latest from Convai empowers users to engage NPCs dynamically, seeking real-time responses and actions. Characters now execute complex tasks, fetch items, and initiate interactions with other NPCs. Even during idle moments in the game, Convai-enabled NPCs engage with each other, expanding the spectrum of their interactions and perceptions within the virtual world.

Designed to integrate seamlessly with game engines like Unreal Engine and Unity, Convai’s latest iteration redefines the behavior and responsiveness of AI-driven characters within interactive environments.

These NPCs now possess enhanced spatial awareness, dynamic character actions, and the ability to engage in meaningful interactions with their environment and other characters. Beyond scripted responses, Convai-powered characters navigate complex instructions, showcase emotional awareness, and participate in organic interactions. It has the potential to reshape the landscape of gaming narratives.

In a strategic partnership, Convai integrated Nvidia Avatar Cloud Engine (ACE) modules, specifically Nvidia Audio2Face and Riva. This integration enhances the realism, believability, and responsiveness of character interactions.

Nvidia ACE is a suite of technologies leveraging generative AI to bring digital avatars to life. Audio2Face facilitates industry-leading lip sync and facial animations, while Riva delivers high-quality, low-latency speech-to-text capabilities.

Continuing their collaboration, Nvidia and Convai showcased the evolution of the Kairos demo, initially unveiled at Computex 2023, now augmented with Convai’s latest advancements and ACE technology, exemplified in a live demo during Nvidia’s CES 2024 online session.

“Generative AI-powered characters in virtual worlds unlock various use cases and experiences that were previously impossible,” said Purnendu Mukherjee, founder at Convai, in a statement. “At CES, we are demonstrating the evolution of Convai’s AI NPC technology, presenting digital characters capable of emotionally-driven conversations, spatial awareness,