NVIDIA announced that leading AI application developers across a wide range of industries are using NVIDIA digital human technologies to create lifelike avatars for commercial applications and dynamic game characters. The results are on display at GTC, the global AI conference held this week in San Jose, California, and can be seen in technology demonstrations from Hippocratic AI, Inworld AI, UneeQ and more.
NVIDIA Avatar Cloud Engine (ACE) for speech and animation, NVIDIA NeMo for language, and NVIDIA RTX for ray-traced rendering are the building blocks that enable developers to create digital humans capable of AI-powered natural language interactions, making conversations more realistic and engaging.
“NVIDIA offers developers a world-class set of AI-powered technologies for digital human creation,” said John Spitzer, vice president of developer and performance technologies at NVIDIA. “These technologies may power the complex animations and conversational speech required to make digital interactions feel real.”
World-Class Digital Human Technologies
The digital human technologies suite includes language, speech, animation and graphics powered by AI:
- NVIDIA ACE — technologies that help developers bring digital humans to life with facial animation powered by NVIDIA Audio2Face and speech powered by NVIDIA Riva automatic speech recognition (ASR) and text-to-speech (TTS). ACE microservices are flexible in allowing models to run across cloud and PC depending on the local GPU capabilities to help ensure the user receives the best experience.
- NVIDIA NeMo — an end-to-end platform that enables developers to deliver enterprise-ready generative AI models with precise data curation, cutting-edge customisation, retrieval-augmented generation and accelerated performance.
- NVIDIA RTX — a collection of rendering technologies, such as RTX Global Illumination (RTXGI) and DLSS 3.5, that enable real-time path tracing in games and applications.
Building Blocks for Digital Humans and Virtual Assistants
To showcase the new capabilities of its digital human technologies, NVIDIA worked across industries with leading developers, such as Hippocratic AI, Inworld AI and UneeQ, on a series of new demonstrations.
Hippocratic AI has created a safety-focused, LLM-powered, task-specific Healthcare Agent. The agent calls patients on the phone, follows up on care coordination tasks, delivers preoperative instructions, performs post-discharge management and much more. For GTC, NVIDIA collaborated with Hippocratic AI to extend its solution to use NVIDIA ACE microservices, NVIDIA Audio2Face along with NVIDIA Animation graph and NVIDIA Omniverse Streamer Client to show the potential of a generative AI healthcare agent avatar.
“Our digital assistants provide helpful, timely and accurate information to patients worldwide,” said Munjal Shah, cofounder and CEO of Hippocratic AI. “NVIDIA ACE technologies bring them to life with cutting-edge visuals and realistic animations that help better connect to patients.”
UneeQ is an autonomous digital human platform specialised in creating AI-powered avatars for customer service and interactive applications. Its digital humans represent brands online, communicating with customers in real time to give them confidence in their purchases. UneeQ integrated the NVIDIA Audio2Face microservice into its platform and combined it with Synanim ML to create highly realistic avatars for a better customer experience and engagement.
“UneeQ combines NVIDIA animation AI with our own Synanim ML synthetic animation technology to deliver real-time digital human interactions that are emotionally responsive and deliver dynamic experiences powered by conversational AI,” said Danny Tomsett, founder and CEO of UneeQ.
Bringing Dynamic Non-Playable Characters to Games
NVIDIA ACE is a suite of technologies designed to bring game characters to life. Covert Protocol is a new technology demonstration, created by Inworld AI in partnership with NVIDIA, that pushes the boundary of what character interactions in games can be. Inworld’s AI engine has integrated NVIDIA Riva for accurate speech-to-text and NVIDIA Audio2Face to deliver lifelike facial performances.
Inworld’s AI engine takes a multimodal approach to the performance of non-playable characters (NPCs), bringing together cognition, perception and behavior systems for an immersive narrative with stunning RTX-rendered characters set in a beautifully crafted environment.
“The combination of NVIDIA ACE microservices and the Inworld Engine enables developers to create digital characters that can drive dynamic narratives, opening new possibilities for how gamers can decipher, deduce and play,” said Kylan Gibbs, CEO of Inworld AI.
Game publishers worldwide are evaluating how NVIDIA ACE can improve the gaming experience.
Developers Across Healthcare, Gaming, Financial Services, Media & Entertainment and Retail Embrace ACE
Top game and digital human developers are pioneering ways ACE and generative AI technologies can be used to transform interactions between players and NPCs in games and applications.
Developers and platforms embracing ACE include Convai, Cyber Agent, Data Monsters, Deloitte, Hippocratic AI, IGOODI, Inworld AI, Media.Monks, miHoYo, NetEase Games, Perfect World, Openstream, OurPalm, Quantiphi, Rakuten Securities, Slalom, SoftServe, Tencent, Top Health Tech, Ubisoft, UneeQ and Unions Avatars.
More information on NVIDIA ACE is available at https://developer.nvidia.com/ace. Platform developers can incorporate the full suite of digital human technologies or individual microservices into their product offerings.
Developers can start their journey on NVIDIA ACE by applying for the early access programme to get in-development AI models. To explore available models, developers can evaluate and access NVIDIA NIM, a set of easy-to-use microservices designed to accelerate the deployment of generative AI, for Riva and Audio2Face on ai.nvidia.com today.
Discussion about this post