NVIDIA Releases Major Omniverse Upgrade with Generative AI and OpenUSD 

Aug. 9, 2023 -- NVIDIA has announced a major release of its NVIDIA Omniverse platform, offering new foundation applications and services for developers and industrial enterprises to optimize and enhance their 3D pipelines with the OpenUSD framework and generative AI.

Credit: Shutterstock

The update to Omniverse, an OpenUSD-native software platform for connecting, describing and simulating across 3D tools and applications, accelerates the creation of virtual worlds and advanced workflows for industrial digitalization. Cesium, Convai, Move AI, SideFX Houdini and Wonder Dynamics are now connected to Omniverse via OpenUSD.

Key highlights from the platform update include advancements to Omniverse Kit — the engine for developing native OpenUSD applications and extensions — as well as to the NVIDIA Omniverse Audio2Face foundation app and spatial-computing capabilities.

“Industrial enterprises are racing to digitalize their workflows, increasing the demand for OpenUSD-enabled, connected, interoperable, 3D software ecosystems,” said Rev Lebaredian, vice president of Omniverse and simulation technology at NVIDIA. “The latest Omniverse update lets developers tap generative AI through OpenUSD to enhance their tools, and it allows enterprises to build larger, more complex world-scale simulations as digital testing grounds for their industrial applications.”

Key Improvements

Updates to Omniverse Kit include:

• New modular app building: A new Omniverse Kit Extension Registry, a central repository for accessing, sharing and managing Omniverse extensions, lets developers easily turn functionality on and off in their application, making it easier than ever to build custom apps from over 600 core Omniverse extensions provided by NVIDIA.
• New developer templates and resources: New application and experience templates provide developers getting started with OpenUSD and Omniverse a major headstart with minimal coding.
• Boosted efficiency and user experience: New rendering optimizations take full advantage of the NVIDIA Ada Lovelace architecture enhancements in NVIDIA RTX GPUs with DLSS 3 technology fully integrated into the Omniverse RTX Renderer, and a new AI denoiser enables real-time 4K path tracing of massive industrial scenes.
• Native RTX-powered spatial integration: New extended-reality (XR) developer tools let users build spatial-computing options natively into their Omniverse-based applications, giving users the flexibility to experience their 3D projects and virtual worlds however they like.

These platform updates are showcased in Omniverse foundation applications, which are fully customizable reference applications that creators, enterprises and developers can copy, extend or enhance. Upgraded applications include:

• Omniverse USD Composer: Lets 3D users assemble large-scale, OpenUSD-based scenes.
• Omniverse Audio2Face: Provides access to generative AI application programming interfaces that create realistic facial animations and gestures from only an audio file, and now includes multilingual support and a new female base model.

OpenUSD Expands Omniverse Ecosystem

NVIDIA also announced a broad range of frameworks, resources and services for developers and companies to accelerate the adoption of Universal Scene Description, known as OpenUSD.

In addition, the company announced new Omniverse Cloud APIs, built by NVIDIA, for developers to more seamlessly implement and deploy OpenUSD pipelines and applications. For example, ChatUSD is a large language model copilot for developers that can answer USD knowledge questions or generate Python-USD code scripts.

New Omniverse connections enabled by OpenUSD are now available, opening more opportunities for industrial enterprises to break data silos in their complex production pipelines.

Expanding their collaboration across Adobe Substance 3D, generative AI and OpenUSD initiatives, Adobe and NVIDIA announced plans to make Adobe Firefly — Adobe’s family of creative generative AI models — available as APIs in Omniverse, enabling developers and creators to enhance their design processes.

Wonder Dynamics is connected to Omniverse with new OpenUSD export support through its AI platform Wonder Studio, which automatically animates, lights and composes computer-generated characters into live-action scenes. New OpenUSD export support will enable artists to generate and export a complete 3D scene — all from a single camera video.

Luma AI’s reality-capture models in USDZ format can be readily imported to Omniverse. Tools from avatar company Convai and character-engine company Inworld AI are connected to Omniverse. With AI tools like Convai, creators can add characters in their digital twin environments that can provide relevant information about the environment and objects, be a tour guide or be a virtual robot. Move AI enables single-camera motion capture with the Move One app, which can be used to generate 3D character animations that can then be exported to OpenUSD and used in Omniverse.

Omniverse users can now build content, experiences and applications that are compatible with other OpenUSD-based spatial computing platforms such as ARKit and RealityKit. Plus, new support for the Khronos Group’s OpenXR open standard expands Omniverse use to more headsets from manufacturers such as HTC VIVE, Magic Leap and Varjo.

Users of SideFX Houdini can also now load Houdini Digital Assets directly into the Omniverse viewport, making Houdini-based connected workflows more seamless. The Cesium extension for Omniverse, called Cesium for Omniverse, enables 3D Tiles, an open standard for streaming massive geospatial datasets in virtual worlds, including those supported by OpenUSD. CGI.Backgrounds now has several ultra-high-definition HDRi maps available in USD Composer. The Cadence DataCenter Design Software, now available via Omniverse, helps users see computational fluid dynamics simulations in the complete context of their digital twin. With the Cadence data center extension, users can plan, test and validate design and operational considerations before implementing them. And the Blackshark.AI world digital twin platform is now connected to Omniverse.

Customers Using Omniverse for Digitalization

Customers are using Omniverse for tasks ranging from simulating robots to training AI models and improving animation.

Boston Dynamics AI Institute is using Omniverse to simulate robots and their interactions to enable the design of novel robotics and control systems. Continental, one of the leading companies in automotive and industrialization of autonomous systems, is using Omniverse in its mobile robots business to generate physically accurate synthetic data at scale to train computer-vision AI models and perform system-integration testing.

Volvo Cars has transitioned its digital twin to be OpenUSD-based, using Omniverse to create immersive visualizations to help customers make online purchasing decisions.

Marks Design, a brand design and experience agency, is using Omniverse and OpenUSD to streamline collaboration and improve its animation, visualization and rendering workflows.

New Omniverse Systems and Partners

NVIDIA is collaborating with global systems manufacturers to bring RTX workstations optimally configured for Omniverse to millions of designers, architects and engineers. The new systems feature up to four NVIDIA RTX 6000 Ada Generation GPUs, bundled with NVIDIA Omniverse Enterprise software, to accelerate OpenUSD world-building, generative AI-enhanced collaborative design and other industrial digitalization applications.

Omniverse users can also take advantage of the new NVIDIA L40S GPU, a powerful, universal data center GPU that accelerates the most graphics-intensive workloads.

Availability

The latest Omniverse release is now available in beta to download for free and coming soon to Omniverse Enterprise.

Learn more about NVIDIA Omniverse and watch NVIDIA CEO Jensen Huang’s SIGGRAPH keynote address on demand.

About NVIDIA

Since its founding in 1993, NVIDIA (NASDAQ: NVDA) has been a pioneer in accelerated computing. The company’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, ignited the era of modern AI and is fueling industrial digitalization across markets. NVIDIA is now a full-stack computing company with data-center-scale offerings that are reshaping industry.

Source: Nvidia