Here Is the Amazing Backstory About Nvidia’s April GTC21 ‘Kitchen Keynote’ with CEO Jensen Huang 

For 14 seconds of Nvidia CEO Jensen Huang’s “kitchen keynote” address at the company’s GTC 2021 virtual conference in April, it turns out that viewers were not actually seeing Huang.

Instead, what they were seeing in that brief stretch was a complex digitalization of Huang that the company created to show off its prowess in GPUs, HPC and cutting-edge technologies.

Nvidia revealed its “magic trick” in an August 11 post on the Nvidia Blog, explaining that during one scene of the GTC21 keynote at the 1:02:41 mark of the one hour and 48 minute event, that 14 seconds of video purporting to be Huang was actually digitally created using Nvidia Omniverse.

It was innocuous at the time – Huang was introducing the company’s Nvidia Grace chip, the company’s first Arm-based CPU designed for terabyte scale accelerated computing – when Huang’s real video image was diced and hacked by video manipulation and then recomposed – this time as a digitalization of the real Huang. It was like watching Captain James T. Kirk being beamed up and down from a planet into the transporter room on the Starship Enterprise in “Star Trek."

It only last 14 seconds, but it was notable because of the techno-trickery that was needed to make it happen.

It was the third time Huang has hosted one of Nvidia’s trademark GPU Technology Conference (GTC) events virtually since 2020 due to the COVID-19 pandemic, each time from his home kitchen.

Only this time, the Nvidia team digitized his kitchen, turning it into a digital twin created from thousands of still photographs, millions of bits of information and more, according to the company.

“Typically, this type of project would take a team months to complete and weeks to render,” the post continued. “But with Omniverse, the animation was chiefly completed by a single animator and rendered in less than a day.”

In this spring’s April keynote, as Huang gave his introduction of Grace, the kitchen behind him and everything in it slid away, leaving Huang alone with the audience to show off an Nvidia DGX Station A100 machine containing Grace CPUs.

It was part of a shared virtual 3D world, or metaverse made up of interactive, immersive and collaborative components. Usually, such things are seen in sci-fi films, but here it was right in a virtual keynote hosted by GPU maker Nvidia.

The work was done by Nvidia artists, researchers and engineers on a tight deadline in the spring, and what they produced was a shared virtual world to tell the latest chapter in the company’s story, according to the blog post.

These are some of the gesture training screen captures created of Nvidia CEO Jensen Huang for the 14 seconds of digitized video in his GTC21 keynote.

“GTC is, first and foremost, our opportunity to highlight the amazing work that our engineers and other teams here at Nvidia have done all year long,” Rev Lebaredian, the vice president of Omniverse engineering and simulation at Nvidia, said in the post.

To create that virtual Huang, the company’s teams did a full face and body scan of him to create a 3D model, then trained an AI model to mimic his gestures and expressions, while applying some AI magic to make his clone realistic, the blog post reported. “Digital Jensen was then brought into a replica of his kitchen that was deconstructed to reveal the holodeck within Omniverse, surprising the audience and making them question how much of the keynote was real or rendered.”

The company also produced a short video documentary, “Connecting in the Metaverse: The Making of the GTC Keynote,” to graphically show the complex steps taken to produce the GTC21 milestone and its metaverse experiment.

Sure, it was techno-wizardry, but for the company, it was also a way of displaying some of the things it has been up to as it works to bring even more powerful computing to the enterprise.

Marc Staimer, president and chief data scientist analyst at Dragon Slayer Consulting, told EnterpriseAI that it may have been gimmicky, but it made an impact.

“It is also entertaining and keeps the audience engaged,” said Staimer. “The metaverse has potential as does augmented reality (AR) and virtual reality (VR). We are just at the very bleeding edge of achieving that potential because of AI and machine learning.”

Ultimately, Nvidia’s use of the technology in the keynote makes sense, said Staimer. “They provide hardware that makes it possible, so they are smart to associate themselves with it. Make no mistake, that technology will become ubiquitous before we know it.”

Another analyst, Karl Freund, founder and principal analyst for machine learning, HPC and AI at Cambrian AI Research, agreed.

“The idea of AR and VR collaboration is pretty compelling, and COVID-19 will linger and make digital collaboration even more important,” said Freund. “It is very cool that nobody even knew that a portion of Jensen’s presentation was digital.”

Addison Snell, the CEO of Intersect360 Research, told EnterpriseAI that it was a notable moment.

“The trick of instantly switching to digital twins of Jensen’s kitchen—and Jensen himself—was the sort of eye-popping-wow performance that we have come to expect from Nvidia with GTC,” Snell said. “Having graphics that are indistinguishable from magic is cool, but more importantly, it inspires innovation, not only in visual effects for movies and games, but for industries that rely on digital simulations, such as manufacturing, pharmaceuticals, and energy.”

Rob Enderle, principal analyst with Enderle Group, was similarly impressed.

“Nvidia’s Omniverse product is a front-end to creating the metaverse, which, eventually, will become a digital twin for much of the world,” said Enderle. “It has the potential to be bigger than the internet largely because it could contain it. This level of technology that can digitally create everything that was is or will be – and anything that can be imagined – has the potential to change how we view and interact with the world.”

Once this technology is further coupled with AI, the resulting metaverse extending to universal or microscopic scale “will allow exploring new places, new worlds, and theoretical places virtually based on actual, projected or imagined constructs and images,” said Enderle. “We can now, relatively cheaply, recreate almost anything digitally, even the impossible. We are about to enter the age of imagination, and Nvidia’s Omniverse is one of the significant foundational elements of the coming world of tomorrow.”

Huang was recently recognized by the Semiconductor Industry Association (SIA) as the recipient of its Robert N. Noyce Award, which is given annually in recognition of a leader who has made outstanding contributions to the semiconductor industry in technology or public policy.