Title: NVIDIA CEO Jensen Huang's Vision for the Future
Author: Cleo Abram

Transcript:
At some point, you have to believe something. 
We've reinvented computing as we know it. What
is the vision for what you see coming next? We 
asked ourselves, if it can do this, how far can
it go? How do we get from the robots that 
we have now to the future world that you
see? Cleo, everything that moves will be 
robotic someday and it will be soon. We
invested tens of billions of dollars before 
it really happened. No that's very good, you
did some research! But the big breakthrough 
I would say is when we...
That's Jensen Huang, and whether you know it or not
his decisions are shaping your future. He's the CEO of
NVIDIA, the company that skyrocketed over the past few
years to become one of the most valuable companies in
the world because they led a fundamental shift 
in how computers work unleashing this current
explosion of what's possible with technology. 
"NVIDIA's done it again!" We found ourselves being
one of the most important technology companies in 
the world and potentially ever. A huge amount of
the most futuristic tech that you're hearing about
in AI and robotics and gaming and self-driving
cars and breakthrough medical research relies on 
new chips and software designed by him and his
company. During the dozens of background interviews 
that I did to prepare for this what struck me most
was how much Jensen Huang has already influenced 
all of our lives over the last 30 years, and how
many said it's just the beginning of something 
even bigger. We all need to know what he's building
and why and most importantly what he's trying 
to build next. Welcome to Huge Conversations...
Thank you so much for doing this. I'm so happy to do 
it. Before we dive in, I wanted to tell you
how this interview is going to be a little bit 
different than other interviews I've seen you
do recently. Okay! I'm not going to ask you any 
questions about - you could ask - company finances,
thank you! I'm not going to ask you questions 
about your management style or why you don't
like one-on ones. I'm not going to ask you 
about regulations or politics. I think all
of those things are important but I think that our 
audience can get them well covered elsewhere. Okay.
What we do on huge if true is we make optimistic 
explainer videos and we've covered - I'm the worst
person to be an explainer video. I think you 
might be the best and I think that's what I'm
really hoping that we can do together is make a 
joint explainer video about how can we actually
use technology to make the future better. Yeah. And 
we do it because we believe that when people see
those better futures, they help build them. So 
the people that you're going to be talking to
are awesome. They are optimists who want to 
build those better futures but because we
cover so many different topics, we've covered 
supersonic planes and quantum computers and
particle colliders, it means that millions 
of people come into every episode without
any prior knowledge whatsoever. You might be 
talking to an expert in their field who doesn't
know the difference between a CPU and a GPU or a 
12-year-old who might grow up one day to be you
but is just starting to learn. For my part, 
I've now been preparing for this interview for
several months, including doing background 
conversations with many members of your team
but I'm not an engineer. So my goal is to help that 
audience see the future that you see so I'm going
to ask about three areas: The first is, how did we 
get here? What were the key insights that led to
this big fundamental shift in computing that we're 
in now? The second is, what's actually happening
right now? How did those insights lead to the world 
that we're now living in that seems like so much
is going on all at once? And the third is, what is 
the vision for what you see coming next? In order
to talk about this big moment we're in with AI 
I think we need to go back to video games in the
'90s. At the time I know game developers wanted 
to create more realistic looking graphics but
the hardware couldn't keep up with all of that 
necessary math. NVIDIA came up with
a solution that would change not just games 
but computing itself. Could you take us back
there and explain what was happening and what 
were the insights that led you and the NVIDIA
team to create the first modern GPU? So in the 
early '90s when we first started the company
we observed that in a software program inside 
it there are just a few lines of code, maybe
10% of the code, does 99% % of the processing 
and that 99% of the processing could be done
in parallel. However the other 90% of the code 
has to be done sequentially. It turns out that
the proper computer the perfect computer is one 
that could do sequential processing and parallel
processing not just one or the other. That was the 
big observation and we set out to build a company
to solve computer problems that normal computers 
can't. And that's really the beginning of NVIDIA.
My favorite visual of why a CPU versus a 
GPU really matters so much is a 15-year-old
video on the NVIDIA YouTube channel where the 
Mythbusters, they use a little robot shooting
paintballs one by one to show solving problems 
one at a time or sequential processing on a
CPU, but then they roll out this huge robot 
that shoots all of the paintballs at once
doing smaller problems all at the same 
time or parallel processing on a GPU.
"3... 2... 1..." So Nvidia unlocks all of this new power
for video games. Why gaming first? The video games
requires parallel processing for processing 
3D graphics and we chose video games because,
one, we loved the application, it's a simulation 
of virtual worlds and who doesn't want to go to
virtual worlds and we had the good observation 
that video games has potential to be the largest
market for for entertainment ever. And it turned 
out to be true. And having it being a large market
is important because the technology is complicated 
and if we had a large market, our R&D budget could
be large, we could create new technology. And that 
flywheel between technology and market and greater
technology was really the flywheel that 
got NVIDIA to become one of the most important
technology companies in the world. It was all 
because of video games. I've heard you say that
GPUs were a time machine? Yeah. Could you tell me 
more about what you meant by that? A GPU is like a
time machine because it lets you see the future 
sooner. One of the most amazing things anybody's
ever said to me was a quantum chemistry 
scientist. He said, Jensen, because of NVIDIA's work,
I can do my life's work in my lifetime. That's time 
travel. He was able to do something that was beyond
his lifetime within his lifetime and this is 
because we make applications run so much faster
and you get to see the future. And so when you're 
doing weather prediction for example, you're seeing
the future when you're doing a simulation 
a virtual city with virtual traffic and we're
simulating our self-driving car through 
that virtual city, we're doing time travel. So
parallel processing takes off in gaming and it's 
allowing us to create worlds in computers that
we never could have before and and gaming is sort 
of this this first incredible cas Cas of parallel
processing unlocking a lot more power and then 
as you said people begin to use that power across
many different industries. The case of the of the 
quantum chemistry researcher, when I've heard you
tell that story it's that he was running molecular 
simulations in a way where it was much faster to
run in parallel on NVIDIA GPUs even then than it 
was to run them on the supercomputer with the CPU
that he had been using before. Yeah that's true. So 
oh my god it's revolutionizing all of these other
industries as well, it's beginning to change 
how we see what's possible with computers and my
understanding is that in the early 2000s you 
see this and you realize that actually doing
that is a little bit difficult because what that 
researcher had to do is he had to sort of trick
the GPUs into thinking that his problem was a 
graphics problem. That's exactly right, no that's
very good, you did some research. So you create 
a way to make that a lot easier. That's right
Specifically it's a platform called CUDA which 
lets programmers tell the GPU what to do using
programming languages that they already know like 
C and that's a big deal because it gives way more
people easier access to all of this computing 
power. Could you explain what the vision was that
led you to create CUDA? Partly researchers 
discovering it, partly internal inspiration and
and partly solving a problem. And you know a 
lot of interesting interesting ideas come out
of that soup. You know some of it is aspiration 
and inspiration, some of it is just desperation you
know. And so in the case of CUDA is very 
much this the same way and probably the first
external ideas of using our GPUs for parallel 
processing emerged out of some interesting work
in medical imaging a couple of researchers 
at Mass General were using it to do CT
reconstruction. They were using our graphics 
processors for that reason and it inspired us.
Meanwhile the problem that we're trying to solve 
inside our company has to do with the fact that
when you're trying to create these virtual worlds 
for video games, you would like it to be beautiful
but also dynamic. Water should flow like water and 
explosions should be like explosions. So there's
particle physics you want to do, fluid dynamics 
you want to do and that is much harder to do if
your pipeline is only able to do computer graphics. 
And so we have a natural reason to want to do it
in the market that we were serving. So 
researchers were also horsing around with using
our GPUs for general purpose uh acceleration and 
and so there there are multiple multiple factors
that were coming together in that soup, we 
just when the time came and we decided
to do something proper and created a CUDA as 
a result of that. Fundamentally the reason why
I was certain that CUDA was going to be successful 
and we put the whole company behind it was
because fundamentally our GPU was going to be 
the highest volume parallel processors built in
the world because the market of video games was so 
large and so this architecture has a good chance
of reaching many people. It has seemed to me like 
creating CUDA was this incredibly optimistic "huge
if true" thing to do where you were saying, if we 
create a way for many more people to use much
more computing power, they might create incredible 
things. And then of course it came true. They did.
In 2012, a group of three researchers submits an 
entry to a famous competition where the goal is
to create computer systems that could recognize 
images and label them with categories. And their
entry just crushes the competition. It gets way 
fewer answers wrong. It was incredible. It blows
everyone away. It's called AlexNet, and it's a kind 
of AI called the neural network. My understanding
is one reason it was so good is that they used 
a huge amount of data to train that system
and they did it on NVIDIA GPUs. All of a sudden, 
GPUs weren't just a way to make computers faster
and more efficient they're becoming the engines 
of a whole new way of computing. We're moving from
instructing computers with step-by-step directions 
to training computers to learn by showing them a
huge number of examples. This moment in 2012 really 
kicked off this truly seismic shift that we're
all seeing with AI right now. Could you describe 
what that moment was like from your perspective
and what did you see it would mean for all of 
our futures? When you create something new like
CUDA, if you build it, they might not come. And
that's always the cynic's perspective
however the optimist's perspective would say, but 
if you don't build it, they can't come. And that's
usually how we look at the world. You know we 
have to reason about intuitively why this would
be very useful. And in fac, in 2012 Ilya Sutskever, 
and Alex Krizhevsky and Geoff Hinton in the University
of Toronto the lab that they were at they reached 
out to a gForce GTX 580 because they learned about
CUDA and that CUDA might be able to to be used as 
a parallel processor for training AlexNet and
so our inspiration that GeForce could be the the 
vehicle to bring out this parallel architecture
into the world and that researchers would somehow 
find it someday was a good was a good strategy. It
was a strategy based on hope, but it was also 
reasoned hope. The thing that really caught
our attention was simultaneously we were trying 
to solve the computer vision problem inside the
company and we were trying to get CUDA to 
be a good computer vision processor and we
were frustrated by a whole bunch of early 
developments internally with respect to our
computer vision effort and getting CUDA to be 
able to do it. And all of a sudden we saw AlexNet,
this new algorithm that is completely 
different than computer vision algorithms before
it, take a giant leap in terms of capability 
for computer vision. And when we saw that it was
partly out of interest but partly because we were 
struggling with something ourselves. And so we were
we were highly interested to want to see it work. 
And so when we when we looked at AlexNet we were
inspired by that. But the big breakthrough I 
would say is when we when we saw AlexNet, we
asked ourselves you know, how far can AlexNet 
go? If it can do this with computer vision, how
far can it go? And if it if it could go to the 
limits of what we think it could go, the type
of problems it could solve, what would it mean for 
the computer industry? And what would it mean for
the computer architecture? And we were, 
we rightfully reasoned that if machine learning,
if the deep learning architecture can scale, 
the vast majority of machine learning problems
could be represented with deep neural networks. And 
the type of problems we could solve with machine
learning is so vast that it has the potential of 
reshaping the computer industry altogether,
which prompted us to re-engineer the entire 
computing stack which is where DGX came from
and this little baby DGX sitting here, all of 
this came from from that observation that we ought
to reinvent the entire computing stack layer by 
layer by layer. You know computers, after 65 years
since IBM System 360 introduced modern general 
purpose computing, we've reinvented computing as we
know it. To think about this as a whole story, so 
parallel processing reinvents modern gaming and
revolutionizes an entire industry then that way 
of computing that parallel processing begins to
be used across different industries. You invest 
in that by building CUDA and then CUDA and the
use of GPUs allows for a a step change in neural 
networks and machine learning and begins a sort
of revolution that we're now seeing only 
increase in importance today... All of a sudden
computer vision is solved. All of a sudden speech 
recognition is solved. All of a sudden language
understanding is solved. These incredible 
problems associated with intelligence one
by one by one by one where we had no solutions 
for in past, desperate desire to have solutions
for, all of a sudden one after another get solved 
you know every couple of years. It's incredible.
Yeah so you're seeing that, in 2012 you're 
looking ahead and believing that that's
the future that you're going to be living in now, 
and you're making bets that get you there, really
big bets that have very high stakes. And then my 
perception as a lay person is that it takes a
pretty long time to get there. You make these bets - 
8 years, 10 years - so my question is:
If AlexNet that happened in 2012 and this audience 
is probably seeing and hearing so much more about
AI and NVIDIA specifically 10 years later, 
why did it take a decade and also because you
had placed those bets, what did the middle 
of that decade feel like for you? Wow that's
a good question. It probably felt like today. You 
know to me, there's always some problem and
then there's some reason to be to be 
impatient. There's always some reason to be
happy about where you are and there's always 
many reasons to carry on. And so I think as I
was reflecting a second ago, that sounds like this 
morning! So but I would say that in all things that
we pursue, first you have to have core beliefs. You 
have to reason from your best principles
and ideally you're reasoning from it from principles 
of either physics or deep understanding of
the industry or deep understanding of the 
science, wherever you're reasoning from, you
reason from first principles. And at some point you 
have to believe something. And if those principles
don't change and the assumptions don't change,
then you, there's no reason to change your
core beliefs. And then along the way there's always 
some evidence of you know of success and
and that you're leading in the right 
direction and sometimes you know you go a
long time without evidence of success and you 
might have to course correct a little but
the evidence comes. And if you feel like you're 
going in the right direction, we just keep on going.
The question of why did we stay so committed for 
so long, the answer is actually the opposite: There
was no reason to not be committed because we are, 
we believed it. And I've believed in NVIDIA
for 30 plus years and I'm still here working 
every single day. There's no fundamental
reason for me to change my belief system and 
I fundamentally believe that the
work we're doing in revolutionizing computing 
is as true today, even more true today than it
was before. And so we'll stick 
with it you know until otherwise. There's
of course very difficult times along the way. You 
know when you're investing in something and nobody
else believes in it and cost a lot of money and 
you know maybe investors or or others would rather
you just keep the profit or you know whatever it 
is improve the share price or whatever it is.
But you have to believe in your future. You have to 
invest in yourself. And we believe this so
deeply that we invested you know tens 
of billions of dollars before it really
happened. And yeah it was, it was 10 long 
years. But it was fun along the way.
How would you summarize those core beliefs? What 
is it that you believe about the way computers
should work and what they can do for us that keeps 
you not only coming through that decade but also
doing what you're doing now, making bets I'm sure 
you're making for the next few decades? The first
core belief was our first discussion, was about 
accelerated computing. Parallel computing versus
general purpose computing. We would add 
two of those processors together and we would do
accelerated computing. And I continue to believe 
that today. The second was the recognition
that these deep learning networks, these DNNs, that 
came to the public during 2012, these deep neural
networks have the ability to learn patterns and 
relationships from a whole bunch of different
types of data. And that it can learn more and 
more nuanced features if it could be larger
and larger. And it's easier to make them larger and 
larger, make them deeper and deeper or wider and
wider, and so the scalability of the architecture 
is empirically true. The fact
that model size and the data size being larger 
and larger, you can learn more knowledge is
also true, empirically true. And so if that's 
the case, you could you know, what what are the
limits? There not, unless there's a physical limit 
or an architectural limit or mathematical limit
and it was never found, and so we believe that you 
could scale it. Then the question, the only other
question is: What can you learn from data? What 
can you learn from experience? Data is basically
digital versions of human experience. And so what 
can you learn? You obviously can learn object
recognition from images. You can learn speech 
from just listening to sound. You can learn
even languages and vocabulary and syntax and 
grammar and all just by studying a whole bunch
of letters and words. So we've now demonstrated 
that AI or deep learning has the ability to learn
almost any modality of data and it can translate 
to any modality of data. And so what does that mean?
You can go from text to text, right, summarize a 
paragraph. You can go from text to text, translate
from language to language. You can go from text 
to images, that's image generation. You can go from
images to text, that's captioning. You can even go 
from amino acid sequences to protein structures.
In the future, you'll go from protein to words: "What 
does this protein do?" or "Give me an example of a
protein that has these properties." You know 
identifying a drug target. And so you could
just see that all of these problems are around 
the corner to be solved. You can go from words
to video, why can't you go from words to action 
tokens for a robot? You know from the computer's
perspective how is it any different? And so it 
it opened up this universe of opportunities and
universe of problems that we can go solve. And 
that gets us quite excited. It feels like
we are on the cusp of this truly enormous change. 
When I think about the next 10 years, unlike the
last 10 years, I know we've gone through a lot of 
change already but I don't think I can predict
anymore how I will be using the technology that is 
currently being developed. That's exactly right. I
think the last 10, the reason why you feel that way 
is, the last 10 years was really about the science
of AI. The next 10 years we're going to have plenty 
of science of AI but the next 10 years is going to
be the application science of AI. The fundamental 
science versus the application science. And so the
the applied research, the application side of AI 
now becomes: How can I apply AI to digital biology?
How can I apply AI to climate technology? How can 
I apply AI to agriculture, to fishery, to robotics,
to transportation, optimizing logistics? How can 
I apply AI to you know teaching? How do I apply AI
to you know podcasting right? I'd love to 
choose a couple of those to help people see how
this fundamental change in computing that we've 
been talking about is actually going to change
their experience of their lives, how they're 
actually going to use technology that is based
on everything we just talked about. One of the 
things that I've now heard you talk a lot about
and I have a particular interest in is physical 
AI. Or in other words, robots - "my friends!" - meaning
humanoid robots but also robots like self-driving 
cars and smart buildings or autonomous warehouses
or autonomous lawnmowers or more. From what 
I understand, we might be about to see a huge
leap in what all of these robots are capable of 
because we're changing how we train them. Up until
recently you've either had to train your robot in 
the real world where it could get damaged or wear
down or you could get data from fairly limited 
sources like humans in motion capture suits. But
that means that robots aren't getting as many 
examples as they'd need to learn more quickly.
But now we're starting to train robots in digital 
worlds, which means way more repetitions a day, way
more conditions, learning way faster. So we could 
be in a big bang moment for robots right now and
NVIDIA is building tools to make that happen. You 
have Omniverse and my understanding is this is 3D
worlds that help train robotic systems so that 
they don't need to train in the physical world.
That's exactly right. You just just announced 
Cosmos which is ways to make that 3D universe
much more realistic. So you can get all kinds 
of different, if we're training something on
this table, many different kinds of lighting on the 
table, many different times of day, many different
you know experiences for the robot to go through 
so that it can get even more out of Omniverse. As
a kid who grew up loving Data on Star Trek, Isaac 
Asimov's books and just dreaming about a future with
robots, how do we get from the robots that we have 
now to the future world that you see of robotics?
Yeah let me use language models maybe ChatGPT 
as a reference for understanding Omniverse and
Cosmos. So first of all when ChatGPT first 
came out it, it was extraordinary and
it has the ability to do to basically from 
your prompt, generate text. However, as amazing as
it was, it has the tendency to hallucinate if
it goes on too long or if it pontificates about
a topic it you know is not informed about, it'll 
still do a good job generating plausible answers.
It just wasn't grounded in the truth. And so
people called it hallucination. And
so the next generation shortly it was, it had 
the ability to be conditioned by context, so
you could upload your PDF and now it's grounded 
by the PDF. The PDF becomes the ground truth. It
could be it could actually look up search and 
then the search becomes its ground truth. And
between that it could reason about what is how 
to produce the answer that you're asking for. And
so the first part is a generative AI and the 
second part is ground truth. Okay and so now let's
come into the the physical world. The
world model, we need a foundation model just like
we need ChatGPT had a core foundation model 
that was the breakthrough in order for robotics
to to be smart about the physical world. It has to 
understand things like gravity, friction, inertia,
geometric and spatial awareness. It has to uh 
understand that an object is sitting there even
when I looked away when I come back it's still 
sitting there, object permanence. It has to
understand cause and effect. If I tip it, it'll 
fall over. And so these kind of physical
common sense if you will has to be captured or 
encoded into a world foundation model so that
the AI has world common sense. Okay and so we 
have to go, somebody has to go create that, and
that's what we did with Cosmos. We created a world 
language model. Just like ChatGPT was a language model,
this is a world model. The second thing we have to 
go do is we have to do the same thing that we did
with PDFs and context and grounding it with 
ground truth. And so the way we augment Cosmos
with ground truth is with physical simulations, 
because Omniverse uses physics simulation which
is based on principled solvers. The mathematics 
is Newtonian physics is the, right, it's the math we
know, all of the the fundamental laws of 
physics we've understood for a very long
time. And it's encoded into, captured into Omniverse. 
That's why Omniverse is a simulator. And using the
simulator to ground or to condition Cosmos, we can 
now generate an infinite number of stories of the
future. And they're grounded on physical truth. Just 
like between PDF or search plus ChatGPT, we can
generate an infinite amount of interesting things, 
answer a whole bunch of interesting questions. The
combination of Omniverse plus Cosmos, you could 
do that for the physical world. So to illustrate
this for the audience, if you had a robot in a 
factory and you wanted to make it learn every
route that it could take, instead of manually 
going through all of those routes, which could
take days and could be a lot of wear and tear on 
the robot, we're now able to simulate all of them
digitally in a fraction of the time and in many 
different situations that the robot might face -
it's dark, it's blocked it's etc - so the robot 
is now learning much much faster. It seems to
me like the future might look very different than 
today. If you play this out 10 years, how do you see
people actually interacting with this technology 
in the near future? Cleo, everything that moves
will be robotic someday and it will be soon. You 
know the the idea that you'll be pushing around
a lawn mower is already kind of silly. You know 
maybe people do it because because it's fun but
but there's no need to. And every car is 
going to be robotic. Humanoid robots, the technology
necessary to make it possible, is just around 
the corner. And so everything that moves will be
robotic and they'll learn how to be 
a robot in Omniverse Cosmos and we'll generate
all these plausible, physically plausible futures 
and the the robots will learn from them and
then they'll come into the physical world and you 
know it's exactly the same. A future where
you're just surrounded by robots is for certain. 
And I'm just excited about having my own R2-D2.
And of course R2-D2 wouldn't be quite the can that 
it is and roll around. It'll be you know R2-D2
yeah, it'll probably be a different physical 
embodiment, but it's always R2. You know so my R2
is going to go around with me. Sometimes it's in my 
smart glasses, sometimes it's in my phone, sometimes
it's in my PC. It's in my car. So R2 is with me 
all the time including you know when I get home
you know where I left a physical version of R2. And 
you know whatever that version happens to
be you know, we'll interact with R2. And so I 
think the idea that we'll have our own R2-D2 for
our entire life and it grows up with us, that's 
a certainty now yeah. I think a lot of news media
when they talk about futures like this they focus 
on what could go wrong. And that makes sense. There
is a lot that could go wrong. We should talk about 
what could go wrong so we could keep it from from
going wrong. Yeah that's the approach that we like 
to take on the show is, what are the big challenges
so that we can overcome them? Yeah. What buckets do 
you think about when you're worrying about this
future? Well there's a whole bunch of the 
stuff that everybody talks about: Bias or toxicity
or just hallucination. You know speaking with 
great confidence about something it knows nothing
about and as a result we rely on that information. 
Generating, that's a version of generating
fake information, fake news or fake images 
or whatever it is. Of course impersonation.
It does such a good job pretending to be a 
human, it could be it could do an incredibly good
job pretending to be a specific human. And so
the spectrum of areas we
have to be concerned about is fairly clear and 
there's a lot of people who are
working on it. There's a some of the stuff, 
some of the stuff related to AI safety requires
deep research and deep engineering and 
that's simply, it wants to do the right thing it
just didn't perform it right and as a result hurt 
somebody. You know for example self-driving car
that wants to drive nicely and drive properly 
and just somehow the sensor broke down or it
didn't detect something. Or you know made it 
too aggressive turn or whatever it is. It did
it poorly. It did it wrongly. And so that's
a whole bunch of engineering that has to
be done to to make sure that AI safety is upheld 
by making sure that the product functions properly.
And then and then lastly you know whatever what 
happens if the system, the AI wants to do a good
job but the system failed. Meaning the AI wanted 
to stop something from happening
and it turned out just when it wanted to do 
it, the machine broke down. And so this is
no different than a flight computer inside 
a plane having three versions of them and then
so there's triple redundancy inside the 
system inside autopilots and then you have two
pilots and then you have air traffic control 
and then you have other pilots watching out for
these pilots. And so that the AI safety 
systems has to be architected as a community
such that such that these AIs one, work,
function properly. When they don't
function properly, they don't put people in harm's 
way. And that they're sufficient safety and
security systems all around them to make sure 
that we keep AI safe. And so there's
this spectrum of conversation is gigantic and and 
you know we have to take the parts, take the
parts apart and and build them as engineers. One 
of the incredible things about this moment that
we're in right now is that we no longer have a 
lot of the technological limits that we had in a
world of CPUs and sequential processing. And we've 
unlocked not only a new way to do computing and
and but also a way to continue to improve. Parallel 
processing has a a different kind of physics to it
than the improvements that we were able to make 
on CPUs. I'm curious, what are the scientific or
technological limitations that we face now in 
the current world that you're thinking a lot
about? Well everything in the end is about how much 
work you can get done within the limitations of
the energy that you have. And so that's 
a physical limit and the laws of
physics about transporting information and 
transporting bits, flipping bits and transporting
bits, at the end of the day the energy it takes 
to do that limits what we can get done. And the
amount of energy that we have limits what we can 
get done. We're far from having any fundamental
limits that keep us from advancing. In the meantime, 
we seek to build better and more energy efficient
computers. This little computer, the the 
big version of it was $250,000 - Pick up? - Yeah
Yeah that's little baby DIGITS yeah. This is 
an AI supercomputer. The version that I delivered,
this is just a prototype so it's a mockup.
The very first version was DGX 1, I
delivered to Open AI in 2016 and that was $250,000. 
10,000 times more power, more energy necessary
than this version and this version has six times 
more performance. I know, it's incredible. We're
in a whole in the world. And it's only since 2016 
and so eight years later we've in increased the
energy efficiency of computing by 10,000 times. 
And imagine if we became 10,000 times more energy
efficient or if a car was 10,000 times more 
energy efficient or electric light bulb was
10,000 times more energy efficient. Our light 
bulb would be right now instead of 100 Watts,
10,000 times less producing the same illumination. 
Yeah and so the energy efficiency of
computing particularly for AI computing that we've 
been working on has advanced incredibly and that's
essential because we want to create you 
know more intelligent systems and and we want to
use more computation to be smarter and so 
energy efficiency to do the work is our number one
priority. When I was preparing for this interview, I 
spoke to a lot of my engineering friends and this
is a question that they really wanted me to ask. So 
you're really speaking to your people here. You've
shown a value of increasing accessibility 
and abstraction, with CUDA and allowing more
people to use more computing power in all kinds of 
other ways. As applications of technology get more
specific, I'm thinking of transformers in AI for 
example... For the audience, a transformer is a very
popular more recent structure of AI that's now 
used in a huge number of the tools that you've
seen. The reason that they're popular is because 
transformers are structured in a way that helps
them pay "attention" to key bits of information and 
give much better results. You could build chips
that are perfectly suited for just one kind of AI 
model, but if you do that then you're making them
less able to do other things. So as these specific 
structures or architectures of AI get more popular,
my understanding is there's a debate between how 
much you place these bets on "burning them into the
chip" or designing hardware that is very specific 
to a certain task versus staying more general and
so my question is, how do you make those bets? How 
do you think about whether the solution is a car
that could go anywhere or it's really optimizing 
a train to go from A to B? You're making bets
with huge stakes and I'm curious how you think 
about that. Yeah and that now comes back
to exactly your question, what are your 
core beliefs? And the question, the core
belief either one, that transformer is the last AI 
algorithm, AI architecture that any researcher will
ever discover again, or that transformers 
is a stepping stone towards evolutions of
transformers that are uh barely recognizable as a 
transformer years from now. And we believe the
latter. And the reason for that is because you 
just have to go back in history and ask yourself,
in the world of computer algorithms, in 
the world of software, in the world of
engineering and innovation, has one idea stayed 
along that long? And the answer is no. And so that's
kind of the beauty, that's in fact 
the essential beauty of a computer that it's able
to do something today that no one even imagined 
possible 10 years ago. And if you would have, if
you would have turned that computer 10 years ago 
into a microwave, then why would the applications
keep coming? And so we believe, we believe in the 
richness of innovation and the
richness of invention and we want to create an 
architecture that let inventors and innovators
and software programmers and AI researchers 
swim in the soup and come up with some amazing
ideas. Look at transformers. The fundamental 
characteristic of a transformer is this idea
called "attention mechanism" and it basically says 
the transformer is going to understand the meaning
and the relevance of every single word with every 
other word. So if you had 10 words, it has to figure
out the relationship across 10 of them. But if you 
have a 100,000 words or if your context is
now as large as, read a PDF and that read a whole 
bunch of PDFs, and the context window is now like
a million tokens, the processing all of it across 
all of it is just impossible. And so the way you
solve that problem is there all kinds of new ideas, 
flash attention or hierarchical attention or you
know all the, wave attention I just read about 
the other day. The number of different types of
attention mechanisms that have been invented 
since the transformer is quite extraordinary.
And so I think that that's going to continue 
and we believe it's going to continue and that
that computer science hasn't ended and that AI 
research have not all given up and we haven't
given up anyhow and that having a 
computer that enables the flexibility of
of research and innovation and new ideas is 
fundamentally the most important thing. One of the
things that I am just so curious about, you design 
the chips. There are companies that assemble the
chips. There are companies that design hardware to 
make it possible to work at nanometer scale. When
you're designing tools like this, how do you think 
about design in the context of what's physically
possible right now to make? What are the things 
that you're thinking about with sort of pushing
that limit today? The way we do it is even 
though even though we have things made like for
example our chips are made by TSMC. Even though 
we have them made by TSMC, we assume that we need
to have the deep expertise that TSMC has. And so 
we have people in our company who are incredibly
good at semiconductive physics so that we have a 
feeling for, we have an intuition for, what are the
limits of what today's semiconductor physics 
can do. And then we work very closely with them to
discover the limits because we're trying to push 
the limits and so we discover the limits together.
Now we do the same thing in system engineering and 
cooling systems. It turns out plumbing is really
important to us because of liquid cooling. 
And maybe fans are really important to us
because of air cooling and we're trying to design 
these fans in a way almost like you know they're
aerodynamically sound so that we could pass the 
highest volume of air, make the least amount of
noise. So we have aerodynamics engineers in our
company. And so even though even though we don't
make 'em, we design them and we have to deep 
expertise of knowing how to have them made. And
and from that we try to push the 
limits. One of the themes of this conversation is
that you are a person who makes big bets on the 
future and time and time again you've been right
about those bets. We've talked about GPUs, we've 
talked about CUDA, we've talked about bets you've
made in AI - self-driving cars, and we're going to 
be right on robotics and - this is my question. What
are the bets you're making now? the latest bet we
just described at the CES and I'm very very proud
of it and I'm very excited about it is the 
fusion of Omniverse and Cosmos so that we have
this new type of generative world generation 
system, this multiverse generation system. I
think that's going to be profoundly important in 
the future of robotics and physical systems.
Of course the work that we're doing with human 
robots, developing the tooling systems and the
training systems and the human demonstration 
systems and all of this stuff that that you've
already mentioned, we're just seeing the 
beginnings of that work and I think the
next 5 years are going to be very interesting in 
the world of human robotics. Of course the work
that we're doing in digital biology so that 
we can understand the language of molecules and
understand the language of cells and just as 
we understand the language of physics and the
physical world we'd like to understand the language 
of the human body and understand the language of
biology. And so if we can learn that, and we can 
predict it. Then all of a sudden our ability to
have a digital twin of the human is plausible. 
And so I'm very excited about that work. I love
the work that we're doing in climate science 
and be able to, from weather predictions, understand
and predict the high resolution regional climates, 
the weather patterns within a kilometer above
your head. That we can somehow predict that with 
great accuracy, its implications is really quite
profound. And so the number of things that 
we're working on is really cool. You know we
we're fortunate that we've created this 
this instrument that is a time machine and
we need time machines in all of these areas that 
we just talked about so that we can see
the future. And if we could see the future and 
we can predict the future then we have a better
chance of making that future the best version 
of it. And that's the reason why scientists
want to predict the future. That's the reason why, 
that's the reason why we try to predict the future
and everything that we try to design so that we 
can optimize for the best version. So if
someone is watching this and maybe they came into 
this video knowing that NVIDIA is an incredibly
important company but not fully understanding why 
or how it might affect their life and they're now
hopefully better understanding a big shift that 
we've gone through over the last few decades in
computing, this very exciting, very sort of strange 
moment that we're in right now, where we're sort
of on the precipice of so many different things. 
If they would like to be able to look into the
future a little bit, how would you advise them to 
prepare or to think about this moment that they're
in personally with respect to how these tools 
are actually going to affect them? Well there are
several ways to reason about the future that 
we're creating. One way to reason about it is,
suppose the work that you do continues to 
be important but the effort by which you
do it went from you know being a week long 
to almost instantaneous. You know that the
effort of drudgery basically goes to zero. 
What is the implication of that? This is, this
is very similar to what would change if all 
of a sudden we had highways in this country?
And that kind of happened you know in the last 
Industrial Revolution, all of a sudden we have
interstate highways and when you have interstate 
highways what happens? Well you know suburbs start
to be created and and all of a sudden you know 
distribution of goods from east to west is
no longer a concern and all of a sudden gas 
stations start cropping up on highways and
and fast food restaurants show up and you 
know someone, some motels show up because people
you know traveling across the state, across the 
country and just wanted to stay somewhere for a
few hours or overnight, and so all of a sudden 
new economies and new capabilities, new economies.
What would happen if a video conference made 
it possible for us to see each other without
having to travel anymore? All of a sudden 
it's actually okay to work further away from
home and from work, work and live 
further away. And so you ask yourself kind of
these questions. You know what would happen 
if I have a software programmer with me
all the time and whatever it is I can dream up, 
the software programmer could write for me. You
know what would, what would happen 
if I just had a seed of an idea and
and I rough it out and all of sudden a you know 
a prototype of a production was put in front
of me? And what how would that change my life and 
how would that change my opportunity? And you
know what does it free me to be able to do and 
and so on so forth. And so I think that the next
the next decade intelligence, not for everything 
but for for some things, would basically become
superhuman. But I can tell 
you exactly what that feels like. I'm surrounded
by superhuman people, super intelligence from 
my perspective because they're the best in the
world at what they do and they do what they 
do way better than I can do it. and I'm
surrounded by thousands of them and yet what it 
it never one day caused me to to think all of a
son I'm no longer necessary. It actually empowers 
me and gives me the confidence to go tackle more
and more ambitious things. And so suppose, 
suppose now everybody is surrounded by these
super AIs that are very good at specific things 
or good at some of the things. What would that
make you feel? Well it's going to empower you, 
it's going to make you feel confident and
and I'm pretty sure you probably use ChatGPT and 
AI and I feel more empowered today, more
confident to learn something today. The knowledge 
of almost any particular field, the barriers to
that understanding, it has been reduced and I have 
a personal tutor with me all of the time. And
so I think that that feeling should be universal.
If there's one thing that I would
encourage everybody to do is to go get yourself 
an AI tutor right away. And that AI tutor could
of course just teach your things, anything you 
like, help you program, help you write,
help you analyze, help you think, help you reason, 
you know all of those things is going to
really make you feel empowered and and I think 
that going to be our future. We're
going to become, we're going to become super humans, 
not because we have super, we're going to become
super humans because we have super AIs. Could you 
tell us a little bit about each of these objects?
This is a new GeForce graphics card and yes and 
this is the RTX 50 Series. It is essentially
a supercomputer that you put into your PC and we 
use it for gaming, of course people today use it
for design and creative arts and it does amazing 
AI. The real breakthrough here and this is
this is truly an amazing thing, GeForce 
enabled AI and it enabled Geoff Hinton, Ilya Sutskever,
Alex Krizhevsky to be able to train AlexNet. We
discovered AI and we advanced AI then AI came back
to GeForce to help computer graphics. And so here's 
the amazing thing: Out of 8 million pixels or so in
a 4K display we are computing, we're processing 
only 500,000 of them. The rest of them we use AI
to predict. The AI guessed it and yet the image is 
perfect. We inform it by the 500,000 pixels that we
computed and we ray traced every single one and it's 
all beautiful. It's perfect. And then we tell the
AI, if these are the 500,000 perfect pixels in this 
screen, what are the other 8 million? And it goes it
fills in the rest of the screen and it's perfect.
And if you only have to do fewer pixels, are you
able to invest more in doing that because you have 
fewer to do so then the quality is better so the
extrapolation that the AI does... Exactly. Because 
whatever computing, whatever attention you have,
whatever resources you have, you can place it into 
500,000 pixels. Now this is a perfect example of
why AI is going to make us all superhuman, because 
all of the other things that it can do, it'll do
for us, allows us to take our time and energy and 
focus it on the really really valuable things that
we do. And so we'll take our own resource which is 
you know energy intensive, attention intensive, and
we'll dedicated to the few 100,000 pixels and 
use AI to superres, upres it you know to
everything else. And so this this graphics card 
is now powered mostly by AI and the computer
graphics technology inside is incredible as 
well. And then this next one, as I mentioned
earlier, in 2016 I built the first one for AI 
researchers and we delivered the first one to Open AI
and Elon was there to receive it and this 
version I built a mini mini version and the
reason for that is because AI has now gone from AI 
researchers to every engineer, every student, every
AI scientist. And AI is going to be everywhere. 
And so instead of these $250,000 versions we're
going to make these $3,000 versions and schools 
can have them, you know students can have them, and
you set it next to your PC or Mac and all of 
a sudden you have your own AI supercomputer. And
you could develop and build AIs. Build your own 
AI, build your own R2-D2. What do you feel like is
important for this audience to know that I haven't 
asked? One of the most important things I would
advise is for example if I were a student today 
the first thing I would do is to learn AI. How do
I learn to interact with ChatGPT, how do I learn 
to interact with Gemini Pro, and how do I learn
to interact with Grok? Learning how to
interact with with AI is not unlike being
someone who is really good at asking questions. 
You're incredibly good at asking questions and
and prompting AI is very very similar.
You can't just randomly ask a bunch of questions
and so asking an AI to be assistant 
to you requires some expertise and
artistry and how to prompt it. And so if I were, 
if I were a student today, irrespective whether
it's for for math or for science or chemistry 
or biology or doesn't matter what field of science
I'm going to go into or what profession, I'm 
going to ask myself, how can I use AI to do my job
better? If I want to be a lawyer, how can I use 
AI to be a better lawyer? If I want to be a better
do doctor, how can I use AI to be a better doctor? 
If I want to be a chemist, how do I use AI to be
a better chemist? If I want to be a biologist, I how 
do I use AI to be a better biologist? That question
should be persistent across everybody. And just as 
my generation grew up as the first generation
that has to ask ourselves, how can we use computers 
to do our jobs better? Yeah the generation before
us had no computers, my generation was the first 
generation that had to ask the question, how do I
use computers to do my job better? Remember I came 
into the industry before Windows 95 right, 1984
there were no computers in offices. And after that, 
shortly after that, computers started to emerge and
so we had to ask ourselves how do we use computers 
to do our jobs better? The next generation doesn't
have to ask that question but it has to ask 
obviously next question, how can I use AI to
do my job better? That is start and finish I think 
for everybody. It's a really exciting and scary and
therefore worthwhile question I think for everyone. 
I think it's going to be incredibly fun. AI is
obviously a word that people are just learning 
now but it's just you know, it's
made your computer so much more accessible. It is 
easier to prompt ChatGPT to ask it anything you
like than to go do the research yourself. And so 
we've lowered a barrier of understanding, we've
lowered a barrier of knowledge, we've 
lowered a barrier of intelligence, and
and everybody really had to just go try 
it. You know the thing that's really really crazy
is if I put a computer in front of somebody and 
they've never used a computer there is no chance
they're going to learn that computer in a day.
There's just no chance. Somebody really has to
show it to you and yet with ChatGPT if you 
don't know how to use it, all you have to do is
type in "I don't know how to use ChatGPT, tell 
me," and it would come back and give you some
examples and so that's the amazing thing.
You know the amazing thing about intelligence is
it'll help you along the way and make you uh 
superhuman you know along the way. All right I have
one more question if you have a second. This is 
not something that I planned to ask you but on the
way here, I'm a little bit afraid of planes, 
which is not my most reasonable quality, and
the flight here was a little bit bumpy mhm very 
bumpy and I'm sitting there and it's moving and
I'm thinking about what they're going to say at my 
funeral and after - She asked good questions, that's
what the tombstone's going to say - I 
hope so! Yeah. And after I loved my husband and my
friends and my family, the thing that I hoped that 
they would talk about was optimism. I hope that
they would recognize what I'm trying to do here. 
And I'm very curious for you, you've you've been
doing this a long time, it feels like there's 
so much that you've described in this vision
ahead, what would the theme be that you would 
want people to say about what you're trying to do?
Very simply, they made an extraordinary impact. 
I think that we're fortunate because of some
core beliefs a long time ago and sticking with 
those core beliefs and building upon them
we found ourselves today being one of 
the most, one of the many most important and
consequential technology companies in
the world and potentially ever. And so
we take that responsibility very seriously.
We work hard to make sure that
the capabilities that we've created are 
available to large companies as well as
individual researchers and developers, across 
every field of science no matter profitable or
not, big or small, famous or otherwise.
And it's because of this understanding of
the consequential work that we're doing and the 
potential impact it has on so many people
that we want to make make this capability 
as pervasively as possible and I
do think that when we look back in a few 
years, and I do hope that what the
next generation realized is as they, well 
first of all they're going to know us because of
all the you know gaming technology we create.
I do think that we'll look back and the whole
field of digital biology and life sciences has 
been transformed. Our whole understanding of of
material sciences has completely been 
revolutionized. That robots are helping
us do dangerous and mundane things all over the 
place. That if we wanted to drive we can drive
but otherwise you know take a nap or enjoy 
your car like it's a home theater of yours,
you know read from work to home and at that 
point you're hoping that you live far
away and so you could be in a car for longer. 
And you look back and
you realize that there's this company almost at 
the epicenter of all of that and happens
to be the company that
you grew up playing games with.
I hope for that to be
what the next generation learn.
Thank you so much for your time.
I enjoyed it, thank you! I'm glad!