#494 – Jensen Huang: NVIDIA – The $4 Trillion Company & the AI Revolution

Jensen Huang is the co-founder and CEO of NVIDIA, the world’s most valuable company and the engine powering the AI computing revolution. Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep494-sc See below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc. Transcript: https://lexfridman.com/jensen-huang-transcript CONTACT LEX: Feedback – give feedback to Lex: https://lexfridman.com/survey AMA – submit questions, videos or call-in: https://lexfridman.com/ama Hiring – join our team: https://lexfridman.com/hiring Other – other ways to get in touch: https://lexfridman.com/contact EPISODE LINKS: NVIDIA: https://nvidia.com NVIDIA on X: https://x.com/nvidia NVIDIA AI on X: https://x.com/NVIDIAAI NVIDIA on YouTube: https://youtube.com/@nvidia NVIDIA on Instagram: https://www.instagram.com/nvidia/ NVIDIA on LinkedIn: https://www.linkedin.com/company/nvidia/ NVIDIA on Facebook: https://www.facebook.com/NVIDIA/ NVIDIA on GitHub: https://github.com/NVIDIA Nemotron: https://developer.nvidia.com/nemotron SPONSORS: To support this podcast, check out our sponsors & get discounts: Perplexity: AI-powered answer engine. Go to https://perplexity.ai/ https://lexfridman.com/s/perplexity-ep494-sc Shopify: Sell stuff online. Go to https://shopify.com/lex https://lexfridman.com/s/shopify-ep494-sc LMNT: Zero-sugar electrolyte drink mix. Go to https://drinkLMNT.com/lex https://lexfridman.com/s/lmnt-ep494-sc Fin: AI agent for customer service. Go to https://fin.ai/lex https://lexfridman.com/s/fin-ep494-sc Quo: Phone system (calls, texts, contacts) for businesses. Go to https://quo.com/lex https://lexfridman.com/s/quo-ep494-sc OUTLINE: (00:00) – Introduction (00:26) – Sponsors, Comments, and Reflections (06:34) – Extreme co-design and rack-scale engineering (09:20) – How Jensen runs NVIDIA (28:41) – AI scaling laws (43:41) – Biggest blockers to AI scaling laws (45:25) – Supply chain (47:20) – Memory (53:25) – Power (58:45) – Elon and Colossus (1:02:13) – Jensen’s approach to engineering and leadership (1:07:38) – China (1:15:51) – TSMC and Taiwan (1:21:06) – NVIDIA’s moat (1:26:43) – AI data centers in space (1:30:31) – Will NVIDIA be worth $10 trillion? (1:40:40) – Leadership under pressure (1:54:26) – Video games (2:01:18) – AGI timeline (2:03:31) – Future of programming (2:17:02) – Consciousness (2:23:23) – Mortality PODCAST LINKS: – Podcast Website: https://lexfridman.com/podcast – Apple Podcasts: https://apple.co/2lwqZIr – Spotify: https://spoti.fi/2nEwCF8 – RSS: https://lexfridman.com/feed/podcast/ – Podcast Playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4 – Clips Channel: https://www.youtube.com/lexclips

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Lex Fridman?

  The following is a conversation with Jensen Huang, CEO of Nvidia, one of the most important and influential companies in the history of human civilization. Nvidia is the engine powering the AI revolution, and a lot of its success can be directly attributed to Jensen's sheer force of will and his many brilliant bets and decisions as a leader, engineer, and innovator. And now, a quick few-second mention of his sponsor. Check them out in the description or at lexfridman.com/sponsors. It is, in fact, the best way to support this podcast. We've got Shopify for selling stuff online, Element for electrolytes, Finn for customer

• 0:41Lex Fridman?

  service AI agents, Quo for a phone system like calls, texts, contacts for your business, and Perplexity for curiosity-driven knowledge exploration. Choose one of them, my friends. And now, onto the full ad reads. I try to make them interesting, but if you skip, please still check out our sponsors. I enjoy their stuff, maybe you will too. To get in touch with me, for whatever reason, go to lexfridman.com/contact. All right, let's go. This episode is brought to you by Shopify, a platform designed for anyone to sell anywhere with a great-looking online store. Now, I know it's an incredible platform for selling stuff. It's a mechanism by which you can buy stuff on the internet. But the thing I like to celebrate is engineering.

• 1:26Lex Fridman?

  They just recently tweeted about SimJim, which runs simulated shopping sessions by the hundreds of thousands daily. I personally love the idea that things at scale, especially now with the LLM models, can be simulated. You basically want to be simulating human behavior, human decision-making, human choice. And this particular context, of course, is shopping. It's really fascinating. And they describe in their blog post how they're leveraging Nvidia Stack to accomplish this task. But you should know, in general, that you can sign up for a $1 per month trial period at Shopify.com/lex. That's all lowercase.

• 2:10Lex Fridman?

  Go to Shopify.com/lex to take your business to the next level today. This episode is also brought to you by Element, my daily zero-sugar delicious electrolyte mix that, as far as I know, has very little to do with the artificial intelligence and GPUs and CPUs and the revolution that we're experiencing in the tech sector. And I think that's beautiful because I got in the chance to train a bunch of world-class fighters, wrestlers, grapplers recently. I'm going to be traveling to parts of the world that doesn't really have much. And I think in those parts of the world is where the mind can reconnect with the things that are truly important, that are truly timeless.

• 2:53Lex Fridman?

  Anyway, in those parts of the world, I often get pretty out there in terms of physical strain and diet and dehydration and so on. So Element's one of the crucial things in my bag, really water and salt. And really nice, delicious, well-balanced salts, meaning sodium, potassium, magnesium electrolytes. Element is my go-to. Watermelon salt, my favorite flavor. Get a free eight-count sample pack with any purchase. Try it at drinkelement.com/lex. This episode is also brought to you by Finn, a powerful AI system that focuses on customer service.

• 3:38Lex Fridman?

  It's trusted by over 6,000 companies. It has a 65 average resolution rate and is built to handle complex, multi-step queries like returns, exchanges, and disputes. This is such a fascinating problem because customer problems, the bulk of them, fall into a very specific set of categories. But there's nuanced details within those categories that make all the difference. And it can be an incredibly frustrating thing for a human being like myself, I swear. I promise, definitely not a robot. Wouldn't tell you if I was, but it's frustrating for, for a human to come to

• 4:19Lex Fridman?

  the customer service process and to know that your problem kinda is like this problem. There is all these details that you can provide about the system you're operating on, the specifics of the puzzle you're trying to solve, but there's details that you just know in your gut that this is, this is important, especially if you kind of thought through the problem. I've been through this quite a bit. You want to have some level of personalization that can get to the tricky aspect, the perspective on the problem that really would lead you down the road to a solution. Anyway, love this problem. Really glad Finn is focusing on it. Go to fin.ai/lex to learn more about transforming your customer service and scaling your support team. That's fin.ai/lex.

• 5:05Lex Fridman?

  This episode is also brought to you by QUO, spelled Q U O, also known as a company which is three letters will win you a game of Scrabble. That is not a joke. It feels like a joke I have made before, but let's run with it. It's a dad joke. It's a bad dad joke. The only, the only thing worse than a dad joke is a bad dad joke. But here we go. Spelled Q U O, a business phone platform for calling and messaging. Basically, you have a bunch of people trying to help a larger group of people and you want to orchestrate how they communicate with each other. And this is just the system that does it extremely well. Period. QUO integrates AI into the whole shebang,

• 5:50Lex Fridman?

  organizing everything, generating summaries, highlighting the next steps, all that kind of stuff. It just does it well. The interface on top of the AI is also really strong. So try QUO for free, plus get 20% off your first six months when you go to quo.com/lex. That's Q U O dot com/lex. This is the Lex Fridman Podcast, and now, dear friends, here's Johnson Huang.

• 6:34Lex Fridman?

  You've propelled Nvidia into a new era in AI, moving beyond its focus on chip-scale design to now rack-scale design. And I think it's fair to say that winning for Nvidia for a long time used to be about building the best GPU possible. And you still do. But now you've expanded that to extreme co-design of GPU, CPU, memory, networking, storage, power, cooling, software, the rack itself, the pod that you've announced, and even the data center. So let's talk about extreme co-design. What is the hardest part of co-designing a system with that many complex components and design variables?

• 7:12Speaker 01

  Yeah, thanks for that question. So first of all, the reason why extreme co-design is necessary is because the problem no longer fits inside one computer to be accelerated by one GPU. The problem that you're trying to solve is you would like to go faster than the number of computers that you add. So you added 10,000 computers, but you would like it to go a million times faster. Then, all of a sudden, you have to take the algorithm, you have to break up the algorithm, you have to refactor it, you have to shard the pipeline, you have to shard the data, you have to shard the model.

• 7:53Speaker 01

  Now, all of a sudden, when you distribute the problem this way, not just scaling up the problem, but you're distributing the problem, then everything gets in the way. This is the Amdahl's Law problem, where the amount of speedup you have for something depends on how much of the total workload it is. And so if computation represents 50% of the problem, and I sped up computation infinitely, like a million times, I only sped up the total workload by a factor of two. Now, all of a sudden, not only do you have to distribute the computation, you have to shard the pipeline somehow,

• 8:36Speaker 01

  you also have to solve the networking problem because you've got all of these computers are all connected together. And so distributed computing at the scale that we do...

• 8:48Speaker 01

  The CPU is a problem, the GPU is a problem, the networking is a problem, the switching is a problem.

• 8:55Speaker 01

  And distributing the workload across all these computers are a problem. It's just a massively complex computer science problem. And so we just got to bring every technology to bear. Otherwise, we scale up linearly, or we scale up based on the capabilities of Moore's law, which has largely slowed because Dernard scaling has slowed.

• 9:18Lex Fridman?

  I'm sure there's trade offs there. Plus, you have a completely disparate disciplines here. I'm sure you have specialists in each one of these, high bandwidth memory, the network and the NVLink, the NICS, the optics and the copper that you're doing, the power delivery, the cooling, all of that. I mean, there's like world experts in each of those. How do you get them in a room together to figure out?

• 9:36Speaker 01

  That's why my staff is so large.

• 9:39Lex Fridman?

  What's the process? Can you take me through the process of the specialists and the journalists? Like, how do you put together the rack when you know the set of things you have to shove into a rack together? Like, what does that process look like of designing it all together?

• 9:52Speaker 01

  There's the first question, which is, what is extreme code design? You're optimizing across the entire stack of software, from architectures to chips to systems to system software to the algorithms to the applications. That's one layer. The second thing that you and I just talked about is goes beyond CPUs and GPUs and networking chips and scale up switches and scale out switches. And then, of course, you got to include power and cooling and all of that because, you know, all these computers are extremely, extremely power hungry. They do a lot of work and they're very energy efficient, but they, in aggregate, still consume a lot of power. And so that's one, the first question is, what is it?

• 10:36Speaker 01

  The second question is, why is it? And we just spoke about the reason, you know, you want to distribute the workload so that you can exceed the benefit of just increasing the number of computers. And then the third question is, how is it? How do you do it? And that's the, that's kind of the miracle of this company. You know, when you're designing a computer, you have to have an operating system of computers. When you're designing a company, you should first think about what is it that you want the company to produce? You know, I see a lot of companies' organization charts and they all look the same. Hamburger organization charts, software organization charts, and car company organization charts, they all look the same. And it doesn't make any sense to me.

• 11:20Speaker 01

  You know, the goal of a company is to be the machinery, the mechanism, the system that produces the output. And that output is the product that we like to create. It is also designed, the architecture of the company should reflect the environment by which it exists. It almost directly says what you should do with the organization. My direct staff is 60 people. You know, I don't have one on ones with them because it's impossible. You can't have, you can't have 60 people on your staff if you're, you know, gonna get work done and-

• 11:52Lex Fridman?

  So you still have 60 reports, you still have a cross-

• 11:55Speaker 01

  More, yeah.

• 11:55Lex Fridman?

  More.

• 11:56Speaker 01

  Yeah.

• 11:57Lex Fridman?

  And most stars at least have a foot in engineering.

• 12:01Speaker 01

  Almost all of them. There's experts in memory, there's experts in CPUs, there's experts in optical, all, yeah, GPUs and architecture, algorithms, design.

• 12:13Lex Fridman?

  So you constantly have an eye on the entire stack? And you're having, do like intense discussions about the design of the entire stack?

• 12:20Speaker 01

  And no conversation is ever one person. That's why I don't do one on ones. We present a problem and all of us attack it. You know, because we're doing extreme code design. And literally the company is doing extreme code design all the time.

• 12:35Lex Fridman?

  So even if you're talking about a particular component, like cooling, networking, everybody's listening in-

• 12:42Speaker 01

  Yeah, exactly.

• 12:43Lex Fridman?

  And they can contribute, well, this doesn't work for the, for the power distribution. This doesn't, this doesn't work for the, for the memory. This doesn't work for this.

• 12:51Speaker 01

  Exactly. And whoever wants to tune out, tune out. You know what I'm saying?

• 12:55Lex Fridman?

  Yeah.

• 12:55Speaker 01

  And the reason for that is because, because the people who are on the staff, they, they know when to pay attention. They're supposed, you know, and something they could have contributed to, they didn't contribute to, I'm going to call them out. You know, and so, hey, come on, let's get in here.

• 13:08Lex Fridman?

  So as you mentioned, Nvidia is this company that's adapting to the environment. So which point can you say? Did the environment change and began, began adapting sort of secretly in the early days from GPU for gaming, maybe the early deep learning revolution to we're now going to start thinking of it as an AI factory. What does Nvidia do? It produces AI. Let's build a factory that makes AI.

• 13:33Speaker 01

  You, I could reason through it just systematically. We started out as an accelerator company. But the problem with accelerators is that the application domain is too narrow. It has the benefit of being incredibly optimized for the job. You know, any specialist has that benefit. The problem with intense specialization is that, of course, your market reach is narrower. But that's, that's even fine. The problem is the market size also dictates your R and D capacity. And your R and D capacity ultimately dictates the influence and impact that you can possibly have in computing.

• 14:16Speaker 01

  And so when we first started out in Excel, as an accelerator, very specific accelerator. We always, we always knew that that had, that was going to be our first step. We have to find a way to become accelerated computing. But the problem is when you become a computing company. It's too general purpose and it takes away from your specialization. I connected two words that are actually have fundamental tension. The better computing company we become, the worse we became as a specialist. The more of a specialist, the less capacity we have to do overall computing. And so that and I connected those two words together on purpose that the company

• 14:55Speaker 01

  has to find that really narrow path, step by step by step, to expand our aperture of computing, but not give up on the most important specialization that we had. Okay, so the first step that we took beyond acceleration was we invented the programmable pixel shader. So that was the first step towards programmability. You know, it was our first journey towards moving into the world of computing. The second thing that we did was we we created, we put FP32 into our shaders. That FP32 step, IEEE compatible FP32, was a huge step in the direction of computing.

• 15:38Speaker 01

  It was the reason why all of the people who were working on on stream processors and, you know, other types of data flow processors discovered us. And they say, hey, all of a sudden, you know, we might be able to use these GPUs as incredibly computationally intensive, and it's now, you know, compliant with IEEE. I can take my software that I was writing, you know, previously on CPUs, and I can, you know, see about about, you know, using the GPU for them. And which led us to create, put C on top of FP32, what's called, we call CG. That CG path took us to eventually CUDA, CUDA, step by step by step.

• 16:20Speaker 01

  Well, putting CUDA on GeForce, that that was a strategic decision that was very, very hard to do, because it cost the company enormous amounts of our profits, and we couldn't afford it at the time. But we did it anyways because we wanted to be a computing company. A computing company has a computing architecture, a computing architecture has to be compatible across all of the chips that we build.

• 16:44Lex Fridman?

  Can you can you take me through that decision? So putting CUDA on GeForce could not afford to do. Can you explain that decision? Why why boldly choose to do that anyway? Can you explain that decision?

• 16:55Speaker 01

  Excellent. That was that was the first, I would I would say that that was the first.

• 17:03Speaker 01

  The first strategic decision that is as close to an existential threat.

• 17:08Lex Fridman?

  For people who don't know, it turned out to be, spoiler alert, one of the most incredibly brilliant decisions ever made by a company. So CUDA turned out to be an incredible foundation for computation in this AI infrastructure world. So just setting the context, it turned out to be a good decision.

• 17:28Speaker 01

  Yeah, it turned out to have been a good decision. I think the, so here's the way it went. So we invented this thing called CUDA and it expanded the aperture of applications that we can accelerate with our accelerator. The question is, how do we, how do we attract developers to CUDA? Because a computing platform is all about developers. And developers

• 17:56Speaker 01

  don't come to a computing platform just because, you know, it could perform something interesting. They come to a computing platform because the install base is large. Because a developer, like anybody else, wants to develop software that reaches a lot of people. So the install base is in fact the single most important part of an architecture. The architecture could attract enormous amounts of criticism. For example, no architecture has ever attracted more criticism than the x86. You know, as a less than elegant architecture. But yet, it is the defining architecture of today. It gives you an example that in fact so many RISC architectures,

• 18:39Speaker 01

  which were beautifully architected, incredibly well-designed by some of the brightest computer scientists in the world, largely failed. And so I gave you two examples where one is, you know, one is elegant, the other one is barely aesthetic. And so yet x86 survived.

• 18:59Lex Fridman?

  Install base is everything.

• 19:00Speaker 01

  Install base defines an architecture. Not, everything else is secondary. Okay, and so there were other architectures at the time. CUDA came out, OpenCL was here. There were, you know, there's several other computing architectures. But the thing that, the decision that we made that was good was we said, hey, look, ultimately it's about install base. And what is the best way we could get a new computing architecture into the world?

• 19:28Speaker 01

  By that timeframe, GeForce had become successful. We were already selling millions and millions of GeForce GPUs a year. And we said, you know, we had to put CUDA on GeForce and put it into every single PC, whether customers use it or not, and use it as a starting point of cultivating our install base. Meanwhile, we'll go and attract developers and went to universities and wrote books and taught classes and put CUDA everywhere. And eventually people discover, and at the time, the PC was the primary computing vehicle. There was no cloud and we could put a supercomputer in the hands of every researcher in school,

• 20:10Speaker 01

  every scientist, you know, every engineering school, or every student in school. And eventually, something amazing will happen. Well, the problem was CUDA increased our cost of that GPU, which is a consumer product, so tremendously, it completely consumed all of the company's gross profit dollars. And so at the time, the company was probably, you know, worth, I don't know, at the time, eight, was it like eight billion dollars or something, six, seven billion dollars or something like that. After we launched CUDA, I recognized that it was going to add so much cost, but it was something we believed in. You know, our market cap went down to like one and a half billion dollars.

• 20:55Speaker 01

  And so we were down, we were down there for a while and, and we clawed our way back slowly. But we carried CUDA on GeForce. I always say that Nvidia is the house that GeForce built, because it was GeForce that took CUDA out to everybody. Researchers, scientists, they discovered CUDA on GeForce, because they were all, you know, many of them were gamers. Many of them built their own PCs anyways. In a university lab, many of them built clusters themselves, you know, using, using PC components. And so that, you know, that's kind of how we got going.

• 21:33Lex Fridman?

  And then that became the platform and the foundation for the deep learning revolution.

• 21:36Speaker 01

  That was also another great, great observation, yeah.

• 21:39Lex Fridman?

  That existential moment, do you remember, like, what were those meetings like? What were those discussions like, deciding as a company, risking everything?

• 21:50Speaker 01

  Well, I had to make it clear to the board what we're trying to do. And the management team knew our gross margins were going to get crushed. So you could imagine a world where GeForce would carry the burden of CUDA, and none of the gamers would appreciate it, and none of the gamers would pay for it. You know, they only pay certain price, and it doesn't matter what your cost is. And so we increased our costs by 50%, and that consumed, and we were a 35% gross margin company. And so it was quite a difficult decision to make. But you could imagine that someday this would go into workstations, and it would go into supercomputers.

• 22:35Speaker 01

  And in those segments, maybe we can capture more margin.

• 22:39Speaker 01

  So you could reason your way into being able to afford this, but it still took a decade.

• 22:46Lex Fridman?

  But that's more like conversation with the board, convincing them. But you psychologically, because Nvidia has continued to make

• 22:55Lex Fridman?

  bold bets that predict the future, and in part, especially now, define the future. So I'm almost looking for wisdom about how you were able to make those decisions, to make leaps like that as a company.

• 23:16Speaker 01

  Well, first of all, I'm informed by a lot of curiosity. At some point, there's a reasoning system that convinces me so clearly this outcome will happen,

• 23:34Speaker 01

  that this will happen. And so I believe it in my mind. And when I believe it in my mind, you know how it is. You manifest a future, and that future is so convincing. There's no way it won't happen. There's a lot of suffering in between. But you've got to believe what you believe.

• 23:54Lex Fridman?

  So you envision the future.

• 23:57Speaker 01

  Yeah.

• 23:57Lex Fridman?

  And you essentially, from an engineering perspective, manifest it.

• 24:01Speaker 01

  Yeah, and you reason about how to get there. You reason about why it must exist.

• 24:08Speaker 01

  And, you know, I reason, we all reason here. The management team will reason about it, all the people that, like, we spent a lot of time reasoning about it. The thing, the thing that, the next part of it is probably a skill thing, which is, you know, oftentimes in leadership, the leadership stays quiet or they learn about something and then they do some manifesto. And it's a brand new year and somehow at the end of the year, next year, we're going to have a brand new plan. Big, huge layoff this way, big, huge organization change this way, new mission statement, brand new logos, you know, that kind of stuff.

• 24:45Lex Fridman?

  Yeah.

• 24:46Speaker 01

  We just never, I never do things that way. When I learn about something and it's starting to influence how I think, I'll make it very clear to everybody near me that, you know, this is interesting.

• 24:59Speaker 01

  This is going to make a difference. This is going to impact that. And I reason about things step by step by step. Oftentimes, I've already made up my mind, but I'll take every possible opportunity. External information, new insights, new discoveries, new engineering, you know, revelations, new milestones developed, I'll take those opportunities and I'll use it to shape everybody else's belief system. And I'm doing that literally every single day. I'm doing that with my board, I'm doing that with my management team, I'm doing that with my employees. I'm trying to shape their belief system such that when I come, the day I say, "Hey, let's buy Melanox."

• 25:45Speaker 01

  It's completely obvious to everybody that we absolutely should. On the day that, on the day that I, that I said, hey, guys, let's go all in on deep learning. And let me tell you why. I've already been laying down the bricks to different organizations inside the company. Every organization and every, everybody, many of the people might have heard everything. Most of the company heard, hears of course, pieces of it. And on the day that I announce it,

• 26:17Speaker 01

  everybody's kind of bought into many pieces of it. And in a lot of ways, I like to announce these things. And I imagine that, that the employees are kind of saying, you know, Jensen, what took you so long? And, and in fact, I've been shaping their belief system for some time. And therefore, leadership, sometimes it looks like you're leading from behind. But you've been shaping their, you know, to the point where on the day that I declared it, a hundred percent buy-in. But that's what you want. You want to bring everybody along. You know, otherwise we announce something about deep learning and everybody goes, what are you talking about? You know, you announce something about, let's go all in on this thing.

• 26:57Speaker 01

  And, and your, your management team, your board, your employees, your customers, they're kind of like, where's this coming from? You know, this is insane. And so, so GTC, in fact, if you go back in time, you look at, look at the keynotes.

• 27:13Speaker 01

  I'm also shaping the belief system of my partners and the industry. And, and I'm using that to shape, you know, the belief system of my own employees. And, and, and so by the time that I announce something, like for example, we just now, we just announced Grok. We've been late, I've been talking about the stepping stones for two and a half years.

• 27:35Speaker 01

  You just go back and you, oh my gosh, they've been talking about it for two and a half years. And so I've been laying the foundation step by step by step. So when the time comes, you announce it, everybody's, you know, what took you so long?

• 27:46Lex Fridman?

  But it's not just inside the company, you're shaping the landscape, the broader global landscape of innovation. Like putting those ideas out there, you really are manifesting reality.

• 27:54Speaker 01

  We don't build computers. We actually don't build clouds. We don't, as it turns out, we're a computing platform company. And so nobody can buy anything from us. That's the weird thing. You know, we, we vertically. Design vertically integrate to design and optimize. But then we open up the entire platform at every single layer to be integrated into other companies, products and services and clouds and supercomputers and OEM computers. And so the amazing thing is I can't do what I do without having convinced them first. And so most of GTC is about manifesting a future that by the time that we my product

• 28:36Speaker 01

  is ready, they're going, what took you so long?

• 28:40Lex Fridman?

  Yeah. So one of the things you've been a believer for a long time is scaling laws broadly defined. So are you still a believer in the in the scaling laws?

• 28:51Speaker 01

  Yeah, we have more scaling laws now.

• 28:53Lex Fridman?

  So I think you've outlined four of them with pre-training, post-training, test time and agentic scaling. What do you think when you think about the future, deep future and the near term future? What are the blockers that you're most concerned about to keep you up at night that you have to overcome in order to keep scaling?

• 29:14Speaker 01

  Well, we can go back and reflect on what people thought were blockers. So in the beginning, we were the first, the pre-training scaling law. You know, people thought, well, rightfully so, that the amount of data that we have, high quality data that we have, will limit the intelligence that we achieve. And that scaling law was an important, very important scaling law. The larger the model, the correspondingly more data results in a better, with a results in a smarter AI. And so that was pre-training and Elias, Suscover, Elias said, we're out of data or something like that, pre-training is over or something like that. The industry panicked, you know, that this is the end of AI. And of course, of course, that's obviously not true.

• 29:59Speaker 01

  We're going to keep on scaling the amount of data that we have to train with. A lot of that data is probably going to be synthetic. And that also confused people, you know? And and what people don't realize is that they've kind of forgotten that most of the data that that we are training, that we teach each other with, inform each other with is synthetic. You know, I it's synthetic because

• 30:23Speaker 01

  It didn't come out of nature. You created it, I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it. And so, so we've now reached a level where AI is able to take ground truth, augment it, enhance it, synthetically generate an enormous amount of data. And that part of post-training continues to scale. And so the amount of data that we could use that is human-generated will be smaller and smaller and smaller. The amount of data that we use to train model is going to continue to scale to the point where we're no longer limited.

• 31:08Speaker 01

  Training is no longer limited by data, it's now limited by compute. And the reason for that is most of the data is synthetic. Then the next phase is test time. And I still remember people, people telling me that inference, oh yeah, that's easy. Pre-training, that's hard. These are giant systems that people are talking about. Inference must be easy. And so inference chips are going to be little tiny chips. And, you know, they're not, they're not like Nvidia's chips. Oh, those are going to be complicated and expensive. And, you know, we could make, and this is, and in the future, inference is going to be the biggest market and it's going to be easy. And we're going to commoditize it. You know, everybody can build their own chips. And, and that was always illogical to me because

• 31:53Speaker 01

  inference is thinking. And I think thinking is hard. Thinking is way harder than reading. You know, pre-training is just memorization and generalization, you know, and looking for patterns and relationships. You're, you're reading and reading versus thinking, reasoning, solving problems, taking un-

• 32:16Speaker 01

  Unexplored experiences, new experiences, and breaking it down into decomposing it into, you know, solvable pieces that we then go off either through first principle reasoning or, you know, through through previous examples, prior experiences, you know, or or just exploration and and search and, you know, trying different things. And that whole process of post of of test time scaling, inference is really about thinking. And and it's about reasoning. It's about planning. It's about search. It's about and so how could that possibly be compute light? And we were absolutely right about that, you know.

• 32:57Speaker 01

  So so test time scaling is intensely compute intensive. Then the question is, OK, now we're at inference and we're at test time scaling. What's beyond that? Well, obviously, we have now created, you know, one agentic person. And that one agentic person has a large language model that we've now we've now, you know, developed. But during test time, that agentic system goes off and does research and bangs on databases. And it goes and, you know, uses tools. And one of the most important things it does is spins off and spawns off a whole bunch of subagents, which means we're now creating large teams. It's so much easier to scale Nvidia

• 33:41Speaker 01

  by hiring more employees than it is to scale myself. And so the next scaling law is the agentic scaling law. It's kind of like multiplying AI. Multiplying AI, we could spin off agents as fast as you want to spin off agents. And so, you know, I you now have four scaling laws. And and as we use the agentic systems, they're going to create a lot more data. They're going to create a lot of experiences. Some of it, we're going to say, wow, this is really good. We ought to memorize this. That data set then comes all the way back to pre-training. We memorize and generalize it. We then refine it and fine tune it.

• 34:22Speaker 01

  Back into post-training, then we enhance it even more with test time, you know, and the agents, agents and agent systems, you know, put it on to the industry. And so this loop, the cycle is going to go on and on and on. It kind of comes down to basically, intelligence is going to scale by one thing and it's compute.

• 34:43Lex Fridman?

  But there's a tricky thing there that you have to anticipate and predict, which is some of these components, it requires different kind of hardware to really do it optimally. So you have to anticipate where the AI innovation is going to lead. For example, mixture of experts with sparsity.

• 35:01Speaker 01

  Perfect.

• 35:01Lex Fridman?

  With hardware, you can't just pivot on a week's notice. You have to anticipate what that's going to look like.

• 35:07Speaker 01

  So good.

• 35:08Lex Fridman?

  That's so scary and difficult to do, right?

• 35:11Speaker 01

  For example, these AI model architectures are being invented about once every six months.

• 35:17Lex Fridman?

  Yeah.

• 35:18Speaker 01

  Right? And system architectures and hardware architectures kind of every three years. And so you need to anticipate what likely is going to happen, you know, two, three years from now. And there's a couple of ways that you could do that. First of all, we could do research internally ourselves. And that's one of the reasons why we have basic research. We have applied research. We create our own models. And so we have, we have hands-on life experience right here. This is part of the code design that I'm talking about. We're also the only AI company in the world that works with literally every AI company in the world. And to the extent that we can, we try to get a sense of, of what are the challenges that people are experiencing.

• 36:00Lex Fridman?

  So you're listening to the whispers across the industry, the alebs?

• 36:04Speaker 01

  That's right. You've got to listen and, and learn from everybody and have a, have a. And then the, the last part is to have an architecture that's, that's flexible, that can adapt and move with the wind. And one of the benefits of, of Kuda is that it's, you know, on the one hand, An incredible accelerator, on the other hand, it's really flexible. And so that balance, incredible balance between specialization, otherwise we can't accelerate the CPU, versus generalization so that we can adapt with changing algorithms. That's really, really important. That's the reason why CUDA has been so resilient on the one hand. And yet we continue to enhance it. We're at CUDA 13.2.

• 36:47Speaker 01

  And so we're evolving the architecture so fast that we can stay with the modern algorithms. For example, when mixture experts came out, that's the reason why we had NVLink 72 instead of NVLink 8. We could now take an entire 4 trillion, 10 trillion parameter model and put it in one computing domain as if it's running on one GPU.

• 37:17Speaker 01

  People probably didn't notice I said it, but if you look at the architecture of the Grace Blackwell racks, it was completely focused on doing one thing, processing the LLM. All of a sudden, one year later, you're looking at a Vera Rubin rack. It has storage accelerators. It has this incredible new CPU called Vera. It has Vera Rubin and NVLink 72 to run the LLMs. It also has this new additional rack called GRAC. And so this entire rack system is completely different than the previous one. And it's got all these new components in it.

• 38:00Speaker 01

  And the reason for that is because the last one was designed to run MOE large language models inference. And this one is to run agents and agents bang on tools.

• 38:11Lex Fridman?

  Obviously, the design of the system had to have been done before Claude Code, Codex, Open Claw. You were anticipating the future, essentially.

• 38:23Speaker 01

  Yeah.

• 38:23Lex Fridman?

  And that comes from what? From the whispers, from the understanding what all the state of the art is?

• 38:27Speaker 01

  No, it's easier than that. You just reason about it.

• 38:33Speaker 01

  First of all, it is reason. No matter, no matter what happens, at some point, in order for that large language model to be a digital worker, let's just, let's just use that metaphor. Let's say that we want the LLM to be a digital worker. What does it have to do? It has to access ground truth. That's our file system. It has to be able to do research. It doesn't know everything. We don't have, and I don't want to wait until this AI becomes universally smart about everything, past, present, and future, before I make it useful. And so therefore, I might as well let it go do research. It's obviously, if it wants to help me, it's got to use my tools.

• 39:14Speaker 01

  You know, a lot of people would say, you know, AI is going to completely destroy software. We don't need software anymore. We don't even need tools anymore. That's ridiculous. Let's, let's use the, let's use a thought experiment. And you could just sit there, enjoy a glass of whiskey, and, and think about all these things, and it would become completely obvious. Like, if I were to create the most amazing role, the most amazing agent that we can imagine in the next 10 years, let's say it be a humanoid robot. If that humanoid robot were to be created, is it more likely that the humanoid robot comes into my house and uses the tools that I have to do the work that it needs to do?

• 39:56Speaker 01

  Or does this hand turns into a 10-pound hammer in one instance, turns into a scalpel in another instance? And in order to boil water, it beams, you know, microwaves out of its fingers? You know, or is it more likely just to use the microwave? You know? And the first time it goes up to the microwave, it probably doesn't know how to use it. But that's okay. It's connected to the internet. It reads the manual of this microwave, reads it, instantly becomes an expert. And so it uses it. And so I, I think the, I just described, in fact, almost all of the. Properties of OpenClaw. You know, that it's going to use tools, that it's going to access files, it's going to be able to do research.

• 40:41Speaker 01

  It has I O subsystem. And when you're done reasoning through it, reasoning about it through it in that way.

• 40:49Speaker 01

  Then you say, oh, my gosh. The impact to the future computing is deeply profound. And the reason for that is I think we've just reinvented the computer. And then now you say, okay, when did we reason about that? When did we reason about OpenClaw? If you take the OpenClaw schematic that I used at G T C. You will find it two years ago.

• 41:13Speaker 01

  Literally two years ago at G T C. I was talking about agentic systems that exactly reflect OpenClaw today. And, and of course, the confluence of, of many things had to happen. First of all, we needed Claude and, and G P T and, you know, all of these models to reach a level of capabilities. So, so their innovation and their breakthroughs and their continued advances was really important. And then, of course, somebody had to create an open source, you know, project that, that was sufficiently robust. You know, sufficiently complete and that we can all, we can all put to, put to work.

• 41:55Speaker 01

  And, and I think OpenClaw did for, did for agentic systems what Chat G P T did for generative systems. And, and I just think it's a very big deal.

• 42:04Lex Fridman?

  Yeah, it's a really special moment. I'm not exactly sure why it captured so much of the world's attention, but it did more than Claude code and Codex and so on.

• 42:14Speaker 01

  Because consumers could reach it.

• 42:15Lex Fridman?

  Sure, yeah, but there's also so much of this is vibes and, and Peter, I had a podcast with him. A wonderful human being. So part of it is also the humans that represent the thing.

• 42:26Speaker 01

  Yeah, no doubt.

• 42:27Lex Fridman?

  Part of it is memes and the, because we're all trying to figure it out. There's really serious and complicated security concerns about when you have such powerful technology. How do you hand over your data so they can do useful stuff? But then there are scary things associated with that. And we as a civilization, as individual people and as a civilization, figuring out how to find that right balance.

• 42:46Speaker 01

  Yeah, we, we, we jumped on it right away and we sent a bunch of security experts this way. And we did this thing called Open Shell. It's, it's already been integrated into, into OpenClaw.

• 42:57Lex Fridman?

  And Nvidia put forward Nemo Claw.

• 43:00Speaker 01

  Yeah, exactly.

• 43:01Lex Fridman?

  They install it super easy. It makes sure that it's secure.

• 43:05Speaker 01

  We give you two out of three rights. Agentic systems can can access sensitive information. It can execute code and it can communicate externally.

• 43:15Speaker 01

  We could keep things safe if we gave you two out of those three capabilities at any time, but not all three. And out of those two out of three capabilities, we also give you access control based on based on whatever rights that you're given by enterprise. And then we connect it to a policy engine that all these enterprises already have. And so we're going to try to do our best to to help Open Claw become a better Claw.

• 43:42Lex Fridman?

  So you eloquently explained how we have a long history of blockers that we thought were going to be blockers and we overcame them. But now looking into the future, what do you think might be the blockers now that it's clear that agents will be everywhere? So it's obviously we're going to need compute. So what is going to be the blocker for that scaling?

• 44:01Speaker 01

  Power is a concern, but it's not the only concern. But that's the reason why we're pushing so hard on Extreme Code Design, so that we can improve the tokens per second, per watt, orders of magnitude every single year. And so in the last 10 years, Moore's Law would have progressed computing about 100 times in the last 10 years. We progressed and scaled up computing by a million times in the last 10 years. And so we're going to keep on, we're going to keep on doing that through Extreme Code Design. So energy efficiency per watt completely affects the revenues of a company. It affects the revenues of a factory. And we're just, we're just going to push that to the limit

• 44:46Speaker 01

  so that we could keep on driving token costs down. As fast as we can. You know, the our computer price is going up, but our token generation effectiveness is going up so much faster. The token cost is coming down. It's just it's it's coming down in order magnitude every year.

• 45:06Lex Fridman?

  So power, that's an interesting one. So the the way to try to get around the power blocker is to try to with the tokens per second per watt, trying to make it more and more efficient. Of course, there's the question of how do we get more power?

• 45:17Speaker 01

  We should also get more power.

• 45:19Lex Fridman?

  That's a really complicated one. You've talked about small module nuclear power plants. There's all kinds of ideas for energy. How much does it keep you up at night? The the bottlenecks in the supply chain of AI like ASML with EUV lithography machines, TSMC with advanced packaging like COAS and SK Hennix with high bandwidth memory?

• 45:40Speaker 01

  All the time and we're working on all the time. No company in history has ever grown at a scale that we're growing while accelerating that growth. It's incredible.

• 45:51Lex Fridman?

  Yeah.

• 45:52Speaker 01

  And it's hard for people to even understand this. In the overall world of AI computing, we're increasing share. And so supply chain upstream and downstream are really important to us. I spend a lot of time informing all the CEOs that I work with. What are the dynamics that's going to cause the growth to continue or even accelerate? It's part of the reasons why to the entire right hand side of me, we're CEOs of practically the entire IT industry upstream and practically the entire infrastructure industry downstream. And there were all there were several hundred CEOs.

• 46:37Speaker 01

  And I don't think there's ever been keynotes where several hundred CEOs show up. And and part of it is I'm telling them about our business condition now. I'm telling them about the growth drivers in the very near future and what's happening. And I'm also describing where are we going to go next? So that they could use all of this information and all of the dynamics that are here to inform how they want to invest. And so so I I inform them that way like I inform my own employees. And then, of course, then I make trips out to them and make sure that, hey, listen, I want you to know this quarter, this coming year, this next year, these things are going to happen. And if you look at the CEOs of the DRAM industry,

• 47:24Speaker 01

  the number one DRAM in the world was DDR memory for CPUs in data centers.

• 47:32Speaker 01

  About three years ago, I was able to convince several of the CEOs that even though at the time HBM memory was used quite scarcely, and barely by supercomputers, that this was going to be a mainstream memory for data centers in the future. And at first, it sounded ridiculous. But several of the CEOs believed me and decided to invest in building HBM memories. Another memory was rather odd to put into a data center is the low-power memories that we use for cell phones. And we wanted them to adapt them for supercomputers in the data center. And they go, cellphone memory for supercomputers? And I explained to them why.

• 48:15Speaker 01

  Well, look at these two memories, LPDDR5, HBM4. The volumes are so incredible. All three of them had record years in history. And these are 45-year companies. And so, you know, that's part of my job is to inform and shape, inspire, you know?

• 48:37Lex Fridman?

  So you're not just manifesting the future and maybe inspiring Nvidia, the different engineers of the company. You're manifesting the supply chain of the future. So you're having conversations with TSMC, with ASML.

• 48:51Speaker 01

  Upstream, downstream.

• 48:53Lex Fridman?

  Upstream, downstream. So that's the thing.

• 48:55Speaker 01

  G E V, Caterpillar. Yeah, that's downstream from us. Yeah, yeah.

• 49:00Lex Fridman?

  Yeah, the whole thing. I mean, but that's so... There's so much incredibly difficult engineering that happens in the entire semiconductor industry. And it just feels scary how intricate the supply chain is, how many components there are. But it works somehow.

• 49:20Speaker 01

  Exactly. The deep science, the deep engineering, the incredible manufacturing. And so much of the manufacturing is already robotics. But we have a couple of hundred suppliers that contribute. The technology that goes into our 1.3 million component rack. Each rack is 1.3, one and a half million components. There are 200 suppliers across the Verarubin rack.

• 49:46Lex Fridman?

  So it's interesting that you don't list that as the thing that keeps you up at night in the list of blockers.

• 49:51Speaker 01

  But I'm doing, I'm doing all the things necessary to.

• 49:54Lex Fridman?

  Okay.

• 49:54Speaker 01

  Yeah, see? I can go to sleep because I checked it off. I said, okay. You know, I go, I can go to sleep and I go, well, let's see. What, um, let's reason about this. What's important for us? Because, okay, let's reason about this. Because we changed the system architecture from the original DGX1 that you remembered to MVLink 72 rack scale computing. What's gonna, what does that, what does that mean? What does that mean to software? What does that mean to engineering? What does that mean to how we design and test? And what does that mean to the supply chain? Well, one of the things that it meant was we moved supercomputer, supercomputer integration at the data center

• 50:38Speaker 01

  into supercomputer manufacturing in the supply chain.

• 50:44Speaker 01

  If you're doing that, you also have to recognize you're going to move one. And if, if you're, if you're, you know, total footprint of whatever data center you're going to build, let's say you would like to have, you know, 50 gigawatts of supercomputers that are running simultaneously. And it takes one week to manufacture that 50 gigawatts of supercomputers. Then each week in the supply chain, the supercomputers are going to need a gigawatt of power. And so, so we're going to need the supply chain to increase the amount of power it has to build, test, to build and test the supercomputers in the supply chain before I ship it. Well, MVLink 72 literally builds supercomputers in the supply chain

• 51:30Speaker 01

  and ships them 2, 3 tons at a time per rack. It used to be, they used to come in parts and we used to assemble them inside data center. But that's impossible now because M V link seventy two is so dense. And so that's an example, and I would have to go into, you know, I've fly into the supply chain, go meet my partners and hey, I said, guess what? So here's what we're going to do with, this is the way we used to build our DGXs. We're going to build them this way. This is going to be so much better because we're going to need them for inference. The market for inference is, you know, coming, the inflection point for inference is coming, it's going to be a big market. And so I first explained to them what's going on, why it's going to happen.

• 52:10Speaker 01

  And then I, then I ask them to make several billion dollars of capital investments each.

• 52:19Speaker 01

  And because they, you know, they trust me and, and I, I'm very respectful of them. And I, I give them every opportunity to question me and I spend time to explain things to people and I reason about it, I draw them pictures and I reason about it in first principles. And by, by, by the time I'm done with them, there's no what to do.

• 52:37Lex Fridman?

  So it's a lot is about relationships and building a shared view of the future.

• 52:41Speaker 01

  Yeah.

• 52:43Lex Fridman?

  But do you worry about certain bottlenecks? I mean, what are the biggest bottlenecks in the supply chain? Are you, are you worried about ASMLs, EV tooling? Are you worried about the, the packaging, cost packaging of TSMC? About how fast it could scale? Like you said, you're not only growing incredibly fast, you're accelerating in growth. So it, it, it feels like every, everybody in the supply chain, and those are certainly bottlenecks, would have to scale up. Are you having conversations with them? Like, how can you scale this faster? Do you worry about it?

• 53:14Speaker 01

  No.

• 53:15Lex Fridman?

  Okay.

• 53:16Speaker 01

  Because, because I told them what I needed, they understood what I need. They told me what they're going to go do and I believe they're what they're going to do.

• 53:23Lex Fridman?

  Interesting.

• 53:24Speaker 01

  Yeah.

• 53:24Lex Fridman?

  That's great to hear. So maybe if we can just linger on the power for a little bit, what are your hopes for how to solve the energy problem?

• 53:32Speaker 01

  One of the areas, like, that I'm, I'm, that I would love, I would love, love us to talk about and just get the message out. You know,

• 53:42Speaker 01

  our, our, our power grid is designed for the worst case condition with some margin.

• 53:50Speaker 01

  Well, 99% of the time, we're nowhere near the worst-case condition because the worst-case condition is a few days in the winter, a few days in the summer and extreme weather. Most of the time, we're nowhere near the worst-case condition and we're probably running around, call it 60% of peak. And so 99% of the time, our power grid has excess power and they're just sitting idle. But they have to be there sitting idle because just in case, when the time comes, hospitals have to be powered and, you know, infrastructure has to be powered and airports have to run and so on and so forth. And so the question that I have is whether we could go and help them understand and

• 54:34Speaker 01

  create contractual agreements and design computer architecture, systems, data centers, such that when they need the maximum power for infrastructure in society, that the data centers would get less. But that's in a very rare instance anyways. And during that time, we either have a backup generator for that little part of it, or we just have our computers shift the workload somewhere else, or we have the computers just run slower. You know, we could degrade our performance, reduce our power consumption and provide for, you know, slightly longer latency response, you know, when somebody asks for, you know, asks for an answer.

• 55:15Speaker 01

  And so I think that that, that way of using computers, of building data centers, instead of expecting 100% uptime and these contracts that are really, really quite rigorous, it's putting a lot of pressure on the grid to be able to, now they're going to have to increase from their maximum. I just want to use their excess.

• 55:36Lex Fridman?

  Mm-hmm.

• 55:37Speaker 01

  It's just sitting there.

• 55:38Lex Fridman?

  Yeah, that's not talked about enough. So what's, what's this, what's stopping there? Is it regulation? Is it bureaucracy?

• 55:44Speaker 01

  I think it's, it's a through-way problem. It starts with the end customer. The end customer puts, puts requirements on the data centers that they can never not be available. Okay. So that the end customer expects perfection. Now, in order to deliver that perfection, you need a combination of backup generators and your grid power supplier to deliver on perfection. And so everybody's got to have six nines.

• 56:15Speaker 01

  Well, I think first of all, right now, we ought to have everybody understand that when the customer asks for these things, you got somebody, you have somebody in your data center operations team disconnected from the CEO. I bet the CEO doesn't know this. I'm going to talk to all the CEOs. Their CEOs are probably not paying any attention to the contracts that are being signed. And so, everybody wants to sign the best contract, of course. And they go down to the cloud service providers and the contract, the two contract negotiators that are, I could just see them now, you know, negotiating these multi-year contracts. Both sides want, you know, the best contract. As a result, the CSPs then have to go down

• 56:58Speaker 01

  to the utilities and they expect the nine, the six nines. And so, I think, I think the first thing is just make sure that, that all of the customers, the CEOs of the customers realize what they're asking for. Now, the second thing is, we have to build data centers that gracefully degrade. And so, if the power, if the utility of the grid tells us, listen, we're going to have to back you down to about 80%, we're going to say, that's no problem at all. We're just going to move our workload around. We're going to make sure that data's never lost, but we can reduce the computing rate and use less energy. The quality of service degrades a little bit. For the critical workloads, I shift that somewhere else right

• 57:38Speaker 01

  away. So I don't have that problem. And so, you know, whoever, whichever data center still has a hundred percent uptime. And so-

• 57:46Lex Fridman?

  How difficult of an engineering problem is that the smart dynamic allocation of power in the data center?

• 57:50Speaker 01

  As soon as you could specify, you could engineer it.

• 57:54Lex Fridman?

  Beautifully put.

• 57:56Speaker 01

  So long as it obeys the laws of physics on first principles, I think we're good.

• 58:00Lex Fridman?

  What was the third thing you were mentioning?

• 58:02Speaker 01

  So the second thing is the data centers. And the third thing is, we need the utilities to also recognize that this is an opportunity. And, and instead of, instead of saying, look, it's going to take me five years to increase my grid capability, if you, if you have, if you're willing to take power of this level of guarantee.

• 58:26Speaker 01

  I can make them available for you next month and at this price. And so if utilities also offered more segments of power delivery promises, then I think everybody will figure out what to do with it. Yeah, but there's just way too much waste in the grid right now. We should go after it.

• 58:45Lex Fridman?

  You've highly lauded Elon's and XAI's accomplishment in Memphis in building Colossus Supercomputer, probably in record time in just four months. It's now at 200,000 GPUs and growing very quickly. Is there something that you could speak to the understand about his approach? That's instructive broadly to all the data center creators that's that enable that kind of accomplishment? His approach to engineering, his approach to the whole management of construction, everything?

• 59:17Speaker 01

  First of all, Elon is deep in so many different topics, yet he's also a really good systems thinker. And so he's able to think through multiple disciplines. And, and he obviously pushes things, questions everything. Whether number one, is it necessary? Number two, does it have to be done this way? And in other words, you know, does it have, does it have to take this long? And, and so, so he, he has, he has, he has the ability to question everything to the point where everything is down to its minimal amount that's necessary. You can't take anything else out.

• 1:00:02Speaker 01

  And, and yet, yet the, the, the, the necessary capabilities of the product retains, you know? And so he's, he is as minimalist as you could possibly imagine. And he does it at a system, system scale. I, I also love the fact that he, he is, he is represented. He, he is, he is present at the point of action.

• 1:00:27Speaker 01

  You know, he'll just go there. If there's a problem, he'll just go there and show me the problem. You know, when you do all of this in combination, you overcome a lot of previous, this is just the way we do it.

• 1:00:41Speaker 01

  You know, I'm, I'm waiting for them. You know, I mean, it's just, everybody has a lot of excuses. And so, and then the last thing is when, when you act personally with so much urgency, it causes everybody else to act with urgency. You know, and, and every supplier has a lot of customers going on. Every supplier has a lot of projects going on. And he, he make, he made, he makes it his business that he's the top priority of everybody else's, you know, projects. And so he, he does that by demonstrating it.

• 1:01:11Lex Fridman?

  Yeah, I've been in a bunch of those meetings. It's, it's fun to watch because really not enough people ask the question like, okay, so can this be done a lot faster and how? Why does it have to take this long?

• 1:01:23Speaker 01

  Yeah, right.

• 1:01:24Lex Fridman?

  And then that becomes an engineering question often. And yes, I think when you get the ground truth of actually, I remember one of the times I was hanging out with him, he literally is going through the entire process of how to plug in cables into a rack. And he's, I was working with an engineer on the ground that's doing that task. And he's just trying to understand what does that process look like? So it can be less air prone. And just building up that intuition from every single task involved in putting together the data center, you start to immediately get a sense at the detailed scale and then at the broad system scale of where the inefficiencies are. And so you can make it more and more and more efficient. Plus you have the big hammer of being able to say, let's do it totally different.

• 1:02:10Speaker 01

  Yeah.

• 1:02:10Lex Fridman?

  And remove all possible blockers.

• 1:02:12Speaker 01

  That's right.

• 1:02:13Lex Fridman?

  Is there parallels in the Nvidia Extreme Systems code design approach that you see in the way Elon approaches systems engineering?

• 1:02:20Speaker 01

  Well, first of all, co-design is a ultimate systems engineering problem. Yeah. And so we approach, we approach the work that we do from that first, from that principle. The other thing that we do, and this is, this is a philosophy that, a thought, a state of mind, I guess, a method that I started 30 years ago, and it's called the speed of light. A speed of light is not just about the speed. Speed of light is my shorthand for what's, what's the limit of what physics can do? And so every single, everything, everything that we do is compared against the speed of light.

• 1:02:59Speaker 01

  Memory speed, math speed, power, cost, time, effort, number of people, manufacturing cycle time. And when you think about latency versus throughput, when you think about cost versus throughput, cost versus capacity, all of these things, you test against the speed of light to achieve all of these different constraints separately. And then when you consider it together, you know, you have to make compromises because a system that achieves extremely low latency versus a cheat, a system that achieves very high throughput are architected fundamentally differently.

• 1:03:43Speaker 01

  But you want to know what's the speed of light of a system that achieves high throughput? What's the speed of light of a system that achieves low latency? And then when you think about the total system, you could make trade offs. And so I, I force everybody to think about what's this, what the first principles, the limits, the physical limits for everything before we, you know, before we do anything. And, and we test everything against that. And so that's a good frame of mind. I don't love the other methods, which is continuous improvement.

• 1:04:21Speaker 01

  The problem with continuous improvement, it First of all, you should engineer something from first principles at the speed, you know, with speed of light thinking, limited only by physical limits and physics limits. And after that, of course, you would improve it over time.

• 1:04:42Speaker 01

  But I don't like going into a problem and somebody says, hey, you know, it takes 74 days to do this today, right now, and we can do it for you in 72 days. You know, I'd rather strip it all back to zero and say, first of all, explain to me why it's 74 days in the first place. And let's know, let's think about what's possible today. And if I were to to build it completely from scratch, you know, how long would it take? Oftentimes, you'd be surprised and it might come to six days. Now, the rest of the six days to 74 could be very well reasoned and compromises and, you know, cost reductions and all kinds of different things, but at least you know what they are.

• 1:05:22Speaker 01

  And then now that you know that six days is possible, then the conversation from 74 to six, surprisingly much more effective.

• 1:05:31Lex Fridman?

  In such incredibly complex systems that you're working with, is simplicity sometimes a good heuristic to, to reach for? I mean, if I can just,

• 1:05:42Lex Fridman?

  I mean, the pod, the Vera Rubin pod that you announced is just incredible. We're talking about seven chips, seven chip types, five purpose-built rack types, 40 racks, 1.2 quadrillion transistors, nearly 20,000 Nvidia dies, over 1,100 Ruben GPUs, 60 exaflops, 10 petabytes per second of scale bandwidth. That's all just one.

• 1:06:05Speaker 01

  That's just one pod.

• 1:06:06Lex Fridman?

  That's just.

• 1:06:08Speaker 01

  Yeah, that's just one pod.

• 1:06:09Lex Fridman?

  I mean, so you have the, and then even the NVL 72 rack alone is 1.3 million components, 1,300 chips, 4,000 pods crammed into a single 19-inch wide rack.

• 1:06:21Speaker 01

  And Lex, we'll probably kind of crank out about 200 of these pods a week, just to put in perspective.

• 1:06:27Lex Fridman?

  The, the amount of different components, I suppose simplicity is impossible, but is that a metric that you kind of? Reach for and trying to design things.

• 1:06:37Speaker 01

  You know, the phrase, the phrase that I use most often is we, we need things to be as complex as necessary, but as simple as possible. And, and so the question is, is all that complexity there necessary? And we ought to test for that and we ought to challenge that. And then after that, everything else above it, you know, is gratuitous.

• 1:06:58Lex Fridman?

  But some of the most incredible semiconductor industry broadly, but what Nvidia is doing,

• 1:07:06Lex Fridman?

  some of the greatest engineering in history. So these systems are just truly, truly marvels of engineering.

• 1:07:12Speaker 01

  It is the most complex computer the world has ever made.

• 1:07:15Lex Fridman?

  Yeah, the engineering teams, I mean, I don't, it's not a competition, but I don't know. If, if it was like an Olympics of engineering teams, I mean, TSMC does incredible engineering. Like I said, ASML at every scale, but Nvidia is going to give them a run for their money.

• 1:07:28Speaker 01

  Yeah.

• 1:07:29Lex Fridman?

  Just incredible, incredible teams.

• 1:07:30Speaker 01

  Gold medalists in every single, in every single sport, all assembled right here.

• 1:07:35Lex Fridman?

  And have to work together and report directly to you. This is wonderful. You've recently traveled to China.

• 1:07:43Lex Fridman?

  So it's interesting to ask you, China has been incredibly successful in building up its technology sector. What do you understand about how China is able to, over the past 10 years, build so many incredible world-class companies, world-class engineering teams, and just this technology ecosystem that produces so many incredible products?

• 1:08:07Speaker 01

  Whole bunch of reasons. Well, first of all, let's, let's start, let's start with some facts. 50% of the world's AI researchers are Chinese, plus or minus. And they're mostly in China, still. We have many of them here, but there's amazing researchers still in China.

• 1:08:26Speaker 01

  They, their tech industry showed up at precisely the right time. At the time of the mobile cloud era, their way of contributing with software. And so this is a country's incredible science and math, really well-educated kids.

• 1:08:45Speaker 01

  Their tech industry was created during the era of software. They're very comfortable with modern software.

• 1:08:54Speaker 01

  China is not one giant economic country. It's got many provinces and cities with mayors all competing with each other. That's the reason why there's so many EV companies. That's the reason why there's so many AI companies. That's the reason why there's so many, every company you could imagine, they all create some of them. And, and as a result, they have insane competition internally. And, you know, what remains is an incredible company. They also have a social culture where, where it's family first, friends second, and company third.

• 1:09:35Speaker 01

  And so the amount of conversation that goes back and forth between, they're essentially open source all the time. So the fact that they contribute more to open source is so sensible because they're probably, what are we protecting? You know, my engineers, their brothers are in that company, their friends are in that company, and they're all schoolmates. You know, the schoolmate concept. It's a, you know, one schoolmate, your brother for life. And, and so they, they, they share knowledge very, very quickly. And so there's no sense keeping technology hidden. You might as well put it on open source.

• 1:10:18Speaker 01

  And so the open source community then amplifies, accelerates the, the innovation process. So you get this rapid, incredible, great talent, rapid innovation because of open source and just, you know, the, the nature of friends. And, and insane competition among, among the company, what emerges is incredible stuff. And so this is the fastest innovating.

• 1:10:46Speaker 01

  Country in the world today. And this is something that has everything that I've just said is fundamental to just how the kids were grown. The fact that they have excellent education, the fact that they, parents want them to do well in school. The fact that their culture is that way. These are, you know, these are just the thing about their country. And they showed up at precisely the time when technology is going through that exponential.

• 1:11:10Lex Fridman?

  Plus culturally, it's pretty cool to be an engineer. It connects to all the components that you're mentioning.

• 1:11:18Speaker 01

  It's a builder nation.

• 1:11:19Lex Fridman?

  It's a builder nation.

• 1:11:20Speaker 01

  Yeah, it's a builder nation. Our country's leaders incredible, but they're mostly lawyers. Their country's leaders, because they're trying to keep us safe, rule of law, governing. Their country was built out of poverty. And so most of their leaders are incredible engineers, some of the brightest minds.

• 1:11:44Lex Fridman?

  To take a small tangent, because you mentioned open source. I have to go to perplexity here, who you've been a fan of a long time.

• 1:11:52Speaker 01

  I love it, yeah.

• 1:11:53Lex Fridman?

  And thank you for releasing open source, Nematron 3 Super, which you can also use inside perplexity to look stuff up. Now, which is 120 billion parameter open weight M O E model. What's your vision with open source? So you mentioned China with deep seek and mini max with all these companies really pushing forward the open source AI movement. And Nvidia is really leading the way in close to state of the art, open source L L M's. What's your vision there?

• 1:12:30Speaker 01

  First, if we're going to be a great AI computing company, we have to understand how AI models are evolving. One of the things that I love about Nematron 3 is it's not just a pure transformer model. It's transformer and S S Ms. And we were early in developing the conditional GANs, which that progressive GANs, which led step by step to diffusion. And so the fact that we're doing basic research. In model architecture and in different domains gives us visibility into, you know, what kind of computing systems would do a good job for future models. And so it is part of our extreme co-design strategy.

• 1:13:14Speaker 01

  Second, I think we rightfully recognize that, on the one hand, we want world-class models as products and they should be proprietary. On the other hand, we also want AI to diffuse into every industry and every country, every researcher, every student. And if everything's proprietary, it's hard to do research and it's hard to innovate on top of, around, with. And so, open source is fundamentally necessary for many industries to join the AI revolution.

• 1:13:56Speaker 01

  Nvidia has the scale and we have the motives to not only skills, scale, and motivation to build and continue to build these AI models for as long as we shall live. And so therefore, we ought to do that. We can open up, we can activate every industry, every researcher, you know, every country to be able to join the AI revolution.

• 1:14:23Speaker 01

  There's the third reason, which is from that to recognizing that AI is not just language. These AIs will likely use tools and models and subagents that were trained on other modalities of information. Maybe it's biology or chemistry or, you know, laws of physics or, you know, fluids and thermodynamics. And not all of it is in language structure. And so, somebody has to go make sure that weather prediction.

• 1:14:57Speaker 01

  Biology, AI, AI for biology, physical AI, all of that stuff stays, can be pushed to the limits and pushed to the frontier. We don't build cars, but we want to make sure every car company has access to great models. We don't, we don't discover drugs, but I want to make sure that Lilly has the world's best biology AI systems so that they can go use it for discovering drugs. And so these three fundamental reasons, both in, in recognizing that AI is not just the language, that AI is really broad, that we want to engage everybody into the world of AI, and then also co-design of AI.

• 1:15:34Lex Fridman?

  Well, I have to say, once again, thank you for open sourcing, really, truly open sourcing Neutron 3.

• 1:15:40Speaker 01

  Yeah, I appreciate you were saying that. We open source the models, we open source the weights, we open source the data, we open source how we created it. Yeah, it's pretty amazing.

• 1:15:49Lex Fridman?

  It's really, it's really incredible. You're originally from Taiwan and have a close relationship with TSMC. So I have to ask, TSMC, I think, also is a legendary company in terms of the engineering teams, in terms of the incredible engineering work that they do. What, what do you understand about TSMC culture and their approach that explains how they're able to achieve this singular unmatched success in everything they're doing with semiconductors?

• 1:16:21Speaker 01

  You know, first of all, the deepest misunderstanding about TSMC is that, that their technology is all they have. That somehow, they, they have a really great transistor. And if somebody shows up another transistor, game over. It's the technology, and of course, you know, I don't mean just the transistor, the metallization systems, the packaging, the 3D packaging, the silicon photonics, that, you know, all of the technology that they have, that technology is really what makes the company special. Their technology makes the company special.

• 1:17:01Speaker 01

  But their ability to orchestrate the demands, the dynamic demands of hundreds of companies in the world as they're moving up, shifting out, you know, increasing, decreasing, pushing out, pulling in,

• 1:17:21Speaker 01

  changing from customer to customer.

• 1:17:25Speaker 01

  Wafer starting, wafer stopping. Emergency wafer starts, you know? All of this dynamics of the world's complexity as the world is shape shifting all the time. And somehow, they're running a factory with high throughput, high yields, really great costs, excellent customer service. They they take their work, they take their promises seriously. When your wafer, because they know that you're helping, they're helping you run your company. When the wafers, when the wafers were promised to show up, the wafers show up, you know, so that you could run your company appropriately. And so their system, their manufacturing system is completely miraculous. I would say then the second thing is their culture.

• 1:18:09Speaker 01

  This culture is a simultaneously technology focused on one hand, advancing technology, simultaneously customer service oriented on the other hand. A lot of companies are very customer service oriented, but they're not very technology excellent. They're they're not at the bleeding edge of technology. Or a lot of companies who are at the bleeding edge of technology, but they're not the best customer service oriented company. And so it just depends on somehow they've they've balanced these two and they're world class at both. And then probably the third thing is the technology that I most value in them, that they created is, you know, this this intangible called trust. I trust them to put my company on top of them.

• 1:18:55Speaker 01

  That's a very big deal.

• 1:18:57Lex Fridman?

  Well, they trust, I mean, there's a really close relationship there that you've established and that trust is established based on many years of performance. But there's human relationships involved there as well.

• 1:19:06Speaker 01

  Three decades, I don't know how many tens, hundreds of billions of dollars of business we've done through them. And we don't have a contract. That's pretty great.

• 1:19:17Lex Fridman?

  Amazing. Okay, there's a story that in 2013, the founders of TSMC, Morris Chang, offered you the chance to become TSMC's chief executive. And you said you already had a job. Is this story true?

• 1:19:31Speaker 01

  Story is true. I didn't, I didn't dismiss it.

• 1:19:36Speaker 01

  But I was, I was deeply honored and, and of course, of course, I knew then, as I know now, TSMC is one of the most consequential companies in history.

• 1:19:45Lex Fridman?

  Yeah.

• 1:19:46Speaker 01

  And, and Morris is one of the, the highest regarded executive and, and business and personal friend that I've, that I've had in my life. And for him to ask is, I was humbled and, and really honored.

• 1:20:07Speaker 01

  But, but the work that I'm doing here is really important. And I've seen, you know, in my mind anyways, in my mind's eye, what Nvidia was going to be and what the impact that we could have. And it was really important work.

• 1:20:23Speaker 01

  And it's my responsibility, you know, my sole responsibility to make this happen. And so I, I, I declined it, you know, not, not because it wasn't an incredible offer. It's an unbelievable offer, but, but I simply couldn't take it.

• 1:20:40Lex Fridman?

  I think Nvidia, both Nvidia and TSMC are two of the greatest companies in the history of human civilization. And running either one, I'm sure is incredibly complicated effort and takes, you have to truly be all in.

• 1:20:54Speaker 01

  Yeah.

• 1:20:54Lex Fridman?

  Everybody at every scale, not just at the CEO level, everybody is really truly all in.

• 1:20:59Speaker 01

  Yeah, no doubt.

• 1:21:00Lex Fridman?

  To accomplish this kind of complexity.

• 1:21:02Speaker 01

  See, now I can help both companies.

• 1:21:03Lex Fridman?

  Exactly. So Nvidia is now the most valuable company in the world. I have to ask, what is the Nvidia's biggest moat? As the folks in the tech sector say, the edge you have that protects you from the competition?

• 1:21:21Speaker 01

  Our single most important property as a company is the installed base of our computing platform. Our single most important thing is the, today is our, is the installed base of CUDA. Now, the reason why.

• 1:21:43Speaker 01

  20 years ago, of course, there was no install base. But what makes... And if somebody came up with a CUDA or Tuda, it wouldn't make any difference at all. And the reason for that is because because it's never been just about the technology. The technology, of course, was incredible visionary.

• 1:22:05Speaker 01

  But it's the fact that the company was dedicated to it, stuck with it, expanded its reach. It wasn't three people that made CUDA successful. It was 43,000 people that made CUDA successful. And the several million developers that believed in us, that trusted that we were going to continue to make CUDA 1, 2, 3, 13, that they decided to port and dedicate their software on top of it, their mountain of software on top of it. And so the install base is the number one most important advantage. That install base, when you amplified with the velocity of our execution at the scale that we're talking about, no company in history had ever built

• 1:22:50Speaker 01

  systems of this complexity, period. And then to build it once a year is impossible. And that velocity, combined with the install base, in the developer's mind, you just can now take a developer's mind. From the developer's perspective, if I support CUDA, tomorrow it will be 10 times better. I just have to wait six months on average. Not only that, if I develop it on CUDA, I reach a few hundred million people, computers. I'm in every cloud, I'm in every computer company, I'm in every single industry, I'm in every single country.

• 1:23:32Speaker 01

  So if I decreed an open source package and I put it on CUDA first, I get these both attributes simultaneously. And not only that, I trust 100% that Nvidia is going to keep CUDA around and maintain it and improve it and keep optimizing the libraries for as long as they shall live.

• 1:23:58Speaker 01

  You could take that to the bank and that last part, trust. You put all that stuff together. If I were a developer today, I would target CUDA first. I would target CUDA most. And that's the reason that that I think in the final analysis is our first. That's even our first.

• 1:24:19Speaker 01

  Core advantage. Our second one is our ecosystem. The fact that we vertically integrated this incredibly complex system. But we integrated horizontally into every single, every single company's computers. We're in the Google Cloud, we're in Amazon, we're in Azure. You know, we're ramping up AWS like crazy right now. We're in new companies like CoreWeave and then Scale. We're in supercomputers at Lilly. We're in enterprise computers. We're at the edge in radio base stations. You know, it's just crazy. One architecture is in all these different systems. We're in cars, we're in robots, we're in satellites, we're out in space. And so, so the fact that you have this one architecture and the ecosystem is so broad. It basically covers every single industry in the world.

• 1:25:04Lex Fridman?

  Well, how does the, how does the CUDA install base evolve into the future with AI factories as a moat? What do you, do you think it's possible that Nvidia of the future is all about the AI factory?

• 1:25:17Speaker 01

  Well, the, the unit of computing used to be GPU to us. Then it became a computer. Then it became a cluster. Now it's an entire AI factory. When I see a computer, when I see what Nvidia builds in the old days, I would, you know, I visualize the chip. And then, and then when I announced a new product, you know, a new generation, like ladies and gentlemen, we're announcing AMPYR today. I pick up the chip.

• 1:25:41Lex Fridman?

  Yeah.

• 1:25:42Speaker 01

  That was my mental model, what I was building. Today, I wouldn't, picking up the chip is kind of still adorable. But it's adorable. It's not, it's not my mental model, what I'm doing. My mental model is this giant gigawatt thing. That has power generation, is connected to the grid, it's got cooling systems and networking of incredible monstrosity. You know, 10,000 people are in there trying to install it. Hundreds of networking engineers in there, thousands of engineers behind it, trying to power it up. You know, powering up one of those factories, as you know, it's not somebody going, it's on now.

• 1:26:22Speaker 01

  Takes thousands of people to bring it up.

• 1:26:24Lex Fridman?

  So mentally, you're actually, when you're thinking about a single unit of compute, you're like, literally, when you go to bed at night, you're thinking now about collection of racks, so pods, not individual chips.

• 1:26:35Speaker 01

  Entire infrastructure. And I'm hoping my next click is when I'm thinking about building computers, it's planetary scale. That'll be the next click.

• 1:26:43Lex Fridman?

  What do you think about the space angle that Elon has talked about doing compute in space for solving some of the, it makes some of the energy issues in terms of scaling energy easier?

• 1:26:58Speaker 01

  Cooling issues is not easy.

• 1:27:00Lex Fridman?

  Cooling, well, there's a large number of engineering complexities involved with that. So Nvidia has also announced that you're already thinking about that.

• 1:27:10Speaker 01

  Yeah, we're already there. Nvidia GPUs are the first GPUs in space. And I didn't realize it was so interesting to, I would have declared it maybe. We're in space, you know, little astronaut suit on one of our GPUs.

• 1:27:29Speaker 01

  But we've been in space. It's the right place to do a lot of imaging, you know, because those satellites have really high resolution imaging systems. And they're sweeping the earth continuously now. And you want centimeter scale imaging that is done continuously for the world, so that you'll basically have real time telemetry of everything.

• 1:27:55Speaker 01

  You don't want to beam that back down to earth. It's just petabytes and petabytes of data. You ought to just do AI right there at the edge, throw away everything you don't need, you've seen before, didn't change, and then just keep the stuff that you need. And so AI ought to be done at the edge. Obviously, we have 24/7 solar if we put it at the polars.

• 1:28:20Speaker 01

  But you know, there's no conduction, no convection. And so you're pretty much just radiation. And but you know, space is big. I guess we're just going to put big giant radiars out there.

• 1:28:33Lex Fridman?

  How crazy of an idea do you think it is? Like, is this is this five years out, 10 years out, 20 years out? So we're talking about blockers for AI scaling.

• 1:28:43Speaker 01

  You know, I'm just so much more practical. I look for where where my next next bucket of opportunities are first. Meanwhile, I'm cultivating space. And so I send I send engineers to go work on the problem. We're starting, we're learning a lot about it. How do we do a radiation? How do we do a degrading performance? How do we deal with a continuous testing and adestation of defects? And and you know, how do we deal with redundancy? And how do we degrade gracefully and things like that? And so we could we could do what about software? How do you think about software and redundancy and performance out in space?

• 1:29:26Speaker 01

  Make it so that so that the computer never breaks. It just gets slower. And I so we could start doing a lot of engineering exploration upfront. But in the meantime, my my favorite answer is eliminate waste. You know, we've we've got all that idle power. I want to evacuate it as fast as possible.

• 1:29:49Lex Fridman?

  Yeah, there's a lot of low hanging fruit here on Earth that we can utilize for the AI scaling. Quick pause. Quick 30 second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We got perplexity for curiosity driven knowledge exploration. Shopify for selling stuff online. Element for electrolytes. Finn for customer service AI agents. And Quo for a phone system like calls, texts, contacts for your business. Choose wisely, my friends. And now back to my conversation with Johnson Kual.

• 1:30:31Lex Fridman?

  Do you think Nvidia may be worth 10 trillion at some point? Let's let's ask it this way. What does the future of the world look like where that where that's true?

• 1:30:46Speaker 01

  I think that Nvidia's growth is is

• 1:30:53Speaker 01

  extremely likely and in my mind inevitable. And let me explain why. We're the largest computer company in history. That alone should beg the question, why? And the reason for, of course, two reasons. First, two foundational, technical reasons. The first reason is that computing went from being a retrieval based file retrieval system. Almost everything is a file. We pre, pre-write something, we pre-record something. You know, we, we draw something, we put it on the web, we put it in a file. And we, we use a recommender system, some smart filter to figure out what to retrieve for you. And so we were a pre-recording, human pre-recording and file retrieving system.

• 1:31:38Speaker 01

  That's what a computer is largely.

• 1:31:41Speaker 01

  To now, AI computers are contextually aware, which means that it has to process and generate tokens in real time. So we went from a retrieval based computing system to a generative based computing system.

• 1:31:55Speaker 01

  We're going to need a lot more processing in this new world than in the old world. We need a lot of storage in the old world. We need a lot of computation in this new world. And so so that's, that's the first part of it. We fundamentally changed computing and the way how computing is done.

• 1:32:14Speaker 01

  The only thing that would cause it to go back is if this way of computation, this way of computing, generating information that's contextually relevant, situationally aware, that is grounded on new insight before it generates information. This computation intensive way of doing computing would only go back if it's not effective. So for the last 10, 15 years while working on deep learning, if at any single moment,

• 1:32:45Speaker 01

  I would have come to the conclusion that, that, you know what? This is not going to work out. I think this is a dead end or it's not going to scale. It's not going to solve this modality, not going to be used in this application. Then of course I would feel very differently about it. But I think the last five years has given me more confidence than the last 10 years, the previous 10 years. The second idea is computers, because it was a storage system, it was largely a warehouse.

• 1:33:15Speaker 01

  We're now building factories. Warehouses don't make much money. Factories directly correlates with a company's revenues. And so the computer did two things. Not only did it change the way it did it, its purpose in the world changed. It's no longer a computer, it's a factory. It's a factory is used for generation of revenues. We're now seeing, not only is this factory generating products, commodities that people want to consume. We're seeing that the commodities are so interesting, so valuable,

• 1:33:59Speaker 01

  so to so many different audiences, that the tokens are starting to segment like iPhones. You have free tokens. You have premium tokens. And you have several tokens in the middle. And so intelligence, as it turns out, you know, is a scalable product. There's extremely high intelligence products, tokens that you could, that are used for specialized things. People be willing to pay. You know, the idea that somebody's willing to pay a thousand dollars per million tokens is just around the corner. It's not if, it's only when. And so, so now we're seeing that the commodity that this factory makes is actually valuable and is revenue generating and profit generating.

• 1:34:43Speaker 01

  How, now the question is, how many of these factories can, does the world need? How much, how many tokens does the world need? And how much is society willing to pay for these tokens? And what would happen to the world's economy? If the productivity were to improve so substantially? What would happen? Are we, are we going to discover new drugs, new products, new services? And so when you take these things in combination, I am absolutely certain.

• 1:35:20Speaker 01

  That the world's GDP is going to accelerate in growth. I'm absolutely certain the percentage of that GDP that will be used for computation will be a hundred times more than the past because it's no longer a storage unit. It's a product generation unit. And so when you look at it in that context, and then you back into what is Nvidia's, what does Nvidia, what does Nvidia do? And how much of that new economics, new industry would we have to benefit to address? I think we're going to be a lot, lot bigger.

• 1:36:00Speaker 01

  And then the rest of it to me is, is it possible for Nvidia to be a, you know, three trillion dollar revenues company in the near future? The answer is of course, yes. And the reason for that is because it's not limited by any physical limits. There's nothing that I see that says, you know, gosh, three trillion dollars is not possible. And as it turns out, Nvidia's supply chain is, the burden is shared by 200 companies. And the fact that we scale out on the backs of, with the partnership of this ecosystem. The question is, do we have the energy to do so?

• 1:36:43Speaker 01

  And surely we will have the energy to do so. And so all of these things combined, that number is just a number, you know? And I still remember Nvidia was a, Nvidia was a, the first time we crossed a billion dollars. I was reminded of, of a CEO who told me, you know, Jensen, it's theoretically impossible for a fabulous semiconductor company to exceed a billion dollars. And, and I won't bore you with why, but, but the, of course, it's illogical and there's a lot of evidence we're not. And then somebody told me, you know, Jensen, you'll never be more than 25 billion dollars because of some other company.

• 1:37:24Speaker 01

  Somebody told me that you'll never be, you know, because, and then so, so the, those, those aren't principle, first principle reason thinking. And the simple, the simple way to think about that is what is it that we make? And how large is the opportunity that we can create? Now, Nvidia is not in the market share business. Almost everything that I just talked about don't exist. That's the part that's hard. You know, if Nvidia was a was a was a $10 billion company trying to take Nvidia's share, then it's easy to see for shareholders that, oh, yeah. If they could just take 10% share, they could be this much larger.

• 1:38:08Speaker 01

  But it's hard for people to imagine how large we could be because there's nobody I could take share from, you know? And so, so I think that that's one of the challenges for the world is is the imagination of the future. But I got plenty of time and I'll keep reasoning about it and I'll keep talking about it. And every single GTC will become more and more real, you know? And and then more and more people talk about it and one of these days, you know, we'll get there, but 100% we'll get there.

• 1:38:35Lex Fridman?

  Yeah, this view of, you know, token factories essentially, this token per second per watt and every token having value, like it's an actual thing that brings value and it brings different kinds of value, different amounts of value to different people with value. That's the actual product that's really can be loosely thought of as the token. And so you have a bunch of token factories and it's very easy, first principle is to imagine a future, given all the potential things that AI can solve, that you're going to need an exponential number more of token factories.

• 1:39:07Speaker 01

  Yeah. And what's really interesting, the reason why I was so excited about it, the iPhone of tokens arrived.

• 1:39:13Lex Fridman?

  What do you call it? Wait, are you saying OpenClaw's iPhone?

• 1:39:15Speaker 01

  Yeah.

• 1:39:16Lex Fridman?

  That's interesting.

• 1:39:17Speaker 01

  Agents.

• 1:39:18Lex Fridman?

  Yeah, agents.

• 1:39:19Speaker 01

  True. Agents in general. The iPhone of tokens arrived. It is the fastest growing application in history. It went straight up.

• 1:39:26Lex Fridman?

  Yeah.

• 1:39:27Speaker 01

  Went straight up.

• 1:39:28Lex Fridman?

  That says something.

• 1:39:29Speaker 01

  Yep. There's no question, OpenClaw is the iPhone of tokens.

• 1:39:33Lex Fridman?

  Yeah, there's something truly, as you know, something truly special happening from about December, where people really woke up to the power of Claude code, of Codex, of OpenClaw. I mean, I've embarrassed to admit that the way here in the airport.

• 1:39:52Lex Fridman?

  I'm this first time I've done this in public. I was programming quote unquote by talking to my laptop. And I was embarrassed because I was pretending like I'm talking to a human colleague. Uh, I'm not sure how I feel about the future where everybody is walking around, talking to their AI. But it's such an efficient way to get stuff done.

• 1:40:14Speaker 01

  And, and it's, it's more likely that your AI is bothering you all the time. And the reason for that is because it's getting stuff done so fast.

• 1:40:23Lex Fridman?

  Yeah.

• 1:40:23Speaker 01

  Is reporting back to you, I got that done. You know, what do you want me to do next? You know, it, that's the part that I think that most people don't realize is that the person who's going to be chatting with them, texting them most is their, is their claws or lobster.

• 1:40:39Lex Fridman?

  What an incredible future. Uh, I read that you attribute a lot of your success to your ability to work harder than anyone and withstand more suffering than anyone. So we can list many of the things that entails. I mean, dealing with failure, the constant engineering problems we've talked about, the, the human problems, uncertainty, responsibility, exhaustion, embarrassment, the near death company moments that you've mentioned.

• 1:41:07Lex Fridman?

  But also the pressure now as the CEO of this

• 1:41:12Lex Fridman?

  company that economies and nations strategize around, uh, plan their, um, financial allocations around, plan their AI infrastructure around. How do you deal with this much pressure? What gives you strength, given how many nations and peoples depend on you?

• 1:41:39Speaker 01

  I'm conscious about the fact that, um, Nvidia's success is very important to United States.

• 1:41:49Speaker 01

  We generate enormous amounts of tax revenues. We establish technology leadership for our nation. Technology leadership is important for national security. National security, not just in one aspect of national security, all aspects of national security. When our country is more prosperous, we could do a better job with domestic policies and helping social benefits. Because we're generating so much reindustrialization in the United States, we're creating mountains of jobs. We're helping shift how we build things back to the United States in so many different plants, chips, computers, and of course, these air factories.

• 1:42:34Speaker 01

  I'm completely aware that, and I have the benefit, and this is a real gift, with mainstream investors, teachers, policemen, who have somehow, for whatever reason, invested in Nvidia or because they watched Jim Cramer, bought some stock and now are millionaires. And I am completely aware of that circumstance. I'm aware of the circumstance that Nvidia is central to a very large network of ecosystem partners behind us and downstream from us.

• 1:43:18Speaker 01

  And so the way I deal with that is exactly what I just did. I reason about what is it that we're doing? What is it causing? What's the impact that has on other people, positively or even through great burden, for example, to supply chain? And the question is...

• 1:43:43Speaker 01

  Therefore, what are you going to do about it? In almost everything that I feel, I break it down. I reason about, okay, what's the circumstance? What has changed? What's hard? And what am I going to do about it? And I break it down, decompose the problem. And the decomposition of these circumstances

• 1:44:09Speaker 01

  turns it into manageable things that I can do. And the only thing that I, after that, I could do is, did you do it? Did you either do it? Or did you get somebody else to do it? And if you didn't do it, you reason that you need to do it and you didn't do it and you didn't get anybody else to do it? Then stop crying about it. You know? And so, and so, I'm fairly, I'm fairly tough on myself. And, but I also break things down so that, so that I don't panic. I can go to sleep because I've made the list of things that needed to be done. And I've made sure that everything that could put our company in harm's way, could put my partners in harm's way, put our industry in harm's way. I've told somebody.

• 1:44:55Speaker 01

  Everything that I feel could put anybody in harm's way, I've told someone. And I've told that someone who could do something about it. And so I've gotten it off my chest. Or I'm doing something about it. And so after that, Lex, what else can you do?

• 1:45:12Lex Fridman?

  So given all the insane intense amount of suffering on the journey of building up Nvidia,

• 1:45:21Lex Fridman?

  you, have you hit low points psychologically?

• 1:45:23Speaker 01

  Oh, yeah. Oh, yeah, sure. All the time. All the time.

• 1:45:28Lex Fridman?

  And there, you just break down the problem into pieces?

• 1:45:33Speaker 01

  Yeah.

• 1:45:33Lex Fridman?

  See what you could do about it?

• 1:45:35Speaker 01

  And, and part of, you know, Lex, part of it, part of it is forgetting. One of the most important attributes of AI learning, as you know, is systematic forgetting. You need to know when to forget some things. You can't memorize everything. You can't keep everything and, you know, you don't want to carry everything. One of the things that I do very quickly is I decompose the problem. I reason about the problem and I, I share the load with it. When I say I tell everybody, I'm essentially sharing that burden as quickly as possible.

• 1:46:08Speaker 01

  Whatever worries me, tell somebody else. Don't just keep it. You know, decompose, don't, don't freak them out. Decompose the problem into smaller parts and get people to, and, and inspire them to be able to go do something about it. But part of it is just, just forgetting, you know, like, a lot of it is, you gotta be tough on yourself. You know, he's just, come on, stop crying about it. Let's get going, you know, and, and then you get out of bed. And then the other part is, is um, you, you, you, you're attracted to the next shiny light, the next future. You know, the next opportunity, the next, okay, that's behind us. What's next? And it's a lot, I think, you know, you watch this with great athletes, they, they just worry about the next point.

• 1:46:51Speaker 01

  The last point is behind them, the embarrassment, the, you know, the setback. You know, and then, and because I do so much of my job publicly. You know, Lex, you do a fair amount of your job publicly too. And so, so I do a lot of my job publicly. And so, um, you know, I, I say a lot of things that, that seem sensible at the time, or funny at the time, mostly it's just because it's funny to me at the time. And then, you know, you reflect on it, it's less funny, but, but.

• 1:47:21Lex Fridman?

  Yeah, trust me, I know. But you basically allow yourself to be pulled by the light of the future. Forget the past and just keep.

• 1:47:29Speaker 01

  That's right.

• 1:47:30Lex Fridman?

  Keep, keep working towards that. I mean, you did say, there's this kind of famous thing, you said that um, if you knew how hard it would be to build Nvidia, uh, it turned out to be, what is it, a million times more hard than you anticipated? Uh, that you wouldn't do it.

• 1:47:48Speaker 01

  Yeah.

• 1:47:49Lex Fridman?

  But isn't, you know, when I hear that, that's probably true about everything worth doing, right?

• 1:47:55Speaker 01

  Exactly. That is, by the way, what I was trying to explain is that there's a, there's a incredible superpower of being, being, being, uh, have the mind of a child.

• 1:48:08Lex Fridman?

  Yeah.

• 1:48:09Speaker 01

  You know, and I say to myself, oftentimes when I look at something and, and almost, almost everything.

• 1:48:17Speaker 01

  My first thought is, how hard can it be?

• 1:48:21Speaker 01

  You know, and so, and so you get yourself into that mode. How hard could it be? And, and nobody's ever done it. It looks gigantic. It's going to cost hundreds of billions of dollars. It's going to take, you know, all this. And you just go, yeah, but how hard could it be? You know? Yeah. How hard could it be? Yeah. And, and so, so you got to get yourself into that state of mind. You don't want to, you don't want to actually over simulate everything and all the setbacks and all the trials and tribulations and all the disappointments. You don't want to simulate all that in advance. You don't want to know that. You don't, you don't, you want to go into a new experience thinking it's going to be perfect. It's going to be great. It's going to be incredibly fun. And then while you're there, you know, you need to have,

• 1:49:05Speaker 01

  you need to have endurance, you need to have grit so that when the setbacks actually happened and those setbacks are going to surprise you, the disappointments aren't going to surprise you. You know, the embarrassments are going to surprise you. The humiliations are going to surprise you. You just can't, now you just got to turn on the other bit, which is just forget about it. Move on, keep, keep moving. And, and to the extent that,

• 1:49:29Speaker 01

  to the extent that my assumptions about the future and why the future is going to manifest, so long as those assumptions and that input doesn't change or didn't change materially, then I should expect that the output won't change. And so my simulated output of the future is still going to happen. And if it's still going to happen, I'm still going to go after it. I believe it's going to, you know, and so there's a combination of two or three human characteristics.

• 1:50:03Speaker 01

  The ability to go into, into an experience fresh minded, the ability to forget the setbacks, the ability to believe in yourself, you know, to believe what you believe and stay, stay true to that belief. But you're constantly reevaluating. This combination of three, four, five things, I think is, is really important for resilience. And, and.

• 1:50:31Speaker 01

  And you know, I'm fortunate that whatever life experiences led to this, I've got kind of those four or five things. And I'm always curious, always learning. I'm always learning from everybody. You know, I'm always asking my, and because I'm humble about everything, I'm always thinking, gosh, they did that so nicely. They did that so wonderfully. You know, I wonder what they're thinking through. How do they, you know, so I'm simulating everybody in a lot of ways. And I'm emulating almost everybody I watch, right? You're empathetic towards, towards everything that they do that you're observing and respect. And so you're constantly learning and, you know.

• 1:51:12Lex Fridman?

  You're now one of the wealthiest people on earth, one of the most successful humans on earth. Is it harder to be humble and to be able to, do you feel the effect of money and power and fame in making it harder for you to sort of be wrong in your own head enough to hear out an opinion of somebody else when it disagrees with you and learn from them? Those kinds of things?

• 1:51:44Speaker 01

  Surprisingly, no. And I would, I would actually go the other way. Because I do so much of my work publicly, when I'm wrong, pretty much everybody sees it.

• 1:51:55Lex Fridman?

  You get humbled.

• 1:51:56Speaker 01

  Yeah. And, and when I'm wrong, when I'm wrong, or it didn't turn out that way, or, you know, I mean, most of the things that I say outside, I'm fairly certain about. And the reason for that is because, because it's going to impact somebody else. And I want to be quite concerned about that and quite circumspect about that. For stuff that I'm reasoning about inside a meeting, you know, a lot of things could turn out differently. And so, but it doesn't ever stop me from reasoning. The way that, the way that I manage and lead, you know, I'm constantly reasoning in front of people. And even when I'm talking to you, you can kind of see me kind of reasoning through things. And I want to make sure that you understand what I'm saying, not because I told you.

• 1:52:42Speaker 01

  Because I'm so humble about what I'm about to tell you. I kind of show you the steps that I got there. And then you could decide whether you believe what I said in the end. And so I'm doing that all day long in meetings. With all of my employees, I'm constantly reasoning through. Let me tell you, let me tell you what, how I see it. And I reason through it. It gives everybody the opportunity to intercept and say, I disagree with that part. The nice thing about reasoning through things and letting, and letting people interact with it is that they don't have to disagree with your outcome. They can disagree with your reasoning steps. And they could pull me in different directions. And then we can reason forward. And so we're, we're kind of, you know,

• 1:53:24Speaker 01

  collective path searching method. And it's really fantastic.

• 1:53:30Lex Fridman?

  Yeah, you have this way about you of when you're explaining stuff, I can feel you actually reasoning on the spot about it with a constant open mind in this where you could, I could feel like I could steer your thinking.

• 1:53:45Speaker 01

  Yeah.

• 1:53:45Lex Fridman?

  And that's, that's really beautiful that you've been able to maintain that after so many years of success and pain. I think sometimes pain makes you close, closes you down a bit.

• 1:53:57Speaker 01

  Yeah.

• 1:53:57Lex Fridman?

  And I think to maintain.

• 1:53:59Speaker 01

  Tolerance for embarrassment, I think is.

• 1:54:01Lex Fridman?

  Yes, that's the tolerance. I mean, that's a real thing.

• 1:54:04Speaker 01

  Yeah.

• 1:54:05Lex Fridman?

  As many years of embarrassing yourself, even those meetings, knowing that there's people around you where you declared one idea and it was shown that that idea was wrong and be able to admit that and to grow from that. That's not, that's very difficult on a human level.

• 1:54:19Speaker 01

  Yeah, well, you know, they knew I was, they knew that recently my first job was, was, you know, cleaning toilets, so.

• 1:54:26Lex Fridman?

  I'm glad you maintain that same spirit of Denny's. The, the work, I mean, that was beautiful. Your whole journey from, starting from Denny's is a beautiful one. Let me ask you about. Video games, so I'm a big gaming fan.

• 1:54:42Speaker 01

  Yeah.

• 1:54:43Lex Fridman?

  So I have to say thank you to Nvidia for many years of incredible graphics.

• 1:54:49Speaker 01

  By the way, it is, GeForce is our still, to this day.

• 1:54:52Lex Fridman?

  Yeah.

• 1:54:53Speaker 01

  Our number one marketing strategy, right? People learn about Nvidia while they're in their teenage years. And then they go to college and they know who Nvidia is. And then in the beginning, it's just, you know, playing Call of Duty, you know, Fortnite. And then later they're using CUDA and then later they're using Nvidia and, you know, Blender and Dassault and Autodesk.

• 1:55:17Lex Fridman?

  I mean, I should say, I mentioned to a friend that I'm talking with you. He said, oh, they make great gaming GPUs.

• 1:55:26Speaker 01

  Yeah, exactly. Exactly.

• 1:55:28Lex Fridman?

  You know, there's more to it. But, but yeah, yeah, people really love the, it really brought a lot of joy to a lot of people. The hardware really brings these worlds to life. There was some controversy around this with DLSS five.

• 1:55:45Speaker 01

  Yeah.

• 1:55:45Lex Fridman?

  Can you explain to me the drama around this? I guess people, gamers online were concerned that it makes games look like AI slop.

• 1:55:55Speaker 01

  Yeah.

• 1:55:56Lex Fridman?

  What do you think of this drama?

• 1:55:57Speaker 01

  Yeah. I think their, their perspective makes sense and I could see where they're coming from. Because I don't love AI slop myself. You know, all of the, the AI generated content increasingly looks similar and they're all beautiful. And, and I can, so I can, I'm empathetic towards what they're, what they're thinking. That's just not what DLSS five is trying to do. I showed several examples of it. But DLSS five is 3D conditioned, 3D guided. It's ground truth, structured data guided. And so, so the artist determined the geometry.

• 1:56:38Speaker 01

  We are completely truthful to the geometry, maintain so in every single frame.

• 1:56:46Speaker 01

  It's conditioned by the textures, the artistry of the artist. And so every single frame, it enhances, but it doesn't change anything. Now, the question is, the question about enhancing. DLSS 5 also lets, because it's, the system is open, you could train your own models to determine, and you could even, in the future, prompt it. You know, I want it to be a tune shader. I want it to look like this kind of, you know, so you can give it even an example, and it would generate in the style of that, all consistent with the artistry, you know, the style, the intent of the artist.

• 1:57:27Speaker 01

  And so, all of that is done for the artist so that they can create something that is more beautiful, but still in the style that they want. I think that they got the impression that the games are going to come out the way the games are, shipped the way they do, and then we're going to post-process it. That's not what DLSS is intended to do. DLSS is integrated with the artist. And so, it's about giving the artist the tool of AI, the tool of generative AI. They could decide not to use it, you know?

• 1:58:03Lex Fridman?

  I think people are very sensitive to human faces.

• 1:58:05Speaker 01

  Yeah.

• 1:58:05Lex Fridman?

  And we're now living in this moment, which I think is a beautiful one, which is, people are sensitive to AI slop.

• 1:58:12Speaker 01

  Yeah.

• 1:58:13Lex Fridman?

  It puts a mirror to ourselves to help us realize that what we seek is imperfections, what we seek is sometimes not perfect graphics. It helps us understand what we find compelling in the worlds we create.

• 1:58:26Speaker 01

  Mm-hmm.

• 1:58:26Lex Fridman?

  And that's beautiful. And as long as it's tools that help us create those worlds.

• 1:58:29Speaker 01

  Yeah, that's right.

• 1:58:30Lex Fridman?

  It's wonderful.

• 1:58:31Speaker 01

  That's right. It's yet another tool. And they want the generative models to generate the opposite of photoreal. Yeah, it'll do that too. And so, it's just yet another tool. I think the gamers might also appreciate that in the last couple of years...

• 1:58:51Speaker 01

  We we introduced skin shaders to the game developers. And many of those games have skin shaders that include subsurface scattering that make skin look more skin-like. And so the industries, game developers are looking for more and more and more tools to express their art. And so this is just yet more, one more tool. They get to decide what to use.

• 1:59:18Lex Fridman?

  Ridiculous question. What do you think is the greatest or most influential game ever made? Maybe from Nvidia's perspective.

• 1:59:26Speaker 01

  Doom.

• 1:59:27Lex Fridman?

  Doom unquestionably. That was the start of the 3D.

• 1:59:30Speaker 01

  I would say Doom from our, the intersection of the cultural implication as well as the industry, turning a PC into a gaming device. That was a very important moment. Now, of course, flight simulation companies were before it. And but they just didn't have the popularity that Doom did to have made the industry turn the PC from an office automation tool into a personal computer for families and gamers and things like that. And so Doom was really impactful there. From a, from an actual game technology perspective, I would say Virtual Fighter. And so we were great friends with both of them, you know?

• 2:00:09Lex Fridman?

  And then there's games more recently. I mean, Cyberpunk 2077, really nice GPU accelerated graphics.

• 2:00:17Speaker 01

  Fully ray traced.

• 2:00:19Lex Fridman?

  Fully ray traced. Also, I like, I personally, I'm a huge fan of Skyrim, Elder Scrolls. And the, you know, it's been released a long, long time ago, but people have released mods.

• 2:00:30Speaker 01

  They love mods.

• 2:00:32Lex Fridman?

  I mean, it's like a different game and it just allows me to replay the game over and over. And it makes you realize that you can re-experience in a totally new way the world you already love. So I do that all the time. One of my favorite things is just walking around Skyrim.

• 2:00:50Speaker 01

  We created this thing called RTX Mod. Yeah, it's a modding tool. And it allows, it allows the community to inject the latest technology into an old game.

• 2:01:01Lex Fridman?

  Of course, like what makes a great video game is not just graphics. It's also story and character development, but.

• 2:01:07Speaker 01

  That's right.

• 2:01:08Lex Fridman?

  Beautiful graphics can add to the immersion, the feeling like it's another place that you're transported to. What's, you said, I think accurately, that the AGI timeline

• 2:01:24Lex Fridman?

  question rests on your definition of AGI.

• 2:01:28Lex Fridman?

  So let's, let me ask you about possible timelines here. Let's, this ridiculous definition, perhaps, of what AGI is, but an AI system that's able to essentially do your job. So run, no, start, grow and run a successful technology company. That's worth.

• 2:01:54Speaker 01

  A good one or a one?

• 2:01:55Lex Fridman?

  No, it has to be worth more than a billion, more than a billion dollars.

• 2:02:03Lex Fridman?

  So, you know, you know how hard it is to do all of those components. So how far are we away from that? So we're talking about OpenClaw that does all the incredibly complex stuff that are required to, to first of all, innovate, to find customers, to sell to them, to manage, to build a team of some agents, some humans, all that kind of stuff. Is this five, 10, 15, 20 years away?

• 2:02:33Speaker 01

  I think it's now. I think we've achieved AGI.

• 2:02:36Lex Fridman?

  You think you can have a company run by an AI system like this?

• 2:02:39Speaker 01

  Possible. And the reason for that is this, you said a billion and you didn't say forever. And, and so for example,

• 2:02:48Speaker 01

  it is not out of the question that

• 2:02:52Speaker 01

  a claw was able to create a web service, some interesting little app that all of a sudden, you know,

• 2:03:04Speaker 01

  a few billion people used for 50 cents and then.

• 2:03:09Speaker 01

  It went out of business again shortly after. Now, we saw a whole bunch of those type of companies during the internet era. And most of those websites were not anything more sophisticated than what Open Claw could generate today.

• 2:03:22Lex Fridman?

  Achieve virality and monetize that virality.

• 2:03:25Speaker 01

  Yeah, it's just that I don't know what it is, but I couldn't have predicted any of those companies at the time either, you know?

• 2:03:31Lex Fridman?

  You're gonna get a lot of people excited with that statement.

• 2:03:34Speaker 01

  Yeah.

• 2:03:35Lex Fridman?

  They're like, what do you mean? I could just launch an agent and make a lot of money.

• 2:03:40Speaker 01

  Well, by the way, it's happening right now, right? You know that when you go to China, you're gonna see, you're gonna see a whole bunch of people teaching their, getting their claws to try to go out and look for jobs and, you know, do work, make money. And, and I'm not, I'm not actually, I wouldn't be surprised if some social thing happened or somebody created a, a digital influencer, super, super cute, or some social application that, you know, feed your little tomagati or something like that. And, and it become, and out of the blue, an instant success, a lot of people use it for a couple of months and it kind of dies away.

• 2:04:20Speaker 01

  Now, the odds of, of, of, you know, a hundred thousand of those agents building Nvidia is zero percent.

• 2:04:30Speaker 01

  And, and then, and then the one part that I will, I will do, and I, and I, I want to make sure we all do, is to recognize that people are really worried about their jobs. And, and I just want to remind them that the purpose of your job and the tasks and the tools that you use to do your job are related not the same. I've been doing my job for 33 years. I'm the longest running tech CEO in the world, 34 years. And the tools that I've used to do my job has changed.

• 2:05:05Speaker 01

  Continuously in the last thirty four years, and sometimes quite dramatically, you know, over the course of a couple, two, three years. And, and the one story that I, I really want to make sure that everybody hears is the story that the first job that every, that computer scientists said, AI researchers said was going to go away was radiology. Because computer vision was going to achieve superhuman levels. And it did. See, the computer vision was superhuman in twenty nineteen, twenty, maybe, maybe a little bit later, twenty twenty. Okay, and so it's been a long time since computer vision has been superhuman. And so the prediction was radiologists would go away.

• 2:05:47Speaker 01

  Because studying radiology scans was thing of the past. AI will do that. Well, they were absolutely right. Computer vision is completely superhuman. Every radiology platform and package today is driven by AI. And yet, the number of radiologists grew. And so the question is why? And we now have a shortage of radiologists in the world. And so, one, the alarmist warning went too far and it scared people from doing this profession that is so important to society. And so it did harm. Now, why was it wrong? The reason why is because the purpose of a radiologist,

• 2:06:32Speaker 01

  the purpose is to diagnose disease and help patients and doctors diagnose disease. And because we're able to study scans so much faster now, you could study more scans, you could diagnose better, you could, you could, um, inpatient faster. We can see people more. The hospitals are making more money. You have more patients in the hospital. You need more radiologists. I mean, the, the amazing thing is it's so obvious this was going to happen. The number of software engineers at Nvidia is going to grow, not decline.

• 2:07:10Speaker 01

  And the reason for that is because the purpose of a software engineer and the task of a software engineer for coding are related, not the same. I wanted my software engineers to solve problems. I didn't care how many lines of code they wrote. You know, but their job, their purpose of their job didn't change. Solving problems, working as a team, diagnosing problems, evaluating the result, looking for new problems to solve, innovation, connecting dots. You know, none of that stuff is going to go away.

• 2:07:41Lex Fridman?

  So you think it's possible that, let's even take coding. You think the number of programmers in the world might increase, not decrease?

• 2:07:47Speaker 01

  Yes.

• 2:07:49Speaker 01

  And the reason for that is this, what is the definition of coding?

• 2:07:54Speaker 01

  I believe that the definition of coding as of today is simply specifying, specification, and maybe if you want to be rather directive, you could even give it an architecture of the software of the year you wanted to write. So the question is, how many people could do that? Describe a specification for a computer to go, telling the computer what to go build. How many people? I think we just went from 30 million to probably 1 billion.

• 2:08:24Speaker 01

  And so every carpenter in the future will be a coder. Except a carpenter with AI is also an architect. They just increased the value that they could deliver to the customer. Their artistry just elevated tremendously. I believe that every accountant is, you know, also your financial analyst, also your financial advisor. So all of these professions have just been elevated. And if I were a carpenter, I see AI, I would just completely go berserk. You know, the services I can bring to my clients, if I were a plumber, completely go berserk.

• 2:09:06Lex Fridman?

  And the people that are currently programmers and software engineers, I think they're at the cutting edge of understanding intuitively how to communicate.

• 2:09:16Lex Fridman?

  With the agents using natural language in order to design the best kind of software.

• 2:09:21Speaker 01

  That's right.

• 2:09:22Lex Fridman?

  So over time, they'll converge, but I think there's still value in getting, I think, learning how to program, like learning what programming languages are, the old kind of programming. What are good practices for programming languages? What are design principles for programming languages for large software systems?

• 2:09:45Speaker 01

  And the reason for that, Lex, and you know, I was just saying for the audience, I think the goal of the goal of specification, the artistry of specification, the goal and the artistry of it, it's going to depend on what problem you're trying to solve. When I'm thinking, when I'm thinking about giving the company strategies and formulating corporate directions and things that we should do, I describe it at a level that is sufficiently specific that people generally understand the direction and it's actionable. They, it's so specific enough that they can take action on it.

• 2:10:28Speaker 01

  But I underspecify it on purpose so that enable 43,000 amazing people to make it even better than I imagined. And so when I'm working with engineers, when I'm working with people, I think about who, what problem am I trying to solve? Who am I working with? And the level of specification, the level of architecture definition relates to that. And, and so everybody's going to have to learn how, where in the spectrum of coding they want to be. Writing a specification is coding.

• 2:11:07Speaker 01

  And so you might decide to be quite prescriptive because there's a very specific outcome you're looking for. You might decide that, you know, this is an area you want to be much more exploratory. And so you might underspecify and enable you to go back and forth with the AI to even push your own boundaries of creativity. And so this artistry of where you are in the spectrum, this is the future of coding.

• 2:11:32Lex Fridman?

  But just to linger on it outside of coding, I think a lot of people, rightfully so, are worried about their jobs, have a lot of anxiety about their jobs, especially in the white collar sector.

• 2:11:45Lex Fridman?

  I don't think any of us know what to do with tumultuous times that always come when automations and new technology arrives. And I just, first of all, I think we all need to have compassion and the responsibility to feel sort of the burden of what the actual suffering feels like for individual people and families that lose their job. I think whenever you have transformative technology, like that's coming with with artificial intelligence, there's going to be a lot of pain. And I don't know what to do about that pain. Hopefully,

• 2:12:23Lex Fridman?

  it creates much more opportunities for those same people of for the same kind of job as the tooling evolves and makes them more productive and makes them more fun, hopefully, as it does in the programming. I've I've been having so much fun programming, I have to say, like, I've never had this much fun. So hopefully, it makes their job automates the boring parts and makes the creative parts, the ones that the human beings are responsible for. But still, there's going to be a lot of pain and suffering.

• 2:12:53Speaker 01

  So my first recommendation before, and this is now how I deal with anxiety. In fact, we just talked about it earlier, enormous anxiety about the future, enormous anxiety about the pressure, enormous anxiety about uncertainty. I first break it down. And then I'm going to tell myself, okay, there are some things you can do something about. There's some things you can't do anything about. But for the stuff that you can do something about, let's reason reason about it and let's go do it. If we were to hire a new college graduate today, and I have a choice between two, one that have that is no clue what AI is, and one that is expert in using AI, I would hire the one who's expert in using

• 2:13:38Speaker 01

  AI. If I had an accountant, a marketing person, the one that is expert in using AI, supply chain, customer service, a salesperson, business development, a lawyer.

• 2:13:53Speaker 01

  I would hire the one who is expert in using AI. And so I would, I would advise that every college student, every, every teacher should encourage their student to be, to go use AI. Every college student should graduate and be an expert in AI. And every, everybody, if you're a carpenter, if you're, you know, electrician, go use AI. Go see what it can do to transform your current job, elevate yourself. If I were a farmer, I would absolutely use AI. If I were a pharmacist, pharmacist, I would use AI. I want to see how, what it could do to elevate my job. So that I could be the, I could be the innovator to revolutionize this industry myself.

• 2:14:38Speaker 01

  And so that, that would be the first thing that I would do. And, and then I would also, I would also help them.

• 2:14:45Speaker 01

  It is the case that the technology will dislocate and will eliminate many tasks.

• 2:14:54Speaker 01

  And because it will automate it, if your job is the task, if your job is the task, then you're very highly going to be disrupted. If your job's purpose includes you certain tasks, then it's vital that you go learn how to use AI to automate those tasks. And then there's the world of spectrum in between.

• 2:15:16Lex Fridman?

  And by the way, the beautiful thing about AI, so the, the, the chatbot versions is you can break down, you have anxiety and you can break down the problem by talking to it. Like I've recently, it's really just incredible how much you can think through your life's problems and through, and I don't mean like therapy problems. I mean, like very practically, okay, I'm worried about my, literally, I'm worried about my job. What are the skills? What are the steps I need to take? How do I get better at AI? Everything you just said, you can literally ask and it's going to give you a point by point plan. I mean, it's just a great life coach, period.

• 2:15:53Speaker 01

  I don't know how to use AI and the AI goes, well, let me show you.

• 2:15:56Lex Fridman?

  Exactly. It's very meta, but it's, it's kind of incredible. So people definitely should.

• 2:16:01Speaker 01

  You can't walk up to Excel and say, I don't know how to use Excel. You're done.

• 2:16:04Lex Fridman?

  I mean, that's really what AI has done for me in all walks of life is that initial friction of being a beginner of using a thing for the first time. I can literally ask about any single thing. What are the first steps I need to take?

• 2:16:18Speaker 01

  That's right.

• 2:16:18Lex Fridman?

  And that that handholding that it does, removing the friction of all the experiences that the world offers is, you know, like I mentioned to you offline, you mentioned I'm going to China and Taiwan.

• 2:16:32Speaker 01

  So awesome. So excited for you.

• 2:16:34Lex Fridman?

  Where do I, what do I, how do I, all of those questions immediately answered. It is beautiful.

• 2:16:39Speaker 01

  Well, when you, when you go to Taiwan, just ask AI, what are Jensen's favorite restaurants in Taiwan?

• 2:16:45Lex Fridman?

  Yeah.

• 2:16:46Speaker 01

  And it will actually tell you.

• 2:16:47Lex Fridman?

  Oh, yeah, yeah. Is it accurate?

• 2:16:48Speaker 01

  Yeah, yeah.

• 2:16:48Lex Fridman?

  All right.

• 2:16:49Speaker 01

  It's all over, all over Taiwan.

• 2:16:52Lex Fridman?

  Well, you're, you're a rock star over there. And, and like we also mentioned offline, maybe our paths will cross, which would be really wonderful.

• 2:16:59Speaker 01

  Computex, Nvidia GTC Taiwan.

• 2:17:03Lex Fridman?

  Do you think there are some things about human nature, about human consciousness? That is fundamentally non computational? Maybe something a chip, no matter how powerful, can never replicate?

• 2:17:19Speaker 01

  I don't know if the chip will ever get nervous. And that's the, you know, of course, the conditions by which that causes anxiety or nervousness or whatever emotion.

• 2:17:32Speaker 01

  I believe that AI will be able to recognize those and understand those. I don't think my chips will feel those. And therefore, the, how, how that anxiety, how that feeling, how that excitement, how that, how that, you know,

• 2:17:52Speaker 01

  all of those feelings manifest in human performance. For example, extremely amazing human performance, athletic performance, you know, average or lesser than average. That, that entire spectrum of human performance that comes out of exactly the same circumstances for different people, manifesting in different outcome, manifesting in different performance. I, I don't think there's anything about, anything that we're building that would suggest that two different computers being presented with all of exactly the same context would perform. Of course, it would produce statistically different outcomes, but it's not because it felt different.

• 2:18:35Lex Fridman?

  Yeah, the subjective, boy, there's something truly special about the subjective experience.

• 2:18:42Lex Fridman?

  That we humans feel. Like I mentioned to you, I was, I was, I was pretty nervous talking to you. Like I mentioned to you that the hope, the fear, the anxiety and just life itself, the richness of life. How amazing everything is, how deeply we fall in love, how deeply our hearts get broken, how afraid we are of death and how much pain we feel when our loved ones pass away. All of that, the whole thing. I know it's very hard to think AI being able to, a computational device being able to do that. But there's so many mysteries about this whole thing that we're yet to uncover that I am open to be surprised. I've been surprised a lot over the past

• 2:19:24Speaker 01

  Yeah.

• 2:19:25Lex Fridman?

  Few months and few years. Scaling can create some incredible miracles in the space of intelligence. Has been truly marvelous to watch, so I'm open to surprise.

• 2:19:35Speaker 01

  And, and it's just really important to, to break down what is intelligence. And the word, that word we use all the time, it's not a mysterious word. Intelligence has a meaning, you know? And, and it's a system that, you know, it's, it's something that we do that includes perception and understanding and reasoning and the ability to do plan and, you know, that, that loop, that loop is, is the fundamentally what intelligence is. Intelligence is not one word that is exactly equal to humanity. And that's, I think is really important to separate the two. We have two words for that.

• 2:20:17Speaker 01

  I'm not, I don't over fantasize about, and I don't over romanticize about intelligence. Intelligence is, and people have heard me say it before, I actually think intelligence is a commodity. I'm surrounded by intelligent people. And I'm surrounded by intelligent people, more intelligent than I am, in each one of the spaces that they're in. And yet I have a role in that circle. It's actually kind of interesting. They're more educated than I am.

• 2:20:50Speaker 01

  They went to better schools than I did. They're deeper than in any in this field that they're in, all of them. I have 60 of them. They're all superhuman to me. And somehow I'm sitting in the middle orchestrating all 60 of them. And so you got to ask yourself, what is, what is it about a dishwasher? That allows that dishwasher to sit in the middle of superhumans? Does that make sense? And so, but that's my point. My point is, intelligence is a, is a functional thing. Humanity is not, not specified functionally. It's a much, much bigger word.

• 2:21:30Speaker 01

  And, and our life experience, our tolerance for pain, our determination. Those are, those are different words in intelligence. And so the, the thing that I, I want to help the audience understand. If I could give them one thing is, is intelligence is a word that we've elevated to very high form over time. The, the word we should really elevate is humanity. Character, humanity, all of those things. Compassion, generosity. All of the things that you say just now. I believe those are superhuman powers. And that now intelligence is going to be commoditized. Because we've spoken about it, the most important thing is your education.

• 2:22:14Speaker 01

  The most, now even, even when they said the most important thing is your education. When you went to school, there's more than just knowledge that you gained. And so, but unfortunately, our society had put everything into one single word. And life is more than one word. And I'm just telling you, my life would suggest that being lower on the intelligence curve than everybody around me, doesn't change the fact I'm the most successful. And so, and, and I think, I think that, that kind of is, I'm trying, hopefully to inspire everybody else that don't let this democratization of intelligence, this commoditization of intelligence.

• 2:22:59Speaker 01

  You know, cause you anxiety, you should be inspired by that.

• 2:23:01Lex Fridman?

  Yeah, I think AI will help us celebrate humans more. And I'm certainly humanity and human first. And I think what makes this world incredible is humans forever will be so. And just AI is this incredible tool that makes us

• 2:23:20Speaker 01

  That's exactly right.

• 2:23:20Lex Fridman?

  Humans more powerful.

• 2:23:21Speaker 01

  That's exactly right.

• 2:23:24Lex Fridman?

  So much of the success of Nvidia and the lives of millions of people that I mentioned depend on you.

• 2:23:33Lex Fridman?

  But you're just one human, like we mentioned, mortal, like all of us. Do you think about your mortality? Are you afraid of death?

• 2:23:44Speaker 01

  I really don't want to die. I have a great life, I have a great family.

• 2:23:51Speaker 01

  I have really important work.

• 2:23:56Speaker 01

  This is, this is not a once in a, once in a lifetime experience suggests that it has been experienced by many people, just not one person. This is a once in a humanity experience, what I'm going through. Nvidia is one of the most consequential technology companies in history. We're doing very important work. I take it very seriously.

• 2:24:22Speaker 01

  And, and so some of the, some of the things that, that of course are, are practical things. Like, how do we think about succession planning? And, and I'm famous in saying that I don't believe in succession planning.

• 2:24:38Speaker 01

  And the reason, the reason for that, the reason for that isn't because I'm immortal. The reason for that is because if you're worried about succession planning, if you're worried, all that anxiety of succession planning, then what should you do about it? Then you break it all way back down. The most important thing you should do today, if you care about the future of your company, post you, is to pass on knowledge, information, insight, skills, experience as often and continuously as you can, which is the reason why I continuously reason about everything in front of my team. Every single meeting is about a reasoning meeting. Every moment I spend inside a company, outside a company,

• 2:25:21Speaker 01

  is about passing on knowledge to people as fast as I can. Nothing I learn ever sits on my desk longer than, you know, a fraction of a second. I'm passing that information, that know, oh my gosh, this is cool. Before I even finished learning all of it myself, I've already pointing it to somebody else. Get on this, this is so cool. You're gonna wanna, you're gonna wanna learn this. And so I'm constantly passing knowledge, empowering people, elevating the capability of everybody around me. So that

• 2:25:54Speaker 01

  the outcome that I, that I seek, that I hope for, is that I die on the job, you know? And, and hopefully I die on the job instantaneously. Yeah. And there's no long periods of suffering, you know? It's a

• 2:26:07Lex Fridman?

  Well, from a fan perspective, given your, your extremely,

• 2:26:15Lex Fridman?

  your enormous positive impact on civilization, of course, I hope you keep going. But also it's just fun to watch what Nvidia is doing. You know, it's just the rate of innovation. And I'm a huge fan of engineering. It's so much incredible. Engineering is continuously being done by Nvidia. It's just fun to watch. It's a celebration of humanity. It's a celebration of great builders. It's a celebration of great engineering. So it represents something special. So I hope you and Nvidia keep going. What gives you hope about this whole thing we got going on? About humanity, about the future of humanity? When you look out and you think about the future quite a bit, when you look out 10, 20, 50, 100 years from now, what gives you hope?

• 2:26:57Speaker 01

  I, I've always had, I've always had a great confidence in, in the, in the kindness,

• 2:27:08Speaker 01

  the generosity,

• 2:27:13Speaker 01

  the compassion, the human capacity. I've always been extremely confident of that. Sometimes.

• 2:27:24Speaker 01

  More so than I should. And, and I get taken advantage of, but it doesn't, it doesn't ever cause me not to. I start with, always, that, that people want, want to do good. People want to help others. And vastly, I am proven right. Constantly proven right. And, and often, it exceeds my expectations. And, and so I have complete confidence in the human capacity.

• 2:28:03Speaker 01

  I think the, the, the thing that, the things that give me incredible hope is what I see as, as I extrapolate, as I, what I see now is possible. And as I extrapolate, based on the things that we're doing, what will very likely happen. And, and, and that there's so many things that we want to solve. There's so many problems we want to solve. There's so many things that we want to build. There's so many good things that we want to do, that are now within our reach and within the reach of my, my lifetime. You just can't possibly not be romantic about that. You know what I'm saying?

• 2:28:47Lex Fridman?

  Yeah. What an exciting time to be alive.

• 2:28:49Speaker 01

  Yeah.

• 2:28:50Lex Fridman?

  Like truly, truly so.

• 2:28:52Speaker 01

  How can you not be romantic about, about, about that? The, the, the fact that, that there is a, there, it's a reasonable thing to expect. The end of disease. It's a reasonable thing to expect. It's a reasonable thing to expect that pollution will be drastically reduced. It's a reasonable thing to expect that traveling at the speed of light. Is actually in our future. And then, you know, not, not for long distances, but short distances. You know, and people ask me how. And well, first of all, very soon, I'm going to put a humanoid on a spaceship. And it's going to be, you know, my humanoid. And, and we're going to send it out as soon, you know, as soon as possible.

• 2:29:35Speaker 01

  And it's going to keep improving and enhancing along the flight. And then when it's time, all of the, all of my consciousness has already been, you know, so much of my life has been uploaded in the internet. Take all my inbox, take everything that I've done, everything I've said. You know, it's been collecting, becoming my AI. And I'm just, you know, when the time comes, you know, we'll just send that at the speed of light. Catch up with my robot.

• 2:30:01Lex Fridman?

  Oh, that's brilliant. I mean, but for me, that's sort of application focused. But also for me, the curiosity, uh, maxing perspective. I just, all of those mysteries, so much fascinating scientific questions there.

• 2:30:16Speaker 01

  Understanding the biological machine is, is right around the corner. It's, it's not 10 years, it's five years probably.

• 2:30:22Lex Fridman?

  And then your biological machine, the, the human mind and cracking physics, theoretical physics open. It's so exciting.

• 2:30:27Speaker 01

  Explaining consciousness, that one would be awesome.

• 2:30:30Lex Fridman?

  And it's all within our reach.

• 2:30:32Speaker 01

  Yeah.

• 2:30:32Lex Fridman?

  Uh, Justin, thank you so much for everything you've done over the years. Thank you for everything you're doing for the world. Thank you for being who you are. Uh, I can tell you're a great human being. And, uh, I wish you incredible success this year. I can't wait. As a fan, I can't wait to see what you do next. And hopefully I'll see you in Taiwan and thank you so much for talking today.

• 2:30:54Speaker 01

  Thank you, Lex. I had a great time. And, and also if I could just say one more thing.

• 2:30:58Lex Fridman?

  Yes.

• 2:30:58Speaker 01

  And thank you for all the interviews that you do, the depth, the, the respect that you go through with, and the research that you do to reveal, you know, for all of us, uh, the, the amazing people that you've interviewed over the years. Uh, I've enjoyed, I've enjoyed them immensely. And, and, and as an innovator to have created this long form, unbelievable. And, and yet, you know, it's just captivating. So anyways, thank you for everything you do.

• 2:31:27Lex Fridman?

  It means the world. Thank you, Justin.

• 2:31:29Speaker 01

  Thank you, Lex.

• 2:31:30Lex Fridman?

  Thank you for listening to this conversation with Jensen Huang. To support this podcast, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback, and so on. And now, let me leave you with the words from Alan Kay. "The best way to predict the future is to invent it." Thank you for listening, and I hope to see you next time.