Nvidia CTO Michael Kagan: Scaling Beyond Moore's Law to Million-GPU Clusters

Recorded live at Sequoia’s Europe100 event: Michael Kagan, co-founder of Mellanox and CTO of Nvidia, explains how the $7 billion Mellanox acquisition helped transform Nvidia from a chip company into the architect of AI infrastructure. Kagan breaks down the technical challenges of scaling from single GPUs to 100K and eventually million-GPU data centers. He reveals why network performance—not just compute power—determines AI system efficiency. He discusses the shift from training to inference workloads, and his vision for AI as humanity's "spaceship of the mind," and why he thinks AI may help us discover laws of physics we haven’t yet imagined. Hosted by Sonya Huang and Pat Grady

• 0:00Speaker 00

  One of the interesting things about Nvidia is the culture of win-win. We are not after taking a bigger piece of existing pie. We are after baking our bigger pie for everybody. And our success is our customer's success. It's not, our success is not the failure of our competition. And I think fusing together conventional computing, human machines, and accelerated computing that provided by Nvidia, it actually gives Nvidia and Intel channels to the market, or expanding the market and serving the markets

• 0:44Speaker 00

  that otherwise was more challenging.

• 1:05Speaker 01

  We are delighted to hear today from one of the legends of the semiconductor industry, Michael Coggan, the CTO of Nvidia. Michael was formerly chief architect at Intel and then co-founder and CTO of Melanox, which Nvidia acquired for $7 billion in March 2019. In the time since, Michael has been a major driver of Nvidia's dominance as the AI compute platform, in large part due to the role of Melanox and Interconnect in driving chips beyond Moore's Law. The AI race is ultimately a silicon race to squeeze the most intelligence possible out of each unit of silicon. And Michael takes us down a journey about how the compute frontier has evolved from squeezing more transistors onto a single chip to bringing together thousands and hundreds of thousands of chips into a single fabric connected by networking

• 1:50Speaker 01

  or an AI data center. Michael has been driving the compute frontier forward for more than four decades, and we're honored to have him on today's show.

• 2:00Speaker 02

  Okay, we're here with Michael Kagan, the CTO of Nvidia, currently the world's most valuable company. Michael, thank you for joining us.

• 2:06Speaker 00

  Thank you, my pleasure.

• 2:08Speaker 02

  So I thought where we could start, our partner Sean likes to make the case about every six months that Nvidia would not be Nvidia without Mellanox. Mellanox is the company that you co-founded some 25 years ago and have been a part of through this day. So can you kind of paint that picture for us? Why is it that the Mellanox acquisition was so critical to Nvidia?

• 2:33Speaker 00

  You know, there was a huge transition in the world in terms of computing and need for computing. And it grows, grows exponentially. And one of the things that we usually estimate linearly, but the world was exponential. And exponential growth now is actually accelerated. It used to be like Murlow, which is a basic silicon. And it was, you know, twice every other year. And regardless, you know, the discussion that Murlow in terms of physics is not quite running anymore. The, once the AI kicked in, which was in the 2010, 2011,

• 3:18Speaker 00

  and kicked in when GPU from graphic processing unit became general processing unit, actually, where it's running workloads where the first time the AI workload was run on the GPU,

• 3:33Speaker 00

  taking advantage of the programmability and parallel nature of this machine. The requirements for performance started to grow up at a much higher coefficient. So the models started to go up in terms of size and capacity, 2X every three months, which requires now 10X or 16X a year, performance growth versus, versus old school of 2X, twice every other year.

• 4:07Speaker 00

  And in order to grow the scale, you need to innovate and you need to develop solutions at much higher scale than just basic component. And that's where network kicks in. That's where the network is and and there is a multiple layers of of scaling performance that requires high-speed networks and high-performance networks. The one is what we call scale up. Basically, if you're going back to the CPU days, scaling up was more low, more transistors, and also some advances in the microarchitecture, like out-of-body execution, and at some point, multi-core, and so on and so forth.

• 4:49Speaker 00

  But so this is the basic building block of computing. In the GPU world, the basic building block is the GPU. And in order to scale it up more than you can do on a single piece of silicon with the lot of advances that we are doing with the microarchitecture and advanced technologies, you actually need to do something on the scale of on the on the sort of multi-core CPU, but on much larger scale. And that's what we are doing with the NVLink. This is a scale-up solution. So our GPU, what we call GPU today, is a rack-sized machine. You need a forklift to lift it.

• 5:31Speaker 00

  So, you know, if you order just GPU on Amazon, just don't wonder that you'll show up this huge rack of. Yeah, people think chip, but it's really a system. Right. And that's just one GPU. Okay, so, basic building block, a very basic computer that application software is running on is this GPU. And it is not just silicon, it's not just hardware, it's not just wires, but there is also a software layer that exposes CUDA as the API. And that's what actually enables to pretty much seamlessly scale. I'm simplifying a little bit the story, but seamlessly scale from one component that

• 6:15Speaker 00

  used to be single GPU all the way up to 72, maintaining the same software interface. And once you get this building block as big as it conceivably can be built in terms of, you know.

• 6:33Speaker 00

  Power, cost, efficiency, then you start scaling out. Yeah. And scale out is, it means you take many of these building blocks, connect them together. And now on the algorithm level, on the application level, you actually split your application to multiple pieces running in parallel on these big machines. Hmm. And that's again where network comes in and there is a, so if you talk about scale up, you basically made

• 7:04Speaker 00

  memory like domain to go beyond single compute node or a single GPU. And that's actually the first thing where Mellanox's technology comes in because before Mellanox acquisition, the scaling up of Nvidia with the NBLINK was limited to a single node machine. Going outside of single compute node, if the seventy-two GPUs are, it's actually thirty-six computers. Each one has two GPUs and they're wired together, so present all this as a single GPU. Getting connectivity outside of the single node, it's not just plugging in the wire to the connector.

• 7:47Speaker 00

  It's a lot of software, it's a lot of technology within the network, how to make multiple nodes to work as the single, the single machine. And that's where Mellanox's first, just immediate in terms of, you know, the, the way we go upstream, that's, that's the first one. The second one is how do you split the operation across multiple machines? And the way to do it, if I have a task that it takes one GPU to do one second, if I want to accelerate it, I split it to ten, to, you know, thousand pieces and send each piece to different GPU. And now in one millisecond, I get done whatever I was doing in the second.

• 8:32Speaker 00

  But, you know, you need to. Communicate this partial uh split, job split, split, split the task. Then you need to consolidate the results. And every time you are, and you, you, you, you run this, you know, multiple times. You have uh multiple iterations or multiple uh applications running to us. So, so that there is a part of doing communication, a part of doing computation.

• 8:56Speaker 02

  Hmm.

• 8:57Speaker 00

  Now, the thing is that you want to split it to as many pieces as you possibly can. Because that's, that's your speed up factor. But then, if your communication is actually blocking you, you, you, you waste time, you waste energy, you wa- wa- waste everything. So what you need to do, you need to have a very fast communication. So you split it to the many, many pieces. And so, eve- each, each piece takes very little time. But then there is a, another piece that is communicated and you need to feed it to this time. So that's, that's just pure bandwidth. And another thing is that when you tune your application, you tune your application so that communication can be hidden behind uh computation.

• 9:40Speaker 00

  And it means that if communication for some reason gets longer, then everybody waits.

• 9:47Speaker 02

  Hmm.

• 9:47Speaker 00

  So it means that what you need to do in the network, you need to have not only just raw performance like what called hero numbers, you know, I can get to that many gigabits per second. I also need to make sure that no matter who communicate to whom, the latency, the, the time it takes is distribution is very narrow.

• 10:06Speaker 02

  Hmm.

• 10:07Speaker 00

  So if you look at uh, at other network technologies or other network products, you know, you, you go to the hero numbers, you know, sending bit from one place to another, it's basically physics.

• 10:17Speaker 02

  Yeah.

• 10:17Speaker 00

  It's so, it's, it's pretty much close to the, to everyone. You know, we are a little bit better, but that's not, that's not, not the big advantage. But when you Do it thousands of times and it takes the same time to do it versus very wide distribution of other technologies, then machine becomes less efficient. So instead of being able to split your job to thousand GPUs, you can split it only to ten GPUs because you need to accommodate for the jitter on the network within the computation phase. So inherently, network determines the performance of this cluster. And we look at this data center

• 11:01Speaker 00

  as basically a single unit of computing.

• 11:03Speaker 03

  Yeah.

• 11:03Speaker 00

  Okay, single unit of computing means that you look at this, you start architecting your components, your software and your hardware, at the point where, you know, this is the data center, this is hundred thousand GPUs that we want to make them work together, we need to make multiple chips, compute chips two, network chips five.

• 11:28Speaker 03

  Hmm.

• 11:29Speaker 00

  Okay, so this is a scale, just, you know, in terms of, you know, what's the impact? Just, and what's the investment you need to make to create this single unit of computing? So that's where Melanox technology came in. And another aspect of this is there is a, we talked about a network that connects the GPUs to run the task. But there is a other side of this machine, which is customer facing. So you need to, this machine needs to save, to serve multiple tenants, and this machine needs to run operating system, you know,

• 12:11Speaker 00

  every computer runs operating system. Another part of the Melanox technologies is what we call Bluefield DPU, Data Processing Unit, which is actually the computing platform to run the operating system of the data center.

• 12:26Speaker 03

  Hmm.

• 12:27Speaker 00

  In conventional computer have a CPU that runs operating system and runs application software. And there is a many things we can talk about, you know, the advantage versus disadvantage, but there are two key things. One is how much

• 12:44Speaker 00

  Time do you spend on your general purpose computing to run the application? You need to, you want to maximize it. And another thing is, how do you isolate your infrastructure computing from the application computing? Because, you know, viruses and, and, and cyber attacks and so on and so forth. And being able to run infrastructure computing on a different computing platform actually reduces significantly the attack front versus, especially in the side channel attacks versus what, what happens if you run it on the same computer. If you remember, there was a virus, well, almost 10 years ago, there was this meltdown

• 13:29Speaker 00

  and all this cyber attacks of the side channel on CPUs. And this cannot happen or the, the attack surface is reduced significantly when you run the different. So on the other side of the network, we have also technology. So that's what makes data center to be more efficient. And I, well, I may be not objective, but I do agree that this merger of Mellanox and, and Nvidia, and it actually goes both ways. I don't think that networking business of now it's Nvidia, previous Mellanox could have been growing that, that significantly. Yeah.

• 14:10Speaker 00

  As, as it grew now, I think we are the fastest growing internet, internet business, you know, let, let, let, let alone NVLink and, and Infiniti. Yeah. But just internet business is the fastest growing business ever.

• 14:26Speaker 03

  What are the things that break as you get to 100,000, maybe eventually a million GPU clusters? And how do you use software to help design around that?

• 14:35Speaker 00

  It's, it's, it's a multi-stage challenge. Okay. One of the things that you need to.

• 14:44Speaker 00

  Need to keep in mind, it's not very obvious for all the engineers that when you design the machine, you think how to operate it. Well, you know, you have these components and they are working and now just let's figure out. Okay, so the thing is that the hardware component works at ninety-nine point nine-nine-nine, whatever percent of the time. And it's usually okay if you are dealing with single box with couple of them. But if you are building hundred thousand component machine, a hundred GPUs machine, which means in terms of components, there is a millions of them, the chance that everything works is zero. So something is definitely broken.

• 15:24Speaker 00

  And you need to design it both from hardware and from software perspective to keep going, to keep going as efficient as you can, to keep, you know, your performance, to keep your power efficiency and of course, keep the service, service running. So this is challenge number one, even before you go to millions. It's a, this challenge actually starts at, you know, a few, few tenths of thousands. That's number one. Number two is when you are running this workloads, it is really important to, it, sometimes you run single job on the entire, entire data center and then it's, you know, you

• 16:07Speaker 00

  need to write the software and you need to provide all the interfaces to the software to place the different parts of the jobs, jobs more efficiently. Building networks at this scale is a very different story than building network, compute network on this scale is very different story than build just general purpose data center network. General purpose data center network is internet, it's not a big deal. Well, it is a big deal, but it is, it is different deal, it's, you are, you are serving, you know, loosely coupled collaborative microservices that create the, you know, service that you, that you see as a, as a customer from outside. Here you are.

• 16:50Speaker 00

  Running one single application on hundred thousand machines.

• 16:54Speaker 02

  Yeah. Is that specific to training workloads or is that also true with inference workloads?

• 16:58Speaker 00

  It's it's true. It's true for everything.

• 17:00Speaker 02

  Okay.

• 17:01Speaker 00

  It depends, you know, to what scale and inference is yet another, yet another topic that we we touched, you know, till recently, training was the key thing, you know, a lot of GPUs and there is a very specific way of training is being done. You basically copy this way another model on multiple, multiple machines or multiple sets of machines and run them then consolidate the results and so on and so forth. On the inference, it's the story is a little bit different. But the thing is that you need to provide the hooks on the hardware and on your low level software, system software for application and for scheduler to place

• 17:42Speaker 00

  the job and place the different parts of the job in the most efficient way.

• 17:47Speaker 02

  Mm hmm.

• 17:47Speaker 00

  On the, you know, and as long as as your machine fits a building, which is about hundred thousand GPUs now talking about, you know, gigawatt, it's all all power driven, okay? The challenge is that for many reasons, reasons, you want to split your workloads across multiple data centers. And sometimes the data centers are at a distance of many kilometers, many miles. It's it may be across the across the continent. And this, you know, it's comes with the yet another challenge, which is speed of light.

• 18:26Speaker 02

  Yeah.

• 18:27Speaker 00

  Okay, now the the latency variance between different parts of your machine is dramatically different. And what is even more challenging is that when you talk about networks, the congestion on the network is one of the key problems that.

• 18:48Speaker 00

  Deteriorate network performance. Yeah. And managing congestion across such a latency differ latency difference. It's not like, you know, in old old telco days, you put some box at the edge of your data center with a huge buffers and it's a shock observer for congestion. Huge buffer is is is not is not good. You know, some bigger is not better. There is a there is famous statement from very famous Wuhan. And so so we need to and these buffers are basically these devices are basically to isolate the external world from the internals.

• 19:29Speaker 00

  And but when you want to run a single workload across data centers that are distant by kilometers, you need to be every machine on one side to be aware what whom does it communicate to, whether it's short, short, short communication, long communication, and adjust all the communication patterns accordingly. So you don't need this big buffers because big buffers is a jitter. Yeah. And so we have a we have a technology, we actually developed it recently, technology. You know, all our Ethernet network is Spectrum X. And this is the device that we designed and developed it based on the spectrum spectrum

• 20:11Speaker 00

  switch that we put on the edge of the data center. And it provides all the information and telemetry needed for for the endpoints to adjust for the congestion. Yeah.

• 20:25Speaker 03

  Can we talk a little bit more about training versus inference? Like, how does the shape of workload differ when you're doing, I guess, I guess backprop is a lot more computationally intensive, forward pass less so. But like, how does how does the workload differ? And then are you seeing customer demand start to shift from pre-training towards inference? Or do you think it's still very training heavy right now?

• 20:46Speaker 02

  And if I could just ask a quick follow up question to that, will people be running inference workloads on the same data centers that they use for training? Or will these end up being two separate? Because they're different optimizations, people end up using two different sets of data centers.

• 21:01Speaker 00

  Okay, yeah, that's, that's, that's a great question. And let me start with the, with the first one. So, so training has two phases. One is inference, which is just for, for propagation and then backpropagation to adjust the weights. And for data parallel training, it's yet another phase to consolidate the results of the weights update across multiple model copies. So till recently, it was a main driver of the compute. Because till not very long ago, it's maybe two years, which is ages in the, in the AI, AI era.

• 21:47Speaker 00

  The inference or AI was mainly perceptual. So you show the picture, that's a dog. You show, you know, the photo of the person and here's that's, that's Michael and that's Sonia. So that's single path and, and that's it. Then came, became generative AI, where actually you get the recursive generation. So when you post the prompt, then it's not just one inference, it's many inferences. Because for every token, when you generate text or you generate picture, for every new token, you need to go through the entire machine all over again. So instead of one, one shot inference, then there is more.

• 22:33Speaker 00

  And then now there is a reasoning, which means the machine starts, you know, sort of thinking.

• 22:40Speaker 02

  Yeah.

• 22:40Speaker 00

  If you ask me what time is it now, I can tell you it's easy, right, what time is now. But if you ask me more, more complicated question, then I need to think. I probably need to wait or compare multiple solutions or multiple pathes. And every such a thing is inference.

• 22:59Speaker 02

  Yeah.

• 22:59Speaker 00

  Every such a thing is inference. And inference itself has actually two, two phases. One is much more computer intensive and the other one is memory intensive. It's, it's what we call pre-fill.

• 23:16Speaker 00

  Because when you do the inference, you have some some sort of background, right? Which is prompt, which is some relevant data that you need to process and create the context for the to generate the answer. And this is very compute intensive, it's not much memory intensive. And the other part is actually generating the answer, which is the decode part of the of the inference, where you generate token by token.

• 23:44Speaker 03

  Uh huh.

• 23:44Speaker 00

  Okay, well, there are some technologies that you can generate more than one token, but it's still, you know, single path is much less than the final answer.

• 23:54Speaker 03

  Hmm.

• 23:55Speaker 00

  So if you combine all these things together, inference inference demand for computing is not actually not less than training. It's it's it's actually even more and and there are two reasons for this. One is that what I explained that there was a much more computing than it used to be for the inference. The other thing is, you know, you train model once, but you infer many times.

• 24:19Speaker 03

  Yeah.

• 24:19Speaker 00

  You know, ChatGPT, you know, billion of people or yeah, it's almost billion of people, right? There's customers, they are pounding them all the time in the same model. They trained at once.

• 24:29Speaker 03

  Now they're making videos, so that's.

• 24:31Speaker 00

  Right, right. Now they're making videos and you can generate and then and you know, everybody is is is do doing the inference. My wife, I think she talks to ChatGPT more than to me. Once she discovered that's that's her best friend. Uh, so so so the in terms of now to your question about about machines, you can infer on the phone. Okay, so there is a definitely going to be a much smaller scale uh installations uh for for inference. Yeah, it's like mobile devices. If you if you look at the data center scale, at the data at the data center scale and on the

• 25:15Speaker 00

  it's it's a efficiency of the programming, programmability is much more viable than optimizations uh in optimizations for hardware. And you know, every every hardware

• 25:33Speaker 00

  Instance, it has its own, its own, its own cost and its own, its own drawback. So as long as, as long as you don't identify, and I don't think, besides, you know, this, we actually did, it's, it's, it's very similar GPU, it's same, it's same programming model as a GPU for pre-fill versus, versus decode. I think, I don't remember when, when, when it happened, but actually we announced that we are building the GPU SKU that is optimized for pre-fill. So you will have, it can, it can do decode and decode GPU can do pre-fill, so, but you

• 26:17Speaker 00

  can equip your, equip your data center with the SKUs or that pre-fill versus SKUs that, that are for decode to, to, to optimize for like typical, typical use.

• 26:28Speaker 02

  Yeah.

• 26:28Speaker 00

  But if your workload shifts for more decode or more pre-fill, you can use either one of them to, to, to compensate. And this is the, this is the importance of programmability. The same interfaces for GPUs, it's based on CUDA and, and TAP, which is, that's what made Nvidia Nvidia before Melanox.

• 26:54Speaker 02

  Yeah, yeah. Can I ask you a question about data center scaling? So for many decades, we had Moore's law and chips got more and more dense and produced better and better performance. And then we ran into the laws of physics and chips just couldn't get more dense because their quantum mechanical properties caused them to break down. And so then we had to scale up to the rack level and now we got to scale out to the data center level. Is there some analogous law of data, data center scaling that says when data centers get too big, the communication overhead causes the performance to break down? Or, or just said differently, or maybe said more simply, is there a natural limit to how big data centers can get?

• 27:35Speaker 00

  I think there is, there is a practical limit of how much energy you can consume within, within given size of the, of the data center.

• 27:44Speaker 02

  But if you were surrounded by nuclear power plants and you, and the energy was available, would the, would the data center itself perform?

• 27:52Speaker 00

  I don't know, I'm not, I'm not an expert in the, in the, in the construction even. But if you surround, you know, there's, there are energy coming in, now the heat, heat is going out. So there is, there is a whole, we are now basically moved pretty much entirely to the liquid cooling.

• 28:09Speaker 03

  Yeah.

• 28:09Speaker 00

  And one of the reasons we did it is to enable much denser, denser compute, compute power. We couldn't build as dense computing as we be, as we're building now with the, with the air cooling.

• 28:21Speaker 03

  Yeah.

• 28:22Speaker 00

  So there's a whole bunch of technologies coming to, to help this more and more denser now. The last big data center, which is like X, X, XAI scale is a hundred or hundred fifty megawatt. Now we're talking about gigawatt data centers, people talking about ten gigawatt data center. So, you know, there is a, looking forward to build much, much bigger data centers and.

• 28:48Speaker 03

  Are you sending the data centers to other space?

• 28:50Speaker 00

  I, I, I, I think, I, well, one of, one of, one of the things that, that determines the speed of data center deployment is, you know, how fast concrete get, get stable.

• 29:06Speaker 03

  Fair enough. So before starting Mellanox, you were at Intel.

• 29:11Speaker 00

  That's right.

• 29:11Speaker 03

  Sixteen, sixteen years?

• 29:13Speaker 00

  Sixteen years.

• 29:13Speaker 03

  You became chief architect. Nvidia and Intel recently announced a partnership. Can you share a little bit about what the vision for that might be?

• 29:21Speaker 00

  The starting point is that computing changed in last decade or a little bit more than decade. Nvidia started as the accelerated computing company.

• 29:35Speaker 00

  Video games was the first. And then it evolved to, to AI, which is the new way of data processing. So you cannot just general one, one human machine just is not capable of, of.

• 29:52Speaker 00

  Being used as a platform to solve the problem like, you know, programming when human machine is just explaining somebody what to do. Okay, I can explain many things and I can explain uh many people what what to do, but I can't I can't explain how to distinguish between cat and dog. Right, so there is a new challenges that AI solves and you need acceleration there. And our partnership with Intel is actually fusing accelerated computing with the general purpose computing. Because general purpose computing is not going away. Everything will be accelerated, but we accelerate the purpose computing. We accelerate the applications. We but so so and you know,

• 30:37Speaker 00

  X R six is the architecture that is dominant there. And you know, it would serve greatly both companies. That's actually one of the interesting things about Nvidia. It's uh it's the culture of win-win. Okay, we are not after uh taking a bigger piece of existing pie. We are after baking our a a bigger pie for everybody. And uh the success, our success is our customer's success. It's not, our success is not the failure of our competition. Our success is uh success of our customers and success of ecosystem.

• 31:20Speaker 00

  And I think fusing together uh conventional computing, one human machines and accelerated computing that provided with Nvidia. It's actually, it's probably open yet another dimension that I'm not sure what it is, but it basically gives, you know, on the practical, you know, short term view is it's uh this gives uh Nvidia and Intel channels to the market or expanding the market and serving the markets that otherwise was more challenging.

• 31:56Speaker 02

  You mentioned the culture of Nvidia. So when Mellanox became part of Nvidia in 2019, the market cap of the combined company was about 100 billion, which is no joke, but the market cap today is about 4.5 trillion.

• 32:10Speaker 02

  And so 45x growth in value in six years is pretty phenomenal. How has that changed the culture of Nvidia? How is Nvidia different today now that it's one of the most admired companies in the world, if not the most admired versus six years ago?

• 32:26Speaker 00

  Yeah, about this, you know, when we just joined, Jensen was in Israel and I presented him, you know, that's I believe that one plus one will be 10. And I actually was off by factor of four.

• 32:42Speaker 00

  So, but Mellanox and Nvidia, in a sense, it's it's sort of similar. The culture is very similar to begin with, but there is some, there is nothing absolutely similar. And I was the only founder that left in Mellanox after Real resigned a few months after the acquisition. And my main focus at the beginning, you know, things what do you think about in the shower? Was how to make sure that this acquisition will succeed.

• 33:17Speaker 02

  Yeah.

• 33:17Speaker 00

  You know, Nvidia paid $7 billion for company that I founded. And, you know, with all the mixed feelings that were there, but once it's done, it's done. Now I have to make it successful.

• 33:33Speaker 02

  Yeah.

• 33:33Speaker 00

  So eventually it worked.

• 33:38Speaker 00

  Most of the Israeli employees are state. I think it's 85 or 90% of original employees are state. Actually. Nvidia grew more than two X in Israel in terms of manpower. Yeah. So we we we growing and we are announced that we are actually going to build a campus in in Israel, new campus for Nvidia. And so that's where I think the the overall, the merger was very, was very successful. I I did, I did my best to make sure it it succeeds.

• 34:21Speaker 00

  And besides the technology that I was looking at, this part, this part of which is sort of technology, but it's technology and theology, and there is many other things to make sure that people are comfortable, that, you know, from being in the center of Melanox, which is the headquarters in Israel, don't feel left, you know, somewhere in the far away. And Jensen, he emphasizes the networking is the critical part of Melanox success, Nvidia success. Yeah. And he's right. Yeah, yeah. So I think it was the, it's, it's, it's considered to be the most successful merger

• 35:06Speaker 00

  in the, in the history of the technology. I am, you guys probably drug this, drug the things better than than I am, but overall, I think it was a great move. Yeah.

• 35:18Speaker 03

  What are the science fiction things that you spend your time thinking about? You just even wondering, like, for example, optical interconnects, do you think, do you think that will exist? Do you think AI will ever be better at physics than than us and better at data center design than us?

• 35:32Speaker 00

  Well, what I, what I thinking, you know, in the, if you look about science fiction is how to make history to be experimental science, you know? You can physics do, try something and then, you know, see if it works and then try something else in history. Time goes one direction, but you have a good simulation of the world, you can make history experiments, and we have an earth to climate simulator. And with this type of technology, we can actually simulate. How what we do today will impact the global warming fifty years from now. Hmm. Okay, so experimental science, you know, you you try something, you see what happens fifty years later.

• 36:13Speaker 00

  Uh, so that's that's that's the science fiction part. Yeah. And, you know, the physics, now we are moving from reasoning and so on and so forth now. We, once we get AI models to understand physics, we actually can learn physics. Yeah. AI can teach us physics because the way we we we uh we get to the laws of physics that we observe, theoretical physics, right, you observe some phenomena and you generalize it and you compose the rule that basically the law, the physics law, that stays underneath of this phenomena.

• 36:57Speaker 00

  And AI is really great of generalizing and data processing and observing. So AI can help us to get to know some laws of physics that we don't even imagine now. Yeah.

• 37:10Speaker 03

  Uh, Moore's law was two X every two years. Huang plus Kagan's law is, what what is the slope and how long do you think you can sustain it?

• 37:17Speaker 00

  Well, the slope is somewhere in the range of ten X or few orders of magnitude a year.

• 37:25Speaker 03

  Okay.

• 37:26Speaker 00

  And that's what we are doing, by the way, now, we are, we are, we are, since about two, three years ago, we accelerated our product introduction from every other year to every year. Now we introduce a new uh new new wave of products every every year. And it's an order of magnitude, higher performance, and it's not, it's not on the cheap level performance. It's on the machine that you can build with this performance. That's what we are looking at, it's a single unit of computing. And how long it will stay, I I don't know, I don't know, but we'll do our best to. To maintain it as long as needed and probably even accelerate. It's it's all about it's all about exponent.

• 38:10Speaker 00

  It's all about exponent. It's hard to imagine, you know, you if you look at this Mula curves or any any role course. They they usually plotted on the logarithmic scale. So it looks like linear. But that's wrong thing to look. You can't predict what's what's going to happen. Who could predict that when iPhone was first introduced or smartphone was first introduced? You know, that's fifteen years ago.

• 38:37Speaker 02

  Two thousand seven.

• 38:38Speaker 00

  Yeah, two thousand seven, seventeen years ago. Okay, who could who could imagine that this smartphone? The least used function, at least for me, is a phone.

• 38:49Speaker 02

  Yeah.

• 38:50Speaker 00

  Unless it's e-commerce, it's it's texting, it's news, it's mail. It's basically running running your life from this machine. It's your authentication, your ID is there. It's so, you know, now who can imagine what's going to happen? You know, ten years from now is all these developments that we are do we are doing today. But we are building the platform for innovation.

• 39:14Speaker 02

  What is the your commentary on who can imagine notwithstanding? What is the most optimistic view of our future with AI that you like to think about? Like, what what could AI do for the world five, ten, fifteen years from now?

• 39:30Speaker 00

  Steve Jobs called computer to be the the bicycle of of mind.

• 39:35Speaker 02

  Yeah.

• 39:36Speaker 00

  Okay, so AI is it's maybe. I don't know if it's if it's probably spaceship. Because there is a lot of things that I would like to do. But I I I just don't have enough time, don't enough resources to do it. With AI, I will have it. And it doesn't mean that, you know, I will do twice as much. Maybe I do, I will do ten times as much. But the thing is that I will want to do hundred times as much as I want to do today. And that's where, and that's where, and you know, you go to, to, to any project leader and nobody says, you know, I have enough, I have enough manpower, I have enough resources. I don't need anymore.

• 40:18Speaker 00

  Okay, if you give him a resource which is twice as efficient, he will do four times more. Yeah. And he will want to do ten times more. So it's like electricity, change the world, right? Instead of using, you know, in the, in the London, you still, still see this ga- gas lamps and this infrastructure to, to, to use the gases, as the source of energy. Who, who could think that, you know, once this electricity was invented, it will change the world that, you know, we can't live without electricity. The same, the same with AI.

• 40:53Speaker 03

  Beautifully said. Thank you so much for joining us today. I love this conversation.

• 40:57Speaker 00

  Thank you. Thank you. Thank you for having me.