Chips, Memory, and Power | Pat Gelsinger

a16z's Raghu Raghuram and Guido Appenzeller sit down with Playground Global General Partner and former Intel CEO Pat Gelsinger to discuss the next wave of semiconductor innovation and the physical constraints shaping the AI buildout. Drawing on his experience designing Intel's 386 and 486 processors, Pat explains how AI could transform chip design, but also why faster design alone won't solve the industry's biggest problems. They examine the bottlenecks in manufacturing, memory bandwidth, advanced packaging, and power, and why today's explosion of specialized AI chips may eventually consolidate around a smaller number of architectures. They also discuss the potential for new memory technologies, the shift from copper to optical networking, and why energy capacity could become a major constraint on AI growth. Finally, they revisit Pat's VMware years to ask what virtualization might look like when infrastructure is built for agents rather than humans.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  In an AI digital age, energy capacity equals economic capacity. Why build a new data center and buy a million GPUs if I can't power them? You're going to see more and more defaults happening on many of those data center projects because the energy won't be there.

• 0:17Guido Appenzeller?

  Whenever you have the technology to make something easy, that means the bottleneck moves somewhere else.

• 0:22Speaker 00

  Nothing's a chip anymore. It's a rack. It took me three months to design it, but it's nine months until I can actually start to use it. Exactly how many major new memories have you had over the last 30 years?

• 0:33Speaker 00

  Memory innovation, for the first time in 30 years, is nigh upon us. I declared the death of copper about 25 years ago. Eventually, I'll be right for all of us hardware guys. This is like Renaissance in front of it. AI inference accelerator chips, and I'm sure I don't even know them all. Why are you guys funding so many of those anyways?

• 0:50Speaker 02

  You've won the fair share too.

• 0:53Guido Appenzeller?

  Historically, there have not been 100 competing processor vendors in any industry ever. Is this a temporary thing? It'll converge back to you? I see it as-

• 1:02Erik Torenberg?

  AI is making it faster to design chips, but actually building them, powering them, and connecting them may be getting harder. In this episode, A16Z's Raghu Raghuram and Guido Appenzeller sit down with Pat Gelsinger, general partner at Playground Global and former CEO of Intel and VMware, to discuss the next bottlenecks in AI infrastructure. Pat takes us back to his early days designing Intel's processors, when engineers were inventing the tools they needed to build the chips themselves. He explains why AI could bring a similar shift today, even as manufacturing timelines, memory bandwidth, and power become harder constraints. They also debate whether the explosion of specialized AI chips will last,

• 1:46Erik Torenberg?

  why Pat believes memory innovation is finally approaching a breakthrough, and how optical networking could change the architecture of AI clusters. In drawing on their shared history at VMware, they close with a question: What would it look like to rebuild virtualization for a world where AI agents, rather than humans, are the primary users?

• 2:08Speaker 02

  I'm here with our esteemed guest and dear friend and former boss, Mr. Pat Gelsinger. Welcome, Pat. Pat is currently the general partner at Playground Global, but as you are very, very well-known in the industry for leading Intel, for being the CEO of Intel, of course, leading VMware and many other things. So, welcome.

• 2:31Speaker 00

  Hey, thank you, Raghu. Great to be with you and Guido. And to me, this feels just a little bit like old home, right? We've superimposed a year of change, right? But you both look the same. I feel energetic, so let's dive in.

• 2:44Speaker 02

  There has never been a day when you stopped being energetic, so there's no use there. But yeah, no, let's start actually from your Intel career. Recently, Andreessen Horowitz created this Horowitz and Recent Academy, which was to find talented people between 16 and 22, and then put them into a modern educational setting so they'd be well-prepared for either doing things on their own or joining the companies of today. You had a similar experience, like you went to a regular trade school and then jumped into Intel when you were, what, 18 or 19, essentially?

• 3:20Speaker 00

  Yeah, it was really a sort of magical period. I'm 16 years old. I accidentally won a scholarship. I go to tech school, right? I skipped my last year and a half of high school, and Intel comes recruiting when I'm 18 years old. And I'd never been on an airplane. I had already fallen in love with computers at tech school. And the interviewer, Ron Smith was his name. He writes on his page, and I was number 12 that he interviewed that day. And if you've interviewed 12 people in a row, you can't tell male from female by the end of it, right? I'm number 12. He says, "Smart, aggressive, arrogant." He'll fit right in. So I got invited to come to Intel, and it just became really glorious. Starting as a technician, moved into the design team at the end of the 286,

• 4:05Speaker 00

  engineer number four on the 386, architect and design manager for the 486. All of that while doing my bachelor's, my master's, and PhD work. So it was like the best career that you could possibly have. You're learning by day, and you're putting it to practice at night.

• 4:22Speaker 02

  That's right. That's right. A very new, completely new era of chips. I bet most of what you learned and what you built, there was a big gap because you're breaking new ground.

• 4:34Speaker 00

  Yeah, in a lot of ways. And it really was a pretty incredible time in the industry. And I remember one of the professor classes, he postulated a new carry look-ahead adder design, right? And one of my classes at Stanford, so I said, "Well, I'm working on the 386 carry look-ahead adder, so let me go try it." So I come back two days later to the professor, and I said, "It doesn't work." And I started an argument with my professor at Stanford. You know, it was like, what, the next two class periods were just me and the professor arguing over why his design didn't work. He had a book that was being published with that design in it. So it was very annoying.

• 5:09Speaker 02

  Oh, it was very time-sensitive for him.

• 5:11Speaker 00

  And we started built-itself test, right? And I remember Ed McCluskey at Stanford was my professor, and he was sort of the father of that. And we had wonderful arguments around why most of his ideas were bad. They weren't practical for real chips, but I was putting built-itself test to work on the 386. John Hennessy was my thesis advisor, and obviously, he did okay. I helped his career. He became president, and he's done okay since then.

• 5:36Speaker 02

  Yeah, yeah. Later on, we'll talk about the SIS courses, this kind of debate.

• 5:40Guido Appenzeller?

  I had the same experience when I was teaching at Stanford. You come with something that you think is the truth, and then a student comes that works in industries, "Well, actually, not quite." So the next lecture, you come back with, "Okay, I looked this up. You're right. This is outdated."

• 5:55Speaker 00

  Yeah, that's part of the reason- Yeah, that interaction, and I think that's part of what makes Silicon Valley so unique. Right? You know, as you have so many companies and entrepreneurship, it's like in the water here, right? Starting things up and new ideas and challenging the professors. And you can't be a professor at a place like Stanford and not expect to get challenged and enjoy it.

• 6:16Guido Appenzeller?

  There's a good foot traffic in and out of that Averi Tower.

• 6:21Speaker 02

  Yeah, no, 486 was your big accomplishment, if you will. Obviously, lots of people worked on it. And you guys broke some new ground in chip design there, and increased what functions a chip was supposed to do and so on and so forth.

• 6:36Speaker 00

  Yeah, it was a pretty magic period as well because it was before what you would think of as the EDA industry.

• 6:43Speaker 02

  Oh, wow. Yes, that's right.

• 6:45Speaker 00

  And the end of the, a little bit of the 386, but the 486 was really the first chip to implement what you would think of as modern EDA techniques. We did a high-level design, right? Description, an RTL description, but there was no Verilog. So we invented HDL, right? The Intel Hardware Description Language. I wrote my own language. Well, you had to build a compiler for that language, right? So we created a compiler for the language. There was no automatic placement route. So we had to invent that, and we worked with Alberto Sangi Avangi, Vincentelli at Berkeley, and some of his students for the first placement, the first routing, the first automated timing management.

• 7:28Guido Appenzeller?

  So they created EDA then to some degree?

• 7:30Speaker 00

  Yeah, to a great degree. That, you know, and that was really one of the hallmarks, that not only was the 486 this compatible pipeline microprocessor, but we ushered in many of the foundations of what became the modern EDA industry. And it was really a pretty magical period of the industry.

• 7:48Speaker 02

  Yeah, yeah.

• 7:49Guido Appenzeller?

  If we look at today, will people look back as Blackwell and saying, "This was of the last generation that was done before AI took over the design process for micros?" Are we in a similar transition right now?

• 8:01Speaker 00

  I think there are certainly aspects of that. And I think when you look at things like jalapeno today, right? You sort of say, "That's sort of a first principles use of AI in the chip design process." Right? Where you throw out a lot of those things. Now, there are pieces of design that are still really hard, right? In that sense, and a lot of the analog, right, aspects, Certi's is probably the best example of that, you know, are still not AI-able, right? You just need lots of silicon data, right, to get those. So you either get so conservative in your analog design that you're able to, I'll say, AI it, or, right, you got to do those to hardware. But now, many of the logic functions can really be done with AI tools and techniques

• 8:45Speaker 00

  in pretty incredible ways, right? Your transistor budgets are big enough, the design tools are getting smart enough, a few experts guiding the tools and how to apply it. And I really think it will be somewhat like the 486 that way, right? We just sort of look back and say, "Yep, that was the beginning of a new era of chip design."

• 9:03Guido Appenzeller?

  Whenever you have the technology to make something easy, that means the bottleneck moves somewhere else. Can we guess at this point where the bottleneck will be in the future? There's some things which it seems like AI is incredibly good at, right? Like creating the software layers, you know, writing kernels, a lot of the-

• 9:21Speaker 02

  I think a lot of this sort of being able to specify the objectives at a higher level, which then gets sort of translated into a very long, right? And so what is the new frontier? What is the thing that will be difficult going forward?

• 9:33Speaker 00

  Well, when you look at today, so let's say Guido and Pat, we're off to do a great new chip together.

• 9:38Speaker 02

  Building the AI.

• 9:39Speaker 00

  Right, right. And it's the one that's going to leap ahead because we have understanding of the AI.

• 9:44Speaker 02

  Twenty twenty six. Four eighty six.

• 9:46Speaker 00

  We have understanding of the AI workloads. We have understanding then of how to compose that and then the right multiply accumulate register structure.

• 9:54Speaker 02

  Perfect visibility with model workloads of tomorrow as well.

• 9:56Speaker 00

  Yeah, and all of that kind of stuff. And let's say we turn our AI tools loose. We are great. Within three months, we have an awesome design, right? For that, we've been a little bit conservative on all the analog components of it. Now, we still have a couple of really major bottlenecks today. One is, right, we still have nine months of silicon processing time.

• 10:19Speaker 02

  Yeah.

• 10:19Speaker 00

  Right, so like that. It's so, I can design the thing in three months, but I can't actually get it into real silicon at scale for nine months. Okay, that sucks. Right, so we have to really see that bottleneck improve if we're going to have this innovation because we know.

• 10:33Speaker 02

  More than evaluation and verification stages or even beyond that?

• 10:35Speaker 00

  Well, you know, first, I can't get anything out of fab, you know, in less than three months. And that's if I have, you know, like supercharged design flows, right? So I said, you know, how do I compress that, right, you know, for it? And then I got to get into advanced packages, 3D packages are getting complex. So that's like another three months, right? You know, and then I have to put it into a rack scale solution because nothing's a chip anymore. It's a rack. So we're nine months. It took me three months to design it, but it's nine months until I can actually start to use it. You know, so how do we start to compress, you know, those aspects of design? And I think we need new forms of lithography, you know, to enable that. We need, you know, design flows that don't require, you know, fifty million dollars of mass cost until I can get things into prototyping.

• 11:20Speaker 00

  So, you know, to me, how do you compress that to a month or two? Right, because I can now do the design in three months. How can I have that done in a month or two? Because if it takes me a year and a half until I actually get scale and software on it, okay, my understanding of the AI workloads is no longer applicable to the chip that I design. Right, you know, so, you know, all of those.

• 11:42Speaker 02

  We're seeing that play out live right now, I think, with AI chips.

• 11:45Speaker 00

  Right, it's almost, you know, and we can look at it historically. It was like, you know, GraphCore wasn't a bad design, but the world moved on. Right, you know, and, right, you know, in that. So all of those aspects somewhat obviate the fact that AI has made the design piece easy. All of those manufacturing scale issues to me become a huge bottleneck. Right, another bottleneck, of course, is memory.

• 12:08Speaker 02

  Yep.

• 12:08Speaker 00

  Right, you know, memory bandwidth.

• 12:10Speaker 02

  We noticed, yeah.

• 12:11Speaker 00

  Right, and, you know, for that, as I've described, HBM is a hideous memory. It's just the best one that we've got.

• 12:17Speaker 02

  I was going to ask you about that.

• 12:18Speaker 00

  It's a terrible memory, you know, I get terrible bit density. I'm limited to shoreline bandwidth, the power, right, issues, thermal issues, DRAM doesn't like to be hot.

• 12:28Speaker 02

  Concentrating everything in one spot.

• 12:30Speaker 00

  Yeah, it's terrible to give.

• 12:33Speaker 02

  Building tunnels for chips.

• 12:34Speaker 00

  And AI is a memory compute workload.

• 12:38Speaker 02

  Yes.

• 12:38Speaker 00

  Right, so, right, you know, for that, so I, you know, I need to, you know, compress that development and embodiment into real scale. You know, I need to fix memory, right, associated with this. So to me, that's another critical bottleneck for us to attack. And power, right, you know, power in, you know, thermal, and most of the power simulation aspects today are bad. Right, you know, you don't have good 3D modeling tools for power, power dissipation, hotspots, et cetera. You know, my voltage in, right, you know, I'm essentially guardbanding almost my full power envelope.

• 13:15Speaker 02

  Correct.

• 13:15Speaker 00

  Right, so essentially I'm losing, you know, forty percent of my power in guardbanding, so I got to get much better at power modeling, you know, and then my power dissipation techniques are bad, right? So, you know.

• 13:27Speaker 02

  The delivery model has to change too.

• 13:29Speaker 00

  Yeah, so all of these things, you know, to me, those are the, you know, the next big bottlenecks, right? You know, how do I, you know, improve each one of those? And now, right, you know, if I've done that, okay, now we're on to an exciting era for what's next. So, you know, thank you for those great AI tools, but, you know, I got big challenges, even when those are fully deployed.

• 13:48Speaker 02

  And so we've seen sort of a rebirth in the number of chips and chip architectures and so on and so forth.

• 13:56Speaker 00

  Yeah, yeah.

• 13:57Speaker 02

  Thanks to these workloads.

• 13:58Speaker 00

  Yeah, it's like there's a hundred AI inference accelerator chips, and I'm sure I don't even know them all.

• 14:03Speaker 02

  Yeah, exactly. So, so.

• 14:07Speaker 00

  Why are you guys funding so many of those anyway?

• 14:11Speaker 02

  You've funded your fair share too. Yeah, so, sorry, you know, they're all, to their credit, making different design assumptions, either in the components that you talked about or in the, in the part of the periodic curve that they are targeting. So do you think there's room for all of these? Or are we in a world where partly because of what Guido talked about, AI tooling, we now have the ability to build more different kind of chips?

• 14:42Speaker 00

  Yeah.

• 14:43Speaker 02

  And the time is enormous, whichever way you.

• 14:45Speaker 00

  Yeah.

• 14:45Speaker 02

  That is even more provocatively. Historically, there have not been a hundred competing processor vendors in any industry ever, right?

• 14:52Speaker 00

  Yeah, yeah.

• 14:53Speaker 02

  So is this a more heterogeneous future? Will we see, is this a temporary thing? It'll convert back to a few?

• 14:58Speaker 00

  Yeah, and, you know, I see it as a more temporary thing and it will convert is what I expect. And there's probably three different reasons that I see that to be the case. You know, one of them is, you know, this emerging heterogeneity that you see in the compute environment where, okay, now I have specialized pre-fill versus decode. Well, you know, and now people are saying, well, we really need a specialized tier for mid-fill, not just pre-fill, right? So now I have early pre-fill and now I have mid-fill, which is, you know, different.

• 15:31Speaker 02

  Maybe speculate and verify if we listen to OpenAI here, right?

• 15:34Speaker 00

  Yeah, you know, and, you know, all of a sudden, my compute fleet becomes more and more granular across the workload. And I think any time that you've seen that in history is not sustainable. And then all of a sudden, right, when, you know, as you look at that, and now people are saying, well, as I go to reasoning models, I want something that looks more like a CPU again. So, you know, all of a sudden, I didn't want that level, you know, that didn't become the predominant portion of my compute capacity. I needed more of it over here, right, and model and model workflows and, you know, so on, you know. So I really don't like, I'll say, at-scale specialization, right? Because I think the workloads are going to continue to moderate, you know, and migrate

• 16:19Speaker 00

  so significantly. First, you know, second is, I think today's models are about to go through some evolutionary breakthroughs as well. Right, you know, I think the LLM is sort of reaching the limits. And now as people look to, how do I really model 3D? Right, you know, where, you know, a flattish LLM doesn't work well when you go to molecules and chemicals and, you know, right, imaging, you know, so I think we'll see limits of that that'll cause different shifts in the algorithmic domain. I also think that some of the most interesting workloads become where you start, you know, I'll say, bringing HPC-like things back into AI-like things, where all of a sudden, you know, sixty-four bit provision, precision counts again, right? So I see.

• 17:02Speaker 02

  What would be an example of that?

• 17:03Speaker 00

  Well, you know, imagine that my AI model is now inducing the five most interesting domains for a chemical analysis. Right, okay, those chemical algorithms are going to run with high precision, right? You know, when, you know, which is going to be the dominant piece of the workload, right? Getting to the five algorithms I need to run or running the five algorithms, right, on different chemical or biological systems? So, you know, I do think that, you know, or as we optimize our AI systems, they will lead us back to some of the things that have been more traditional, high-performance computing. So I see that aspect of workload as well. So, you know, all of that said is, you know, this extreme specialization to me in the

• 17:45Speaker 00

  compute architect, I sort of don't like it because I don't think it's going to be what the workloads will look like two, three, four years from now. So that's one reason I don't see all of these AI chips as they get more and more specialized for portions of the compute workload to necessarily be right. You know, second is, right, there's like a hundred of them now, right? You know, there's multiple optical ones, there's probabilistic ones and so on. And you know they're not all going to win, and they're not all going to win because at the end of the day, you have to get scale.

• 18:14Speaker 02

  Yeah.

• 18:15Speaker 00

  Of these things. And scale requires, you know, you have to win, you have to get capital, you have to get workloads on to it. So I think it sort of defies logic that you're going to have a hundred of these things. So I see them, you know, coming back as a result of, you know, simple, you know, capital market share, et cetera, you know, which are the winning teams, you know, winning designs, winning architecture. So I see the workload driving that, you know, I see the natural, you know, industry consolidations, you know, that will occur. And then I also expect that the winners will pick some winners. Right, you know, where, you know, an OpenAI, an Nvidia, an Anthropic will say, I like this

• 18:54Speaker 00

  one because it isn't just hardware, it's how do the hardware and software code evolve?

• 19:01Speaker 02

  Yeah.

• 19:01Speaker 00

  Right, you go forward, and I do expect that there's going to be, hmm, you know, here's this 10 I could pick for this, I like that one. And it does require investment in the software and the workload evolution to take advantage of those platforms. So for those three reasons, yeah, we're going to see, you know, a narrowing of the field in that sense.

• 19:20Speaker 02

  And the deployment complexity as well.

• 19:22Speaker 00

  Yeah.

• 19:22Speaker 02

  Sorry, go ahead.

• 19:23Guido Appenzeller?

  I mean, it makes total sense, and I think I agree in most part. Let me still try to stir a little bit of controversy here, just to.

• 19:28Speaker 00

  You, Guido, stirring controversy?

• 19:31Guido Appenzeller?

  No.

• 19:31Speaker 00

  Totally not me.

• 19:32Guido Appenzeller?

  You guys have changed it, right? Yeah. So, so obviously, a hundred companies might say so consolidate, right? And, and I completely agree with your argument that at the end of the day, the large consumers will pick winners to some, some degree, right? Today, I think, to be a first tier chip startup to some degree, whether you're, you're that or not, is defined by how much traction you have with, with the, with the really big guys. That's it. And, and there's also an argument to say, like, look, in any kind of transition period, you probably have a bit more of an explosion of different advantages over time that'll shake off. Now, all that said, I think there's an argument you can make that basically says, look, historically, we standardized on, you know, one or two architectures because, you know, different instruction sets were hard, you had to build multiple compilers, you know, like the, there

• 20:17Guido Appenzeller?

  was basically, um, the, the, the software complexity dictated that you can only have a certain number of hardware platforms. That part seems to be changing with AI a little bit, right? Today, you know, writing an optimized kernel for, I mean, I think Kalapena was a great example, but that basically said, look, humans can't really program this thing anymore, but an agent swarm overnight can do it, that's all we need, right? So we're basically, you can say, like, look, if you can build a chip that has a certain capability, it's now much, much easier to build the software layers on top that fully leverages that. Could a result of that be that we have a little bit more heterogeneity, just because the, you know, it's easier to program that heterogeneity going forward?

• 20:53Speaker 00

  And, and I, I do think it changes some of, you know, you know, you know, the, you know, where the bottleneck lands, right, changes in that regard. You know, the first bottleneck is the one I'd point to that we've already talked about. Still takes a year and a half to build these things.

• 21:06Speaker 02

  Yeah.

• 21:06Speaker 00

  Right, you know, and then the second bottleneck is, not only does it take a year and a half to get these things into racks and scales and so on like that, now to make them useful, okay, well, could I borrow 50 billion? Right, I mean, you're not talking about small capital expenditures, you're talking about very large capital expenditures and data centers, you know, power, you know, commitments and so on like that. Well, I gotta build this thing at scale, right, associated with it, so, and that's going to defy a lot of heterogeneity as a result. Now, you know, clearly, big players, and we saw this with what, you know, Nvidia did with Grok, okay, you know, we're going to sort of own some of that heterogeneity, right? And, you know, you know, two years from now, you're not going to tell what Grok is inside of Nvidia, right?

• 21:49Speaker 00

  It's just going to be like another, right, you know, someone like that. So you'll see the architectures, you'll sort of hide the heterogeneity, right, uh, inside of it, which to me is just fine, right? You know, I mean, that's what layering, that's what abstraction has always done associated with it. Yeah, but I think those forces are powerful ones to, I'll say, expose a lot of the heterogeneity? No. Right, I think it will get abstracted underneath, you know, some of these, but at the end of the day, it's still hard to build these things at scale.

• 22:17Speaker 02

  Earlier, you talked about HPM being hideous memory.

• 22:21Speaker 00

  By far the best one we have today.

• 22:23Speaker 02

  Exactly. The thing about democracy.

• 22:27Speaker 02

  We can't recall when there was the last memory innovation, it might have been Optane, I think you probably had something to do with it.

• 22:33Speaker 00

  Yeah, that one died, so.

• 22:34Speaker 02

  That one died.

• 22:36Guido Appenzeller?

  We killed it.

• 22:36Speaker 02

  Yeah, we killed it. So, do you think memory innovation is around the corner, or what needs to happen for, obviously, we understand the scale problem because it's even more.

• 22:50Speaker 00

  Yeah.

• 22:50Speaker 02

  Larger there.

• 22:51Speaker 00

  Yeah.

• 22:51Speaker 02

  But if you keep the scale aside, even from a technology point of view.

• 22:55Speaker 00

  Yeah, and let's, you know, uh, you know, look at that history. I've probably personally been associated with at least five different new memory architectures, Optane just being one of them, right, you know, that did not see the light of day, right, uh, as well. I'm probably familiar with close to 100 that have happened in the industry over the last 30 years. And exactly how many major new memories have we had over the last 30 years?

• 23:22Speaker 02

  Zero.

• 23:23Speaker 00

  Right, you know, which, you know, okay, DRAM, SRAM, Flash, okay. What else is there? DRAM, SRAM, Flash, yeah. You know, so I, I do think.

• 23:30Speaker 02

  Flash stacking comes for a little bit.

• 23:32Speaker 00

  Barely.

• 23:33Speaker 02

  Yeah.

• 23:33Speaker 00

  Right, you know, but anyway, right? So, uh, you know, in that, it, you know, it has been, right, you know, a disappointing that we haven't created, you know, the next physics, you know, that enable, you know, low cost, high performance, resilient, manufacturable memory, right, you know, for it. So I do think, and, and now, and, and over the last 30, 40 years, you know, memory was a terrible industry, right? Because you would make money one year and then you'd lose money for the next four, right? So let's go do R and D in an industry that makes money one out of five years, right? You know, it's just always, you know, it's just been so terribly difficult because it's

• 24:15Speaker 00

  been such, you know, driven by such violent commodization cycles.

• 24:20Guido Appenzeller?

  That has changed a little bit, though, recently.

• 24:21Speaker 00

  Over the last five years?

• 24:25Speaker 00

  Happy days, man.

• 24:27Guido Appenzeller?

  All three big memory vendors are in the, in the top 20 most valuable companies on the planet.

• 24:32Speaker 00

  Yeah.

• 24:32Guido Appenzeller?

  That's a crazy change.

• 24:33Speaker 00

  It's just so, right, you know, who, who would have thought, right, uh, associated with it? And anybody who looks at the AI workloads and it's always, you know, it's much harder to, you know, you know, it's easy to predict the past, but it's a little bit harder to predict the future, right? And, you know, in that, you look at it and say, AI is a memory work.

• 24:52Speaker 02

  Exactly.

• 24:52Speaker 00

  Right, you know, inside of it. So I think there is the combination of capital and technology needs that will justify memory innovation. Now, there's two domains that I'm excited about, uh, you know, for, you know, one is, you know, this idea, you know, I do think that the shoreline bandwidth limitations of HPM is a fundamental issue.

• 25:14Speaker 02

  Yeah.

• 25:14Speaker 00

  Right, so with that, the idea, you know, and obviously, you know, we have a company, Dematrix, that's doing this, but I think others will be, you know, whether it's, you know, cerebrus, you know, and their techniques.

• 25:24Guido Appenzeller?

  The future is stacked, yeah.

• 25:26Speaker 00

  Somehow, you gotta bring memory and.

• 25:27Speaker 02

  In more ways than one.

• 25:30Speaker 00

  You know, you gotta bring memory and compute together.

• 25:32Speaker 02

  Yeah.

• 25:32Speaker 00

  Right, you know, in, uh, fundamental, uh, new structures. I do think that, given the workload now has such capital being deployed against it, you know, in the AI space, that I do think we'll actually be able to break through some of those physics challenges as well, uh, as well. So I do believe, and, you know, I've just funded.

• 25:53Speaker 02

  You're substrate something, yeah.

• 25:55Speaker 00

  You know, I've just funded a new memory company as well, that's still a stealth, but, uh, you know, I, you know, you know, I'm quite excited about some of the innovations that will occur in new memories, and you're going to see new materials, uh, you know, for memory, you know, a lot of people looking at things like Faro electrics, uh, you know, finding non-capacitive, you know, uh, high-density memories, finding memories that can, you know, be high-performance, high-density and stackable, right? But, you know, thing, you know, and people are looking at, uh, low-cost Flash, you know, for that, I'm not particularly enthusiastic about that idea myself. I think, you know, speed and performance of the underlying cell are somewhat problematic with a Flash or an MRAM structure, you know, but I do think there's, memory innovations

• 26:40Speaker 00

  are coming for the first time in 30 years, and I do think that's going to be an exciting space. And, you know, we'll have a couple of companies there, I'm sure.

• 26:47Speaker 02

  Yeah.

• 26:47Speaker 00

  Hopefully, we'll do a few of them together.

• 26:49Speaker 02

  Yeah.

• 26:50Speaker 00

  Maybe, you know, what do you think, should we do? Okay, you know, but, uh, we'll, we'll find some things to do here, but I think memory innovation for the first time in 30 years is nigh upon us.

• 26:59Speaker 02

  Yeah, yeah.

• 27:01Guido Appenzeller?

  It's amazing what a couple of $10 billion of market cap can do to a hundred billion. I don't even know what you say, trillion, okay, sorry. Off of orders of bank.

• 27:08Speaker 00

  You know, I think the memory industry has now increased by market cap by two and a half trillion dollars, you know, over the last four years. I think that's about right.

• 27:17Speaker 02

  That is, yeah.

• 27:18Guido Appenzeller?

  Of course, just invested there and not done any of the venture stuff.

• 27:20Speaker 02

  Exactly. Use your mind.

• 27:25Speaker 02

  How long, what's your view? Sorry, go ahead.

• 27:27Guido Appenzeller?

  Yes, to expand a little bit on the stacking part. If you have to predict, how quickly will our chips get taller? Is this going to be like, you know, we're going to go to two and then sit there for a while? Or will we have, you know, the 16 layer sandwich, you know, happily mixing logic memory and SRAM?

• 27:46Speaker 02

  HPM pulled down this.

• 27:48Guido Appenzeller?

  HPM is going down at Mofa, but how do you see this evolve?

• 27:52Speaker 00

  Yeah, you know, I think there's, you know, the problem with stacking, right? Is now essentially the value of the stack, right? Becomes greater as you go to higher dimensionality of the stack, which means the yield of any individual layer in the stacking process has to keep coming. You know, essentially it can't go up linearly.

• 28:14Guido Appenzeller?

  Yep.

• 28:14Speaker 00

  Right? You know, it has to go up exponentially. And improving the manufacturing and defect characteristics in an exponential way, you know, when you get to eight or 16 high stacks gets to be so, so hard when you run the math on that. Associated with it, you know, your manufacturer flows have to be so close to perfect.

• 28:32Guido Appenzeller?

  They could see designs that take into account a certain model. Yes. Like we've seen it for like a long time for two dimensions.

• 28:39Speaker 00

  Of course, but remember that was the nature of DRAM. You always have bad block replacement and, you know, spare lines and so on like that. So I'm not in any way, right, you know, minimizing that. But assuming you're doing that, you still need the yields to be so good. You know, if I have a cracked die, I don't care how much resilience you've built into it. You can't, you know, it's a bad die, right? So, you know, those kind, you know, so as a result of that, I'm not, you know, a crazy 16, 32 stack, you know, kind of guy. I think there's going to be, you know, nice three, four, five stacks, you know, that just end up sort of being the sweet spot.

• 29:14Guido Appenzeller?

  The mid rises, yeah.

• 29:15Speaker 00

  Yeah, right, you know, for, and I expect for, you know, part of it's manufacturing, part of it's complexity associated with it, you know, where's those yield sweet spots associated? So that's sort of where I think most things land. But even there, right, you know, so let's say I now have, you know, I now have a great compute tile, right? You know, how big does that tile need to be? I mean, everything's become reticle limited now, right? You know, can I go to more chiplets that better yield characteristics, better thermal? You know, and then you're going to say, I want a memory stack on that. You know, is it a two stack memory, a four stack memory? You know, I don't see it becoming an eight stack memory, you know? So to me, two or four sort of right, you know? But then power delivery needs to get integrated into that.

• 30:00Speaker 00

  So that becomes sort of another element. And I want that in the Z dimension, right? Associated with it because, you know, I want my power rails coming into this as well, right? And then I need, you know, fairly intelligent RDL layers to distribute signals across that platform. So to some degree, you know, my canonical thing is already about eight layer stack. When you add up all of those pieces associated with it. And I think that's about all you can do, right? At least as we understand manufacturing today. And then of course, you know, we expect optics to come in. So you're going to have more three, five material structures, you know, that are going to be closer to this complex. Cause I want, you know, my I O getting, you know, essentially, you know, deeply integrated into that construct as well.

• 30:44Speaker 00

  You know, so my silicon centric pieces need to get complemented by some of these other three, five materials. And how do I get optical connectivity built into this? How do I get power rails, you know, built into that? You know, that is a, you know, I'll say that is an engineering feat and a manufacturing nightmare. You know, to bring all those pieces, but that's where the physics is taking us over time. So because of that, you know, throwing a 16 high stack into me, you know, it's like, that's crazy. You know, a two or four high memory stack, you know, I think that's probably where the sweet spot will be.

• 31:18Speaker 02

  And you didn't talk about the thermal dissipation and new materials needed for that.

• 31:24Speaker 00

  Yeah, yeah. And obviously there's going to be a lot of, you know, you know, first thing, right? You just can't have some of the, you know, the physics hotspots that you have today. So you have to be more intelligent about how you use the heat and spread the heat. But, you know, then you're going to be saying, okay, how do I both get power in and dissipate power out of those environments? And by the way, if I can, you know, get much more efficient power in at the voltages I want with much better, you know, power responsiveness in my voltage regulation, you know, then I don't create as much thermal chimneys.

• 31:54Speaker 02

  Yeah.

• 31:54Speaker 00

  But I do have to think, okay, how do I cool these things? And, you know, what role do new materials, diamonds and other things have been postulated as ways to, you know, do a better job of the thermal cooling? How do I then have better, you know, liquid cooling techniques and others, you know, sitting right on it? You know, with good thermal characteristics, so yeah. You know, we, we engineers are becoming plumbers, right?

• 32:18Speaker 02

  Exactly. Yeah, you don't worry about M E C, the chip scale.

• 32:22Speaker 00

  Yeah.

• 32:23Speaker 02

  Let me ask you about an operate phenomenon, which is some people have said, hey, let's get the memory off of the package and just go all optical between the compute and the memory. And that'll give you more degrees of freedom and more ways of building things. What's your view on that as opposed to stacking that's other extreme?

• 32:42Speaker 00

  Yeah, you know, I think the more O E O, right? Uh, interfaces that, you know, it's, you know, optical to electrical to optical to, you know, the more of those that you demand in your core compute complex, you know, to me is just hard, right? Uh, in that sense, and you're always going to have losses associated with it. You know, people are, you know, working hard to come up with more clever techniques to decrease, you know, both the cost and the power, right? You know, how, how much of the power am I wasting as I go through my, you know, my DAX and, you know, conversions? So I'm not a fan of those, you know, because I think it gives you access to large pools of memory, right? But I burn a lot of power getting there.

• 33:25Speaker 02

  Yeah.

• 33:25Speaker 00

  Right? And remember my, you know, the essentially, you know, the femtojoules per compute is now actually pretty amazingly good, right? The femtojoules per bit of communication is comparatively a thousand times worse, right? And if I'm spending that much to go get to large pools of, so I don't like those approaches. You know, myself, I think they somewhat defy the underlying physics of the solution.

• 33:51Guido Appenzeller?

  I mean, and then there's sort of the other approach to say like, look, can we actually integrate compute and memory directly? Right? Shift some of the compute into the memory.

• 34:00Speaker 00

  Some of the PIM approaches.

• 34:02Guido Appenzeller?

  Yeah, exactly. Right? So what, what are your thoughts about that?

• 34:05Speaker 00

  I'm not a fan of those either, right? Because I think workloads, you know, because now I have to take my workload and I have to like, push these little pieces of compute into the memory, right? You know, so that, you know, I don't need as much bandwidth of memory coming into my real compute, right? And that puts constraints on workloads. And again, you know, I could be wrong on this, particularly some of the design flexibility and the intelligence that we can now bring, you know, through the, the design aspects. But most PIM ideas, you know, have been around for 25, 30 years. And I think 25, 30 years from now, they'll still be hanging around, right? In that sense, you know, give me great memory bandwidth, you know, that I can have, you know, high density, uh, high performance and get it close to the real compute engines.

• 34:49Speaker 00

  I expect that's going to be a more generalizable solution across many and evolving workloads over time.

• 34:55Guido Appenzeller?

  Why? So we're still stuck with denser and denser chips, more compute, more logic, more power.

• 35:01Speaker 00

  And much better I O. Now, I am a, I am a huge fan, right, of optic, you know, moving to optics for all I O function.

• 35:09Guido Appenzeller?

  Yeah.

• 35:10Speaker 00

  Right? You know, I declared the death of copper about 25 years ago. Eventually I'll be right.

• 35:14Guido Appenzeller?

  You're still right.

• 35:15Speaker 00

  Yeah, yeah, one day soon, uh, I'll be right. But I think is that, you know, to me, all I O should go to optical, right? You know, but that core compute memory complex, no, right?

• 35:27Speaker 02

  So when do you, the first step for going on optical is to scale up even more so than scale out.

• 35:35Speaker 00

  Yeah, yeah.

• 35:36Speaker 02

  What's your views on that?

• 35:39Speaker 00

  Yeah, you know, and I think we're now seeing.

• 35:41Speaker 02

  What would be the, like, the application of the workload driver, if you will?

• 35:44Speaker 00

  Well, I think the workload driver is the one in front of us.

• 35:47Speaker 02

  Okay.

• 35:47Speaker 00

  Right? I mean, you know, today, you know, our scale up environments, we're putting tons of copper, right? You know, and essentially copper is becoming a wave guide that's getting shorter and shorter, right? And all of a sudden, you know, to make my copper work at, you know, five meters is more expensive than making optical work, right, at 100 meters, right? You know, so, you know, I think the physics is driving us there today, you know? And, you know, so if that's the case, Pat, why haven't we moved there already?

• 36:16Speaker 02

  Yeah.

• 36:17Speaker 00

  It's a complex supply chain that, you know, has never been tested at scale. And I think now, you know, I think everybody is looking at, you know, some form of N P O, C P O, et cetera, you know, in the 28, 29 timeframe, because, you know, I can't scale up to large radics, you know, compute clusters from my scale up environment without making that move. So, you know, clearly Nvidia has indicated that. I think everybody else in the industry expects, you know, that's the conversion point. You know, in reality, I should have never built an N V L 72. You know, it's an engineering marvel and it's a manufacturing nightmare, right? You know, it took the industry, you know, one extra year and a half to digest that beast, right?

• 36:59Speaker 00

  And it really is amazing that, you know, we're able to get it to scale and so on. But, you know, that really proved that had we had a more mature optical supply chain, we'd be much more advanced today. We'd be able to build better scale up environments today, you know, better, you know, power, better cost, better energy. But we didn't have the maturity of the optical supply chains to allow that. And those things are not trivial to get resolved. There's a lot of work to happen, but I think 28, 29 is the.

• 37:28Speaker 02

  Card capacity.

• 37:30Guido Appenzeller?

  Oh, good as I think.

• 37:30Speaker 02

  Well, yes, yes.

• 37:31Guido Appenzeller?

  We're calling. Again, you have experienced how impact optics work in networking, right? And I think many of that will translate nicely to GPUs.

• 37:40Speaker 00

  Yeah, I do think, but all of a sudden, you know, it's like, okay, these things have to work, you know, at 10 million GPU scale. Wow, right, you know, and it has to fit in packages. And, you know, there's just a lot of things there, you know, and many forms of optical don't like bad thermals. Well, guess what? Bad thermals is the nature of the game for it. You know, so we have thermal problems to deal with, you know, package integration problems. We didn't have enough, you know, laser capacity in the industry. So a bunch of things, but I don't see anything there fundamental. It's just a lot of hard work at this point to pull all of those pieces together.

• 38:12Guido Appenzeller?

  In terms of the networked architecture, does also mean a switch, what's more circuit switch networks, just because, you know, you can't scale these, these very fast optical networks indefinitely.

• 38:24Speaker 00

  Well, you, you know, I like, you know, Nick McCune, you know, another Stanford professor here, a good friend of mine, you know, as well, you know, I mean, you know, a little bit often, Guido, you know.

• 38:36Speaker 02

  Was he a thesis advisor?

• 38:37Speaker 00

  He was my thesis advisor.

• 38:40Speaker 00

  Yeah, yeah, so, but, you know, to me, he describes it well, you know, we built, you know, networks for, to be able to handle any packet going anywhere with no knowledge of where it might go. When you think about AI, you know, it's almost exactly the opposite, right? You know, we're doing flows that are large and predictable. So, and that says, just the very description says, hmm, that looks more like a switch than like a packet. So I do think this confluence of moving to optical, right? And, you know, having a workload that is far more predictable, manageable, you know, large and flow, you know, to me, you know, I do see that, yeah, we will go all the way

• 39:22Speaker 00

  to bringing optical into the compute complex and we will go all the way to OCS or some form of optical switching, you know, with more predictable flows. You know, to me, that is the right, you know, architecture. And then you sort of say, is that scale up or scale out? And to some degree, if my scale.

• 39:41Speaker 02

  I was gonna say, it sort of dissolves the boundary there.

• 39:42Speaker 00

  Yeah, right. If my scale up is big enough, then yeah, you know, just saying, okay, what's the rate of my scale up environment? And when do I want to hook it up to other clusters that are really big? And to me, that ends up being a pretty beautiful answer, you know, to the future. But I'm looking at somebody who's way more expert on this than myself, so.

• 40:00Guido Appenzeller?

  No, I, I, look, I always found scale up was a scale up to some degree in artificial distinction. You understand where it comes from, right? As we're kind of using different protocols, one is front end, one is backend network and so on. So right now, practically speaking, it is a, it is a big difference. If you take a step back and look at it from a system architecture perspective, it achieves the same goal, you know, with slightly different flavors, slightly different protocols that over time I think will converge.

• 40:24Speaker 00

  Yeah.

• 40:25Speaker 02

  I mean, especially for the AI training workload.

• 40:29Guido Appenzeller?

  Yeah, although, I mean, training and inference is actually look fairly similar.

• 40:34Speaker 02

  The inference corp is less.

• 40:35Speaker 00

  Yeah, now I'm going to, you know, right, does your brain stop learning? Right, you know, you know, by the way, is another characteristic of workload evolution, I do think the, you know, the separation of training and inferencing is going to become, you know, less so going forward, not more so, right? And that's a little bit of the argument against, you know, you know, more and more heterogeneous architectures is I think continuous learning, you know, I think there'll be algorithmic domains that will say, yes, continuous learning, you know, becomes the nature of the algorithm where, yeah, I do want my inference environment to be updating my model weights because, you know, that's giving me more and more refinement, you know, evolution across, you know, workloads, moving into environments that are less formalized training as well, while our smaller data

• 41:20Speaker 00

  sets in those environments. So, you know, I'm not sure that's a.

• 41:24Speaker 02

  But the number of nodes you need in order to be useful is a smaller diameter for inference than for training, that's what it means.

• 41:32Guido Appenzeller?

  Today, that's true, yeah.

• 41:33Speaker 00

  Yeah.

• 41:33Guido Appenzeller?

  I mean, I, I still think there might be a role for heterogeneity coming from a different angle, which is that different types of AI workloads require very different underlying goal, like some are more compute limited, some are more, more memory bandwidth, have a small diffusion workload or so, right? The optimal card for that looks very different from, you know, a, a top of the line, you know, 15 trillion dollar, 50, sorry, 50 trillion weight LLM, right? That needs to operate at massive scale, right? So it's, I think that, that might drive heterogeneity, if I'm thinking.

• 42:03Speaker 00

  I, I think there's some room, room for that, you know, but I'd also say, okay, now I'm talking about putting a million GPUs to work in my data center, right? And you say, well, you know, for that particular diffusion workload, you know, I should really put something optimized this way, right? So I'm now going to start, you know, putting, you know, a thousand of those in, right? And for that workload, I need, you know, 2,000 of these in and, you know, for some, you know, and all of a sudden, you know, you don't get some of the, you know, the flexibility of scale, right? That you would say, and, you know, how much does it cost me to manage and upgrade and, you know, connect and, you know, fault domain, you know, all those types of things.

• 42:41Guido Appenzeller?

  The agents will do all the upgrades, so that's no longer a problem.

• 42:44Speaker 00

  Yeah, yeah, yeah, thank you. Everything gets easy, so good, I don't care.

• 42:47Speaker 02

  Yeah, so fleet management is done.

• 42:51Speaker 02

  So we talked about what, compute, memory, networking, power is the next one.

• 42:58Speaker 00

  Yeah, yeah.

• 42:59Speaker 02

  And you touched upon it with vertical delivery inside the chip, on the chip. But, I mean, there's 800 volt DC, there's power delivery to the data center.

• 43:11Speaker 02

  There's an entire ecosystem all the way to

• 43:15Speaker 02

  power generation and

• 43:19Speaker 02

  the social, political dimensions of that and so on and so forth.

• 43:22Speaker 00

  Yeah, yeah.

• 43:23Speaker 02

  So where does innovation come in and where does like its pure execution?

• 43:28Speaker 00

  Well, you, the, you know, and if we start at, you know, first principles here, you know, our, our nation has done a terrible job with its energy capacity, right? You know, and essentially we went through 10 to 15 years where essentially I was taking coal offline at the rate I was adding renewables and essentially the nation was flatlined in terms of overall energy capacity. Not good, because in a AI digital age, energy capacity equals economic capacity, right? So essentially my economic capacity as a nation, flat for 15 years if you buy that thesis, right, which I think is very, you know, provable. Over the last five years, I've, you know, we've seen this enormous influx into just

• 44:13Speaker 00

  give me more capacity. But, you know, I think, you know, even with that, we've gone to maybe increasing our national energy capacity 4% per year?

• 44:23Speaker 02

  Something like that.

• 44:24Speaker 00

  Going from, you know, essentially zero to one, right? Wow, 4X or 4%, right? So to me, you know, that, you know, it's a bad situation. So we just need more energy capacity for it, so number one. And obviously we have some companies in that space, you know, and we talked about nuclear upgrading as one of those with Alva. And, you know, I think nuclear, right, you know, as a base load is, you know, a fabulous thing that we want to go build more of. You know, unfortunately, all of our renewables have, you know, deep dependencies in China, which is very unfortunate, you know, for us, you know, where, you know, you know, gas turbine lead times are only eight years at this point, right?

• 45:08Speaker 00

  So, you know, you know, I mean, we really have a difficult environment to scale up rapidly, but one, we just need more capacity. And fundamentally, energy capacity somewhat dampens how you fork we can get on AI, right? Because, you know, why build a new data center and buy, you know, the million GPUs if I can't power them, right? And I think you're going to see more and more defaults happening on many of those data center projects because the energy won't be there.

• 45:35Speaker 02

  So you think it'll be a significant headwind?

• 45:37Speaker 00

  I think it will be a headwind. And I think you're already starting to see some of the first indicators of that, the oracle, you know, comment, you know, was just, I, I think that's the first of what you're going to see many. Now, it's not like we're going to slow down as a result, but I think people will start saying, do I really have the energy, you know, to underwrite the capital commitments that I'm making, you know, for, you know, cement for data center racks and GPU purchases? So I do think it's, you know, going to be a, a, you know, an upper bound. But we need more energy capacity and we need to get more innovation into that and we need more of those supply chains in the US. Nuclear, one of my favorites, you know, when was the last nuclear reactor that came online in the US, right? You know, it's 20 years ago, right?

• 46:21Speaker 00

  You know, we just stopped, you know, a very capable industry. So that's one we need more, you know, we need to get much more efficient at delivery, you know, the power networks. And as you said, you know, moving an 800 volt DC data center, absolutely, you know, right? You know, and being able to eliminate so many conversion steps along the way because of, of historical, you know, reasons and, you know, standardization reasons, okay, we got to fix this, you know, an 800 volt DC, the revenge of Edison is upon us.

• 46:52Speaker 02

  Yes, I was just gonna say.

• 46:54Speaker 00

  Yeah, I like to say, so, you know, I do think that ends up being, we need better power conversion, you know, we have a couple of companies, one of those vertical, you know, vertical GAN, you know, I think will be, you know, a killer technology, you know, to be able to do 800 to 48 or 800 to 12 or even 800 to five in one conversion step, right? Which just gives you a lot more energy, you know, we touched on.

• 47:17Speaker 02

  Solid state transformers, you know?

• 47:18Speaker 00

  Yeah, you know, all of those are, you know, solid state transformers, switches, et cetera, you know, rebuilding, you know, the power delivery network. And I think that will, you know, have a lot of, you know, innovation, you know, sort of like, okay, power's cool again, right? You know, for, and then you're going to need all of those in the cooling systems as well, right? You know, and so there'll be a bunch of innovations in that, you know, so, you know, you know, turbines and refrigerants and, you know, all of a sudden they get to be, you know, exciting new technologies again. So, you know, for all of us hardware guys, this, yeah, this, you know, it's like Renaissance and brother, this is gonna be.

• 47:53Speaker 02

  After 25 years, I had to go back and read about Cardinals.

• 47:56Speaker 00

  Cardinal efficiencies, oh yeah, this is like good stuff. How does that first law of thermodynamics work again?

• 48:02Guido Appenzeller?

  The physics, I think we have something after all here.

• 48:06Speaker 00

  You know, so, so it's going to be, you know, an exciting time that way, you know, but, you know, you know, we also say, you know, we need to change the physics, right? Because seam loss hasn't improved, you know, the essential power per ter, you know, power per teraflop for the last five generations of Nvidia chips, you know, flatline, not okay, right? And that's where some of the things, you know, we have a superconducting company, Snoke, Snoke, right, which affords the opportunity to fundamentally change the physics, you know, a thousand times better power performance. So I do think some of those innovations are upon us in the near future.

• 48:40Speaker 02

  I've been a terrible time manager here for the past hour. So, so many topics yet to cover, we didn't cover, but we gotta cover at least one before we leave.

• 48:50Speaker 00

  Okay.

• 48:50Speaker 02

  Because now.

• 48:51Speaker 00

  You're in charge.

• 48:52Speaker 02

  Yes, so I thought.

• 48:57Speaker 02

  Now VMs are back. How do you think about that? The pharmacy of VMware.

• 49:03Speaker 00

  Well, you know, I, I.

• 49:05Guido Appenzeller?

  I feel excluded from this conversation.

• 49:07Speaker 02

  No, no, no.

• 49:08Guido Appenzeller?

  Pharmacy as a beer. But we go on.

• 49:10Speaker 02

  Part of it was I was going to talk about networking as well. So, but I don't.

• 49:14Speaker 00

  But, you know, hey, you know, I, I think the, you know, every innovation has always led to the next abstraction. Right. And I think many of these ideas and, you know, right? You know, I always like to say, you know, you know, what's, what's new in our world today? You know, data structures, algorithms, you know, and abstraction. You know, we bring it back to those, you know, foundational. And hey, you know, I think, you know, the virtual machine abstraction, you know, whether it's done from the infrastructure level or from the application level, you know, I think it's still a foundational abstraction model, you know, that deserves to always have a place in the compute hierarchy. So, you know, so what do you think? One of us go back and run VMware?

• 49:52Speaker 02

  Yeah, that's what I tell, I think I told Gud as well. It's time to go back and run VMware again.

• 49:57Speaker 00

  Yeah, yeah. You know, and, you know, and I do think some of these abstractions, you know, but you also think about it and like one of the things we did at VMware, we, you know, essentially, you know, managed every aspect of the computing hierarchy.

• 50:10Speaker 02

  Yeah.

• 50:10Speaker 00

  Right. You know, and you manage the workload. You manage the network. You manage the storage, you know, system and how do you abstract those? And I think in the AI context, right? We think about, you know, these agents swarms. Well, who's going to manage the agents? Who's going to create the security profiles around all of the agents? Who's going to be managed? The performance of the agents, you know, essentially every fundamental element of virtualization and management needs to get recreated in this next computing hierarchy. And there's going to be lots of wonderful companies that you'll sort of break through for how those things will be, you know, done.

• 50:44Speaker 02

  Okay. Do you want to end it on networking?

• 50:48Guido Appenzeller?

  No, actually, I love to stay with the, with the VMs for a second. And the most interesting thing for me is that we're now building VMs so that they're usable for agents as opposed to usable for humans. And what we've seen in some other companies is that the change is a lot. The form factor that changes a lot, how you market them, right? Whether you want the agent to make the pick for, for, for you VM offering. It, it changes a lot how much complexity you can have. It changes how, how quickly the startup should be. Humans are a lot more patient than, than agents.

• 51:15Speaker 00

  Uh huh.

• 51:15Guido Appenzeller?

  Um, so it's, if you have to rebuild VMware today, how would it look different?

• 51:22Speaker 00

  Yeah. And, and I do think.

• 51:23Guido Appenzeller?

  The, the VMware for agents.

• 51:25Speaker 00

  Yeah. And I, and I do think there's, you know, does there need to be a VMware for agents for humans? Right? You know.

• 51:33Guido Appenzeller?

  I see what you mean. Yes.

• 51:33Speaker 00

  Right. You know, in the sense that, you know, I need humans.

• 51:36Guido Appenzeller?

  Let's stick to the agents. They're way more.

• 51:39Speaker 00

  Yeah. But, but, you know, I think it's an important point because I do think that there needs to be, you know, and I'll use, you know, Dario's constitution, right? Or guardrails, right? Or being able to set policies. And I think then you, you really say, okay, every aspect of the virtual machine as we know, we love it today, it is servicing agents. Right? As opposed to servicing hardware and humans. Right? So I think, you know, that becomes your fundamental design constraint on this side. You know, how do I make agents great? Right? How can I make them secure? How can I make them performant? You know, you know, uh, you know, how do I abstract them? How can I migrate them? All of those things, you know, have the new embodiments, you know, V motion for agents, right?

• 52:24Speaker 00

  It's like every one of these will have the new embodiment here for them. But then I also need to think about how can humans be setting the policies, describing the constitutions, you know, getting the, you know, dashboards, et cetera. And I think that's sort of, you know, where the interesting thing is because you need to do both of those, right? Not just one of those, but the fundamental, you know, I'll say the hard design stuff like we did for, you know, virtual machines was associated abstracting the hardware in a performant way for applications or operating systems. Here it's, you know, abstracting the hardware and the operations for agents. It's a fundamental thing that you're serving.

• 52:59Speaker 02

  It was great discussing this and thank you so much, Pat, for coming in and we'll do it again.

• 53:04Speaker 00

  Well, two of my favorite people. Anytime.

• 53:06Speaker 02

  All right.

• 53:07Guido Appenzeller?

  That was awesome. Thank you.

• 53:08Speaker 00

  Thank you.

• 53:11Erik Torenberg?

  Thanks for listening to this episode of the A16Z Podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or a review and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts and Spotify. Follow us on X at A16Z and subscribe to our Substack at A16Z.substack.com. Thanks again for listening and I'll see you in the next episode.

• 53:37Erik Torenberg?

  As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax or investment advice, or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com/disclosures.