The Infrastructure Behind the Machine Age

Ben Horowitz, Martin Casado, Raghu Raghuram, and Erik Torenberg discuss the launch of a16z's new Machine Age Fund and the infrastructure buildout behind AI, from chips, memory, and networking to power, cooling, and data centers. Why a dedicated fund now? The group argues that the bottleneck in AI is increasingly shifting from the models themselves to everything beneath them. Hyperscaler CapEx is surging, critical components are booked years in advance, and each new generation of reasoning and agents requires dramatically more compute. They unpack why this cycle looks different from previous infrastructure booms and how AI is turning problems once constrained by engineering into problems that can increasingly be attacked with capital and compute. They also explore where the next generation of infrastructure companies could emerge, why founders are returning to hard technical problems across hardware and systems, and what it will take to rebuild the computing stack for the Machine Age.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Ben Horowitz

  We have a whole new technology that's the most important technology ever. And you need a whole new infrastructure.

• 0:05Speaker 01

  Normally, when we talk about the infrastructure world, we're talking about the servers, the storage and the network. Here it goes all the way down to the mines, copper mines. That's how widespread this thing is going to be.

• 0:15Martin Casado?

  It used to be when you built something, it was an engineering problem. And here it feels like it really is a resource limitation. So whether it's tokens or not, we're pouring a ton of money into systems. And then those systems are producing a result. And right now, we're bottlenecking those systems' ability to actually match the resources we're pouring into them.

• 0:32Speaker 01

  The leading memory controller said the demand they have today, it will take them three years of capacity to supply.

• 0:38Erik Torenberg?

  If this fund does what we think it will do, how do we see the world in five to 10 years?

• 0:42Ben Horowitz

  America wins in the infrastructure day, and that would be awesome.

• 0:47Speaker 04

  Today, A16Z is announcing the Machine Age Fund, a new fund dedicated to the infrastructure powering the next era of AI. I'm joined by Ben Horowitz, Raghu Raghuram and Martine Casado to explain why we're launching it now and why we believe the next major bottleneck in AI isn't necessarily the model. It's everything underneath it. Chips, memory, networking, power, cooling and data centers are all being pushed beyond what they were originally designed to handle. At the same time, AI is changing an old rule of technology. Throwing money at an engineering problem didn't necessarily make it move faster. Increasingly, capital can be converted directly into compute

• 1:30Speaker 04

  and compute into more capable intelligence. We unpack what that shift means, where new infrastructure companies can break through and why a new generation of founders is returning to some of the hardest problems in computing.

• 1:46Speaker 04

  Ben, Martine, Raghu, welcome.

• 1:48Martin Casado?

  Thank you.

• 1:49Ben Horowitz

  All right, thank you.

• 1:50Erik Torenberg?

  I want to start with a Mark quote to introduce this new fund. This is the biggest technological revolution of my lifetime. This is clearly bigger than the internet. The comps on this are the microprocessor, the steam engine and electricity, or maybe the wheel. Guys, the Machine Age Fund, please introduce it. Ben, start us off.

• 2:08Ben Horowitz

  Well, basically, what's happened is we have a whole new technology that's the most important technology ever. And what happens every time there's a dramatic new way of using all of the things that we love, infrastructure, you need a whole new infrastructure. And never has it been more high impact as it is on this one. So not only do we need new chips, new system software, we need new ways of doing power, we need to replace copper. I mean, like, it's absolutely everything. So it's a very exciting time.

• 2:47Ben Horowitz

  So particularly for the kind of hardware aspects of this new era, we needed a new approach.

• 2:54Speaker 01

  Yeah, I would agree. Normally, when we, at least in the computing, when we talk about the infrastructure world, we're talking about the servers and the storage and the network. Here it goes all the way down to the mines, copper mines. That's how widespread this thing is going to be. And that's number one. And number two, I think what we have seen over the last three years is a steady increase of the capabilities of the models, where the model is no longer the bottleneck. And in fact, using AI, these models are getting better, faster and faster and faster. Now the bottleneck is all what I call south of the model. And so that's why we need to work on that.

• 3:32Martin Casado?

  The only thing I'd add very quickly is we tend to follow founders. And we've been watching over the last couple of years as the number of very strong teams going after complex hardware problems has increased. I don't know the actual numbers, but I was trying to estimate it over the weekend. So I think we'd get maybe 5% of the deals from top founders would come in, would be harder before. Now it's the north of 20% or 30% right now. So it's like the founder community, which tends to be much smarter than the VC community, has identified this as a very active area for innovation and they're responding.

• 4:00Ben Horowitz

  I think 5% is probably generous.

• 4:02Martin Casado?

  Yeah, it's very low. It's very low. Yeah, 3%. Yeah.

• 4:05Erik Torenberg?

  And explain some of the macro conditions that have led to this change in terms of the surplus of founders pursuing these ideas. Like, what are they seeing that's a need?

• 4:12Martin Casado?

  Well, I mean, the obvious is the demand for AI is basically infinite. And as a result of that, every part of the supply chain is under duress. I mean, everything, including like materials used to make things like memory. It's also very interesting, there's something unique about AI, which because the demand is infinite and growth is infinite, what you tend to worry about is the margin of companies, which is how efficient it is. Like, normally you worry about growth. Can I just, can I just get people to buy this stuff? You don't have to worry about that here. The question, can you do this in a way that's profitable? And a lot of the efficiencies are actually strictly a physical limitation of hardware. And so even the business model of the AI wave is really putting a lot of stress on the existing systems. So they weren't built for AI, they weren't built for those workloads.

• 4:54Martin Casado?

  And I think there's just this, this global observation that we actually need to change the core components to get that efficiency, to help drive the growth and to drive the value of the businesses.

• 5:04Erik Torenberg?

  Yeah. And how do we know that demand is actually outpacing supply here rather than this being another hype cycle?

• 5:13Speaker 01

  Yeah, there are a lot of numbers. Firstly, it is that some of the smartest judges of demand are calling huge purchase orders. I mean, if you look at the hyperscalers, right? Their cap expand has been exploding. Next year, supposedly it could reach a trillion dollars collectively across the bay. Hyperscalers this year, it's about $700 billion, right? And if you think about the hyperscalers' position in the industry, they see demand from everywhere, right? They see obviously the frontier labs wanting their compute. They see the AI native companies. They see the enterprise. They see the US geography, the international geography.

• 5:57Speaker 01

  So if anybody has visibility, it is them. And they're being jacking up their capex, like, it's never been seen before, right? So that's a clear, clear sign. And secondly, if you look at the companies that we see on a day-to-day basis, they are all ripping, all the application companies, the growth is insane. The frontier labs, the growth is insane, it's been documented. So I would say on the demand side, the signals have never been clear that this is not a hybrid.

• 6:27Ben Horowitz

  And prices are going up. Like, we've never seen prices go, like, up on chips.

• 6:32Martin Casado?

  GPU prices went down.

• 6:33Ben Horowitz

  They always go down.

• 6:34Martin Casado?

  Yeah.

• 6:34Speaker 01

  As long as it goes down, if you look at the price curve, it's like this and then it's back to this. I mean, only like 5%, 10% of the market has stopped today.

• 6:45Martin Casado?

  I mean, if you look at the supply across the board, it's basically all booked out to 2028. I mean, it's so bad, we've actually seen multi-day auctions for a few thousand GPUs. The other side of that, of course, is demand. And as Raghu said, we've seen the fastest growing companies we've seen in the history of the industry.

• 7:02Speaker 01

  And also the unit of work that AI can do, the value of that unit of work keeps increasing. But underneath the cover, the number of tokens that are consumed is going by orders of magnitude, right? If it's 100 tokens for chat or agent, it's thousands of tokens, right? So you've got expansion on both sides of demand. One is the unit of work is becoming more and more consumptive of tokens. And then secondly, the number of people there for that are going to be benefited. It's not just the developers, it's going to be all knowledge workers and then all of beyond that. So that's what we see.

• 7:38Erik Torenberg?

  You said the key components in supply are sold out to 2027, maybe 2028. What does it mean for an entire industry to be sold out that far?

• 7:46Martin Casado?

  I don't know if this has ever happened before. Do you guys recall? I mean, remember in the internet days, when we were doing massive build out, the majority that was actually being put in the ground was speculative and was dark. Remember the dark fiber in here? Basically, every GPU that's been created is already pre-sold.

• 8:03Ben Horowitz

  Yeah, we weren't quite there. I mean, there was a lack of bandwidth, like, in the 98, 99 timeframe. But there wasn't that much real demand for it because there just weren't that many people on the internet. So it was a two-sided thing. And the companies were all rushing there and needed more bandwidth theoretically, but there weren't the users on the other side to consume it necessarily. And then to really consume a lot of bandwidth, you have to do high bandwidth things like video, which weren't really viable for a number of reasons that had nothing to do with how much bandwidth was in the data center. So it smelled similar, but it wasn't this.

• 8:47Ben Horowitz

  This is like, we're flat out and people are reselling GPUs for four times what they bought them for and this kind of thing. Like, it's just not, and then we're also out of power and cooling. And then on top of that, it's really hard to build because there's these incredible political headwinds going into it. So it's really unprecedented in my career that we've had anything like this.

• 9:14Martin Casado?

  Yeah. I want to give you a quick anecdote. So I was talking to a CFO of a large company, a large public company, who had historically been very resistant about going into the cloud. So they had a lot of servers. And they were doing an inventory check and they realized that the memory in their servers had increased so much it could fund the entire migration to the cloud. So I just feel like we're in a very unusual situation now.

• 9:35Ben Horowitz

  That's right. We're out of many things. Power, cooling, memory, GPUs, like, you name it, we're out of it.

• 9:42Speaker 01

  So the flagship conference for the industry is the one called Hot Chips. It's going on in Stanford. And the leading memory controller said the demand they have today, it will take them three years of capacity to supply. It is just today. It's not even future demand.

• 10:05Erik Torenberg?

  So in terms of being about everything simultaneously, is it because people just underestimated how good the models would be? How useful they would be? They just couldn't have resumed the demand?

• 10:14Martin Casado?

  Well, I don't even think it's that. I mean, this stuff came out of nowhere, right? We're only four years into this. So even if we had a perfect oracle, once it started working, I don't think.

• 10:23Ben Horowitz

  We kind of built the capacity fast enough.

• 10:24Martin Casado?

  The capacity, there's no way. And we're talking about like chip cycles, which tend to be three to four years. We're talking about breaking ground and building data centers, which is four to five years.

• 10:35Ben Horowitz

  And connecting, breaking down and building them and having a power source. So like, you either have to build your own power or like, usually both. You've got to build your own power and have a power source, which is not easy.

• 10:48Speaker 01

  Yeah, if you have an industry that's used, if it's growing at 20%, 30%, it's a great growth rate, right? And it's being connected to an AI software industry that's like triple digits of the base. So you can see the disconnect, right? So it's just wide net.

• 11:05Erik Torenberg?

  And so why didn't this fund exist, you know, five years ago or seven years ago? Or why was it not a great category to invest in the same way, Brad?

• 11:15Ben Horowitz

  Well, I would say we're probably, I'd like to think for just in time, but you know, we probably would have been well suited to have it at least a couple of years ago.

• 11:25Martin Casado?

  Yeah, also, you could actually point on basically every epoch to an independent company that came up, right? Clearly, the move from the client, from a mainframe to the client server, we saw a bunch of companies come up. The move to the internet, this is why we've got Cisco and Juniper. Even in the mega data centers, which by the way, was largely driven by the incumbent cloud providers verticalizing, you saw the arising of Arista. So there has been the ability to invest in silicon and hardware, but it's been relatively minor because the change has been relatively minor, like one chip company, one switch company, where here, everything is. And so I agree with Ben, we're probably, you know, we probably could have started a little bit earlier, but the amount of change is so high now that it's just an obvious thing to do.

• 12:10Ben Horowitz

  And the other thing is, the demand for intelligence is so vertical with really no end in sight. I mean, because every company that's adopted it is growing very fast in its usage. And then, most companies haven't adopted it to a high degree. And then, consumers are just getting started. And so it's going to probably, the demand for tokens is probably going to grow close to 1,000% a year, which you cannot grow supply that fast. Like, we're

• 12:49Ben Horowitz

  not, like, the amount of just work we're going to have to do across the board to get to the point where we can grow like infra at that kind of rate is, is pretty vast. So I think there's a lot of investing opportunity on the way. And by the way, the other thing is, like, all the architectures of the hardware systems were built for a whole different era of computing. And so more than just, we need more capacity, we need capacity to build, there's lots of opportunities to build different kinds of infrastructure.

• 13:27Speaker 01

  Yeah, they're all reaching the physics limits for what they were designed, right? Like what Ben was talking about, proper and so on and so forth. And you could go across every one of these categories, and you could find, okay, this is the limit of this type of technology. So now, you got to keep some technical breakthroughs to get to the next one.

• 13:52Erik Torenberg?

  Yeah. I want to dive deeper on that advanced side for a second. As we've moved from chatbots to reasoning to agents to, to multi-agents, each step has multiplied the number of tokens a single task takes up by orders, increasing orders.

• 14:07Ben Horowitz

  Yeah, nobody likes to use AI more than AI.

• 14:12Erik Torenberg?

  Why does that keep happening instead of leveling off? And do you just see that happening indefinitely, just continuing to?

• 14:18Martin Casado?

  Well, so there's a couple of aspects of that. The first one is, is for sure, right now, if you look at like, the, the way we're achieving scaling, the way we're doing it is through a lot of inference. So through a lot of token, right? If you think about what RL is, you know, it's a lot of inference. If you think about chain of thought, it's a lot of inference. If you think about long running agents, of course, it's a lot of inference. And so that's just basically been one of the approaches that we've been using to, to scaling. I think if you want to step back and say, kind of, what is the macro trend here? It used to be when you built something, it was an engineering problem. You throw a bunch of engineers at it, and that doesn't scale, and that would have a natural law of engineering physics, which is what the mythical man month came from. And here, it feels

• 15:01Martin Casado?

  like it really is a resource limitation. So whether it's tokens or not, we're pouring a ton of money into systems, and then those systems are producing a result. And right now, we're bottlenecked on those positions, those systems' ability to actually match the resource we're pouring into them. And so I think, like, tokens right now is probably where we're on the scaling curve. But we don't have a natural regulator like engineering like we did before. So I think we should expect this to continue, and we have to build a supply to support it.

• 15:28Ben Horowitz

  Yeah, like, the similar way to think about it is, any problem that you have can be solved with enough infrastructure.

• 15:38Martin Casado?

  Basically.

• 15:39Ben Horowitz

  GPUs and power and money. And so until we run out of problems, we're not going to run out of demand. And that's the, that's the challenge.

• 15:52Speaker 01

  I mean, AI's answer to getting better and better is to use more AI, right? Inference is one basic building block that it keeps using over and over and over again. And so that's why these tokens multiply.

• 16:04Martin Casado?

  Yeah, even the auto catalytic effect. So even the idea of using AI to create more AI, like creating GPU kernel, of course, is just using more AI as part of the process. So again, one way that we think about it is, in the past, money would come in, you have an engineering problem, we know that it takes two years, normally fails, you know, it's a national governor, and then you get the product on the other end. This, there's, there's, there's nothing between the money going in and then the hardware, you know, creating intelligence. And so now, we're just limited by our ability to create supply. It's a very, very different dynamic.

• 16:38Speaker 01

  Give me more GPUs and our solid is gone.

• 16:41Martin Casado?

  Yeah, that's right.

• 16:42Speaker 01

  The cycle.

• 16:43Martin Casado?

  As long as you have the money, the GPUs and the data, you know, for the foreseeable future, you'll be able to scale these things.

• 16:49Erik Torenberg?

  And it's fascinating, because, you know, over the last decade, it feels like there's so many, you know, people and the pervasive sentiment was, there's too much money going to startups, we're overfunding these startups, there's too much money in venture capital. Say more about what that means, because there used to be this sort of skepticism that the more money you put into, into the industry, that, you know, there would be bigger outcomes. And now, you know, we're saying at the offsite that there's, to some degree, the, the market is as big as we collectively contribute to it.

• 17:18Ben Horowitz

  Yeah, so this is, this, look, the one thing we all knew in startup world is that if I have a two-year lead on you, and you try and cast me by hiring 1,000 engineers, you're going to wreck your company. Like, that never works. It's a mythical man month. Nine women can't have a baby in a month. That, like, that's it. Like, that never works. Okay, now that works. But it's not hiring 100, 1,000 engineers, it's taking $3 billion and like, lighting up a magnificent cluster. And then all of a sudden, you know, whatever Grok can come out of nowhere and like, oh, all of a sudden it's real, or Kimmy, or what

• 18:01Ben Horowitz

  have you. It's just like these leads, you can throw money at the problem, and you can throw money at almost any problem, and that works. And so that is just completely different than anything we've ever lived through. So we're, by the way, we're all psychologically adjusting to this.

• 18:22Erik Torenberg?

  The ChatGPT app has a billion weekly actives. There's about 30 million developers who are using, you know, relatively a big portion of compute demands. How do we think about commute, compute demands needs now and in the future in light of what people are actually doing with AI?

• 18:39Speaker 01

  That's the progression, right? So ChatGPT was a casual app, and it's coding for professionals, right? Now, using coding, you have built amazing tools for knowledge workers. So that's the next frontier. And now there are over a billion knowledge workers in the line, right? And that, it's a long ways to go for that demand. And by the way, the work that they do, all this work, grown automation, so on. And then you get to the back office, which is all the agents. So progressively, each of these things unlocks, I would say, an utter magnitude more demand. I mean, just at the start of this.

• 19:18Ben Horowitz

  Well, and now you have Grokbot, which is kind of, you know, what happened with coding is kind of happening with all use of computer via Grokbot. And so we're in a whole nother wave of demand, and most certainly, there's going to be more to come. So it does seem quite unlimited at the moment. And we haven't even gotten into embodied AI or robots, which are going to be another source of demand.

• 19:49Speaker 01

  Mark, Mark is an expert, but I- My understanding is Grokbot uses computer use, which is just like, you know, being sitting inside the computer type of way.

• 19:58Martin Casado?

  I literally used it over the weekend to to update my credit card with a bunch of services that I'd been like lazy to do and cancel a bunch of subscriptions. I mean, this is not coding or whatever. This is true computer use.

• 20:08Speaker 01

  All of a sudden, you're like, possible knowledge workers, except they're all sitting inside of the computer doing work.

• 20:15Martin Casado?

  I do, I do think that, that Marc Andreessen is right. It's like the right analog here is like the steam engine or electricity in the following way. Like we've, we've introduced this new thing that you can turn to work. And there's some very obvious applications now, but there's probably thirty, forty years of throwing computer problems. I didn't think with a clear reward signal and we were just starting like we've got language and code, that's it. And just starting computer use. But like, what else are we looking at? We're looking at in terms of science, materials, biology. I mean, of course creativity is a massive use. And so, listen, we're at the very, very early part of a very long journey. And we've removed this key bottleneck, which is, you know, traditional software engineering. Now, of course, you know, bottlenecks will move and there'll be kind of more complexity elsewhere.

• 20:59Martin Casado?

  But I think we're at it very early in a very long run of throwing computer problems. So let's expect, you know, this compute need to persist for decades.

• 21:08Erik Torenberg?

  But because we mentioned it, Marty, talk about Grokbot. Because we, we're talking at the offsite about how, you know, what struck you about it. Obviously, we're involved in every possible way you could, you could be involved. But what, um, you know, what, what did you find so interesting about?

• 21:21Martin Casado?

  So I think we've, I think we've, as an industry, gone through kind of multiple realizations for how AI enters our lives, right? And, and very early on, we're like, okay, well, you add AI to a product and it's like, whatever, it's like a search bar. And then you kind of, you know, you, you do chat with it and it chats back. So that's kind of the traditional way to do it.

• 21:42Martin Casado?

  And then OpenClock kind of showed up and that was earlier in the year. And with OpenClock, I say, okay, well, maybe like it just being like Google, but better. Maybe that's not the full embodiment of it. How about we'll have it be a standalone thing, but it'll be an extension of you and it'll share your keys and it'll know your passwords and it'll just kind of do stuff that you would do, right? So it's kind of an extension of you, but it's more like a human, an extension of you. And then when I think Grokbot got really right is, no, how about it's just actually an employee? So now you have this thing that's an entity and it doesn't have like special access to your keys or whatever. It has its own computer and it has its own browser. And because these are the smartest models in the world, it can do whatever an employee can do. And it's kind of interesting because now actually if I, if I want something done, my

• 22:25Martin Casado?

  first thing I, I think is like, well, can Grokbot do it for me? And, and often the answer is yes, even if it's something you wouldn't, you know, expect it. So the obvious ones are like, whatever, it'll like manage my calendar, you know, like book a meeting. But there's also not, uh, non-obvious ones as well. Like, so for example, I'll have it, um, uh, read through my email and do triage. And, and I did, I don't tell it how to do that, but it will know to check with me before actually doing the triage. So like, these things are sophisticated enough that you can give it relatively high level tasks and it'll do kind of, you know, like sophisticated things as a result.

• 22:59Erik Torenberg?

  Ben, I know you're thinking a lot, a lot about this and how this, you know, works in the organization. You think a lot about culture, of course. What are your thoughts here?

• 23:06Ben Horowitz

  Well, I mean, I think if you just look at us, um,

• 23:12Ben Horowitz

  you know, it's like having a new kind of employee and there's going to be a lot of them. And we have to just like, you know, with our, we spent many, many, many years figuring out how to work with our kind of regular human employees. And now we've got these other kinds of employees. And, you know, there is a learning curve with them. So, um, they can burn a lot of tokens and spend a lot of money and get nothing productive done. They can forget stuff. They can make stuff up. You know, they can have good behavior. They can have bad behavior. They can, they can create security problems.

• 23:54Ben Horowitz

  So like there, there, there's all those aspects to it, but like they can also be like super duper productive. And so I think figuring out how to integrate them in, have them work nicely with the people that they're working with, um, the actual humans, uh, is all something that we're learning how to do. I mean, I, I don't want to sit up here and say, I've cracked the code. We've got this marvelous loop and, um, the whole firm is just completely automated now and I'm going to slowly get rid of all the humans because I can't like, that's not at all where we are. We're much more going like, okay, how do we make all our humans superhuman without like wrecking the place, um, because the bots get out of control?

• 24:37Speaker 01

  Yeah. And it's interesting because we've done, we've tried a couple of different ways. It's how best to get agents into the system, if you will. And eventually it was Martin's insight, just treat them as people and get it done. And that's what we're doing. That's turned out to be the most durable way of getting this thing going inside of an organization.

• 25:00Erik Torenberg?

  I want to go back to the supply side and go deeper into the, the bottlenecks. You know, we, we were talking about how, you know, in terms of the data centers, the chip architecture, system software, facilities themselves, that none of them were designed with, with AI mind. What would it look like for them to be designed with AI? Like what is sort of the mental model for thinking about what that could mean?

• 25:19Speaker 01

  Yeah. So, um, I mean, I think it's tough to statement that you just say, hey, original model, um, of infrastructure on any of these models has to change. You can go set category by column and see varied bricks, right? And then you start unlocking the bottlenecks in any one of these things. So eventually you had to get to a system where if you look at what an inference engine does, right? It takes up a lot of memory. It generates new tokens along with the compute. And so you can just think about, how do I optimize all of this? What does memory need to be? What does a computer need to be? How do they need to talk to each other? How much power does each of them need?

• 26:03Speaker 01

  And if they need all of these power, how do you call each of these, right? And then how do you put the collections of these things together? That is the exercise that's underway in the industry right now with a lot of the founders. So they're breaking down the problem until it's fundamental components and saying, what is the exact nature of the computer that's getting done? Okay, how's could be matrix multiplications? How do I optimize my computer in that kind of a scenario? And they all need memory progressively to generate these tokens. What is the best way of hierarchically arranging this memory, right? And then how does the power consume? I mean, then you've got to connect it together. What are the ways of connecting it on the same chip, but across chips and across data centers?

• 26:46Speaker 01

  How much power does each of these data transmission take? So you have to progressively break it all down and rebuild it from these fundamental building blocks. And that's what we see and the way, and that's where we see that generally.

• 26:58Martin Casado?

  Let me give you an interesting mental model to think about and like how the landscapes change. So, um, uh, so today to build a frontier model costs, let's say, three to five billion dollars, right? So, and that's, you know, and, and that's to train it. And so the inference has to pay back at least that, of course, right? You know, in order for any of this stuff to be viable, let's say two times that. So let's say that now inference has to, to make $10 billion. So if you can save 20% of efficiency on that, that's $2 billion and you can easily build an ASIC for $2 billion, right? So it's, so we've actually gotten to this interesting point in the industry where it

• 27:39Martin Casado?

  actually makes sense to build an ASIC per model, just because the amount, amount of capital investment in that model. And then unlike traditional software, traditional software has a lot of state and a lot of, you know, it's very dynamic. These models are fixed, the model weights are fixed. And so we don't know if the world goes to per model ASICs, but it gives you a great mental model of how you would evolve the architecture to be far more bespoke for these massive capital investments we're doing. Like, I don't think in the history of the industry, we've ever created a digital artifact with something like $5 billion that went directly into that artifact. And so, you know, like this, I think is going to put the greatest demands on hardware that we've ever seen.

• 28:21Erik Torenberg?

  To that end, rack power requirements are moving from roughly five to 10 kilowatts to 100 to 50 kilowatts. Compute density is climbing, something like 70x. Cooling is moving from air to liquid as a requirement. What are the investment opportunities as a result of this?

• 28:37Ben Horowitz

  Well, first of all, when you get to that level of power per rack, AC power doesn't work anymore. So like, that's a pretty wild thing. So now, now you're into DC power, which by the way, also requires its own cooling. And is like, uh, by the way, super fucking dangerous, uh, which is kind of ironic because this was Edison promoted DC power by claiming how dangerous AC power was and demonstrating it by like electrocuting animals and things.

• 29:13Martin Casado?

  The horse, yeah.

• 29:14Ben Horowitz

  The horse, yeah. Yeah. So, but he was right, but around his own kind of power, which is extremely powerful is the good news. So, you know, just starting with power, uh, yeah, that's going to be like very, very different. I think with cooling, so there, and this gets into.

• 29:36Ben Horowitz

  So yes, we're going air cooling to liquid cooling. I think we're already at liquid cooling for any city or data center. Like, that's already kind of a done thing. But it gets into, okay, you know, given the political environment and so forth, you like, liquid cooling isn't enough. It's got to be eco-friendly liquid cooling. And, you know, kind of DC power is not enough. It's got to be

• 30:03Ben Horowitz

  power that contributes to the power of society, not takes away from it. And so you have data centers who have been behaving badly. Small percentage, actually, probably 10%, wasting a lot of water.

• 30:20Ben Horowitz

  You know, not as much as pistachios or almonds and so forth as people demonstrate on the internet, but like, they could be a lot more efficient with that. And then there are ones that, you know, kind of are parasites of power and don't contribute power back. I think all that's gonna end, it's gonna have to end just because like that, we've kind of gone through a one-way door on that. So that requires like a level of engineering that, you know, many haven't invested in yet. So that's coming.

• 30:53Ben Horowitz

  And then, you know, like, if racks are that dense, there are other things that, like, the way the floors are designed have to support that kind of weight. You know, that kind of thing is, is actually for real.

• 31:11Ben Horowitz

  And then, you know, I think that there's, you know, you just need a lot of everything. And so there's gonna be, you know, also by, the things are really loud. So you have to build the data center with thicker walls, or you're gonna disturb the peace in the neighborhood, which is not gonna be acceptable. Like, I don't think any state's gonna allow that. And so a lot of the ways people have architected and designed the buildings themselves are already completely obsolete. Like, once we get to Feynman, a much smaller percentage of the data centers that we have today work.

• 31:47Speaker 01

  In fact, I mean, everybody talks about memory prices, but one of the fastest areas where price is increasing is reinforced concrete for their send. The other thing that happens when these data centers are sending 800 volts to the rack is it's become so dangerous, number one. But secondly, we don't have enough electrical contractors that have the expertise to deal with 800 volts inside the data center. Because this is high voltage.

• 32:18Ben Horowitz

  Only 2% of electrical engineer, or electricians in the US have been certified on DC power. So like, that gives you an idea. Now, Meta's got a whole program to train people up and so forth, which is great. It's like a new job corps where they train people for free to do this job. But, you know, it's funny, AI is taking all the jobs. AI is gonna create a lot of new electricians.

• 32:43Erik Torenberg?

  Yeah, I think we're just doing something in space, too.

• 32:45Speaker 01

  The big guys that own the big cloud data, big data centers, they all are furiously experimenting with robots, right, to do the work of assembling or putting servers into the data center, etc. And so you will see that increasing as a result of the evolution of AI.

• 33:06Martin Casado?

  By the way, to be, to be clear, on the, on the actual fund that we're raising, our focus is on computer science infrastructure. So anything a model runs on, that's computer science, right? So think, you know, chips, network, interconnect, storage, all the way down, probably to the electricity.

• 33:25Erik Torenberg?

  Yeah. And say more about the robotics arm in terms of what we'll be doing versus maybe American dynamism, or how to think about that?

• 33:31Martin Casado?

  Yeah, for sure. So, you know, again, we think that any platform that, that AI will run on, like, what are the, the, the great breakthroughs that AI does is it allows computers to interact with the physical world, right? It can see, it can hear, it can talk, right? And this means new platforms, right? And the simplest way, people will say edge device, but that doesn't really mean anything, right? I mean, it could be a mobile device, it could be a CDN, it could be a laptop, but it also could be an embodied, you know, device that goes around. And so again, we, we, as, you know, as infrastructure-focused investors, don't do heavy regulated industries or more verticalized industries, but any sort of computer science platform that's gonna push AI further out, we're quite interested in.

• 34:17Erik Torenberg?

  Yeah. Going back to the data centers, by 2028, new data centers are gonna need something like 44 gigawatts of additional power against maybe 25 gigawatts of expected grid additions.

• 34:28Ben Horowitz

  Hold on, hold on, we use that word gigawatt.

• 34:31Martin Casado?

  No, it's like, it's like, oh, well, I have 100 gigawatts.

• 34:35Ben Horowitz

  Martin, what's a gigawatt?

• 34:36Martin Casado?

  I mean, I mean, how big is it? It's multiple football fields. I mean, it's massive. It's 50,000, 50,000 people.

• 34:44Martin Casado?

  What do you mean, what is it power?

• 34:45Ben Horowitz

  Like, like.

• 34:46Martin Casado?

  The equivalent is like 50,000 houses?

• 34:49Ben Horowitz

  City, yeah.

• 34:50Martin Casado?

  50,000, 50,000 homes, 50,000 homes. It's like, I, I grew up, I grew up in Flagstaff, Arizona, which is a town of 40 to 60,000 people, depending on the universities. We have less than a gigawatt of power consumption. I mean, this is.

• 35:03Ben Horowitz

  So you can basically light up and air condition your entire town for a gigawatt.

• 35:07Martin Casado?

  Yeah, I mean, this is.

• 35:08Ben Horowitz

  He's just throwing them around.

• 35:11Martin Casado?

  No, but by the way, everybody talks about the gigawatt. There's very few gigawatt data centers that are actually up. And we've got a long way to go.

• 35:18Erik Torenberg?

  But then, why can't utilities and hyperscalers just build faster?

• 35:23Ben Horowitz

  Oh, there's so many things. Well, there's, first of all, right now, you need humans to build them. So there's just like the regular construction. But much more than that, you need permits. You need access to power that you can plug in. So you're either, you're doing a combination of, you've got to get access to power, which is a massive kind of regulatory bidding struggle. There's very limited kind of amounts and things you can tap into in terms of natural gas, power grids, what have you. But then you also have to build your own power. And guess what? We've got shortages of transformers and turbines and everything that goes into

• 36:08Ben Horowitz

  that. So it's just, you know, like, you've got to get all that stuff. It's not, this is not a software problem. It's not just like a bunch of engineers, can't you like, work weekends and that type of stuff like this? Not that that works anyway, but

• 36:25Ben Horowitz

  there are real bottlenecks in this. And these, these lead times are not that easy to compress. And look, we have the best minds in the world trying to figure out how to compress them. But it's not easy. It's not easy. And the demand is not slowing down. So we're already behind. The demand is growing, you know, 10x a year right now. And the, the supply just can't grow that fast.

• 36:54Martin Casado?

  By the way, it is so bad that right now, if we have new companies going for GPUs, it's often in Mexico or Australia or another country, just because it is so difficult in the United States.

• 37:05Ben Horowitz

  Yeah, we're creating huge, both job and long-term economic opportunity in other countries by banning data centers here. I think, like, the right answer would be to set a standard where a data center contributes back to the community. Like, that power gets better, there's no noise, there's no water issue, and it's adding jobs. Like, that ought to be the standard. And then everybody ought to be just held to that standard. And by the way, like, there are, there are data centers that do that now. Like, that's not a, you know, like a futuristic dream or something. Rates, energy rates have gone down, like,

• 37:46Ben Horowitz

  every year they're there. And the reason is, they provide their own power. They give power to, to the state during the day. And then at night, they borrow power from the state when the state doesn't need it. Because you're always, the way power plants work is you're always generating peak capacity. And since a data center has steady capacity during day and night, and a city goes way up in the day and way down at night, that's a symbiotic relationship.

• 38:16Erik Torenberg?

  Zooming out, why do we think the, you know, we were batting around the name for a little bit. Why do we think machine age is a, is a compelling term for, for what we're doing here?

• 38:29Martin Casado?

  Well, listen, let me, let me take it. So the first one is, I think Ben's absolutely right. Artificial intelligence was the wrong word. Like, we shouldn't have called it, it's machine intelligence.

• 38:39Erik Torenberg?

  Say more about that. Why is that?

• 38:41Martin Casado?

  Because it's not how humans think necessarily, right? I mean, it is a cache of how humans thought, is a collection of humans' thoughts. But like, today, we don't know how to take a, an AI with no knowledge and put it out in the world and have it reconstruct language, right? Like, that's not what we've done, right? We've, we've built something that can learn off of everything we've already learned and then use that in a productive way. And listen, AI is a general term that goes back 70 years in computer science formally that applies to many different things. And of course, it's got a lot of baggage, either from science fiction or from, you know, Nick Bostrom, who wrote about it or whatever. And so, so the first one is just an acknowledgement, like, this really is machine intelligence. And then you want to emphasize the machine part of it. I mean, there's kind of this

• 39:26Martin Casado?

  deep irony. And this is from, you know, the, the software seeding the world people that, you know, you've really come to a place where you pour money into something and then you're limited by the actual machines below it. And so I think it is kind of a nod to like, the hardware component is so significant in this wave, and we want to acknowledge that.

• 39:43Speaker 01

  Yeah, I think that's what's gonna create the next breakthroughs in technologies is the quality of the machines I'm on it. And so that's, that's basically the reason for the name.

• 39:53Martin Casado?

  It's also a cool name.

• 39:55Ben Horowitz

  It sounds good. Futuristic.

• 39:58Erik Torenberg?

  Yeah. The, given how much has been spent on AI infrastructure to date, and how capex intensive these businesses can be, are we past the point where new companies can break in at, at, at sort of material levels? You know, why not incumbents like Nvidia, Corey, etc. just take the lion's share of these markets?

• 40:18Speaker 01

  They're all doing, well, there's no question about it, right? But to our discussion earlier, when you're get, you need fundamentally new innovations to keep the growth continuing or the pace of improvement continuing, whether it's tokens per second, per dollar, or tokens per watt, tokens per rack, right? Or power, you take any metric. If you want to have a 10x on those metrics, you gotta have new innovation. And new innovation traditionally comes from Brazilian founders thinking about solving the problem from first principles in a different way. Right? And that's what's needed here for the next jump in innovation.

• 41:01Martin Casado?

  I mean, this is the law of markets, right? I mean, let's assume that the, the existing silicon incumbents are multi-trillion dollars in market cap, which is absolutely the case. Even 5% of that is a massive private company, massive private company, right? We're talking, you know, annual. And you could say, well, but Nvidia could do that. They could, but why would they? If they're focused on things that are in the 90%, which is also driving the same amount of growth. And you always ask these questions. We ask these questions during the cloud days, right? Like, why wouldn't Amazon did this? You ask these questions during the Microsoft days. Why wouldn't Microsoft do this? There's a very natural law of markets is that once you get to a certain scale, there's tremendous opportunity for innovation at the margins.

• 41:42Ben Horowitz

  Yeah, there's a funny quote from our partner, Alex Rampell. He had this startup called TrialPay. And he was trying to sell it to, or sell, it's his services to Meta and then Facebook. And Dan Rose, who was the head of corp dev at the time, said, Alex, that's great. You're, it sounds like you can collect a lot of silver bricks, but I'm like, I have so many gold bricks, I can't even pick them all up.

• 42:08Martin Casado?

  Yeah.

• 42:08Ben Horowitz

  So the last thing I'm doing is looking at a silver brick. And I think Nvidia is in that position.

• 42:13Martin Casado?

  100%.

• 42:13Erik Torenberg?

  Yeah. We were talking about as we're basically model providers, that if you're, you know, in the sweet spot of, of what OpenAI or Anthropic can do, you know, one of their main sort of interest areas, that might be a tough place to be, but anything outside of those maybe, you know, three to five areas might, you know?

• 42:29Martin Casado?

  As markets, as markets expand, they fragment, right? And it happens all the time. And remember in the early days of Ford, there was the 1913, there was the Rouge River Plant. Literally this, you know, this was like made cars, like in went like water, coal and rubber trees and out came cars.

• 42:45Ben Horowitz

  By the way, he bought a whole rubber tree plantation in the Amazon jungle. And then there's a great book called Fordlandia, where so, cause he wanted to like own like the complete vertical thing, where he created this city called Fordlandia in the Amazon jungle, which had like, was all Americanized, bandstands and ice cream and all this kind of stuff. And it actually worked for a while until he made people like show up to things on time. And then they were like, screw this, get the fuck out of here.

• 43:20Martin Casado?

  So, so now, if you look at the car industry, of course, there's multiple levels of supplier and there's a bunch of companies and this always happens. So, you know, as markets expand, they fragment, there's a lot of, and then, and then once that growth slows down, they tend to consolidate. The consolidation can either be acquisition or it can be like new challengers rise up. And that, that is the everlasting cycle of private markets.

• 43:40Ben Horowitz

  Yeah, cause the use cases are, are multiplying and there's no way, like if you're the biggest company, you can get to the biggest use cases, but there's so many use cases and all, as Martina was saying, very valuable use cases that it's just very hard to get to in a great way.

• 43:57Speaker 01

  Yeah, even influence used to be one simple architecture, right? It no longer is, it's like, it's so complex now. So it's inevitable that you can optimize things in a different way.

• 44:07Martin Casado?

  By the way, here's a very interesting thing, like people don't, often don't understand that like margins kind of fell out of the standard way of doing the technology with software, right? Like it wasn't really a technology problem. Like, once you got the business working, you tended to have pretty good margins, cause that's just kind of how software works, certainly when you shift it, but even as a service. And that's not necessarily the case with AIs. We may actually be entering an era where the optimization in the hardware is absolutely meaningful to the upside of the business in a way that we haven't seen in the past. So there's a lot of opportunity here.

• 44:43Erik Torenberg?

  Let's get deeper in talking about the types of companies we'll be investing in. Maybe we could start by either illustrating the, the, the subsectors, or if we can, talk about a few or a couple investments that we've made, and there's some that haven't been announced yet, but why do you want to take us down?

• 44:59Speaker 01

  Yeah, I mean, the subsectors, as we've been talking about, well, there's every one of these categories, right? The obvious ones are computer chips. But these days, it's not enough to build a chip, you need to build a full system, right? And then therefore what goes into the system is potentially memory innovation, there is potentially networking innovation, there's potentially power chips and so on and so forth. So each one of these categories are categories where you can see public company style companies emerging. And those are all the things that we're looking into.

• 45:33Speaker 01

  And then, once you put it all together, there's a lot of software around it to automate all of these things, to manage these fleets and so on and so forth. So that is another important area. So these things keep building on each other, but every one of these categories is important.

• 45:48Erik Torenberg?

  Talk about what's different about these kinds of companies from the usual company. I mean, one thing you could tell from the companies we announced is their first rounds have been massive, you know, hundreds of millions. Is it a different kind of founder or what else is different as we think about just the practice of, you know, building and investing in these kinds of businesses relative to our traditional software?

• 46:06Ben Horowitz

  Well, I think the big thing is, you hit on one of the big things, which is a lot of money goes in before they get to a product. And that's just kind of the nature of it. Now that's true on big models too, but I would say that's a little more of a known path, whereas this has got a little more risk and a little more money than, than some of the other things that we've done. But, and you know, like a lot of, a lot of the chip founders are here from the past. You know, like the guys who know how to make memory, they're not young.

• 46:45Martin Casado?

  Yeah.

• 46:47Ben Horowitz

  So it's, you know, that part is different too, but it's kind of exciting, you know?

• 46:52Speaker 01

  Yeah, the other thing about these founders, they've all got to be systems founders. So what I mean by that is you can't just be a researcher or a great computer scientist, right? You gotta be able to architect and design the chip or the system, whatever it is. Then you gotta think about how is this thing really actually getting manufactured, right? Who's going to be supplying this? And a whole bunch of these downstream things, which normally if you're building software, you don't know how to think about all of these things. So to really, the best founders, of course, Jensen is the Michael Jordan of this, right?

• 47:33Speaker 01

  They think the entire ecosystem right from the get-go before they start designing the chip, right? Because of the nature of the bottlenecks and all these things that have to come together. So that's, that's a big characteristic that is different.

• 47:46Martin Casado?

  There's two environmental factors that are important too. The first one is the labs are so desperate that they will engage with startups. And so like, we actually have quite a bit of signal early on because they're, you know, the labs are inking deals with companies before they actually have hardware available. And that's a big, big shift than, you know, five years ago, right? Like you just didn't go and, you know, sell your kind of janky hardware thing to Google or whatever. So that, that's a shift. The second one is the, the capital availability has loosened up a lot. I think there's general consensus that, that, you know, it is the time that to reshape this stuff. And so follow-on rounds, there's a lot of capital available, which, you know, of course you want to be investing into areas where there's capital available. And so the atmospherics are also just different.

• 48:26Erik Torenberg?

  Patrick Carlson, you know, remarked a few years ago, he said, hey, it feels like there's less younger founders today, in the way that Zuck, you know, in college building the next Facebook or Gates, you know, in the same way with Microsoft. And of course, you know, the Michael Truel's of the world, there, there's still, you know, some young founders building iconic companies, but it does seem, you know, to your point, that there's more older founders building these, these, these companies or, or less 20 year olds. I'm curious if you, it resonates and why?

• 48:56Ben Horowitz

  Well, I think it's Raghu's point that if.

• 49:00Ben Horowitz

  If you're building something that has like a very complicated supply chain, has to manufacture things and is technically complicated, that, you know, some experience helps. And, you know, if you look at Elon or Travis Kalanick, their companies, when they were young, were software companies. It wasn't till they got like a lot, even those guys, the best guys, needed some experience in building a company's building technology and so forth to kind of graduate to the much more kind of complicated or I would say elaborate domains.

• 49:42Ben Horowitz

  There's just much more, there are many more moving parts in these things. And so, look, when you're learning how to build a company, it's hard enough if you completely understand the product. If you don't completely understand the product and have to learn it while you build the company,

• 50:00Ben Horowitz

  that's just such a steep learning curve for a brand new entrepreneur. So I think that what we're seeing is you see Michael, on the one hand, who is a very young guy, brilliant, but what he built was kind of a pure software AI thing. And then on the other end, you have like an Elon or a Travis who can, who's got enough experience. I think Michael could probably do that, you know, 10 years from now. But today, that would have been hard.

• 50:28Martin Casado?

  And it's important to remember, like, it's been defocused by the entire industry and academia for the last 20 years, right? It just, there just hasn't been the same opportunity. Like, it's been there, but like, it's never been a growth area. The growth areas have been, you know, software, networking, things like that. And so I also think we just have a positive of people coming out of the universities or having experience at large companies that have done this. And there's just not that many. Like, you don't go intern and like build a chip. So, but a lot of that's changing now. Like, listen, we're going to create a whole generation of, of, you know, founders that come from these new companies that will know how to do this. And, you know, they'll be hired in much more junior. And like, I would say, actually, one of the greatest legacies of Elon towards this is, is of course, he's created these great companies,

• 51:11Martin Casado?

  but the amount of entrepreneurs that have come out of SpaceX that have, are changing the entire industrial complex, maybe even greater legacy than the companies themselves. And I think we're going to see the same thing for, for computer science and hardware.

• 51:25Speaker 01

  Yeah. As a matter of fact, one of our investments was started by two founders in their twenties. But if you go walk through their offices, you see the experienced people as well. So it's ideal combination here.

• 51:38Ben Horowitz

  Yeah, yeah, it doesn't necessarily have to be the founder with experience, but that founder better be able to tap into that experience.

• 51:46Martin Casado?

  Get the people with it, yeah.

• 51:47Ben Horowitz

  Yeah, yeah, well, and then be able to work with them. And, and they have to be good and all these kinds of things. It's complicated.

• 51:56Erik Torenberg?

  Speaking of experience, this is a big new fund we're launching and there's no new GPs. We're sort of collecting, it's because you guys have a lot of experience and the rest of the group, you know, in this field that has been kind of latent and dormant.

• 52:11Martin Casado?

  Yeah. Well, it's kind of funny, I think we almost had to be warned against it almost just because like our backgrounds are from, it's hard. And I think the reason that we needed a reminder is because all this has been so much in our careers, like systems and hardware, we're kind of drawn to that. And so, listen, we've been clearly invested in hardware over the years, right? We're in SpaceX, we're in Android. And these are very early checks, we're in Astralis, we're in Waymo. You know, so we even, even early on, we did a number of those investments. But like, you know, this is because it's so much in our DNA. And so I don't think this is necessarily need to increase the team of competencies, just for us.

• 52:46Erik Torenberg?

  If this fund does what we think it will do, how do we see the world changing or looking like in five to 10 years?

• 52:52Ben Horowitz

  Well, you know, hopefully, America wins in the infrastructure game. Then we have lots of like, super eco-friendly, efficient data centers out there and lots and lots and abundance of chips and abundance of memory and abundance of power.

• 53:10Ben Horowitz

  And, you know, that would be awesome. And I think, like, we, it goes back to like, we really think America is a special place and we're important not only to everybody here, but anybody in the world who wants to kind of make a contribution and do something bigger than themselves. It's kind of the best place to come with nothing and do something profound. So we'd like to keep that going. And I think that doesn't continue to go if we lose our lead in technology. I think we'll be in another era and it'll be another country. And maybe they have a different set of values around that.

• 53:51Erik Torenberg?

  Thanks for listening. Appreciate your fun, Martin, Ben, our group, thank you.

• 53:56Ben Horowitz

  Thank you, thank you.

• 54:00Speaker 04

  Thanks for listening to this episode of the A16Z Podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening and I'll see you in the next episode.

• 54:25Speaker 04

  As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax or investment advice or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com/disclosures.