Dylan Patel – Anthropic & OpenAI will have most of the world’s compute by 2028

Had a lot of fun chatting again with my twin brother Dylan Patel. We went through lab economics over the next few years - the shift from inference to training as RSI draws near; and how Anthropic and OpenAI are on track to control most of the world’s usable FLOPs within the next few years (because they can monetize compute better and thus outbid everyone). And then we discuss whether the >$10T of total AI capex we’ll see by the end of the decade will cause a sovereign debt crisis, where hyperscaler debt raises interest rates, drives non-AI exposed countries into bankruptcy, and crashes non-AI equities. One question we weren’t able to resolve is whether there’s anything that can counter all the forces barrelling towards centralization in this industry - the economies of scale in training, the scarcity of compute, and eventually continual learning and RSI. Watch on YouTube https://youtu.be/aV26V1UvkJw; read the transcript https://www.dwarkesh.com/p/dylan-patel-3. Sponsors * Grok Bot https://x.ai/bot has been quite helpful with my search for a new editor. I created a recruiter bot and described the type of editor I was looking for. That bot then spun up a handful of subagents that combed through my emails and X DMs, read the end credits of various documentaries I like, and figured out who edits for some of my favorite YouTubers. It took all of those results, and then delivered me a shortlist of candidates that matched my criteria. Try Grok Bot for yourself athttps://x.ai/botx.ai/bot http://x.ai/bot * Antithesis https://antithesis.com/dwarkesh lets you add time travel to your software testing toolkit. Since the Antithesis platform is fully deterministic, everything that happens inside of it is perfectly reproducible. So if your software crashes, you can rewind to the exact right moment, freeze time, and investigate. Or you can test different hypotheses by perturbing the system: kill a node or disable a feature, see what happens, then reset the trajectory and try something else. Learn more athttps://antithesis.com/dwarkeshantithesis.com/dwarkesh http://antithesis.com/dwarkesh * Jane Street https://janestreet.com/dwarkesh is hiring for two separate ML internships right now, one focused primarily on research and one focused on engineering. In both cases, interns are expected to contribute to real work, not contrived exercises: one common project is adapting a frontier LLM paper to financial markets, which tend to come with a ton of different gnarly challenges. Importantly, you don’t need any finance background to apply. 2027 applications are open now atjanestreet.com/dwarkesh https://janestreet.com/dwarkesh Timestamps (00:00:00) – Two labs will soon control most of the world’s compute (00:07:01) – $6 billion in fab capex enables $1t+ of end revenue (00:13:08) – Compute prices will rise if the labs outbid everyone (00:18:22) – Which layer will capture most of the surplus? (00:25:40) – What could slow down progress? (00:29:43) – Labs are shifting compute from inference to R&D (00:33:27) – China gets less than 10% of new compute, but its labs need less (00:48:48) – Will AI cause a sovereign debt crisis? (01:07:52) – Will the world’s future workforce belong to a few companies? This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com https://www.dwarkesh.com?utm_medium=podcast&utm_campaign=CTA_1

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Dwarkesh Patel?

  Okay, I'm back with Dylan Patel, founder of Semi Analysis. Our version of a family Thanksgiving dinner is a regular yearly podcast. We are not actually related.

• 0:08Dylan Patel?

  Don't tell the people this. It will destroy the myth.

• 0:12Dwarkesh Patel?

  Walk me through. So basically, where the world economy is headed is more and more becoming a function of where like lab economics are headed, where like the compute market is headed, et cetera. So I want to understand where the crazy future ends up within a few years. But let's start with just where we are today. So walk me through lab compute and lab revenue right now and maybe projecting out a year or two.

• 0:36Dylan Patel?

  Yeah, so when we go back to last year, even at the end of the year, most of GDP growth in America was just AI infrastructure. And as we look towards this year, about a third of the compute coming online is for the labs, for OpenAI and Anthropic. Now, it may be built by others and then rented to them, but it's at the end customer, it's them. As we go forward into the future, the numbers for computer ballooning, right? We're at, you know, a little bit over a trillion dollars of capex this year. As we go out into 28, it's going to be more than $2 trillion. The labs are also taking an increasing percentage of this. And so ultimately, you've got a very interesting situation where the labs are going from companies that spend,

• 1:21Dylan Patel?

  you know, tens of billions of dollars a year, to hundreds of billions of dollars a year, to forecasting to spend trillions of dollars a year even towards the end of the decade. And this is at least some of the contracts they've begun signing with their partners. And so this requires a big reshaping of what happens with their economics, right? You know, so up until now, they have been companies that mostly lost money. Anthropic started turning a profit in Q2. It's believed at some point in Q3, OpenAI could potentially start turning a profit even with the big rise of Codex and 5.6 and all this. But if we go back a year ago, everything that they, all the money they had was venture-funded losses, right?

• 2:03Dylan Patel?

  If we go back to even the beginning of this year, it was venture-funded losses. They've now turned the corner and are actually starting to profit. Now, that doesn't mean they're not taking in new capital. The new capital is still coming in to accelerate the growth further. But ultimately, there's more and more of their business is being funded off of their own revenue rather than capital injections into them. Over the last, you know, year and a half, their margins have really skyrocketed. You know, the base cost of compute tends to be around 10 or 13 or 15 million dollars per megawatt. The most interesting aspect about what's happening now is before, again, they were generating, if they served a model, right?

• 2:42Dylan Patel?

  GPT-4 being served on, you know, Nvidia Hopper GPUs was generating negative gross margin for OpenAI. But now, when OpenAI serves GPT-5.6 or Anthropic serves OPUS 5 or Mythos, Fable 5, their revenue generation has passed well beyond the sort of incremental 10, 15 million dollars per megawatt. In the case of Anthropic, the revenue has gone as high as $50 million per megawatt. And what that now enables them to do is, hey, if I spend 10 bucks on inference capacity, I actually generate 50 bucks of revenue, and then I can turn around and incrementally spend all of that profit on training.

• 3:24Dwarkesh Patel?

  One thing I'm very interested in understanding is how you see decentralization of compute happening at the labs or the relative ratio of compute that goes to the world versus goes to the labs. Where if you say right now, a third of marginal compute is going to the labs, by when is it over half of the incremental compute in the world is going to the labs? And by what point do the labs have basically a vast majority of the world's compute?

• 3:50Dylan Patel?

  Yeah, so earlier this year, you know, the beginning of this year, Anthropic, OpenAI started at 2 for OpenAI and less than 2 for Anthropic. End of this year, they're both above 5. So they've 3, 4x compute as a whole. When you, when you look at the incremental compute added, that's about 30% of the compute added this year. And as we step forward to next year, given what's already been signed and penned and inked, you've got something even more dramatic, right? You've got Anthropic, OpenAI are taking as much as 40 to 50% of compute next year. And this centralization doesn't look like it's slowing down or stopping. In fact, it looks like it's only accelerating. Now, who's building that compute for them will change.

• 4:34Dylan Patel?

  You know, next year, a big new entrant is, for example, SpaceX is building a ton of compute. And they're actively going to lease quite a bit of it to Anthropic and OpenAI, most likely, because they're the ones who can, who have the marginal capability to pay the highest price. In addition, OpenAI and Anthropic are also starting to build their own compute. OpenAI with their own chips, Anthropic with TPUs that they're purchasing from Google and deploying with FluidStack. And so when you ask, hey, when does half of the world's incremental new compute go to just OpenAI and Anthropic? I mean, it's really, by the end of next year, it's already half of the incremental compute is going to Anthropic and OpenAI.

• 5:09Dwarkesh Patel?

  Because compute is growing so fast, incremental compute is going to be basically most of compute. So it's very soon, you're saying maybe within a year and a half or two years, and most of the world's compute is owned by two labs, or at least is serving the demand from two labs. How long do you think? So there's this trend where maybe world compute in gigawatts doubles every year, but the compute at the frontier labs triples every single year. But if you keep the current trend going, it goes from like 2 at the beginning of this year to close to like 6 at the end of this year, just multiplying out by 3. 18 by the end of 2027, 54 by the end of 2028. Are you like, okay, at that point, they simply can't continue tripling given the amount of world compute? Or how, how do you see the world compute situation over the next few years?

• 5:52Dylan Patel?

  Yeah, so if the incremental compute adds this year 30 gigawatts, next year 50 gigawatts, and the year after that 70, roughly, you end up with this really interesting phenomenon, which is, okay, well, a new watt deployed this year is significantly more efficient than the watts deployed two years ago. So actually, you know, a humongous percentage of the world's compute was deployed this year. Even though it didn't double the number of watts deployed, I'm deploying GB300s and TPV7s and Tranium 3s, which are way, way, way more efficient, you know, 3x, 5x more performance per watt than the prior generation chips. And so ultimately, you've got a huge ladder here. So Anthropic and OpenAI take on, you know, 45% of compute next year. You've got them in, let's say, December 27.

• 6:37Dylan Patel?

  They have taken on half of the world's incremental new compute. But that half of the world's new incremental compute is actually at a higher performance than everything else before it.

• 6:44Dwarkesh Patel?

  Right.

• 6:45Dylan Patel?

  So you've got another multiplier on that. So by the time you're in like towards the end of 2028, if this trend continues, which I see nothing that's stopping it, you've got them just controlling most of the usable, you know, flops in the world on their own.

• 7:01Dwarkesh Patel?

  The thing I'm confused about is why you think we only add 80 gigawatts in 2028 if we enter a world in which the price, the value of compute increases so much?

• 7:10Dylan Patel?

  That's the upper bound, by the way. That's the, that's the like, I'm so fucking bullish.

• 7:14Dwarkesh Patel?

  Right, okay, so let's, let's do some chain of thought here. So when I interviewed you a few months ago, you said, in order to make a gigawatt of, I think Vera Rubens, you need, what's it called? You need 55,000 N3 wafers, 6K N5 wafers, and 170K DRAM wafers. I don't know, those numbers might have changed.

• 7:34Dylan Patel?

  I'm gonna troll you, but the way you said wafers was so fucking idiot, pafers.

• 7:39Dwarkesh Patel?

  By the way, when we first, when we first moved to the US, I had the V W thing pretty bad. And I was a vegetarian.

• 7:45Dylan Patel?

  Vegetarian, I remember you told me about this.

• 7:48Dwarkesh Patel?

  Inside in Nerd Dakota, I was in elementary school, and I'd be like.

• 7:51Dylan Patel?

  Can I get a wedgie?

• 7:52Dwarkesh Patel?

  Can I get a wedgie? Anyways, so that's for one gigawatt, right?

• 8:00Dylan Patel?

  Yeah.

• 8:01Dwarkesh Patel?

  Now, I had to run your wafer fab equipment model and figure out how much tooling, how much the tooling costs to produce a gigawatt of compute basically every single year. And it was at like 3 to 4 billion dollars. Now, suppose you add in, you know, clean rooms and shell and everything else at the fab. So $6 billion of like fab capex produces every single year a gigawatt. And a gigawatt produces right now $100 billion of revenue. But also, that 6 billion in capex is producing a gigawatt every single year. And that gigawatt is producing $100 billion every single year. So even over the course of five years, so you know.

• 8:45Dwarkesh Patel?

  The first gigawatt has generated five years of profits. The second gigawatt that the FAB is produced has generated four years of profits and so on. Six billion of capex at the FAB level will have generated over a trillion dollars of end AI revenue.

• 9:01Dylan Patel?

  Yeah, there's a lot of OpEx along the way. There's a lot of other capex, like the data center, the power.

• 9:06Dwarkesh Patel?

  And you had to pay like, you know, the OpenAI for the R and D.

• 9:09Dylan Patel?

  There's a lot of different people who need money here.

• 9:11Dwarkesh Patel?

  So but.

• 9:11Dylan Patel?

  But yeah, it's, it's, there's a huge.

• 9:13Dwarkesh Patel?

  Take away half of it for all this, all these middlemen. That still means there's a hundred X discrepancy between FAB capex and end revenue generated. More than that, actually, really, but we're just being very conservative. And as a result, this is capitalism, right? Like you would imagine that people are going to figure, like, we're going to be, you're, you're, you have this huge discrepancy. Or you can turn $1 into $100 and you're, they're not going to figure out a way to make more mirrors.

• 9:41Dylan Patel?

  I mean, they are.

• 9:42Dwarkesh Patel?

  Right.

• 9:42Dylan Patel?

  Just these mirrors take some time to bake, right?

• 9:44Dwarkesh Patel?

  But the, the emergency are so big, we're like, Anthropic and OpenAI, like, we could make a trillion dollars right now. But we're just bottlenecked on the mirrors that go into the ASML machines. They'd spend, okay, how, how can we make more mirrors if we spend $100 billion on this, right? That's the situation we're going to be in pretty soon. And I'm just like, we're not going to be able to solve that supply constraint. That just seems quite hard to imagine.

• 10:04Dylan Patel?

  No, there's definitely, you, you, you've seen people do funny arbitragers here where they buy like turbines and then they try and resell them.

• 10:12Dwarkesh Patel?

  Right.

• 10:12Dylan Patel?

  Because the value of a turbine is way more because it's the thing bottlenecking a data center. You know, I, I think like if anyone had like $400 million and the ability to convince ASML to sell them an EUV tool, they should totally just go buy one and, and wait, wait, wait, and then sell it for north of a billion dollars, right? But, but ultimately, like, yes, capitalism will cause these things to expand, but it's, it's a whip, right? It takes a long time for the whip signal to get to the tail end of, of that. And so the supply chain doesn't react immediately. In fact, you go to talk to, to someone at Carl Zeiss, they're like, yeah, yeah, yeah. We need to make 100 EUV tools by the end of the decade. I think when we first had our, when we had our episode earlier this year, they didn't even think they needed to make that many,

• 10:55Dylan Patel?

  enough mirrors to make 100 EUV tools a year. And, and so now they're, they've like, sort of, they're like, okay, we need to do that. But in reality, you know, because of all the economics of what's going on, it, it should be even more. But it takes so long to, to.

• 11:10Dwarkesh Patel?

  Suppose, suppose that every single company in the firm, sorry, in the stack, got private equityed. Like somebody came in who was super AGI-pilled and was like, we're going to maximize production. How fast, like, what do you think the physical constraints on making more things would be? Because the reason I ask is we're pretty soon going to be in a world where the lab revenue or just AI cash flows, because obviously the accelerators also have these huge cash flows, will be so big that you can just fund extreme expansion of all this production from cash flows themselves.

• 11:40Dylan Patel?

  Yeah, I, I do agree generally, there's obviously some physical constraints. The way the supply chain is expanding currently, the 100 is roughly still the right number.

• 11:50Dwarkesh Patel?

  For 2030?

• 11:51Dylan Patel?

  100 ASML tools for 2030, but you know, if you said, Carl Zeiss, here's $10 billion, please fucking just expand production, that would change things. And you would have to do this with every company in the supply chain. I don't think it'll happen this year. I don't think it'll happen next year. I don't have, I don't think it'll happen the year after, because the world is capital constrained.

• 12:09Dwarkesh Patel?

  But in a world where, say, the top labs are generating, let's say even combined, a trillion dollars in revenue next year. They're not able to say 10.

• 12:17Dylan Patel?

  I don't think they're going to do that, but.

• 12:19Dwarkesh Patel?

  Yeah, well, or hundreds of billions at least, right? It just seems like they realize where the world is headed. I feel like they could just make.

• 12:25Dylan Patel?

  So, so the thing is, the labs can spend hundreds of billion, they're going to generate hundreds of billions of revenue next year. But ultimately, capex next year is like $2 trillion. So you've got this big mismatch, right? You know, the, the wafer fabrication equipment supply chain will do, you know, two, you know, something on the order of $200 billion. The data center market supply chain will do even more. The accelerator supply chain will do even more. You know, the energy supply chain will do a number, you know, you, you sum all this up, it's going to be, you know, well north of $2 trillion of capex. So the labs have not yet gotten to the point where their cash flows can fund this stuff.

• 12:59Dwarkesh Patel?

  Of course, yeah, yeah, yeah. I mean, obviously they will like never get to that point, right? Because they want to keep.

• 13:03Dylan Patel?

  Yeah, you reinvest.

• 13:04Dwarkesh Patel?

  You, you want to keep, make your capex higher than your returns. But the key question I really want to understand is if, yeah, if the current continues to be like north of 50 gigawatts per lab by the end of 2028. So between them, they'd have 100 gigawatts. Those gigawatts, as you're saying, drive manyfold more throughput or more performance by 2028 than they are now, right? Because the hardware's gotten better, so not only have like flops for watt increase, but also the hardware gets better at working with AI workloads. Okay, so 100 gigawatts for the labs end of 2028. How much is like world compute?

• 13:39Dylan Patel?

  I think that may be a little difficult, given 2028, you start to have, they've taken 70, 80% of incremental compute, and I'm not sure what happens to markets then, right? You know, how much does the price of compute skyrocket for them to actually be able to buy 70, 80% of compute? Is, you know, Google or Meta or my Amazon willing to sell even that much? Also, one caveat when we're sort of talking about these gigawatt numbers is, you know, when Amazon is serving bedrock anthropic models, that counts as anthropic compute in sort of our worldview, because it is effectively at the end of the day counted as revenue for anthropic, even though like there's a revenue share and credit back and all that.

• 14:19Dylan Patel?

  But ultimately, in 2028, it's, it's, you know, if they get to 100 gigawatts combined, they have done really disruptive things to the market, because anyone can make money off of 10 to 15 million dollars per megawatt compute today. You literally, like, I kid you not, it's not that hard. Go get a GB 300 rack, go download the Kimi Weights, go download VLM or SG Lang, set it up. You know, Codex and Fable can actually help you do this. It's pretty simple. I mean, it's not like it's, you know, it's not trivial, but it's not like rocket science. And go put it on open router. It's very simple. And, and you'll start generating more revenue than you're paying for the compute.

• 14:59Dylan Patel?

  And so this is, this is sort of already led to this compute pricing 10 to 15 million dollars per megawatt start to inflect up. And to get to that 100 gigawatts in 2028, you have to believe that the labs can outpay for compute, because anyone can make money at 10 to 15. You, you know, does compute now get to $25 million a megawatt? Does it get to $40 million a megawatt?

• 15:21Dwarkesh Patel?

  But as you're saying, it's already the case that the labs are generating way more revenue per megawatt than everybody else. If they stay as far ahead as they are currently, you'd expect that to be the continuing case. If there's like some kind of recursive self-improvement where the AI labs are like relatively uplifted, where they have models internally, they're not releasing externally, they're helping them make the next model better. You'd expect that to be even more the case. And you're already seeing this, where like SpaceX or whoever's like slightly further behind will just sell compute to the highest bidder. If they can't internally monetize it as well as the labs. So I feel like it's, continue expecting them to be able to gobble up, like bid for larger and larger shares of the compute.

• 15:54Dylan Patel?

  I think that is my worldview, that they will continue to gobble up more of the compute. But ultimately, they can't do it at, at current pricing or anywhere close to it.

• 16:02Dwarkesh Patel?

  Sure, sure.

• 16:03Dylan Patel?

  They, they do have to start paying 25, 30, 50 million dollars a megawatt to really gobble up 70% of the world's compute in 2028. To get to that 100 gigawatts by 2028, which is a very sort of aggressive goal. The other aspect of this that's really challenging is we've already seen a huge slowdown for the AI labs, right? This, this regulation that they advocate for is actually slowing down the labs a lot more than it slows down, you know, sort of the open source Chinese language models. You know, OpenAI not releasing Astra, OpenAI stopping training for two weeks, Anthropic not releasing what their safety assessment said is Model 2, which is widely believed to be the next version of Methos. They're clearly not releasing their best models, and in which case their revenue per megawatt stalls

• 16:47Dylan Patel?

  or even can start to decline again because other models are competitive again. So it's not that they're falling behind, it's just that they're not releasing their best stuff. What if there is some regulatory impact that prevents them from releasing their best models? Now their revenue per megawatt does not climb as fast, then their ability to buy that incremental compute for a higher price than everyone else starts to diminish, and then maybe they can't get to that 100 gigawatts is, is sort of, in, in a world where safety doesn't matter, I, I do believe that's exactly what happens, right? They can start generating $100 million per megawatt or more, and they can pay $50 million a megawatt. And no one else has any logical reason to do anything with their compute besides say, please, Dario, take everything off of my hands.

• 17:28Dylan Patel?

  But there are, you know, forces at play that, which we cannot describe, that, that would potentially slow this down.

• 17:37Dwarkesh Patel?

  Yeah, yeah, yeah. I mean, I, I think a good intuition pump is just, what if the AI models were literally as good as a fully automated software engineer? They're not currently there yet, right? Like, I think they're far from just being able to fully automate the job of like a full white collar worker. But white collar workers earn, you know, six figures or north of that a year. And if you have a gigawatt that can sustain a population of like, say, a million of white collar workers, just let's say roughly, right? That's like, you could then off the back of that.

• 18:11Dwarkesh Patel?

  That would be 100 billion. That's actually surprisingly low.

• 18:15Dylan Patel?

  Yeah, 100K per person, million population, yeah.

• 18:17Dwarkesh Patel?

  Yeah, yeah, yeah. I don't know. But it would be many hundreds of billions of dollars if you get like full AGI per gigawatt.

• 18:23Dylan Patel?

  I think the other aspect of this is, and we've continued to see this, the most of the value capture is not happening, right? Like, most of the value that these models generate does not get given to OpenAI and Anthropic. Thankfully, so far, it is mostly just being given to the users, right? Jane Street with their exclusive contract with OpenAI for GPT-5.6 ultra-fast mode, or Jane Street where they're like, one of Anthropic's biggest customers, is generating way, way, way, way more value out of the tokens they're paying for than Anthropic is generating in terms of profit, right? Because they get to, you know, make money off of the market. Or Meta, who at one point was rumored to be as much as 10% of Anthropic's business,

• 19:06Dylan Patel?

  you know, they're generating way more efficiencies by optimizing their ad algorithms or what have you, and getting engagement time 5% longer and, you know, all these things. They're making way more money off of using these models than Anthropic. And so ultimately, you know, and that's what's required. So, sure, if you had a million new software engineers, the cost per software engineer would also fall.

• 19:29Dwarkesh Patel?

  One thing I'm confused about is, does the market come into equilibrium? And if it comes into equilibrium, would you just expect the price of compute to equal whatever Anthropic and OpenAI can generate from it? Or be very close to it, but like a small amount of markup for Anthropic and OpenAI? Like, right now, it's really weird that there is a 4X or more difference between what compute sells for and how much money Anthropic can make from it. And in a world where the revenue per gigawatt continues to increase, if Anthropic's ability to monetize a gigawatt doubles or triples or something, it'd be weird if then the gap continued to increase? And so Anthropic, just by like having some software, having some weights, can take something that costs them $10 and then turn it into $100?

• 20:12Dylan Patel?

  Yeah, so there's a bit of, this is always a fun question, right? Which is, where does the value go in AI? AI is generating all this value. You've got, you know, the end user, which I think we all agree is generating more value than anyone else, hence they're paying a lot for these models. Bot then you have, you know, the app layer. Well, so far, the app layer has generated very little value. And you've got the model layer, which, again, up until a year ago, was generating negative gross margins and is now generating massive positive gross margins. And looks like it's on the path to generating, you know, $100 million per megawatt. So turning, you know, $10, $15 into $100, as you said. But if we go back again a year ago, the hardware supply chain was generating all this gross margin while literally everyone else was losing money on it.

• 20:56Dylan Patel?

  OpenAI and Anthropic were just plowing VC money in. And as were many other startups and many of these hyperscalers are building infrastructure without knowing if there was going to be a payoff. So ultimately, you had this like, you know, negative value being created on the model layer almost, if you will, because they were selling the tokens for less than it cost them on the infra side. And all the values being created used at the chip, the fab. Initially, in 2023, the memory guys were making no money off of, you know, HBM or memory for AI, even though theoretically, the value they were delivering was humongous. Now you've got, well, actually, TSMC makes way less value than the memory guys. Is that actually how much, you know, they're capturing less value, even though...

• 21:36Dylan Patel?

  So the value capture shifted around a lot, which is very fun for people tracking the market or participating in the market, like Jane Street is an example.

• 21:46Dwarkesh Patel?

  You don't have to fuck them that hard.

• 21:52Dylan Patel?

  So, you know, what happens, you know, going forward? Does Anthropic and OpenAI, you know, they've slowly started to balloon in value capture? Do they balloon and take all the value capture? Well, that was a thought, and then Elon showed, actually, no, I can sell my compute for $25 million a megawatt or $40 million a megawatt to Anthropic and Google. Even if it's a short-term thing, I've sold it for this price, and I'll recoup my entire capex in a year.

• 22:18Dwarkesh Patel?

  So what's your prediction of how much the relevant tranche of compute, like B300s or whatever, that sold for 40B a gigawatt? SpaceX sold for 40B a gigawatt to Google. What does that sell for at the end of next year?

• 22:31Dylan Patel?

  I think most compute will still continue to transact at sub $20 billion a gigawatt.

• 22:35Dwarkesh Patel?

  Even at the end of next year?

• 22:36Dylan Patel?

  Because all of it has to be financed. For compute that you can build without financing, right? If Meta can build compute, Microsoft, Amazon, SpaceX can build compute without finding a customer, just saying, "Fuck it, I'm going to build this compute", and then turn around and wait till it's already built, they now control what's going on. So most compute is contracted well before it's built.

• 22:58Dwarkesh Patel?

  Yeah.

• 22:58Dylan Patel?

  And so this is sort of what Elon took advantage of in the market is, he actually had all this compute, and he was like, "Hey, Anthropic, I know you're making like $60-plus billion per gigawatt. Why don't you just buy my stuff for a crazy amount of money?" And obviously, you know, it's not like Elon decided this or Anthropic decided this. It's sort of, markets figured itself out. Other people, you know, you go to a random cloud, they're like, "Okay, I'm going to build a gigawatt of compute or 100 megawatts of compute. I'm going to spend the capex. I need to turn around and find a customer. If I want to find a customer, I need to find the capital. Who's going to give me the capital and the customer? The customer has to sign a deal, and then I take the customer's commitment to the credit markets, and I raise the capital." And so there's this sort of like completely different power structure

• 23:40Dylan Patel?

  where Meta, who is effectively hoarding compute, them and SpaceX are plausibly the like number three. And the only plausible number three is because they're hoarding all this compute. They're using their balance sheets and capabilities to build compute, to build compute without an end customer that's monetizing at a huge degree. And they have an actual balance sheet, so they can go to the credit market and being like, "Hey, guys, you build a gigawatt, you can make, you know, your margin. Not a crazy margin, but you can make a good margin, and I now have all this compute." And now Meta and SpaceX have this optionality of looking around and being like, "Is my internal use case going to make me more money? Or should I go out there and sell it to Anthropic, OpenAI at crazy margins?"

• 24:18Dwarkesh Patel?

  Yeah.

• 24:18Dylan Patel?

  So now we've sort of entered a regime where SpaceX and Meta are saying, "Actually, I'm going to build the compute, and I can start to rent it out for not 13. I can sell it for 25, 50, and more."

• 24:31Dwarkesh Patel?

  So as I've been doing video essays and other formats, I've been looking to hire a new editor for the podcast. But actively searching for editors has been quite time-consuming because the vast majority of candidates don't fit the profile that I'm looking for. So I created a recruiter in GrokBot to see if it would help. I gave it a huge context dump where I monologued basically everything that I wanted, and then it spun up four other bots to narrow in on different parts of the search. One went through the last year of my email for relevant inbound. One searched my X feed and DMs. One went through the end credits on various documentaries I like. And the last one looked for editors who work for some of the YouTubers that I follow. GrokBot then took all these different candidates that these subagents had found. It filtered them against my criteria and then delivered for me a final shortlist to review.

• 25:14Dwarkesh Patel?

  To be honest, the first batch had a few good candidates I wanted to see, but mainly a bunch of duds. But after I gave GrokBot some more feedback about what it was missing, it came back with a new list of candidates that I'm actually extremely excited about. I ended up saving this whole workflow as a routine. So every week now, GrokBot checks my inbound email and X DMs for promising new candidates to potentially interview. If you want to try GrokBot yourself, go to x.ai/bot. What do you think their revenue per gigawatt is by the end of 2027? Like, for Anthropic or OpenAI by the end of 2027?

• 25:44Dylan Patel?

  I think it's highly dependent on who has the best model, if they're allowed to keep releasing their best models. But I don't see why it wouldn't be 50-plus million dollars a megawatt.

• 25:52Dwarkesh Patel?

  By the end of 2027?

• 25:53Dylan Patel?

  Oh, by the end of 2027?

• 25:54Dwarkesh Patel?

  Yeah.

• 25:55Dylan Patel?

  That's where it gets more challenging, but I think it could get to, you know, higher than that. It's like 70, 80 million dollars a megawatt blended across the company, if not higher.

• 26:03Dwarkesh Patel?

  Yeah.

• 26:03Dylan Patel?

  Yeah. And so I think, if that's the case, right? Then what happens to the price of compute? Well, if I'm Anthropic, incremental compute is worth it. Maybe I spend $40 million a megawatt on SpaceX compute.

• 26:16Dylan Patel?

  And if I'm SpaceX, you know, I look to the supply chain, I'm like, "Well, you know, I've struck this deal with Jensen where he's now all of a sudden using Twitter."

• 26:25Dylan Patel?

  And, you know, Elon's saying they're exclusive to Nvidia, but why doesn't Jensen raise his prices? And then, you know, SK Hynix and Micron and Samsung looked at Nvidia and are like, "Well, why don't they raise their prices?" So I think the value capture, there's a bullwhip effect here, right? Where just because someone has risen the prices doesn't mean the entire supply chain rebalances immediately.

• 26:45Dwarkesh Patel?

  Yeah.

• 26:45Dylan Patel?

  But over time, the supply chain will rebalance, and things will cost more and more. And, you know, to get that incremental capacity, you sort of have to, right? So TSMC raising prices very slowly, but memory companies raising prices very quickly. You know, substrate companies raising prices very quickly. Different parts of the supply chain raise, you know, Elon wouldn't have sold if it was 15. But he's selling because it's 25-plus, so obviously, he rose his prices really quickly.

• 27:08Dwarkesh Patel?

  Yeah. I'm sort of surprised you think like revenue per gigawatt doesn't increase way more than even like 100 per gigawatt by the end of next year.

• 27:15Dylan Patel?

  When does RSI happen? When does it take off, right? I think...

• 27:18Dwarkesh Patel?

  Or even if RSI doesn't happen, the current rate of progress continues? If you just look at how much progress have you made in, let's say, the last... Year and a half, like what was the model from a year and a half ago?

• 27:26Dylan Patel?

  I mean, my problem with this is the best model that exists in the world was trained in February.

• 27:30Dwarkesh Patel?

  Okay, you're saying maybe we just won't be able to release the best model?

• 27:33Dylan Patel?

  OpenAI says they're not training models for two weeks, man, what the hell?

• 27:35Dwarkesh Patel?

  Yeah, I mean, there's another, there's one thing like internally, are they getting enough use for it? Especially like bit of the price of computer or another is like, does AI progress as a whole slowdown because of regulation?

• 27:43Dylan Patel?

  Yeah, but they're not even allowed to use this like new model and like Astro is not widely deployed internally even.

• 27:49Dwarkesh Patel?

  But still, I don't know, just like, if you have like a model that is, what was the model released? Like, let's say the beginning of last year, like GPT.

• 27:58Dylan Patel?

  4.0, is that 4.0?

• 28:00Dwarkesh Patel?

  Yeah, that's like you're talking about a 4.0 to Fable size or Mythos 2 size leap by this point. Again, by the end of 2027.

• 28:08Dylan Patel?

  Yeah, but Mythos 2 is not out.

• 28:09Dwarkesh Patel?

  Yeah, or like even Mythos, right? Like that leap again.

• 28:12Dylan Patel?

  Even Mythos is not allowed to be out, right? They've neutered it.

• 28:15Dwarkesh Patel?

  Yeah, yeah, yeah.

• 28:15Dylan Patel?

  Like, we can't, we can't use it to optimize inference performance, or you can't use it to optimize all sorts of things.

• 28:20Dwarkesh Patel?

  Right, yeah, maybe there's like some slowdown in AI progress or the deployment of AI. That means that the revenue per gigawatt can be lower, but that's the only way I could see it being only 100 per megawatt by the end of next year.

• 28:31Dylan Patel?

  Yeah, I mean, as long as the model gets better, the value generated out of it gets better, obviously. Who captures the value is still up for debate, but ultimately, everyone's going to raise their prices.

• 28:40Dwarkesh Patel?

  Yeah, yeah.

• 28:41Dylan Patel?

  Because they can, and it's super inflationary, especially if the method of regulation is right now, so far, it's just don't release the models. But more and more, the method of regulation is New York's banning data centers, Texas is holding memoratoriums. Ohio is saying you have to, or at least trying to say you have to like pay everyone's property tax in a certain radius. These sorts of things are going to decrease supply and increase cost, and that's going to get passed on as well. So you start to end up in a spot where progress does slow, at least in the external sense.

• 29:10Dwarkesh Patel?

  Yeah.

• 29:11Dylan Patel?

  Even if the models internally keep getting better and better, I see no reason why like, you know, again, like in a takeoff scenario. Why would Anthropic not have their best model six months ahead of what is externally available because of safety and regulation? But also, you know, the competitive advantage. And then that six month difference, if progress accelerates, is actually a bigger differential. So that's the thing that would cap revenue per megawatt gains to a much lower growth than we've seen in the first half this year.

• 29:41Dwarkesh Patel?

  Here's something I'm very interested in, as these companies go public and they're accountable to investors. And they, let's say, end of next year, they have, I don't know, close to 20 gigawatts, so like 10% of their compute, 2 gigawatts. Let's say they want to go from 60% compute to training, to 70% compute to training. And their investors are like, well, if you're able to generate $100 billion per gigawatt, you're basically saying no to like $200 billion of revenue in order to increase your training compute. As investors are like, what the fuck? You're already spending so much on training, why are you spending even more on training? As a public company, do you think, yeah, what do you think would happen if they're just like, no, we will keep increasing

• 30:22Dwarkesh Patel?

  the share of compute we spent on training to offset the increase in revenue that each gigawatt of compute is giving us?

• 30:30Dylan Patel?

  Yeah, so this is sort of what I personally believe that the labs are going to allocate less and less compute to inference over time, which I think is very non-consensus, right? Everyone's sort of, the standard belief of most people is, oh, most compute will go to inference. Most of it will go to forward passes for training, not maybe necessarily revenue generating inference. But ultimately, you end up with, if they're generating, you know, $30, $40 million per megawatt today, you allocate 40% to inference. If you now get to generating $60, $70 million per megawatt, do you still allocate 40% to inference and generate all this profit? And then do dividends and share buybacks? Or do you go build AGI?

• 31:11Dwarkesh Patel?

  Yeah, yeah.

• 31:12Dylan Patel?

  And I think the obvious answer from Anthropic and OpenAI, and not just at the executive level, but also their board, is go build AGI because it's way more profitable.

• 31:22Dwarkesh Patel?

  Right.

• 31:23Dylan Patel?

  And so ultimately, you're going to see them ratchet up their percentage of compute dedicated to training.

• 31:31Dwarkesh Patel?

  While each increment of compute is getting more and more profit generating if they had dedicated to inference.

• 31:37Dylan Patel?

  Right, and so the whole point is, well, okay, if I'm selling tokens, is Anthrop, is OpenAI releasing ultra-fast mode for just external? Or are they doing it internally too? And it turns out, no, actually, I'm going to allocate it to internal and external. Because my internal, you know, value that I'm generating from super fast AI or the best AI model is way more than what someone externally is.

• 32:00Dwarkesh Patel?

  Yeah, yeah.

• 32:01Dylan Patel?

  And so ultimately, sure, I could generate $100 million per megawatt, but if I turn that towards AI research, what is the incremental progress that I get? And then what does that do towards my future earnings potential, the discounted cash flows of whatever the hell I've done, right? And so, you know, they're not going through that calculation, but ultimately, it's, it makes more sense to dedicate more and more compute internally. And the only reason to, you know, have inference compute be so large is so you can grow your training fleet.

• 32:27Dwarkesh Patel?

  Right, right, right. I think this is an interesting economics question that I feel like we can have the models digest of. What is, what would have to be true about a world where they reduce fraction of compute spent on inference?

• 32:37Dylan Patel?

  I think they have been over the last three months already.

• 32:39Dwarkesh Patel?

  Interesting.

• 32:40Dylan Patel?

  I think, I think at parts of this year, they were increasing fraction of compute. So let's just take month by month. You would agree that every month Anthropic has added more compute than the prior month. There might be some noise when they like sign a SpaceX deal or whatever, but in general, the amount of compute is a curve up. And so in January, they added less compute than December. And yet, their revenue adds, you know, skyrocketed, and then they've sort of plateaued. They're only adding, you know, they're not adding $25 billion of ARR every month now. And so that means the marginal megawatt they're getting is going higher percentage to R and D than is inference.

• 33:15Dwarkesh Patel?

  Interesting, yeah.

• 33:15Dylan Patel?

  And so they are factually increasing their compute towards R and D today.

• 33:19Dwarkesh Patel?

  Yeah, yeah, yeah.

• 33:20Dylan Patel?

  Yeah, I think this is like self-evident, if you like look at what they're doing enough.

• 33:26Dwarkesh Patel?

  Yeah, so if I look at the numbers you said of like how fast world compute grows, here's some things I want to understand. So it seems like if I added the numbers you just said, it would be over 200 gigawatts of world compute by the end of 2028, right?

• 33:38Dylan Patel?

  Yeah, globally.

• 33:39Dwarkesh Patel?

  Okay, and how fast can that continue growing? Like global, global AI compute after 2028?

• 33:45Dylan Patel?

  Yeah, so 30 this year, 50 next year, 70 in 28. 29 should be like on the order of 90 to 100.

• 33:53Dwarkesh Patel?

  Like then just 100 and more every single year or something?

• 33:55Dylan Patel?

  I think, I think the slope can continue to go upwards, and it's hard to predict anything more than four years out, given, who knows what's, you know, are we in RSI regime or, you know, when is the world economy growing at 10% a year? Because if you're at 100 plus gigawatts a year, you're at absurd revenue GDP growth.

• 34:14Dwarkesh Patel?

  Right, if you think there's 200 gigawatts globally in 2028, how much is in China by that point? And how does like, yeah, how does Chinese compute continue increasing through this whole trend? Because if, if the RSI stuff kicks off in the West before China has a large amount of compute, maybe we're living in a different world than when it doesn't.

• 34:34Dylan Patel?

  Yeah, so China today, so if we, if we sort of level set back to 2022, the US was adding about 45 to 50% of the world's compute. China was adding about 30 to 35% of the world's compute, and the rest being taken up by the rest of the world. Since 2022, we've had big regulations against China and a dramatic increase in America. So today, 70% of watts are being deployed in America. And, you know, China is really a very small number. It's sub 10% of watts being deployed for data center AI compute is in China. And as we step forward, they're still at a very small number.

• 35:16Dylan Patel?

  Their domestic production is quite small, their purchasing from Nvidia is still quite small. And a lot of that ends up in other places as well, right? You know, Malaysia or what have you. So ultimately, China domestically still continues to have 10, sub 10% of incremental new compute. So in 2028, it might start to inflect up, I think. But it's pretty easy to say China will have like 30 gigawatts of AI compute or less.

• 35:41Dwarkesh Patel?

  By 2028?

• 35:42Dylan Patel?

  Yeah, in 2028.

• 35:43Dwarkesh Patel?

  Okay, and then how fast does their hockey stick go up?

• 35:46Dylan Patel?

  I do think in 2028, they have a big uplift in what compute they're able to deploy. 2026, they're still mostly relying on a lot of the smuggled chips. You know, a lot of the chips that TSMC made for companies that they thought weren't Huawei, but ended up being Huawei. Or a lot of HBM that Samsung is shipping, you know, sort of, but in 27, fabs start to go up, and 28 especially, fabs start to go up from SMIC and CXMT and such, where domestic production is actually reaching many millions of units a year. And now they're incrementally adding, you know, 5, 10 gigawatts in just 2028 of domestically produced chips.

• 36:23Dwarkesh Patel?

  Right.

• 36:23Dylan Patel?

  Those chips are definitely worse than the chips that Nvidia will have in 28, or Google will have in 28, or OpenAI will have in 2028.

• 36:32Dwarkesh Patel?

  So even, even the gigawatt number overstates things you're saying. It's like 30 gigawatts, but it's really much worse chips. Bot then how, yeah, how does it, if you think the world is going to add 100 gigawatts in the following year? Or some, you know, I know you said you can't really say that far out. How much is China able to add the subsequent year? Basically, I want to know, did they just hockey stick at the point at which they are able to start shipping large amounts of compute? Or is it still going to be less than US plus allies?

• 37:01Dylan Patel?

  There's a lot left to, you know, whether or not the US passes the MATCH Act, whether or not X tools continue to get export controlled, how fast China can build their new equipment that they're starting to be able to produce domestically. But ultimately, you know, China, China is definitely going to hockey stick. If there's anything, China's really good at is scaling manufacturing really, really quickly. And, you know, I imagine, you know, China's, China will start to be able to extract more and more purchasing of even foreign ships into, into domestic China, or at least close the gap in what the US is allowing, you know, Nvidia to sell them or what have you.

• 37:41Dwarkesh Patel?

  But do you think China could do adding 50 gigawatts by 2029? Marginal incremental gigawatts in 2029?

• 37:48Dylan Patel?

  I think that's, I think that's completely reasonable.

• 37:50Dwarkesh Patel?

  Yeah.

• 37:52Dylan Patel?

  And part of that could also be purchased from foreign.

• 37:54Dwarkesh Patel?

  Yeah.

• 37:56Dylan Patel?

  But yeah, I think it's completely reasonable that China in 2029 can do 50 gigs. But if most of those are domestic chips, there is some factor there where that 50 gigawatts is really worth as much as 20 gigawatts in America. Or from, from American chips.

• 38:10Dwarkesh Patel?

  Right, right, right. So yeah, you're actually projecting a world where maybe the leading lab in 2028 has more compute than China will have in like. All of China will have in 29 or even 30. If you weighed gigawatts by their quality.

• 38:24Dylan Patel?

  Implying that there's nothing done to slow down the US labs.

• 38:28Dwarkesh Patel?

  Yeah, that's right.

• 38:28Dylan Patel?

  Clearly, the government is starting and politicians are starting to do that.

• 38:32Dwarkesh Patel?

  Yeah, yeah, yeah.

• 38:32Dylan Patel?

  Whereas China is not going to slow down AI. In fact, the only thing they're going to do is accelerate it.

• 38:36Dwarkesh Patel?

  So honestly, I, when I interviewed Jensen and asked about export controls, I am a libertarian person. I'm like, I wasn't genuinely sure what I thought about this issue. I was steel manning what is like the opposite view that he has. And because I think it's important to hash out ideas. But I'm like, yeah, maybe there's a world where if you just cooperated with China, it would be better for us, especially since they control so much of the supply chain and the other things that will be needed for robotics and other things. But I didn't realize the compute situation was as fucked as you're saying. Like actually, the export controls do seem to have like really, if they ship the amount that you're saying, that's a huge difference. By the time we have automated coder and even getting into like automated researcher, China is like way far behind on the compute stock.

• 39:19Dwarkesh Patel?

  And so if that ends up being the case, that would have worked. I think that's actually a notable success.

• 39:24Dylan Patel?

  I would say the only caveat there is some of it is export controls. But some of it is also just financial systems, right? American financial systems are more willing to YOLO into startups than Chinese financial systems. But once Chinese financial systems choose an industry to focus on, they'll subsidize it a hell of a lot more.

• 39:45Dwarkesh Patel?

  Yeah.

• 39:46Dylan Patel?

  And so the Chinese semiconductor industry has significantly more subsidies than the rest of the world's semiconductor industries combined, which, which points to like, you know, if takeoff is not as fast as sort of you're implying, but actually it takes longer, then ultimately China will catch up drastically on the semiconductor side, which then is compute at some point. The other aspect of this that I would think is like, I think is like noteworthy is Chinese companies today are not that far behind in AI models, at least perceivably by the public, relative to the amount of compute they have, right? The leading Chinese labs have 100, 200 megawatts total of compute at most. ByteDance Seed being the one outlier where they have significantly more than that.

• 40:29Dylan Patel?

  But you know, Kimi is not running, you know, a gigawatt or anywhere close to it.

• 40:34Dwarkesh Patel?

  Yeah.

• 40:35Dylan Patel?

  Whereas Anthropic is, you know, nearly five gigawatts by the end of the year, right? Or more, sorry. And so, you know, the question is sort of, well, does it matter? And I think right now it doesn't matter that much, this difference in compute. Because, you know, when we break down the compute ratio or budget of a lab, historically, it's been, you know, let's say 60, or so far, it's been like 60% training, 40% inference, but that training gets broken down further. And that is actually like 50% of the compute is research, like 10% of the compute is development, and then 40% is inference. And what I mean by research and development is, you know, researchers are generating ideas, testing new architectures, testing new data mixes, testing new hyperparameters, whatever it is they're doing,

• 41:20Dylan Patel?

  new attention techniques, blah, blah, blah. But ultimately, when they do the training run, when Anthropic trains Methos, it's sub 200 megawatts, right?

• 41:29Dwarkesh Patel?

  The pre-training, the whole thing?

• 41:29Dylan Patel?

  The pre-training. It's sub 200 megawatts for, call it, two months. And then the RL is even less.

• 41:36Dwarkesh Patel?

  But you think the RL was less to compute than the pre-training?

• 41:38Dylan Patel?

  At least in terms of single-site inference, I mean, single-site of pre-training, yeah.

• 41:43Dwarkesh Patel?

  But total compute was probably higher, right?

• 41:45Dylan Patel?

  Total compute, but it's like sequential, right?

• 41:47Dwarkesh Patel?

  Yeah.

• 41:47Dylan Patel?

  So at most, the most they ever used at one point in time was maybe 200 megawatts. And then in reality, they had multiple gigawatts. So most of their compute was going to the research, not the development of a model. And there's reasons for this, right? You can't, it's hard to coordinate all these clusters. It's hard to co-locate all of them. It's hard to do multi-site training. It's hard to do RL. You know, generating even more rollouts during RL does not necessarily make it better. There's all sorts of reasons why you may not be able to leverage, you know, all two gigawatts that you have for training onto training, right? Actually, I can only leverage 200 megawatts. As we get closer, as we get further and further down, implement automated coding, automated researcher,

• 42:30Dylan Patel?

  I actually expect the percentage of the compute budget that goes to research versus training to start to, and to start to like, become a lot more fuzzy or even higher for training. Also, things like continual learning, right? All of these things start to mean that more and more is actually going to training the model.

• 42:48Dwarkesh Patel?

  All right, so if you have, if you end up in a world where you're doing 100 gigawatts a year, at current prices, that would be 5 trillion of capex every single year.

• 42:57Dylan Patel?

  And then stack on the fact that you have to build the power plants way before then, slash, it's a 30-year asset. You stack on the fact that the data centers are, you know, 15, 20-year asset, and you have to build that then too. So the 5 trillion, you know, you have to account for future years' growth. It's actually going to be more like 7 or 10 trillion of capex.

• 43:14Dwarkesh Patel?

  I didn't understand, because you're not including the fact that like, that this include the fact that there's not the infrastructure for the power generation or whatever in the data center itself.

• 43:24Dylan Patel?

  Right, exactly.

• 43:25Dwarkesh Patel?

  Yeah, yeah.

• 43:25Dylan Patel?

  And the data center itself is, when you talk about AI capex, people are saying 40, 50 billion dollars. But that's really just the critical IT.

• 43:35Dwarkesh Patel?

  Yeah.

• 43:36Dylan Patel?

  Right? The servers, the networking, the fiber, the transceivers, optical communications, all this sort of stuff. It doesn't account for the data center itself or the power plants themselves, which are being built ahead of time.

• 43:48Dwarkesh Patel?

  Yeah, yeah, yeah.

• 43:50Dylan Patel?

  If I'm building 100 gigawatts this year and 150 gigawatts next year, well, then all of the buildings for that 150 gigawatts need to be built in capex this year. And if I'm building 200 gigawatts the year after that, all those power plants need to be spent, you have to buy the turbines this year.

• 44:03Dwarkesh Patel?

  Yeah, yeah, yeah.

• 44:04Dylan Patel?

  Right? And so you've got this like, this like, actually, it's much bigger than even 5 trillion dollars if you're building 100 gigawatts.

• 44:10Dwarkesh Patel?

  Right, so very plausibly incremental capex every year is getting close to 10 trillion dollars.

• 44:20Dylan Patel?

  By the end of the decade.

• 44:21Dwarkesh Patel?

  Right, which is going to be like, close to a 10th of the world economy. And like a third of, if all of it's going up in the US, it's like, well, the US economy will have grown as well, but still, and the current size of the US economy will be like a third to a quarter of the US economy, would just be going towards data centers. And as I say that out loud, I'm like, maybe, maybe you're right, and we just won't allow it. And that's the reason this doesn't happen, right? Because like, for this exponential continue, just like a quarter of the world, a quarter of America's economy is just building data centers.

• 44:48Dylan Patel?

  Yeah, I mean, I believe in capitalism and reallocation of resources towards the most profitable thing, but at the same time, politics exist.

• 44:55Dwarkesh Patel?

  Yeah, yeah.

• 44:56Dylan Patel?

  And credit markets exist and capital markets exist. So to enable, let's say, that 100 gigawatts by 2030, or let's even like, let's even like pare it down to 2028, where it's like 3 or 4 trillion dollars of capex across all of these items. You know, a couple, you know, over, you know, 2.5 towards compute, IT capex, and then another 1 to 2 on data center and energy, and all this supply chain downstream, like semiconductors and all that stuff. So if you're at, if you're at 3 or 4 trillion dollars of capex, where does all this cash come from? No one is generating that much cash from the business yet, right? Hyperscalers, they funded all of the growth up until now, Google, Microsoft, Amazon, Meta. They funded a huge percentage of it. They were more than half of compute.

• 45:40Dylan Patel?

  But they now don't generate cash. They actually spend everything on capex. And in addition, they raised debt and spent everything on capex, right? You've seen Meta do it, even Amazon, even Google. You know, Microsoft will be there soon. Everyone is raising debt to pay for their capex. So now, who is the incremental person to pay for this? That was not doing it before. In the case of like Google, it was pretty simple for them to stop doing buybacks, or Meta stopped doing buybacks, and turn around and buy computer infrastructure. And that doesn't have a huge effect on the market, but it does have some effect. But as you, as you step forward to 2028, where the hyperscalers are now raising hundreds of billions of dollars of debt, and then all of their supply chain is raising hundreds of billions of dollars of debt, who pays for this?

• 46:25Dylan Patel?

  And so there's a few different ways. You know, there's the semiconductor companies like Nvidia and Broadcom and the memory companies turning around and deciding to fund some of this capex. There's the traditional infrastructure investors who are turning around and gathering capital and investing in infrastructure, and instead of bridges, it's data centers. And then lastly, there's everyone in the economy who's realizing, maybe I shouldn't buy a home, or maybe I shouldn't invest in credit for a home that's helping people buy homes, or maybe I shouldn't buy government debt. I should just buy hyperscaler debt, or I should just buy this data center's debt, or I should buy Anthropic's debt, because Anthropic's willing to pay 20% you know, rates for, you know,

• 47:09Dylan Patel?

  the incremental billion dollars to build their capacity, because they know their revenue from it's going to be huge. And they're going to pay 20% because it's still better than renting it from SpaceX for 50 billion dollars a gigawatt. So you've got all of this contention, but if you now do this, the whole world economy is like really shifted around.

• 47:26Dwarkesh Patel?

  Antithesis is a deterministic software testing platform that enables perfect reproducibility. It also unlocks some pretty insane approaches to debugging, like time travel. With Antithesis, you can jump to any point in the trajectory and start from there. So when there's a crash, you can rewind to the exact moment that something went wrong and freeze the entire system. The application, the database, even the environment itself. This lets you do something that would otherwise be impossible, which is to observe every part of a distributed system at the exact same instant. Time travel also allows you to add telemetry and logging to an event that has already happened. For example, you can rewind to 5 seconds before a crash and decide to capture all the network traffic.

• 48:09Dwarkesh Patel?

  Most powerfully, Antithesis gives you a live terminal into your system that you can use to perturb anything you wish. Kill a node or disable a feature, and then hit play and see what happens. Then go back and try something else. In production, you often only get one shot on goal with this sort of destructive analysis. If you restart a deadlock service, for example, the exact deadlock you needed to study disappears. But with Antithesis, the original timeline is always reproducible, so you can test as many hypotheses as you need. And if you don't want to do all this time traveling yourself, you can just have your agents do it for you via the Antithesis API. Go to antithesis.com/thewarcash to learn more.

• 48:48Dwarkesh Patel?

  So you and I have been debating off air for the last few days, whether there will be a sovereign debt crisis as a result of AI. And the logic is this. AI is, you have a situation where, as we were mentioning, very little investment turns into a lot of money, right? So the rate of return-

• 49:07Dylan Patel?

  What a fucking problem, dude. Oh, my God. Can't believe it.

• 49:13Dwarkesh Patel?

  No, it is a huge problem for everybody else who can't turn a little money into a lot of money, right? So the rate of return is incredibly high. Even at the data center level, you know, if you build, if you like, build a data center and you're like, get rented out to Anthropic and Urban AI for like, 10x what it costs you on a depreciated basis to build it. It's fucking crazy. And so you turn $1 into like $2 or $10 or something at the end of the year. That raises the rate of interest higher. Now, if the rate of interest goes higher, and if it does that for the entire economy, and people are borrowing more and more money, they're competing against the other lending that the government would have done, or that other companies would have done, or that you as a consumer or a mortgage buyer would have done,

• 49:56Dwarkesh Patel?

  then that's just making it basically more expensive for everybody else to borrow. This has huge implications for tons and tons of people. Sorry, I'm gonna go on a bit of a monologue here. But we've been thinking about this together. So I think the US will be fine at the end of the day because they can, if the data centers are built in America, you can fundamentally just like tax the data centers. But the way the current tax system is set up, you know, corporate income is like less than 10% of federal revenues, and 80% plus is payroll taxes and income taxes, which, as more and more automation happens, will shrink. At the same time, on the spending side, currently 20% of tax revenue spending

• 50:39Dwarkesh Patel?

  goes towards paying, servicing the debt, basically, paying interest payments on the debt. Now, a lot of the debt is short duration, so it refurbishes. Every five years it rolls over. Why are you fucking laughing?

• 50:51Dylan Patel?

  Because, you know, it's like things we've learned, you've learned in the last month.

• 50:57Dwarkesh Patel?

  Yeah, like it's any different for you? Like you got a degree in fucking financial economics?

• 51:01Dylan Patel?

  I did. I did. The internet thinks I'm a beekeeper. Few months, few months, few months.

• 51:10Dwarkesh Patel?

  This is our business, Dylan.

• 51:12Dylan Patel?

  I know, I know, sorry, sorry.

• 51:16Dwarkesh Patel?

  And so- Now I'm self-conscious, fuck.

• 51:21Dylan Patel?

  No, it's good, you're doing good, I just think it's funny. Million people, listen to this guy who just learned about debt this month.

• 51:31Dwarkesh Patel?

  So you go from 20% of, suppose interest rates rise 1%. Then the, over a five-year basis, the amount of, the fraction of tax revenue that goes towards servicing the debt, basically, goes from 20% to 25%. If it rises five percentages, that would go towards like north of 40%. But if you take into account the fact that the government is borrowing $2 trillion every single year, then that goes from like 40% to like north of 60%. So 60% of tax revenue, basically, just goes towards paying interest payments on the debt. No, I think the US is gonna be fine because also the tax base will increase if we let data centers get built in America. Other countries are absolutely fucked, in my opinion. I was just looking at which countries have a lot of debt, have very little tax revenue,

• 52:15Dwarkesh Patel?

  and also a lot of their debt is serviced quite often. And those countries, like Pakistan or Nigeria or something, I think are just gonna be very fucked in this new interest rate regime.

• 52:24Dylan Patel?

  So this, this crowding out effect is actually like, the thing that I've like, is the reason it's not like YOLO 1 billion gigawatts.

• 52:32Dwarkesh Patel?

  Yeah, yeah.

• 52:32Dylan Patel?

  Right? You've got, you've got all these industries and countries that use a lot of debt. Whether it's, you know, all these impoverished countries that you mentioned earlier that are just gonna default. You've got like consumer packaged goods, right? Like, all of these like companies that make things you see at Trader Joe's or wherever, use a lot of debt. All these telecom companies use a lot of debt. And banks use a lot of debt. And so if interest rates go up, in the market, not necessarily the government's set interest rate, but in the market, the spread of interest rate between what the government says their federal rate is versus what everyone else is charging because Amazon wants to raise $100 billion of debt next year, or whatever the hell the number is. You know, probably less, but

• 53:17Dylan Patel?

  you end up with this like really challenging problem of, where does the cash come from? There is some level that is funded by cash flows, and cash flows keep going up. But the logical thing to do is to invest way more than your cash flows because then the returns in the future years will be amazing. So you have this delta. And then what's pushing down on the delta is all of these other things, right? There's regulations against data centers, regulations, you know, consumers getting mad, politicians getting mad, regulations against AI. The AI labs not releasing their latest models because of safety reasons. All of these things, and, and interest rates going up are an influence on all of these things. So all of these things bend the curve from what does capitalism want in terms of just pure, simple economics,

• 54:01Dylan Patel?

  to what is the complex system that we have want, and bends it lower and lower and lower, to where not as many gigawatts as should be built, will be built.

• 54:09Dwarkesh Patel?

  Well, the interest rate is part of capitalism, right?

• 54:11Dylan Patel?

  Yeah, but like, you know, like in the simple economic model versus like the more complex what we have.

• 54:15Dwarkesh Patel?

  Yeah. What is the rate at which you think Amazon or Anthropic or whatever will be releasing bonds for debt next year? They do hundreds of billions of dollars of debt. What is the rate, what is the average rate?

• 54:26Dylan Patel?

  I, I don't think Amazon will do hundreds of billions of dollars of debt.

• 54:30Dwarkesh Patel?

  The total, let's say the big tech-

• 54:31Dylan Patel?

  The hyperscalers in total will raise, and, and all the clouds.

• 54:34Dwarkesh Patel?

  Yeah, yeah.

• 54:35Dylan Patel?

  In, in the modeling that we do, we have about $11 trillion of CapEx from 2024 to 2029.

• 54:42Dwarkesh Patel?

  Total.

• 54:43Dylan Patel?

  Total. And if you do, you know, if you fund a lot of this with cash flows and as much as you can, you still end up with north of $5 trillion of credit that needs to be issued for this $11 trillion plus buildup.

• 54:54Dwarkesh Patel?

  You don't think the AI revenue continues even 3x year over year?

• 54:57Dylan Patel?

  AI revenue does go up. I don't think it can go up forever. I don't, you know, like, just like, without like certain constraints being hit, I think certain labs will have certain incentives. And, and labs are not the ones building all the compute in many cases, even though they're increasingly trying to go that way.

• 55:13Dwarkesh Patel?

  They'll have all these cash, like if their revenue keeps increasing, whatever, that's fine. But if, how much did you say their revenue will be? You think they'll not have that much revenue?

• 55:21Dylan Patel?

  No, I'm just saying till 2029, there's, you know, something on the order of $11 trillion of CapEx. And six of that is funded with cash. And five of that is funded with debt. And if that's the case, $5 trillion of debt being raised across the whole ecosystem does make interest rates go up. And then what prevents that? You know, there's a couple of things. One, do labs increase their revenue per megawatt more and keep inference allocations large? In which case, they're taking all this profit, they're accumulating all the profit across the S&P 500 because everyone's paying to, you know, reduce their costs. Of course, their profits will also go up, but, you know, cash has to come from somewhere. So there's an upper limit on how fast their revenue can grow versus the value they deliver into the world. And there's a diffusion aspect of the technology.

• 56:06Dylan Patel?

  But ultimately, labs' revenue keep going up. They can't cash flow fund everything. The optimal scenario is you actually use credit as much as you can to fund because even if cash flows from the labs fund a lot of stuff, you want to build more than that. And so there is some amount of credit that gets built. Our current modeling has $5 trillion of credit and $6 trillion of cash-funded infrastructure investments through 29. And when you take that, you, you've sort of got, you know, this is not enough compute relative to what this demand growth is from the AI models. And so you've got the obvious answer, which is revenue per megawatt keeps going up.

• 56:42Dwarkesh Patel?

  Yeah, that, that makes sense. So how much do you think interest rates will increase by 2029 as a result of all this?

• 56:48Dylan Patel?

  Dude, you know, I was just vibing a number. But if you're vibing a number out, you know, growth in the world and economy is growing up a lot, so why wouldn't interest rates

• 56:57Dwarkesh Patel?

  Yep.

• 56:57Dylan Patel?

  for Amazon go from, you know, from where they are today? I think Meta pay, okay, let's like, so this is gonna be extremely vibed out, but recently Meta's raised at like 5% to 6%. I don't see why they wouldn't pay 8%. Because they would happily pay 8% because the return from the compute that they're gonna build is humongous.

• 57:18Dwarkesh Patel?

  Right.

• 57:18Dylan Patel?

  And the market won't want them to, but if they, they'll want to pay 8%. The flip side is, if they pay 8% versus the 5 they do, 5.5, 6 they do today, you know, 2.50 bips increase, that makes everyone else in the economy also pay 2.50 bips more.

• 57:34Dwarkesh Patel?

  Yeah, yeah.

• 57:35Dylan Patel?

  Which then causes a lot of things, right? Banks will scream because if their credit spread goes up, then their assets don't, their, their, their debt themselves reprice is faster than their assets reprice. And you ultimately end up with, they're losing tons of money if their credit spread blows up.

• 57:55Dwarkesh Patel?

  The other consequences of this are, this is a point you made, but the, if interest rates rise, the discount rate increases, which means that the discount cash flows of all equities crater, which means that even though the stock market as a whole might be doing fine, like S&P 500 will be fine, any individual stock will probably have just like cratered in value, especially the Buffett, like Berkshire type, you know, pay good cash flows for 30-year type stocks.

• 58:23Dylan Patel?

  Yeah, it's like, it's like, why would I pay this much for, you know, you know, Johnson & Johnson?

• 58:28Dwarkesh Patel?

  Right.

• 58:29Dylan Patel?

  You know, like, they're, they're seen as a stable stock, good cash flows, they'll return their cash flows over time. Or a railway company, like, why the fuck would I invest that much if my discount rate isn't 3% or 5%? It's now 8% or 10%.

• 58:44Dwarkesh Patel?

  And, and for developing countries, what's, Basil Hopperen, who's a good friend and he's an economist, he made this point that we'll see a second Volcker shock. So in the 80s, to fight inflation, Fed Chair Paul Volcker raised interest rates like more than 5%. Or it's like something like 8%. Real interest rates, 8%. And that caused some 40 different countries, mostly Latin America, to default in that decade. And I think that will probably happen again. In fact, okay, now we're getting into like singularity talk. So we've been talking about what happens if interest rates-

• 59:16Dylan Patel?

  I think this all happens before singularity, by the way.

• 59:17Dwarkesh Patel?

  Yeah, that's what I'm saying, that's what I'm saying. So we were talking about like, you know, before singularity, interest rates rise 2%, 3%, etc. At some point, I think it's very likely that the world economy will be doubling every single year.

• 59:31Dwarkesh Patel?

  Okay, this is not happening in five years, but it'll happen eventually. It's just like, there will be, there's this, there's researcher David Binder who's done great work on this. But basically, if you look at like input-output tables in a fully automated economy, just like, what would it take to like double the entire stock of things in the economy?

• 59:50Dylan Patel?

  Yeah, if economy grows at 3% a year, then it's like, you know, rule of 70, it's like 20 something years.

• 59:56Dwarkesh Patel?

  Right, but then, but he was like, okay, well, right now we're bottlenecked by the fact that there's people, and you can't like double people every single year. But in a world where like, you can also double labor force every single year. How fast can the economy grow? And I think it could double every single year, or at the very least, it'd be like 10s of percent every single year. Okay, the rate of interest should be pretty close to the growth rate. It won't be exactly that because of consumption, but it should, it should be pretty similar. So then we'll go into a world, I think in the 2030s, where the rate of interest is like 10s of percent. And like, I don't know, part of my brain is like, it might be hundreds of percent. But like, okay, let's say it's at least 10s of percent. I'm just like, okay, every country that is not involved in the production of AI defaults. Every stock that is not an AI stock is like worth basically zero, because discounted cash flows are worth nothing.

• 1:00:38Dwarkesh Patel?

  If the federal government can't figure out a way to tax AI, you know, the servicing the debt is more than the current tax revenue. All these other effects that I'm sure we're not even pricing in, like you can't get a mortgage, et cetera, et cetera. Because fundamentally, what is happening in this world? Like, this is all nerd speak, right? But like, let's step back, what's happening?

• 1:00:58Dylan Patel?

  Just now it started the nerd speak?

• 1:01:00Dwarkesh Patel?

  We'd be entering a regime, we're just, we're in a totally different growth regime, basically. And the economy is basically saying, hey, you like paying people, you, the government borrowing money to pay people pensions, the opportunity cost of that is extremely high now, because that money could be spent building a robot factory that builds a robot factory that builds a robot factory. And so the opportunity cost of capital is going to increase a ton. And that just like, that's fundamentally what the cause of all of these things we're talking about.

• 1:01:28Dylan Patel?

  Yeah, so, so as interest rates go up, equity markets get pummeled.

• 1:01:32Dwarkesh Patel?

  Yeah.

• 1:01:33Dylan Patel?

  And even AI companies, right? People are like, you know, some people who really believe in AI are like, why does Micron or Hynix or Kyocia trade at 2 or 3x earnings? And it's like, well, if you're really AI-pilled, everything in the economy should trade at like 2 or 3x earnings. And if you're not AI-pilled, then sure, they're, they're over-earning. So it's sort of like a, an argument for why, like, I think memory is gonna do great, but you know, memory, memory stocks shouldn't, you know, 10x or whatever, again. Because if they were, if we're in the market where there's that much demand for memory, which means AI's caused this drastic change in the economy, then everything should trade at like 2 or 3x multiples, and the stock market should fucking crash.

• 1:02:14Dwarkesh Patel?

  Right.

• 1:02:14Dylan Patel?

  Right? And so, in a sense, like Meta trading, I don't know, I think Meta trades at like something, they're like 1.5 trillion dollar company. It's like, what? Silly? They're worth way more than that, at least in like a logical sense, you just look at their cash flows. And, and like all the infrastructure they're hoarding, and all the compute that they're gonna be able to sell for crazy amounts of dollars per watt, either as tokens, because their lab works, or just Anthropic and OpenAI. Ultimately, it becomes a question of like, you have to reallocate all the capital to the AGI, and you do that by pricing everyone else out. And, and, and so the limiter on AGI is not how fast can the research engineers, like, you know, our roommate Sholto can crank the gears.

• 1:02:55Dylan Patel?

  It's actually just like, how much does the rest of the world let that happen?

• 1:03:00Dwarkesh Patel?

  Right.

• 1:03:00Dylan Patel?

  Because they're gonna regulate, they're gonna obviously increase interest rates. They're gonna say, no data centers, they're gonna say, stop building fabs. They're gonna say, oh shit, every company's equity value is tanking, so how can I pay for AI? You know, to increase my business? Well then, you know, like, okay, then Anthropic and OpenAI have to start like building their own stuff. And obviously, they're gonna eventually focus, you know, they're building their own chips already, or at least designing their own chips, and it'll expand out there. You know, they're contracting their own data centers and building their own infra in the next couple of years. You know, there's sort of like, how does this reallocation of the economy happen? But there's a lot of downward pressure on it not being, you know, just straight takeoff. Even if the models were capable of it.

• 1:03:43Dylan Patel?

  Which, which I think you and I believe, we're in a world where models are capable of that. But slow takeoff is, is at least my hope, possible, because everything in the economy and regulatory world, like governments saying, don't release your models, governments saying, actually, you can't even use your models internally that much, because that's gonna happen soon. They're already saying you can't release your models.

• 1:04:04Dwarkesh Patel?

  Which is actually, the thing I'm most worried about is, you know, a singularity, which external deployment is, is actually helping, right? So the fact that we're preventing external deployment.

• 1:04:13Dylan Patel?

  Well, does that prevent singularity?

• 1:04:15Dwarkesh Patel?

  I mean, right now, it would at least do more revenue, because the models aren't capable of RSI. But I'm worried about a world where it's 2030, and the government's like, we're gonna wait 6 months before we can release your model to the public.

• 1:04:26Dylan Patel?

  6 months, 100x, let's go!

• 1:04:28Dwarkesh Patel?

  Yeah, and that 6 months, they're just like, they do like recursive self-employment internally. They just have all kinds of crazy shit happening in the company. Meanwhile, the rest of us are stuck with models that are like, at current pace, years behind.

• 1:04:39Dylan Patel?

  Yeah.

• 1:04:39Dwarkesh Patel?

  So here's my thought. Okay, suppose that the whole world gets in on this conspiracy to like try to slow down AI.

• 1:04:47Dylan Patel?

  I don't think it's a conspiracy, it's like, it's outwardly written.

• 1:04:50Dwarkesh Patel?

  Yeah, yeah.

• 1:04:50Dylan Patel?

  You know, from like, every politician.

• 1:04:53Dwarkesh Patel?

  Suppose they basically prevent an entire, they slow down AI by a year. They, if compute is increasing 2 to 3x every single year, they prevent a whole year of AI deployment, such that you're a year behind where you would otherwise been. During RSI, you're getting 3 to 6 years of AI progress in a single year.

• 1:05:12Dylan Patel?

  Well, but they can't, they don't just limit compute, right? They also limit the lab's ability to release the model internally, right? We saw that, Anthropic had to stop giving Mythos to foreign employees for a bit.

• 1:05:23Dwarkesh Patel?

  I didn't, I didn't know that was true, like internally as well?

• 1:05:26Dylan Patel?

  I mean, that's what they claimed, and they claimed-

• 1:05:27Dwarkesh Patel?

  I thought that was like just a different checkpoint that was not Mythos, but it was basically Mythos.

• 1:05:29Dylan Patel?

  Yeah, yeah, yeah, but I mean, like, stuff like that is not gonna be allowed either, right? Like, the government is dumb, but they're not that dumb, right? Like, you know, I would hope at least. You know, governments are gonna not want companies, at least the US government has the cards here, they're not gonna want Anthropic to use Mythos for internally. They're gonna be like, hold the fuck on, right? Like, slow down, you know, because, because all of these regulatory reasons. Everyone who's elected is gonna hate AI. Even the people who are elected already hate AI. All the constituents, you're gonna literally have like, I bet you at some point, your parents are gonna call you and be like, Dwarshika, you're doing a terrible job. You're making every AI progress happen faster than I like it.

• 1:06:10Dwarkesh Patel?

  It's my podcast?

• 1:06:11Dylan Patel?

  It's gonna happen, it's gonna fucking happen.

• 1:06:13Dwarkesh Patel?

  I'm accelerating AI progress.

• 1:06:15Dylan Patel?

  I mean, maybe, you educate people, right? And maybe if they're smarter, they're progressing AI faster. But anyways, like, you're going to have real-world constraints on the progress and development and deployment of AI, even though, you know, it will happen eventually. It's like, we could tear ourselves apart before we get there.

• 1:06:33Dwarkesh Patel?

  Jane Street is hiring for two separate ML internships right now. One focused on ML engineering, and the other focused on ML research. I sat down with Alok, who helps run the research track, to learn more about that program.

• 1:06:45Speaker 02

  I think this domain is fundamentally understudied. Often, we have unanswered questions within our deep learning research team, where we don't understand some, say, some market participants' behaviors, or certain dynamics of how trading happens. These unanswered questions make for really good intern projects, because they are ultimately topics that we care about and just haven't gotten around to figuring out yet.

• 1:07:06Dwarkesh Patel?

  So even as an intern, you'll be contributing to real research, not working on some sort of contrived exercise. The Jane Street team follows frontier LLM research closely. A relatively common intern project is adapting a recent paper to financial markets, which come with their own set of gnarly problems.

• 1:07:22Speaker 02

  Ultimately, we're trying to model thousands of interconnected irregular time series. The signal-to-noise ratios are extremely low, because we have a lot of competitors trying to do the same. So we have this like adversarial, non-stationary, extremely high-dimensional problem that we were trying to solve.

• 1:07:36Dwarkesh Patel?

  To be clear, you don't need to know anything about finance in order to be a good fit. As long as you have a background in ML research, Jane Street can teach you the rest. Their 2027 internship applications are open now. Apply at janestreet.com/thewarcash. One thing I find crazy about these scenarios is just how much of the world's future labor supply ends up in very few companies? And also, how fast that labor supply grows year over year? So if like, compute at the frontier, you know, in flop terms, is growing 4 or 5x a year. And further, the compute required to achieve the capabilities is like decreasing 3x a year.

• 1:08:20Dwarkesh Patel?

  So the computer at the frontier, or the, basically, the effective AI population size at the frontier lab is increasing 10x year over year. And so, that doesn't really matter that much right now, because AIs are not good enough to do full jobs, or be like, as autonomous as people in their capacity to do work, or pull off schemes, or whatever. But if the current trend continues, you have a world where OpenAI goes from having, say, 10 million basically AI laborers this year, to 100 million the next year, to a billion the year after that. And then, pretty soon, even if compute scaling slows down, it doesn't take many more years before each company individually has more labor equivalents than there are people on Earth.

• 1:09:04Dwarkesh Patel?

  And I think that's like a thing that is very plausible by the end of this decade. That there's more AI labor, more effective population within a single lab, than there are people on Earth. So we talk often about centralization of power because of nationalization or whatever, but we don't think enough about the fact that we're actually moving very fast into a regime where most AI labor, or sorry, most, most people, like in terms of like the work output or something, is just like concentrated within two labs who are consuming more and more of the world's compute. And so if, if these AIs are misaligned, then most of the world is misaligned, basically, because like most of the world's minds are there. But even if they're not, it's just very few companies have like a lot of influence or a lot of control.

• 1:09:46Dylan Patel?

  Yeah, it's sort of, there was the whole spat recently where it's like, I think Gavin Baker was like, Anthropic, Dario believes that there's only going to be one company in the world. And then, you know, Shulto and Dario came out and were like, no, no, no, we didn't say that. But ultimately, you know, if you believe in RSI, you believe in the labs are the most effective user of compute and can generate the most value from the compute, then the only thing that's going to happen is centralization of compute. And if you believe in, you know, sort of AI researchers, RSI, AGI, then all of this exists. All of this is the base.

• 1:10:20Dwarkesh Patel?

  This is even true if there's no RSI. The current effective, like, effective population at the frontier is currently increasing 10x year-over-year for a given level of capabilities. Right? So if you get to the level of capabilities, which is a human, a very competent remote worker, or like a very competent software engineer, a very competent researcher, that population of those would like 10x year-over-year at the current rate of capability.

• 1:10:44Dylan Patel?

  I see. And without RSI, then once you have RSI, it's even crazier.

• 1:10:46Dwarkesh Patel?

  Maybe growing like 100x year or 1,000x year, where they're like, intelligence is increasing, but the population isn't increasing, or some mixture of the two, right?

• 1:10:53Dylan Patel?

  Yeah, I mean, I guess, I guess like, what world do you see, Dwarkesh, where everything is not centralized? Because it seems to me that every force is screeching towards centralization.

• 1:11:05Dwarkesh Patel?

  Right.

• 1:11:05Dylan Patel?

  And that's scary as hell.

• 1:11:07Dwarkesh Patel?

  Yeah.

• 1:11:09Dylan Patel?

  I don't, I don't, you know, I would love for it not to be centralized completely. But maybe that's, that's the whole point of a machine, a machine that loves grace, right? Is, is, it is everything and it makes our lives great.

• 1:11:20Dwarkesh Patel?

  Yeah, it's so hard to think about the future, but I agree with you that, I think the fundamental problem is that lab, AI training has huge economies of scale, because any effort you spend into training an AI for a specific skill or a specific set of knowledge gets amortized across billions of sessions or billions of users. Furthermore, so that's like one effect. The other effect is if you're slightly ahead in the AI race and compute is in shortage, you can charge a much higher markup because you can better economize this scarce resource. So there's like two effects which are, give more and more to the person who's like ahead in the AI race. There may be more, right? So if there's, if models are learning from deployment and one model is like deployed much more widely than another one, it's getting much more like real world data.

• 1:12:05Dylan Patel?

  Yeah, your point, your point is taken that like, whether it's user deployment and continual learning, whether it's training, having these economies of scales, whether it's the incremental progress that the best AI model helps you to make the next AI model, RSI, all of these things.

• 1:12:21Dwarkesh Patel?

  Oh, sorry, I didn't even mention RSI.

• 1:12:22Dylan Patel?

  All of these things point to centralization.

• 1:12:24Dwarkesh Patel?

  So I think one of the big intellectual projects, honestly, that, yeah, we should spend some time thinking about async, or at least I'll spend some time thinking about, is what is a vision of like a decentralized, broadly empowered future after AGI that takes these economies of scale seriously? The alternative vision is that the government controls it. And maybe you think that you can trust the government more because it's not a private corporation.

• 1:12:47Dylan Patel?

  I don't trust the government and I don't trust Dario and I don't trust Sam.

• 1:12:50Dwarkesh Patel?

  Yeah, yeah, that's a problem, right? But there's no, at least, obviously, obviously it's like very easy to be wrong about the future and you don't anticipate a key effect or something that changes everything. But ex ante, it's very hard to see a reason why there, or like, how we avoid a scenario where we have to choose one source of centralization.

• 1:13:08Dylan Patel?

  It's why capitalism worked, right? It's the decentralized decision-making and decentralized power. And why super centralized capitalistic economies actually grew slower than super decentralized capitalist economies to some extent. You have to have rule of law and all this. Bot then AI flips all this on its head.

• 1:13:22Dwarkesh Patel?

  Right.

• 1:13:22Dylan Patel?

  And ultimately, you're like, actually, private ownership is probably not the most efficient economy and therefore grows slower than an AI economy, which is centralized.

• 1:13:32Dwarkesh Patel?

  It's still private ownership, but it's like, how many firms are really involved in this, this, like, this share of the economy that's not, what, like 5% of the economy or something like that in the US? Sorry, 1 trillion divided by 30, less than that, sorry. But yeah, maybe 2%, of the economy right now. It's like Nvidia's a huge share of it and Anthropic and OpenAI and these hyperscalers. And obviously, there's other firms involved, but like a large share of the AI stuff is just happening from very few companies. So it's like, it could be private property, but like very few companies are involved.

• 1:13:58Dylan Patel?

  I mean, this is what the structure of the market is doing. So, you know, what, what can prevent it? I don't know. Unless AI progress slows down, unless governments regulate the fuck out of it, this is all that happens. In which case, you know, we're headed for a world where either we have super concentration of resources and we pray that that one company gets everything right, or we have governments slow everything down and people slow everything down and, and you have a slowdown of progress somehow, hopefully. And, and there is a more of a balance of power. And even as we go towards an AGI, ASI, RSI, everything along the way will still lead to, someone's going to allocate, going to capture more resources. So, so it's kind of hard for a framework in which AI doesn't lead to super concentration.

• 1:14:41Dwarkesh Patel?

  Yeah.

• 1:14:42Dylan Patel?

  Now, the one positive thing here is that today Anthropic does not capture most of the value. So we can talk all we want about, oh, you know, they went from $20 million per megawatt to $100 million per megawatt, but they're still paying 13 for a lot of the compute they're buying. But at the end of the day, the reason they've gone to $100 million per megawatt is because Jane Street is capturing $300 million per megawatt or $500 million per megawatt. Where Dwarkesh, from researching his podcast and learning about credit, is capturing, you know, how many dollars per megawatt? Now, how much can you use? Tough.

• 1:15:18Dwarkesh Patel?

  Yeah, yeah, yeah.

• 1:15:19Dylan Patel?

  But, you know, I think, I think that's the, like, one saving grace, is that the rest of the economy maybe profits so much more from Anthropic.

• 1:15:26Dwarkesh Patel?

  No, no, but the whole logic you were laying out earlier of them reallocating inference to AI R and D, the whole logic of that is that the returns to labor inside AI labs is much higher than the returns.

• 1:15:39Dylan Patel?

  This is my cope.

• 1:15:40Dwarkesh Patel?

  Returns outside, yeah.

• 1:15:41Dylan Patel?

  This is my cope. I agree. In, in all scenarios of the world, you know, there's 80,000 worlds and only one of them, Anthropic doesn't own the whole world, is, is that, is that, you know, again, power concentrates because I don't want to send the tokens outside, they're more valuable inside. And so it's the same thing, right? Why would I let Jane Street, you know, make all this money off of these degenerate options traders?

• 1:16:01Dwarkesh Patel?

  Hey, there's, there's some options there, come on. Jesus Christ.

• 1:16:05Dylan Patel?

  No, I think it's great. I think it's great. It's a good value for the world to make it an efficient market.

• 1:16:09Dwarkesh Patel?

  Yeah, yeah, yeah.

• 1:16:10Dylan Patel?

  You know, Jane Street making all this money off of getting the worldview correctly, making money off of degenerate options traders, whatever it is, you know, why would Anthropic allocate compute to that? If Anthro, if, if, if the end, you know, monetization that Jane Street has per megawatt is 200, so they're willing to pay Anthropic 100, well, what if Anthropic can just generate hundreds of millions of dollars per megawatt by using that compute internally? And that's, that's what's happening.

• 1:16:36Dwarkesh Patel?

  There you go. Well, on that somber note, I guess, I guess, I guess we'll meet again when the RSI is officially kicked off.

• 1:16:44Dylan Patel?

  You're not going to have me on your podcast again for like two months?

• 1:16:50Dwarkesh Patel?

  All right. Cool. Thanks, dude.