China’s catching up to US AI… Here’s why it won’t matter

Lennart Heim, a researcher and information scientist at RAND Corporation, joins Azeem Azhar to unpack a provocative claim: China is catching up with US AI capabilities, but it doesn't matter.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Azeem Azhar?

  We are really lucky to be joined by Leonard Heim. He said China is catching up with the US in AI capabilities and that it didn't matter. What's America's big advantage?

• 0:09Speaker 01

  Having eight times more of the resource, which is the most important.

• 0:12Azeem Azhar?

  Let's start on why compute matters and what it drives.

• 0:16Speaker 01

  I think it's fair to say compute is like the currency of AI. Every time it's being produced requires computing power. If you have less compute, you've got less AI workers. And population size is pretty important for economic growth.

• 0:26Azeem Azhar?

  There is this question about a world where compute resources are highly concentrated. Typically in political systems, we don't like concentration of power.

• 0:34Speaker 01

  Let's look at a status quo. I think like 60 to 70% of all supercomputers are in the US. I'm not an antitrust lawyer, but I think this is at least something we should look into.

• 0:42Azeem Azhar?

  This also explains, in a sense, why the EU and why India and also the UAE are pushing for their own compute infrastructure. So this sense of this concentration risk is really being noticed increasingly over the last five years. How do you imagine it plays out?

• 0:59Speaker 01

  I don't think the plan is like every country is going to like redevelop the whole tech stack and come up with their own solution. Again, maybe I'm a little bit too much kumbaya. It would be nice if we can just get together and just say like, here are the standards, here's how the democratic world is doing AI.

• 1:19Azeem Azhar?

  We are really lucky to be joined by Leonard Heim. He is a researcher and information scientist at RAND, the OG of think tanks. He's been an advisor of EPOP, which is an AI research group, which readers of Exponential View know that we rather like. I have been following his work for a long time and he recently wrote an essay on the China Talk Substack arguing two things. He said China is catching up with the US in AI capabilities and that it didn't matter. So that is something to talk about. Leonard, let's get into that key claim. China may match individual US model capabilities this year, but that won't matter for America. What's America's big advantage?

• 2:04Speaker 01

  So the argument which I'm putting forward in this essay is basically inspired what happened with DeepSeek. Everybody freaked out. Some people were just expecting there's like this really big lead and it's like really, really hard to build competitive AI. Don't get me wrong, not everyone can do it. But I think like actually more actors than many people assume can build competitive models. In particular, being like running behind is actually easier than many people expect. What I mostly work on is the importance of compute. And I'm just always asking like, well, there's these many data centers, there are millions of GPUs out there. How are they being used? Sure, tens of thousands of them are being used for training systems. And this is what we see more and more is AI being deployed, AI being used. And what I took, what I'm trying to put forward to tease is just like, to some degree, like guys, actually it's more complex.

• 2:46Speaker 01

  We cannot only look at model benchmarks and say, this, this is the country which is winning. This is the country which is losing, or we only have a three month gap. There's more to it. In particular, if you think about like, how do expert controls buy it? So what I'm putting forward there is just like, actually, you use compute for training. They only use part of the compute for training. They will probably build competitive models in China this year, particularly with test time compute being on there. But given the compute might be more important in the future, or you want to deploy it at scale to get your AI agents, or whatever the future holds. This is eventually where I think expert control holds, and I think where the US at least has a big advantage. How to then leverage this is another question.

• 3:20Azeem Azhar?

  Wonderful. Okay, so we've got a sketch of the argument. We need to do some level setting. Let's start in the engine room and on why compute matters and what it drives. So what exactly is compute in this context?

• 3:36Speaker 01

  Yeah, I think we use compute loosely here to refer to many different things. But I think more broadly, what we mean with this is like processing units, integrated circuits which do the computations, right? And when we talk about AI, we talk about GPUs or AI chips. It started in 2012, LXNet used two GPUs. Now we see Elon Musk building clusters with 200 thousands of them. I think he's currently right now talking about upgrading to a million GPUs. And I think it's fair to say compute is like the currency of AI. You know, when you're in Silicon Valley, people bolster of how much GPUs they have and how big their model is. And again, it just became like this key ingredient for AI for training the systems, but eventually deploying it. It's like the water acid is for the fish or the air this far. It's like this is what these systems need to be run and to be developed.

• 4:21Azeem Azhar?

  Every time I talk to a researcher at a frontier lab, they're always complaining about the lack of compute. Even though these labs have more compute than anyone else, because there is so much work to be done. And the more compute you have, it's not just that you do things faster. It's actually that you can do new things, right? You can do things at a higher degree of complexity, which may develop new capabilities. One of the things I found staggering is that there is this new exponential in the computer industry that is far faster than the Moore's law exponential, which is the amount of compute that is being used on these frontier models. Can you give us a flavor for what that is like today and how that has evolved over the past few years?

• 5:06Speaker 01

  Absolutely. Yeah, I loved it to bring it up because that's actually how it got started for me. I think in two thousand twenty, I think OpenAI put out the blog post AI and compute. And they're like, look, we looked at how much computation we needed to train a system and it's doubling every three point four months. I was like, wow, Moore's law is described as like the fastest exponential ever, doubling every two years. Something doubling every three point four months is quite, quite staggering. And what I then did, which basically also the original story of Epoch, I found some colleagues submitted like the study of like, well, actually let's add some more systems. Because the original OpenAI paper only like eight or 10 AI systems there and we then upgraded to a couple of hundreds. And what we then found, okay, it's not doubling every three point four months anymore. It's doubling every six months. That's still crazy, right? Crossing like multiple orders of magnitude over the last few years.

• 5:50Speaker 01

  Again, going from two GPUs to two hundred thousand or going from like spending a couple of hundred bucks on computational power on this to literally millions, potentially to billions. And why are people doing this? Well, because the systems get better. It's not like we want to burn money, right? It's just like we train big AI systems, train in more data. And if you do this, you just require more computing power. Has been what's been called scaling law since this has been, I think, the easiest way to describe AI progress over the last decade.

• 6:16Azeem Azhar?

  It really is a sort of a staggering outcome that we have this quite predictable law. That as you double the compute capacity, you get not quite a doubling, right? It's a sublinear relationship to the capabilities of the model, but it's predictable. And the power of a predictable upfront relationship is that you can start in some sense to talk about your ROI. You can kind of go to your CFO and say, well, if you give me this X hundred million dollars, I'm pretty confident that I can push the system this far and give us this result. And then we can work out whether it makes sense financially or strategically. Although I think at the moment, that calculus doesn't exist. It's just get the compute, run bigger models, get higher capabilities.

• 7:00Azeem Azhar?

  Is that your sense of what the world is like right now?

• 7:03Speaker 01

  I think to some extent, I mean, like, if you need to justify millions being spent on something, that's probably easier than justifying billion, right? So and if OpenAI and others, and it's a five hundred billion and for Stargate, I updated some math, right? Again, but the math here is beyond just training bigger models. Even if you just stay at current capabilities, we need to deploy them. More and more people will be using them, AI agents might be working, they gotta need to be running somewhere. And that's to some degree, the nice thing about compute, you're not making a specific bet saying, oh, large language models are the thing. You're basically just saying like, look, having more of something is useful.

• 7:34Azeem Azhar?

  One of the things that OpenAI, I think Sam Altman said, was that in a few years, the bulk of the compute would be on inference, right? That is the deployment of these models. And certainly in the modeling that exponential view has done, we had come to a similar conclusion. What is your assessment of how the AI compute specifically splits now between the research and development of training new models? And actually executing them when when we think about these leading frontier labs?

• 8:03Speaker 01

  It's hard to say because of course that publicly don't tell us. There have actually been some papers coming out of Google where they talked about the environmental footprint. And Nvidia, but I think asking Nvidia on AWS, the question is like, what is an AI workload? Are your becoming their systems you use for Instagram also AI workload? So if you look at it this way, then AI is like computers predominantly being used for deploying systems, right? This might look different if you look at a frontier lab. Right, but I think like over time, you see like the share of deployment getting higher. And before ChatGPT, there was not a lot of deployment. There were not many users, right? There were like some nerds playing around with the API. But since then, again, I think many people saying the fastest growing app ever. You need to serve millions of users, probably up into the hundreds of millions of users now.

• 8:45Speaker 01

  You need tons of GPUs there. And if everybody tries to Ghibli for the image, which we saw, they had a GPU crunch.

• 8:50Azeem Azhar?

  OpenAI practically doubled its users in a few weeks because of the Ghibli image redrawing that took place. By the way, not something I succeeded in. I somehow made a mistake with my prompt every time. But there's also this new shift in how the systems are designed, right? So back in the old days, so ancient history, twenty twenty four, the idea was that you did pre-training compute, which was that you train this enormous model once, and then you would have this model that you could run inference against, which was much less compute-intensive. Give listeners a sense of just how much compute was required, because these numbers are pretty astronomical to train a GPT four or a Claude.

• 9:29Speaker 01

  I mean, we were talking about history, right? We started with like two GPUs, which was the breakthrough with LXNet. And now we go up, I think the biggest system we know with numbers of GPUs is around a hundred thousands, which is like grog free. It goes back to so deep, see, how do we measure this in terms of money? Let's say there are two ways of measuring it. There's like how much you need to pay for getting these GPUs, like you want to buy them. And Nvidia's stock is going up there for a nice margin. So again, if you buy a hundred thousand chips, each chip costs you forty thousand dollars, right? So we go up into the billions. So just like get the AI chip, which makes up the large majority of the cost for building such a data center. This might change in the future if the AI chip landscape becomes more competitive right now. But there is a reason right now why Nvidia is like so high on the stock market. So it costs you billions to just get the GPUs. Of course, it's better. A lot of times you don't need to actually build your own cluster.

• 10:13Speaker 01

  You can just go to a cloud computing company and rent the GPUs, right? So you tell them, hey, could I get thirty thousand GPUs, say for six months? If you do it this way, then current systems cost them like the three-digit millions for the computing power alone.

• 10:24Azeem Azhar?

  The three-digit millions, wait, we talked about a hundred thousand chips at forty thousand each, which is four billion dollars. So you're saying three-digit millions, in other words, as an amortized cost of

• 10:34Speaker 01

  Indeed.

• 10:35Azeem Azhar?

  The amortizing the chips over Five years, the electricity they need to run them and the very low salaries, no doubt that the AI scientists are paid.

• 10:43Speaker 01

  Deepseek puts out this number, it costed us six million. This does only include the amortized cost for buying the GPUs. It does not include the salaries. It doesn't include all of your failures and experiments. We at least know from one paper, I think it was the science paper where Hugging Face played a big role, where they said the total amount of compute spend was actually three X the amount we spend on the final training round. You make mistakes, do like de-risking training rounds. I'm not going to spend a hundred billion on your architecture if you haven't tested. So like I start with a small architecture, scale it up, see if it works. That's the whole idea, right? This trial and error and you've wrote computer and someone you say, sure, why not? You know, let's burn it and hopefully we get a good system out of it.

• 11:18Azeem Azhar?

  One last question on the compute before we move into the sort of the heart of the discussion. So we talked about pre-training compute, which is these really expensive, risky, frankly, training runs that are building the biggest, most complicated pieces of software we've ever known. They run to seven, eight figures in cost and they can go wrong. But we then started to move with the O one model that OpenAI released to the idea of inference compute or test time compute, which is that you allow the model to provide an answer and then explore many, many more answers before it gives you a final result, which allows a sort of a reasoning-like capability.

• 12:00Azeem Azhar?

  So could you just explain the sort of relative weight of test time compute? And what it has meant for the traditional prompt, right? How much more compute expensive is an O one prompt over a traditional GPT four prompt?

• 12:13Speaker 01

  So everything which you said before is still true. We train these big pre-trained models, but now we do another thing. We make these model reason. I think the best way to think about it is just like, people are not surprised. If I get you an hour to think about something, you come up with a better solution than if I get your off-the-cuff answer, right?

• 12:26Azeem Azhar?

  Yeah.

• 12:27Speaker 01

  And it's basically, well, this was always the case, right? Like a couple of years ago, somebody said, oh, if I prompt the model and tell it to think step by step, it produces a better result. All I told it to think step by step. Again, not surprising. Every teacher knows this. If you have a good prompt, your students produce better results. What we basically did with O one and like this new whole new test time paradigm is we help these models to reason, we train them all to reason. So we have a big pre-trained model again, spending millions on this, then we do another step on it. We can call it post-training. Some people might even call it mid-training now, where we do some type of reinforcement learning on reasoning. This particularly works well for coding and math. Why is that? There are little truths. We can verify it. There's a right and wrong. I work on policy. Kind of hard to say what's right and wrong here, actually.

• 13:08Azeem Azhar?

  Well, I mean, your ideas surely are right and.

• 13:10Speaker 01

  Well, yeah, I I don't know. Let's see.

• 13:12Azeem Azhar?

  Yeah, but you're right. So maths and coding, we can verify them. And they lend themselves very well to this inference time compute. Let's get some numbers out there. I mean, we're talking about, I mean, roughly speaking, you might have a back and forth with a traditional L L M. And it might be a few thousand tokens where a token is roughly a word. It's a bit less than a word. In a test time compute model, that might run into tens, if not hundreds of thousands of tokens being computed. And every token is a computing opera, I mean, millions, tens of millions of computing operations.

• 13:42Speaker 01

  Indeed, it's just the same operation, the modus things, but ideally at the end you get like these high quality tokens, you know, like those who actually have the answer. I think my favorite example is actually the R A G I benchmark, where like O one, when it came out, like had like this really impressive record. To achieve this record, I think they spent twenty thousand dollars on compute time on each task. They produced a couple of, I think, hundreds of thousands of tokens. They probably produced like five times a Harry Potter book to fill in a pixel. Again, this goes back to our L L M to do it, but I think this gives people an idea here. And they did two things. They did this reasoning. It's not that the model reasoned over five Harry Potter books. It just did many, many reasoning attempts. And you're just like, oh, I tried here, I tried here. And then you just look which is the better one. Ideally, you have some tree search over it. Right? So all of these things are now combined, but the cost can be kind of staggering, right? And I think the key implication is here to achieve the best capabilities at the beginning.

• 14:28Speaker 01

  You spend a ton of time on appearance, but again, A I moves forward or exponentially move forward, everything gets cheaper over time.

• 14:33Azeem Azhar?

  Well, everything gets cheaper over time. And you have hinted at this because DeepSeek, as we know, came out with similar quality performance to many American models, but it was much cheaper to run. So what is the role of algorithmic efficiency, software optimization and general optimization in all of this? Because I think, you know, we all know that getting a G P T four quality response is two or three hundred times cheaper today. Than it was eighteen months ago. I mean, it's staggering. We're talking about exponentials. That is a tremendous exponential. Why is that happening?

• 15:07Speaker 01

  History of humanity, you know, we do something and then we later learn how to do it cheaper. Just like economies of scale, we build bigger models. And I think that's the case for like computer science since forever, right? And I think for AI, it's just like staggering because we have like these basically brute force approaches. We've got a big model, we do it and then later we get smarter. We have like this compute efficiency improvement. And again, the odds are really fast, right? It's roughly three X per year. So like basically the cost to achieve a given capability is like three X cheaper at the end of the year. Because we just like, again, this is really hard to measure. Do you do it with loss? Do you do it on a specific benchmark? It gets tricky, but the rough trend line is there. And then DeepSeek is, I think, caught many by surprise because they just didn't think about it. They just felt like, oh, it's always cost us a hundred million. DeepSeek is perfectly on the trend line, basically.

• 15:50Speaker 01

  If you look at it, just like it sits directly where it's supposed to sit. The exception might be test and compute. This is actually a bigger increase in terms of compute efficiency in counter star by improvement, which we've seen.

• 16:00Azeem Azhar?

  Okay, so what we've done now is we've covered off the importance of compute, the difference between training and inference compute. We've also identified that we get much, much better, more efficient at getting the same performance over time. So that gets us to this question of the geopolitics of compute, right? So there is this competition between the US and China. We can discuss a bit later as to why there needs to be such a competition. But your argument was that the total compute capacity is what will ultimately give the US an advantage. So is the core idea that it's not about the best frontier, most performant model that people are pursuing for the headlines?

• 16:41Azeem Azhar?

  But rather about the rate of diffusion, which is an argument that Jeffrey Ding, who's I think a mutual connection of both of ours, has made. Or an argument that Eric Schmidt made with a colleague in the New York Times today. That it's really about diffusion that's going to drive the economic and strategic benefits of AI.

• 16:59Speaker 01

  Yeah, I think I don't want to go forward and say it is this, right? I'm just saying it's like, look, there has been this hyper focus on benchmark capabilities, on chatbot arena to measure the AI US rates. And it's like, man. I think it's more complicated. Eventually, it really depends what you want to measure and what you're worried about. If I care about national security, I measure different things than if I care about like economic diffusion, right? To some degree for national security, I might not even care about how many users are using a chatbot to get better cooking recipes. Right? What I care a lot about, if you have, I don't know, a million drones embedded with AI or like two drones embedded with AI, right? So the argument which I'm just putting forward is more just like, we need to understand how Xbox controls, which GS government is doing on these AI chips are actually working.

• 17:43Speaker 01

  What to expect and what not to expect, right? I think there's a world with like pure raw model capabilities, matter of time. It depends how you think the AI future is gonna go, right? If there's gonna be an oracle in the future and it just tells me the truth to the universe, hell yeah. And that's like, what, 10,000 tokens? Amazing. I think that's quite not plausible. I think the future will just look way more complicated and it's actually about people using the system. Now, we are saying this like having a car or something, but you actually need to drive it, right? And then it's not you driving, everybody needs to drive it, right? And that's how I expect it to happen. And I think the best example of things what is AI agents and workers. People claim there will be in the future, these drop-in remote workers and AI is literally gonna do my job because I'm just sitting in front of a computer all day. I mean, sometimes I like do talks at whatever, I guess, maybe that's my culture.

• 18:26Speaker 01

  These things, if that's true, it just depends on how much computing.

• 18:29Azeem Azhar?

  I would say that we've run some experiments here and a few weeks ago, while I was sitting with one of my colleagues, he accesses his AIs through an API rather than through the apps, which means he can count his token use. And he was going off just doing some little task and in a 15-minute period, he used up 400,000 tokens, which is about 320,000 words, something like that, which, you know, is far beyond what you would do as a human talking to any machine, even with a brain interface. And I think as I see enterprise applications, as I go around the world and talk to companies, you increasingly see that they have lots of autonomous or semi-autonomous workflows running, where AI systems are talking to AI systems at quite high rates.

• 19:11Azeem Azhar?

  And those are well beyond the kind of consumption capability of a human. Or I generally have an AI listen in to all of my meetings and so you're running at tens of thousands of tokens a day before I do anything. So that there is clearly this sort of core foundational demand that it is quite hard to get one's head around. And you're familiar with OpenRouter. I mean, OpenRouter is a, it's a sort of aggregator of. L L M A P I's and it sees a very, very small portion of the market. But their year to year growth in token usage has been thirty eight to forty X forty fold, right? Based on their public data, there is this burgeoning explosion that I'm sure is going to happen.

• 19:53Azeem Azhar?

  And now that Google Gemini, for sake of argument, summarizes every email in Gmail. Well, every one of those is across a billion users.

• 20:00Speaker 01

  Indeed. I mean, and this is nothing new, right? Like, I'm talking to you via an iPhone. There's a lot of thinking which went into this iPhone, lots of tokens, if you want to put it this way. And at the end of it, I get like one product. This is just how things are being produced and they're being like hidden behind layers. And this is what we're just seeing more and more with models. And again, every token that's being produced requires computing power. And then if you have less compute, you have less of this. If you have less compute, you've got less A I workers. And population size is pretty important for economic growth, right?

• 20:26Azeem Azhar?

  Take us through the argument that you actually concluded, because you concluded with some things that are born out of data and the extrapolations of that data, ultimately, that there would be more compute in the U S and not a sort of sufficient amount in China to enable a twenty thirty economy. Is that a reasonable assessment?

• 20:44Speaker 01

  I'm not sure what sufficient means. I'm just saying like, if like one country has more, this definitely helps it accelerate faster, right? So the argument which I'm putting forward there is just like, look, guys, it's more complex. Like, what do extra controls do and what they don't do? We have more compute, how can you leverage more compute? Well, actually, you have more A I workers, you can leverage A I there. Again, if you go to the policy discussion, this does not necessarily mean you're winning. You need to use it the right way. What is the right way? That's kind of the question which I'm putting forward, right? Where do we expect it to help, where we don't expect it to help? That's the question which I'm asking there. But if we go back to just like, we think A I is going to be like the driver of the economy in the future. Then having eight times more of the resource, which is the most important. It seems pretty important to me, right?

• 21:26Speaker 01

  This will just help you and you might get even the flywheel effect where you just take off to some degree.

• 21:31Azeem Azhar?

  Have the Biden era export controls been helpful in this set of scenarios that you have posited?

• 21:38Speaker 01

  Absolutely. And notably, these are not Biden era export controls. Actually, Trump started it. Trump made sure that ASML, this obscure company in the Netherlands, which building the machines which put the chips, don't go to China anymore. This was in 2018. And then we had entity listing controls on Huawei, one of the leading AI chip producers in China, right? And the related companies there. And then Biden just added on top of this with making sure the AI chips don't go there. And I think this is a large part why we currently see less compute in China versus in the US.

• 22:09Azeem Azhar?

  I will posit a different reason on that while I acknowledge all of the points you've made. You know, the compute has to be secured by people who know what they are doing. It's not just a case of having billions of dollars. And the US and China both have a small number of firms that do know that. But ultimately, it's expensive and you have to be able to access the capital markets. And the balance sheets of Google and Meta and Amazon are far, far larger and deeper than those of Baidu and Tencent and so on. And so you already had an ability to do more CapEx, buy more chips. And in the context of the moment, we had that moment just before the Ant Financial IPO,

• 22:51Azeem Azhar?

  which was pulled, where the Chinese government really sat on its entrepreneurs and a chill came into the Chinese market. Their valuations declined, the access to capital declined, the venture capital dried up, the balance sheets weakened. And in a way, I'm sure someone, a historian, will look at this, but my argument would be that more constraints were applied at that point. Because even today, with the US hyperscalers and the big tech firms spending $320 billion on CapEx, a little bit more than half of which will go on GPUs. That's above the capabilities of the Chinese big tech firms anyway. I'm sure that they have been, it's been very useful, particularly the SML thing, which actually, in a sense,

• 23:32Azeem Azhar?

  prevents China or slows China's ability to build its own semiconductor industry, which has been a strategic priority for many years. But in terms of procuring chips, there was just much more money in the US.

• 23:45Speaker 01

  Absolutely, yeah. I think, I mean, to some degree, this also might largely explain why we have like more frontier companies here. Right? So I do, I would like, I would not argue against it, that it's not one of the reasons. But even if you have the money and you can just buy like less good chips, that's definitely going to hit you in the long run, right? And again, depending on where this goes, something to point about extra controls, this has been the confusing thing with deep sea. It takes some time to bite and we had many failures, right? The first times they got the numbers wrong, those are the chips they train deep sea on, right? So we basically have only working X controls since, what is it, October 2023? It's a little bit more than a year, right? You don't put a cluster every new year. So I expect we'll see like a major impact going forward if certain enforcement and things are being patched.

• 24:26Speaker 01

  And again, all of this also depends on will Nvidia's chips continue to get exponentially better, right? To some degree, the good and thing about computing is it gets exponentially better. Who cares about the computer that built five years ago? I wouldn't use this that anymore, right? It's like my chips now are like 10 times better and I produce 10 times more of them. But if this would level off, then this might look actually different.

• 24:45Azeem Azhar?

  You do hear, I mean, you hear Jensen say, I'm the chief revenue destroyer in the sense that he keeps bringing out new chips that are better than the old version, but you see the hyperscale is saying, we're sweating chips at six years old and seven years old and they're still sort of fully used in a sense, they have to say that to the market. What we have here though is we have a challenge, we have a real bifurcation here, which is national security needs, which in a way predicated on how good the frontier models can be. And there are the economic needs which are predicated on the ability to deploy and diffuse high quality, reliable AI into your firms. That feels like it's a really difficult pair of objectives to optimize for. You know, there's a line, you know, he who chases after two rabbits will get neither.

• 25:30Azeem Azhar?

  How do policymakers make sense of these things? Cause they're quite conflicting.

• 25:36Speaker 01

  They are. I mean, this is the the concentrate of, right? Security comes at a cost, but this is also known for every company. And every company has ideally a CISO who takes care that your IP is secure, right?

• 25:48Azeem Azhar?

  But this this isn't so much about security coming at a cost, right? Security can be having the equivalent of biological secure levels for your labs, for AI labs, and that that is a cost that comes out. This is something that's slightly unknowable, which is will scaling over five years create this super intelligence that builds self reinforcing loops that take you up and away? And if it does, it's an existential national security issue. And therefore, you need to think in those terms. The other part of the argument, which is the economic argument, is that it's broad diffusion and use that matters. And it feels like those are really slightly at odds with each other.

• 26:25Speaker 01

  Yeah, I mean, look, there is like the view you just laid out, like it's a national security. It was just like, well, the AI, it takes off and then it's gonna kill us all. I think that's a national security risk. I think there are many other national security risks, which are between AI is all good and between AI is gonna kill us all. There are many different risks, right? We talk about cyber risk, we talk about biological misuse, we talk about loss of control scenarios, right? And they all need to be managed. And I think they're actually not too different from like normal security risk, which we're trying to manage all day long. If the US is trying to decide who to sell fighter jets to, pretty similar on this one, right? And then you're totally right. Eventually, if you decide to not sell chips to China, you might lose revenue, right? That's just the case. I think this is just like, again, what the government, again, that has been bipartisan consensus on has decided.

• 27:09Speaker 01

  It was like, yep, we take this revenue in, but eventually in the long run, that's the thing we wanna do here. Because AI might gonna kill us all, China might take over the world, whatever your favorite threat scenario is. I think there's like a polarity of like national security arguments, why people are doing this. And again, yeah, my job as a policymaker is like to calculate the costs of both, right? All I would say is, AI industry has been doing pretty fine so far, right? And again, when we talk about export control to China, that's one market. There's still the rest of the world, the majority of the market is still available. And if you just compare these semiconductor export controls, they're minor compared to what's happening right now with the terrorists, right? This is the way bigger economic deal, and it's still being done for many reasons, right?

• 27:47Azeem Azhar?

  I want to sort of dig into this a little bit because, you know, in a way, there is a cost to export controls. There is a cost to controlling this. The cost comes as follows. It's about where the government chooses to intervene in a market, which has a doctrinal effect on where a nation operates and thinks about itself. It has a fundamental impact on the certainty and the risk profile of a particular industry and therefore its ability to secure capital. It has a cost to Pala over any country that becomes dependent on, say, for a sake of argument, Nvidia chips, where they see that there is a line that is easily movable at whim in pencil

• 28:29Azeem Azhar?

  about whether they'll be able to get updates or support and whether that investment in CUDA and that infrastructure goes. So it's more than a blanket, sort of simple, clean, clinical cut. It's also one that spills over in many different directions. Then on the other side, we have this issue of to what extent does it stimulate innovation in other places in China in this case? We haven't really talked about Huawei much, but Huawei came out with this cloud matrix server unit. It has a sort of performance that's similar to Nvidia's top tier system, except it's four times as energy expensive. There are these other calculuses, right, that say, what are the costs that we have to bear? How do you think about that?

• 29:13Speaker 01

  Let's go back to the Huawei system, I think, because it's like a nice example of exactly the point which I'm trying to make. They will be able to build systems which are on paper similar to the American system. The difference, just like I was just saying, it eats way much power. It requires three times more chips, right? And like you have all of these inefficiencies. Does this mean they cannot not train, let's say, GPT four point five model? Absolutely, they can't probably. This is kind of what I'm expecting, right? The difference is just like while TSMC is churning out five million chips per year, they're churning out less than a million chips and they need three times more chips to achieve the same performance, right? This is exactly the argument which I'm making here. And then, you're right, there are costs to this, right? These export controls, they used to say small yard, high fence. I don't think that's the case, right?

• 29:56Speaker 01

  To some degree, when it originally started, two thousand and twenty-two, this was before ChatGPT. And I give the government like a lot of credit for foresight here. Like knowing that computers cool, if like people would have invested into N two into Nvidia, they were still early. So they had this foresight. And then it was just like, oh, we only hit on these advanced AI chips which are being used for servers. Sure, AI was a niche application. But again, if you if you look at like people like us, we believe AI is the future, AI is going to be everywhere. Then this is not a small yard anymore, right? If I use AI to literally plan my day, cook my food and everything else gets organized around me. Then you have collateral damage which hits on the broader economy. And this is just a thing which is simply true, right? And this is generally the challenge with governing to do use technologies on these kinds of things.

• 30:40Speaker 01

  What I would say, though, is China still has access to AI services. They can still use Microsoft Azure, they can use ChatGPT, right? So it really depends where on the value chain you want to intervene. What the US government wants to do is like, hey, you shouldn't be able to build your own competitive big systems and like you make human rights abuse, a bunch of other nasty stuff with it. You want to use AI which is like monitored by us and like again, you cannot build biological weapons with it. Sure, go for it, that's fine, right? And I think that's the best way then to think about it. And here, again, the key is chips can't go into China. They can access cloud computing right now without any restrictions. And any engineer, it's not like I need a computer like in my basement to train a model, I go to a cloud provider. So right now, they can technically do it.

• 31:24Speaker 01

  You're totally right, they're the whims of the US government. If they decide tomorrow to cut them off the cloud, that's a problem. But again, that's historically been the case for the monetary system, the same. This is just how national security works and how you're trying to govern, right? And those things are trying to be balanced, but I think eventually we can find a good middle ground here. And ideally, it's not only the US doing all of this. It's the US and its allies and partners with the semiconductor industry, the key allies there, building a broader multilateral AI governance framework, which is built with democratic values. That's at least what I'm aspiring to.

• 31:53Azeem Azhar?

  Let's talk about technology change in all of this, right? We're having to hold a couple of distinct ideas in our head. One is this idea of the very powerful national security model. I'm not thinking about one that kills us, but I'm thinking about one that runs the million drones rather than the 20 drones. The other is the economic side of this, which is the ability to scale this out across an entire economy and all the workers and give them all all superpowers. One of the things that strikes me over a five year period of time is that isn't long enough for certain things to change. I mean, the CTO of AMD, Mark Papermaster, has said that by 2030, he expects most inference loads to be run on device, right? So on the the smartphone.

• 32:33Azeem Azhar?

  Because in reality, to undertake 28, 20, 29, 20, 30 tasks, the quality of a smartphone chip and the RAM that's on that device will be enough to, you know, to do that, to act as the software coding agent and so on and so forth. So to some extent, at that point, the battle is not about these super sexy $40,000 GPUs or their competitors. It's about what can exist on device for which China can build lots of devices, the great chips, that's the mobile phone industry. But it might also be about new architectures that emerge predominantly for inference, right? Because we sometimes we think the training and inference are the same, but they're not. They they require different things of the the chip architectures, the memory bandwidth, the latency and so on.

• 33:15Azeem Azhar?

  So that also changes the way in which the leakiness or the effectiveness of export controls, given what's needed for the inference and the deployment.

• 33:24Speaker 01

  Absolutely. Well, I mean, we just talked about history. We talked about going from two chips to 100,000 chips. The two chips was not a transformer architecture. At some point, we developed transformers, you know, some point we had a new loss function. Me as a technologist, I love when people develop new papers, but I'm interested in the macro trends. That's why I just love like line going up compute Moore's law, more transistors per area generally to how do we do it? Completely different architectures over time, right?

• 33:50Azeem Azhar?

  Right.

• 33:51Speaker 01

  So if you find the right abstraction layer, again, you see these exponentials being the case there. Then some people argue like, oh, we have a new architecture that does this. And they're like, true, that's what happens all the time. This is what we call increasing algorithmic efficiency and compute efficiency. I think it's quite unlikely that somebody tomorrow pulls out a new architecture and says like, oh, look, AGI on a smartphone. I don't think that's the case. I think we will first build AGI on a big cluster. And then a couple of decades later, we'll build it on smartphone. This is just what we always see for these kinds of things. So the thing to acknowledge here is diffusion, right? A given capability gets cheap over time. I used to need to run it on a big server. No, I can run it on my smartphone. That's definitely the case, right? So, yes, we will see more workloads being on edge. For privacy reasons, we see Apple begging bad on these kinds of things.

• 34:34Speaker 01

  But even Apple is kind of acknowledging, it's like, damn. There are certain questions, we can't do them locally. You're gonna run out of battery power. We sent them up to the cloud and they came up with a really fancy architecture to preserve privacy.

• 34:45Azeem Azhar?

  It was super fancy, yeah.

• 34:46Speaker 01

  And these are the things I'm expecting, right? So like how you just say, like, if I want to set a reminder, sure, my L L M on my smartphone can. If I ask for the meaning of life, well, hopefully it goes up to the server and like it's processing and generates five books and tells me what the meaning of life is. Right? So I expect this trend to generally continue. They would just see these kinds of things. While, again, one curve is going up, the best capabilities will always be in the biggest servers roughly. Whereas a given capability goes down over time and will diffuse across. And that's definitely national security. You're totally right, right? If G P T five poses a national security risk, they can control it for, what, a year or two? And then every guy in a garage with five Mac minis potentially could reproduce it, right? And that's just the thing we need to deal with eventually.

• 35:28Azeem Azhar?

  Well, and eventually may in that case be very soon if we're talking about G P T five. Okay, we've got a few minutes left. I want to jump into just some thinking about the future and what the future ought to look like. When DeepSeek released its R one model and V three, not much fanfare from DeepSeek, but lots of people were very excited at the time. One unique feature was that it's an open source and open weights model. And Marc Andreessen, who is a Silicon Valley bench capitalist, really behind the Build America movement, you know, viewed this as a gift. I think I forget the exact text of the tweet, but it was something like, you know, this is a gift to humanity or or something similar. And what that does is that helps to diffuse A I capabilities because anyone can get those weights.

• 36:09Azeem Azhar?

  They can do things with them, customize them and build their own applications. And at this point, you know, Google and OpenAI have been getting progressively more more closed. So there is this question about, as we look at the future. Between a world where compute resources are highly concentrated, concentration lends a course to control. But in many places, typically in political systems, we don't like concentration of power, right? We like decentralized power. We like checks and balances. We like some competition. We also like that in economic systems, right, in markets. There is this simple two by two, right? Or maybe it's just two boxes, two paths. A world of compute concentration and a world of compute abundance.

• 36:52Azeem Azhar?

  And a world of closed source models, living with that concentrated compute and a world of capable open source models. Living on more available compute. What are the considerations for each of those scenarios? You know, how should people making their own mind up on what will ultimately be a political choice for us? Hopefully, think about those two.

• 37:11Speaker 01

  Well, that's an easy question, which will be a political choice for us, right? Or if one needs to intervene. I think the best way to start with is just let's look at a status quo. Most compute is largely concentrated. Where is it concentrated? The large majority sits in industry. This didn't always used to be the case, right? Academia was really strong in traditional high performance computing. This lead was eradicated over the last decade, basically. And secondly, where is this compute concentrated? Within industry, within our hyperscalers. Amazon Web Services, Microsoft Azure, Google Cloud, Oracle, they own the large majority of it. If you look at Nvidia's shareholder output, they need to like tell if a big customer, like if it's mostly for big customers. That's what you see. That's simply the case. These are all American companies. Which countries is it mostly concentrated? Most in the US, bad news.

• 37:56Speaker 01

  We just published a data set where we see like, I think like sixty to seventy percent of all supercomputers are in the US, right? Like Europe, ten percent, China's never ten percent, something on these lines. I think roughly this is also like what I expect all of AI computer resources being there. So that's the status quo, right? So if people just say like power to the people, they're worried about more concentration. It's like, look, the status quo, it's already concentrated across all these companies. And there's a good reason for investigation. I think there is ongoing investigations into the cloud stack, right? All of these cloud companies have just been getting bigger and bigger and bigger. Again, I'm not an antitrust lawyer, but I think this is at least something one should look into what we could do about it. So those are the computer resources being concentrated. Argument against it is like this doesn't mean who's using it. We both right now use computing power by Substack. Well, I don't think Substack has a cluster.

• 38:41Speaker 01

  It's probably Amazon Web Services, right? So it's something that I can just access it. The worry is just like, what if they cut us off? Right? What in the future? I need my AI lawyer and they turn my AI lawyer off and I'm just like on my own. That's a big problem, right?

• 38:53Azeem Azhar?

  Right.

• 38:53Speaker 01

  And I think that's where we just need to make sure like which role will AI play in the future? And then I think it's even more important. I don't want compute in the future. I want AI. If I have ten GPUs in my garage, but I don't have the leading AI model, that's a way bigger problem, right? So I think that's the thing which we need to think about, like what kind of services do people need access to? Right? And then is it a computer with AI model or not? If my mom needs a lawyer, I'm sorry, she's not gonna set up a cluster and spin up the new DeepSeek R2. Ain't gonna happen, right? There, I think it's like way more likely you potentially want access to AI systems. Even here, right now, whoever can afford the more tokens already has a lead on this, right?

• 39:30Azeem Azhar?

  Well, there are more, there are more countries, of course, than the US and China. So this also explains in a sense why the EU and why India and also the UAE are pushing for their own compute infrastructure. And that can be either be physically owned by local companies that fall under their jurisdiction or it can be the right kind of contracts with a hyperscaler. And then you've got this emerging tension that we've seen and certainly Microsoft has come and spoken about, talked about it in the last couple of weeks regarding Europe and the sense that their assets in other countries, sovereign nations now want to know, under whose legal purview does this reside? How certain can we be of this?

• 40:11Azeem Azhar?

  So this sense of this concentration risk is really being noticed increasingly over the last five years. I guess one question is, how do you imagine it plays out?

• 40:20Speaker 01

  You're totally right. You see more and more countries kind of waking up to this computing, right? Like I just said in the beginning, computers survive in San Francisco. I was just in Brussels. It's also vibe there. They're just a non-AI factory, right? I was sort of hanging around London around the time they announced the future of compute review in the UK. So like, I think people walk up to this computer here earlier. And to some it's easier. I think as a government, it's easier for me to set up a cluster than to actually build a leading AI company. This is my worry. I'm just like, guys, I don't want you to overly focus on compute, where there's a kind of force feeding people computers. Like, wait, are you actually solving any problem? You need a whole tech stack strategy. You want to build a fint AI company? Sure, go for it. Do you need more than just compute?

• 40:59Azeem Azhar?

  There's a problem here that I think is really hard for these governments to wind their way through, which is that the AI layer is going to be the fundamental infrastructure layer of the global economy. AI systems will be our interfaces as citizens, consumers, whatever role we take, to the bulk of the digital services that we need, and increasingly so. And so if the iPhone ended up being a political issue, which in a sense it was, it's something I wrote about in my first book a few years ago, the AI layer will be even more so. And I think that governments will have to figure out how they can articulate what they need and expect

• 41:42Azeem Azhar?

  and what kind of guarantees they need from that layer. Especially, as you say, it's concentrated in American hyperscalers, delivered admittedly through their European subsidiaries, but ultimately all routes lead back to Washington in some sense. And that seems like a really difficult, thorny problem for them to address over the next few years.

• 42:02Speaker 01

  Absolutely. I mean, you were saying, it's like actually not a new one, right? When like Snowden came out, I was like, oh, god damn, all of our data is going over there. We want like more sovereign cloud. So like to some degree, I think it's not a new problem. Just repackage with like a new tech layer on top of it. The thing where I think it just differs, I do think AI is a fairly big deal, probably the biggest deal in our century. So we really need to get it right here. And so like, I'm excited about all of these governments having these plans, right? And I want more techies in there and like doing it. But I'm just saying, I've not seen a sovereign AI strategy, if you want to call it this way, which actually makes me feel like you're actually tackling the root cause here, right? And eventually, what I believe here is, I don't think the plan is like every country is gonna like, redevelop the whole tech stack and come up with their own solution. I think eventually what I want, again, maybe I'm a little bit too much kumbaya.

• 42:47Speaker 01

  It would be nice if we can just get together and just say like, here are the standards. Here's how the democratic world is doing AI. We like have these scales, right, to work on this. And like you're not at the whims of somebody just shutting off your AI tomorrow because they just feel like it. I don't think that's great. I don't think that's how we should go about it, right?

• 43:01Azeem Azhar?

  That is super kumbaya. So I'm gonna have to pick that, pick that up. Number one, what did you have for lunch? I want some of that to feel that great. But number two, I mean, the issue that you have there is this sort of original sin of the internet, which is that we never really figured out who was going to ultimately make the rules. And it was fine in the late eighties and early nineties when it could be, you know, IETF and IANA and so on. But now it's a much, much bigger issue. And the decision ultimately needs to rest from the view of a national government or the EU, which means that for that to be cast iron, those rights have to be subsidiarized by the US government in a way that is non-reputable to whoever else is using that. And I actually don't know what that mechanism is anyway.

• 43:46Azeem Azhar?

  And even if that mechanism existed, I'm not sure I can see the political circumstances in the next few years where it's politically tractable. So let's start with that first question, which is what is that mechanism?

• 43:57Speaker 01

  I mean, like, we made progress on these kinds of things, right? There are certain data localization laws. And this at least stops like other governments to literally raid the data center, grab the hard drive and take your data. There's also encryption. So we've got like methods to like to something defend and like there's nothing new. We've been doing this with different types of cloud act initiatives and more, right? So that's already going on. I think for AI we want the same. And ideally, you just wanna be part of the broader AI supply chain. I think there's something we're having these discussions because Europe is just like, where is Europe on the tech stack? I see it all the way below with ASML, right? Where they're producing the machines which produce the chips. But across the L C tech stack, it looks like not that well. I don't think US government can say tomorrow like, oh, actually Taiwan, Netherlands, South Korea, meh, we don't want any of it. Then they got a problem.

• 44:41Speaker 01

  Right? Because these are key allies in the semiconductor supply chain. So I'd be more way more keen for countries to pick like their part in the supply chain, right? This is just how it works. Be it a local community, be it on a global scale, to pick it there and then localize certain things. But I feel like they've got a say or they got potentially some bargaining there if they eventually needed to. And then again, I think just building data centers gives you at least a physical control over this computing power. And then we should just ask the question, which part of the tech stack do you wanna localize? What is the most important? I don't have the answers here.

• 45:11Azeem Azhar?

  I'll make two comments about that, right? So the first is I like what you've said about pick a point where you can be the winner and you can be the best and therefore everyone needs you in the team. It's a little bit like how the primes in the US, you know, build even a hammer in thirty nine different states.

• 45:27Speaker 01

  Yeah.

• 45:27Azeem Azhar?

  So that there's always a congressman fighting for your side and, you know, the wood is done here and the plastic is done there and the screws are done there. You could, you know, argue that, but of course, as you say, three countries right now are part of that, which is the Netherlands, South Korea for memory and Taiwan for the chips. So that that is quite an interesting approach. And you're right, I haven't seen that in any solid AI strategy and you've read more than me. But on the second point about the physical data centers, I think one of the problems there is Moore's Law or Huang's Law, whatever you want to call it. Because essentially, given the price performance improvement of chips every year, on a static amount of CapEx every year, you're sort of adding the same amount of compute as you already had.

• 46:08Azeem Azhar?

  And so even if you have physical data centers and you can't swap out the chips because the chips are sort of physical, they have different cooling requirements, they have to build new data centers for the new families of chips. Even if you do have the data centers, in two years, if you get cut off for whatever reason, in two years, they're going to be anemic, they're not going to be able to run anything and they're going to be irrelevant in that new modern AI economy. So you have to be able to guarantee the consistent supply of uprating of the chips. And it's not just going to be hyperscale data centers, these big ones like Stargate and Colossus, it will be metro scale and edge data centers as well that will need kind of constant replenishment.

• 46:50Azeem Azhar?

  So it feels like that may be an indicator of your commitment to be a customer and therefore create incentives to not be cut off.

• 46:56Speaker 01

  Absolutely.

• 46:57Azeem Azhar?

  But it doesn't feel like it's as strong a defense as the proposal that you have, which is let's become the best at photo resist in the world, which I think exists in Japan right now.

• 47:05Speaker 01

  I mean, two points, let's flip it around. You just said like, well, you build a data center and the next year it's outdated. Nice thing is about you can enter the race pretty late, you can get a good share of compute, right? If I now build a data center with like B two hundreds and Nvidia's newest chip over H one hundreds, here we go, right? It's way easier. And if you look at like these compute shares, and again, like me crunching all these numbers, you just see how like to some people, bolatay there are, right? If you just build like a couple of big data centers, you're something in the lead. You're right, there's no guarantee, next year changes again, but it goes both ways here, right? And then you're right, you need to always upgrade and get the newest chips. This is where the semiconductor supply chain again plays a key role, right? And again, the US government just identified AI chips. Oh, nice, they're like these four countries which play like this important role here. We got like this nice authority, we can just decide where it goes.

• 47:50Speaker 01

  But again, why did we have all of this outcry? For example, like Poland and the diffusion framework not being the one, same with Portugal. It's exactly for this reason, the vibes are off. The vibes are off if you just feel like you're subject to and they can cut you off at any single point in time. And it's a product that we need to replace every five years, right? And this is where I wish for more bilateral and multilateral to have these engagements, right? But I think we need to talk a lot about AI here. I think we found this like across many different industries where we found a way and ideally we can work together.

• 48:19Azeem Azhar?

  That is a positive note, Len, for us to bring this conversation to an end. And I'm so disappointed to do it because I'm really enjoying it. So I hope we will get a chance to do this in person or in the near future. Just have one fun quickfire question for you. Don't overthink it. Who makes a more fun dinner companion? An AI governance person or an AI developer?

• 48:39Speaker 01

  I mean, speaking for me personally, I love the developers, you know? I'm in DC. We got lots of high level talk with little substance. And you know what I love about engineering? There are truths, things either work or they don't. And again, this is my bias as an engineer. So right now, maybe I'm just speaking what I need right now after spending a lot of weeks here in DC and not being in the Bay Area for a long time. I would just love to hang with my AI developer and ask him a bunch of questions.

• 49:00Azeem Azhar?

  Well, you know what? The other thing that has worked has been our conversation this afternoon. Thank you so much for making the time. You have a great weekend and thank you to everybody who tuned in.