@Asianometry & Dylan Patel — How the semiconductor industry actually works

A bonanza on the semiconductor industry and hardware scaling to AGI by the end of the decade. Dylan Patel runs Semianalysis https://www.semianalysis.com/, the leading publication and research firm on AI hardware. Jon Y runs Asianometry https://www.youtube.com/@Asianometry, the world’s best YouTube channel on semiconductors and business history. * What Xi would do if he became scaling pilled * $ 1T+ in datacenter buildout by end of decade Watch on YouTube https://youtu.be/pE3KKUKXcTM. Listen on Apple Podcasts https://podcasts.apple.com/us/podcast/dylan-patel-jon-asianometry-how-the-semiconductor/id1516093381?i=1000671564456, Spotify https://open.spotify.com/episode/6q1XODE2L5bqqBwe7434S7?si=XPbFrc8HRQedPM-tqHAnGQ, or any other podcast platform. Read the full transcript here https://www.dwarkeshpatel.com/p/dylan-jon. Follow me on Twitter https://twitter.com/dwarkesh_sp for updates on future episodes. Sponsors: * Jane Street https://jane-st.co/dwarkesh is looking to hire their next generation of leaders. Their deep learning team is looking for FPGA programmers, CUDA programmers, and ML researchers. To learn more about their full time roles, internship, tech podcast, and upcoming Kaggle competition, go here https://jane-st.co/dwarkesh. * This episode is brought to you by Stripe https://stripe.com/, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue. If you’re interested in advertising on the podcast, check out this page https://www.dwarkeshpatel.com/p/advertise. Timestamps 00:00:00 – Xi's path to AGI https://www.dwarkeshpatel.com/i/149705443/xis-path-to-agi 00:04:20 – Liang Mong Song https://www.dwarkeshpatel.com/i/149705443/liang-mong-song 00:08:25 – How semiconductors get better https://www.dwarkeshpatel.com/i/149705443/how-semiconductors-get-better 00:11:16 – China can centralize compute https://www.dwarkeshpatel.com/i/149705443/china-can-centralize-compute 00:18:50 – Export controls & sanctions https://www.dwarkeshpatel.com/i/149705443/export-controls-and-sanctions 00:32:51 – Huawei's intense culture https://www.dwarkeshpatel.com/i/149705443/huaweis-intense-culture 00:38:51 – Why the semiconductor industry is so stratified https://www.dwarkeshpatel.com/i/149705443/why-the-semiconductor-industry-is-so-stratified 00:40:58 – N2 should not exist https://www.dwarkeshpatel.com/i/149705443/can-we-afford-moores-law 00:45:53 – Taiwan invasion hypothetical https://www.dwarkeshpatel.com/i/149705443/taiwan-invasion-hypothetical 00:49:21 – Mind-boggling complexity of semiconductors https://www.dwarkeshpatel.com/i/149705443/mind-boggling-complexity-of-semiconductors 00:59:13 – Chip architecture design https://www.dwarkeshpatel.com/i/149705443/chip-architecture-design 01:04:36 – Architectures lead to different AI models? China vs. US https://www.dwarkeshpatel.com/i/149705443/architectures-lead-to-different-ai-models-china-vs-us 01:10:12 – Being head of compute at an AI lab https://www.dwarkeshpatel.com/i/149705443/being-head-of-compute-at-an-ai-lab 01:16:24 – Scaling costs and power demand https://www.dwarkeshpatel.com/i/149705443/scaling-costs-and-power-demand 01:37:05 – Are we financing an AI bubble? https://www.dwarkeshpatel.com/i/149705443/are-we-financing-an-ai-bubble 01:50:20 – Starting Asianometry and SemiAnalysis https://www.dwarkeshpatel.com/i/149705443/starting-asianometry-and-semianalysis 02:06:10 – Opportunities in the semiconductor stack https://www.dwarkeshpatel.com/i/149705443/opportunities-in-the-semiconductor-stack This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com https://www.dwarkesh.com?utm_medium=podcast&utm_campaign=CTA_1

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Dwarkesh Patel?

  Today, I'm chatting with Dylan Patel, who runs Semi Analysis, and John, who runs the Asianometry YouTube channel.

• 0:07Dylan Patel?

  Does he have a last name?

• 0:08Dwarkesh Patel?

  No, I do not. No, just kidding. I'm John Y. What's right is it? I'm John Y.

• 0:12Dylan Patel?

  Wait, why is it only one letter?

• 0:15Dwarkesh Patel?

  Because Y is the best letter.

• 0:18Dylan Patel?

  Why is your face covered?

• 0:21Dwarkesh Patel?

  Why not? No, seriously, why is it covered? Because I'm afraid of looking myself get older and fatter over the years.

• 0:30Dwarkesh Patel?

  But seriously, it's like anonymity, right? Anonymity. Okay. Yeah. By the way, so you know what Dylan's middle name is? Actually, no. No, he told me.

• 0:38Dylan Patel?

  What's my father's name?

• 0:40Dwarkesh Patel?

  I'm not gonna say, but I remember.

• 0:41Dylan Patel?

  You could say it. It's fine.

• 0:43Dwarkesh Patel?

  Sanjay?

• 0:45Dylan Patel?

  Yes. What's his middle name?

• 0:46Dwarkesh Patel?

  Sanjay? That's right. Wow! So I'm Dwarkesh Sanjay Patel, he's Dylan Sanjay Patel. It's like, literally, my white name.

• 0:53Dylan Patel?

  Yeah. It's unfortunate my parents decided between my older brother and me to give me a white name. And it could have been Dwarkesh Sanjay. Like, you know how amazing it would have been if we had the same name? Like, butterfly effect and all, it probably would have all turned out the same way, but like...

• 1:09Dwarkesh Patel?

  Maybe it would have been even closer. We would have met each other sooner, you know? Who else is named Dwarkesh Sanjay Patel in the world?

• 1:13Dylan Patel?

  Yeah, yeah, yeah, yeah.

• 1:15Dwarkesh Patel?

  All right, first question. If you were Xi Jinping and you're scaling-pilled, what is it that you do?

• 1:19Dylan Patel?

  Don't answer that question, John. That's bad for AI safety.

• 1:22Speaker 02

  I would basically be contacting every foreigner. I would be contacting every Chinese national with family back home and saying, "I want information. I wanna know your recipes. I wanna know..."

• 1:32Dwarkesh Patel?

  What kind of like AI lab foreigners or hardware foreigners? Honeypotting OpenAI?

• 1:36Speaker 02

  I would basically, like, I... This is totally off-cycle, but like, this is off the reservation. But like, I was doing a video about Yugoslavia's nuclear program. Nuclear weapons program. Started absolutely nothing. One guy from Paris. And then, one guy in Paris, he showed up and he was like, and then he had, who knows what he did? He knows a little bit about making atomic nuclear weapons, but like, he was like, "Okay, well, do I need help?" And then the state's secret police is like, "I would get you everything." And then like, "I shouldn't do that. I was getting you everything." And for like, a span of four years, they basically, they drew up a list, "What do you need? What do you want? What are you gonna do? What is it gonna be for?" And they just, state police just got everything.

• 2:21Speaker 02

  If I was running a country and I needed catch-up on that, that's the sort of thing that I would be doing.

• 2:25Dwarkesh Patel?

  So, okay, let's talk about the espionage. So... What is the most valuable piece of, if you could have this blueprint, like this, this like one megabyte of information, do you want it from TSMC, do you want it from Nvidia, do you want it from OpenAI? What is the first thing you would try to steal?

• 2:43Dylan Patel?

  I mean, I guess you have to like stack every layer, right? And I think like the beautiful thing about AI is because it's growing so freaking fast, every layer is being stressed to some incredible degree. Of course, China has been hacking ASML for over five years and you know, ASML is kind of like, oh, it's fine. The Dutch government's really pissed off, but it's fine. Right? I think it's a, they already have those files, right? In my view. It's just a, it's a very difficult thing to build, right? I think, I think the same applies for like fab recipes, right? They can poach Taiwanese nationals very like, not, not that difficult, right? Because TSMC employees do not make absurd amounts of money. You can just poach them and give them a much better life and they have, right? A lot of SMIC's employees are TSMC, you know, Taiwanese nationals, right?

• 3:28Dylan Patel?

  A lot of the really good ones, high up ones especially, right? And then you go up like the next layers of the stack and it's like, I think, I think, yeah, of course there's tons of model secrets. But then like, you know, how many of those model secrets do you not already have and you just haven't deployed or implemented, you know, organized, right? That's the, that's the one thing I would say is like China just hasn't, they clearly are still not scale-pilled in my view.

• 3:51Dwarkesh Patel?

  So these people are, I don't know, if you could like hire them, it would probably worth a lot to you, right? Because you're building a fab that's worth tens of billions of dollars and this talent is like, they know a lot of shit. How often do they get poached? Do they get poached by like foreign adversaries or do they just get poached by other companies within the same industry but in the same country? And then, yeah, well, like, why doesn't that like sort of drive up their wages?

• 4:16Dylan Patel?

  I think it's because it's, it's very compartmentalized. And I think like back in the 2000s, prior to TSMC, before SMIC got big.

• 4:25Speaker 02

  It was actually much more kind of open, more flat. I think after that, there was like after the Ahmung Song and after all the Samsung issues and after all the the SMIC's rise, when there you literally saw.

• 4:36Dylan Patel?

  I think I think you should tell that story, actually. The the the TSMC guy that went to Samsung and SMIC and all that, I think you should tell that story.

• 4:42Speaker 02

  There are two stories. There's a guy, he ran a semiconductor company in Taiwan called Worldwide Semiconductor. And this guy, Richard Chang, was very religious. I mean, all the TSMC people are pretty religious. But like, he in particular was very fervent and he wanted to bring religion to China. So after he sold his company to TSMC, huge Cooper TSMC, he worked there for about eight or nine months and he was like, all right, I'll go to China. Because back then, there was the relations between China and Taiwan were much more different. And so he goes over there, Shanghai says, we'll give you a bunch of money. And then Richard Chang basically recruits half of like a whole bunch, it's like a conga line of like Taiwanese line, just like they get on the plane, they're flying over. And generally, that's actually a lot of like, a lot of like acceleration points within China's semiconductor industry.

• 5:27Speaker 02

  It's from talent flowing from Taiwan. And then the second thing was like Ahmung Song. Ahmung Song was a, is a nut. And I've met him, I've not met him, I've met people who work with him, and they say he is a nut. He is, he is probably on the spectrum and he's, he does not care about people, he does not care about business, he does not care about anything. He wants to take it to the limit. The only thing, that's the only thing he cares about. He worked for TSMC, literal genius, 300 patents or whatever, 285, goes, works all the way to like the top, top tier. And then one day, uh, he decides, he loses out on some sort of power game within TSMC and gets demoted.

• 6:05Dylan Patel?

  And he was like head of R and D, right, or something?

• 6:06Speaker 02

  He was like one of the top R and D, he was like second or third place.

• 6:09Dylan Patel?

  And it was for the head of R and D position, basically?

• 6:11Speaker 02

  Correct, more of the head of R and D position. He's like, I can't deal with this. And he goes to Samsung and he steals a whole bunch of talent from TSMC, literally, again, conga line, goes and just emails people, say, we will pay. At some point, some of these people were getting paid more than the Samsung chairman, which not really comparable, but like, you know what I mean?

• 6:29Dylan Patel?

  Isn't the Samsung chairman usually like, like part of the family that owns Samsung?

• 6:33Speaker 02

  Correcto.

• 6:34Dylan Patel?

  Okay, yeah, so it's like kind of relevant.

• 6:35Speaker 02

  So it's a bit like, he goes over there and he's like, well, I'm like, we will make Samsung into this monster. We forget everything, forget all of the stuff you've been trying to do, like incremental, toss that out. We are going to the leading edge and that is it. They go to the leading edge, the guys like, they win Apple's business.

• 6:54Dylan Patel?

  They win Apple's business, they win it back from TSMC, or did they win it back from TSMC? They had a portion of the...

• 7:00Speaker 02

  They had a big portion of it, and then TSMC, Morris Tang, is like, at this time was running the company, and he's like, I'm not letting this happen, because that guy, toxic to work for as well, but also goddamn brilliant, and also like very good at motivating people. He's like, we will work literally day or night, sets up what is called the Nightingale Army, where you have, they split a bunch of people and they say, you are working R and D night shift. There is no rest at the TSMC fab. You will go in, there was, as you go in, there'll be a day shift going out. They called it the, it's like, you're burning your liver, because in Taiwan, they said like, if you get old, like, as you work, you're sacrificing your liver. They call it the Liver Buster.

• 7:45Speaker 02

  So they basically did this Nightingale Army for like a year, two years. They finished FinFET, they basically just blow away Samsung. And at the same time, they sue Naomong Song directly for stealing trade secrets. Samsung basically separates from Naomong Song and Naomong Song goes to SMIC.

• 8:06Dylan Patel?

  And, and, and so Samsung like at one point was better than TSMC. And then, yeah, he goes to SMIC and SMIC is now better than, well, or not better, but they caught up rapidly as well after.

• 8:14Speaker 02

  Very rapid. That guy's a genius. That's a guy's a genius. I mean, I, I don't, I don't even know what to say about him. He's like 78 and he's like... Beyond Brilliant does not care about people.

• 8:24Dwarkesh Patel?

  Like, yeah, what is research to make the next process node look like? Is it just a matter of like 100 researchers go in? They do like the next N plus one, then the next morning, the next 100 researchers go in.

• 8:38Speaker 02

  It's experiments. They have a recipe and what they do, every recipe, a TSMC recipe is the culmination of a long, long years of like research, right? It's highly secret and the idea is that you're what you're gonna do is that you go, you look at one particular part of it and you say, experiment, run the experiment. Is it better? Is it not? Is it better or not? Kind of a thing like that.

• 8:59Dylan Patel?

  You're basically, it's, it's, it's, it's multi-variable problem that each, every single tool, sequentially, you're processing the whole thing. You, you turn up knobs up and down on every single tool. You can increase the pressure on this one specific deposition tool or.

• 9:11Dwarkesh Patel?

  And what are you trying to measure? Is it like, does it increase the yield or like, what is it that?

• 9:14Dylan Patel?

  It's not, it's yield, it's performance, it's power. It's not just a one, it's not just better or worse, right? It's a multi-variable search space.

• 9:22Dwarkesh Patel?

  And what do these people know such that they can do this? Is it, they understand the chemistry and physics?

• 9:26Dylan Patel?

  So it's a lot of intuition, but yeah, it's, it's PhDs in chemistry, PhDs in physics, PhDs in EE.

• 9:32Speaker 02

  Brilliant geniuses, people, and they all just.

• 9:34Dylan Patel?

  And they don't even know about like the end chip a lot of times. It's like, oh, I am an etch engineer and all I focus on is how hydrogen fluoride etches this, right? And that's all I know. And like, if I do it at different pressures, if I do it at different temperatures, if I do it with a slightly different recipe of chemicals, it changes everything.

• 9:52Speaker 02

  I remember like, someone told me this when I was speaking, like, how did America lose the ability to do this sort of thing, like etch and hydrofluoric acid or that? I told them like, but he told me basically was like, it's, it's very apprentice, master apprentice. Like you know in Star Wars Sith, there's only one, right? Master apprentice, master apprentice. In, it used to be that there is a master, there's apprentice, and they pass on this secret knowledge. This guy knows nothing but etch, nothing but etch. Over time, the apprentices stopped coming. And then in the end, the apprentices moved to Taiwan. And that's the same way it's still run, like you have the NTU and NTHU, Qinghua University, National Qinghua University. There's a bunch of masters, they teach apprentices and they just pass this secret, sacred knowledge down.

• 10:36Dwarkesh Patel?

  Who are the most AGI-pilled people in the supply chain? Is there anybody?

• 10:40Dylan Patel?

  I gotta have my phone call with Colette right now.

• 10:42Dwarkesh Patel?

  Okay, go for it.

• 10:43Dylan Patel?

  Sorry, sorry.

• 10:44Dwarkesh Patel?

  Could we mention to the podcast that Nvidia is calling Dylan to update him on the earnings call?

• 10:50Dylan Patel?

  Well, it's not exactly that, but...

• 10:52Dwarkesh Patel?

  Go for it, go for it. Dylan is back from his call with Jensen Huang.

• 10:57Dylan Patel?

  It was not with Jensen, Jesus.

• 10:59Dwarkesh Patel?

  What did they tell you, huh? What did they tell you about next year's earnings?

• 11:01Dylan Patel?

  No, it was just color around like a hopper blackwell and like margins. It's like quite boring stuff.

• 11:07Dwarkesh Patel?

  I'm sure.

• 11:08Dylan Patel?

  For most people, I think it's interesting, though.

• 11:10Dwarkesh Patel?

  I guess we could just start talking about Nvidia. You know what?

• 11:12Dylan Patel?

  No, I think we should go back to China. There's like a lot of points there.

• 11:15Dwarkesh Patel?

  All right, we covered the trips themselves. How do they get like the 10 gigawatts data center up? What else do they need?

• 11:21Dylan Patel?

  So I think there is a true like question of how decentralized do you go versus centralized, right? And if you look in the US, right, as far as like labs and such, the, you know, OpenAI, XAI, you know, Anthropic, and then Microsoft having their own effort, Anthropic having their own efforts despite having their partner, and then Meta, and you know, you go down the list. It's like there is quite a decentralization. And then all the startups, like interesting startups that are out there doing stuff. There's quite a decentralization of efforts. Today in China, it is still quite decentralized, right? It's not like Alibaba, Baidu, you are the champions, right? You have like DeepSeek, like, who the hell are you? Does government even support you like doing amazing stuff, right?

• 12:05Dylan Patel?

  If you are Xingping and scale-pilled.

• 12:07Dwarkesh Patel?

  Interesting.

• 12:08Dylan Patel?

  You must now centralize the compute resources, right? Because you have sanctions on how many Nvidia GPUs you can get in now. They're still north of a million a year, right? Even post-October last year sanctions. We still have more than a million H20s and other hopper GPUs getting in through, you know, other means, but legally like the H20s. And then on top of that, you have your domestic chips, right? But that's less than a million chips. So then when you look at it, it's like, oh, well, we're still talking about a million chips. The scale of data centers people are training on today slash over the next six months is 100,000 GPUs.

• 12:43Speaker 02

  Yeah.

• 12:43Dylan Patel?

  Right? OpenAI, XAI, right? These are like quite well documented and others. But in China, they have no individual system of that scale yet, right? So then the question is like, how do we get there? You know, no, no company has had the centralization push to have a cluster that large and train on it yet, at least publicly like well-known. And the best models seem to be from a company that has got like 10,000 GPUs, right? Or 16,000 GPUs, right? So it's not, it's not quite a, quite as centralized as the US companies are. And the US companies are quite decentralized. If you're Xi Jinping and you're scale-pilled, do you just say XYZ company is now in charge?

• 13:23Speaker 02

  Yeah.

• 13:23Dylan Patel?

  And every GPU goes to one place? And then, then you don't have the same issues as the US, right? In the US, we have a big problem with like being able to build big enough data centers, being able to build substations and transformers and all this that are large enough in a dense area. China has no issue with that at all because their supply chain adds like as much power as like half of Europe every year, right? Like or some absurd statistics, right? So they're building transformers, substations, they're building new power plants constantly. So they have no problem with like getting power density. And you go look at like Bitcoin mining, right? Around the Three Gorges Dam, at one point at least, there was like 10 gigawatts of like Bitcoin mining estimated, right?

• 14:05Dylan Patel?

  Which, you know, we're talking about, you know, gigawatt data centers are coming over, you know, 26, 27 in the, or 26 in the US or 27, right? You know, sort of, this is an absurd scale relatively, right? We don't have gigawatt data centers, you know, ready, but like China could just build it in six months, I think. Around the Three Gorges Dam or many other places, right? Because they have, they have the ability to do the substations, they have the, they have the power generation capabilities. Everything can be like done like a flip of a switch, but they haven't done it yet. And then they can centralize the chips like crazy. Right now, oh, oh, a million chips that Nvidia is shipping in Q3 and Q4, the H20. Let's just put them all in this one data center. They just haven't had that centralization effort.

• 14:45Speaker 02

  Well, you can argue that like the more you centralize it, the more you start building this monstrous thing within the industry, you start getting attention to it. And then suddenly. You know, lo and behold, you have a little bit of a, you have a little worm in there suddenly. While you're doing your big training run, oh, this GPU off, oh, this GPU, oh, no, oh, no, oh, no.

• 15:06Dylan Patel?

  I don't know if it's like that easy to hack.

• 15:08Dwarkesh Patel?

  Was that a Chinese accent, by the way?

• 15:10Dylan Patel?

  Just to be clear, John is, is East Asian? He's Chinese?

• 15:12Speaker 02

  I am of East Asian descent.

• 15:14Dylan Patel?

  Half Taiwanese, half Chinese?

• 15:16Speaker 02

  Right, that is right.

• 15:16Dylan Patel?

  But like, I think, I think, I don't know if that's like as simple as that to like, uh, cause, cause training systems are like fire, like they're, they're water, is it water gated? Fire walled? What, what is it called? Not firewall. I don't know. There's a word for that where they're not like.

• 15:30Speaker 02

  They're air gapped.

• 15:31Dylan Patel?

  Air gapped.

• 15:32Speaker 02

  I think they're Chinese wall.

• 15:33Dwarkesh Patel?

  You're going through like the, all the like the four elements.

• 15:36Dylan Patel?

  Dirt, fire, like dirt, it's protected.

• 15:38Dwarkesh Patel?

  Water, fire.

• 15:39Dylan Patel?

  If you're huge, big, you're scale pilled. Fuck the air benders, fuck the fire benders, you know? We got the avatar, right? Like, you have to build the avatar. Okay. Um, I think, I think that's possible. Um, the question is like, does that slow down your research? Do you like crush, like, crack people like DeepSeek? Uh, who are like clearly like not being, you know, influenced by the government and put some like idiot like, you know?

• 16:06Speaker 02

  Idiot bureaucrat at the top, suddenly he's all thinking about like, you know, all these politics and he's trying to deal with all these different things. Suddenly you have a single point of failure and that's a, that's, that's bad.

• 16:19Dylan Patel?

  But, I mean, in the, in the, on the flip side, right? Like, there is like obviously immense gains from being centralized because of the scaling laws. Right? And then, and then the flip side is compute efficiency is obviously going to be hurt because you can't do, you can't experiment and like have different people lead and try their efforts as much if you're less centralized, uh, more, more centralized. So it's like, there is a balancing act there.

• 16:39Dwarkesh Patel?

  The, the fact that they can centralize, I, I didn't think about this, but that is actually like, uh, cause you know, even if America as a whole is getting millions of GPUs a year. The fact that any one company is only getting hundreds of thousands or less means that there's no one person who can do a single training run as big in America as if like China as a whole decides to do one together. Um, the, the 10 gigawatts you mentioned near the three words. Just damn, is it like literally like, how widespread is it like a state? Is it like one wire like how?

• 17:10Dylan Patel?

  I think like between not just the dam itself, but like also all of the coal, there's some nuclear reactors there, I believe as well.

• 17:18Dylan Patel?

  Between all of and and like renewables like solar and wind, between all of that in that region, there is an absurd amount of concentrated power that could be built. I don't think it's like, I'm not saying it's like one button, but it's like, hey, within X mile radius, right? Is more more of like the correct way to frame it. And that's how the that's how the labs are also framing it, right? Like, I think in the US.

• 17:40Dwarkesh Patel?

  If they started right now, like, how long does it take to build? The biggest, the biggest AI data center that in the world?

• 17:47Dylan Patel?

  You know, actually, I think, I think the other thing is like, could we notice it? I don't think so, because the amount of like factories that are being spun up, the amount of other construction, manufacturing, etc., that's being built. A gigawatt is actually like a drop in the bucket, right? Like a gigawatt is not a lot of power. 10 gigawatts is not an absurd amount of power, right? It's okay. Yes, it's like hundreds of thousands of homes, right? Well, yeah, millions of people, but it's like, you got 1.4 billion people, you got like, most of the world's like extremely energy intensive like refining and like, you know, rare earth refining and all these manufacturing industries are here. It would be very easy to hide it. It'd be very easy to just like shut down like, I think the largest aluminum mill in the world is there and it's like, it's like north of 5 gigawatts alone.

• 18:31Dylan Patel?

  It's like, oh, what, what, could we tell if they stop making aluminum there and instead started like making, you know, AIs there or making AI there? Like, I don't know if we could tell, right? Because they could also just easily spawn like 10 other aluminum mills, make up for the production and be fine, right? So like, there's many ways for them to hide compute as well.

• 18:49Dwarkesh Patel?

  To the extent that you could just take out a 5 gigawatt aluminum refining center and like build a giant data center there, then I guess the way to control Chinese AI has to be the chips. Because like everything else, they, so like, how do you like? Just like walk me through how many chips do they have now? How many will they have in the future? What will the like, how many is that in comparison to US and the rest of the world?

• 19:11Dylan Patel?

  Yeah, so in the world, I mean, the world we live in is they are not restricted at all on like the physical infrastructure side of things in terms of power, data centers, etc., because their supply chain is built for that, right? And it's pretty easy to pivot that, whereas the US adds so little power each year and Europe loses power every year, the the Western sort of industry for power is non-existent in comparison, right? But on the flip side is quote unquote Western, including Taiwan. Manufacture chip manufacturing is way, way, way, way, way larger than China's, especially on leading edge where China theoretically has, you know, depending on the way you look at it, either zero or a very small percentage share, right? And so there you have you have wafer, you have you have equipment, wafer manufacturing,

• 19:54Dylan Patel?

  and then you have advanced packaging capacity, right? And where the US can control China, right? So advanced packaging capacity is kind of a shot because the vast majority, the largest advanced packaging company in the world was Hong Kong headquartered, they just moved to Singapore, but like, that's effectively like, you know, in a realm where the US can't sanction it, right? A majority of these other companies are in similar places, right? So advanced packaging capacity is very hard, right? Advanced packaging is useful for stacking memory, stacking chips on co-OS, right? Things like that. And then then the step down is wafer fabrication. There is immense capability to restrict China there, and despite the US making some sanctions,

• 20:35Dylan Patel?

  China in the most recent quarters was like 48% of ASML's revenue, right? So, you know, and and like 45% of like applied materials and you just go down the list. So it's like, obviously, it's not being controlled that effectively, but it could be on the equipment side of things. The chip side of things is actually being controlled quite effectively, I think, right? Like, yes, there is like shipping GPUs through Singapore and Malaysia and and other countries in Asia to China, but you know, the amount you can smuggle is quite small. And then the sanctions have limited the chip performance to a point where it's like, you know, this is actually kind of fair, but there is a problem with how everything is restricted, right?

• 21:14Dylan Patel?

  Because you want to be able to restrict China from building their own domestic chip manufacturing industry. That is better than what we ship them. Um, you want to prevent them from having chips that are better than what we have. And then, or, and then you want to prevent them from having AIs better. The ultimate goal being, you know, and if you read the restrictions, like very clear, it's about AI. Um, even in 2022, which is amazing, like at least the Commerce Department was kind of AI pilled. It was like, is, is you want to restrict them from having AIs worse than us, right? So, starting on the right end, it's like, okay, well, if you want to restrict them from having better AIs than us, you have to restrict chips. Okay. If you want to restrict them from having chips, uh, you have to let them have at least some level of chip that the West also, that is good, better than what they can build internally. Um, but currently the, the restrictions are flipped the other way, right?

• 21:58Dylan Patel?

  They can build better chips in China than, uh, we restrict them in terms of chips that Nvidia or AMD or an Intel can sell to China. Um, and so there's sort of a problem there in terms of the equipment that is shipped can be used to build chips that are better than what the Western companies can actually ship them.

• 22:15Dwarkesh Patel?

  Uh, John Dillon seems to think the export controls are kind of a failure. Do you, do you, do you agree with him or?

• 22:19Speaker 02

  That is a very interesting question because I think it's like.

• 22:24Dwarkesh Patel?

  Why, thank you.

• 22:24Speaker 02

  Like, what do you?

• 22:27Dylan Patel?

  Dwarkesh, you're so good.

• 22:29Speaker 02

  Yeah, Dwarkesh, you're the best. I think it's, I think failure is a tough word to say because I think it's like, what are we trying to achieve, right? Like, and so they're talking about AI, right?

• 22:39Dwarkesh Patel?

  Yeah.

• 22:39Speaker 02

  When you do sanctions like that, it's, you need like such a deep knowledge of the technologies.

• 22:46Dylan Patel?

  You know, just taking lithography, right? If your goal is to restrict China from building chips and you just like boil it down to like, hey, lithography is 30% of making a chip, so are 25%. Cool. Let's, let's sanction lithography. Okay. Where do we draw the line? Okay. Let me ask, let me ask, let me figure out what, where the line is. And if I'm a bureaucrat, if I'm a lawyer at the Commerce Department or what have you. Well, obviously, I'm gonna go talk to ASML and ASML is gonna tell me, this is the line. Because they know like, hey, well, you know, this, this, this is, you know, there's like some blending over.

• 23:15Speaker 02

  There's like, they're, they're like looking at like, what's gonna cost us the most money, right?

• 23:18Dylan Patel?

  And then they are constantly say like, if you restrict us, then China will have their own industry, right? And, and the way I like to look at it is like, chip manufacturing is like, like 3D chess or like, you know, a massive jigsaw puzzle in that if you take away one piece, China can be like, oh, that's the piece, let's put it in. Right? And currently this export restrictions, year by year by year, they keep updating them. Ever since like 2018 or so, 19, right? When Trump started and now Biden's, you know, accelerated them. They've been like, they haven't just like, take a bat to the table and like break it, right? Like, it's like, let's take one jigsaw puzzle out, walk away, oh shit, let's take two more out. Oh shit, right? Like, you know, it's like, instead if they like, they, you either have to go kind of like

• 24:02Dylan Patel?

  full bat to the frickin' like table slash wall, or, or chill out, right? Like, and like, you know, let them, let them do whatever they want. Cause the alternative is, everything is focused on this thing and they've make that. And then now when you take out another two pieces, like, well, I have my domestic industry for this. I can also now make a domestic industry for these. Like, you go deeper into the tech tree or what have you.

• 24:23Speaker 02

  It's a very, it's art, right? In a sense that there are technologies out there that can compensate. Like, if you believe, the belief that lithography is a linchpin within the system is, it's not exactly true. Right? At some point, if you keep pulling, keep pulling a thread, other things will start developing to kind of close that loop. And like, I think it's, it's, it is, that's why I say it's an art, right? I don't think you can stop Chinese semiconductor industry, the semiconductor industry from progressing. I think that's basically impossible. So the question is, the Chinese government believes in the primacy of semiconductor manufacturing.

• 25:01Speaker 02

  They used, they've believed it for a long time, but now they really believe it, right?

• 25:05Dylan Patel?

  To some extent, the sanctions have made China believe in the importance of the semiconductor industry more than anything else.

• 25:12Dwarkesh Patel?

  So from an AI perspective, what's the point of export controls then? Because even if like, if they're going to be able to get these, like if you're like concerned about AI and they're going to be able to build-

• 25:21Dylan Patel?

  Well, they're not centralized though, right? So that's the big question is are they centralized? And then also, you know, there's the belief, I don't really, I'm not sure if I really believe it, but like, you know, prior podcasts, there have been people who talked about nationalization, right? In which case, okay, now you're talking about-

• 25:35Dwarkesh Patel?

  Why are you referring to this ambiguously?

• 25:37Dylan Patel?

  Well, I think there's a couple-

• 25:38Dwarkesh Patel?

  My opponent.

• 25:40Dylan Patel?

  I love my opponent, you know? No, but I think there have been a couple where people have talked about nationalization, right? But like, if you have, you know, nationalization, then all of a sudden you aggregate all the flops. It's like, no, there's no fucking way, right?

• 25:51Dwarkesh Patel?

  Yeah.

• 25:52Dylan Patel?

  China can be centralized enough to compete with each individual US lab. They could have just as many flops in 25 and 26 if they decided they were scale-pilled, right? Just from foreign chips for individual model.

• 26:04Dwarkesh Patel?

  And like in 2026, they can train a 1E27, like they can release a 1E27 model by 2026.

• 26:11Dylan Patel?

  Yeah, and then a 28 model, you know, 1E28 model in the works, right? Like, they totally could just with foreign chip supply, right? Just a question of centralization. Then the question is like, do you have as much innovation and compute efficiency wins or what have you get developed when you centralize? Or does like Anthropic and OpenAI and XAI and Google like all develop things? And then like, secrets kind of shift a little bit in between each other and all that? Like, you know, you end up with that being a better outcome in the long term versus like the nationalization of the US, right? Uh, if that's possible and like, or, you know, and what happens there. But China could absolutely have it in 26, 27 if they just have the desire to. And that's just from foreign chips, right? And then domestic chips are the other question, right?

• 26:52Dylan Patel?

  600,000 of the Ascend 910B, which is roughly like 400 teraflops or so.

• 27:01Dylan Patel?

  You know, so, so if they put them all in one cluster, they could have a bigger model than any of the labs next year. Right? I have no clue where all the 7910Bs are going, right? But I mean, well, there's like rumors about like some, they are being divvied up between the like major Alibaba, ByteDance, Baidu, etc. And next year, more than a million. And it's possible that they actually do have, you know, 1E30 before the US. Because data center is not as big of an issue. 10 gigawatt data center is going to be, I don't think anyone is even trying to build that today in the US. Like even out to 27, 28. Really, they're focusing on like linking many data centers together. So there's a possibility that like, hey, come 2028, 2029, China can have more flops delivered to a single model.

• 27:45Dylan Patel?

  Even ignoring sort of, even once the centralization question is solved, right? Because that's clearly not happening today for either party. And I would bet if AI is like as important as, you know, you and I believe, that they will centralize sooner than the West does. So there is a possibility, right?

• 28:04Dwarkesh Patel?

  Yeah. It seems like a big question then is how much could SMIC either increase the product, like increase the amount of wafers? Like how many more wafers could they make? And how many of those wafers could be dedicated to the night? Because I assume there's other things they want to do with these semiconductors.

• 28:17Dylan Patel?

  Yeah, so there's like two points, parts there too, right? Like, so the way the US has sanctioned SMIC is really like stupid kind of. Is that in that they've like sanctioned a specific spot rather than the entire company. And so therefore, right, SMIC is still buying a ton of tools that can be used for their 7 nanometer and their, you know, call it 5.5 nanometer process or 6 nanometer process for the 910C, which releases later this year, right? They can build as much of that as long as it's not in Shanghai, right? And Shanghai has anywhere from 45 to 50 high-end immersion lithography tools is what's like believed by intelligence as well as like many other folks.

• 28:58Dylan Patel?

  That roughly gives them as much as 60,000 wafers a month of 7 nanometer, but they also make their 14 nanometer in that fab, right? And so the belief is that they actually only have about like 25 to 35,000 of 7 nanometer capacity. Wafers a month, right? Doing the math, right? Of the chip die size and all these things, because Huawei also uses chiplets and stuff, so they can get away with using less leading edge wafers, but then their yields are bad. You can roughly say, you know, something like 50 to 80 good chips per wafer with their with their bad yield, right? With their bad yield.

• 29:35Dwarkesh Patel?

  Why do they have bad yield?

• 29:36Dylan Patel?

  Because it's hard, right? You know, you're...

• 29:38Speaker 02

  Even if it was like, you know, everyone knows the number, right? Like, it's a thousand steps, even if you're 99% for each, like 98 or 99%, like in the end, you'll still get a 40% yield overall.

• 29:48Dwarkesh Patel?

  Interesting.

• 29:49Dylan Patel?

  I think it's like, even with like 99, if I think it's like, I think, I think it's, if it's six sigma of like, or of like perfection, and you have your 10,000 plus steps, you end up with like, yield is still dog shit by the end, right? Like, yeah.

• 30:01Speaker 02

  That is a scientific measure, dog shit percent.

• 30:04Dylan Patel?

  Yeah. Yeah, yeah, as a multiplicative effect, right?

• 30:08Dwarkesh Patel?

  Yeah.

• 30:09Dylan Patel?

  So yields are bad because they have hands tied behind their back, right? Like, A, they are not getting to use EUV, whereas on 7 nanometer, Intel never used EUV, but TSMC eventually started using EUV. Initially, they used DUV, right?

• 30:25Dwarkesh Patel?

  Doesn't that mean the expert control succeeded? Because that, they have bad yield because they have to use like...

• 30:30Speaker 02

  Success is, again, they still are determined. Success is mean they stop. They're not stopping.

• 30:36Dylan Patel?

  Going back to the yield question, right? Like, oh, theoretically 60,000 wafers a month, times 50 to 100 dies per wafer with yielded yielded dies? Holy shit, that's, that's millions of GPUs, right? Now, what are they doing with most of their wafers? They still have not become skillpilled, so they're still throwing them out like, let's make 200 million Huawei phones, right? Like, oh, okay, cool, I don't care, right? Like, as, as the West, you don't care as much, even though like, Western companies will get screwed, like Qualcomm and like, you know, and MediaTek Taiwanese companies. So, so obviously, there's that. And the same applies to the US, but when you, when you flip to like...

• 31:12Dylan Patel?

  Sorry, I don't fucking know what I was gonna say.

• 31:16Speaker 02

  Nailed it!

• 31:18Dwarkesh Patel?

  We're keeping this in.

• 31:19Dylan Patel?

  That's fine, that's fine, that's fine.

• 31:21Dwarkesh Patel?

  Hey, everybody. I am super excited to introduce our new sponsors, Jane Street. They're one of the world's most successful trading firms. I have a bunch of friends who either work there now or have worked there in the past. And I have very good things to say about those friends, and those friends have very good things to say about Jane Street. Jane Street is currently looking to hire its next generation of leaders. As I'm sure you've noticed, recent developments in AI have totally changed what's possible in trading. They've noticed this too, and they've stacked a scrappy, chaotic new team with tens of millions of dollars of GPUs to discover signal that nobody else in the world can find. Most new hires have no background in trading or finance.

• 32:03Dwarkesh Patel?

  Instead, they come from math, CS, physics, and other technical fields. Of particular relevance to this episode, their deep learning team is hiring CUDA programmers, FPGA programmers, and ML researchers. Go to jainestreet.com/dwarkech to learn more. And now, back to Dylan and John. 2026, if they're centralized, they can have as big training runs as any one US company.

• 32:31Dylan Patel?

  Oh, the reason why I was bringing up Shanghai? They're building 7-nanometer capacity in Beijing, they're building 5-nanometer capacity in Beijing, but the US government doesn't care. And they're importing dozens of tools into Beijing. And they're saying to the US government and ASML, "This is for 28-nanometer, obviously, right? This is not bad." And then, obviously, you know, like, in the background, "Yeah, we're making 5-nanometer here, right?"

• 32:51Dwarkesh Patel?

  Are they doing it because they believe in AI or because they want to make Huawei phones?

• 32:55Dylan Patel?

  You know, Huawei was the largest TSMC customer for like a few quarters, actually, before they got sanctioned. Huawei makes most of the telecom equipment in the world, right? You know, phones, of course, modems, but of course, accelerators, networking equipment. You know, you go down the whole, like, video surveillance chips, right? Like, you kind of like go through the whole gambit.

• 33:13Dwarkesh Patel?

  Yeah.

• 33:13Dylan Patel?

  A lot of that could use 7- and 5-nanometer. Do you think the dominance of Huawei is actually a bad thing for the rest of the Chinese tech industry? I think Huawei is so fucking cracked that, like, it's hard to say that, right? Like, Huawei outcompetes Western firms regularly with two hands tied behind their back.

• 33:32Dylan Patel?

  Like, you know, like, what the hell is Nokia and Sony Ericsson like trash, right? Like compared to Huawei and Huawei is not allowed to ship cell to like European companies or American companies and they don't have TSMC and yet they still destroy them, right? And same applies to like the new phone, right? It's like, oh, it's like as good as like a year old Qualcomm phone on a process node that's equivalent to like four years old, right? Or three years old. So it's like, wait, so they actually out engineered us with a worse process node? You know, so it's like, oh, wow. Okay. Like, you know, Huawei is Huawei is like crazy cracked.

• 34:06Speaker 02

  Why do you think that culture comes from?

• 34:09Dylan Patel?

  The military because it's the PLA.

• 34:10Speaker 02

  It is the we it is generally seen as an arm of the PLA. But like, how do you square that with the fact that sometimes the PLA seems to mess stuff up?

• 34:22Dylan Patel?

  Oh, like filling water in rockets? I don't know if that was true. I'm denying, I'm not denying. There is, there is like that, like, like, like crazy conspiracy, not conspiracy. It was like, you don't know what the hell to believe in China, especially as a not Chinese person, but like.

• 34:34Speaker 02

  Nobody knows, even Chinese people don't know what's going on in China.

• 34:37Dylan Patel?

  There's like, you know, like all sorts of stuff like, oh, they're filling water in their rockets. Clearly they're like incompetence. Like, look, if I'm the Chinese military, I want the Western world to like believe I'm completely incompetent because one day I can just like destroy the fuck out of everything, right? With all these hypersonic missiles and all this shit, right? Like drones and like, no, no, no, we're filling water in our missiles. These are all fake. We don't actually have 100,000 missiles that we manufacture in a facility that's like super hyper advanced and Raytheon is stupid as shit because they can't make, you know, missiles nearly as fast, right? Like, I think like, that's also like a flip side is like, how much false propaganda is there, right? Because there's a lot of like, no, SMIC could never, SMIC could never, they don't have the best tools, blah, blah, blah. And then it's like, motherfucker, they just shipped 60 million phones last year with this

• 35:20Dylan Patel?

  chip that performs only one year worse than like what Qualcomm has. It's like, proof is in the pudding, right? Like, you know, there's, there's a lot of like cope, if you will.

• 35:30Speaker 02

  I just wonder where it comes from. I do really do just wonder where that culture comes from. Like, there's something crazy about them where they're kind of like, everything they touch, they seem to succeed in. And like, I kind of wonder why.

• 35:40Dylan Patel?

  They're making cars. I wonder if it's going on there. I think, I like, if, like supposedly, like, if we kind of imagine, like, historically, like, you think they're getting something from somewhere?

• 35:50Dwarkesh Patel?

  What do you mean?

• 35:51Dylan Patel?

  Espionage, you mean? Yeah. Well, obviously. Like, East Germany and the Soviet industry was basically, was just, it was like a conveyor belt of, like, secrets coming in, and they're just, use that to run everything. But the Soviets were never good at it. They could never mass produce it.

• 36:03Dwarkesh Patel?

  How would espionage explain how they can make things with different processes?

• 36:07Dylan Patel?

  I don't think it's just espionage. I think they're just, like, literally cracked. It has to be something else. They have the espionage, without a doubt, right? Like, ASML has been known to have been hacked a dozen times.

• 36:16Dwarkesh Patel?

  Right, right.

• 36:17Dylan Patel?

  Or at least a few times, right? Um, and they've been known to have people sued who made it to China with a bunch of documents, right? Not just ASML, but every fucking company in the supply chain. Cisco code was literally in, like, early Huawei, like routers and stuff, right? Like, you go down the list, it's like, everything is, but then it's like, no, architecturally, the Ascent 910B looks nothing like a GPU, it looks nothing like a TPU. It is like its own independent thing. Sure, they probably learned some things from some places, but like, it is just like, they're good at engineering. It's 996. Like, wherever that culture come from, they, they, they do good. They do very good.

• 36:48Dwarkesh Patel?

  I know. Well, another thing I'm curious about is, like, yeah, where their culture come from, but like, how does it stay there? Because, uh, with American firms, or any other firm, um, you can have a company that's very good, but over time it gets worse, right? Like Intel, or many others. Um, I guess Huawei just isn't that old of a company, but, uh, like, it, it's hard to, like, be a big company and, like, stay good.

• 37:07Dylan Patel?

  That is true. I think it's like, I think, like, but, I think a lot, a word that I hear a lot in, with regards to Huawei is a struggle, right? And China has a culture of, like, the Communist Party is, like, really big on struggle. I think, like, Huawei, in the sense, they sort of brought that culture into some, into their, in the way they do it, like you said before, right? They, they, they go crazy because they think that in five years that they're gonna fight the United States, and those, like, literally, everything they do, every second, is like, their country depends on it, right? It's, it's like, it's the Andy Grovian mindset, right? Like, shout out to, like, the base to Intel, but like, only the paranoid survive, right? Like, paranoid Western companies do well. Why did, why did Google, like, really screw the pooch on a lot of stuff? And then why are they, like, resurging kind of now?

• 37:50Dylan Patel?

  It's because they got paranoid as hell, right? But they weren't paranoid for a while. Um, if Huawei is just constantly paranoid about, like, the external world, and like, oh, fuck, we're gonna die, oh, fuck, like, you know, they're gonna beat us, uh, our country depends on it, we're gonna get the best people from the entire country, that are like, you know, the best at whatever they do. And tell them, you will, if you do not succeed...

• 38:12Speaker 02

  You will die, or like our country, your family will die, your family will be enslaved and everything, like it'll be terrible.

• 38:17Dylan Patel?

  By the evil western pigs, right?

• 38:19Speaker 02

  Evil western, yeah, capital, or not capital, they don't believe in, they don't say that anymore. But it's like, you know, everyone is against China, China is being, it's being defiled, right? And like, they're saying like, if you, that is all on you, bro. Like, if you can't do that, then like you.

• 38:35Dylan Patel?

  If you can't get that fucking radio to be slightly less noisy and like transmit like 5% more data, we have a great palace fire all over again. The British are coming and they will steal all the, all the, all the trinkets and everything, like that's on you.

• 38:49Dwarkesh Patel?

  Uh huh. Um, uh, why isn't there more vertical integration in this semiconductor industry? Well, like, why are there like, this subcomponent requires this other subcomponent from this other company, which requires a component from another company, like, why, why is more of it not done in-house?

• 39:00Dylan Patel?

  The way to look at it today is it's super, super stratified and every industry has anywhere from one to three competitors. And pretty much the most competitive it gets is like 70% share, 25% share, 5% share. Um, in any layer of like manufacturing chips, anything, anything, chemicals, different types of chips. But it used to, used to be vertically integrated, like in.

• 39:22Speaker 02

  Well, the very beginning it was integrated, right?

• 39:23Dwarkesh Patel?

  And where did that stop?

• 39:25Speaker 02

  What happened was, you know, the funniest thing was like, you know, you had companies that used to do it all in the one. And then suddenly, sometimes a guy would be like, I hate this, I think I know, I know how to do better. Spins off, does his own thing, starts his company, goes back to his old company, says, I can sell you a product that's better, right? And that's the beginning of what we called the semiconductor manufacturing and equipment industry. Like basically.

• 39:46Dylan Patel?

  Like in the seventies, right? Like everyone made their own equipment.

• 39:48Speaker 02

  Sixties and seventies, like the spinoff, all these people, and then what happened was that the companies that accepted, you know, these outside products and equipment got better stuff. They did better. Like you can talk about a whole bunch, like there are companies that were totally vertically integrated in semiconductor manufacturing for decades. And they are, they're still good, but they're nowhere near competitive.

• 40:07Dwarkesh Patel?

  One thing I'm confused about is like the actual foundries themselves, there's like fewer and fewer of them every year, right? So there's like more, maybe more companies overall, but like the, like the final people like who, uh, make the, make the wafers, there's less and less. And then it's interesting in a way, it's similar to like the AI Foundation models, where you need to use like the revenues from like a previous model in order or like your market share to like fund the next round of ever more expensive development.

• 40:37Speaker 02

  When TSMC launched the foundry industry, right? And when they started, there was a whole wave of like Asian companies that funded semiconductor foundries of their own. You had Malaysia with Solterra, you have Singapore with Charter, you had, there was one, there's one semiconductor where I talked about earlier, there's one from Hong Kong.

• 40:55Dylan Patel?

  Bunch in Japan.

• 40:56Speaker 02

  Bunch in Japan, like they all sort of did this thing, right? And I think the thing was that when you're going to leading edge, when the thing is that like it got harder and harder, which means that you had to aggregate more demand from all the customers to fund the next node, right? So technically, in the sense that what it's kind of do is aggregating all this money, all this profit to kind of fund this next node to the point where now like there's no room in the market for an N2 or N3, like there technically you could argue that economically, you can make an argument that like N2 is a monstrosity that doesn't make sense economically and which should not exist in some ways without the immense single concentrated spend of like five players in the market.

• 41:39Dylan Patel?

  I'm sorry to like completely derail you, but like there's this video where it's like there's an unholy concoction of meat slurry.

• 41:46Speaker 02

  Yes.

• 41:47Dwarkesh Patel?

  What?

• 41:49Dylan Patel?

  Sorry, there's like a video that's like, ham is disgusting. It's an unholy concoction of like meat with no bones or collagen. And like, I don't know, like he was like, the way he was describing two nanometer is kind of like that, right?

• 42:00Speaker 02

  It's like the guy who pumps his right arm so much and he's like super muscular. The human body was not meant to be so muscular.

• 42:08Dwarkesh Patel?

  What's the point? Like why is two nanometer not justified?

• 42:11Speaker 02

  I'm not saying N2 is like N2 specifically, but I say N2 as a concept. The next node should technically like right now.

• 42:20Speaker 02

  There is a, there will come a point where economically, the next node will not be possible. Like, at all, right?

• 42:24Dylan Patel?

  Unless, unless more, you know, technology spawned, like AI now makes, you know, one nanometer or whatever.

• 42:29Speaker 02

  Yeah, there was a long period of time.

• 42:31Dylan Patel?

  Yeah, yeah. Viable, right? So, so like, right before AI spawned. And makes it viable and what's, as in like, it makes it worth it? So, every, every two years, you get a shrink, right? Yeah. Uh, like clockwork, Moore's law. Um, and then, five nanometer happened. It took three years, holy shit. And then, three nanometer happened, it took three, or no, sorry, is it three nanometer or five? Took three years. Holy shit, like, is Moore's law dead? Right? Like, cause TSMC didn't, and then what did Apple do? Even on the third year of three, of, of, uh, or sorry, once three nanometer finally launched, they still only, Apple only moved half of the iPhone volume to three nanometer. So this is like, now they did a fourth year of I, of five nanometer for a big chunk of iPhones. Right? And it's like, oh, is the mobile industry petering out?

• 43:14Dylan Patel?

  Then you look at two nanometer and it's like gonna be a similar, like, very difficult thing for the, for the industry to pay for this, right? Apple, of course, they have, you know, because they get to make the phone, they have so much profit, they can funnel into like, more and more expensive chips, but finally, like, that was, that was running out, right? It was to, how economically viable is two nanometer just for one player? TSMC, you know, ignore Intel, ignore Samsung. Just, you know, because Samsung is paying for it with memory, not with their actual profit. And then, Intel's paying it from it from their former CPU monopoly.

• 43:45Speaker 02

  Private equity money.

• 43:46Dylan Patel?

  And now, and now private equity money and debt and subsidies.

• 43:50Speaker 02

  Laid off people's salaries, you know?

• 43:51Dylan Patel?

  But like, anyways, like, you know, there's, there's a strong argument that like, funding the next node would not be economically viable anymore if it weren't for AI taking off. Right? And then generating all this humongous demand for the most leading edge chip.

• 44:05Dwarkesh Patel?

  So, and how, how much, uh, how big is the difference between seven to five to three nanometer? Like, is it, is it, like, does it, is it a huge deal in terms of like, who can build the biggest cluster?

• 44:13Dylan Patel?

  So, there's the, there's this simplistic argument that like, oh, moving a process node only saves me X percent in power, right? And, and that, that has been petering out, right? Uh, you know, when you move from like ninety nanometer to, uh, eighty something, right? Or seventy something, right? It was like, it was, you got two X, right? Denard scaling. was still intact, right? But now, when you move from 5 nanometer, 3 nanometer, first of all, you don't double density. SRAM doesn't scale at all. Logic does scale, but it's like 30%. So all in all, you only saved like 20% in power per transistor. But because of like data locality and movement of data, you actually get a much larger improvement in power efficiency by moving to the next node than just the individual transistor's power efficiency benefit. Because, you know, for example, you're multiplying a matrix that's like, you know,

• 44:58Dylan Patel?

  8,000 by 8,000 by 8,000. And then like you can't fit that all on one chip. But if you could fit more and more, you have to move off chip less. You have to go to memory less, et cetera, right? So the data locality helps a lot, too. But, you know, the AI really, really, really wants new processed nodes because of A, you know, power used is a lot less now.

• 45:17Dwarkesh Patel?

  Yeah.

• 45:17Dylan Patel?

  Uh, higher density, higher performance, of course. But the big deal is like, well, if I have a gigawatt data center, I can now, how much more flops can I get? If I have 2 gigawatt data center, how much more flops can I get? If I have a 10 gigawatt data center, how much more flops can I get, right? Like you, you look at the scaling and it's like, well, no, everyone needs to go to the most recent process node as soon as possible.

• 45:34Dwarkesh Patel?

  I, I want to ask the normie question. For like everybody's, I want to phrase it that way. Okay, I want to ask a question that's like.

• 45:41Dylan Patel?

  Fucking normie. Not for you nerds. I, I think, I think John and I could communicate to, to like the point where you wouldn't even know what the fuck we're talking about.

• 45:52Dwarkesh Patel?

  Okay, the, uh, suppose Taiwan is invaded or Taiwan has an earthquake. Nothing is shipped out of Taiwan in, from now on. What happens next? The rest of the world, how would it feel its impact? A day in, a weekend, a month in, a year in?

• 46:06Dylan Patel?

  I mean, it's, it's terrible thing. It's a terrible thing to talk about. I think it's like, can you just say it's all terrible? Everything's terrible because it's not just like leading edge, leading edge. People are focused on leading edge, but there's a lot of trailing edge stuff that like people depend on every day. I mean, we all worry about AI. The reality is you're not going to get your fridge. You're not going to get your cars. You're not going to get everything. It's terrible. And then there's the human part of it, right? It's all terrible. Can we, like, it's, it's depressing.

• 46:32Dwarkesh Patel?

  I think.

• 46:32Dylan Patel?

  And I live there. Yeah. I think day one, market crashes a lot, right? You're going to think about like, I think, I think the big, like, big six, six, six biggest companies, magnificent seven, whatever the heck it's called, are like 60, 75% of the S and P 500 and their entire business relies on chips, right? Google, Microsoft, Apple, Nvidia. You know, you go down the list, right? They're, they're Meta, right? They all entirely rely on AI.

• 46:58Dylan Patel?

  And you would have a tech reset, like extremely insane tech reset, by the way, right? Like, so market would crash, a week, a day in, a couple of weeks in, right? Like, people are preparing now, people are like, oh shit, like, let's start building fabs, fuck all the environmental stuff, like, war's probably happening. Yeah, yeah. But, but like, the supply chain is trying to like, figure out what the hell to do to refix it. But, six months in? Dis supply of chips for making new cars, gone, or sequestered to make military shit, right? You can no longer make cars. And we don't even know how to make non semiconductor induced cars, right? Like, this unholy concoction with all these, like, chips, right? You, uh... Cars like 40% chips now. Like, it's just chips on, in the tire. There's like, there's like 2,000 plus chips, right?

• 47:42Dylan Patel?

  Every Tesla door handle has like four chips in it, it's like, what the fuck? Like, why? Like, like, but like, it's like, it's like shitty, like, microcontrollers and stuff, but like, there's like 2,000 plus chips, even in an ICE vehicle, like, internal combustion engine vehicle, right? And every engine has dozens of dozens of chips, right? Anyways, this all shuts down, because, not all of the production, there's some in Europe, there's some in the US, there's some in Japan, there's some in Singapore... They're gonna bring in a guy to work in, on Saturday, until 4. Yeah, yeah, I mean, yeah. So, so you have like, TSMC always builds new fabs, that old fab, they tweak production up a little bit more and more, and new designs move to the next, next, next node, and, and old stuff fills in the old nodes, right? So, you know, ever since TSMC has been the most important player, and not just TSMC,

• 48:26Dylan Patel?

  there's UMC there, there's PSMC there, there's a number of other companies there, Taiwan's share of like, total manufacturing has grown every single process node. So, in like 130 nanometer, there's a lot, and, including like, many chips from like, Texas Instruments, or Analog Devices, or like, NXP, like, all these companies, 100% of it is manufactured in Taiwan, right? By, you know, either TSMC or UMC or whatever, but then you like, step forward and forward and forward, right? Like, 28 nanometer, like, 80% of the world's production of 28 nanometer is in Taiwan. Oh, fuck, right? Like, you know, and everything in 28 nanometer is like, what's made on 28 nanometer today? Tons of microcontrollers and stuff, but also like, every display driver I see, like, cool, like, even if I could make my Mac chip, I can't make the chip that drives the display. Like, you know, you just go down the list, like, everything, no fridges, no, no automobiles,

• 49:11Dylan Patel?

  no, no weed whackers, because that shit has, my toothbrush has fucking Bluetooth in it, right? Like, why? I don't know, but like, you know, there's like, so many things that like, just like, poof, we're tech reset.

• 49:20Dwarkesh Patel?

  We were supposed to do this interview like, many months ago, and then I like, kept like, delaying, because I'm like, ah, I don't understand any of this shit. But like, it is like a very difficult thing to understand, but I feel like with AI, it's like, it's not that.

• 49:32Dylan Patel?

  No, you've just spent time.

• 49:33Dwarkesh Patel?

  Sure, but I also feel like it's like less complex. It feels like it's a kind of thing where like, in an amateur kind of way, you can like, you know, pick up what's going on in the field. In this field, like, the thing I'm curious about is like, how, how does one learn the layers of the stack? Because the layers of the stack are like, there's not just the papers online. You can't just like look up the tutorial on how the transformer works or whatever. It's like, it's like many layers of really difficult shit.

• 49:57Dylan Patel?

  There are like eighteen year olds who are just cracked at AI, right, already, right? And like, there's high school dropouts that get like jobs at OpenAI. This existed in the past, right? Pat Gelsinger, current CEO of Intel, went straight to work. He was, he like grew up in the Amish area of Pennsylvania and he went straight to work at Intel, right? Because he's just cracked, right? That is not possible in semiconductors today. You can't even get like a job at like a tool company without like, at least like a freaking masters in chemistry, right? And probably a PhD, right? Like, like, of the like seventy-five thousand TSMC workers, it's like fifty thousand have a PhD or something insane, right? It's like, okay, this is like, there's like some, there's like a next level amount of like, how specialized everything's gotten.

• 50:39Dylan Patel?

  Whereas today, like, you can take like, you know, Sholto, you know, he, when did he start working on AI? Not that long ago. Not to say anything bad about Sholto. No, no, no, but he's cracked. He's like Omega cracked at like what he does. What he does, you could pick him up and drop him into another part of the AI stack. First of all, he understands it already. And then, second of all, he could probably become cracked at that too, right? Whereas that is not the case in semiconductors, right? You, one, you like specialize like crazy. Two, you can't just pick it up.

• 51:09Dylan Patel?

  You know, like Sholto, I think, what did he say? He like just started like.

• 51:11Dwarkesh Patel?

  He was a consultant in McKinsey and at like night, he would like read papers about robotics. And like run experiments and whatever.

• 51:18Dylan Patel?

  Yeah, and then, and then like he like was like, like people noticed, he was like, who the hell is this guy? And why is he posting this? Like, I thought everyone who knew about this was at Google already, right? It's like, come to Google, right? That can't happen in semiconductors, right? Like, it's just not like conducive, like, it's not possible, right? One, archive is like a free thing. The paper publishing industry is like abhorrent everywhere else. And you just like cannot download IEEE papers or like SPIE papers or like other organizations. And then, two, at least up until like.

• 51:50Speaker 02

  Late 2022, or really early 2023, in the case of Google, right? I think, what, the Palm inference paper? Up until the Palm inference paper, before that, all the good, best stuff was just posted on the internet. After that, you know, it's kind of a little bit clamping down by the labs. But there's also still all these other companies making innovations in the public. That, and, and like, what is state of the art is public. That is not the case in semiconductors. Semiconductors have been shut down since 1960s, 1970s, basically. I mean, like, it's kind of crazy how little information has been formally transmitted from one, one country to another. Like, the last time you could really think of this was like 19, maybe the Samsung era, right?

• 52:26Dwarkesh Patel?

  So then how do you guys keep up with it?

• 52:27Speaker 02

  Well, we don't know it. I don't personally, I don't think I know it. I don't, I mean, I-

• 52:31Dwarkesh Patel?

  You don't know it?

• 52:32Speaker 02

  It's crazy because like, like, there is a guy, there's like, I spoke to one guy, he's like a PhD in etch or something. The world, one of the top people in etch, and he's like, "Man, you really know, like, lithography, right?" And I'm just like, "I don't feel like I know lithography." But then you've talked to people who know lithography, they're like, "You've done pretty good work in packaging, right? Nobody knows anything."

• 52:51Dwarkesh Patel?

  They all have gelman amnesia.

• 52:53Speaker 02

  They're all, they're all in this, like, single well, right? They're, they're, they're digging deep. They're digging deep for what they're getting at. But they, but, you know, they don't know the other stuff well enough. And in some ways, I mean, nobody knows the whole stack. Nobody knows the whole stack.

• 53:05Dylan Patel?

  The, the, like, the stratification of just like manufacturing is absurd. Like, the tool people don't even know exactly what Intel and TSMC do in production, and vice versa. They don't know exactly how the tool is optimized like this. And it's like, how many different types of tools there are? Dozens. And each of those has like, an entire tree of like, all the things that we've built, all the things we've invented, all the things that we continue to iterate upon, and then like, here's the breakthrough innovation that happens every few years in it, too.

• 53:31Dwarkesh Patel?

  So if, if that's the case, if like, nobody knows the whole stack, then how does the industry coordinate to be like, um, uh, you know, in five, in two years, we want to go to the next process, which has gate all around, and for that, we need X tools, and X technologies developed by whatever.

• 53:46Speaker 02

  That's really fascinating. It's a fascinating social kind of phenomenon, right? You can feel it. I went to Europe the, earlier this year, Dylan was like, had allergies. But like, I was like, talking to those American people- It's like gossip. It's gossip. You start feeling the, you start feeling people are coalescing around like a something, right? Early on, we used to have like Semitec where people, all these American companies came together and talked and they came and they hammered out, right? But Semitec in reality was dominated by a single company, right? And but then, you know, nowadays is a little more dispersed, right? You feel, you feel like it's like, it's like, it's like, it's a blue moon arising kind of thing. Like they are going towards something, they know it.

• 54:29Speaker 02

  And then suddenly the whole industry is like, this is it. Let's do it.

• 54:33Dylan Patel?

  I think it's like God came and proclaimed it. We will shrink density 2X every two years. Gordon Moore, he made an observation and then like it didn't go nowhere. It went way further than he ever expected because it was like, oh, there's line of sight to get to here and here. I'm like, and he predicted like seven, eight years out, like multiple orders of magnitude of increases in transistors. And it came true, but then by then the entire industry was like, this is obviously true. This is the word of God and every engineer in the entire industry, tens of millions of people, like literally, this is what they were driven to do. Now, not every single engineer didn't believe it, but like people were like, yes, to hit the next shrink, we must do this, this, this, right? And this is the optimizations we make.

• 55:12Dylan Patel?

  And then you have this abstrification, every single layer and abstraction layers, every single layer through the entire stack to where people. It's, it's, it's an unholy concoction. I mean, you keep saying this word, but like you, no one knows what's going on because there's an abstraction layer between every single layer. And on this layer, the people below you and the people above you know what's going on. And then like beyond that, it's like, okay, I can like under, try to understand, but like not really like.

• 55:38Dwarkesh Patel?

  But, but I guess that doesn't answer the question of like, uh, when IRDS or whatever, I don't know, was it 10, 20 years ago? I, I watched your video about it where they're like, we are EUV is the net is like, this is, we're going to do EUV instead of the other thing. And this is the path forward. If, how do they do that if they don't have the whole sort of picture of like different constraints, different trade offs, different blah, blah, blah?

• 55:59Speaker 02

  They kind of, they argue it out. They get together and they talk and they argue. And basically at some point, a guy somewhere says, I think we can move forward with this.

• 56:09Dylan Patel?

  Semiconductors are so siloed and the data and knowledge within each layer is A, not documented online at all.

• 56:16Speaker 02

  No documentation.

• 56:18Dylan Patel?

  Because it's all siloed within companies. Um, B, it is, there's a lot of human element to it because a lot of the knowledge like as John was saying is like apprentice master apprentice master type of uh knowledge or I've been doing this for 30 years and there's an un- an amazing amount of intuition on what to do just when you see something. Um, to where like AI can't just learn semiconductors like that. But at the same time, there is a massive amount of talent shortage and ability to move forward on things, right? So like the technology used on like, like most of the like equipment in semiconductor tool fabs runs on like Windows XP. Right? Like the each tool has like a Windows XP server on it.

• 56:59Dylan Patel?

  Or like, you know, like all the chip design tools like have like Centos, Centos like version six. Right? And like, that's old as hell, right? So like, there's like so many like areas where like, why is this so far behind? At the same time, it's like so like hyper optimized. It's like the, the tech stack is so broken in that sense. They're afraid to touch it. They're afraid to touch it. Yeah, because it's an unholy amalgamation. It's unholy, it should not be work. It should not work. This thing should not work. It's literally a miracle. Uh, so you have all the abstraction layers, but then it's like, one is there's a lot of breakthrough innovation that can happen now stretching across abstraction layers. But two is because there's so much inherent knowledge in each individual one. What if I can just experiment and test at a thousand x velocity or a hundred thousand x velocity?

• 57:43Dylan Patel?

  And, and so some examples of where this is already like shown true is some of Nvidia's uh AI layout tools, right? And, and Google as well, like laying out the circuits within a small blob of the chip with AI. Um, some of these like RL design things, some of the, there's a lot of like various like simulation things. Design or is that manufacturing? Um, it's all design, right? Most of it's design. Manufacturing has not really seen much of this yet, although there is start, it's starting to come in. Inverse lithography, maybe. Yeah, ILT and say, maybe. I don't know if it's AI. That's not AI. Yeah. Uh, anyways, like, there's like tremendous opportunity to um bring breakthrough innovation. Simply because there is so many like layers where things are unoptimized, right?

• 58:25Dylan Patel?

  So you see like all these like, oh, single digit mid, you know, low uh double digit like advantages just from like RL techniques from like. AlphaGo type stuff like or like not from AlphaGo but like like five, six, seven, eight year old RL techniques being brought in but like generative AI being brought in could like really revolutionize the industry, you know, although there's a massive data problem.

• 58:47Dwarkesh Patel?

  So and can you give those, can you give the possibilities here in numbers in terms of maybe like a flop per dollar or whatever the relevant thing here is like how much do you expect in the future to come from process node improvements, how much from just like how the hardware is designed because of AI, if you like had to, we're talking specifically for like GPUs.

• 59:08Dylan Patel?

  Yeah.

• 59:09Dwarkesh Patel?

  Like if you had to disaggregate future improvements.

• 59:11Dylan Patel?

  I think, I think, you know, it's first, it's important to state that semiconductor manufacturing and design is the largest search space of any problem that humans do because it is the most complicated industry that anything that humans do. And so, you know, when you think about it, right, there's, there's a 1E10, 1E11, right, 100 billion transistors.

• 59:33Dwarkesh Patel?

  Yeah.

• 59:34Dylan Patel?

  On, on leading edge chips, right, Blackwell has two 20 billion transistors or something like that. So what is, and those are just on off switches, and then think about every permutation of putting those together, contact ground, et cetera, drain source, blah, blah, blah, with wires, right, there's 15 metal layers, right, connecting every single transistor in every possible arrangement. This is a search space that is literally almost infinite, right, you could like, the search space is much larger than any other search space that humans know.

• 1:00:01Dwarkesh Patel?

  And what is the nature of the search, like what are you trying to optimize over?

• 1:00:03Dylan Patel?

  Well, useful compute, right, what is, you know, if you're, if the, if the goal is optimize intelligence per picajoule, right, and, and intelligence is some nebulous nature of like the, what the model architecture is.

• 1:00:16Dwarkesh Patel?

  Yeah.

• 1:00:17Dylan Patel?

  But and then, and then picajoule is like a unit of energy, right, how do you optimize that? So there's humongous innovations possible in architecture, right, because vast majority of the power on a H100 does not go to compute. And there are more efficient like. Compute, you know, ALUs or Ethnological Unit like designs, right? But even then, the vast majority of the power doesn't go there, right? The vast majority of the power goes to moving data around, right? And then when you look at what is the movement of data? It's either networking or memory. You know, you have, you have a humongous amount of movement relative to compute and a humongous amount of power consumption relative to compute.

• 1:00:59Dylan Patel?

  And so the, so how can you minimize that data movement and then maximize the compute? There are 100x gains from architecture. Even if we like literally stop shrinking, I think we could have 100x gains from architectural advancements. Over what time period? The, the question is how much can we advance the architecture, right? The, the challenge, the other challenge is like the number of people designing chips has not necessarily grown in a long time, right? Um, yeah, like company to company it shifts, but like within like the semiconductor industry in the US and the US makes, you know, designs the vast majority of leading edge chips, the number of people designing chips has not grown much. Um, what has happened is the output per individual has soared because of EDA, Electronic Design Assistance Tooling, right?

• 1:01:43Dylan Patel?

  Now, this is all still like classical tooling. There's just a little bit of inkling of AI in there yet, right? What, what happens when we bring this in is the question and, and how you can solve this search space somehow with humans and AI working together to optimize this so it's not most of the power is data movement and then the logic, the, the compute is actually very small to flip side. Um, the compute is first of all, compute can get like 100x more efficient just with like design changes and then you could minimize that data movement massively, right? So you can get a humongous gain in efficiency just from architecture itself. And then process node helps you innovate that there, right? And, uh, power delivery helps you innovate that. Uh, system design, chip to chip networking helps you innovate that, right?

• 1:02:26Dylan Patel?

  Like memory technologies, there's so much innovation there and there's so many different vectors of innovation that people are pursuing simultaneously, uh, to where like Nvidia gen to gen to gen will do more than 2x performance per dollar. Uh, I think that's very clear and then like hyperscalers are. Probably gonna try and shoot above that, but we'll see if they can execute.

• 1:02:46Dwarkesh Patel?

  There's like two narratives you can tell here of how this happens. One is that these AI companies who are training the foundation models, who understand the trade offs of like how much is the marginal increase in compute versus memory worth to them? And what trade offs do they want between different kinds of memory? They understand this, and so therefore the accelerators they build, they can make these sort of trade offs in a way that's like most optimal. Or and also design like the architecture of the the model itself in a way that

• 1:03:19Dwarkesh Patel?

  reflects like what are the hardware trade offs? Another is Nvidia, because it has like, I don't know how this works, presumably they have some sort of like know how. Like they're accumulating all this like knowledge about how to better design this architecture and like also better search tools or so on.

• 1:03:36Dwarkesh Patel?

  Who has basically like better mote here in terms of will Nvidia keep getting better at design? Getting this 100x improvement or will it be like OpenAI and Microsoft and Amazon and Anthropic? Or designing their accelerators will keep getting better at like designing the accelerator?

• 1:03:52Dylan Patel?

  I think that there's a few vectors to go here, right? One is you mentioned, and I think it's important to note, is that hardware has a huge influence on the model architecture that's optimal. And so it's not a one way street that better chip equals, you know, the optimal model for Google to run on TPUs, given a given amount of dollars, a given amount of compute, is different architecturally than what it is for OpenAI with Nvidia's stuff, right? It is like absolutely different. And then like even down to like networking decisions that different companies do and data center design decisions that people do. The optimal, like if you were to say, you know, X amount of compute of TPU versus GPU, compute optimally, what is the best thing? You'll diverge in what the architecture is. And I think that's important to know, right?

• 1:04:34Dwarkesh Patel?

  Can I ask about that real quick? The so earlier we're talking about how China has the H20s or B20s? And there there's like much less compute per memory bandwidth and like the amount of memory, right? Does that mean that Chinese models will actually have like very different? Architecture and characteristics than American models in the future.

• 1:04:55Dylan Patel?

  So you can take this to like a very like large conclude like leap and it's like oh, you know, neuromorphic computing or whatever is like the optimal path and that looks very different than like what a transformer does, right? Or you could take it to like a simple thing, which is like the level of sparsity and like coarse grains, sparsity, I like experts and all this sort of stuff. The arrangement of what exactly the attention mechanism is because there are a lot of tweaks. It's not just like pure transformer attention, right? Or like, hey, demod, like how wide versus tall the model is, right? That's like very important, like demod versus, you know, number of layers, right? These are all like things that like would be different, like and I, and like, I know they're different between like, say, a Google and an OpenAI and what is optimal.

• 1:05:37Dylan Patel?

  But what really, it really starts to get like, hey, if you were limited on a number of different things, like, like China invests humongously in compute and memory. You know, which is like basically the memory cell is directly coupled or is the, the compute cell, right? So these are like things that like China's investing hugely and you go to conferences like, oh, there's 20 papers from Chinese companies slash universities about compute and memory. Or like, you know, hey, like because the flop limitation is here, maybe Nvidia pumps up the on-chip memory and like changes the architecture because they, they still stand to benefit tens of billions of dollars by selling chips to China, right? Today, it's just like neutered American chips, right? Uh, neutered chips that go to the US, but like it'll start to diverge more and more

• 1:06:20Dylan Patel?

  architecturally because they'd be stupid not to make chips for China, right? And Huawei, obviously, again, like has like their constraints, right? Like where are they limited on memory? Oh, they have a lot of networking capabilities and they could move to like certain optical, like networking technologies directly onto the chip much sooner than we could, right? Because that is what's optimal for them within their search base of solutions, right? Because this whole area is like blocked off.

• 1:06:44Speaker 02

  It's kind of really interesting to see, to think about like the development of how Chinese AI models will differ from American AI models because of the, because of these changes or these.

• 1:06:54Dylan Patel?

  And it applies to use cases, it applies to data, right? Like, American models are very important about like, let me learn from you, right? Let me be able to use you directly as a random consumer, right? That is not the case for Chinese model, I assume, right? Uh, because there's probably very different use cases for them. Uh, China's crushes the West at video and image recognition, right? At ICML, like Albert Gu at, you know, of Cartesian, like state space models, like every single Chinese person was like, can I take a selfie with you? Man was harassed. In the US, like you see Albert and he's like, it's awesome. He invented state space models, but it's not like state space models are like, like here. But that's because state space models potentially have like a huge advantage in like video and image and audio, which is like stuff that China does more of.

• 1:07:37Dylan Patel?

  And that is further along and has better capabilities in, right? So it's like.

• 1:07:40Speaker 02

  Because of all the surveillance cameras there.

• 1:07:42Dylan Patel?

  Sorry?

• 1:07:43Speaker 02

  Because of all the surveillance cameras there.

• 1:07:44Dylan Patel?

  Yeah, that's the, the quiet part out loud, right? Uh, but like, there's already divergence in like capabilities there, right? Like, you know, you look at image recognition, China like destroys American companies, right? Um, on that, right? Cause, cause the surveillance, you have like this divergence in tech tree and like people can like, start to design different architectures within the constraints you're given.

• 1:08:03Speaker 02

  Yeah, yeah.

• 1:08:03Dylan Patel?

  Um, and, and everyone has constraints, but the constraints different companies have are even different, right? Um, and so like, Google's constraints have shown them that they built, they built a genuinely different architecture. But now if you look at like Blackwell and then what's like said about TPV six, right? They're, I'm not going to say they're like converging, but they are getting a little bit closer in terms of like, how big is the mat mole unit size? And like some of the like topology and like world size of like the scale up versus scale out network. Like there is some like convergence slightly, like not saying they're similar yet, but like already they're starting to. But then there's different architectures that people could go down and path. So you see stuff like from all these startups that are trying to go down different tech trees, because maybe that'll work. Uh, but there's a self fulfilling prophecy here too, right?

• 1:08:48Dylan Patel?

  All the research is in transformers that are very high arithmetic intensity, because the hardware we have is very high arithmetic intensity. Uh, and transformers run really well on GPUs and TPUs and like, you sort of have a self fulfilling prophecy. If all of a sudden you have an architecture which is. Theoretically, it's way better, but you can get only like half of the like usable flops out of the mo- out of your chip. It's worthless because even if it's 30%, you know, compute efficiency win, it took twice, i- i- it's half as fast on the chip, right? So there's all sorts of like trade offs and like self fulfilling prophecies of what do, what path do people go down?

• 1:09:20Dwarkesh Patel?

  John and Dylan have talked a lot in this episode about how stupefyingly complex the global semiconductor supply chain is. The only thing in the world that approaches this level of complexity is the Byzantine web of global payments. You're stitching together legacy tech stacks and regulations that differ in every jurisdiction. In Japan, for example, a lot of people pay for online purchases by taking a code to their corner store and punching it into a kiosk. Stripe abstracts all this complexity away from businesses. You can offer customers whatever payment experience they're most likely to use wherever they are in the world. And Stripe is how I invoice advertisers for this very podcast.

• 1:10:03Dwarkesh Patel?

  I doubt that they're punching in codes at a kiosk in Japan, but if they are, Stripe will handle it. Anyways, you can head to Stripe.com to learn more. If you were made head of compute of a new AI lab, if like SSI came to you, the Ilya Sescover new lab, and they're like, Dylan, we give you $1 billion, you are head of compute. Like, help us get on the map, we can compete with the frontier labs. What is your first step?

• 1:10:28Dylan Patel?

  Okay, so the constraints are you're a US/Israeli firm, because that's what SSI is, right? And your researchers are in the US and Israel. You probably can't build data centers in Israel because power is expensive as hell and it's probably like risky, maybe, I don't know. So still in the US, most likely. Most of the researchers are here, so, or a lot of them are in the US, right? Like, Paul or whatever. So I guess you need a significant chunk of compute. Obviously, the like, the whole pitch is you're gonna make some research breakthrough that's like compute efficiency win, data efficiency win, whatever it is. You're gonna make some breakthrough, but you need compute to get there, right? Because your GPUs per researcher is your research velocity, right?

• 1:11:12Dylan Patel?

  Obviously, like, data centers are very tapped out, right? Not in terms of tapped out, but like, every new data center that's coming up, most of them have been sold, which has led people like Elon to go through this, like, insane thing in Memphis, right? I'm just trying to like, I'm just trying to, I'm just trying to square the circle, yeah.

• 1:11:26Dwarkesh Patel?

  On that question, I kid you not, in my group house, like, group chat, there like, there have been two separate people who have been like, I have a cluster of H100s and I have like, a long lease on them, but I don't, like, I'm trying to get, sell them off. Is it like a buyer's market right now? Because it does seem like people are trying to get rid of them.

• 1:11:45Dylan Patel?

  So I think like, for the Ilya question, is like, a cluster of like, 256 GPUs, or even 4K GPUs, is kind of, it's kind of cope, right? It's not enough, right? Yes, you're gonna make compute efficiency wins, but with a billion dollars, you probably just want the biggest cluster in one individual spot.

• 1:11:59Dwarkesh Patel?

  Sure.

• 1:12:00Dylan Patel?

  And so like, small amounts of GPUs, probably not like, you know, possible to use, right? Like, for them, right? Like, and that's what most of the sales are, right? Like, you go and look at like, GPU list, or like, VAST, or like, Foundry, like, or 100 different GPU resellers, the cluster sizes are small. Now, is it a, is it a buyer's market? Yeah, last year, you would buy H100s for like, $4 or $3, like, if you, you know.

• 1:12:26Dwarkesh Patel?

  An hour?

• 1:12:26Dylan Patel?

  An hour, right? For shorter term or midterm deals, right? Now, it's like, if you want a six-month deal, you could get like, $2.15 or less, right? Like, and like, the natural cost, if I, if I have a data center, right? And I'm paying like, standard data center pricing, to purchase the GPUs and deploy them, is like, $1.40, and then you add on the debt, because I probably took debt to buy the GPUs, or cost, equity cost of capital, gets up to like, $1.70 or something, right? And so you see deals that are like, the good deals, right? Like, Microsoft renting from Core Weaver, like, $1.90 to $2, right? So people are getting closer and closer to like, there's still a lot of profit, right? Because the natural rate, even after debt and all, this is like, $1.70. So like, there's still a lot of profit when people are selling in the low 2s, like GPU companies, people are deploying them, but it is a buyer's market in a sense

• 1:13:11Dylan Patel?

  that it's gotten a lot cheaper, but cost of compute is gonna continue to tank, right? Because it's like, sort of like, I don't remember the exact name of the law, but it's effectively Moore's Law, right? Every two years, the cost of transistors halved, and yet, the industry grew. Right? Every, every six months or three months, the cost of, of intelligence. You know, like OpenAI and GPT, GPT four, uh, what, February 2023? Right? A hundred and twenty dollars per million tokens or something like that was roughly the cost. And now it's like 10? Yeah. Right? It's like, it's like the cost of intelligence is tanking, partially because of compute. Um, partially because the model's compute efficiency wins, right? I think that's a trend we'll see, and then that's going to drive adoption as you scale up and get, make it cheaper and scale up and make it cheaper. Right, right.

• 1:13:55Dylan Patel?

  Uh, anyways, what you're saying, if you're a head of computer of SSI. Okay, head of computer of SSI. That was very intense. There's, there's obviously no free data center lunch, right? In terms of, you know, and, and you can just, you know, take that based on like the data we, we, we have shows that there's no lunch, free lunch per se. Like immediately today, you need the compute, um, for a large cluster size or even six months out, right? There's some, but like not a huge amount because of what X did, right? XAI is like, oh shit, we're going to go like, we're going to go, um, buy a Memphis factory, uh, put a bunch of like generators outside, like mobile generators usually reserved for like natural disasters, a Tesla battery pack, drive as much power as we can from the grid,

• 1:14:35Dylan Patel?

  tap the natural gas line that's going to the natural gas plant, like two miles away, the gigawatt natural gas plant, like just like, send it and like get a cluster built as fast as possible. Now you're running a hundred K GPs, right? I know. And that cost, that cost about five billion, right? Four billion, right? Not, not, not, not one billion. So the scale that SSI has is much smaller, by the way, right? Um, so, so their size of cluster will be, you know, maybe one third or one fourth of the size, right? So now you're talking about 25 to 32 K cluster, right? There, you still don't have that, right? No one is willing to rent you a 32 K cluster today, no matter how much money you have, right? Even if you had more than a billion dollars. So you now, it, it makes the most sense to build your own cluster one, uh, instead of renting it,

• 1:15:17Dylan Patel?

  or get a very close relationship like a, uh, OpenAI, Microsoft with CoreWeave, or OpenAI, Microsoft with Oracle slash Crusoe. Um, the next step is Bitcoin, right? Um. So OpenAI has a data center in Texas, right? Or it's going to be their data center. It's like they kind of contracted all that. CoreWeave, there is a 300-megawatt natural gas plant onsite powering these crypto-mining data centers from the company called Core Scientific. And so they're just converting that. There's a lot of conversion, but like the power's already there, the power infrastructure's already there. So it's really about like converting it, getting it ready to be water-cooled, all that sort of stuff, and convert it to 100,000 GB 200 cluster. And they have a number of those going up across the country.

• 1:16:01Dylan Patel?

  But that's also like tapped out to some extent because Nvidia's doing the same thing in Plano, Texas for a 32,000 GPU cluster that they're building.

• 1:16:10Dwarkesh Patel?

  Nvidia's doing that?

• 1:16:11Dylan Patel?

  Well, they're going through partners, right? Because this is the other interesting thing is the big tech companies can't do crazy shit like Elon did.

• 1:16:18Dwarkesh Patel?

  Why?

• 1:16:18Dylan Patel?

  ESG.

• 1:16:21Dwarkesh Patel?

  Oh, interesting.

• 1:16:22Dylan Patel?

  They can't just do crazy shit like, because this-

• 1:16:23Dwarkesh Patel?

  Actually, do you expect Microsoft and Google and whoever to like drop their net-zero commitments as the scaling picture intensifies?

• 1:16:31Dylan Patel?

  Yeah, yeah. So like this, this like, this like, what XAI is doing, right, is like, it's not, it's not that polluting, you know, on the scheme of things, but it's like, you have 14 mobile generators and you're just burning natural gas onsite on these like mobile generators that sit on trucks, right? And then you have like power directly two miles down the road. There's no unequivocal way to say any of the power is because two miles down the road is a natural gas plant as well, right? There's no way to say this is like green. You go to the CoreWeave thing, it's a natural gas plant, is literally onsite from Core Scientific and all that, right? And then the data centers around it are horrendously inefficient, right? There's this metric called PUE, which is basically how much power is brought in versus how much gets delivered to the chips, right? And like the hyperscalers, because they're so efficient or whatever, right?

• 1:17:16Dylan Patel?

  Their PUE is like 1.1 or lower, right? I.e. if you get a gigawatt in, 900 megawatts or more gets delivered to chips, right? Not wasted on cooling and all these other things. This, this like Core Scientific one is going to be like 1.5, 1.6, i.e. even if I have 300 megawatts of generation onsite, I only deliver like 180, 200 megawatts to the chips.

• 1:17:39Dwarkesh Patel?

  How fast solar is getting cheaper and also the fact that like, you know, how the reason solar is difficult elsewhere is like, you know, you're like, you gotta like power the homes at night. Here, I guess it's like theoretically possible to like, uh, figure out, you know, only like run the clusters in the day or something.

• 1:17:56Dylan Patel?

  Absolutely not. That is, that's not possible.

• 1:17:59Dwarkesh Patel?

  Because, because it's so expensive to have these GPUs?

• 1:18:01Dylan Patel?

  Yeah. So, so like, when you look at the power cost of a large cluster, it's trivial in, and to some extent, right? Like, um, you know, like the, the meme that like, oh, you know, you can't build a data center in Europe or East Asia because the power is expensive. That's not really relevant. What's the, or power is so cheap in China and the US, that's where the only places you can build data centers. That's not really the real reason. It's the ability to generate new power for these activities is why it's really difficult. Um, and the economic regulation around that, but the, the real thing is like, if you look at the cost of ownership of a GP, of, of an H100, let's just say, you gave me, you know, a billion dollars and I already have a data center, I already have all this stuff. I'm paying regular rates for the data centers. I'm not paying through the nose or anything, paying regular rates for power, not paying through the nose. Power is sub 15% of the cost.

• 1:18:47Dylan Patel?

  Um, and it's sub 10% of the cost actually, right? The, the biggest, like 75 to 80% of the cost is just the servers, right? And this is on a, like a multi-year, including debt financing, including cost of operation, all that, right? Like when you do a TCO, total cost of ownership, like it's like 80% is the GPUs, 10% is the data center, 10% of the power, rough, rough numbers, right? So it's like kind of irrelevant, right? Whether or not you like, like how expensive the power is, right?

• 1:19:12Dwarkesh Patel?

  Yeah.

• 1:19:12Dylan Patel?

  You'd rather do what Taiwan does, right? When like power, like what do they do when, when there was droughts, right? They like, like force people to not shower.

• 1:19:21Dwarkesh Patel?

  They basically reroute the power from, when there was a, when there was a power shortage in Taiwan, they basically rerouted power from the residential.

• 1:19:27Dylan Patel?

  And this will happen in a capitalistic society as well, most likely because like, fuck you, like why, you're not going to pay X dollars per kilowatt hour? Because to me, the marginal cost of power is irrelevant. Really, it's all about the GPU cost and the ability to get the power. I don't want to turn it off eight hours a day.

• 1:19:42Dwarkesh Patel?

  Maybe let's discuss what would happen if the training regime changes and if it doesn't change. So like you could imagine that the training regime becomes much more paralizable, where it's like about like coming up with some sort of like search or synthetic, like most of the compute for training is used to come up with synthetic data or do some kind of search and that can happen across a wide area. In that world, how fast could we scale? Like, just like, let's go through the numbers on like year after year. And then suppose it actually has to be, you would know more than me, but like suppose it has to be the current regime. And like, just explain what that would mean in terms of like how distributed that would have to be. And then how plausible it is to get clusters of certain sizes over the next two years?

• 1:20:24Dylan Patel?

  I think, I think it like is not too difficult for Ilya's company to get a cluster of like 32K in like of, of Blackwell.

• 1:20:32Dwarkesh Patel?

  Forget about Ilya's company.

• 1:20:33Dylan Patel?

  Okay, okay, fair enough, fair enough.

• 1:20:36Dwarkesh Patel?

  Like 2025, 2026, 2027.

• 1:20:38Dylan Patel?

  2025, 2026, there's, before I like talk about like the US, I think it's like important to note that there's like a gigawatt plus of data center capacity in Malaysia next year now. That's like mostly ByteDance. But like, there's like, you know, and power wise, there's like, there's the humongous damming of the Nile in Ethiopia and the country uses like one third of the power that that dam generates. So there's like a ton of power there to like.

• 1:20:59Dwarkesh Patel?

  How much power does that dam generate?

• 1:21:00Dylan Patel?

  Like, it's like over a gigawatt. And the country consumes like 400 megawatts or something trivial.

• 1:21:05Dwarkesh Patel?

  And is like, are people bidding for that power?

• 1:21:07Dylan Patel?

  I think people just don't think they can build a data center in fucking Ethiopia.

• 1:21:11Dwarkesh Patel?

  Why not?

• 1:21:11Dylan Patel?

  I don't think the dam is filled yet, is it? No, I mean, they have to like, the, the dam could generate that power. They just don't.

• 1:21:16Dwarkesh Patel?

  Oh, got it.

• 1:21:17Dylan Patel?

  Right? Like, there's a little bit more equipment required, but that's like not too hard. Why don't they?

• 1:21:22Dwarkesh Patel?

  Yeah.

• 1:21:24Dylan Patel?

  I think there's like, like true security risks, right? If you're China or if you're the US lab, like to build a fucking data center with all your IP in fucking Ethiopia. Like, you want AGI to be in Ethiopia? Like, you want it to be that accessible? Like, people you can't even monitor, like, like being the technicians in the fucking data center or whatever, right? Or like powering the, the data center, all these things. Like, there's so many like. You know, things you could do to like, you could just destroy every GPU in a data center if you want, if you just like fuck with the grid. Right? Like, pretty, pretty like easily, I think. People talk a lot about it in the Middle East. There's a hundred K A G B two hundred cluster going up in the Middle East, right? Um, and the US like, there's like clearly like stuff the US is doing, right? Like, uh, the, you know, um, G forty two is the UAE data center company, cloud company.

• 1:22:09Dylan Patel?

  Their CEO is a Chinese national, or not a Chinese, he's Chinese, basically Chinese allegiance, but, uh, open, I, I think OpenAI wanted to use a data center from them, but instead like, the US forced Microsoft to like, I feel like this is what happened is forced Microsoft to like do a deal with them, um, so that G forty two has a hundred K GPU cluster, but Microsoft is like administering and operating for security reasons. Right? And there's like Omnivum in like Kuwait, like the Kuwait, like super rich guy spending like five plus billion dollars on data centers. Right? Like you just go down the list, like all these countries, Malaysia has, you know, you know, ten plus billion dollars of like data center, you know, AI data center build outs over the next couple of years, right? Like,

• 1:22:49Dylan Patel?

  and you know, go to every country, it's like this, this stuff is happening, but on the grand scheme of things, the vast majority of the compute is being built in the US. And then China, and then like Malaysia, Middle East, and like rest of the world. Um, and, and if you're in the, you know, going back to your point, right? Like you have synthetic data, you have like the search stuff, you have like, um, you have all these post-training techniques. Uh, you have all this, you know, all this ways to soak up flops, or you just figure out how to train across multiple data centers, which I think they have, um, at least Microsoft and OpenAI have figured, or OpenAI has figured it out. Um, their actions. So Microsoft has signed deals north of ten billion dollars with fiber companies to connect their data centers together.

• 1:23:32Dylan Patel?

  There are some permits already filed to show people are digging, um, you know, between certain data centers. So we think with fairly high accuracy, we can say, we, we think that there's five data centers, massive, not just five data centers, sorry, five like regions that they're connecting together, which comprises of many data centers, right? What will be the total power usage of the? Depends on the time, but easily north of a gigawatt, right? Which is like close to a million GPUs. Uh, well, the Each GPU is getting more power, higher power consumption too, right? Like, it's like, you know, the rule of thumb is like GPU H100 is like 700 watts, but then like total power per GPU all in is like 1,200, 1,300 watts, 1,400 watts. But next generation Nvidia GPUs are, um, it's 1,200 watts for the GPU, but then it actually ends up being like 2,000 watts all in, right?

• 1:24:16Dylan Patel?

  Like, so there's a little bit of scaling of power per GPU. But like, you already have 100K cluster, right? Uh, OpenAI in Arizona, XAI in, uh, Memphis and many others already building 100K clusters of H100s. You have multiple, at least five, I believe, GB200, 100K clusters, being built by Microsoft slash OpenAI slash their partners, uh, for them. Um, and then, and then potentially even more, 500K GB200s, right? Is a gigawatt, right? And that's like online next year, right? And like the year after that, if you aggregate all the data center sites and like how much power? And you only look at net adds since 2022 instead of like the total capacity at each data center, then you're still like north of multi-gigawatt, right?

• 1:24:59Dylan Patel?

  And they're, so they're spending 10 north, 10 plus billion dollars on these fiber deals with a few fiber companies, Lumen, Zayo, like, you know, a couple of other companies. And then they've got all these data centers that they're clearly building 100K clusters on, right? Like old crypto mining site, uh, with CoreWeave in Texas, or like this Oracle Crusoe in Texas, and then like in Wisconsin and Arizona, and, and, and, you know, a couple of other places, there's a lot of data centers being built up. Uh, you know, and, and, and providers, right? QTS and Cooper and like, you know, you go down the list, there's like so many different providers and self-build, right? Data centers I'm building myself.

• 1:25:31Dwarkesh Patel?

  So, so, uh, uh,

• 1:25:34Dylan Patel?

  Gigawatts, yes.

• 1:25:35Dwarkesh Patel?

  Let's just like give the number on like, okay, 2025, Elon's cluster is gonna be the big, like, uh, it doesn't matter who it is.

• 1:25:42Dylan Patel?

  So, so then there's the definition game, right? Like Elon claims he has the largest cluster at 100K GPUs because they're all fully connected.

• 1:25:48Dwarkesh Patel?

  Rather than who it is, like, I just want to know, like, how, how, how many, like? I don't know if it's better to denominate in

• 1:25:54Dylan Patel?

  Hundred thousand GPUs this year.

• 1:25:56Dwarkesh Patel?

  Okay. For the biggest cluster.

• 1:25:58Dylan Patel?

  For the biggest cluster.

• 1:25:59Dwarkesh Patel?

  Next year.

• 1:25:59Dylan Patel?

  Next year, three hundred to five hundred thousand, depending on whether it's one side or many, right? Three hundred to like seven hundred thousand, I think is the upper bound of that. But anyways, like, you know, there's, it's, it's, it's about like when they tiered on, when they can connect them, when the fibers connected together. Anyways, three hundred, three hundred to like seven, five hundred thousand, let's say, but those GPUs are two to three X faster, right? Versus the hundred K cluster. So on an H one hundred equivalent basis, you're at a million chips next year.

• 1:26:26Dwarkesh Patel?

  In one cluster.

• 1:26:27Dylan Patel?

  By the end of the year, yes. No, no, no. Well, so one cluster is like the

• 1:26:30Dwarkesh Patel?

  But you know what I mean.

• 1:26:30Dylan Patel?

  The wishy washy definition, right? Multi-site, right? Can you do multi-site? What's the efficiency loss when you go multi-site? Um, is it possible at all? I truly believe so. What is it? Whether it's a, whether what's the efficiency loss is the question, right?

• 1:26:43Dwarkesh Patel?

  Okay. Would it be like twenty percent loss, fifty percent loss?

• 1:26:46Dylan Patel?

  Great question. This is where like, you know, this is where you need like the secrets, right? Of like, and Anthropic's got similar plans with Amazon and you go down the list, right? Like people

• 1:26:53Dwarkesh Patel?

  And then, and then the year after that.

• 1:26:55Dylan Patel?

  The year after that is where

• 1:26:56Dwarkesh Patel?

  Twenty, this is twenty twenty six.

• 1:26:58Dylan Patel?

  Twenty twenty six, there is a single gigawatt site, and that's just part of the like multiple sites, right? For, for Microsoft.

• 1:27:05Dwarkesh Patel?

  The Microsoft five gigawatt thing happens in twenty

• 1:27:07Dylan Patel?

  One gigawatt, one site in, in twenty twenty six, but then you have, you know, a number of others. Uh, you have five different locations, each with multiple, some with multiple sites, some with single site. You, you're easily north of two, three gigawatts. Um, and then the question is, can you start using the old chips with the new chips? And like, the scaling, I think is like, you're going to continue to see flop scaling like much faster than people expect. I think as long as the money pours in, right? Like, that's the other thing is like, there's no fucking way you can pay for the scale of clusters that are being planned to be built next year for OpenAI. Until, unless they raise like fifty to a hundred billion dollars.

• 1:27:43Dylan Patel?

  Which I think they will raise that, like, end of this year, early next year.

• 1:27:46Dwarkesh Patel?

  Fifty to one hundred billion?

• 1:27:47Dylan Patel?

  Yes.

• 1:27:47Dwarkesh Patel?

  Are you kidding me?

• 1:27:48Dylan Patel?

  No.

• 1:27:49Dwarkesh Patel?

  Oh, my god.

• 1:27:50Dylan Patel?

  This is like, you know, like, Sam has a superpower, no? Like, it's like, it's like recruiting and, like, raising money. That's like what he's like a god at.

• 1:27:58Dwarkesh Patel?

  Will ships themselves be a bottleneck to this killing?

• 1:28:00Dylan Patel?

  Not in the near term. Uh, it's, it's more getting back to the concentration versus decentralization point. Because, like, the largest cluster is a hundred thousand GPUs. Nvidia has manufactured, uh, close to six million hoppers, right, across last year and this year, right? So like, what? That's fucking tiny, right?

• 1:28:16Dwarkesh Patel?

  So then, why is, uh, Sam talking about the seven trillion to build foundries and whatever, like?

• 1:28:20Dylan Patel?

  Well, this is, this, you know, like, draw the line, right? Like, log, log lines. Let's fuck, a number goes up, right? You know? If you do, if you do that, right, like, you're going from hundred K to three hundred to five hundred K, where the equivalent is a million, you just ten X year on year. Do that again. Do that again, or more, right? If you increase the pacing.

• 1:28:37Dwarkesh Patel?

  Wait, what is do that again? So like twenty twenty-six, like, the number of H one hundred equivalents?

• 1:28:40Dylan Patel?

  You know, if you increase the globally produced flops by like thirty X year on year, or ten X year on year, and the cluster size grows, or the cluster size grows by, you know, three to five to seven X, and then you do your start, you get multi-site going better and better and better, you can get to the point where multi-million chip clusters, I either can, even if they're like regionally not connected right next to each other, um, are, are right there.

• 1:29:06Dwarkesh Patel?

  And in, in terms of, uh, flops, like, it would be one E, what?

• 1:29:10Dylan Patel?

  I think, I think one E thirty is like very possible, like twenty-eight, twenty-nine.

• 1:29:14Dwarkesh Patel?

  Wow.

• 1:29:15Dylan Patel?

  Yeah.

• 1:29:15Dwarkesh Patel?

  Um, and one E thirty, you said, by twenty-eight, twenty-nine?

• 1:29:18Dylan Patel?

  Yeah.

• 1:29:19Dwarkesh Patel?

  And so that is literally six orders of magnitude. Um, that's like one hundred thousand times more compute than G P T four.

• 1:29:26Dylan Patel?

  The, the, the other thing to say is, like, the way you count flops on a, on a training run is really stupid. Like, you can't just do, like, param, active parameters times tokens times six, right? Like, that's, that's really dumb. Because, like, the paradigm, as you mentioned, right, is like, and you've had many great podcasts on this, like, synthetic data and, like, R L stuff, post-training, like, verifying data, and like, all these things, generating and throwing it away, like, all sorts of stuff, search, like, inference time compute, all these things, like, aren't counted. In the training flops, so you can't like say 1e30 is a really stupid number to say because by then the, you know, the actual flops of the pre-training may be X. But the data to generate the, for the pre-training may be, you know, way bigger or like the search inference time may be like way, way bigger, right?

• 1:30:07Dwarkesh Patel?

  Right. But also the, like, because you're doing this sort of adversarial synthetic data where like the thing you're weakest that you can make synthetic data for that. It might be way more sample efficient. So like, even though

• 1:30:18Dylan Patel?

  The pre-training flops will be irrelevant, right? Like, I actually don't think pre-training flops will be 1e30. I think more reasonably it'll be like the total summation of the flops that you deliver to the model.

• 1:30:27Dwarkesh Patel?

  Right.

• 1:30:28Dylan Patel?

  Uh, across pre-training, uh, post-training, um, synthetic data for that pre-training data and post-training data. As well as like some of the inference time compute efficiencies, like, could be like, it's more like 1e30, right?

• 1:30:40Dwarkesh Patel?

  Interesting. So suppose you, you really do get to the world where like it's worth investing. Okay, actually, if you're doing 1e30, how, is that like a trillion dollar cluster, a hundred billion dollar cluster, like?

• 1:30:51Dylan Patel?

  I, I think it'll be like multi-hundred billion dollars.

• 1:30:56Dwarkesh Patel?

  And then?

• 1:30:56Dylan Patel?

  But then like, it'll be like, I, I like truly believe people are going to be able to use their prior generation clusters and alongside their new generation clusters. Um, and obviously like, you know, smaller batch sizes or whatever, right? Like, or use that to generate and verify data, all these sorts of things.

• 1:31:10Dwarkesh Patel?

  And then for 1e30, um, right now, I think 5% of, uh, TSMC's N5 is Nvidia or like whatever percentage it is. By, by 2028, what percentage will it be?

• 1:31:22Dylan Patel?

  Um, again, this is like a question of like how scale-pilled you are and how much money will flow into this and how you think progress works. Like, will models continue to get better or does the line like not, does the line slope over? I believe it'll like continue to like skyrocket in terms of capabilities.

• 1:31:37Dwarkesh Patel?

  In that world.

• 1:31:38Dylan Patel?

  In that world, why, why wouldn't like, not a 5 nanometer, but like of 2 nanometer, A16, A14, these are the nodes that'll be in that timeframe of 2028. Used for AI, I could see like 60, 70, 80% of it, like, yeah, no problem.

• 1:31:52Dwarkesh Patel?

  Given the fabs that are like currently planned and are currently being built, that is, is that enough for the 1e30 or will?

• 1:31:57Dylan Patel?

  I think so, yeah.

• 1:31:58Dwarkesh Patel?

  So, so then like the chip code doesn't make any sense. Cause like the chip, like the chip go stuff about like, we don't have enough computer, doesn't make any sense.

• 1:32:04Dylan Patel?

  I think, I think like, the plans of TSMC on 2 nanometer and such are like quite aggressive for a reason, right? Like, to be clear.

• 1:32:16Dylan Patel?

  Apple, which has been TSMC's largest customer, does not need how much 2-nanometer capacity they're building. They will not need A16, they will not need A14, right? Like, you go down the list, it's like, Apple doesn't need this shit, right? Although they did just hire one of Google's head of system design for TPU, but, you know, so they are gonna make an accelerator, but, you know, that's besides the point, an AI accelerator, but that's besides the point. Like, Apple doesn't need this for their business, which they have been 25% or so of TSMC's business for a long time, and when you just zone in on just the leading edge, they've been like, more than half of the newest node, or 100% of the newest node almost constantly. That paradigm goes away, right? If you believe in scaling, and you believe in like, the models get better, the new models

• 1:32:58Dylan Patel?

  will generate, you know, infinite, not infinite, but like, amazing productivity gains for the world, and such on and so forth, then if you believe in that world, then like, TSMC needs to act accordingly, and the amount of silicon that gets delivered needs to be there, so 25, 26, TSMC is like, definitely there, and then on a longer time scale, the industry can be ready for it, but it's gonna be a constant game of like, you must convince them, constantly that they must do this. It's not like a simple game of like, oh, you know, if people work silently, it's not gonna happen, right? Like, they have to see the demonstrated growth over and over and over and over again, across the industry.

• 1:33:35Dwarkesh Patel?

  Who wants to see it, investors, or companies, or who?

• 1:33:39Dylan Patel?

  More so like, TSMC needs to see Nvidia volumes continue to grow straight up, right? And, oh, and Google's volumes continue to grow straight up, and, you know, go down the list. Chips, in the near term, right, next year, for example, are less of a constraint than data centers, right? And likewise for 2026. The question for 27, 28 is like, you know, always when you grow super rapidly, like, people wanna say, that's the one bottleneck, because that's the convenient thing to say. And in 2023, there was a convenient bottleneck, COAS, right? The picture's gotten much, much like, cloudier, not cloudier, but we can see that like, you know, HBM is a limiter too, COAS is as well, COASL especially, right? Data centers, transformers, substations, like, all, like-

• 1:34:25Dylan Patel?

  Power generation, batteries, like UPSs, like CRHs, like water cooling stuff. Like, all of this stuff is now limitations next year and the year after. fabs are in 26, 27, right? Like, you know, things will get cloudy because like, the moment you unlock one, oh, like, only 10% higher, the next one is the thing. And only 20% higher, the next one is the thing. So today, like, data centers are like 4% to 5%. Of total US? Of total US. When you think about, like, as a percentage of US power, that's not that much. But when you think US power's been like this, and now you're like this. But then you also flip side, you're like, oh, all this coal's been curtailed, all these, like, oh, there's so many, like, different things. So like, power is not that crazy on a, like, on a national basis. On a localized basis, it is, because it's about the delivery of it. Same with the substation transformer supply chains, right?

• 1:35:08Dylan Patel?

  It's like, these companies have operated in an environment where the US power is like this, or even slightly down, right? And it's like, kind of been like, you know, like that, because of efficiency gains, because of, you know, so, you know. So anyways, like, there have been humongous, like, weakening of the industry. But now, all of a sudden, if you tell that industry, your business will triple next year if you can produce more. Oh, but I can only produce 50% more. Okay, fine. Year after that, now we can produce 3X as much, right? You do that to the industry, the US industrial base, as well as the Japanese, as well as, like, you know, all across the world can get revitalized much faster than people realize, right? Like, I truly believe that people can innovate when, when given the, like, need to.

• 1:35:49Dylan Patel?

  It's one thing if it's like, this is a shitty industry where my margins are low, and we're not growing really, and like, you know, blah, blah, blah, blah, blah, to all of a sudden, oh, this is the sexy, I'm in power, and I'm like, this is the sexiest time to be alive, and like, we're, we're gonna do all these different plans and projects, and people have all this demand, and they're like begging me for another percent of efficiency advantage, because that gives them another percent to deliver to the chips. Like, all these things, or 10% or whatever it is, like, you see all these things happen, and innovation is unlocked. And, you know, you also bring in, like, AI tools, you bring in, like, all these things. Innovation will be unlocked, production capacity can grow, not overnight, but it will on six months, 18 months, three-year timescales. It will grow rapidly. And you see the revitalization of these industries.

• 1:36:34Dylan Patel?

  So, but I think like getting people to understand that, getting people to believe because, you know, if we pivot to like, you know, I'm telling you that Sam's going to raise 50 to 100 billion dollars because he's telling people he's going to raise this much, right? Like literally having discussions with sovereigns and like Saudi Arabia and like the Canadian Pension Fund and like, not these specific people, but like the biggest investors in the world. And of course, Microsoft as well, but like he's literally having these discussions because they're going to drop their next model or they're going to show it off to people and raise that money. But, but if, because this is their plan.

• 1:37:05Dwarkesh Patel?

  If these sites are already planned and like they've already-

• 1:37:07Dylan Patel?

  The money's not there, right?

• 1:37:08Dwarkesh Patel?

  So how do you play, how do you like plan a site without-

• 1:37:11Dylan Patel?

  Today, Microsoft is taking on immense credit risk, right? Like they've signed these deals with all these companies to do this stuff, but Microsoft doesn't have, I mean, they could pay for it, right? Microsoft could pay for it on the current time scale, right? Oh, what's, what's, you know, their capex going from 50 billion dollars to 80 billion dollars, direct capex, and then another 20 billion across like Oracle, CoreWeave, you know? And then like another like 10 billion across their data center partners. They can afford that, right? To next year, right? But then that doesn't, you know, like this is because Microsoft truly believes in OpenAI. They may have doubts like, holy shit, we're taking on a lot of credit risk. You know, obviously they have to message Wall Street, all these things, but they are not like, that's like affordable for them because they believe they're a great partner to OpenAI, that they'll take on all this credit risk.

• 1:37:56Dylan Patel?

  Now, now, obviously OpenAI has to deliver, they have to make the next model, right? That's way better. And they also have to raise the money and I think they will, right? I truly believe from like how amazing 4.0 is, how small it is relative to four, uh, the cost of it is so insanely cheap. It's much cheaper than the API prices lead you to believe. And you're like, oh, what if you just make a big one? It's like very clear what's going to happen to me on the next jump, that they can then raise this money and they can raise this capital from the world. This is, this is intense. That's very intense.

• 1:38:26Dwarkesh Patel?

  John, if, if, actually if he's right, or I don't know, not him, but like in general, if we're, uh, if like the capabilities are there, the revenue is there.

• 1:38:34Dylan Patel?

  Revenue doesn't matter.

• 1:38:35Speaker 02

  Revenue matters.

• 1:38:37Dwarkesh Patel?

  Is there any part of that picture that still seems wrong to you in terms of like displacing so much of TSMC production, uh, wafers and like, uh, power and so forth? Does any part of that seem wrong to you?

• 1:38:47Speaker 02

  I can only speak to the semiconductor's part, even though I'm not an expert. But I think the thing is like TSMC can do it, like they'll do it. I just wonder-

• 1:38:56Dylan Patel?

  Though he's right in that in a sense of 24, 25, that's covered.

• 1:39:00Speaker 02

  Yeah.

• 1:39:00Dylan Patel?

  But 26, 27, that's that secret point where you have to say, can the semiconductor industry and the rest of the industry be convinced that this is where the money is? Like where's money is like, and that means is there money? Is there money by 24, 25?

• 1:39:14Speaker 02

  How much revenue do you think the AI industry as a whole needs by 25 in order to keep scaling?

• 1:39:18Dylan Patel?

  Doesn't matter. Compared to smartphones, compared to smartphone. But I know he says it doesn't matter.

• 1:39:22Speaker 02

  I'll get to it, why?

• 1:39:24Dylan Patel?

  You keep, I know.

• 1:39:25Speaker 02

  Hey, what is smartphones like Apple's revenue is like 200 something billion dollars, so like?

• 1:39:28Dylan Patel?

  Yeah, it needs to be another smartphone sized opportunity, right? Like even the smartphone industry didn't drive this sort of growth. Like it's crazy, don't you think?

• 1:39:35Speaker 02

  So today, so far, right?

• 1:39:37Dylan Patel?

  Only, the only thing I can really perceive, yeah, girlfriend.

• 1:39:39Speaker 02

  But like, but you know what I mean? It's not there.

• 1:39:44Dylan Patel?

  I want a real one, damn it.

• 1:39:49Dylan Patel?

  So, so, so like few things, right? The return on invested capital for all of the big tech firms is up since 2022.

• 1:39:56Speaker 02

  Yeah.

• 1:39:56Dylan Patel?

  And therefore it's clear as day that them investing in AI has been fruitful so far. Right?

• 1:40:02Speaker 02

  Wait, wait.

• 1:40:02Dylan Patel?

  For the, for the big tech firms.

• 1:40:05Dylan Patel?

  Return on invested capital. Like financially, you look at the, you look at Metas, you look at Microsoft's, you look at Amazons, you look at Googles. The return on invested capital is up since 2022, so it's.

• 1:40:15Speaker 02

  On AI in particular?

• 1:40:16Dylan Patel?

  No, just generally as a company. Now, obviously there's other factors here, like what is Meta's ad efficiency? How much of that is AI, right?

• 1:40:21Speaker 02

  Super messy, that's a super messy.

• 1:40:22Dylan Patel?

  Super messy thing, but here's the other thing. This is Pascal's wager, right? This is a matrix of like, do you believe in God? Yes or no? If you believe in God, yes or no, like hell or heaven, right? So if you believe in God and God's real and you go to heaven, that's great, that's fine, whatever. If you don't believe in God and God is real, then you're going to hell.

• 1:40:41Speaker 02

  This is the deep technical analysis you'll subscribe to semi analysis for.

• 1:40:45Dylan Patel?

  This is just me riffing. Can you, can you imagine what happens to the stock if Satya starts talking about Pascal's wager? But this is psychologically what's happening, right? This is a if I don't and and Satya said it on his earnings call. The risk of under investing is worse than the risk of over investing. He has said this word for word. This is Pascal's wager. This is I must believe I am AGI pill because if I'm not and my competitor does it, I'm absolutely fucked. Okay, other than Zuck, who seems pretty convinced. Sundar said this on the on the C on the earnings call. So Zuck said it, Sundar said it. Satya's actions on credit risk for Microsoft do it. He's very good at PR and like messaging, so he hasn't like said it so openly, right? Um, Sam believes it. Dario believes it. You look across these tech titans, they believe it.

• 1:41:29Dylan Patel?

  And then you look at the capital holders. Uh, the UAE believes it. Saudi believes it. How do you know the UAE and Saudi believe? Believes it like all these major companies and capital holders also believe it because they're putting their money here. But but that's like, how can like, it won't last. It can't last unless there's money coming in somewhere. Correct, correct. But then the question is, the the the simple truth is like GPT four costs like $500 million to train. I agree. And it has generated billions in reoccurring revenue. But in that meantime, OpenAI raised $10 billion or $13 billion and is building a, you know, a model that costs that much effectively, right? Right. Um, and and so then obviously they're not making money. So what happens when they do it again?

• 1:42:10Dylan Patel?

  They release and show GPT five with whatever capabilities that make everyone in the world like, holy fuck. Obviously, the revenue takes time after you release the model to show up. You still have only a few billion dollars or, you know, $5 billion of revenue run rate. You just raised 50 to 100 billion dollars because everyone sees this like, holy fuck. This is going to generate tens of billions of revenue. But that tens of billions takes time to flow in, right? It's not an immediate click. But the time where Sam can convince and not just Sam, but like people's decisions to spend the money are being made are then, right? Like so therefore, like you look at the data centers people are building. You don't have to spend most of the money to build a data center. Uh, most of the money is the chips, but you're already committed to like, oh, I'm just going to have so much data center capacity by 2027 or 2026.

• 1:42:53Dylan Patel?

  That it's I'm never going to need to build a data center again for like three, four, five years if AI is not real, right? That's like basically what they're all their actions are, or I can spend over 100. billion dollars on chips in twenty six and I can spend over a hundred billion dollars on chips in twenty seven. All right. So this is the, these are the actions people are doing and the lag on revenue versus when you spend the money or raise the money, raise the money, spend the money, build, you know, there's like a lag on this. So this is like, you don't necessarily need the revenue in twenty twenty five to support this. You don't need the revenue twenty twenty six to support this. You need the revenue in twenty five, twenty six to support the ten billion dollars that OpenAI spent in twenty three or Microsoft spent in twenty three slash early twenty four to build the cluster, which then they trained the model in mid twenty, you know, for early twenty four, mid twenty four, which they then released at the end of twenty four,

• 1:43:38Dylan Patel?

  which then started generating revenue in twenty five, twenty six.

• 1:43:40Speaker 02

  I mean, like, the only thing I can say is that you're, you, you look at a chart with three points on a graph. GPT one, two, three, and then you're like.

• 1:43:47Dwarkesh Patel?

  And even that graph is like, um, the investment you have to make in GPT four or GPT three is a hundred X. The investment you had to make in GPT five or GPT four is a hundred, like, so revenue, if, like, um, currently the ROI could be positive, but like, um, and this very well could be true. I think it will be true, but like, the, uh, the, uh, the revenue has to like increase exponentially, not just like.

• 1:44:08Speaker 02

  Yeah, of course, of course. I, I, I, I agree with you, but I also, I agree in Dylan's that it can be achieved ROI, like semiconductor, TSMC does this. Invest sixteen billion dollars and expects ROI does that, right? That's, I understand that. That's, that's fine. Lag, all that. The thing that I don't expect is that GPT five is not here. It's all dependent on GPT five being good. If GPT five sucks, if GPT, GPT five looks like, it doesn't blow people's socks off, this is all void.

• 1:44:38Dylan Patel?

  What kind of socks you wearing, bro?

• 1:44:40Speaker 02

  Show them.

• 1:44:43Speaker 02

  AWS. GPT five is not here. It's late. We don't know.

• 1:44:48Dylan Patel?

  I don't think it's late.

• 1:44:49Speaker 02

  I think it's late.

• 1:44:50Dwarkesh Patel?

  I, I, I want to zoom out and like go back to the end of the decade picture again. So if you're, um, if this picture.

• 1:44:56Speaker 02

  Hold on, hold on, we've already lost John. We've already accepted GPT-5 would be good, hello?

• 1:45:02Dylan Patel?

  Yeah, you got it, you know? Yeah, yeah, it's like, bro, like, life is so much more fun when you just, like, are delusionally, like, you know?

• 1:45:10Speaker 02

  You were just ripping bong hits, how are we?

• 1:45:14Dylan Patel?

  When you feel the AGI, you feel your soul.

• 1:45:16Speaker 02

  This is why I don't live in San Francisco.

• 1:45:20Dylan Patel?

  I have tremendous belief in, like, GPT-5.

• 1:45:22Speaker 02

  Why?

• 1:45:23Dylan Patel?

  Area, because, like, what we've seen already. Um, I think the public signs all show that this is, like, very much the case, right? Uh, what we see with, beyond that is more questionable, and I'm not sure, because I don't know what, I don't know, right? Like, I don't know. Um, we'll see how, like, how much they progress. But if, like, things continue to improve, uh, life continues to radically get reshaped for, you know, many people, uh, the, you know, it's also like, every time you increment up the intelligence, the amount of, like, usage of it grows hugely. Every time you increment the cost down of that amount of, that amount of intelligence, the amount of usage increases massively. As you continue to push that curve out, that's what really, like, matters, right? And, and it doesn't need to be today, it doesn't need to be a revenue versus, like, how much

• 1:46:08Dylan Patel?

  CapEx. In any time in the next few years, it just needs to be, did that last humongous chunk of CapEx make sense for OpenAI or whoever the leader was? Or, and then how does that flow through, right? Or were they able to convince enough people that they need to, they can raise this much money, right? Like, you think Elon's tapped out of his network with raising $6 billion? No. XAI is gonna be able to raise 30-plus, right? Easily, right? I think so. Um, you think Sam's tapped out? You think Anthropic's taped out? Anthropic's barely even diluted the company relatively, right? Like, you know, there's a lot of capital to be raised in just from, like, call it FOMO if you want, but like, during the dot-com bubble, people were spending, uh, the private industry flew through, like, $150 billion a year. We're nowhere close to that yet, right?

• 1:46:52Dylan Patel?

  We're not even close to the, the dot-com bubble, right? Why, why would this bubble not be bigger, right? And if you go back to the prior bubbles, PC bubble, semiconductor bubble, mechatronics bubble, uh, throughout the US, each bubble was smaller, I don't know, you know, you call it a bubble or not, it, why wouldn't this one be bigger? How, how many billions of dollars a year is this bubble right now? Uh, for private capital? Yeah. Uh, it's like 55, 60 billion so far, for, for this year. It can go much higher, right? And I think it will next year. Okay, so, uh, uh, let me think of...

• 1:47:23Speaker 02

  You need another bong rip.

• 1:47:26Dylan Patel?

  You know, at least like finishing up and looping into the next question was like, uh, you know, prior bubbles also didn't have the most profitable companies that humanity's ever created investing, and they were debt financed. This is not debt financed yet, right? So that's the last like little point on that one. Whereas the 90s bubble was like very debt financed. This is like-

• 1:47:42Speaker 02

  It was disastrous for those companies.

• 1:47:43Dylan Patel?

  Yeah, sure, but it was so many, so much was built, right? Um, you gotta blow a bubble to get real stuff to be built.

• 1:47:49Dwarkesh Patel?

  It is an interesting analogy where like, uh, with, even though the dot com bubble obviously burst and like a lot of companies went bankrupt, they in fact did lay out the infrastructure that enabled the web and everything. So you could imagine in AI, it's like some, a lot of the foundation, a lot of companies or whatever, the like a bunch of companies will like go bankrupt, but like they will-

• 1:48:07Dylan Patel?

  You could argue with all-

• 1:48:08Dwarkesh Patel?

  Enable the singularity.

• 1:48:09Speaker 02

  During the 1990s, at the turn of the 1990s, there was immense amount of money invested in like MEMS and like opticals, optical technologies because everyone expected the fiber bubble to continue, right? That all ended at 2003, 2002, or issue, right?

• 1:48:22Dylan Patel?

  And that started in 94?

• 1:48:23Speaker 02

  Hasn't been revitalization since, right? Like that's, you could risk the possibility of a-

• 1:48:28Dylan Patel?

  Bro, Lumen, one of the companies that's doing the fiber build out for Microsoft, the stock like fucking Forexed last month or this month.

• 1:48:34Speaker 02

  And then how's it done from 2002 to 2024?

• 1:48:36Dylan Patel?

  Oh no, horrible, horrible, but like, we're gonna rip, babe.

• 1:48:39Speaker 02

  You could, you could, you could freeze AI for another two decades.

• 1:48:43Dylan Patel?

  You, sure, sure, possible. Or people can see a badass demo from GPT-5, slight release, raise a fuckload of money. It could even be like a Devon like demo, right? Where it's like complete bullshit, but like it's fine, right? Like, should I should?

• 1:48:57Speaker 02

  Edit that out, edit that out.

• 1:48:59Dylan Patel?

  No, it's fine, it's fine, dude. I don't really, I don't really care. Um, you know, it, it, it's, it's, the capital is gonna flow in, right? Now, whether, whether it deflates or not is like an irrelevant concern on the near term because you operate in a world where it is happening. And being, you know, being, uh, you know, what is the Warren Buffett quote, which is like, you can be, I don't even know if it's Warren Buffett.

• 1:49:21Speaker 02

  You don't know who's, you don't know who's swimming naked until the tide goes out?

• 1:49:24Dylan Patel?

  No, no, no, the one about like, um, the market is delusional far longer than you can remain solvent, or something like that.

• 1:49:30Speaker 02

  That's not Buffett.

• 1:49:30Dylan Patel?

  That's not Buffett?

• 1:49:31Speaker 02

  Yeah, yeah, that's, uh, John Maynard Keynes.

• 1:49:33Dylan Patel?

  Oh shit, that's that old?

• 1:49:35Speaker 02

  Yeah.

• 1:49:36Dylan Patel?

  Okay, uh, okay, so Keynes said it, right? It's like you can be, yeah, so this is the world you're operating in. Like, it doesn't matter, right? Like, what, what exactly happens? There will be ebbs and flows, but like, that's the world you're operating in.

• 1:49:47Speaker 02

  I reckon that if an AI bubble hap, if AI bubble pops, each one of these CEOs lose their jobs.

• 1:49:54Dylan Patel?

  Sure. Or if you don't invest and you lose, it's a Pascalian wager and you're a... That's much worse. Across decades, the largest company at the end of each decade, like the largest companies, that list changes a lot.

• 1:50:06Dwarkesh Patel?

  Yeah.

• 1:50:06Dylan Patel?

  And these companies are the most profitable companies ever. Are they going to let that list, are they going to let themselves like lose it? Or are they gonna go for it? They have one shot, one opportunity, you know, to make themselves into, you know, the whole M and N song, right?

• 1:50:19Dwarkesh Patel?

  I want to hear like the story of how both of you started your businesses or you're like, the thing you're doing now. John, like how, like what, how did it begin? What were you doing when you started the?

• 1:50:32Dylan Patel?

  You're gonna talk about your textile company?

• 1:50:34Speaker 02

  Oh my god, no way.

• 1:50:36Dylan Patel?

  Please, please.

• 1:50:36Dwarkesh Patel?

  Wait, are you joking?

• 1:50:38Dylan Patel?

  I guess if he doesn't want to, we'll talk about it later.

• 1:50:40Dwarkesh Patel?

  Okay, sure.

• 1:50:41Speaker 02

  I think like I used to, I mean, the story's famous. I've told it a million times. It's like Asianometry started off as a tourist channel.

• 1:50:47Dwarkesh Patel?

  Yeah.

• 1:50:47Speaker 02

  So I would go around, kind of like I was, I moved to Taiwan for work and then...

• 1:50:52Dylan Patel?

  Doing what?

• 1:50:53Speaker 02

  I was, I was working in cameras. And then like I told...

• 1:50:58Dylan Patel?

  What was the other company you started?

• 1:51:02Speaker 02

  It tells too much about me.

• 1:51:03Dylan Patel?

  Aw, come on.

• 1:51:04Speaker 02

  Um, I'd like, I worked in, I worked in cameras and then basically I went to Japan with my mom. And mom was like, hey, you know, what are you doing in Taiwan? I don't know what you're doing. I was like, alright, mom, I will go back to Taiwan and I'll make stuff for you. And I made videos. I would like go to the Chiang Kai-shek Park and be like, hi, mom, this park was this, this. Eventually at some point you run out of stuff, but then it's like a pretty smooth transition from that into like, you know, history of Chinese history, Taiwanese history. And then people started calling me Chinanometry. I didn't like that, so I moved to other parts of Asia. And now like, and then...

• 1:51:39Dwarkesh Patel?

  So what year did you like start, like what year was like people started watching your videos? Let's say like a thousand views per video or something?

• 1:51:46Speaker 02

  Oh my gosh, that was not, I started the channel in 2017 and it wasn't until like 2018 that, 2019 that it actually... I labored on for like three years, first three years with like no one watching. Like I had got like 200 views and I'd be like, oh, this is great.

• 1:52:00Dwarkesh Patel?

  And then were you were the videos basically like the ones you have? But sorry, backing up for the audience who might not, I imagine basically everybody knows Asianometry, but if you don't, like the most popular channel about semiconductors, Asian business history, business history in general,

• 1:52:16Dwarkesh Patel?

  even like geopolitics history and so forth. And yeah, I mean, it's like, honestly, I've done like research for like different AI guests and different, like whatever thing I'm trying to be, I'm trying to understand like how does hardware work? How does AI work? It's like, this is like my-

• 1:52:31Dylan Patel?

  How does a zipper work? Did you watch that video?

• 1:52:33Dwarkesh Patel?

  No, I haven't watched that one.

• 1:52:34Dylan Patel?

  It was like, I think it was a span of three videos. It was like Russian oil industry in the 1980s and how it like funded everything. And then when it collapsed, they were absolutely fucked.

• 1:52:42Dwarkesh Patel?

  Yeah.

• 1:52:42Dylan Patel?

  And then it was like, the next video was like the zipper monopoly in Japan. The next video was about ASL.

• 1:52:47Speaker 02

  Not a monopoly. Strong, strong holding in a mid, in a mid-tier size. There's like the luxury zipper makers. Asianometry is always just kind of like stuff that I'm interested in. And I'm like interested in a whole bunch of different stuff and I like, like, and then the channel, for some reasons, people started watching the stuff I do. And I still have no idea why. To be honest, I still feel like it's, I still feel like a fraud. I sit in front of like Dylan and he's, I feel like a fraud, legit fraud, especially when he starts talking about 60,000 reefers and all that. I'm just like, I feel like I should be know, I should know this, but like, you know, in the end it's, but, but that, you know, I just try my best to kind of bring interesting stories out.

• 1:53:26Dwarkesh Patel?

  How do you make a video every single week? Cause these are like-

• 1:53:29Speaker 02

  Two a week.

• 1:53:30Dylan Patel?

  You know how long he had a full-time job?

• 1:53:32Speaker 02

  Five years, six years.

• 1:53:33Dylan Patel?

  Oh, sorry, a textile business.

• 1:53:35Speaker 02

  And yes.

• 1:53:36Dylan Patel?

  And a full-time job. Wait, no, full-time job, textile business and Asianometry until like for a long, long time.

• 1:53:41Speaker 02

  I literally just gave up the textile business this year.

• 1:53:44Dwarkesh Patel?

  And like, how are you doing research and doing like making a video and like twice a week? I don't know. I like do these fucking, I'm like fucking talking. This is all I do. And I like do these like once every two weeks. Sorry.

• 1:53:54Dylan Patel?

  See, see the difference is Dwarkesh, you go to SF Bay Area parties constantly. And Dwarkesh is, John is like locked in.

• 1:54:02Dwarkesh Patel?

  Yeah, yeah.

• 1:54:02Dylan Patel?

  He's like locked in 24/7.

• 1:54:04Dwarkesh Patel?

  He's got like the GSMC work ethic and I've got like the Intel work ethic.

• 1:54:07Dylan Patel?

  If I don't, I got the Huawei ethic. If I do not finish this video, my family is, it will be, will be pillaged.

• 1:54:15Dwarkesh Patel?

  He actually gets really stressed about it, I think, like not doing something like on his schedule. Yeah.

• 1:54:20Speaker 02

  It's very much like I do, I do two videos per week. I write them both simultaneously.

• 1:54:26Dwarkesh Patel?

  And how are you scouting out future topics you wanna do? This is just like, you know, you just like pick up random articles, books, whatever. And then you just, if you find it interesting, you make a video about it.

• 1:54:34Speaker 02

  Sometimes what I'll do is I'll Google a country, I'll Google an industry and I'll Google like what the country is exporting now and what it used to export. And I compare that and I say, that's my video. Or I'll be like, or but then sometimes also just as simple as like, I should do a video about YKK. And then it's also just, but then it's also just a simple.

• 1:54:52Dwarkesh Patel?

  Simper is nice. I should do a video about it.

• 1:54:54Speaker 02

  I do, I do. It literally is.

• 1:54:56Dwarkesh Patel?

  Do you like keep a list of like, here's the next one, here's the one after that?

• 1:55:00Speaker 02

  I have a long list of like ideas. Sometimes it's as vague as like Japanese whiskey. No idea what Japanese whiskey is about. I heard about it before. I watched that movie. And then so I was just like, okay, I should do a video about that.

• 1:55:14Dwarkesh Patel?

  Yeah.

• 1:55:14Speaker 02

  And then eventually, you know, you get to a, you get.

• 1:55:17Dwarkesh Patel?

  How many research topics do you have in the back burner, basically? Like you're like, I'm just kind of reading about it constantly and then like in a month or so, I'll make a video about it.

• 1:55:24Speaker 02

  I just finished a video about how IBM lost the PC.

• 1:55:27Dwarkesh Patel?

  Yeah.

• 1:55:27Speaker 02

  So right now I'm de, I'm unstressing about that, but then I'll kind of move right on to. Like the videos do kind of lead into others. Like right now, this one is about IBM PC, how IBM lost the PC. Now it's next is how Compaq collapsed, how the wave destroyed Compaq. So technically, I'll do that. At the same time, I'm dual lining a video about Qubits. I'm dual lining a video about uh, the self, directed self-assembly for semiconductor manufacturing, which I'll read a lot of Dylan's work for. But then like, like a lot of that is kind of like, it's just, it's in the back of my head. And I'm like producing it as I, as I go. Dylan, how do you work?

• 1:56:07Dwarkesh Patel?

  How does one go from Reddit shit poster to like running a, like a semiconductor research and consulting firm?

• 1:56:13Dylan Patel?

  Yes.

• 1:56:14Dwarkesh Patel?

  We'll start with this shitposting.

• 1:56:15Dylan Patel?

  It's a long line, right? Like, so immigrant parents grew up in rural Georgia. So when I was eight, I begged for, or seven, I begged for an Xbox. And when I was eight, I got it, 360, right? Uh, they had a manufacturing defect called the red ring of death. Um, there are a variety of fixes. I tried them, like putting a wet towel around the Xbox, something called the penny trick. Those all didn't work. My Xbox still didn't work. Uh, my cousin was coming the next weekend and like, you know, he's like two years older than me. I look up to him. He's like in between my brother and I, but I'm like, oh, no, no. We're friends, you know, you don't like my brother as much as you like me. My brother's more like jockey types. It didn't matter. Um, so like, he didn't really care that I broke, that the Xbox is broken. He's like, you better fix it though. Right? Otherwise, parents will be pissed. So I figure out how to fix it online. It ends up, you know, I tried a variety of fixes.

• 1:56:59Dylan Patel?

  Ended up shorting the temperature sensor, um, and that worked for long enough until Microsoft did the recall, right? Um, but in that, you know, I stay, I, I learned how to do it out of necessity on the forums. I was a nerdy kid, so I liked games, but whatever. But then like, there was no other outlet. Once I was like, holy shit, this Pandora's box, like what just got opened up? Um, so then I just shitposted on the forums constantly, right? Um, and, you know, for many, many years, and then I ended up like moderating all sorts of reddits when I was like a tween teenager. Um, and then like, you know, as soon as I started making money, you know, you know, grew up in a family business, but didn't get paid for working, right? Of course, like yourself, right? But like, as soon as I started making money at like, and like, I got my internship and I like internships, I was like 18, 19, right?

• 1:57:41Dylan Patel?

  I started making money. I started investing in semiconductors, right? Like, I was like, of course, this is shit I like, right? Um, you know, everything from like, uh, and by the way, like the whole way through, like as technology progressed, especially mobile, right? It goes from like very shitty chips in phones to like very advanced, every generation, they'd add something. And I'd like read every comment. I'd read every technical post about it. Uh, and also all the history around that technology. And then like, you know, who's in the supply chain? And it just kept building and building and building. Went to college, did data science-y type stuff, uh, went to work on like hurricane, earthquake, wildfire simulation and stuff for a financial company. But before that, like, but during college, I was still like, I wasn't shitposting on the internet as much. I was still posting some.

• 1:58:22Dylan Patel?

  But I was like following the stocks and all these sorts of things, the supply chain, all the way from like the tool equipment companies. Uh, and the reason I like, like those is because like, oh, this technology, oh, it's made by them. You know, you kind of.

• 1:58:32Dwarkesh Patel?

  Do you have like friends in person who were into this shit or was it just online?

• 1:58:36Dylan Patel?

  I made friends on the internet, right? Oh, that's dangerous. Nah, I've only ever had like literally one bad experience and that was just because he's drugged out. Right, like? A one bad experience online or? Like, meeting someone from the internet in person. Everyone else has been genuine, like you, you have enough filtering before that point. You're like, you know, even if they're like hyper mega like autistic, it's cool, right? Like, I am too, right? You know, no, I'm just kidding. Um, but like, you know, you go through like the, um, you know, the layers and you look at the economic angle. You look at the technical angle. Um, you read a bunch of books just out of like, you know, you could just buy engineering textbooks, right? And read them, right? Like, what's, what's, what's stopping you, right? And if you bang your head against the wall, you learn it, right?

• 1:59:18Dylan Patel?

  And then why we were doing this, was there like, did you expect to work on this at some point? Or was it just like pure interest? No, it was like, it was like obsessive hobby of many years and it pivoted all around, right? Like, at some point, I really liked gaming and then I got moved into like, I really liked phones and like rooting them and like underclocking them and the chips there and like screens and cameras. And then back to like gaming and then to like data center stuff. Like, cause that was like where the most advanced stuff was happening. So it's like, I liked all sorts of like telecom stuff for a little bit. Like, it was like, it like bounced all around, uh, but generally in like computing hardware, right? Um, and I did data science, you know, you could, I, I said I did AI when I interviewed, but like, you know, but it was like bullshit, multi-variable regression, whatever, right?

• 2:00:01Dylan Patel?

  Um, those simulations of hurricanes, earthquakes, wildfire for like financial reasons, right? Like, anyways, um, you move, I moved up to like, um, you know, I was still, you know, I worked, I had a job for three years after college and I was posting and like whatever, I had a blog, anonymous blog for a long time. I'd even made like some YouTube videos and stuff. Most of that stuff is scrubbed off the internet, including internet archive because I asked them to remove it. Um, but like, you, you, um, in 2020, I like quite quit my job and like started shitposting more seriously on the internet. I, I, um, moved out of my apartment and started traveling through the US and I went to all the national parks, like in my truck slash like tent slash, you know, also stayed in hotels and motels like three, four days a week. But I'd like, I started posting more frequently on the internet.

• 2:00:44Dylan Patel?

  Um, and I'd already had like some small consulting arrangements in the past, uh, but it really started to pick up in mid 2020, like consulting arrangements from the internet from my persona. Like, what kinds of people, investors, hardware companies, like? Um, there were like- It was like, it was like people who weren't in hardware that wanted to know about hardware. It would be like some investors, right? Some couple of VCs did it, but some public market folks. You know, there was times where like companies would ask about like three layers up in the stack. Like me, because they saw me write some random posts and like, hey, like, can we, and blah, blah, blah. There's all sorts of like random, it was really small money. And then in 2020, like it really picked up and I just like, I was like, why don't I just arbitrarily make the price way higher? And it worked. And then I started posting, I made it a new, I made a newsletter as well.

• 2:01:29Dylan Patel?

  And I kept posting. Quality kept getting better, right? Because people read it, they're like, this is fucking retarded. Like, you know, here's what they actually write or, you know, like, you know, over, over more than a decade, right? And then in 2021, towards the end, I made a paid post because someone pay and like, you know, for a report or whatever, right? Ended up, that ended up doing like, I went to sleep that night. It was about, it was about photo resist and like the developments in that industry, which is the stuff you put on top of the wafer before you put in the litho tool. Lithography tool. Did great, right? Like I woke up the next day and I had like 40 paid subscriptions. I was like, what? Okay, let's keep going, right? And like, let's keep, let's post more paid, paid, sort of like partially free, partially paid.

• 2:02:09Dylan Patel?

  Did like all sorts of stuff on like advanced packaging and chips and data center stuff and like AI chips, like all sorts of stuff, right? That I like was interested in. And that was interesting. And like, I always bridged economically because I read all the companies earnings for like, you know, since I was 18 and, you know, 28 now, right? You know, all the way through to like, you know, all the technical stuff that I could. 2022, I also started to just go to every conference I could, right?

• 2:02:35Dylan Patel?

  So I go to like 40 conferences a year, not like, not like trade show type conferences, but like technical conferences, like, like an arc, chip architecture. Photo resist. You know, AI nips, right? Like, you know, ICML.

• 2:02:46Dwarkesh Patel?

  How many conferences do you go to a year?

• 2:02:48Dylan Patel?

  Like 40.

• 2:02:49Dwarkesh Patel?

  So you like live at conferences?

• 2:02:50Dylan Patel?

  Yes. Yeah. I mean, I've been a digital nomad since 2020 and I've basically stopped and I moved to SF now, right? But like, kind of, kind of, not really.

• 2:02:58Dwarkesh Patel?

  You can't say that, that the government, the California.

• 2:03:00Dylan Patel?

  No, I'm not, I don't live at SF. Come on. But I basically do now, right?

• 2:03:04Dwarkesh Patel?

  California Internal Revenue Service.

• 2:03:09Dylan Patel?

  Like, do not seriously joke about it.

• 2:03:11Speaker 02

  They're gonna send you a clip of this podcast and be like, forty percent, please.

• 2:03:16Dylan Patel?

  I am in, I am in San Francisco, like, sub-four months a year, continuously, and, you know.

• 2:03:22Speaker 02

  Exactly a hundred and whatever days.

• 2:03:23Dylan Patel?

  Exactly a hundred and seventy-nine days, let's go, right, like, you know, um, over the full course of the year. Uh, but no, like, you know, go to every conference, make connections at all these, like, very technical things, like, international electron device manufacturing, oh, lithography and advanced patterning, oh, like, uh, very large-scale integration, like, you know, um, all, you know, circuits conference, you just go every single layer of the stack. It's so siloed, there's tens of millions of people that work in this industry, but if you go to every single one, you try and understand the presentations, you do the required reading, you look at the economics of it, you, like, are just curious and wanna learn. You, like, you can start to build up, like, more and more, and the content got better, and, like, you know, what I followed got better, and then,

• 2:04:04Dylan Patel?

  like, started hiring people in twenty twenty, in early twenty twenty-two as well, um. Or, or might have been, yeah, yeah, like, mid, mid twenty twenty-two, started hiring, uh, got people in different layers of the stack, but now today, like, you fast-forward now today, right, like, uh, almost every hyperscaler is a customer, not for the newsletter, but for, like, data we sell, right, um, you know, most, many major semiconductor companies, many investors, right, like, all these people are, like, customers of the data and stuff we sell, um, and the company has people all the way from, like, ex-Cymer, ex-SML, all the way to, like, ex, like, Microsoft and, like, an AI company, right, like, you know, like, and, and then through the stratification, you know, now there's fourteen people here in, like, the company, and, like, all across the US, Japan, Taiwan, Singapore, France,

• 2:04:48Dylan Patel?

  um, US, of course, right, like, you know, all over the world, and across many ranges of, like, and, and hedge funds as well, right, ex-hedge funds as well, right, so you got, kinda have, like, this amalgamation of, like, you know, tech and finance expertise, and we just do the best work there, I think, right?

• 2:05:03Speaker 02

  Are you still talking about a monstrosity?

• 2:05:06Dylan Patel?

  An unholy concussive.

• 2:05:10Dylan Patel?

  So like, when we saw, we saw, you know, we have data analysis consulting, etc. For anyone who like really wants to like get deeper into this, right? Like, we can talk about like, oh, people are building big data centers. But like, how many chips is being made in every quarter? Of what kind for each company? What are the subcomponents of these chips? What are the, what are the subcomponents of the servers, right? We try and track all of that. Follow every server manufacturer, every, every component manufacturer, every cable manufacturer, like, just like all the way down the stack, tool manufacturer. And like, know how much is being sold where and how, how and where things are. And project out, right? All the way out to like, hey, where's every single data center? What is the pace that it's being built out? This like, this is like the sort of data we want to have and sell.

• 2:05:54Dylan Patel?

  And, you know, it's, the validation is that hyperscalers purchase it and they like, like it a lot, right? And like, AI companies do, and like semiconductor companies do. So I think that's the sort of like, how it got there to where it is, is just like try and do the best, right? And try and be the best.

• 2:06:10Dwarkesh Patel?

  If you were an entrepreneur who's like, I want to get involved in the hardware chain somewhere, like, what is, like, what is, if you, if you could start a business today, somewhere in the stack, what, what, what would you pick? John, tell them about your textile business.

• 2:06:25Speaker 02

  I think I'd work in memory. Something in memory. Cause I think like, if you, if this concept is like there, like you have to hold immense amounts of memory. Immense amounts of memory. And I think memory already is tapped, like technologically. HBM exists because of limitations in DRAM. I said it correctly. I think like, it's, it's fundamentally, we've forgotten it because it is a commodity, but we shouldn't. I think it's, breaking memory is going to, would change, could change the world in that scenario.

• 2:06:59Dylan Patel?

  I think the context here is that Moore's Law was predicted in 1965. Intel was founded in 68 and released their first memory chips in 69 and 70. And so Moore's Law was, a lot of it was about memory. And the memory industry followed Moore's Law up until 2012. Where it stopped, right? And it became very incremental gains since then, whereas logic has continued and like people are like, "Oh, it's dying, it's slowing down." At least there's still a little bit of like, you know, coming, right? You know, still more than 10%, 15% a year CAGR, right? Of growth and density slash cost improvement. Memory is like literally like been like since 2012, like really bad. Uh, so, and, and when you think about the cost of memory, you know, it's been, it's been considered a commodity, but memory integration with accelerators, like this is like something that, I don't know if you can be an entrepreneur here though.

• 2:07:44Dylan Patel?

  That's the real challenge is because you have to manufacture at some really absurdly large scale or design something which in an industry that does not allow you to make custom memory devices.

• 2:07:53Dwarkesh Patel?

  Or use materials that don't work that way.

• 2:07:56Dylan Patel?

  So there's a lot of like work there that I don't, so I don't necessarily agree with you, but I do agree it's like one of the most important things for people to invest in. You know, I think there's, it's, it's really about where is your, where are you good at? And where can you vibe and where can you like enjoy your work and be productive in society, right? Because there are a thousand different layers of the abstraction stack. Where can you make it more efficient? Where can you use, utilize AI to build better and make everything more efficient in the world? And produce more bounty and like iterate feedback loop, right? And there is more opportunity to today than any other time in human history in my view, right? And so like just go out there and try, right? Like what engages you? Because if you're interested in it, you'll work harder, right?

• 2:08:37Dylan Patel?

  If you were like have a passion for copper wires, I promise to God, if you make the best copper wires, you'll make a shitload of money. And if you have a passion for like B2B SaaS, I promise to God, you'll make fuckloads of money, right? I, I don't, I don't like B2B SaaS, but whatever, right? It's like, whatever. Um, you know, whatever you have a passion for, like just work your ass off, try and innovate, bring AI into it, um, and, and let it, you try and use AI yourself to like make yourself more efficient and make everything more efficient. And I promise you will like be successful, right? I think that's really the view is not necessarily that there's one specific spot, because every layer of the supply chain has, you go, you go to the conference there, you go to talk to the experts there, it's like, dude, this is the stuff that's breaking and we could innovate in this way.

• 2:09:22Dylan Patel?

  Or like these five extraction layers, we could innovate this way. Yeah, do it. There's so many layers where this is, we're not at the Pareto optimal, right? Like there's so much more to go in terms of innovation and inefficiency.

• 2:09:32Dwarkesh Patel?

  All right. I think that's a great place to close. Dylan, uh, John, thank you so much for coming on the podcast. I'll just give people. And then the reminder, Dylan Patel, semianalysis.com, that's where you can find the technical breakdowns that we've been discussing today. Asianometry YouTube channel. Everybody will already know of Asianometry, but anyways.

• 2:09:54Dwarkesh Patel?

  Thanks so much for doing this. This was a lot of fun. Yeah, thank you. Yeah. Thank you.