⚡️ Which GPU Clouds Are Actually Good? | ClusterMAX 3.0

Watch the full show on YouTube: https://www.youtube.com/watch?v=MWX36ZYnsm0

• 0:04Speaker 00

  Hi, Jordan. Congrats on launching ClusterMax. What's your brief history as a personal introduction and also journey into building ClusterMax?

• 0:16Speaker 01

  Sure, I joined Semi Analysis last summer, last June, and been working on the technical staff. We've grown a lot over the past year. Anyway, I guess personal background is that I spent 10 years working at Hewlett Packard Enterprise. I was designing a lot of the hardware systems that these NeoClouds were buying back, you know, before ChatGPT launched and then with the big craze in 22, 23, when a lot of them came into the market. So a big Semi Analysis reader, read every word, knew a bunch of the guys in like group chats on the side, and saw a lot of my opinions represented in ClusterMax 1 when Dan and the team put it out.

• 0:58Speaker 01

  And the best way to get a job at Semi Analysis is to find something that Semi Analysis is doing and criticize it, and then Dylan hires you to fix it, which was what happened in my case.

• 1:09Speaker 00

  You gotta tell us, what were the criticisms of ClusterMax 1 and 2?

• 1:14Speaker 01

  Well, I did 2 all by myself, so I can give you the critiques.

• 1:17Speaker 00

  No criticisms there.

• 1:22Speaker 00

  What changed from 1 to 2 and then, you know, let's let's kick into what's new in 3?

• 1:25Speaker 01

  Did you see it on screen, guys? It's perfect as is. I don't know what you would do. Criticize it now. Okay, yeah, ClusterMax 1, I mean, there was just a lot of providers missed, like 24 total providers, and a lot of them, big ones, you know, they should have been testing and weren't. Small team, didn't have the time. And then there's a big focus on Slurm. To me, Kubernetes was really important, need to, you know, test that as well.

• 1:52Speaker 01

  In terms of like criteria and stuff, we haven't changed a lot, but yeah, all the way back to ClusterMax 1, you know, the rankings page looks a lot different, or there's just a lot less names. And releasing the market view also, I believe at the time it was around 120 total providers, yeah, 111 emerging NeoClouds there in the middle. And we didn't release the full market view this time because we paywalled that behind our NeoCloud model, where we track all of these providers. A lot of them are public, a lot of them are going IPO soon, so we've got this whole financial model for the industry now, but yeah, 3.0, we're up 323 total providers that we track. And I have some people this morning in my inbox reaching out saying that they've got some 3,000 GPUs and they want us to test and we gotta, you know.

• 2:38Speaker 01

  Anyway, it's been a big evolution. I think the big thing in terms of testing is that with ClusterMax 1, you know, all the testing was hands-on. This is pre-CLOUD code, pre-Codecs, beginning serious. And so we did a lot of the testing hands-on by like just running benchmark scripts manually. And now we've been able to develop like this whole repo over the past year and a half of industry standard benchmarks, custom benchmarks, stuff for reliability and burn-in. You know, it's it's like very automated now where we can launch this stuff with the coding agents and have them run and fix a bunch of issues autonomously without us involved.

• 3:17Speaker 01

  And yeah, I mean, just the breadth of coverage as well as the depth of coverage, I would say, has improved over time.

• 3:25Speaker 00

  I mean, I guess introduce it. What is ClusterMax? What is NeoCloud ratings and rankings? What are like the main things you guys are actually testing for here?

• 3:36Speaker 01

  Yeah, so there's 10 criteria described on the website. Basically, a NeoCloud we define as anybody who rents access to compute, sells access to compute. So NeoCloud implying that there was hyperscalers and legacy clouds that existed before ChatGPT. This is AWS, Google, Oracle, Azure, also some of the old school clouds like a Rackspace or DigitalOcean or even like a Cloudflare, Akamai, anybody who's like running a big data center footprint and renting access out. There'd be a cloud and then the NeoClouds are the new ones that have come along. We say everybody's becoming a NeoCloud because what makes NeoClouds different is that they're specifically focused on AI.

• 4:17Speaker 01

  So they deploy GPUs and other accelerators for AI workloads, this is training, inference, increasingly RL. And that's what we try and test on is who's going to give you the best performance, security, reliability, and just like ease of use, pricing, availability. We try to go through comprehensive assessments instead of individual benchmarks. Back when ChatGPT came out, a bunch of people were reaching out to Semi Analysis saying like, hey, I'm trying to decide between five providers, like how should I think about this? How should we go about it? They were, you know, bothering Dylan with the same questions over and over because like, hey, I just raised $20 million, I'm about to go give 10, 15 million of it to a provider.

• 5:03Speaker 01

  You know, it's my whole seed round. Like, this is a really critical decision for me. I need to get everybody's feedback. And so instead of Dylan and team like answering that question individually, custom every single time, I was like, why don't we write an article to give our like coarse grain, first thoughts, here's our like quick tier list, rankings, podium, and then you can engage us for more detailed stuff, you know, consulting DD, whatever it is for your specific, you know, requirements and we can dig in further from there.

• 5:35Speaker 00

  There's a lot there. The one that comes to mind is the criteria of the 10 criteria. The one that comes to mind is the one that you guys quoted, which is Ilya Sutskever saying every, all of them have terrible cybersecurity.

• 5:51Speaker 00

  What's, what is the state of cybersecurity in NeoClouds?

• 5:55Speaker 01

  I mean, you said it, terrible, yeah. It depends who we're talking about. Look, yeah.

• 6:02Speaker 00

  And also I'll just say hi to Dylan, who is just during this.

• 6:06Speaker 01

  Good enough, Dylan.

• 6:07Speaker 02

  So sorry for being late.

• 6:09Speaker 01

  Dylan, what's the state of NeoCloud cybersecurity right now?

• 6:12Speaker 02

  So, so I mean, we, you know, I think security is really bad. Even back from the first ClusterMax, there was pretty significant security issues. People not properly implementing network security controls, and so we were able to see other people's jobs, see other people's data, see other people's storage, et cetera, in the first generation of ClusterMax. We escalated to the firm as well as Nvidia, you know, and ranked that company underperform, of course, and strongly recommended people don't use them. And in the second generation of ClusterMax, we saw similar stuff. I think a lot of people got better because we highlighted stuff. But in the third generation of ClusterMax, we also tested, I think, you know,

• 6:56Speaker 02

  Jordan and his team tested this and saw pretty major security issues. So we posted a newsletter titled "Most NeoClouds Suck at Security."

• 7:07Speaker 02

  And then, you know, a handful of hours later, Ilya tweeted about most NeoClouds security sucking. They're obviously a very security-conscious firm. You know, they have a lot of folks who are from the former cybersecurity world. So I imagine they've done their own work on a number of clouds there to see and test. And it's it's pretty clear, right? Like, it's so bad. In one case, there was, we were literally able to see national intelligence of a certain country's like stuff, right? Like they had multiple national security agencies and military-style intelligence stuff running in that NeoCloud. And it's like, good God, we're just like here to test the cluster and we can like see this stuff.

• 7:52Speaker 02

  Imagine if we were malicious. I'm sure there were people who were malicious or could have been malicious on that cluster as well. So thankfully they fixed it, I think. But like, you know, we shouldn't be the one that has to find whether or not someone is

• 8:08Speaker 02

  you know, good or bad at security. But you know, I think the industry generally needs that service. And so that's, that's a big area of our criteria.

• 8:16Speaker 01

  Yeah, and I think on the technical side, like this is, our checks are very simple. Is your software version up to date? Are your drivers up to date? Like, is it configured correctly? I mean,

• 8:31Speaker 01

  these are relatively simple checks, but when people are running versions of software that are 2, 3 years old and they have well-documented CVEs online, it's like you don't need a frontier model to create a POC exploit of this.

• 8:51Speaker 01

  You just need to know the right place to look. And so as part of the article, we also put out a free CLI. People can install ClusterMax, Cmax, and then use it to check the software versions on their clusters and identify, you know, a quick, like you get a link in the terminal output to show you where to upgrade stuff, trying to help people keep the software up to date. This is the bare NeoCloud security.

• 9:13Speaker 02

  The other thing is the network keys, right? There's InfiniVAN and Ethernet have various like, at least on InfiniVAN side, it's like M keys, P keys, all these things are those properly configured and deployed.

• 9:24Speaker 02

  It's, it's, it's shocking how bad most NeoClouds are though. Like, it is really bad.

• 9:29Speaker 00

  Well, you guys, I guess, anything interesting, any major jumps, any improvements that they've made from 1, 2, 3? Anything you want to highlight?

• 9:39Speaker 02

  Our bar on ClusterMax is, you know, its first generation was like super low. And each generation, we've made it harder and harder. Hence, the trend has been for people to get downgraded. There have been a few folks who have been upgraded, of course. Google's done a really good job at getting upgraded across the ClusterMax generations. I think they started out as bronze and moved to silver and now gold. Or just silver, sorry. Oh, no, gold. Yeah, yeah. So Google's consistently upgraded. While there have been a number of clouds who have been downgraded across the generations because they sort of stalled. They improved a little bit, but they didn't improve a lot. And we keep trying to raise the bar. And teach everyone, hey, these are all the things you should do to be better.

• 10:24Speaker 01

  Yeah, and the bar for this time was like latest and greatest. So are you deploying GB300 and V L72 in 2026? You know, this is incredibly popular chip. It's what all the frontier labs want to buy. It's what all of the Neo Labs want to buy. And the typical customer that comes to us and says, I've raised $100 million. I want to spend it on compute. They want a GB300 and V L72 cluster. And so some of the providers that were in gold originally do not offer GB300 and V L72. Why not? Well, you got to contend with a whole bunch of different things. There's a new network and V L72 implies it's a scale up domain in the rack. You got to go to ARM CPUs. It's a Blackwell GPU. You have an 800 gig network. It's mandatory direct liquid cooling.

• 11:10Speaker 01

  So from the physical data center to how you deploy the operating system, how you like manage the drivers in the network. Just about everything's different. And when people say, oh, we just like made a strategic business decision, choice not to do GBs. Well, what happens when Vera Rubin comes next year? Who are you going to trust? And so the trend is like, if people are going out and they're deploying 10,000 GB300s. And we test the cluster and it works. And we engage with the support team and we give them feedback on some things we think we have issues with and they fix it in an hour, not two weeks. They're moving up the rankings. Other people that are not deploying these latest and greatest GPUs or they're doing it and it's completely broken. It's taken them a long time to fix it.

• 11:52Speaker 01

  Or it's we find security issues or we hear from customers huge issues with reliability, which is probably the number one criteria that like these top customers care about. They're moving down the list. So that's a bit of the dynamic there.

• 12:08Speaker 03

  So assuming most of these, you know, bronze, silver honorable mentions are good to rent from. What's the state of availability, cost, compute? So say you are in Neo Lab, you just raised tens of millions of dollars. You want to start to rent. How does that market look now compared to one, two, three years ago? Things getting better? Worse?

• 12:28Speaker 02

  Good fucking luck.

• 12:32Speaker 02

  It's really tough. It's the toughest it's ever been to rent compute.

• 12:38Speaker 02

  Obviously Anthropic and OpenAI have been buying smaller and smaller cluster sizes. But also all of the inference providers are more and more profitable. It was one thing when say Base10 was running at 10% gross margins, which I think might have been like six months ago, right? On inference or nine months ago. But it's another thing when, you know, Base10 fireworks together and many other companies, even small companies like Morph are running at 60% gross margins on open models using V L M or S G L A and using just a little bit of optimizations beyond that. And able to do an awesome job. And then, you know, companies like Infraact and Riddicark as well. So, so anyone who can like get access to a GPU can immediately make money.

• 13:22Speaker 02

  And so ultimately it's like, well, great. I'm a Neo Lab who wants to train an interesting, cool new model. Well, I'm also competing against people who make money now. Whereas before I was mostly competing against money losing, you know, OpenAI, money losing Anthropic, money losing labs. Very few people are making money off of GPUs. Now everyone can make money off of GPUs. And so it's just very, very difficult to get any sort of compute.

• 13:43Speaker 03

  What level are the labs? Yeah, what level are the labs going down to? Compared to like major, you know?

• 13:51Speaker 02

  They used to only go like 8K, you know, 8K GPUs from a provider. You know, and it's steadily gone down. We've even seen as small as 1,000 GPUs get rented by the labs now. And so 1,000 GPUs is closer to what a Neo Lab wants. And so you've squeezed out the market a lot. And then when you look at like the Base10s, the Fireworks, the Togethers and all these other inference providers. They're cooled like, they're cooled like four nodes. Like they're modal, right? These guys are cooled like even four nodes at a cloud. So ultimately it's like, you know, obviously their orchestration is tough. Modals talked a lot about that actually, right? Their orchestration across like 20 different clouds. Well, it's, yeah. So ultimately it's like a very,

• 14:36Speaker 02

  it's challenging for anyone to get any sort of compute if, you know, modal can make money hand over fist off of even four nodes from a company.

• 14:44Speaker 03

  Yeah, it's bad. I mean, like even individual, you can't spot rent single nodes anymore. Like down the whole stack is cooked. What about the state of the labs? How do you segment the large players, OpenAI, Anthropic, Google? Where's the split of who has the most compute? Anything interesting on the roadmaps of where you think who's investing the most?

• 15:07Speaker 01

  I mean, it's clearly OpenAI and Anthropic that are leading the way in terms of both renting from like large scale bare metal hyperscalers. In other words, you know, OpenAI is a driver of CoreWeave's business, also Microsoft. They're also exploring with other hyperscalers. They're doing self-filled as well, like massive stuff with Oracle. And then Anthropic, obviously they got their big AWS partnership with Rainier. They've also got a lot of TPUs with Google Cloud. They've signed stuff with CoreWeave. They've signed stuff with, you know, others. And so they're, they're the big driver. I mean, you just look at ARR and something has to drive that. And so it goes to the compute side.

• 15:46Speaker 01

  With that.

• 15:46Speaker 02

  Another aspect of this is, you know, a lot of these providers like, like a Crusoe have fantastic bare metal, right? But that's not what we're testing in ClusterMax. We're not testing bare metal. I mean, it is part of the criteria. But a lot of the criteria is a managed cluster, right? Managed Slur, managed Kubernetes, all of that sort of stuff. So it's a different level of service that people want on top of the GPU. The original ClusterMax was titled, also like a big chunk of it was called how to rent a GPU. We're also just teaching people what they should do to rent a GPU.

• 16:19Speaker 02

  And in our view, most of these smaller folks, you know, if you're super small, you don't care about, you know, your node being bare metal. If you're super, super big, you want bare metal because you're going to put your own, you know, fork of Slur slash Kubernetes and do your own stuff on managed services and such like that. But most customers want a managed Slur, managed Kubernetes. And for the cloud to like hot swap nodes for them. When one fails, you know, the biggest customers will manage stuff like this. But, you know, the very, the midsize and smaller customers want that sort of stuff managed for them. And so ClusterMax is focused around that. So companies like Crusoe will rent bare metal, right? They have massive contracts with companies.

• 17:02Speaker 02

  Or they'll do great in data centers or they'll, they're starting to do well in, you know, inference API. But on managed services, they're not doing well. And that specifically is what ClusterMax is targeting is the managed service layer, right? Cause it's otherwise it's like too complicated to, you know, it's, it's many different areas people want to rent from. Some people want bare metal. Some people want managed services. Some people want tokens out. And so ultimately it's a range of options.

• 17:28Speaker 02

  As far as, you know, even Google is renting some compute from folks, right? They signed a deal with CoreWeave. They signed a deal with SpaceX. So, so pretty large amounts of compute that they've signed over, I think over $100 billion worth of compute that they've signed from various customers. Nebius also has deals with the labs and such. And so,

• 17:47Speaker 02

  but these guys are only going for bare metal. OpenAI and Anthropic are definitely lion's share buying the most. Google, Google is definitely falling behind. At this point, OpenAI and Anthropic have as much compute, if not a little bit more. By the end of the year, they'll both have more compute dedicated to R and D than DeepMind. So yes, Google still ultimately has more compute than OpenAI and Anthropic for now. But a lot of that is sold out to other places and dedicated to other offerings. But DeepMind's R and D compute versus OpenAI and Anthropic's R and D compute. The two labs have more compute now because they've reached that scale. So this whole like, oh, Google has the most compute they'll win is, is actually not really the case anymore, right?

• 18:29Speaker 02

  They still have a cost advantage because they build the TPU and it's much lower cost. But ultimately it's, it's, it's less, DeepMind has the less compute than OpenAI and Anthropic.

• 18:41Speaker 03

  Yeah, it's a, it's a big, big shift. Like one of the

• 18:45Speaker 03

  the biggest insights for me at A I E was running into some DeepMind researchers and they were very depressed because they're like, look, our leadership keeps selling the compute that we want to other people.

• 18:56Speaker 00

  Yeah.

• 18:58Speaker 03

  I mean, imagine being XAI, right? Like.

• 19:01Speaker 00

  Well, yeah, you know, I thought SpaceX was a NeoCloud. What the hell? Why are we unavailable?

• 19:06Speaker 01

  They're on here hard. Yeah, well, I think as we can test, they'll move up the rankings, you know? They'll be a competitor. I think SpaceX and the Google deal, for example, is super interesting as well as an anthropic one, because it goes to what Dylan was saying about the fact that the labs are considering becoming NeoClouds on this opportunistic basis. Meta's considering the same thing. You see Mistral and Poolside in there as well. The companies have this skill set on building data centers and managing them. And then when there's an opportunity to monetize the compute, either through renting it to other people or through serving inference, they want to pursue that, but only to a certain level. And they always want the optionality, bring the compute back and use it for those research

• 19:51Speaker 01

  or hero runs as soon as they can afford to. They want to use the maximum amount of compute they can afford to research right now. And all the extras are, you know, opportunistically making the money to fund all that research. And so SpaceX's contract with Google, for example, has mutual 90-day cancellation rights. So SpaceX's research starts to work, they can pull the compute back. Google stops making money off of that compute on some inference endpoint, they pull the compute back. It's a really interesting structure where both are kind of, you know, in this little standoff saying who's gonna blink first, who believes in AGI more.

• 20:33Speaker 01

  As going.

• 20:34Speaker 02

  I think the other aspect of, at least for SpaceX, right? If you're an XAI employee and you got bought by SpaceX and then SpaceX sold all the compute, at least you're not depressed because like, you know, you're like, okay, look, Elon's building a shitload more, I'm gonna have access to this. And he's doing this because it helps us be more financially solve it so for the IPO.

• 20:58Speaker 02

  With Google, it's like, why the hell am I selling the compute? You know, I have, I'm already a very profitable company. Why can't we just use the compute to build models and sell those models at way higher margin instead of giving it to Anthropic or OpenAI or other firms? You know, SSI is also a customer of Google.

• 21:15Speaker 01

  Dylan, what do you think about the dynamic of like, Google funding the people that leave to go do a NeoLab? To buy GCP. In other words, like Dave Silver, Jeff Dean, they spin out, they get funding from Google Ventures to then go spend on GCP.

• 21:33Speaker 02

  I mean, I think it's a fantastic move if you look at just GCP as its own company, right? Thomas Kurian continues to pump his, you know, his revenue, his operating margin, his operating income. But for Google as a whole, it's really dumb, right? Like, you should not incentivize your best researchers leaving. You should not incentivize.

• 21:51Speaker 01

  So Jeff Dean has a breakthrough.

• 21:53Speaker 02

  Sold at $20, $30 million a megawatt when you could run models on it for $50 million a megawatt.

• 21:59Speaker 01

  So if Discovery Loop has a big breakthrough, do you think Google is able to keep them in their orbit? Or do you think Jeff Dean goes and takes that breakthrough and does something without Google being exposed to any of the upside? Like, they would be an investor, they're obviously the cloud service provider.

• 22:15Speaker 02

  I mean, go look at Anthropic, right? Like, yeah, true, Google did invest in Anthropic. But if Google didn't invest in Anthropic, would Anthropic be here today? Because Google is a really early believer.

• 22:28Speaker 02

  Like Opus 3 and Sonnet 3 and 3.5, like those, I think, were just trained on Google's infra for sure.

• 22:36Speaker 02

  You know, whereas, and then they later brought on Amazon, which was, you know, Amazon's also great, but at least Amazon, you know, is just like, they don't have a business of training models and selling them, right? In the same way, they have some folks, but like, nothing, nothing crazy. But Google does. And so it's like, did Google like stand up ultimately their biggest competitor and fighter? Now Anthropic is not only renting from Amazon and Google, but also from numerous other clouds. They have deals with like 10 plus clouds now. And then Anthropic is also building their own data centers, leasing their own data centers, starting to fill them up, working with FluidStack to deploy TPUs in their own data centers. So sure, Google's still getting revenue, but it's like, well, I used to rent from you. Now I'm buying TPUs from you, which is less margin for you.

• 23:22Speaker 02

  And now in the future, I'm gonna, I'm now building my own chip. I'm gonna do everything without you. So like, it's pretty clear what happened with Anthropic. Jeff Dean, if he does something amazing in Discovery Loop, then why would they continue? Why would they be stuck to Google, right? They can use any cloud. Switching clouds is ultimately not impossible. It's not, it's not, you know, data gravity and all this stuff that people used to talk about with clouds in the pre-AI era is a lot less relevant in the post, in the AI era, because AI can just help you migrate everything. And a GPU, a GPU is not just a GPU, but like, you know, a GPU is largely workable as a fungible asset across clouds if you're good.

• 24:02Speaker 01

  So Rex, what do you think? You think Google's cooked, man?

• 24:04Speaker 00

  I mean, I think you guys are much more of an expert than I am. And there's like, yeah, this, Google's just like a conglomerate of a bunch of different things, right? So I do think like the data advantage, I mean, there's data gravity as NeoCloud, but the data advantage of Google is the thing that most people could keep coming back to, right? And there's just organizational challenges and like, you know, now there's new leadership. You know, God bless Decore, who has the hardest job in the world to fix this. But like, I mean, he's a very, very capable guy. And like, I think, I think they, they know, like, you know, it's like, not a secret that like,

• 24:47Speaker 00

  they do need a reset. And to me, Demis, like, basically stepping out of the leadership seat was just like a complete black swan. Like, I just like, I never thought it was possible. Like, maybe I just don't know him personally. Where like, he's like, yeah, actually, like, I just care a lot about Isomorphic. But like, I mean, you know, he founded it, like, he's, you know, he's sort of like the spiritual leader of DeepMind. And so like, this is probably the first major frontier lab transition, leadership transition, fully. Can we say that?

• 25:23Speaker 01

  There were like two days where Sam left, right?

• 25:25Speaker 00

  But, yeah. He never left in anyone's like hearts.

• 25:31Speaker 00

  But yeah, this is the Tim Cook of like.

• 25:36Speaker 02

  Ilya leaving OpenAI?

• 25:38Speaker 01

  No, not the leader.

• 25:41Speaker 00

  Yeah, it was like, kind of like the, yeah, yeah, I mean, yes, okay. So like, obviously people split, right? Anyway, so that's my rough take. I don't think I'm gonna have like spicy takes to you guys. I did want to pick up on one thing. This is like just the chance to like catch up on like the last six months of like crazy news. You guys mentioned Poolside. Have you never actually written about Poolside? Because they're not that big. But they do have.

• 26:07Speaker 00

  They got bought by, their employees got bought by Nvidia for 12 billion. What's going on?

• 26:13Speaker 01

  Okay, yeah.

• 26:15Speaker 02

  It's really cool, right? Because Nvidia is paying for all these researchers so they can make Nematron better.

• 26:21Speaker 01

  No, this is defense.

• 26:22Speaker 02

  Poolside is, Poolside, Poolside, like, you know, from the beginning, like, they just like, all the cloud, NeoClouds kind of sucked. And so they started building a lot of their own infra and they got better and better at building their own infra. And then they were trying to deploy their own clusters and had a partnership with CoreWeave on this and stuff like that. But they've just been slowly moving down the stack, maybe the first AI lab to become, you know, running their own infra. Even before OpenAI and Anthropic were trying to build their own infra, Poolside was trying to.

• 26:49Speaker 00

  Which is wild as a side project. It's like, people have to do this as a full-time job. This is a side project.

• 26:56Speaker 02

  Well, now it's their full-time job, right? Because they sold all their researchers to Nvidia for Nematron. Because Nvidia clearly wants Nematron to be a lot better. I think they've sort of lost hope in a lot of the open source folks. I mean, sure, Reflection is still trying, Thinking Machines, open source some stuff. But ultimately, I think Nvidia is losing sort of hope in American open source, so they think they just have to do it themselves. So they've bought a bunch of employees. And yes, I thought it was $7 billion, not 12. But anyways, regardless, I thought it was like a $7 billion and then a $1 billion investment out of 12.

• 27:33Speaker 00

  We have the, we have the details.

• 27:34Speaker 01

  Yeah, yeah.

• 27:38Speaker 01

  License and employee, they're six. The number's six.

• 27:40Speaker 02

  Always, always take the bigger number. But you know, I think at the end of the day, it's like Nvidia can say, well, actually, we only paid one-fourth the price. Because all of this money is just going back to buying GPUs. So did they pay, you know, $7 billion, six for the employees and one in an investment? Or did they actually pay, you know, one and a half? Because all seven of that is going into buying GPUs. And in fact, them doing this is going to incentivize other people to invest a bunch of money into them. And so actually, it's probably a positive thing to hire all these people. Like, they actually didn't spend any money because now they've created a new customer, a new cloud that's going to build a massive cluster, right? Like, imagine, you know, they've got $7 billion now of cash. They raise another few billion dollars. Let's just say they have $10 billion of cash on the balance sheet.

• 28:26Speaker 02

  And they're able to get a loan to value of like 75%, you know, 25% down and 75% loan. So now they have $40 billion, they can build a gigawatt. Awesome. This is another gigawatt scale NeoCloud now. So they've just made a new competitor.

• 28:41Speaker 00

  We have that lower down in the article. It's, yeah, I mean, I don't know, I don't know if that, I mean, this is the playbook. If this is the playbook, honestly. Lower down, lower down, with the guy who's leaving it, Robert Bonar. And like, if this is the playbook, a lot more people should be doing this.

• 29:02Speaker 03

  Should be doing what?

• 29:04Speaker 00

  Should be doing deals with Nvidia like this because Nvidia will king make you just because they want more Neo Clouds.

• 29:11Speaker 01

  Yeah, I mean, not everybody releases the second best open American model.

• 29:15Speaker 00

  Yes, true.

• 29:16Speaker 01

  And spends years building it. I mean, we're kind of brushing over the fact that Poolside did build something interesting. And to me, both this and Hugging Face are defensive acquisitions by Nvidia, who does not want their competition snapping up a lab that actually has something going for them and having some momentum where they can build the real experience that's required to build real training and inference clusters. In other words, it's so important for building an infrastructure company that you actually understand how the workload works. And you have opinions that are informed by training models and running. It kept represented in the, if not crypto miners who decide, hey, I found some site.

• 29:57Speaker 01

  I got some power. I have some money. They don't have the ability to execute like somebody who has proven they can train the second best American open model in the world, which Poolside did, right?

• 30:10Speaker 00

  Yeah.

• 30:10Speaker 03

  Say more on the...

• 30:13Speaker 03

  I was gonna say, you mentioned Hugging Face in there. How do you guys see the Hugging Face acquisition? Anything you want to comment on there?

• 30:18Speaker 01

  I think it's quite similar. I think it's defensive. I don't think that Nvidia wants the influence in the ecosystem of other companies coming in and owning somebody that's like critical.

• 30:29Speaker 02

  There's a galaxy brain take here, right? Which is now, now Hugging Face, you know, can sue the fuck out of OpenAI for hacking them. And now Nvidia can, has that hammered? I don't think so. I don't think so. But I think, I think, I think Hugging Face is obviously like, you know, leagues ahead of everything else. Some, some folks on my team have started submitting a bunch of PRs for ModelScope to work across the ecosystem.

• 30:58Speaker 03

  By leagues ahead in what sense?

• 31:01Speaker 02

  Just like for, for being the open source repository for.

• 31:05Speaker 03

  I see.

• 31:06Speaker 02

  Yeah, just like what Hugging Face does.

• 31:09Speaker 02

  But then ultimately, a lot of them are, you know, like, it's, it's, it's, you know, the question is like, does Hugging Face, you know, there's a way you can just like deploy Hugging Face models on Amazon Trainium if you want, right? You can just click a button and deploy them. Does that go away? And is it only your GPUs from here on out? Or is there any closing up that happens? We'll see, but ultimately, it's, it's, it's, I don't know. I also love that the price was the emoji code.

• 31:39Speaker 03

  That much.

• 31:39Speaker 02

  So sick.

• 31:40Speaker 01

  And the team's incredible. Look, I think we're kind of brushing over the fact that Hugging Face and Poolside have incredible talent in both these organizations. And like, a lot of people would see that's the motivation for Nvidia to acquire them. I say it's defensive because if you look at Nvidia's business, they're generating almost $50 billion of free cash flow a quarter. So 200 billion a year. And there is a corporate strategy team at Nvidia that gets some amount of that money and says, what, what, we need to deploy it, right? They can make investments. They can make acquisitions. And 10% of free cash flow to acquire Hugging Face and Poolside? I mean, it's a no brainer to me that these would be good assets on your team at working

• 32:22Speaker 01

  at least not for your competition. Let's say Hugging Face stays independent and they follow the letter of the law in the acquisition and they keep supporting all these other chips and stuff. If they were acquired by somebody else, they would be actively working against Nvidia's interests. And I think that's not what they want. So I think you're going to see more of this. If companies grow up a little bit, they build this team, 100 or 200 incredible people.

• 32:46Speaker 01

  They want to bring them into, make sure that they stay in the Nvidia ecosystem as opposed to going and competing against them with somebody else.

• 32:54Speaker 03

  Yeah, it makes sense not to discredit anything Poolside's done. We're pretty big fans. We had them on the pod a month before they got acquired. For people interested, the Model Factory worked, approached to training foundation models. Very, very good way that they built all this out.

• 33:09Speaker 00

  Yeah. To me, you know, coming, you know, maybe, maybe this brings us back to ClusterMax. The way that you, that all Neo Clouds should be proving, like ClusterMax 4 is like, just train a frontier model, please. Yourselves, right? And then like show that everything works.

• 33:26Speaker 02

  I don't know if everyone, everyone should have the skillset to train a frontier. That's just like.

• 33:30Speaker 00

  No, but if. And there's a lot of open recipes and stuff, right?

• 33:33Speaker 03

  We need a mosaic.

• 33:35Speaker 01

  Let's, let's talk about what it takes to get there, right? If you're going to train, I think, also, I think Poolside and probably Hugging Face have probably both enter caned acquisition options in the past to the tune of similar money. And it's.

• 33:49Speaker 00

  Yeah. So specifically Nvidia tried to buy both side, sorry, Nvidia tried to buy Hugging Face at the start of the year, right? Guys, in case people didn't know.

• 33:58Speaker 01

  Well.

• 33:59Speaker 03

  I mean, I think also Nvidia is on a bit of acquisition craze. There were rumors of Thinking Machines and a whole bunch of Nvidia acquisition.

• 34:07Speaker 01

  Yeah, I know. Of course, founders kind of have the, they have the option to like cash in when they want. For a lot of these people that have built these things, right? So when you decide to cash in, I think Poolside is probably looking out there and going, okay, I've built the 10,000 GPU cluster I needed to get Laguna out. But if I want to compete, I need a research cluster that's 100,000 GPUs, 400,000 GPUs. Like, how do I keep up with the open frontier into next year? I think that's pretty hard to keep going. And the money comes and they're like, okay, sure, let's just cash in and build it with the other people. That, that seems fine. But the, the other thing worth pointing out is like on the technical side, Poolside's papers are really good on the infrastructure stuff. I've talked to the team.

• 34:52Speaker 01

  If you're going to train a frontier model, you need to have health checks on your cluster. You need to have high reliability. You, you can't just have this model like not make progress as you're training and constantly be failing. Poolside has that. They have like an incredible suite of, they just have incredible infrastructure. So the, the idea that people with that talent would be able to provide that as a service to other people, I think is like, just obviously true. And there's only limited people that have that experience, as we're saying, like even when we get into the silver and bronze tier here. People are not training frontier models on most of the silver tier and all the bronze tier, let's say.

• 35:31Speaker 01

  A lot of inference, bit of RL, a lot of research, but I wouldn't trust any of those guys frankly to build, except Azure, right? But to build 100,000 GPU cluster, like who has proven they can do that already?

• 35:43Speaker 00

  Yeah.

• 35:44Speaker 01

  No five companies?

• 35:45Speaker 00

  Yeah. Yeah. Okay, I do want to maybe dive in on one or two things that maybe you don't, don't actually cover. Maybe the best way, given the time that we have left, is to just address the criticism.

• 36:00Speaker 00

  You know, you've really, obviously, this is a very valuable list. Like, people do make purchasing decisions off of this. Periodic lads did come out and say like, yeah, actually like, you know, this, this helps us decide what, what we spend on.

• 36:16Speaker 00

  Yeah, I mean, you know, what is the criticism that most people have?

• 36:20Speaker 02

  I think, I think there's a lot of criticism for a lot of things. Like first and foremost, this is testing managed clusters. So that does not mean that a company is not good at other things. Right? Crusoe was downgraded for managed clusters, but they're one of the best data center builders in the world.

• 36:40Speaker 02

  If not like the best, they have, they have nearly the most contracted or the most contracted gigawatts in their pipeline and a bunch of sites that they're building on really rapidly. But this is not a criticism against their, you know, data center building capabilities. It's only a criticism against their managed cluster offerings that they were, they're still one of the better ones, right? But they were downgraded, right? You know, likewise, you know, other companies together has great inference endpoints that has nothing to do with managed services.

• 37:07Speaker 02

  You know, and, and so I think there's a lot of confusion around what this is testing, right? This is for managed clusters, not for data center construction or bare metal clusters or for inference endpoints. Which just already become a blurrier line because everyone is doing these things, right? The other question, the other big, major criticism we have is like, oh, that, you know, people are like, oh, are they taking money for this? It's like, no. We do not take money for any ranking. That would be illegal. I myself have personal investments in Crusoe. I have personal investments in FluidStack. And neither of those companies, you know, I did SPVs in both of them. Neither of those companies are in the top, you know, five clouds.

• 37:49Speaker 02

  Because, because that's not what, you know, I'm, I'm not here to be biased, right? Jordan will say he's like, he's like, he has free reign to like, do whatever he wants to do technically. He doesn't even know all the companies we do contracts with. If I look at the top 20 companies in the world, there are five companies that are not our customers. You know, like Eli Lilly and Saudi Aramco and stuff like that. But ultimately, it's like, those are much bigger customers than Core Weaver Nebius could ever be, right? Because, you know, they, they just have way, they're way bigger companies with way bigger budgets.

• 38:20Speaker 02

  You know, and so like AWS is a bigger customer than Core Weaver Nebius, for example, right? And yet that has nothing to do with the rankings, right? Everyone in the infra supply chain buys our stuff. It just is what it is, right? If you're tracking data centers, you want to know other people are doing in their data centers. If you're tracking accelerator supply chains, if you're trying to understand the cost of various kinds of chillers, if you're trying to understand the supply chain for networking and optics, you're going to buy our research and our data services. That does not mean, and you'll often commission us to do work beyond that, right? You know, but that doesn't mean that we're going to ever bias our rankings.

• 38:54Speaker 02

  You know, the, so, so sort of, I think, you know, there's a lot of criticisms around that, but ultimately, like, first of all, if we're taking money and ranking people and not disclosing it, that'd be illegal. But second of all, right? Like, you know, it makes no sense, right? The, the value we have to the industry is that we tell the truth and what we believe.

• 39:11Speaker 02

  If, if we didn't do that, then no one would care what we would say and we'd be screaming into the void. So our reputation is like what matters the most, right? Over a quick buck.

• 39:20Speaker 00

  Yeah, maybe one way to put it is like, you're, you're, you're a buy side service and people, and people are accusing you of taking money for the sell side. And you don't understand that buy side is like probably 1,000 times larger in terms of revenue for you than, than the sell side.

• 39:35Speaker 00

  I mean, I think people trying to poke at it's a paid list, you know, and that's just one IQ. The broader point is like, okay, people are misunderstanding what you're trying to rank here. And a lot of the claims are just, well, it's crazy that this is a better cloud. X is a better cloud than Y, right? But that's, that's not what you're actually testing.

• 39:56Speaker 01

  I hear two clear criticisms about the writing from people where I listen to the criticism because I, you know, respect their work and I, you know, they're not like anonymous, just shouting into the void. And those criticisms are, you guys didn't write enough. You should have released more data, you should have written more. Or this is 30,000 words, I can't read this all.

• 40:23Speaker 01

  So it's really hard for us to balance between those two.

• 40:25Speaker 00

  But it's a new, yeah, you're evaluating Neo clouds, of course, like.

• 40:31Speaker 01

  So for, for the, for the former, I'm, you know, interested in that criticism and we, we want to do better. And so we have this research product called the Cloud TCO Model that anybody can come and license. It's a yearly fee. We write notes every week about the trends in the industry. We have a dashboard, which we took a redacted screenshot of below where you can get all of our data. All of our performance data is on there. Yeah, it's, it's like, you know, you, you get us working on the research for you year round, weekly, rather than just for this individual, you know, kind of once a year release or every six months or something is what we do. So that's for the people who want more detail.

• 41:16Speaker 02

  There's another crowd of criticisms, which is like, you know, it's oftentimes someone who like owns one of the stocks. And we rank them poorly or positively, right? Like, I remember last time people were like, like, by the way, like, our view on a company doing better or worse in the stock market has nothing to do with Jordan and his team, right? There's, you know, we, we do, we do estimate revenue for all these companies going on a forward basis. And then you can like license our data and see what you think that should mean you should trade. We never recommend us to trade. But there have been times where like, for example, we thought Nebius was better than CoreWeave. And the reason we thought Nebius was better than CoreWeave, even though we think CoreWeave is still a better cloud than Nebius, is because Nebius's contracts are shorter term.

• 42:00Speaker 02

  And so with prices of GPUs spiking, Nebius gets to take the upside better, right? Like, that's something that we've released in our research in a separate, separate group of people, not Jordan and the technical staff that are working on ClusterMax. Or, or hey, like Iris Energy has this massive data center that's going to come online. We think it's going to get leased. And we would put a report out about that while at the same time saying you should never use Iris Energy's, Iron's managed services. And so like, it's like, it's, it's, it's, it's completely different things, right? You know, they sold that data center as a bare metal to Microsoft. And so we were, you know, our research on the institutional research side for someone who might be buying stocks is completely different from like, hey, ClusterMax is really made for people like Periodic Labs

• 42:41Speaker 02

  or Applied Compute or companies like that who have publicly came out and said they like, like ClusterMax and what we do there. You know, even OpenAI said they like ClusterMax and we have like a quote from them on, on that. We have a variety of different buyers who said they really appreciate the methodology and that we are pushing the bar up for all the clouds. And so I think it's, it's, it's, you know, if you're trying to buy stocks based off ClusterMax, that's like completely wrong. It has nothing to do with stock performance.

• 43:07Speaker 01

  Yeah, it's a hundred percent, man. This is strictly for buyers. This is for helping people buy compute. It is not for people to pick what stock to Yolo all their savings into.

• 43:21Speaker 00

  I mean, it, it will correlate at some kind of limit. I, I think like, I think there's a tendency to have literally, it's literally correlated. Oh my God, you guys are so, okay, all right, all right, all right. Jeez, oh my God.

• 43:35Speaker 02

  Right, like someone, someone who's terrible at running a cloud but has a gigawatt data center that they're building. And, and it's like actually credible, like a gigawatt data center. Well, they could be the worst ranked ClusterMax and they could still make shitloads of money. It like, it like, doesn't matter.

• 43:50Speaker 01

  At the current price, at the current price of the stock, many names are cheap, many names are expensive, like.

• 43:56Speaker 00

  Yeah, I, I do, I do, I do get it. Fundamentals versus, you know, one offering that contributes to your overall fundamentals, it contributes to the overall valuation and direction.

• 44:06Speaker 01

  It's a data.

• 44:07Speaker 00

  I, I used to be a hedge fund guy, I get it, you know? I'm just saying like, this is like, you guys are the domain experts and people do value the rating on their elements. But obviously this is way more buy side and bias in, you know, the, the Neo Lab.

• 44:22Speaker 01

  Yeah, I'm certain to just keep stressing this, but we genuinely have a core research product where people report on stocks and give opinions about the names. And many of the guys that work on that research, look, they ask us questions, but then they come out and they say people in the underperforming or unavailable tier were positive on the name. This is different than what we put out publicly and say you should rent clusters. And maybe I need another disclaimer on the future one that's like, this is not a cosign of the current valuation of their stock.

• 44:50Speaker 00

  It could be, you know, it could be really interesting for like a two by two or like a two dimensional chart where like, here's the ClusterMax ranking and then here's the semi-analysis like stock ranking. And like, it, it, the differences are going to be interesting.

• 45:04Speaker 01

  By the research. Maybe worth saying though, there is a lot more coming, which is that Dylan mentioned it. Individual names and logos on here have businesses that are much larger than just managed clusters, which is what we're raking here. And so we are going to be addressing that on the newsletter for inference endpoints, for bare metal, for our infrastructure, for sandboxes, for, you know, part of our L infrastructure, but kind of different. We're going to be writing about all of that and stay tuned for that because I think people will start to have a more holistic view of these businesses that compete in different market segments across inference, managed clusters and bare metal. And we've got a lot of focus on managed clusters here, but frankly,

• 45:47Speaker 01

  so many people are reaching out to us for help deciding on which endpoint provider to go with or which hosted RL training provider to go with, or which sandbox provider to go with. And that's a different question than who to go with for your managed cluster.

• 46:00Speaker 00

  Yeah, so, so is the solution more MAXs, more benchmarks?

• 46:03Speaker 01

  Yeah, that's the thought.

• 46:04Speaker 00

  Yeah. Okay. How many do you have? You have inference.

• 46:09Speaker 03

  You have one on like inference.

• 46:11Speaker 02

  When you go shopping, when you go to the grocery store when you're hungry, it's the same way, right? We want to do everything, we'll see, we'll see, hopefully we can execute on all of it.

• 46:18Speaker 01

  Yeah, maybe to be clear though, inference X today, the name is because it's for inference performance, but it's about chips. It's about comparing chips and systems. And we're going to be releasing something called Endpoint X where we test the serverless inference endpoint providers and kind of dig into those details on, because I think there's a very similar methodology. We know there's a similar methodology that we should apply where we, we go beyond just performance. We, you know, we look at cost, we look at reliability, we look at security, we look at monitoring infrastructure. There's all of this stuff that goes into an inference endpoint that says, just because somebody hits a higher target on batch one, you know, contrived tests, doesn't necessarily mean they're the right partner

• 47:02Speaker 01

  to scale up your inference requirements with when you need all that extra.

• 47:07Speaker 02

  Yeah, there's, there's providers with better performance, like raw, better performance, better optimization engineers. But then they're like APIs less stable, it's flakier. You know, there, there's, there's all sorts of reasons why it may not be as good for the endpoint, end customer, even though it's like more tokens per second per user and lower cost.

• 47:27Speaker 00

  Yeah, amazing. Do you ever, how will you cover the, let's call it the, the non-Nvidia universe? So, which is rising.

• 47:40Speaker 03

  You have some in here, right? Like TensorWave is AMD Cloud.

• 47:44Speaker 02

  Yeah, that's another one where I explicitly have an investment, by the way, guys. And, and we still rank them poorly. You know, time until like that investment in the first ClusterMax, I put, we put them as underperform, right? I think like a lot of this bias against like, it's, it's, yes, I might make an investment. I'm going to talk to them and try and help them, but at the end of the day, the ranking is the ranking. So we, we do want to extend beyond, but ultimately like, you know, FluidStack is not going to rent TPUs to us, right? Because they're renting everything to Anthropic, right?

• 48:14Speaker 01

  Maybe.

• 48:15Speaker 02

  We'll see.

• 48:15Speaker 00

  Maybe.

• 48:16Speaker 02

  I hope, I hope.

• 48:16Speaker 01

  I'm paying them.

• 48:17Speaker 00

  Yeah, it's kind of unexcusable that they're unavailable, like, come on, guys, like.

• 48:21Speaker 02

  No, I mean, it's, it's, it's valid at the same time, right? It's like, look, it's unavailable to us, so we can't, we can't, we can't rent it. Won't be able to rent it either, unless you're really, really freaking cool. Because they're selling everything, like SpaceX is not going to rent to us. They're not going to rent to you, SpaceX, they're not going to rent to, you know, random companies. They're going to rent massive.

• 48:41Speaker 01

  Dylan, I think, I think you need to work on your, I think you need to, this is the first time I'm ever giving you this feedback, but I think you need to work on your self-confidence, man. We, we can get it in there. I think we can get in there with these guys, yeah.

• 48:54Speaker 02

  Well, I think over time, like today.

• 48:56Speaker 01

  Probably a lot of these providers, even FluidStack and, and people that we know who are at SpaceX AI now, see value in reading the article and they see value in getting an outside opinion. Even if they're not necessarily going to provide the service publicly that they offered us, they, you know, they, they enjoy a third, you know, a, a technical opinion that's outside the company that's on us, that's going to give them real feedback. And we're, we're here to help for all of that stuff.

• 49:24Speaker 01

  To answer the question about multi silicon, it's going to be huge next year. Like with the Vera Rubin generation with Nvidia, we're going to see it because they're going to bring the LPUs in from Croc, the LPX systems, right? So even if we're just testing Nvidia, we're going to have multi silicon to test. AMD obviously has their own thing. Cerebras is becoming a Neo cloud for OpenAI, building their own data centers, right? Moving from just inference endpoint stuff to like really managing clusters of their chips for everybody else. We see Samenova coming to market, lots of the chip startups like Positron, Etched, Maddax, like lots of people are launching stuff like this. There's rumors, you know, obviously Fluid Stack's got TPU outside of GCP. There's more stuff coming there. Tranium, lots, AWS can produce a lot of chips.

• 50:07Speaker 01

  Maybe they won't self-build all of it in the future. They've got announcements on heterogeneous Tranium with Cerebras and stuff like that. So I think there's going to be a lot more for that next year. And we're going to have to contend with multi silicon as part of a comprehensive offering from the leading Neo clouds. They will need to be able to support heterogeneous compute in the future.

• 50:29Speaker 00

  And that's all focused on inference mostly, right? Like because of it's, you know, it's like.

• 50:34Speaker 02

  I mean, Anthropic, for example, uses TPUs for training primarily. So in that case, it's sort of the opposite, right? They don't actually, most of their inference is on Tranium and GPU. Most of their training is actually, especially like, you know, Fable 5 was trained on TPUs, for example, for the pre-training. And then the post-training and inference is mostly Tranium and GPUs. So it's a multi silicon. I would say in most cases, other silicon offerings are for inference, but there are some cases where people are using it for training.

• 51:04Speaker 01

  Yeah, but I think the key thing is inference versus training is not this hard to split now where there's lots of forward passes during RL. When you're doing research and when you're actually post-training the model. And so.

• 51:17Speaker 02

  In fact, most of the workload is forward passes, right? And so you might call that classically inference, even though it's for, you know, forward passes for reinforcement learning.

• 51:26Speaker 00

  Yeah, inference is part of the loop, yeah.

• 51:28Speaker 01

  There's a whole bunch of benefits of using, let's say, cheaper chips capable of high throughput for the forward pass in RL. But then there's a bunch of challenges with numerical correctness if you're using two different chips. Even just two different generations of Nvidia GPUs. B200 versus B300, you might, you have different amounts of memory, different batch sizes. There's, there's all these effects. So I absolutely expect there to be heterogeneous silicon used across training and inference. Pre-training, probably a homogenous cluster. People probably not going to risk that at like 400,000 GPU scale, but we'll see.

• 52:02Speaker 00

  Yeah, but like, you know, RL is like as big or sometimes bigger than the pre-trained workload right now, right? And so that actually means that you need like maybe a new form of disaggregation where like you have different kinds of compute in the same data center. That you shuttle back and forth.

• 52:19Speaker 02

  It doesn't even have to be the same data center, right? People are doing RL across many, many sites, right? This data center is doing token generation, that data center is doing token generation. And it's actually fine to do different types of chips there. It's just a very complex infra problem because you're going to shard your model differently and weight updates happen at different times. And there's all this.

• 52:38Speaker 00

  It's speed of light.

• 52:39Speaker 01

  For people listening to this right now are like, God, no, please, no.

• 52:44Speaker 00

  You have to.

• 52:45Speaker 01

  Data center, multi silicon RL updates on the fly. God, please, no.

• 52:50Speaker 00

  You're fighting speed of light. How are you going to transfer all this, this data?

• 52:54Speaker 03

  I mean, there's interesting architecture stuff being done on disaggregated, like decentralized RL. The MIMO 2.6 that just came out, it's a good open paper on how they super scale up the RL side of things. But on that point as well, you mentioned a lot of the custom chips. Anything on Huawei Chinese chips? Do you guys look at them?

• 53:18Speaker 02

  Yeah, we're tracking them. The Huawei chips are really getting better, much faster. For a while, they were like, you know, basically when they were banned by the US government, it took them like a few years to reset. You know, and the Ascend 910, B and C. But now they've launched the 940, 950 and 960 in pretty quick succession. They, you know, we're trying to sort of get access to them and test them along all bounds, right? Whether it be renting them so we can test the performance, whether it be, you know, looking at the specs and the manufacturing capacity and things like that. Also trying to access them so we can do a reverse engineering analysis out of our Oregon reverse engineering lab.

• 54:00Speaker 02

  We're trying to do work across all of these dimensions, you know.

• 54:04Speaker 03

  How's the, how's the availability of their clusters? Like are the labs okay without Nvidia? Any, any early signs?

• 54:14Speaker 02

  In general, yeah, I mean, people are super, I mean, the labs are using, you know, Anthropic uses four different.

• 54:20Speaker 03

  No, no, I mean, like the Chinese labs using their own chips from.

• 54:25Speaker 02

  So I think it was like GLM said explicitly, right? That they had 50,000 chips that were not Nvidia. They're a Chinese-made AI accelerators for inference of their newest model. As we extend out to other firms, you know, CameraCon, Ilvatar, Inflame and many others. There's, there's a handful of different companies. They're all pushing up the curve, you know, obviously they're further behind Nvidia. Huawei is definitely the best of them all. And then Baidu is spinning out their chip business as a completely separate company. So there's a lot of developments in the Chinese chip ecosystem and, and especially for inference, I think, you know, the Chinese, you know,

• 55:05Speaker 02

  as frontier models become, and even, even now the open source models become better and better at writing kernels. It's easier and easier to port stuff.

• 55:15Speaker 02

  And so ultimately you're, you're going to have more diversity and heterogeneity, which is why Nvidia has moved to this super fast pace of trying to release new chips every year. And new systems every year because they know they have to run as fast as possible at the speed of light. Otherwise they will get caught up too. Because the CUDA mode is graphically being deteriorated.

• 55:33Speaker 01

  But it's just volume, right, Dylan? Like the, these chips do work today, but they can't produce enough of them to really serve all their domestic demand. So.

• 55:45Speaker 02

  Yeah, I mean, it's, it's, you know, you know, if you look, if you do an H100 equivalents, or let's say if you do an GV200 equivalents. Nvidia is making, you know, millions and millions of chips. Google's making millions and millions of chips. Amazon's making millions and millions of chips. AMD is making millions of chips, right? But then if you look at the Chinese companies, really Huawei is the only one that even in gross chips, they're making a million plus. And then the Huawei chips are behind, so then you have to discount them for that. Everyone else is still in the hundreds of thousands of units and scaling production is very difficult, especially given export controls and stuff. I think China will ramp production rapidly across next year and the year after. And they will get to even tens of millions of chips produced by 2028. But it does take time.

• 56:29Speaker 00

  Yeah, I mean, let's not forget that they're the first to have a national plan around all this stuff. You know, when they get going, they get going in real scale, like unlike we've ever seen in human, human history. Okay, lots to discuss here. Congratulations on Questor Max and all the stuff that you guys are doing. I actually, I think we managed to hit on like many of the major topics of like the year so far. It's just great to get a catch up on like the hardware lab that you guys are building in Oregon. I'd actually love to see it. We should, we should do like some kind of trip.

• 56:57Speaker 02

  Yes, fly out, fly out. We will give you a tour.

• 57:00Speaker 00

  Yeah, yeah, give me a heads up whenever you are going and then we'll do a tour. I want to.

• 57:04Speaker 01

  Dylan will give you a ride in his orange Jeep.

• 57:08Speaker 00

  We, I want to go to Abilene, like let's, let's just do all these things. You know, I mean, obviously you guys have been there as well. Anyway, you guys got to go. It's great to catch up. Congrats on everything. And also like, I think there's just a lot of more work to do. Jordan gave us a sneak peek at the kind of things you're doing. Endpoint X sounds interesting. And like the, the market is really segmenting, right? Like to me, that's, that's what you guys are identified early and are raising the standard for everyone. So I think for, you know, on behalf of everybody, like thank you for doing this.

• 57:39Speaker 01

  Thank you, man.

• 57:39Speaker 02

  Thank you.

• 57:40Speaker 01

  Yeah.

• 57:41Speaker 00

  Good to talk to you guys.

• 57:42Speaker 02

  Bye.