Distributed Training, Decentralized AI: Prime Intellect's Master Plan to Make AI Too Cheap to Meter

Vincent Weisser and Johannes Hagemann, founders of Prime Intellect, join a conversation on the Cognitive Revolution to delve into distributed training, decentralized AI, and their vision for a future where compute and intelligence are widely accessible. They discuss the technical challenges and advantages of distributed training, emphasizing how such systems can democratize AI technology and create a more equitable future. The founders also describe their broader goal of creating a public utility for compute and intelligence and touch on their collaborative work in biosafety and scientific research to illustrate the practical applications of their vision for decentralized AI. SPONSORS: Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers. OCI powers industry leaders like Vodafone and Thomson Reuters with secure infrastructure and application development capabilities. New U. S. customers can get their cloud bill cut in half by switching to OCI before March 31, 2024 at https://oracle.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive Shopify: Shopify is revolutionizing online selling with its market-leading checkout system and robust API ecosystem. Its exclusive library of cutting-edge AI apps empowers e-commerce businesses to thrive in a competitive market. Cognitive Revolution listeners can try Shopify for just $1 per month at https://shopify.com/cognitive CHAPTERS: (00:00) Teaser (01:02) About the Episode (05:43) Welcome to the Cognitive Revolution (05:55) Exploring Decentralized AI (06:46) A Positive Vision for the Future (08:19) The Risks and Rewards of AI (08:56) Superintelligence and Its Implications (13:22) The Future of Work in an AI-Driven World (17:09) The Role of Billionaires in an AI Future (Part 1) (20:41) Sponsors: Oracle Cloud Infrastructure (OCI) | NetSuite (23:21) The Role of Billionaires in an AI Future (Part 2) (30:20) The Compute Market Landscape (Part 1) (35:10) Sponsors: Shopify (36:30) The Compute Market Landscape (Part 2) (47:49) Decentralized Compute Fabrics (51:25) Regulatory Challenges in Europe and the US (53:28) Policy Regrets and the EU AI Act (54:30) The Impact of Overregulation on AI (57:00) Frontier AI Labs and Safety Plans (01:00:02) Open Source vs. Closed Models (01:06:19) Scientific Progress with AI (01:14:56) Distributed Training in AI (01:35:29) Challenges in Model Interpretability (01:40:06) Supervised Fine-Tuning and Reinforcement Learning (01:45:19) Future of Compute and Infrastructure (02:01:02) NVIDIA's Market Dominance and Competition (02:05:22) Decentralized Training and Open Source Collaboration (02:09:58) Governance and Incentives in Decentralized AI (02:14:19) Conclusion and Call for Collaboration

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Execution is cheap. Ideas are worth everything, right? In a world where you can just like, it almost inverts to the current reality. And I think it would just lead to like billions of startups. We don't buy like hundreds of billions of computers. So in that sense, like we're not a hotel, we're more like Airbnb or like we're more marketplace sitting on top even of other marketplaces.

• 0:19Nathan Labenz?

  Why does this matter? Multiple reasons, right? It's like in the limit, you know, it could create a sort of truly decentralized AI infrastructure that nobody can control.

• 0:29Speaker 02

  Obviously, they have a lot of coming up in terms of like improving all that algorithm, right? Um, I think what we've done so far is just what we've realized those pseudo gradients that actually stand after those hundreds of steps. So it's not the actual gradients of the model. It's a difference between the beginning of the rates and the end state of the rates. Um, after all those inner step updates.

• 0:48Speaker 00

  In the intelligence age, almost, you want to own a piece of a super intelligent system that is able to generate value where you have actually had like access like through your ownership in it to the compute to the intelligence.

• 1:03Nathan Labenz?

  Hello and welcome back to The Cognitive Revolution. Today, I'm excited to share my conversation with Vincent Weisser and Johannes Hagema, founders of Prime Intellect, whose mission is to make intelligence too cheap to meter by building foundational technology to support decentralized, collectively owned AI. Vincent and Johannes stand out for offering a positive vision of a future in which a wide range of AIs empower everyone simultaneously, amplifying each individual's abilities and improving societal resilience, while all actors implicitly check and balance one another's power. At the same time, they've articulated an ambitious master plan and shipped a number of notable milestone projects in pursuit of this goal. Part one of their plan is to build an international market for compute.

• 1:47Nathan Labenz?

  And as of this writing, you can rent an H200 for a dollar forty nine an hour via their website, primeintellect.ai.

• 1:55Nathan Labenz?

  Part two is to build software frameworks for distributed training. And in late November, they released Intellect One, proving that distributed training can scale up to at least the 10 billion parameter level. Part three is to train high-impact science models. And the Medagene One model, developed in collaboration with the Nucleic Acid Observatory and others, and designed to be useful for pandemic detection but architecturally incapable of generating new pathogens, is one of the best examples of a defense-favoring AI project that I've seen anywhere. Part four is to launch a decentralized protocol for collective ownership of AI models and to collaboratively build towards aligned AGI that benefits all of humanity. While that still remains in front of them to do, given their track record to date,

• 2:40Nathan Labenz?

  I would not bet against them making a meaningful contribution. We spent much of the first half of this conversation unpacking their vision. To be honest, I'm still not sure how realistic it is to expect that we can maintain a stable societal equilibrium with AI changing everything everywhere all at once. But then again, to be real, this is happening very fast, and I don't think anybody has articulated a credible big-picture plan so far. If that's true, and we're mostly just going to keep developing this technology as fast as possible and hope that the resulting AIs end up being mostly harmless by default, I do find a lot to like in their vision for a more decentralized and hopefully resilient balance of power, as opposed to a world dominated by a few major AI players.

• 3:26Nathan Labenz?

  In the second half of our conversation, we get into the technical details, including both the fundamental challenges and recent progress in distributed training. Johannes walks us through the three main parallelization strategies used in model training: data, pipeline, and tensor parallelism, and discusses strategies like DeepMind's Deloco, which reduces communication overhead by allowing training nodes to process hundreds of steps before needing to aggregate gradients and sync model states. That they've managed to use this and a number of other optimizations to train a 10 billion parameter model across a globally distributed network of compute resources is impressive. And the latest from Google, called Streaming Deloco, which was released just after we recorded, suggests that they have not yet hit any fundamental limits.

• 4:12Nathan Labenz?

  It might still be very difficult to aggregate enough compute to train foundation models from scratch in a distributed fashion. But the recent shift toward reinforcement learning, which is far more inference-heavy and thus friendlier to distributed training approaches, strongly suggests that any number of network groups can probably muster enough compute to train whatever models they might like to reinforce into existence. Or, as Anthropic's Jack Clark put it in a recent edition of Import AI, "We might soon live in 'a world of models trained continuously in the invisible global compute sea.'" That world would almost certainly be simultaneously weird and beautiful and scary, but barring extremely draconian measures, the likes of which are well outside the Overton window

• 4:54Nathan Labenz?

  today, it seems like the sort of thing that technologists can and will create and that governments will have a very difficult time preventing or controlling.

• 5:04Nathan Labenz?

  Perhaps in the end, we can only hope that DACC, the Strategy of Accelerating the Differential Development of Decentralized Defenses, as exemplified by the MetaGene 1 model, will ultimately win out.

• 5:17Nathan Labenz?

  As always, if you're finding value in the show, we'd appreciate it if you'd share it with friends, write a review on Apple Podcasts or Spotify, or leave us a comment on YouTube. We always welcome your feedback and suggestions via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. For now, I hope you enjoy this discussion about a positive vision for and the technical underpinnings of decentralized AI development with Vincent Visser and Johannes Hagemann of Prime Intellect.

• 5:46Nathan Labenz?

  Vincent Visser and Johannes Hagemann, founders of Prime Intellect. Welcome to the Cognitive Revolution.

• 5:52Speaker 00

  Thanks for having us.

• 5:54Speaker 02

  Thanks for having us.

• 5:55Nathan Labenz?

  Yeah, I'm excited for this conversation. You know, for a while, I have been really wondering about Distributed training and if it's going to work, what the trade offs are going to be. And just generally, the the concept of decentralized AI, I think, has been quite fascinating. And when I recently mentioned on an episode that I'm looking for the right people to talk to about this, your names came to my attention. So I appreciate you being willing to do this and definitely look forward to getting into some of the nitty gritty details of all the work that you have done to realize this decentralized AI vision. And just before even getting into that, though, one of my common refrains is that a positive vision for the future is currently the scariest resource.

• 6:38Nathan Labenz?

  And I'm just always amazed by how little of that we hear, including from people leading the frontier AI companies. We get very vague visions of what's going to be good about the future. One thing that has struck me about the two of you, as I've explored your work in preparation for this, is that I think you do have a positive vision for the future. So I'd love to start off by giving you an opportunity to articulate that positive vision for us a little bit.

• 7:02Speaker 00

  So I can kick it off to frame it also in kind of like, starting with what we plan to build with Prime and then more broadly, like how I see a positive vision of the future with AI. So basically, really, like, our goal is to make intelligence and compute too cheap to meter. And I think we're entering that era right now. And really, I think the the key goal for us, I think, which is would be a key part to make it utopian is to make it widely accessible and open to everyone. And I think really, like, the goal is to see almost like, if we zoom forward, right, a few years or decades ahead, the biggest risk would be if we have this super powerful intelligence and compute,

• 7:42Speaker 00

  but it's not accessible to everyone, but rather to the select few and kind of like the big states, the big tech giants and individuals. So I think that's really how I see the the most positive future. And I think what does enable it really is like, I think it empowers like every human and creator to leverage their intelligence, rather than like be able to do more with less, right? It's like basically have agents at their disposal, like create more advanced science, more like have better access to medicine. Right to all these things. So I think it's actually, I think, really like has the potential to usher us into kind of an era of like widely distributed abundance.

• 8:21Speaker 00

  If done right, but if done wrong, it also has the risk that it actually kind of like creates like much harder to access intelligence age, let's say, right? And I think that's the most important piece. I think that basically large parts of the society is like able to participate in this like upgrade.

• 8:38Speaker 02

  Yeah, we're pretty much aligned on that, right? I think it needs to be widely distributed in a sense, right? The most dangerous outcome is that if there's only one superintelligence, right? Like they say, this is zero, obviously, but the distributed race is to go where we are going forward. So that's how we basically think about the future of from source AGI as well as ASI.

• 8:58Nathan Labenz?

  It's interesting that you mentioned AGI and ASI in that in that answer. I think, you know, so many AI worldviews or expectations really diverge on how powerful do people expect the AIs to become? And on what timeline? How would you compare and contrast your own expectations to what we've heard recently from Altman's and Dario's? Who seem to be increasingly confident that we are going to see superhuman AI in the next few years?

• 9:29Speaker 00

  Yeah, I think it's basically like most of the views put forward for me are very plausible. Maybe like even since the last fifteen years, like since reading like the similarities near and Bostrom, like when it came out. And I think basically we're still on that path, right? I think it's actually surprising how well those predictions held up. I think the most common views of the big lab leaders, like from them as Daryl, Sam to folks like Leopold, I think. Like I would put a basically high probability mass on their scenarios. And I think they also base it off, like, obviously, the research progress that we've seen over the last, like, I think the biggest question, I think, is like, how general it will be in like.

• 10:10Speaker 00

  The broader sense, right? And I think a lot of people I talked to also at the labs, like have high confidence that we'll get to superhuman coding, superhuman math, maybe superhuman software organizations that run autonomously, right? But that doesn't mean we get superhuman humanoids like tomorrow, right? Like basically, I think a lot of these things will look probably like autonomous cars or something, where it's easy to make radical progress, it's hard to get to a hundred percent like accuracy and robustness, right? I think basically in autonomous cars, it is not a hundred percent robust, but like ninety-five percent robust, right? Like as as build users. And I think we'll see the same with superintelligence, where it's like the deployment of it, right? Like will happen if it has extremely high robustness in its application.

• 10:54Speaker 00

  So I think it's not enough to have a superhuman investment agent. Like it can't screw up one percent of the time and lose you all your money. And I think like that's how we'll see probably progress shape up is that like actually, I think it will move much faster in some areas, like we've also seen, right? Like with the new like inference time compute scaling paradigm. And I think we'll see it to like move slightly slower than like some of the biggest optimists, maybe like make it out to be in trickier, like harder to simulate areas. But I think really the user framework, I think it's like even like folks like Steve Worfram on kind of like how computational reducible basically different areas are, right? It's it's more difficult to solve all the diseases than to write a bunch of code autonomously.

• 11:36Speaker 00

  So I think we'll run into the limits of what you can compute, right? And I think that's probably one of the things, like the only maybe bottleneck I see in the next ten years is like that you can't simulate everything, you can't compute everything, but a surprisingly large amount, right? And you can approximate maybe reality with like pretty good simulations, including for things like biology. So yeah, I'm quite optimistic that like, um, I think most people are not optimistic enough, especially in the face of like all the progress, in the face of like the infrastructure build out. So I think we're on track for like superintelligence and like for the next decade.

• 12:14Nathan Labenz?

  So can you maybe put a little bit more color on the future that you envision like?

• 12:21Nathan Labenz?

  I hear you on the accessibility part. I definitely think people are underestimating what's coming. But I'm more unsure as to like, you know, is it going to be good by default, bad by default, you know, what are the hinge points? Obviously, a lot of questions there. It's funny you mention like Kurzweil too. I've increasingly been using the phrase Kurzweil's revenge recently. Because in retrospect, I look back and basically say there was really no other way this was going to go. You know, it seems like the fact that so many different algorithms seem to work. And the fact that we have the compute at scale and the data at scale. Like, there was really no trajectory where we were going to have these compute and data resources and not have somebody figure out, you know, a workable algorithm.

• 13:06Nathan Labenz?

  So I think in that very fundamental sense of like, you know, if you have the physical and informational inputs, like, somebody will figure out how to unlock the lock and will get reasonably and potentially very powerful AIs. That seems to have been like very well borne out. Do you have a vision for daily life? Like, am I working a job in your future? You know, imagined successful scenario? And is there like a stable equilibrium? I really struggle to envision a stable equilibrium. I think the decentralized AI, you know, line of thought in general sort of tries to get there. But I would love to have a more concrete, you know, detailed, richer vision of that than I do, to be honest.

• 13:49Speaker 00

  Yeah, yeah, I think like it really depends on different dimensions and axes and also on the timeframe, right? And like personal preferences. I think in many ways, people that are like financially independent, right? They could basically retire, but they could also still work on a startup or like do other things. And I think that's playing out, right? It's like every billionaire in society is working like probably harder and not like less hard on like their philanthropic efforts, right? Like their entrepreneurial efforts and other efforts. And I think we'll see basically more people like behaving like as if they're like financially independent. So it's like, I think in that sense, there's already a glimpse of like, I think what like, how abundance manifests, right? It's like billionaires are definition living in abundance. But I think.

• 14:33Speaker 00

  Like for a lot of people, obviously, they have different priorities in life. So I think actually, good recent book I read on this was like from Nick Bostrom, like How Like Life and Meaning in a Self-World, and I think like some of his points, I think make a lot of sense, where it's like, there's so many other aspects to life that people like care about, like be it having children or like doing art, like creating things, right, like understanding the world, right, like reading, like all these things, that I think people will be able to spend more time in, right, like even if one sees other, like almost like jumps in in human progress, right, like in the industrial age or now age to a knowledge, like basically, in those shifts, I think people were able to basically

• 15:15Speaker 00

  like get more freedoms and ultimately have an easier time to reach financial independence, as one example, but also more broadly, almost like do whatever they want to do. Most people I know could stop working, they don't, right, it's like, and and I don't see them to completely stop working once you have AGI, and I think, like, what I think is interesting is like, even seeing this now, with a lot of people working on this technology, I think to your point, there's the camp of the people that are like too optimistic or too pessimistic, and and they kind of like give up, so they like resign from their AGI jobs because they're like, oh, time to enjoy the last two or three years before their superintelligence, but also on the other side, but it's like, kind of like pessimists were like, oh, the end is near, and I think that's like a dangerous

• 15:57Speaker 00

  and kind of like unlikely scenario to play out, where it's like, I think the nature of jobs, I think, will change, but I think another good line of argument that I've heard, which I agree with, is like, there will be probably like a return to very like human labor, and people might still go to a restaurant, and they might still go to a theater, they might not drive an Uber, like it drives itself, right, it's like, so basically, I think there will just be like a different sort of labor, right, it's like, that will become even bigger, like basically, everything where you and I would like be even post AGI, like pay for a human, right, because it's just more joyful to have a human around than like a steel robot, you know, like if you want your kids to be taken care of, maybe you don't want

• 16:39Speaker 00

  your optimist robot to do it, but like a human you you trust and can understand, but I think that that's how it will play out, like I think in many ways, it won't actually be absurdly different from today, it's just that I think there will be way less. Like knowledge work, that isn't an exception, right? Like, it will be much more curation. And even like, I think all of the knowledge workers will just like still work. They'll just work in very amplified ways on different functions. And they have an army of people working for them for free, basically, which are the two next NDAIs, right?

• 17:12Nathan Labenz?

  The one point on that I want to push on a little bit more is like the billionaire point working today. Cause I, I feel like there's some, maybe you can resolve it, but I feel like there is some contradiction. But the sense that like, why are they working today? I think in part it's because they feel like they can make a meaningful contribution, right? They want to make a positive difference.

• 17:34Nathan Labenz?

  Does your vision sort of imply that for the people in that position, that there is not an AI that they can hand that responsibility off to, to do a better job? Cause you might imagine, you know, Bill Gates or Dustin Moskowitz or whatever might say, you know, my goal is to eliminate these diseases. I could sit here and run this foundation, but there is a super intelligence that could probably make even better, you know, grant evaluation decisions or whatever. And are, are you imagining that they don't have that? Or that they just choose not to use it for some reason? That part is always a little weird to me because it feels like if you have a super intelligence, what are even the billionaires doing?

• 18:14Speaker 00

  Yeah, totally. I think there was a good interview between Sam Altman and Bill Gates exactly on this. And you could see also like Bill Gates also grappling with it, being like, oh, you get so much meaning out of being so good at like fighting malaria and like, hey, if a super intelligence can do it for him, he would lose some meaning because he's so good at it, which I think is the wrong worldview, right? It's like, hey, if you really care about it, be happy if you can like hand it over to your employee, to your agent or whatever, to do it better for you. I think the best almost like founders or like philanthropists or whatever, they build a team that is way better than they are on every dimension, like basically. Maybe they are, they have broad skills, but like ultimately ideally they hire people that are like

• 18:54Speaker 00

  super intelligent compared to them on something very specific. I hope Elon has someone who's way more intelligent than he is on propulsion, on autonomous cars, right? And AI and systems, right? Like on across all those efforts. And I think that's more how we'll play out where it's like, like basically execution is cheap, ideas are worth everything, right? In a world where you can just like, it's almost inverts to the current reality. And I think it would just lead to like billions of startups, right? And billions of movies or much more, right? It's like basically just an explosion of content companies, autonomous creations. It ultimately still like a human is either like seeking or creating. I'm kind of extrapolating from like where we are today, right?

• 19:35Speaker 00

  And like how I, I see and try to adapt to what I see as plausible in a year and five years and 10 years. Like I think there's still a lot of things we'll want to do, like we'll want to work on. And I think to your point, I think like the ultimate, like even this like almost like master of the pyramid of needs, right? It's like there's things like impact status, right? And like meaning and fulfillment that are like timeless. And even if you have super intelligence, you might still get like meaning out of specific things, right? That you're like, see. And I think I see it almost more like humanity will just have like a gigantic workforce of like AIs and agents that like do a lot of the work, but like it will basically hopefully stay on, on top of it. And like basically

• 20:15Speaker 00

  manage that and, and direct it towards the, the aims like humanity cares about, right? It's like, it's a bit like in, in some ways, our biggest corporations or states are like super intelligences with a lot of agents in it. And they're directed like through democracy, through like shareholder capitalism or whatever, like into specific directions, right? And I think actually the biggest risk, right? Is that we'll just have like nation states and like for-profit corporations have the super intelligence instead of like everyone to, to do like everything they want to do.

• 20:44Speaker 03

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 20:49Nathan Labenz?

  When you said hopefully, the one interjection I was gonna ask is, do you maintain a P doom? Do you entertain that conversation at all?

• 20:56Speaker 00

  I think it's not extremely useful as a concept. I think basically, I think a lot of people are way too confident in like weird sci-fi scenarios and kind of like probabilities, techno probabilities. But then if you ask them to like unfold them, like they can't really. It's like, like I've seen too many people, you know, like, they're like, oh yeah, my P doom. It's like 20% and I'm like, why? Oh, because this person also has it. And it's like, basically, it like usually falls apart. I think it's, I think a mind virus. Like, I think it's a really dangerous concept, actually. Like, I wouldn't put strong confidence in anyone putting out random numbers. And I think there's almost like, it's really about like probability math, right? Probability is like between 0 and 100%. And it's like, the universe will go on for billions of years.

• 21:40Speaker 00

  Like, will something at some point kill it? Sure. Like, when, how, and what kind of like the order of events is? I think it's like so, so hard to know that I think it's almost not worth pondering. And I think one of the problems, like the utilitarian philosophy brought to the space is to have very high confidence and impossibility to know future scenarios and events. And then basing like policy, all the actions. So I think what is more important, and I think the ironic thing is like that in this worldview, I think like one needs to realize that like on shorter timeframes, the probabilities are stacked in favor of good outcomes.

• 22:21Speaker 00

  And then even the actions, like by, for example, on AI policy, like sometimes have the exact opposite consequence, where basically it's like someone even with a low P doom, like, it's like, oh yeah, that's why we need to regulate it heavily in Europe. Okay, goodbye AI in Europe. Like, oh, we need to have regulated heavily in the US. And suddenly like things move outside of this. So I think there's basically, yeah, like a cascade of like very bad, almost like epistemics and reasoning that follows from people going too deep on P dooms.

• 22:59Nathan Labenz?

  I've certainly seen some of that. At the same time, I do sort of worry that things could get out of control. I answer that question when I'm asked for my own P Doom. I usually say five to ninety five percent or like ten to ninety percent.

• 23:14Nathan Labenz?

  And yeah, what I mean by that is basically just nobody has said anything to me that seems compelling enough that I shouldn't worry about things getting totally out of control, nor that, you know, it's so likely to happen that I should, you know, check out and spend the rest of my life at the beach or whatever. So, but I do worry about the, you know, the possibility of things getting out of control. Do you think that that worry is misplaced? Or, or do you share it to some degree? I don't think quantifying it is super useful, but I do think like considering it is is still useful.

• 23:47Speaker 00

  Like I see it almost like the industrial revolution brought like created chaos, like created havoc, maybe like civil war and like some ways and like had second thought all the consequences. Was it that good? Sure. And like, I think, or like the internet, right? It's like created like some like complexities, political consequences and everything else. And I think still like ultimately, I think that AI like almost like benefits like far outweighed risks. And I think also some of the paths to avoid the risk become risk themselves. But it's like basically like kind of like in a world where like everyone is worried about like the existential risk and then like calls for like one world government. I think probably the one world

• 24:32Speaker 00

  government is like the bigger risk than actually kind of like the like sci fi scenarios that people are worried about. So I think all these things are very complex systems, right? And ultimately, I think it's hard to know what will be specific risk, right? And I think the biggest risk is actually people losing their autonomy, like putting life on autopilot with AGI, losing their freedoms because they handed over happily to the state and to over regulation. So I think really that's my biggest worry is basically that even the most well intentioned people. Actually create almost like the worst outcome scenario unknowingly and unintentionally. And then also in some ways being misused by people that seek power.

• 25:15Speaker 00

  And I think like a lot of the people that like were fighting for alignment, safety and policy have, I think, been co-opted by big labs and big tech that has their own goals and their own intentions. And you can see this really well, right? Like with the big tech giants and the big tech AGI labs. Like, and I know stories about like almost all of the AGI labs that are extremely worrying of like things happening behind the scenes and how they power seek. Even the people that this whole community kind of reveres. So I think that's the thing that I'm really worried about. It's basically it's like seeing some more details. And I think it's like the biggest risk, I think, is basically like full centralization of superintelligence by a few nation states and big tech giants.

• 25:58Speaker 00

  And basically completely disenfranchising large parts of humanity.

• 26:03Nathan Labenz?

  I don't know, like, it seems like it's all in play in a way that it's never before been in my life. But I do hear you very much on the idea that a crazy imbalance of power or just like, you know, one superintelligence in one organization or one company or, you know, one even potentially even one individual's hands does seem like a really problematic situation. You know, in in biology, right? Like I've one lesson I've learned over and over again in life is anything that is sufficiently concentrated is dangerous. You know, like all the drugs, you know, become dangerous when you take them from like the leaf that it grew in on the tree.

• 26:46Nathan Labenz?

  That was fine, but then when you, you know, purified it down to a hundred percent and started huffing that like, that's when you really got into problems, right? And that seems like it kind of keeps coming up over and over again. So I am quite sympathetic to the idea that if we can maintain. A sort of buffered solution, sort of balance of power that, you know, where everybody is sort of checking everybody else, much like things work today, right? Nobody has like the ability to dominate the world unilaterally. If we can maintain that as we like bring AI online across, you know, all aspects of life. I think that does sound really appealing. Of course, I don't really know like what that also could bring.

• 27:28Nathan Labenz?

  It seems, you know, anything is possible there, but I do find a lot to like in that vision of the sort of all, you know, everybody rising simultaneously. Everybody's interests, you know, kind of keeping one another in check. No single actor, you know, being able to make a mistake that kind of throws everybody else into a huge problem. Let's talk about the company and your kind of master plan for getting there. I love a good master plan. And again, you've got one. So one thing I wasn't able to tell from my research is like, what kind of a company is it? Is it like just normal corporation or something else? So maybe, you know, give us a little bit of the foundational backstory and then take us through your master plan.

• 28:06Speaker 00

  So basically like almost like our broader goal really, right? It's like to make computer intelligence kind of like too cheap to me. And like really how we want to do that is like in multiple stages where basically in the first one, we started with aggregating global compute and creating an efficient compute market. But also the developer interfaces for it, like an API command line interface. And basically ways for people to just like, no matter what kind of computer they're looking for, like be it one H one hundred or thousand H one hundred, so they can basically make a request. And then they they find like where they can find the chips computer they're looking for. So that's basically how we started and how we launched two or three months into the company creation. So basically that really was like also the biggest thing we initially did and which gained a lot of traction.

• 28:51Speaker 00

  And then really building on top of this and kind of like in conjunction is building basically all decentralized training and other approaches like distributed synthetic data generation and others, which really can then like leverage as the global like compute fabric, right? It's like you can envision if there's extremely cheap compute in one place or another and it dynamically shifts, it's extremely beneficial if you can. Train decent rise in a full way. Because then you can basically just save massive amounts. But also really leverage the global compute. Especially when there's idle compute available somewhere in the world. So basically, it ultimately reduces the cost of intelligence, reduces the cost of compute. Because you can make more efficient use of this resource. That's kind of like step number two. And we also made a lot of progress there and can go into more detail.

• 29:36Speaker 00

  And connected with it, which is very closely connected, it's like. The goal is really to train kind of like leading open models collaboratively. That anyone in the world could contribute their idle compute. Like to this network and also get rewarded for it. Basically make the highest possible return on their compute. Because it gets fully utilized. And then if it's for example idle, if it's not utilized. It can contribute this to like frontier models. So I'd like say to continuous improvement of like all the R one model from Deepseek. Which is one thing we're looking into. And I think there's basically like that's almost the third pillar. Which really connects to the like more broadly to the like peer to peer computer and intelligence. Where the goal is anyone in the world could contribute compute.

• 30:20Speaker 00

  And then one in the world can use that compute. And it's kind of like an extremely efficient market for compute. And on the other side, like I think on a more abstract level. The same can be said about like intelligence itself. Basically, be it like any AI API. Like it also becomes sort of a market, right? It's like there's different people hosting Lama models, Deepseek models, the diffusion models. And they're also competing ultimately that their like cost is purely compute. Right? So so basically you can lower the cost of of those intelligence endpoints, if you like. Like of agents of Lama of like other models by having the most efficient compute market. So basically like an efficient peer to peer compute market also enables like. Efficient peer-to-peer intelligence sitting on top of it.

• 31:05Speaker 00

  And really, I think the goal is to create like a a system that is almost more like a protocol, like parts of the internet that like can basically be maintained, like almost like a public good. And people can just like use it trustlessly and permissionlessly. And I think the closest to structure is like something like Ethereum, like foremost more like smart contracts, which like really the goal, right, is there to like create a technology, make it openly accessible, have a like foundation, like structure to support it. And then over time, like it really can be owned by anyone. Anyone can use it like for the cheapest cost possible. And I think that that's a broader vision. And I think to your question, how we structure this, kind of like it's a Delaware C Corp. But really, the setup is more akin to Ethereum

• 31:50Speaker 00

  in the sense that we're creating a foundation and basically giving out grants for people to develop this in the open, right, like through like open source approach to basically fully like make this accessible to anyone. And anyone could fork this system, improve this system, contribute back to the system, participate in it. And that's really the broader design principles, almost more creating like a public utility to some extent. And also in conjunction with some of those systems like Ethereum, where anyone can also create their own agents and commercialize it. And the agent can make money for you, right? So you're, I think that's also like going back to this utopian vision, like how we see this play out is that like anyone in the world can like contribute to it, own a piece of it, and it basically just like generates like revenues

• 32:34Speaker 00

  like for the end users that participate in that system.

• 32:38Speaker 03

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 32:43Nathan Labenz?

  A lot of points I want to follow up on there.

• 32:47Nathan Labenz?

  Maybe let's start with the compute market. And we'll work our way forward in time through the master plan. For starters, how would you describe the compute market today? I think we've had a sort of much hyped cycle of people raising large equity rounds and just plowing all of that proceeds directly into GPUs, in some cases like Nvidia, even just taking equity positions in exchange for GPUs. It seems like that seems like it's over. Are those the kinds of companies that are now like saying, geez, we maybe like overbought a little bit and have a have more, you know, we maybe can't use the cluster we rushed out to buy as much as we thought we could.

• 33:29Nathan Labenz?

  And now they're contributing to your marketplace or like, yeah, I mean, I have a million questions about the compute marketplace, but tell me what you think is interesting that's going on.

• 33:38Speaker 00

  I think if you zoom out, GPUs were a tiny, tiny market, right? Like pre chat GPT and they're growing at exponential rate. And I think there's no slowdown inside. I think basically it's, and you see this, obviously it's it's powered on the sense that like, obviously this week was kind of like anonymous, right? Like 100 billion plus in like compute commitments from individual companies and that's. And I actually think the trend right now is rather that there's much more demand for compute, including a much longer tail of companies, startups seeking that compute. But I think to your point of like, kind of like how does market works and structures, right? Like how I would, I would describe it, right? It's like a lot of the like, we basically more than a year ago, right? Like we're in a period where like the supply for H100 was extremely constrained.

• 34:23Speaker 00

  So basically the the big guys that paid like extra got got more of them and and kind of like got it got them with priority, but like the open eyes of the world. And then it also shows you kind of like the problem, right? It's like those big tech giants got priority because they they paid more and have like gigantic long orders and clusters. But I think what we've seen shift is basically it's like that there's obviously now more supply. So basically it's like the production started to catch up a bit. And obviously also new generation, for example, concrete with Nvidia rolling out. So I think basically the lot of those more startups on on the point of like raising capital and deploying it. Bought long contracts, right? They bought like one, two or three year contracts and they paid hundreds of millions. And and they had, basically, it's like if you wanted to train them all and you needed a thousand GUs,

• 35:08Speaker 00

  you had to buy a two year contract to get it. It was impossible to rent them on demand. And I kind of like fast forward like now, like on platforms like ours, you can rent a thousand H one hundreds like on demand. And that wasn't possible even three to six months ago, really. And I think it showcases you that basically from the supply, which the majority was kind of like man for long, long term contracts, right? Like which from a capital perspective are more de-risked and and almost more like structured financial products, right? It's like Core Weave buying like tens of billions in in compute, giving them to OpenAI, taking out loans against the GUs. Like that model, I think, is evolving that basically more and more of those compute is actually moving to the on-demand uh supply.

• 35:49Speaker 00

  So basically, it's like really, I think, also what powered all our growth and positioning is that like a year ago or two, it was impossible to find an H one hundred. And that was very useful what we offer in a sense that that's also why we started it. Every AI startup and developer we talked to, right, like was had a very hard time to find H one hundreds. But we were able to find them going directly to data centers, talking to some billionaires that bought like tens of thousands of H one hundreds and we're looking to like sell them on demand. And but fast forward, like now there's even more data centers, there's even more clouds that are appearing. And really, it's a market that is insanely fragmented. So I think to the outsider, it's not so obvious that basically there's hundreds of clouds, there's thousands of data centers, right? And there will be more of them. And like there will also be more chips than like Nvidia chips, right?

• 36:33Speaker 00

  So really, you need to aggregate a market that is like by design so fragmented. Like basically Nvidia keeps it fragmented because they don't want to have like an overly powerful buyer, right? Like if OpenAI or Microsoft would be the only buyer, they wouldn't be in a good spot. But then on the other side, their biggest clients also compete with that, right? It's like. Google, Amazon, like all the others, big buyers, they also produce their own chips, right? They compete on with the various chips. So basically, they need to be careful and really gave a lot of allocations to more folks like Lambda Labs, Core Weave, basically like in some ways smaller clouds than than the really big tech clouds because they're more aligned in many ways. So basically, that's why the supply is kind of like distributed across America, across the world in hundreds of data centers. And really, as an end user,

• 37:18Speaker 00

  you kind of want to discover like, hey, I need 200 of them. Where can I find them right now? Like, and it's not so easy and clear because also the obviously the demand and the supply is shifting in real time, like with all of these platforms. So that's kind of like the broader backdrop. And I think the other piece is like, that I think the there was obviously some of those like very big and well-known cases of like startups that raised a lot, then started pre-training models. And I think the thing is, that's continuing to be the case, right? It's like there's like more AGI labs today than a year ago, right? It's like a very obviously well-known case. Out of all my ideas now, like four AGI labs I can count, that were in the round like one or two years ago. There will also be like a lot of applications

• 38:00Speaker 00

  that need more and more compute, right? And especially some new use cases, like video, like the more heavy reasoning or coding agents. And in general, like agents coming on to the scene this year, they'll need much more compute. So I don't think there will be a slowdown in compute. I think on the other hand, like would become like the biggest chunk of GDP probably, like almost in this ramp up to AGI or ASI, and probably just like power all the different intelligence use cases.

• 38:25Nathan Labenz?

  So would it be fair to say that you are playing a role that's like an aggregator of people that are also selling direct, like you, kind of like I'm, I don't know, to pick one, like a kayak type of aggregator where I could go by directly from the airlines, or I could go to kayak and they give me this sort of menu of all the different airlines, as opposed to like an eBay where it's like, hey, I've got three H100s here, you know, can I plug them into your marketplace? It's, it sounds like it's more like the former. Is there, are those good, you know, frameworks from which to think about it?

• 39:00Speaker 00

  Yeah, I think like ultimately it's not so different from the biggest marketplace in the world like Amazon or something, right? Where it's like anyone in the world can like basically sell compute on our platform. And also like anyone that is selling compute, we're in touch with it. Like from the hyperscalers to individual data centers, to clouds, to billionaires that have compute, to startups that want, that have too much compute, to startups. They want to buy compute and put it back onto the platform when they don't need it, right? Like to, to get cost savings out of it. So I think the best analogy is just like a lot of the platforms, right? Like are kind of like little isolated islands and we try to create the ocean that connects all the islands.

• 39:38Speaker 00

  So it's like basically you have some compute in, in one cloud, you have some in the other, you have some in, in one big tech, some more in the other. But basically it's very, very hard to orchestrate all of it. So it's like literally a lot of the people we talked to, like even start off with two people where, where building their own APIs to like query the availability of all the GPUs, then they have like different APIs. And like, that's basically all the work that we're like simplifying for the end user. Is that you can just like orchestrate this kind of like global compute fabric. And I think the other pieces that like a lot of them are almost more like a PE, like they're basically structured financial products. They borrow a billion to buy a billion of compute and lend money against the compute, right? It's a very high capex business.

• 40:22Speaker 00

  And I think what like what's very different is like we don't buy like hundreds of millions, billions of, of compute. So in that sense, like we're not a hotel, we're more like Airbnb or like in, in that sense, right? Like we're more marketplace sitting on top even of other marketplaces. So basically some of the marketplace we talked to, they're trying to do something similar. They're, they're not going for the full broad spectrum, right? For example, they don't work with the, the hyperscalers or with like some of the larger clouds. They just work with like individual data centers. So basically we're trying to just like aggregate all of them, right? Also with the time and onboard all of them onto one market.

• 40:58Nathan Labenz?

  What's the smallest unit of compute that would make practical economic sense for somebody to contribute to the marketplace today?

• 41:07Speaker 00

  Yeah, so basically, we have every chip that is in demand. It's kind of like a market where it's like the majority of the demand falls right now for like H100, it's like GH200, it's like kind of like A100 still to some extent of 3090s. But like, there's obviously, over time, I think, reasons believed that like, for example, for synthetic data generation, you could even use your like MacBook or something. That's not our primary focus right now, but I think it's like, there's compute everywhere, right? And it's not just in the data centers in H100s, even though that's our primary focus. Like, over time, I think, the goal is basically that you can literally leverage all the compute out there. Obviously, some of it is like much more useful for the different AI workloads, right? But also, it's like, now, for example, that we're shifting to this synthetic data paradigm, suddenly, some new compute starts to become useful, right?

• 41:52Speaker 00

  Like, now that Nvidia is chipping their own, like, almost like home GPUs, right? Those become potentially interesting over the next few years, like, especially if like a lot of people get them. So I think, like, all, like, the other, I think, it's like, all to compute, but like, the computer that is in highest demand, like, will be more successful on the market, right? So it's like very market-driven in that sense.

• 42:14Speaker 02

  Exactly. For users on the platform, obviously, all kinds of compute resources would be useful, right? That they, yeah, need for their development purposes and stuff like this. For the more distributed training component of it, right? Contributing resources there. It's very much more focused on the high-capacity GPUs, right? Like, an H100, an A100 node, a full node, or even full clusters, right? Just because of memory requirements and other components needed to actually do distributed training across those. So there, the focus is more on higher-end GPUs. There's the use cases for like the 4090s, your smaller clusters and stuff like this, right? For large-scale synthetic data generation and other things. For training, you need quite a bit of memory requirements, and that's why it's more focused on A100 and H100s also for our, like, distributed training ones.

• 42:58Nathan Labenz?

  Part of why I'm interested in sort of the economics or the sort of practicality of contributing to the market is it seems like in the long term, we may be headed for a fight about like who can compute, you know, under what controls, under what circumstances, with what oversight, whatever, you know, the sort of trillion dollar data center is one idea where it's like, it's all super aggregated, it's all super centralized. Obviously, that also becomes from like a geostrategic standpoint, hard to defend, but it is like probably relatively easy for a government to control. You can sort of fence the perimeter and know who comes in and out. On the other end, there's not really anything like this for AI yet, but there is Bitcoin

• 43:41Nathan Labenz?

  and Ethereum, and these are things that are truly like not something people can shut down. It would be very difficult at this point for even probably, I mean, tell me if you think I'm wrong about this, but it would seem like even the United States government, even if it really wanted to, would have a really hard time shutting down those networks. So I'm kind of wondering like, where are we on that spectrum today with these sort of global decentralized compute fabrics like, because if it's, you know, you have to be at a, you know, significant scale and it's basically a data center. It seems like we're, you know, obviously not at the extreme of the trillion dollar data center, but a lot closer to that where it like could be, you know, controlled or governed or whatever. Whereas if you really could get down to, I got a Mac mini on my desk that's idle and

• 44:26Nathan Labenz?

  I'll, you know, rent that to you for five cents an hour, potentially not even for the money, but because I want to contribute to Yeah. a, you know, global network that nobody can shut down, then we sort of are in a very different regime. So where are we today and where do you think we're going?

• 44:42Speaker 02

  To maybe touch on the first point you mentioned, the trillion dollar cluster. I don't think it's going to be one single cluster, even the Stargate round, where we distributed in a way, right? All the big takes are looking into how to train across there, distributed training clusters as well. They obviously have a bit more favorable environment than we are in. They have cables between the data centers, 40 gigs of interconnectance, at least 100 gigs, which we obviously don't have right now in like a globally distributed setting. Even there, I would argue, it's also not that easy to control that you just have like one single location that you have to prevent somebody access to it. It's it's gonna be distributed across the globe. If you wanna build something like a star gate cluster because of energy requirements, you can't build a

• 45:23Speaker 02

  ten gigawatt cluster right now, but we're gonna be limited to to multiple like five hundred megawatt clusters or something like this over the next couple of years.

• 45:31Speaker 00

  Yeah, and I think what I would add is like with this distributed training run, like we trained across Europe, Asia. I think both things can be true at the same time, right? It's like computers extremely just real like it sits in Europe, it sits in mainly America, it sits in Asia already, it sits in other parts of Asia like China. But also there's a ton of compute in India, there's a ton of compute in like other almost like more free like nations like be it Singapore, Malaysia, and and a lot of the computers shifting to the places where is the most free, right? Like where where you basically can generate revenues, we we are not encumbered by like stringent regulation, right? So like places like Malaysia and Singapore, right? Like seeing a boom in compute. And I think that's one piece. I think the other is like we'll see these big data centers, but like

• 46:15Speaker 00

  there's a gigantic long tail of compute, right? Like basically, there's a lot of data centers, like hundreds of thousands or tens of thousands of GPUs. Most don't have actually a hundred thousand or million, right? Like those you can count on one hand, right? As they have like fifty to twenty thousand GPUs. It's like more like the XAI, OpenAI and Microsoft clusters. But there's so much more compute, and I think the other is to your point, all the lower powered compute sitting in people's home, right? I guess a ton of people with forty nineties, thirty nineties across the world, right? Like in every part of the world. So I think it's not only extremely naive to try to regulate compute, but it's also extremely dangerous. David Deutsch said on a recent interview, it's like regulating code or math, like it was like basically it's like this

• 46:58Speaker 00

  on one side extremely dangerous, but also extremely naive in a sense that basically it has the exact opposite effect, right? It's like that leads the compute to shift to nations that are even less Easy to regulate and and observe. It's like it shifts to the the boundaries and to the actors that don't obey by regulation, right? And to an extent, you could see this even with Bitcoin and Ethereum. And I think the exact same thing will play out. Basically, a lot of it went to the cheapest regions in the world where you could mine, right? A lot of it was in China. And then China banned it. And then a lot of it moved to the US. And and a lot of the business associate moved to the US. But the energy was cheap in other regions, so it also moved to Iceland or something, or like parts of Africa. And I think like it's how I see this play out as well, basically, where ultimately

• 47:43Speaker 00

  like you're maybe the best example, like we're both from Europe. And I think we both left Europe. And I think one of the reasons is because Europe is known to be extremely diesel and like co-opted by those regulators who ultimately don't have the best intention of the citizens, of the future and of the economy of Europe in mind. But they basically over-regulate and kind of like, like Europe didn't even have an AI community, right? Now it's dead. Like ultimately, it was not even really breathing and alive. But like it's non-existent now, right? And I think it's a it's a best example, zooming in on a nation that completely fundered their participation in the new like intelligence age by a few co-opted like regulators.

• 48:26Speaker 00

  And I know those people, right? Like that regulated, they had the best intentions. They didn't plan to destroy the future of a whole continent, but they did. And I think that's a very cautious lesson. And I think fortunately, like I'm very excited for America because ultimately, America doesn't repeat that mistake. I think it was on track to repeat it under the last administration. And I think fortunately, like we're on a bright path, like for optimistic future. Otherwise, the US would have said goodbye to their their future and their stake in AGI. And I think that would have been extremely dangerous, would have pushed it even harder to Asia than it already is right now with progress like DeepSeek. So I think basically it's important to think through the second, third order consequences, right? And like what ultimately goes.

• 49:09Speaker 00

  I think there's been a lot of policy people that I know, like some of the also most powerful ones that really even regret some of the policy they they are kind of like argue for. I know some of the people indirectly funded some of the biggest ones. And they're not big fans and they they publicly stated or will continue to publicly state it. And I think you would be surprised that like. How many of the people that actually enabled a big policy regret having done so and stopped doing so now? And I think that will continue to be the case, I think be more public in the future because now the old window shifted.

• 49:41Speaker 00

  What policies do you think people are regretting? I think the EU AI Act will go uniquely into history as being one of the worst ideas for not only Europe, but also having completely zero impact. It didn't really improve anything for anyone, like literally nothing, but it made it worse for smaller AI startups. Like, obviously, it's like a general problem Europe has. It's very proud to regulate and it's very proud to over-regulate industries that are very tiny to non-existence, right? So I think the EU AI is a very concrete example. I think even the California Act is an example. Even the people that were arguing for it, they said like they should have done it on US level instead of the, just like on the level of California. But I think those two things stand

• 50:24Speaker 00

  out for me as the two most well-known policy initiatives. And I know some of the key policy people at the big AI labs had stopped being there and regretted it. And I know like some of the by far biggest funders of policy, if you talk to them, they regret that they made those donations. And I think they will be more of that and will be more public in the future. So I think there's a lot basically where people realize the outcomes of policy, like funding super limied, like great, there should be more of that. Ultimately, funding policy quite quickly goes wrong, right? Like quite quickly gets corrupted, quite quickly has the unintended consequences. And it can happen that it goes horribly wrong. And I think it was to an extent the case, I think, with a lot of the government, like the policy.

• 51:06Speaker 00

  But I think on the other side, it's like the majority proposal is compute governance, right? And people like Jack Clark also commented on our work and said like, hey, I think those folks need to update. I think compute governance in the face of the December change working makes zero sense, especially how it's done right now. So I think. That's the other piece. Policy proposals were very slow and lacking, but also very naive. And ultimately, I think not actually in the spirit of like freedom, not in the spirit of democracy, not in the spirit of like basically free markets. And I think that's very problematic. Like ultimately, I think it's like, and David Deutsch actually did a really good interview on this. And I think he basically said, it's like, if you're like ultimately worried about like basically unsolvable problems,

• 51:51Speaker 00

  but it's like the only path forward is like to solve them. And it's like ultimately AI can help solve those problems. Like AI, like we proved with also like one of our first model releases can help make the world safer with regards to biosafety. So I think maybe the the philosophy more broadly underlying this, I think well encapsulated by what Vitalik wrote, if you read it on like D I C of kind of like basically accelerating defensive technologies and differential progress, like democracy and decentralization, that ultimately is the path, probably the brightest future in this direction. It doesn't include governments overregulating. It's rather making sure that ultimately the good actors are supported,

• 52:31Speaker 00

  while the bad actors are stifled, right? And I think like, like policy and regulation completely has the opposite effect. I just like the best and most well intentioned actors, they'll follow it. The bad actors, they don't care. And I think that's the reality to an extent. Yeah, with how it's already playing out with like governance.

• 52:50Nathan Labenz?

  I feel you on the counterproductive nature of many regulations for sure. I'm personally worried about that right now with all of our policies that are directed toward China. I'm like, you know, it's, I just can't shake the feeling, no matter how many arguments I hear to the contrary, that it feels like the actions we're taking now are going to make this situation worse. You also mentioned super alignment, and that was one, one other thought I wanted to kind of pick up on. What do you think is reasonable for society to expect of the frontier labs? I mean, I was a supporter. I went back and forth on SB 1047. Initially, I thought, yeah, it seems okay. Then there was a moment where they like, there were many drafts, of course, right? And then there was one moment in time where it seemed like this sort of.

• 53:36Nathan Labenz?

  Panel or committee or commission, whatever it was officially called, was going to be kind of over empowered. And I was like, man, I don't know. It doesn't sound that great to have these like five people that are appointed by who exactly? The governor? And then they're going to get to make all these decisions. So then I was kind of out on it when that was the version that was being considered. And then they cut that from the final version. And then I sort of would summarize the final version as basically like, if you are doing frontier AI work, you have to have a safety plan. You have to publish that safety plan. And you at least have to be like, open to a certain amount of scrutiny. That to me, in the end, seemed like pretty reasonable. And, you know, as things go, and I'm very mindful of the risk of unintended, or even outright

• 54:19Nathan Labenz?

  opposite consequences from what policymakers intend. That seemed like a pretty light touch, like not super crazy sort of thing. You know, even even if it's not like law, you could frame it as law, policy, regulation, whatever, or you could just frame it as like, normatively, in your opinion, what does it mean for a frontier AI developer to be acting responsibly in today's world? Because it does seem like they're taking sufficiently, potentially disruptive steps that like, they should be proceeding with caution. They should be trying really hard to act responsible. But like, what does that mean in your mind? Or what should we expect of them or demand of them?

• 55:00Speaker 00

  Yeah, I think like, the key aspect is this differentially accelerating defense. But that's basically that ultimately, you and I can't defend against like, an AI if we don't have one ourselves with intent to like, whatever, like, defending against like, cyber, like bio, and other things. So I think, going back to the frontier labs, I think, a lot of the way how it's currently being done, I think it's like, very reasonable, right? It's like, they do right teaming. They have their own goals to like, publish something that is safe, right? And that they basically can't easily get misuse. So I think ultimately, the, it's a bit like, you can create a technology, right? And like, the burden is to an extent on the user, right?

• 55:42Speaker 00

  It's like, a lot of things you can misuse, right? Like, be it the internet, a car, or anything. And ultimately, most people don't, right? Like, most people use it very lawfully and carefully, right? And I think the same, we should assume with AI is that like, the majority of humanity by far, like, will use it for the best purposes. And I think that's that's like, a very tiny minority that will misuse it.

• 56:05Speaker 00

  And I think that's really where the focus should be, right? It's like basically stopping misuse of generally like technologies, right? But I think the other is like, it it depends, I think, on on the flavor of of GI research. Where I think that there was like very reasonable steps being done by like all the labs on like safety and alignment and all of these things. And I think the benefit really with like o- um, open source is that you have actually more oversight. You have more transparency of like, just by design, like people are able to jailbreak that and create solution with spotted jailbreaks. They are able to basically just very quickly like make those systems much more robust. So basically, I think it's like, if I think about a system, right?

• 56:43Speaker 00

  It's like that I want to hand my healthcare into the occasion of the kids and like basically a a key of parts of like societal functions. I want to make sure that actually we can see the code, we can verify the code, we can run tests against it, right? Like we can battle test it. It's actually like, you know, like a black box, like we were served it by the AGI labs, doesn't instill a lot of confidence in me. So I think there's basically, I think fair demands for people to like get more insights into like those front end models than just like an API endpoint, right? Basically, like limited to access the capabilities that actually those models have, right? And I I know you you, I think also the red team, right?

• 57:23Speaker 00

  Some of the most in the past, like I'm curious on on also your perspective, but I think there's a lot one can like find out, like even using those closed black boxes. But ultimately, you can't look into the black box, like you can't understand what's really happening. And I think that's the problem, right? It's like, I think we would be in a way safer si- situation if all the front AGI models would be completely open source. And like similar to like the internet, right? It's like, and and a lot of the building blocks of the internet are extremely open, like similar for blockchains, like all the code, everything is open. They're extremely safe, right? It's like ultimately, they're safer for being fully open and having kind of like this almost like adversarial environment where people try to break them, but they can't, right?

• 58:04Speaker 00

  Like in most cases, and I think that's really like would be the safest path also for this technology. And I think. In the face of this, like ultimately, I think there is some like given they have black boxes. I think closed malls should have actually more stringent regulation than open source malls. Because ultimately you can't like basically, I think they should open their code to like a bunch of people that like look deeply into it, that do tests beyond just having an API endpoint. So I think the jury's like really out on like if closed malls basically get to the same rigorous like testing and like transparency that like open source models have by nature. But yeah, I'm curious how you think about it also from where teaming them.

• 58:46Nathan Labenz?

  I'm of mixed mind on this. You know, my background has always been pretty libertarian politics, you know, much always skeptical of regulation, generally pro capitalism, free markets, you know, within a pretty wide range of like of outcomes. I do wonder how well the sort of open source label really applies to models. Cause I do think, you know, models are not that much less of a black box. They can be somewhat less of a black box when you have the weights. Cause I'm a very small angel investor in Goodfire, for example, and they're doing interpretability on Lama 3.7 DB or 3.1 or 3.3, or whatever it is exactly. So there is some degree to which that black box obviously started to get cracked open.

• 59:31Nathan Labenz?

  But it's still not like we're anywhere close to having the ability to really say, you know, we now know that like Lama 3 point whatever is, you know, fully under control or like won't do bad things or, you know, anything along those lines. I find myself always looking for some way to square these, you know, to get the best of both worlds or to square the circle where I'm like, I do think more than API access and just generally more access for safety and academic researchers at the big labs seems like it should be a pretty significant priority. The what access I had was like purely API, very limited, very little information. That's gotten a little bit better, but I would like to see it get better still.

• 1:00:15Nathan Labenz?

  But then I also do kind of worry that like, and I think the sort of R1 moment is kind of a good example of this that. We don't know exactly what the trajectory of these technologies is going to be. And so there is something kind of risky about open sourcing frontier stuff because you can't really take it back. And, you know, if you open sourced Llama 70B and then somebody comes along and does some gain of function research on 70B, as they did with R1, right? I mean, in the R1 paper, they found that they were able to take all these like midsize open source models to like dramatically higher levels of reasoning capability in a way that, you know, probably if you'd asked them,

• 1:00:56Nathan Labenz?

  the people that made those models like, hey, what do you think is the best ever that somebody will be able to do if they fine tune this, you know, on these math benchmarks? They'll be like, oh, well, probably not that good, right? And now all of a sudden, whoa, there's like a huge unlock and it's sort of something that society as a whole is like, that's just now the norm, right? So I do wish there was some way to kind of enable the access, enable the research, enable the understanding, but also give us some way to like, take certain things back if we do make mistakes. And so far, I think we're mostly just protected by the fact that like the AIs that are out there are just not that powerful. And so, you know, we haven't had anything where we really needed to take it back. But it certainly seems realistic to me that in the not too distant future, you know, you could have a sequence of events where

• 1:01:41Nathan Labenz?

  something that's a bit more powerful gets open sourced and it seemed fine at the time, but then there's like another unlock of post-training or whatever that sort of takes it to a whole other level. And then it's like, geez, you know, this now becomes a real problem. And yet there's not really anything that can be done about it. I think you guys actually have one really good example, and maybe you'll just jump to that now because you mentioned the D act philosophy. And honestly, one of the best examples of an open source AI project that I think fits that description is the recent work that you guys put out. And I was really interested to see too, that it was in partnership with Secure Bio. So you want to tell us about the, the work that you've done in, in biology recently that kind of illustrates

• 1:02:26Nathan Labenz?

  how we can maybe get the best of both worlds, at least in some, you know, domains?

• 1:02:31Speaker 00

  Yes, yeah, so basically for context, we we trained or supported basically training of a meta genomic foundation model for early pandemic detection in wastewater and can also be more broadly extended. Actually, I think two or three things are interesting. On the one side, it's how almost like a little computer took, I think it was like twenty, thirty thousand dollars of compute to create a state of the art foundation model to detect pandemics earlier. And so we'll continue to train it in a this real fashion to improve it, like there's much more data we can train it on. But I think the other interesting fact is that basically there's a lot of defense favoring open source models that we built.

• 1:03:10Speaker 00

  And I think a lot of basically very like objectively like high impact positive modes haven't been built yet, right? It's like the best examples is like a lot of different scientific foundation models, right? It's like if you look at something like AlphaFold, it has very little misuse risk and it's gigantic upside, right? Like for humanity. So even from a like almost like utilitarian perspective, it's like the it's very heavily skewed towards like positive outcomes. Yeah, like almost like to explain it how it works, right? It's like basically the data is mainly just a lot of different wastewater data to detect what's in the water. And if you can count on like or basically detect like the outbreaks of pandemics early, which was actually the way how COVID was detected in the earliest disentiation was through wastewater detection.

• 1:03:53Speaker 00

  And I think it's how you can even like monitor and spread like to grow the spread of those risks. So I think that's a good example where it's like if we put those things in place, right? It's like basically we we are in a much safer world, right? In general, right? Like irrespective of AGI, like COVID probably didn't happen due to like intelligent systems. It actually happened due to like actually like probably at this stage, like quite quite factually through misuse. Or like basically it's like mistakes by nation state actors, right? It's like by US and China and like in their involvement in gain of function research, right? And ultimately it lacked the antibodies for society and the the defense mechanisms, right? Like to avoid it. So I think that's really like how we also go into this like utopia and a safe AGI future.

• 1:04:36Speaker 00

  It's like if we look very concretely at the risk, right? And biosafety being one of them and then just solve them. Like so basically even if there is a next COVID or pandemic, which surely there will be, that's actually not a problem because we have the vaccines, we have the early detection, we have all the other defenses to basically avoid this spiraling into a problem. But I think, yeah, for the for the model, basically was like collaboration with UCL and kind of like the like nucleic acid observatory and like the safety and A team. And they basically did all the research work and we helped them really on the one side with compute, but also on more like the training side, right? Making sure that a good model comes out of this. And I think it's a good example where there's like a lot more things like this, where even like

• 1:05:20Speaker 00

  where in the millions of compute, you could really transform scientific progress and human progress more broadly. And I think that's really obviously also, I think, to going back to the YouTube envision, right? It's like it's intelligent, like solving intelligence to solve everything else. And I think the most important thing to solve really is science. And I think it's like solving all the diseases, solving natural risks, but also things like climate, right? And energy and a bunch of other pieces that I think will just create an enormous amount of human flourishing and progress. So I think it's actually very key to our broader goal is to help solve science and really also with a lot of the connections we made in that world. I think a key piece of this is almost like working towards autonomous AI researcher and autonomous scientists that can safely advance different areas of science.

• 1:06:05Speaker 00

  And it's just bottlenecked by almost like the amount of compute you can feed it to do reasoning and like solve those problems and like simulate them. So I think that's actually a world that we're like already starting to move into. It's like a world in which like AI will like massively accelerate science, like scientific progress more broadly.

• 1:06:24Nathan Labenz?

  Yeah, I've been tracking that general phenomenon in a lot of different domains and it is like everything else, it's really ramping up. What I really found interesting about this particular model, and I'll just read a couple highlights off of your blog post, it's called Metagene one.

• 1:06:43Nathan Labenz?

  And then I'd be interested to hear like what other sort of ideas you think have similar shape in terms of their, you know, defense favoring properties. But this one I thought was really thoughtfully designed because of course we've heard from so many, you know, warriors, myself included, from time to time, like, geez, you know, do we really want, you know, expert virologists in everybody's pocket? That seems like maybe not a great, you know, technology to make freely available in a way we can't take back. This is like almost the exact inverse of that where I think at least what jumps out to me as the key architectural decision is that the model only has a 512 token context length. And so that's like, obviously a lot shorter than any genome and it's not going to be able

• 1:07:28Nathan Labenz?

  to be used to generate full genomes. There's been a lot of talk, I'm not sure how much exactly like research there's been, although I did just see one paper that came out that I want to circle back to around like creating AIs that are myopic in general, which is like, you know, to say, we don't want these things necessarily thinking like super big, long-term plans. We don't want them doing super huge things that we don't understand. We want them to do really well at the thing that we put in front of them that is there for them to do. And this is that, right? 512 tokens, you can only do so much, but that is enough to detect anomalies. And so you've created something there that it does seem overwhelmingly likely that you could put this in everybody's pocket. And really, the only thing they could do with it would be monitor

• 1:08:13Nathan Labenz?

  their local wastewater for anomalies. And that just seems like, and I love projects like this that are sort of the unilateral provision of a global public good. That's one of my favorite phrases. And this really, I think, fits that bill where you're like, for the people that need this, it's incredible. For the general sort of balance of attack and defense, this seems extremely useful. And yet, it's really hard to imagine how anybody could really abuse it. And that is baked into the artifact itself. Now, when you mentioned a general purpose scientist, like, I'm not so sure that that trait is always there. Like, what other projects do you have in mind that have that set of properties?

• 1:08:54Speaker 00

  Like, going to this point, I think that's really the philosophy, right? It's like figuring out what could be the biggest contribution. Also, that almost has a disproportionate impact to the amount of computer takes, right? It's like, there's other things you could do with hundreds of millions that have less impact, you know, where it's like, really about figuring out what's the lowest hanging fruit for human progress, what is differential progress. I think it's even like leads, and I shared with like, different people in biosafety about like the work we did. They're very excited. To like open up like almost like a distributed kind of like pathogen detection and monitoring effort, right? Like where where you could at some point have like millions of places, right? Like analyze those samples, which I already do, but like obviously not on a granular like individual home basis, but more like on like a city basis, right? Or airports and things like this.

• 1:09:38Speaker 00

  But I think like more broadly, I think like we we basically want to make sure, right? Like that we that we build like the the key pieces and the key research needed to scale progress radically, right? And I think basically it's like almost like figuring out how to create a systems with a human in the loop to do science autonomously or AI research autonomously in a safe fashion, right? I think also the the path to things like super lining, right? It's like at the core of those philosophies being like realistically, the only way to solve this superintelligence alignment or the only reasonable path to have an autonomous scientific system solve programmable pandemics is to have the right guardrails and the right like humans in the loop. And and just the benefits of open source and a lot of participa- uh participants to drive them in the best direction.

• 1:10:23Speaker 00

  So I think um we have basically multiple things planned in this direction, including other scientific foundation models that that basically just like generally just like advance progress. Like for example, like a virtual cell foundation model is is one idea or things like a simple autonomous scientist where where you also have some of those like mechanisms in place where humans can uh like interact with with AI scientists. But I think those are the most promising paths to also just like radical positive human progress. I think it's like what needs to do this like iterative deployment, right? Like where one like builds out in small steps and then like improves, better tests it and scales it. And then makes it accessible to everyone, right? To have basically the biggest potential impact.

• 1:11:09Speaker 00

  Ready to switch gears to distributed training?

• 1:11:13Nathan Labenz?

  We've covered a lot of ground, but now I think might be a good time to just dive into the technical details of that. So again, just to remind myself and everybody, why does this matter? I mean, multiple reasons, right? It's like in the limit, you know, it could create a sort of truly decentralized AI infrastructure that nobody can control. It's also just a challenge to build clusters beyond a certain scale. There's also this spare compute notion, major implications for compute governance. So lots of reasons, I think, to care about distributed training. You guys have an excellent blog post, which is nine months old, but I still found very well worth reading today. And I think that's like, honestly, a rare accomplishment in AI that you could have something that is still

• 1:11:56Nathan Labenz?

  worth reading nine months later. So I commend you for for that. I'd love to kind of just walk through that and talk through the details. And then, you know, we can go beyond the, you know, the time of the blog post too, because you guys have actively contributed to the frontier since then. Well, let's just start maybe with what makes distributed training hard? Like, why is it, you know, why was it ever a question as to whether or not it would work?

• 1:12:23Speaker 02

  Yeah, this is mainly about the bandwidth requirements you have for your usual distributed training, right? Even distributed training inside a single cluster is quite hard to do. So you have to make sure that for every kind of like parallelization strategy you use to like parallelize your workloads across all the G PUs you have in one data center, that you do it efficiently in a sense that it works efficiently for the different parallelization strategies, right? And yeah, some of them have more memory or more bandwidth requirements than others, right? So one of the ones that is also used in our local technique, for example, is normal data parallel training, where you basically do your forward pass, do your backward pass of your whole model, and then you only send the gradients at the end in a very efficient, like all reduced manner across all those nodes in the network,

• 1:13:08Speaker 02

  right? And there are other techniques that have more memory requirements. So one of those is like tensor model parallelism, where you basically split up the whole weights of your model across different G PUs. And then you have to do communication between different G PUs for every single layer, basically, in your in your transformer model, right? And another technique we also mentioned in the blog post is pipeline parallelism, where you have different stages. So you only have to communicate between those stages. And on all those stages, there are different blocks of layers of your transformer. So you only have to sync the last activation hidden state to the next stage and to your next pipeline stage, which has less memory requirements than your tensor model parallelism,

• 1:13:52Speaker 02

  but still a lot more memory, a lot more bandwidth requirements than your data parallelism. And those are the three different techniques you can basically go for. There's a couple more other things that people figured out over the last couple of years, but that's the main one. And a lot of people have been going for distributed training across like non-co-located clusters now. This is data parallel regime, which in a normal setting is still way too communication intensive to actually do it. Yeah, with interconnect, that is slower than 100 gigs for large models. So even in this regime, you have to find techniques to have less communication between the nodes. And yeah, one of the techniques, yeah, we've been working on quite heavily and have been building on top of is this

• 1:14:35Speaker 02

  the local technique by Begoob Dikman, distributed low communication training, which is basically a technique where you train on those different islands of devices in a data parallel fashion. But instead of like syncing every step in your training now, you only have to sync every couple of hundreds of steps, basically. So you sync only a hundred, couple of steps, like those pseudo gradients and this mechanism. And then you have an auto-optimization loop to merge those results, basically, from all those different nodes. And yeah, it basically works as efficient as in a centralized cluster with obviously some limitations in a sense on how many actual nodes you can scale to and where it actually works well. Like at the beginning of the training, it doesn't work as well. But in later stages of training, it's pretty much as efficient as your usual data parallel training.

• 1:15:21Nathan Labenz?

  Okay, let me take that back entirely from the top. Make sure I understand it. You can then correct me and elaborate on the things that I'm either getting wrong or simplifying. But I do think this is really important for people to understand. So I guess in terms of just the fundamental challenge of like a naive setup, right? Data parallelism is the first kind of parallelism. And that just means if I'm going to train a giant model, obviously, you know, if I want to do, for example, 15 trillion tokens, as you know, some famous open source models have done, I'm not going to do one token at a time. The whole game is right. I need to, the reason I have these big clusters is I need to parallelize this out.

• 1:16:02Nathan Labenz?

  I can't possibly, you know, have one model do a forward pass, take the gradient, update, you know, and just keep using this one copy of the model. You'd never get there. Now we can just say data parallelism is basically you make a bunch of copies of the model. You train all these different copies, or maybe I should say you run the forward passes and take the backward pass gradients on all these independent copies of the model, which are each processing their own data. And then the aggregation step is the part where a lot of stuff has to fly around. And I think it's underappreciated that the gradient is basically the same footprint as the model itself, right?

• 1:16:47Nathan Labenz?

  Because if I've got 100 billion parameters sitting on a server somewhere and I take a backward pass, in the limit, I'm at least, you know, the naive approach is I'm have one adjustment to each one of those 100 billion parameters. So I have 100 billion adjustments that I want to make to the model. And now if I have 10,000, you know, GPUs wide, each having processed its own data and

• 1:17:15Nathan Labenz?

  Found its gradient in order to learn from that particular data point that it just crunched. Now I've got 10,000 times 100 billion numbers that I need to bring to some aggregate form. So then I have the sort of cumulative gradient from this training step. Now I can go apply that to all the copies of the model and then I can take the next step. Right. And then I'm not sure how much we need to add on to that in terms of tensor parallelism or pipeline parallelism to to help people at least develop like a decent intuition. I don't ma- maybe there's more you would add, but it seems like that what I just described feels to me like the really fundamental issue and then tensor and and pipeline parallelism

• 1:17:57Nathan Labenz?

  also is important because like your model is too big for a single GPU, then you got to start to split your model. And so that just adds even like more complexity and overhead. But even if you had like giant GPUs that could fit all the parameters onto one, you would still have this data parallelism problem where you're just like, I have to run a bunch of copies and I have to figure out some way to aggregate the changes that each instance of the model wants to make so that I can make like the global change and that the overall model can keep getting better. Pause there. Is there anything more that in terms of like fundamental intuition that somebody who's not going to do this themselves, but wants to understand generally what's going on needs to know?

• 1:18:40Speaker 02

  No, I think that's great. It probably makes sense as part of this conversation to stay in the data parallel regime because the other one is even harder to do in a distributed fashion, right? Like there are techniques for this too, obviously, but yeah, as as part of this conversation makes sense to to stay in the data parallel regime and understand that use case. But yeah, as you said, all those models have a lot of memory requirements too, right? You can't train your 100 billion parameter model on a 4090 because it takes too much memory. Even the parameters are too big, right? And especially for the 20 use case, which people don't always realize, right? You have even lost more memory requirements than for inference, right? So in in training, you need your model parameters. You have your gradients, which are the same size as your parameters. And then you also have a huge optimizer state, right?

• 1:19:23Speaker 02

  Uh, thing to to update your your model parameters, right? And those kind of things take a lot of memory, which is yeah, one of the reasons, for example, in our 10 billion parameter run, we started with intellect one. Yeah, the biggest thing at the time we were able to to fit on a node basically was around like a 10 billion parameter model in terms of memory requirements for training, right? And then doing data parallel training across those.

• 1:19:49Nathan Labenz?

  Basically, the momentum term helps stabilize training overall, right? You like you could take the gradients without the momentum term. But then you have convergence problems because everything's sort of thrashing around and the momentum term basically sort of keeps your individual updates from going too far in in unusual directions. But again, you have one of those momentum things to keep track of for each parameter, right? So essentially you have a multiple for however many parameters you have, which are numbers, you have a multiple of that number of numbers that you need to keep track of during the training process and then figure out how to aggregate across.

• 1:20:28Nathan Labenz?

  So when you go to something like Deloco, which is the distributed training regime or training scheme, I guess, maybe that DeepMind put out and that you've elaborated on since then. Is it as simple as an insight that like actually you can just run more forward passes and kind of keep track of things locally and you just simply don't need to aggregate every step and it still works? Or is there more to it than that?

• 1:20:53Speaker 02

  For sure, it's a very empirical result, right? Just showing that it basically works by doing this. It's also not the first work into this direction, obviously, right? There's been a lot of the whole like federated learning literature has been doing a lot of things in this direction as well as as a general algorithm is more called local S G T type of approaches, right? And where you are able to do those local steps and then in a sim can have an outer step basically. So it's just been showing to empirically work really well. From from the really mind on the things they said in their paper to like a 400 million parameter model, which is yeah, in real spec, not that large for large language models these days, right? We scaled it up to the 1 billion parameter size where it was still working and then to the 10 billion parameter size, right, with the intellect one run.

• 1:21:38Speaker 02

  Unfortunately, obviously, didn't have a baseline run to compare to, because at that scale, the the costs are a bit too large for like a small startup to actually run baseline operations, right, on those kind of models. But yeah, as a general intuition, you can do those local steps and don't have to sync as often. It works a little bit less at the beginning of the training, which is also one of the things that the local paper showed and we showed later on, right? Where at the beginning, it's very clear the gradients have to move in a specific direction, right? And then the merging of those doesn't really help too much, right? But in the later, like, very flat regime of the loss, Deloco is working really well.

• 1:22:14Nathan Labenz?

  Okay, let me help me understand that a little bit better, because I almost had the opposite intuition at first, like, when you're learning very quickly, it would seem like maybe you need to share those learnings sooner, but it seems like you're saying the opposite, like, I don't know, I'm confused. Help me, let me try, help me explain that a little better.

• 1:22:31Speaker 02

  At the beginning, there's not much signal because they all go into the same direction, right? And then later on in training, there's a lot of signal coming from all those different data parallel ranks. And then the technique just works better in the in the later stages of training. It's also very much empirical in the sense that we've seen Deloco converge a little bit slower at the beginning and then pretty much catch up to your your baseline experiments with just doing their parallel training.

• 1:22:57Nathan Labenz?

  Yeah, okay, that's interesting. So there's not much major trade off?

• 1:23:01Speaker 02

  And there's certainly still things to solve in a sense, right? Like, there are three things we are obviously work on since we've been doing intellect one, right? For the intellect truth around like around like three major components. For the one part, the communication requirements are still too large for like training a 100 billion parameter model with this technique. We would have to have even less communication and have been working on different techniques there to quantify student gradients and other research directions. So that's one of the things where it has some limitations still, right? Other parts are just memory requirements. So if you take larger nodes of H100s or multiple nodes, obviously it would fit there. So this requirement is gone.

• 1:23:41Speaker 02

  But to actually make it possible, so everybody could join at least with like some memory requirement from A100 node and H100 node. So there's still pretty much a limitation to scale this one. And as well as then on the number of workers. So the local currently still has some diminishing returns in terms of how many workers can you actually scale to. And when you you stint the gradients after like the 500 steps of pseudo gradients, right, the signal has lost a little bit if you have too many workers. So what we've empirically tested within like one run, we're scaling well to like 16 workers, for example, right. But to truly make a more distributed 21 happen, we have to scale to hundreds and thousands of workers, right?

• 1:24:21Nathan Labenz?

  Yeah, that's interesting. So and the the intuition there is is simply that because these things are like each kind of over learning in whatever random directions they happen to be over learning. And then if you have just too many of those, the whole thing becomes noisy. And because these like over learnings are sort of happening in in various directions, they're kind of cancelling each other out, the signal is getting lost.

• 1:24:49Speaker 02

  Yeah, it's not like it doesn't work at all anymore. So we've been scaling to two more workers in the sense in our testing, but it's just not as compute efficient anymore as your usual data parallel training. So that's what you want to aim for. You want to you can take some trade offs because in a distributed global setting, the the flops are maybe cheaper, right? Because we can run spot instances, we can have people contribute their own compute resources and so on. So the flops are cheaper, the bandwidth is more extensive, but we still want to be on par in a sense to how efficient it is to to train in a in a centralized cluster.

• 1:25:22Nathan Labenz?

  Yeah, gotcha. So you mentioned like quantizing gradients. I assume you've sort of similar, maybe even the same idea as just saying like, you could sort of round probably a lot of gradients down to zero, perhaps, and sort of say, like, anything below a certain size, you know, we just don't have to send across the wire at all, maybe. What other what other tools are in your toolbox there that are?

• 1:25:48Speaker 02

  Yeah, we obviously have a lot of coming up in terms of like improving on that algorithm, right? Um, submitting a paper they see. Um, I think what we've done so far, right, is uh just what we've realized those pseudo gradients were actually sent after those hundreds of steps. So it's not the actual gradients of the model. It's a difference between the beginning of the weights and the end state of the weights, um, after all those inner step updates. And what we've realized is that those two grand is actually um pretty easy to uh well to uh quantify. So what we've done so far for the intellect one run, uh, we haven't been sending those in like uh thirty two bit precision, but we've uh been sending those in eight bit precision, which gives us another like four X uh reduction in computation requirements. And for the intellect one run,

• 1:26:31Speaker 02

  it was enough to basically send every hundred of a hundred steps. And so we had a total communication reduction of uh four hundred X, right? Which was enough to still train very efficiently across the whole globe, right? With the interconnect we had there for a ten billion premiere model. And if you do more inner steps, basically, um, find more quantization techniques, different things on top of it, we could probably scale that to even other modes.

• 1:26:58Nathan Labenz?

  Yeah, that's interesting. The other thing that really stood out to me was the Deepako scheme where, and this I think is interesting for multiple reasons. I mean, I had previously been kind of studying mixture of experts in general and, you know, interested to learn that like, just like so many other things, like the sort of relationship between the experts is like generally very opaque. There's not, you know, you should not model the experts as like actually being domain, you know, subject matter experts in a recognizable area of human pursuit or whatever. On the contrary, you know, just like everything else, it's like, why did that token get routed to that expert or set of experts like, you know, we don't know. But this Deepako thing starts to

• 1:27:43Nathan Labenz?

  segment data by like longer sequences and therefore, you know, it's not like you have to juggle things on a token by token level. That seems to me like Both potentially interesting from an efficiency standpoint, but also from a, you know, interpretability standpoint and potentially on some of these like gradient sync questions. I wonder if you were to say like, what if we sent all of our science, you know, information over to this, you know, sub cluster and we sent all of our like literature over to this sub cluster, would that have a chance at sort of allowing these things to do like more locally before they need to aggregate and potentially would it also create resulting

• 1:28:27Nathan Labenz?

  architectures where there actually is like a separation of concerns, more like a traditional software project and less like, you know, a giant spaghetti black box that we are accustomed to? How much promise do you think there is in that sort of like, let's say semantic data segmentation sort of idea?

• 1:28:43Speaker 02

  Yeah, great question. And yeah, that's I think what a lot of people are getting around, right? In terms of how they think about mixture of experts, right? That they actually have some semantic routing in there that it actually routes to specific experts for a specific domains, which is yeah, just generally it doesn't seem to be the case, right? Just seems to be a more efficient way to compute to get a lower loss, right? So the usual mixture of expert way where you basically, which yeah, it's still a lot behind a lot of the models, right? Like even the newer deep sea models, right? Where you route per token level. So for every token, you route to a different experts of MLP. And the Tarko way, once you're allowed to actually distribute the whole mixture of experts, which is not really possible if you route to a different expert for every token.

• 1:29:27Speaker 02

  So you have to do it not in such a granular way, but you, you can only do it on the sequence level. So for every sequence, you can route to a different expert, which yeah, as you said, theoretically would allow for like routing to different experts for actually domain experts in a way. Unfortunately, I'm, I'm not too bullish on the whole way of like routing it by the domains, basically, right? I think it's just a mixture of expert that doesn't work as well as what we've been seeing with the normal token level mixture of experts. So, um, yeah, would love to see if somebody wants to replicate the Tarko in an open way to, yeah, see more research happening, right? And see if it works as well to actually use a routing on a bit more of a domain level.

• 1:30:10Speaker 02

  And but yeah, my intuition is, is not to, yeah, that I'm not too bullish on it.

• 1:30:16Nathan Labenz?

  And is that because you just feel like it's too clever and clever things don't work? Or is there what motivates? Because I mean, it would really be in some ways a huge win if you could get a semantic segmentation between experts, right? For multiple reasons, but just on the DeepSeek V3 R1 architecture, if I recall correctly, it's 671 billion parameters of which I think 37 are active at any given point in time. So it's like 6% or something of the, you know, overall parameters in the model are used in any one forward pass. If you could do like sufficient level of semantic segmentation across experts, you

• 1:31:00Nathan Labenz?

  know, you could have the sort of, you know, virologist removed, you know, package that I think that's another way I kind of try to think about like squaring the circle. If you could localize certain highly sensitive, you know, sets of information or knowledge in the model and then distribute a version that like just doesn't have the virology knowledge as sort of the base, you know, open source case, that might be a great way to be like, look, everybody has everything you really need and you just don't get the virologist. And like, you probably, you know, shouldn't complain too much about that if you're, you know, a normal user. But yeah, so what can you tell me more about why you think that's ultimately not going to work?

• 1:31:38Speaker 02

  I think it's got to work in a sense, right? You can definitely do it. I'm not as sure about that it works as well as just letting the model learn, right? So adding like this inductive bias on like, hey, we want to route those to those different domains is probably not as efficient to just let some mix of expert learn on their own where they want to route it to. And just empirically has been shown the mix of expert is not learning where to actually that to route it by a domain, right? But rather in a completely different way that is not as interpretable for us, right? Which is unfortunate for our interpretability research, obviously, right? It would have learned it if we would have created a better model. So that's, that's my kind of view on it.

• 1:32:19Speaker 00

  Yeah, I think like to add to that, I think there's almost like two things that I think are interesting on the one side that like one can build those architectures that are slightly better set up for like a distributed setting. I think on the other side is that we're obviously entering a new scaling paradigm, right, like of like inference time compute, which scales much better, right? It's like basically you get much more reasoning for much less compute and it's actually perfectly set up for decentralized trading, like almost like by coincidence, right? So I think basically the parkour I think was in many ways interesting. I think looking into the present and future, I think it's it's more like the R1 scaling paradigm is actually perfect for a decentralized setting on the one side. I think on the other side, it's like basically it's very little communication requirements, right?

• 1:33:01Speaker 00

  Because a lot of it is scaling through synthetic data and through and for some compute. And I think on the other side, I think to your interoperability point, like the RL chains, right? Like it's good if you can read them, it's good if you can see them, right? Which you can't for like O1 or O3, right? Even for interoperability and like safety and whatnot, and even as an end user to understand how it's reasoning. So I think it's actually quite a, in some ways, interpretive paradigm. And I think that's also what like, um, I hope my iPhones try to make you believe, right? It's like the like, it shows you how it's reasoning, right? And you can align and kind of like edit out specific reasoning. You can add some things into the reasoning, right? Like some safety checks or some general checks.

• 1:33:45Speaker 00

  I think on the other side though, to be realistic, right? It's still like a Chicagoth way of reasoning, right? It's like basically suddenly starts reasoning in Chinese, like drops some like random bits that like you also can't really understand. I think it's it's just like, almost like the emergence, like where it's like scaling laws don't like scale reasoning in the exact same way as human reasoning, right? Like they they take some like different paths through all this like this multi-dimensional space, right? Of math, like that that is kind of like foreign to us, but it's still probably there's there's patterns and structure in those systems, right? Just like maybe not, um, they don't map like anthropomorphically like one-on-one to human reasoning and like language.

• 1:34:27Speaker 00

  So yeah, I think that's kind of like the paradigm we're in and the paradigm we are like planning to scale is almost more like inference time compute paradigm in a decentralized fashion.

• 1:34:38Nathan Labenz?

  Yeah, a couple of interesting points there. In terms of just very practically on the

• 1:34:44Nathan Labenz?

  why is the RL, like the R1 style training more favorable? It's, I don't know if you have like better numbers on this, but because I don't think it was in the R1 paper. But basically, you're doing a lot more forward passes relative to the number of backward passes. Instead of having to do a backward pass for every document that gets, you know, processed through the model. Because you want to initially in pre-training, right, you're doing every, I know you guys understand this, this is more for the audience. You know, every single token, you're kind of able to adjust on here. It's like, did you, did you get it right? Did you get it wrong?

• 1:35:28Nathan Labenz?

  Take the best of N samples and maybe also the worst and like you only have to do the backward pass and, and look at the gradient for like these sort of extremely good or, or extremely bad generations. And so that just like dramatically reduces the overhead in terms of the bandwidth that's required, right? Any, anything else that needs to be added there is any ratio in the R1 paper? I don't think it said how many generations they're doing and like what their sort of best of strategy is. But you may know from other sources, what is kind of the norm there?

• 1:36:00Speaker 02

  Yeah, like unfortunately for, for this time, like obviously DeepSeek is always pretty transparent in what they're doing, right? For the R1 paper, there were a couple of missing details, definitely on the, yeah, how they actually trained that thing, right? How did the infrastructure look like and how long did they train for what's the actual communication requirements were, right? But in general, like the, the two things that are needed for the DeepCigar one model in a sense, right? Like one of the things they've shown that is that supervised fine tuning on just a bunch of different reasoning chains from yeah, task and identifiers that you've just generated with a different kind of model.

• 1:36:37Speaker 02

  And it's helping quite a bit for like actually getting a better R1 at the end with their reinforcement learning approach that is layered on top later, right? And the two components, first of all, the supervised fine tuning to generate those large scale synthetic data sets is very much favorable for a split training setting, right? Well, split network, right, that you can just generate those reasoning change.

• 1:37:03Speaker 02

  Yeah, for the Latta models on like H100, H200, H200 nodes. And then, yeah, you can even like use the smaller disturbed models for our statistic data generation. And yeah, just have more supervised fine tuning data to train on. And this improves the performance of the statistical model. And then the second stage, right, where you actually do the GPO reinforcement learning on top of it. Yeah, they don't have the details there, but generally looking at what other people have been doing in the reinforcement learning regime. On top of language models, where you have a bunch of different rollout phases, right? You generate for 256 questions, different rollouts.

• 1:37:46Speaker 02

  And then only do an update step after you've done all those rollouts or do a backward pass on the afterwards. And yeah, accepting or rejecting a bunch of those. We've seen that you have like minutes of forward passes, basically, for every backward pass. And or it could be hours if you have a lot of rollout phases. So still very early on there, obviously, but yeah, it definitely looks like the whole paradigm seems to be much more favorable for distributed training.

• 1:38:13Nathan Labenz?

  Yeah, it also probably is worth mentioning that like, there's not going to be a single ratio of forward passes to backward pass, because you're also sort of interacting there with like curriculum learning type things or sort of what problems are you trying and what is your success rate on them? Because you need some signal, right? So if you're doing, if you're doing problems that you only get right one in a thousand times, then you need like a thousand to get one that's right, that allows you to get the reinforcement signal to proceed. So that's part of why the sampling strategy is important. And they have some interesting details on that. And I think in both that and the Kimi paper, there's like

• 1:38:55Nathan Labenz?

  a sort of algorithm that's like trying to make sure that the difficulty of the problem is well calibrated so that, you know, so that you're doing hard enough problems that you're still meaningfully learning, but you're doing, you know, problems that are in range enough that you don't have to try a thousand times before you get it right once. So you can learn from that. So I do think that's important to keep in mind, too.

• 1:39:18Nathan Labenz?

  What else? I mean, there were a number of other different papers that you run through in the blog post beyond what we've talked about so far. What else do you think is worthy of highlighting? Or, you know, what from your work since that blog post nine months ago, would you say is what people most need to know about?

• 1:39:36Speaker 02

  Yeah, a good question. If I think back about the blog post, like one of the other things mentioned in there is, for example, this swarm parallelism idea. It's a paper by somebody we've been collaborating with a lot to Max Verbeen, who's currently at TogetherAI. And he's done like really tremendous work in the whole distributed training space over the last six years and has been doing his PhD in it. And yeah, one of the techniques he's been coming up with there is this called swarm parallelism, where you also not only have your data parallel across the world, basically, but you can also do like pipeline parallelism across the world, which at least for like small sequence length, it's actually not that communication intensive, right? Because you only have to think the last activation

• 1:40:17Speaker 02

  state of your pipeline stage. So you only have to think something which for transformer model, but also for the different hybrid variants that are out there right now is always of the size, like your sequence length times your batch size times your hidden dimension, which at least for like small sequence length is not that big, or for also small in size and batch size. So that's something you can also potentially do over the internet. But then you also get into other things to be cautious of, right? In a data parallelism, for example, don't have to think about latency, which in one single data center is not an issue. But across the world, latency is a huge issue. And in your data parallelism,

• 1:40:58Speaker 02

  you don't really have to think about it because you have to do the update step so little that we have like 100 milliseconds of latency. It doesn't add much overhead. But for pipeline parallelism, we have to do that communication quite often with like very small sizes of tens as we have to send there. So that's one of the things to be cautious of there. But yeah, otherwise, it's also a great technique to actually scale that also to larger model sizes, which is one of the limitations we have right now in terms of how much, how large of a model can you actually fit on one node?

• 1:41:32Nathan Labenz?

  So does that swarm type thing start to open up the possibility of like truly end user like retail compute being contributed to training projects in the future? Cause my laptop can't hold like the full DeepSeek V3 or R1, but it might be able to hold like one of the experts or like one of the layers. How far from that or how realistic or how far from that do you think we are right now?

• 1:42:00Speaker 02

  Yeah, definitely would be a possibility for for smaller models, right? You can't still train a large model across a bit, have too many pipeline stages, right? As well as then the other like bottlenecks you run into with latency and so on in like a home GPU type setting, right? But yeah, theoretically it should allow for it. The one thing that makes me a bit bearish on doing this in a whole distributed fashion is how the current research paradigm has been moving, right? Like one of the things we've been doing is we create ever larger sequence length because we need those for the R1 style reasoning components, right? So whenever we increase our sequence length, it doesn't make a difference for your data parallel training.

• 1:42:41Speaker 02

  So your data parallel training, you have to only send the gradients, which aren't growing with your sequence length, but in pipeline parallelism, your activations, they grow by the sequence length. And if we go for a million sequence lengths to train something like an OSA type model, your activations are going to be way too large to actually send those. So I think there are better techniques. So one of the things we've been rather focused on is offloading a lot of the optimizer states, for example, on a single node, which there are techniques out there, zero offload and a lot of more modern approaches to make that efficient and train like a hundred billion model on a single node of H100s or A100s.

• 1:43:17Nathan Labenz?

  Can you describe offloading a little more? It's essentially moving the momentum terms off of the high bandwidth memory to

• 1:43:28Nathan Labenz?

  Other storage?

• 1:43:30Speaker 02

  Yeah, two, two stores on your node, basically, right? To CPU RAM, right? So that's how offloading works in a sense. And you certainly want to offload the optimizer stage without taking the most amount of memory, right? Like we touched on it earlier, in your user, like, mixed business training, you have your Adam W, right? You have your parameters, you have your gradients, which are stored in mostly 60 bit precision, right? But your optimizer, you have to store your parameters, a copy of it, you have to store in full precision. You have to restore your gradients again in full precision, a copy of it. You have the momentum, as you mentioned, and you have the variance also in full precision. So it takes up like way much more memory than your parameters and gradients.

• 1:44:14Speaker 02

  So if you're able to offload that efficiently while still keeping really good training efficiency, that's probably the way to go. And then you can fit like some of the largest models, basically, on a single node. Obviously, you still won't be able to train a DeepSeek R1 with 650 billion parameters on it. But at least to go into the 100 billion parameter regime.

• 1:44:35Nathan Labenz?

  One thing I've been thinking about a decent amount, I'm sure you've seen this blog post from Yi-Tai, who, you know, was at Google and then left to go start Reca and is now back at Google. I thought the blog post about like buying CPU or buying GPUs, I should say, in the wild, was a really illuminating one. And basically, the long and the short of it was like, he was saying, I had no idea how spoiled I was at Google because all the infrastructure there just works. And, you know, then you get out into the real world and you're like trying to buy from this cluster or that cluster. And this was like a year ago, so, you know, you didn't exist yet, maybe. And there was also just a more sort of frothy time in general in trying to buy compute.

• 1:45:18Nathan Labenz?

  But, and also people hadn't like worked out problems nearly as much, but it sounded like a real jungle. Like, at what point, if ever, do you expect us to kind of be in a similar place with GPUs as we are with CPUs? By which I mean, the hyperscalers years ago now, right? Basically figured out how to Sort of abstract away the differences in underlying hardware and create like consistent containers that, you know, they don't really have to worry too much. Any, you know, there's enough layers of abstraction. They don't really have to worry exactly what physical device is this running on. They just kind of know like the virtual environment that it works on and that can be consistent. And, you know, that's good enough. And on top

• 1:45:59Nathan Labenz?

  of that, they have the sort of, you know, fault tolerance of like, one computer craps out or whatever, somebody spills a drink on it, it doesn't matter. They just kind of route around those problems. And, you know, that's great, right? That is why we have all the tremendous uptime that we have, so on and so forth. It strikes me that we're not there with GPUs. Is it like, just because software takes time to mature, and we're just not there yet, and it will get there? Or is there something more fundamental that I may not appreciate? Because then I've heard these, you know, in contrast, I've heard these stories of like, you know, one GPU goes down and it like stops your training run. When I hear that, I'm like, wow, that's really weird. Like, that's a, that's a dramatic

• 1:46:40Nathan Labenz?

  departure from, you know, 10 years ago, like, large scale, you know, web products were supported, right? With Facebook 2013, or 2014, 2015, I think was already, you know, starting to get pretty far down that path. So, what do you make of that? You know, what should I expect in the future?

• 1:47:01Speaker 02

  Yep. Yes, you've mentioned it. For tolerance is a huge part there, right? And it's definitely on the software as well as hardware stack right now that it's, yeah, not perfect yet in the whole GPU regime, right? If you train across like, or like do a training one across like 100,000 GPUs in a single cluster, for example, like the, I don't know, Grok, XAI cluster or something like this, right? There will be a node that fails every, at least a couple hours, right? Maybe less. So, right now, a lot of the trigger and training frameworks are not actually made for for tolerance. So, if training one is just going to crash to this one node of those, thousands of nodes, if I got a crash. And yeah, you have to continue your training from a last checkpoint, for example.

• 1:47:44Speaker 02

  And yeah, all the big labs, a lot of them have it figured out. The other ones that don't have it figured out, they're racing to figure it out now because otherwise they won't be able to train their models across the largest cluster. And in our case, it's even more crazy in a sense. We definitely need for tolerance. We need even like more than that. We need onboarding of different GPU nodes during the training. And yeah, that's what we've been, we've done with 20 framework called Prime. We've also open source, right? Where we have this for tolerancy. So all the dial local rings, all the data pair of rings we have in our training for tolerant. Whenever somebody drops off, the training is not stopping. You can still continue training. And yeah, that's why I like also all the big labs are interested in like solutions there in a sense, right?

• 1:48:29Speaker 02

  How do you actually make this for tolerance to happen? And yeah, over the long term, I think it's going to be fixed, obviously on the software side, as well as on the hardware side. We're going to go more into like actually a commodity for compute, which is definitely not the case right now. There's, yeah, you've mentioned the RACAP paper where they basically showed there was like a hundred X difference in terms of different GPU providers for how reliable their GPUs were, which I don't think it's definitely, there's probably also always going to be a bit of a better provider. But it's definitely got to turn more on a commodity that can actually just run on this compute for tolerances being handled and hardware is going to improve as well.

• 1:49:07Nathan Labenz?

  When you do something like the 10 billion parameter model that you have trained in a distributed way, how much of your time is going toward managing that mess and making sure that it works with heterogeneous hardware and, you know, things coming online and offline? And like, do you have like an SLA sort of agreement with your providers that, you know, they're not just going to yank things out from you? Or are you giving them the freedom to say, no, well, you know, these are preemptable nodes? Like, we'll take what we can get as we can get it. How much of, you know, so yeah, a little more there? And how much of your time and how much of the return comes from that kind of improvement versus like the sort of de loco, like more theoretical information management kind of improvement?

• 1:49:52Speaker 02

  Yeah, I think for the initial like intellect one run, it was a lot of improvements more on the hardcore engineering type work to actually get that done, right? To get photon to the framework as well as then covering all the other edge cases you have. Having stuff like that you can have like a life recovery where somebody can drop off, but they could also join again or yeah, just join in the middle of training one, right? Which is definitely not supported in any of the other training frameworks right now. So our focus was was definitely more on the engineering side. And to be full of Sean Spur, there was still at the beginning, a lot of things going wrong. So where we basically had to like fix stuff on the fly where a lot of the edge cases we haven't thought about, right? Which can actually go wrong in such a globally distributed setting had to be fixed on the fly.

• 1:50:37Speaker 02

  And the training stopped for a couple of hours and then we continued with all the notes and later on. So that's basically how we've been handling for the intellect one. And I think at the end of the training one, we we had a pretty stable solution in a sense, right? That a lot of the components are fixed. I'm sure once we started like two, there's still going to be some of the issues where training ones have to be restarted because of edge cases. So you can never be a hundred percent sure, but yeah, that's how we've been handling it.

• 1:51:03Nathan Labenz?

  That narrative strikes me as a decent metaphor for the AI phenomenon as a whole. It's like a lot of edge cases we haven't anticipated and we may have some crashes and hopefully we'll recover and hopefully we'll get to a stable equilibrium at some point. I guess, let me try to summarize where where are we on distributed training? There is a mess of hardware management, which it sounds like you've come quite a long way toward taming, but maybe not fully tamed yet. There's also just the fundamentals of information management and the challenges of bandwidth. It seems like the low hanging fruit has been picked where it was like, actually, you could just do this in a less bandwidth intensive way and it still works.

• 1:51:47Nathan Labenz?

  And now we're getting into like things where there's probably more tangible trade offs where, yeah, you could do this, but then it's not going to converge as fast. Or we're not necessarily competitive if we make certain trade offs. It seems like we're, and you know, in terms of scale, you've got the 10 billion, it seems like you can see a path to 100 billion. But a path to a trillion maybe is not yet clear. Is that decent summary?

• 1:52:11Speaker 00

  I think there's still a path to almost like AGI and even like in this paradigm that almost like goes beyond purely just maxing out parameter counter like where I think even the R1 shows us, right? It's like scaling that paradigm might get us to AGI and it might not require a trillion of active tokens or parameters. And I think that for me quite plausible path where it's like there's other pieces of the puzzle that are quite relevant to solve. And I think we're largely on track to solve decentralized training. It will always have like some trade-offs, but that come also with huge benefits, right? It's like because you can train across the whole world, you're not relying on like drawing

• 1:52:53Speaker 00

  absurd amounts of energy in one place and having like an absurdly large cluster in one place, which you can see play out in real time, right? Like with all the big labs, which is also why they're all extremely interested. Like a bunch of them reach out to us, right? It's like because it's the key thing that they are trying to solve, right? Like this weird training. So I think in that sense, like it's actually something which even the big labs and everyone basically is tackling, like it's not just all this fringe, like decentralized AI, decentralized training ecosystem. It's OpenAI, it's Gemini, it's all of them, right? And I think that's something which people haven't fully appreciated yet about this weird training. Obviously, as Janos mentioned earlier, I have like a slightly different

• 1:53:35Speaker 00

  paradigm, right? Like they in some cases have very fast interconnect between their clusters. But it is surprising. It's like even if you look at OpenAI, it's like, it's not like they're next to each other, right? It's like the plan is like they're actually spread throughout the US, kind of like the data center built out like that they plan.

• 1:53:50Nathan Labenz?

  You need decentralized training to win over Congress to fund the massive build out because they all need a little, little bit in each of their respective states and districts.

• 1:53:59Speaker 00

  Yes, and I think on the other side, this also unlocks in just like global communication, right? It's like having fiber like under the oceans, right? Like connecting the continents, like having starlink, like all three things like opens up theoretically even like compute being in the middle of the ocean, right? It's like if you suddenly can communicate and it's like we had people doing exactly this, like reach out to us because the cost of energy is so much lower if you have like wave energy in the ocean. It's like Suddenly, basically, like a different, almost like even infrastructure paradigm opens up where you don't need to compute necessarily in like the different hubs, but you can have them in extremely rural areas as well. And I think that future is playing out.

• 1:54:39Speaker 00

  And I think the question is probably how fast that infra build out will continue, right? It's like it's at a trillions per dollar is like now. I think it's probably gonna get like to the tens of trillions. But Microsoft alone is spending eighty billion a year like on data center build out. Oh, may I obviously now like hundred billion plus. And I think there's a lot of others that like are under reported, right? It's like the governments have gigantic infrastructure build out that are very under reported, like including US government. So I think there's much more compute already being built than even people know like about, like and then it's reported. And there's way more being built. Yeah, that would be hard to even like fully have a overview of, right? It's like deep sea is the best example, right?

• 1:55:23Speaker 00

  It's like there was like yesterday, like a saying by like scale founder that like supposedly they have like fifty thousand plus eight hundred, even though they under report the number because they have export restrictions, right? So there's all these things in place where it's like, good luck getting the actual number of what the defense department is right now building up on the GBU side. I know it's like in the six figure like hundreds plus, like it's probably in the seven figures already. It's like gigantic clusters, like that no one know, like they are very unknown. And I think that's playing out for China, it's playing out for like a lot of different places where like basically there will be specific spots where there will be also an explosion just in in compute, which we'll tap into, right? A good example was like TikTok was one of the biggest providers of compute capacity to a lot of these different platforms and ecosystems.

• 1:56:07Speaker 00

  Because they bought so many GUs, like I think over six hundred thousand. And at times they didn't need them, right? And they gave them to the market, basically. And I think those things will continue to play out as basically as like every corporation and every nation state starts ramping up their like data center build outs.

• 1:56:26Nathan Labenz?

  It's gonna be quite something if sea steadying becomes a thing because people wanna compute in the open ocean for like wave energy, you know, hey, we may be headed to the cyberpunk future.

• 1:56:36Speaker 00

  Yeah, I think more what is like the energy there is already so much cheaper like order of magnitude that it will kind of like happen even from that angle probably to some extent. Yeah, so I think it would like a computer will be to an extent everywhere where it's like allowed and has like geographic advantages, right, like where it can tap into cheap energy, where it's like easy to maintain.

• 1:56:57Nathan Labenz?

  Yeah, with Starlink, you could even have a decent, you know, it sounds like you could have a little floating compute. You talk about islands of compute. I mean, you're you could be really talking true genuine islands in the, you know, middle of the Pacific of compute. That would be quite something to contemplate. What does this mean for like Nvidia? I mean, this is not a stock show by any means, but like I looked at Nvidia, which of course has blown up and as of today, $3.6 trillion market cap. And then just for one comp, I looked at AMD, which is actually down a third over the last year or so, and it's a $200 billion market cap right now. So you've got an 18x ratio of Nvidia to AMD.

• 1:57:38Nathan Labenz?

  I wonder what you guys think of like Nvidia's prospects for continued dominance of this market? I mean, a sort of naive read of everything that we've talked about would be, you know, the sort of abstraction is coming. And if abstraction comes, you know, then that would seem to be almost synonymous with not the kind of moats or profit margins that Nvidia enjoys today.

• 1:58:01Speaker 00

  I think like two or three things can be true at the same time, right? I think one is that this market is exploding. So I think it was like non-existent basically five years ago, and now it's like in a trillions and we'll go to tens, probably hundreds of trillions. I think it's hard to fathom how gigantic this market will be in a sense where it's like everyone in the stack is poised to be like one of the most valuable corporations in history, right? It's like beer, then media, ASML, TSMC, AMD and everyone else. But I think on the other side, it's like, I think it's also very profitable, right? Like the kind of the monopoly really that Nvidia holds right now, right? It's like they make a hundred, like over 100% of the profits, right? Uh uh in AI industry right now because everyone is burning through money, right?

• 1:58:42Speaker 00

  Like while they stack money, I think their margins and their like basically having like ma- ma- majority of like AI profits flow to them. I think that will probably over time shift slightly way, right? It's like their margins might compress while their revenue is like ten to hundred X, right? So it's like it they they'll probably still be a bigger business, right? Especially in kind of like the optimistic scenarios. But like the thing is, I think AMD hasn't been the strongest competition, but they're obviously the clear number two, right? It's like they think founded famously on like a lot of different fronts, including on software and also not really building up the ecosystem that I think India built up. I think my other like almost like read of it is that at some point like India is such

• 1:59:24Speaker 00

  a big free cash flow that they can like reinvest them into like the next generation of chips and and ultimately extend the lead. I think to be honest, the strongest competition, which is also fumbling it in like a different dimension is Google with TBUs, which obviously also like has a very like close to their chest internally. Obviously, the other big techs like the Amazons and Apples, they won't start adopting uh Google TBUs, right? But they're also doing their own efforts, right? It's like Amazon has their own chips, Apple has their own chips. So I wouldn't actually underrate almost like the chip efforts from the big tech giants, like more in terms that they can force adoption and everything else. I think basically like AMD will also go from close to zero percent market share to slightly higher, but like probably it won't uh overtake India anytime soon. And I think there will be, there's other more specialized chip providers, right?

• 2:00:09Speaker 00

  There's specialized ASICs just like made for transformers, like the Grox and Edges of the world. I think they also are basically at close to zero market share right now, but they'll probably gain some market share, similar how like a lot of the other chip providers will. So I think over time, like zooming fast forward, I think often will be probably as a class of companies be worth more, like the margins will be compressed, but the the revenues will be way larger. And I think there's hopefully basically more competition to your point on like the software stack, which we plan to compete to, right? Like to help commoditize those efforts in a sense where where basically the margins ultimately, I think in a perfectly efficient economy, right? It's like get compressed to a kind of cost of capital with like some risk premium, right?

• 2:00:53Speaker 00

  And I think that's hopefully even with like superintelligence in long run where we had it, right? So they're actually over time, like their margins is opportunity of superintelligence to compress them with like cheap alternatives, right? Over time, so I think. That's like, I think the high level read of it is that they were able to stack those modes, right, from hardware to software to the interconnects to like vendor relations and all of these other things, which is like not easy to break, at least right now, for like, I would say, pretty weak competition from like AMD and even weaker from Intel, right? Like, they're not even in the race at all.

• 2:01:28Nathan Labenz?

  Yeah, Elon might have something to say about that, I guess, from what I hear recently, but time will tell. Are you familiar with Imad's Intelligent Internet project? I did an episode with him not too long ago. Of all the other things I've seen in the world, I would say you guys like vibe match most with him. Are you in touch? Or are there other things?

• 2:01:49Speaker 00

  We're quite close with him. I think still in my chats, like, I can't fully wrap my head around yet what exactly he's planning to do, but it's, I think it's also evolving, and it hasn't fully launched yet. But I think maybe some differences, right? And then some similarities is we're focusing a lot on P2P compute and intelligence as foundational pieces of the puzzle. So I think that that's like a difference. I think the others, as far as I know, ultimately, we put a big focus on decentralized training, right, and fixing and making that work. He seems more interested in like slightly different paths or like kind of orthogonal to it. So basically, I think it shares maybe like a, almost like broader vision, but I think the way how we get there is like slightly different for

• 2:02:34Speaker 00

  us. But then also, I think we really tried, obviously, compared to not just him, but like a lot of the efforts, just like ship in like small increments, like as quickly as possible, like different pieces of the puzzle and open them up to the open source community. So I think he'll do the same probably soon, like looking at his work before with stability. So I think he definitely knows and has a track record of building relevant open source models, right? Which isn't the case for others and maybe entering decentralized training or decentralized AI. So like, I do think he'll be able to create some cool open source AI models and create an interesting community, which we obviously did quite well with stability in many ways. So I think that's probably like superpower that he'll continue to play into. And yeah, we're

• 2:03:16Speaker 00

  Quite collaborative with him and in touch, but I think it's like the same case with a bunch of other efforts in the space. Where we would just try to figure out how they can support each other. It's like the different use cases where people can. Build agents, they can build their own models on top of us. Or Emars network can like needs compute and like it can draw it from our network. And I think that's kind of the future that I'm envisioning is that similar to what we've seen with something like Ethereum where it's really the big power and why it ultimately I think won in many dimensions. Is that you have to basically like co-ownership, right? Of the whole network and protocol. It's fully open source. You have thousands of people like collaborating, having strong incentives to make it successful instead of just like.

• 2:03:58Speaker 00

  Five or 10 team members and a few shareholders. Like it's basically a paradigm where ultimately. It's it's kind of almost like the open and decentralized community competing against the closed source. Labs, I think, and that union basically of players is growing, including some of the foundational open source pieces like PyTorch and and the Llama community that we are very open. And in close touch with. So I think all of those things can stack on each other and build on each other. So like he can leverage our progress on decentralized training and same for everyone else and and the other way around. So I think that's really, I think the. Basically it's the strong tailwind that space has. It's like we can build on our own like. And those things. And I think.

• 2:04:41Speaker 00

  That's really how the most progress will happen in like a very short period of time and how I think open source is now like officially right, like catched up with closed source like since our one, like it closed the gap, right? It's like most closed source companies like the X A Is and then topics are like now officially behind like open source. And and I think that's like an update that people that that shifts people's perception. Like I think a lot of people. Didn't thought it was possible like even a year ago or or two years ago. And now it's kind of like definitively proven that like. Open say I lead like melted towards zero, right? It's like the one or two year lead is now like gone.

• 2:05:17Nathan Labenz?

  Certainly down to months, not years. I would still say, I think, I mean, it is tricky because there I have had really good experiences with DeepSeek. It's only been a few days. A lot of people seem to be very enthused about it. I still think if you could only give me one, I would take a one over R1. And certainly there's, you know, we know O3 is coming too. So there is something there. I would also bet that internally at DeepMind and Anthropic, there is something, you know, at least on the R1 level, even if we haven't necessarily seen it. We have seen Gemini flash thinking and Dario's recent comments about what he's seen internally. And Anthropic certainly suggests they've got something cooking that's moving that direction. In terms of like the long-term vision or sort of structure of your company, in a way, kind of going back to the beginning,

• 2:06:02Nathan Labenz?

  you guys have all these compute partners that contribute to the marketplace, but then they also can contribute to these decentralized training runs that you're doing. Are they just contributing in kind? And like, what is the governance model? Or, you know, I'm a little bit fuzzy on how, what's in it for them? What is the sort of incentive design that you've done to bring people together and create sustainability on that level?

• 2:06:26Speaker 00

  I think there's different phases to it, right? And I think up till now, like everyone contributed in kind. And ultimately it was like more like friends and sponsors. So I'd like basically like the folks like Hagen Phase or Dylan Patel from Seminarless. But also just like a bunch of like in the internet, right? Like this model and jointed. I think basically almost like again drawing the analogy to something like Ethereum. Really the goal of this is to create like a public utility that like anyone that contributes to it, I can have a piece in and can direct it. And I think we're basically in a process to like finalize the details and work this out. But like it will have like a nonprofit foundation structure that governs kind of basically almost like this provisioning of this public utility.

• 2:07:11Speaker 00

  And I think it's in many ways very counter to just like pure like startup corporations. It just in structure, right? Like it's basically setting up to provision like a utility in a very efficient way, but also in a very open and decentralized way that is permissionless. So it's to your point, it's like a Vitale can't stop Ethereum, right? If you wanted to, and I think that's kind of like by design and ultimately they're not the only one contributing to it. There's hundreds of teams contributing to something like this. Infrastructure, like everyone using it, everyone like making pull requests to it. And I think that's basically what we're setting up for. And I think that then basically the structure is like that we as a company almost are one of the contributors to that.

• 2:07:55Speaker 00

  Like public infrastructure and utility. But I think it's very, very different from like a traditional corporation in that sense.

• 2:08:05Nathan Labenz?

  So do you envision like a, a token or a currency of some sort where I contribute compute? And then I get some sort of claim on future governance that I could potentially even resell?

• 2:08:19Speaker 00

  The, the broader goal right is to like create like an alternative system and real like in that sense, like, like a theorem that is like fully permissionless, fully like tokenized and everything. But like, so yeah, in the, in this direction, even though like it's very like, we're very light and can't also come to much on like a concrete timeline and everything. Basically, it's a broader question and discussion where it's like almost how economies, right? And, and even currencies reshape in kind of like this like super intelligence age. And I do think the default outcome, right? It's like almost a continuation of like fear dying, like to an extent or being like at least overly inflated with like UBI like measures, right?

• 2:09:02Speaker 00

  Which can potentially be funded by the windfall from super intelligence, right? Like, but still means that ultimately people want to flee into like less inflating harder assets. And I do think like a currency that is backed by compute is way harder than a currency, like fear currency that's backed by nothing. And also then like a cryptocurrency that is backed by like, like even like something like Bitcoin. If you really look at it, like it's not that hard of an asset and currency, right? Like you don't get much utility out of it. So I think. It's kind of like a bit how we're thinking about it. It's like in the intelligence age almost, you want to own a piece of a super intelligent system that is able to generate value, where you have actually had like access like through your ownership in it to the compute,

• 2:09:46Speaker 00

  to the intelligence. And how that exactly can be built, right? Like and how we are approaching it, we'll like share more about soon. But ultimately, I think it basically needs to happen in stages where it's ultimately really guarantees also that like anyone like can get involved and have has a piece of it. But ultimately, where it's also truly permissionless, right? And like in the spirit, almost like of the early like cyberpunk internet and not like of the late stage like platform monopoly, big techs. And I think that that's really the broader spirit is like making sure that, you know, we lost kind of like the internet to to big tech platform monopolies. I want to make sure that we don't lose like super intelligence to them again. I think it's kind of like the default outcome if we don't kind of like veer off in a in an alternative path.

• 2:10:32Nathan Labenz?

  I think you just brought us to a perfect ending point with that comment.

• 2:10:38Nathan Labenz?

  Maybe one little extra bonus would be, what are you guys looking for? You know, that could be hiring on your team. I know you got a jobs page with a number of listings. It could be compute contributors or anything else. But what sort of bat signal do you want to put out into the world?

• 2:10:54Speaker 00

  Yeah, I appreciate it. Like, I think all the ones you mentioned, like basically, we're actively hiring across like a lot of different roles, like a lot of on the AI research side, on just like general developer side, but even on things like marketing, design and every other front. So like for anyone that is interested to reach out, feel free to do so. I think similarly, like with the model collaboration we did, which we didn't touch on quite actively collaborating with like dozens or hundreds of like different open source AI researchers, like leading scientific institutions, leading like AI labs at universities. So basically, we're quite supportive of basically supporting the highest impact initiatives, basically with compute, but also with hands-on support to actually build and scale their models.

• 2:11:37Speaker 00

  So that's another area. And then, yeah, in general, always happy to connect with anyone interested to like go deep on this and connect or collaborate with us. And there's a lot of surface areas for collaboration, as you also mentioned, for people that have either compute and want to contribute to the models that they are or the agents that they're excited about. Like, that's kind of like really what we're planning to enable.

• 2:11:58Nathan Labenz?

  This has been excellent. Vince Visser and Johannes Hagemann, founders of Prime Intellect, thank you both for being part of the cognitive revolution. It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please, don't hesitate to reach out via email at [email protected], or you can DM me on the social media platform of your choice.