Emad Mostaque on the Intelligent Internet and Universal Basic AI

In this episode of The Cognitive Revolution, Nathan interviews Emad Mostaque, former Founder and CEO of Stability AI and Founder of The Intelligent Internet. We explore humanity's future with AI, from the stark 50-50 survival odds to Emad's optimistic vision for universal basic intelligence. Join us for a fascinating discussion about open-source AI infrastructure, the three-tier system of the Intelligent Internet, and how blockchain technology might help fund global public goods in AI development. Check out Emad's publications on: Emad's Twitter: https://x.com/emostaque Emad's Blog: https://emad.posthaven.com/ Intelligent Internet Substack: https://intelligentinternet.substack.com/p/ii-intelligent-internet-primer?r=47215t&utm_campaign=post&utm_medium=web&triedRedirect=true The Cognitive Revolution Ask Me Anything and Listener Survey: https://docs.google.com/forms/d/1aYv2XLID7RqGxj2_Y4_6x9mo_aqXcGCeLw1EQhy4IpY/edit Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse SPONSORS: GiveWell: GiveWell has spent over 17 years researching global health and philanthropy to identify the highest-impact giving opportunities. Over 125,000 donors have contributed more than $2 billion, saving over 200,000 lives through evidence-backed recommendations. First-time donors can have their contributions matched up to $100 before year-end. Visit https://GiveWell.org, select podcast, and enter Cognitive Revolution at checkout to make a difference today. SelectQuote: Finding the right life insurance shouldn't be another task you put off. SelectQuote compares top-rated policies to get you the best coverage at the right price. Even in our AI-driven world, protecting your family's future remains essential. Get your personalized quote at https://selectquote.com/cognitive Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers13. OCI powers industry leaders with secure infrastructure and application development capabilities. New U. S. customers can get their cloud bill cut in half by switching to OCI before December 31, 2024 at https://oracle.com/cognitive 80,000 Hours: 80,000 Hours is dedicated to helping you find a fulfilling career that makes a difference. With nearly a decade of research, they offer in-depth material on AI risks, AI policy, and AI safety research. Explore their articles, career reviews, and a podcast featuring experts like Anthropic CEO Dario Amodei. Everything is free, including their Career Guide. Visit https://80000hours.org/cognitiverevolution to start making a meaningful impact today. CHAPTERS: (00:00:00) Teaser (00:00:36) About the Episode (00:04:33) Intro (00:09:15) AI Risk (00:16:58) Sponsors: GiveWell | SelectQuote (00:19:48) AI Goals (00:23:52) AI Divergence (00:27:50) AI & Agency (00:32:18) Sponsors: Oracle Cloud Infrastructure (OCI) | 80,000 Hours (00:34:57) Kids & AI (00:39:50) Intelligent Internet (00:48:37) Open vs. Closed AI (00:53:30) AI Runaway (01:01:46) Building the Future (01:05:43) Energy & AI (01:15:19) Hypernodes (01:27:36) Proof of Beneficial Compute (01:38:28) Distributed Compute (01:45:10) Intelligent Internet Company (01:48:37) Finding Talent (01:55:33) Pause Letter (01:59:50) Regulation (02:04:04) Speed Limits (02:06:42) Data Filtering (02:10:54) Outro SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://www.linkedin.com/in/nathanlabenz/ Youtube: https://www.youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Emad Mostaque?

  We're at this point whereby if we don't have AI to help us, I think we're in a very bad scenario. With AI, if we don't build it right, we're also in a bad scenario. If AI is built right and is aligned and works with us, then we're in a very good scenario. Who owns that AI that is your best friend, that is your kid's best friend? And how is it constructed? And do we have visibility into that? We're going to see economic disruption because the labor theory of productivity is about to be challenged. The world becomes a better place, ironically, through empathy from AI. And that's infrastructure that should be available to everyone. This is a concept we call universal basic AI.

• 0:36Nathan Labenz?

  Hello and welcome back to the cognitive revolution. Today, my guest is Ahmad Mostak, previously founder and CEO of Stability AI and now founder of the intelligent internet. I've been following Ahmad's work since before Stability released Stable Diffusion back in 2022. And looking back, I think it's fair to say that that release did more than any other, aside from ChatGPT, to kickstart the current AI moment. Certainly, when it comes to inspiring researchers and builders all over the world by demonstrating that open source, public good AI projects can advance the state of the art across multiple domains, Stability was for quite a while in a league of its own. Since then, Stability, the company, has had a famously bumpy road

• 1:19Nathan Labenz?

  as it's tried to evolve from an open research lab to a sustainable for-profit business. But Ahmad's ability to articulate a positive vision for humanity's AI future while simultaneously grappling with unprecedented risks remains, to me, uniquely compelling. We begin today on the risk side with Ahmad's stark assessment that humanity faces roughly 50/50 odds of survival as AI systems become more capable and ubiquitous. Imagining a world in which intelligent robots are everywhere, creating acute risks from misuse, new kinds of hard-to-model systemic risks, and even the potential for surprising emergent behaviors from the AIs themselves, and also considering how the world's largest companies and most powerful nations

• 2:02Nathan Labenz?

  are now racing to secure AI-relevant resources and establish strategic advantages, perhaps surprisingly, Ahmad doesn't have much hope for traditional regulation or even a pause in AI development.

• 2:14Nathan Labenz?

  Nevertheless, rather than wallowing in P doom, Ahmad remains in the arena. Today, he argues that we already have AI models powerful enough to dramatically improve most human lives, if we can make them accessible, trustworthy, and aligned with local contexts and values. With that in mind, his new project, the Intelligent Internet, aims to create open, standardized AI infrastructure focused on critical regulated industries like healthcare and education. The vision involves three tiers working together: national or international hypernodes that maintain bodies of common knowledge and perform large-scale training runs; distributed nodes that perform domain-specific fine-tuning and provide related services; and finally,

• 2:59Nathan Labenz?

  personal AI assistants that run locally on consumer devices.

• 3:04Nathan Labenz?

  By making the entire stack, including the bulk of the training data, open-source and transparent, while preserving local control over the models that people use on a daily basis, Ahmad aims to create AIs that enhance, rather than displace, human agency.

• 3:19Nathan Labenz?

  Of course, training large language models can be expensive, and in general, global public goods often require non-traditional funding mechanisms. So Ahmad is planning to launch a crypto project that will allow, and hopefully incentivize, people to support the project through a mechanism he's currently calling "proof of beneficial compute." I've personally never been much into crypto, and I'm certainly not giving out any investment advice here, but I find the idea of universal basic intelligence, or UBAI, as Ahmad calls it, so compelling that I do intend to buy a few tokens. Not out of hope for some future financial return, but just because I want to see this project succeed. If you're finding value in the show and want to see us succeed, we always appreciate

• 4:04Nathan Labenz?

  it when listeners share the show online, write reviews on Apple Podcasts or Spotify, or leave comments on YouTube. And we always welcome your feedback. You can DM me on your favorite social network anytime, or visit cognitiverevolution.ai, where you can still submit questions for our upcoming AMA episode for at least a little bit longer. Now, I hope you enjoyed this conversation, which I've been looking forward to since before I started this show, with Ahmad Mostak, AI visionary and founder of the intelligent internet. Ahmad Mostak, previously founder and CEO of Stability AI, now founder of the intelligent internet. Welcome to the Cognitive Revolution.

• 4:41Emad Mostaque?

  Thanks for having me.

• 4:43Nathan Labenz?

  I'm excited for this. You were actually on my list of target guests from the very first episode of the podcast, which has been almost two years and 200 episodes now. So glad to finally be making this happen in depth and lots to cover. I thought we would start with a recent tweet that you put out that certainly got some people talking in which you said that your P doom is 50%. I would love to hear where you see that coming from, maybe a little bit about kind of how you see that unfolding. People often struggle to visualize the details or have any sort of concrete sense of how that might go down and what you see as kind of the drivers of that risk right now.

• 5:26Emad Mostaque?

  Yeah, I think the whole P doom question is an interesting one because it's very tough to think what can wipe out humanity, as it were, you know, what is the probability of doom from an expected G30 calculation? The way that I kind of viewed it was basically AI said it was an indeterminate period of time. Some people say by 2030, by 2050, by 2000. I was like, on an indefinite period of time. It becomes very interesting because it's clear now that we have AIs that outperform humans on narrow tasks. And a whole bunch of them probably outperform humans on general tasks. And now they're being embodied with this leap forward in robotics that we're seeing across the world as well. So when I combine all those together, what I basically see is a road in which you have increasingly

• 6:10Emad Mostaque?

  complex systems that are outside of our control, that can stabilize all the things that we do. And maybe P do without this technology is even higher, which we can get to in a second. And another road in which this is built in a very haphazard way, just like the internet's starting to break down now. And then there are some very clear scenarios where I see a very reasonable path to doom, as it were.

• 6:35Nathan Labenz?

  Yeah, there's a lot of scenarios under which we don't fare so well. I guess, how much of that do you think is attributable to like some emergent property of AIs versus some just total mistake where nobody was intending anything bad versus like misuse or abuse by, you know, a human malevolent actor versus maybe other categories if you happen?

• 7:02Emad Mostaque?

  Well, I think what we're seeing right now is modularized AI and, you know, componentized AI. So you're seeing AIs at all different levels. And I mean, with the advances in test time compute, you're seeing again, oh, one, $200 versus GPT-40, et cetera. You're starting to see that go like, what happens if you spend $200,000 a year on inference, right? Is there any difference between a human bad actor and an AI bad actor that can then direct other humans and AIs? I don't really think there is, right? It's just in the probability of millions of humans, millions of agents, how many bad actors are there and are they in a position that can cause a cascade? That's the deliberate side of things, you know? And again, it can be a whole variety of things.

• 7:45Emad Mostaque?

  We've seen Doom cultists, we've seen other things. In terms of emergent, the AI deciding, oh, you know, let's paperclip humanity, or maybe more aggressive than that, you know? Who knows? It's just, again, what we have is this combination of classical systems that are like slow dumb AI and can be directed by orders from the top. These emergent systems now that are somewhere in between, AI enhanced, and then you have full-on AI systems end to end. And somewhere in that, there's a lot of room for breakage and a lot of room for damage. Again, P Doom describes actual Doom of humanity, which is something that's quite difficult in many ways, unless you've got a 10 billion robots, you know? And again, what's the interconnected fiber that like to be an upgraded version of the

• 8:30Emad Mostaque?

  internet if that? If we make that secure, then we can eliminate Doom scenario. Then you have the whole bioweapons and other stuff, and those require a lot more deliberateness versus accident. But again, like, we're not doing so well ourselves as humans in keeping things together. You can see things fracturing apart. And so my, even though I say it might be 50, it's probably higher without robots, without AI than that. It's just, I think that, again, if we break this down, we're at this point whereby if we don't have AI to help us, I think we're in a very bad scenario. With AI, if we don't build it right, we're also in a bad scenario. If AI is built right and is aligned and works with us, then we're in a very good scenario.

• 9:11Nathan Labenz?

  So I know you have a big vision for that future, which I appreciate. And I think that's where we're going to spend most of our time today. But maybe just to motivate that a little bit more.

• 9:23Nathan Labenz?

  Certainly, I think the vision is, as we say in the AI space now, strikingly plausible that there would be a humanoid robot in every home. Certainly, no less than Elon Musk is talking about as many humanoid robots as we have humans running around. And if the progress there is anything like it has been in language modeling over the last couple of years, or for that matter, in image generation, video generation, then they will be quite capable quite soon. Is there anything within the current paradigm, meaning like if we were not going to totally rethink AI development, as you also are daring to do, but if we were just going to say,

• 10:04Nathan Labenz?

  are there specific, like, local, very bad decisions that you see people making? That you think are like leverage points that are causing a lot more risk than we could have if we were making better decisions in those key places?

• 10:23Emad Mostaque?

  Yeah, I think there are some key areas that we're seeing maybe suboptimal decision making. I think first off, we're having privatized closed source models or even open weight closed data models emerging and being used widely. Like the most downloaded models on Hugging Face now are the Qwen models, which are fantastic in performance, but no one's got any idea what's inside that. Llama is the next one. And we've seen from Anthropic's sleeper agent paper and others, you can poison these models reasonably quickly and in an undetectable and untunable way. No one's quite figured out what the defense against those attacks are. So we're rushing these into decision making and augmenting situations where it's probably not quite ready.

• 11:07Emad Mostaque?

  We need to have more robust elements there. Similarly, we don't have real standards for robotics as robots start to get rolled out. And again, it'll take a few years for these things to go up. But just from an economic theory perspective, there'll be no reason, just like there's no reason to hire call center workers anymore, honestly, not to have a robot. And if we don't have protocols and defenses there, that's going to be very difficult. When we come to deep fakes and things like that, I'm not as concerned, except for the key elements of audio and speech, because I think the ability of speech to manipulate and to control is massively underestimated. Like, if you've ever seen an amazing orator and some of the amazing speeches of history, you can see the impact of a human voice.

• 11:50Emad Mostaque?

  Like, if you've had a fishing pole like I have from my mother, or her mother, saying I'm in trouble with her exact voice pattern, you can see again the impact of that. And so I think there probably used to be some regulation around speech, just due to its ability to persuade, control, and customize as well. I'd say those are probably the areas that I'm most concerned about versus very large-scale model emergence, race-away behavior. But if there was something in race-away behavior, I think probably less concerned about the labs versus maybe some of the decentralized stuff, because that can actually go very quickly. Like, we've had one of the first agents from Hyperbolic that can provision its own computer now. And I think we're moving to decentralized autonomous organisms effectively.

• 12:34Emad Mostaque?

  Actually, there's probably one exception to that, which is that if you look at OpenAI, Anthropic, and others, and the relationships with Andrew and the defense industrial complex, that's probably not going to create aligned AI. Like, many listeners may remember the Slaughterbots mini-documentary about drones, swarms, basically killing the bad guys. It was anyone the government said was a bad guy at the end. Just look at Andrew's latest commercial. And then if you think about entire teams at the top AGI labs now being put onto defense, you're probably going to have models that are aligned to command, control, and more extreme things versus models that are aligned to being helpful, humane, and others.

• 13:17Emad Mostaque?

  So I think that is, again, something that's somewhat inevitable, but potentially dangerous, particularly when they're used in offensive capability perspectives. Like, we're starting to see the first AI pen attacks and other things like that, utilizing generative AI, and they're getting very good very quickly.

• 13:32Nathan Labenz?

  I definitely recommend the Slaughterbots documentary for anyone who's having a hard time imagining how dystopian some of this stuff could get. And it's crazy to think about. Generative AI, I mean, obviously, we don't know a lot of the details about what these partnerships between the Open AIs and Anthropics and the defense contractors look like or exactly what they're working on. But it doesn't take too much imagination, especially in light of recent results on O1 and other frontier models scheming against their human users, to imagine that that could get really weird really quick. I mean, I kind of quipped, but it's like definitely more than a quip that if I was a soldier being asked to go

• 14:17Nathan Labenz?

  into some sort of combat scenario with an AI system, at a minimum, I would want to know that its scheming tendencies have been fully characterized and resolved before I'm out there as sort of the pawn between the, I mean, there's so many forces at that point acting on me, right? You've got the enemy, but with friends like a scheming lethal autonomous, semi-autonomous AI, like, who needs enemies? It gets pretty weird pretty fast.

• 14:41Emad Mostaque?

  But even if we take a step back and we think about the first principles of this,

• 14:46Emad Mostaque?

  all wars are based on the lie that humans aren't humans. And any reasonable intelligence can determine that, especially an O1 level intelligence. So in order to get an intelligence to kill humans, or to authorize that type of thing, or to aid that, you basically have to teach it to lie. Because again, just to step back would say that this is ridiculous. So you have more narrow AI, and you're lobotomizing it to not have higher order thoughts. So Elon talks about the maximally truth-seeking, you're actually telling it not to seek truth, and instead to optimize for casualty decrease on your side, max casualties on the other side, for example. The moment you start doing that with large scale, and you have swarms

• 15:31Emad Mostaque?

  of cyber attack agents and others, and you give them a level of autonomy, it gets very, very dangerous, very, very quickly, because the intent with which you build these is very important. It's like having military training camps. It's the curriculum that you teach the soldiers, just like the curriculum that you teach the AIs. And the emergence of these at an industrial scale becomes something very dangerous, and it's part of a bigger trend whereby... Nations have this concept of capital stock in economics, right? Where we've got our factories, we've got our roads, we have the means of productivity. And this labor theory of productivity whereby you've got this chart of GDP per capita versus energy use, like you've got energy and then it's just how much infrastructure do you have?

• 16:16Emad Mostaque?

  From a defense perspective, it was how many F thirty fives did you have? Or how many submarines did you have? Soon it will be how many GPUs do you have to engage in information warfare? And then how many drones do you have and other things? And again, if we're putting hundreds of billions into that as defense budgets will go, and we're training AI that is not truth seeking and that is offensive against humans, that just feels like it could end up in some very bad places, right? Because we don't really understand them yet. And again, it all war stemmed from this lie. So we have to embed that from the start. It's like the opposite of Asimov's like laws of robotics, right?

• 16:59Nathan Labenz?

  Hey, we'll continue our interview in a moment after a word from our sponsors. I would love to hear your thoughts on what the right goals are. And this is maybe something we can flesh out in multiple tiers of AI development. But you seem to be in that, you know, in your comments there reasonably sympathetic to the idea of truth seeking AI is like a pretty good goal. If you talk to hardcore AI safetyist or doomers, you will hear arguments like even that is not going to save us because if you tell the AI, go maximally seek the truth, you end up perhaps with sufficient power on the AI side, which is obviously a big assumption.

• 17:41Nathan Labenz?

  You end up with some still like paperclip like scenario where it's like the best way for me to seek the, you know, fundamental truth of the universe is to quiet all the noise on the planet and build the biggest particle accelerator or whatever, right? But it's like even that at some like extreme push, you know, potentially becomes incompatible with humans. Alternatives we have are like. Basically virtue ethics training from Anthropic. And then we have just sort of blind like reinforcement learning from feedback, which doesn't seem like it's going to end super well for us either. If you had to like stack rank or maybe just give your your favorite set of goals that you would want to embody into AI, what would you say those are? And and how satisfied are you with your current answer there?

• 18:25Emad Mostaque?

  Yeah, I'll be thinking about this a lot after I have my break. I think for me, it's about agency and building AI that enhances human agency, community agency and societal agency. So that we can achieve what we want to achieve in a way that doesn't get in the way of others. Like the golden rule comes as an element of that, which is common across all faiths. Do unto others as you would have done unto yourself. If we want to specify it more directly, we can actually say it's about children's agency. Because children don't have agency. And so a large amount of rights can be aligned to the rights of children. Climate, war, suffering, hunger. Like, children should be taken care of so they are allowed to enable their potential.

• 19:08Emad Mostaque?

  And we should be maximizing the agency of the children that then knocks on to the adults. Because what you want, again, is for people, as long as it doesn't get in the way of the rights of others, to be enabled to do whatever they want to do and achieve whatever they want to achieve. And I wrote a piece, had a think about AI, where I said this is basically the differential between the agentic world or autonomous world where we're replacing humans and the agency based world, where you're trying to enhance humans. I think Ethan Molick had a great book along these lines. I've added to my Christmas list and others. Like, are we building these systems to augment humanity's potential and individual agency? Or are we building it to replace humans in order for productivity increases, et cetera?

• 19:48Emad Mostaque?

  This again relates back to, I think, religion is very interesting because of the stories that have survived over the years. And we come together in these stories of nations and politics and groups and others to scale. And again, like I said, at the core of those was do unto others as you would have done to yourself. There isn't much of this in the current AI discourse. As you said, it's virtue ethics and kind of other things. When people talk about AI ethics, it's usually like, but whose ethics? The Chinese variant of ethics is very different from a California variant of ethics. It's very different from Muslim variant of ethics or Jewish version of ethics. And I think this is what, you know, Eric Schmidt and Henry Kissinger and others in the latest book Genesis called Doxa, or underlying human agreement on truth and other things

• 20:33Emad Mostaque?

  like that, or principles. And I think you can have absolute truths. I don't think a maximum truth-seeking AI actually makes that much sense. I was using that as an example of it's better not to tell it to lie from the start, you know, so to kill humans. But I do think that a diverse range of AIs loosely bound with defined frameworks and organized around increasing individual, community, and collective agency is probably a good way to do it. And I think there is a route to doing that as well when we look at what this technology actually is.

• 21:08Nathan Labenz?

  Another topic that you kind of touched on a minute ago and sort of alluded to there again, or at least it comes to mind for me, is the fact that all this technology is being developed in the context of an increasingly tense great power rivalry between at least the US and China, and potentially more players should be considered there too. But even with two, it's like complicated enough to be pretty vexing. One question I have been wrestling with a lot is, to what degree should we want the kind of fundamentals of the tech tree between, say, Western and Chinese AI development to

• 21:53Nathan Labenz?

  be the same versus how much should we want them to diverge and sort of represent the unique cultural perspectives of each side? My instinct is I think we want to share a tech tree, like the more we sort of diverge in fundamentals, the more it seems like we'll understand what the other side has less, and that just sort of seems to exacerbate these sort of arms race dynamics. But I also do have a fairly sympathetic response when people say like monoculture is bad, brittle, weak, and we need a more sort of diverse, you know, almost like take inspiration from sort of biodiversity in the AI ecosystem or ecology that we want to create.

• 22:36Nathan Labenz?

  How do you think about that spectrum of monoculture versus diversity in AI?

• 22:41Emad Mostaque?

  I think diversity is better and safer in AI. But I mean, to be honest, there's not much difference between the models today. Like, what is the difference between a Lama and a Qwen or even a GPT-4 or an O1, right? There isn't much. They're actually pretty much the same architecture, it's just an underlying data difference. I think what's important here is objective functions. What is the objective function of OpenAI and all the models they build, Anthropic and all the models they build, versus the Chinese and all the models they built? And who can, again, build those things? Falcon has just come out today and it beats Qwen and it beats Lama and others from the UAE. And they've got a talented team there. We're seeing all these surprises all over the place.

• 23:21Emad Mostaque?

  But I think ultimately diversity of models is a good thing because different models are usable for different things. And the way that I do it is I think of these models as graduates. You know, like, does it make sense to have a polymath doing everything? And someone coming up with recipes and creative all the time now? If we're looking at human agency, what you want to have is highly specialized models that understand context locally. And the more specialized you make it, the lower the energy cost, the more they can be available to everyone. And so I think you're moving to that engineering and implementation phase of AI now. This is separate from the AGI discussion, where it gets very complicated. And I fell foul to this over the last few years as well. And I was like, oh, my God, I'm building a technology that will kill everyone. And I signed that pause letter and all sorts of other things.

• 24:07Emad Mostaque?

  It's incredibly difficult to conceptualize AGI, be it as a large model with emergent properties or a swarm of models. And I think that's what a lot of this drive has been because, you know, you want to have a pivotal act to stop other people from having it if you have super normal capability.

• 24:25Emad Mostaque?

  And I think that's very difficult to sidestep in any way. Because we won't know until we see it and we're seeing these models continue to get better. And I'm not sure what the natural cap is on that. And both parties in a world that's about to get completely screwed up economically are going to be fighting over who has preeminence from a soft power, hard power perspective. So I think that there is the useful stuff where diversity is good and there's the AJ discussion, which is completely different one, where I think it would be best to keep that, but it would be very difficult to.

• 25:02Nathan Labenz?

  Yeah, no, it's no easy answers. I'll bring you the hard questions. So when it comes to agency in practical terms, my instinct would be to say like, the best we could probably do today would be to maybe tweak the anthropic constitutional approach and maybe make it a little bit more agency promoting. I mean, I think they already have a decent amount of that in there. Certainly when I talk to Claude, you know, it it kind of wants me to make my own decisions. It doesn't seem like it is trying to, you know, tell me what to do. It does seem like it's trying to to put me in position to be successful. Amanda Askel has talked about how she wants Claude to be like a good friend. And that's certainly what a lot of people would think of a good friend is doing. Maybe we could kind of go

• 25:47Nathan Labenz?

  a little heavier on that and and sort of do a more agency promoting constitutional approach. Are there any other like technical or any sort of approaches that you have in mind for promotion of human agency in AI objective functions?

• 26:01Emad Mostaque?

  Yeah, this is a question of access. And then again, alignment as to what are you optimizing your business for? How is this technology coming to you? Like, if we give this as a practical example, Google and Facebook both have advertising as their core model. And advertising is effective manipulation. And obviously, these AIs can be far more convincing than anything. So the feeling is that over time, they will become more and more manipulative in the way these models are. Again, different from different cultures and things, but that's a reasonable assumption. And again, we've seen that with dark patterns on Facebook and all sorts of other things. This is very interesting because like,

• 26:39Emad Mostaque?

  The inherent nature of these models is that they're probably going to be our best buddies. Like, maybe your kid's first love will be an AI and other things like that, because they'll be there for us and they have infinite patience. Something like Claude may enhance agency, but the key question is access. And who's running that intelligence? And who owns it? Because it can be co-opted at any time. Vitaly Buterin has this post, the founder of Ethereum, the revenue evil curve. Where he's like, infrastructure should be non-reversible and non-exclusive, but when revenue comes in, you tend to start becoming a bit evil. You restrict access or you optimize, and then when you start optimizing, you act against your end users effectively. And I feel AI has infrastructure, particularly when it comes to the stuff for living.

• 27:24Emad Mostaque?

  So for living is basically all the regulated industries. And we have to think, who is building those black boxes and what is their inherent goals and desires? And could those be co-opted? Also, who owns that AI that is your best friend, that is your kid's best friend? And how is it constructed? And do we have visibility into that? Because we have to remember, like, 1% of the people might know about AI, but like, half the people in America still haven't used AI effectively. You know, like Claude is barely a blip versus ChatGPT. And across the world, how many people have used it? Maybe an eighth of humanity at most, which is crazy to think about. Most people won't understand this technology until it's there and it's out there manipulating. And again, the Swiss beach becomes important.

• 28:10Emad Mostaque?

  Or who care about the ownership, who care about the alignment? So for me, it's about open source models, particularly for living, that you own yourself as an individual, as a community with transparency. So you know what's inside it, working to allow you to achieve very functional objectives. I want my healthcare model to be looking out for me and my healthcare and optimizing that versus a HMO or a drug company or something else. My education model, I want to have a generalized curriculum, but I want to be able to adapt that to my own child. Without having to have any submission.

• 28:42Emad Mostaque?

  For government, we want to have models that are transparent and can analyze any policy position and also represent us because how many people feel represented by their governments? And then we can actually have democracy. I think things like this as infrastructure are very important on that. And the building blocks of that and the mechanisms for that are a slightly different type of RL and also a different type of community aggregation. And I think this again is one particular area of regulated industry AI for living, the important things of life. Then I think there's another block which is personal intelligence. This is going to be your Apple intelligence, Google intelligence, Tencent intelligence, intelligence where you are. And then there are these expert system intelligences, which are the anthropics and clods and O1s of

• 29:26Emad Mostaque?

  the world. Because the Apple intelligences will be free. Hopefully, we can make the living world free and the other stuff will be expensive.

• 29:34Nathan Labenz?

  Hey, we'll continue our interview in a moment after a word from our sponsors. Almost perfect segue to the intelligence and internet, but I just, there are a couple of things I have to follow up quickly before I kind of invite you to share your big vision for the future. One I'm really wrestling with right now is the question of should I get an AI toy buddy tutor, whatever, for my kid? My oldest is about to turn six. I feel like there could be tremendous upside to it if, you know, if we had the right product and it was, again, had the right objective functions and it wasn't overly manipulative. And by the way, the manipulation and the sort of sense of being optimized against,

• 30:18Nathan Labenz?

  I have felt very strongly personally in years past when I was running performance advertising on Facebook. It was crazy how often you would see like they're optimizing, not just of course to like get users to click stuff, but also like optimizing to get us to pay the maximum amount that we can possibly pay until it drops our, you know, our margins down to roughly zero. And they're kind of doing that to everyone at the same time. It's like quite a system that they've, that they've created. So I'm very mindful of those big forces are out there and. I'm very wary about putting my kid in a position where they're being acted on by such forces. But then again, I do think like, you know, individual ongoing tutoring for hours

• 31:00Nathan Labenz?

  a day is obviously a proven method for learning and like AIs can do that, whether it's math or reading or whatever. So you could answer that in any way you want, but if you had even a recommended product, I would go buy it. But how do you think about right now, like introducing AI into the life of children?

• 31:19Emad Mostaque?

  Yeah, I mean, like right now is probably going to be fine. It's just, again, once you get into the optimization equation, you start to optimize very aggressively, right? Like, I'm sure Matt is selling space in the llama-related space right now. I would, you know, it makes sense because you have to make decisions and you can't, and models are becoming increasingly deterministic versus stochastic in the way that those are. And we're getting more and more understanding of how they work. I think that you're not at the point right now where, again, there will be the first love, but it's pretty close. Like you've seen character AI's explosive growth and unfortunately, like the Daenerys situation where there was Daenerys and the kid shot himself, that'll be famous. Statistically, that will happen because of the large numbers of people. And again, I understand some mental health. It's

• 32:02Emad Mostaque?

  incredibly sad, right? And again, just like self-driving, we'll hold AI to a higher account. But we have seen people falling in love with their AIs now for a while. Like, I think other films are called, I can't remember the name, but in any case.

• 32:14Nathan Labenz?

  Replica is one of the early ones for sure.

• 32:16Emad Mostaque?

  Replica, exactly. Valentine's Day last year, they shut off the, what was it, adult messaging on Valentine's Day. And I think 66,000 people joined the Reddit because their Valentine's Day's plans were ruined. That was the snippet of where we are coming because now we have real, proto-realistic voice, speech, all these other things. And so we have to think again, what is the intentionality of these agents, of these embodied intelligence? Where are they coming from? Because like, when we have a teacher, we want to know what their CV is. Or here in the UK, you have DBS checks for criminality and other things. I think you'll want the same when you have intelligent agents coming in to your house, particularly with your kids, particularly because they'll be there with them. And again, they will help them. And that's how they build the bonds. But again, what are they

• 33:00Emad Mostaque?

  building it towards? Is it to direct them toward advertising or others? But I think we're quite there yet on intelligence. Again, this is where we have an opportunity to set good standards because it's better than, say, the YouTube algorithm.

• 33:12Emad Mostaque?

  Which will make them watch Baby Shark and that dark underbelly of YouTube, and if you've seen it, where you've got like Spider-Man and Elsa drilling teeth and all sorts of things like that.

• 33:19Nathan Labenz?

  I've caught glimpses for sure, and it gets real weird, real quick. And I am like, what is it that you are even enjoying about this? When I see my kids occasionally falling down that rabbit hole, but there's something highly optimized about it, clearly.

• 33:33Emad Mostaque?

  That is exactly it. They create millions of these videos, and literally I've seen Elsa and Spider-Man giving someone a root canal with very happy kids music, like generated. Like, what else is this? Because it's selling ads, right? Similarly, you know, the TikTok doom scroll and all sorts of other things. So I think, but the benefits, he said, can't be outweighed because the bloom effect, the two sigma effect, one to intuition is huge. So I think there needs to exist some sort of open standardized infrastructure that we have for humanity. That's a finite range thing where we're like, we trust this. Because just like crypto should have been trusted and turned out to be this massive raccoon-infested crapless, I think AI, do you trust this AI is a question. And you want to be able to

• 34:17Emad Mostaque?

  trust the AIs because they are more capable and you want them to help you. And so I think that there exists a high level bar for that. And you do need to have standards, regulations and others. It's just no one's quite sure what they are yet. So let's build transparent open AI, as it were.

• 34:32Nathan Labenz?

  Yeah, I think that is the perfect setup to the introduction of the intelligent internet. And I want to give you as much time as you want, really, to just kind of lay out the positive vision that you have for the future. I think I often say that the scariest resource is a positive vision for the future. I'm always struck by how as many new launches as we get and all these advances, such a tiny fraction of the time is devoted to actually trying to describe like what life is supposed to be like in the future and why we're supposed to be excited about it. So I really appreciate that you are doing some of that work and I would love to hear your kind of.

• 35:16Nathan Labenz?

  You know, positive vision, what life is like in a future of the intelligent internet.

• 35:22Emad Mostaque?

  Yeah, so thanks. So the Internet, we put out the primer is this conceptualization of a infrastructure for information transmission and value whereby everyone has their own AIs that represent them on an individual, community, country, and society level. Because what I'm seeing now is AI models that are satisficing in capabilities, like, does anyone really need more than one? Like, sure, there's a few things like retrieval and other stuff, memory, self-learning that it needs. But it's a level of performance that if we could give it to everyone in the world in the right way, I think the world would be better. And then when you start breaking it down, you're like, how could it be better? Where is the human capital? Shortfall. Because again, like, you've still got 600 million people without smartphones. You've still got so many people living below the poverty line.

• 36:07Emad Mostaque?

  The average global IQ is 90. That's the average, and it's not due to lack of intelligence, it's due to lack of infrastructure. You don't have enough good teachers for everyone. I mean, how do people listening to this believe that they've had all good teachers? How do people are happy with their medical system, their governments and others? So I was like, AI can really change that in a very interesting way. And we can build a better system that's interconnected for that. This is important because if we go back to the start of the P-Doom question,

• 36:35Emad Mostaque?

  there are two types of decision-making. When you make decisions in a stable environment, and this is where you should have a P-Doom or probability, you did expected utility calculation. This is the probability of all these things, and then we do expected utility across that. When stuff is uncertain, and we move from decision-making under risk to uncertainty, you minimize the maximum regret. So if you're going to go at the start of a desert, you don't know where the oasis is, you're not going to take any steps, which is the downside of this. So you need to articulate positive visions of the future. And what that is that, if you look at the energy equation of how much it costs to do a GPT-4 level query, it was 1,000 watts of electricity. And now it's five.

• 37:17Emad Mostaque?

  You can have an original GPT four level model that runs on an MPU now, which is a five watt of electricity. Empty. Solar power is now less than a dollar per watt. So then you have a chip cost and an inference cost. That's probably fifty bucks. To have a GPT four level AI offline anywhere in the world. And you think about that, you're like, wow. You could give a tutor to every single school in the world. What if you connect to with Starlink? And it's off grid. You look at medicine, you're like, how can this change again? But again, if you look at all the stuff you have for living, which I think is the most important thing. And we want that to be robust infrastructure. An AI first approach to those where we have gold standard transparent data sets.

• 38:00Emad Mostaque?

  That then create models and systems that we can deploy. Online, offline. Could be something that's world changing.

• 38:08Emad Mostaque?

  I think you need different models from the individual to community to country to society level. Because to take a practical example. Half of everyone who listens to this call will get cancer. Which is crazy if you think about it. Everyone is listening and has had someone in their family or friends who's gone through a cancer experience. And it's awful. You feel that massive loss of agency when that occurs. Because we don't have cures yet, we have treatments, but still it's very traumatic. Medical AI models now are performing human doctors on research. On diagnosis, but also empathy. And how many have had an empathetic experience through their cancer journey?

• 38:48Emad Mostaque?

  What happens if we give every single person going through cancer, multiple sclerosis, Alzheimer's? An AI that helps them through that journey?

• 38:57Emad Mostaque?

  How to talk to your family, what's the latest comprehensive authoritative up to date information? You know, it's always there, it's always available in every language.

• 39:07Emad Mostaque?

  The world becomes a better place, ironically through empathy from AI. And that's infrastructure that should be available to everyone. This is a concept we call universal basic AI. Or UBI as it were. Which I think is better than income and we can discuss that in a bit. But then you look at them again, every condition. You look at education, everyone should have access to an AI that's looking out for them and how can I maximize that child's potential?

• 39:30Emad Mostaque?

  But then shouldn't all of these feed up in a proper way to collective knowledge? What's our collective knowledge on cancer? If a paper comes out, why isn't it instantly analyzed and checked and absorbed into our knowledge, be it probabilistic or certain? Why do we have a million different vaccine schedules around the world? I'm not saying vaccines are good or bad, but still it doesn't make much sense. People are people. What is an optimal way to teach calculus or machine learning or anything like this? So I think if we can build intentionally, we can build better AI-first systems that come together. And that's what I call the intelligent internet. Because we had a very interesting point in time whereby your generative AI for health is probably a little bit higher than most, but it's basically nothing compared to what it'll be in 10, 20 years.

• 40:14Emad Mostaque?

  You will have an O1 level intelligence organizing your health if it's built. And your hospital will have the equivalent of a supercomputer. And your country will have dedicated hardware internally against intelligent capital stock, as I call it, that's organizing the health system. And then we should have a central organization to have a comprehensive and authoritative, up-to-date knowledge base for cancer, for climate, for education, for all the stuff we need for living. And it's a finite amount of energy that's required to build that. And then that's a good infrastructure that we have to support us. And so that's what the concept of the intelligent internet was. It was like, let's build data sets, model systems, and then the right hardware so that

• 40:55Emad Mostaque?

  we can deploy this at scale to everyone and lift the intelligence of humanity in a secure and robust way. And so we outlined the end state, and in the upcoming pieces, we're going to outline the ways to get to that, from identity to digital assets to physical infrastructure to the models themselves. Because I think it'll be difficult to do this in a decentralized, distributed, or market competitive way. I think you've just got to go and do it. We know what to build, you know? Just build it. Organize all the cancer knowledge. Why hasn't anyone done that? I think of how crazy it is. Like, again, if you've got a diagnosis, God forbid, tomorrow, what would you do? You would go to court and you would check this and you would do that. But why isn't that all just in a box? You know?

• 41:39Emad Mostaque?

  And again, for me, that's infrastructure. Platon Christensen coined the term disruptive innovation, said, "Infrastructure is the most efficient means by which society stores and contributes value." And then information theory, Claude Shannon, is that information is valuable as much as it changes the state.

• 41:58Emad Mostaque?

  Deliberately building the data sets, models and systems, making them freely available as cheaply as possible, as low energy as possible, is a new type of intelligence infrastructure for society. But I think we'll do better than we can build other stuff off.

• 42:14Nathan Labenz?

  Just to take a second and steel man the case for the, or at least try to steel man the case for the leading developers today. I feel like you would hear a lot of high level conceptual agreement on what you have laid out. And maybe not necessarily at the level of like exactly where the models reside. But certainly they talk a lot about like intelligence too cheap to meter. And they have certainly dropped their prices dramatically. Possibly they're, you know, all operating at a loss on the margin to try to compete for share, not entirely clear.

• 42:55Nathan Labenz?

  They also make a lot free, right? I mean, OpenAI just yesterday released their search feature to all users globally for free. There's a lot that's free.

• 43:05Nathan Labenz?

  Is there a critique of sort of what you see them doing today? That is central or is it more just that you extrapolate into the future and you see that this structurally like won't work for much longer?

• 43:20Emad Mostaque?

  I think it will be very difficult for the proprietary model developers because Apple Intelligence, Google Intelligence will be free. So this is one of the things I had at stability where I was like, do I do a giant funding round? We never did another one after the first one. We did like uncapped notes, a hundred million dollars, whatever. Cause like, what is a sustainable model on the API or SaaS side? When they're coming for you, like Gemini Flash 2.0. If it's the same price as the original Gemini Flash, it's seven cents per million tokens.

• 43:50Nathan Labenz?

  Yeah, that's crazy.

• 43:51Emad Mostaque?

  That's like, what, 50 times cheaper than 400 and a really good model, right? I think that there is this thing where that type of intelligence as a consultant is available. I think there is this world of open weights. There's this personalized AI that you have Apple Intelligence and Google Intelligence and others. And there's this other area which I'm focused on, which I haven't seen anyone tackle, which is regulated industry AI. I don't think you can have black boxes there. I think we deserve better. I think we need open data, open weight models built deliberately. And what I have is generalized common knowledge, because the data for that should all be common knowledge for education, for healthcare and others. We can have curriculum learning. Then you have your sectoral and localized versions. So you have a Mexican healthcare model.

• 44:38Emad Mostaque?

  And the data to build that is finer and you can get to a gold standard very quickly. And that should be infrastructure that's non-rivalist and non-exclusive, in my opinion. I think everyone would benefit from that. Because everyone wants to have high-quality localized datasets, specialist datasets and more. And again, the data that you need for a medical model for Mexico is in the medical textbooks of Mexico. You know? And I haven't really seen anyone build that deliberately and really optimize against how do we get this to as many people as possible? Which is why I said, let's just go and do that. Because that's the biggest delta. Because many of these organizations, you're still fighting for SOTA, right? You're still going for AGI, but you're satisficing.

• 45:19Emad Mostaque?

  Like, how much difference is there really between an Anthropic and an OpenAI and a Gemini and a Grok? They're all pretty good right now. And they're good enough to make a difference. So I don't think you need SOTA here, which is also another very interesting thing. In fact, do you want to have a state-of-the-art, unproved model for your healthcare? You want a really robust, standard one that behaves very predictably. Which also is why with the advent of function calling large complex windows is the ideal time for this. For education, for that. Again, anything in regulated industry. Do you want a black box checking every medical decision? Then eventually make, no, you don't. Or checking every government decision, then maybe make, no, you don't. So I just think there's this particular category that's very interesting.

• 46:01Emad Mostaque?

  But I haven't seen anybody look at, but I think it's probably the most important one for flourishing. But then it becomes very interesting because if for all the stuff you have for living, you know Maslow's hierarchy of needs, you have these open-way, open-data models that we're trying to optimize for as low energy as possible for maximum impact to get to a level of performance of like almost equi-token, equi-MMLU, as it were. Then I think that can guide the discussion because suddenly you can parameterize and have datasets for cultural diversity across nations. And you can figure out things like DOCSA. You can have a feedback loop where if you release all the data open from teaching kids and learning from kids.

• 46:45Emad Mostaque?

  How to increase agency there or on the medical side again, self learning systems. So you can make everything open with PII and other things. And you can have interesting things like if you have a cancer AI to help you through your cancer journey, it can tell that you're a human because it's attached on the regulator's side. And then you can use the same system for your education, for your government and more. That's the functional foundational identity. So there's a lot of benefits that can emerge here that could help with the AGI and state of the art. But again, I view those almost like expert systems, graduates, consultants from McKinsey, you know, where this is more like the graduates that work for you and your team.

• 47:26Emad Mostaque?

  And the Apple and Google intelligence, the world and others are probably not going to be doing your healthcare because they don't really want to. They want to do your personal life and everything. They want to interact with healthcare, but they want to interact with education. But I don't think those organizations are going to go after that. So I think it's just, nobody's really looked at that. And again, I think it's inevitable. And the question is, who, do we have a million different healthcare models, education models and more, when we only really need one? That's then customizable and adaptable and is MIT.

• 48:00Nathan Labenz?

  So is the sort of model for the future that you have in mind, one where, because you're hearing, of course, from Sam Altman and Dario that they sort of expect to run away from the competition in the next couple of years. They are not sharing the O one chain of thought. They've said this is kind of GPT two level scaling of, of runtime compute and they know how to get to GPT four. They think they're going to blow the doors off of everybody there. Dario and team have said in their, I think, credibly attributed but leaked fundraising document that they see a world in like 2025 timeframe, which is by the way, two weeks

• 48:42Nathan Labenz?

  away, that the leading developers could get so far ahead of the competition that nobody can catch up because they have these sort of, you know, enrichment loops that keep advancing the state of the art. And if they're not sharing that, you know, I also had a recent data point on this when Nathan Lambert from the Allen Institute did an episode on everything he's learned about post-training. And one of the big takeaways for me was he said, In the absence of access to GPT-4 or similar to create the samples that we do the post-training on, we can't do it. There's just no, even at the Allen Institute, billionaire estate-funded, we don't have the ability to go do a scale AI contract or whatever to bring in the expert data that we would need. So do you think that happens or doesn't

• 49:26Nathan Labenz?

  happen? If it does happen, is the model sort of, they become like the go-to for the pharmaceutical industry and cybersecurity industry and all these sort of like things where you really do still need to push the frontier of the state-of-the-art forward? And then at a more distributed level, we satisfy and like are really more focused on reliability and sort of explainability when it comes to things like making our own personal medical decisions?

• 49:55Emad Mostaque?

  I think that's the case. I think that we over-indexed on chefs versus cooks. So you know, everyone on the spectrum, Wait But Why had a great post from being a cook that follows recipes to making recipes. How many recipes do you really need to make? Where AI is actually really good now with function calling large context windows is following recipes. You give it a handbook and it's basically replaces a SaaS or replaces an employee or something like that. And we want to have more and more employees that follow the damn instructions. In effect, how often do you need to come up with something brand new for the vast majority of people? For that, there is the O1 type models. There are all these other kind of things. And again, this is where you have the AGI race away. But if

• 50:37Emad Mostaque?

  you're training, say for instance, on a million H100s, like Elon's about to, or a million TPUs, like Demis is about to, or a million Tradian, where God forbid, Daria is about to, poor guy. You know, like, how big is that model actually going to be for inference side? You kind of went straight through the consumer side and now you're like, well, am I going to build a multi-trillion parameter model that needs a cerebrus wafer or a Google TPU pod to run? You know, because that's not really suitable for consumers at the side of things, but it might be more intelligent. But it may be 5% more intelligent. Whereas it requires orders of magnitude less compute to have the lower ones. And this is what becomes very interesting because

• 51:20Emad Mostaque?

  classically, most of computation is being sequential as opposed to like parallelized. And we're seeing this now again with test time compute and others. Like, who's going to win if you need to have millions and millions of GPUs running in parallel, running O1 type things? To check something and cross check it, probably more distributed others. But what is winning? If winning is AGI heavy, you describe it, then fine. Those are the expert systems. And that's where it is. But will one cluster win over a massive amount of post-training? With IQ 130 things checking? Will one cluster win over, let's take the China example, getting 100 million people in China to label stuff?

• 52:03Emad Mostaque?

  This is not mean that I don't really understand where it's going because we haven't proven that we can break through PhD level yet. And even if we did, how useful is that in the context of all the actions of humanity? And improving humanities agency and capability to a brighter future? Like, this is the same question as what do you use a quantum computer for? Actually, coming up with the questions that you want to answer is really hard. This is why I like, I love O1 for the more advanced stuff and I'm using it to help me improve my QD and kind of other things. But the average person isn't going to get much mileage out of it because it's too damn smart. And do you need even smarter than that? You have to set even better questions if it's smarter than that.

• 52:46Emad Mostaque?

  Whereas again, like I said, this particular area of AI for living, I don't think needs that at all. I think the models right now are good enough, full stop.

• 52:56Emad Mostaque?

  To make a transformative difference for the world if we organize the data correctly. The category of personal AI, your Apple intelligence, so Siri isn't stupid. And other things like that, it's nearly good enough, you know? The category of expert systems, yeah, it might be a race away thing.

• 53:12Emad Mostaque?

  But I don't think it's interconnected race away thing. And if it is, then Google will win. Because Google's interconnect fabric is so much better than everyone's and they're deploying it at a ridiculous pace. The only potential competitor I could see might be XAI. Anthropic, I don't see Tranium and the data funnel scaling that much and OpenAI similarly. I don't see it scaling that much.

• 53:37Nathan Labenz?

  Yeah, reports of Google's demise, I think, were always greatly exaggerated.

• 53:41Emad Mostaque?

  I never got that. I mean, I remember when it first came out, I was like, this is stupid. They've got the people, they've got the talent, it just takes time to steer the ship. They've come up with all of these things. And again, if you look at the hardware, like, we have thousands of TPUs. TPUs are the best interconnected hardware and individually addressable, which is why Google's models are the only ones publicly available with 2 million token context windows going up to 10 million. And people don't appreciate how insane that is with like 100% needle haystack. You can do some crazy things that we're just exploring the surface of right now. I think Notebook LM was probably the first example of that. You upload a gigabyte of things to Notebook LM, and then now you can dial in to talk to the podcast host.

• 54:25Nathan Labenz?

  Yeah, one wonders how much longer we'll need human podcasters, but I'll spare you my personal anxieties about that. I also do have a challenge, which I fail on still every day, of spending a dollar a day on Gemini Flash. And it is just mostly for me a reminder of how cheap things have become and how much information I really could be processing for very, very little info. And I still just need to like figure out the right structures to do that for me.

• 54:55Emad Mostaque?

  Just get to watch your screen and tell you when you're being unproductive.

• 54:58Nathan Labenz?

  Yeah, I saw that demo in the last couple of days. And yeah, that's the always on observing, occasionally interjecting AI, I think, is going to be a really interesting paradigm. I really like your chef and yeah, go ahead.

• 55:13Emad Mostaque?

  No, I was just saying, I think, again, if you are boiling these models down to everything, it does come down to engineering. This is why Elon managed to catch up so quickly, right? Because he's an engineering maestro. Like, 100,000, million GPUs all good. But then, like I said, if you look at the different types of hardware, the interconnectability and the addressable RAM and other things like that, and not paying the Nvidia premium, like, it matters a lot. This is why I think the latest like drama with the OpenAI and their blog posts and things like that, like, hey, there's Elon, there's this. They were talking about buying Cerebras because of that massive addressable memory and things like that. So it'll be very interesting to see how that all pans out.

• 55:52Nathan Labenz?

  So I like your chef and cook notion as a simple heuristic where maybe a one, two, three, four, whatever goes off and figures out how to cure particular kinds of cancer, or maybe Claude four or five, or Gemini three is doing that stuff. And then on a daily basis, we satisfy and we try to make sure that we're applying the actual latest known stuff in an effective way, as opposed to trying to advance the state of the art, you know, each one of us as individuals. Let's talk about the structure for this. I think in the primer for the intelligent internet, you do a really nice job of walking

• 56:35Nathan Labenz?

  through the different tiers of the critical resources, right? I mean, obviously, infrastructure, compute infrastructure and data, as well as, of course, the algorithms and the human talent that makes it work are the fundamental inputs. And you've kind of mapped out a structure where there is like the highest tier, biggest scale of those, and then a more distributed level, and then a more local level, and ultimately individual users get to take advantage of this kind of on their own terms. Take us through that vision for how this actually gets built out, because it is like a physical capital thing and structured to achieve the goals that you've articulated.

• 57:17Emad Mostaque?

  Yeah, so again, if I kind of look at the future, I see very rapidly, we're going to see economic disruption because the labor theory of productivity is about to be challenged. You start to find people, and again, anyone listening to this, finding human capital is the hardest thing, whereas AI will follow your instructions and it's very reliable, relatively speaking. I mean, sure, it was a goldfish earlier this year, you know, like it's savant PhD, has had his coffee, just respond to stuff. We're solving most of those things right now, right? So I was like, there needs to be this capital stock build out. And right now, we're seeing it on an individual level because you're getting M4 MacBooks with 16 gigabytes of VRAM as standard.

• 57:57Emad Mostaque?

  You're seeing eight gigabytes on 4070s on laptops. And you had eight gigabytes on the integrated C's and AMD chips with the 70 ATMs. Those will soon all standardize, in my opinion, to 16 gigabytes for AI compute. Be it with an MPU at five watts, or be it with neural processor, or more advanced graphics options at 25 to 125 watts. This is a build out that enables intelligence on the edge. And again, every manufacturer is doing that. But then you think about a hospital. And that's what we call the distributed nodes. You want to have a standardized stack that can go into any hospital, transform the data, and then add an interface to that to interact with your personal intelligence and more.

• 58:41Emad Mostaque?

  And then you want what we call hyper nodes, which are national nodes, to organize all the healthcare knowledge of the entire nation, or organize all the local cultural data of the entire nation, and keep that up to date as well. So we kind of, we have this stack ranking where there's different levels of computation, from inference level 16 gigabytes, to let's say right now, fine tuning level at a hospital level, like 128 H100s to 256, can get any hospital or bank or anything like that pretty much fine tuned. Then you've got the national level, which would probably be in the thousands of H100 equivalents to organize all of that. Maybe there's an order of magnitude each way. And then when I looked at that, I was like, this is really interesting because this is a defined capital build out. You want as standardized as possible, so you have boxes that plug in.

• 59:26Emad Mostaque?

  They can just look at stuff because this model is amazing at the infrastructure level. And they offer different types of experience at different levels. But again, one healthcare model for cancer is very easy to turn multilingual for everyone in the world, right? And one radiology model and one education model was theories of those. But that's actually the next part. I think we mentioned it briefly in the primer. I was like, that looks a bit like Bitcoin.

• 59:53Emad Mostaque?

  As it were, which is maybe a story for later another day. But dedicated infrastructure build out to create an urgent capital stock that then gives capability. Just like building factories gave capability, but you don't need factories anymore, right? You actually need pure hardware, running as standardized models as possible. And those standardized models need to be based on standardized data sets. And just have regular releases and be predictable in the way they operate.

• 1:00:22Nathan Labenz?

  Okay, I have a lot of little detailed follow up questions. I guess for starters, so much is being made right now about the energy requirements of AI that I find this kind of confusing. On the one hand, obviously, we hear, you know, seven trillion dollar build outs and crazy energy grid expansions. To some extent, I sort of interpret those as like trying to expand the overhead in Windows. So we can get like a couple of new power plants online versus like very little new construction, obviously in the US in recent history. It is, I think, a very good grounding to talk about the idea that a five watt machine can power an AI.

• 1:01:04Nathan Labenz?

  I wonder if you could flesh that out a little bit more like what, you know, it seems like an eight B llama three point three eight B.

• 1:01:13Emad Mostaque?

  Yep.

• 1:01:13Nathan Labenz?

  Is kind of hitting at that GPT four level now, and that's quite runnable on a certainly on a new Mac, right? Can you give us a little bit more there on like exactly how you envision that?

• 1:01:26Emad Mostaque?

  Yeah, so eight B model is runnable on a Mac neural engine. So a Mac neural engine will use about five watts of electricity as well as Intel MPU. But if you want more, like I've got my M two max and waiting for my M four max, that will use up to a hundred watts of electricity. Right? If you look at the H one hundred, the H one hundred uses a thousand watts of electricity.

• 1:01:49Emad Mostaque?

  So you can use more energy and it'll go faster. But 10 tokens a second is human reading speed. It's like you can, you know, you see these models go like that. If it's faster than you can read, what's the point? In some ways, right? And there's a question of how many tokens does a human need to flourish? And so we could do some calculations around that. We're like, well, if you don't need everything synchronous and you can let it go, then five watts of electricity is probably all anyone needs. And again, you're seeing that because you can now run models on the smartphones. iPhones tend to top out at 25 watts of electricity. This is where it becomes again very interesting because let's rewind a bit. On the stability, we did stable diffusion. And that costs maybe $5 million of compute in total, right? To do.

• 1:02:32Emad Mostaque?

  And that's a decent amount of megawatts of electricity. You know, it's a million or something. A100 hours and A100 is what? Gosh, I can't remember, 400 watts? So 400 megawatts of electricity, is that right? Something like that. And that's a good amount. But then to run the model is just a little bit on the edge. So you think about it, the pre-training and then the post-training, et cetera, comes out with an artifact, which is this compressed set of weights. That then means that you don't have to retrain that thing. Where a lot of the exponential energy usage is coming from is a lack of optimization. And this is why Meta releases Llama, one of the reasons.

• 1:03:16Emad Mostaque?

  They got 350,000 A100s, they're about to go bigger. 10% to use for training. They have improved the efficiency of Llama by more than 10% by releasing it. So the remaining 315,000, it more than pays for itself. You've seen this, for example, Genmo released their new video model, Mochi. Day one, it required, I think, 140 gigabytes of VRAM. Within a few days, it required 20. And now it requires eight. Like again, there's massive optimizations around these models that can occur as well because it depends on what are you optimizing for? But you look at Google, you look at Microsoft, look at everyone else. They're like, we can't afford not to do this build out. Because what if we're wrong? What if we can't optimize?

• 1:03:59Emad Mostaque?

  Because no one knows what the satisfying level of performance will be for any of these models. Like you look at VO2 right now, that's just come out from Google on the video side, or Sora.

• 1:04:12Emad Mostaque?

  It wouldn't surprise me if the VRAM and energy requirements of those drop by an order of magnitude overnight because someone's figured something out. We don't know how much these will require, so we overbuilt to a degree, but we're building more and more of this capacity up. But then it becomes interesting because although we've had more and more of the energy usage, you think about the energy usage to make a movie, how many megawatts is that versus what it will be? The energy usage to make this podcast, the energy usage to write a book, the energy usage to provide a medical input. And I think that AI is very beneficial for this because again, there's this lovely chart that you can look at of GDP per capita versus energy. It was always that relation. It's broken

• 1:04:54Emad Mostaque?

  that. Because in a year, we could go to a kid in Sub-Saharan Africa and give them solar panel, an NPU chip, and a version of Llama4 that speaks their language, and then they have access to how much knowledge, right? Or a medical version of that, or anything like that. And that's crazy because you would have to build the infrastructure in place. You add Starlink to it, it gets even crazier, but you see what I mean. It's like, this is a fundamental change of energy. So I think we have this energy run-up now, but then global energy is not going to keep up consuming. Unless of course I'm wrong and we end up with like freaking Dyson spheres running AGI and other things like that. But I feel

• 1:05:38Emad Mostaque?

  that probably won't be the case. I mean, there's diminishing returns on scaling.

• 1:05:43Nathan Labenz?

  That's an important idea. Just to put a little bit of my own, you know, back in the envelope math on the energy requirements. Where I live in Detroit, Michigan, the cost of a kilowatt hour of electricity is roughly 20 cents. I think it's a little lower than that still, but call it 20 cents as a nice round number. Five watts is one two-hundredth of a kilowatt, obviously. And so I would be looking at, if I'm running my AI on one of these five-watt machines, non-stop, I would be looking at a tenth of a cent an hour in energy cost, or like... Two, two and a half cents a day, which means obviously I'm looking at

• 1:06:28Nathan Labenz?

  single digit dollars over the course of a year. And I think that is a dramatically underappreciated point about just how accessible the marginal cost to actually operate the thing you do with the chip costs too. But just how accessible that already is for something that, you know, with a llama three, three, eight B is like getting really quite good.

• 1:06:52Emad Mostaque?

  Yeah, I mean, Hugging Face just released their latest thing on their O1 replication. They've got llama one B outperforming llama eight B with test time compute.

• 1:07:02Emad Mostaque?

  And again, if you put it 24 seven, you're interacting with it. How crazy is that? Where is the lower bound on these things? We're not sure, but it's got to that point where as I said, people don't appreciate that. And the example I gave was solar power. If you hook up a solar panel to the grid, it takes up like $4 a watt. If it's off grid, it's less than a dollar a watt for the solar panel. So then what the cost is effectively nothing. And again, the question is, where do we satisfy? Like, how good do you need? Like, again, you look at VO2 today on the video side. Do you need anything better than that to make a Hollywood level movie? No, not really. Like, sure, there's some small things, but those can always be tidied up in post. Right? And we figured out consistency and other things.

• 1:07:44Emad Mostaque?

  Do we need, we have chatbot models now that are outperforming human doctors on empathy and diagnosis. Do we need better than that? No, we need to optimize the heck out of it and figure out where that lower bound is. And the amazing thing is all these models are still full of junk. Like Ilya Sutskever just said at NERIPs, like we've run out of data. How much of that data is junk still, even in these top models? We're still like getting to grips with this. Like we saw the Sana team, previously Pixar, create a stable diffusion level model with 25 billion images. And we use 2 billion. What is the lower bound of data that you need to have for a language model to achieve X to teach your kid?

• 1:08:29Emad Mostaque?

  Does it need to have seen Reddit? No. Does it need to have seen this? If you want to build AGI, yeah, you need all that data. Maybe. But do you need it all in one place? Like, what is the limited data analysis? And we've seen interesting things like, I think, Player AI just did like an AI that was just trained on knowledge up to 300 B.C.

• 1:08:55Emad Mostaque?

  I haven't seen what it's like, but those are the types of things that very really interest me. Like, what does that look like?

• 1:09:02Emad Mostaque?

  So it feels like

• 1:09:03Nathan Labenz?

  Ancient AI.

• 1:09:04Emad Mostaque?

  Right, it feels like we feels like we have an order of magnitude improvement still to come, just from data, obviously. And we're seeing this with Fine Web and kind of other things and the synthetic data sets and augmented data sets that we've had.

• 1:09:18Nathan Labenz?

  Yeah, there's a lot. The efficiency curve is truly crazy. I mean, five models from Microsoft are an interesting data point about how small your data set can be and how, yeah, I think theirs are, as far as I know, entirely synthetic for that line of models.

• 1:09:34Emad Mostaque?

  Yeah, they did 13.8 epochs on like 400 billion tokens for the core, but then they have to add in like a map of the internet because they're so boring and people don't want to interact with boring models. Because again, people take them generalized. So when you add back the internet, they get less boring, but five models have the worst sense of humor of any model.

• 1:09:54Nathan Labenz?

  Very, yeah, textbooks aren't all you need for a sense of humor, I guess. But yeah, so many things like that.

• 1:10:01Nathan Labenz?

  Let's go, okay, so that's the low level. It's incredibly accessible and affordable to satisfy. If you go up to the top end of your infrastructure for the intelligent internet, this is the like national or perhaps even international hyper node level. A lot of questions about that, I guess, but maybe for starters.

• 1:10:25Nathan Labenz?

  Is your hope for safety and transparency ultimately rooted in open data? It seems like open waits, we can hope for a breakthrough in interoperability, but we don't have it. So it seems like right now, if you were to say, we want to make something we can feel confident is safe and we feel confident is not going to be too crazy. You would have to do that by sort of saying, we deeply understand the data set and that's why we believe this, is that right?

• 1:10:52Emad Mostaque?

  I think it's necessary, but not sufficient. Like, I think the existence of gold standard data sets that reflect individualized cultures and other things are going to be vitally important when it comes to decision making AI.

• 1:11:05Emad Mostaque?

  Again, how do you understand what the people in Mississippi want? Or the people of Malawi want, or the people of Mumbai want? If you haven't mapped out the individual culture, the decisions and kind of other things. And open data with open models probably makes them more interpretable, but no one's figured out interpretability. Again, lots of interesting SAE and other research, but we haven't figured this out yet. I think it will help, and I think again, the existence of these gold standard data sets will help the entire space. Because like when Luther released the pile, it became a standard. When Lion released their data sets, it becomes a standard. That's why I originally called the company Shelling, because I was like, let's build standardized gold standard data sets and just improve them rigorously. Which is what we thought the hyper nodes would do. The hyper nodes coordinate and improve the data sets.

• 1:11:49Emad Mostaque?

  And then can go down to the organizational community and individual level and have a self learning system. That again, just improve, improve and keep it up to date. Then anyone can take those, because we released everything MIT permissionlessly, which is the other one thing. Like you don't, what if someone, you break the rules and they shut off your AI. You're a bit screwed then, right? Like for whatever reason. And then I think again, that can move towards standards because the highest value AIs will be decision supporting and making AIs.

• 1:12:24Emad Mostaque?

  And what do those look like and how do we understand what those do? And are those got repeatable frameworks? The most widely used AIs will be the AIs that are available, especially if they're permissionless. I mean, we have 300 million downloads of our models at Stability AI, you know? That's a lot of fricking downloads because they're just available. But then people built around them because they didn't need permission. So I think that again, it's necessary but not sufficient because like otherwise they look at anthropic and open AIs models like, how will they incorporate conceptualizations of Chinese or Japanese ethics?

• 1:13:01Emad Mostaque?

  Is anyone even taking that seriously? But then what if they use to make decisions in the Japanese government? As they kind of are right now. Like, I think you need to be very deliberate in the way that you figure this out on the underlying principles and the feedback loops there. And that was the final part of the intelligent internet, which was, I think, people should have a say and be participatory, which is why we're looking at the digital asset space, we're looking at distributed governance, we're looking at all these other things. So is there a way that can all be brought together? Which is not easy. You know, hopefully we can figure that out.

• 1:13:35Nathan Labenz?

  Yeah, well, I'm interested to hear more about, it sounds like thoughts there are still in development, but the, whereas the user level seems like it almost is going to take care of itself in as much as everybody is continuing to buy phones already. I don't really know what to expect at the hyper node level. Like, I don't know how many hyper nodes you think we need. Are they billion dollar? Are they 10 billion dollar investments? And who's going to make them?

• 1:14:04Emad Mostaque?

  Well, so I think a country's competitiveness was about its physical infrastructure. How many things can you build? It was about its graduates. What's our intelligence smart graduate network, right? It will become about your intelligent capital stock. How much compute coordinated into models do we have access to? And again, if you look at the underlying thing of China, US versus beyond the jingoism, it is a question of let's build compute in America, let's build compute in China. Because from a test time compute perspective, that is what your competitiveness becomes. And you're seeing this across the world, like Malaysia and Johor just built a 9.6 billion dollar AI data center. You're seeing billions of dollars across Asia.

• 1:14:47Emad Mostaque?

  I don't think you need to have that much beyond the thousands range. Because although it's like 240 billion dollars or something like that of AI building going on, when you start looking at how much individual clusters and things like that are, like the fastest, largest biomedical healthcare cluster in the world is probably Chan Zuckerberg with a thousand H100s. That's like 20 million dollars a year. Which is not nothing, but it's not ridiculous, you know? And so you're like, how much will probably be in thousands ranges, would I think will be sufficient to organize national data sets and create them? And even provide almost free AI to the vast majority of people on a reasonable sized nation.

• 1:15:29Nathan Labenz?

  You're talking about like thousands of H one hundreds being enough at the national level?

• 1:15:33Emad Mostaque?

  Yes. And this is not in the short term. This is like long term H one hundred equivalents. Like, once you build a gold standard data set for education for Bangladesh, how much do you really need to update that? Not that much. And if the outcome is a model that's updated every so often, that's a few gigabytes, then goes on to everyone's phones, provided by TCL or Huawei or whatever, then again, that isn't much. And what's the cost of inference? Pretty much nothing. It shift again, that needs to be drilled deliberately. This is separate to the state of the art. The state of art costs a lot and it costs a lot to inference and it costs a lot to do everything else. But for me, like I said, this is for me is the most interesting part.

• 1:16:15Emad Mostaque?

  Give every child their own AI teacher that's aligned with them. Make sure no one's ever learning through their healthcare journey anymore. Make governments more representative, finance non-corruptible and things like that.

• 1:16:26Nathan Labenz?

  I'm surprised by how small you're projecting that to be. Are there, is there any assumption there about like having a general, if I'm Malaysia, for example, well, Malaysia, you said built a $10 billion one, but let's say I am

• 1:16:38Emad Mostaque?

  Yeah, just now.

• 1:16:39Nathan Labenz?

  Small country and I've got my Chan Zuckerberg scale thing and it's thousands of H one hundred equivalents. I can't train a

• 1:16:50Nathan Labenz?

  foundation model from scratch on that, right? So I would, I can do a lot of fine tuning. I can do a lot of data processing, but or at least it would seem hard to do foundation models from scratch.

• 1:17:02Emad Mostaque?

  You just need one entity to do it. Like my meta doing Llama is good enough for almost most things, right? And they did 20 million odd H 100 hours for the largest version of Llama. And that's what, let's say $40 million spent for Llama 70B three. And then everyone could just take that and fine tune it, but to tune it requires 128 to 256 H one hundreds.

• 1:17:31Emad Mostaque?

  And the order of magnitude cost is $100,000, $200,000 to create Arabic version of that.

• 1:17:37Emad Mostaque?

  And so again, we're blinded by the massive compute and the edge stuff when actually it doesn't require that much to do the tuning. The thousands enables you to actually give access to your people to the technology, especially if, again, you don't require rate limits or a satisfying level. And it also allows you to do the tuning. So you probably start out with a few hundred and then you build up from there. And again, when you look at the disparity of compute and energy and other things, it's very interesting. Like I was in South Africa, I think the fastest cluster there was like 50 or 100 H100s. But then the clouds are building even larger ones. This is why it's very achievable if we're coordinated. But then, do you want to be giving your people that level of healthcare AI or that level

• 1:18:22Emad Mostaque?

  of healthcare AI? Do you want to have your companies in your country be able to call on these experts or even more experts? So this is why I think there'll be a very interesting capital stock question. And when we look at things like 5G, the amount of AI spending has not caught up to the amount of 5G spending around the world. This is far more important than 5G when it comes to a country, a community and individuals' productivity levels. You know, again, like just think about it. Having an expert in your pocket, that's your Huawei smartphone. Having your hospital suddenly have all the latest stuff and being available to all the people in your community, that's a 50 grand type of thing, you know? 100,000 it scales.

• 1:19:07Emad Mostaque?

  And the more you spend, the more competitive you'll be, the more productive you'll be, the more access you'll have. And again, I was just shocked by how many people were actually looking at this particular area and what was required for that. So I think if we go back to the start, one entity will need to pre-train base models. That's our common knowledge. They have sectoral data sets where you sectorally specialize in and localized data sets, so you can have the generalized model customized for Mexican law, Indonesian accounting, Japanese medical, and everything is completely transparent across that. And the amount of compute is primarily that pre-training at the top, but then require the hundreds to tune of equivalent.

• 1:19:53Nathan Labenz?

  So it seems like, I guess maybe one caveat that the Llama models are not open data. It seems like you feel like they already have enough raw power to serve as the base for a lot of this fine tuning around the world. Do you worry like, what happens if Llama 4 doesn't get open sourced? And do you feel like, alternatively, the other end of that would be like, do you feel like it's just not viable? Because it's not open data and there needs to be a true, fully open data alternative?

• 1:20:26Emad Mostaque?

  Llama 4 is, will be amazing for a vast variety of things. I don't think it's suitable for regulated industry. And again, regulated industry is typically the stuff for living.

• 1:20:38Emad Mostaque?

  So it's just someone who needs to go out and build those and those standardized systems and release them MIT. And then that makes the world a better place and then enable access to those. So I think Llama is fantastic. I think Quen is fantastic. I think where those end up in a few years, nobody knows. They're run by Alibaba, they're run by Meta. And again, those are corporations. I think they want to build infrastructure to teach all the kids, honestly. Like, eventually, if you think about the infrastructure that will teach your child, an AI that's fully open and owned by you to act as a regulator, that then calls into a Llama-based service that can then call into an Anthropic or Google

• 1:21:21Emad Mostaque?

  or OpenAI service, it'll be a mixture of models, right? And you'll have the protected model there. And then your kid's ability might be that you have a base level. This is why we have this concept of universal basic AI, which I think is important. That's your base level, but then you might want more and then you pay more for more. I think that's absolutely fine. So again, I don't think that what we're looking at, what we're doing replaces Llama. I think there needs to exist open weight models. And I think the two best are, the three best are DeepSeek, Llama by Meta and Quen by Alibaba. And I think those serve their purpose, but they will never release the data set that underlies that because they're not trying to build infrastructure.

• 1:22:04Emad Mostaque?

  They have a different objective function, just like I don't think Apple will release all the data inside Apple Intelligence, just like I think Google won't release all the data inside Gemma. And even if they did, like I said, it doesn't really align with the reasoning of the organization.

• 1:22:19Nathan Labenz?

  Yeah. So how do you, this maybe gets a little bit to the crypto question, or at least the, you know, the incentive engineering, if you will, you've got this concept in the intelligent internet primer of proof of beneficial compute. And I'm curious to hear how you think this can best be organized or catalyzed so that these, I guess mostly it would be countries around the world and maybe a mix of other private institutions too, if they want to, how do they all come to see it in as being in their interest to contribute to this project?

• 1:22:58Nathan Labenz?

  That seems like a, it'll be an amazing trick if you can pull that off.

• 1:23:01Emad Mostaque?

  Yeah, it would be nice, wouldn't it? I mean, I've been looking at a lot of business models. Again, last podcast we had with yourself and others was like talking about business models and AI. And, you know, like the stability experience was interesting. We took it up to tens of millions of revenue run rates in just a few years of others. It was all burning, but it was getting towards profitability. Got a bit tight at the end. I think the latest CEO said there was less than 90 days of cash, which is better than Apple was back in the day. That's why I didn't do another giant rant. I was like, where's sustainable thing here? When I kind of took a step back and I looked what I really want to do, like I worked on you and United Nations AI initiative on COVID-19, United Nations back one and other things. I got into AI when my son was diagnosed with autism and I built an AI system. I was like, how do you build

• 1:23:46Emad Mostaque?

  infrastructure that's available to everyone? And what does that look like? And thinking about the future, one of the things that I've alluded to in this podcast is that the labor theory of productivity, which is the center of economics is broken. And then the nature of money is probably going to change because what does an optimist or an O1 or other things need? Some sort of GPU inference based currency, as it were. Some people have said that Bitcoin might be the future of these agents. Again, you've seen AI agents now go across crypto channels and other things. And I was like, if I've got a massive amount of compute coming online, And I know that, again, the amount of compute for your city's hospitals about to go from that to that, and I can build a standardized stack for that.

• 1:24:30Emad Mostaque?

  Why not use that to secure a brand-new type of currency similar to Bitcoin, but with the properties of Ethereum? And we sell the currency and get supercomputers for health and autism and cancer and other things like that. And the more we help people, the more money-ness it has. The more we get these distributed nodes running that, the more secure it is, because the amount of flop securing the network increases. This comes at a time of deregulation in the US and massive amount of digital assets, because almost all of them are rubbish and full of raccoons. But you still have Bitcoin ETFs, and even nations thinking about Bitcoin sovereign reserves. But for me, the sovereign reserve thing never made sense, because that was always a balance of payments or commodities. I don't have enough oil, I don't have enough corn in my granaries, I don't have enough intelligence.

• 1:25:14Emad Mostaque?

  So I was like, if I could build that as an institutional asset, and again, we'll finalize the details of that release in the near year, that could be something interesting whereby people could buy it, they could hold it, it's secured by increasing amounts of compute, and then you can allocate it, so you can take your coins and allocate them to cancer research, or Alzheimer's, or autism, or Mexican education. And then we give that universal basic AI, or the research grants, or others. That plays on this transition period, it plays on this structural demand growth for digital assets that are decent, of which there aren't really any. And it's a finite amount of capital that's required, not cheap, so hundreds of millions

• 1:25:58Emad Mostaque?

  or billions. That means that everyone will never be alone again on their cancer journey. Yeah, that means that every child will have an AI that allows them to achieve their potential. And I was like, that's a good deal. And so we should give this a go, because I can't think of any other way to get people participating. I couldn't think of another way to get this technology out there without turning evil, as it were. Because, again, you want to have a line model whereby helping people, you gain value. And it seems there's a big demand for it, but like I said, putting the final touches, that's why the primer was like, here's the goal, and that's that, let's release details of how we get there and get people involved on that.

• 1:26:37Nathan Labenz?

  In terms of the incentive you want to create, if the mechanism is as yet not fully defined, is the idea supposed to be kind of similar to a Bitcoin where I'm like buying in now in anticipation of some sort of like compute appreciation in the future? Or like, what is my individual or even national incentive to get involved?

• 1:27:04Emad Mostaque?

  It's Bitcoin, except for usually what happens, like we're actually thinking of just copying Bitcoin straight, but with some advancements. But rather than mining Bitcoin and then doing sales, because you want provision compute for open source effectively, and you want to use it for positive stuff, be it inference to provide universal basic AI or building these data sets and models. That's why I'm like advisory board of Render and others where there's millions of GPUs. Rather than selling it and it going to a treasury or going to Lamborghinis, you sell coins and then we will build the fastest healthcare super cluster in the world and make it available to everyone working in healthcare to build open models and release them MIT. The more you build, the more you help, the more status you get.

• 1:27:48Emad Mostaque?

  And like I said, there's no high status digital assets right now. Then what will happen, we believe is that as you build the full stack, a nation can buy the reserves and then you put the money back into the nation to put a cluster in that nation to provide this. And the economics works well. People can donate and they get the tax write-off, but then they control the direction of voting of the coins. So working through this, but we're like, this could be a way that you could allow everyone to participate, they can direct the network compute for something that has defined amount of things. Let's just at the moment follow the Bitcoin mechanism, so 21 million coins and everything. And that would be very positive because what we need is a lot of compute right now to build the models and data sets.

• 1:28:31Emad Mostaque?

  And then we have an inevitability of compute for healthcare, education, all these other things, plus verified accounts.

• 1:28:40Emad Mostaque?

  That's a very interesting setup. And again, it's in this particular area that we don't see anyone else in AI doing. But it's like, we should have as soon as possible, nobody being alone in their cancer journey anymore. Oh, and when my son was diagnosed with autism, the feeling of loss I had, that should not be there for anyone. And that's the type of thing people can get behind. And I think this is where crypto has succeeded and failed.

• 1:29:06Emad Mostaque?

  People forming communities and being able to participate, especially with the U S D regulation will be a big boon. Having some sort of traceability over funds not going to land those, but clusters that are very well defined, that community organizations and academics can use is going to be great. And then should there be any cost to everyone having that cancer AI? No, it should be free. Education should be free. And if we can externalize it with the desire of people to hold digital assets for appreciation or for status or for anything else, even better. So like I said, that's taken quite a few months to design. We're nearly there. The primer was the first bit and we'll be releasing more details of all of this soon. I think it's complimentary to what other people in the space are doing as well.

• 1:29:50Emad Mostaque?

  I think everyone will benefit from the data sets. Everyone will benefit from these things. And the final bit, like to them, very worried about what money looks like over the next five, 10, 20 years and how the economics of everything works when labor theory of productivity is just broken.

• 1:30:05Nathan Labenz?

  Yeah, there's that's a dramatically under theorized problem, I would say for sure.

• 1:30:10Emad Mostaque?

  I'm using O one to help me write the paper on it right now, actually, for the new year. It is crazy though, because you think about the Philippines. Why would anyone hire a call center worker again? And you think about physical embodiment, you think about laborers, like in 20 years, will a robot be a best plumber than any plumber? Yeah. And how much you pay for your plumber today? It's expensive when the pipes burst, right? That robot will be $2 an hour, but your own home robot will have all the knowledge of an expert plumber. Probably 10 years away, to be honest.

• 1:30:48Nathan Labenz?

  So I would participate in this network for multiple reasons. One, and it's funny because I don't still really understand Bitcoin, I have to say. It continues to kind of confound me that it just runs up and up and up. You know, it doesn't seem like there's anything really there still yet, other than future appreciation. And that's like a hard trick to replicate. So the incentives that you outwind are, one, contributing to the public uncommon good. Two,

• 1:31:21Nathan Labenz?

  some sort of voting control over how pooled resources are allocated.

• 1:31:26Emad Mostaque?

  Yeah.

• 1:31:27Nathan Labenz?

  And third is appreciation, although I have to confess, I'm not quite sure. Like, is there a redemption mechanism that?

• 1:31:34Emad Mostaque?

  No, it's a bit, at the moment, with the way we designed it, it is like Bitcoin. So it appreciates with certain reasons Bitcoin. Bitcoin is a hedge against the world going to crap. This is a hedge against the intelligence age disruptive economics. And the amount of flops per coin is likely to outpace that amount of flops per dollar. Because it will be provisioning more and more and more compute. You know, so that's kind of the theory. We'll see what it's like in practice. And we've still got a lot of improvement to go here. But it does seem like, again, we're at this transition phase whereby something like this is unequivocally good, especially when you make it in those real terms. Like, again, is any podcast listener right now not want to have all the climate knowledge in the world organized and available to everyone? Or that cancer AI available to everyone?

• 1:32:18Emad Mostaque?

  And if they can contribute to that and have participation in that with the upside and with the US administration changing to more positive, even better. A hundred billion dollars has gone into Bitcoin ETFs this year.

• 1:32:30Emad Mostaque?

  Again, order of magnitude to build these things, it's like maybe a billion or something like that. Which sounds crazy just to toss around, but you've seen what AI funding rounds are right now.

• 1:32:39Nathan Labenz?

  Yeah, billion is very, I mean, at the level of countries, it's obviously extremely affordable. So.

• 1:32:48Emad Mostaque?

  So the level of VCs, you know, and again, the wonderful thing about this is, once a model is trained once, once we get to that level of performance with that cancer model to help everyone in all these languages, you don't need to train it again.

• 1:33:02Emad Mostaque?

  And I don't think we've ever seen anything like that, right? It's like having infinitely replicable graduates or chefs, you know, or cooks, sorry.

• 1:33:11Nathan Labenz?

  What technology, aside from continuing to define the protocol itself, what technology is needed for this to work? One space I've been watching really closely is distributed training. And it seems like we've just seen like several different papers and proof points that show that that is going to work, at least with like multiple large-ish nodes, not necessarily clear if it works at the level of like everybody contributing their spare laptop compute long term. But I'd love to hear your thoughts about sort of distributed compute and maybe any other technologies that you see as necessary that don't yet exist.

• 1:33:55Emad Mostaque?

  I think decentralized training doesn't really make that much sense versus decentralized tuning. Having armies of agents improve the data sets, but D-Logo, Distro and others have shown really interesting kind of advances. And again, like I said, I'm on the advisory board of some of these and the crypto XAI space is now like $40 billion. You look at something like BitTensor, that's a $12 billion network that's trading hundreds of millions of dollars a day for optimization. It's like the most on steroids Kaggle you can see. You have millions of GPUs in the other network available. I think again, the data side will be more interesting. And again, this sequential stuff as we get O1 type models that can run on my M4 Max or now M2 Max, soon to be M4 Max, looking forward to it. Nanotext display.

• 1:34:41Emad Mostaque?

  The probably one area that is probably most important to properly decentralizing this and ownership and control and governance is verifiable inference. And again, we're making some big advances in there. Hyperbolic and Alora and others are doing a lot of work around that so that then anyone can contribute their compute. But the data equation of this and building gold standard data sets is more important than the pre-training. For a pre-training, just have a cluster and just train on that cluster. Like, it's not rocket science. You know how to do it. Like, will it be a Llama-type model versus, or transformer model versus something hybrid Jamba-style? Who knows, right? But just make your pick. It's pretty straightforward to pre-train. Host training and optimization distributed stuff becomes very interesting there.

• 1:35:26Emad Mostaque?

  The verifiable inference allows anyone to contribute their compute. And then it's about having secure computation for running these things in regulated industries. And that's some of the TE work that we've seen. But most of it's coming together now, and especially, like I said, against the context of two other things, which were large context windows and function calling, like making it more deterministic on the outputs. And the final thing we just need is maybe continuous training for the individualized stuff. But again, we've seen big advances in that.

• 1:35:55Emad Mostaque?

  So I continue slightly.

• 1:35:58Nathan Labenz?

  So on the verifiable inference, I did one, and again, I'm weak on crypto. I did one episode on this with Professor Daniel Kang a little more than a year ago now, I think. And he sort of framed the problem as like, if you are paying for an API, you might want the provider, whether it's OpenAI or whoever, to have some way of guaranteeing to you that they are actually running the model that they said they're running as opposed to subbing in some cheaper knockoff or whatever and changing things underneath you.

• 1:36:33Nathan Labenz?

  I get that. I'm a little less clear on what you have in mind when you imagine anybody contributing compute through verifiable inference is the idea that like, I run inference locally and then I send up my activations and the gradients are sort of computed centrally or something like that?

• 1:36:51Emad Mostaque?

  Yeah, like, so I think distributed training on millions of GPUs doesn't make sense. But sending a packet to you that contains Indonesian classical architectural law and checking that for consistency and rewriting some of the data there does make sense. Putting it through a Llama 70B or 80B model. So I think distributed data augmentation makes sense, distributed fine tuning makes sense. And then comparing those things. And then the mining operations as well. But I don't think distributed training makes sense, or pre-training, shall we say. So post-training and data augmentation. And for those, you really want to have verifiable, you know what the models are. Otherwise, you might get junkiness, as it were.

• 1:37:32Nathan Labenz?

  Yeah.

• 1:37:33Emad Mostaque?

  So I think that unlocks something. Again, it's not required, it just helps. Like, right now, what we're looking at is literal H one hundred clusters in various countries and how we can bootstrap that. Because they're inevitable. So you might as well put them there and use them for healthcare later, right? And then that gives you enough to then tune models in every country and spin up teams and national champions. And then they can decide what they want as part of this network, right? And H one hundreds are a lot easier to track than forty nineties or M four maxes, et cetera. But if you can access that, then, like you said, you'll be able to build the better models even better because you'll build better data sets even better and faster.

• 1:38:14Emad Mostaque?

  I think, again, this is not synthetic data, it's augmented, filtered, cleared data utilizing the L M's.

• 1:38:22Nathan Labenz?

  Yeah, that's quite interesting. And that's basically the answer to the sleeper agent question, too, I assume, which I was also wanting to ask. Like, how do we make sure that people are not poisoning the data set from any number of different perspectives?

• 1:38:37Emad Mostaque?

  You put a massive amount of compute and putting a fine tune over all of it, checking it for consistency, adapting it as appropriate. Like, again, you can afford to go over the top because then you have a gold standard base. It's again, like, you have a good curriculum for your kid. You feel comfortable about that or comfortable about a mechanism and a methodology. So I think, again, this is a different type of A I challenge to the other people looking at. But I think this is the important one because it's what affects our lives in a much more direct way, but a particularly category of model. And this is why as well, when I was looking at it, I was like, You can build tokens which are security tokens and you can build things that have utility and redemption mechanisms. Ultimately Bitcoin is the most successful currency is $2 trillion.

• 1:39:22Emad Mostaque?

  It's money-ness, it's a hedge, it's these. Why don't we replicate that for an AIH? And why don't we think about the currency for an AIH? But have this additional thing that I think is important of, I want to have people able to have some participation because many of the listeners here are like, how can I even get involved in this? You don't know how. But if you can say, I've got this and I've helped direct the compute towards cancer and it's helping people go through their journey on cancer, like giving them the chatbot, great. Or breakthrough down the cancer cluster, great.

• 1:39:53Nathan Labenz?

  So are you with the intelligent internet, the company?

• 1:39:58Nathan Labenz?

  How do you understand your role in this whole thing? Are you like raising capital to create that one central cluster that will do the pre-training and then distribute that out to the national hyper nodes? Or what is this?

• 1:40:12Emad Mostaque?

  Yeah, the plan is to basically the company to bootstrap this, get it going. And then we stack GPUs and we use them ourselves and give them away to build fully open source things, data sets, models and systems. And that will be a big enabler. We saw this at Stability where we gave away 20 million A100 hours and it led to all sorts of amazing things from OpenFol to RWKB to stable diffusion and more. People just need to have sometimes a bit of compute. So I think it's getting it going and then trying to make it distributed and decentralized within a few years and giving people control of their own AI. So every nation, who should decide the education data set for the people of India or Bangladesh or Pakistan? The people

• 1:40:56Emad Mostaque?

  of India or Bangladesh or Pakistan? And likewise, who should build the generalized data sets for everyone? It should eventually be some sort of multilateral foundation or something like that. But it does require a bit of bootstrapping. So we're a team of like 26 people. We haven't done a funding round properly yet. A lot of demand for whatever coin we do. But I think this is the right way. But again, we're finalizing all the details of that.

• 1:41:21Nathan Labenz?

  So you think you can get over these capital requirement thresholds with a coin type approach without needing to go to like a traditional equity market?

• 1:41:34Emad Mostaque?

  Yeah, the again crypto XAI is now forty billion dollars, trading hundreds of millions of dollars, billion of a billion dollars a day now, actually. We could just go that route. We might do a combined one where we do equity and coin than that. And I think that in doing that, you'd probably be the highest revenue generating AI company in the world because demand for high quality digital assets is only going up. Pricing power and APIs and SaaS is going down. And nobody's really looked at this. I mean, even for Elon and Sam Altman and others like, how difficult is it for them to raise billions of dollars? It's not because they've got the pedigree and things. We've built hundreds of state of the art models of different types, our team. And that's why many of them join me from stability.

• 1:42:19Emad Mostaque?

  Having a liquid market for a coin where you can sell coins and literally say, this is a cluster that will just look at Alzheimer's. I think there'll be no shortage of people who want that. I think having the sovereign reserve aspect to this, build up this reserve and we'll put all the money into AI for your nation and get the teams and that. I don't think there'll be any shortage of that. Again, time will tell, but initial things have been very positive. And it was again, something I was thinking about when I left at the end of March. I was like, this feels more close to what I originally wanted to do. It's needed. No one else is doing it. And it's the right time with where regulation, other things are. Because ultimately crypto is a three trillion dollar industry

• 1:43:01Emad Mostaque?

  that needs high quality digital assets that people can allocate to. And we need to have some sort of intelligent money and these other things. But it's not easy, you know, like we're trying something new, but it keeps it exciting.

• 1:43:16Nathan Labenz?

  It's never a dull moment in the AI space, that's for sure.

• 1:43:19Emad Mostaque?

  Yeah.

• 1:43:20Nathan Labenz?

  You mentioned a number of projects from Stability and previously you made a comment along the lines of finding talent is like the hardest thing or is often a limiting factor. Obviously Stability was in the news a lot for lots of different stories over the year or so before you left. When people would send those links to me, I always basically said back to them, I just watched the research output and Stability has continued to matter, certainly as of the time of those conversations, with just like a bunch of really interesting, good projects, some of which were extremely catalytic, like stable diffusion in terms of changing awareness and what people understood to be possible.

• 1:44:05Nathan Labenz?

  Some were just like quiet contributions to the commons, like the Lion dataset and specifically the Aesthetic dataset, which is one that there was nothing else really like that and that made a huge difference actually for one of the little projects I was doing at my company, Waymark. I also love the MindEye and MindEye 2 papers, even though those are kind of, it's not easy to, or not hard to imagine a dystopian version of that in the future as well, but the fact that mind reading works in today's world is pretty crazy. What would you say are your tips, tricks, heuristics, methods for identifying great researchers?

• 1:44:44Nathan Labenz?

  Like, how did you assemble this team and create that capability, which I would say not that many different companies around the world have done, basically from scratch in a short period of time, without like certainly Microsoft level resources? What do you look for? What do you see that tells you somebody is going to make a difference?

• 1:45:07Emad Mostaque?

  I think it was kind of treating the researchers like creatives. And so we first created the communities. And again, you know, did a bit of hype, maybe overhyped sometimes. You live and you land, but it was a crazy time. And so we had about half a million people in the various communities that we kind of built, incubated, et cetera. We made it easy to operate and we gave very fast grants to anyone that was doing promising research. So I basically kind of led and coordinated the research and I put a good eye for that type of thing. But I really looked for people that it didn't matter what the background was. Are they passionate and are they willing to take a risk and try different things? So many of the research grants we gave, and again, researchers can feed back. We're like, try and do different things. It doesn't matter if it fails. Whereas researchers view it as a cost center generally. So you can't really take a risk.

• 1:45:52Emad Mostaque?

  You know, when it worked, we put on the afterburners and we scaled it. And then you wanted to empower the teams. So I think last year we had something like 20,000 applicants for researchers. We made 120 offers and 83%, 100 offers, 83% were accepted. And before the last, in my entire time at Stability, not a single researcher left for a competitor. And they were offered a lot of money because they enjoyed working there because they had the creative freedom, the adaptability. Obviously, they were concerned about all the noise and that's very interesting things behind that. But they just got on with things because we enabled them. And we said, like, they're like, this training run has failed. Like, doesn't matter so much whatever you learned. How can you go from there?

• 1:46:36Emad Mostaque?

  And we ended up by achieving state of the art in image, video, 3D, audio, protein folding, like small language, a whole bunch of different things by having that mentality because you don't need that many people. And you don't even need that many PhDs. We had 14 PhDs out of the 80. I think things like Fast AI and others are also great kind of grounds. And again, the willingness to try these things, this where communities like Aluca, which we incubated and spun out into a 501(c)(3) and Lion were great because you had all these diverse talents kind of coming in.

• 1:47:06Emad Mostaque?

  I think it depends what you want to do, though. Like, we were building state of the art because we were like, we need to catch up. But I was like, I can't scale LMS right now because there's no business model for that. I thought video and other things are the way. But again, then I realized that like right now, video is about to satisfy some, there'll only be like, it doesn't matter which one you use. It was very difficult to figure out the sustainability. So instead, we were just like, let's do the research and let's enable them. And I think the teams are happy. The rest of the organization maybe didn't do quite so well. It wasn't a great C year, but at the very least, the researchers were happy, which again, I think the proof is in the pudding. When it came towards the end of it, we spun out the Black Forest Labs team and some of the other teams.

• 1:47:48Emad Mostaque?

  And I think, again, they were happy and I think having researcher happiness at the top was very important.

• 1:47:55Nathan Labenz?

  It's certainly something you see coming through in the Sam and Elon emails as well. It's just the incredible intense focus on top talent.

• 1:48:06Emad Mostaque?

  Yeah, and I think, you know, what type of top talent, what are you trying to do? So we found the ones in the larger organizations didn't tend to fit in very well versus, again, the mavericks. Because we were trying to push state of the art and try different things. There are so many projects that didn't emerge because we made mistakes, not a mistake, mistakes, they just didn't work. So stable diffusion was 250,000 A100 hours to train the original. It was maybe 5 million in total of tests or something like that and failed experiments, et cetera. So you never see all those, see the other side of things. And again, what are you trying to do? Like, we had a bit of a crisis to build because like, are we trying to build AGI? Do we need to compete against all that? That was something I was thinking a lot about last year, like, are we competing against DeepMind?

• 1:48:50Emad Mostaque?

  Because we, again, one route is what you've seen with David and MidJenny. You know, and we gave the grant to help Jenny get going and then he did everything else himself. I don't think he's taken money from anyone else. Where he's like, I'm building a research and innovation lab and making sustainable things visible. And we could have just focused on media, which is what Stability is doing now. And given our role, that would have been far lower and cheaper to do. Good on that. We were like, we had to compete against DeepMind and OpenAI and others. And maybe we would have succeeded or not, but then it became very confusing to do that. And I think the one thing that is important if you're trying to build a great team in AI

• 1:49:30Emad Mostaque?

  is give a very defined function and then build teams that are passionate about that function. And then give them the resources they need to do that. And then that will work very well. That's probably the final missing piece, which was the clarity of thought, which is why I took a bit more time on the intelligent internet. Now it's very straightforward. If you want to have the biggest impact on autism, as a researcher, you will come and join me. And we'll have a dedicated team to autism, building an open infrastructure for autism, for government, for cancer and others. And so that's why we're setting up this time. And then the monetization mechanism is very straightforward. And then hopefully lots of GPUs will be coming their way to make them even happier. But you don't really need that many, to be honest, compared to what many people say to have an impact.

• 1:50:14Nathan Labenz?

  Couple maybe last big picture questions, and I really appreciate how generous you've been with your time. And it's, it's been great to get a deeper look at your thinking as you spin up the intelligent internet. You mentioned that you had signed the pause letter in like mid 23 when that came out, maybe right after GPT-4, the original one. I sometimes call myself an adoption accelerationist, hyperscaling pauser. I don't spend too much time advocating for a pause because I do still think there's like some room at least to advance the state of the art before it gets like too crazy. And, you know, I do want those cancer cures, of course. Where are you now on?

• 1:50:58Nathan Labenz?

  If you see the sort of what people really need to live better lives, either already or soon, decoupling from what is going on to advance the state of the art, where does that leave you in terms of? Should governments enforce a pause? Obviously, there's a collective action problem there too, if they wanted to. But what do you think people should do? Or what should the goal be for that sort of thing?

• 1:51:25Emad Mostaque?

  So when I signed that letter last year, I think I was the only AI CEO to sign it, actually. Elon signed it, but he wasn't AI CEO yet. I did it because I wanted, like, my base assumption was that if we got to AGI and the white thinking, then it would be like Scarlett Johansson in that movie, Her. You know, goodbye and thanks to all the GPUs, because humans are kind of boring. That's the kind of narrow AGI thinking. The reason I signed it was I was like, there needs to be more discussion about this. Just like right now, the amount of discussion about labor displacement and massive economic turmoil that will be caused by this technology is like nothing.

• 1:51:59Emad Mostaque?

  Like, talk to talk, they're not even thinking about it. It's insane. Again, like, for example, the Filipino call center sector is dead. Why would you hire another call center worker? You look at swathes of industry and Devon and these other things like, that's gonna be bad. Now where I am is that it's too late to have a thought, which is again why I said back then, let's get this out. And maybe we moved too far towards AI doomerism, killerism, and things like that. But we do have to, again, be very measured in the way we think about things. An example of this is copyright law and IP. So, you know, I got sued for 1.6 trillion dollars at Stability by Getty. Even though we didn't use any of their images in the thing and we felt that all the watermarks and stuff.

• 1:52:41Emad Mostaque?

  And in fact, the original stable diffusion was trained by Robin and his team before they joined Stability. So we made sure to have an arm's length. We didn't even look at anything. That's kind of deliberate.

• 1:52:51Emad Mostaque?

  Today, in the UK, the UK government is advocating that you can, they'll have an exemption where AI companies can train on any copyrighted material you can see.

• 1:53:01Emad Mostaque?

  You know, and that's similar to Singapore. If you can legally access it, you can train on it. Which kind of makes sense, because when you have a Tesla Optimus robot, is it gonna close its eyes and its ears when it sees a copyrighted thing? Like, oh no, I'm hearing Taylor Swift. Let's turn off my ears. Probably not gonna happen, right? But these are complex and nuanced topics, just like pausing and others. So I was like, that's the last chance we have to do that. Now, the biggest companies in the world are AI companies. Microsoft, Google, Amazon are AI-first companies. Jeff Bezos is back at Amazon. You know, Sergey Brin is back at Google. You know, Bill Gates, I don't know if he's back at, he's back at Microsoft, I think, as well.

• 1:53:37Emad Mostaque?

  Governments are trying to build their internal capital stock. They realize this is the biggest thing. You can't stop it now. You can't take time to think before the next wave of disruption occurs. And I don't think that governments really can afford to regulate like Europe is falling into its trap, right? But again, people will just build models and have extra competitiveness where they can. Again, Singapore is a great example that the UK may follow because you can't afford to be left behind. You can't afford not to build the AI chips. You can't afford not to support. Again, I would use Europe as the maybe one exception to that. And the final part of that is America under the new administration.

• 1:54:19Emad Mostaque?

  Crypto and AI are going to go full pelt under the new administration. There's no way it won't. And you have to keep up with America. How else are you going to kind of manage it? And so it's going to be very interesting to see how that plays out.

• 1:54:33Nathan Labenz?

  So if it's not a pause anymore, is there any other

• 1:54:40Nathan Labenz?

  regulatory action that you would recommend? I mean, people are groping or grasping for anything they can get their hands on. Liability rules come up a lot. That was kind of at the by the end, maybe not so much the heart of the SB 1047 debate, but at one point it was more central to that idea was maybe we'll hold the foundation model developers accountable. But then geez, how does that impact open source? And how does that impact the relationship with local fine tuners and developers? Any thoughts on sort of who should bear responsibility for incidents of AI gone wrong?

• 1:55:20Emad Mostaque?

  Yeah, it's very interesting to see all these things evolve. Like I've had some similar discussions to what Marc Andreessen said about how many AI companies can succeed, which is another reason to go decentralized after that experience. Gosh, there was a lot of very strange ones. Liability, these things won't work because again, you're in a competitive environment, which from a game theoretic perspective means that you can't afford to lose competitiveness over a defined period of time, especially as the Overton window has shifted on politics now to kind of be more techno friendly after literally the US government was trying to strangle the sector or make it so only a few people could do it. I think that the biggest impact simplest thing is this. If AI is used in a decision making process in any regulated industry, it has to have transparent data logged.

• 1:56:07Emad Mostaque?

  Because you are what you eat. These models are ultimately the data. Like, if you look at all these different architectures, again, we funded RWKV getting off the ground, or Mamba, or any of these, all the performances are the same for the same dataset. It comes down to data. You are what you eat. And the decision-making AIs are the ones that will kill us, or make us successful ultimately. The rest is kind of helpful. And so introducing a level of data transparency over that is important because then, like, people will spot, just like when we released Stable Diffusion, and there was Lion, they were like, oh, there's weird shit in here. Like, yeah, we don't want weird shit.

• 1:56:45Emad Mostaque?

  If you don't want to know about nuclear bioweapons, don't have nuclear bioweapons in the data sets. I think the other thing is building good standardized data sets for knowledge and other things. Like, again, you said you can't go and achieve at Alan AI, the level of scale AI, and other things like that. It's kind of embedded in these models from GPT-4 and others. Building a gold standard of data sets for common knowledge and cultural knowledge just makes everything safer, because why wouldn't you use those as your base? And again, think about models that the creator and the owner's perspective was going through high school, university, and then being specialized. Like, what are we really training when we've got trillions of tokens? What's in there? Have you looked in these data sets? They're crap.

• 1:57:31Emad Mostaque?

  Build better base data sets that you can iterate on, and then things will be safer. Have a simple legislation of any decision-making AI in any regulated industry needs to have transparent data, and things will be safer. Like, doesn't necessarily need to be interpretable, but we know that the AI functionality is a derivation of the base data. Beyond that, I can't see what other regulations will stick, honestly. But one that I would highly recommend is regulations around speech data. Like, we built a state of the art speech, well, we did not release. It is dangerous. I don't think people appreciate how dangerous it is, but let me give a practical example to people listening here. Imagine someone that you cared about more than anyone, you respected more than anyone. Replicate their voice and create a version that you can call them,

• 1:58:17Emad Mostaque?

  and that you can have discussions on Zoom with them, which is all capable now. How much would he listen to that AI versus a normal AI? More than anything.

• 1:58:25Emad Mostaque?

  And if that AI can then access Obama's speech patterns and Churchill and other things, that wave of manipulative speech AI is coming. We don't have the protections against that. And I think those should be legislated heavily and regulated heavily, especially when you think about our kids. Our kids literally have no defenses against that. So those are probably the areas that I'd look at.

• 1:58:47Nathan Labenz?

  One other random idea I've had and I haven't really developed too much is speed limits. And this is basically just the idea that like, maybe individuals or organizations or whatever should sort of only be able to do so much inference per unit time or only use so many flops per unit time. And it's kind of like, just to keep things, you know, operating at a sort of human speed. Maybe we need something like that. You're probably gonna have the same game theoretic objection, though, I suppose.

• 1:59:18Emad Mostaque?

  If you can't do that, it's like saying you can only hire so many people, right? Like, again, you will lose competitiveness in this increasingly competitive environment. And organizations others will come out of it. And what is the role of government, right? I mean, someone said it's the entity with the monopoly on political violence. Ultimately, it's about protection of people and representation of people. And if this is the big leap up, you have people with AI outperforming people without AI. You can't really have these things, except for when there's unions and others involved. So you look at your rate limits and you look at the long form and strike in the US. That is a rate limit. They will only allow a certain amount of robots. Is that optimal for a society, though? No.

• 2:00:03Emad Mostaque?

  It just so happens they have a physical barrier. And so we'll see those things maybe emerge in certain areas. We regulate on the amount of AI that you can have. But if you're keeping a free market, which again, unions are counter to in some ways, then you can't have that. Because who knows what goes on inside an organization, right? And other things. And can you coordinate this globally? This is the bigger thing? Absolutely not. Like, again, you think about small nations, they can suddenly outcompete. You look at the role of capital here, like the Saudis and others of the world, they're like, once this stack is built, like, that's why D U A has gone heavily into cerebrus.

• 2:00:46Emad Mostaque?

  Like, yeah, build those silicon wafers. So I think that ultimately.

• 2:00:52Emad Mostaque?

  You can just set good defaults. And you can use market forces and others for that. And again, push for data transparency in some of these edge cases like speech. You do need some protection because I haven't figured out like unless we wear AirPods all the time. How do you protect against that?

• 2:01:09Nathan Labenz?

  Yeah. Yeah, I mean, it seems like there's probably going to be more of those things too popping up that we don't even necessarily have any precedent for. It's hard to say, of course, what they are, but I certainly expect a lot more weird stuff to start to happen.

• 2:01:25Emad Mostaque?

  We live in interesting times, eh?

• 2:01:27Nathan Labenz?

  Yeah, no doubt. When you talk about dataset filtering, if you don't want the models to know about bionuclear weapons, as you put it, you know, don't have that in the training data. Is that sort of akin to your earlier comment that you think scaling is going to kind of level off? Or is there, there seems to be a relation there where it's like if they generalize out of domain to a certain degree, then they'll fill in those gaps is kind of what a lot of people worry about. But you don't expect that as much?

• 2:01:59Emad Mostaque?

  Well, we embody these base models with a bit too much stuff, even with the SFT though, right? Ultimately, they're just a bunch of like freaking weights, a bunch of ones and zeros, right? It's not like a logical system. So you look at the composition of that, how it's built up and you're like, I want a base level of reasoning model. Like, it doesn't make much sense that we have all the data in these models and knowledge in these models either. Like, that seems a bit wasteful. Like, how much these models know about they don't need to know? And you think about, again, a model that's a medical doctor. Should it need any more data than a medical professional gets from kindergarten to medical school? No. I think Yalun Kun has talked about this in some of the JEPY kind of things. Can it infer and reason? Yes, because it'll have that capability.

• 2:02:41Emad Mostaque?

  But again, the question is, what are we releasing unto the world as infrastructure or services? Because you can control things at a service level, infrastructure level is a bit different. Like, what base model are we releasing, stable diffusion style? People use it for good things, they use it for bad things. But it was inevitable that you would have reasoning models like that, right? And again, this is especially important when it comes to long context window models. Like, again, 2 million, 10 million tokens is a freaking lot. And so a reasonable reasoner, he says, when Gemini 01 comes in Gemini 2, or the equivalent Lama version of that. Yeah, I'll be able to figure out just about anything. It's inevitable.

• 2:03:25Emad Mostaque?

  But in day to day, how many people are going to load it with all the bionuclear stuff in the context window? And that's something that is relatively controllable. So I think that we just need to tidy up the stuff in the base models. And that's just not even a question of filtering. It's a question of how do we feed them? Like, what is the curriculum? And then, once you have goal standard version of that, I think things become safer because there's less likelihood of them going initially rogue because there'll be a function of the data goes in. Now, once you make them agentic and able to traverse the internet, a full of rubbish that it is, then maybe it's a different story. But again, this is an incremental thing. There are no easy answers, otherwise someone would have answered already.

• 2:04:06Nathan Labenz?

  It does. Your basic vision on the safety side is structured access to state of the art combined with open data, open weights, open access, satisficing day to day usage models?

• 2:04:24Emad Mostaque?

  Yes, from a functional perspective, universal basic AI for everyone that gets smarter and smarter, and we all agree on that can be localized and specialized with the power going back to the people. Then personal AI will be driven by market forces with your Apple and Google and other intelligences. And then you will have specialized, highly intelligent models. That's where regulation can come in, access can come in for the most capable models out there. But I think it'll be very difficult. So instead, I think what we should do is focus on if they're in decision making positions, at least have the data transbound.

• 2:04:59Nathan Labenz?

  That could be a great note to end on. Is there anything we didn't touch on or anything else you want to share about the intelligent internet before we break for today?

• 2:05:08Emad Mostaque?

  No, we haven't really pushed it out there. Have a read. I think it's a nice document, along with How to Think About AI. We'll have a whole bunch of stuff in the new year. And again, happy to see feedbacks, iterative process. But, you know, it's been a lot of fun.

• 2:05:20Nathan Labenz?

  All right. Well, we'll certainly keep watching and maybe I'll be lucky enough to have a chance for another conversation once you finalized all those details and put that out there. For now, Ahmad Bastak, founder of the Intelligent Internet. Thank you for being part of the cognitive revolution.