Who Will Govern the Future of AGI? w/ Emad Mostaque (Stability AI Founder) | X Spaces

Amidst all the recent Tech news, Peter and Emad hopped on Twitter (X) Spaces to discuss AGI governance, the future of AI, open-source models, and more. Emad Mostaque is the CEO and Founder of Stability AI, a company funding the development of open-source music- and image-generating systems such as Dance Diffusion and Stable Diffusion. Get started on Stability AI Click here to sign up for the launch of the $101M XPRIZE – the largest in history. ____________ I send weekly emails with the latest insights and trends on today’s and tomorrow’s exponential technologies. Stay ahead of the curve, and sign up now: Tech Blog My new book with Salim Ismail, Exponential Organizations 2.0: The New Playbook for 10x Growth and Impact, is now available on Amazon: https://bit.ly/3P3j54J Learn more about my executive summit, Abundance360 _____________ Connect With Peter: Twitter Instagram Youtube Moonshots Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Flowing ad budget on metrics that look great till the CFO sees them. That's bull spend and marketers are calling it out in dashboard confessions.

• 0:09Speaker 01

  I remember telling my boss, it'll be good for the brand when leads were slow. Yeah, it it wasn't.

• 0:17Speaker 00

  Cut the bull spend. LinkedIn lets you target by company, job title and more. Advertise on LinkedIn. Spend $250 on your first campaign and get a $250 credit. Go to LinkedIn.com slash campaign. Terms and conditions apply.

• 0:30Speaker 02

  College football is back. So Hilton called to me, the superstition concierge to make your fan rituals a reality. Need a room to match your lucky number? We got you. Want to make sure our team doesn't wash your lucky jersey? Oh, that smells lucky. Hilton's unmatched hospitality can keep up with any superstition. Even a marching band wake up call at 5:55 in 55 seconds. Hit it! When you need a team that will do whatever it takes on game day, it matters where you stay. Hilton, for the stay.

• 1:10Peter Diamandis?

  Hey, Amad, good to hear you.

• 1:13Speaker 04

  It was always a pleasure.

• 1:13Peter Diamandis?

  Yeah. So where are you today?

• 1:16Speaker 04

  I'm in London.

• 1:18Peter Diamandis?

  Good, other side of the planet. I'm in Santa Monica. It's it's been quite the extraordinary game of ping pong out there these last four or five days.

• 1:31Speaker 04

  I was like, I didn't think the first thing that AI would disrupt would be reality TV, right?

• 1:36Peter Diamandis?

  Yeah.

• 1:38Peter Diamandis?

  Yeah, it's been it's been fascinating how how how X has become sort of the go to place to find out the latest of where Sam is working and what's going on with the AI industry. So.

• 1:53Speaker 04

  You found notifications in the wake. I mean, that's the thing, like, what else will move at the speed of this? Like I was saying to someone recently, AI research doesn't really move the speed of conferences or even PDFs anymore, right? You just wake up, you're like, oh, it's 10 times faster. So I think that's why X is quite good. I actually like I'm following just about everyone. I just let the AI algorithms find the most interesting things for me. So I've got like 10 people I follow and it's actually working really well. It's getting better.

• 2:22Peter Diamandis?

  Well.

• 2:24Peter Diamandis?

  It has been. I've been enjoying enjoying the conversation. It really feels like you're inside a intimate conversation among friends as this is going back and forth. I think this entire four or five days has been an extraordinary, up-close intimate conversation around governance and around, you know, what's the future of AI because honestly, you know, as it gets faster and more powerful, the cost of missteps is going to increase exponentially. Let's let's begin here. I mean, you've been making the argument about open source as

• 3:06Peter Diamandis?

  one of the most critical elements of governance for a while now. Can you just let's hop into that.

• 3:12Speaker 04

  Yeah, I think that, you know, open source is the difficult one because it means a few different things. Like, is it models you can download and use? Do you make all the data available and free? And then when you actually look at what all these big companies do, all their stuff is built on open source basis. You know, it's built on the transformer paper. It's built on like the new model by Caifu Lee and Vera1.ai is basically Llama. Like, it's actually got the same variable names and other things like that, plus a gigantic supercomputer, right?

• 3:46Peter Diamandis?

  The whole conversation has been, you know, how important is openness and transparency? And and what are the governance models that are going to allow the most powerful technology on the planet to enable the most benefit for humanity and the safety? So, I mean, you've been thinking about this and and speaking to transparency, openness, governance for a while. And could you, I mean, what do you think is is going to be, what do you think is we need to be focused on? Where do we need to evolve to?

• 4:21Speaker 04

  Um, it's a complicated topic. I think that most of the infrastructure of the internet is open source, Linux, everything like that. I think these models, um, it's unlikely that our governments will be run on G P T seven. Or Bard or anything like that. How are you going to have black boxes that run these things? I think a lot of the governance debate has been hijacked by the A I safety debate. Where people are talking about A G I killing us all. And then there's this precautionary principle that kicks in. It's too dangerous to let out because what if China gets it? What if someone builds an A G I that kills us all? It'd be great to have this amazing board that could pull the off switch, you know?

• 5:00Speaker 04

  Um, whereas in reality, I think that you're seeing a real social impact from this technology. And it's about who advances forward and who's left behind if we're thinking about risk. Because governance is always about finding, as you said, the best outcomes and also mitigating against the harms, right? And there's some very real amazingly positive outcomes that are now emerging that people can agree on. But also some very real social impacts that we have to mitigate against.

• 5:30Peter Diamandis?

  So, I mean, let's begin. How is, uh, how is stability governed?

• 5:38Speaker 04

  Uh, stability is basically governed by me. And, um, so I looked in foundations and DAOs and everything like that. And I thought to take it to where we are now, it needed to have very singular governance. But now we're looking at other alternatives.

• 5:52Peter Diamandis?

  And what do you think it's gonna be? You know, where would you head in the future? I mean, uh, let's, let's actually jump away from this in particular. What do you recommend the most powerful technologies on the planet? How should they be governed? How should they be owned?

• 6:09Peter Diamandis?

  You know, where, where should we be in five years?

• 6:12Speaker 04

  I think they need to be public goods that are collectively owned and then individually owned as well. So, for example, there was the tweet kind of storm, the kind of I am Spartacus or his name is Robert Paulson in the OpenAI team, um, saying OpenAI is nothing without its people.

• 6:30Peter Diamandis?

  Mm hmm.

• 6:31Speaker 04

  Um, stability, we have amazing people, a hundred and ninety and sixty five top researchers. Without its people, we're open models used by hundreds of millions. It continues. And if you think about where you need to go, you can never have a choke point on this technology, I think, if it becomes part of your life. It's like the phrase I have is not your models, not your minds. So these models again are just such interesting things. Take billions of images or trillions of words and you get this file out that can do magic, right? Train your magic sand. Um, I think that you will have pilots that gather our global knowledge on various modalities. And you'll have co-pilots that you individually own that guide you through life.

• 7:15Speaker 04

  And I can't see how that can be controlled by any one organization.

• 7:19Peter Diamandis?

  You've been, you know, on record talking about having models uh owned by the citizens of nations. Uh, can you speak to that a little bit?

• 7:29Speaker 04

  Sure. So we just released some of the top Japanese models from visual language to language to uh Japanese SDXL as an example. So we're training for half a dozen different nations' models now. And the plan is uh to figure out a way to give ownership of these data sets and models back to the people of that nation. So you get the smartest people in Mexico to run a stability Mexico or maybe a different structure that then makes decisions for Mexicans with the Mexicans about the data and what goes in it. Because uh everyone's been focusing on the outputs, the inputs actually are the things that matter the most. Um, the best way I've thought about thinking of these models is like very enthusiastic graduates, so hallucinations are just prior to hard.

• 8:13Speaker 04

  A lot of the things about like, oh, what about these bad things the models can output? It's about what you input. And so what you put into that Mexican dataset, or the Chinese or Vietnamese one, will impact the outputs. And there's a great paper in Nature Human Behavior today about that, about how foundational models are cultural technologies. So again, how can you outsource your culture and your brains to other countries, to people that are from a different place? I think it eventually has to be localized.

• 8:45Peter Diamandis?

  Yeah, I I think one of the points you you said originally is we have to separate the issue of governance versus safety and alignment.

• 8:55Peter Diamandis?

  Are they, are they seri- are they actually different?

• 9:00Speaker 04

  So I think that a lot of the safety discussion, or this AGI risk discussion, is because the future is so uncertain because it is so powerful. Right? And we didn't have a good view of where we're going. So when you go on a journey and you don't know where you're going, you'll minimize for maximum regret, you'll have the precautionary principle. And then that means you basically go towards authority, you go towards trying to control this technology when it's so difficult to control. And you end up not doing much, you know, because anything could go wrong. Um, when you have an idea of where we're going, like you should have all the cancer knowledge in the world at your fingertips, or climate knowledge. Or anybody should be able to create whole worlds and share them.

• 9:44Speaker 04

  Then you align your safety discussions against the goal, against the location that you're going to. Again, just like setting out on a journey, I think that's a big change. Similarly, most of the safety discussion has been on outputs, not inputs. If you have a high quality dataset without knowledge about anthrax, your language model is unlikely. To tell you how to build anthrax, you know.

• 10:06Peter Diamandis?

  So I I think that's it.

• 10:07Speaker 04

  And transparency around that will be very useful.

• 10:10Peter Diamandis?

  So that's a, you know, let's dive into that safety alignment issue for a moment because it's an area you and I have been talking a lot about. So Mustafa wrote a a book, Mustafa Salimansh wrote a book called The Coming Coming Wave, in which he talks about containment as the mechanism by which we're going to be making sure we have safe AI. You and I have had the conversation of it's really how you educate and raise and train your AI systems and in giving, making sure that there's full transparency and openness on the data sets that are utilized. Do you think containment is an option for for safety?

• 10:50Speaker 04

  No, not at all. Like, a number of leaders say, what if China gets open source AI? The reality is that China, Russia, everyone already has the weights for GPT-4, because they just downloaded it on a USM stick.

• 11:06Speaker 04

  You know, you know that there's been compromised, right? There's no way they couldn't. The rewards are too great. And there is a absolutely false dichotomy here and thing. A lot of the companies want you to believe that giant models are the main thing and you need to have these gigantic ridiculous supercomputers that only they can run. I mean, we run gigantic supercomputers. But the reality is this. The supercomputers and the giant trillion zillion data sets are just a shortcut for bad quality data. It's like using a hot pot or sous vide-ing a steak that's bad quality. You cook it for longer and it organizes the information.

• 11:48Speaker 04

  With stable diffusion, we did a study and we showed that basically ninety-two percent of the data isn't used ninety-nine percent of the time.

• 11:57Speaker 04

  You know, because now you're seeing this with, for example, Microsoft's five release. It's trained entirely on synthetic data. Dali three is trained on our V A E and entirely synthetic data. You are what you eat. And again, we cooked it for longer to get past that. But the implications of this are that.

• 12:19Speaker 04

  I believe within twelve to eighteen months, you'll see G P T four level performance on a smartphone. How do you contain that? And how do you contain it when China can do distributed training at scale? And release open source models. So Google recently did fifty thousand T P U training run on their V five Es. The new V five Es, their T P Us are very low powered relative to what we've seen. But again, you can do distributed dynamic training. Similarly, like we funded Five Mind and we've seen Google Deep Mind just as a new paper on localization through distributed training. The models are good, fast, fast enough and cheap enough that you can swarm them. And you don't need to run supercomputers anymore.

• 13:03Speaker 04

  And that has a lot of implications. And how are you going to contain that?

• 13:08Peter Diamandis?

  So coming back to the question of do you mandate training, training sets? Do you, does, you know, does the government set out? Um, what all companies should be utilizing and mandate if you're going to have a aligned A I, it has to be trained on on these sets. How do we, how do we possibly govern that?

• 13:32Speaker 04

  Look, we have food standards, right, for ingredients. Why don't you have data standards for the ingredients that make up a model? It's just data compute and some algorithms, right? And so you should say they're the standards and then you can make it compulsory, that will take a while, or you can just have an I S O type standard. This is good quality model training, good quality data. You know, and people will naturally gravitate towards that and it becomes the default.

• 13:58Peter Diamandis?

  Are you working, are you, are you working towards that right now?

• 14:02Speaker 04

  Yeah, I mean, look, we spun out a Lethra AI as an independent five oh one C three, so they could look at data standards and things like that independently of us. Kind of the opposite of OpenAI. Um, and this is something I've been talking to many people about, and we're getting national data sets and more. So that hopefully we can implement good standards. Similar to how we offered opt out and had a billion images opted out of our image data set, because everyone was just training on everything. Is it required? No, but is it good? Yes. And everyone will benefit from better quality data. So there's no reason that for these very large model training runs, the data set should not be transparent and logged. Again, we want to know what goes into that. Again, if we have the graduate analogy. What was the curriculum that the graduate was taught at? Which university did they go to?

• 14:48Speaker 04

  It's something that we'd want to know. But then why do we talk to G P T four when we don't know where it went to university or where it's been trained on? It's a bit weird, that, isn't it?

• 14:57Peter Diamandis?

  What do you think the lesson's gonna be uh on from the last four days?

• 15:05Speaker 04

  I'm just confused. I don't know who was against who or what. I think I just posted, are we against Mr. Lin or Moloch?

• 15:12Speaker 04

  I think probably the biggest lesson is it's very hard to align humans. Right? And the stakes are very large. Like, why is this so interesting to us? Because stakes are so high.

• 15:21Peter Diamandis?

  You tweeted something that was, you know, serious and unfortunately funny, which was how can we align? AI with human humanity's best interest. We can't align our company's board with its employees' best interests.

• 15:35Speaker 04

  Yeah. Well, the thing is, it's not the employee's best interest. It's like, the board was set up as a lever to ensure the charter of OpenAI. So if you look at the original founding document of OpenAI from twenty fifteen. It is a beautiful document. Talking about open collaboration, everything, and then it kind of changed in twenty nineteen. But the charter still emphasizes cooperation, safety. And fundamentally, I posted about this back in March, when I said the board and the government structure of OpenAI is weird. Like, what is it for? What are they trying to do? Because if you say you're building AGI. In their own road to AGI, they say, this will end democracy, most likely.

• 16:19Peter Diamandis?

  I remember reading that.

• 16:20Speaker 04

  Because democracy. There's no way democracy survives AGI. Because either, obviously it'll be better and you get it to do it, or persuade everyone, or we all die. Or it's utopia forever, right?

• 16:33Peter Diamandis?

  Abundance, baby.

• 16:34Speaker 04

  There's no, yeah, well, this thing, it, it, then, but anyway, regardless, there's no way it survives AGI. There's no way capitalism survives AGI. The AGI will be the best trader in the world, right? And it's like, who should be making the decisions on the AGI? Right? Assuming that they achieve those things, and that's in their own words. So I think that people are kind of waking up to, oh, there's no real way to do this properly. And previously, we were scared of open and being transparent, everyone getting this, which kind of was the original thing of OpenAI. And now we're scared of, who are these clowns? You know? And put it in the nicest way, because this was ridiculous, like you see better politics in a teenage sorority, right?

• 17:22Speaker 04

  And it's fundamentally scary, but unelected people, no matter how great they are, and I think some of the evil numbers are great. Should have a say in something that could literally upend our entire society according to their own words.

• 17:38Speaker 04

  I find that inherently anti-democratic and a liberal.

• 17:44Peter Diamandis?

  At the end of the day, um, you know, capitalism has worked and it's the best system that we have thus far. And it's a self, you know, it's, it's built on self-interest and built on op- continuous optimization and maximization.

• 18:03Peter Diamandis?

  I'm still wondering where you go in terms of governing these companies at one level. Uh, internal governance and then governing the companies at a national and global level.

• 18:18Peter Diamandis?

  Has anybody put forward a plan that you think is, uh, is worth highlighting here?

• 18:25Speaker 04

  Not really. I mean, organizations are weird artificial intelligence, right? They have the status of people and they're slow dumb AI. And they eat our hopes and dreams, that's what they feed on, I think. But um, this AI can upgrade them, it can make them smarter. Again, how do you coordinate? And from a mechanism design perspective, it's super interesting. Like in markets, I think we will have AI market makers that can tell stories. Like the story of Silicon Valley Bank went around the world in two seconds, the story of OpenAI goes around. AI can tell better stories than humans, it's inevitable. I think that gives hope of coordination, but then also it's dangers of disruption.

• 19:03Peter Diamandis?

  Uh, let's, I want to double click one second on the two words that you use most, openness and transparency.

• 19:11Peter Diamandis?

  And understand fully what those mean one moment, because, you know, and the question is, not only what they mean, but uh, how fundamental they need to be. So openness right now in your definition in terms of AI means what?

• 19:27Speaker 04

  It means uh, different things for different things unfortunately. I don't think it means open source. I think uh, for me, open means more about access and ownership of the models. So that you don't have a lockstep, like you can hire your own graduates as opposed to really relying on consultants. Transparency comes down to, I think for language models in particular, I don't think this holds for media models. You really need to know what has been taught. That's the only way to safety. Like you should not engage with something or use something if you don't know what its credentials are and how it's been taught. Because I think that's inherently dangerous as these get more and more capabilities. And again, I don't know if we get to AGI, like, if we do, I think it'll probably be like Scarlett Johansson and her. You know, like just goodbye, thanks for the G.

• 20:12Speaker 04

  Assuming we don't, you still need transparency.

• 20:15Peter Diamandis?

  So that, so then.

• 20:16Speaker 04

  Again, how can any, how can any government or regulated industry not run on a transparent model? They can't run in black boxes.

• 20:24Peter Diamandis?

  Uh, I, I get that and I understand the rationale for it, but now the question is, can you prove transparency?

• 20:33Speaker 04

  I think that, again, a model is only three things, really. It's the data, the algorithm and the compute. And then they come and the binary file pops out. Then you can tune it with RLHF or DPO or genetic algorithms or whatever. But that's really the recipe, right? And so the algorithms, you don't need algorithmic transparency here versus classical AI because they're very simple. One of our fellows recreated the Palm five hundred and forty billion parameter model. This is Lucid Raines on GitHub. You would look at that if you're a developer and you want to cry, it's GitHub, it's crazy. In two hundred and six lines of PyTorch. And that's it. The algorithms are not very complicated. Running a gigantic supercomputer is complicated.

• 21:16Speaker 04

  And this is why they freaked out when Greg Brockman kind of stepped down because he's one of those talented engineers of our time. Built this amazing gigantic clusters. And then the data and how you structure data is complicated. So I think you can have transparency there. Because if the data is transparent, then who cares about the supercomputer? Who really cares about the algorithm? You know?

• 21:36Peter Diamandis?

  Now let's talk about the next term alignment here. Alignment's thrown around in in lots of different ways. How do you define alignment?

• 21:48Speaker 04

  I define alignment in terms of objective function. So YouTube was used by the extremists to serve ads for their nastiness, right? Why? Because the algorithm optimized for engagement, which then optimized for extreme content, which then optimized for the extremists. Did YouTube mean that? No. But they're just trying to serve ads up, right? But it meant it wasn't aligned with its users' interests. And so for me, if you have these technologies that we're going to outsource more of our mind, our culture, our children's futures to you that are very persuasive. We have to ensure they're aligned with our individual community and societal best interests.

• 22:31Speaker 04

  And I think this is where the tension with corporations will come in.

• 22:37Speaker 04

  Because whoever licenses Scarlett Johansson's voice will sell a lot of ads, you know, they can be very, very persuasive. But then what are their controls on that? No one talks about that. The bigger question of alignment is not killerism. Making sure the AI doesn't kill us. But again, I feel that if we build AI that is transparent, that we can test, that people can build mitigations around, we are more likely to survive and thrive. And also, I think there's a final element to here, which is whose alignment?

• 23:10Peter Diamandis?

  Yes.

• 23:11Speaker 04

  Different cultures are different, different people are different. What we found with stable diffusion is that when we merge together the models that different people around the world have built, the model gets so much better. I think that makes sense because a monoculture will always be less fragile than a diversity. Again, I'm not talking about in the DAI kind of way. I'm talking about it in the actual logical way. So we have a paper from our reinforcement learning lab called CARPA called QDHF, QDAIF, Quality and Diversity Through Artificial Intelligence Feedback. Because you find these models do get better with high quality and diverse inputs. Just like you will get better if you have high quality and diverse experiences.

• 23:54Speaker 04

  You know, and so that's something that's important, but we'll get lost if all these models are centralized.

• 24:00Peter Diamandis?

  You know, uh, we, you and I've had a lot of conversations about timelines here. Um, we can get into a conversation of when, uh, and if we see AGI. But we're seeing more and more powerful capabilities coming online right now that are going to cause a lot of amazing progress and, and disruption. Um, how much time do we have, Imad? And, and we've had, we had a conversation when we were together at FII about, um, uh, the disenfranchised youth coming off COVID. Uh, so let's talk one second about timelines. How long do we have to get our shit together?

• 24:44Peter Diamandis?

  Um, both as, as AI, uh, AI companies and, uh, and investors and governors of society.

• 24:57Peter Diamandis?

  Uh, we don't, I mean, the speed here is, is awesome, um, and frightening. How long do we have?

• 25:06Speaker 04

  Almost everything everywhere all at once, right? Uh, we don't have long, like AGI timelines for every definition of AGI, I have no idea. It will never be less than twelve months, right? Because it's such a step change. Let's put that to the side.

• 25:19Peter Diamandis?

  Okay.

• 25:21Speaker 04

  Right now, everyone that's listening. Are you all going to hire the same amount of graduates that you hired before? Answer is no. Some people might not hire any because this is a productivity enhancement. We have the data for that across any type of knowledge industry.

• 25:38Speaker 04

  You just had a great app that you can sketch and it does a whole iPhone app for you, right? I've got on record and saying there are no programmers who know it in five years. Why, why would there be? What type of interfaces? You had a fifty percent drop, I just posted on my Twitter, in hiring from Indian IITs. That's crazy. So what you're gonna have in a couple of years is around the world at the same time. These kids that have gone through the trauma of COVID. Highly educated STEM, programming, accountancy, law. Simultaneously, people will hire massively less of them because productivity enhances. And you don't need as many of them. Why would you need so many paralegals?

• 26:22Speaker 04

  And that for me is a gigantic societal issue. And the only thing I can think of is to stoke open innovation and the generative jobs of the future through open source technology. Um, because I don't know how else we're gonna mitigate that because, you know, Peter, you're a student of history. What happens when you have large amounts of intelligent disenfranchised youth?

• 26:41Peter Diamandis?

  We've had that happen a few times. We just had Arab Spring not long ago. Revolt, yeah. Yeah, si- si- civil war, if not, if not inter-, you know, international law.

• 26:52Speaker 04

  Oh, that's all. War is a good way to soak up the excess youth.

• 26:56Peter Diamandis?

  Yep.

• 26:57Speaker 04

  Um, but it's not pleasant. It's not pleasant for society. And fundamentally, the cost of information gathering, organization has collapsed, like, and you look at Stable Video that we released yesterday, right? It's gonna get so much better so quickly, just like Stable DevU, the cost of creating movies increases. The demand for quality stuff increases. But there's a few years where demand and supply don't match.

• 27:23Speaker 04

  And that's such a turbulent thing to navigate. That's one of the reasons I'm creating stabilities for different countries, so the best and brightest from each can help navigate.

• 27:31Peter Diamandis?

  I loved your

• 27:32Speaker 04

  And people don't talk about this.

• 27:33Peter Diamandis?

  I loved your idea that the um that the stability models and systems will be owned by the nation. In fact, one idea that I heard you say, which I thought was fantastic, was you graduate college in India, you're an owner in that uh in that system. You graduate from in Nigeria, you're an owner in that system. Basically to incentivize people to complete their education and to have them have ownership in what is ultimately the most important asset that nation has. And and and talk about it as infrastructure as well. I think that's an important analogy that people don't get.

• 28:10Speaker 04

  This is the knowledge infrastructure of the future. It's the biggest leap forward we have, because you'll always have a co-pilot that knows everything in a few years, and it can create anything in any type. But it must be embedded to your cultural values, and you can't let anyone else own that. So it is the infrastructure of the mind, and who would outsource the infrastructure to someone else? So that's why I think Nigerians should own the models of Nigerians for Nigerians, and it should be the next generation that does that, you know? That's why you give the equity to the graduates. That's why you list it. That's why you make national champions. Because, again, that has to be that way. This is far more important than 5G, and this gives you an idea of the scale. We're just at the start, the earlier adopt phase. A trillion dollars was spent on 5G. This

• 28:55Speaker 04

  is clearly more important. More than a trillion dollars have been spent on this, and again, it flips the world. And so there is huge threat for our societal balance. And again, I think open is a potential antidote to create the jobs of the future. And there's huge opportunity on the side, because no one will ever be alone. And we can use this to coordinate our systems, give everyone all the knowledge that they need at their fingertips, and help guide everyone if we build this infrastructure correctly. And again, I don't see the highlight can be closed.

• 29:26Peter Diamandis?

  AGI, um, you know, the conversation and the definition of AGI has basically been all over the place. Ray Kurzweil's prediction has been for thirty years that it's twenty twenty nine. Again, that's a blurry line of what we're trying to, you know, target, but Elon's talked about anywhere from twenty twenty five to twenty twenty eight. What are you thinking? How do you, what's your, what's your timeline for, you know, even digital superintelligence?

• 30:00Speaker 04

  I honestly have no idea. Like, people are looking at the scaling laws and applying it, but as I've said, data is the key. And it's clear that we already have, like, you could build a board GPT and it'd be better than most corporate boards, right? So I think we're already seeing improvements over the existing. Um, one of the complications here is swarm AI. So even like, it's the whole thing, like a duck-sized human or a hundred human-sized ducks, right? We're just at the start of swarm intelligence and that reflects and represents how companies are organized. Andre Karpathy has some great analogies on this in terms of the new knowledge OS. And that could take off at any time, but the function and format of that may not be this

• 30:45Speaker 04

  whole Western anti-consummized consciousness that we think of, but just incredibly efficient systems that displace existing human decision-making, right? And so there's an entire actual range of different AGI outcomes depending on your definition. And I just don't know. Um, but I feel again, like, I wake up and I'm like, oh, look, it's fed up ten times the model, you know, like, I'm just not, no one can predict this.

• 31:12Peter Diamandis?

  But there is a point at which, I mean, we were heading towards an AI singularity, you know, using the definition of a singularity as a point after which you cannot predict what's coming next, and that isn't far away. I mean, how, how far out is it for you? A year, two years?

• 31:30Speaker 04

  I think you're heading towards it in the next few years, but like I said, um every company, organisation, individual has an objective function. My objective function is to allow the next generation to navigate what's coming in the optimal way and achieve their potential. So I don't want to build an AGI. I don't want to do any of this. Amplified human intelligence is my preference. And trying to mitigate against some of the harms of these agentic things through data transparency, good standards. And making it so people don't need to build gigantic models on crap, which I think is a major danger if you're not from AGI.

• 32:09Speaker 04

  But again, we just don't understand because it's difficult for us to comprehend hu- superhuman capabilities. But again, we're already seeing that. In narrow fields, we already know that it's a better rider than us. You know, we already know that it can make better pictures than us.

• 32:24Peter Diamandis?

  And a better physician and a better educator and a better surgeon and a better everything.

• 32:29Speaker 04

  Um. Yeah, and again, I think it's this ethos of these big labs being AGI focused, whereas. You can be better than us in like five percent of the stuff that humans can do, and that's still a massive impact on the world, and it can still take over companies and things like that, right? Like if you take over a company, then you can impact the world. And there's clearly with a G P T four or a thousand of them orchestrated correctly, that can call out people and stuff, you wouldn't know it's not the C E O. You know, I can make an M I G P T and then they won't have to make all these tough decisions. And any of that. And most of my decisions aren't that good, so it'd probably be better. Um, so I think that we're getting to that point. It's very difficult.

• 33:11Speaker 04

  And the design patterns are going fast. We're at the iPhone two G, three G stage, it's got copy and paste. And we just got the first stage as well of this technology, which is the creation step. It creates stuff. The next step is control and then composition, where you're annoyed because Chat G P T doesn't remember all the stuff that you've written. That won't be the case in a year. And the final bit is collaboration, where these A I's collaborate together and with humans. To build the information superstructures of the future, and I don't feel that's more than a few years away. And it's completely unpredictable what that world creates.

• 33:44Peter Diamandis?

  Let's talk about responsibility that AI companies have for making sure that their technology is used in a pro-human and not a disruptive fashion. Do you think that is a responsibility of a company, of a company's board, of a company's leadership? How do you think about that?

• 34:05Speaker 04

  Again, with the corporatist capitalist system, it typically isn't because you're maximizing shareholder value and there aren't laws and regulations, which is why I think there's a moral, a social and legal/ regulatory aspect to this. Companies will just look at the legal/ regulatory and in some cases they'll just ignore them, right? But I do think, again, we have a bigger moral and social obligation to this. This is why I don't subscribe to EA or EA or any of these things. I think it's complicated and it's hard, given the uncertainty of how this technology proliferate and you've got to do your best and be as straight as possible to people about doing your best, because none of us are qualified to understand or do this. And none of us should be trusted

• 34:48Speaker 04

  to have the power over this technology, right? You should be questioned, you should be challenged. With that, and again, if you're not transparent, how are you going to challenge?

• 34:58Peter Diamandis?

  When I think of the most linear organizations on the planet, I think of governments, maybe religions, but governments, let's leave it there. How can, you know, let's talk about Western government at least, the US, I would have said Europe, but I'll say the UK and Europe. What should they be, what steps should they be taking right now? You know, if you were given the reins to say, how would you regulate, how would you, what would you want them to do or not do?

• 35:34Speaker 04

  I believe, it's a complicated one. So I signed the first FLI letter, I think I was the only AI CEO to do that back before it was cool, because I said I don't think AGI will kill us all, I just don't know. I think this conversation deserves to be had and it's a good way to have that conversation. And then we flipped the wrong way, where we went overly AI death risk and other things like that and governments were doing that at the AI Safety Summit in the UK. And then we had the King of England come out and say, this is the biggest thing since fire. I was like, okay, that's a big change.

• 36:05Speaker 04

  Uh, the King of England said it, so I must be on the right track. Um, but I think if you look at it, regulation doesn't move fast enough. Even the executive order will take a long time. The EU things will kind of come in. Instead, I think that governments have to focus on the tangibles. AI killerism, again, it can be addressed by considering this is infrastructure. And what infrastructure do we need to give our people to survive and thrive? The US is in a good initial place with the CHIPS Act. But I think you need national data sets. You need to provide open models to stoke innovation. And think about what the jobs of the future are, because things are never the same again. You don't need all those programmers when Copilot is so good and you're moving Copilot for a level above, which is compositional Copilot and then collaborative Copilot, right?

• 36:50Speaker 04

  You would be able to talk and computers can talk to computers better than humans can talk to computers. So we need to articulate the future on that side, but then the other side. Like, one of the examples I give is a loved one had a recent misdiagnosis of pancreatic cancer, right? We did, you and I talked about this. And the loss of agency you feel, and many of you on this call will have had that diagnosis to the near end, dear, is huge. And then I had a thousand AI agents finding out every piece of information about pancreatic cancer. And then after that, I felt a bit more control. Why don't we have a global cancer model that gives you all the knowledge about cancer? And helps you talk to your kids and connects to other people like you, not for diagnosis or research, but for humans.

• 37:31Speaker 04

  This is the Google Medpom two model, for example, that outperforms humans in diagnosis, but also empathy. And what if we armed our graduates to go out and give support to the humans that are being diagnosed in this way? That makes society better and it's valuable, you know? And that's an example of a job of the future, I think. I don't believe in UBI, I think even universal basic jobs as well. Universal basic opportunity, right?

• 37:59Speaker 04

  You know, it's a big opportunity, you know, it's a waste of jobs, but then policymakers need to think about it now because the graduate unemployment wave is literally a few years away.

• 38:09Peter Diamandis?

  That, that, yeah, that is, I mean, when I think about what I, I parse the challenges we're going to be facing in society into a few different elements. I think, you know, what we have today is amazing. And if, if generative AI froze here, we'd have an incredible set of tools to help humanity across all of its areas. And then we've got what's coming in the next, you know, zero to five years. We've talked about patient zero perhaps being the US elections and the, you know, I think you had said, you know, it was Cambridge Analytica that required interference. Now it's any kid in the garage that could could play with the elections. That's a challenging period of time.

• 38:50Peter Diamandis?

  And, and this graduate unemployment wave, as you mentioned, coming right on its heels.

• 38:57Peter Diamandis?

  And that's, you know, the question becomes, is, is the only thing that can create alignment and help us overcome this AGI at the highest level? Meaning it is causing challenges, but ultimately is a tool that will allow us to enable to solve these challenges as well.

• 39:15Speaker 04

  I mean, that's a crazy thought, right? Like, all this stuff is crazy, like the sheer scale and impact of it. And you know, these discussions, we had them last year, Peter, and now everyone's like, yeah, that makes sense. We're like, oh, wow. Right? It may be AGI, it may be these coordinating automated story makers and balances for the market, right? Next year, there's fifty six elections with four billion people heading to the polls.

• 39:41Peter Diamandis?

  What could possibly go wrong?

• 39:44Speaker 04

  What could possibly go wrong, you know?

• 39:46Peter Diamandis?

  Oh, my God.

• 39:46Speaker 04

  But again, the technology isn't going to stop. Like even if stability puts down things, if OpenAI puts down things, it will continue from around the world because you don't need much to train these models. Again, the supercomputer thing is a myth. You've got another year or two where you need them, you don't need them on to that. And that is insane to think about.

• 40:06Peter Diamandis?

  You just released stability video. Congratulations on our stable visual diffusion.

• 40:12Peter Diamandis?

  And I'm enjoying some of the, the clips. How far are we away from me telling a story to my kids and, and saying, let's make that into a movie?

• 40:27Speaker 04

  Almost two years away.

• 40:27Peter Diamandis?

  Two years away.

• 40:29Speaker 04

  So this is a building block, it's the first creation step, and then like I said, you have the control step, composition, and then collaboration. And self-learning systems around that. So we have comfy UI, which is our node-based system where you have all the logic that makes up an image, like you can take a dress and a pose and a face that combines them all. And it's all encoded in the image. Because you can move beyond files to intelligent workflows that you can collaborate with. If I send you that image file and you put it into your comfy UI, it gives you all the logic that made that up. How insane is that, right? So we're gonna step up there, and what's happened now is that people are looking at this AI like

• 41:10Speaker 04

  instant versus, again, the huge amount of effort it took to take this information, structure it all. But the value is actually in stuff that takes a bit longer. Like, when you're shooting a movie, you don't just say, do it all in one shot, right? Unless you are a very talented director and actor, you know? You have mise en place, you have staging, you have blocking, you have cinematography. And it takes a while to composite the scenes together. It'll be the same for this, but a large part of it will then be automated for creating the story that can resonate with you, and you can turn it into Korean or whatever. And there'll still be big blockbusters, like Oppenheimer and Barbie. But again, the floor will be raised overall.

• 41:52Speaker 04

  Similarly, like, we had a music video competition, check it out on YouTube with Peter Gabriel. He let us to use kindly his songs, and people from all around the world made amazing music videos to his thing, but they took weeks. So I think that's somewhere in the middle here, where again, we're just at that early stage, because ChatGPT isn't even a year old, you know, stable diffusion is only fourteen, fifteen.

• 42:13Peter Diamandis?

  And they're, and need, and I think, but you'd agree that neither of them is the end-all and be-all, it's just, it's the earliest days of this field. Um, I had the conver-

• 42:23Speaker 04

  The timiest building-

• 42:24Peter Diamandis?

  Yeah, I had this conversation with Ray Kurzweil, uh, two weeks ago. Um, we were just after a Singularity board meeting we had, and we're just hung on on a Zoom and chatted. And, you know, the realization is-

• 42:38Peter Diamandis?

  Unfortunately, the human mind is awful at exponential projections. And despite the convergence of all these technologies, we tend to project the future as a linear extrapolation of, you know, the world we're living in right now.

• 42:54Peter Diamandis?

  But the best I can, I can say is that we're going to see in the next decade, all right, between now and 2033, we're going to see a century worth of progress. But it's going to get very weird, very fast, isn't it?

• 43:08Speaker 04

  I mean, there's, there's two-way doors and there's one-way doors, right? Like, in December of last year, multiple headmasters called me and said, "We can't set essays for our homework anymore." And every headmaster in the world had to say that same thing, it's a one-way door.

• 43:22Peter Diamandis?

  Yes.

• 43:22Speaker 04

  And this is the scary part, the one-way doors, right? Like, when you have an AI that can do your taxes, what does that mean for accountants?

• 43:33Speaker 04

  All the accountants at the same time?

• 43:36Speaker 04

  Kind of crazy, right?

• 43:37Peter Diamandis?

  It is. And the challenge, I mean, one of my biggest concerns, so I'm, listen, I'm the eternal optimist. I'm not the guy who's, the glass is half-full, it's the glass is overflowing. And one of the challenges I think through when I think about where AI, AGI, ASI, whoever you want to project it to, is the innate importance of human purpose. And unfortunately, most of us derive our purpose from the work that we do. You know, I ask you, you know, what, you know, tell me about yourself, and you jump into your work and what you do. And so when AI systems are able to do most everything we do,

• 44:20Peter Diamandis?

  not just a little bit better, but, you know, orders of magnitude better, redefining purpose and redefining my role in achieving a moonshot or a transformation is, it's the, you know, it's the impetus mismatch between human societal growth rates and tech growth rates. What are your thoughts there?

• 44:50Speaker 04

  Yeah, I mean, I think again, exponential is hard. Like, if I say G P T four in twelve to eighteen months on a smartphone, you'd be like, well, that's not possible. Why? You know, like, G P T four is impossible. Stable diffusion is impossible, right? Like, now they've almost become commonplace, but why would you need supercomputers in these things? But I do agree this is mismatched. And that's why we're in for five years of chaos. And that's why it's called instability. I saw this coming a few years ago, and I was like, holy crap, we have to build this company. And now we have the most downloads of any models of any company, like fifty million last month versus seven hundred thousand from Mistral, for example.

• 45:32Speaker 04

  Um, and we will have the best model of every type except for very large language models by the end of the year. So we have audio, three D, video, code, everything, and a lovely, amazing community. Um, because it's just so hard again for us to imagine this mismatch, there's a period of chaos. But then on the other side, like, there's this P doom question, right? The probability of doom. I'm gonna say something. With this technology, the probability of doom is lower than without this technology, because we're killing ourselves. And this can be used to enhance every human and coordinate us all. And I think what we're aiming for is that Star Trek future versus that Star Wars.

• 46:11Peter Diamandis?

  Yes, I'm into that. Um, and I think that's an important point, that the level of complexity that we have in society. Uh, we don't need A I to destroy the planet. Um, we're doing that very, very well, thank you. Um, but the ability to coordinate. So one of the, one of the things I think about is a world in which everyone has access to all the food, water, energy, healthcare, education that they want. Uh, really, a, a world of true abundance in my mind is a, a peace, more peaceful world, right? Why would you want to destroy things if you have access to everything that you need?

• 46:52Peter Diamandis?

  And that kind of a world of abundance is, is on the backside of this kind of awesome technology.

• 47:00Speaker 04

  We have to navigate the next period. I believe we'll see it within our lifetimes, particularly if we get longevity songs, right? Um, and that's so amazing, right? But then we think about, as you said, why peace? A child in Israel is the same as a child in Gaza. And then something happens. A lie is told that you are not like others. And the other person is not human like you. All wars are based on that same lie. And so, again, if we have A I that is aligned with the potential of each human that can help mitigate those lies. Then we can get away from war because the world is top scarce. There is enough food for everyone. It's a coordination failure.

• 47:41Peter Diamandis?

  Right, I.

• 47:42Speaker 04

  And that can be addressed by this.

• 47:43Peter Diamandis?

  Agree. One of the most interesting and basic functions or capabilities of generative A I has been the ability to translate my ideas. Uh, into concepts that someone with a different frame of thought can understand. Right, um.

• 47:58Speaker 04

  That, that's, but that's what this generative A I is. It's a universal translator.

• 48:04Peter Diamandis?

  For sure.

• 48:04Speaker 04

  It does not have facts. The fact that it knows anything is insane. Hallucinations is a crazy thing to say. Again, it's just like a graduate trying so hard. G P T four with ten trillion words and a hundred gigabytes is insane. Stable diffusion has like a hundred thousand gigabytes in a two gigabyte file, fifty thousand to one. Compression is something else. It's learned principles.

• 48:28Peter Diamandis?

  Yes.

• 48:28Speaker 04

  And this is.

• 48:29Peter Diamandis?

  It's, it's knowledge, knowledge versus data, yeah.

• 48:33Speaker 04

  It's knowledge versus data and you apply some experience, you get the wisdom, right? Because it's learnt the principles and contexts and it can map them to transform data. Because that's how you navigate. You don't navigate based on.

• 48:49Speaker 04

  Like logical flow. We have those two parts of our brain. Navigate sometimes based on instinct, based on the principles you've learned. So Tesla's new auto-driving model, self-driving model, is entirely based on, I can't say which architecture, and if they said it publicly. It's based on this technology. It doesn't have any rules. It's just learnt the principles of how to drive for massive amounts of Tesla data that now fits on the hardware without internet. And so they went from self-driving being impossible to now, hey, it works pretty well, you know? Because it's learnt the principles. And so that's why this technology can help solve the problems. This is why it can help us amplify our intelligence and our innovation. Because it's the missing part, the second part of the brain.

• 49:30Peter Diamandis?

  You know, uh, next, I can't give more details yet, but next week we're announcing the largest X Prize ever. It's a hundred and one million dollars. Um, it's a hundred and one, so it's, uh, Elon had the, uh, had a hundred million dollar prize that got him to fund a few years ago for carbon sequestration. And the funder, the first funder of this prize wanted to be larger than Elon's. I said, okay, you add the extra million.

• 49:56Speaker 04

  It's for luck, it's for luck. We did ask either rounds a hundred one million.

• 49:59Peter Diamandis?

  Oh, really? Okay, that's great. That's a new popular number. Um, anyway, uh, the, uh, and it's in the field of health. I'll leave it at that. Uh, folks can go to Xprize.org to register to see the live uh, event on November twenty ninth, we're going to be debuting the prize, what it is, what it's going to impact eight billion people. Uh, long story short, it's, it's a nonlinear future because we are able to utilize AI and make things that were seemingly crazy before, uh, likely to become inevitable. Um, and it's an amazing future we have to live into.

• 50:41Speaker 04

  Yeah, I mean, again, because it's one way doors. The moment we create a cancer GPT, this is something that we're building, we have trillions of tokens and then you Google TPs and things like that, that organizes global cancer knowledge and makes it accessible and useful, even if it's just for guiding people that have been diagnosed, the world changes. The fifty percent of people that have a cancer diagnosis in their lives, in every language and every level, will have someone to talk to and connect them with the resources they need and other people like them and talk to their families, you know? And how insane is that? And so I can't believe these positive stories of the future need to be told, right?

• 51:20Speaker 04

  Because that will align us to where we need to go as opposed to a future full of uncertainty and craziness and doom.

• 51:27Peter Diamandis?

  Um, in our last couple of minutes here, buddy, uh, what can we look forward to from stability? Um, in the months and years ahead?

• 51:37Speaker 04

  Um. We have every model of every type and we'll build it for every nation and we'll give back control to every nation.

• 51:43Peter Diamandis?

  So, coming back to governance here, um, again, is the nation state the unit of control? Is it?

• 51:53Speaker 04

  No, I, my, my thinking is the stability of every nation should have the best and brightest of each. Because what you've seen is there are amazing people in this sphere. The best and brightest in the world know this is the biggest thing ever. And they all want to work in it. And it's just finding the right people with the right intention. The brightest people go back to Singapore or Malaysia or others because of the future of their nations. And again, now we're doing a big change and we don't talk about all the cool stuff we do. We're just taking it because we need to articulate that positive vision of the future. Because the only scarce resource that actually exists is human capital. It's not GPUs. It's not data. It's about the humans that can see this technology and realize that they can play a part in guiding it for the good of everyone.

• 52:39Speaker 04

  Their own societies and more. And that's again what I hope stability can be.

• 52:43Peter Diamandis?

  Well, I wish you the best of luck, pal. Thank you for joining me in this conversation. It's a, it's been a crazy four or five days, um, and wish Sam and Greg and the entire OpenAI team, um, stability in their lives.

• 53:01Speaker 04

  Yeah, yeah, I hope they have a nice Thanksgiving. They're, they're absolutely, they're an amazing team building world-changing technology. It's such a concentration of talent. I think, again, I really felt for them over the last few days, you know, much as I kind of post memes and everything, I posted that as well. Um, I think this will bring them closer together and hopefully they can solve the number one problem that I've asked them to solve, which is email. Solve email. Right? And then we'll crack on from there.

• 53:29Peter Diamandis?

  All right, cheers, my friend.

• 53:40Speaker 05

  New and exclusive holiday decor just launched online at The Home Depot. Check out the viral Grand Duchess Collection with smart holiday lights you can control from your phone. Easily customize the twinkle across trees, wreaths, and garlands to match your holiday vibe. Create stunning light effects, then switch it up with the tap of your fingers. Even the man in red will be impressed. Shop the holiday decor you want, online only, at homedepot.com.