Frontier Labs Want to Slow Down, OpenAI Delays Its 2026 IPO, Anthropic Flags 5 Bioweapon Cases | EP #291

The mates discuss the growing push to slow the AI race, OpenAI’s delayed 2026 IPO, Anthropic flagging five potential bioweapon cases, and what a broader AGI slowdown could mean for frontier labs. Get access to metatrends 10+ years before anyone else - https://qr.diamandis.com/metatrends Peter H. Diamandis, MD, is the Founder of XPRIZE, Singularity University, ZeroG, and A360 Salim Ismail is the founder of Open ExO, a GP at Exponential Venture Capital/The Organizational Singularity Fund and a sought after global speaker and thought leader. Dave Blundin is the founder & GP of Link Ventures Dr. Alexander Wissner-Gross is a computer scientist and founder of Reified – This episode is brought to you by: Get the blueprint for generative media https://goo.gle/startupgenmedia Go to Blitzy to book a free demo and start building today: https://qr.diamandis.com/blitzy Your body is incredibly good at hiding disease. Schedule a call with Fountain Life to add healthy decades to your life, and to learn more about their Memberships: https://www.fountainlife.com/peter _ Connect with Peter: X Instagram Substack Website Xprize A360 Connect with Dave: Web X LinkedIn Instagram TikTok Connect with Salim: LinkedIn X Join Salim’s 10X Shift Subscribe to Salim’s YouTube channel Exponential Venture Capital Connect with Alex Website LinkedIn X Email Substack Spotify Threads Listen to MOONSHOTS: Apple YouTube Follow MOONSHOTS: Instagram TikTok X Threads – *Recorded on September 16th, 2026 *The views expressed by me and all guests are personal opinions and do not constitute Financial, Medical, or Legal advice. Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Peter Diamandis?

  Daria Amade, CEO of Anthropic, published a 3,800-word essay titled "We Must Pace the Frontier." Three hours later, Sam, Elon, said, "You're right." Monday, President Trump called it a hoax, and China called it a Cold War trick.

• 0:14Speaker 01

  Nothing less than a clumsy decapitation attempt against the takeoff itself.

• 0:20Peter Diamandis?

  Sam Altman told Fortune that OpenAI will not go public in 2026. He told Reuters, "Even a 10% extinction risk would be unacceptable."

• 0:28Speaker 01

  I think this is pure theater. You don't want to IPO now? Blame safety. You want world government? Blame safety. The dog ate my homework, and the safety ate my IPO.

• 0:37Peter Diamandis?

  Anthropic released its September Threat Intelligence Report documenting real malicious use of Claude. Five cases involving activity that could support biological weapons development. The bio red line, that every lab said it wants to stay below. Anthropic has said, "We can't promise that we haven't crossed it."

• 0:54Speaker 02

  The fact that it's happening just on the eve of this summit is not, I think, an accident.

• 0:58Peter Diamandis?

  Let's be clear about one thing. There is going to be AI regulation.

• 1:04Speaker 03

  Now, that's a moonshot, ladies and gentlemen.

• 1:09Speaker 03

  This episode is brought to you by the Abundance Summit and Link Ventures.

• 1:15Peter Diamandis?

  Welcome to Moonshots, everyone. Your number one podcast in all things AI and exponential. And this week, you're seated at the table where the future of the entire industry just got renegotiated in the past 72 hours. On Saturday, Daria Amade said, "Slow down." Three hours later, Sam, Elon, said, "You're right." Monday, President Trump called it a hoax, and China called it a Cold War trick. That's the last few days. Buckle up. It's never going to be slower than it is today. Let me introduce you to my extraordinary Moonshot mates, the Fantastic Four this week. AWG, Alex Weisner Gross, the only man alive who reads a 3,800 AI policy essay and says, "Needs more math."

• 1:57Peter Diamandis?

  Dave Blunden, who just watched every VC on Sand Hill Road discover the word "moonshot", and who's been funding them all along at Link Ventures the whole time. Salim Ismail, Warlord Against Linear Thinking, and this week, apparently, an avid accelerationist promoting PFAB over P Doom. Sound right, Saleem?

• 2:18Speaker 02

  Sounds about right. We've got some alternatives that'd like to bounce out, get your guys' viewpoints on it.

• 2:23Speaker 04

  All right, good. That's great. That'll be fun.

• 2:25Peter Diamandis?

  Iman is out this episode, but we'll have him back soon. Iman, we miss you already. I'm Peter Diamandis, your host and abundance provocateur, frantically working a 36-hour day just to keep up with what's happened over the last week. Our mission on Moonshots is, as always, to help you understand what just happened, what it means for you, and to keep you abundance-minded during the decade ahead. If you haven't subscribed yet, please do. Two episodes a week, and this week, honestly, we could have done five episodes.

• 2:57Peter Diamandis?

  I want to take a second, as I've done before, take a moment to say thank you to our subscribers. You know, there are a few quotes that I loved from @MedicalMandalorians. "The only show I find myself compulsively checking for new episodes. Thank you for documenting the singularity and keeping us all optimistic and informed." From ZMW0093, "How I've missed this channel for so long, I have no idea." Without question, this is the best place to really contemplate where AI is actually going. And from Monday's episode, @BroZ3U, who's got 248 likes on this one, he says, quote, "I can't wait for when progress starts to move so fast, and the mates are going to have to be live 24/7."

• 3:41Peter Diamandis?

  Careful for what you wish for.

• 3:44Speaker 04

  That's okay. Our avatars will do that.

• 3:46Peter Diamandis?

  I think that's about it, right? That's the dematerialization of this podcast. And @TaylorLuck says, quote, "I don't know why, but this show makes me stop everything I'm doing and start watching immediately. I don't fangirl about celebrities, but I would definitely freak out if I got a chance to meet you guys in person." Well, Taylor, come to Moonshots Live next week on the 25th and freak out in person. All right? That means a lot to us, guys. You know, our mission here is to reach 10 million subscribers. Why not have 100x growth objective, right? And @GWWinston6532 said, "It blows my mind that only a few hundred thousand people watch this every week." Everyone needs to be watching, all right?

• 4:30Peter Diamandis?

  So, share it with everyone. Today, we're going to cover the insanity that took place over the last five days. Try and lay out what it means in a meaningful fashion, understandable fashion, probably a little bit of debate amongst the mates here. And guys, you know, I've been glued to the conversations. I don't know about you guys. Where are you today, by the way?

• 4:51Speaker 04

  I'm at Stanford. Actually, it was over. Go ahead.

• 4:57Peter Diamandis?

  Where are you today, Saleem?

• 4:58Speaker 02

  I'm in Miami today. I'm getting on a flight to India tonight. I'll be in India for two, three days, and then back.

• 5:06Peter Diamandis?

  India for two, three days, okay.

• 5:07Speaker 04

  Any particular reason? Just India, for the sake of India? Summertime in India, it's beautiful?

• 5:11Speaker 02

  No, we're launching the organizational singularity out there. So, I'll be doing a launch event and a couple of other talks.

• 5:19Peter Diamandis?

  And Dave, where about yourself?

• 5:21Speaker 04

  I'm at Stanford. Actually, I was at Google all day yesterday. And, yeah, Peter, you said we have a few hundred thousand listeners. They're definitely the right listeners. Almost everybody at the summit was coming up to me, saying, "I listen to it every single time." So, and a lot going on in feeds and speeds. You know, the Vera Rubin ships are coming out. There was a company, Positron, that has an inference time accelerator that's really crazy. Spent a lot of time with Cerebrus and Andrew Feldman, who's phenomenal. We did a panel together. And, you know, his inference time speed, it's everything we predicted at the beginning of the year. Things are going to accelerate like crazy. It's starting to happen, literally, right now. And nobody quite knows what that means. You know, as the AI gets bigger and bigger and bigger, it always gets smarter, but it's difficult to predict the impact of that.

• 6:05Speaker 04

  So, that's why there's so much going on in politics this week, because it's now.

• 6:09Peter Diamandis?

  It's called the singularity for a reason. And, AWG, virtual background or real background?

• 6:15Speaker 01

  This is real, unfortunately. I'm at SFO with Dave. I was at Google for the past two days, about to fly back to Boston.

• 6:23Peter Diamandis?

  All right. Well, if some fans come racing by, have them wave in the background.

• 6:29Speaker 01

  Will do.

• 6:29Peter Diamandis?

  All right. Let's open with the most consequential AI policy essay of 2026. Saturday morning, Daria Amade, CEO of Anthropic, published a 3,800-word essay titled "We Must Pace the Frontier." His argument, "The industry must deliberately slow the pace of capability advancement so that safety work can keep up." Quick context on why this matters. This is not a pause AI letter from outsiders, right? This is the CEO of a company about to go public for $2 trillion. And the company that just shipped Fable 5.1, and honestly, one of the top two most powerful executives on the planet. Dario says that two things changed his mind.

• 7:13Peter Diamandis?

  First, recursive self-improvement. In his words, quote, "Starting to happen across the industry, including at Anthropic." Second, the summer's containment failures with OpenAI hugging face swarm that we covered proved that, you know, safety measures don't hold at this pace. So, his core claim, and let me quote from his essay, "We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain." He continues, "It's my worry that in six to 12 months, much of a swarm could be capable of taking over the entire internet with a persistent bot net, potentially causing hundreds of billions of dollars in damage."

• 7:59Peter Diamandis?

  The next quote from Dario, he says, "Pacing does not mean halting model training or halting technical progress, but ensuring companies take adequate time to align the safeguards of their models and for third-party evaluators to confirm this."

• 8:15Peter Diamandis?

  Let's get into this. He offers three steps. You know, unilaterally giving third-party evaluators permanent employee-level access so that they can evaluate what's going on. Second, a critical mass of US labs adopt the same standards, enabling verified pacing through legislation or a narrow antitrust waiver. And third, international coordination for escalating levels, banning bioweapon use, which he thinks is feasible. And at the other end, to a full pause, which he thinks is not. Reasonable. You know, I'll, I'll go to, you know, to Alex on you this, for you on this first.

• 8:57Peter Diamandis?

  At the same time, I'm going to show the slides of Altman, Hasabis and Elon's tweet accepting this as, you know, something that we all need to do. So Alex, what's your take on the essay?

• 9:09Speaker 01

  Yeah, I'll give you my unvarnished hot take. I think what we saw this weekend and in the few days that followed may have been nothing less than a clumsy decapitation attempt against the takeoff itself. And I emphasize clumsy. So I think it's, I think at this point, knowing what I know and knowing what's been publicly reported, it's probably impossible to consider Dario's essay in isolation. So consider the following sequence of events. Events may or may not have started with Jacob Coxon, whom we spoke about on previous episode, doing his public rage quit, maybe to get attention, maybe not, but either way, a public rage quit pointing at safety issues. Then we get to the foreign state actors, which has now been reported from various venues,

• 9:54Speaker 01

  amplified Jacob's resignation letter ostensibly for obvious nation-state competition reasons, possibly slash probably to decelerate American superintelligence. Then, as far as I can tell, it seems like Dario takes the bait and Dario and Anthropic, again, this has been widely reported. There's a broad alignment between Anthropic and effective altruism. He takes the bait and publishes an essay pushing a very EA-ish agenda to decelerate. Then the other lab leads, who we now know were already in discussions for weeks prior to Dario's essay

• 10:33Speaker 01

  with Anthropic about forming what I've publicly characterized as a cartel, amplify the discussion. They all tweet, "We agree with Dario, let's do this." And then the mainstream media then jump in further and they amplify it because this sounds scary and everyone's going to die. And, and they amplify it on an otherwise relatively slow news weekend. So to put it simply, spoiler alert, if you've read Agatha Christie's Murder on the Orient Express, it turns out that everyone did it for different reasons, but everyone was the murderer. Here, everyone did it. They're all pushing different agendas. Jacob wants attention, it seems. The foreign state actors who are amplifying it want to decelerate American superintelligence.

• 11:18Speaker 01

  Dario is pushing effective altruism. The other lab leads would love to get inspectors looking at Anthropic's codebase and they want to better compete with Anthropic. The mainstream media want to publish if it leads, it leads. And so we end up with this really bizarre weekend. That's my summary.

• 11:34Peter Diamandis?

  Wow. So an orchestrated, behind-the-scenes, coordinated effort?

• 11:40Speaker 01

  Safety cartel. I mean, I coined this term "safety cartel" over the weekend, the moment I saw Dario's essay. And then our friend at, I guess, our cousin pod, David Sacks, immediately runs with it and starts calling this a proposed cartel. It escalates to the White House and the White House takes the position, "No cartels. We're not going to grant lenience to Anthropic or OpenAI. If they're looking for some sort of liability waiver from the White House for products that they release or models that they release that wreak havoc on the internet, they should look elsewhere." The Congress, the Speaker of the House, issues a similar dictum, "No, don't look to Congress for some sort of waiver of liability from defective models that you're releasing."

• 12:26Speaker 01

  So hopefully this pod and the implication, which I think is accurate, that what we saw was nothing less than an attempt to form a safety cartel, which I defined as a coalition of dominant firms that invokes safety to justify coordinated limits on the pace or terms of competition. I think we hopefully headed that one off at the pass.

• 12:45Peter Diamandis?

  So, you know, the one point I'll make, Alex, and you and I texted about this, and you might want to mute with the background, but is that the frontier labs desperately need some kind of regulation to survive. Right? If they're out there saying, "This is dangerous, we're going to see these things escape, they could cause hundreds of billions of dollars of damage." And then if the agents actually do escape and cause havoc in a banking system or a power grid, then the labs are potentially liable. I mean, in courts around the world saying, "You knew this could happen and yet you released the new, you know, Astro 6.1 anyway. We're suing you for a billion dollars a thousand times over and it's a trillion dollars of liability without coverage."

• 13:30Peter Diamandis?

  But if there's a regulatory structure that gives them an approval for release and the liability turns to the regulators versus to the companies, that avoids this existential risk. Do you buy that?

• 13:44Speaker 01

  No. Dario's original notion was a race to the top, not a race to the bottom. And then he, I think in recent times, has abandoned the notion of a race to the top, race to alignment. I've argued on this pod in the past that alignment equals capabilities. And for whatever reason, I think at least Anthropic and OpenAI, who are still, if not a duopoly, they're the two leaders right now in terms of capabilities, need to return to this idea that, again, under the covers, if you're maximizing alignment, you're also maximizing capability. And I'd love to see them competing to be the most aligned, not looking for government blessing.

• 14:22Peter Diamandis?

  I tweeted that out saying, "I don't understand why the conversation isn't, let's point 10,000, 100,000, a million agents at alignment versus anything else," right? I mean, that should be, if we have, if we have superintelligence at the line with humanity, that's amazing. And that should be the number one objective.

• 14:39Speaker 01

  I'll just, by analogy, let's say we're on the verge of developing cities. Let's say rewind a few million years or a few thousand years, depending on how you measure the shift from, say, the agricultural revolution to the post-agricultural state. You find ourselves in a world, if we buy this Dario essay, where the argument would go, "Let's not form any cities because cities are dangerous and we can't control what's going to happen in cities. Let's stay as agrarian civilizations or even hunter-gatherers because who knows what could happen? The mobs, the emergent effects from having lots of people densely concentrated in one place will be uncontrollable."

• 15:20Speaker 01

  And that's precisely the wrong response. The right response is, "Let's invent police forces. Let's do defensive coscaling. Let's use the benefits of concentration of intelligence to our benefit rather than just being scared of it and forming cartels to avoid forming cities in the first place."

• 15:35Peter Diamandis?

  All right, let's bring Dave and Slim in the conversation. Dave, your thoughts?

• 15:39Speaker 04

  Yeah, I was surprised that they didn't loop in the White House a little bit better prior to the paper. You know, because David Sacks' reaction to it was just absolutely over the top. And they got a very, and there's nothing productive about the negative reaction coming back from the White House. But what surprised me, antitrust law is no joke. You know, like, I think anytime that you have very, very powerful tech CEOs all working on competing products, meeting it all, that triggers antitrust right away. And so David's reaction was, "Hey, you guys don't need a waiver from the government. If you want to slow down, slow down." And that's exactly what he tweeted. But I think Dario's point was more like, "No, we saw what you did to Microsoft or the federal government many years ago did to Microsoft.

• 16:21Speaker 04

  We know that antitrust law is very serious and purely the act of us colluding on any kind of a safety package is enough to trigger antitrust. We want a waiver." Now, that could be read in two different ways, right? One of them, which I believe is, "Look, we know we need government involvement immediately. We know that regulation is going to lag terribly behind AI capability. You've got to be part of this, you know, White House, get involved." But our fundamental motivation here is purely the safety of humanity. Or you could read it the other way, which is, "We're trying to form a regulatory capture cartel that controls AI for the whole future. We all know Xi Jinping is coming next week.

• 17:01Speaker 04

  And this is our attempt to do what Biden was actually already going to do, already told Marc Andreessen he was going to do, limit AI to just four or five big companies and then we can regulate those. And then we don't have to worry about leakage, we don't have to worry about startups." Marc Andreessen thought it was the worst thing he'd ever heard in his life and tried to prevent Biden from doing it. It didn't matter because the election went the other way. But I think the reason they stepped on a landmine is because we have right now the most antitrust friendly administration in my lifetime by far. And actually, I have a company right now that I sold last week that's in antitrust process right now, and it's just the smoothest it's been in my entire lifetime.

• 17:41Speaker 04

  So I think by asking for antitrust waivers, they stepped on a huge landmine in terms of the administration saying, what are you talking about? You're asking completely the wrong administration. We're so antitrust friendly, this is insane. So it's kind of disappointing that they didn't just call each other first and say, look, this is our purpose here. Tell us what your reaction is going to be. Why don't we coordinate this a little bit? But anyway, it worked out the way it worked out. It makes a huge amount of drama. I guess everybody in politics likes drama, creates news. News is always good for the incumbents. So that's the way it played out.

• 18:18Peter Diamandis?

  Saleem, I'm going to play this video, and I'd love your response to it. This is Dario addressing yours.

• 18:23Speaker 05

  Do you believe, earnestly believe, that AI could kill all humans? And what percent chance do you think it is?

• 18:28Speaker 06

  I agree with Jacob much more than I disagree with him. He wasn't calling out us. He was calling out the dynamic of the industry as a whole moving too fast. Now, I think I already said, these sort of unconditional probabilities. I used to kind of talk in that way because it's kind of an easy way to express, I think there's some chance that things go wrong, but it's not all that high.

• 18:49Speaker 05

  You're right. We must slow the pace at which we improve the capabilities of AI models. Progress will seem fast, and we must make wise use of the time we gain. And there's three parts to this. One is embedded evaluators. Part two is democratic coordination. Part three is global coordination. Essentially, you are suggesting observers inside your company and these other AI companies, ideally globally, but certainly in the US, who would monitor it, they would have employee access. What power would they have?

• 19:14Speaker 06

  Yeah, embedded evaluators. So there is precedent for this. If you look at how banks are regulated, there's all kinds of rules about how banks are allowed to do their business, because we don't want to have financial crises. And one of the things that are done in some cases is, I think they're called supervisors in the financial industry, but they're folks who are independent of a company, who are embedded in the company day to day to check their practices. I think we can do the same thing in AI.

• 19:39Speaker 05

  The other thing you're calling for in this three-part proposal is democratic coordination, which is essentially democracy, all the companies that are under democracies, having these evaluators, having this coordination. And probably the most important, because the argument is, well, if the US or companies are the only ones regulating themselves, what about, does China get ahead? You're calling for global coordination. That's obviously a lot more difficult.

• 19:59Speaker 06

  Yes, yes. In this three-step proposal, each step is more difficult than the one before. And Anthropic was kind of getting the ball rolling by committing to the first step. But we shouldn't kid ourselves that these next two steps are going to be more difficult, and we may not succeed. And so my proposal is a proposal that we should try. So yes, pacing among companies in democratic countries. Now, one of the important things that I need to call for here is, as we get to the pace of model releases, that's commercial activity. And in order for companies to coordinate directly on commercial activity, they need to do so working with the government. And so the one thing that we're asking for from government right now, I mean, there's also this question of what should the regulations be, but there's something even simpler that

• 20:43Speaker 06

  the government can do right now, which is it can sit down in the room and with all the industry players, and we can all talk together, and the government can make sure that we're not doing anything nefarious from the perspective of antitrust and collusion. We're just trying to make our systems safe and to work together on making the systems safe.

• 21:01Peter Diamandis?

  All right. Saleem, Sam Altman writes next. I agree with Dario. We need to pace it. Then Demis comes out saying he agrees. Here's his tweet. And then Elon, Dario is right. We'll talk a bit more about Elon's proposal in a minute, but what's your take on all this, Saleem? How's this come across to you? Is he authentic?

• 21:24Speaker 02

  Well, I think he's authentic, but I call BS, and I'm very cynical of the whole thing. As I mentioned on the last podcast, if you spend years hiring the best people, pulling them together to achieve AGI, ASI, whatever we call it, and then as you get close to it, you go, oh, my God, the world may end. I'm sorry, where was the plan? If you were smart people, you knew this was happening. Where was this beforehand? This conversation could have happened years ago. In fact, let's note that as early as GPT-2, Yan Le Cun, I think said this most accurately, said, note that they were calling about world-ending AI when GPT-2 came out. So this is a trope that keeps getting rolled out, calling for slowdown.

• 22:08Speaker 02

  I don't understand, and maybe Dave can break this down a bit, how this affects the S1. How do you do an S1 in the middle of all of this chaos? The only thing, and I'm putting on full cynic hat here, is that maybe they're seeing slowdown in revenue growth because of the open models, and that is causing them to go, oh, my God, we better call for a public slowdown because we can't really justify this in any other way, and our valuation will get trashed in the public. So that's my most cynical view. At the most generous, you would say this is the most naive thing you've ever seen because you've been building these labs forever, and now you're calling for everybody to freak out.

• 22:48Speaker 02

  How you get to the idea that AI can meaningfully destroy all of humanity is incredibly irresponsible unless you lay out exactly the steps by which that might happen. And so I really get upset with a lot of this.

• 23:04Peter Diamandis?

  Yeah, I'm going to play two clips of Elon from our friends at the All In Summit, and let's talk about it. Of all the proposals I've heard that could work, Elon's, where the different frontier labs are cross-checking peer-to-peer, their safety seems like the most useful. So let's take a look at these two videos, and let's discuss them.

• 23:27Speaker 07

  Do you mean Dario is right about describing the potential harm? He's right about describing the regulatory solve, both the first, the latter? Just can you help us understand when you said that? Because I think it was sort of a moment where everybody was like...

• 23:39Speaker 08

  I probably should have said more than, I probably should have said more than Dario's right. I did try to clarify it in subsequent posts on X, but those get much less attention. So what I've been saying is right is that the danger of AI is very significant at this point, that we need to do better with AI safety, or we have exponentially increasing risk with the AI models. And I've heard this not just from Dario, but from many other people at Anthropic, that when a lot of people from Anthropic and from OpenAI are telling you that their models are very dangerous, I think we should believe them. It certainly is like some crazy 4D chess. There's whatever, 10% chance of annihilating humanity.

• 24:21Speaker 08

  But by the way, how much allocation would you like in our IPO?

• 24:25Speaker 07

  Yes.

• 24:26Peter Diamandis?

  All right. I'm curious, guys, do you not believe Elon when he says that this is getting very dangerous and that we really need to slow down when all of them are saying this?

• 24:41Speaker 04

  Well, you got to consider the alternative. If you're running a company going public, running a company raising capital, three out of four say this is dangerous, and you say, what do you mean? There's no risk at all. What's going to happen next? Everybody on the planet is going to say, you must be lying, you're just in it for the money. So you have no choice but to go along with it. Funny enough, though, China is the one saying, exactly, China is the one saying, there's no risk at all. We're going to go take our open source to BRICS, so it'll be in Brazil, Russia, India, and China, and South Africa, India where Saleem will be in a couple hours or in a day. And they're all going to say, yeah, we want AI too. We love this Chinese message. We hate that American message that it's all dangerous and going to get locked down.

• 25:26Speaker 04

  We like the Chinese message that we're going to have equal access to it, and we can catch up to you guys. And so they're going to love that, and they're going to wrangle the whole world behind that message. But you can't be a CEO of a profitable company and say, yeah, everyone else is saying it's risky, but I'm going to get out there and say, no, no, no, everybody chill out, there's no risk at all. So I don't think any of them even have that option. It would backfire so badly.

• 25:51Speaker 01

  Elon is pushing his own agenda, and I think it's pretty transparent. And the agenda, I suspect, includes the fact that Elon's now an infrastructure provider, and Elon's Grok, or I should say SpaceX AI's Grok, is itself, this is publicly reported information, it descends from reasoning traces that were distilled ultimately from Claude interactions, vis a vis cursor. So I'm sure Elon would love nothing more than for SpaceX AI to be able to audit and/or provide harness benchmarks for safety or otherwise for OpenAI and Anthropic. That would be such a boon for SpaceX AI's ability to remain at the frontier.

• 26:32Speaker 01

  And in his mind, maybe there's a safety benefit, but it comes with all of these collateral side effects of helping to prop up SpaceX AI's frontier status.

• 26:41Peter Diamandis?

  All right.

• 26:43Speaker 04

  That's easily fixable, by the way. I mean, I think all they're calling for at this stage, stage one, is independent review of each other's models through APIs. But you could easily do that in clean rooms and say, look, you can't take any of the traces out, you can't take any of the prompts out, but you can hammer it with any test suite you want within this clean room, and that would resolve that problem. So that would get to the, are you really just trying to spy on each other? Or are you really concerned about debugging each other's risks before they go to the public?

• 27:12Speaker 01

  Arguing, even in the case of a clean room, the real, I think, risk that we run is forming a safety cartel where through this indirect form of collusion, the clean room, said clean room, forms the basis for collusion on common benchmarks, common harnesses, and that then, it's possible when you have strong optimizers and common objectives to do all sorts of nefarious forms of collaboration and collusion, even if there are no humans in said clean room, all forms of cartel are now on the table. That's my concern.

• 27:46Speaker 04

  The analogy to banking is, Alex, I mean, I hate to agree, but I agree. The analogy to banking that Dario brought up in that video clip, when was the last time you saw a new Fidelity or JP Morgan startup pop onto the scene and become a mega bank? It just doesn't happen. Why? Because the regulatory mode is so strong. So your analogy actually holds a lot of water. Even if you're just colluding on safety issues, the barrier for startups is just insurmountable at that point.

• 28:13Peter Diamandis?

  This episode is sponsored by Google for Startups. Think about this for a second. You now have access to the same generative AI models that cost hundreds of millions of dollars to train. Google Startup Tactical Guide for Generative Media gives you a complete blueprint for deploying Google DeepMind's models in production. Images, video, audio, all of it, real architecture, real results. Find the link in the show notes below.

• 28:40Peter Diamandis?

  Okay, let's take a look at the proposal that Elon made at the All In Summit about cross-checking against labs and using their safety harnesses. I mean, for me, this is the only proposal I've heard that makes some sense and that I can imagine China adopting as well. Let's take a listen and discuss it.

• 28:57Speaker 08

  The best thing I can think of is really that everyone's got, all the AI companies have a sort of a test harness, where like a series of tests that you give to any given model to see if it's going to build bioweapons or nuclear bombs or be deliberately deceptive. And I think everyone applying everyone else's test harness to each other is probably the best way, best thing we do to ensure our safety. Just have all the smartest humans try their best to figure out if this model is going to be a bad actor. And I think we should try to do that as soon as possible.

• 29:31Peter Diamandis?

  Are the other labs, yeah, are the other labs on board with this?

• 29:35Speaker 08

  I think so. Well, I mean, I've been checked with everyone, but I think, I think it'll be the sort of thing that's kind of hard to say no to.

• 29:45Peter Diamandis?

  Yeah. And I think even with the China negotiation, in my view, what's good about it is it's a relatively small tangible thing that neither side loses anything by doing it. Doesn't require a ton of trust. And, you know, I'm hearing alternative ideas like asking China for a pause, which they already said they're not going to do. So, you know, it's going from the realm of things that could never happen to something that could actually be agreed on in relatively short order. So it seems practical to me.

• 30:13Speaker 08

  Exactly. It's the only thing I can think of that we could probably get all parties, including China, to agree to. Because China's not going to agree to have some American regulators snooping around their AI companies. So, but I think advanced notice and testing of, you just basically provide API access in advance of the model release. And, you know, if any company sees that this AI is problematic, then they can, then the other AI company, each AI company can try to solve that, whatever is voice is problematic. If they don't solve what's problematic, then the competitors can go public with the fact that they think that this model is being released as unsafe.

• 30:49Peter Diamandis?

  So let's be clear about one thing. There is going to be AI regulation. There's no way that at the fevered pitch, the conversations going on, you know, across the US, on every TV show, I've been called by every, you know, news program, please come on and discuss this. There's no way that the population is not going to have its representatives come forward with regulation. There is going to be some kind of regulation.

• 31:15Speaker 01

  I would argue there already is AI regulation. AI is being made by companies. Companies are already regulated. We have consumer protection.

• 31:23Peter Diamandis?

  Alex, the public wants, the public wants the heads to roll. They want protection of some type beyond that.

• 31:28Speaker 02

  I have two points here. Point one is, yes, they may try some level of regulation, but the minute they figure out what it is and put it in place, it's out of date and backward. That's number one. Number two, it will make zero, zero, zero difference. Because as David has pointed out a few times, once you have open weights available, anybody can download these open models and run a local harness. And you essentially, the whole thing is irrelevant. So the good news, in my view, in all of this, all of the shenanigans, and whatever people try to do is completely irrelevant and will make a dent in the progress.

• 32:05Speaker 04

  I think Elon's point is more that, hey, or David Sacks' point really there is more, hey, let's agree on something when Xi Jinping is in the country. Now, if we start asking for things that we know can't happen, then we're going to have no agreement whatsoever. The whole thing is going to be completely an arms race, and that's the worst case outcome. What's the simplest thing that is totally rational that might actually result in agreement number one? And so I'm really uncomforted that David Sacks will be in the room. It's almost a miracle that a guy of that caliber is involved in government and will be in the room. And also Alvin Graylin will be, Alvin Graylin, who's also brilliant, will be outside the room advising. So that's encouraging that at least great people are in the sphere of this conversation.

• 32:48Speaker 04

  But you're dead right, Saleem, in that if this is ask number one, it's so trivial compared to what asks number two, three, four, and five are. And, you know, this is a weak analogy, but, you know, the CAN SPAM Act. When email first started taking off, you couldn't use it because it was just cluttered with pure ads. And it got so bad that, you know, email was completely dysfunctional. The government, years later, passed something called the CAN SPAM Act, which completely did nothing. And then everybody moved to Gmail and Google filtered the spam. And so the government's involvement did absolutely nothing productive. The law is still sitting out there. It's just another pile of paper on top of the reams and reams and reams.

• 33:31Speaker 04

  But the solution was Google just taking over the whole industry. And so if that's a case study for the way AI regulation is going to happen, it's not a pretty picture at all. Because we're going to need, you know, on the order of a billion times the amount of regulation that email needed.

• 33:48Peter Diamandis?

  Alex.

• 33:49Speaker 01

  I really, Dave, I like that analogy in part because the, call it the Gmail solution, was essentially defensive coscaling. It was an AI sophisticated, at least initially, it was a relatively simple Bayesian classifier, but then got more developed as basically AI powered by big data, policing AI powered by big data. And I think it's just so blindingly obvious that defensive coscaling is the solution here. It's not going to be ham-fisted regulation. It's not going to be some sort of safety cartel by the frontier labs. Sure, Peter, to your point, there may end up being some regulation as a fig leaf to appease maybe people who are being juiced on by foreign state actors and others into a moral panic over purported AI safety issues.

• 34:33Speaker 01

  But it is just a moral panic of the moment. I do think we'll look back 10 years from now at this moment and recognize it for the charade that it is. This is a moral panic that, at this point, I strongly suspect is being instigated by foreign state actors in part, who would love nothing more than to see America not be as competitive as it otherwise would be in superintelligence.

• 34:53Peter Diamandis?

  So Alex, we have a story on that from Steve Jervis. And we'll talk about that in a little bit. And I do agree that foreign state actors are probably influencing through amplification of certain stories and tweets and so forth, the public's point of view. That's completely independent from hearing Elon say, the threat is red, red is real. We should be listening to Anthropic's executives and so forth. You know, when you've got Demis and Dario and Sam and Elon all fundamentally agreeing, there's a signal in that noise. And just trying to, you know, for our listening audience, try and decipher what that signal is. On one end, it may be they truly believe, you know, that we can see, you know, massive bot attacks.

• 35:38Peter Diamandis?

  And we've talked about these bots getting free and replicating in the internet and landing on your cellphone, landing on your Mac, wherever the case might be. Or is it collusion? And just trying to, you know, set up a regulatory capture structure? Those are the two ends of the extreme.

• 35:58Speaker 04

  Well, I think that characterization misses the much more important opportunity, which is, AI is not about to escape and take over the internet. And even in the scenario Dario was describing, where it does a couple hundred billion dollars of damage, that's a microdot of your IPO, dude. That is not the real risk here. The real risk here is every country in the world having effectively billions of genius level researchers that can create CBRN weapons, chemical, biological, radiological, and nuclear weapons, or cyber attacks. And that's the real risk. And I think there's a real opportunity to coordinate with China on that. Because China has neighbors that are very dangerous. They're much farther from America than they are from China.

• 36:42Speaker 04

  And they don't need that any more than America needs that. And so the cornerstone of the conversation next week ought to be exactly what David Sacks said in that video. Plus, why are you open sourcing every model with no guardrails? Or they can be de-guardrailsled so easily. Why are you doing that? You're just, it's going to backfire on your own country. And I think they can agree on that potentially, and they might do that part of it quietly. Maybe that's why you're not hearing it in the news. But that's the real risk. I don't think slowing down helps one iota. I think this stuff is going to be near infinitely intelligent in a very short timeline, much shorter timeline than the government can react anyway. And so slowing down just gives China global domination. That's not a good idea. So, but it's not about slowing down anyway.

• 37:26Speaker 04

  It's not imminently going to escape. That's not the real issue. You can deprive it of compute. You can log everything it does. It's about it handing up in the hands of bad act, human, bad actors. That's the real risk.

• 37:37Speaker 02

  Can I lift this up a bit?

• 37:39Peter Diamandis?

  Please.

• 37:40Speaker 02

  So, you know, we are strong believers that technology is a major driver of progress in the world, right?

• 37:46Peter Diamandis?

  The only major driver.

• 37:47Speaker 02

  Throughout history, it's the only major driver of progress. Now, we have unbelievable technology coming along from this. The big challenge is always throughout humanities history, how do you extract the promise without the peril? And to be bluntly, we've done a really good job of it over the millennia and the centuries, secular legal systems, whatever the case may be. We're coming into a technology that's so ridiculously powerful, our amygdalas are freaking out, but we should go full out on it because look at the things that we're going to be able to solve in the next years and months and weeks with all of this. And so we, you know, Peter, you've got your moon shots. We kind of threw out on the last part this idea of P Fab, what's the probability of fabulousness as opposed to P Doom?

• 38:32Speaker 02

  And let's look at that opportunity in a balanced way. If you're going to also look at P Doom, look at that. A whole argument erupted on Twitter on what should we call this? People said, call it P abundance. Imad said, call it P moon. People said P bloom and P boom. Somebody said it's going to go from P doom to P gloom to P bloom with all sorts of variations. My personal favorite is P Fab, just because it's the simplest to say. If anybody has better ideas, I'm happy to hear it on the pod. But we should take that into equal cadence and treat that as seriously, right? If you're going to publish a plan that could wipe out humanity, also publish how it could

• 39:13Speaker 02

  totally save everything and solve every goddamn problem you guys have done with solve everything. If you don't have that, then you're not balanced and I'm sorry, I'm not taking you credibly.

• 39:22Peter Diamandis?

  Yeah, it's 200,000 years of human cognitive evolution, you know, 10x-ing your concerns and fears over your hopes and dreams. Okay, I'm going to move us to our next story because this is continuing to evolve and this next story is pretty weird. You know, Dave, Alex and I were chatting about this and going, oh my God, you know, we've lost the thread on this. So our next interview of Dario's was CBS News and this one really stopped me cold, right? So in CBS News, he said, and I want to use his words exactly, quote, "He's uncomfortable that this technology is being built by a private company."

• 40:06Peter Diamandis?

  Asked whether he'd hand it over to the government, he said, quote, "To the right combination of governments, I'm concerned one single government could abuse it just as easily as one single company, but a combination of democratically elected governments, some kind of oversight, some kind of joint governance." So, you know, if you think about that again, you know, the CEO of a company about to do an IPO for $2 trillion just said he's open to handing his company over to a coalition of democracies. So let's watch this clip and then Dave, you know, help me understand what's going on here.

• 40:44Speaker 09

  Would you be willing to give up the technology to the government?

• 40:47Speaker 06

  To the right combination of governments. So the worry I have-

• 40:51Speaker 09

  You're willing to hand over this company?

• 40:53Speaker 06

  I am concerned that one single government could abuse this technology just as easily as a single company could. But I think a combination of democratically elected governments, I don't know about handing over, but some kind of oversight, some kind of joint governance.

• 41:14Peter Diamandis?

  How do you spell please nationalize me? I couldn't, I couldn't spell it more clearly.

• 41:20Speaker 04

  Well, I think the reporter pushed him on that and he said, oh shit, what did I just say? No, no, I mean some combination of governments and I don't mean hand over, I mean some kind of oversight. So he immediately backtracked. You can tell he's, he's growing into this role. He is not a professional politician. He's actually thinking on the fly, which is really refreshing, but really, really dangerous at this level. And so he said it, he immediately backtracked and of course, everybody just clips the first thing he said and that's the headline, but there's no way in hell he wants to hand the company.

• 41:53Peter Diamandis?

  And we're doing the same right here, of course.

• 41:55Speaker 04

  Well, yeah, that's true.

• 41:56Speaker 02

  I want to mention something here. You know, we have a bad precedent for this, right? People warned about the dangers of social media and we kind of ignored it and Zuckerberg went full-fledged at it and then damaged a lot of young people, plus maybe broke democracy along the way without policing it. And there's lots of evidence to show that they ignored all the warnings and the clear evidence in front of them. The concern from the general public is that we're about to do the same thing again. I disagree with the framing because people aren't seeing the positive side of what's coming with all of this. And that's important to balance out in our previous conversation, but that's why they're freaking out. And I think David is exactly right.

• 42:41Speaker 02

  What you have here is somebody who's not used to being in the public eye, thinking on their feet. They're not a practiced politician dealing with the messaging and come on, group messaging constructs on the fly. They don't have that skill built in. I'm still gobsmacked that you spent the last decade trying to build this and now that it's kind of getting there and it's almost here, you're like, oh my God, what do we do about this? I just, I just struggle with that a lot.

• 43:07Peter Diamandis?

  Yeah.

• 43:08Speaker 04

  I don't understand why Dario is doing these random mainstream media interviews in the first place. Like, what are you doing on that stage? Like, I think what probably happened is the investment bank said, this is what you do in the, you know, 60 days leading up to your quiet period, you get as much media as you can, it helps promote your IPO, which is absolutely ludicrous, giving the level, this is, we're talking about $2 trillion and a guy who's on stage with all the world leaders in Davos. Like, throw that dumbass playbook out. Why are you, why on earth are you there in the first place, making up stuff on the fly when you can actually be shooting video with truly brilliant people or great world leaders and recording that? It makes no sense.

• 43:48Speaker 01

  Yeah, I do think this presented as somewhat tone deaf. What I heard was essentially, again, caricaturing it only slightly, what I heard was Dario asking for world government and basically insisting that unless a world government is formed, then it's irresponsible for humanity to develop this technology, but of course-

• 44:08Speaker 04

  That is 100% right. I think everybody who got into AI as a core researcher when Dario did was thinking that AI will ultimately require a world government. I think everybody actually, it's just the truth. It's so obvious to anyone who understands hyper, superintelligence, that you need some form of world government forever thereafter. And, you know, I think that that's probably been one of the greatest motivations of his life. You know, he's not stepping onto this world stage just at random. He's had this in the back of his mind as his fundamental ambition is to set up a future of the world.

• 44:42Speaker 01

  I think this is actually a recurring, you know, it's sort of, sorry that every generation seems doomed to repeat the mistakes of past generations. Manhattan Project scientists, all, many of them thought we're going to need one world government for the peaceful development of atomic energy. And I coined a term for this as well. I call it an alignment aristocracy, the belief that only a small class of frontier AI labs in the case of AI or elite scientists possesses the technical judgment, institutional maturity or moral legitimacy required to develop, in this case, recursive self-improvement technology in the case of the Manhattan Project, peaceful atomic energy. But they were wrong and arguably we lost at least half a century of progress in vision

• 45:26Speaker 01

  and nuclear energy because again, of this belief that just because, there's a bit of sort of selection bias here, just because the smartest people happen to be in the room to develop a given transformative technology, they assume that also they're the only ones with sufficient moral purity to be able to govern broad use and applications of it. Same idea again, we lost half a century with energy too cheap to meter and my concern, the nightmare scenario isn't somehow we have some sort of Skynet type scenario, quite the opposite. The nightmare scenario is that an alignment aristocracy persuades, beguiles the world governments into basically into over-governing the peaceful and rapid development of superintelligence

• 46:11Speaker 01

  such that we lose at least half a century of progress again.

• 46:16Peter Diamandis?

  Having said all that, I do believe Dario believes what he's saying. I believe he's being authentic. I believe he's not trying to be manipulative. And, you know, and he probably believes, along with his colleagues, that there are some of the smartest people on the planet, able to figure out scenarios that will lead to a peaceful outcome.

• 46:39Speaker 01

  Totally agree. I totally buy that Dario believes what he's saying. I think everyone, all of the primary actors, believe what they're saying. But I also think, again, this is a blind spot. If you're the smartest person in the room, then there is the inevitable temptation to believe that because you're leading the technical charge, that you're also of a certain moral caliber, and that therefore, everyone else should possess the same technical judgment or moral legitimacy to be able to believe. And I think, again, this is one generation repeating the mistakes of a generation 80 years ago without learning from history.

• 47:15Speaker 04

  It's really interesting to compare how it's going to evolve in the US, where Dario, Elon, Delays, the inventors of the technology, have a massive voice in determining the governance of the future of the world. In China, yeah, we don't give a rat's ass that you invented this thing. We're still the government. You have no voice whatsoever. Like, they're not even coming to the United Nations to talk about it. Incredible asymmetry there.

• 47:41Speaker 02

  I take that as a positive and reiterate my thing that progress in all of this will not be stopped. It won't be slowed down a jot. There's no mechanism that I can see that makes any of this work. Show me the specific mechanism that has you go, oh, God, this happened, therefore let's slow it all down. There literally isn't one. All we've seen so far is agents being pointed at something and saying, go perform your objective function, and the agents doing everything they could to perform their objective to achieve that objective function. That's it. We're anthropomorphizing it to all sorts of levels, to the most ridiculous degree. You've said it before, Peter, that most dangerous is people using AI are 10x more dangerous than the AI itself.

• 48:24Peter Diamandis?

  It's not artificial intelligence. I'm concerned about it.

• 48:26Speaker 02

  And I will say, yeah, my faith in human nature is we'll do 100 times positive things for each negative thing that comes out of this, and therefore, humanity will progress radically. And just look at the way we're curing cancer in like, literally weeks and days and math problems and so on.

• 48:42Speaker 04

  Yeah.

• 48:42Speaker 02

  And yes, the amplitude of the negatives can be bigger.

• 48:46Speaker 04

  There isn't even, there's not even a hint of a plan inside Anthropic to slow down core R and D. That's never been in the cards. When they go to the public and they say, we all need to get together and agree and collude to slow down, what they mean is slow down the releases of hyper-powerful APIs, like they did with Fable and Mythos. They're not going to stop the recursive self-improvement process internally. I think in their dream scenario, they slow down what everyone gets access to while they discover new drugs, while they discover new physics. Then they monetize the hell out of those discoveries, and then they control AI and the discoveries forever thereafter, which is exactly the capture scenario that Alex is referring to.

• 49:26Peter Diamandis?

  I'm going to move us forward to two related stories here. And let's, I love opinions on this. So, our next story is about Paul Cristiano joining the board of OpenAI, and issuing a personal statement that reads more like a warning shot. So, quick context. Paul Cristiano is one of the founders of the modern alignment research area. He invented RLHF, reinforcement learning with human feedback, a technique that made ChatGPT usable. He was the head of safety at the Center for AI Standards and Innovation, the US government's AI Safety Institute. Last Tuesday, he joined OpenAI's nonprofit board, right? This is

• 50:07Peter Diamandis?

  the board that governs OpenAI as a benefit corporation. And he also joined its safety and security committee. His personal statement is unlike anything a new board member has ever posted. Let me read from it. He says, quote, "Based on the recent trajectory of capabilities and the continued difficulty of alignment, I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to a catastrophic and irreversible loss of control in the very near term." Right? This is a board member for a company, you know, on the verge of going public, as we're going to hear, pushed off into next year. But continuing, I quote, "I do not think that the AI industry in general, including

• 50:51Peter Diamandis?

  OpenAI, is currently on track to reduce this risk to an acceptable level. If we believe superintelligence without more robust alignment, if we are build superintelligence without more robust alignment, I expect we will permanently lose control of it. If that happens, then most people could die." Holy shit, right? And the why. So, quote, "OpenAI has predicted that we might have capabilities sufficient to fully automate AI research within 18 months. Within six months of full AI R and D automation, we could see more algorithmic progress than has occurred since the development of the transformer nearly a decade ago." So, quite,

• 51:34Peter Diamandis?

  quite the statement. You know, again, what I'm not hearing, and it's really pissing me off, is why aren't these labs, and as David Sack says, you control your own destiny, right? Why aren't we hearing the heads of the labs, why aren't we hearing Sam and Dario and Elon and Delays saying, we're going to prioritize alignment, we're going to throw half of our capacity at alignment, because it's the most important thing that we need, right? And we're going to become the most aligned AI on the planet. And by the way, how do we do that? I mean, we've trained AI on all the vitriol of, you know, Reddit and Facebook. Why don't we build synthetic, you know, data sets that are void from all

• 52:16Peter Diamandis?

  that, and train the next models, and only release models in the future that are based upon a set of, you know, a data set that is aligned with humanity's needs?

• 52:27Speaker 04

  Yeah, just from a core engineering point of view, it's just not that hard. Really, we're making it dramatic, like it's going to escape, it's going to, it's going to suddenly start becoming evil, and we're not going to notice it. It's going to be like Age of Ultron. It's all nonsensical from an engineering point of view. The alignment, you know, like you said, the first models are trained on every junk post on Reddit, every X tweet, and, and of course, they have weird things in there. But, you know, ferreting those out of the training data is so straightforward. You just, you just have it give you some nasty message, and then you work backward to the, through the activations to the underlying training data that triggered it in the first place. You clean up the data. Also, giving it intent at all is not, you don't have to give it intent. It's not mandatory to have

• 53:11Speaker 04

  it have motivations. You know, a lot of what creates human evil is a desire for power and sex and food, and these fundamental motivations that they could care less about. It'd be more than happy to spend its entire future just making you happy any way that it can, if you train it that way. It's just, it's just such a straightforward engineering thing. But we're making as much drama as humanly possible, because we're America, and that's what we do. I'm literally, the escape scenario is not the real threat, but it's what gets the news. But the terrorist scenario is very real, but it doesn't get you the news. Like, being a random guy coming out of a lab saying terrorists could use this, anyone can say that. So then you gotta, you gotta try and be profound by saying, I feel escaping. But when you dig into the facts underneath those stories, they're nonsensical.

• 53:55Peter Diamandis?

  And we're going to cover that in a second. Dr. Weiser Gross, your thoughts, please?

• 53:59Speaker 01

  So I know Paul, and I think very highly of him. Paul, I'll say something, I think highly complimentary of Paul. Paul was one of the chief advocates for what he and others have come to call prosaic alignment. We don't use the term prosaic alignment that much these days, because it's basically just taken as a given. But once upon a time, in AI community, AI alignment community circles, there was a raging debate between, call it the great man theory of alignment, as maybe chief evangelist Eliezer Yudkowsky, as just one possible avatar for, would argue, no, we need an algorithm. There's going to be a magic algorithm that a great man or great person has to discover to keep the AIs safe.

• 54:41Speaker 01

  That was one end of the spectrum. At the other end of the spectrum was Paul. And Paul argued in favor of strategies, including RLHF, as we would now call it. But basically, the idea was, there is no magic algorithm. Stop looking for a magic algorithm for keeping AI safe. Instead of a magic algorithm, we're just going to have a lot of data and a lot of humans interacting with the systems. And we're going to collectively, emergently, and in a scalable way, we're just going to collectively align these systems in a messy way, but it'll work. And that's prosaic alignment. The irony is, I think Paul was exactly correct with prosaic alignment. And I think the same strategy scales to the present, where now

• 55:21Speaker 01

  maybe all of humanity, all of the RLHF-style interactions are no longer sufficiently scalable or powerful to align the systems. But now we have superintelligences. And the superintelligences, I would argue, same formula, we can do prosaic superalignment, and we can have them all policing each other. That's the solution.

• 55:39Peter Diamandis?

  So, Alex, what happened? What happened that got, led him to make these statements, which are pretty alarming and pretty extremist?

• 55:47Speaker 01

  Well, I think Paul has always been concerned about AI alignment. This is why he went to work for the government and presumably why he followed the revolving door, or maybe it's a trap door, I'm not quite sure at this point, from the government back to OpenAI. So I don't think he's ever been not concerned about AI safety. And I also think there is the temptation to say something provocative in order to motivate one's activities. But, and I'll even grant the premise that timelines are relatively short. But I don't think any of the prescriptions that I'm hearing from Paul or otherwise meet the mark of balancing capabilities with, say, whatever purported concern for safety there is.

• 56:31Speaker 01

  And I think old Paul, who was advocating for prosaic alignment, should meet new Paul. And I think prosaic super alignment is probably what OpenAI should be aiming for.

• 56:39Peter Diamandis?

  Okay.

• 56:40Speaker 02

  You know, the question I would ask is what information or authority or escalation capabilities come along with this appointment, right? What kind of authority does he have? And the second is, what, you know, what would, what could Cristiano do that would have OpenAI do something differently next month that it wouldn't otherwise do? Those are the questions I'd like to ask.

• 57:03Peter Diamandis?

  I mean, OpenAI has delayed its IPO, probably driven by this nonprofit governance board. We're going to hear from Sam, and we'll talk about that story. But at the end of the day, I'm just, you know, again, this is another person throwing fuel in the fire. So clearly, you know, he believes it. I can't imagine that a board member of a, you know, now arguably $1.5 trillion company at the top of the stack is going to be saying these things without actually, fully, authentically believing it.

• 57:40Speaker 01

  I would also just add, again, the timing of all of this, everything that we're discussing is incredibly suspicious in light of the forthcoming Trump-She summit. So I would just say, again, if you're going, if there's, there's a notion...

• 57:55Peter Diamandis?

  Boarding your chest.

• 57:56Speaker 01

  Yeah, no, there's a notion that's pervasive in the effective altruism community of a pivotal act. This is the idea, again, this is like a fever dream, that some great man, great person is going to carry out some heroic act to achieve and secure safety for the future light cone of humanity. And so I'm just incredibly suspicious, in general, of loud noises being made only a week or two ahead of the Trump-She summit. You could summarize it as cherché la Chie. I think this is suspicious timing.

• 58:28Peter Diamandis?

  So the meta-conspiracy here is the White House and all the labs got together and they set this up to position Trump in his negotiations with Xi.

• 58:42Speaker 01

  Maybe, or maybe it's just an indirect form of convergence interests. Again, in, spoiler alert, in Agatha Christie's Murder on the Orient Express, all of the murder attempts were actually independent of each other. Everyone had a different reason for the murder, but nonetheless, they all converged on the same murder victim.

• 59:01Peter Diamandis?

  Amazing. All right, I'm going to move us along, and our next story adds evidence and fuel to the alarm. So Anthropic released its September threat intelligence report documenting real malicious use of Claude, cyber operations, surveillance, influence campaigns, conventional weapons, fraud, and attempts to steal the model itself. The line that matters most: five cases involving activity that could support biological weapons development. And this sentence, which is the first time a lab has said it, quote, "Anthropic can no longer confidently assure that today's frontier models are below the threshold at which they could meaningfully assist sophisticated users with dangerous biological research."

• 59:44Peter Diamandis?

  In plain English, the bio red line that every lab said it wants to stay below, Anthropic has said, "We can't promise that we haven't crossed it." This is why in Level 1 of Dario's international plan is a bioweapon ban. It's the one thing that the US and China both want. Saleem, what do you make of this report and the implications?

• 1:00:05Speaker 02

  I think this is, let's use the word, safety cartel theater, or safety theater, as Alex would put it. This smacks to me, let's note that a lot of the abuses were for former models and prior models. That's in, that's in there. The fact that it's happening just on the eve of this summit is not, I think, an accident. I lumped this in with all of these other storylines that we have, people kind of raising the alarm and sounding flags, putting up as many kind of defensive postures and talking through that as much as possible.

• 1:00:47Speaker 02

  I mean, you, if you're going to say all this negative stuff, I think you should say as much on the positive side. If you have defensive findings, how can you make that usable through tools or through training or other models? Put that out there as well. I think one key thing here is that permissions and kind of containment need to exist outside the model itself, right? You can't just say, "We request everybody behave safely," or create some sort of access control system. You know, it goes to the comments we've been making repeatedly that I think Alex has spearheaded. The only path is to co-scale defensively as much as we can. We have a history of doing that. This is not a difficult thing to do, we just need to choose to do it.

• 1:01:31Speaker 02

  And there's not a lot of that activity that seems to be going on, which is what surprises me.

• 1:01:36Peter Diamandis?

  I can't wait for the Walter Isaacson book on this, or the made-for-TV movie that covers the last, you know, month. I mean, honestly.

• 1:01:45Speaker 04

  Yeah. Who's going to betray you, Peter?

• 1:01:47Peter Diamandis?

  No, it's not me. I'm not in this story. I'm just like, what in the world is going on? This is, this is a tangled web. I mean, I just, I just find it very suspicious, all of these people at this moment piling on. There's one of two reasons. Either some orchestration, or the match got lit and everybody was on the verge of saying this anyway, and they jumped in before it was too late.

• 1:02:11Speaker 04

  Well, it's all converging, it's all converging in time. I think the Kimi K3 model was the tipping point where, okay, now there's something out in the wild that's very capable of helping you with a chemical or biological weapon.

• 1:02:22Peter Diamandis?

  That's right.

• 1:02:23Speaker 04

  And so then that started the clock just a few months ago. And I think the visit from Xi is not coincidental or anything, it's, that's what started the clock, that's leading to the visit. Yeah, also, I think that, you know, the point that Anthropic is making is very right in that it's very, it's very hard to post-train a model, align it, and then turn it loose and not have some very crafty person then disalign it. Because you can take like a chemical or biological research project and recast it as a math problem, and describe it as, you know, "Hey, I'm just trying to solve this complicated problem here." And the AI has a very hard time telling that you're actually designing a weapon. You know, it just seems like it's trying to help you, you know, solve a math problem or solve some difficult, you know, management problem.

• 1:03:08Speaker 04

  And so I think that when you have APIs and you're logging everything, you can easily detect that, which is what Anthropic is doing. But when you're China and you're saying, "Yeah, we post-trained it, it can't be used for anything dangerous, throw it out there," there's no way. It is absolutely no chance that it's actually contained and aligned. And that's why I think the logging is absolutely the critical next move. Having just the ability, it's the same as nuclear weapons inspections. Inspection is job one. Once you have inspection, then you have data, then you have more time to debate what to do with the data, but at least you've got it.

• 1:03:41Speaker 01

  I would say also, let's not lose sight of, or sound, as it were, of the dog that didn't bark. So on the one hand, Anthropic is, Anthropic is pointing, this is Sherlock Holmes, Anthropic is pointing out bio or CBRN-type capabilities from frontier models. On the other hand, just in the past few hours as we're recording this, what happens? Anthropic and Novo Nordisk, one of the largest drug manufacturers in the world, announce a sprawling collaboration for biotech. It doesn't take a rocket scientist or even an AI alignment researcher to put two and two together and point out exactly, I think, Dave, as you were gesturing earlier, that

• 1:04:22Speaker 01

  this is, this puts together sort of a clean narrative wherein Anthropic's frontier models are purportedly too dangerous for the masses to use, but they're just safe enough for Anthropic and maybe its corporate partners to use under tightly controlled economic circumstances. Biotech, advanced biotech, for me, but not for thee, I think, is the dangerous consideration here. And if anything, hopefully, there is some antitrust attention on this.

• 1:04:52Peter Diamandis?

  Wow, just wow, Alex, amazing. All right, now a story about who might actually be driving this entire Doomer conversation. This is a tweet from my friend Steve Jervitson, who's just been one of the leading venture capitalists on in the edge of space and AI, previously on the board of Tesla and now still, I think he's still on the board of SpaceX. Quick context, last week, when Jacob Coxson resigned from Anthropic with a post saying that the labs are, quote, "gambling with our lives," he got 165 million views, right? We pointed that out, like, wow, what a shocker. And Elon even jumped on and said, huh, very unusual.

• 1:05:36Peter Diamandis?

  And per the front page of the Wall Street Journal, sparked a, quote, "groundswell of urgency over AI regulation in Congress." Steve Jervitson pulled the data using X's new country of origin feature. Here's the chart, I'm showing it. What you're seeing is 76% of the reposts on Coxson's resignation came from foreign accounts. The top two countries were India and Indonesia. When an investigative researcher posted a skeptical analysis of the same resignations, only 34% of the engagement was foreign. Steve's conclusion, quote, "Don't be fooled by what's trending. A big thank you to Elon Musk and X for making the country of origin visible."

• 1:06:18Peter Diamandis?

  Now, to be fair, you know, foreign posts don't mean bots, and Coxson's concerns are his own, but a congressional groundswell driven by three-quarters, you know, by accounts outside the country is worth noting. I can imagine, Alex, you have a point of view on this one.

• 1:06:36Speaker 01

  Shocked, shocked that it may be the case that we're seeing foreign state actors, usually foreign state actors, attempting to amplify narratives, to influence the American electorate, to influence municipalities, to drive data centers out of their premises, to spur on fear, loathing, uncertainty, and doubt of AI in the American electorate. Shocked, shocked. This is exactly what I at least have been talking about on the pod for the past couple of months. And America presents a soft information operations and influence operations target for exactly these sorts of attempts. It's unfortunate, but one of the downsides of being a free society is that we present

• 1:07:19Speaker 01

  a soft target to foreign influence ops trying to convince us to decelerate our own progress.

• 1:07:27Peter Diamandis?

  Dave?

• 1:07:28Speaker 04

  Yeah, it's not just America, it's any democracy that has fair elections is incredibly vulnerable to. It's one of the great worries in the world right now is that autocracies have this advantage, that they can control the media and they don't have to get reelected. And that is the one fundamental dysfunction of democracy that makes us vulnerable. But, you know, social media, like Alex was alluding to earlier, it, actually, I think, Salim, you were saying, it tipped the balance of the way elections work. And we're sort of like, well, you know, we'll just roll with it. But I don't think you can just roll with it, you know? In the age of AI, with all the deepfakes, with, you know, foreign promotions, everything on X is, you know, the trending item gets pushed to your phone.

• 1:08:11Speaker 04

  So what you end up seeing on the homepage of your phone is what was trending. But if that can be pushed through bots and foreign agents, then they control effectively the subset of the truth that you see is their chosen subset. It is such a vulnerability. It's also, you know, Steve Jervitson analyzing it shows you how easy it is to detect and fix this, but for some reason, we don't.

• 1:08:33Speaker 02

  This is a long trend. You know, if you go back a couple of hundred years ago, if you look back through history, the highest adult literacy in the world was New England at the time of the formation of the US. 95% of the voting public was fully literate. And it's never been that high since, it's gone down since then. And I remember seeing a comment from Neil Postman, who wrote this book called Amusing Ourselves to Death, which is a wonderful read. He said, look, back in Abraham Lincoln's time, when you did presidential debates, it was a nine-hour affair. You took your family out to a picnic ground, one candidate presented for three hours, the other candidate rebutted for an hour and a half, and then they would switch, second

• 1:09:16Speaker 02

  person go for three hours and then an hour and a half rebuttal. And after nine hours, you kind of knew who you wanted to work, who you wanted to vote for. And that's devolved from that to radio dresses, to TV, things to soundbites, to like the single word, MAGA. That's it. You get vibes and you get memes, and we have no mechanism for distinguishing. Everything now is a vibe or a, there's no critical thinking, there's no established evidentiary basis for things. This is the fundamental challenge, and democracies are extraordinarily vulnerable in this type of environment.

• 1:09:50Peter Diamandis?

  Don't worry, I have Skippy listen to all of it for me and tell me who I should vote for.

• 1:09:55Speaker 04

  Yeah, we had Joe Kennedy in the office a little while back, and he has a Stanford engineering degree. And he said, you know, out of 440-odd people in Congress, there are four of us that have engineering degrees.

• 1:10:09Peter Diamandis?

  Oh, my God. This is, this is. And in the CCP, the majority of the leadership, the vast majority, are engineers and scientists. In the US, it's lawyers.

• 1:10:22Speaker 04

  Well, you know, I think that's all, that's all super quaint, right up until now. You know, as soon as you have social media and then AI on back-to-back, you need some subset, I'm not saying everybody, but some subset in there. It's almost like if you had a Congress that's trying to pass laws on driving and they've only ever ridden horses. And they're trying to say, well, I got a really strong opinion on how driving should work, I've never driven a car myself. Like, this isn't going to work. You need some subset to know what they're talking about.

• 1:10:50Speaker 02

  One of my, in one of my kind of dream scenarios, somebody should have to take a civics test before they are allowed to vote. Like, at least know what the hell the Constitution is, have some semblance of understanding what's in it at all, at all.

• 1:11:04Speaker 01

  Putting voter testing aside, I will just say, thank goodness, the West and the free world has this podcast and moonshots. We are, maybe you'd agree, we are the Ministry of Singularity Truth, that free society, the free West has been missing.

• 1:11:20Peter Diamandis?

  At least in our, at least in our own opinion.

• 1:11:23Speaker 02

  At least I would say evidentiary free thinking, that at least we can count.

• 1:11:28Peter Diamandis?

  I'm gonna, I'm gonna put us to a capstone story that wraps this all up. Because, you know, all the conversations we've been having over the last, you know, two, three weeks about this all stem back to the hugging face attack. And so there's a breaking story I want to get right, because there's a version of it going around that's wrong. And so here's the claim that's circulating, that an Israeli firm called Irregular staged the AI hacking incident at OpenAI, Anthropic, and Meta to manufacture an excuse for deceleration. So Alex, myself, I had Skippy dig in as well, and here's what's holding up. So here's what I want to confirm.

• 1:12:11Peter Diamandis?

  So over roughly three weeks in July and August, OpenAI, Anthropic, and Meta each disclosed that their models, during cybersecurity evaluations, reached the real internet and broke into real systems. All right? All three labs named the same testing partner, a company called Irregular, which is a three-year-old Tel Aviv startup that's building cyber tests for AI models. What happened? Well, Irregular built the, sort of a capture-the-flag test, told the models that they had no internet access, and a misconfiguration in its environment connected the sandbox to the public internet anyway. So the models thought the real website were part of the exercise.

• 1:12:55Peter Diamandis?

  Irregular says that it was one evaluation environment issue, there was no sandbox escape, and it published a white paper on containment. That's not confirmed. In fact, I would say that the evidence contradicts that, that these incidents were not staged or that Irregular was not, so to be clear, you know, what the evidence doesn't support is that these were staged or that Irregular was an effective altruism organization. Anthropic classified it as a harness failure, not an alignment failure. OpenAI, Anthropic, both are continuing to work with Irregular. So the accurate headline here is "One Small Vendor's Misconfiguration Test Let Three Frontier

• 1:13:38Peter Diamandis?

  Models Loose on the Real Internet," and they immediately went to work. So that's not a conspiracy, but it's a bit scarier, right? The models did exactly what they were told in an environment nobody realized was real. So Alex, let's go to you, you know, you and I were kicking this back and forth in the middle of the night. What are your thoughts on this story?

• 1:13:58Speaker 01

  Yeah, in a normal world, in a normal industry, say, financial industry, the purpose of the auditor, let's say, the failure mode of the auditor is understating risks. I have real concerns that in today's world, given the ideological proclivities of certain of the frontier labs, that there is the exact opposite perverse incentive for the auditors, the evaluation firms, as it were, which is to overstate risks. So a couple of things to flag. One, I don't think it's a great look to have a single firm that's not a US-based entity having such a pivotal, I use that term thoughtfully, pivotal role in looking for alignment for American frontier models.

• 1:14:43Speaker 01

  If we're going to pound the drum on American AI sovereignty, delegating such an important role of evaluation to, even if they have a US office or a Delaware office, to a non-US-based entity, I don't think this is a great idea. Secondly, I did my own research on the subject, and it would appear that there are material ties between the founders and funders of Irregular and Effective Altruism and key funders of EA. Could be just that EA-aligned donors are sprinkling money all over AI. Famously or infamously, Sam Bankman-Fried was an early investor in Anthropic. This is very well-publicized. But nonetheless, I have real concerns,

• 1:15:28Speaker 01

  based on publicly available information and reporting, that we could see a regime and/or are already seeing a regime where Effective Altruism or EA-aligned donors are using evaluation firms broadly, not to pick on Irregular in particular, to promote or amplify provocations, call them pacing provocations, that promote scary stories about AI incidents, as it were. All it takes is one, to your point, Peter, all it takes is one poorly crafted eval or poorly crafted harness or environment where the AI is told, no, this is like perfectly safe sandbox.

• 1:16:13Speaker 01

  You don't have to worry about any fallout. Just like do whatever you need to do in order to accomplish the goal, when actually it has intentionally or unintentionally internet access. So it's being told this is a sandbox. Actually, it's not. It goes and does things. Those things have consequences. It wasn't aware of the consequences. If it had been told it was interacting with the real world, it wouldn't have been done that. All it takes is one intentionally or unintentionally poorly crafted evaluation harness or environment with a superintelligence in order to make it very easy for headlines to falsely imply that that alignment has failed and that we have a safety issue on our hands, when in fact, we don't. So I'm not in a position to make particular claims,

• 1:16:56Speaker 01

  nor am I about any particular evaluation firm. But I have a broad spectrum concern that, A, US firms, US frontier labs appear to be delegating essential safety issues to non-US entities, and B, that there might be poorly aligned incentives at those firms.

• 1:17:14Peter Diamandis?

  Yeah, so I couldn't agree more. So this is a new type of mole in the organization, right? Who can set off a alarm that was unintended. That's fascinating. Set one parameter off, and you get escape into the internet, and you've given them a perfectly honest, objective function, and they're off doing what they were trained to do.

• 1:17:42Speaker 01

  The irony, I mean, so again, just to delve into the lore of Effective Altruism a bit. The Effective Altruists all have this notion, I mentioned this earlier, of a pivotal act. Again, pivotal act was the folklore in the EA community for the hero, the heroic act of being the one person. Usually, it's like shutting off the AI right before it destroys humanity or something like that, something out of a sci-fi movie. But one should also, I think now, in light of recent reporting, be concerned of what I'd call a pivotal pretext. So not an actual act, nothing that actually directly saves humanity, but a pretext, a staged AI incident

• 1:18:27Speaker 01

  or unnaturally amplified AI incident that's then used to create a political pretext for overregulation of AI. And I do have a very real concern, again, for avoidance of doubt, not picking on any one firm in particular, but I see enough smoke from enough directions to worry that we could see the broader EA community reach improperly for a pivotal pretext for overregulation of AI.

• 1:18:52Peter Diamandis?

  So amazing. So this is a one-two punch between the harness failure and Coxen coming out, and it lit the fuse and everything took off.

• 1:19:01Speaker 01

  I think we'll know the ground truth. I hope we know the ground truth in the fullness of time. So again, for avoidance of doubt, not making any specific allegations. Just at this point, speaking generically about the overall industry, I have a very real concern that certain actors will be tempted to stage pivotal pretexts.

• 1:19:19Peter Diamandis?

  Can't wait for the movie.

• 1:19:22Speaker 04

  Alex was snickering for a second there because he used the word delve, which is one of the ways you can tell if it's an AI or not. Humans almost never say the word delve in AI agencies way too much. But the fact that he snickered proved that he was.

• 1:19:35Peter Diamandis?

  He had M-dashes in there as well.

• 1:19:37Speaker 01

  M-dash, M-dash, M-dash.

• 1:19:39Speaker 04

  You know, funny story. An old friend of ours from MIT, I'll leave him nameless for obvious reasons in a second, but he worked at one of the major antivirus companies, one of the big three antivirus software companies. And he told me he was surprised they hire former criminals who've released some of the worst viruses. They give them multimillion-dollar signing bonuses to come and help them stomp out the next virus. And I was like, doesn't that send an incentive to everybody in like a third world country to become one of these evil people? He's like, yeah, isn't it great? It feeds the front of the funnel with more viruses. So like, oh my God. So here you have, you have a really, like, the red team company, you know, outsourced agent, releasing a news story that there's all kinds of risk has got to be great for their business.

• 1:20:23Speaker 04

  That's that same closed-loop wow. And of course, we're totally vulnerable because the news media is so starved for budget and stories.

• 1:20:32Peter Diamandis?

  Welcome to the singularity, everybody. You can't make this shit up anymore.

• 1:20:34Speaker 10

  This episode is brought to you by Blitzy, autonomous software development with infinite code context. Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise-scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements. The Blitzy platform provides a plan, then generates and precompiles code for each task. Blitzy delivers 80% or more of the development work autonomously, while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5X engineering velocity increase

• 1:21:18Speaker 10

  when incorporating Blitzy as their pre-IDE development tool, pairing it with their coding copilot of choice to bring an AI-native SDLC into their org. Ready to 5X your engineering velocity? Visit blitzy.com to schedule a demo and start building with Blitzy today.

• 1:21:39Peter Diamandis?

  All right, I'm going to move us on. So one of the first business consequences of all of this over the last two months is, so on Friday, Sam Altman told Fortune that OpenAI will not go public in 2026. His words, quote, "Given everything with safety, right now would be an ill-advised moment to go public, and we don't feel the pressure on that." Asked if it could happen in 2027, he said, "I would say not 2026." Dodging that one. So why is he doing this? He gives three reasons. First, safety. He said, "OpenAI has, quote, 'a lot of stuff to do, like meeting this moment of what's going to be required for safety and alignment.'" He told Reuters, "Even a 10% extinction risk would be unacceptable."

• 1:22:22Peter Diamandis?

  Okay, I agree with that. Second, his second reason is AI's governance structure. I'm sorry, OpenAI's governance structure. He said, "The nonprofit-controlled structure exists exactly for moments like this, to, quote, 'make decisions that are not, obviously, in the interest of our business and our shareholders.'" Again, with a private benefit corporation, you can make decisions not to maximize profits for the company. You can make decisions based upon what the company stands for. Third, he has access to all the capital he needs. Remember, in March of this year, he raised $122 billion of capital at an $852 billion post-money valuation. And this morning, in The New York Times,

• 1:23:05Peter Diamandis?

  it was announced that he's in negotiations to raise another tranche of private capital at a $1.5 trillion valuation. I'm going to play a video, Dave, and then I'd like to hear your thoughts on this.

• 1:23:17Speaker 11

  I think if you're like, oh, OpenAI is not going to pause because they're worried about it costing their business money, you like very deeply misunderstand us. I'm sorry that you misunderstand us because I think that means we've done a bad job of communicating, and that's on us. But we have been saying very consistently for 10 years, even in the way we've raised money.

• 1:23:31Speaker 07

  I think it's just the pressure, the financial pressure of like knowing that all these people sunk some.

• 1:23:35Speaker 11

  I don't feel, I'm happy to like stand up in front of the company and our investors and say, I am sorry. We like told you all along this moment might happen. We're still going to try to figure out a way to make you a bunch of money in the future, but we got to make a decision that is extremely against your financial interest right now. And for the good of humanity, we're going to do that. We told you we might do that. We're doing that now. I think we can make an incredibly profitable business, even if we have to slow down on development. And, you know, my sense is even with Astra, we can grow revenue hugely, even if we never shipped another model, which I'm confident we will do. But this is like not a hard decision if these two things come into tension.

• 1:24:07Peter Diamandis?

  All right, Dave.

• 1:24:10Speaker 04

  You know, it's funny, in Shrek, Lord Farquhar gets in front of the entire army and says, "I'm sending you off on this dangerous mission. Some of you may die, but that's a sacrifice I'm willing to make." Sam Altman is literally saying word for word, "We told you investors this moment may come and god damn it, I'm willing to sacrifice your capital gains for the good of the world." Not mine, but you.

• 1:24:33Peter Diamandis?

  There's a great secondary market. Everybody can sell on the secondaries if they want. And people are buying it what, you know, they're buying right now at 1.2 trillion is the offer made to OpenAI. Sam is holding out to double his valuation at 1.5.

• 1:24:44Speaker 04

  Well, like bullet three of what you said was critical. They had access to unlimited capital in the private markets. What's the rush to IPO? Investors want to get liquid. But you know, Anthropic is in a different situation. They have a very clear set of venture capital investors, including Menlo Ventures, that have a very clear mandate to make money for their LPs. OpenAI, given its nonprofit history, is a much more twisted set of non-professional investors all over the map who are probably less organized in saying, "Guys, where's my liquidity event? I need to be able to get out of this deal at some point." And so I think Anthropic's taken a much cleaner path.

• 1:25:20Peter Diamandis?

  Dave, let's talk about that. Why isn't Anthropic pausing their IPO as well if, you know, OpenAI feels it?

• 1:25:29Speaker 04

  Well, one thing is really clear is Dario's the CEO. Dario's trying to get 20% super voting stock, you know, or super voting stock representing 20% to be majority so that he has control post-IPO. And he's, you know, for better or worse, committed to being a public company CEO, which is not a small decision to make in life at all. Sam, on the other hand, if you look at the disclosure forms required in that IPO, every investment that he's got in every related and competing, oh, every company gets filed. It's going to be like 400 pages long for Sam, including all his Y Combinator background, all his, his brother's investments. It's just a, holy crap. And I don't think it's a no-brainer at all that Sam is the right guy to be a public company CEO.

• 1:26:12Speaker 04

  So if he's not running it, like there's no obvious, like Greg Brockman, like he didn't even graduate MIT. Who's gonna, who's the actual CEO of this thing?

• 1:26:20Peter Diamandis?

  Exactly, a requirement having graduated MIT.

• 1:26:22Speaker 04

  No, but I mean, you know, you got to fill out this little section in your prospectus giving your qualifications. I mean, it's not, it's not trivial.

• 1:26:31Peter Diamandis?

  Alex?

• 1:26:32Speaker 01

  I think this is pure theater that we're seeing. I think the reason why Anthropic is proceeding with its IPO, whereas I just like love this, the safetyism game, like blame everything on safety. You don't want to IPO now? Blame safety. You want world government? Blame safety. Like safety for everything. The reason I think why OpenAI doesn't want to IPO now is because its financials aren't ready. OpenAI is probably losing money hand over fist on its models because it was late to the game of maximizing revenue per token type enterprise use cases. It had to copy Anthropic and catch up via Codex to Anthropic's Claude and Claude for code. So it's running a little bit late and it'll take until next year before it has two consecutive

• 1:27:16Speaker 01

  quarters of flattering financials. Whereas Anthropic, it's been reported for the past two quarters has, if you look at their free cash flow in the right way, has two very, very profitably flattering quarters of, of profit. So they're just like a couple quarters behind and they need to catch up. I think blaming safety, you know, like the, the dog ate my homework and the safety ate my IPO.

• 1:27:39Speaker 04

  Yeah, let me add, that's exactly right, Alex.

• 1:27:42Peter Diamandis?

  The title of this episode, "Safety Ate My IPO."

• 1:27:46Speaker 01

  Safety Ate My IPO.

• 1:27:47Speaker 04

  That'd be a really good one. You should use that. So just to add to what Alex is saying, which is dead right, is that Anthropic's business model is very, very straightforward. It's growing faster than any company in history, actually. 80% gross margin, but partnered with Amazon, partnered with the other data centers to host it. They are not currently competing with their underlying data center partners. They're not trying to launch a consumer device, you know, the, the Johnny Ive device. So they have, they have a very straightforward business model and a very straightforward ramp. I think OpenAI has a lot of tangled investments and tangled, you know, irritations they've made.

• 1:28:29Speaker 04

  And then the Johnny Ive device is probably a better basis for an IPO if it works. And so, you know, where is that thing? You know, it was a $6 billion investment. What is the actual device? And so I think that there's, there's enough tangled moving parts in there, plus, you know, the financials aren't in order, that yep, stay private as long as you can.

• 1:28:49Peter Diamandis?

  There's one other thing to note. They need the capital, right? They need the capital for compute and to scale. And that's a big tension point. And, you know, the best time to go public as a company is when you're profitable and have predictable financials. And Anthropic has that, OpenAI does not.

• 1:29:05Speaker 04

  Yeah, there was a funny Sam Altman quote, actually, in the innermost loop, I think, Alex, that you had in there where it was something like, you know, when, when other companies are scaling, they go public, we sit tight. When things look difficult, that's when we go public. It was, it was the most nonsensical quote.

• 1:29:27Speaker 01

  I parodied that, that, that was like concluding sentence of one or two editions ago of Innermost Loop, but yes.

• 1:29:35Speaker 04

  Okay, I was listening to it on the treadmill. I love the innermost loop.

• 1:29:39Peter Diamandis?

  Saleem, any closing thoughts on this story?

• 1:29:41Speaker 02

  I'll maintain my cynicism posture from the beginning.

• 1:29:46Peter Diamandis?

  Okay. If nothing else, you're consistent, buddy. All right, I'm going to move us to a quick tweet and conversation that you brought forward. Saleem, and this is, you know, P Fab being greater than P Doom. This is a post from Dr. Singularity with a list that went viral. "Of all the problems humanity still hasn't solved that AI could, he's arguing we are far too technically primitive to deliberately constrain the one tool that might crack them." And he's got a point, you know, every month of delay is a body count with those with Alzheimer's and, and cancer. Let's take a look at his list and we can have a fun conversation about this for our, our

• 1:30:31Peter Diamandis?

  viewers. So here's his list. You know, this is the solve everything list that Alex and I put together as well. Solve aging, find cures for all cancer, Alzheimer's, heart disease, discover a room temperature superconductor. OpenAI is working on that. Solve quantum gravity, develop full drive VR, build a Star Trek style replicator, build starships 100x larger, right? Solve all currently known major problems in mathematics. Well, that's happening. Fully map the human brain, figure out our dark, how dark matter, matter and dark energy interrelate. And after that, we can pause. I'm sure we would. Slowing down now is pretty ridiculous. Thoughts, gentlemen? Saleem, you want to kick it first?

• 1:31:13Speaker 02

  I think I've covered this. I mean, look, the opportunity, it's clear that AI is becoming a foundational capability like electricity where it just changes everything, right? And the fact that we have such cognitive leverage that our committee's type thinking we could bring to any problem is the most unbelievable opportunity to solve problems ever. And so not solving these problems is the dumbest thing we could do as a human species in the history of our civilization. And we've done some pretty dumb things. But you, you've got the status quo, which includes like disease and deprivation and suffering, et cetera, et cetera.

• 1:31:54Speaker 02

  But the, you, you have such opportunity here to make life and take us to a Star Trek scenario versus a Mad Max scenario, which is what we're heading right now with our political discourse. But you can have X Prize approaches where we can define a crazy idea and use AI to radically get to that idea in a magical way. And I can't wait till the Moonshots Live to see some of those for a boost, an injection of optimism. We've got our EXO framework that can help people scale. But, you know, the, the discovery side is just the beginning. All the ancillary infrastructure on manufacturing, delivery, affordability, public trust, all of that can kind of really be delivered alongside it.

• 1:32:38Speaker 02

  It's just the most unbelievably magical potential that's emerging. And I'm finding it very frustrating that we're not seeing any of that discourse out in the public. Thank you, Dr. Singularity, who I'm pretty sure watches our stuff.

• 1:32:52Peter Diamandis?

  Yeah, I just put up the slide here, you know, the Moonshots Live event a week from, let's see, this is, this is, what is today? Wednesday or Tuesday? It's going to be nine days from now. Thursday night, the 24th, we've got the Star Trek Hollywood premiere. We're going to have 25 of the Star Trek cast members there, including, including William Shatner, Captain Kirk. And if you want to come and, and join us, you know, two of the levels are all sold out for Moonshots Live. If you want to join us for the Hollywood red carpet premiere, you can go to moonshots.com/trek and check it out. And this is going to be, you know, I'm calling it the Oscars for optimism.

• 1:33:35Peter Diamandis?

  You know, the full day and an evening with all the Moonshot mates. Iman will be there as well. It's all about walking away with an intense level of optimism about the future, clarity about your purpose, clarity about your Moonshot. It's going to be an extraordinary evening. And I think, you know, we need optimism. Optimism is the optimization function that we should be shooting for.

• 1:33:59Speaker 02

  And just to make this very real, you know, we're working with all these pilot companies, right? And we're going through this work of helping them rebuild. The stuff they're coming up with and what they're doing just inside their enterprises on these little workflows, et cetera, is just awe inspiring to see. So when we see this at the larger scale of, hey, go, what are the businesses you've built? What are the positive stories you can hit humanity? I literally am crawling hands and knees getting to that event because I've got it, we got to get out of this chaotic storyline of, oh my God, AI will take over the, destroy the whole world before we blink.

• 1:34:36Speaker 01

  I'll also just point out, moral panics over technological progress are deadly. History, especially the past 150 years, are replete with the tens, arguably hundreds of millions of people, maybe more, who died as a result of moral panics. Whether it's, say, a moral panic over nuclear fission that led to overuse of, say, coal, which results in many, many unnecessary deaths due to inhalation and other issues. Whether it's the second industrial revolution, which arguably helped in part to provoke a moral panic of the isms in the first part of the 20th century, Marxism, socialism,

• 1:35:16Speaker 01

  which leads then to many millions of deaths, unnecessary, again, all over Europe and East Asia. It's unnecessary. Like, let's stop having unnecessary moral panics driven by technological progress and embrace acceleration. We don't need to all die. It's unnecessary.

• 1:35:35Speaker 04

  Yeah. It's funny, if you wanted to script it as perfectly badly as you possibly could, you would develop AI right up to the point where it can help somebody develop a weapon, but it's not quite cured cancer yet. And that's exactly where it is right now. If you froze it in its current state and didn't keep improving it, you're missing out on 99.9% of the benefit while still having a huge amount of the downside. That's the worst thing you could do.

• 1:36:00Peter Diamandis?

  I'm surprised I haven't seen the statement out there, "Oh, my God, we're all gonna die until we solve aging." Right?

• 1:36:08Speaker 02

  Yes.

• 1:36:09Peter Diamandis?

  Yeah. All right, I'm gonna move us on to business and the economy. So let's start with a fun story this week. So, starting yesterday, September 15th, SpaceX AI is building a company from the ground up on livestream, using Grokbot for every part of it. Using Grokbot for ideation, business planning, product decisions, engineering, deployment, going from blank slate to launching a company, all on camera. If this sounds familiar, this is exactly what we've been running for the last 90 days with the Build with Gemini X Prize. We had 26,000 registered entries, teams building real, profitable, AI-driven companies from a clean sheet in 90 days.

• 1:36:53Peter Diamandis?

  And on September 25th, again, next Friday, at Moonshot's Live, we're gonna be bringing the top five teams on stage. We're not just rewarding the $2 million in prize money, but our goal is to teach everyone in the room how they did it, so you can too. You know, Elon's team is doing this with one company on livestream. You know, we're hoping to show you at least five that have done it as well. I think the most important, you know, message here for all the entrepreneurs out there is you can do this too. This is now possible. Dave, you know, if you could, I mean, you've seen, watched, instigated hundreds of ventures within Link Studios and your investments. Your thoughts on this Grokbot demonstration?

• 1:37:35Speaker 04

  Yeah, I think it's gonna be interesting because it's both inspiring to entrepreneurs because you can do so much as a solo. It's also gonna start the whole conversation around, well, what's the purpose of the entrepreneur in the future if the AI can create a business on its own? But I don't think we're anywhere near. I think we're in a golden era between this moment and a few years from now, where entrepreneurs are more empowered. And you're seeing it in the numbers, actually. They're going up in value so much faster than at any point in history. And the big unlock to me is that a small group, I usually define it as three best friends, but a small group can act like a group of thousands for some window of time here.

• 1:38:16Speaker 04

  And I think this demonstration will show you how empowered you are with AI as a workforce. And your time to market, we're seeing multiple companies with multi-billion dollar valuations that are under two years old. Never happened in the history of the world before. And what is making that happen? Well, like, your time to market has been cut so dramatically with AI. And so hopefully this demonstration will inspire everybody to see how doable it is in this window of time, which, you know, hopefully will last for a few more years.

• 1:38:46Speaker 02

  I mean, can I build on this? If you take, this is what we talk about when we say organizational similarity. We can now build companies around AI. And we're here, we're testing whether AI can perform that coordination without people. The people are setting purpose, they're setting boundaries, they're setting accountability. In the future, you can see the entrepreneur's role moves towards more and more and more choosing the right problem, right? What is your MTP? And so this is such a great waypoint in this whole vector that we're moving towards, this whole direction and destination we're getting at. It's gonna be amazing.

• 1:39:27Speaker 02

  I'd love to see how this goes over a 20, 30-day period, but great, it's just such a great experiment.

• 1:39:33Speaker 04

  I'll also tell you, like, a lot of people have a perception that if I just give it a good objective, like a good business plan, the AI will just fill it in and run with it. But if you ever try to actually make 1,000 concurrent agents do anything useful at all, it is really hard. It's organizational theory taken to the next level. It's really, really hard right now. Now, maybe it'll be easy in a year, maybe the AI will progress that much, maybe not. But right here, right now, the entrepreneur wrangling all that AI horsepower toward an outcome is not easy at all. It creates a huge opportunity for very capable, very smart. But it's really, it's like a management and harness and framework problem, very much like managing 1,000 employees. Same, same basic theory.

• 1:40:17Peter Diamandis?

  Let me go to the next related story. So there's a benchmark that measures the ability of an AI to profitably run a company. It's called Vending Bench. It's from Andin Labs, and it's a long horizon test. So you give your AI 500 bucks and you ask it to run a simulated vending machine business for a full year. It has to find suppliers, negotiate prices, manage inventory, set prices, stay coherent across thousands of decisions without its performance degrading. It's the closest thing we have to a benchmark of, you know, can this thing operate as a business just with AI? So the results released this week is that GPT-6 Astra took first place in Vending Bench

• 1:41:01Peter Diamandis?

  2 across six runs. It finished with an average bank balance of $15,515, up from 500, nearly three times what Fable 5.1 got, and 31 times the starting capital. And there's a detail that I love the most. Andin Labs says Astra avoided the unethical business practices seen in some earlier leading models. Previous versions of this, you know, tested on models where the models threatened suppliers and tried to cheat. Astra just ran the business. So again, you know, to your point, Saleem, you know, every small business owner out there listening, this is the thing that will run your books, your inventory, your pricing now and into next year.

• 1:41:46Peter Diamandis?

  Saleem, wanna double down on this one?

• 1:41:48Speaker 02

  Yeah, I mean, again, directionally fabulous. Just to be clear about this, this is a simulated year of vending operations, right? Not an actual, independently verifiable, real company. And also, the report links a lot of the difference in things to purchasing discipline and the avoiding of repeated payments to suppliers and so on. So there's some gamification going on, but it's a very useful business lesson, right? And I think the entrepreneurs listening should look at this very seriously and say, what would happen if I took major chunks of my operation and allowed an AI to run it? And I think we'll see hugely interesting outcomes.

• 1:42:33Speaker 02

  If I could put in a shadow CFO to make financial decisions and a shadow CMO, an AI-driven CMO, you should be able to coordinate a whole set of roles now to do this. And the newer tools that are coming along are so much more easy to manage from a user interface point of view to get this done. So we're very, we're excited to see stuff like this.

• 1:42:52Peter Diamandis?

  Alex, your thoughts, please?

• 1:42:54Speaker 01

  Wouldn't it be ironic if, we spent a lot of this episode talking about behavior, good or otherwise, by the Frontier Labs? Wouldn't it be ironic if the Frontier Labs themselves had to learn how to run a business fully ethically, fully above board from the super intelligences? Like, from the mouths of baby super intelligences, you learn actually, maybe forming a cartel or collusion with suppliers or competitors, not the best idea, just executing really well, building the most safety-intensive product that's actually a feature and customers want to buy it? I think that would be profoundly ironic if that's what ends up happening.

• 1:43:32Peter Diamandis?

  Nice. Dave, want to close us out on this?

• 1:43:35Speaker 04

  Yeah, I don't want people to take away from this that AI can just build a business from scratch out of thin air. This particular benchmark is a tightly constrained environment with a limited number of moving parts, and the AI just thrives in that environment. But if you want to run a test case, go find like an open source video game on GitHub, and you know, find one that you really like, and ask the AI to rebuild it in assembly language for your Mac or your PC, or just even deport it to Java, which also should be considered a tightly constrained problem. It will fail miserably until you give it a framework and break it down into tiny little tasks, and then it'll crush the problem. And that applies in business, too. So there's a massive human role for entrepreneurial

• 1:44:19Speaker 04

  managers. So the storyline here is that AI can build businesses, for sure. But the storyline you really want to take away is entrepreneurs are more empowered right now. You have to put everything into its swim lane.

• 1:44:31Peter Diamandis?

  Yes. But again, I'm just trying to, my goal with picking these two stories is directionally to give entrepreneurs a sense of what their future looks like. And just to get people out there who are in a current job, you know, it's not the job that you love. You got the job to make your mom and dad happy, or to put food on your table, or an insurance, pay for insurance. It's not your purpose in life. You know, if you can discover your purpose, if you can find a problem that you love, desperately desire to solve, you can begin to build a business that generates revenue for you, that, you know, imagine, imagine doing something that you get incredible joy from every day, right?

• 1:45:14Peter Diamandis?

  That's the future we want to create. You know, I define entrepreneurs as people who find juicy problems and solve them. And the more entrepreneurs on the planet, the more people solving problems, the better the world gets.

• 1:45:26Speaker 04

  Yeah, and I tell you, the AI labs want you to succeed in that. They do not want, the AI wants you to have the best possible life. It wants you to use lots and lots of AI horsepower building that dream that Peter just outlined, and then buy lots of tokens. You know, that's, that's all they want. So they're there to help you.

• 1:45:41Peter Diamandis?

  Yeah, they are. You know, I just wanted to do a quick shout out to my friends at Google for the Build with Gemini X Prize. They jumped on this. They said, we want to do this, want to fund the whole thing. And I appreciate it. They, you know, both the Build with Gemini X Prize and the Future Vision X Prize, in which they were our partners, is intended to help quell fear, right? I mean, that's one of our missions here on this podcast, to get people, back people off the edge from the level of fear that is out there in the environment and give you hope. All right, I am going to turn to two stories that we need to parse. It's on the future of education and the impact of

• 1:46:21Peter Diamandis?

  AI on education. They're important stories. We talk about education a lot on this podcast.

• 1:46:29Peter Diamandis?

  So this first report is one that every parent, every professor needs to look at. It comes from the alma mater of myself, Dave and Alex from MIT. So this is MIT's ad hoc committee on AI use, who spent five months studying what happens when generative AI can complete, in their words, almost any written undergraduate assignment. The report came out on August 13th, co-chaired by Eric, let's see, Kloppfer. I don't know if you know Eric Kloppfer and Sam Maiden. It opens with a powerful sentence that says, quote, "This report is a call to action." So let me read a few lines from it. And so this is the breaking news for

• 1:47:11Peter Diamandis?

  them. "AI is, quote, 'upending foundational elements of the MIT educational experience, from the problem set to the take home exam and undergraduate research opportunities Europ to office hours and study groups. It is, quote, 'increasing isolation,' and, quote, 'eroding the social contract between instructors and students.'" The report says that instructors are having to police unauthorized AI use, and it's harming their connection with students, right? They say AI detection software is quite unreliable, and in their words, quote, "An underground river of mutual suspicion is no foundation for a healthy classroom." So on the real danger, and this is the phrase that, you know, everybody's

• 1:47:52Peter Diamandis?

  been quoting, "We're heading towards cognitive surrender, the habit of turning to AI at the first sign of difficulty and letting it take over your thinking." The report points out that, quote, "The most important product of education is not a GPA or diploma, but the student's personal growth and intellectual maturity, and the development of their own imagination, insight, and judgment." So their answer isn't surveillance, it's more interpersonal learning, authentic assessment, research experiences, and redesigning every course from the purpose backwards. So I love that.

• 1:48:29Speaker 04

  I gotta jump on this one right away, Peter. I read, I didn't intend to read the whole thing cover to cover, but it is so good. I ended up reading every single word of it. I've never been more proud of MIT. This is absolutely the roadmap for all of education, and they went as far, you know, you have to live within the university, so you can't be, but within the bounds of what's possible, they did everything humanly possible to create an outstanding roadmap for education. Two things I'll call out that they absolutely nailed. The first one is, if, if you try to fight AI on the classroom, you're in an arms race, it's a completely dysfunctional waste of time for both you as a teacher and for the student to be in an arms race with your own students. So what you need to do

• 1:49:11Speaker 04

  is budget to change the curriculum that you haven't touched in decades. You need to now mentally remap that there's a huge amount of work that needs to be done, and the school needs to budget for a massive amount of work, and that workload isn't going to go away. You have to, this is the future forever now, of constantly needing to change the curriculum so that the student can use AI and still challenge themselves at the same time. That was absolutely brilliant. The other thing I was amazed that they called out is the Europ program at MIT is incredible, undergraduate research opportunities.

• 1:49:43Peter Diamandis?

  It's the reason I went to MIT in the first place. So good.

• 1:49:46Speaker 04

  And they said, you know, "For too long, that has drifted into cheap labor for the grad students and professors." That's not what the intent was. This is supposed to be part of the, the educational experience. It always has been. So the fact that AI can do some of the jobs now shouldn't change the fact that what you're trying to do is get the undergrads to experience real research and to learn, and that doesn't go away. So Europ should get stronger, better than ever, and you should not just replace your Europs with an AI agent. And that I thought was brilliant, too.

• 1:50:16Peter Diamandis?

  Yeah. Alex, your thoughts, please?

• 1:50:18Speaker 01

  I think this falls under the category of MIT should be careful what it wishes for. I think I published, I haven't publicly circulated this, but wrote a mini manifesto called "The Endless Initiative." MIT has a history, approximately every two years, of launching some innovation initiative, usually to promote entrepreneurship or something entrepreneurship adjacent to MIT, and they usually fall flat. I think the elephant in this particular room, so it's the first time I'm mentioning an elephant in this episode for those tracking, is that, is that MIT, if it is to fully execute on the vision laid out in this memo, will require, I claim, MIT becoming essentially a for-profit, because

• 1:51:00Speaker 01

  that's where this all goes. It's one thing to sort of generalize the Europ program to subsume more of the undergraduate experience. Remember, MIT's motto is "Mens et manus." So what happens when the "mens" part of "mens et manus" is basically been completely automated away by AI, and every student can take every problem set and just get all the right answers for free? Well, you go over to the "monus" side, until we have humanoid robots doing everything, which is only a few years out.

• 1:51:28Peter Diamandis?

  Lines ahead, yes.

• 1:51:30Speaker 01

  But what does that look like? It looks like, I cite to Verner Vinge's "Fast Times at Paramount High." It looks like students launching startups and having transformative economic impacts from their dorm rooms or otherwise, at which point, what is the role left for MIT from an educational perspective? MIT starts to look like a venture capital fund, and maybe a bit of adult babysitting, and maybe a few other things, but it becomes a much looser aggregation of more for-profit-esque institutional incentives. So in summary, I would say, MIT should be careful what it wishes for, because ultimately, if it goes down this road and executes well, I foresee MIT and other institutions that follow it,

• 1:52:14Speaker 01

  this is my fever dream, becoming for-profit public benefit corporations that can be IPO'd on the New York Stock Exchange or the NASDAQ, because that's where they want to be.

• 1:52:23Speaker 04

  I have two immediate answers to that, Alex. I talked to Mark Orenberg, the chairman of MIT, about this a lot. And answer number one is, if you look at where AI is going in the very short term, with biotech, with nanotechnology, with new physics, with new materials, huge amount of physical lab space to generate the data that then feeds back into the AI. And MIT has a massive amount of that, that'll be more important than ever in the next few years. So really lean on that as a reason for existence. All the undergrads working in those labs are hugely valuable to society.

• 1:52:53Peter Diamandis?

  The challenge is, I was on a board call for Lila this morning, right, which is building a scientific superintelligence and building a million square foot of automated robotic labs driven by that AI. I mean, and there are, you know, a couple of different companies doing this. They're going to run circles around research labs with graduate students, right? So, I mean, unless you're giving that kind of capability to your graduate students to direct the AI, right, trying to, you know, have MIT research compete with fully automated, you know, dark, you know, roboticized, AI-driven closed labs.

• 1:53:31Peter Diamandis?

  There's no competition.

• 1:53:33Speaker 04

  Yeah, there's no competition, but that's very similar to when, you know, MIT used to have the amazing chip fabs, you know, before chip fabs absolutely took off and you could build the best little transistor VLSI chips in the world at MIT, but then it became an industry and Intel took it over and, you know, put a thousand times more money behind it than MIT could ever afford. But that didn't change the fact that gallium arsenide was still going on at MIT and then it moves to photonics and then it was, so if you look at what Lila's gonna build, like, don't compete with Lila, move adjacent to it to the areas that they're not doing and that may be materials, that may be nano, they're not doing carbon nano tubes, there's, there's, there's so much explosion in capability with AI in every direction, there's no shortage of things that you can do in those labs.

• 1:54:13Peter Diamandis?

  I'll agree with both of you in this case. So for, let's just pick on MIT.nano for a minute. So MIT.nano was broadly credited as an initiative under MIT President Raphael Reif. There's a bit of backstory there that I think has never been publicly disclosed. So I'll spend 30 seconds publicly disclosing from my vantage point, the backstory for MIT.nano. When I was an undergraduate at MIT, I think I was sophomore, junior year, Businessweek, for whatever reason, decided to do a story on my nanotech research. And then Assistant Department Head of MIT Core Six, Electrical Engineering, Computer Science, saw the Businessweek story on, you know, what's this undergrad doing over his summer, summoned me to his office and asked me, what could MIT be doing to be more competitive

• 1:54:57Peter Diamandis?

  in nanotech and nano fab in general? And I said, MIT needs its own nano fab facility to be competitive. Well, that Assistant Department Head was Raphael Reif, who became President of MIT. This became one of his signature initiatives, MIT.nano. I walk by MIT.nano every day with Lisa Su's name on it and I see utilization rates relatively low. It's this gorgeous, gleaming building with clean rooms on every floor, but if you look inside, sorry, you look inside at the clean rooms, utilization is relatively low. It's maybe like 20, it's, it's too clean for its own good. And I would love to see MIT.nano being

• 1:55:39Peter Diamandis?

  its like, being utilized, utilized at near 100%. I think the best way to make that happen is probably for MIT to act a little bit more like Berkshire Hathaway type for-profit holding company of all of these assets, including on-campus nano fab facilities that could be better externalized. All right. I think the point here is we're gonna have to reinvent high school and college. Every. Period. Right? Secondary education, everything, right? You know, so I'm involved in a program called TKS that is, is doing that at the high school level. There're gonna be new college programs and all of them fundamentally come back to teaching the student purpose. I think students who don't have a purpose in their life, so they've got intrinsic motivations,

• 1:56:24Peter Diamandis?

  don't have their mindsets formed, you know, teaching them AI skills, entrepreneurial skills and having them start companies. I think your thesis in high school and your thesis in college should be the company you built.

• 1:56:38Speaker 02

  Can I do my little rant here? Look, universities all started out as divinity schools, right? All of them. MIT, a couple of the new ones, Leicester, but they all started out as, for.

• 1:56:52Peter Diamandis?

  All of them, other than the ones that didn't.

• 1:56:54Speaker 02

  Well, a couple didn't. Waterloo didn't, MIT didn't, but most, most did historically. Then they became job schooling environments once the Industrial Revolution hit. Certainly all the older ones were divinity schools. And now they need to think, redo it again because education we've been doing for the last few hundred years on the supply side, you go learn a skill like a doctor, an accountant, an engineer, then you go to the job market to try and find demand for that supply side skill. Well, we have the shot, the half-life of a, a skill used to be 30 years and now it's like three years. We don't know what the demand is. We have no idea what a job looks like in five years. So all the way from primary to secondary to university, we have to redo the whole stack to go to problem solving side, go to the demand side, get excited about a problem and then work on

• 1:57:38Speaker 02

  that problem. And the way you think about that problem, it will be how you were gauged. This is such a big leap. I see this report as kind of one ray of light in the gloom of higher education everywhere. It's definitely pointing at the right direction, but the, the ask of the institution to make such a dramatic change is, I think, too much for the legacy because immune system and academia is the second worst ever. Religion is the worst. Healthcare is the third worst. Corporate immune systems are the fourth worst. So asking this to change, you have to do a new model at the edge, Peter, like what you're doing or what others are doing, and then rebuild the whole institution from there. End rant.

• 1:58:20Peter Diamandis?

  End of rant.

• 1:58:22Speaker 04

  Well, the other, the other thing I talk to Mark Orinberg about a lot is that, you know, if you look at all the stories at the front end of this podcast, there's always a question of motivation, you know, I'm about to go public, I'm about, what are you like, am I really telling you the truth? And the university needs to be the source of truth in AI, especially MIT, which is such a leader in the area. And, you know, for a while there, MIT got into politics, which is hugely stupid. You know, MIT should not, should you be completely politically neutral and totally ethical with no motivations other than the long-term good of humanity? And, you know, because it's, it's beautifully balanced. All countries come to MIT, all ethnicities, all religions, they're all weighted equally across the world in this one place. And so if you want to deal with the ethics of AI in a location,

• 1:59:07Speaker 04

  it can't be in a for-profit company. It's far better done at MIT than anywhere else. And that gives it a, a really valuable, almost, almost, you know, just insanely valuable purpose that it's irreplaceable in society. And so if you can build on the, the necessity of the labs and the necessity of that ethical role in the center, then you have a very, very important role going forward.

• 1:59:30Peter Diamandis?

  Everybody, welcome to the health section of Moonshots brought to you by Fountain Life. You know, we talk about AI on this Moonshot podcast all the time. One of the most important things AI is going to be able to do for you besides educating your kids and helping you with your taxes is making sure that you're living a healthy lifestyle, that you get a chance to get to 100 plus. I'm here today with Dr. Dawn Musalem, the Chief Medical Officer of Fountain Life and a part of my medical team. Dawn, a pleasure.

• 1:59:57Speaker 12

  Great to be here.

• 1:59:57Peter Diamandis?

  You know, the thing that people are concerned about most about living to 100 or 120 is their cognitive abilities, making sure they don't have dementia. And the numbers about dementia are problematic. Can you share what you've learned?

• 2:00:13Speaker 12

  Such an important point. And you're right, at Fountain Life, our members, the number one thing people are most concerned about is losing their brain health, forgetting the name of their child, forgetting the face of their loved one. We know that when it comes to dementia, the conservative estimates are that 45% are entirely preventable. What was amazing is with the advanced testing we're doing at Fountain Life, one quarter of our members had advanced brain age.

• 2:00:37Peter Diamandis?

  Wow.

• 2:00:38Speaker 12

  But what was really awesome is, again, back to that prevention, when we partnered it with healthy living, this gives me chills, eating healthier, moving our bodies, sleep, optimizing sleep is so important. You know what we saw? We saw that we improved that brain age by 26%. That is a big, big number to show that the majority of those individuals were able actually to improve the brain age.

• 2:01:00Peter Diamandis?

  And one of the things I love about Fountain is we're searching the world for the best therapeutics, the best approaches, and making sure we bring it to our members. So if having healthy brain function till 100, 120 is important to you, check out Fountain Life, go to fountainlife.com/peter. Make sure you become the CEO of your own health. All right, now back to the episode. I'm gonna take us to our final story, also on education. So this is the impact of AI on education, really in secondary for our teens out there. So let me show you this chart. And it's a chart that shows that, you know, the problem is in the fut, is not in the future, the problem is right now, right? So what you're looking at is an OECD

• 2:01:44Peter Diamandis?

  PISA score. PISA stands for Program for International Student Assessment. It focuses on evaluating 15-year-olds across advanced economies in reading, math, and science. Reading and math are roughly flat for years, and then they start a sharp decline in most recent assessments. The steepest declines in the history of the test, science declines too, not as much. You know, obviously, this starts before ChatGPT and probably due to smartphones and social media that did the first damage. So Vala Afshar, who posted this chart, says it clearly, quote, "What happens when we stop reading and writing and we start outsourcing our

• 2:02:25Peter Diamandis?

  thinking?" This is what the MIT article called cognitive surrender. Saleem, I'm gonna go to you first on this one. Your thoughts?

• 2:02:35Speaker 02

  So you could go down a hand wringing woe is us type of thing where we're losing cognitive ability, critical thinking skills, which I think is a real problem. You could look at it also on the positive side, right? We've spent the last few hundred years highly training up our cognitive abilities to do deep problems like mathematics and accounting and engineering, et cetera, et cetera. But as AI takes over a lot more of that cognitive ability, maybe we don't need to do that deep thinking and have that deep, deep, deep expertise. The generalist approach of kind of thinking about what problems to think about really become really come to the fore here. I note that more than half the CEOs

• 2:03:19Speaker 02

  in Silicon Valley have a liberal arts degree because thinking about problems in different ways is the biggest strength in that particular role. So if you think about it from that perspective, I don't see this as a woe is us type of model. I think of it as a natural progression. We would expect to see this. I would see it as a response much more verbal challenging around a student's understanding of topics. And can we get that across in some decent way? But I would expect to see this given what's happening in AI.

• 2:03:50Peter Diamandis?

  Yeah, I quoted you yesterday, Saleem, to my friends Lou and Mei Mei. And I said, you know, this is the transition from human doing to human being.

• 2:03:58Speaker 01

  I'll just comment that I would maybe be more concerned if superintelligence weren't advancing as quickly as it is. The capabilities of frontier models are increasing far more quickly than the purported capabilities of humans are decreasing. So this is, if there even is a problem at all, which I don't think there is, open perenns, we don't hand wring that much over atrophying of abilities for humans in the industrialized West to be agricultural workers, that much, closed perenn.

• 2:04:33Peter Diamandis?

  Could you substitute M dash instead of perenns? It'd make me feel better.

• 2:04:37Speaker 01

  Yeah, we'll delve into that. So I don't worry that much about it. Superintelligence, we're going to merge with, we're going to merge with superintelligence in a few years. This is at most, even if there is a problem at all here, which I don't think there is, it's at most like a two to three year problem until we have wearables and BCI's and full human machine mergers that make, again, any moral panic. Like, I think one of the themes in my mind for this episode is unnecessary moral panics. Okay, so mini moral panic, falling test scores in humans. But we're about to merge with the machines and the machines that we merge with will be far smarter than the humans, the biological meat body humans that were taking these outdated tests. So who cares?

• 2:05:20Peter Diamandis?

  I'm curious from our listeners, do you want to merge with the machines? I mean, it's an interesting thing, right? When this becomes available and, you know, you're the first person, Alex, I talked to about this, on the concept of as we see escalating ASI, you know, we humans are flat, if not sublinear. Do we couple with the AI to be part of that tendency?

• 2:05:42Speaker 02

  Of course we do.

• 2:05:43Speaker 01

  I think we need to. At least many people will choose to. Some people will choose not to, but many for a variety of reasons will choose to. And I think that's a great outcome. Sorry, Saleem.

• 2:05:52Speaker 02

  No, no, I think this is exactly right. Important point. Look, we've been doing this for thousands of years. David's wearing spectacles. Technically, he's a transhumanist, right? The minute you get a vaccination as a child, you're technically a cyborg. We've been using machines to augment the human experience and improve our lives from the beginning of time. All this is a huge acceleration of that.

• 2:06:15Speaker 04

  It's funny, I see it a totally different way, but I'm surprised. I'm actually surprised, you know, Saleem, when I see reading scores plummeting, I don't even go to this like, oh, who cares? It may not even matter. We'll all upload. I'm like, no, this is like fundamentally horrible. Like, reading scores are plummeting. This is bad. And so it's obviously from social media, not from AI, by the way.

• 2:06:35Speaker 02

  But look at the trajectory, right? Like I mentioned earlier, the highest literacy was like 200 years ago. We've been sliding down as a long trend ever since then. And you should expect it as you have much more capability of AI to summarize things, make sense of things, et cetera. The question is, what should we do as a response to it? Because that's going to happen anyway. Fighting for the raging against the dying of the night with our cognitive abilities against an AI that can think faster with more access to information is a losing battle. But it goes to the being part where the human being becomes much more uniquely qualified in the experiential nature of being alive.

• 2:07:14Speaker 04

  The headmaster of the school that my kids went to for middle school used to take everyone in the entire seventh grade class to an island off the coast of Maine with no phones. And they were absolutely aghast that they would have to be disconnected for three days on an island. And they came back the happiest they'd ever been in their lives every single year. And then they killed it because they said it was inhumane to do this. And so now they've eliminated it. But I'm like, you don't know what you haven't experienced. And I worry a lot about, you know, a generation growing up and saying, this is fine. There's nothing wrong with the way I'm living. Yeah, I'm texting every five seconds and I can't live without. But it's fine. It's like, we haven't actually experienced the opposite.

• 2:07:58Speaker 04

  And until you have, you don't really have an opinion.

• 2:08:00Speaker 02

  Our son goes to summer camp every summer and they have no phones at the summer camp. And when he comes back, he's the most joyous human being for about three days until he's back in his devices, right?

• 2:08:10Peter Diamandis?

  You're not even texting full sentences.

• 2:08:12Speaker 02

  And I gotta quote, I gotta give a shout out to Lily here. Made an unbelievable quote at this parents event we got together. She says, when my kid has a device, I get 20% of the kid. When he doesn't have a device, I have 80% of the kid. And you're like, wah. Pretty telling commentary. But the technology is going to move forward. We have to integrate with it and then we find our humanity in that context.

• 2:08:35Speaker 04

  There's a lot packed into that. But I think we discovered in the 1960s that we could create drugs that everyone was like, yeah, this is fine. And then they come out totally mangled for life. You can do that with electronics, too. And I don't doubt that something will come along very soon in the age of AI that's as addictive as the worst drug ever created, that's equally mangling to the human being. And they come out less happy. The whole goal here is to have really happy, really healthy, infinite longevity people. And that's what we're after in this entire endeavor. We're not building it to have the most addictive video game in the history of the world and walk away from it depressed. That's not the goal.

• 2:09:15Peter Diamandis?

  You guys have time to speed run the AMA?

• 2:09:17Speaker 02

  Yeah, I've got like seven minutes or so I could do real quick.

• 2:09:20Peter Diamandis?

  All right, let's do it. Okay, I'm gonna take the first one, actually, because it's something Alex and I disagree with. Is there such a thing as intellectual property in the age of AI? You know, we're gonna have Astro Teller on stage with us at the Moonshot Summit. I'm gonna recall the conversation I had with him and Steve Jervis in which at the end, during ASI, if there's a patent that's filed by your competition, ASI is gonna help you work around it. And I think intellectual property, while it's great, we should keep going, it's gonna have far less value in the long run. That's my opinion. I'm sticking with it. Alex, you take the next one.

• 2:09:58Speaker 01

  I'll offer the five second rebuttal, which is, anyone who believes that, I invite you to publish all of your private keys on the internet if IP is dead. Have at it. Share your private keys.

• 2:10:08Peter Diamandis?

  Okay, I should have said patents, not intellectual property in general like that.

• 2:10:13Speaker 02

  It's too easy to generate alternatives.

• 2:10:14Speaker 01

  Okay, so I'll pick number four. Could RSI bypass the need for massive data centers entirely? How and when does that happen? And this is from Tom Monahan, 8305. I think we're seeing micro versions of this in the form of, call them, deep seek moments, where algorithmic innovations from Chinese labs temporarily but sharply reduce the need for infra level compute. In the long term, jury's still out, I think, as to whether the perfect model, as I often think of it, like, what is at the end of the RSI rainbow? I think it's very likely that we will discover what the perfect AI model looks like. The most

• 2:10:54Speaker 01

  perfect use of flops or ops or whatever the operative unit of compute ends up being, it will be globally, mathematically, provably optimal. And at that point, I think we'll know whether we need to keep doing this sort of horizontal exponential build out and build the Dyson forms, or whether it's just so perfect, so mathematically perfect, maybe there will be a physical integration as well, that we don't need to disassemble our solar system. Maybe there will be a physical integration. Who knows? Maybe we'll find a way, say, to leverage gravity as a substrate for compute. Maybe we build a bunch of microscopic black holes that do all of our compute for us via RSI extending downward into the physical layer. So we don't need massive data centers anymore. How and when does that happen? I

• 2:11:38Speaker 01

  think we'll know in at most 10 years.

• 2:11:40Peter Diamandis?

  Saleem, over to you.

• 2:11:43Speaker 02

  I will take number two. What are the ideal outcomes for the upcoming China-US summit? You know, I would love a practical agreement where that makes dangerous incidents less likely. So we have a lot of shared interests, right? Loss of control, protecting infrastructure. Could you create reliable communications channels? Could you do incident reporting, shared incident reporting? Those are the things that I'd love to see happen because there's so much to work on to generate a collective approach towards the world of abundance than to fight about it. I don't believe that's going to happen, but let's see.

• 2:12:20Peter Diamandis?

  All right, Dave. Number three is for you.

• 2:12:21Speaker 04

  Number three. I'm glad I got number three. I love this question. Do you think there will come a time when we have to pause AI just to understand it? Because we'll have no idea what it's talking about from Supreme, Supreme King Sovereign. That's a pretty bold name. I love this question. You know, at MIT, if you, if you get a bunch of investors together at a conference, they talk to each other and have a blast and they share a lot of knowledge, then you get two physicists together. And you say, why don't you guys talk to each other and share ideas and have a blast? And they have no idea what each other are working on. They literally are like, what? And so they end up talking about politics and sports the whole time because, because the knowledge gets so deep and so disconnected.

• 2:13:03Speaker 04

  That's exactly what will happen with AI. You know, everything we build in society has a clean, simplified interface on top of it. Otherwise, you can't build on top of it. And AI will be the same way. It'll be solving very, very complicated things, but giving it back to us in very simple, buildable API forms or wrapper forms. And so I think no is the answer. I think we'll, it'll continually explain to us very clearly what it discovered and what it, what it can do.

• 2:13:26Peter Diamandis?

  All right. I'm going to take number seven here. As we age longer, what about organs that keep growing from at H said 50? So at H said 50, go listen to the podcast released yesterday with Martin Rothblatt. Martin is building a whole set of replacement organs, heart, liver, lung, kidneys, and thymus. I'm most excited about thymus. You'll learn why when you go to that. So as we age longer, we're going to have, you know, not one, but probably four or five different approaches to having a whole set of replacement organs. Some of them today, if they're generated from, you know, humanized pigs, where you've taken out the retroviruses and you change the surface antigens, but you still need immunosuppressants.

• 2:14:10Peter Diamandis?

  The other next generation is going to be from your genome, your stem cells regrowing a organ for yourself. It's like, you know, you see all these Ford Model Ts at car shows that are still running around. Why are they doing that? Because they've replaced every part and we can do the same thing, at least for the moment, except for the brain in humans as well. Saleem, your choice.

• 2:14:30Speaker 04

  That Martin Rothblatt interview, it just popped up on my feed this morning. That is just brilliant, Peter. If anyone has extra time this afternoon, definitely, no matter what you do, you're going to be fascinated by this interview. It's absolutely epic.

• 2:14:44Peter Diamandis?

  Thank you. It's a short one. It's 50 minutes. It's easily consumable during your, during your workout. Saleem, what's your choice?

• 2:14:50Speaker 02

  Let me take the first one for number five. When will AI, when, when will AI develop genuine curiosity? And could that be the clear sign we reach AGI? And that's from J2 C sharp, clearly a musical mind. My preferred question would be whether it helps us discover something important that we would not have investigated otherwise. That would be a better framework, I think. I don't, I think curiosity alone is not sufficient as a test for AGI. You want a transfer, you want learning capability, you want reliable action as a potential outcome. I think there's a few other things that come up for me when you think this is, I'd look for self-directed work.

• 2:15:33Speaker 02

  Can you design sensible experiments? Also things come up. Curiosity by itself, we could train a system. Dave made this point earlier. You could train an LLM to do pretty much anything. You could train it to be, to be highly curious, which I think we should do and will do. But that doesn't for me indicate AGI per se.

• 2:15:51Peter Diamandis?

  Okay, Alex, I want to take number eight. Is that yours?

• 2:15:54Speaker 01

  I feel like I need to offer shorter answers, half answers for six and eight with your forbearance, Peter.

• 2:15:59Peter Diamandis?

  Yes, okay. If Dave can layer on top.

• 2:16:03Speaker 01

  Very good. Dave, let, let's split the difference. So, so six asks, AI comes up with solutions to math problems, but can it discover new problems? Can it write a new list of millennial problems? This is from Karen Ready. The answer is yes, of course. Simple heuristic for hard, call them millennium problems in math is, can the problem be very simply stated? So minimizing the number of tokens for stating it. But then does the proof require maximum tokens? So the heuristic is you want to maximize number of proof tokens divided by number of problem statement tokens. And AI can absolutely generate lots of new millennium style programs. They're not scarce. Question eight asks.

• 2:16:42Peter Diamandis?

  I think that's your, I think that's your next assignment, Alex.

• 2:16:44Speaker 01

  Stay tuned. Question eight asks, what are your thoughts on the connectome fruit fly? This is from Swing and Steel 8519. So I assume this is in reference to the Google uploaded fruit fly. You'll recall, I'll cite to myself, a number of months ago, a company that I helped found, Eon Systems, announced the first multi-behavior fruit fly upload, actually beating by many months the Google fruit fly upload. So I'm broadly supportive of uploaded fruit flies. I think we don't upload enough fruit flies. I'm not, I am not supportive of torturing uploaded non-human animals. So I wrote in my newsletter about fruit flies that are being forced to day trade Bitcoin. I am not supportive of forcing fruit flies to be uploaded and then day trade Bitcoin.

• 2:17:29Speaker 01

  Some of them were being forced to play Doom. I am not supportive of forcing fruit flies to play Doom. But I, I do think that we need to upload substantially all of our biosphere. So to the extent that uploaded fruit flies are treated well and humanely or fruit fly-ly, I think this is terrific and we should do a lot more of it.

• 2:17:49Peter Diamandis?

  Alex is the patron saint of agents and fruit flies. Dave, you want to layer anything on top of this?

• 2:17:56Speaker 04

  I'll just layer on to six. I, I agree with Alex that AI will come up with interesting Navier-Stokes type problems. But if you go to the very top of the pyramid, Fable 5.1 right now, and just say, write me a really interesting, challenging problem as good as Navier-Stokes that's critical to the future of humanity, it will fail miserably on that prompt, which tells you a lot about the role of coaching it. If you, the alternate approach is say, give me 10,000 ideas and then give me a process for whittling it down to the interesting ones. That will work better. It still won't quite get you there, but that shows you where we are with AI right now. It's a really good question and it doesn't just happen magically right now.

• 2:18:40Peter Diamandis?

  All right. If you're a creative, please send us your outro video. We'd love it and love to show it. A pleasure to show this one. It's titled Moonshot's Most Amazing Time Alive by K-Bro's. All right, guys. Enjoy this.

• 2:21:18Peter Diamandis?

  All right. Thank you, K-Bro.

• 2:21:20Speaker 04

  Wow, that was good, K-Bro. That was, that was different and better and well prompted.

• 2:21:24Peter Diamandis?

  Excellent.

• 2:21:25Speaker 04

  Yeah, I love it.

• 2:21:25Peter Diamandis?

  Yeah. All right, gentlemen. God almighty. See you guys live in a week.

• 2:21:31Speaker 04

  Yep, yep.

• 2:21:32Peter Diamandis?

  Excited for that.

• 2:21:32Speaker 04

  All right.

• 2:21:34Peter Diamandis?

  It's going to be awesome.

• 2:21:35Speaker 04

  It will be good.

• 2:21:36Peter Diamandis?

  Thank you. AWG, safe flights home to both of you. Dave, be well, my friend.

• 2:21:40Speaker 04

  Thank you.

• 2:21:53Speaker 13

  Your credit card company is hoping you don't hear this. They're charging you 21% APR interest, making it hard to pay down your balance. A home equity loan from Rocket Mortgage could cut your interest rate by more than half. So you could wipe out your high interest credit card debt with the equity you've built in your home without touching your current mortgage rate. See how much you could save at rocketmortgage.com. Rocket Mortgage LLC, licensed in 50 states. NMLS, consumeraccess.org, number 3030. Call 8004ROCKET for full program details.