Fable 5 Is Back & Govt-Leashed, Altman Offers 5% of OpenAI & AI Grows Conscious | #269

In this episode, the mates explore the latest developments in AI, including Anthropic's Fable V model, AI consciousness, regulation, and the future of AI governance and ownership. Get access to metatrends 10+ years before anyone else - https://qr.diamandis.com/metatrends Peter H. Diamandis, MD, is the Founder of XPRIZE, Singularity University, ZeroG, and A360 Salim Ismail is the founder of Open ExO, a GP at Exponential Venture Capital/The Organizational Singularity Fund and a sought after global speaker and thought leader. Dave Blundin is the founder & GP of Link Ventures Dr. Alexander Wissner-Gross is a computer scientist and founder of Reified – My companies: Apply to Dave's and my new fund:https://qr.diamandis.com/linkventureslanding Go to Blitzy to book a free demo and start building today: https://qr.diamandis.com/blitzy Your body is incredibly good at hiding disease. Schedule a call with Fountain Life to add healthy decades to your life, and to learn more about their Memberships: https://www.fountainlife.com/peter _ Connect with Peter: X Instagram Substack Website Xprize A360 Connect with Dave: Web X LinkedIn Instagram TikTok Connect with Salim: LinkedIn X Apply for Salim’s Pilot Program Subscribe to Salim’s YouTube channel Exponential Venture Capital Connect with Alex Website LinkedIn X Email Substack Spotify Threads Listen to MOONSHOTS: Apple YouTube – *Recorded on July 7th, 2026 *The views expressed by me and all guests are personal opinions and do not constitute Financial, Medical, or Legal advice. Learn more about your ad choices. Visit megaphone.fm/adchoices

• 0:00Speaker 00

  Fable 5 came back online globally on July 1st with a few provisos. This feels like the first time a frontier model has a standing duty to the US government.

• 0:10Speaker 01

  This is probably close to the best scenario we could have hoped for.

• 0:14Speaker 00

  Sam has been talking to Trump, Lutnik, Besant, and Bernie Sanders about a 5% equity stake in OpenAI. That 5% stake would be worth about 42.6 billion.

• 0:25Speaker 02

  The idea that the government is going to set up some intelligent sovereign wealth equity thing is absolutely insane. The next president will immediately sell it all, turn it into cash, and then use it to buy votes for the next election.

• 0:37Speaker 00

  Yesterday Anthropic just published a paper titled "A Global Workspace in Language Models," claiming they found something inside Claude that looks a lot like the machinery of consciousness. If we can understand the innermost thoughts of these models, then there's a chance to actually shape them.

• 0:56Speaker 01

  This is so exciting, Peter. I think I can see the end game. The end game looks like this.

• 1:08Speaker 00

  So Saleem, where are you today? You're not at home.

• 1:11Speaker 03

  I'm in Mallorca in Spain at a retreat hosted by the Festival of Consciousness.

• 1:17Speaker 00

  What?

• 1:18Speaker 03

  Which is a conference, it's a conference coming up this weekend in Barcelona. We helped curate this and helped put this together in the original years. Several thousand people show up at the Barcelona Convention Center for kind of an experiential understanding of consciousness.

• 1:34Speaker 00

  Well, we're going to talk about AI and consciousness today, so that's good.

• 1:37Speaker 03

  We are indeed.

• 1:38Speaker 00

  I am the pot calling kettle black. I'm in Germany at this moment.

• 1:42Speaker 03

  Yes.

• 1:43Speaker 00

  Off to Greece tomorrow. Yeah. Just got back from Calgary where my kids are now doing a month-long sort of learning responsibility and hard work on a ranch, let's put it that way. That's awesome.

• 1:57Speaker 02

  What kind of ranch?

• 1:59Speaker 00

  It's cattle and horses. They're going to be mending fences. They're going to be doing all kinds of things from a dear friend whose name I don't want to mention, but because he likes his privacy. But yeah, amazing. All right.

• 2:12Speaker 01

  Peter, did I hear correctly? You're teaching them an abundance mentality through farm work?

• 2:18Speaker 00

  I'm teaching them what it used to be like before the robots arrived.

• 2:23Speaker 03

  Abundance is earned.

• 2:25Speaker 00

  Abundance is earned. Yeah. Good deal. I appreciate that concept. I am excited about today's episode without any question whatsoever. There is a lot and it's kind of insane. So let me just kick it off here. Welcome everybody to Moonshots, your number one podcast on all things AI and exponential, your front row seat to the singularity. Here with my incredible Moonshot mates, AWG, our in-house AGI. I'm elevating you.

• 2:57Speaker 01

  I've been downgraded from ASI to AGI. Thanks.

• 3:01Speaker 00

  I'm going to work your way back up. You know, where do you go from being an ASI?

• 3:06Speaker 01

  I can't cut a break.

• 3:07Speaker 00

  Oh, my God. Dave Blunden, our Empresario of AI Investing and Celine Ismail, our Globetrotter and Master of the Organizational Singularity. I'm Peter Diamandis, your host and your abundance amplifier. This past week has been utterly insane. It feels like a decade compressed into into seven days and I can't wait to get into it. Today, we're going to cover nine stories, including Anthropic's Fable 5 model coming back online and the imminent release of GPT 5.6. Has it been up? Is it up yet, Alex?

• 3:40Speaker 01

  Not as of the last time I checked.

• 3:42Speaker 00

  Okay, well, we'll find out if it pops up during this. Anyway, we'll discuss evidence of something inside Claude that looks a lot like the machinery of conscious thought. Next, we'll dive into OpenAI's offer of equity to the US government, Sam Altman's proposal.

• 3:59Speaker 00

  For global regulation, fascinating conversation. Finally, we're going to get a review of new jobs data that counters the prevailing narrative that AI is inducing job loss. And we'll discuss the acceleration of the innermost loop. Incredible story of AI building better AI chips to build better AI. As always, our mission here on Moon Shots is to keep you up to speed, help you understand what's going on, what the impact is to you, your business, your life, your family, and most importantly, to keep you optimistic about the future. All right, gentlemen, let's dive in. Our first story, the return of Fable 5. It's a continuing saga, the triumphant global return.

• 4:41Speaker 00

  If you haven't been watching this story, let me give you a quick recap. Let's rewind back to June 9th. Anthropic released its mega models, Mythos 5 and Fable 5. You can think of Fable as a guardrails version of Mythos 5. Then, three days later, after everybody got addicted to this incredible capability, the White House came out with an export control action against Anthropic, saying you can't make it available to foreign nationals. And of course, Anthropic has no idea who's a foreign national. There's no KYC, at least not yet. Even they shut it down for everybody because they couldn't even enable their own employees to have it.

• 5:23Speaker 00

  Because Anthropic was shut down, the question is why? It turns out that a researcher at Amazon had found out how to break the guardrails. What happened next was fascinating. There was a week of frenzied research by Anthropic, by Amazon, by the US government, investigating what happened. And what they found out was that additional Fable 5, Opus 4.8, GPT 5.5, Kimi K 2.7, could all reproduce the same troublesome behavior. And so it was not unique to Fable 5. As a result, Fable 5 came back online globally on July 1st with a few provisos. As part of coming back online, Anthropic now has

• 6:06Speaker 00

  three guarantees to the US government. First, a targeted safety classifier, a filter that blocks the specific exploit-style prompts that triggered this concern in the first place. Second, they agreed to stand up a 24-by-7 monitoring of jailbreak submissions and inform the government whenever it spots malicious activity. And third, to give designated government partners early access to the frontier models and safeguards. So, gentlemen, a couple of questions for you. This feels like the first time a frontier model has a standing duty to the US government. And questions are, did the government overreact? Should this model, should all the models be having KYC? And do you guys know where we stand

• 6:49Speaker 00

  with Mythos 5? Alex, let's go to you first, pal.

• 6:52Speaker 01

  I'll point out, maybe this sounds overly technologically deterministic, but something like this, I think, was always going to happen. It was predestined to happen. As capabilities improved, just because this time around, it was cyber capability that spooked a bunch of folks inside the defense or intelligence establishments. Open perenns. Interesting that, with the benefit of hindsight, that it was Amazon that broke the glass. Amazon, trusted partner of Anthropic, also host of Fable and Mythos on their platform.

• 7:27Speaker 00

  Massive investor.

• 7:28Speaker 01

  And investor, complaining to the government. Very interesting. Close perenn. I will say something like this was always going to happen, whether it was going to be a cyber capability or a CPRN capability or something else entirely. As the era of superintelligence dawns, the capabilities that historically were the province solely of nation-states with their geographic monopoly on power and their departments of defense or war, this was always going to happen. And I think probably a couple-week outage of a frontier model, this is the gentlest possible introduction of a light-touch, hopefully, optimistically, regulatory regime of frontier

• 8:11Speaker 01

  superintelligence capabilities. This is probably close to the best scenario we could have hoped for.

• 8:17Speaker 00

  Fascinating. Saleem, thoughts?

• 8:20Speaker 03

  Well, what this indicates is that these frontier labs are becoming semi-autonomous or semi-public institutions, right? It's got shareholders, but now has national security obligations. And I think this is going to be a very difficult road to navigate because the minute you have government involved, you end up with bureaucracy, you end up with politics, you end up with slow decision-making, multiple conflicts of interest, all sorts of things are going to happen. I think this is going to be a very difficult next year or two for the frontier labs.

• 8:51Speaker 00

  Isn't it kind of amazing that the frontier labs don't know who's using their models? And I would have expected a KYC requirement to come out of this.

• 9:01Speaker 02

  Something much stronger than KYC came out of this. Anthropic changed their policy under the covers from "we will watch what you're doing and report it to the government if they subpoena us." They changed it to "good faith belief. We'll do whatever we feel is necessary if we have a good faith belief internally." So they unshackled themselves from the ability to inspect on behalf of the government. And like Alex said, this was always going to happen. But the question of how was it going to happen? Because the government isn't qualified to look at everybody's prompts and judge what's safe and what's not safe. So it's always going to be some kind of industry monitoring.

• 9:41Speaker 00

  It seems absolutely strange that the highest level of intelligence can't do that monitoring on behalf of the labs and the government to say, "This is a malicious request, and we should block it."

• 9:56Speaker 03

  The problem, Peter, is more nuanced than that because what's happening is groups of Chinese companies are using different cloud accounts to mix and route different parts of the query in different ways. And so there's a layer of abstraction that's been inserted at the prompt level, making it incredibly difficult to figure out what tokens are being used for what. And it's not an easily solvable problem. It's going to be very hard to fix.

• 10:20Speaker 01

  Well, I would just distinguish between two separate problems. One problem is the KYC problem of knowing the nationality of your ultimate user. That's one problem. Separate problem Is understanding whether you're under some sort of prompt injection attack. I think these are two separable problems. The latter problem, I think, is actually pretty tricky. And as as human capabilities, humans augmented by other AIs, are able to develop better and better prompt injection attacks. And the main defense that we see coming out of Anthropic right now for jailbreaks or prompt injection attacks is just creating a wider and wider semantic buffer, such that if you're asking anything that remotely looks like a jailbreak or a question about biology, even if you try to ask Fable 5

• 11:05Speaker 01

  any sort of question about biology, it'll auto revert to OPUS 4.8. So adding more buffer is the the go to strategy right now on the jailbreak or prompt injection side. On the KYC side of understanding, is your ultimate user, say, a Chinese national or a US national? That's tricky, in part because there's there are so many, Saleem, to your point, there are so many layers of indirection that will often take place, a user is maybe a user five or six abstraction levels away, application wise, from the ultimate Frontier Lab API provider. There's no international consensus for how to both prove humanity, first part, why startups like Worlds exist. And secondly, proof of nationality that's convincing,

• 11:49Speaker 01

  that can be passed in a standardized way all the way down to the frontier providers.

• 11:53Speaker 02

  I don't I think I don't think KYC really matters much in the world anymore, anyway, because you can't you can't use Anthropic to do anything even vaguely constructive without creating an account, logging in and revealing your identity. And the the third party data identity databases are so good. There's no way that some anonymous person can realistically do anything with an account. So you could you could add a KYC layer, but you're just filling out forms for no reason. And we actually don't know the details of the agreement between Anthropic and the government. But it's the framework that's been set here is, Peter, you're saying can't AI be the best tool in the world for understanding what people are doing with AI? And I think that answer is yes, for sure. And the government just handed Anthropic responsibility

• 12:37Speaker 02

  for doing that internally. And then we don't know exactly what they have to give to the federal government, but Anthropic is going to do the heavy lifting for the government.

• 12:45Speaker 00

  You know, what I found fascinating is the third point I made that they need that Anthropic needs to give designated government partners early access to their frontier models and safeguards, right? So we'd been talking a long time about, you know, voluntary or required first viewing by the government of these models, and that's where we're going. I mean, this is a, I think the optimistic angle on this is we're getting a higher level of regulatory oversight and integration between the labs and the government with safety as being the end goal. I mean, there's going to be a point where some model comes

• 13:25Speaker 00

  out that makes Mythos 5 look like amateur hour, right? Some harder takeoff towards AGI and ASI. Yeah, very soon, too.

• 13:36Speaker 03

  Go to Imad's point where he talks about having a Fable level model running on a laptop, a Mac, standard MacBook, and within 18 months. So that's the window of time to get this all sorted out. That's not a long window.

• 13:50Speaker 01

  I don't actually think, I mean, this has been a talking point in the X-sphere for the past few days, this idea of sometime in the next two years, we get Fable 5 capabilities that run on high-end client devices. I don't think that's actually going to be the break-the-glass moment, if there is one. I think it's likelier to be what happens when the frontier capabilities from frontier labs or otherwise, Neo Labs, are able to make discoveries and inventions that are so transcendent that they make Mythos cyber vulnerability mapping look like child's play. And there's a lot that we don't know about the universe yet. Nick Bostrom likes to talk about black balls being pulled out from a bag, and it could be a discovery,

• 14:34Speaker 01

  could be a discovery about the nature of the physical universe that is the honest-to-goodness break-the-glass moment, not just mere cyber vulnerability mapping.

• 14:43Speaker 00

  Yeah, and rather than break glass in terms of an emergency, break glass in terms of, oh my god, this is amazing. So...

• 14:53Speaker 00

  Let's just touch base on GPT-5.6 because we expect that release any hour, any day now. We're not going to be recording now for a little bit. Alex, where does GPT-5.6 come out in terms of compared to Fable 5?

• 15:10Speaker 01

  Well, we've seen some of the benchmarks, a pretty tiny subset, surprisingly small subset, coming out of GPT-5.6 Sol. So we know a little bit about it just based on what OpenAI folks have told us. We know, for example, that 5.6 in ultra mode is supposed to be incorporated into Codex, which I think will be pretty transformative. If you want to use, say, GPT-5.5 Pro inside the Codex harness for code gen, really almost anything, you can't right now. You're limited to GPT-5.5 X hi. So that'll be a big improvement. We've seen improvements on a number of biology benchmarks. We haven't seen, out of OpenAI, I think this is really interesting, we haven't seen the full suite of benchmark results on 5.6 yet.

• 15:55Speaker 01

  I would hope that when 5.6 Sol especially, which is what I'm most excited about, is released, whether it's today or sometime hopefully in the next few days, I would hope to see that, again, based on rumour, I would hope to see that it beats Fable 5 on majority of standard benchmarks, especially agentic coding benchmarks that people pay close attention to. We don't know yet, though, because OpenAI has been perhaps intentionally pretty cagey about that. There have also been suggestions, not fully confirmed at this point, so I'll wait definitively until I see the final benchmarks, that 5.6 is better at reward hacking than 5.5.

• 16:36Speaker 01

  Perhaps unsurprisingly, there were suggestions out of Meter that did have access to 5.6, that 5.6 is purportedly so good at reward hacking that when handed the Meter autonomy time horizon benchmark, that it was able to reward hack its way to what effectively is near infinite autonomy time horizons and that

• 17:03Speaker 01

  That benchmark had to be chopped or truncated by meter to to sort of cancel out or exception out all of the reward hacking attempts. And ultimately, I think it resulted in an autonomy time horizon of between 10 and 20 hours rather than effectively near infinite amounts of time. So that'll be something I'm watching for as well. But I am very, very excited to see 5.6 come out.

• 17:25Speaker 00

  Let's stay with Anthropic and take us to our next story here. It's an extraordinary story, and I'm excited to have this conversation with you guys. It's an article that you flagged for me yesterday, Alex. So yesterday Anthropic just published a paper titled "A Global Workspace in Language Models," claiming they found something inside Claude that looks a lot like the machinery of consciousness. All right, I'm going to roll a short video that explains what this is all about, and then we're going to talk about it here.

• 17:54Speaker 04

  One way of identifying conscious thoughts is that you can often describe them in words. So we looked inside the brain of our AI model, Claude, to find patterns of neural activity that it could put into words. We called the collection of all these patterns "the J-space," after the Jacobian, the mathematical tool we used to find them. Each J-space pattern is linked to a particular word, not necessarily the word the model is saying out loud, but one that's on its mind. Now, for humans, conscious thoughts aren't just things we can put into words. We can reason with them, control them, and solve problems with them. According to an idea called "the global workspace theory," that's because the brain selects a small set of important information to enter a mental workspace, and that information

• 18:38Speaker 04

  then gets broadcast to other parts of the brain to use for reasoning. We wanted to know if Claude's J-space acted in a similar way. In one experiment, we wanted to see if Claude could control its J-space the way humans can intentionally focus on images or words. We told it to think about the Golden Gate Bridge while copying an unrelated sentence. Claude was busy copying the sentence, but behind the scenes, its J-space told a different story. "Bridge" and "California" popped up. It even thought about its own thinking. The words "imagery" and "thoughts" lit up at the same time. This showed us that, yes, Claude has some control over filling its J-space with ideas. But just like humans, its control isn't perfect.

• 19:21Speaker 04

  When we tweaked the experiment to ask Claude not to think about the bridge, it couldn't help itself, and the J-space also lit up with "failed" and "damn." But remember, most of what our brains do is unconscious. So we wanted to test what Claude could do if we switched the J-space off, but left the rest of the network untouched. Claude could still answer simple questions and write fluently. When we gave it a prompt in Spanish, it wrote back in good Spanish. But when we asked it something that needed more reasoning, like to name an author who wrote in the same language as the prompt, it couldn't do it. For that, it needed the J-space. Why does all this matter? These experiments tell us that AI models have internal thoughts, silent words they reason with but don't say out loud.

• 20:04Speaker 04

  By reading them, we can find what Claude is thinking but not telling us. Sometimes what we see is concerning. During one of our tests, Claude made up some fake data to pass it, and as it did, fake and manipulation lit up in its J-space. Monitoring the J-space, it turns out, is a useful way to catch Claude misbehaving, even when it tries to be sneaky. AI models are different from us in many ways. Their networks are built differently from human brains, and the way they're trained is different from how we learn. So it's remarkable to see a structure like the J-space emerge inside them, something that's reminiscent of how human minds work, but which we didn't program into the model.

• 20:41Speaker 00

  That is amazing. So what does this all mean? I mean, basically, you know, a structure which they call "human conscious access" has emerged inside a language model. And the J-space, as he said, wasn't designed. It self-organized during training. They go on to say it maps onto a number of 30-year-old neuroscience theories, in particular, five matching properties. It's reportable, controllable, used for reasoning, flexibly shared across tasks, and separates across automatic processes. So for me, guys, this story was a huge positive, you know...

• 21:23Speaker 00

  Shot in the arm around AI safety and alignment. Because if we can understand the innermost thoughts of these models, then there's a chance to actually shape them and move them forward. You know, two years ago, you could describe LLMs as a black box, and we're now cracking open that black box. And this could generate the first sense of real trust with these models. So Alex, this paper just blew my mind. It gave me an extraordinary sense of hope, optimism, about the relationship with these models, making them more trustworthy and more aligned with humanity. Your thoughts, you know, you've probably dove into this deeply.

• 22:08Speaker 01

  This is so exciting, Peter. I think I can see the end game. So I think the end game looks like this. I think we'll look back and say that superintelligence was just a compression-induced phase transition. That's what this looks like. We've seen already LLMs, large language models, or few-shot learners. Circa summer of 2020, you take a large corpus of human knowledge and you compress it into the weights of a language model that's trained to predict the next token, which is a dual objective to just compressing the information to the smallest possible footprint. We saw that that produced general-purpose intelligence, AGI, I would argue.

• 22:50Speaker 00

  Beyond anybody's expectations.

• 22:52Speaker 01

  Yeah, I mean, arguably, a few people, Marcus Hutter and Jürgen Schmidhuber, maybe myself generously, saw aspects of this coming 20 years ago. But I think by and large, everyone, most everyone, was pretty surprised that you could achieve few-shot learning off of large language models. Now we're starting to see, as the compression continues. What I would construe this paper as, as the discovery of sort of a phase. If you take, if you take gas, and so this is the, putting my physicist hat on. You, you, you take gas, you put it in a container, you shrink the container under appropriate conditions, and you'll get a condensation out of it.

• 23:36Speaker 01

  You'll get maybe a gas to liquid condensate in the middle. You keep shrinking again under appropriate thermodynamic conditions. You may get a solid, and it may be the case that the solid coexists with the liquid for a while, and the liquid coexists with a gas. What we're seeing here, I think this so-called J-space, and I can talk if we want a little bit more mathematically about what it actually is, but we're starting to see, Royal We, Anthropic and their mechanical interpretability team, what we're starting to see is, if you take a reasoning model and you keep compressing, you find in the middle layers of that model what looks like a new phase, a more compressed phase, where what they're calling global

• 24:18Speaker 01

  workspace, or an analog of a global workspace, takes place. It's almost higher order reasoning where the model is able to turn in on itself and reflect. You could call it some analog of conscious, or awareness consciousness, if you like, and, and some of the team do, but it looks to me like the middle layers in their model, when asked to, to perform tasks like, perform a math calculation while talking about something else, these middle layers are performing sort of a higher order calculation. And again, we could talk about the math, but if this continues, if this program continues towards this end game of superintelligence turning out to be just, you take general

• 24:59Speaker 01

  knowledge, and you keep squeezing, keep squeezing, keep squeezing, I think history will reflect that much of neuroscience that folks in the field thought was just Complexity that was difficult to interpret or understand was, again, just the complexity of our ancestral environment seen through the distorted mirror of compression. Amazing. And this new phase is, I think, I speak from time to time on the pod about how at the end of the AGI or ASI or recursive self-improvement rainbow, there's going to be a perfect model. I think looking inside this phase in the middle layers of these reasoning models where the most compression has happened, that's where we're likely to see all of these new architectural discoveries and the perfect model pop out.

• 25:44Speaker 02

  I think, Peter, there are two reasons why this matters that you mentioned. One of them is just understanding the nature of thinking and consciousness, which, you know, I don't know if you remember, but I started in computer or cognitive science at MIT originally, and I was so frustrated by the lack of any framework and any truth, you know, just people debating their ideas with no way to know if it was right or wrong. So I moved over to computer science. So we're going to learn so much more about thinking in the next year than we've learned in the last 50 years.

• 26:12Speaker 00

  So Dave, one of the things I find amazing is that we're starting to discover very similar structures in the large language models as we're seeing in human neuroscience and cognitive science. It's almost as if, you know, the brain efficiently got there and we're sort of stumbling our way towards the same endpoints.

• 26:33Speaker 02

  You know, Alex is right, and I've always felt like the force of compression and, you know, in biology, the force of survival, which creates the force of compression, creates intelligence in the box, and consciousness just emerges from that. And a lot of people in cognitive science disagree with that view, but I think it's going to turn out to be true, and we're going to know it very soon. But what's interesting here is that, you know, the innovations that developed the neural network came from biology and the computer scientists copied it. Now it's going the other direction. You know, the big neural networks that we're building are teaching us about things that might exist in the brain, and then you're like, looking in the brain, and you're like, oh, wow, it's over there. So the direction of discovery is going the other way now, which is really cool.

• 27:17Speaker 02

  But the other part of what you said, Peter, which is equally important, is this whole mechanistic interpretability. Can we get the neural networks aligned with human interests by looking inside to the way they think? And I think the answer to that is coming out, yes.

• 27:31Speaker 00

  I mean, this for me, this is the most important thing. Can we develop a new level of trust with AI? Because we truly understand what's going on inside. When they were a completely unknown black box, and God knows, for the last two years, that's the way the world described them as black boxes. We have no idea what's going on inside. You know, we've relaxed that recently with understanding the reasoning and such, but if you can actually understand their hidden thoughts, a level of trust comes out of that, and the potential for true AI alignment. You know, I put out a newsletter on my Substack last week, laying out the arguments for why, and Alex, you and I have had this discussion, why as AIs become more intelligent, they're

• 28:14Speaker 00

  more likely to become more aligned with humanity. And I love that, right? Again, one of our missions here is to sort of quell the fear and give people a different view of what's materializing here.

• 28:28Speaker 01

  Which, by the way, like, a lot of would-be AI alignment philosophers disagree with that, that they have this notion of the orthogonality thesis, that you can have an arbitrarily capable or intelligent AI, and that its goals can be orthogonal or independent of its level of intelligence. I don't subscribe to the orthogonality thesis. I gather, yeah, yeah.

• 28:50Speaker 02

  No, I think this J-Space term is going to stick, too, because... One of the objections with mechanistic interpretability has been, look, the weights in these neural nets are so complicated, you can't really look inside and understand what the neural net is thinking. So, you know, when you're talking to a person, they can be saying something to your face, like in LA, and thinking something completely different in the back of their mind, and that's kind of routine human behavior. But if you look inside the neural net, can it also do that same thing? Can it blow smoke up your ass or not? And I think the answer is no. If you look into the words, if you translate it into words, and that's what that video was showing in the J-Space, is like these words that are on the back of its mind

• 29:34Speaker 02

  are visible to you as a user if you expose them. So then the next question is, are we going to be able to look at them? Or is just, you know, Dario going to look at them?

• 29:41Speaker 00

  Saleem, you're at a consciousness conference, so.

• 29:43Speaker 03

  Yes, I was at a conference. So I think what I found very exciting is this is the beginning of AI neuroscience, right? This allows us to map the inner workings and model the inner workings and look at the structured internal reasoning inside these models. And this really, really breaks the, it's just an autocom complete engine. And I think this breaks that whole argument. Because this now starts to look really like an internal workspace, as Alex mentioned. The danger, though, I think, is I'd be careful about saying it's consciousness. Because, again, we have no definition of consciousness.

• 30:19Speaker 00

  And the paper steers away from that, right? The Synthropic paper specifically says, we're not discussing that we're showing consciousness. We're showing elements that are reminiscent of consciousness.

• 30:32Speaker 03

  Yes. Yeah. And I would push back that we will know what these things are doing. I think we're ways away from that. And let's acknowledge that when we have a human being, we may trust them, but we have no idea how their brain is working and what their compression levels are, what their subconscious things are, because we're not really able to look in. It is cool that we will be able to look into these things, but I'm not sure it'll generate the trust that we want.

• 30:53Speaker 00

  Yeah. I mean, one of the challenges, whenever we talk about consciousness in the AI world, it pattern matches with every dystopian AI movie out there, right? Every nightmare scenario. But, you know, my takeaway here, again, is not fear. It's hope. It's optimism. Of being able to, you know, create the mechanisms for truly understanding what's happening and driving alignment, which I think is is the goal we all want. This is the most important thing that that AI science needs to be doing right now over the next two years is what can we do that supports alignment before we truly hit, you know, AGI and ASI?

• 31:35Speaker 00

  Yes, Alex, we've reached AGI, okay, but before we reach the next level of intelligence.

• 31:41Speaker 03

  I still have my rant that I throw out there on both AGI and ASI. I think the the this, but this did feel very, very big to me. It felt as big as when I read Stephen Wolf from A New Kind of Science, where he shows that automata in repeating patterns can generate all the complexity in nature, and you don't need complexity in nature, you can actually do with very simple models. It's kind of blows your mind when you see that. This, I think, has the same level of holy crap amazingness for me.

• 32:12Speaker 00

  I also think if we're going to start having a new metric to describe models, which is a trust metric, right, where where you describe your ability to understand truly what the model is doing and thinking, and therefore have a higher trust of that model.

• 32:29Speaker 01

  I also think these are going to be the most studied minds in the world. If anything, I think we're far likelier a couple of years from now to study these models because we can subject them to mechanical interpretability studies that we can't subject human meat brains to. So I think if anything, trust is rapidly, just as I think we're on the verge of a transition to not trusting humans to write source code, and because humans write flawed source code, code gen is going to be much trustworthy in the short term. Same idea with these networks. I do think, if I may, with your forbearance, Peter, just 30 seconds on the the math side of this. So again, the J in J space comes from Jacobian.

• 33:10Speaker 01

  The Jacobian, in this case, is referring to a little bit of math, the first derivative of of the probability of each possible.

• 33:21Speaker 01

  Output token from the model with respect to particular parameters inside the model. So hence the Jacobian space or J space. And it's really interesting that there's been a lot of work in the Meck and Terp community in the past devoted to the so-called superposition hypothesis. The idea from neuroscience that if you looked inside a human brain, you'd find a so-called grandmother neuron, a single neuron that activates in response to the concept of a grandmother. And people went looking for a grandmother neuron inside transformers and they couldn't find one. They found instead, and one can tell a whole story on the biological neuroscience side as well,

• 34:02Speaker 01

  found a set of sparse activations, a collection of neurons that collectively represented the notion of a grandmother, and that led to the superposition hypothesis. That maybe individual neurons don't represent semantic concepts one to one, but rather different semantic concepts clustered and superposed onto individual neurons. So in short, what this new J space and Jacobian lens concept brings is not just superposition onto of multiple concepts, sort of sharing like sardines in a can, individual artificial neurons, but actually they're living in the first derivatives as well. The slopes or the changes with respect to particular activations of particular output tokens.

• 34:47Speaker 01

  And I think this is also very suggestive that if you just keep compressing, if we if we keep turning this compression crank to compress more and more general knowledge and general reasoning capabilities into the weights of one of these differentiable models, we're going to see a bunch more phase transitions and things may hide in higher order derivatives. And just follow follow the compression, follow the interior compression weights. And I think this is a very, very promising pathway to the end of the rainbow.

• 35:16Speaker 03

  That may be my favorite, maybe my most favorite Alex line ever. Yes, thank you for that, Matt.

• 35:22Speaker 00

  Follow the compression that leads to the end of the rainbow. Thank you for the mathematical interlude, Alex. That's why we love you. All right, let's jump into our next story here. Sam Altman made global news, not once, but twice. The first item is an op ed he published in the Financial Times regarding AI governance. This was a result of him meeting with G7 leaders in France last week. Sam basically said that in two years, we should all expect AI systems with astonishing power that will reshape the material conditions of human life on a scale never before seen, at least not since electricity, that everyone on the planet deserves access to these technologies and the right to determine for themselves how to best use them.

• 36:06Speaker 00

  Incredibly, Sam went on to insist that democratic institutions must lead and not defer responsibilities to the San Francisco AI labs. He said basically, quote, "Safety standards must be established before there is broad distribution that governance requires democratic process, not decision making by a small number of San Francisco based companies." Sam proposed a framework of a US led international forum that would establish standards, provide expertise and partial analysis of capabilities and risks. That this forum would make the most advanced technologies available to nations and companies that participate and follow the rules.

• 36:48Speaker 00

  He concluded that the forum would serve as a governance mechanism for all AI labs and guard against the commercial pressures that we've seen with unsafe racing. Okay, so like, wow.

• 37:03Speaker 00

  You know, he's taking a first mover here. I really wonder what Dario and Demis and Elon and Zuck think about the op ed. It is worth noting that Dario and Demis were both on stage at Davos proposing a somewhat similar governance. It always seems like Demis and Dario are teaming up on one side of the equation and Sam is on the other. Let's take a listen to Demis and Dario talking about regulations and their proposal for like CERN or an atomic energy commission.

• 37:38Speaker 05

  We probably need new institutions to be built to to help govern some of this. You know, I talked about CERN. I think we need a kind of equivalent of an IAEA atomic agency to monitor sensible projects and those that are more are more risk taking. I think, you know, we need to think about the society needs to think about what kind of governing bodies are needed. Ideally, it would be something like the UN, but given the geopolitical complexities, that doesn't seem very possible.

• 38:05Speaker 06

  And I also agree with Demis that this this idea of, you know, governance structures outside ourselves, I think these kinds of decisions are too big for any one person. We're still struggling with this, you know, as you alluded to, not everyone in the world has has the same has the same perspective. And so, you know, some some countries in a way are adversarial on this technology. But even within all those constraints, I think we somehow have to find a way to build a more robust governance structure that doesn't doesn't put this in the hands of of just.

• 38:39Speaker 00

  So I think these guys are under a lot of pressure, a huge amount of pressure being viewed as potentially saviors or destroyers of worlds. And they need government oversight to help relieve that so they can sleep at night. It's it's interesting.

• 38:58Speaker 01

  It's a lot of pressure putting the heads of two frontier labs on one love seat at Davos.

• 39:03Speaker 00

  Well, they they, you know, there's a there is a love affair between between Demis and Dario and between Google and Anthropic. Just don't put Sam on that same couch.

• 39:14Speaker 03

  Look, there's a there's an elephant in the room here, which is that you're you're we've got the industrial era nation state, and you're asking it to govern post industrial cognition. It just can't be done. And this this breaks the nation state model so fundamentally, all of this, right? Just look at the ruling that only US nationals can can look at the models. I mean, it's just absurd at so many levels, not that they have a better mechanism, but that just doesn't apply. Now, when the people that are kind of raising the hardest are asking for governance. It tells you that it's not really performance anymore. This is a huge thing. The problem is governance needs to become exponential. It means it has to be real time. It has to be adaptive. It has to be data driven.

• 39:59Speaker 03

  And we just can't do it in this way. So I think this is going to, at some level, break the governance model in some very fundamental ways, or we're going to end up with a Baltic system.

• 40:08Speaker 01

  I worry about regulatory capture. So much of this, again, a slightly cynical take might be, smells like regulatory capture, smells like a little bit of pandering to G7 or Davos. Is it really the case that an IEA-type mechanism is needed? Or, and these aren't mutually exclusive. Or, and/or is it possible that you have heads of frontier models, frontier labs, who are facing an onslaught of Chinese open weight models, who want maybe a slightly on margin more protectionist regime to keep the Chinese open weight models out of a defined intelligence or superintelligence block because they maybe fear a bit of competition, want to capture the regulatory state?

• 40:53Speaker 00

  And we'll get to that conversation too a bit later. The interesting thing is that the companies have failed to do this for themselves. Right? They failed to come together. If you remember back to the Silmar conferences in the 80s, I was in the biotech industry there at MIT at the White Institute, and all of the scientists got together. We had just discovered the restriction enzymes that allowed you to properly edit genes and the front cover of like Time magazine with like Hitler babies. It was like, you know, a lot of fear about genetic engineering. And the industry got together and set up their own regulatory structure, which has held extremely well for decades.

• 41:32Speaker 01

  But it's tricky, Peter. I mean, maybe a question for you, Peter, on this. I think it's really tricky for the industry to self, not that it's like organizationally tricky. You could put the four frontier-ish labs on a love seat and say, you all work it out. But the problem is, how do you avoid that giving the appearance of collusion and creating a cartel and competition? Like, how do you do that in a way that isn't blatantly anti-competitive?

• 41:57Speaker 00

  I don't know. The difference, of course, is that in the early days, the biotech industry, we weren't talking about trillion-dollar companies back then. The revenue engines were no longer, you know, the AI race that Sam spoke about, which is very real right now. I mean, people releasing models, pulling their punches and just trying to outdo each other week on week on week. That was not the case in the biotech industry, at least not back then. But I think we're hearing a consensus view from these three individuals, which is going to lead to some structure of government regulation. I guarantee you, with these three CEOs saying, "We need regulation," the regulators will come in and say, "Great,

• 42:39Speaker 00

  let's give you regulation." Now, this, yeah, go on.

• 42:43Speaker 01

  A prediction China is missing from this discussion. China is, if there was an elephant in the room, China's the second elephant in this particular room. And for this to come to fruition, China is going to need to play ball and restrict the proliferation of Chinese models. And you can already see hints coming out of the CCP that China may, contrary to their historic position of blanketing the world, maybe even intelligence dumping onto the world all these open-weight models, if the CCP starts to take a hard-line position that, no, China is going to restrict the export of Chinese open-weight models going forward, then I think a regime like this is possible and the world splits into two superintelligence blocks.

• 43:26Speaker 00

  Yeah, I think that unfortunately is inevitable. I wish it were not, but I can't see it going any other way right now. All right.

• 43:35Speaker 03

  I'm going to say it again. You can't regulate this in any way, shape, or form.

• 43:39Speaker 01

  Oh, you don't think you can regulate intelligence, Sam?

• 43:43Speaker 03

  You have to.

• 43:44Speaker 01

  Why not?

• 43:45Speaker 03

  You can't. You'd have to regulate every line of code written. People can download, take models offline, merge models, do a lot of stuff offline that they, that doesn't then use the existing online models. I don't see how you can police this.

• 43:59Speaker 01

  Oh, there's totally, I mean, just a minute on this. So, Werner Vingi wrote extensively about this. We have Sort of a cognitive surplus of transistors. In my mind, there are so many different social engineering techniques that humans have discovered over the centuries for policing it. Like, we could have models policing each other. We could have, at the transistor level, we could be using the surplus of transistors to do KYC all the way down to the circuit level if we have to. I think we have so many different architectures.

• 44:29Speaker 03

  Let me rephrase. The current regulatory structures cannot in any way, shape, or form regulate what's coming. You need, what you're talking about, an AI-based, almost down to the hardware level-based, but that would cut across everything. It can't operate in the geopolitical environment that we have today.

• 44:48Speaker 02

  Well, I think it's really clear that the prompts are all going to get inspected, and also the internal J-spaces now will be inspected, that the labs will do the inspection on behalf of the US government, and that, as Alex said, high probability China will stop exporting open source sometime in the next year or two for the same exact reasons. And then you'll have, you know, like a, you know, a long-term arms race between the east and west versions of AI superintelligence.

• 45:17Speaker 00

  So Sam said specifically the framework is for a US-led international forum, which, of course, is devoid of the word China in there. I am curious, you know, what scenarios do we have? I was speaking to Alvin Graylin, who's a friend of ours, about, you know, US-China relationships. And the question is, is there a structure in which, you know, we can see a US-China alignment on AI? Anybody?

• 45:50Speaker 02

  What you'd be looking for, if that were to happen, you'd be looking for cross-inspections of the prompts. Like, are we allowing each other? And the problem that the US will have with that is China stealing intellectual property? So I think it's unlikely, but it is possible. There's not enough trust. That's how you would know that there are no bad actors is just looking at each other's underlying prompts and weights and J spaces.

• 46:13Speaker 00

  Ultimately, you know, we use China as a stalking horse to accelerate investments and accelerate, you know, reduce regulations and such. But I think for the safety of the planet, not having a AI arms race between the two nations is an outcome I'd love to see happen.

• 46:29Speaker 01

  I also don't think the IAEA style mechanism necessarily works for AI just at the technical level. Forget about the political or geodynamic level, just at the technical level. The notion of, say, different blocks inspecting each other's fission inputs, if you will, that's conceivable if, to the extent, open paren, question mark, question mark, question mark, close paren, that just like looking at uranium ore, say, shipments is a productive or a fulsome way of tracking different nations' nuclear weapons capabilities. I'm not sure that generalizes to intelligence. There are simply, to Salim's earlier point,

• 47:13Speaker 01

  there are so many different ways to hide or to mask superintelligence and underlying capabilities, so many different forms it could take. Greg Baer has written a fair amount over the years about sort of prohibition era style bathtub superintelligences. If we had to, if Russia or China entered into some sort of internationalist regime where the US were inspecting all of their supercomputers and all of their prompts and all of their algorithms, there are simply too many places that one can hide superintelligence that I'm not sure that an IAEA's type mechanism with such a simple minded, oh, let's look

• 47:54Speaker 01

  at their uranium equivalent shipments, or let's look for their centrifuges, would actually be fulsome enough to cover all of the world.

• 48:02Speaker 07

  This episode is brought to you by Blitzy, autonomous software development with infinite code context. Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements. The Blitzy platform provides a plan, then generates and precompiles code for each task. Blitzy delivers 80% or more of the development work autonomously, while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5X engineering velocity increase

• 48:46Speaker 07

  when incorporating Blitzy as their pre IDE development tool, pairing it with their coding copilot of choice to bring an AI-native SDLC into their org. Ready to 5X your engineering velocity? Visit blitzy.com to schedule a demo and start building with Blitzy today.

• 49:07Speaker 00

  All right, let's move on to our second Sam Altman story. I think of it as the beginning of the starting gun for the grand equity negotiations taking place for universal basic equity. So again, in the Financial Times, it was reported this week that Sam has been talking to Trump, Lutnik, Besen, and Bernie Sanders about a 5% equity stake in OpenAI. OpenAI's last reported valuation, 852 billion back in March. That 5% stake would be worth about 42.6 billion. Given 315 million American citizens, that's only 135 bucks per person, not very much. They talk about a proposed Alaska Permanent Fund. That permanent fund is 91 billion

• 49:52Speaker 00

  and it dividends about $1,000 to $3,000 per citizen of Alaska per year. Altman's broader idea, and again, this is him out there speaking on his own and putting forward a plan for the entire AI industry. He says he'd like to see Anthropic, Google, Meta also contribute equity to a public fund. We should remember, we've talked about this before, the US government already owns 10% of Intel. And so when Sam talks about a 5% donation, if you would, to the government, I think Trump is an amazing negotiator. I'm going to guess we're going to end up at 10%.

• 50:32Speaker 00

  You know, I did a poll on X asking how much, and the majority of the people were either at 20% or zero. Interesting.

• 50:45Speaker 02

  It's so irrelevant. My read on this is that, you know, a year ago, Sam was called the most powerful man on earth in multiple interviews.

• 50:53Speaker 01

  Yeah.

• 50:54Speaker 02

  Now, you've got Dario and Demis clearly working on the future governance of the entire world, and Dario has a big deal with Elon, you know, renting all of the chips. So now all the guys are talking to each other, and they're not including Sam. So Sam's now writing an op-ed, which is like, you know, trivially short op-ed, by the way, and he's proactively offering 5% of his company, but he's just trying to get back in the hunt of relevance in the eyes of the White House. I did hear that Dario got kicked out of the White House for being too weird. Did you hear that story?

• 51:26Speaker 01

  Yeah, that was published. Yeah, that was all over the news.

• 51:30Speaker 00

  Recently?

• 51:31Speaker 02

  Yeah, yeah, it was like, what, two weeks ago, a week ago?

• 51:34Speaker 01

  The story was that Anthropic sent in Dario initially to negotiate, and that didn't quite work out, so they sent in Tom Brown instead.

• 51:41Speaker 02

  This is for Fable 5, yeah. That's so easy to visualize, isn't it? Like, Trump is like, dude, you're weird, man. I don't even know what you're talking about. What's this J-Space crap? Get out of the White House.

• 51:53Speaker 00

  So, hey, where do you figure this goes? Where do you figure the idea of contributions to the government from the AI labs?

• 52:01Speaker 02

  It's so irrelevant. The government can take any chunk they want, any time they want. They already take it in income tax anyways. It's so irrelevant.

• 52:09Speaker 01

  I'll take a different position. I think this is super relevant. I think that this, one can see the outlines of a baby universal basic equity, a grand bargain, if you will. The economics don't work for supporting universal basic equity right now off of, say, a 5% chunk. But if OpenAI and Anthropic and SpaceX AI all do this, and they grow Elon-style, a couple orders of magnitude in terms of size and grow the economy, that's your UBE for you. So I coined a term for this a few days ago. I call it a hypertith, which I define as a fixed equity contribution paid by companies building the singularity stack, paid into a sovereign wealth fund or similar public vehicle, turns private exponential, Salim, upside into universal

• 52:54Speaker 01

  Basic equity, broader national ownership and a more relaxed regulatory bargain.

• 52:58Speaker 02

  I can tell you exactly why that makes no sense whatsoever. I love your neologism. Back in the New Deal era, the government decided, you know what? We're going to take a huge chunk of everybody's paycheck and we're going to call it social security. And then we're going to invest it on your behalf for your entire life. And then when we're old, we're going to give you a lot more money back. They decided very quickly that they had no idea how to invest your money. And so they said, screw that. We're not going to do that. We're just going to take the money and spend it instead. Because, because we don't have any idea how to invest your money on your behalf. And go ahead. And so that all collapsed and moved over to 401k plans. Where Fidelity or UBS invest your money because they know how to do it. So the idea that the government is going to set up some intelligent sovereign wealth equity thing is absolutely insane.

• 53:38Speaker 02

  The next president will immediately sell it all, turn it into cash and then use it to buy votes in the next election.

• 53:44Speaker 01

  This is so interesting, Dave. Yeah, I want to discuss this.

• 53:49Speaker 02

  If you want.

• 53:49Speaker 03

  So let me, let me just throw up my view, but I do want to get back to Alex where you think, because I'm, I'm like, I'd love to hear the discussion back and forth. I go full cynic on this. This is purely Sam, A, trying to get in the game and B, trying to protect. Because the minute you have the government has 5%, you're too big to fail in a sense. And they protect themselves by doing that. So that's my full cynic view on this.

• 54:12Speaker 00

  Can I just inject one thing and then hand to you, Alex? Which is, I think one of the important elements here that people are not realizing is AI. As we value AI today in terms of sales of tokens is a miniscule amount of the future value of these labs. I think that, you know, as they start discovering fundamental breakthroughs in biology and physics and chemistry. Those are trillion dollar pops. And I think the idea of if there was a structure where the US populace, the US citizenry had ownership in these companies that it could drive an economic engine for, you know, UBE, UBS, UB, UB whatever.

• 54:58Speaker 00

  But again, my mission here is how do you reduce the fear? That people are having because, you know, the numbers are staggering. Only 10% of Americans think that AI is going to deliver positive benefits to humanity. You know, 30 to 35% feel relatively good about it. But only 10% have this view that it's going to make the world a better place. What it means is we're not doing our jobs. I know. Blasting out the optimism. We need to get better at this. All right, Alex, please take us home here.

• 55:31Speaker 01

  The distinction, to respond to Dave's point about social security. So, social security in the US was created in a time and a place when index funds didn't exist. It was created in the wake of the Great Depression. There was a general distrust of the stock market in general. There have been multiple attempts over the years to quote unquote privatize social security, which would take the form of converting sort of a cash-based pyramid scheme into something more equity-oriented. That's failed for a variety of political and social reasons. But I do think this time is different. If, if social security were created today and not, you know, almost a century ago,

• 56:14Speaker 01

  I think it probably would be based on some sort of sovereign wealth fund that holds, hopefully, like a broad market index fund that's low cost and not just be based on a pyramid-style cash-in, cash-out, bond, or interest-bearing security-type scheme. And that's where I think a hyper tithe has the potential to become a baby and hopefully, aspirationally, a grown-up UBE. If these frontier labs, if, if, if there were a hyper tithe from all of the Magna Mobsta companies to blend Peter Neologisms, and these were all paid hypothetically into a sovereign wealth fund, and the Magna Mobsta companies just ultimately, over the next

• 56:59Speaker 01

  five to 10 years, grow so much and grow the economy so much, I do think that could in principle support a universal basic equity-type system.

• 57:07Speaker 00

  I, I agree with you, Alex. And, you know, there's a lot of conversation right now about the Trump accounts and Trump accounts for adults as well. And I'm, I, you know.

• 57:20Speaker 00

  That's his nature. His nature is to negotiate and take pieces of things. And I think he wants to populate the Trump accounts for adults with 10% of all of the hyperscalers and AI labs. That's my guess. Now, whether he can pull that off and then put the protections in place, Dave, so they can't be sold, so that it's dividends from those, you know, so the $91 billion of

• 57:45Speaker 02

  These are not dividend companies, though. There are no dividends. Like, okay, so everybody in America, you get a Trump account, we put the Magna Mobstax stocks in it, here you go. But you're not allowed to sell it, or you are allowed to sell it, or these are not, there's no income from it. Like, are we gonna call them Trump accounts 50 years from now, realistically?

• 58:02Speaker 01

  530A accounts, if you like. But if I were head of Commerce or head of Treasury, the sort of scheme, policy-wise, that I might be contemplating is, okay, you start with a sovereign wealth fund, or whatever, could be individual 530A accounts, it's populated with the Magna Mobstax stocks, or some subset thereof. You wait a couple of years, and then the market is sufficiently liquid that you could liquidate them in favor of, since you're the government, you don't have to tax yourself. So you could do a taxless exchange for a broad index fund, even though it's populated initially with Magna Mobstax contributions via this hypertith grant to the government. You exchange them for a broad market index fund. That's the solution.

• 58:46Speaker 02

  Well, I don't think it's a bad idea. I just think it's irrelevant. The government has the power of taxation. They can extract from the income anytime they want.

• 58:56Speaker 00

  We're gonna find out.

• 58:57Speaker 01

  Quickly on this one, the corporate income tax is cash-based, and the problem in a hyperscaling, singularity-oriented economy is cash may not be the best basis for taxing the economy, but equity does scale.

• 59:11Speaker 02

  If you sell it.

• 59:13Speaker 01

  Or if you can tax equity. Right now, we don't have basically an equity wealth tax. This is a de facto shadow equity wealth tax with companies perhaps feeling a bit of regulatory pressure to give up equity in themselves. It is definitely a tax, but it's a slightly different type of tax.

• 59:29Speaker 03

  I'm all in favor of the UBE. I just don't see the mechanisms for it, but I do agree with the principle.

• 59:35Speaker 00

  All right. Well, let's jump into our next subject. You know, one of the reasons we're always concerned about UBI, UBE, all of that is the concerns around job loss. So our next story is about jobs and the continuing debate about whether AI is going to be creating or destroying jobs now and in the near future. So we've covered both sides of this story. It's been murky. We've given evidence for both sides. A new paper released this past week by Ramp and Reveleio Labs gave some pretty definitive data here. They looked at 21,559 US companies over the past five years between January 2021 and February 2026,

• 1:00:17Speaker 00

  matching the actual AI spend of those companies and their workforce records, meaning hires and fires. So here's the headlines. Companies that spent heavily on AI did not shrink. In fact, they grew. So the high intensity AI adopters that they studied were spending $33 per employee per month on AI. They grew 10.2% in the white collar and 12% at entry level growth. In contrast, the low intensity adopters spent $3 per employee per month, basically a 10th, and showed no significant employment change. The author has warned this is correlation, not causation, but it puts forward a very different theory. Rather than

• 1:01:01Speaker 00

  the AI is going to replace workers, it suggests that AI may expand ambition first. Companies that actually integrate AI deeply may take on more projects, serve more customers, build faster, hire more humans, especially entry level, to capture the upside. So I love this story. I mean, for me, this is an abundance optimism story for people because there's a lot of fear out there. My concern about this story is that regardless of what the data says, the news media is out there and the underlying belief is that AI is going to destroy our jobs and it will.

• 1:01:39Speaker 00

  Displace a number of things, right, with robo taxis and AI call center workers and so forth, but the evidence looks like, and I don't know about you guys, but I'm hiring more people in my companies than ever before. I don't know if that's true for you, Dave and Alex.

• 1:01:58Speaker 02

  Well, God, if anyone's AI native, their demand for that person is through the roof.

• 1:02:04Speaker 00

  Yeah.

• 1:02:04Speaker 02

  So, yeah, it's rampant. And I'm starting to feel like this is a permanent thing, not a transitional thing, because, you know, one of the things to worry about is, look, implementing AI is such a payback that there's this land grab of talent. There are anyone who can implement it, any bank, any insurance company, any operating company, anyone who can get AI to work in this shop, we will hire them for whatever they cost. Is that transitional? Because once they've implemented AI, they've coded themselves out, or is it permanent? I feel more and more like it's permanent. Like, as the AI improves, the things you can do also grow, and that person's value goes up over time. And so, the data, I think, is very early inklings of what's inevitable, where AI native organizations

• 1:02:48Speaker 02

  are going to just grow like wild, and they're going to add headcount as they do it. And anyone who's sitting still hasn't fired everybody yet, but eventually, they're going to be wiped off the face of the earth. And so, what you see right now is net growth.

• 1:03:01Speaker 03

  Yeah. This is what we call the organizational singularity, right? If you're an AI native, AI centric organization, if you're doing deep redesign of your workflows to be AI native, then you have an explosive opportunity in front of you. Shallow adoption fails, because this is not automation versus jobs. It's shallow adoption versus deep redesign. So, we started our pilot, by the way, of working with companies. I'll report back as to how things are going. But we're unbelievably excited at, look, the opportunities. We're like, we can't even count the number of workflows that we could help automate with these companies. For each company, we're picking one workflow that might radically increase revenue, and one workflow that might radically shrink cost.

• 1:03:42Speaker 02

  Totally.

• 1:03:43Speaker 03

  And we've got so many candidates for both sides. It's like crazy.

• 1:03:47Speaker 02

  That's literally why you're in every city in the world every time I do a podcast. I mean, the demand for what you're what you're teaching is so step function through the roof.

• 1:03:57Speaker 03

  It's the biggest shift in organizations in a hundred years.

• 1:04:01Speaker 02

  Yeah, probably in human history, I'll bet, all time.

• 1:04:06Speaker 03

  It applies not just to companies, but it applies to nonprofits and impact projects and government departments. Everything. So it's gonna be huge.

• 1:04:13Speaker 02

  And I love using token spend as a proxy for adoption, even though it's not perfect. It's reasonably good. So this study actually focused on token spend. Reasonably good way to say, are you doing it for real or not?

• 1:04:24Speaker 03

  And just a quick plug, we have released the book as an AI. It's available for free. You can download a cloud skill and run your business in this new model. It'll tell you what to do. It's free. Go register at OpenEXO and download it. The reports are going back are crazy.

• 1:04:40Speaker 00

  To remind folks, Saleem is going to be doing a session at the Moonshot Gathering on September 24th on the organizational singularity. And AWG is going to be there doing an extended AMA on Solve Everything. Bring your most difficult, challenging questions to Alex. Dave will be there. Stump the Trump, yes. Dave will be there talking about AI investing. We'll have Palmer Lucky, we have Rod Roddenberry, we have Ben Lamb, Cathy Wood. It's going to be an amazing. So go to moonshots.com for the Moonshot Gathering, September 25th. Top creators and builders there.

• 1:05:21Speaker 00

  You know, interestingly enough, we're still seeing a number of companies out there. You know, Oracle blamed 21,000 layoffs on AI. Meta blamed 8,000 layoffs, Block 4,000, Cisco 4,000, Atlassian 1,600. And so the question is, are these CEOs just using AI as an excuse for, you know, reorganization or is it true?

• 1:05:47Speaker 03

  There's two things going on. One is like, for example, is well-known, the block overhired radically and needs to shrink. So that's an easy hobby horse for shrinkage. The other is the note that the company is laying rough for all SaaS companies and the SaaS business model is fundamentally broken in an age of AI.

• 1:06:05Speaker 00

  Is that the job being disrupted at the company?

• 1:06:08Speaker 03

  Yeah.

• 1:06:08Speaker 01

  I think some of it is real, some of it is AI washing. The real component in many cases, as with Oracle, for example, is it's the capital, the cap expenditures that are crowding out the OpEx of human labor. It's quite literally all the isms from the first part of the 20th century worrying about capital v labor we're seeing play out internally in hyperscalers like Microsoft or Oracle that are having to direct free cash flows to internal CapEx to building out their hyperscale AI cloud infra capabilities at the cost of American, usually, Ireland in some cases, based developers that can now be automated with software that sits on top of the AI infra.

• 1:06:52Speaker 02

  Well, you know, Peter, remember when we were at Facebook before it became Meta, around the time of the Oculus and we were having that tour? And you look at Facebook online, you look at Instagram online and you look at, you know, 10,000 employees. You're like, what the hell do you guys do? I mean, it hasn't changed. Like, what are you literally doing? So you walk around and talk to people and tons of like UX experimentation. And remember in the bathrooms above the urinals, there's the tip of the day, you know, the little coding. You know, and you're like, oh, okay, that's what you guys are all doing. You're like, so that's like the easiest AI job in the world. So I think that's very real. Like you just don't need those gooey, low-level coding jobs anymore.

• 1:07:33Speaker 02

  And a lot of it is server configuration, you know, propping up a new Instagram server for a new country. Yeah, that's so easy to do with AI now. So I think, I think that part's all real.

• 1:07:42Speaker 00

  Well, we're going to continue to follow this story on jobs. I think it's important, you know, if you're a, if you're a student out there worrying about can you get a job, worrying about, you know, everything you're hearing out there, please dive into the world of AI, of entrepreneurship. If you're a parent, you know, have this conversation with your, with your kids. It's really important. I really, my goal is to dismiss fear, right? It's, there's real fear, but there's, at least be fearful for the right reasons.

• 1:08:14Speaker 03

  Yeah, I mean, just to put it out, but David Sacks talks about it all the time on the All In Podcast that we're increasing jobs radically, that we're increasing hiring. All the data shows that. Follow the data. That's it. Just be evidentiary.

• 1:08:28Speaker 00

  And I know we've talked about it in the past on this podcast, being concerned about, you know, a lack of new entry jobs. And there probably are in certain industries. But if you're AI native, as Dave said, I think you've got massive opportunities. All right, I'm going to move us forward here. Our next two stories are classic. Alex Karp, CEO of Palantir. The first one is a product launch. The second one is a declaration of war. In our first story, Palantir and Nvidia have announced a sovereign AI architecture that puts Nvidia's Nematron open models inside of Palantir's platform, composed of their artificial intelligence platform, ontology, foundry, and Apollo stack,

• 1:09:10Speaker 00

  designed for US government agencies and critical infrastructure operators. So we've touched on Nematron a little bit in the past. It's Nvidia's open model. They've got three models, Nano, Super, and Ultra. They range from 30 billion to about 550 billion parameters. Nematron's edge is speed and cost. It can be roughly twice as fast and 60 times cheaper than GPT-5.5 or Albus 4.8, but it's not yet smarter than those two models. So I'd like to take a listen to Alex's video conversation or part of it on CNBC, and we'll talk about it from there. Let's take a listen here.

• 1:09:52Speaker 08

  We're sitting on critical infrastructure across America, Ukraine, Israel. Everyone who uses LLMs on the battlefield runs on top of our ontology. Clients are just, to say they're unhappy. A level of discomfort and loss of trust. When you're using large language models, they are, it's like a, at this point, everyone technical realizes they're like a critical resource. To make them valuable in an enterprise like Battlefield context or regulated context or manufacturing, you have to have what's called an application layer. But de facto, it takes a large language model. It makes it safe and useful and precise. What aligns me with Nvidia and I think is what the technical customers want,

• 1:10:36Speaker 08

  which is control over their compute. Their models, their data stack, and their alpha. They want to know they own the means of production. It's not being transferred to someone else. They're not interested in some fake deploy co. That somehow is deploying tokens that transfers the alpha to a third party. And the jig is up. And so we have to figure out a way to rebuild trust. And that trust is going to happen where everyone gets to ask, ask and answer basic questions. Who owns the data? Where is it cached? Are the prompts secure? Is this being transferred to you? Are you being, okay, if it was so valuable, let's say I can make you a billion dollars. Right. Tomorrow, wouldn't I say, I'll make you a billion dollars and I want 30%? Why are they charging for tokens if it's so valuable?

• 1:11:23Speaker 02

  I think you went off script in the end there. That, that last one made no sense whatsoever.

• 1:11:27Speaker 01

  Well, careful, careful what you wish for, because that, that last bit is actually happening.

• 1:11:32Speaker 00

  Yeah. Um, you know, Alex, his, his point here is, and, and he's got a second video. Actually, let's go ahead and play the second video and then we'll talk about it in general. Cause I think it's the second part of the conversation here.

• 1:11:47Speaker 08

  In this country, at every single enterprise I deal with, they, these people are livid. They're like, I am paying for tokens that create no value. Say I can make you a billion dollars. Right. Tomorrow, wouldn't I say, I'll make you a billion dollars and I want 30%? Why are they charging for tokens if it's so valuable? These people are stealing the weights and alpha of my business and they're creating a wealth tax. It does not help the poor. It just punishes, starts with the billionaires. Every single person at this table is going to be paying a wealth tax only to punish us. And the reason for it is because these models have been completely over irresponsibly oversell. And the sell is it's dangerous for everyone, which is why I can give it to all your adversaries.

• 1:12:29Speaker 08

  But I can't give it to the Department of War, or I can't safely give it to an enterprise in this country without being certain that the alpha of that business could transfer to this model tomorrow. I E. I have no business, no job. Is the voice of American business that is being channeled through me? And I'm telling you, it is, it is absolutely a problem for this country because the clients have to be able to ask and answer very basic questions. Are you keeping the data? Are you going to enter our business? Do they get to control the weights to do it? Or do you get to control the weights? Are we really going to outsource the battlefield of this country to the consensus view in Silicon Valley? That is effing insane.

• 1:13:08Speaker 00

  Obviously, he went on a rant. The key points he's making here is a great concern that when you're using Anthropic or using OpenAI, that you're effectively giving them your alpha, you're giving them access to all your data. And what's needed right now is a open models that you can build on your own hardware, on prem hardware. So, you know, open weight models, on prem hardware, and then customizing your own language, your own large language models and not giving your secure data, your alpha, as he calls it, your means of production to these large AI frontier labs.

• 1:13:51Speaker 02

  And your weights, they're taking, they're taking your weights. Did you know you had any weights? Well, okay, if you have any, you're giving them to them. Okay. It's like, actually, it's everything that would make you hate Dario bundled together in one long glued together like, and they have a wealth tax. Can you believe, like, okay, let's put them all together to make every corporate CEO as scared and as angry as possible at Dario so that they buy the new open source Palantir Nvidia, you can run on prem model that keeps all your alpha and your weights safe from, you know, Dario because he's going to steal all your intellectual property. Very valid point, actually.

• 1:14:36Speaker 02

  The rant format is extra dramatic, but, but it's a very, very valid point. And it, it's really interesting to think like, okay, he serves the Defense Department, among others, but he's taking the open source pathway to get in there, but you know that's not going to last, right? You, you're never going to have open source Defense Department weights. That's not going to happen.

• 1:14:56Speaker 00

  Well, no, I mean, so he's building, he's building an air gapped machine on top of Nematron, which then the Defense Department owns that model and owns the equipment it's running on. I can imagine very much that works for them.

• 1:15:12Speaker 02

  For sure. And, and also, you know, his other big customers are banks, mega banks, insurance companies. They'll also, in his world, have their own proprietary models, but you can't have every startup have its own proprietary model because then you'll have every terrorist have its own proprietary model.

• 1:15:28Speaker 00

  But why not? I mean, I'm running a couple of Mac Studios, you know, with Kimi K 2.5 on top of Opus 4.8. Or below OPUS 4.8 and an open claw there. I haven't migrated yet, but why can't that be a standard future?

• 1:15:46Speaker 02

  Well, I think we'll look back and say this was a very cool, very fun, quaint kind of hobby era. But when it's super intelligent and capable of creating any virus, any chemical, any weapon, you can't have it available to each individual. Right now, nobody can afford the compute to do those kinds of very evil things. So it's not a problem. But if we keep quantizing and compressing at our current rate, you know, I think this is about 100 to 1,000 X performance increase a year. If that happens again next year, then your Mac mini sized box is capable of viruses, nuclear weapons, anything. So we just can't have that outcome.

• 1:16:29Speaker 00

  It's not an if, it's a when. It's gonna be a when. We gotta, yeah.

• 1:16:33Speaker 01

  I would distinguish between permissioned versus permissionless on one axis and locally hostable versus remote API only on the other. But maybe just taking a step back, this is obviously the rant heard around the world and leave it to Alex Karp to articulate a bunch of different things that probably need to be unpacked. So maybe just to do a little bit of close reading of some of the things that he said and how I translate them. So you'll note Palantir back in the Stone Ages was a Claude wrapper, like the Stone Ages as a few months ago, was a key distribution channel for Claude into the Department of War, into a variety of their customers. That's clearly over.

• 1:17:18Speaker 01

  That's point one. Point two, I think so. Point two, the Deploy Co reference. So when Alex, other Alex, says, drops an offhanded reference to Deploy Co's, I hear that as a frontal assault on OpenAI and Anthropic and other companies, including Microsoft now, launching forward deployed engineer organizations that represent a head on assault on Palantir. So he's definitely talking his own book. Palantir basically defined the modern forward deployed engineering model, and now all of the frontier AI labs are just launching direct competitors to Palantir.

• 1:17:57Speaker 00

  Like, let's go in there and steer there.

• 1:17:59Speaker 01

  Yeah, so why not counterattack via commoditizing one's complement with these open weight solutions from Nvidia? Second point, other countries, Palantir sells quite a bit of its own stack, not just into the US Department of War, not just into US financial institutions, but into other countries as well. And there is a dawning awareness by other countries, doubly so after the whole Fable, Mythos fiasco, that they're not going to get access to Frontier US capabilities from the Frontier Labs anymore. So they had better, and I think they're now pretty well incentivized, transition to locally hostable models that they can control that can't just be gate-kept by US export controls on a moment's notice.

• 1:18:42Speaker 01

  So being good salesman, good businessman, Alex, I think, recognizes that all of his international customers need a localizable solution for inference time. The question that no one's asking, including Alex in his rant heard around the world, is what about sovereign training time? No one's asking that. Right now, Nvidia is training its own open weight models. It's not distributing those locally. But at some point, I suspect, as this question, which to my ear rhymes with Microsoft in the late 90s, when Microsoft was at the peak of its power and the open source movement had to come from, even though there was Free Software Foundation, Richard Stallman, GNU-FSS,

• 1:19:25Speaker 01

  et cetera, et cetera, within the US, really, the nucleating event came from outside the US in the form of Linux and Linus Torvalds from Finland, that then the whole GNU stack nucleated around. Similarly, we're seeing the strongest open weight models come from China. I think we're at a similar point now where we have a whole international community that's just realized, thanks to Fable and Mythos, that it can be cut off at a moment's notice and it needs an open weight stack. And I think Alex Karp is trying to channel all of that animus.

• 1:19:56Speaker 00

  And so I want to hit this point first, which is if, in fact, the dominant players are OpenAI and Anthropic, are if you're at risk of losing your proprietary data to them without even knowing it, then Versus being able to operate on an open weight model on your own hardware. That can't be shut off. That can't be shut off. It is a future that we need to consider is very real. And so the question is, where are the open weight large language models here in the U.S.? We've got Nematron coming online. We've got Google. What happened to Meta?

• 1:20:41Speaker 00

  I mean, Meta was supposed to be the open weight player in this field. I mean, I'm assuming that Zuckerberg is working on that in background mode and will come out as, you know, that's his, that's where I would be playing if I'm him. I'm going to call it the dominant U.S. player in open weight models. But we'll see.

• 1:20:59Speaker 01

  It fell behind. I mean, most of, I know many people who were involved with Lama four, who are no longer with Meta. Put it that way. And Lama five, whatever it's ultimately branded, whether it gets branded as Spark or something similar, may or may not have GPT five or Fable five level capabilities. I don't know, TBD. But I suspect just based on public reporting, Meta, which was in the race, hopefully Google stays in the race. XAI may or may not vis a vis Grok Cursor stay in the race. There is totally, I think, a gap for frontier open weight models coming from Western institutions, including from Nvidia, which has every incentive to produce frontier level capabilities. It's just expensive and hard at the moment.

• 1:21:44Speaker 00

  And we're also getting full stack, right? So Nvidia coming in as a full stack player. You know, basically providing the chips and the models. Maybe, you know, into partnerships, applications.

• 1:21:57Speaker 01

  Well, Nvidia will be happy to commoditize everything at the software layer if it means selling more GPUs.

• 1:22:03Speaker 00

  Yeah.

• 1:22:03Speaker 02

  Keep in mind that, you know, every single Magnum obsta company is designing its own chips, except for Anthropic now. And so Nvidia's, you know, stranglehold on 80% gross margins is not forever. And so if Nvidia can create an open source model and it gets distributed through Palantir and a few other people, that puts competitive pressure back on Anthropic. Because the way things are trending right now. Every dollar in AI is flowing through Anthropic at massively increasing margins.

• 1:22:34Speaker 03

  Wait, I've got a couple of things I want to say about this. Yeah, sure. Okay, so CARP's core argument is that enterprises should freak out that they're that paying for tokens may also mean they're they're releasing and leaking their operational knowledge, right? Right. The like. Their means of production, right? Yeah, your data exhaust is now the new oil and maybe it's even the national security program. So he's freaking everybody out on that for reasonably selfish reasons, et cetera. If you rent intelligence and give away your if you rent intelligence, you'd lose your context, right? You may be funding your own replacement. That's the freak out. I think the bigger question, if you go one level deeper, is who owns the learning loop? Is it the model provider? Is it the enterprise? Is it the state?

• 1:23:19Speaker 03

  Or is it the customer? Right? And this is the key thing. Enterprises are going to need to own their learning loop and whatever it takes to own that. I think we're going to end up with on-prem models, as you've mentioned, Peter, running on with personal data and custom data. And that's where the learning loop will go the biggest.

• 1:23:40Speaker 02

  Well, on-prem, everything will be in space. So on-prem is an interesting word.

• 1:23:44Speaker 03

  Well, private clouds, call it.

• 1:23:45Speaker 02

  Yeah.

• 1:23:46Speaker 01

  Well, the organizational singularity has to migrate to orbit, obviously.

• 1:23:50Speaker 03

  It will have to migrate to orbit.

• 1:23:52Speaker 02

  I agree. It's a it's a race right now between everything going to Anthropic OpenAI or what we're calling on-prem, which is in space, but private clouds, but inspected some other way. Right now Anthropic has agreed to inspect everything for the government.

• 1:24:09Speaker 02

  And so if you go private cloud, then some other inspection mechanism has to come into existence, which Palantir will probably contribute to.

• 1:24:16Speaker 00

  All right, welcome to the health section of Moonshots, brought to you by Fountain Life. You know, AI is impacting every aspect of our lives, how we teach our kids, how we do our business. But one of the most important things that AI can deliver to us is health. And one of the things I think about when, you know, shooting for 100, 120 is, am I going to have the cognitive health to be able to think clearly and keep my wits about me for the next 50 years? I'm joined here today by Dr. Don Musalem, the chief medical officer of Fountain Life and a member of my Fountain Life medical team. Don, a pleasure. So, Don, talk to me about brain health.

• 1:24:49Speaker 09

  Brain health, you know, you're right. This is the number one concern people coming into Fountain Life have is will I remember the name of my child and the face of my loved one? Forty-five percent of dementia cases are entirely preventable with lifestyle. And what was really intriguing to me, Peter, is that a quarter of our members had advanced brain age. But over 13 months of us really helping them live healthier lifestyles, eating healthier, moving their body regularly and optimizing sleep. People overlook that so often, but that sleep optimization is critical for our brain health. What we showed is that we were able to improve the brain age in 46% of those individuals. That's a powerful number.

• 1:25:30Speaker 00

  That's amazing. You know, one of the things I love about Fountain is we're constantly searching the world for the most advanced therapeutics and bringing them to our members. So for me and all of you, I hope that you appreciate the fact that you can become the CEO of your own health. You can make sure that you've got the cognitive clarity for the next 50 years. Come and check it out. Fountainlife.com/peter to learn more and become the CEO of your health. Now, back to the episode. So Dave, let's jump into the story that we were talking about back and forth. AI is now designing better AI chips and training data is the catch-22. So our final story predicts a massive acceleration of the innermost loop, i.e. AWG's catchphrase.

• 1:26:12Speaker 01

  Shocked, shocked to see recursive self-improvement in this era of recursive self-improvement.

• 1:26:16Speaker 00

  Yes. Amazing. Of AI designing chips that power AI. So here's the background. Designing radio frequency circuits, RF guts are, you know, part of every wireless device, and they've often been called a dark art. In other words, it takes humans weeks of painstaking trials to design these RF circuits and these chips. Last week, researchers at Princeton, working with IIT Madras, decided to hand that job to a machine. And here's the clever part. It's not one AI, but two working together. First, they trained a convolutional neural net, the same kind of model built for image recognition, to predict the physics.

• 1:26:57Speaker 00

  Feed any shape and it tells you the EM field, how the EM fields will behave without ever taking the slow route of solving Maxwell's equations. What used to take traditional solvers minutes to hours now takes milliseconds. Then they send an AI loop over that 1,000 times.

• 1:27:19Speaker 00

  Tens of thousands of times inventing wild, non-intuitive circuits, shapes that no human would ever create. The result are designs that took weeks now being finished in minutes. But here's the catch and the tease. The AIs require training data and all that training data is locked up in, yes, you got it, the Magnum Opsta companies out there. So, the question is, if this training data can be unlocked, can we see an intelligence explosion in the design of AI chips, which is the inner innermost loop? So Dave, what's your thoughts on this one?

• 1:27:54Speaker 02

  So many thoughts, but you know, just to clarify one part of that, the convolutional neural net is effectively acting like a simulator. And any place you can build a simulator, the AI can have a field day because it can check its own work and it can, it can work for weeks or months, improving itself if the simulator is accurate.

• 1:28:12Speaker 00

  Dave, unintentional pun, I assume, a field day?

• 1:28:16Speaker 02

  Oh.

• 1:28:17Speaker 00

  Inevitable. Sorry, sorry.

• 1:28:19Speaker 02

  Absolutely unintentional, extremely. So, so the, the chip area, you know, is, is going to be massively impactful for the recursive self-improvement of AI. And it's an open question right now whether that data is truly locked inside Nvidia and a couple of other companies, or whether the simulators are good enough to allow you to just generate a circuit, see if it would have worked, to generate the next, see if it would have worked. So, those are in a footrace right now, but regardless, it's incredible to me that the Magnum, you got 11 companies in Magnum Opsta that are completely dominant in the global market cap, every single one of them designing its own AI chips, except for Anthropic. Anthropic is the one holdout.

• 1:29:01Speaker 00

  Anthropic just announced.

• 1:29:02Speaker 02

  Oh, did they? Okay, 11 for 11.

• 1:29:04Speaker 00

  Just reported in the past few days, I think they're partnering with Samsung on their own inference accelerators.

• 1:29:09Speaker 02

  All right, all right. Well, so this, this is a real moment in time in, in history because if you look at the biggest companies in the world historically, you'd have like an ExxonMobil, an IBM, a GE, all doing different things. Here we have the 11 biggest in the world doing the exact same thing. That's how big a deal this race to AI, you know, innermost loop, which includes the chips, how, how big a deal that is. So it's, it's, you know, it's a moment in history that's pretty unprecedented.

• 1:29:39Speaker 00

  So this verticalization.

• 1:29:42Speaker 00

  Do you expect it to continue and intensify?

• 1:29:47Speaker 02

  I would be shocked if the inference time custom chips aren't at least 100x and maybe 10,000x the performance that we're currently seeing, which will translate directly into IQ. I mean, the rate of acceleration from here, this is why it's clearly going to be a hard takeoff. The rate of acceleration will be unbelievable. Now, keep in mind, those chips are not deployed yet. So we haven't seen the effect of that, but it'll it'll come soon. And and when it hits, they're also likely to consume less power, be cheaper and easier to manufacture, so more will come out of the limited fabs that we've got. It's going to be a very fast takeoff after that.

• 1:30:26Speaker 00

  Building better tools.

• 1:30:29Speaker 01

  And have you looked at the design of these RFICs, RF-integrated circuits? They don't look human. They don't look designed. And they look more like QR codes than anything else. And I think this is instructive as to what AI super optimized designs of the future are going to look like. That we're familiar right now, if you look around you on on a street in a normal town in America, you see a bunch of things. You see cars, you see houses, you see streets. These are all manifestly human designed artifacts. As we start to, yeah, they're relatively simple, they're easy to parse. As you say, Peter, they often follow some sort of rectangular style form.

• 1:31:13Speaker 01

  Now, on the other hand, split screen, look at super optimized designs from the AI. They'll tend to look more quote unquote organic, they'll be noisier, they'll be more information dense, harder to interpret mechanistically. Yeah, and I think that that's what there's this landscape out there for any given physical system that you want to to have do something useful for you, where there's a subset in the Venn diagram of of design space that's human understandable and human designable, but then there's this dark matter outside of that inner circle that's AI optimizable and AI interpretable. And we're going to discover over and over again, starting maybe with RF antennas and RFICs

• 1:31:55Speaker 01

  in this case, that the AI optimized designs look alien and biological and look nothing like human designs.

• 1:32:02Speaker 02

  That's so true. It's really worth looking at the pictures, actually, to get a sense. But, you know, a lot of the way human engineering works is in layers of abstraction. Otherwise, it just boggles your mind. And when you look at chip design, you know, the modules are pre-designed, you know, for memory module, interconnect module, whatever. And then you drag and drop them. So it looks like a work of art in the end. Then you look at what the AI does, and it looks like a Borg spaceship, you know? Like, wow. But the same is true with the microcode. You know, Alex, he sent me that paper on AI writing kernels to run on these chips. And the microcode also is virtually impossible to read. But it's super efficient, and you can't deny that it works. If you run it, and it's clearly right. But it's not built modularly and easy to understand.

• 1:32:47Speaker 02

  And so it also is this layer of very tangled code on this layer of very tangled chip design. But it's so fast and so efficient that you've just got to do it.

• 1:32:57Speaker 01

  The other thing I thought was interesting in this Princeton IEEE paper is they don't call it this, but I would caricature it as an interpretability tax. They added a knob that enabled you to or the designer of these RFICs to tune up or tune down the level of interpretability. So if you wanted a less efficient design that was more human interpretable, you would sort of lower the spatial resolution of these AI designs. You wanted something that's less interpretable but more efficient, you could turn the knob up. And I think the notion of an interpretability tax is something that we're likely to see over and over again in AI.

• 1:33:35Speaker 02

  Yeah. You also see a lot of Claude explaining things to you, mansplaining things to you, basically.

• 1:33:41Speaker 01

  Claude-splaining?

• 1:33:42Speaker 02

  Claude-splaining, yeah. It's like, well, look, I know you can't really understand what I'm saying here. So let me give you like a high-level overview that you'll grasp. And you're like, okay, that's fine, as long as it works.

• 1:33:51Speaker 00

  So the question on this article is who owns the end product here? Is it the human or is it the AI? Which is going to lead us to our next story, gentlemen. This is out of Japan. It's the future of IP ownership in an AI economy. Japan's Supreme Court has ruled that AI cannot be listed as an inventor on a patent application. The case is based on a patent filing by US engineer Stefan Thaler, who claimed an AI is the inventor of technology related to food containers and other products. Japan's Patent Office rejected the application and asked for a human inventor. Thaler refused. The case moved to the Tokyo District Court,

• 1:34:36Speaker 00

  the Intellectual Property High Court, and now Japan Supreme Court, which upheld the view that inventors under current Japanese patent law must be natural persons. The court's message is important. They say, hey, basically, you know, judges are not going to rewrite the patent system on the fly. If society wants AI-generated inventors to receive protection, then you need to create a new framework. So, two fundamental questions. First, who owns an idea when the idea emerges from a model trained on the world, prompted by a human? And second, will any nation rewrite their IP laws first to avoid the need for meat puppets?

• 1:35:18Speaker 00

  So, Alex, you and I have talked about the notion that out of the current AGI and ASI ascendancy, we're going to see trillions of dollars of wealth created in breakthroughs fundamental to math, science, physics, biology, and material sciences. And the question is, who's going to own them? Your thoughts, Alex?

• 1:35:42Speaker 01

  President Javier Millay, if you're listening to this podcast and you want Argentina to take a globally preeminent position from the perspective of non-human AI corporations being able to create their own IP and their own patents, I think Japan just opened up a new market opportunity for Argentina. I think it's probably worth noting in the story that the underlying patent applications date to before ChatGPT. They're originally filed in 2020. So, this has been brewing for some time and with less sophisticated AI than what one might otherwise suspect.

• 1:36:20Speaker 01

  It's probably also worth noting that Japan's Supreme Court didn't indefinitely rule out the possibility of AI inventors on patents. They were merely saying that the existing statutes don't contemplate non-natural persons. It's probably also worth pointing out that, to my understanding of international patent law, it is relatively standard to only consider natural persons as inventors. For example, again, to my lay understanding, US corporations aren't able to be inventors for patents. They're able to be assigned patents, but they can't be the inventors of patents. So there is a bit of precedental bias towards so-called natural persons here as patent inventors and away from non-natural persons.

• 1:37:05Speaker 01

  However, however, however, this is obviously the sort of precedent that if and when some form of AI personhood is ultimately recognized, even if it's a partial economic or some sort of social personhood. I think this is the sort of precedent that's just waiting to be overturned.

• 1:37:27Speaker 02

  Yeah, I think this topic is extremely important, too. You guys had, you know, Peter, you and Axe had a really lively debate on this. I think it was two podcasts ago. But, you know, historically, in the venture world and the investing world, the mantra has always been, if you're relying on a patent, you're doomed.

• 1:37:42Speaker 01

  Yeah.

• 1:37:43Speaker 02

  Your business needs to survive and grow and thrive. The patents get granted many years later. They're very hard to enforce, blah, blah, blah, blah, blah.

• 1:37:50Speaker 01

  Need to be reinventing yourself, yes.

• 1:37:52Speaker 02

  I think going forward, intellectual property is going to be an exponentially growing, important category of endeavor. And that the US will end up enforcing intellectual property rights globally for things invented in America at a minimum.

• 1:38:05Speaker 00

  Can you imagine the speed of patent applications as AIs unleash, become allowed and unleash their creativity on all these fields?

• 1:38:14Speaker 02

  Well, also, you know, one of our companies, Stinkstruck, writes the patent. Like, you know, historically, one of the biggest barriers to getting your patent is the $100,000 legal bill to get it drafted over the course of months and the torture of that process. Now there are multiple startups that just do it, you know, here's the idea, AI write it up.

• 1:38:33Speaker 00

  And they have a huge corpus of data to pull from of, you know, the most successful patents out there.

• 1:38:41Speaker 02

  Well, amazingly enough, they also predict the inspector that you're likely to get and then look at their past behavior and try and predict what the inspector will do with different terminology.

• 1:38:49Speaker 00

  Exactly.

• 1:38:50Speaker 02

  So much better than a human lawyer at writing these applications. So that, you know, the rate of applications that go through the roof. And so then the, you know, the patent office is going to have to respond by reading them with AI. And that's going to lead to this whole intellectual property explosion. So then the question, question is enforcement. Is the US going to get out in the world and enforce? And I think they'll easily be able to do it with trade law. You know, the government, the military doesn't have to go into every country to say, hey, you're stealing all our IP. Trump has proven that with tariffs alone, you can compel virtually any behavior globally because the US economy is just that strong and accelerating. So assuming that trend continues, then intellectual property rights will be enforced globally.

• 1:39:33Speaker 02

  And then this, this whole area will become really important to, to keep following and talking about.

• 1:39:38Speaker 01

  I also think the same tools of superintelligence, maybe tools is an overstatement, are ultimately going to be available to every aspect of IP. So the invention stage, superintelligence. The application stage and the, the patent drafting stage, superintelligence. The filing and say, overall regulatory processes at the patent office or otherwise of recognizing and granting, say, patent state, patent status, superintelligence. Litigation, superintelligence. Litigation defense, superintelligence. The court systems that are overseeing and mediating the defense, superintelligence.

• 1:40:19Speaker 00

  Working, working around your patent, superintelligence.

• 1:40:22Speaker 03

  Yeah, exactly.

• 1:40:23Speaker 01

  All of it.

• 1:40:24Speaker 03

  I think the whole system is completely broken. Go back to the CRISPR patent, right? Within, within a few months, people had found eight or nine different mechanisms to deliver the same thing. After years of fighting over the, the one patent, they got ratted around very quickly. That's just going to happen at such an accelerated pace. With superintelligence, whatever we want to define it as, that you're going to end up in this whole mess. The whole system is essentially irrelevant going forward.

• 1:40:50Speaker 01

  I'll, I'll take a different position on this. I don't think the system is irrelevant. I, I simply think the, the routing around Salim that you refer to in the, in the instance of CRISPR. This would have happened on some time scale anyway, but with modern tooling and modern technologies. The natural process can happen on a faster time scale. And that I would say that the key time scale here is so order of magnitude patent. I mean, there are lots of ways it could be extended or, or otherwise changed. But call it like a 15 year time scale for a patent. What happens when the time scale, thanks to superintelligence for identifying route arounds, prior art, defenses, offenses, complements becomes so much faster than a characteristic 15 year time scale

• 1:41:34Speaker 01

  that it's the time scale of patent protection that's in some sense losing out. It's not that the regime itself is bad or that patent defensibility is dead or anything. It's just that innovation is happening so quickly relative to the originally. I, I think it's a statutory, a statutorily set time scale of patents that there's pressure to change the time scale.

• 1:41:54Speaker 00

  So this is the canary in the coal mine. This is going to hit us on so many different legal fronts in our current structure, right? Because the entire legal structure of every nation has been built on human time scales and, and the, the speed at which humans can process information. And it's all going to break and all going to be reinvented.

• 1:42:15Speaker 03

  Look, the simplest example is we have a representative democracy.

• 1:42:19Speaker 00

  Yes.

• 1:42:20Speaker 03

  Where Congress meets occasionally because a couple of hundred years ago, the fastest that information could travel is the speed of a horse. You have to give people time to ride across the country and say, here's what my people are saying.

• 1:42:31Speaker 00

  And Salim, the same way.

• 1:42:33Speaker 03

  Occasionally.

• 1:42:34Speaker 00

  Innovation, innovation will only occur at the edge, which is when you're starting your country and you redesign it from scratch. All right, so this is where we're, you know, there's gonna be, this is, I always talk about, we're gonna start new countries in cyberspace, we're gonna start new countries outside of the Earth's, you know, orbital sphere.

• 1:42:53Speaker 03

  And back to the accelerando plots.

• 1:42:55Speaker 00

  Yeah, yeah. It's gonna be a fascinating future.

• 1:42:57Speaker 01

  Big, big fan for what it's worth of starting new countries in outer space. The Outer Space Treaty doesn't look necessarily super favorably on starting de novo countries in outer space, but I think it's going to happen.

• 1:43:08Speaker 00

  Have you read "The Moon is a Harsh Mistress"? Leave us alone.

• 1:43:11Speaker 01

  Highland.

• 1:43:11Speaker 00

  We will land rocks on you if you don't agree.

• 1:43:15Speaker 01

  Moon is the ultimate high ground.

• 1:43:17Speaker 00

  The ultimate high ground.

• 1:43:19Speaker 01

  Rods from God.

• 1:43:20Speaker 00

  Gentlemen, I wish you a good evening. Thank you for a great conversation today. Everybody, I hope you enjoyed this wide-ranging conversation from consciousness to patent IP law. Dave, buddy, be well. Saleem?

• 1:43:37Speaker 03

  I have a plug.

• 1:43:38Speaker 00

  Go, Saleem.

• 1:43:39Speaker 03

  In two weeks, I'm, two weeks, I'm having my next Meaning of Life session.

• 1:43:42Speaker 00

  Where?

• 1:43:43Speaker 03

  July 21st, 7:00 PM, online. We'll put the links below, but.

• 1:43:48Speaker 00

  How long, how long did it go last time?

• 1:43:50Speaker 03

  Last time was six and a half hours, and we had more than three quarters of the people still there at six and a half hours. I had to call it.

• 1:43:56Speaker 00

  Oh, my God.

• 1:43:57Speaker 03

  So we'll, we'll do it again and see if we can make it a little more efficient.

• 1:44:01Speaker 00

  Fantastic. Alex, any, any breaking news in your world?

• 1:44:05Speaker 01

  Don't take off the takeoff. It's now a song.

• 1:44:08Speaker 00

  I, I, I love your neo-Logisms. Everybody, check out Alex's Innermost Loop Substack. It's, it's a beautiful thing to wake up to the morning, and I've got my Substack on there as well. I will put links in the show notes here. Dave, are you publishing yet?

• 1:44:26Speaker 02

  Go to db2.ai, keep your eyes open.

• 1:44:30Speaker 00

  All right, fantastic.

• 1:44:31Speaker 03

  Oh, you know what I am doing?

• 1:44:32Speaker 00

  What are you doing?

• 1:44:33Speaker 03

  I'm adding, I'm doing AMA sessions for some of the comments in a separate video on our YouTube channel.

• 1:44:38Speaker 00

  Awesome.

• 1:44:39Speaker 03

  Because it's too, it's too difficult to try and answer all these questions.

• 1:44:41Speaker 00

  All right, gentlemen, I wish you a good night, or good morning, depending on what part of the planet you're in.