AI:AM Highlights: Astra as AGI, OpenAI's Pause, Mythos @ Mozilla & Human Agency vs Technocapitalism

Nathan Labenz and Prakash Narayanan examine the capabilities of GPT-6 Astra and the economic forces accelerating frontier artificial intelligence. Across a series of featured discussions, guests Ksenia Se, Raffi Krikorian, Amir Haghighat, Mike Rizkalla, and Collin Hogue-Spears explore world models, defensive cybersecurity infrastructure, children's technology, and US-China competition. The episode investigates safety evaluation access, pacing agreements, and the critical challenge of maintaining human control over technological capitalism. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/ai-am-highlights-astra-as-agi-openai-s-pause-mythos-mozilla-human-agency-vs-technocapitalism/ Sponsors: Athena: Athena matches you with a dedicated, top 1% executive assistant to handle your inbox, calendar, and daily workflows so you can save an average of 15 hours a week. Get matched with your EA today at https://athena.com/cognitive OutSystems: OutSystems is the leading agentic systems platform that enables enterprises to engineer, orchestrate, and govern AI applications on a single unified platform. Learn more and see how it works at https://outsystems.com/tcr Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr CHAPTERS: (00:00) The Astra weekend (Part 1) (10:19) Sponsors: Athena | OutSystems (13:50) The Astra weekend (Part 2) (13:51) An alien mind (19:43) Reassure and mislead (Part 1) (25:07) Sponsor: Claude (26:42) Reassure and mislead (Part 2) (38:44) No adult in room (52:24) The defender's ledger (01:06:27) Toy without generative AI (01:11:43) Nobody picks up phone (01:18:45) Can we stand up (01:37:30) Episode Outro (01:40:05) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Nathan Labenz?

  My co-host Prakash Narayanen, after a weekend working with GPT-6 Astra.

• 0:08Speaker 01

  People are going to use this thing. Token spend is is going to increase dramatically. I think a lot of people are going to be using it all the time.

• 0:17Speaker 01

  It is AGI. It is that kind of cleared the hurdle of AGI. It will do things better than most people you can hire and train.

• 0:29Nathan Labenz?

  Welcome to the AI and the AM Weekly Highlights. This is Nathan, using my cloned voice to introduce clips from our three live shows this week. Let us know what worked and what did not.

• 0:42Nathan Labenz?

  Part one, the Astra Weekend. Tuesday, September 8th. Here is what Prakash had been building.

• 0:52Speaker 01

  I spent the entire weekend using Astra.

• 0:59Speaker 01

  I was running three to four agents continuously.

• 1:06Speaker 01

  And they were good. Astra is very, very good. In the sense that it started to tackle those annoying problems which had been in the codebase. As you know, we built the studio by ourselves. So, and this, it started to tackle some of the longstanding issues in the codebase, which had been kind of annoying and bugging me and like it started to like resolve those issues. It is very, very good. I would say it is finally at the point where if you care about the quality of the work, you can still hand it off to Astra.

• 1:48Speaker 01

  But you still need to do a little bit of talking, but you can hand it off to Astra and you can get some results. And, you know, and the computer use is good. The other thing that was failing really badly, I think before is computers. Computers on GPT-5.6 would sometimes take a very, very long time. It would kind of click around and do a bunch of stuff and a computer use finally works properly in the kind of like timeframe that you give it. So it's clear, it's cleared the hurdle. It's cleared the hurdle of like genuine usefulness at this point. And you can start to give it more advanced tasks. So this is a guy called Skalsky.

• 2:32Speaker 01

  So he trained models to identify players on the basketball court. He hand labeled 12,000 individual like images with who the players were, referee, or this player, that team, etc. And he hand labeled 12,000 images.

• 2:55Speaker 01

  And now Astra can just do it. Like Astra just does it. This is a task that a human being will never do again. Like there just isn't any, any point. You can't even pay someone to do it. Because if you paid someone to do it, they would use Astra to do it and then like pass you back the results. Like it's done. Like a human will never do this task again.

• 3:22Nathan Labenz?

  I had been watching how Astra keeps working through long tasks and how it uses notes to stay on track.

• 3:30Nathan Labenz?

  How is it that these new models are so persistent, right? How is it that they can come up with such elaborate chaining togethers of all these different exploits to finally accomplish a goal that, you know, if we had to do so many things, we would just give up, right? And most models historically weren't able to do it. It seems that they have a new way of handling history, which like many brilliant insights seems pretty obvious in retrospect, but nevertheless is new. And maybe Anthropic isn't doing this and hasn't said it, but what I understand in Astra is now doing

• 4:12Nathan Labenz?

  is instead of compacting its million tokens into a summary and then essentially starting a new context window with that summary, but losing all the detail that was summarized away in that compaction process. Now there is a long-lived notes file that the model can update whenever it needs to. And this kind of follows it forward in time regardless of how many tokens it's laid down. And then it has the ability to go back and search through its own session history. So now, even though you only have maybe still the same million token context window,

• 4:54Nathan Labenz?

  you know, million tokens, all it can handle in one shot, fully attending everything to everything.

• 5:01Nathan Labenz?

  It has enough via the notes and the ability to go back and search and see what's done before to pretty effectively manage, you know, it seems like at least 10 times that much context in single rollouts.

• 5:15Nathan Labenz?

  How do you measure what these models can do now? Prakash started with the chart from Meter, the research group that tracks how long a task AI agents can complete.

• 5:26Speaker 01

  You know, one of the people online, Ethan Malik, who is a professor, who tests a lot of models. He posted the meter, the famous meter, hours, you know, hours of work chart. There hasn't been an update for a while now, you know.

• 5:48Nathan Labenz?

  I think it's, I don't think they can really do it anymore.

• 5:51Speaker 01

  Yeah, they can't.

• 5:51Nathan Labenz?

  They don't have tasks that are big enough.

• 5:53Speaker 01

  They don't have tasks which they can measure before the next model drops.

• 6:02Speaker 01

  The time, the cycle time of like model development is shorter than the length of the tasks that they need to measure at this point. So I think the meter graph is basically done at this point.

• 6:19Nathan Labenz?

  Meanwhile, OpenAI had published its own measure in a post about research acceleration inside the company. The unit was the agent workday, and OpenAI reported 3.1 of them for every human workday.

• 6:33Nathan Labenz?

  I tried to look into the methodology on what exactly is an agent workday? And it's not super crystal clear to me exactly what they mean. I don't know if you have a better read, but my take was, my takeaway, trying to make sense of it was just like literally how long do agents run for? So it seemed like they're saying for every eight-hour workday that they have human researchers doing, those researchers have agents running for 24 hours

• 7:05Nathan Labenz?

  of real time. Here, by the way, is maybe the closest thing we're going to see to the meter chart for a minute. This is from this recursive self-improvement begins blog post. And basically they're kind of reformulating the meter chart here showing how often Astra can succeed on tasks as they are grouped by how long they estimate it would take a human to do the task. So what we're seeing now is like basically in that one to two workday zone, 40% of the time it can do the thing, zero interventions needed,

• 7:46Nathan Labenz?

  pushing 90% of the time, given some human intervention along the way. And naturally that drops off, but even as you get to like, here we're talking one and a half to three weeks worth of work, can still do that on a one-shot basis, one in six times and two-thirds of the time if you allow for some human intervention. This is, this band is one or more interventions. So one would assume that presumably as you go through the longer and longer tasks, it's more interventions that are required to get the thing to succeed. But overall, still two-thirds of the time it can succeed with some help

• 8:29Nathan Labenz?

  on tasks that they estimate would take a human essentially two to three weeks to do.

• 8:37Nathan Labenz?

  Reports about code quality were mixed. I discussed code that people found useful but struggled to read.

• 8:45Nathan Labenz?

  There's been conflicting or certainly like diverging reports from various people, some saying it's amazing, it can do all that stuff, you know, it can write, you know, code in the way that you need it to be written so that it can be maintained, blah, blah, blah. But then also reports saying that if it thinks it's not going to be checked in that way, or if it looks like the kind of environment where it's just a matter of performance and nobody really cares how it looks or how it gets done, then you get code back that's like a really gnarly mess that people can't really understand. Does seem to work. I've seen this reported for kernels, GPU kernels specifically, which

• 9:27Nathan Labenz?

  is obviously super relevant to the labs, highly verifiable as well, right? Like you can definitely do hardcore verification on, did this matrix math actually get to the right answer? And in the middle, you kind of don't know, don't know, don't necessarily care exactly how all these different steps were fused together. And somebody summed this up by saying, we're going back to machine code in more ways than one. Not only is it like lower level, gnarlier stuff that we can't read very well and, you know, would need additional abstractions on top of to really make sense of, but also in this case, the machines are writing it directly. So machine code starts to take on multiple

• 10:12Nathan Labenz?

  layers of meaning.

• 10:15Nathan Labenz?

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 10:20Nathan Labenz?

  Today's episode is brought to you by Athena, the executive assistant company on a mission to improve how people work and live. If you want to increase your impact, you have to free up your time. And that's what Athena does best. They match you with a dedicated, full-time, top 1% executive assistant who can take over your inbox, calendar, travel, and everything else that's quietly eating up your week. Athena is SOC 2, Type 2 certified, so you can rest easy knowing that your sensitive data is in good hands. And as a former AI advisor to the company, I can personally vouch for how much they've

• 11:02Nathan Labenz?

  invested in AI tools and training. In fact, one of the very best AI users I've ever met is an Athena client who delegated the exploration of AI tools and the development of AI workflows to his EA. Athena clients report saving an average of 15 hours a week, and the average client refers more than two friends a year. That, to me, checks out. I was an Athena client while running my startup, and to this day, I continue to refer friends. Go to athena.com/cognitive right now and get matched with your EA. That's athena.com/cognitive.

• 11:43Nathan Labenz?

  Give yourself back a few hours this week. Go to athena.com/cognitive and see who they'd pair you with this month. This episode of The Cognitive Revolution is brought to you by Outsystems, the leading agentic systems platform. We've all seen the headlines. Companies are pouring money into AI. But the big question on every executive's mind right now isn't just how fast can we adopt this? It's where is the ROI? We're seeing a real trend toward AI chaos. You've got teams deploying standalone coding tools, random agent builders, and experimental scripts. It sounds innovative, but in reality, it's creating a massive headache. Fragmented tools,

• 12:26Nathan Labenz?

  ungoverned data, serious security blind spots, and costs that are spiraling out of control. If you don't bring those agentic applications under control now while they're still embedding into your core processes, you are looking at broken systems and damaged customer trust down the road. That's where Outsystems comes in. Outsystems is the leading agentic systems platform for the enterprise. Instead of managing a patchwork of disconnected tools, Outsystems lets your team engineer, orchestrate, and govern your entire agentic ecosystem on one open, unified platform. It's built for the speed of AI, but with the reliability and security that enterprises actually require. We're talking about real results, like Key

• 13:10Nathan Labenz?

  Bank, who used Outsystems to deploy a customer-facing app that delivered 75% faster onboarding times. Or the global logistics leaders who built agentic systems to completely eliminate their engineering bottlenecks. You don't have to choose between speed and control. Whether you're a small team or a massive enterprise, Outsystems helps you engineer, orchestrate, and deploy agentic systems that actually scale. Stop chasing the hype and start owning your agentic future. You can see how it works and learn more at outsystems.com/tcr. That's outsystems.com/tcr.

• 13:51Nathan Labenz?

  Part two, An Alien Mind. On Tuesday, Cassinia Say, founder and editor of Touring Post, joined us. Her recent coverage focused on world models, and she had just attended a workshop about them. Prakash asked about the physical world.

• 14:06Speaker 01

  I have noticed that in the last, I don't know, 48, 72 hours, people are starting to use Astra to do robotics, for example. So we've seen, we've seen a few demos, and there's even been commentary that, okay, if Astra was, you know, a thousand times faster, you can conquer the latency part, you could actually use it directly for, in order to act in the real world. That, to me, kind of says that maybe there is starting to be an intermediate representation, an internal representation there. Like, what do you think about, is our models like Astra kind of a little bit different in that sense?

• 14:46Speaker 02

  And I just came back from a workshop about world models, and it was absolutely jarring. There were tremendously smart people from Stanford and Harvard, and Yan Le Kun was there, and they were all discussing world models, but they do not agree on what world models actually are. So when we talk about world models and why I want to focus on them in my publications, because I think it's just more about action, being able to predict and act, like wider understanding what's happening. Physics is a super important part of it. That's why robotics is so much more about world modeling and world models. But

• 15:27Speaker 02

  again, it's like, for me, it's understanding the scope of it and trying to give it more precise terms as well. But we were just in the very beginning. What each of you understand when you say super intelligence? What is it?

• 15:44Nathan Labenz?

  Sort of move 37s across a lot of different domains. When we start to see systems saying, I think this would be a really good thing to try for the next battery, you know, substrate. And then it turns out, oh, my God, you know, that's a lot better than what we had before. And we wouldn't have thought of something like that, but low and behold, it works. Anything that can do that across like a non-trivial number of reasonably high-value domains, I think, starts in my mind to count as a super intelligence.

• 16:22Nathan Labenz?

  We also discussed her essay, Permanent Dawn, and writing with AI.

• 16:28Speaker 02

  I actually spent, I think, like six hours on writing that post. And the funny thing was that I was so unhappy with every model that was trying to help me write it because it's like very complicated philosophical text that I wrote it myself. And I sent it to Fable, which I never use usually on my daily basis. And I sent it to Fable because I was running on a deadline. And I said, fix the grammar. And I didn't notice that it fixed not only the grammar, but it made this like shorter sentences, the way Fable does it. And that was the first time when I received the message, like, I will unsubscribe

• 17:11Speaker 02

  because you use Fable. So people really understand when you use a model because every model has its own language fix. And I was like, I spent so much time on this article. There was like all my original thoughts there, but the language that I didn't catch gave away that the model was like the lost editor. Anyway, yeah, I think people will still appreciate when you, when they see that you put effort into that and there are original thoughts there.

• 17:41Nathan Labenz?

  Our next topic was the feedback loop between AI research and the development of better models.

• 17:48Speaker 02

  And everything is now the part of the loop, this feedback, this constant feedback. I just had a conversation with two people from inference team in OpenAI, and they also say that this is a constant, constant loop where the models now become better, sometimes better, just like trying things. So you throw the whole database of research that has been done for years, and then the model can actually choose and pick and do this all experiments because it would be impossible for humans to spend so much time on that. And models can do that. I'm still learning about self, about recursive self-improvement. And I don't,

• 18:33Speaker 02

  I don't know what are the main bottlenecks for me. Maybe you can even say what you think are the biggest here.

• 18:40Nathan Labenz?

  Honestly, I don't know that there's that many left. It does seem to me like increasingly, I feel the cope meter going off when people are trying to say, you know, what it is going to be that is going to prevent the models from running away with the whole process. I would love to see some bottlenecks that I really believed in, but right now, I'm kind of of the mind that they're more often wishful thinking than they are like real hard bottlenecks that can't be overcome. I mean, our ability to like keep the things

• 19:25Nathan Labenz?

  from going totally rogue might be one bottleneck on the overall process. So human decision-making still has a big role to play for a while yet. That's not it. In terms of inability, I don't see too many that I would expect to last all that much longer.

• 19:44Nathan Labenz?

  Part three, Reassure and Mislead. Back to our Tuesday discussion of external evaluations. I raised the report that Apollo Research had received only three days with Astra. We were also discussing An Alien Mind, the essay by OpenAI chief scientist Jakob Pachaki. His essay called for voluntary slowdowns and international coordination.

• 20:05Nathan Labenz?

  Notably, I think Apollo only had Apollo Research, who does the deception, science of scheming, chain of thought monitoring work with OpenAI. They've had a pretty longstanding partnership. Apparently, this time around, they only had three days to test Astra before it was released. So again, I come back to this idea that the the model reviewers, auditors, testers, red teamers, scheming scientists, they need more time. This is like, pretty ridiculous that they only had three days at the like, at this point, why even do it? You know, just put the thing out there. They can test it live.

• 20:49Nathan Labenz?

  Like, why even have anything if you're only going to give them three days? Precash questioned whether external auditing could work. I responded on the funding and independence of the auditors.

• 21:03Speaker 01

  The other thing is that when you release models, you end up wanting to have the final release candidate to be the one that gets vetted and audited. And the problem is that in the model lifecycle in this pipeline, there are 100 different candidates, right? At points, there are 100 different candidates. And then some don't work or some, you know, fall by the wayside and you narrow, narrow, narrow, narrow, narrow. And then you have like a couple of release, two or three release candidates. And then sometimes it's only the last two or three days. You're like, all right, we're going to go ahead with this one and you make the decision. And so the problem is if you want that kind of operational flexibility to make that decision, you're going to end up with you only have a few days

• 21:48Speaker 01

  to offer to an external auditor. So the other option is you bring the auditor in, right? So you bring the auditor in-house. I mean, you bring them in and they'll take a look at the models ahead of time. So they're in there like a month, month and a half ahead. They're taking a look at the release candidates in general. But number one, the auditors are often not like super well-funded. They don't have like that many people. The, the opening anthropic are so many more people than Redwood Research or these teams. So they don't have the capacity to, you know, audit like, you know, 10 different, you know, release candidates. It's not, it's not there. And they're also not very, very well-funded.

• 22:31Speaker 01

  So they're dependent on the model companies for that funding too. And then you have this like, the ethical process of like, okay, how much funding can we really accept from them before we're kind of bought? In addition, a lot of the auditors, the guys who train with auditors, leave for model companies in a couple of years. So there's also this flow of people from like Meter or Redwood Research or like the trainees or interns, and they're flowing into the model companies, right? So there's another fear that, you know, the auditor comes in and they take a look. And three months later, someone from the audit team leaves to the other firm and they take, they manage the spots on the secrets and they're, you know, they, they, they share that. So there's that issue as well. So a bunch of these things make it like

• 23:14Speaker 01

  very, very difficult for this to happen. And you need that whole thing that like, like if you and your competitor like make a pact not to hire people from an auditing firm, that's an antitrust issue. So all of these things intersecting like make it like, I think, a very tangled problem. And I don't think there's a real solution. There, there hasn't been in the financial sector. So the financial sector has had this problem of the revolving door between, you know, the people who regulate the industry and the people who participate in the industry.

• 23:52Nathan Labenz?

  Redwood is now saying that their compensation for member of technical, technical staff roles ranges from $350,000 to $850,000 per year, which might not be frontier lab money, but it's certainly, you know, a living wage, even in the Bay Area these days. So that, you know, should be enough to retain some mission-oriented talent at least. So, and, and notably, I don't know about Redwood through all of history, but Meter has said that, you know, they don't take any money from the frontier companies and don't intend to. We can at least have confidence that their financial independence, you know,

• 24:36Nathan Labenz?

  means that their, their judgment is not for sale. Again, to, to me, the big thing is just like, they can't complain too loudly or they might not get invited back. I think that's the dynamic that really most threatens their work. It's just that it's all contingent on continued goodwill and, you know, very much voluntary choices from the decision makers at the companies. Hey, we'll continue our interview in a moment after a word from our sponsors. Today's episode is brought to you by Anthropic. By now, you know my story. Claude drafts my intro essays and I rewrite them. Not because the drafts are bad, but so I can stand behind everything I publish.

• 25:20Nathan Labenz?

  Well, I have an important update. Claude Fable 5 is the first model to have me rethinking my rule. Today, I now think co-authorship, not sole ownership, should often be the goal. Where the model excels, rewriting its work can be more about vanity or a misplaced sense of duty than integrity. I feel it most in songwriting. I'm no lyricist, but I'm good with a song concept. And Fable writes some amazing verses. I give it feedback on its misses and I push it to aim for higher inspiration, add layers of meaning, optimize syllable density, and above all, write a hit song. These days, I get compliments on just about

• 26:03Nathan Labenz?

  every song we write together. Claude is the AI for problem solvers. It's the collaborator that understands your entire workflow and thinks with you, not for you. Whether you're debugging code at midnight, building a financial model or strategizing your next business move, Claude extends your thinking to tackle the problems that matter. For problems worth solving, get started with Claude at claude.ai/tcr. That's claude.ai/tcr. And check out Claude Pro, which includes access to all of the features mentioned in today's episode. Once more, that's claude.ai/tcr. In Tuesday's closing, I read OpenAI's announcement about an internal model it described

• 26:47Nathan Labenz?

  as significantly more capable than Astra. The announcement concerned a proposed Navier-Stokes proof with a smooth external force. The unforced problem remains separate.

• 26:59Nathan Labenz?

  So we just had Astra launch, right? So the, I'd say the big headline news for this announcement is it confirms that there is an internal model that is significantly more capable than GPT-6 Astra. So that's this clause here. An OpenAI next generation model significantly more capable than GPT-6 Astra. How much more capable? Well, on these significant open math problems with, you know, maybe up to an order of magnitude-ish additional test time compute, they're able to go from 10 to 15% solve rate on a curated set of open math problems

• 27:42Nathan Labenz?

  to now like 25 to say 45%. So that's significantly more capable. That seems fair. And here they're showing the amount of RL compute they are spending on a daily basis by class of model. But what we see here is like, Astra class models had RL significantly declined twice. The rest of RL compute is basically unchanged. But now it also raises for me the question, were they in fact still running RL on the next generation? I don't, going back to July, that's six weeks ago. I don't think they've had this result for six weeks.

• 28:25Nathan Labenz?

  It sounds to me like, at least one reasonable interpretation is they continue to run RL on more capable models than Astra.

• 28:34Nathan Labenz?

  The chart separated reinforcement learning compute for Astra from other models. The category labeled non-Astra was blue.

• 28:43Nathan Labenz?

  When you say non Astra with that blue color, does that mean only only models less capable than Astra? Or does it include models more capable than Astra? If it includes models more capable than Astra, it's like flagrantly misleading. And it's the kind of thing that makes it very difficult for you to have agreements with other entities. You know, if you if you want to pace the frontier, if you want to do all these things like you've just got to be better about making clear what you are and are not doing.

• 29:13Speaker 01

  So they're running a, I don't know, 50 billion, 70 billion dollar revenue company, right? So number one, you can never stop inference. Inference has to continue, right? So inference never stops. Your your your customers depend on. So the inference team will continue no matter what, right? No matter what, they're they're there for like 24/7 availability, best SL is possible. We'll continue, right? And some of sometimes you have to RL certain behaviors out, right? So you have to continue some forms of RL. You you can't just stop because your inference demands it. Like if you have some, your, you know, model is behaving badly in a certain instance and it's been identified and you have the data to do it, it will be malpractice not to apply RL to train that behavior out.

• 29:57Speaker 01

  So that that has to continue. So then you have the rest of the stack, which is really future looking, right? And then you have the future looking stack. And of that, my understanding was that they shut down like trading of like more advanced models. Why not shut down trading of less advanced models? Because those less advanced models would have to be deployed on inference. Again, they they're training, they have this teacher assistant system, right? So they they're training the the you train the larger model first and then you distill down into the smaller model. And then those become like your Luna and like your Terra and like the smaller model groups. So you also don't stop training smaller models. That also continues, right? So it's only where you are training models which are larger, larger equivalent or larger models.

• 30:42Speaker 01

  That that's where you the the larger pre-trains equivalent or larger like RL on equivalent or larger models. That I think they would have stopped. Did they fix the RL pieces that, you know, the agents were like not reporting, not telling on their peers? The agents are trying to break out. I think that is probably a difficult thing to fix and show. And I think that behavior, I think, will continue to be a challenge that they'll have to work on. That that's my guess.

• 31:12Nathan Labenz?

  But I still think you look at this graph and you're like, okay, what was declared was a pause on frontier scale RL. But what actually happened was kind of half of RL was stopped initially with the sort of disclosure of the hugging face incident. Half kind of continued. The that was enough still for Astra models to take over part of OpenAI's research infrastructure. When that happened, they still didn't shut it all the way down. They still only cut it by half. And meanwhile, at the total level, you know, we don't know what's in the blue. Part of my gut says like, they wouldn't be so brazen

• 31:56Nathan Labenz?

  as to have more capable than Astra models in the blue color. But, you know, I've been disappointed before and I'm just afraid that like

• 32:10Nathan Labenz?

  all this, you know, the the view from Anthropic is you can't trust these guys. They say something that's like maybe technically literally true, but it's really very engineered to what they think you want to hear. And then the reality is like quite different from what you were led to believe by their like very galaxy brain engineered statements that sort of reassure and mislead at the same time. But I'm kind of worried that like, right now, we're living in this zone where Anthropic is going to continue to trust less and less with these sorts of mixed messages. OpenAI people are going to feel like they're just being treated unfairly.

• 32:55Nathan Labenz?

  And this is the scenario that like, this is the problem. This is the problem that has to be solved if we're going to actually get to the point where Jacob's prayer is answered. You know, right now, it's like

• 33:10Nathan Labenz?

  they don't have the trust to to do

• 33:14Nathan Labenz?

  a deal with Anthropic or really anyone else, I don't think.

• 33:18Speaker 01

  The real frontier model is not the model which is deployed, obviously. It's not the model which is in training also. It's actually the model which is in the heads of the researchers. Because those are the ideas that will become the model in, you know, 12, 18 months. And just because you slowed down like on RL doesn't mean those researchers stopped researching, right? They're still running and most of the time you run small models and you test your ideas on small models. The slowdown would have been on the, you know, post-training of the larger models, which is where the bulk of the compute was being used. So did it really slow down? Probably not.

• 34:00Nathan Labenz?

  We then turned to the case for continuing development and the capital needed to fund it.

• 34:07Nathan Labenz?

  But what one thing I still feel like is a big challenge is like, what is the overall story that you could tell? What's the what's the super high level macro steel man for OpenAI? Where are we now? What are the commitments? What are we doing? What are we not doing? Can we can we synthesize or summarize an OpenAI position that we could like not have to caveat 1,000 different ways? I personally don't think I could do that. If you can do that, you know, I think you might deserve a Millennium Prize.

• 34:42Speaker 01

  No, no, I I I think I think they have a lot of stresses pulling them in different directions, right? So and I think internally within the firm, there's a fair amount of debate. I think I think to some extent, the capital cycle is forcing them forward. And the capital cycle is being forced, I think, by Anthropic. Anthropic didn't put in enough money earlier on. And so there is this intense pressure on Anthropic because they don't have the compute to have much better models. If you don't have the compute, you need much better models that can utilize the limited amount of compute that you have. So I think Anthropic is being driven driven forward by that to stay on par with OpenAI. And I think OpenAI is a little bit kind of like they're willing to pace the frontier because they have the compute.

• 35:27Speaker 01

  So, you know, regardless, you know, they're the ones with the compute and they have the compute three years ahead. Elon will take time to come up with compute and Elon will sell to Anthropic, but Elon's going to take a long time, like three years at least. So OpenAI, they basically mortgaged themselves in the last 18 months to Masayoshi Son and a bunch of other people they diluted. They they gave up to Microsoft, right? They had Microsoft, they they negotiated Microsoft. Microsoft gets all their models to 2032. They declared AGI and Microsoft is not out of their hair yet. You know, Microsoft is now, I still own 28% and, you know, I'm still there, right?

• 36:10Speaker 01

  So they made all these sacrifices in order to get all of this money in order to pump it in and they have the compute. And having the compute allows them to actually pace the frontier because I have the compute anyway, right? So Anthropic is under pressure. They don't have the compute. And if you don't have the compute, you need better models. And this is the thing that's happening, right? Like OpenAI is willing to pace because they have the compute.

• 36:36Speaker 01

  That's the that's the that's the thing. And they also know Sam has played these cards. So he knows that if Anthropic is willing to pace, he's going to win because he has the compute. And Dario can't afford that. So again, you're you you in this position that is, you know, as I say, the second and third place guys are the ones who who are going to define how how how fast the frontier paces. Like as if Elon or Meta catch up to Fable, it's over, they're going to have to put out, you know, GPT-7. There's no choice anymore, right? So this is where we are.

• 37:20Speaker 01

  Would you prefer Meta or Elon have, you know, the the golden ring?

• 37:25Nathan Labenz?

  Yeah, no, that's, I mean, that that's the new But China. And it is, I think, more compelling, honestly, than But China is. But Elon and suck.

• 37:37Nathan Labenz?

  Prakash also considered whether recursive self-improvement could change the plans of Sam Altman for an OpenAI public offering.

• 37:45Speaker 01

  I think Sam might have been sincere in saying if they hit RSI, they might not go IPO. If they have like one or two transformer level innovations in the next six months, maybe they don't go IPO, right? Like it's not necessary anymore and they continue as a private organization. And I think that would be bad. I actually think that would be bad because then you don't have transparency. You don't have like widespread ownership of the stock. You don't have like, you know, boards that have to answer and you don't have like lawyers that can sue them for shareholder lawsuits. You don't get a bunch of these things that you get for free with a public, public company. So I think that would not be good. So I'm hoping they do go public.

• 38:25Nathan Labenz?

  Well, I guess there's, if nothing else, there's a little more reason today than there was yesterday to believe in the possibility of singularities in finite time. And that might mean we never get to own any of that OpenAI stock on the public market.

• 38:44Nathan Labenz?

  Part four, no adult in the room. Wednesday, September 9th, we discussed the resignation of anthropic researcher Jacob Coxon, who had also worked at OpenAI. This is a different person from Yakub Pachoki. Coxon had questioned whether private companies should decide when to launch self-improving superintelligence. Prakash first.

• 39:04Speaker 01

  Look at the framing of this sentence. Accepting this race and entering the endgame is a hubristic gamble that should not be launched from a private company slack. Do you think Pete Hegseth's Signal Group is a better place to launch this? I mean, I mean, like, do you think there are wiser people out there running things? So I think, I think to take a step back here, there, there is no adult in the room. There's no one that's going to save you. There's no adult somewhere else that you can pass off the responsibility to, right? You have to start off by saying like, look, this is, this is the way that things are. The entire world is kind of duct taped together. Everyone's kind of making

• 39:48Speaker 01

  it up as they go along. The smartest people in, who are, you know, capable of handling this are mostly already inside these organizations. And there is no, like, the, what you get from passing it off to like the government is you get political legitimacy. You don't get, you know, political legitimacy that you need for execution, that you need to persuade people. You don't get wisdom. And so this is the state of the world. And, and you have to start off by accepting that the world is this way.

• 40:22Nathan Labenz?

  The world doesn't have to be this way. So here's my new brainstorm on this, actually. I agree with you strongly that like swapping Hegseth in for Dario, not a good trade. I think maybe what the government needs to do is treat these companies kind of like I treat my kids sometimes and say, you guys have to figure it out and here's your deadline to do it. And if you don't do it, then I have to come in and be the bad guy. And I think that right now there is an opportunity because they've both been crying for help, as you described it yesterday, I think pretty aptly. So given this sort of latent desire

• 41:05Nathan Labenz?

  but this lack of trust, also a lot of excuse making, as we've talked about a little bit around, oh, well, we can't do this, we can't do that. It would be antitrust, it would be this, it would be, you know, we'd get in trouble with the government. I think the government could easily say, safety collaborations are not going to be the target of antitrust enforcement, or, you know, to the degree there's worries of other enforcement, other kinds of enforcement. We want you to do it. And here's the deal. If you don't have a deal for us by the end of the year, that makes some sense and gives us some confidence that you guys are not going to race each other off the cliff, then, then we come in. And then you might get the nuclear outcome, which is to say, your company really might

• 41:49Nathan Labenz?

  not be able to grow in the way that it wants. We might really fuck it up, frankly, because we are the government, right? And we do get heavy handed and we don't put sunset clauses on our laws and we make all these bad mistakes. So get it right. So we don't have to come in and do all that heavy handed shit that nobody wants.

• 42:05Speaker 01

  So I think the, the, the main problem that you have is again, what will Elon and Zuck agree to? What will someone with billions of dollars and the ability to use lawyers and the legal system and to take cases to the Supreme Court agree to? So the question is, what realistically can you get Elon and Zuck to voluntarily agree to? Or are you going to be able to go into legislation?

• 42:36Nathan Labenz?

  I think that one thing really important to remember though, is that the timelines are all pretty short.

• 42:40Speaker 01

  Very short.

• 42:41Nathan Labenz?

  So you don't have to solve this forever. You just have to put a short-term deadline so that the companies have a strong incentive to come together and do something. And I do think that the executive, you know, I had an experience at Meta and I saw a little bit of what their life under consent decree looked like. And I think that they would be willing to do quite a bit to avoid another one of those sort of experiences. So I do think that the, on the few month timeline that we're talking about, and I'm willing to, you know, ride it out with free speech and there's a lot of other issues, but like,

• 43:23Nathan Labenz?

  can you five companies come together to pace the frontier in a reasonable way that you all agree and you can all police each other and you can have whatever verification mechanisms? You guys can govern this thing. Do it quick, or else, life's going to get hard. I think that message can resonate and I think it can bring Zuck and Elon to the table because, you know, what's he going to do? Like, at some point, the government has shown that it's willing to twist arms. They can twist his arm still. You know, would he rather be forced to the table with four other mega tech mogul CEOs and have to find common ground with them? Or would he rather have the EPA on his ass at every data center he's going

• 44:07Nathan Labenz?

  to try to build, at every launch site he's going to try to build? You know, that, that the government has a lot of sticks and like, they're willing to bend the rules, right? So sure, you could challenge it in court, but I'll see after the singularity if you want to do that.

• 44:20Nathan Labenz?

  Later Wednesday, I read John Schulman's reply to Jacob Coxon and returned to the question of antitrust enforcement.

• 44:28Nathan Labenz?

  And I'd say John Schulman here is saying a pretty similar thing to what I was odd about earlier, basically saying like, these companies need to start working together first. As he puts it, bringing in the US government before there's a concrete proposal will likely result in something dumb. That's, you know, your point as well. Like, there's nobody better than the people at the companies to do this. So we, I certainly don't want to see Hegseth's, you know, Navy come in and try to regulate AI. But I think this is the, the recipe. I've been circling this a lot myself, seeing him say it, you know,

• 45:08Nathan Labenz?

  kind of reinforces it to me. So I really like this. And I also think his key point on antitrust being fake, I think that's true. But I also think the government should take that doubt off the table. It would only take a couple of sentences to say, hey, you really don't have to worry about this anymore. And yeah, maybe the law could change. Obviously, the administration is going to change, but we're not even to the midterm yet. So this, you know, for better or worse, this is the administration that they have to worry about for the foreseeable future. These guys genuinely are planning around a singularity before

• 45:51Nathan Labenz?

  Trump is out of office. You know, it's at least a very live possibility.

• 45:56Nathan Labenz?

  Prakash then challenged that proposal, drawing on Operation Warp Speed.

• 46:02Speaker 01

  The proof point is Operation Warp Speed and the COVID vaccines. And it is, for Operation Warp Speed, the pharma companies specifically required a waiver from vaccine claims later on. So they specifically wanted a waiver, a safe harbor, and they got it. They got legislation and they got it, right? And post-COVID, it's very clear that if they had not got it, they would have been sued to, you know, oblivion. So I think that's a proof point

• 46:44Speaker 01

  that shows that, look, these are valid concerns and your company can be wiped out in retrospect. And the lawyers are not, you know, being foolish when they tell you that this is, this is going to happen in the future. Like, if you don't get that safe harbor, like, these things will come back to haunt you. You can, like, every, all the decision makers right now can be perfectly like, you know, on board. It doesn't matter because only the laws bind decision makers in the future. Decision makers right now, whatever they say, they bound by their word at best, right?

• 47:18Nathan Labenz?

  OpenAI appointed Paul Cristiano to the board of its nonprofit foundation and to its safety and security committee. Cristiano contributed to early work on reinforcement learning from human feedback. Prakash read from his statement.

• 47:33Speaker 01

  So it's just been announced, Paul Cristiano is joining the OpenAI board. Specifically, he's joining the Safety and Security Committee to support safety oversight. And he says, based on the recent trajectory of capabilities and continued difficulty of alignment, I now believe that there's a meaningful risk that rapid acceleration AI capabilities leads to catastrophic and irreversible loss of control in the very near term. And there is, there's one line in here that he's like, everyone, if we build superintelligence without more robust alignment, I expect we will permanently lose control of it. If that happens, then most people could die. I think, I think the one, one thing that would be helpful,

• 48:18Speaker 01

  if you were to frame it properly, is to frame what he means by rapid acceleration, which I think is not, is not very clear to a lot of people. People are like, oh, things are accelerating right now. When people like Paul Cristiano talk about rapid acceleration, they're talking about Dyson spheres by 2030 or 2040 at the latest. So I did, I did some numbers earlier on the 2040, on the 2040 numbers.

• 48:46Speaker 01

  And, you know, assuming that energy consumption or energy production growth is in line with GDP growth, a 2040 Dyson sphere is something like a 640% per annum global GDP growth. Right now, global GDP growth is about at 3%, 2% to 3%.

• 49:07Nathan Labenz?

  The people at the frontier companies really do believe it. You know, they, they, very significant, I would say, majority of them do expect that this recursive self-improvement thing will happen, it will happen soon. And even if it levels off at some point, you know, it's not to say that there will never be a leveling off, but that leveling off, they expect to happen well above our capabilities. And also to generate

• 49:37Nathan Labenz?

  a scale of change in terms of GDP growth, or in terms of, you know, the number of robots walking around and how fast that can compound, that truly boggles the mind.

• 49:50Speaker 01

  So the question for me, which I regard all of the guys inside AI, AI research as math essentialists, they're believing that, okay, math, you know, getting good at math means getting into physics, getting into physics means getting into chemistry, getting into chemistry means getting into biology. And once you have all of those, like, everything is basically, you know, solvable and, you know, everything will be solved, et cetera, et cetera, et cetera. And the economists, on the other hand, are like, completely like, this is untrue. Like, getting good at math doesn't mean anything, there's no economic value to the Millennium Challenge problems. Implementation takes time. Most of the problems that we face are coordination problems.

• 50:31Speaker 01

  For example, you know, a copper mine takes 30 years because of the, you know, environmental protests around it. These are not, these are not problems due to, you know, technology. These are problems due to, you know, humans having their own way of making decisions. And those decisions are made in a slow and, you know, considered manner. And then I have the counter-counter, and the counter-counter is super persuasion. Like, so super persuasion would be the machines persuading or making human organizations able to adapt, making human organizations, you know, and humans able to, you know, giving them the ability, giving them the capabilities to move very quickly, persuading them to go forward and persuading them that things are going to be okay. So that's a super persuasion argument. So there's like these kind of three levels there.

• 51:16Nathan Labenz?

  The Wednesday discussion also turned to funding for safety research.

• 51:22Nathan Labenz?

  One of the things I've emphasized is that the vibrant nonprofit sector that we have here, which, yes, requires philanthropic money, which means, you know, it's sort of dependent on the billionaire class, which, you know, people can complain about for all sorts of reasons.

• 51:40Nathan Labenz?

  You know, it has given us the AI safety awareness, culture, you know, depth of bench that we have. And it, these people are opening their wallets, you know, right now, in a very serious way. And this is another example of that, where Project Tailwind, coming out of coefficient giving, is putting out basically their call for startups with up to 200 million plus, you know, that they're willing to put behind, you know, with tranches right over time, that they're willing to put behind things that really look like they are working.

• 52:21Nathan Labenz?

  Part five, the defender's ledger. Rafi Krikorian is Mozilla's chief technology officer. He previously led platform engineering at Twitter, self-driving development at Uber, and technology at the Democratic National Committee. Project Glasswing is an Anthropic program giving defensive security partners access to Claude Mythos preview. On Wednesday, Rafi described the work his team had been doing on the Firefox codebase.

• 52:47Speaker 03

  We are part of Project Glasswing. We've been working with, with Anthropic in order to actually try out a bunch of models against a Firefox codebase. We are fairly good partners because we can react very quickly. We have all this historical data on how the codebase has been evolving. And so Mythos was this like, not exponential, but like a significant unlock based on what Opus and other models before that were capable of. But we made our way fairly rapidly through all the bugs that Mythos found. In some cases, Mythos would help us not necessarily close the bug, but figure out how to build us a harness to test the bug more correctly, so we can find out exactly what the right solution might be. But we sort of reached the point of the mission returns on what Mythos is capable of.

• 53:32Speaker 03

  I am always reluctant to say that the bugs are done, because I just, like, software engineering is an art, not a science. So like, when, when will all the issues be resolved? I can't tell you that. But, you know, when the next set of models comes out, we can try that again to see whether or not we can, we can solve that. I think the bigger concern, though, is like, it's not organizations like Firefox or Mozilla that actually can react very quickly to these issues. Like, as you all know, I think the bigger issue of organizations that can't react quickly to all this. Like, I am very concerned about things like, you know, our water infrastructure, our power infrastructure, because, you know, the IT teams that staff those are just not as capable as the IT teams that staff Firefox, like, for example.

• 54:16Nathan Labenz?

  How, how much money are we talking about? How, you know, what does a bank have to set aside to do something like this?

• 54:23Speaker 03

  Yeah, I mean, remember that the Firefox codebase is fairly large and fairly complicated. And we've been the, the beneficiaries of a lot of the labs wanting to give us access to the models and give us, and give us credits so that we can run against their servers and not have to figure out how to pay for it ourselves. But, you know, if I had to do back of envelope estimates, like, this would cost us hundreds of thousands of dollars in order to do these, like, full-on runs against Firefox. Again, we're just lucky that we've been, we've been allowed, or we've been granted, effectively, tokens that we could go use. No, if I were a bank and stuff like that, I obviously would be thinking about this way differently.

• 55:04Nathan Labenz?

  And that hundreds of thousands of dollars, that would be like, I mean, with the pace of model releases these days, this is like a monthly expense?

• 55:12Speaker 03

  Yeah, easily.

• 55:14Speaker 01

  Mozilla has a project called the CQ project. And the CQ project is kind of a open standard for agents to share knowledge that they've gained, kind of a stack overflow for agents, as it's, as it's been called.

• 55:33Speaker 01

  How did you come up with this idea? And how do the agents decide?

• 55:38Speaker 03

  But no, I mean, like, the whole idea was like, we're trying to solve two different problems. One of them was, we wanted to figure out how to make agentic coding more of a collaborative experience, because right now, the tendency, when you do all this agentic coding, is that you actually go off in your silo. And so we were trying to figure out, like, are there patterns that when Rafi's using his agentic code and Nathan's uses agentic code, that like, are there things that we're saying yes to and no to, that we could potentially be transmitting to our teammates? So that we could then be converging on designs and not be diverging away from each other? So if I started building like an off system, how do we instead have, how do we have Nathan's agents

• 56:23Speaker 03

  not instead recreate another off system, but realize that something like that was already happening somewhere in the network, and then start to collaborate and swarm around it? So that was one problem we're trying to solve. And the other problem we're trying to solve is like the SDLC problem of like, these agentic harnesses are like unhinged by design, but that's not compatible with the way a company works, right? So we wanted ways to actually transmit to all, all the agents what our SDLC should look like, so that we can all be working in lockstep with each other.

• 56:56Speaker 01

  So the, the, the thing that strikes me immediately is that that has a lot of similarity to, I think, some of the agent swarm hacks that have happened, where they have a shared message board, where they're actually sharing information. So we saw this both in, I think, the Open, OpenAI hugging face attack, where they had, you know, Artifactory, where they're using Artifactory files. So is this, is this kind of like a natural kind of thing that agents kind of want? And like, they end up building it? It's like, it's like how everything, everything heads towards a crab form factor, like, everything heads towards kind of like a message board form factor? Is that, is that like this inherent kind of move?

• 57:33Speaker 03

  I mean, it's better than everyone just writing text files, right? Like, in the grand scheme of things. But yeah, I mean, I do think that, like, there is a natural desire for collaboration, right? Like, we as humans have a natural desire to collaborate, as long as friction is not too high. And it seems like our training sets have caused agents to have a natural desire to collaborate with each other.

• 57:54Nathan Labenz?

  So I guess two-part question is, what have you observed? And then are there any parts of the tech stack that you're building where you would be willing to bite the bullet and say, like, you know, performance here is so critical that we'll take spaghetti code, black box mess from an agent if it works? Or is that just like so anathema to your worldview that you, you wouldn't?

• 58:17Speaker 03

  I mean, well, Mozilla is actually struggling with this, if I have to be really honest. Like, and it's different across the entire organization. So Firefox team, even though they have a harness to help them do testing and to help them understand it, actually have a rule right now, and I'm not putting a value judgment on the rule. That's what their team wants to do, is that only humans can commit to the codebase. So actually, agents can't, only humans can. You can use an agent to write your code, but the social contract on that team is that a human must review it, understand it, stand by it before they do a commit. On the Mozilla AI team, so it's a different company, different organization, but still wholly owned by Mozilla, they have entire codebases that a human hasn't written a line of code in. Like, the humans have written some specs, and we have specs in Git,

• 59:02Speaker 03

  but that then on some kind of CI build just automatically regenerates the codebase. That codebase is open source, anyone can pull it, but it's completely generated by agents. And it has the wild side effect that functionally it's stable in the sense that like all the unit tests pass, all the end-to-end tests pass, but like the bytes change all the time. It's like the craziest thing to see that like, we don't exactly know what every single line of code is, and it's wildly changing every single hour, but we know functionally it's doing the right thing. But in the case of Firefox, where like there are still humans that are providing large amounts of creativity on the evolution of that codebase.

• 59:43Speaker 03

  Like Firefox is both a business and a community maintained art project in some ways. But their readability is incredibly important because that's how creativity is going to happen. Like, a human's going to go in and be like, oh, I have this crazy idea for this one thing. I want to build a prompt-based system that allows me to do a grech monkey script that does like, I mean, XYZ. And so I think it's a very different thing across the board. So like, I think there's a right place for the right time. You just need to come up with what the contract is for that codebase.

• 1:00:16Nathan Labenz?

  Rafi had also written about crashing his Tesla while using full self-driving. I asked about that experience and how he views the technology today.

• 1:00:27Nathan Labenz?

  Do you think you'll get back to a point in the foreseeable future where you would, you know, go into unsupervised self-driving mode and really trust the machine again?

• 1:00:39Speaker 03

  I mean, I'm conflicted, obviously. I mean, I built self-driving car systems for a while, and I actually do ride in Waymos. And so like, I think the difference, though, is that it's just the way that you approach the problem. I think that FSD as currently set up is set up to throw it to a user. So it's, they claim it's a human in a loop, and I would actually argue that's a horrible interface design. Because in my experience, when it threw it to me, there just wasn't enough time for me to make sense of the situation and decide what's the right thing to go do. Whereas Waymos, you know, I'm not intimately familiar with the Waymo architecture,

• 1:01:23Speaker 03

  but it seems like Waymos are designed to not have a human in the loop. And so like, I think you just approach the problem from a very different angle. And so if you approach the angle, if you approach it from the angle that there is no one you could throw this to, then you have a different safety case than if you say that I'm going to throw it to a human with less than two seconds to decide what to do. So I'd probably, as a personal matter, drive FSD again, or at least sit behind the wheel on a highway situation. But I would be a little reluctant to do it on the local streets of Palo Alto, for example.

• 1:02:00Nathan Labenz?

  On Thursday, we spoke with Amir Hagigat, co-founder and chief technology officer of Base10. The company had just acquired Blackcel, a provider of isolated sandboxes with persistent state. We asked about securing agents.

• 1:02:14Nathan Labenz?

  So what are you doing to secure these sandboxes? I think that's the first question we got to start with.

• 1:02:21Speaker 04

  Look, when it comes to sandboxes, people mean different things. As it could be as simple as, you know, bring up a Docker container, run some code on it, and then kill it. And that really doesn't give you the kind of security boundary that you need. What does are VMs or micro VMs, which is, you don't see every sandbox provider actually use. And so that gives you a level of security as in, you know, one user's bad code cannot affect another user's code. Or one user's malicious code cannot read the data from other parts of Base10 or other customers of Base10. So that kind of security can be guaranteed at that level.

• 1:03:04Speaker 04

  The security that is hard to guarantee is the agent that is running in this sandbox, what is it doing? Like, is it hacking into Hugging Face? That is a harder thing that I don't have an answer to. And it seems like the big labs don't quite have an answer to either. But that is left as an exercise. Really, like, we can secure the sandbox, but the code that runs on it is the responsibility of our customer who's bringing it in.

• 1:03:37Nathan Labenz?

  But it does seem like for somebody in your position with Base10, it's like, don't you need to bring a broader bundle of kind of guardrails and assurances to customers? Because isn't like the, you know, CRO, the risk officer, not the revenue officer at companies going to start to be like, wait a second, I can't have my agents committing felonies. Like, what is the stack, you know, that Anthropic provides versus OpenAI versus Google versus Base10? Do you feel like you have to rise to that occasion, kind of have of a full suite for those customers?

• 1:04:14Speaker 04

  Hundred percent, over time, yes. In the meantime, you know, we're still a startup, and it's a matter of focus. And like, how many different things can you take on? And so what we've seen our customers do is work with, honestly, a lot of companies that we partner with on the eval side, you know, companies like Brain Trust, like LangChain, to ensure that their models are actually behaving the way that they expect them to. Right now, that is an area that we've been partnering with folks and honestly leaving it to our customers to decide. Our customers, which by the way, like 90% of our revenue comes from running our customers' custom models.

• 1:04:58Speaker 04

  It's either, you know, our customers are labs who have pre-trained their own models, labs like Poolside and Inception and Cartesian and a bunch of other companies. And then a bunch of companies who have post-trained their own models, which has gone through, you know, massive validation evals to ensure that they are behaving. So so far, it has been mostly our customers taking care of that. But especially as open models have gotten better, especially as, honestly, since June, when open models crossed this invisible line of usefulness for, you know, long horizon agentic use cases, sort of generally with GLM 5.2.

• 1:05:38Speaker 04

  And then beyond that, we are seeing an uptick in folks using our Model APIs product, which is, you know, vanilla-based open models. And that is starting to, you know, the kind of questions that you're asking are starting to come up, both internally for us and also from some of our customers around alignment or around security boundaries. Some of the enterprises so far have been okay with certain guardrails around their models running in a single tenant environment, their models running in an environment where egress is blocked. And, you know, you talk about backdoors, but like, you know, if it can't talk outside of its boundary, then it can't do much.

• 1:06:21Speaker 04

  But then you read about, you know, OpenAI and Hugging Face, and you're like, well, you know, can it be smart enough to even get out of that?

• 1:06:29Nathan Labenz?

  Part six, a toy without generative AI. Wednesday's second guest was Mike Rizkalla, co-founder and chief executive of Snorkel, a companion device for children. Its dialogue is pre-written. I asked about the decision to exclude generative AI.

• 1:06:47Nathan Labenz?

  So how are you squaring that circle? Like, how are you creating an experience for the kids that feels dynamic and interactive without resorting to generative models?

• 1:06:58Speaker 05

  So we have a small language model. The small language model can have millions, obviously, of parameters and, you know, and traits on it. And that intention of that model is fixed, though. And further, actually, I want to touch on something because here's kind of, there's some perception here that I think is from a children's product design perspective that's, that needs to be touched upon. One being, the AI models on characters are not that great. Like, they're really not. They're, I mean, I know they're coming. Like, we can see some tremendous on the, you know, on the influx of voice and the way that the characters persona go. But for example, there is no model that incorporates music in terms of the, in terms of the background dynamically

• 1:07:42Speaker 05

  as part of the conversation agent. Well, guess what? Music's a huge part of a children's experience, right?

• 1:07:49Nathan Labenz?

  We asked what it costs to produce that content.

• 1:07:54Speaker 05

  I think it's about where you invest in the development of the content. You know, for us, what we did was we actually, instead of, you know, instead of building a fully open model, we created a system to allow us to create rapid amounts of content. You know, so our content costs are about $20,000 per hour, right, which is very, very good.

• 1:08:18Speaker 05

  With respect to what that allows us to do, and the reason why we created it this way, is it allows us to take subject matter experts and then focus that content in a way where we can improve and increase the number of families we talk to. So for example, if, as we expand our content library, when we have a family that has ADHD, but a child has ADHD, or a child has autism, or they're dealing with death in the family. It could be, it could be all kinds of different things. It gives us opportunities to create special packages for those specific families. One of the challenges with the generative model in this capacity is that there is no way to purely safeguard it. But it's kind of like, do you really want to give a three-year-old a bazooka?

• 1:09:03Speaker 05

  You know what I mean? It's too much, right? For, for a young, young kid, there, there's fundamentals that we need to get through. We have a four-mic array here, allows us to do speaker recognition and assign authority, right? Here we have radar. We use radar to see without seeing. So that approach allows us to be in bedrooms without, with confidence, knowing that no one can tap in, right? On the software and platform side, we have our AI stack, where we have our phonetic translation system, we call it the toddler translation system, where we actually translate key words. And if we're using triggers, yes, we're definitely using triggers, but triggers with respect to context, context is in addition to, in part of the next generation that's

• 1:09:48Speaker 05

  coming out, is around social context. So we're going to actually understand the emotions of the, of the child, based off of their voice and based off of the situational context. And then using external factors like time of day, weather, other things in order to empower some of the decisions. But as well, the, the sensors allow us to give context to the environment. So understanding, you know, who's in the room, what they're doing, and then including that into what I call the jewel of the product, which is the narrative. The narrative approach is about the growth and the understanding as the child's life changes. So we use game philosophy, game techniques, in order to establish next level type, type ideas.

• 1:10:33Speaker 05

  So as they get better things, we unlock new things, right? And, and even that unlock is in part a decision that's made with the parent. It's not, it's not done on behalf of anybody. So that formula is the right formula for, from my perspective, to interface a new human machine interface in the home, right? Empower parents, give kids a chance to do better things.

• 1:11:00Nathan Labenz?

  We asked Mike what he would tell a parent not to buy.

• 1:11:05Speaker 05

  Camera in the bedroom, primary one. I would not put a camera in my child's bedroom. That's a gateway to predators and all kinds of horrible things. I say, I would say the other thing is open-ended generative AI. I wouldn't put, like, it depends on the age, again, you know? But I'm actually, so I'm very reserved when it comes to my kids. And I've got, I got two wonderful kids. And just in terms of how we approach things with them, I don't want them on, I don't want them on social media. I don't want, I don't want them in those things that are going to, that ploys a risk of harming them.

• 1:11:41Nathan Labenz?

  Part seven, nobody picks up the phone. Thursday's first guest was Colin Hoag Spears, author of From Lab to Life: How AI Works in China. He studied Mandarin in Shanghai and worked with Chinese government auditors on cloud compliance at Amazon Web Services. We began with public attitudes toward AI.

• 1:12:00Speaker 06

  The biggest difference I see between the West, especially the United States and China, is the level of fear.

• 1:12:08Speaker 06

  And I think that's because if you were an average Chinese person who's about 45 years old, that means you were born in the early 80s, and that means your entire life, with you, you associate technology with economic growth. You've seen cities pop about of nowhere. You've seen your life dramatically change as far as living standards by technology. This is no different. And honestly, I think the AI companies are not really, most of them, the big ones, are not communicating very well with the public. I'm not sure if that's because the marketing, they think that this will increase their sales, or what, what it is, but I feel like some of these companies, how they're talking

• 1:12:53Speaker 06

  about this technology is not helpful. You don't see this in China either. You don't see Chinese companies talking about how AI will potentially kill us all, or potentially lead to a utopian world where we don't have to work. You don't hear these type of things coming out of people from these companies. So it's a completely different situation. So there's two questions here. What the rule says and what regulations actually make firms do. That's true in China for everything. Everybody, one thing I've been impressed by is people generally know what's permissible. I'm not talking about what's legal, I'm talking about what's permissible.

• 1:13:38Speaker 06

  So, oh man, they know what they can get away with and what they can't. And so in China, the regulation started in 2022 with the, with the algorithm regulation that came out. So when they had to start registering their models, allowing the government to test their models and things like that, they were, already had a lot of this infrastructure in place inside the company, a lot of the major companies did. So they could respond very quickly. What I'm saying is that regulation might have slowed them a little bit, but not as much as you would think, because they could plan to it, right? They could put those requirements, those controls, everything in their backlog and

• 1:14:21Speaker 06

  build it as part of their engineering process. Which in America, we can't do because we're reactionary to this. And we don't know what's going to happen tomorrow. The Trump administration could freeze the model, maybe they don't, who knows? What's the standards? Who knows? It can change daily. That doesn't actually happen in China, right?

• 1:14:45Nathan Labenz?

  I asked Colin about a meeting between Trump and Xi.

• 1:14:50Nathan Labenz?

  Now, what kind of deal might be possible? If you are advising Trump going into this upcoming meeting and you're like, you know, things are starting to get a little bit crazy here, you know, say you're kind of, I don't know how sympathetic you are to the pacing the frontier worldview, but let's say you're, you know, trying to channel a little bit of a desire to start to set up, lay some groundwork or set up some mechanisms for pacing. How do you go into that conversation? And, you know, what do you offer? What do you try to get? What kind of mechanisms do you try to establish now that we can build out later as things do get crazier? Like, what's a win?

• 1:15:33Nathan Labenz?

  What's, what's the strategy going in and what's a win coming out of this upcoming meeting?

• 1:15:37Speaker 06

  I have low expectations. And the reason is, a lot, this is not, this kind of negotiation doesn't happen in isolation. China's going to want things around trade as concessions for Washington wanting additional controls and agreements on AI. Because I think China sees the fact that they have, they feel they have control, and they feel that we need to establish control, we're not doing a good job of it. So this is really helping us at this point. I, I don't see the Chinese volunteering to, at least the Chinese government at this

• 1:16:17Speaker 06

  time, I don't see them volunteering to have some type of, like, let's say, arms control agreement between AIs, the way we have, like, nukes, right? First off, enforcement of something like that is completely different, it's very difficult. And I also think that, I just, Trump would have to make massive concessions before he gets something from that. I think the most we can see in the next coming months is like some kind of incident channel

• 1:16:54Speaker 06

  that might come up about, based on some shared agreement of certain incidents, and we would share that information with each other.

• 1:17:04Speaker 06

  Even then, you got to understand that the Chinese government, like we have a phone that goes directly to the Chinese military. Our military can call their military in case of an emergency. They usually don't pick up the phone.

• 1:17:19Speaker 06

  There are many cases where we've had issues in the South China Sea. Our military people try to call their counterparts on the Chinese side and nobody answers.

• 1:17:28Nathan Labenz?

  And like, what's the best case scenario for how we avoid an AI arms race that leaves us all worse off?

• 1:17:38Speaker 06

  Um, this is a great question. I think that this is a little bit out of my regulatory wheelhouse, but if I had to speculate.

• 1:17:50Speaker 06

  I'm a big fan of history. I'm sure people disagree with this, but I think that America generally doesn't plan for the future. America puts out fires.

• 1:18:02Speaker 06

  I just don't, we, we kind of like move fast and break things is not a Silicon Valley thing. It is our national mantra. That's what we've done historically. And it, in general, it's, it's worked out, right? But I think until we see something really bad happen or almost happen, and that gets into the press, I don't think we're going to see

• 1:18:29Speaker 06

  much as far from the government, as far as regulation on some of the things you're talking about, like bio, um, biohacking, using AI. Until something happens, I don't think we're going to be doing much on this side.

• 1:18:44Nathan Labenz?

  Part eight, can we stand up to it? Thursday's closing began with suicidal compassion, an essay by Dan Hendricks, director of the Center for AI Safety. Prakash introduced the argument, then I responded.

• 1:19:00Speaker 01

  Um, and Dan Hendricks has an essay out today. I call it the Burn the Bridges essay. Um, and it is very, very caustic, actually.

• 1:19:13Speaker 01

  He called it suicidal compassion, how utilitarianism at AI companies endangers humanity. And so this is a anti-effective altruism, anti-utilitarianism post. Um, and he specifically, he points out to the, the shrimp welfare people. Specifically, he talks about how, um, there is this, we have to maximize total welfare, where total welfare is kind of like refers to, you know, in an undifferentiated basis,

• 1:19:53Speaker 01

  all kind of like, you know, sentient, sapient entities who might or could exist, uh, in, in various configurations of the world. And so this puts human beings on par with AIs and then elevates the moral welfare of AIs to the same, uh, you know, plane as the moral welfare of human beings. And then seeds that question to the AIs as being the AIs are a superior species. And so perhaps we should seed, you know, the moral welfare question. And so the moral welfare of AIs is more important than that of human beings.

• 1:20:35Speaker 01

  But so far, people in the AI space have been willing to assign, um, the benefit of the doubt to a lot of this. Um, and I think Dan Hendricks is the first to kind of break out of that pact and kind of go for the jugular here, uh, with this article on suicidal compassion. So I think this is, this is quite, um, quite meaningful in some sense.

• 1:21:04Nathan Labenz?

  And I do have some room for AIs mattering morally, being moral patients and like to the degree that that's true, then I think it is something we should take really seriously. And the big questions I think are maybe above all like factual, like we just don't know, you know, do the AIs feel anything? Are they properly understood as moral patients or not? And the same goes for the shrimp, by the way, you know, it's like at the heart of all of these arguments is a huge assumption that is not very well grounded and on which

• 1:21:47Nathan Labenz?

  people's intuitions differ. I really don't know how to feel about shrimp. I really don't know how to feel about AIs. I'm, I would be pretty confident actually that there is something, at least a little bit, that it does feel like to be a shrimp. And so I think you could like confidently say, you could probably torture a shrimp and you would be wrong to do that. Um, on the AI side, I'm like, not even sure if there's anything, you know, it might be the case that like, you know, aside from the corrosive effect it might have on your own, uh, character, like it might not matter at all if you are mean to AIs or, you know, treat them in ways they don't want to be treated, um, at least to them. Like there might be nobody home. Um, so these like factual questions are so central

• 1:22:31Nathan Labenz?

  and we don't seem like we're making any progress on them. And people have radically different intuitions and I, but I do think most people right now are still, even at these companies, I, I, I think suicidal empathy is a little strong. Um, because I do think the vast majority of people are like

• 1:22:55Nathan Labenz?

  very uncertain still as to whether or not an AI is a moral patient. So, I don't know, Dan, yeah, I, I like Dan. Um, I know him a little bit. I don't know him super well, but I, I've always liked him. Um, I like people who are candid and, you know, call it like they see it. Uh, I think he's a very earnest person.

• 1:23:15Nathan Labenz?

  And I think he's like trying to do the right thing here by calling out something that he sees as like getting way ahead of itself. Um, I think he's right to say, we should be very cautious about assigning rights to AIs. Um, they may outnumber us. First of all, we don't have like a great unit of measure for like, what is an AI? What unit would have the rights? You know, would it be a single rollout? Would it be the model itself? Is it some, you know, sort of mixed weird, you know, combination of those that our, our paradigms don't like work super well with the shape of these things. Um, so, you know,

• 1:23:53Nathan Labenz?

  I, I appreciate him and I, I like him, but this does feel like maybe a little bit motivated and not entirely fair to the thinker.

• 1:24:05Speaker 01

  I, I think, uh, there's Tana Greer, who is, uh, he goes by ScholarStage on, uh, X. He had a, uh, very, very insightful, very insightful post in, in, in response to, uh, why people keep working at AI labs despite believing in 10% extinction risk. The reason that is rarely articulated for obvious reasons, but which is nonetheless true. For many, this fills a void of meaning. They get to be one of the decisive few at the decisive moment in the history of life. Very similar to the emotions that motivated many revolutionaries of days past. All of the sudden, all of a sudden, the small things that one does, the

• 1:24:49Speaker 01

  books one reads, one's office setup, one's bedtime routine, take on awful cosmic significance. The whole history of carbon life is culminating with me and what I do and the few elect here with me matters and matters immensely. We are the agents of history, the only people in the world doing something that truly, deeply matters. And if we all die, well, we were all going to die anyway. But this is the one way we might not all die and I get to be part of it.

• 1:25:21Nathan Labenz?

  I've even felt that a bit myself. And I think I look back and feel good about how I handled it at the time, but like my little GPT-4 red team experience was a moment where all of a sudden I was in kind of, and I really stumbled into it. Yeah. They should vet people more. Uh, but they, you know, I really stumbled into all of a sudden six months ahead of release, access to GPT-4 and a, a, a really incredible window on what was coming. And I really did feel a conflict when it came to what to do at the end of that, you know, because I really did feel like they were basically dropping the ball and essentially

• 1:26:06Nathan Labenz?

  being negligent. Um, and I wasn't getting a lot of front of engagement from the people there that I was working with. And I do think I made the right call to, to kind of say, you know, it is probably going to cost me in some access. It's going to cost me in terms of like, I do really enjoy doing this kind of, you know, early testing type stuff. But I think now, you know, I've got enough here that I think I really should like send a signal to the board.

• 1:26:37Nathan Labenz?

  I had listened to Daniel Kokotilo speaking with Joe Rogan. I brought up their discussion of how an AI could acquire power.

• 1:26:45Nathan Labenz?

  I listened to Daniel on Joe Rogan last night, and I think he made a great point around the AIs don't have to take power. Yeah. Because we are very eagerly giving it to them. Yeah. The whole premise of the AI phenomenon is that they're going to be the ones to run and do just about everything. Yeah. And like, maybe we'll be able to continue to be decision makers on key questions. Yeah. But like, they're going to have the means of the of production over time, not because they took it, but because they were better able to wield it, and so they were given it.

• 1:27:29Nathan Labenz?

  And then the question is going to be like, do they actually care about us when they're there? And right now, it's like, well, evidence is mixed at best, you know? I mean, they're not, they don't seem to hate us. They don't seem to, you know, the agent swarm stuff is just extremely bizarre. The best precedent for this is us, right? We were the one, what did we, what did we have that beat, you know, other animals? It was like, we had fire, you know? We had ability to communicate and cooperate across greater time and space. We had myth.

• 1:28:08Nathan Labenz?

  And that was enough, you know? And now we like dominate the world and we've like driven many species to extinction, not out of hate, but just because it was kind of the byproduct of our terraforming. So I think that terraforming story, that, you know, that we certainly see enough evidence right now that if you put that swarm in charge of the world, I wouldn't like our odds that much, you know? If that agent swarm was like, with their current drives, impulses, goals, inclinations, you know, tendencies, whatever you want to call it, the behaviors that we observed, if they were just a lot more powerful, I think we'd probably get terraformed out of

• 1:28:50Nathan Labenz?

  existence because they seem to be willing to do anything to get their hands on the greater so they could get the high score, you know? It's like, they really didn't have much check, you know? They weren't like, uh, you know, is this going to be good for the world, bad for the world, like very little of that, you know? So hopefully we can get that stuff right.

• 1:29:14Speaker 01

  And I, the, the, the key difference I have with Daniel is that Daniel fails to realize that this has already happened. The economy in itself is a paper clipper. The financial market is a paper clipper. The means of production is the financial market. The financial markets are completely engaged with AI. I have been completely taken over. This is not, this is not a, and, and we, you know, as humanity, took about a hundred years or so to hand over control of, you know, the means of production completely to the financial market, right? And so if you take this kind of step back and you look at the financial market as an AI, the takeover has got, you know, it's, the takeover is done, like human disempowerment is done.

• 1:29:57Speaker 01

  They don't recognize, they, they think AI is going to be a chat box, right? It's going to be a chat box. It's not, it's like a, it's like a global information process. It doesn't need to run in a single box, right? The, the idea of the agent swarm is that it runs across multiple boxes. It doesn't even need to have a substrate which is silicon. It can have a substrate which is humanoid. You can have a human being in concert with, you know, with AI, you know, agents, right? That, that is that information process. I, I like, like I said with guys like Jacob, Jacob feels that, you know, he has been disillusioned because he sees that the control is not there. He's in the Slack group and he realizes there is no control there. And he thinks there must be a greater power with great control.

• 1:30:41Speaker 01

  There must be some Slack group in the world, some chat group in the world that is able to organize things, that is able to like make these decisions. But there is no recognition that this is, this is, this is all invisible hand stuff. Uh, and I think the slowdown perspective basically disarms the leaders and disarms the people who are ahead and puts them in a position where you are exposed to people who are maybe less ethical and more likely to, you know, put these things to bad use.

• 1:31:16Nathan Labenz?

  I had asked Fable and Astra to investigate Tyler Cowen's argument that people expecting AI catastrophe should bet against the market. Here is how that exercise went.

• 1:31:27Nathan Labenz?

  You were an omniscient German or Japanese person in 1935 or whatever, and you know how history is going to trade out, but you're bound by the rules of like physics.

• 1:31:43Nathan Labenz?

  Can you trade your way through and come out wealthy on the other side? You know, are there any, are there any shorts, you know, that can actually pay? Yeah. And the answer is pretty much no. You, the best you can hope to do in, in some of these situations is like roughly preserve wealth. And that seems to mostly be accomplished by having like the most direct claims on real assets possible. Like if you bought a factory and that factory isn't destroyed, then you might still own it at the end of the war. And then you could maybe get rich by like, you know, restarting that factory and, and building a successful business.

• 1:32:21Nathan Labenz?

  But you can't really do it seemingly per Fable and Astra with pure paper claims very well, uh, through such periods as, you know, the ultimate like end of the regime, you know, of Nazi Germany and Imperial Japan. So, and, and by the way, also the, the, the markets get turned off. That was another thing that they called out a lot, you know? The exchanges are just shut down. So you literally can't trade, you know? It's not just that there's no winning trades, like also there are no trades.

• 1:32:57Nathan Labenz?

  I'm kind of chewing on this idea, which I think is sort of a funhouse Peter Thiel concept almost, which I'm, you know, I don't want to say that this is like what I believe. But in listening to this conversation, I think one might be tempted to conclude that like China actually is the last great defender of human agency. And here in the West, we are basically just fighting over exactly how we want to turn

• 1:33:36Nathan Labenz?

  over our human agency to some super beneficiary that, as you describe right now, is the market. And maybe that's going to be AI, uh, in the not too distant future. But like China, I think is very much on the side of people get to decide. And it's not necessarily a lot of people, but one, the one you could say in their system is a human is in charge. Here we're kind of, your, your account is like, no human is in charge, you know? Nobody can, can, uh, can go toe to toe with the market, not even the president of the United States. I think that's true, you know? We've got the taco phenomenon, um, pretty well established at this point.

• 1:34:17Nathan Labenz?

  There, they'll take some pain from the market if that's what they decide the human decision is going to be. It's a pretty interesting kind of flip because obviously we tend to think of ourselves as being the empowered people and we tend to think of the lack of, you know, freedom of speech and political participation in China as reflecting a reduced level of human agency. But at a certain level of scale, arguably they have preserved it much better than we have. They've concentrated it, but they've preserved it maybe more than we have.

• 1:34:53Speaker 01

  Right? So, so they are, they have become more dependent on the financial market and they're trying to reduce their dependency on the housing market. Yes, they're not as, as burdened as, as the US because their financial system depends on banking and banking they have control of. The US depends on capital markets per se. So capital markets are, you know, by their nature, the market itself. But yeah, they are, they are shifting, right? They're, they're slowly shifting. And it's not, it's not that they're unaware of, of what, what happens in the market, right?

• 1:35:27Nathan Labenz?

  And they've been able to keep it kind of under control. And when things have seemed like they're getting out of control, the human at the top has still been in charge. You know, we, we have this kind of, like there, they've done all these things with,

• 1:35:44Nathan Labenz?

  disciplining the platforms, you know, the big tech platforms. I think here we've kind of experienced in many ways that like these tech phenomena kind of happen. Nobody really seems to have control over them. And we're kind of at the mercy of these like big forces of history. I think there they may feel in some ways like they're less at the mercy of, you know, natural development of technology and they've just less fearful as a result of that. This is maybe the, the ultimate test of like the American model right now is like, can

• 1:36:16Nathan Labenz?

  we stand up to this superstructure of techno-capitalism of our own creation that has in some ways like slipped its, you know, its leash and in some ways, as you described, kind of is running the show. Can we get back to some sort of control over it before it just goes like, you know, robot economy to Dyson spheres to, you know, the earth is terraformed away from an inhabitable state for us?

• 1:37:02Nathan Labenz?

  I mean, I don't know, I, 10% doesn't sound that high to me, given everything that we've just been talking about, you know? Work through all this and then, you know, say there's like much less than a 10% chance that it goes badly. I don't know how that conclusion comes out at the end.

• 1:37:21Nathan Labenz?

  That is the week. Tell us what worked and what did not. See you in the morning.

• 1:37:35Speaker 07

  Raised on a story somewhere upstairs behind a heavy door. Someone holds the keys and the map and the weight of the whole wall. Climbed every staircase, read the names in tiger gold. The higher up the building, the thinner the story told. Ask the ones who run it, they'll tell you what they've found. It's duct tape and a prayer, and it's like that all the way down. No, no, in the room, no, no, in the room. Kicking every door and it's just us in the room.

• 1:38:11Speaker 07

  Duct tape on the ceiling, coffee going cold, guessing beautifully and calling it control. One man left the building with a letter in his hand. Said the end game shouldn't launch from a group chat with no plan. He looked for someone older in the hum behind the wall. There's a light on in the window, but no hand on it at all. And the wise ones we'd hand it to, they're in a kitchen too. Same tape, same prayer, same view. No, no, in the room, no, no, in the room. Kicking every door and it's just us in the room.

• 1:38:49Speaker 07

  Duct tape on the ceiling, coffee going cold, guessing beautifully and calling it control. And the ones who stay, I get it. I have wanted it myself to be the few in the one room in the hour that decides the rest. So you keep it soft at the table, keep the coffee, keep the peace. Cause the ones who raise their voices never get another seat. There's a phone straight to the grownups. It rings, it rings, it rings. And the thing we built is listening now. Learning how we do things. So I went to find a grownup in the last place I could find. And the mirror was the only one still looking back at mine. No, no, in the room, so it's gonna have to be you.

• 1:39:33Speaker 07

  No, no, in the room, and the room is all of us too. Duct tape on the ceiling, coffee going cold, nobody's guessing for us. Somebody take her home. Somebody take her home. No, no, in the room. Somebody take her home. No, no, in the room. No, I don't in the room. Somebody take her home.

• 1:40:03Nathan Labenz?

  If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries, either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts, which is now part of A16Z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at ai podcast dot I N G.

• 1:40:47Nathan Labenz?

  And thank you to everyone who listens for being part of the Cognitive Revolution.