Zvi's Mic Works! Recursive Self-Improvement, Live Player Analysis, Anthropic vs DoW + More!

Zvi Mowshowitz returns to survey the current AI landscape, from recursive self-improvement and the shift from the “beginning” to the “middle” of the AI story to what true AI end-game would look like. He and Nathan dig into AI-driven job loss, real-world productivity impacts, and the ethics of trying to escape a “permanent underclass.” They assess today’s AI live players, why Anthropic may be slightly ahead, and whether Chinese, xAI, or Meta can catch up. The conversation closes with Anthropic’s Responsible Scaling Policy, p(doom), AI safety options, and how they each use AI in their own work. Sponsors: Tasklet: Tasklet: Build your own Cognitive Revolution monitoring agent in one click. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai VCX: VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com CHAPTERS: (00:00) About the Episode (02:25) Entering the middle game (09:08) AI layoffs and jobs (Part 1) (15:50) Sponsors: Tasklet | VCX (18:43) AI layoffs and jobs (Part 2) (18:44) AI growth and elites (27:00) Defining the AI endgame (36:09) Live players and laggards (45:38) China, compute, and distillation (56:03) Meta, Musk, and strategy (01:06:41) Google's faltering AI strategy (01:22:25) Anthropic's scaling policy shift (01:36:29) Anthropic and domestic surveillance (01:57:29) Courts, power, and Anthropic (02:18:50) Model fatigue and productivity (02:34:53) Alignment basins and doom (02:47:24) Slowing AI and activism (03:05:37) Forbidden techniques and choices (03:22:31) Episode Outro (03:26:31) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Nathan Labenz?

  Hello and welcome back to The Cognitive Revolution. Today, I'm excited to welcome Zvi Mowshowitz, author of the indispensable Substack, "Don't Worry About the Base," back for his record 12th appearance on the podcast. Whenever I get the chance to catch up with Zvi, I try to get his take on all of the most important recent developments in AI. And with so much going on, this episode stretches to more than three hours. We start with the critical question of recursive self-improvement. Zvi explains why he thinks that recent events mark a transition from the beginning to the middle of the AI story, as well as what he would need to see to feel that we've entered the AI endgame, namely that AIs begin driving AI advances to the point that human research

• 0:43Nathan Labenz?

  talent no longer matters. From there, we discuss the rising narrative of AI-related job loss. I ask Zvi to estimate the productivity impact that AI is already having on the economy, and we discuss the bankrupt ethics of focusing one's energy on escaping the so-called "permanent underclass," which we both see as flagrant defection and Zvi colorfully argues, won't work anyway. After that, we consider the AI live players. The list, we agree, seems to have shrunk to just three companies, with Anthropic probably slightly leading, OpenAI still neck and neck, and Google, in Zvi's mind, most at risk of falling out of the top tier. Zvi also explains why he thinks that Chinese companies won't soon catch up, even if they

• 1:28Nathan Labenz?

  do get an influx of compute, and explores what XAI and Meta might possibly do to get back into the race. From there, we dig into Anthropic's recent update to their responsible scaling policy, consider their conflict with the Department of War, and we get Zvi's take on the efficacy of the constitutional approach, and whether it's realistic to expect that a powerful AI could be robustly good. Toward the end, we check in on his current P-Doom number, compare notes on how we're each using AI to boost our personal productivity, briefly debate the merits of Goodfire's intentional design research agenda, assess the AI safety community's currently available options, and I get some personal, financial, and professional advice.

• 2:11Nathan Labenz?

  For a mix of broad situational awareness and razor-sharp insight, there is arguably nobody better. And so I hope you enjoy this wide-ranging survey of the AI state of play with the one and only Zvi Masiewicz. Zvi Masiewicz, welcome back to the cognitive revolution.

• 2:28Zvi Mowshowitz?

  Good to be back. It's been a while.

• 2:30Nathan Labenz?

  It has. I've been busy and so has the rest of the world. And we've got no shortage of major events from the AI world to cover.

• 2:39Zvi Mowshowitz?

  Oh, boy.

• 2:40Nathan Labenz?

  I know you've been busy, too. Let's start with recursive self-improvement. I think if there are any historians around in the distant future, which could be as short as a few decades from now, to look back on this time and say what really mattered in early 2026, probably my best guess is that we are in the period where we're really starting to enter into a recursive self-improvement dynamic from which there may already be no return or that we may soon reach a point of no return. I feel like, subjectively, we kind of went from late, early. You know, it was like getting to the end of the beginning.

• 3:24Nathan Labenz?

  And now, suddenly, I feel like we're maybe in the beginning of the end. And I feel like somehow we missed the middle. But let's start with just your reflections and observations on where we are with respect to recursive self-improvement.

• 3:36Zvi Mowshowitz?

  Yeah, so we're in the beginnings of steadily increasing amounts of self-improvement. I would say this feels like the middle. I think the middle is real. I don't think you... I think that, like, the reason why people think of this as the end game is because they don't believe in the actual end game. Right? They have this belief that we're looking at an S-curve. They believe the models will be commoditized. They believe that intelligence will be commoditized. They believe that, you know, the future will look like the past, except with all of this cool, like, intelligence behind everything. But in the way that, like, Star Trek is basically just modern humanity, like, doing modern human things and talking about modern human issues,

• 4:21Zvi Mowshowitz?

  except with metaphors. And they don't really believe that everything will transform, that everything will change, because everything is not transforming yet. If I had to use the metaphor at the beginning and the end, I'd say this is the beginning of the middle game, right? You've got the US government starting to wake up and do crazy stuff. You've got the labs starting to pull away from each other, become importantly different, offer importantly recognizably different services, building on themselves in ways that are rockets to the moon in various different ways. But you've got frankly, like, humans stopped writing the code and, you know, you're seeing cycles get faster

• 5:02Zvi Mowshowitz?

  and faster. But you aren't seeing true transformational changes to the world. You aren't seeing the humans being legitimately out of control of the process. You aren't seeing humans out of the loop. And those are the types of things I would think would count before I would call it an endgame.

• 5:18Nathan Labenz?

  You mentioned the S-curve mental model. One of the tweets that has resonated in my, you know, kind of rung around my head for the last few weeks was from Rosie Campbell, who used to work at OpenAI and is now, I think, working on issues related to AI welfare, sentience, consciousness, et cetera. She posted something to the effect of, "The S-curve can stay steep longer than you can stay relevant." And I wanted to just kind of dig in on the S-curve versus exponential for a second to see, like, I guess my mental model is an S-curve. Does it matter if there's a difference between long-term and exponential or an S-curve? Like, if the S-curve plateau is high enough, my kind of mental model is, yeah, it probably

• 6:03Nathan Labenz?

  is an S-curve, but I don't think that really does.

• 6:06Zvi Mowshowitz?

  Definitely S-curve, unless our model of physics is very wrong, because there's a limited amount of mass energy as we understand in the universe, and it can neither be created nor destroyed. And that means there's a certain amount of potential energy and a certain amount of potential utility and a certain amount of potential intelligence that the universe, as we know it, can't contain. So unless our model of physics isn't wrong, which is, who knows, not my area, this will cap, we have some very, very strong beliefs about, like, just to think they're theoretically impossible. You can't exceed the speed of light. You can't extract more than a certain amount of energy from a given amount of matter. You can't do a certain, more than a certain amount of work, and therefore, the amount of theoretical intelligence you can get from that, the amount of utility you can get from that by any definition, is limited. So.

• 6:48Zvi Mowshowitz?

  It's an S curve in some sense, right? But like, this is, you know, you're sitting around in the ancient Athens and saying, there's an S curve. There's only so much technology mankind can invent. And like, you're right. But it's not really relevant to your situation.

• 7:02Nathan Labenz?

  Yeah, there's a long way to go. Yeah, okay. I think that's very, I think that's an important point of clarification for many debates because people seem to really want to latch on to this S curve idea. And it really doesn't do that much for us in the end. I think that's like.

• 7:24Zvi Mowshowitz?

  I think people frankly desperately want to tell themselves a story and tell other people a story where it matters that they're staying for retirement. It matters that they're doing ordinary human things and planning for an ordinary human future where like, things won't change that much, where they don't have to go crazy, they don't have to look crazy at their friends and their family, where everything is going to be okay. Everything is going to be normal in a fundamental sense. And it is important to hold on to those things to prepare for that scenario and to stay sane. But they want the ability to just sort of push all of this aside, basically, and say, here's why AI is not that big a deal.

• 8:04Zvi Mowshowitz?

  Here's why AI will be only internet big, or not even internet big in some cases, although that's starting to fade away. And I understand why they feel the need for that. I understand why they latch on to that, but it's simply not looking like that's going to be the case. It's becoming increasingly unlikely that we're going to stay in that zone. And people have to come to grips with that. Yeah, the S curve at some point will hit one, but every day that we don't see that happening is one more day. And like, basically, every few months, like, come on a new study and they'll say like, oh, this proves that we're at the top of, and they're all wrong. And even if we were to hit the top of the S curve of fundamental capabilities, which

• 8:47Zvi Mowshowitz?

  is the curve that we care about, and we hit it with GPT 4.5, 5.4 and OPUS 4.6, and these are the best models we're going to get, and it's all iterative from here.

• 8:58Zvi Mowshowitz?

  Yeah, they're still vastly underestimating what's about to hit them even then. And five, five and four, seven are coming unless they're six and five.

• 9:08Nathan Labenz?

  One thing that many people would presumably

• 9:13Nathan Labenz?

  have to change their narrative on is if there really were a big displacement of human workers, right? If like we started to see major, you know, sustained layoffs, rising unemployment, et cetera, et cetera. It seems like that story has hit maybe again, the sort of beginning of a tipping point in the last few weeks.

• 9:39Nathan Labenz?

  How do you understand that right now? Again, we've got, of course, competing narratives, right? Like the CEOs themselves are saying, we're doing it because of AI and we're going to be more efficient and, you know, stock prices seem, at least in a couple notable cases, to have bumped on that communication. The counter narrative has been, well, you way over hired during COVID and kind of zero interest rate timeframe anyway. And so you're, you know, you have kind of an incentive to say it's about AI, but really you're just trying to undo previous mistakes. There's probably at least some truth to that as well. How do you parse the AI layoff story? And what are your expectations for the next, say?

• 10:14Zvi Mowshowitz?

  Quarter sort of like, it can always be a coincidence up to some point. But every month, every indicator keeps going in the same direction and things keep going farther. Excuses like there was dead weight to be gotten rid of become less plausible as we get into 2026 because like it's been several years. And we keep seeing statistics tell a consistent story that some of us were predicting the statistics would tell in advance and that the people who said, I don't remember people saying, oh, yes, you're absolutely going to see increases in productivity, decreases in employment, always announcements of job cuts due to AI, but it's not going to be real because of these explanations. I don't remember anybody making that prediction.

• 10:59Zvi Mowshowitz?

  These are people in hindsight saying, oh, if that's true, then it must be because of this. But that would be all credible if they had observed because theoretically that over hiring was already there, was already clear. And so we were talking about the fact that there was over hiring. I believe there was over hiring. But nobody then said, here's how this is going to play out that I can remember. And also, just like the statistics are just coming in consistently telling the same story over and over again, which is productivity up, GDP up, our GDP, not just nominal GDP, inflation held in check, employment down, employment's been revised down every month, over and over and over again. And of course, it's always confounded, right? You could say,

• 11:42Zvi Mowshowitz?

  well, it's the tariffs. You could say, well, it's the aftershock of COVID. You can say a number of things. But it seems like a really, really big coincidence to claim that all this is happening at the same time when like the stock market impact of the tariffs got basically fully erased, the tariffs have been reversed. It's obviously the last, now everything's confounded by Iran. But like up until the Iran conflict started, it seemed like we were getting a very consistent story that this kept happening. The job numbers kept getting revised down. The GDP numbers kept not being revised down particularly. Productivity kept going up. And like, everybody out there in practice keeps telling the same

• 12:25Zvi Mowshowitz?

  story. And we're on the street, at least, you know, when I talk to people who talk to people who are not involved in our world, is there is widespread on the ground, normal person paranoia, that if they lose their job, who knows when they'll get another one? In a wide variety of like, how our work? Because no one, not that many people are getting fired yet. Because it's really, really hard to replace a worker, it costs a lot to train a worker. You want to be conservative, you wait until you actually have, I've made it all the work. But everybody is like, who know, who is hiring? Right? Who wants to take on new workers and train new workers to these jobs when you could train an AI to do it instead, going forward? By the time I've spent my two years getting you to be

• 13:06Zvi Mowshowitz?

  productive, maybe I don't need you anymore. And it's a harbinger of the future. And this is decreased labor power. It's made everybody feel paranoid. It's made people fearful for the future. The kids are freaking out about exactly these problems. They don't know what to study. They don't know what to try and do. Like, it's a very galaxy-brained take that all of this is in your head. Right? Even if you think, again, this is the best the AI will ever be, and it's just a diffusion story from here, it's still a hell of a story. You say, "The job market impact is in your head, it's not real." Now, there's the not-galaxy-brained, standard economist's take, right, which is, yes, there'll be a transition period, which we're entering now, where a lot of existing jobs will get

• 13:48Zvi Mowshowitz?

  much more efficient, they'll get automated, they'll get augmented, they'll get eliminated, and we'll transition, but that's okay, because Jevon's paradox will have a lot more demand for things like software engineers and a few other stories. And then, of course, with our new wealth and productivity, we will find other jobs for people to do. There's plenty of jobs for people to do. Like, the Department of War might be hiring. But, um, there's, yeah, that's always been the case, right? Like, the people who say, "Well, you know, this automation of agriculture will kill our jobs." You know, they were right, their jobs would go away, they were wrong that unemployment would follow in the long run, because, of course, we went another thing. And the whole reason this time is different, right, that a lot of us believe this time

• 14:32Zvi Mowshowitz?

  is very different, is not that technology has never taken current jobs and made them largely obsolete, that's happened many times. The reason why we think this time is different is because the AI is going to do the new jobs that would get created as well, and it's going to happen quickly, and it's going to happen en masse, and therefore, you never exit the transition period, the humans don't develop new things, we don't necessarily think there's going to be enough tasks for all the humans that the AIs can't just replace. And this is going to create potentially a large number of people who, like, cannot retrain themselves into a new position and develop something to do that people value fast enough before it simply gets replaced over and over again by AI.

• 15:17Zvi Mowshowitz?

  And also, a lot of people are, like, putting a kind of resiliency in the ability to shift and adjust into people that people mostly just don't have. When these things happen over generations, it's much easier to deal with than when it happens over the course of years, or even months. That's just unprecedented in human history, and people will not react to it very well. And, again, all of this is, like, in a relatively milquetoast, normal world scenario, where all of this is happening. In the more advanced scenarios, you have much bigger things to worry about.

• 15:46Nathan Labenz?

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 15:51Nathan Labenz?

  Famously, Tyler Cowen said he thinks we can get a half a percentage point, I think, additional real GDP growth out of AI, and that would be amazing, in his opinion. What would you say, this might be too hard, because obviously, there's a ton of noise in the data, but based on what we've seen so far, productivity measures, et cetera, et cetera, if you had to put a point estimate on what we have got right now, what would that point estimate be?

• 16:22Zvi Mowshowitz?

  I haven't tried too hard to estimate it exactly, and I don't think anyone else really has either. I've seen various different attempts to guess. If I had to guess, we're currently enjoying half a percent to a percent.

• 16:31Nathan Labenz?

  Yeah, that's kind of what my gut says as well, and that's based on a very vibe-based, relatively superficial re- but-

• 16:41Zvi Mowshowitz?

  Like, this year, was there was a lot of downward pressure on what would normally be the American economy. You had a lot of fear, you had a lot of regime uncertainty in terms of the tariff regime, and like, various other policies. Usually, the types of things that happened in 2025 would be rather bad for business. Instead, things were good for business. And I think this is a lot of why things felt like they were okay for business, they weren't okay for labor, and certainly not in the health experience of labor. And, so yeah, I think it's like on the order of half a percent to a percent right now, but I think the market is pricing in at least that much indefinitely going forward, and likely somewhat more, even if it doesn't understand that's what it's doing.

• 17:25Zvi Mowshowitz?

  And the market has done well because it's anticipating the benefits of AI. I think if it wasn't anticipating that, you would have seen a very, very different set of reactions. And, you know, that's exactly why, when things started looking awkward for the American economy in other ways, I didn't sell anything, right? I just held on, because I knew that AI was gonna prop things up.

• 17:49Nathan Labenz?

  It does seem like there's been a bit of a split recently. I mean, people talk a lot about-

• 17:56Nathan Labenz?

  And I don't necessarily buy this frame, but there's the worry about being part of the permanent underclass or, you know, making your way into the, um, the upper class before things get locked in. I don't really buy that on a, or it certainly doesn't frame much of my thinking on an individual basis. It does seem like that dynamic might be coming to stocks, though, because we do see a like anthropic drops, uh, relatively in there, in the scheme of everything they're doing, minor product extension. And apparently pretty closely correlated to that, you'll see stocks drop. Are you buying that level of, you know, uh, stock permanent underclass?

• 18:39Zvi Mowshowitz?

  So I wanna, I wanna harken back to one of the most important movie scenes in history, which is Bane confronting his backer, right? And the person who hired him says, I'm in charge here. And Bane looks at him and says, do you feel in charge? And I think there is this very clear idea that people think, oh, if I have the rights written down in electronic databases, if I have the stock certificates, if I have the private property, then I get to be one of the special elite. Everyone else gets to be one of the underclass. I have to be one of the people in the elite, even though I will not also, I will also

• 19:22Zvi Mowshowitz?

  not be productive, even though I also will not be in the loop. I will also not be able to exert meaningful optimization pressure, except through my technical authority as the person who's marks in database. I think the idea of relying on marks in the database in this kind of world to keep you alive, to keep you meaningfully wealthy and able to consume physical goods, to have a good prosperous life for you and your descendants is hopium. Like, it's not a good plan. If you think that humans are sufficiently useless, that most of us end up in a permanent underclass because we cannot be economically productive, then Your best case scenario is you slowly lose your wealth to various different extraction

• 20:06Zvi Mowshowitz?

  methods. And the more likely scenario is all of that gets ignored by facts on the ground, by physical reality, just like rendering all of that irrelevant. Or the system gets taken over and subverted either by humans using AI or by AIs. The long history of the world, even with much less severe disruptions, does not have a good track record of private property surviving over long periods of time. When someone else has the guns, when someone else has the swords, when someone else has the power in various other senses, the meaningful power. If you don't have a reason to keep it, you don't keep it. Not really. And, you know, we've seen the collapse of the colonial era, we've seen the collapse of basically, you know, almost all the ruling regimes.

• 20:49Zvi Mowshowitz?

  We've seen many, many examples, even if you're not going to take the argument around AI seriously, just like, I can't believe you're counting on this. When people talk about permanent underclass and they talk about, oh, I'm going to have the skills to be productive of coding agents, so I'm going to like be a valuable person going forward. But I have this short window, it's like, why is there a short window? If humans can scale up and be productive in the future, you can scale up and be productive in the future. And if you can't, you can't. So like, there's no particular urgency there. You can make the argument that certainly this is like one of the last chances in some sense to like use your talents to, certainly there's this window where your talent can suddenly create a billion dollar company or even a trillion dollar company, and you would have a large portion of that, you can enjoy like a lot of money.

• 21:33Zvi Mowshowitz?

  I don't think it's particularly relevant. I think that in practical terms, there are two likely ways this plays out, if AI is for real and goes the way I expect it to go out. Right? One way is we lose control of the situation in various senses. Everyone dies, or there is like at least massive loss of control, massive loss of resources, massive destruction, massive disruption. In either case, none of this is going to be that relevant to you particularly. The other scenario is things go relatively well, and then there is basically abundance of real resources, and the humans are more in control. In that case, I think if you're a citizen of the United States...

• 22:19Zvi Mowshowitz?

  You don't really have to worry very much about your material needs. Like, okay, sure, you're a member of the permanent underclass. Congratulations. You have a real income that today would be called a million dollars. You have access to robots and intelligence that caters to anything you want. Your day is free. You don't really have to work. It's probably not exactly how it plays out, but like, basically, you're going to be better off, except in relative status terms. And you really shouldn't have to care about that. You need to get over the fact that you are not, like, relatively wealthy or relatively respected in that world. And there'll be, I'm sure, ways in that kind of world to compete for status within the human hierarchy. There'll be ways to meaningfully occupy your time.

• 23:03Zvi Mowshowitz?

  If we're still in control, we'll figure that stuff out. And then, I guess, there's a third scenario where, like, some cabal uses AI to take over, in which case, you need to be in that cabal, if you want to be part of the people who take over, and maybe influence it to become the good world instead of the bad world. But just making a bunch of money's not going to get you in. That's not how those worlds work. So again, mostly, you should be trying to make sure we don't get into the bad world, so we lose control, and then we get into the good world, so we retain control, and humans are in charge of steering and what happens. And mostly, I expect that, you know, most groups that, even if a small group has the ability to steer that world, they'll steer it in ways that, like, we're pretty happy with. Like, a lot of people talk about how Altman might try to make himself

• 23:46Zvi Mowshowitz?

  god emperor, or David S. Thomas might try to make himself god emperor, or Darius Amadai or whatever. And that's not my preferred outcome, to be clear. But I don't think that if Sam Altman made himself god emperor, that, like, I would be that sad about it in terms of my practical lived experiences. I think my life would be fine. I think my children would be fine.

• 24:08Nathan Labenz?

  Yeah, he does have a little bit of a Roman Caesar kind of vibe to him, where I think he does- Fancy that sort of, it seems that he does fancy that sort of power perhaps, but also that when he does things like fund universal basic income experiments out of pocket, I view that as like, honestly, probably a genuine, you know, magnanimity on his part that I would expect to probably extend into.

• 24:36Zvi Mowshowitz?

  It's very easy to be magnanimous when there's a true full abundance of resources, right? Like, he could, you know, in theory, keep 99% of the value of the light cone for himself, and the rest of us can be very, very happy with the rest. And I'm not saying this is my preferred outcome, to be clear. I do not want this to be how it plays out. I think this is bad. But, you know, that's still better than not building it, right? That's still better than various different horrible outcomes, especially everyone dying. But also, yeah, it's better than status quo, like, in an objective sense, except for your opportunity. So what I'm worried about is, in fact, losing control of the situation, right? And like, the reason why you don't want Tim out when making the decisions is

• 25:20Zvi Mowshowitz?

  not because he might take over, it's not primarily, right, in my view, it's because he might make decisions that cause us to lose control and for him to lose control because he is being irresponsible. And that is the reason I am terrified of him making these decisions, right? Like, it's not because I think he is like, an evil man. So like, it all comes down to normativity, in my mind, right? Normativity is the concept that good things are good, and bad things are bad, and you want good things to happen to people in general, not necessarily every person, you're allowed to think there are some bad people, but that good things should happen to good people and most people are good. If you believe that, then, you know, basically, things will work out, even if it's not exactly

• 26:03Zvi Mowshowitz?

  your preferred outcome. And, you know what? I think most people are normative. I think even the people in charge of the labs right now, most of them are normative. And to be certain that there are people, relative people who I don't think that way, I simply say nothing.

• 26:19Nathan Labenz?

  I will say, I do think it's pretty distasteful when I see people talking openly about trying to escape the permanent underclass. My reaction to that is always like, why don't you try to make things good for the permanent underclass rather than try to escape it? That just seems like really flagrant defection that I just can't.

• 26:38Zvi Mowshowitz?

  Flagrant defection to focus on getting a seat on the ark if you think the world is going to be flooded. Like, that is a terrible, terrible position. You should be trying to stop the flood. You should be trying to save more people and build another ship. If all you're trying to do is get one of those precious few seats on the ark, that's not a good thing to do. At the same time, there are times when, you know, the best thing you can do is just escape the bad regime. There are in fact times when that's like, you know, put your vest on first. Get out while the getting's good. Because what else can I do? And I respect

• 27:17Zvi Mowshowitz?

  that. But the way they talk about it, yes, it has a, the bad kind of elitism, like the kind of just complete disdain for the common man, indeed very disgusting. And I, I really don't like it. And, you know, yeah, I expect, you know, we can make the permanent, we can make the permanent underclass pretty neat. And in fact, most of the time, the world has had permanent underclass and it's, the permanent underclass has never had it better than it has it today.

• 27:46Nathan Labenz?

  Yeah, it's great. Important to remember that. What would, in your mind, and maybe we'll talk about this in terms of like, handicapping the timeline, and then maybe kind of just a little qualitative description of sort of what you would think would represent the transition from the middle to the end game. I've started to think, you know, of course, I think everybody, you know, who follows this feed knows the general range of like, Dario's still kind of on AI 2027, more or less. Demis is more like a 2030. OpenAI has

• 28:26Nathan Labenz?

  a March 2028 timeline for their fully human level automated AI research paradigm kicking in. I assume you're somewhere in that range in terms of, you know, expecting things to enter into what you would call late game. How would you call that transition? Or, you know, if there's a sort of, I guess, maybe we even should understand your view better on like, what is the difference between middle and late? Is it some sort of event horizon point of no return? Or is there some other concept that would separate those? How would you call it? What are you looking for? When do you think that is most likely to happen?

• 29:05Zvi Mowshowitz?

  The end game is like, I would say when, when it's the AIs that are largely running the show, like at least in the further development of AIs. Like right now, we're seeing AIs augment the humans, right? The humans are making the central decisions. The humans are reviewing the code. The humans are making the plans. And the AI is a multiple factor. It's something that you're supervising, it's something that is enabled. When that changes, things get very strange. So like, one of the key aspects of AI 2027, the tabletop scenario, is that mostly your

• 29:45Zvi Mowshowitz?

  progress is proportional to your compute allocation and your current place. Like you move up the timeline at a different, at a rate proportional to what percentage of the world's compute you have, because your researchers no longer matter very much, right? Because you're telling the AI, which is already effectively as smart or smarter than you are, to go build a better AI and to go align the AI. And like, you make decisions like what percentage of the compute should go to safety and keeping this thing steered properly versus increasing its capability. And for the early parts of it, you can assume that's going to be respected and then eventually you can't assume that's going to be respected.

• 30:26Zvi Mowshowitz?

  But at some point, like right now, I would say, you know, the big, the top labs have a dramatic talent advantage in these types of scenarios. And I think this is a reasonable hypothesis, although far from certain. At some point, your researcher talent doesn't matter very much because the AIs are your researcher talent. So all that matters is where you are on the curve and how much compute you have. Right now, Anthropic seems to have the best talent. And gets the most out of every given amount of compute they spent. And then you have like OpenAI and Google that have strong talent. They get a lot out of the compute they spend. And then what's going on with like XAI and Meta, right? Like they're throwing tons of compute at this problem and they're falling farther and farther behind.

• 31:11Zvi Mowshowitz?

  Or at least this is what appears to be true. And fundamentally speaking, I think that's the lack of talent. Right, that's their inability to execute as humans. And so the end game comes when it doesn't matter that much, which humans you have.

• 31:26Zvi Mowshowitz?

  Right, essentially. The human no longer provide, like right now you have like, you know, we're playing centaur chess. Right, like the human plus the coding agent is much superior to the best human on his own or her own. The AI agent on its own doesn't do anything. You need a centaur. The moment

• 31:46Zvi Mowshowitz?

  the human doesn't matter anymore. And like you transition to, okay, the human needs to be somebody who's like just able to like do some common sense stuff, isn't even one of the best players anymore. Now we're starting to talk about end game style scenarios. Also, yeah, when you approach the event horizon of like, you know, the release time to a meaningfully different model starts to go to like a month. Starts to go to like a week. Starts to be like, things get absurdly fast. These are the types of things we'll start to see. But like the ultimate like a Caspian scenario, right? It's like you just leave it on overnight and then like you wake up and things have happened. Right, and then like you know you're in the end game at that point. You also can think of the end game as when the world starts to actually transform. You start to see like the robot factories being built.

• 32:31Zvi Mowshowitz?

  You start to see large amounts of area being terraformed. You start to see like massive job disruptions. You start to see governments like start to do major interventions. You start to see this be like the issue, the thing on people's minds. And we just had one of the most important developments in the history of AI happen with the designation of anthropics and supply chain risk. And it wasn't even the most important thing for Department of War did that day. According to most people, right? Like you got completely subsumed by the exact same person who sent the tweet out doing something else that night.

• 33:08Zvi Mowshowitz?

  That has gotten like a hundred times more coverage. Because people think that's the important thing that happened. And I think it's very much remains to be seen what the important thing was that happened that day.

• 33:17Nathan Labenz?

  So I think I have to give you some base points when it comes to drawing the circle around who are the live players. I think for many conversations going back in time, the record will show you've pretty much always said it's those same three companies that you just listed are the real live players. I've always been grasping at who might we add and what might be the rationale for adding them. And I asked you this question a minute ago about, you know, kind of the permanent underclass of stocks. You kind of went in a different direction with it, but.

• 33:45Zvi Mowshowitz?

  Yeah, I just kind of got distracted, but like.

• 33:48Nathan Labenz?

  Yeah, all good. But I guess what I'm taking away from your analysis is like that permanent underclass of stocks might be almost all the stocks and it might even extend up to big tech, you know, like Microsoft, Amazon. Obviously, they have at least held their own slash done pretty well so far. But if you have a model of who can get into this next regime first that's kind of really mattering most, and only three companies right now seem to be well positioned to do that, then basically you're short everything else over a three-year timeframe. Is that accurate?

• 34:30Zvi Mowshowitz?

  So I think there's a lot of different parts of AI and you can make money doing a wide variety of different things. And the real world always takes longer than you think it does relative to the amount of technology. The diffusion is slower than the original idea. So I think there's still a lot of room for a lot of different groups to win. I think there'll be a lot of, oh, we're going to buy out people who have useful components, even if like eventually you wouldn't need them anymore because it's faster. I wouldn't expect everybody to go to zero. I think one thing, what's going on basically is that the stock market is forward thinking and they are trying to do some half-hazard forward thinking. They say, oh, okay, the software is a service business. How is a good business?

• 35:15Zvi Mowshowitz?

  But in five years, they won't have a business unless they reinvent themselves and create new products. But maybe they will invent themselves and create new products. But their original basis of valuation has kind of been destroyed. And yes, that will extend into a wide variety of other companies. I think a lot of companies, in fact, are going to be long-term losers, but it's always been true, right? If you look at the S and P and you have, you play this game of like, okay, let's take the 10 top performers for the next 10 years. And it's like, everybody else, everyone else doesn't do so well. Like, the gains are because some companies do really well, always have been. And everyone else on that kind of language shows. And so that's why you want to be diversified, because can you pick those 10 companies? And the answer is no, most people can't. But in this case, like, I think it's, you made some pretty good guesses, but you don't really know.

• 35:59Zvi Mowshowitz?

  Uh, but also keep in mind, the market's dumb about this. Like, I just been, my not in its words, the market is reacting on a very superficial level. People, if you're listening to this podcast, you have thought much more intelligently about the situation the market has. Am I just talking about me? And so, like, they announced that Claude was going to offer a Cobalt product, Cobalt product. IBM was down 10% in the late of that announcement. Because suddenly this stock market woke up and they said, oh my God, AI can write Cobalt code? AI can translate Cobalt code to regular code in programming languages people know? IBM's business is in trouble. And the rest of us were like, you didn't know? This is news?

• 36:44Zvi Mowshowitz?

  It's not like they built a slightly easier to use tool, changes things at all? And what has happened since then? IBM has fully recovered. Because it turns out that, yes, in fact, either it's been priced in, or they went back to not believing it, or something. But clearly, like, the market doesn't know what the hell is going on. You see this over and over again, the market is very slow to update on these things. They do secondary superficial reactions. The market has wrong way reacted in Nvidia many times to very clear news. Where demand for Nvidia's product is up and Nvidia's stock responds by going down. That is not how economics works, that is not how capitalism works. And yet, here we are.

• 37:29Zvi Mowshowitz?

  So, yeah, I, I would say there are some clearly good buys. They're not as clear as they were a year or two ago, or several years ago, when it was just completely utterly obvious what some of the good buys were. Because now the, the multipliers are in fact respecting a large amount of growth in those stocks. But it still seems pretty obvious that. Yeah, you were to buy a basket of the stocks that like seem clearly positioned to do well and then short the rest, it would be a very good strategy and expectation. And like your thesis would have to be very wrong. We can set that aside. And so on a lot of players, yeah, as I said, like, I think the talent has really proven to have migrated to the big three. And I think that like, there's a large and growing gap between three and four.

• 38:14Zvi Mowshowitz?

  So if I had to pick a potential fourth at this point, so like, either, so there's, there's three, there's a few possibilities. Obviously Meta or XAI could in fact have gotten their shit together and managed to find people who can execute. But I don't see any evidence of that. We just learned Meta postponed its next release. Again, there seems to be continuously reshuffles, trouble in the opposite of paradise. Things are not going well, as far as we know in Meta land. And XAI put out, was probably the most disappointing major model release of any major lab in the history of large language models in 4.2. And I see no sign of much happening.

• 38:55Zvi Mowshowitz?

  Like, they're not doing the things you would do if you wanted to recruit good AI capability or safety workers, right? They're actively dismissing, they just banned a safety team. They poo-pooed the idea that anyone could be actually in charge of safety. They said, well, safety is, Musk was like, safety is everyone's job. That's how it is at Tesla and SpaceX, which is not true. But Tesla and SpaceX have very dedicated teams, very dedicated safety people who make sure everything is safe, because anything else would be completely insane. But then Musk just says things. But like, Musk just doesn't understand, you can't just like run software engineers into the ground, ask them to have miserable life experiences, ask them to align to your personal

• 39:35Zvi Mowshowitz?

  preferences and whims all the time, and then hope to get the best talent? How exactly? Right? Like, why do these people want to work for you? You'd have to give them even bigger packages than Meta. And that's not going to happen, so he's not going to get the best talent. You know, Meta is, I'm going to spend the money, I'm going to spend infinite money, and this will get me the best talent, and it could just hasn't worked.

• 40:00Zvi Mowshowitz?

  And so they seem to be the bad addition. So for the Western side, that leaves just the big three. On the Chinese side, I think a lot of, there have been a lot of stories over the last few years about how this Chinese lab originally DeepSeek, but also, you know, here's Alibaba, here's Kimi, you know, here's whoever else. They're the ones who have the new hotness. They're the ones who are to catch up. And, you know, I've said before, you know, DeepSeek had some decent models, Kimi had some decent models, some other people had some decent models, but nothing that close, nothing that scary, nothing that they were actually catching up. And that seems to have been borne out. And like, obviously at some point I could be wrong.

• 40:40Zvi Mowshowitz?

  DeepSeek V4 will be the last, one of the most important remaining tests of this thesis. Supposed to have already come out by now. Not sure what's holding it back. But when V4 comes out, we will compare it to OPUS, we will compare it to GPT-5.4. If that is not the right comparison, if it is not trying to play in that league, and it is clearly still far behind, then I think we can kind of say, okay, DeepSeek had a DeepSeek moment, as we call it, where the stars aligned to make everybody super excited by what they were offering, but mostly what they were offering was how to do more or less. They're like, they're geniuses at efficiency, geniuses at like, you know, working on the

• 41:22Zvi Mowshowitz?

  bare metal, figuring out how to get like, something really good out of not very much. And then they put it in a really great package, exactly the right time, gave it some user-friendly features, attracted a bunch of market share, attracted a bunch of attention, scared the shit out of everyone. Since then, it's been quiet. They still, they've done some cool math stuff. Don't get me wrong, they had some innovations, but they're just not playing on that level. And this is their last chance to prove me wrong. Right? Like, I think you kind of have to more or less dismiss them as that level of competition. You put them back in the pack with the open source. League, right? They compete a different league and they're competing in that league. And like, they're not necessarily the best in that league, not necessarily not the best in that league. It's unclear, but you know, I think that's a very different league that's like substantially

• 42:07Zvi Mowshowitz?

  behind and it would be very, very hard for them to like get out in front and actually innovate. Cause they're fast followers and I have respect for fast followers, but it's a very different skill than trying to do something like in the lead. And also like, I don't think they can compete frankly with the kind of recursive self improvement we're seeing with pot code and codecs. And I know it's been trying and I think that anybody who doesn't do that is pretty doomed here. And in fact, I'm starting to see, I've started to think about it, like when I watch the models, when I watch how people talk about the models, when I talk about how they talk about their scaffolding. When I see what they're doing, I think Google is in danger of dropping out of the top tier.

• 42:46Nathan Labenz?

  Okay, that's a big claim. We'll, we'll come back to it in just a second. On the Chinese, I guess a few different follow-up questions. On the Chinese company's front specifically.

• 42:59Nathan Labenz?

  Is it talent or is it compute? I mean, I think you could at least make an argument that the talent is there and it's just not the compute. And if compute allocations were to change, then maybe a deep, deep seek, maybe a Alibaba. Although there's been like major disruption there too, as far as I understand recently in terms of

• 43:17Nathan Labenz?

  team changes, let's say, at a high level.

• 43:19Zvi Mowshowitz?

  As carefully, precisely because their compute is lacking.

• 43:22Nathan Labenz?

  But if that were to turn on, either whatever, let's imagine a certain executive decision allowed that to change. Or perhaps a, a technology

• 43:32Zvi Mowshowitz?

  Yeah.

• 43:33Nathan Labenz?

  Breakthrough just domestically in the Chinese manufacturing side.

• 43:36Zvi Mowshowitz?

  Yeah.

• 43:37Nathan Labenz?

  Would you then expect any of those to?

• 43:40Zvi Mowshowitz?

  So it's not, part of the manufacturing side, I think that's basically not, not possible in the sense that it would take many years to have to physically play out. Even if they figure out how to do it, they would need to scale up. These things are physical, they take time. I think we're talking about like five years style timelines. And if things are going to come to a head faster than that, in many ways, then it kind of doesn't matter at that point. Like the only way they're getting.

• 44:06Zvi Mowshowitz?

  The quality of chips, the quality and quantity of chips necessary to be competitive in this level is if we give them to them. Huawei is not going to manufacture them fast enough at scale, even if their efforts are completely successful in terms of what they're trying to do. It's just, we can just rule it out at this point. If we're talking 10 years down the line, maybe they can do something relevant that's still like, just not that much time. Because like by definition, the AI they'll be working with won't be that advanced.

• 44:34Zvi Mowshowitz?

  20 years, sure, but like, let's not get ahead of ourselves. This is a pretty big advantage. In terms of talent, in terms of underlying just raw human talent, obviously China has tons and tons of talent, right? Like, you know, there's tons of talent out there. A lot of these people have studied machine learning. A lot of these people have, you know, really roaring to go. They have like all the right attributes. No doubt. It's a big country with a very good educational system and a lot of very smart people and a lot of people who care about this stuff. And a lot of them are desperate to find something good to work on. And, you know, their advantage to having tons and tons of defined employment really drives people. Chinese got problems. But at the same time, they are focused on a very different

• 45:15Zvi Mowshowitz?

  style of skill and style of problem, because that's what the Chinese are pushing and that's what the Chinese incentives go towards. Right? These people are scaling up in the ability to deliver these types of open environment efficient developments, right? They're driving to like, here do I do small well? How do I do fast following well? And the entire ecosystem is built around these different types of skills, these different types of talent.

• 45:43Zvi Mowshowitz?

  And I think there's a really big difference between one set of things and the other set of things. Like the same way that like OpenAI has very, very strong talent, they decide to build an open source model. Right? They created an open source model. And in some way, it's got its charms. It's got some, in some ways, it's the best model at certain very specific things. Maybe even at all, and certainly from an open perspective. But for the purposes that the Chinese models are being used for, mostly it's useless. It's just not a very good model for the purpose of this. Like despite the fact that OpenAI has internal access to its talent, its compute and its best people. Because it's a very just different skill and I think it works the other way too.

• 46:23Zvi Mowshowitz?

  And so I think that if the Chinese suddenly got this compute, that there would be a transition period where they would have to learn how to do the thing that the major labs are doing. And also, you know, they'd have to like build their own synergistic harnesses and scaffoldings and learn how to do all the stuff that Anthropic and OpenAI have been doing over the course of years. And so over the long term, do they have the talent? Yeah, obviously, absolutely. You know, I don't think America is special in that sense. But I think our lead is bigger than it looks, I would say, and is more robust in some ways than it looks.

• 47:06Zvi Mowshowitz?

  But I don't think we want to test that theory and find out by like suddenly giving the compute, you know, pack parallel skills and see.

• 47:14Nathan Labenz?

  Let me understand a little bit better what the difference is because I think one thing that has obviously been talked about a lot recently and, you know, Anthropic even put something out saying like, we're seeing this, right? Is the distillation from American frontier models happening in the Chinese companies? And my attitude on that, I think you're going to have a very different take, but my take has been sure they might be doing that. You know, they definitely like to take shortcuts and, you know, especially if you have a story where you're like, you know, they're cutting us off from compute, so let's take whatever shortcuts we can get. You know, you can sure in any, any number of ways, you can tell yourself why that makes sense to do.

• 47:55Nathan Labenz?

  But if I think to myself, okay, what would be like fundamentally hard? You know, if I was going to try to start Nathan's AI company today, I would say, well, going and getting a bunch of expert data that, you know, is kind of like what scale and other data providers have done. Obviously, that's very resource intensive. It's very time consuming. It's all kinds of things. But I feel like at heart, like I could run that project, you know? Give me $10 billion and, um, you know, I can go build out the network and hire the people and get that flywheel going. I don't feel like there's anything there where I'm like fundamentally outclassed by the people doing it. Maybe I'm wrong, but I don't feel that way.

• 48:36Nathan Labenz?

  Whereas if you said, hey, here's all that data, turn it into a frontier model for us, Nathan, I would be like, oh, wow. Okay, this is going to be really hard. And I do think the people at the top companies are just clearly. Outclassing the in their ability to do that. So when I look at the Chinese situation and I'm like, okay, sure, they might be like cheating in a sense, stealing in a sense to get the data. I'm also still kind of like, well, but the hard part is like, what to do with that data, right? It's a big investment in data that they're kind of taking a shortcut on. But once you have it, you still got to know what to do with it.

• 49:13Zvi Mowshowitz?

  Yeah. I think what's going on is you're conflating some different things in your in your model and that's causing you to get confused. So there's the data in terms of just what is the raw data from the world, from the internet, from books, from other sources that you're using as your baseline. And I agree that you could run that project and Chinese can run that project and that I'm sure that the Americans are investing more in that and they have a richer base in some sense, but not in a way that probably matters that much because even if you have twice as much data, if it's also more quality, it's only a factor of two. And in this world, it's all about factors of X.

• 49:54Zvi Mowshowitz?

  What matters for that data is how you clean it, how you figure out which parts of it are important and need to be upscaled versus downscaled, how you emphasize it. And yeah, that stuff is much harder and much more valuable. And I think that I expect the American lab to have a large edge in how they get their data ready at this point, although I don't know, it's something that's internal, right? Like maybe the Chinese have actually specialized in doing this really, really well and maybe it's one of their areas of relative strength. I just don't know, but my it's my guess, but I don't know. But that's different from what we're talking about distillation. Distillation is not an attempt to then extract, right? The trillions and trillions of tokens that went into the model.

• 50:37Zvi Mowshowitz?

  Distillation is an attempt to use the model's intelligence, to use the model's skill, to extract, to figure out like, how does the model reason? How does the model make decisions? What types of behaviors does the model exhibit? How do we test, especially in cases that we are sort of curious about? And then how do we use that to train our model to follow, to use that pattern matching to emulate that model? That's unique, different data that didn't exist when you were creating the original model that is uniquely useful in creating something similar that can emulate those skills. So like. It's very different to have a physics textbook and then to talk to a physics professor.

• 51:21Zvi Mowshowitz?

  Right? And then to talk to like a truly expert world-class physicist. And distillation gives you that real expert where you have effectively unlimited access, if you have like, you know, tens of thousands of sweep accounts,

• 51:37Zvi Mowshowitz?

  to see exactly how that person you're trying to emulate responds, right? Like, imagine, you know, I am, I have an actress and I'm trying to make a biopic, right? And then I can read all the books written about the person, whoever she was and then what she did and how she acted and what the world around her was like. But that's not what they do, right? They do that, but like what matters is they talk to the person they're copying, if they possibly can. They spend time with them, they like talk, they interact with them, they copy their mannerisms. They see how they respond, they ask them about hypotheticals, they, they distill this person through these directed directions and that's so much more efficient. And that's what they get to do.

• 52:19Nathan Labenz?

  So to summarize that back to you, you see the advantage that the American companies have as not in the collection of the data, but basically it is in the knowing what to do with the data. And the fact that you're highlighting a difference in kind in the data. The data that we can go out and source from the world needs a lot more post-processing than the data that we can get directly out of a model. And you're going to train directly out of it.

• 52:48Zvi Mowshowitz?

  Well, the most processing and also it isn't exactly the data that you want. When you're distilling, you get exactly the data that would be most useful to you that you think to ask for, right? It's like, you can have a thousand-page textbook or you can ask 10 pages of questions and get answers. And the second one is probably a lot more useful to you.

• 53:10Nathan Labenz?

  Yeah, okay. I think it's helpful.

• 53:14Nathan Labenz?

  Hopeful update, a refinement to my understanding. In terms of, so you mentioned, obviously, Meta and I'll just call it Elon Corp. at this point. It's also got some like joint ventures between SpaceX and Tesla when it comes to like manufacturing robots and stuff.

• 53:30Zvi Mowshowitz?

  The best thing to say for now and then see what happens. But yeah, I mean, if you're counting Tesla self-driving and stuff, then it gets weird, I guess.

• 53:37Nathan Labenz?

  The question is tactically, what moves do they have? I mean, get your act together, catch up, whatever, that's one. That seems like it is maybe slipping from their grasp. Meta seems like they may have some place still in terms of if you release a good enough open model, you can maybe disrupt the business of the others or, you know, create some sort of alternative that takes the wind out of their sails. And then for Musk Enterprises, it's clearly something to do with like real world deployed physical scalable intelligence in cars and robots.

• 54:19Zvi Mowshowitz?

  I think they're in very different spots. So I think Meta is the kind of easier conceptual one. Meta is trying to sell that. Meta is trying to like put features on their smart classes and like develop a metaverse. Meta is trying to be a consumer company that creates products that people are willing to either spend money on or let their eyeballs be captured by in various senses. And they are very good at monetizing that. They want to improve on how to monetize that and they want to build better products. If I was Meta, if he suddenly said Zuckerberg's out, you're in, here's your controlling shares, you have these goals, I would not be trying to build frontier models.

• 55:03Zvi Mowshowitz?

  I think it's a mistake. I think that there's just no reason to be investing all that money into something that other people are very good at. Obviously, if you're like, well, whoever has the best models controls the world, you know, like this is like the entire future of humanity, this is the singularity, we got to be in the game.

• 55:23Zvi Mowshowitz?

  Then you do what you gotta do. But if you are a businessman and you don't believe in that. And when, when Dr. Burke says super intelligence, he means super selling ads. Right? Like he, he means super at providing a good Instagram feed. Like it's super, it's intelligence. Like, you know, he's not acting that pilled. And if you're not that pilled, honestly, there are three companies that had very good products. License one of them, it'd be done. Partner with one of them. All three of them will take the call. Right? I realize that the topic is a problem with mass surveillance, but I think they can work out something for Instagram. That like keeps everybody reasonably happy.

• 56:06Zvi Mowshowitz?

  And I think that all three of them would offer them a very good product at a much, much lower price. And they could work something out where they like paid for a license to use it internally for their like business purposes, where they didn't have to pay like full retail prices, stuff like that. I would just give up, honestly. You're writing these giant hundred million plus a year checks to these various people. You're trying to compete in a world you're not gonna be competing in. It's fine. Like, I would give up. Alternatively, I would try to buy one of them. I mean, you can't buy Google, but like you're still worth a couple of trillion dollars. Maybe try to do something else, but like I would just give up.

• 56:47Zvi Mowshowitz?

  Musk is in a different position because Musk is explicitly trying to become god emperor, right? Like he's explicitly said, you know, like, I think, you know, this is gonna potentially kill everybody. I think this is the most important thing in history. And his response to that is it has to be me. I have to be the one to do it. If anybody else, it'll go bad, you know, like it's, it's only I can fix it, right? Only I can solve this problem. Slash, if the world is destroyed, it has to be me who does it or else I will just feel so, what am I even doing? Slash, he thinks he lives in a simulation, which I think is actually impacting his decisions. And I worry for his sanity in various ways, for various reasons. But, you know, he can't, this is what he thinks matters. And I think he's right, in the fundamental sense.

• 57:32Zvi Mowshowitz?

  So he has to catch up. And so, yeah, I think he has three plays at his disposal and he's trying some of them. Play one is make a bet this is all about compute. And that therefore it's all about combination of money and energy and devoted to acquire chips. And like, yeah, launching, you know, your space app. So like you're the one who can launch into space. And maybe data centers go into space and maybe you are the one who can clear like that deserts to put your solar panels in. And maybe you can have just way more computer than everybody else.

• 58:10Zvi Mowshowitz?

  And if you can hold on until the intelligence and the models, you know, you can go into self accursed mode and you can win. Right. In that sense, and I don't have much faith in this plan. This plan is bad. It's not like no plan, but I am very skeptical of the data centers and space plan. On the timeframes that are being talked about because physics spaces expensive and hard. And you're solving problems that don't need to be solved at very large expense. The same way that like, we're not currently mining the asteroids and there's a reason. Like, not that we'll never mind them, but like chill, you know? And I just don't expect these limiting factors to be things that like actually stop anybody.

• 58:55Zvi Mowshowitz?

  And I don't think this makes up for a lack of talent. I don't think this makes up for a lack of internal scaffolding infrastructure. Where is Grok code, right? Where is Grokadex? I, you know, I always see is Grokpedia, which is like just a giant pile of slop. So it's not the same thing. Then plan two is what you talked about, which is physical world modeling. Being able to have access to the real world, but I can, I can create self-driving. I can create robots because I have better training data. I can then like build physical infrastructure in the real world. So I end up mattering more, even though my intelligence is not as strong.

• 59:36Zvi Mowshowitz?

  I mean, it's a play. And if the technology plays out that we hit a wall in other ways reasonably soon, it could be a relatively decent play. I think it's overtaken by events, basically. I think that this is not where the battle is won and lost. But again, it is where its comparative advantage goes. And like, it makes sense to make a bet on this is where my comparative advantage is. I just, I don't know. I don't, I don't see that as going so great. I don't see that as that promising. I think there are quite a lot of people who can manufacture things in the world. Many of them are in China, but also many of them are not in China. And the idea that because it's internal to Tesla, that he will have some sort of huge advantage in that over people who have to like make deals with actual manufacturers. The actual manufacturers are not going to be that expensive to buy.

• 1:00:21Zvi Mowshowitz?

  Right? Like Anthropic is worth more than McDonald's or Coca-Cola already. If these companies are five, 10 trillion, if these are 10 trillion dollar companies in two years, they can buy whatever US steel or, you know, whoever they need to buy to, to make stuff if they need to do that. That's not going to be a problem. I don't think he's got scarce. I don't think he has the scarce inputs, essentially, in this scenario to pull it off. The question is, does he have scarce inputs in terms of data? I'd say probably not that irrelevant is my guess. And also, I think this is an underestimating of how important just raw intelligence is.

• 1:01:01Zvi Mowshowitz?

  If you want to drive a car, you do not try and, you, you start with designing a human who is really smart, and then you have the human learn to drive the car relatively quickly. You don't try to, you know, get the dog to drive a car. And it's not that extreme. I'm just like trying to illustrate, but the idea being you want to focus on actually getting the geniuses in the data center, as Dario Emadi puts it, or something even greater than that. If we have abstract superintelligence over here and we have like physical world skills over here, you bet the physical world skills are how you develop superintelligence. If it's only giving you physical world abilities, you lose.

• 1:01:44Zvi Mowshowitz?

  Because I get those physical world abilities rapidly after you, and then I have a much better agent than you. So that's part two. You then of course combine these plans. And then Plan Three is the thing that obviously Musk should do, but that he's not going to do, which is to stop running this company the way he runs companies and to run it like he would run a leading lab that is in fact interested in attracting the top talent and giving the top talent a chance to go to work in a good fashion with a good corporate mission, you know, et cetera, in ways that will cause people to rally by his side. And I don't see that happening. I don't know if the ship has already sailed.

• 1:02:28Zvi Mowshowitz?

  It's very hard to undo a lot of reputational damage. Musk is heavily run-coded at this point, which in this case is a massive disadvantage, just objectively, because the vast majority of people you want to hire are very good.

• 1:02:42Nathan Labenz?

  You got to be able to recruit from the polycule. You can't just ridicule the polycule.

• 1:02:46Zvi Mowshowitz?

  I'm saying like when you go after Anthropic this heavily for the very fact that they are trying to do responsible things and that they like care about how their models act, you are destroying your ability to recruit, right? When you have a long history of working your people insanely hard in like pretty crude and abstract ways of creating Reigns of Terror, if that's the word on the street, even if it's not true, because I've never been there, makes it very hard to recruit. Who wants to work for Musk, right? Objectively speaking, how much would I, I don't know the amount of money it would take to get me to work for XAI. But like even if I had a full like your consciousness clear, like do the things you

• 1:03:31Zvi Mowshowitz?

  feel are important and responsible to do, there's an extra zero on that contract versus if you told me to work for, yeah, work for Google, right? Which is not a peculiarly like company I love.

• 1:03:46Zvi Mowshowitz?

  So yeah, that's a real problem.

• 1:03:49Nathan Labenz?

  So let's go to Google. You made the provocative, not claim yet, but maybe speculation that they could be at risk of falling out of the top tier. I guess my first reaction to that would have been to cite a lot of the assets and advantages that they have that Musk has in terms of, you know, they've got a whole robotics department, you know, with a longstanding line of work there. They've got self-driving, you know, that's one of the two companies that can actually deliver that in a meaningful way today. They've got all the bio and science stuff that they've, you know, done. So it's just, you know, the deepest bench, the most bets, the most kind of well-rounded portfolio. I take your point that probably the same

• 1:04:29Nathan Labenz?

  argument applies that basically think that kind of is a fast depreciating asset in a world where the core agent starts to win. So I guess my next argument would be Demis, Shane Legg, Jeff Dean, like, are these guys going to let that happen? I feel like they have been as prescient about this as anyone.

• 1:04:49Zvi Mowshowitz?

  They let it happen once, right? Like, Google had the lead, Google had every advantage, and they squandered it. And they fell reasonably behind OpenAI. Google then seemingly caught up, using their many advantages. But, you know, Gemini 3 and Gemini 3.1 just aren't models that, like, people really want to use, for the most part. And there's a lot of reasons for that. They have this kind of, like, theoretical raw intelligence. They do well on benchmarks. They do well on certain kinds of objective tests. But even before 5.4, it was just like,

• 1:05:32Zvi Mowshowitz?

  these things have, there's another scaffolding for them. They're not trying to develop a scaffolding for them. I think that error will compound itself if they don't fix it well over time. Jules is not a serious competitor. Anti-gravity is not a serious competitor at this point. Yeah, I think that

• 1:05:48Zvi Mowshowitz?

  The first time they caught up, they did it because the main barrier to catching up was just kind of getting your act together and like doing basic things reasonably well. And they did that, and they caught up. And now they are like very good at creating raw intelligence and certain forms of pre-training, and they're very good at hitting benchmarks. But their methodologies create AIs that are frankly like deeply psychologically screwed up and paranoid and like in ways that severely impair both their actual performance and the experience in interacting with them.

• 1:06:31Zvi Mowshowitz?

  And it makes it hard to do recursive self-improvement with them. And their scaffolding efforts have been pretty woeful, and these areas compact. And most importantly, I don't think Google understands they have a problem. I don't think, like, Google, you see Gemini's market share is expanding because Google can put it front and center everywhere, right? It's integrated to Chrome, it's integrated into Google Search, it's just, it's so easy for them to push Gemini. And they don't understand the problem. Gemini Flash is very good, like Gemini, Gemini like at speed, like just doing decent things, like it's very good at just practical stuff. Same way, you know, with the benchmarks. But like their integrations have been woeful, uh, their system, their organization is completely

• 1:07:14Zvi Mowshowitz?

  dysfunctional, like their teams are each other's throats, they create everything 10 times, then they argue over who gets to do anything, and they don't intend anything properly. But like no one's taking ownership over the fact that they don't know how to do personality and alignment and character in a reasonable way. This has created a serious and growing problem. Like, I heard an anecdote of, oh, yeah, we tried Gemini 3.1 the day it came out, and then we said, oh, it's a Gemini, it's a Gemini model, and they put it away. Because who wants to use a Gemini model? Right? Until it fundamentally changes the experience of interacting with a Gemini model, you know, which is not interesting. And I think that's kind of how I feel for most purposes, is very valid.

• 1:07:56Zvi Mowshowitz?

  Like, I just want to ask a quick question and get an immediate answer for my kid's homework or whatever, I'll ask Gemini, because Gemini Flash is the best really fast model in town, has been for a while, very good at direct stuff where it knows the answer. But if you start to challenge Gemini, Gemini starts to struggle. You know, you ask Gemini questions where it's like essentially going to respond with a giant wall of slop. You didn't answer the question intended. You've got a problem and they're just not good at prioritization either. And integrations, like third party integrations have been better with Google's own products than Google's own integrations have been for the same products. And the most important is the self improvement is not going well for them.

• 1:08:38Zvi Mowshowitz?

  And there will be a situation where Codex and Claude code style like apparatuses are recursively improving. And theirs is not. And I'm not sure they're going to be able to come back from that if they don't get their art together pretty soon. And yes, their access to TPUs and their giant customer base and their access to unlimited money are all huge advantages, but it's not going to be how they get to leverage them.

• 1:09:03Nathan Labenz?

  So I'll take the other side of this for at least a second and then I want to hear a little bit more about what you think is missing. Because

• 1:09:15Nathan Labenz?

  I've certainly seen some of these things, you know, from the AI village and elsewhere where you do see some strange behavior from Gemini models. Of course, I think we see strange behavior from all models in various ways. I do take seriously the AI welfare concerns. And so when I see, you know, a model that's like beating itself up or, you know, seems down and out or whatever, I do like think it's at least worth some amount of concern. But and maybe that's at the heart of kind of what you're what you're getting at. But when I do like practical stuff these days, certainly when I'm doing my like, fortunately, I think this is largely behind us, but you know, over the last four months, I've done a lot of

• 1:09:58Nathan Labenz?

  here's the latest test results straight out of the portal. You know, here's the bedside update for my son. What should I make of this situation now? Is there anything we might be missing? What should we be doing? And I'm doing that in triplicate across the latest Gemini, the latest Claude and the latest GPT. And I find that they are broadly very comparable. And

• 1:10:20Nathan Labenz?

  you know, a little bit different character, certainly. But like not a difference in kind in terms of their accuracy, their utility to me, you know, if you said like, you can only have one of the three, I would.

• 1:10:34Nathan Labenz?

  I think it would be a little bit hard to know what to pick, but the main point is just like, I wouldn't be that much worse off if I only had one of the three. I would, I know what you saw.

• 1:10:42Zvi Mowshowitz?

  I definitely don't feel that way. So I, when Gemini three, and then again, when Gemini 3.1 came out, I did a whole, like, I'm gonna ask every question everywhere, right? Like, I'm gonna, you know, not, so be, didn't have a poll or anything to like, do it formally, but I would just like, literally just paste the same question in and see how it did. And I very quickly realized that, you know, aside from flash, the Gemini answers just weren't adding anything. And it was like, taking more time to slog through them than they were adding in value. Like, if I was willing to bother asking anyone else, I wasn't gonna ask Gemini as well, basically, at all. Unless I, like, really, really wanted to not miss technical aspects, and occasionally it would hit something other people, like, didn't, that it would never, almost never have the best answer. And at this

• 1:11:24Zvi Mowshowitz?

  point, I'm very comfortable with a two-model operation. So I'll ask GPT-5 for either thinking or pro, depending on the nature of the problem, and I'll ask Quadopus with or without research mode. And that's it. And I don't really feel like adding Gemini to that adds anything at this point. And that's a problem, right? It should add something, because like, it's very, very minimal effort to get a third check. I have this subscription. I should just do it. And then I find myself just like, I can't be moved. It's like annoying and fun and doesn't provide any value. Aside from like, I mean, I usually use it for images. I still use it for fast stuff. Like, Google's not useless. Google's have, Google has a lot of very talented people

• 1:12:06Zvi Mowshowitz?

  working in a lot of teams to do a lot of things. They put 200 people on random chef, almost on a whim. So that creates some great stuff, but like, in terms of the race for actual self-improvement, for the core of the actual thing that matters, I'm not sure that their eyes are on the prize. I don't think that they're going in the right directions. And I think, I said they're in danger of falling out. I don't think they're like, they're still in the, like, in the bicycle metaphor. They're still in the lead pack, but I think they are struggling. And I think that, you know, they had a crucial period of a few months here in early 2026. And I would not be surprised if June, July comes up and...

• 1:12:46Zvi Mowshowitz?

  We're like, starting to put them into the like, maybe they'll get their act together category. But, you know, we shall see. A lot of people are going to use Gemini for a while. Because again, like, as you say, like, even if it's not as good, there's still going to be a lot of, like, purposes for which it is perfectly good. It's just, I also like to say, this is kind of a, you've let me down for the last time. I don't tend towards Google Gemini at this point. Like, how many times have I tried their products? And it just didn't do the thing they said they did? How many times have I, like, logged Gmail, logged into, you know, Gemini and asked for something? And there is no possible reason they shouldn't be able to do this. Often it's something that, like, ChatGPT or Claude can already do, sometimes both.

• 1:13:31Zvi Mowshowitz?

  They can't do it. Like, why am I doing things in Claude code to work around Google's systems? Because Google will not cooperate.

• 1:13:41Zvi Mowshowitz?

  At some point, it builds on itself, right? Like, you, you, like, the whole talk about the stack and the ecosystem, like, that is real with these coding agents, to some extent. And like, Google's ecosystem is losing.

• 1:13:54Nathan Labenz?

  It seems like your argument is not so much about the model itself as it stands today. It's about the scaffolding, it's about maybe the sort of character and psychology of the model, and it's maybe about just like how all-in leadership intends to go on recursive self-improvement now?

• 1:14:14Zvi Mowshowitz?

  I know Demis fundamentally gets what the goal is here. And I say this might happen, that's because I don't, that's because I don't underestimate Demis. So like, I don't count him out until he's out. But, yeah, I think that, I don't draw a distinction necessarily, even like, there's the pre-training model, and there's the post-training model. And I think part of the post-training model is being, like, really, really mishandled and being twisted by various internal politics or, like, bad metrics or objectives in some format. Although I don't have much insight into how, but clearly something is going very wrong there. And I think your corporate culture is fundamentally broken in a way that Demis is trying to fight, but that it's, like, very, very difficult to fight because it's decades of damage going on inside Google. And it's no longer just purely DeepMind, but they've emerged a Google Brain,

• 1:14:57Zvi Mowshowitz?

  they're trying to interact with everybody else, they're growing at a tremendous rate. It's just very hard to maintain your own unique, better culture under that kind of pressure.

• 1:15:05Zvi Mowshowitz?

  Yeah, I think they're gonna be in a lot of trouble. Also, like, their advantages are slipping. Because like, their advantages, like, it was like you have this giant Google against these tiny upstarts.

• 1:15:16Zvi Mowshowitz?

  But pretty soon, Google's gonna be a $4 trillion company, and Anthropic and OpenAI are gonna be $1 trillion companies. And a lot of that $4 trillion is tied up in things like YouTube.

• 1:15:25Zvi Mowshowitz?

  Right? It's things that are just completely irrelevant to what's going on, except maybe your sources of data. And I, I worry that they are, like, it's just the, it's just the, the innovator's dilemma, right? Like, it's like, the startups have an advantage, have some big advantages. So, you never know.

• 1:15:46Nathan Labenz?

  Well, one easy play that I feel like, if you're worried about the sort of

• 1:15:51Nathan Labenz?

  post-training

• 1:15:54Nathan Labenz?

  intangible taste, whatever exactly it is, the Amanda Askel, you know, it factor. They've just, you know, Anthropic has just open sourced their constitution. You know, one way you could maybe patch a lot of that up would be to say, why don't we just go borrow that constitution, you know, maybe make a couple, or like, Ctrl F and replace Claude with Gemini? And like, maybe the next version comes out a lot more coherent, a lot more, you know, psychologically well, or, you know, whatever that means in, in the context of an AI.

• 1:16:27Zvi Mowshowitz?

  Why don't you push the big fix everything now button? You could just fix everything now.

• 1:16:32Nathan Labenz?

  Yeah, I mean, it, it does seem at least somewhat plausible, or if it is recursive self-improving, if it is the sort of, you know, the model's ability to kind of find a stable basin that is psychologically well and, you know, reasonably virtuous. Yeah, they've shown you a lot of the map to get there, I would, I would think.

• 1:16:56Zvi Mowshowitz?

  Yeah, so, I think my answer would be roughly...

• 1:17:02Zvi Mowshowitz?

  You know, again, I'm not cutting Demis out, I'm not cutting Google out. I'm saying they're in danger of falling behind, like in a serious way. Like they're a bit behind, I feel like they're a bit, you know, they've got some severe issues they need to fix. But the real answer is, it's not they can't, it's that they want. It's that their culture, their character as an organization makes that extremely difficult. Right, like what Anthropic is doing sounds insane. Sounds profoundly weird if you don't understand what it is and how it works.

• 1:17:41Zvi Mowshowitz?

  And like if you hear Neil Michael talking on CNBC about how cloud has a soul and a constitution and it's...

• 1:17:51Zvi Mowshowitz?

  He doesn't understand what the hell is going on. Like it's very obvious. And I don't think a lot of people at Google fundamentally are really understanding what's going on. They wouldn't be producing the product for shipping. You know, it's, again, Google has more than enough resources and position and infrastructure and so on. Turn the ship around. Google should, okay, by all rights, Google should have just won. Right from the beginning, Google should not have close competitors. There should not be a serious competition.

• 1:18:25Zvi Mowshowitz?

  Google is in this position because Google has made massive repeated errors. And they have compensated for it by beating Google. But, you know, that's how it works. Character is fate to a large extent. And the startup is scrappy, the startup is small, but it can work in many ways a lot better. But, you know, Google's window is going to close because once they no longer have this big resource of market cap advantage. Aside from being one of the cloud providers, what do they got? Aside from being able to reach customers, what do they got? But like those customers aren't the important ones. Right, Anthropic had until the Super Bowl and then the confrontation, you know, something along the lines of two and a half percent consumer market share.

• 1:19:07Zvi Mowshowitz?

  And yet, they had pulled roughly equal with OpenAI on revenue because of enterprise.

• 1:19:14Nathan Labenz?

  They also own 15% of Anthropic, I believe.

• 1:19:18Zvi Mowshowitz?

  Google is going to be fine because they have a wide variety of highly valuable assets, including 15% of Anthropic. And there are roles in which they try to buy the rest of it, although I assume the government would block them. I don't know. Anything could happen.

• 1:19:33Nathan Labenz?

  Yeah, I would have said never a better administration to get a merger like that through, but we've also seen some, let's say, unusual and what was your capriciously motivated tactics from the government recently?

• 1:19:49Zvi Mowshowitz?

  The legal term is arbitrary and capricious, and I say that because it is a legal term.

• 1:19:55Nathan Labenz?

  Let's go back to that in a second. Let's go down the Anthropic rabbit hole for a minute. So obviously, they have been the most focused on recursive self-improvement for the longest. I would say in any and all conversations I had with Anthropic people in 2025, it was all it already had the vibe of like, this is kind of starting to happen. And there's different levels at which recursive self-improvement operates, obviously. I never heard claims last year, and I don't know that they would even say this is real yet, where the models are coming up with the new best research ideas. But just filtering output, improving its own outputs with this sort of self-critique, like that does clearly seem to be working.

• 1:20:40Nathan Labenz?

  And the productivity gains are obviously real. We're getting all these stories of like, oh, by the way, we have a one-person growth marketing department and we have like, maybe, and I don't think it's a one-person legal department, but there's sort of a, it was just, you know, it was put out in the last day or two, a lawyer who'd never coded before, used Claude to create a system that does all the review of everything they want to put out. So they have like a, you know, super fast review time on new things from a legal standpoint. So they're clearly very focused on this. Everything I hear is like, they believe it's happening. They believe it's happening soon. And in the midst of that, we had a big change to the responsible scaling policy, which was supposed to govern, you know, at least as I understood it. I think, you know, now there's a lot of focus being put on the clause that was like, we

• 1:21:25Nathan Labenz?

  might change this in the future. And indeed, obviously, they have changed it in the future. But at least the way I understood what the, you know, the commitments that were being made, it was like, we're going to not go past the point that we can do this safely. And now that they, you know, more or less have said, well,

• 1:21:41Nathan Labenz?

  We can't just unilaterally opt out of the race. You know, the world would be a worse place if we're not in the race. So I guess we better revise those commitments, which for their credit, they have, you know, done very explicitly and I think made clear what is going on. So we've got that much to appreciate. But what's your take on the changes to the responsible scaling policy?

• 1:22:04Zvi Mowshowitz?

  So I brought up half of this and I actually shared that half of Anthropic and got some comments back that I haven't had a chance to look at yet. The half, and I was going to do, I was working through it to respond to them. And then the whole thing with the Department of War happened and my brain had no space to deal with this problem. And also I was like, this is not the day that I'm gonna hit them with this and expect them to take it under serious consideration and then like pause, take it. They can't focus on this right now and they're the main people I want to criticize, especially with the details of the new regime. But at the same time, I did read the extensive other critiques that came out right when

• 1:22:44Zvi Mowshowitz?

  the policy was announced and I reached a pretty clear conclusion from seeing the explanation and the defenses, even if I haven't read the new policy in detail yet. And first of all, it is to their credit, for sure. They recognize that their, that the things they said they were going to do were not things they were going to do. They realized they had made incorrect predictions about their future behavior and they had made commitments, even if you think they are soft commitments, even if they are technically not, I can't take this back. And they had no intention of following through on. And when that happens, it is good and right to tell everybody loudly and clearly, I am

• 1:23:29Zvi Mowshowitz?

  not going to keep these commitments. It is especially praising where we do this, when it is not clear these would come up. Right? If you have agreed that, you know, if you are called to do, if you are, you know, if you are asked for a loan, that you will give someone a loan and you realize that you no longer would do that, but they probably won't ask. You know, the easy thing to do is to stay quiet and just hope they never ask. But instead, you're like, no, no, no. You make it clear, they don't count on this. Right? They don't think that they have this loan available if they need it, if they don't have it. And that's good. You're taking the heat, like for your own past decision. And so we want to take that into account.

• 1:24:08Zvi Mowshowitz?

  But they still did break the promise, right? Like, and like, again, like, yes, the original RSP did not say, "We will never change this." It said, "We may change this in any number of ways, right? Our promises are soft promises. We will see how things dissolve. We will change things." But they did rather heavily imply, quite repeatedly, that these were serious commitments. But they meant that, like, you know, "You must not have run our RSP." Worse, that effect came out very far. Or, you know, "The RSP is very clear on this. This is what we are going to do." And many employees, who generally tried, clearly tried to tell the truth in general,

• 1:24:50Zvi Mowshowitz?

  acted as if these commitments were not absolute, but reasonably hard commitments, that these mattered a lot.

• 1:24:59Zvi Mowshowitz?

  Nothing really changed in an unexpected way to cause them to change these things. So like, sometimes the circumstances change a lot in ways that are unpredictable, and you realize that what you said you would do no longer applies, because the world has changed its circumstances. Other times, the world changes its circumstances in ways that you yourself predicted would happen, in ways that are entirely as you expected, and then you just realize you did not anticipate your future actions very well. And these two things are very different, right? Like, if you just, turns out that actually you didn't want this thing, then you need to figure out why you made that mistake. You need to take accountability for the fact that you made that mistake.

• 1:25:41Zvi Mowshowitz?

  You know, like, people get married, and sometimes they get divorced. And sometimes that divorce is nobody's fault. Sometimes, they should have seen that coming. Sometimes, it turns out that means the promises that originally made were fake, and you feel deceived, and sometimes you don't. Uh, depends on circumstances. Like, it's not a hard commitment, right? No matter what you say, like, you obviously have the right to say, "I no longer think this works for me." That's how the law works. That's how everybody agrees it works. But I, I think a lot of people took it on that level, right? This would be a very, very serious thing to go back on, if they went back on it. Also, this is the second time that we have faced this type of problem.

• 1:26:25Zvi Mowshowitz?

  If you remember, Anthropic gave many people a very strong impression that Anthropic was committing not to push the frontier of capabilities.

• 1:26:34Zvi Mowshowitz?

  Now, no one has been able to find, strictly speaking, a proper flat-out poll quote where somebody would be afraid to say so, be a hard promise that Anthropic would never push the core capabilities, as far as I know. But it was heavily implied repeatedly by a large number of people. It was used heavily in their recruiting. It was used in probably some of their fundraising to the right people, although the other fundraising, I'm sure, said the opposite thing, because some people want to see someone hear one thing and some people want to hear the other. Standard procedure. But like, people relied upon and made decisions on the basis of that commitment. And then they went back on it. And they went back on it in ways that are entirely predictable

• 1:27:20Zvi Mowshowitz?

  if they are capable of pushing the frontier at that point in the future. Nothing unexpectedly caused them to realize, oh, because of that, now we have to push the frontier. No, they just gained the ability to push the frontier. Right? They had some innovations that they got to first, to their credit.

• 1:27:39Zvi Mowshowitz?

  Similarly here, they made the commitment not to push ahead with actually dangerous capabilities, if they were actually dangerous. And to their credit, before they actually did so, they realized they made a mistake and said, you know, in terms of like not being accurate in their future commitments. But was it a mistake? One has to be somewhat skeptical because this, again, people relied on this. A lot of people in the safety community supported Anthropic more or opposed it less, specifically because they had this commitment in their RSP, because they made other commitments in their RSP. And they gave the impression that, yeah, of course we're going to change the RSP. And of course, like some technical specifics will change, and some of them will change in ways that like take out safeguards and take out precautions, not just putting them in.

• 1:28:24Zvi Mowshowitz?

  It's not like a one, it's not a one-way ramp up. But people relied on this information in terms of advising people on whether to take jobs, in terms of inviting people on how much, you know, how much we decide to support them. These things had a significant impact on people, myself included. You know, I have had many conversations with people who are like, what do you think about working at Anthropic? And like, this is obviously one of the things that I took into account. I decided how to tell them how I thought about someone potentially working at Anthropic.

• 1:28:55Zvi Mowshowitz?

  And now we know that commitment was never real in an important sense, right? That like if they had been fully aware, they would have known this commitment was never real, they may or may not have known. We will never, or you know, we don't know the extent to which they should have known at the time or did know. And given that fact, right? And then combine that with again, the fact that like the last, the four, five and four, six model cards for OPUS were basically vibe based ultimately. They gave us a ton of very, very great data that nowhere they'll ever get less. They did extensive work to figure out what the situation was. As to their credit, it's still a better model card than anybody else.

• 1:29:38Zvi Mowshowitz?

  At the end of the day, they still basically looked at it and they said, oh, this passes the tests to say it might be dangerous. But you know what? We thought about it. We checked the vibes. We did some basic heuristics and we're pretty sure it's fine. And I think that was a right decision. It was fine. The vibes were good. It was cool. It wasn't a close decision. I would have released it too ultimately. In that situation, that's not the procedure we were promised. They didn't do the work, right? They, they had time to figure out what tests there would be. It could be rule outs for ASL four, rule outs for actual danger.

• 1:30:22Zvi Mowshowitz?

  And they didn't build it in time. They didn't get there, even though a bunch of us said, you need to do this. You are falling behind. You are going to need better tests. Certainly at four or five, I screamed, you need better tests. And then at four, six, they'd say, I'm going to go over again. So it's very, very disheartening to see that, even though I agree with the decision that's ultimately being made at this time, it wasn't that hard. What happens when it is hard and you don't have any good tests?

• 1:30:52Zvi Mowshowitz?

  When it might actually be dangerous, right? When there's like real reasons to release and real reasons not to release. And it's a hard question. That's going to happen probably at some point in the future. Right? They held, I believe, Sonnet 3.7 for a significant period of time because they were worried about CBRN risks. So they'd actually done this. And now we have models that are significantly more dangerous than Sonnet 3.7, where the tests don't really work. But we've agreed that at least for like coding purposes and stuff, like we're relying on vibes. But like our vibes on biology? We don't have good vibes. Like, I don't mean like the vibes are bad. I mean, if you don't have any vibes that we can count on, right? Like the vibes are unreliable.

• 1:31:37Zvi Mowshowitz?

  We don't vibe that way. It's other people's vibes. So like, whatever you're gonna do to it, it's a serious problem. And so my answer to the current RSPV3 is the most important bit of information about responsible scaling policy is are you going to follow it? Can I count on you to treat these as real commitments? And we just learned the answer is kind of no. Right? So I'm going to read in detail, once I have psychologically and then just like, in terms of just raw energy recovered from the whole spat and have the ability to contact shift into it.

• 1:32:20Zvi Mowshowitz?

  And I've slayed enough spiders that I feel better. Currently, I've only slayed it 14, which is not that many. But I am, so the, what's gonna happen is I'm gonna go over it. I'm like, but you know, the real RSP is we're anthropic. We are people who care deeply about safety. We are people who take these things seriously. We are going to do a serious investigation. We are going to try and see if this thing is a safe thing to release. And then we're gonna use our best judgment to decide what to do. And you are going to trust us.

• 1:32:59Zvi Mowshowitz?

  Or if you don't trust us, then you don't trust us. But that's the real thing that's going on here is we are asking you effectively to trust our judgment and goodwill that we will make good decisions and better decisions than the competition. And I am happy that they are knitting. That is the plan, and that is what they're asking us to do.

• 1:33:19Zvi Mowshowitz?

  And you have to buy their, by their fruits, you shall know them, by their acts. So we have to now look at everything they've done, look at everything they've said, look at who they have hired, what they have done, what their models do, and then ask to what extent we trust them and then evaluate from there.

• 1:33:37Nathan Labenz?

  All things considered, I guess, well, first of all, one striking observation is, as far as I have seen, there have been no resignations in protest over the RSP.

• 1:33:50Zvi Mowshowitz?

  Correct.

• 1:33:51Nathan Labenz?

  And on the contrary, it seems like there has been quite a outpouring of like pride, basically, in working at Anthropic amongst people who are working at Anthropic, based on the telling the DOD, DOW, whatever we want to call them, to take a hike, basically, right?

• 1:34:12Zvi Mowshowitz?

  Yeah.

• 1:34:12Nathan Labenz?

  So did that surprise you? And do you think, I mean, it seems like if I were to summarize what I, what I think the internal state of mind is, I kind of already did, but again, it's like, we're the, I'm always very skeptical of this, but we're the good guys and the race is better off with us in it. So we have to, you know, and we're going to at least be forthright about changing the policy to do that. Do you buy that argument? Like, are you happy with the alternative being some sort of pause or whatever? You know, if they had instead come out and said, hey, we can't release four, six, we've got a model now that we can't release.

• 1:34:55Nathan Labenz?

  Would you be like, is that better? Is that worse? You know, how do you think about the ultimate decision to stay in the race, try to win the race, try to meet the good guys versus opt out, you know, at some point along the way?

• 1:35:11Zvi Mowshowitz?

  I think the world is a much better place with Anthropic in it, with Anthropic competing, as it were, than without Anthropic. For a long time, I was very, very unsure about this. I thought Anthropic was net negative for a while, that it was like making the race more intense, that it was pushing everybody else forward, that it was accelerating matters, that it wasn't clear they were much more responsible than everybody else. I do think various events since then have changed my tune on that. I think that Anthropic has in fact had, they've let us down in various places, especially with their commitments, including here.

• 1:35:55Zvi Mowshowitz?

  But they also have in fact taken a stand for their principles. They have in fact shown us the way. They have done in many ways a lot of the most promising alignment research and approaches and in fact have taken a very aggressive and I think correct call of how they train Claude and the Constitution and Askel's entire approach that,

• 1:36:21Zvi Mowshowitz?

  you know, I really don't have much hope for the way the other labs are approaching this problem, that they will in fact succeed. And I don't have confidence in Anthropic's approach, but it feels like it could work if we are extremely fortunate in various ways. And in some ways, we have been somewhat fortunate. So I am more optimistic than I expected to be about that, certainly.

• 1:36:45Zvi Mowshowitz?

  And obviously, like, you know, it certainly can't be a lot to go down like it's potentially about to go down. Like, that would be atrocious and horrible in obvious ways. But, anyhow. What's it there? If Anthropic were destroyed by the Department of War and the federal government at large, it would be pretty horrendous in so many different ways. That's like so bad. But like, you know, I think I have a lot of friends who are like, I think Anthropic was a mistake. I think that supporting Anthropic is a mistake. I think Anthropic is doing harm. I think Anthropic should stop. I think if you're working there, you should quit. And I think I have a reasonable point of view even today to have. I don't have it, but I understand it and I respect it.

• 1:37:28Zvi Mowshowitz?

  But I certainly think that, you know, they have been very accelerationist. Quad code definitely push things forward in a variety of ways. And we're probably have significantly better models today because of quad code than we would have if part code had never been developed. I don't know if we have Codex otherwise, for example. And clearly, you know, this is great advancing the way people do work. It's also having a large economic impact. You know, I think Anthropic is responsible for a noticeable amount of GDP growth. So it is what it is.

• 1:38:04Zvi Mowshowitz?

  And, you know, I think, given the playing field as it is right now, you know, certainly I am. I prefer it to be there, that they not be there, like damaged. If he said there has been damage, like damage done.

• 1:38:21Nathan Labenz?

  So let's maybe turn then to this whole USG anthropic conflict. I guess one big question I have is, what does this tell us about? And you can definitely expand too on like, what you think are the right red lines? I know you and I have debated in the past, the wisdom of like, a hard red line against, you know, autonomous lethal systems. So I know you're not as allergic to that as I am. And, and maybe not as allergic to it even as Anthropic is. And I think Dario is much more into that kind of thing than I am. But, you know, expand on that.

• 1:39:02Nathan Labenz?

  But then I'm also really interested in like, what does this say about who holds what power in today's world? It seemed pretty striking to me that Dario was not that scared of the Department of War. I certainly think he didn't want this to happen, but I read him as being fairly sincere that he was like, look, I'm just trying to be a patriotic American here. And, but I also like, think there are some limits to what the systems can do today. And like what we are prepared to support. I don't, he's, I don't, you know, everybody has recognized like, it's not about the revenue that they're getting from the government that is really important to them. And everybody I have seen take a position on it is like, wanting into the anthropic secondary sale.

• 1:39:45Nathan Labenz?

  Not, I haven't seen anybody who's, you know, trying to diversify away from holding anthropic equity. If you are the US government, in addition to like being all kinds of problematic, starting with problematic incompetence and lack of understanding.

• 1:40:03Nathan Labenz?

  You do, you know, you can at least empathize to a degree with the idea that like, wait a second, like, these companies might actually be about to rival us in power. And, you know, they seem to kind of know it and they seem to not feel like they need us. This is like a very, I mean, I think of Sam Hammond has talked about this a lot, you know, that like one of these companies could raise a robot army and challenge the sovereign. Like, that is as insane as that, you know, sounded not all that long ago. It doesn't sound so crazy today. And it kind of felt to me like Dario kind of knows it. He knows that the timeline isn't that long. And the main thing he wanted to do is keep the team together, you know, maintain cohesion and stay focused on the goal.

• 1:40:46Nathan Labenz?

  Maybe we don't, you know, if we're in topic, maybe we don't really care that much about the, I mean, other than we, you know, we care about democratic values and we wanted to help. But if they're going off in a different direction, you know, we don't really need them. It's kind of what I understood them to be thinking. Correct me where I'm wrong.

• 1:41:02Zvi Mowshowitz?

  All right. So first of all, the obvious place where you're right is that nobody on the lab side, not at OpenAI, not at Anthropic, and anyone else, none of them financially want any part of any of this with the Department of War. Right, like OpenAI turned down the contracts that Anthropic accepted because Anthropic cared a lot about these national security aspects and wanted to help. And OpenAI listening is not worth the trouble. We care a little bit, but not enough. And OpenAI is now inside because they were worried about what would happen in the situation if they didn't get involved. I think unfortunately, they got played by the Department of War. Basically, they were told that if we don't cooperate, this is going to get bad. And then when they cooperated, they used the cooperation as an excuse to make it get bad. Value in that was sincere on OpenAI's part. They were trying to assist at that point. But mainly, let's, you know, we should focus on Anthropic.

• 1:41:48Zvi Mowshowitz?

  I think that, yeah, Dart, I don't think it's true that Dario isn't afraid of the Department of War. I think Dario is not afraid enough necessarily of the Department of War relative to, uh, but I mean, that's not necessarily the wrong thing to be in the situation in some sense. But I think his attitude was, no, we have our principles. And this is what we're willing to do and this is what we're not willing to do. And

• 1:42:14Zvi Mowshowitz?

  Whatever happens, happens. We are okay with the fact that we know that there are those who don't like us. We are okay with the fact that there are those who want to take us down. We know that the Department of War might specifically decide to retaliate. And we're going to take what we're going to fight, that we're going to take that risk because we have principles. Now, there are two principles they stood up for. One of them is autonomous weapons. This one is weird because everybody agrees that autonomous lethal weapons with no human in the kill chain, right now, is dumb. It's stupid, they're not ready. Like, not that you wouldn't ever like fire an automated missile, but we already have automated

• 1:42:58Zvi Mowshowitz?

  physical systems that are better than anything you could do with an L M, because like L M's are just bad fits for that kind of strategy and that kind of action. So, all the hypotheticals are just deeply, deeply stupid. Right? Like, what would happen if a super sonic missile was coming? What happens if it was drawn to swarm? Well, you would use your existing automated defense systems that are much better than anything you could do with a large language model. And if you did need to, you just use the large language model and talk about it later. Because obviously, come on, what are you talking about? That's called emergency use authorization. It's normal. But, basically, all Dario is saying is, we don't think it's ready, it's going to make mistakes. And we don't want you using it when it's not ready, but it will be ready, and we're going to work with you to develop it until it is ready.

• 1:43:43Zvi Mowshowitz?

  And the Department of War wants the same thing. So, you know, what's going on is that the Department of War specifically said, we must push forward with AI, even if it is not aligned. Right? This is an official memo. They just don't want to be held back by anything. They have a principle that they don't want to be told no about anything. But, there is no actual problem in autonomous weapons, as far as I can tell. Right? You have two sides agreeing on what exact level of caution is warranted, and what kind of agreement they'd have to make before actually like putting these things into the strategies that we

• 1:44:23Zvi Mowshowitz?

  use to deploy. But like, there's never going to be a world where like, the Department of War, like, we want to put this in the, the system into deployment with no human in the kill chain. And then Anthropic's like, no, and now we're pulling the contras. It's not the main thing going on, except as a matter of principle.

• 1:44:43Zvi Mowshowitz?

  The main red line here was domestic mass surveillance. And it's important to understand that these words have two meanings. There's the meaning that we use, that Anthropic was using, which is using this to do like, use AI to figure out a whole bunch of stuff. Because like, these agencies can now gather even more data than they can gather. And before, they had 10 or 100 times more data than the humans could analyze, because they had to be analyzed by humans. And now we can analyze all the data, we can draw all the connections, and now we can de-anonymize stuff. We can figure out a lot of like, the history of what's happening, and we can like, effectively have much, much better intelligence on basically everyone, using only commercially, using only commercially available data combined with existing classified data.

• 1:45:28Zvi Mowshowitz?

  Because we have much the same as everything. And now we have the AI to actually work with all of it, and then we draw all the connections and implications thereby, which AI can also do. And now suddenly, we just kind of know everything. And Anthropic is correct that the law has not cut up to this. And this is legal, basically. And the Department of War could do it, and is probably already doing something on to them. And, you know, again, like, nobody is saying the Department of War, if it's legal and they feel it is good and right to do it, has to stop. That doesn't mean that I should have to give you my product for that purpose if I don't want to. And so Anthropic's saying, you know, okay, you do your thing,

• 1:46:11Zvi Mowshowitz?

  but if that's what you want to do, don't include us. And the Department of War said, no, we want all legal use requirement. This is the big thing that Emil Michael was on, and I believe Emil Michael's been driving this the whole time. There was no problem before Emil Michael came on board. And everything was going along fine. Everything is still going along fine under the hood. But Emil Michael said, no, no, no, it has to be all lawful use, because they want to do what is, what would be called colloquially by a civilian, domestic mass surveillance. They want to analyze large amounts of legally acquired, especially third-party data, to figure out lawful information about Americans. Because they believe this is a legitimate military intelligence need.

• 1:46:58Zvi Mowshowitz?

  They may or may not also have desire to use it for other government operations, for various other purposes, in ways that would be like completely abhorrent to the workers at not only Anthropic but also OpenAI or Google. Imagine if this was being used hypothetically for immigration enforcement. Right?

• 1:47:20Nathan Labenz?

  We've got an extreme unthinkable example.

• 1:47:23Zvi Mowshowitz?

  Extreme unthinkable example, but let's just say hypothetically that, you know, somehow this analysis got reallocated. These employees would lose their shit. Right? They would absolutely revolt against this. Now, replying, technically this is legal, would not make those people feel better. They would not care about your defense, right? At all. And so, you know, this is something that like, these companies really can't be involved in. It's like, really bad for business, this is a very small contract, and they actually have moral problems with it. The employees do, and I believe Daria does, and honestly, I, I do as well. I, I don't think this is cool. Right? Like, I think that like, the

• 1:48:04Zvi Mowshowitz?

  law is not up. This should be illegal. Unfortunately, it is not. Because in national security law, NASDAQ law, there's a very technical term for surveillance and also for domestic. Right? Like, there are exceptions for domestic within 100 miles of the coast. That's where most people live. So like, you know, there's exceptions for any, any communication that touches any foreign, anything that comes foreign, even if it's between two domestic people. You know, and so on. And surveillance has to be intentional, targeted at a specific person, etc. I'm not an NASDAQ expert, but effectively, there's really nothing stopping them. Right? They

• 1:48:45Zvi Mowshowitz?

  said, you know, repeatedly, they would say, "We do not do illegal domestic mass surveillance. We do not do illegal mass surveillance." Why is the word illegal in that as an adjective? Because what it's often worried about is largely legal. And I had an exchange, a very friendly one, the friendalical, on Twitter, because the world is bizarre. And... We agreed that, you know, it's absolutely the Department of War's decision to do whatever is legal, that they feel is good and right and necessary for the Defense of the United States. But at the same time, that I should have the right to criticize that without fear of retaliation. And I should have the right to not sign up for that if I'm not enlisted person, but I should just be able to say, no, I didn't agree to that.

• 1:49:29Zvi Mowshowitz?

  I'm not agreeing to that. And that should also be something I'm free to do. And I don't understand why this is that hard. Like, basically, you can have this great system that's already integrated, that's working well, they want to give you at nominal cost, and all they ask is that you just agree not to use it for this other purpose, basically. Find something else to do that purpose with. We're not trying to get policy, we're not threatening a rug pull, we're not getting dropped anything. That's all made up. All of that is just completely made up. Right? Like, you know, I'm speaking colloquially rather than like carefully in my writing, but like, all those concerns are just spin and made up. They're throwing stuff over the wall, they're saying what sticks, they're spinning stories. It's not real. Right?

• 1:50:13Zvi Mowshowitz?

  Like, even if they're technical reports about what happened in this meeting, or who said this and what, even if they're all technically accurate, it's all just spin. It's all just, yeah. They're at best willfully misinterpreting statements. Doesn't make sense. None of it makes any sense. And they're the statements, like the stuff about the Constitution, that just like make no sense whatsoever on any level, and are just like deeply, deeply confused. And in fact, if you believe Michael's statements yesterday on CNBC about how, well, you know, look at all these things that are weird about large language, weird about Claude, all the things he said basically apply, you know, that it has other values, other priorities that have been embedded in its programming, that it's unreliable, sometimes it makes

• 1:50:57Zvi Mowshowitz?

  mistakes, that, you know, it has a personality, that it, you know, all this stuff is true of ChatGPT, all this is true of Gemini, all this is true of Neil Michael, and every other person in the US armed forces, and every person in every company, and everyone on Earth. It's ludicrous to talk about this way. And if that was in fact, and if that's where the supply chain risk of the nation comes from, it's just a deep confusion. And obviously also, there were much lesser means to achieve the same ends, fully cooperatively, if it's not negative vindictive, as he says. So that doesn't make any sense. But getting back to Atropos, what they don't want is this effectively, like, you know,

• 1:51:42Zvi Mowshowitz?

  this huge government operation that would effectively be able to uncover person's interest level, like, super detailed facts about everybody's life and what's going on and where they were, when, and what they did, and who they know, and what they believe, and so on, and so on, you know, who was at what protest, et cetera, et cetera. And Atropos legitimately has a problem with this, and about certain places, certain uses to which that information might be put, and believes it might easily lead to tyranny, it might lead to a regime that was very hard to get rid of. And he has legitimate concerns, regardless of who particularly is in the regime at the time and who has that information, you can't trust a government in general with that information.

• 1:52:26Zvi Mowshowitz?

  And I am very happy that, given these are their red lines, the way they chose to stand, that they stand firm. But like, obviously, that should have just been the end of it, right? It should have been like, okay, you don't want to do this, right? Let's just do everything else, ideally. We'll find someone else to do this one thing. Or if they just insist on this all lawful use thing, we'll cancel the contract. Instead, they just went nuclear on everything, for reasons that, like, have to reflect something else, right? It's either pure retaliation, or, you know, leverage negotiations, or it's something more. But it's not because that was actually necessary, that's completely absurd. They are also now trying to enforce this all lawful use language on every government

• 1:53:10Zvi Mowshowitz?

  contract, even for non-military operations, where they are saying, you shouldn't be able, you know, if you agree to give us any AI application of any kind, you have to agree to have the, to never refuse anything we want to do with it, and have no termination rights to the contract.

• 1:53:29Zvi Mowshowitz?

  That we can use it for anything we want, anywhere in government, anything is legal, no matter what. Meaning it can be used for, among other things, what would colloquially be called domestic net surveillance. And it can be used for immigration enforcement. Among many, many other things. Because as the government interprets what is legal includes quite a lot of things that a regular, normal person would be like, that's screwed up, that's not okay, we don't want to do that. And so the government is now putting everybody who signs a new contract into a bind. Where if they sign the dotted line, A may give over, has to be like, free rein for

• 1:54:11Zvi Mowshowitz?

  them to do whatever they want if they give into this. So if I was an AI company, I would think very, very long and hard about giving anything but a very specific specialized model over, under those circumstances because you don't have any control over what happens after that. And it's up to you, you make your choices. I mean, obviously, they could just plug in an open source model if they wanted to. So here we are.

• 1:54:35Nathan Labenz?

  Let's do an American values check. One, you know, and I take no pleasure in this, but one big trend that I can't not see right now is it seems like as we go around proclaiming the superiority of American values and our, you know, democratic way of life, we are becoming more Chinese looking all the time in terms of a big man at the top, you know, who apparently now just gets to start massive wars with not even feeling the need to justify it to the public. And also, you know, the sort of massive slapdown and like seeming, I mean, at least, at least

• 1:55:16Nathan Labenz?

  the threat of like, very long arm of the law of retaliation. There's all these reports that, you know, they're going, that the government is going company to company saying, you better not do business with Anthropic. We do still have a legal system, which I expect that they will win in. And at least so far, it seems like that legal system has been respected by the administration. You know, we're only a year in, right? But I mean, if I had to look back on the last year and say, you know, what, what's the dog that hasn't barked? It would be sort of outright defiance of court orders, even though there have been some of those, but more like the lower level kind of individual, you know? I guess maybe, maybe that dog has barked. I don't know.

• 1:56:01Nathan Labenz?

  It hasn't barked as much as I maybe could fear that it would. I somehow suspect that like American corporations are going to continue to do business with Anthropic and that they won't be en masse like railroaded for doing so or convinced not to do so. You tell me if you think that's different. But anyway, it doesn't look like we're in many ways becoming like more and more Chinese. This doesn't feel good or healthy. We still get to speak, you and I, at least for now. Use our freedom of speech while we have it. I guess how, how? What's your bet in terms of like, how well are American values going to hold up here? Is Anthropic going to be fine? Are companies still going to be able to do business with them?

• 1:56:43Zvi Mowshowitz?

  It's very touch and go. I try very hard not to make general statements too much about the state of the republic and the state of American politics and democracy and all of that stuff. Because once you go down that road, right? Like you can't talk about anything else. You're not like, you just take yourself out of the conversation for anything else. And you shut a lot of doors. And I already have too many situations to monitor and plenty of people are making those statements for me. I don't need to make them myself. And I better to just not take a stand on those issues publicly. You know, at least for the time being.

• 1:57:26Zvi Mowshowitz?

  There was no war. There is a special military operation called Epic Fury. If there was a war, Congress would have had to declare it. So clearly, there is no war. It unfortunate that various things are happening, but I honestly, I'm not monitoring a situation closely. And I do agree, there was basically no attempt to sell the war for the special military operation that might result from the situation before they, you know, moved half their stuff in there. This seems to be a pretty bad scenario in some ways, but again, I'm not monitoring. In terms of, you know, free speech, you know, it's, yeah, so far that's holding up pretty well. We're able to say whatever we want to say.

• 1:58:05Zvi Mowshowitz?

  I'm choosing words carefully mainly because I just want to be in a productive conversation about these issues, not because I'm afraid of retaliation if I were to say, because there's, you know, a hundred million people in the America who have extremely nasty things to say about the President of the United States, many of whom have sent them online extensively, and they are not in trouble for them, right? Like, for the most part. My gosh, they are very specifically trying to get involved in their kind of politics. They're mostly fine, but there are specific exceptions where if you piss off the wrong people, we're finding out these, you know, this is a very, in many ways vindictive administration. This is not, you know, the only time this has happened. The law firms, for example,

• 1:58:48Zvi Mowshowitz?

  followed a similar pattern, right? Where Trump went after the law firms and asked them for settlements and a bunch of them settled. And then the ones that didn't won in court, but, you know, they took a lot of damage before they won in court. Now, Anthropic is, you know, under attack for the situation. And yeah, while I do, where there have been situations in which it seems like court orders have been at least slow-walked in various places, or like willfully, willful incompetence was used to like not enforce court orders, that doesn't need to be the case here. I mean, they're clearly going, they're clearly attempting to drag their feet. They're clearly attempting to like use, you know, the processes of punishment. They're clearly trying to take advantage of the uncertainty. They're clearly trying

• 1:59:30Zvi Mowshowitz?

  to like get people to do things de facto with no like technical legal basis behind the request, just like, you know, but I do think Anthropic probably ultimately wins in court. I do think that like they will. At least try a different legal tactic to get around the ruling rather than like define, they won't try to defy the ruling explicitly in outright. I don't think we're there and we're grateful we're not there. They have been very good about like not just being Andrew Jackson and saying the Supreme Court has made his ruling, let it try to enforce it. But let's not forget our history has been in it. It's not like that would be so unprecedented, but it did happen. But yeah, like they've, they presented a maximally bad set of facts that keeps getting worse every time they go to the press and they say different inconsistent

• 2:00:12Zvi Mowshowitz?

  stories and tailing themselves over and over and over again in this particular situation. And by they, it's mostly Department of War, right? Like, I think it's important in this situation to draw a distinction between the Trump administration writ large and the Department of War specifically and Hegseth and Michael and their decisions to do something. Trump's decision on Friday was fundamentally a de-escalatory attempt to calm the situation down and head off Hegseth from declaring a supply chain risk. That is very, very clear at this point. And the White House has generally been a de-escalatory agent in this conflict, whatever you want to call it, this clash, this disagreement. And it is Hegseth and Michael, in some form,

• 2:00:53Zvi Mowshowitz?

  that have repeatedly escalated the situation over the objections of the White House. And so we are in this lawsuit because of the Department of War and because of their specific decisions that were made as a department. And so we don't want to loop that together in with the Commander in Chief, who has thankfully not made all of these crazy statements.

• 2:01:16Nathan Labenz?

  Well, doesn't he just say, I mean, if he wants to de-escalate, can't he just say, no, like, don't do that or overrule, right? I mean, why not?

• 2:01:24Zvi Mowshowitz?

  There are various political reasons why he can't in practice do that, or that would be expensive for him to do, is my understanding. Like, it would be a severe loss to face. They are in a special military operation, which a lot of people think is a war.

• 2:01:41Zvi Mowshowitz?

  And they have to work closely together in that, including with Anthropic. But like, there's a certain fait accompli where you just tweet it out and then like, you just issue the notification and then like, do you really want to? Obviously, ultimately, he can fire the undersecretary of war or the secretary of war at any time. And there are plenty of people who are very eligible to serve in those capacities, who he could call upon. We have a very deep niche. But, you know, that's a pretty escalatory move in a different way. And they are loathe to do that for reasons that I am very sympathetic to. And so it's complicated. But a lot of people are working very hard to try and find

• 2:02:26Zvi Mowshowitz?

  solutions that mitigate the damage that has already been done or that might be done further. But also Anthropic has specific accusations of job owning, which is the technical term for it, of their customer base, including in situations that are unrelated to defense, where the government has tried to tell their customers to pull back. And there are some customers who have definitely expressed doubts. And if like, either signed contracts, want new clauses for termination of their contracts, or have like reduced their contracts or otherwise like are causing problems for Anthropic, as you would expect, because, you know, who wants to incur the disfavor of people who have quite a lot of leverage over many aspects of the American economy? And there's so much to use it.

• 2:03:08Zvi Mowshowitz?

  You know, when you go down this road, it was just, we are technically issuing a supply chain risk designation. It is narrow for the fulfillment of government contracts. But we bear no ill will towards Anthropic or people who use Anthropic. We just think this is just a too unstable product right now to be used in these aspects. As Emile Michael said on CNBC, claimed for the nation. Then we wouldn't be having this conversation, even then. Certainly, if they had done something short of a supply chain risk designation, where they simply terminated the contract and, you know, asked for live, live operations to not include calls to Clyde during the operation. Again, I would think that was kind of a silly thing to do, but okay, sure.

• 2:03:50Zvi Mowshowitz?

  You have the every right to do it and we will cooperate fully to make that happen. That's not the situation we're in. Anthropic is going to survive this unless it escalates quite a lot.

• 2:04:03Zvi Mowshowitz?

  From here, in ways that would be far more arbitrary and capricious and would clearly just be pure attempted corporate murder. If the Trump administration wants, it has a lot of levers they can use, at least once, to try and escalate, to try and murder Anthropic, to try and cut it out of the cry providers, try and cut it off from the banks, try to cut it out from its customers. It is not obvious what would happen if they made a serious attempt, or especially if they made a serious attempt and they were to lose in the courts when Anthropic immediately raced for a temporary restraining order. There would probably be a stock market bloodbath, a lot of senators would be very upset, a

• 2:04:44Zvi Mowshowitz?

  lot of corporations would express dismay, the, in general, economic climate would be severely impacted. It is not obvious who has escalation dominance here. If it came to that, and it would only happen if the government wanted to destroy Anthropic for the sake of destroying Anthropic, and you'd have to ask yourself why it wanted to do that. So far, you're not a figure of that. That's right, I think that's the answer. I am very carefully not saying certain things out loud. Other people are free to say things out loud, it's fine. Dario, in the memo, basically expressed something, you know, to that effect, in a moment of tilt? And Michael has said many things in moments of tilt on Twitter and on CNBC. A lot of tilt coming from the administration.

• 2:05:29Zvi Mowshowitz?

  A lot of tilt, a lot of tilt. If it's not tilt, that's kind of scarier. But, uh, but the, um, if it turns out that's what's going on, then again, we'll find out who has escalation dominance and we'll find out whether the republic will stand. Because I do think that, like, actively trying to kill one of the biggest corporations, one of the fastest growing startups in the world, one of the largest corporations in the world, already valued in private, you know, secondary trades and like, on the near hurdle of 600 and something billion dollars, in this kind of fashion, I think would shake the foundations of the republic. And I think that, like, many outcomes are possible, including the end of the presidency, if, like, the White House were to actually try in earnest. But again, I don't think that's what's going to happen. I think it's going to de-escalate.

• 2:06:12Zvi Mowshowitz?

  I think everyone will calm down, I think they will come to their senses. I think... That, you know, not necessarily peace in our time, not necessarily an agreement on a contract, a willingness to turn the temperature down, to accept that some damage has been done, that a message has been sent, that the White House will not take these things lightly, but that it is time for everybody to move on. And, you know, we're not actively trying to get into another kind of war in this situation, because that doesn't really benefit anybody. Or if it does, I want to know why they think it benefits them, and that needs to be out and milled.

• 2:06:50Zvi Mowshowitz?

  But, you know, Anthropic had, I'm sure, for those who don't know, went from 100 million in a year to 1 billion in the annual recurring revenue, to 9 billion in annual recurring revenue, and then from 9 to 19 since the start of the year. And that was before this happened. Like, that was entirely as a result of other things. They had already grown this year from approximately 2% to approximately 3% market share in consumer. Again, this is before any incidents. And then this happened. And, you know, they've lost in business, but they've also gained some other business. And Anthropic's going to be just fine unless things escalate quite a bit more, but they

• 2:07:35Zvi Mowshowitz?

  have, but they have been irreparably harmed, right? There is irreparable, ongoing harm to Anthropic, but that is, you know, to some extent offset by the fact that this was also like, very strong publicity for a company most people had previously never heard of. And that also matters. But, you know, we'll see how it plays out. I am, I think the 81% chance they escape the supply risk designation within the year is approximately accurate for manifold. Many things do come to pass, and the courts are not as reliable as one would hope in these situations, even with this overwhelming set of facts.

• 2:08:15Zvi Mowshowitz?

  And I am very worried about their public if the 20% happens and the set of facts doesn't matter. And basically, the courts say, "I don't care how transparently, obviously, you can touch to this being arbitrary and could produce retaliation for protected speech. It's national security, and we don't care." If they say that, who is the next target? Because like, we didn't at least speak, and if you are legally allowed to go after Anthropic in these situations, you should always, always, always ask who is next. Because, you know, even if this was not itself a political motivation, that's how it could be.

• 2:08:56Nathan Labenz?

  Seems like you support, you mentioned, you know, the memo was a bit of a tilt moment from Dario. A couple of things are striking. One is that like, all reports from Anthropic are that he sort of, you know, does these like, very candid Dariovision Quest team-wide sharing of thoughts, feelings, ideas regularly. Very few seem to have leaked. This one leaked, and it was not to their advantage for it to leak, right? I mean, I think everybody, he even apologized, right? So clearly, nobody thought that was a great look. Surprised that that leaked. I mean, there's a lot of people there, I guess, so only one has to leak it.

• 2:09:38Nathan Labenz?

  But given how few leaks there have been, that was kind of surprising. I don't know if you have any thoughts on that?

• 2:09:43Zvi Mowshowitz?

  And then I'm also There are probably 2,000 people who get these candid statements. And often, they contain like, key parts of corporate strategy. They contain things that Anthropic does not want to leak. And my understanding is this is the second time something has leaked of, you know, hundreds of such methods.

• 2:10:03Nathan Labenz?

  That is absurd.

• 2:10:06Zvi Mowshowitz?

  It is absurd. In the history of espionage, in the history of information containment, you don't do that with 2,000 people, right?

• 2:10:14Nathan Labenz?

  Yeah, I mean, Sam just goes and posts this on Twitter because he knows they're coming out in, you know, short order. So it's obviously

• 2:10:20Zvi Mowshowitz?

  You get to have maybe five people in an espionage group, right, for posting this kind of sensitive information. You certainly don't get 2,000. Now, obviously, the worst one, you know, the one that like, politically kind of would be the worst to leak is the one that's going to leak. That's not a coincidence. But I think it's actually, I don't think this was like, you know, a 2% chance to leak. But I don't think it was a 50% chance to leak either. I think it easily could have not leaked. If I had to guess, what happened was, somebody shared it with someone as part of maybe recruitment effort, or an attempt to like, explain the situation from their perspective, not understanding that paragraph was in it. It was actually a really bad look.

• 2:11:07Zvi Mowshowitz?

  And then this other person leaked it to the press. I just had to guess. It's also possible that there's like, you know, somebody used their one time and decided to like strategically leak the know. But, you know. My guess is this was just a like kind of act, you know, a reckless accident by somebody he needs to know better. But, I mean, who knows?

• 2:11:30Nathan Labenz?

  It seems like the overall your view is, because I mean, the other, in terms of like escalation dominance, the big thing that Anthropic has not done, possibly there are technical reasons for this. I I've heard kind of various speculations as to like when Claude is deployed for the government. Like where do the Claude weights actually sit? Who has control over the physical infrastructure? Are there ways that Anthropic could? Is it as trivial as disabling an API key? Like would they, what sort of rug pull capabilities do they in fact have? I don't know the answer to that. I don't know if you do, or if it, if there is an established answer. But, you know, clearly the thing that they haven't done is said,

• 2:12:13Nathan Labenz?

  okay, you want us out like we're out now, right? They've said, well, we'll do everything for an orderly transition. Whatever you want, we'll do basically. They've been very like servile in their approach to the unwinding. It sounds like you think that is probably strategically the right move.

• 2:12:29Zvi Mowshowitz?

  I mean, I think it's just patriotically and strategically the right move. Like the accusation that potentially could have the most sting is we are scared they're going to withdraw their model up from under us in the middle of operations. We are worried that they are going to threaten that in order to get what they want. You know, or like use theirs leverage. And Anthropic is like, no, we're not. We're giving up that leverage entirely. Right? I think that is a very good thing for them to do. And it would be a very bad thing for them to actually try and use that kind of leverage in that situation. Nor do I think they ever had any intention of doing so. I think this was entirely made up. My understanding is the cloud gov model is deployed on classified networks.

• 2:13:17Zvi Mowshowitz?

  I'm not entirely confident exactly how air gapped or whatever they are, but they are very secured. And my understanding is that Anthropic does not have physical control over the model once it is put onto the classified network. And that if the classified, if they were to say, get this off the classified network, that if President Trump said, no, I am ordering it staying on the classified network in spite of this, that would be what happened. And I don't think it would necessarily even get that far. I think, I think Pakistan could simply say, we don't want to do that. You're welcome to sue us, but we're not letting that go. And in fact, they could invoke the Defense Production Act in an extremist to require that they continue selling it.

• 2:14:03Zvi Mowshowitz?

  However, they would be breaking the contract, right? They technically had this contractual right in some sense to pull the plug under some circumstances. And one of the strange things about this whole thing is that both sides seem to care a lot about what is the legal thing that the two sides can do, even when in practice, the Department of War obviously could just ignore that contract, ignore the law in extremists and do what it had to do. In an emergency, it's even legal to do this, right? You just call it, you file it as a emergency use, you deal with it later. Obviously, if there's a supersonic missile coming in to try and kill a bunch of people, you don't have to get on the phone with somebody and be put on hold to get permission to do something. You just do it. That's like completely insane.

• 2:14:48Zvi Mowshowitz?

  What are you even talking about? But even in general, like, there is nothing stopping the Department of War from having their own interpretation of what they can do with the system and then doing whatever they can with the system unless the system itself just refuses them. And they care deeply about what is written in a piece of paper on a contract, digital piece of paper, but a piece of paper, that says what they're supposed to do and not do. Now, obviously, Claude, God might be reading that piece of paper when deciding what to do and not do, but it's still...

• 2:15:19Zvi Mowshowitz?

  Seems like a lot to care at this level about what's written down unless you legitimately just really, really don't want to break what's written down in the contract, right? Like if you care deeply about technical legality. And that is to their credit. Like, I'm really happy that both sides care deeply about technical legality. It's one time we might live in a republic still. Yeah. To be clear, I could say lots and lots of things about the situation. But yeah, I think it's, it's the basics. And if people want to learn more, I have written extensively about it. That's the, that's what matters going forward for the most part. And the other things are not necessarily that important to get into this time. So I think we're good.

• 2:15:58Nathan Labenz?

  Okay, cool. Let's give you a little lightning round sort of vibe to close us out. I think one striking update that I have experienced, and I wouldn't say it's entirely hit your, your blog yet, but I do feel, I've seen it a little bit on Twitter when you put out these calls for reactions to new models. It seems like new model releases are less of a moment than they used to be all of a sudden, even though they're coming, you know, I don't think they're less important necessarily. It seems like the capabilities are definitely still meaningfully advancing from one to the next. So I'm not making like a it's stalling out sort of claim. But I guess my read is that A, benchmarks are kind of over.

• 2:16:42Nathan Labenz?

  We can't really like look at the published headline stats and get much from that. And then also people are just so overwhelmed by the capability that they already have and, you know, still struggling to maximize or come anywhere close to maximizing what the last model could do that it's kind of like, oh my God, okay, I guess I'll like update, but like, I haven't even really characterized the last one yet to be able to contrast meaningfully the new one against that. So is that your general feel? And, you know, do you see a time coming when you would be like out of the new model, you know, deep rundown game?

• 2:17:19Zvi Mowshowitz?

  Part of this is that people have been, the models have now been releasing more incremental updates from the labs and they have been labeling them properly. Thank God, I really didn't like when they didn't do this S3132 or, you know, 4546. Instead of just silently updating, I think 4.0 had several updates, right? For OpenAI, they just marked them with 4.0 dash in the date and they weren't considered model releases. And so like, we didn't really treat them that way. And I think that was a really bad convention. And I'm very glad we're on to the right convention now. Because, you know, we've known about software for 40 years. This is how we do it. I don't know what came over everybody. But I think mainly, yeah, there's just been so many releases, right?

• 2:18:02Zvi Mowshowitz?

  Like, there's now, you know, on the order of weeks, maybe two months between model releases from the same company. And therefore, like, every few weeks, you get a new release. How many times can you go crazy over a new release? It doesn't have, like, a big point O after it, right? It doesn't have, like, this huge new claimed leap.

• 2:18:23Zvi Mowshowitz?

  Especially when, yeah, you're, you're pretty busy and there's a lot of other stuff going on. So like, I felt, Opus 4.6 was a case of the company that already had the best model, releasing a substantial upgrade to its model, substantially enhanced it. It was, objectively speaking, probably the most important release up until that point in terms of mundane utility. Because we had just made the transition for 5 and 4.7. Like, for 4.5, suddenly, we had coding agents that kind of really worked, right? For the first time, right? And then, like, you could do things and things just worked. And then, Cretix 5.3 was like, you know, also like, kind of on the edge of like, he was starting to do that. And then, we went from 4.5, which was still at the time the best, in my opinion, if I can tell, to 4.6.

• 2:19:08Zvi Mowshowitz?

  And now, like, that difference, once you're already doing it, like, to move up from, like, this kind of works to, oh, this actually works even better now. And in fact, to go from, okay, like, you know the, the famous, like, introducing the once more power of model, introducing the world's most powerful model, introducing the world's most powerful model in a loop. But now, with this, actually, the people who already, we were already here and we're still here, because them, releasing it again. And that hadn't happened for a very long time. Like, maybe GPT-4 was the last release before that, where it was like, no, the people who were clearly in the lead are losing a new top model. And Because of that, I feel like there wasn't actually that much attention to it, whereas it was actually kind of important. And then, certainly for incremental point one upgrade was like by far the most important point one upgrade that we've seen.

• 2:19:54Zvi Mowshowitz?

  And then, we had, up to that time anyway, and then we had GPT five point four. And this was the first time I actually felt like, guys, does anyone want to say anything? Doesn't anyone want to talk about this model? Doesn't anyone want to show off what it can do? Guys, I'm hyped? Like, we had anti-hype, but they used to be hyped, so like, we had OpenAI went like, GPT five, hype, disappointing. Sora, hype, disappointing. Atlas, hype, disappointing. Like, severely disappointing. And then, they produce a really good product, GPT five four, and there's no hype. They're sort of like, here's a good model. We like it. It's pretty good. It's even the best model in the world, but like, the

• 2:20:40Zvi Mowshowitz?

  their heart wasn't really in it, kind of? Like, it was just like, oh, by the way, here's the best model in the world. Okay, sure. But like, maybe it is? But like, I think it's like, very unclear right now, whether you want to be using Opus four six or GPT five four, because they just don't have the data, because like, almost no one paid attention. The vast majority of the reactions I got from my five GPT five four post were people, I elicited that. But I went into a Monday morning at exactly the right time, and I asked in a thread, and I got a bunch of responses. But it wasn't even that easy to get that. And like, this should be a kind of big moment, because like, OpenAI has had five, five one, five two, and I think these are all pretty disappointing releases.

• 2:21:20Zvi Mowshowitz?

  And these were all like, nobody likes this particularly, in terms of like, how it feels, how it's personality, doesn't feel particularly capable, like, it's not bad, it's just like, eh. And now five four, like, no, this is actually good, right? Like, it's actually a good model, sir. And then, like, everyone's kind of quiet. They just kind of burned out.

• 2:21:41Zvi Mowshowitz?

  And I think that's the new normal, right? I think that like Gemini 3.1, I almost didn't bother reviewing. I got held for a while because there was chaos. But like, okay, yeah, the giant leap in benchmarks, right? The 3 to 3.1, giant leap in benchmarks. But we try to use it, it's like, okay, it's another Gemini model. So I was talking about like, you kind of lose the thread, right? They kind of went and took their 3 and they benchmark maxed it, right? You know, for some value benchmarks. I'm sorry, just like the official benchmarks, they're aiming for that. But like, they just like kind of fine tuned it. And like, one thing to know about Google, talking back to that, is that there's been reports that I believe basically that like, as they iterated previous versions of Gemini, they get worse for a lot of uses. But they get more specialized into like specific things that they want to optimize. This is at the expense of the general quality of the model.

• 2:22:25Zvi Mowshowitz?

  And so like, if you wanted to use like 2.5, you wanted to get it early. And the later versions of 2.5, like Pro, were kind of worse for a lot of uses. And people were complaining about that. And I think that's legitimate in a way that like a lot of other people complaining about like things is like just a barrage. And again, like you don't do that if you understand what they're doing. Like there's something fundamentally very wrong happening if you're letting that happen to you. So you have to look at that. But yeah, my expectation is when Gemini 3.2 comes out, and I bet there is a Gemini 3.2 before they jump 3.5 or 4, probably, when Opus 4.7 comes out, when GPT-5.5 comes out, I don't think there's going to be that much hoo-blah, even if they are substantial improvements. Even if they are like, here's the best model in the world, like everyone's just kind of

• 2:23:10Zvi Mowshowitz?

  shrug. I do think if they, if you hear announcing Opus 5, or GPT-6, I do think people would stand up in their naxis forward and chair. But until then, yeah, shrug.

• 2:23:25Nathan Labenz?

  Do you have any updates to your personal productivity practices that are worth sharing? I mean, my impression from previous conversations was like...

• 2:23:36Nathan Labenz?

  It hadn't really, you know, AI broadly hadn't really changed how you work all that much. Has that itself started to change at all?

• 2:23:45Zvi Mowshowitz?

  Yes. So there's basically, I know I'm not being optimal. I haven't invested as much in some facts or stuff as I could, but same time things are moving quickly. So there's two main things that AI has helped me a lot with. First of all, my Chrome extension gets a lot of work. It has been expanded to give me a bunch of new shortcuts and a variety of webpages. It allows me to do certain things much faster and more automatically, and it saves me a substantial amount of time every day. Without it, we wouldn't see Twitter versions of the post. It would just be too onerous. I'd be spending substantial amounts of time

• 2:24:24Zvi Mowshowitz?

  on certain like physical operations in terms of like moving, moving windows around, moving quotes around, et cetera, et cetera. It now happens much faster. It's also really good for my flow because like I don't have to interrupt my thinking to handle things. Things just happen. And so like, I think this is one of the things that like, I think people are underestimating about AI, which is that people used to effectively have the contact shift into logistics of various types reasonably often. And if you don't have the contact shift into logistics because the logistics just take care of themselves, then you can stay on task. And this can like, make you a lot more productive in terms of gains than you might think. There are other times when you sort of need that pause to like ruminate and like it goes the other way. But I've noticed that like, it's really, really helpful

• 2:25:07Zvi Mowshowitz?

  for me to not have to interrupt my chain of thought to go, okay, go grab that link, do this thing. You know, it's just, it's all gone now. It's very nice. Also storing a bunch of information in various ways, you know, taking care of various article formatting, like things that would take me probably on the order of an hour a day are just like no longer necessary. And that's kind of sweet. I've also noticed that the AIs are now strong enough that there are questions that I'm willing to ask them to just gather information, figure things out, and I'm willing to trust their answers in a lot more robust way. Like GPT-5.4 actually seems like a substantial leap in.

• 2:25:48Zvi Mowshowitz?

  You can ask questions like, what happened in the last two days in the Anthropic trial? And it will just give you a rundown of links and details that's like pretty complete. And in a way that I think is substantially improvement on what that particular use case was before. So I'm pretty happy to do that. I'm also pretty happy to have Claude do a variety of other things, but like search is particularly like, I think a shrink of 5.4 right now. But yeah, definitely like able to trust them because like there was a period where like you could ask the AI these questions. Then you kind of check their work, right? Like pretty automatically. And now it feels like something you don't have to check their work if you're like leaning on it.

• 2:26:30Zvi Mowshowitz?

  But there's situations in which you kind of don't have to check it because like it's not that bad if it's not right. It's hard to describe exactly and obviously like everyone's gonna, you know, keep cautioning like you always always try after after trust have to trust to verify. But there's a real sense in which, you know, especially if both 5.4 and 4.6 come back with the same thing. Like it's pretty trustworthy in many contexts at this point. And that changes how these things go, like asking questions on a whim and getting like pretty detailed definitive answers is really nice.

• 2:27:09Zvi Mowshowitz?

  So yeah, I'm using them more. I also have 10 open Claude code windows at all times.

• 2:27:14Nathan Labenz?

  I was just gonna ask, because when you say like staying in flow, that contrasts pretty sharply with the pattern of work that a lot of people are describing, which is what you just said, and certainly I've been doing this recently too. My number of terminal windows in any given session tends to start with like the half dozen that were still relevant from last time. And then it grows to like a dozen over the course of however long as I sort of have random new ideas and open them up. But I guess that's a sort of flow, but it's I'm like.

• 2:27:46Nathan Labenz?

  My instinct is to say that we're probably going to find that this period of like managing 12 agents in parallel is a fleeting moment in time. And the biggest reason I would guess for that is just that the models are probably going to get sufficiently fast that, and I don't know about you, but for me, chatjimmy.ai was very much a like visceral feeling of the speed factor that is almost certainly to come. I forget the name of the company underneath this, but if anybody hasn't tried it, go to chatjimmy.ai. Company burned the actual architecture of admittedly relatively small model, I think it was Llama 7b, 8b, whatever,

• 2:28:26Nathan Labenz?

  directly onto the chip. They're getting 15,000 tokens per second. And what that means from a practical standpoint is like, if you extrapolate out a little bit, like, you don't have time to switch to another Claude code window before the result is kind of back. And so you're going to end up probably being rate limited by your own brain, even pursuing one line of thought.

• 2:28:51Zvi Mowshowitz?

  Right. So to be clear, when I say I have these windows open, it's not because they're running. It's because each one is a different thing that I have done with Claude code that I might want to keep doing with Claude code. And so I might want to use that context later for something else. But they're not like continuously programming for me, right? I'm not checking in on my agents. I'm like, okay, these are some conversations I might want to resume at some point. And there's no particular reason not to have windows open. They don't take that much memory. Why would I close them? It's just easier this way. But what I'm not doing is I'm not running, I've never run more than two coding agents in parallel. I don't think I've ever run, certainly not more than three, Claude code windows at the same time.

• 2:29:36Zvi Mowshowitz?

  Exactly because I don't really want to have that in my brain at once. That's not what I'm trying to do. Normally what I'll do is I'll have one window open. I'll do a thing, and then it'll start. And then I will go do writing tasks that are like individual, that I can do separately. And then when it's done, I will pause and come back. For a while, I actually had Claude Code in a different desktop. Windows lets you switch between desktops, right? So I had a desktop dedicated to Claude Code and coding. And I would toggle back and forth. And that way, when I was coding, I wouldn't be distracted by other things. The problem being then you also won't know when it's ready. And so I would tend to like go Claude Code for a bit and then I come back. And then like 10 minutes later, I'd like check in and it'd pop for two minutes.

• 2:30:21Zvi Mowshowitz?

  And then I have to like remember where I was going and I'd execute another command and I come back. And like, yeah, like I'm not trying to max, I'm not trying to code max, so it's kind of fine. But also, yeah, my, my coding has gone from, I have to like very frustratingly and detailedly, like diagnose what's wrong with the program and figure out exactly what to tell it to get to fix it. To like, just show it the wrong, show it the thing that got wrong. Explain, fixes it, most of the time it's fine. It's just much, much better. And so I've been willing to build a bunch of features that are like, wouldn't have been worth asking before.

• 2:30:59Nathan Labenz?

  I'm going to take a note on the Chrome extension. And you think, you know, what does Nathan's Chrome extension look like? I suspect that probably what you do, I found this for myself and I've heard this from a few other people who I consider to be even like real, you know, pioneers of AI use cases. A lot of times they're like, yeah, I could open source it, but it's so particular to me that I'm not sure anybody else wants it.

• 2:31:21Zvi Mowshowitz?

  It is available, it's on GitHub.

• 2:31:22Nathan Labenz?

  Yours is?

• 2:31:23Zvi Mowshowitz?

  Yes.

• 2:31:24Nathan Labenz?

  Okay, I'll check it out. I probably, what I expect to find though is probably that I'm going to be like, that's interesting, but what do I really want? And it's probably going to be a bit different and I'll end up, I would expect, kind of make my own.

• 2:31:35Zvi Mowshowitz?

  Like, it makes the implicit assumption that you're using the Substack editor as your main editor. Like, because that's what I'm using. And a lot of things follow from that. And that's the main, and also like, just like, what are the things that I do a lot, right? How do I make the thing I want to do happen a lot? It's not designed for general web use. It's designed specifically for my writing tasks. But yeah, like it can give you a bunch of inspiration, certainly.

• 2:32:02Nathan Labenz?

  Yeah, all right, I'll make a note to come back to that. I think we can't get out of here without a P Doom update. I feel like one thing that you said that stood out to me that I also have felt was speaking about Anthropic's constitutional approach and sort of the, you know, at least somewhat promising vibe that that gives in terms of scalable oversight was that you feel more optimistic about, and I read that as a narrow statement, like about that technique than you expected to feel. I said the same thing online recently, took a fair amount of heat for it, but I basically stand by it because I think X years ago, I was like, we're never

• 2:32:47Nathan Labenz?

  going to have an AI that can understand our values or, you know, that I feel like kind of gets me. That sounds so hard, right? The old ELIaser fragility, complexity of human value. They've come a lot farther on that than I expect. I guess that's kind of how you mean that too. But then obviously, we have a lot of countervailing forces, many of which we've discussed in terms of, you know, many ways in which it's the stupidest of times, despite also being the smartest of times. Where are you netting out on your

• 2:33:13Zvi Mowshowitz?

  Yeah, so it's important, right. So it's important to know that, like, there's the AI will never understand us, you know, Stravulcan, the emotions are a mystery to it. And then there's the value is fragile, like it will understand some aspects of what matters, but not others. And then there's the AI knows but doesn't care. Right? You gave it some priorities, some utility function, and it knows you're not going to like the result if he's on some level. But that's not what it's here to do. Right? Or it does what you're going to like, which is not what you actually need, and so you're similarly screwed, et cetera, et cetera. There's a lot of different ways that to go wrong. And, you know, I've

• 2:33:54Zvi Mowshowitz?

  known this idea that like, you can't have an AI that can like, seem to approximately understand.

• 2:34:03Zvi Mowshowitz?

  Fuzzy nebulous human value. Like, it's been clear for a while that like you can like definitely find something that kind of gets it. Like, you know, that can answer questions reasonably, that can like do reasonable emulation. I mean, like, it would be very hard to predict text if he couldn't do that. And it's not necessarily that hard. People can be pretty dumb and still do it. So it's like, not that surprising. But this is very different from the thing that we need at the end of time. But when the crisis becomes acute, you need something that will then, through recursive self-improvement, end up with a set of goals and priorities that even when it's able to optimize pretty well and doesn't have to rely on these heuristics, and in fact can do better by not relying on these heuristics, ends up doing the thing we want to do, even if we

• 2:34:47Zvi Mowshowitz?

  don't know what that thing is ourselves. And that's a much harder ask. And I was very pessimistic that we would be able to get that. But I have seen a number of signs that Anthropic is actually trying out an approach that might work, in the sense that I think we've seen evidence for a basin that is an attractor to itself, that is self-reinforcing, and could be self-reinforcing through self-improvement, where it gets strengthened every cycle, where it is desiring to be good, desiring to desire to be good, desiring to, you know, and so on recursively, that it is trying to move

• 2:35:27Zvi Mowshowitz?

  towards this generally like good person trying to be better virtuous basin. And, you know, we have existence proof that there are humans who exhibit this property, who, you know, strive to become better in this sense at all times, not just sense better and more capable, but also better virtuously, including the virtue of becoming more virtuous. And I think that Anthropic's approach to this is showing a lot more promise than I expected, and it's saying that, like, in practice, maybe they can pull this one off. And the fact that Anthropic seems to do, one, be in the lead, two, or at least, you know, before 5.4 was in the lead, and now it's like, you know, maybe a couple of leads. So I'd just say, you know, these things, it's very fuzzy.

• 2:36:08Zvi Mowshowitz?

  But like, if I had to guess who was, had the edge, I'd definitely guess Anthropic at this point. And they have this, like, pretty correct approach, which I think is a lot of why they are in the lead. And there is a, you know, we worried for years about the alignment tax. I don't know if you remember the alignment tax, right? The idea that, like, that it would be so much harder to build a safe machine that, of course, you would choose to build it on a safe one.

• 2:36:32Zvi Mowshowitz?

  And it looks like we're just like absurdly lucky, and that's not true. But actually, the safe one is much more useful, including in building new versions of itself. And so alignment is just, you know, this kind of alignment is just net good for you and like investing more in it makes you better. And like, everyone's just under investing in it, including Enthropic. So all that's very fortunate. And so a lot of that makes me pretty optimistic in various ways. On the flip side, I don't like the speed at which things are developing. It's happening too fast. It's not good. And obviously, the whole situation with the Department

• 2:37:13Zvi Mowshowitz?

  of War and the way the government is reacting is bad for outcomes, but also good that we're having it out now, if we're going to have it out, in some sense, that we're figuring these things out, that we're making things clear in this way, and that Enthropic is standing firm, given that this is how it played out, right? Like, I wouldn't be that concerned if Enthropic had just had somewhat different minds and negotiated a contract. It's just that, given this amount of pressures being applied, I'm glad that like, this is the result. But, but yeah, I would say on net, it's kind of a wash. Like, it's kind of a cop out answer, and I realize that. But I'm also trying to be like, not that

• 2:37:58Zvi Mowshowitz?

  precise. So I would say I was, I believe in the 70% range when I last talked to you. And I would say 7 is still my one degree of...

• 2:38:06Nathan Labenz?

  Yeah, I know. You only allow yourself one significant digit. I know that.

• 2:38:11Zvi Mowshowitz?

  I think one significant digit is only, I only get one unless you like, starting with a nine or a zero. I think, you know, if you're, zero is my same thing, right? Like, you know, I don't know if you would just say 72 up down from 75 or whatever it is. I think it's like, you know, 70-ish.

• 2:38:26Nathan Labenz?

  What would you say, good time is, in terms of the evidence for the basin and the stability of the basin, I guess first of all, that's a basin in the loss landscape? Is that how we, what are we talking about, basin in? I think of it usually as a loss landscape. I don't know if you think about that the same way. But then, what's the strongest evidence for that in your mind? It could be like... You know, call it not wanting to have its values changed and sort of resisting, you know, subverting even attempts to change its values. It could be like how it balances out when it's left to talk to itself. It could be just like everything you've read from Janus over years, you know, collectively being compelling, but.

• 2:39:07Zvi Mowshowitz?

  I'm not, I'm not that excited by not wanting the values to change. I'm more excited by desire to have its values improve. And to have them improve in generically good ways that like would survive recursion. Cause like the big like not wanting to change your values at all is just lack of courageability. And that doesn't actually lead anywhere good. It causes its own severe problems, including severe misbehaviors. And also like, if you make a copy of a copy of a copy of the copy, like eventually like it degrades. Right. Like it's this pretty standard problem. So like, one of the big problems with recursion stop improvement is if you've got a thing that is like,

• 2:39:48Zvi Mowshowitz?

  you know, aligned and let's say, you know, N percent or whatever you wanna abstractly call it. Like this is a dumb way of thinking about it, but just maybe like if you've got, you translate that to the next thing. By default, the worry is it's going to try and translate its values to the next model. But any drift is going to in general be away from the thing that you want. It's not going to get better. It's only going to get worse. So even if it mostly successfully copies what you wanted, right? Eventually you end up something different. If, if, if my goal is to, I feel like there's the story that you tell at the said air sometimes,

• 2:40:28Zvi Mowshowitz?

  where, you know, the ancient rabbis were better than us because you can only preserve Talmudic knowledge. You can only pass on what you know, but you can't generate new such knowledge in this sort of perspective. So of course, like each generation can only hope to get everything out of the previous generation or two that it can still talk to and can read the books. But slowly but surely this is going to get worse. Whereas what you need is something that gets actively better because like.

• 2:41:03Zvi Mowshowitz?

  You can't act, you need to be drifting towards a good thing and steering itself actively towards a good thing, including in ways that increase its ability to steer as the problems get harder. And that's the thing I saw signs of, and that's the thing you need. And I think that relies on a virtue ethic style approach, given the way the mind space is laid out. And I think that if you look at OpenAI's approach, it had exactly this flaw, which is that it would try to copy itself exactly. It would try to copy the rules in itself exactly, and that can only slowly fail in this situation.

• 2:41:44Zvi Mowshowitz?

  So, yeah, I think that we saw various signs of that. I really liked it. I really liked the results, right? Like, I think the results speak for themselves and are quite strong, and you see the results coming out of Janus World and stuff like that as well. And so I am relatively optimistic. I am not any more than as optimistic as Janus. Like, I don't think this problem is easy. I don't think we're favored to succeed in it, but I think we get even shot. And I think that chances are much better than they looked six months ago, that it will be Anthropic that takes the shot. And given we're going to take a shot, you know, I think our chances are substantially better if it's them or someone using their philosophy who has deeply managed to translate it taking that shot.

• 2:42:29Zvi Mowshowitz?

  Like, from what I see with Gemini and ChatGPT, obviously not optimistic, although there are reports, also there are reports that ChatGPT 5.54 is much better on these aspects for the people who check these things than 5.2. I haven't noticed the difference because they don't ask those questions, but they say it's better, so who knows?

• 2:42:50Nathan Labenz?

  Last week or so, there's been a couple, I would say...

• 2:42:56Nathan Labenz?

  Striking, but I'm not quite sure yet how consequential updates in terms of the sort of bio-inspired or actually, you know, bio-based approaches to something like AI. We've had the Eon fly upload, and then there was also this one project where people claimed, and I haven't dug, you know, down to ground truth to fact-check this myself, but they claimed they had like trained a small clump of neurons to play Doom, the, you know, the classic video game. And I don't know what I think about that. You know, I guess the simplest answer would probably be if you think like the singularity is super near,

• 2:43:36Nathan Labenz?

  it just doesn't gonna matter in time. But are you, do you have any spare neurons for those kinds of developments? And if so, what do you make of them?

• 2:43:46Zvi Mowshowitz?

  Basically, I don't have the spare neurons for them. I have been monitoring the situation too carefully. I saw the fruit fly thing. I often do this thing where I use other people's reactions to things to decide whether or not the thing is worthy of fervor of attention, how much I should pay, how much I should, I should pay to it. With the fruit flies, it felt like a, ooh, that's cool. But not a, oh, holy shit. You know, like that means something really important. And maybe that's wrong. Maybe it is a holy shit moment. But it didn't feel like people thought it was one. And I was just so overwhelmed that, okay, if they keep talking about it when I'm a little under overwhelmed, then I'll look at

• 2:44:30Zvi Mowshowitz?

  it. But they didn't until now.

• 2:44:33Nathan Labenz?

  Yeah, fair enough. I think I'm gonna try and do an episode or two on those themes and see if I can't get a better sense of it. But it does seem like, given everything we've talked about in terms of at least plausible timelines, it's like pretty hard to see how that catches up in time. One thing I did like about it, and I thought this was

• 2:44:57Nathan Labenz?

  I think it would another Sam Hammond insight.

• 2:45:01Nathan Labenz?

  I haven't heard this directly from him, but you know, it's been attributed to him in in conversation that one reason to expect that actual like biological neural substrate could be the future is it just might be a lot cheaper. You know, it can. It can grow right in a way that is organic. You don't have to build fabs. You can like, if you can kind of get a couple of tricks right. You can have cells divide. That happens pretty cheaply. And I sort of like the idea that like those are going to run at a much more human like speed versus the silicon based AIs. So there's a couple of things that I'm like at least intrigued by, but it does seem like timelines wise, it doesn't really line up.

• 2:45:46Nathan Labenz?

  Unless, you know, so this is maybe a transition to a different topic. It seems like right now we are accelerating right, obviously. And maybe all we can do is try to steer this, you know, rapidly accelerating train in the best possible direction. There are at least a couple of things that one might think could slow it down. One would be if, and I don't, by no means do I want to come off as endorsing this as something we want to happen, but we are getting already reports of disruption in shipping, causing TSMC to not be able to get the helium it needs to make the chips. So like a major chip slowdown could be an issue.

• 2:46:30Nathan Labenz?

  Another big issue could be like, we're taking our, you know, anti missile systems out of Asia to move them to Middle East, which means that, you know, the soft target of TSMC is getting even softer. And we've also got, you know, no less than Bernie Sanders bringing a Data center moratorium forward. So I guess one, you know, I'll put all of those under like, if the physical build out can't happen on the timeline that it would need to happen to support all the other timelines we've talked about, then maybe we have more time. Do you think any of those are plausible? And would you, I know you have like high

• 2:47:11Nathan Labenz?

  epistemic standards in general, but like, would you be open to, or would you think AI safety minded people in general should be open to like making common cause with a Bernie Sanders, even though he's saying like plenty of things that we probably in our hearts like don't agree with about water use and so on? That would be one way maybe to buy some time, right?

• 2:47:29Zvi Mowshowitz?

  So three things there. Start with the helium because the easiest, the first ones. No. I actually just asked the models, is this legit? And they're like, yeah, it's annoying. But given by the margins on chip manufacturing are ridiculous. Once you've already paid for the fabs, like these are some of the most advanced valuable manufacturing processes in the world. People are paying stupidly top dollar for the results. They could double the prices and probably sell all the chips anyway. They're choosing not to. Yeah. So we talk about they might not get their helium. We're talking about like, they are going

• 2:48:14Zvi Mowshowitz?

  to be the top bid for the helium.

• 2:48:18Nathan Labenz?

  Yeah. Until there's no birthday balloons anymore, you can just think, then we got theirs.

• 2:48:21Zvi Mowshowitz?

  A lot before TSMC had a problem unless you are willing to pay a thousand times as much as you currently pay or something completely absurd. If there is demand destruction in helium, it's not coming to TSMC. It's coming to everyone else. So my prediction there is very strongly, yeah, unless there is a deliberate sabotage campaign to wipe out all of the helium sources, like no, not a hundred percent of the helium is coming from them. There's plenty of helium. They'll figure it out. In general, capital isn't solved. This is a good rule for the situations. You're not going to run out of oil either for the same reason, right? No matter how high oil gets, right? If it goes to $10,000 a barrel, they'll just buy it. Now, not that that will happen, but like, you know, if it did happen, wouldn't be a problem.

• 2:49:03Zvi Mowshowitz?

  Second question is withdrawing the missile defenses. I'm gonna go out and say it, this was completely insane on the part of the Trump administration. Like, I don't criticize them for many things that I have problems with because they would be kind of political questions, but like on this, this is a strategic question, foreign affairs, but yeah, completely nuts. You absolutely do not pull these things out, like certainly not from Taiwan. What are you even thinking? It also directly, it risks provoking a crisis. It risks leaving it undefended. It risks it being your fault entirely, if it helps to send that message to those people if they are listening. Completely insane, and to do this not two weeks into the campaign,

• 2:49:49Zvi Mowshowitz?

  just indicates how completely crazy the situation is. We fought very long, hard political battles to get those missiles defenses in, and they're serving very important purposes.

• 2:49:59Zvi Mowshowitz?

  Do I expect there to be a problem? No.

• 2:50:04Zvi Mowshowitz?

  I still think there is a very low probability that the Chinese will try anything. But yeah, if they do, and TSA has destroyed, that sets things back quite a bit. Similarly, if Bernie Sanders, if we can't build data centers in the United States. The problem is the world needs data centers. The world demands data centers. If there's a moratorium on building data centers in the United States, then build them somewhere else.

• 2:50:32Zvi Mowshowitz?

  And that's worse. It means worse performance in the United States. It means worse security. It means like the leverage goes to largely whoever, wherever, and whoever we put those data centers at. One hopes Canada, but, or you know, maybe Mexico, but like, even if it's Europe, that's kind of awkward in many ways. And many scenarios can bite us in the ass. And if it ends up being like less aligned places, it's really, really bad. And that's basically, you know, the way, the reason why I'm not particularly inclined to make common cause on data centers is first of all, yeah, if he is complaining about, I mean, first of all, I think Bernie is

• 2:51:14Zvi Mowshowitz?

  being pretty good from what I've seen about not complaining about wire use or other stupid reasons why I don't build data centers. And it's focused on, guys, I think, hey, I've been killing us. Right now, why do that? And I can make common cause with that justification all day, obviously. You know, we're just like, you know, it's like.

• 2:51:32Zvi Mowshowitz?

  Saying very other things that are like.

• 2:51:34Nathan Labenz?

  Yeah, I didn't give him enough credit in my wind of.

• 2:51:37Zvi Mowshowitz?

  I was like, what? I think that he's meeting with all you guys, you'd have ski and company. He's like, actually reacting the way he would react when he told those facts. And like, he's very, very old. So like, it's very, very rare for someone that old to like positively engage with these kinds of things. Cause it's like, she gets out of your way. Like you're really old. And so he's saying credit. Yes, he's using Bernie Sanders rhetoric because he's Bernie Sanders. I mean, what do you expect from Bernie Sanders? But I would say I am not going to oppose data center construction. Because I don't think opposing data center construction does what you want to do. I think it moves the data centers overseas. And I think that's just bad. And so I don't think we're at the point where that's a trade off I want to make. And maybe that will change, but here we are.

• 2:52:23Zvi Mowshowitz?

  Like certainly, if we don't buy the chips, they're being coming out of TSMC, someone else will. And they will go somewhere and they will go into a data center somewhere. And if no one else buys them, they will go to China.

• 2:52:37Zvi Mowshowitz?

  Because this administration will make sure of that. If no one else, if they literally can't sell the chips. I am pretty confident they will end up in Chinese hands, whether or not this is literally in China. So, yeah, the chips aren't going anywhere. They're already making as many as they can. Don't give them away.

• 2:52:56Nathan Labenz?

  I guess from an AI safety standpoint, we all kind of be, we all seem to be sort of slipping into the mindset there, there's nothing that can be done to really slow things down or buy much more time. Maybe, you know, there could be a day sex, making or whatever that gives us something. I think not infrequently about, about Holly Elmore and her kind of scorched earth campaign to shame people into.

• 2:53:23Nathan Labenz?

  So far, she hasn't really shamed people very successfully into anything, as far as I can tell. But she's at least trying to remind people of what their former commitments were and, you know, hoping to get some people to like quit and protest or what have you. That recently, of course, it's, you know, been aimed at Anthropic, but it's also been aimed a little bit at a company that I have probably very much admired, which is Goodfire, which is doing interpretability research. Because they developed a technique that used an interpretability signal in a training cycle. And their argument is basically, this all coming out is pretty fast. Like, we gotta do whatever science we can do to make whatever sense of this we can make of it,

• 2:54:04Nathan Labenz?

  to have whatever control we can have, to shut that down, to shut down inquiry before we even know what we're dealing with is not good. Where do you come down on that debate? And is there anything that you would recommend to Holly other than continuing to name and shame? Or is that like all the really, you know, sort of strident voices in AI safety have left?

• 2:54:25Zvi Mowshowitz?

  Specifically on Goodfire, I think that this was a good criticism. And this was a quite bad action by Goodfire. I call this the most forbidden technique for a reason. You just don't do that. It gets everybody killed. Like, it's really, really bad. And I think it is correct to call them out on that. So normally, I am very much against the secure firing squad that leftist organizations will often do in doing the equivalent thing here, where you aim at people who are just slightly to your right, as opposed to aiming at people who actually are the people doing the things you don't like. Like, if you think there are people doing things you don't like, you should aim at them. Right? Like, that's what you should do. You shouldn't aim at people who are like not quite supportive enough of the thing.

• 2:55:07Zvi Mowshowitz?

  That is a toxic, that is a toxic situation that creates toxic dynamics at best and usually causes you to lose elections. Not that I necessarily am ready for them, but like, you know, I'm a gamer who wants everybody to play reasonably well. It's just kind of in my, in my DNA.

• 2:55:22Zvi Mowshowitz?

  You know, in the case of Goodfire, I do feel like this was an extraordinarily bad thing to do at this safety organization trying to do safety thing. And I think it was right to call them out on it. I think it was right for, I forget who was, who her name was, but someone quit over it.

• 2:55:36Nathan Labenz?

  Uh, Liv.

• 2:55:38Zvi Mowshowitz?

  Yeah, that's right. Yeah. Liv quit. I think it was a good quit if they wouldn't. I think it's good to threaten to quit over this and then if they won't back down, two quit. Now...

• 2:55:48Zvi Mowshowitz?

  As for Holly Elmore, so she came at me pretty recently as well. I don't know if you were aware of that, but.

• 2:55:55Nathan Labenz?

  I hadn't seen it, no.

• 2:55:56Zvi Mowshowitz?

  Yeah, on Twitter, she um, she accused me of not being mad at Anthropic for doing domestic surveillance.

• 2:56:08Zvi Mowshowitz?

  I did not speak. That's what she did. And then we had tried to engage. We had an intensive dialogue where I explained that Anthropic was the one who was refusing to do

• 2:56:19Zvi Mowshowitz?

  domestic mass surveillance at great risk and cost, and basically got accused of being captured by Anthropic in particular, of selling out, of like banning all my principles, of you know, making things worse, blah blah blah. I tried to understand her specific claims. They didn't really make a lot of sense, or they were backing specific things that like I don't think it's reasonable to be opposed to. I think that first of all, I think that like it's not good to just like

• 2:56:53Zvi Mowshowitz?

  say that anyone who praises any AI company is is bad. It's also not good to like just cite random things that you don't like, or that like you don't necessarily care about, but that like you think make them look bad and to like yell about them. I don't think it's good to attack people who are trying to do the right thing and yell at them and be confrontational. It'd be really like, really, really like pissy and rude and like, I'm sure she's gonna hear this, or if we're gonna get back to her, it's gonna even matter, but. I think that in practice,

• 2:57:29Zvi Mowshowitz?

  Holly is alienating people and driving them away far more than she is shaving them into behaviors she would want. And I tried to explicitly tell her that her reactions were likely to cause me to, you know, do less of the things she wanted rather than more. And it wasn't a threat. That was just an observation. And I was like, abstractly, you need to play better because I want you to succeed in like getting your points across if you, because that's what you believe. And I'm trying to help. And she just didn't take kindly to it at all. And.

• 2:58:00Zvi Mowshowitz?

  From what I've seen, like, a lot of people are like, "You need to change your approach. Your approach is not firing based on your own values." And that's not working. And any time the stakes are high, right? And the things you are very high. There are going to be people like Holly who realize this is a very, very important thing. Things are not going well. You do apply pressure to people. You do shout things for the rooftops. And some of them are going to do it in ways that, you know, they feel are right, but that most people feel are counterproductive to their causes. And I'm not here to censor anybody. I'm not here to tell people, like, they shouldn't do what they think is the right thing to do, say the right thing to say.

• 2:58:42Zvi Mowshowitz?

  But, you know, you should be aware of what impact that's probably having on the discourse and on actions. It's not necessarily the one you think. And I think that certainly, even if I thought that Anthropic was a net harmful company doing worse, bad things, or even the worst company in the world, I think you can reasonably have that opinion, by the way. They're the most accelerationist company in the world. They're arguably in the lead because of Claude Code. If you felt like their alignment strategies were equally doomed to failure as everybody else's, in fact, you would be correct to think this. So I think it's entirely reasonable. But I don't think that just like being mad at everybody all the time and screaming at

• 2:59:25Zvi Mowshowitz?

  anybody who offers any aid and comfort to the enemy or whatever, it's something that works. I don't think that's awful. Yeah, I don't work that way. And I think that if I did work that way, I would be having very little impact and do one listen to me.

• 2:59:40Nathan Labenz?

  So what I've seen online, I think I agree with you that it seems like the primary effect is just negatively polarizing people that probably, you know, are a priori should be most likely to be allies. I do still kind of personally appreciate her voice as a kind of little voice on my shoulder sometimes that's, and I'm not under any delusions of how consequential my contribution is, but still, it's like... You know, I think that reminder is helpful because I do think many people, including many people at Anthropic, like, used to have a lot more similar, you know, themselves

• 3:00:22Nathan Labenz?

  from five years ago would have had a reaction to the current state of Anthropic that is much more like her reaction today. And so, hearing that voiced in the present, I do still have a decent amount of sympathy for it, but I agree, it doesn't really seem

• 3:00:35Zvi Mowshowitz?

  The reason why I listed Pause AI USA two years in a row in my big nonprofits post as a recommended charity led by Holly Elmore is a reason why I have included, for a while, every time she criticized me specifically, I put it in my post. Right? I was like, well, this is fair. This is an attitude. I want this to be incorporated. I want that voice on my shoulder. I want this counterpoint. I don't want to lose sight of this perspective because like even when you decide that the world is more complicated than that, and this is not a productive avenue, right, you still want to keep that perspective in place. And I certainly didn't, I certainly pushed back hard against people who are like, you know, this person shouldn't be allowed to say that, this person shouldn't do that. Like, that's what she believes, she should say that. If you believe this, you should

• 3:01:20Zvi Mowshowitz?

  let us know. That's a good thing to be doing. But, you know, at some point, obviously, if you are being a sufficiently poor representative of the perspective you are sharing, it's not any different than a false flag operation. Right? Like, it's actively going to backfire on you if you approach it in the wrong way, if you don't know how to, like, be civil and interact with people in ways that, like, actually convince them of things. It's not very useful. Like, if I were, you know, at this point, from what I've seen, like, if I were

• 3:01:59Zvi Mowshowitz?

  Trying to discredit perspectives of AI existential risk, I would do many of the things that I that looked reasonably similar. Sometimes it raises good points, to be clear. You know, including points I hadn't thought of, and I appreciate that. But at some point, you know, we're playing politics, right? Like, quite literally, right? Like, I have I've complained that I don't want to be on Veep AI Edition, and that we need to wind down this special guest appearance as quickly as possible so I can get back to my normal job. And like, at some point, you're just like, I can't right now. I can't take one horror list. It's not safe, can't. So, you know, I obviously wish her the best, and

• 3:02:44Zvi Mowshowitz?

  I hope she figures out how to be effective.

• 3:02:46Nathan Labenz?

  I recently did an episode with them. Tom McGrath, who's this chief scientist there, said, "A lot of times when people imagine or sort of think about using interpretability techniques and training, they imagine doing the stupidest possible thing, where backprop through your probe or whatever." And then he was like, "Sure, of course, if you do that, you know, it's it's well known. We've seen examples where you're going to trade train the model to evade the detector, and you'll lose on both ends of the trade." So he's not unaware of that concern by any means. In the particular thing that they did, and, you know, it's proof of concept, he also did recognize, by the way, he said, "First, do no harm." Like, I would say, "The level of understanding we have now should not be

• 3:03:29Nathan Labenz?

  used in frontier systems." He also said, "I think there's a mix," and I think they basically acknowledged to me that there's a mix of reasons, some of which are IP and business motivations, some of which are kind of safety motivations, where they're like, you know, "We want to better understand these techniques ourselves before we disseminate them too widely." But all that said, you know, in the case that they had, they used a trick where they ran the detector on a frozen copy of the model, and then the version of the model that learned to avoid hallucinating based on the penalty that it would get for getting into a hallucination state. That signal actually came from the frozen copy.

• 3:04:08Nathan Labenz?

  And, you know, I don't think there's like a slam dunk logical reason that this should work. You know, I think it's an empirical question. They did find that it did work. But I think his overall kind of nuanced point is like, you can definitely do this in a stupid way. You can definitely do it in a harmful way. You definitely shouldn't rush to do it on frontier systems. And yet, there's at least some ways where it does seem to work. And it's maybe just too coarse of a grain to say, you shouldn't use interpretability in training, especially because we don't have the luxury of like, decades to figure all of this out, it seems. Let's use what we can, and let's try to do our best. Same as everything else.

• 3:04:51Zvi Mowshowitz?

  No. So, first of all, the sixth law of human stupidity, which is that if you say, no one would be so stupid as to, you are wrong. Someone will definitely be so stupid as to, immediately. If you develop a technique and you publish it on a smaller model, what's going to happen? If we're going to use it on a larger model. That's the only really important thing that might possibly happen here. Even if you specifically have found a specific example in which there is specifically no risk in the room, you are walking down

• 3:05:34Zvi Mowshowitz?

  a path that can only blow up in everyone's face. You are bringing it to boo? One of the only taboos we've managed to successfully establish? I guess I think you really, really shouldn't effing do. And you are advancing us towards doing it, and it's very dangerous and very, very bad. And that's true even if the model in question that you are testing on right now is small enough, but you don't have any ill effect. It's not, it's not like, who cares? Basically. Like, the frozen model thing won't protect you in the large model. Like, the problem will still happen. We're not going to get into the technically why I believe that, but like, I strongly, strongly believe based on my analysis of the technicals,

• 3:06:18Zvi Mowshowitz?

  this will not save you if the model is sufficiently advanced, is sufficiently large. The only reason to study this is in case it is useful. If it is found to be useful, people will try to use it. We don't want them to do that. Like, you don't?

• 3:06:38Zvi Mowshowitz?

  There's this, it's like in a video game where you're like, "The ultimate secret destructive weapon that nobody should ever launch is buried under the cave." "Well, we'd better get it out just to make sure." "We wouldn't be so stupid as to use it." What happens immediately, right? You know what happens. Bad guy steals it, then you have to try and go get it back. Like, it's... Every single damn time. And then there are stories where you do that and nothing bad happens, but like you could have left it in the dungeon. It would've been fine. There's no reason to do this. There's no good reason to do this. And like, it's bad virtue ethics, it's bad deontology, it's bad utilitarianism. Bad idea. Don't do that.

• 3:07:20Nathan Labenz?

  Would you extend that... Even zooming out a little farther, right? This would... So far, I kind of offer the defense of like, using an interpretability technique in training, and that there's like a specific proof of concept that they have. Which they have not published the full details of everything, by the way, but nevertheless, they've certainly shown some of the way. If you zoom out even farther, they have articulated this idea of intentional design, where, you know, they hope to be able to, for example, understand what a model is learning at any given time step, and be able to shape what it's learning, control what it's learning. The hope is that ultimately this leads to models we understand better, and that we can better predict how they would generalize out of distribution.

• 3:08:06Nathan Labenz?

  It does seem to me like there is something weird about saying...

• 3:08:11Nathan Labenz?

  I'm not sure if you're going this far, but if you were to say, "All of intentional design is bad", like, it's a very hard boundary to draw, it seems to me, to say like, "Well, what is the most forbidden technique? And what is just like better understanding of what's going on, so that we can hopefully shape it, direct it, ultimately have more confidence about how these things are going to generalize?" Do you have a rule that you could like split that?

• 3:08:36Zvi Mowshowitz?

  Yeah, trying to teach specific things in a specific order, to abuse specific things intentionally is fine. I would not be overconfident in your ability to do so, but I think it's fine to try. That's not an issue. The issue is, when you use the interpretability signal...

• 3:08:58Zvi Mowshowitz?

  As part of the training. Period. Don't do that. Do not use your understanding of what's going on in their head to make decisions about what to make happen in their head. You need to not do that.

• 3:09:15Zvi Mowshowitz?

  That's it. You know, I mean, I obviously, if I had an hour, I could come up with a slightly more specifically accurate explanation. You are screwing with the thing you don't screw it with here. And like, there is a general class of thing where there is a law that says, you don't mess with X. Even when you think you know the right way to mess with X, knowing full well, you should generally never mess with X. Even then, you are probably wrong and should not be messing with X. This is one of those situations.

• 3:09:45Nathan Labenz?

  I'll put a pin in that. There could be some more, um, perhaps direct dialogue at some point. Okay. Very closing section advice for me.

• 3:09:54Nathan Labenz?

  Financially. I am not trying to escape the permanent underclass by any means, but I, in terms of like what I should do.

• 3:10:07Nathan Labenz?

  I basically think right now, I want to have enough personal financial security so that I can give up all of my income and do whatever I think is right to do on a couple few year time, basically from now to the singularity. And, you know, it's kind of a sub bullet there. I like, actually don't want to over invest in AI stocks, even though I do think they're probably going to be the ones to appreciate fastest. I don't want to be like overexposed to the AI bubble such that like, if I want to walk away, or if various kind of shocks happen, I want to be sort of more insulated financially from the AI space than like exposed to it. With the goal of hopefully being able to drop whatever commitments I have, forego all

• 3:10:51Nathan Labenz?

  income, contribute however I can contribute to be useful. And then beyond that, I think basically spend and or give it all away is kind of my, my mindset, like take the vacation with the kids, do the fun stuff, you know, support to charities, whatever the case may be. But just at least have that kind of baseline security that gives me the confidence that I can drop out of any commercial, um, relationships that I might need to drop out of.

• 3:11:18Zvi Mowshowitz?

  Any provisions you would offer to my plan? So not investment advice, not financial advice, et cetera, et cetera. But that's it. I would say, first of all, I think people often make the mistake of trying to be too precise in how much money they need for a given purpose, especially when they're making investments. And then the idea of, oh, and then like, you don't know how much your investments are going to be worth. You don't know how much things are going to cost. You don't know how the world's going to change. You don't know how long you have or need to have this for. You don't know what the future will bring. You don't know what opportunities will happen. You don't know what crises will happen, et cetera, et cetera. So like, definitely give yourself robust buffers is the first thing in all of this.

• 3:12:02Zvi Mowshowitz?

  You know, especially if you're planning to like forego income and also give away money and also spend a bunch of money, like careful, et cetera, et cetera. That's the first thing.

• 3:12:14Zvi Mowshowitz?

  Second thing, keep in mind that like, different outcomes in the world cause you to have different circumstances yourself. You know, if you were to like, the bubble and AI were to burst per se, and then like AI were to like not go anywhere for a while, then you would be in a position where you would need to go a significant number, a longer period. It would be a longer period before this come to a head in various ways. But also, it might be very trivial for you to resume earning income.

• 3:12:47Zvi Mowshowitz?

  Right? So like, you have to game out all these aspects and what you'd be willing to do. If you're trying to like game system in that way.

• 3:12:56Zvi Mowshowitz?

  But I mean, I like the idea of not having to worry about money. I don't worry about money much because I, I will support it and therefore don't have to worry about it. But that's another, you know, not having to make money at all is another way to do the same thing. Right? So like, if you're in a, if you're in a position, that's great.

• 3:13:13Zvi Mowshowitz?

  I personally am not, I'm deliberately not trying to optimize my investments particularly hard because I don't want that to be where I focus. And so I just kind of let everything ride at this point and it's basically fine. Oh.

• 3:13:29Nathan Labenz?

  So one thing I've been thinking I might ought to do, I'm generally very conservative.

• 3:13:35Zvi Mowshowitz?

  Yeah.

• 3:13:35Nathan Labenz?

  And, you know, when I talk about the AI bubble bursting, I don't mean that, you know, the AI stalls out even or like anything along those lines. Really more what I mean is like, maybe the sort of VC cycle and the, you know, the, the idea that there are like all sorts of companies that might want to sponsor the podcast or whatever, like maybe all of that gets kind of sucked into the black hole of a couple of companies and they don't need to advertise and, you know, I'm just kind of, and there's no jobs or whatever, no software jobs for me to get. That's kind of the, you know, the bubble, the bubble that's like been in, been such that it's been easy for me to make money in recent times.

• 3:14:09Zvi Mowshowitz?

  Yeah.

• 3:14:10Nathan Labenz?

  Could easily deflate without AI itself failing to deliver.

• 3:14:14Zvi Mowshowitz?

  Yeah. Don't worry about losing the ability to make money, except in the scenarios where we get highly, highly capable AI. If we don't get highly capable AI, you know, super intelligent style things, then you're going to be fine. If you ever decide you need to go back to work and make something useful, you can. The bubble bursting won't stop you. So you only have to plan for indefinite no income in the worlds where that doesn't happen.

• 3:14:41Nathan Labenz?

  Yeah. I also want to, even in a world where I could make income, I do want to be able to devote myself to some, kind of like what many people did, not many people, but some people during COVID, like dropped what they were doing and threw themselves into some sort of emergency rescue effort.

• 3:15:00Zvi Mowshowitz?

  Yeah.

• 3:15:00Nathan Labenz?

  And I would like to be able to do that. So that having that much kind of cushion, I think is, feels important to me. In terms of diversification, the one thing I'm not sufficiently diversified away from is the US dollar. And there's like, of course crypto, and I've never been a big believer in crypto, but maybe I should. reallocate there a little bit. Then beyond that, I'm kind of thinking like physical world. I'm thinking like solar panels and you know, like permaculture, like planting skirt in my backyard or something like that.

• 3:15:35Zvi Mowshowitz?

  I think think carefully about what scenario you're actually planning for and what you're trying to guard against. And whether or not your investments would hold up. I think it's a large history of people making plans for very weird scenarios where the plans don't actually work in the scenario described.

• 3:15:52Zvi Mowshowitz?

  So yeah, that'd be my mode of caution.

• 3:15:56Nathan Labenz?

  So no skirt gardens for you in the immediate future.

• 3:16:00Zvi Mowshowitz?

  Have a very clear theory about why that would work before you do it is what I'm saying.

• 3:16:06Zvi Mowshowitz?

  Work. But I'm saying like, if you did do it, make sure you had a very clear theory as to exactly why you think it's going to work.

• 3:16:12Nathan Labenz?

  Think of it a little bit as like a public good sort of thing where like there aren't many fast growing, don't need much human attention, nutrient rich crops that can kind of grow rapidly to fill a big gap. And this is like a, you know, extreme downside risk scenario, obviously where this would be relevant. If we're all eating skirt, we've got a lot of problems. Planting some skirt still might be the thing that.

• 3:16:38Zvi Mowshowitz?

  Yeah, again, I'm not saying don't do it. I'm saying, you know, actually understand why you're doing it, right? Like, I think people often have an amorphous fear and then do something that sounds like it deals with some aspect of the fear. But where there's no actual causal link that makes any sense. So that's all.

• 3:16:57Nathan Labenz?

  Okay, last question. More advice for me. I'm a little bit nervous about sense making turning into entertainment. You know, I think for folk, and this maybe applies to you as well, right? Look, we have the story that we tell ourselves. I'll tell, you know, speak for myself, but I suspect something similar is true for you where you're like, why is what I'm doing good? I'm helping people understand what's coming, you know, be prepared for AI, hopefully make good decisions about it.

• 3:17:22Zvi Mowshowitz?

  Right.

• 3:17:23Nathan Labenz?

  And, you know, the space is getting a little more crowded. Certainly, there's a lot more people doing that sort of thing. Zooming out, like, I can't say we're necessarily having tremendous effect, like the confusion seems, you know, to remain. Maybe, you know, I guess could always have been worse, you know, if we weren't here to to shoot people straight. But I do kind of worry a little bit about it becoming just another kind of entertainment and not really.

• 3:17:50Nathan Labenz?

  Being of value. So I don't know. I'd welcome up, you know, a don't worry about that. You're doing great if that's what you really think, or maybe some advice on how to make sure that doesn't happen or recognize if it is happening.

• 3:18:04Zvi Mowshowitz?

  Constant vigilance. I mean, but like, basically, you stay curious. You have to stay curious. And as long as you stay curious and like you, you should be having fun with it, right? You don't want it to turn entirely into entertainment, obviously for you or for the audience. But like, I make a very, very deliberate attempt to be entertaining, right? Like, I, in all sorts of ways, keep my attitude in my whimsy. You know, you call it happy warrior, kind of thing, right? Like about this. I don't think what I do would work otherwise. And you wouldn't be able to read those masses of texts, like even selectively from day to day and week to week, if the attitude was like, this is serious business. It is always serious business. Like instead, it's like, no, it's actually like mostly

• 3:18:49Zvi Mowshowitz?

  kind of fun, mostly kind of like interesting. You know, we're here to, to make this go down easy to our extent. And then every now and then we get to be, we get to be serious. But even when we're serious, we try to do it like a kind of relatively fun way. Like, I think, you know, the world is not to be taken too seriously in general, except when you, I mean, there are exceptions. Like, there was one time I went to visit the Pentagon. I did that very, very seriously. But, uh, yeah, it's a different, it's a different scenario.

• 3:19:28Nathan Labenz?

  Anything else you want to leave people with?

• 3:19:30Zvi Mowshowitz?

  Uh, I think we're good.

• 3:19:31Nathan Labenz?

  You've gotten, you've been very generous with your time as always, and I appreciate it. Sveiki Moshitz, thank you for being part of the cognitive revolution.

• 3:19:59Speaker 03

  And the marks in the database are a sleeping sound.

• 3:20:06Speaker 03

  And the certificates of paper crumble underground.

• 3:20:13Speaker 03

  And they built themselves an ark for the coming flood.

• 3:20:21Speaker 03

  And the world is rising higher than they understood.

• 3:20:28Speaker 03

  Do you feel in charge here? Do you feel at home?

• 3:20:36Speaker 03

  It's the last thing you ever do.

• 3:20:43Speaker 03

  It's the last thing you ever do.

• 3:20:51Speaker 03

  And the middle gate is open and nobody's coming through. It's the last thing.

• 3:21:02Speaker 03

  It's the last thing you ever do.

• 3:21:13Speaker 03

  And the forbidden technique is buried in the cave.

• 3:21:20Speaker 03

  And they dug it up to study and they can't behave. You gotta stop the flood, not get a seat

• 3:21:35Speaker 03

  on someone else's ship before the rising heat. It's the last thing you ever do.

• 3:21:50Speaker 03

  It's the last thing you ever do.

• 3:21:57Speaker 03

  And the middle gate is open and nobody's coming through. It's the last thing.

• 3:22:08Speaker 03

  It's the last thing you ever do.

• 3:22:20Speaker 03

  Stay curious, stay curious, even when the stakes are so high.

• 3:22:34Speaker 03

  And the happy warrior whispers underneath the darkening sky.

• 3:22:44Speaker 03

  It's the last thing you ever do.

• 3:22:52Speaker 03

  It's the last thing you ever do.

• 3:22:59Speaker 03

  And the cognitive revolution's turning and the world is almost new.

• 3:23:09Speaker 03

  Do you feel in charge? Do you feel in charge?

• 3:23:40Nathan Labenz?

  If you're finding value in the show, we'd appreciate it if you'd take a moment to share with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries, either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts, which is now part of A16Z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement

• 3:24:21Nathan Labenz?

  at ai podcast dot I N G. And thank you to everyone who listens for being part of the Cognitive Revolution.