Dario Amodei — "We are near the end of the exponential"

Dario Amodei thinks we are just a few years away from AGI — or as he puts it, from having “a country of geniuses in a data center”. In this episode, we discuss what to make of the scaling hypothesis in the current RL regime, why task-specific RL might lead to generalization, and how AI will diffuse throughout the economy. We also dive into Anthropic’s revenue projections, compute commitments, path to profitability, and more. Watch on YouTube https://youtu.be/n1E9IZfvGMA; read the transcript https://www.dwarkesh.com/p/dario-amodei-2. Sponsors * Labelbox https://labelbox.com/dwarkesh can get you the RL tasks and environments you need. Their massive network of subject-matter experts ensures realism across domains, and their in-house tooling lets them continuously tweak task difficulty to optimize learning. Reach out at labelbox.com/dwarkesh https://labelbox.com/dwarkesh. * Jane Street https://janestreet.com/dwarkesh sent me another puzzle… this time, they’ve trained backdoors into 3 different language models — they want you to find the triggers. Jane Street isn’t even sure this is possible, but they’ve set aside $50,000 for the best attempts and write-ups. They’re accepting submissions until April 1st at janestreet.com/dwarkesh https://janestreet.com/dwarkesh. * Mercury https://mercury.com/personal-banking’s personal accounts make it easy to share finances with a partner, a roommate… or OpenClaw. Last week, I wanted to try OpenClaw for myself, so I used Mercury to spin up a virtual debit card with a small spend limit, and then I let my agent loose. No matter your use case, apply at mercury.com/personal-banking https://mercury.com/personal-banking. Timestamps (00:00:00) - What exactly are we scaling? (00:12:36) - Is diffusion cope? (00:29:42) - Is continual learning necessary? (00:46:20) - If AGI is imminent, why not buy more compute? (00:58:49) - How will AI labs actually make profit? (01:31:19) - Will regulations destroy the boons of AGI? (01:47:41) - Why can’t China and America both have a country of geniuses in a datacenter? This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com https://www.dwarkesh.com?utm_medium=podcast&utm_campaign=CTA_1

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Dwarkesh Patel?

  So we talked three years ago. I'm curious, in your view, what has been the biggest update of the last three years? What has been the biggest difference between what I felt like last three years versus now?

• 0:08Speaker 01

  Yeah, I would say, actually, the underlying technology, like the exponential of the technology, has gone, broadly speaking, I would say, about as I expected it to go. I mean, there's like plus or minus, you know, a couple. There's plus or minus a year or two here, there's plus or minus a year or two there. I don't know that I would have predicted the specific direction of code. But actually, when I look at the exponential, it is roughly what I expected in terms of the march of the models from like, you know, smart high school student to smart college student to like, you know, beginning to do PhD and professional stuff, and in the case of code, reaching beyond that. So, you know, the frontier is a little bit uneven. It's roughly what I expected.

• 0:51Speaker 01

  I will tell you, though, what the most surprising thing has been. The most surprising thing has been the lack of public recognition of how close we are to the end of the exponential. To me, it is absolutely wild that, you know, you have people, you know, within the bubble and outside the bubble, you know, but you have people talking about these, you know, just the same tired old hot-button political issues and like, you know, around us, we're like, near the end of the exponential.

• 1:21Dwarkesh Patel?

  I want to understand what that exponential looks like right now, because the first question I asked you when we recorded three years ago was, you know, what's up with scaling? How does it work? I have a similar question now, but I feel like it's a more complicated question because, at least from the public's point of view.

• 1:36Speaker 01

  Yes.

• 1:37Dwarkesh Patel?

  Three years ago, there were these, you know, well-known public trends where, across many orders of magnitude of compute, you could see how the loss improves. And now, we have RL scaling, and there's no publicly known scaling law for it. It's not even clear what exactly the story is of, is this supposed to be teaching the model skills, is this supposed to be teaching meta-learning? What is the scaling hypothesis at this point?

• 1:58Speaker 01

  Yeah, so I have actually the same hypothesis that I had even all the way back in 2017. So in 2017, I think I talked about it last time, but I wrote a doc called "The Big Blob Of compute hypothesis. And, and, and, you know, it wasn't about the scaling of language models in particular. When I, when I wrote it, GPT one had, had just come out, right? So that was, you know, one among many things, right? There was, back in those days, there was robotics. People tried to work on reasoning as a separate thing from language models. There was scaling of the kind of RL that happened, that, you know, kind of happened in AlphaGo. And, uh, you know, that, that happened at Dota, at OpenAI. And, um, you know, people remember Starcraft at DeepMind, you know, the AlphaStar.

• 2:42Speaker 01

  So, uh, it was written as a more general document. And, and the specific thing I said was the following that, and, you know, it's, it's very, you know, Rich Sutton put out the bitter lesson a couple of years later. But, you know, the, the hypothesis is, is basically the same. So, so what it says is all the cleverness, all the techniques, all, all the kind of, we need a new method to, to do something like that, doesn't matter very much. There are only a few things that matter, and I think I listed seven of them. One is like how much raw compute you have. The other is the quantity of data that you have. Then the third is kind of the quality and distribution of data, right? It needs to be a broad, broad distribution of data. The fourth is, I think, how long you train for.

• 3:27Speaker 01

  The fifth is you need an objective function that can scale to the moon. So the pre-training objective function is one such objective function, right? Another objective function is, you know, the, the kind of RL objective function that says like you have a goal, you're going to go out and reach the goal. Within that, of course, there's objective rewards, like, you know, like you see in math and coding. And there's more subjective rewards, like you see in RL from human feedback or kind of higher order, higher order versions of that. And, and then the sixth and seventh were things around kind of like normalization or conditioning, like, you know, just getting the numerical stability so that kind of the big blob of compute. Flows in this laminar way instead of instead of running into problems.

• 4:10Speaker 01

  So that was the hypothesis and it's hypothesis I still hold. I don't think I've seen very much that is not in line with that hypothesis. And so the pre-training scaling laws were one example of kind of what we see there. And indeed, those have continued going like, you know, you know, I think, I think now it's been, it's been widely reported like, you know, we feel good about pre-training like pre-training is continuing to give us gains. What has changed is that now we're also seeing the same thing for RL, right? So we're seeing a pre-training phase and then we're seeing like an RL phase on top of that.

• 4:50Speaker 01

  And with RL, it's actually just the same like, you know, even even other companies have published, like, you know, in some of their, in some of their releases have published things that say, look, you know, we trained the model on math contests, you know, AIME or the kind of other things. And, you know, how well, how well the model does is log linear and how long we've trained it. And we see that as well. And it's not just math contests. It's a wide variety of RL tasks. And so we're seeing the same scaling in RL that we saw for pre-training.

• 5:26Dwarkesh Patel?

  You mentioned Richard Sutton and the bitter lesson.

• 5:29Speaker 01

  Yeah.

• 5:30Dwarkesh Patel?

  I interviewed him last year and he is actually very non LLM pilled. And if I'm, I don't know if this is his perspective, but one way to paraphrase this objection is something like, look, something which possesses the true core of human learning would not require all these billions of dollars of data and compute and these bespoke environments to learn how to use Excel or how does an, you know, how to, how to use PowerPoint, how to navigate a web browser. And the fact that we have to build in these skills using these RL environments hints that we're actually lacking this core human learning algorithm. And so we're scaling the wrong thing. And so, yeah, that is raises a question, why are we doing all this RL scaling if we

• 6:15Dwarkesh Patel?

  do think there's something that's going to be Human-like inability to learn on the fly.

• 6:18Speaker 01

  Yeah, yeah. So I think I think this kind of puts together several things that should be kind of thought of thought of differently.

• 6:25Dwarkesh Patel?

  Yeah.

• 6:26Speaker 01

  I think there is a genuine puzzle here, but it may not matter. Um, in fact, I would guess it probably it probably doesn't matter. So let's take the RL out of it for a second. Cause I actually think RL and it's a red herring to say that RL was any different from pre-training in this matter. Um, so if we if we look at pre-training scaling, um, it was very interesting back in, you know, 2017. When Alec Radford was doing GPT one, if you look at the models before GPT one, they were trained on these data sets that didn't represent a wide, you know, distribution of text, right? You had like, you know, these very standard, you know, kind of language modeling benchmarks.

• 7:09Speaker 01

  And GPT one itself was trained on a bunch of, I think it was fanfiction, actually. Um, but you know, it was it was like literary, you know, it's like literary text, which is a very small fraction of the text that you get. And what we found with that, you know, and in those days, it was like a billion words or something. So small data sets and represented a pretty narrow distribution, right? Like a narrow distribution of kind of what what you can see, what you can see in the world. And it didn't generalize well. If you did better on, um, you know, the, the, you know, I forget what it was. Some some kind of fanfiction corpus, um, it wouldn't generalize that well to kind of the other task. You know, we had all these measures of like, you know, how well does how well does a model do at predicting all of these other kinds of texts?

• 7:53Speaker 01

  You really didn't see the generalization. It was only when you trained over all the tasks on the, you know, the internet, when you, when you kind of did a general internet scrape, right? From something like, you know, common crawl or scraping links on Reddit, which is what we did for GPT two. It's only when you do that, that you kind of started to get generalization.

• 8:13Speaker 01

  And I think we're seeing the same thing on RL, that we're starting with first very simple RL tasks, like training on math competitions. Then we're kind of moving to, you know, kind of broader, broader training that involves things like code as a task. And now we're moving to do kind of many, many other tasks. And then I think we're going to increasingly get generalization. So that, that kind of takes out the RL versus the pre-training side of it. But I think there is a puzzle here either way, which is that on pre-training, when we train the model on pre-training, you know, we, we use like trillions of tokens, right? And, and humans don't see trillions of words. So there is an actual sample efficiency difference here.

• 8:55Speaker 01

  There, there is actually something different that's, that's happening here, which is that the models start from scratch and, you know, they have to get much more, much more training. But we also see that once they're trained, if we give them a long context length, the only thing blocking a long context length is like inference. But if we give them like a context length of a million, they're very good at learning and adapting within that context length. And, and so I don't know the full answer to this, but, but I think there's something going on that pre-training, it's, it's not like the process of humans learning. It's somewhere between the process of humans learning and the process of human evolution. It's like, it's somewhere between, like, we get many of our priors from evolution.

• 9:39Speaker 01

  Our brain isn't just a blank slate, right? Whole books have been written about. I think the language models, they're much more blank slates. They literally start as like random weights. Whereas the human brain starts with all these regions. It's connected to all these inputs and outputs. And, and so maybe we should think of pre-training, and for that matter, RL as well, as, as being something that exists in the middle space between human evolution and, you know, kind of human on, on the spot learning. And as the in-context learning that the models do as, as something between long-term human learning and short-term human learning. So, you know, there, there's this hierarchy of like, there's evolution, there's long-term learning, there's short-term learning, and there's just human reaction.

• 10:24Speaker 01

  And The L O M phases exist along this spectrum, but not necessarily exactly at the same points. That there's no analog to some of the human modes of learning. The L O Ms are kind of falling between the points. Does that make sense?

• 10:39Dwarkesh Patel?

  Um, yes, although some things are still a bit confusing. For example, if the analogy is that this is like evolution. So it's fine that it's not that sample efficient. Then like, well, if we're going to get the kind of super sample efficient agent from in context learning, why are we bothering to build in? You know, there's R L environment companies which are, it seems like what they're doing is they're teaching it how to use this A P I, how to use Slack, how to use whatever. It's confusing to me why there's so much emphasis on that. If the kind of agent that can just learn on the fly is emerging or is going to soon emerge or has already emerged.

• 11:10Speaker 01

  Yeah, yeah, so I, I mean, I can't speak for the emphasis of anyone else. I can, I can only talk about how we, how we think about it. I think the way we think about it is the goal is not to teach the model every possible skill within R L, just as we don't do that within pre-training. Right within pre-training, we're not trying to expose the model to, you know, every, every possible, you know, way that words could be put together. Right, you know, where it's, it's rather that the model trains on a lot of things. And then, and then it reaches generalization across pre-training. Right, that was, that was the transition from G P T one to G P T two that I saw up close, which is like, you know, the, the model reaches a point. You know, I, I, I like had these moments where I was like, oh yeah,

• 11:55Speaker 01

  you just give the model like, you just give the model a list of numbers. That's like, you know, um, you know, this is the cost of the house. This is the square feet of the house. And the model completes the pattern and does linear regression. Like not great, but it does it. But it's never seen that exact thing before. And, and so to, you know, to, to the extent that we are building these R L environments, the, the goal is, is very similar to what is, you know, to what was done five or 10 years ago with pre-training with, we're trying to get a, we're trying to get a whole bunch of data, not because we want to cover a specific document or a specific skill, but because we want to generalize.

• 12:35Dwarkesh Patel?

  I mean, I, I think the. Framework you're laying down obviously makes sense like we're making progress towards AGI. I think the crux is something like, nobody at this point disagrees that we're gonna achieve AGI in the century. And the crux is, you say we're hitting the end of the exponential. And somebody else looks at this and says, oh yeah, we're making progress. We've been making progress since 2012 and then 2035 will have a human-like agent. And so I want to understand what it is that you're seeing, which makes you think, yeah, obviously we're seeing the kinds of things that evolution did or that human, within the human lifetime learning is like in these models. And why think that it's one year away and not 10 years away?

• 13:13Speaker 01

  I, I actually think of it as like two, there's kind of two cases to be made here. Or like two, two claims you could make, one of which is like stronger and the other of which is weaker. So I think starting, starting with the weaker claim, you know, when, when I first saw the scaling back in like, you know, 2019, you know, I wasn't sure, you know, this was the whole, this was kind of a 50 50 thing, right? I thought I saw something that was, you know, and, and my claim was this is much more likely than anyone thinks it is. Like this is wild. No one else would even consider this. Maybe there's a 50% chance this happens. On the basic hypothesis of, you know, as you put it, within 10 years, we'll get to, you know,

• 13:58Speaker 01

  you know, what I call kind of country of geniuses in a data center. I'm at like 90% on that. And it's hard to go much higher than 90% because the world is so unpredictable. Maybe the irreducible uncertainty would be if we were at 95% where you get to things like, I don't know, maybe multiple, you know, multiple companies have, you know, kind of internal turmoil and nothing happens. And then Taiwan gets invaded and like all the, all the fabs get blown up by missiles and, and you know, and then.

• 14:27Dwarkesh Patel?

  Now you're drinking the Stadia.

• 14:28Speaker 01

  Yeah, yeah, you know, just you could construct a scenario where there's like a 5% chance that it, or you know, you can construct a 5% world where like things, things get delayed for, for, for, for, for 10 years. That's maybe 5%. There's another 5% which is that I'm very confident on tasks that can be verified. So I think, I think with coding, I'm just except for that irreducible uncertainty. There's just, there's, I mean, I think we'll be there in one or two years. There's no way we will not be there in 10 years in terms of being able to do it end to end coding. My one little bit, the one little bit of, of. Fundamental uncertainty, even on long time scales, is this thing about tasks that

• 15:08Speaker 01

  aren't verifiable, like planning a mission to Mars, like, you know, doing some fundamental scientific discovery, like CRISPR, like, you know, writing a novel. Hard to verify those tasks. I am almost certain that we have a reliable path to get there, but like, if there was a little bit uncertainty, it's there. So on the 10 years, I'm like, you know, 90%, which is about as certain as you can be. Like, I think it's crazy to say that this won't happen by 2035. Like, in some sane world, it would be outside the mainstream.

• 15:48Dwarkesh Patel?

  But the emphasis on verification hints to me as a lack of belief that these models was generalized. If you think about humans, we are good at things that both which we get verifiable reward and things which we don't. You're like, you haven't.

• 16:05Speaker 01

  No, no, no, this is why I'm almost sure. We already see substantial generalization from things that verify to things that don't verify. We're already seeing that.

• 16:14Dwarkesh Patel?

  But it seems like you were emphasizing this as a spectrum which will split apart, which means we see more progress. And I'm like, but that doesn't seem like how humans get better.

• 16:23Speaker 01

  The world in which we don't make it, or the world in which we don't get there, is the world in which we do all the things that are verifiable. And then they like, you know, many of them generalize, but we kind of don't get fully there. We don't fully, you know, we don't fully color in this side of the box. It's not a binary thing.

• 16:43Dwarkesh Patel?

  But it also seems to me, even if in the world where generalization is weak, when you only see verifiable domains, it's not clear to me in such a world you could automate software engineering, because software, like, in some sense, you are quote unquote, a software engineer.

• 16:56Speaker 01

  Yeah.

• 16:57Dwarkesh Patel?

  But part of being a software engineer for you involves writing these like long memos about your grand vision about different things.

• 17:01Speaker 01

  Well, I don't think that's part of the job of SUI. That's part of the job of the company. But I do think SUI involves like, design documents and other things like that. Which, by the way, the models are not bad. They're already pretty good at writing comments. And so with. Again, I'm making like much weaker claims here than I believe to like, you know, to kind of set up a, you know, to distinguish between two things like, we're already almost there for software engineering. We are already almost there.

• 17:27Dwarkesh Patel?

  By one metric, there's one metric, which is like how many lines of code are written by AI? And if you use, if you consider other productivity improvements in the course of the history of software engineering, compilers write all the lines of software. And but we, there's a difference between how many lines are written and how big the productivity improvement is.

• 17:42Speaker 01

  Oh, yeah.

• 17:43Dwarkesh Patel?

  And then like, we're almost there, meaning like, how big is the productivity improvement? Not just how many lines are written.

• 17:49Speaker 01

  Yeah, yeah. So I actually, I actually, I actually agree with you on this. So I've made this series of predictions on code and software engineering. And I think people have repeatedly kind of misunderstood them. So let me, let me, let me, let me lay out the spectrum, right? Like, I think it was like, you know, like, you know, eight or nine months ago or something, I said, you know, the AI model will be writing 90%, 90% of the lines of code in like, you know, three to six months, which, which happened at least at some places, right? Happened, happened at Anthropic, happened with many people downstream using our models. But, but that's actually a very weak criterion, right? People thought I was saying like, we won't need 90% of the software engineers. Those things are worlds apart, right?

• 18:33Speaker 01

  Like, I would put the spectrum as 90% of code is written by the model. 100% of code is written by the model, and that's a big difference in productivity. 90% of the end-to-end SUI tasks, right? Including things like compiling, including things like setting up clusters and environments, testing features, writing memos, 90% of the SUI tasks are written by the models. 100% of today's SUI tasks are written by the models. And even when, when that happens, doesn't mean software engineers are out of a job. Like, there's like new higher level things they can do, where they can, they can manage. And then there's a further down the spectrum, like, you know, there's 90% less demand for SUIs, which I think will happen, but like, this, this, this is a spectrum.

• 19:18Speaker 01

  And, you know, I wrote about it in the adolescence of technology, where I went through this kind of spectrum with farming.

• 19:26Speaker 01

  And so I actually totally agree with you on that. It's just, these are very different benchmarks from each other, but we're proceeding through them super fast.

• 19:34Dwarkesh Patel?

  It seems like in part of Eurovision, it's like going from 90 to 100. First, it's going to happen fast. And two, that somehow that leads to huge productivity improvements. Whereas when I notice even in greenfield projects that people start with Claude Code or something, people report starting a lot of projects. And I'm like, do we see in the world out there a renaissance of software, all these new features that wouldn't exist otherwise? And at least so far, it doesn't seem like we see that. And so that does make me wonder, even if, even if like I never had to intervene on Claude Code, there is this thing of like, there's just, the world is complicated, jobs are complicated, and closing the loop on self-contained systems, whether it's just writing software or something,

• 20:15Dwarkesh Patel?

  how much sort of, how much broader gains we would see just from that? And so maybe that makes us, this should dilute our estimation of the country of geniuses.

• 20:24Speaker 01

  Well, I actually, I like, I like simultaneously, I simultaneously agree with you, agree that it's a reason why these things don't happen instantly. But at the same time, I think the, the effect is going to be very fast. So like, I don't know, you could have these two poles, right? One is like, you know, AI is like, you know, it's not going to make progress, it's slow, like, it's going to take, you know, kind of forever to diffuse within the economy, right? Economic diffusion has become one of these buzzwords that's like, a reason why we're not going to make AI progress, or why AI progress doesn't matter. And, and you know, the other axis is like, we'll get recursive self-improvement, you know, the whole thing, you know, can't you just draw an exponential line on the, on

• 21:07Speaker 01

  the curve, you know, it's, it's, we're going to have, you know, Dyson spheres around the sun and like, you know, you know, so many nanoseconds after, you know, after, after we get recursive, I mean, I'm completely caricaturing the view here, but like, you know, there, there are these two extremes. But what we've seen from, from the beginning, you know, at least if you look within Anthropic, there's this bizarre 10x per year growth in revenue that we've seen, right? So, you know, in 2023, it was like zero to 100 million. 2024, it was 100 million to a billion. 2025, it was a billion to like nine or 10 billion. And then

• 21:47Dwarkesh Patel?

  Billion dollars with your own product so you could just like have a clean 10B.

• 21:50Speaker 01

  And the first month of this year, like that exponential is you would think it would slow down, but it would like, you know, we added another few billion to like, you know, to, to, to, we added another few billion to revenue in January. And, and so, you know, obviously that curve can't go on forever, right? You know, the GDP is only so large. I don't, you know, I would even guess that it bends, that it bends, bends somewhat this year. But like, that is like a fast curve, right? That's like, that's like a really fast curve. And I would bet it stays pretty fast, even as the scale goes to the entire economy. So like, I, I think we should be thinking about this middle world where things are like extremely fast,

• 22:32Speaker 01

  but not instant, where they take time because of economic diffusion, because of the need to close the loop. Because, you know, it's like this fiddly, oh, man, I have to do change management within my enterprise, you know, I have to like, you know, uh, you know, I, I, I like, I set this up, but, but, you know, I have to change the security permissions on this in order to make it actually work. Or, you know, I had this like old piece of software that, you know, that like, you know, checks the model before it's compiled and, and, and like released. And I have to rewrite it. And yes, the model can do that, but I have to tell the model to do that. And it has to, it has to take time to do that. And, and, and so I think everything we've seen so far is, is compatible with the idea

• 23:15Speaker 01

  that there's one fast exponential, that's the, the capability of the model. And then there's another fast exponential that's downstream of that, which is the diffusion of the model into the economy. Not instant, not slow, much faster than any previous technology, but it has its limits. And, and, and, and this is what we, you know, when I, when I look inside Anthropic, when I look at our customers, fast adoption, but not infinitely fast.

• 23:44Dwarkesh Patel?

  Can I try a hot take on you?

• 23:45Speaker 01

  Yeah.

• 23:45Dwarkesh Patel?

  I feel like diffusion is COB, that people use to say when it's like. If the model isn't able to do something, they're like, oh, but it's like a diffusion issue. But then you should use the comparison to humans. You would think that the inherent advantages that AIs have would make diffusion a much easier problem for new AIs getting on boarded than new humans getting on boarded. So an AI can read your entire Slack and your drive-in minutes. They can share all the knowledge that the other copies of the same instance have. You don't have this adverse selection problem when you're hiring AIs, because you can just hire copies of a vetted AI model. Hiring a human is so much more hassle, and people hire humans all the time, right? We pay humans upwards of $50 trillion in wages because they're useful, even though

• 24:26Dwarkesh Patel?

  it's like, in principle, it would be much easier to integrate AIs into the economy than it is to hire humans. I think the diffusion, I feel like, doesn't really explain.

• 24:34Speaker 01

  I think diffusion is very real and doesn't have to, you know, doesn't exclusively have to do with limitations on the AI models. Again, there are people who use diffusion as kind of a buzzword to say this isn't a big deal. I'm not talking about that. I'm not talking about, you know, AI will diffuse at the speed that previous. I think AI will diffuse much faster than previous technologies have, but not infinitely fast. So I'll just give an example of this, right? There's Claude code. Claude code is extremely easy to set up. If you're a developer, you can kind of just start using Claude code. There is no reason why a developer at a large enterprise should not be adopting Claude

• 25:19Speaker 01

  code as quickly as, you know, an individual developer or developer at a startup. And we do everything we can to promote it, right? We sell Claude code to enterprises, and big enterprises, like, you know, big financial companies, big pharmaceutical companies, all of them, they're adopting Claude code much faster than enterprises typically adopt new technology, right? But again, it takes time. Like, any given feature or any given product, like Claude code or like CoWork, will get adopted by the, you know, the individual developers who are on Twitter all the time, by the, like, Series A startups.

• 26:02Speaker 01

  Many months faster than they will get adopted by like, you know, a like large enterprise that does food sales. There are a number of factors, like you have to go through legal, you have to provision it for everyone. It has to, you know, like it has to pass security and compliance. The leaders of the company who are further away from the AI revolution, you know, are forward looking, but they have to say, oh, it makes sense for us to spend 50 million. This is what this Claude code thing is. This is why it helps our company. This is why it makes us more productive. And then they have to explain to the people two levels below and they have to say, okay, we have 3,000 developers, like here's how we're going to roll it out to our developers.

• 26:45Speaker 01

  And we have conversations like this every day, like, you know, we are doing everything we can to make Anthropic's revenue grow 20 or 30 X a year instead of 10 X a year. You know, and, and, and again, you know, many enterprises are just saying, this is so productive, like, you know, we're going to take shortcuts on our usual procurement process, right? They're moving much faster than, you know, when we tried to sell them just the ordinary API, which many of them use, but Claude code is a more compelling product. But it's not an infinitely compelling product. And I don't think even AGI or powerful AI or country of geniuses in the data center will be an infinitely compelling product. It will be a compelling product enough, maybe to get three or five or 10 X a year growth,

• 27:28Speaker 01

  even when you're in the hundreds of billions of dollars, which is extremely hard to do and has never been done in history before, but not infinitely fast.

• 27:34Dwarkesh Patel?

  I, I buy that it would be a slight slowdown and maybe this is not your claim, but sometimes people talk about this like, oh, the capabilities are there, but because of diffusion. Otherwise, like we're basically at AGI and then.

• 27:45Speaker 01

  I, I don't believe we're basically at AGI.

• 27:48Dwarkesh Patel?

  I think if you had the country of geniuses in a data center, if your company didn't adopt.

• 27:52Speaker 01

  If you had the country of geniuses in a data center, we would know it.

• 27:55Dwarkesh Patel?

  Right.

• 27:56Speaker 01

  We would know it if you had the country of geniuses in a data center. Like everyone in this room would know it. Everyone in Washington would know it, like, you know. People in rural parts that might not know it, but we would know it. We don't have that now. That's very clear.

• 28:12Dwarkesh Patel?

  As Dario was hinting at, to get generalization, you need to train across a wide variety of realistic tasks and environments. For example, with a sales agent, the hardest part isn't teaching it to mash buttons in a specific database in Salesforce. It's training the agent's judgment across ambiguous situations. How do you sort through a database with thousands of leads to figure out which ones are hot? How do you actually reach out? What do you do when you get ghosted? When an AI lab wanted to train a sales agent, Labelbox brought in dozens of Fortune 500 salespeople to build a bunch of different RL environments. They created thousands of scenarios where the sales agent had to engage with a potential customer, which was role-played by a second AI. Labelbox made sure that this customer AI had a few different personas. Because when you cold-call, you have no idea who's going to be on the other end.

• 28:57Dwarkesh Patel?

  You need to be able to deal with a whole range of possibilities. Labelbox's sales experts monitored these conversations turn by turn, tweaking the role-playing agent to ensure it did the kinds of things an actual customer would do. Labelbox could iterate faster than anybody else in the industry. This is super important because RL is an empirical science. It's not a solved problem. Labelbox has a bunch of tools for monitoring agent performance in real time. This lets their experts keep coming up with tasks so that the model stays in the right distribution of difficulty and gets the optimal reward signal during training. Labelbox can do this sort of thing in almost every domain. They've got hedge fund managers, radiologists, even airline pilots. So whatever you're working on, Labelbox can help. Learn more at labelbox.com/vorkesh.

• 29:42Dwarkesh Patel?

  Coming back to concrete predictions, because I think because there's so many different things to disambiguate, it can be easy to talk past each other when we're talking about capabilities. So for example, when I interviewed you three years ago, I asked you a prediction about what should we expect three years from now. I think you were right. So you said, "We should expect systems, which if you talk to them for the course of an hour, it's hard to tell them apart from a generally well-educated human." Yes. I think you were right about that. And I think spiritually, I feel unsatisfied because my internal expectation was was that such a system could automate large parts of white-collar work. And so it might be more productive to talk about the actual end capabilities you want such a system. So I will basically tell you where I think we are.

• 30:25Dwarkesh Patel?

  So let me ask you a very specific question so that we can... Figure out exactly what kinds of capabilities we should do soon. So maybe I'll ask about it in the context of a job I understand well, not because it's the most relevant job, but just because I can evaluate the claims about it. Take video editors, right? I have video editors and part of their job involves learning about our audience's preferences, learning about my preferences and tastes and the different trade offs we have and just over the course of many months, building up this understanding of context. And so the skill and ability they have six months into the job, a model that can pick up that skill on the job, on the fly. When should we expect such an AI system?

• 31:03Speaker 01

  Yeah, so I guess what you're talking about is like, you know, we're doing this interview for three hours and then like, you know, someone's going to come in, someone's going to edit it, they're going to be like, oh, you know, you know, I don't know, Dario like, you know, scratched his head and you know, we could edit that out.

• 31:18Dwarkesh Patel?

  And you know, there was this like long, there was this like long discussion that like is less interesting to people.

• 31:24Speaker 01

  And then, you know, then there's other thing that's like more interesting to people. So, you know, let's, let's, let's kind of make this, this edit. So, you know, I think the country of geniuses in a data center will be able to do that. The, the way it will be able to do that is, you know, it will have general control of a computer screen, right? Like, you know, and, and you'll be able to feed this in and it'll be able to also use the computer screen to like, go on the web, look at all your previous, look at all your previous interviews, like, look at what people are saying on Twitter in response to your interviews, like, talk to you, ask you questions, talk to your staff, look at the history of kind of edits, edits that you did and from that, like, do the job.

• 32:01Dwarkesh Patel?

  Yeah.

• 32:02Speaker 01

  So I think that's dependent on several things. One, that's dependent, and, and, and I think this is one of the things that's actually blocking deployment, getting to the point on computer use where the models are really masters at using the computer, right? And, you know, we've seen this climb in benchmarks and benchmarks are always, you know, imperfect measures, but like, you know, OS world is, you know, went from, you know, like 5%, you know, like, I think when we first released, you know, uh, computer use like a year and a quarter ago, it was like maybe 15%, I don't remember exactly, but we've climbed from that to like 65 or 70%. And, and, you know, there may be harder measures as well, but, but I think computer use has to pass a point of reliability.

• 32:46Dwarkesh Patel?

  Can I just have a follow up on that before you move on to the next point? I often, for years, I've been trying to build different internal LLM tools for myself. And I often, often I have these. Text in, text out tasks, which should be dead center in the repertoire of these models, and yet I still hire humans to do them just because it's if it's something like make identify what the best clips would be in this transcript and maybe they'll do like a seven out of 10 job at them. But there's not this ongoing way I can engage with them to help them get better at the job the way I could with a human employee. And so that missing ability, even if you solve computer use, would still block my ability to like offload an actual job to them.

• 33:23Speaker 01

  Again, there's there's this gets back to what we're talking about before with learning on the job where it's it's very interesting. You know, I think I think with the coding agents, like, I don't think people would say that learning on the job is what is what is, you know, preventing the coding agents from like, you know, doing everything end to end like they keep they keep getting better. We have engineers at Anthropic who like, don't write any code. And when I look at the productivity to your to your previous question, you know, we have folks who say this this GPU kernel, this chip, I used to write it myself. I just have Claude do it. And so there's this, there's this enormous improvement in productivity. And I don't know, like, when I see Claude code, like,

• 34:06Speaker 01

  familiarity with the code base, or like, you know, or a feeling that the model hasn't worked at the company for a year, that's not high up on the list of complaints I see. And so I think what I'm saying is we're like, we're kind of taking a different path.

• 34:22Dwarkesh Patel?

  Don't you think with coding, that's because there is an external scaffold of memory which exists instantiated in the code base, which I don't know how many other jobs have coding made fast progress precisely because it has this unique advantage that other economic activity doesn't.

• 34:37Speaker 01

  But but when you say that, what you're what you're implying is that by reading the code base into the context, I have everything that the human needed to learn on the job. So that would be an example of whether it's written or not, whether it's available or not, a case where everything you needed to know, you got from the context window, right? And that and that what we think of as learning, like, oh, man, I started this job, it's going to take me six months to understand the code base, the model just did it in the context.

• 35:07Dwarkesh Patel?

  Yeah, I honestly don't know how to think about this because there are people who qualitative report what you're saying. There was a meter study, I'm sure you saw last year.

• 35:16Speaker 01

  Yes.

• 35:17Dwarkesh Patel?

  Where they Had experienced developers try to close pull requests in repositories that they were familiar with, and those developers reported an uplift, they reported that they felt more productive with the use of these models. But in fact, if you look at their output and how much was actually merged back in, there's a 20% down lift. They were less productive as a result of these models. And so I'm trying to square the qualitative feeling that people feel with these models versus, one, at a macro level, where is this renaissance of software? And then, two, when people do these independent evaluations, why are we not seeing the productive benefits we would expect?

• 35:53Speaker 01

  Within Anthropic, this is just really unambiguous, right? We're under an incredible amount of commercial pressure and make it even harder for ourselves because we have all this safety stuff we do that I think we do more than other companies. So like, the pressure to survive economically while also keeping our values is just incredible, right? We're trying to keep this 10x revenue curve going. There is zero time for bullshit. There is zero time for feeling like we're productive when we're not. Like, these tools make us a lot more productive. Like, why do you think we're concerned about competitors using the tools? Because we think we're ahead of the competitors and like, we don't want

• 36:37Speaker 01

  to sell, we wouldn't be going through all this trouble if this was secretly reducing our productivity. Like, we see the end productivity every few months in the form of model launches. Like, there's no kidding yourself about this. Like, the models make you more productive.

• 36:55Dwarkesh Patel?

  One, that, people feeling like they're more productive is qualitatively predicted by studies like this. But two, if I just look at the end output, obviously you guys are making fast progress. But the fact, you know, the idea was supposed to be with recursive self-improvement is that you make a better AI, the AI helps you build a better next AI, et cetera, et cetera. And what I see instead, if I look at the U OpenAI DeepMind, is that people are just shifting around the podium every few months. And maybe you think that stops because you've won or whatever. But why are we not seeing the person with the best coding model have this lasting advantage if, in fact, there are these enormous productivity gains from the last coding model?

• 37:36Speaker 01

  So, no, no, no. I mean, I think it's all like... My my model of the situation is there's there's an advantage that's gradually growing. Like, I would say, right now, the coding models give maybe, I don't know, a like 15, maybe 20% total factor speed up. Like, that's my view. And six months ago, it was maybe 5%. And so and so it didn't matter. Like, 5% doesn't register. It's now just getting to the point where it's like, one of several factors that that kind of matters. And and that's gonna that's gonna keep speeding up. And so I think six months ago, like, you know, there were several, there were several companies that were at roughly the same point because, you know, this this

• 38:21Speaker 01

  wasn't, this wasn't a notable factor. But I think it's starting to speed up more and more. I, you know, I would, I would also say there are multiple companies that, you know, write models that are used for code. And, you know, we're not perfectly good at, you know, preventing some of these other companies from from from using, from from from kind of using our models internally. So, you know, I think, I think everything we're kind of everything we're seeing is consistent with this kind of, this kind of snowball model where you where, you know, there's no hard, again, my, my, my, my, my theme in all of this is like, all of this is soft takeoff, like, soft, smooth exponentials, although the exponentials are relatively steep. And so and so we're seeing this snowball

• 39:04Speaker 01

  gather momentum where it's like, 10%, 20%, 25%, you know, 40%. And as you go, yeah, AMDALL's law, you have to get all the like, things that are preventing you from from closing the loop out of the way. But like, this is one of the biggest priorities within Anthropic.

• 39:22Dwarkesh Patel?

  Stepping back, I think, before in the stack, we were talking about, well, when do we get this on the job learning? And it seems like the coding, the point you were making, the coding thing is, we actually don't need on the job learning. That you can have tremendous productivity improvements, you can have potentially trillions of dollars of revenue for AI companies, without this basic human ability, maybe that's not your claim, you should clarify. But without this basic human ability to learn on the job. But I just look at like, In most domains of economic activity, people say, I hired somebody, they weren't that useful for the first few months, and then over time, they built up the context understanding. It's actually hard to define what we're talking about here. But they got something, and then now they're a powerhorse and they're so valuable to us.

• 40:05Dwarkesh Patel?

  And if AI doesn't develop this ability to learn on the fly, I'm not, I'm a bit skeptical that we're going to see huge changes to the world.

• 40:13Speaker 01

  Yeah, so I think, I think, I think two things here, right? There's the state of the technology right now, which is, again, we have these two stages. We have the pre-training and RL stage where you throw, you throw a bunch of data and tasks into the models, and then they generalize. So it's like learning, but it's like learning from more data and not, you know, not learning over kind of one human or one model's lifetime. So again, this is situated between evolution and, and, and human learning. But once you learn all those skills, you have them. And, and just like with pre-training, just how the models know more, you know, if, if I look at a, a pre-trained model, you know, it knows more about the history of Samurai in Japan than I do. It knows more about baseball than I do.

• 40:58Speaker 01

  It knows, you know, it knows more about, you know, low-pass filters and electronics than, you know, all, all of these things. Its knowledge is way broader than mine. So I think, I think even, even just that, you know, may get us to the point where the models are better at, you know, kind of better at everything. And then we also have, again, just with scaling the kind of existing setup, we have the in-context learning, which I would describe as kind of like human on-the-job learning, but like a little weaker and a little short-term. Like, you look at in-context learning, you give the model a bunch of examples, it does get it. There's real learning that happens in context. And like, a million tokens is a lot. That's, that's, you know, that can be days of human learning, right?

• 41:42Speaker 01

  You know, if you think about the model, you know, you know, kind of reading, reading a million words, you know, it, you know, it takes me, how long would it take me to read a million? I mean, you know, like, days or weeks at least. So you have these two things, and, and I think these two, these two things within the existing paradigm may just be enough to get you the country of genius in a data center. I don't know for sure, but I think they're going to get you a large fraction of it. There may be gaps, but I, I certainly think, just as things are, this, I believe, is enough to generate trillions of dollars of revenue. That's one. That's all one. Two is this idea of continual learning, this idea of a single model learning on the job. I think we're working on that, too.

• 42:26Speaker 01

  And I think there's a good chance that in the next year or two, we also make, we also solve that. I, again, I, you know, I think you get most of the way there without it. I think the trillions of dollars of, of, you know, the, I think the trillions of dollars a year market, maybe all of the national security implications and the safety implications that I wrote about in Adolescence of Technology can happen without it. But I, I also think we, and I imagine others, are working on it. And I think there's a good chance that, that, you know, that we get there within the next year or two. There are a bunch of ideas. I won't go into all of them in detail, but, you know, one is just make the context longer.

• 43:09Speaker 01

  There's, there's nothing preventing longer context from working. You just have to train at longer context and then learn to, to serve them at inference. And both of those are engineering problems that we are working on and that I would assume others are working on as well.

• 43:21Dwarkesh Patel?

  Yeah. So this context length increase, it seemed like there was a period from 2020 to 2023 where from GPT-3 to GPT-4 Turbo, there was an increase from like 2,000 context lengths to 128K. I feel like for the next, for the two-ish year since then, we've been in the same-ish ballpark.

• 43:36Speaker 01

  Yeah.

• 43:36Dwarkesh Patel?

  And when model, context lengths get much longer than that, people report qualitative degradation in the ability of the model to consider that full context. So I'm curious what you're internally seeing that makes you think like, oh, 10 million context, 100 million context, to get human, like, six-month learning, billion, billion context.

• 43:54Speaker 01

  This isn't a research problem. This is a, this is an engineering and inference problem, right? If you want to serve long context, you have to like store your entire KV cache, you have to, you know, you know, it's, it's, it's, it's difficult to store all the memory in the GPUs, to juggle the memory around. I don't even know the detail, you know, at this point, this is at a level of detail that, that, that I'm no longer able to follow, although, you know, I, I knew it in the GPT-3 era of like, you know, these are the weights, these are the activations you have to store. But, you know, you know, these days, the whole thing is Flip because we have M O E models and and and kind of all of that. But um, and and this degradation you're talking about like again, without getting too specific like a question I would ask is like.

• 44:38Speaker 01

  There's two things. There's the context length you train at, and there's a context length that you serve at. If you train at a small context length and then try to serve at a long context length, like maybe you get these degradations. It's better than nothing. You might still offer it, but you get these degradations. And maybe it's harder to train at a long context length. Yeah. So you know, there's, there's a lot.

• 44:56Dwarkesh Patel?

  I I want to at the same time ask about like maybe some rabbit holes of like, well, wouldn't you expect? That if you had to train a longer context length, that would mean that um, you're able to get sort of like less samples in for the same amount of compute. But before maybe it's not worth diving deep on that. I I want to get an answer to the bigger picture question, which is like, okay. So um, I don't feel a preference for a human editor. That's been working for me for six months versus an A I that's been working with me for six months. What year do you predict that that will be the case?

• 45:31Speaker 01

  I my, I mean, you know, my guess for that is, you know, there's, there's a lot of problems that are basically like, we can do this when we have the country of geniuses in a data center. And so, you know, my, my, my, my, my picture for that is, you know, again, if you, if you, if you, if you know, if you made me guess, it's like one to two years, maybe one to three years. It's really hard to tell. I have a, I have a strong view. Ninety nine, ninety five percent that like all this will happen in 10 years. Like that's, I think that's just a super safe bet.

• 45:59Dwarkesh Patel?

  Yeah.

• 45:59Speaker 01

  And then I have a hunch, this is more like a 50 50 thing, that it's going to be more like one to two, maybe more like one to three.

• 46:06Dwarkesh Patel?

  So one to three years, the country of geniuses, um, and the slightly less economically valuable task of editing videos.

• 46:14Speaker 01

  It seems pretty economically valuable, let me tell you. It's just there are a lot of use cases like that, right? There are a lot of similar ones.

• 46:20Dwarkesh Patel?

  So you're predicting that within one to three years. And then generally anthropic is predicted that by late 26, early 27, we will have A I systems that are quote. Have the ability to navigate interfaces available to humans doing digital work today. Intellectual capabilities mashing or exceeding that of Nobel Prize winners. And the ability to interface with the physical world. And then you gave an interview two months ago with Dealbook where you're emphasizing your your company's more responsible compute scaling as compared to your competitors. And I'm trying to square these two views where if you really believe that we're going to have a country of geniuses, you you want as big a data center as you can get. There's no reason to slow down. The TAM of a Nobel Prize winner that is actually can do everything a Nobel Prize winner can

• 47:05Dwarkesh Patel?

  do is like trillions of dollars. And so I'm trying to square this conservatism, which seems rational if you have more moderate timelines, with your stated views about AI progress.

• 47:16Speaker 01

  Yeah, so so it actually all fits together. And and we go back to this fast, but not infinitely fast diffusion. So like, let's say that we're making progress at this rate. You know, the the technology is making progress this fast. Again, I have, you know, very high conviction that like, it's going, you know, the, you know, we're we're going to get there within within a few years. I have a hunch that we're going to get there within a year or two. So a little uncertainty on the technical side, but like, you know, pretty, pretty strong confidence that it won't be off by much. What I'm less certain about is, again, the economic diffusion side. Like, I really do believe that we could have models that are a country of geniuses, a

• 48:00Speaker 01

  country of geniuses in a data center in one to two years. One question is, how many years after that do the trillions in, you know, do the do the trillions in revenue start rolling in? I don't think it's guaranteed that it's going to be immediate. You know, I think it could be one year, it could be two years, I could even stretch it to five years, although I'm like, I'm skeptical of that. And so we have this uncertainty, which is, even if the technology goes as fast as I suspect that it will. We don't know exactly how fast it's going to drive revenue. We know it's coming, but with the way you buy these data centers,

• 48:45Speaker 01

  if you're off by a couple of years, that can be ruinous. It is just like how I wrote, you know, in Machines of Loving Grace. I said, look, I think we might get this powerful AI, this country genius in the data center. That description you gave comes from the Machines of Loving Grace. I said, we'll get that 2026, maybe 2027 again. That is, that is my hunch. Wouldn't be surprised if I'm off by a year or two, but like, that is my hunch. Let's say that happens, that's the starting gun. How long does it take to cure all the diseases, right? That's, that's one of the ways that like drives a huge amount of, of, of economic value, right? Like you cure, you cure every disease. You know, there's a question of how much of that goes to the pharmaceutical company, to the AI company, but there's an enormous consumer surplus because everyone, you know, assuming we can get access for everyone,

• 49:29Speaker 01

  which I care about greatly, we, you know, we, we cure all of these diseases. How long does it take? You have to do the biological discovery. You have to, you know, you have to, you know, manufacture the new drug. You have to, you know, go through the regulatory process. And we saw this with like vaccines and COVID, right? Like it, there's just this, we, we got the vaccine out to everyone, but it took a year and a half. Right? And, and so my question is, how long does it take to get the cure for everything? Which AI is the genius that can in theory invent out to everyone? How long from when that AI first exists in the lab to when diseases have actually been cured for everyone? Right? In, in, you know, we've had a polio vaccine for 50 years.

• 50:12Speaker 01

  We're still trying to eradicate it in the most remote corners of Africa. And, you know, the Gates Foundation is trying as hard as they can. Others are trying as hard as they can, but you know, that's difficult. Again, I, you know, I don't expect most of the economic diffusion to be as difficult as that. Right? That's like the most difficult case, but, but there's a, there's a real dilemma here. And, and where I've settled on it is it will be, it will be, it will be faster than anything we've seen in the world, but it still has its limits. And, and so then when we go to buying data centers, you know, you, again, again, the curve I'm looking at is, okay, we, you know, we've had a 10x a year increase every year.

• 50:54Speaker 01

  So beginning of this year, we're looking at 10 billion in, in, in annual, in, you know, rate of annualized revenue at the beginning of the year. We have to decide how much compute to buy.

• 51:06Speaker 01

  And, you know, it takes a year or two to actually build out the data centers to reserve the data centers. So basically, I'm saying like, in 2027, how much compute do I get? Well, I could assume that the revenue will continue growing 10x a year. So it'll be one, one hundred billion at the end of 2026 and one trillion at the end of 2027. And so I could buy a trillion dollars. Actually, it would be like five trillion dollars of compute because it would be a trillion dollar a year for five years, right? I could buy a trillion dollars of compute that starts at the end of 2027. And if my, if my revenue is not a trillion dollars,

• 51:51Speaker 01

  if it's even 800 billion, there's no force on earth. There's, there's no hedge on earth that could stop me from going bankrupt if I, if I buy that much compute. And, and so even though a part of my brain wonders if it's going to keep growing 10x, I can't buy a trillion dollars a year of compute in, in, in, in, in, in, in, in 2027. If I'm just off by a year in that rate of growth, or if the, the growth rate is five x a year instead of 10x a year, then, then, you know, then you go bankrupt. And, and, and so you end up in a world where, you know, you're supporting hundreds of billions, not trillions, and you accept, you accept some

• 52:33Speaker 01

  risk that there's so much demand that you can't support the revenue, and you accept still some risk that, you know, you got it wrong and it's still slow. And so when I talked about behaving responsibly, what I meant actually was not the absolute amount. That, that actually was not, you know, I think it is true, we're spending somewhat less than some of the other players. It's actually the other things like Have we been thoughtful about it? Or are we YOLOing and saying, oh, we're going to do $100 billion here, $100 billion there? I kind of get the impression that, you know, some of the other companies have not written down the spreadsheet, that they don't really understand the risks they're taking. They're just kind of doing stuff because it sounds cool. And, and we've thought carefully about it, right?

• 53:18Speaker 01

  We're an enterprise business, therefore, you know, we can rely more on revenue. It's less fickle than consumer. We have better margins, which is the buffer between buying too much and buying too little. And so I think we bought an amount that allows us to capture pretty strong upside worlds. It won't capture the full 10x a year. And things would have to go pretty badly for us to be, for us to be in financial trouble. So I think we thought carefully and we've made that balance. And, and that's what I mean when I say that we're being responsible.

• 53:50Dwarkesh Patel?

  Okay, so it seems like it's possible that we're, we actually just have different definitions of the country of a genius in a data center. Because when I think of like actual human geniuses, an actual country of human geniuses in a data center, I'm like, I would happily buy $5 trillion worth of compute to run actual country of human geniuses in a data center. So let's say JP Morgan or Moderna or whatever doesn't want to use them. I've got a country of geniuses, they'll, they'll start their own company. And if like they, they can't start their own company and they're bottlenecked by clinical trials, it is worth stating with clinical trials, like most clinical trials fail because the drug doesn't work, there's not efficacy, right?

• 54:23Speaker 01

  And I make exactly that point in, in Machines of Love and Grace. I say the clinical trials are going to go much faster than we're used to, but not, not instant, not infinitely fast.

• 54:33Dwarkesh Patel?

  And then suppose it takes a year to, for the clinical trials to work out so that you're getting revenue from that and can make more drugs. Okay, well, you've got a country of geniuses and you're an AI lab and you have, you could use many more AI researchers. You also think that there's these like self-reinforcing gains from, you know, smart people working on AI tech. So like, okay, you can have the, you can have the data center working on like AI progress.

• 54:57Speaker 01

  Is there more gains from buying? Like substantially more gains from buying a trillion dollars a year of compute versus $300 billion a year of compute.

• 55:07Dwarkesh Patel?

  If your competitor is buying a trillion, yes, there is.

• 55:09Speaker 01

  Well, no, there's some gain, but then, but again, there's this chance that they go bankrupt before, you know, again, if you're off by only a year, you destroy yourselves. That's the, that's the balance. We're buying a lot. We're buying a hell of a lot. Like, we're not, we're, you know, we're buying an amount that's comparable to that, that, you know, the, the, the, the biggest players in the game are buying. But, but if you're asking me, why, why haven't we signed, you know, 10, 10 trillion of compute starting in, starting in mid 2027? First of all, it can't be produced. There isn't that much in the world. But, but second, what if the country of geniuses comes, but it comes in

• 55:52Speaker 01

  mid 2028 instead of mid 2027? You go bankrupt.

• 55:55Dwarkesh Patel?

  So if your projection is one to three years, it seems like you should want $10 trillion of compute by 2029?

• 56:03Speaker 01

  2020 and maybe 2020.

• 56:05Dwarkesh Patel?

  By latest?

• 56:05Speaker 01

  Like, I mean, you know,

• 56:07Dwarkesh Patel?

  But like, are you interested? Like, it seems like even in your, the longest version of the timelines you state, the compute you are ramping up to build doesn't seem in accordance.

• 56:16Speaker 01

  What, what, what makes you think that?

• 56:17Dwarkesh Patel?

  Well, you, as you said, you want the 10 trillion, like, human wages, let's say, are on the order of 50 trillion a year.

• 56:24Speaker 01

  If you look at, so, so I won't, I won't talk about Anthropic in particular, but if you talk about the industry, like, the amount of compute the industry had, you know, the, the, the amount of compute the industry is building this year is probably in the, you know, I don't know, very low tens of, you know, call it 10, 15 gigawatts next year. I, you know, it goes up by roughly 3X a year. So like, next year's 30 or 40 gigawatts. And 2028 might be 100, 2029 might be like 300 gigawatts. And like, each gigawatt costs like, Um, maybe 10. I mean, I'm doing the math in my head, but each gigawatt costs maybe $10 billion, you know, or border 10 to 15 billion dollars a year.

• 57:09Speaker 01

  So, you know, you kind of, you know, you put that all together and you're getting about, about what you described. You're getting multiple trillions a year by 2028 to 2029. So you're, you're getting exactly that. You're getting, you're getting exactly what you predict.

• 57:22Dwarkesh Patel?

  That's for the industry.

• 57:24Speaker 01

  That, that's for the industry. That's right.

• 57:25Dwarkesh Patel?

  Suppose Anthropic's compute keeps 3Xing a year and then by like 27, you have, or 27, 28, you have 10 gigawatts and like multiply that by, as you say, 10 billion. So then it's like 100 billion a year, but then you're seeing the TAM by 2028, 29.

• 57:43Speaker 01

  Again, I don't want to give exact numbers for Anthropic, but, but these numbers are too small. These numbers are too small.

• 57:48Dwarkesh Patel?

  Okay. Interesting. I'm really proud that the puzzles I've worked on with Jane Street have resulted in them hiring a bunch of people from my audience. Well, they're still hiring and they just sent me another puzzle. For this one, they spent about 20,000 GPU hours training backdoors into three different language models. Each one has a hidden prompt that elicits completely different behavior. You just have to find the trigger. This is particularly cool because finding backdoors is actually an open question in frontier AI research. Anthropic actually released a couple of papers about sleeper agents and they showed that you can build a simple classifier on the residual stream to detect when a backdoor is about to fire. But they already knew what the triggers were because they built them. Here you don't and it's not feasible to check the activations for all possible trigger phrases.

• 58:32Dwarkesh Patel?

  Unlike the other puzzles they made for this podcast, Jane Street isn't even sure this one is solvable, but they've set aside $50,000 for the best attempts and write-ups. The puzzle's live at janestreet.com/thorash. And they're accepting submissions until April 1st. All right. Back to Dario. You've told investors that you plan to be profitable starting in 28. And this is the year where we're like potentially getting the country of geniuses at a data center. And this is like going to now unlock all this progress and medicine and health and et cetera, et cetera, and new technologies. Wouldn't this be particularly, exactly the time where you'd like want to reinvest in the business and build bigger countries so they can make more discoveries?

• 59:15Speaker 01

  I mean, profitability is this kind of like weird thing in this field. I like, like, I don't think, I don't think in this field profitability is actually. A measure of, you know, kind of spending down versus investing in the business. Like, let's, let's just, let's just take a model of this. I actually think profitability happens when you underestimated the amount of demand you were going to get, and loss happens when you overestimated the amount of demand you were going to get, because you're buying the data centers ahead of time. So, think about it this way. Ideally, you would like, and again, these are stylized facts, these numbers are not exact, I'm just trying to make a toy model here.

• 59:56Speaker 01

  Let's say, half of your compute is for training, and half of your compute is for inference. And, you know, the inference has some gross margin that's like, more than 50%. And so what that means is that if you were in steady state, you build a data center, if you knew exactly, exactly, exactly the demand you were getting, you would, you know, you would, you would, you would, you would get a certain amount of revenue, say, I don't know, let's say you pay $100 billion a year for compute, and on $50 billion a year, you support $150 billion of revenue, and the other 50 billion, the other 50 billion are used for training. So, basically, you're profitable, you make, you make, you make $50 billion of profit.

• 1:00:40Speaker 01

  Those are the economics of the industry today, or, sorry, not today, but like, that's where we're, where we're projecting forward in a year or two. The only thing that makes that not the case, is if you get less demand than 50 billion, then you have more than 50% of your, your data center for research, and you're not profitable. So, you, you know, you train stronger models, but you're like, not profitable. If you get more demand than you thought, then your research gets squeezed, but, you know, you're, you're, you're kind of able to support more inference, and you're more profitable. So, it's... Maybe I'm not explaining it well, but the thing I'm trying to say is you decide the amount

• 1:01:19Speaker 01

  of compute first and then you have some target desire of inference versus versus training, but that gets determined by demand. It doesn't get determined by you.

• 1:01:29Dwarkesh Patel?

  What I'm hearing is the reason you're predicting profit is that you are systematically underestimating, underinvesting in compute, right? Because if you actually like-

• 1:01:37Speaker 01

  No, no, no. I'm saying, I'm saying it's hard to predict. So, so these things about 2028 and when it will happen, that's our, that's our attempt to do the best we can with investors. All of this stuff is really uncertain because of the cone of uncertainty. Like, we could be profitable in 2026 if the, if the revenue grows fast enough and then, and then, um, you know, if we, if we overestimate or underestimate the next year, that could swing wildly. Like, I, I, what I'm trying to get is you have a model in your head of like, the, the business, invest, invest, invest, invest, get scale and, and, and kind of then becomes profitable. There's a single point at which things turn around. I don't think the economics of this industry work that way.

• 1:02:19Dwarkesh Patel?

  I see. So if I'm understanding correctly, you're saying because of the discrepancy between the amount of compute we should have gotten and the amount of compute we got, we, we were like sort of forced to make profit, but that, that doesn't mean we're going to continue making profit. We're going to like reinvest the money because well, now AI has made so much progress and we want the bigger country of geniuses. And so then back into revenue is high, but losses are also high.

• 1:02:42Speaker 01

  If we, if we predict, if every year, we predict exactly what the demand is going to be, we'll be profitable every year. Because growth, because spending, spending 50% of your compute on, on 50% of your compute on research, roughly, um, plus a gross margin that's higher than 50% and, and correct demand prediction leads to profit. That's the profit, that's, that's the profitable business model that I think is kind of like there, but like obscured by these like building ahead and prediction errors.

• 1:03:13Dwarkesh Patel?

  I guess you're treating the 50% as a- Uh, as a sort of like, you know, just like a given constant. Whereas in fact, if you if AI progress is fast and you can increase the progress by scaling up more, you should just have more than 50% and not make profit.

• 1:03:25Speaker 01

  Here's what I'll say. You might want to scale up it more. You might want to scale it up more. But but but, you know, remember the log returns to scale, right? If if 70% would get you a very little bit of a smaller model through a factor of 1.4X, right? Like, that extra $20 billion is, you know, that each each dollar there is worth much less to you because because because the log linear setup. And so you might find that it's better to invest that that that it's better to invest that $20 billion in, you know, in in serving inference or in hiring engineers who are who are who are kind of better, who are who are kind of better, who are kind of better at what they're doing. So the the reason I said 50%, that's not that's not exactly our target.

• 1:04:08Speaker 01

  It's not exactly going to be 50%. It'll probably vary vary over time. What I'm saying is the the the like log linear return, what it leads to is you spend of order one fraction of the business, right? Like, not 5%, not 95%. And then it then it that, you know, then then you get diminishing returns because of the because of the log.

• 1:04:29Dwarkesh Patel?

  I realize that I'm like convincing Dario to like believe in AI progress or something. But like, okay, you you don't invest in research because it has diminishing returns, but you invest in the other things you mentioned.

• 1:04:39Speaker 01

  Again, again, we're talking about diminishing returns after you're spending 50 billion a year, right?

• 1:04:45Dwarkesh Patel?

  Like, this is a point I'm sure you would make, but like, diminishing returns on a genius is could be quite high. And more generally, like, what is profit in the market economy? Profit is basically saying the other companies in the market can like do more things with this money that I can't.

• 1:05:01Speaker 01

  Put aside Anthropic. I'm just trying to like, because I, you know, I don't want to give information about Anthropic. Is why I'm giving these stylized numbers. But like, let's just derive the equilibrium of the industry, right? I think the so so so why doesn't everyone spend 100% of their,

• 1:05:19Speaker 01

  you know, 100% of their compute on training and not serve any customers, right? It's because if they didn't get any revenue, they couldn't. Couldn't raise money. They couldn't do compute deals. They couldn't buy more compute the next year. So there's going to be an equilibrium where every, every company spends less than 100% on, on, on, on, on training. And certainly less than 100% on inference. It should be clear why you don't just serve the current models and, and, you know, and, and, and, and never train another model because then you don't have any demand because you'll, because you'll fall behind. So there's some equilibrium. It's, it's not going to be 10%. It's not going to be 90%. Let's just say as a stylized fact, it's 50%. That's what I'm getting at. And, and, and I think we're going to be in a position where that equilibrium of how much you spend on training

• 1:06:02Speaker 01

  is less than the gross margins that, that you're, that, that, that you're able to get on compute. And so the, the, the, the underlying economics are profitable. The problem is you have this, this hellish demand prediction problem when you're, when you're buying the next year of compute. And you might guess under and be very profitable, but have no compute for research. Or you might guess over and, you know, you're, you're, you're, um, you, you are not profitable and you have all the compute, compute for research in the world. Does, does that make sense? Just as a dynamic model of the industry?

• 1:06:37Dwarkesh Patel?

  Maybe, maybe stepping back, I'm like, uh, I, I, I'm not saying I, I think the country of geniuses is going to come in two years and therefore you should buy this compute. To me, what you're saying, the end conclusion you're arriving at makes a lot of sense. But, uh, that's because like, oh, it seems like country of geniuses is hard and there's a long way to go. And so the stepping back, the thing I'm trying to get at is more like,

• 1:07:02Dwarkesh Patel?

  it seems like your worldview is compatible with somebody who says, uh, we're like 10 years away from a world in which like we're generating trillions of dollars.

• 1:07:08Speaker 01

  And that's just, that's just not my view.

• 1:07:10Dwarkesh Patel?

  Yeah.

• 1:07:10Speaker 01

  That is, that is not my view. Like, I, I, so, so I'll like, I'll like make another prediction. It is hard for me to see that, that there won't be trillions of dollars in revenue before 2030. Um, like, uh, I can, I can construct a plausible world. It takes maybe three years. So that would, that would, you know, that would be the end of what I think it's plausible. Like in 2028, we get the, the real country of geniuses in a data. You know, the revenue's been been, you know, the revenue's been going into the maybe is in the low hundreds of billions by by by by 2028. And and and then the country of Unis accelerates it to trillions. You know, and and we're basically, we're basically on the slow end of diffusion. It takes two years to get to the trillions.

• 1:07:54Speaker 01

  That that that would that that would be the world where it takes until that would be the world where it takes until 2030. I I I suspect even composing the technical exponential and diffusion exponential will get there before 2030.

• 1:08:06Dwarkesh Patel?

  So you laid out a model where anthropic makes profit because it seems like fundamentally we're in a compute constrained world. And so it's like, eventually we keep growing compute.

• 1:08:16Speaker 01

  No, I think I think the way the profit comes is again, and you know, let's let's just abstract to the whole industry here. Like we have a, you know, let's just imagine we're we're in like an economics textbook. We have a small number of firms. Each can invest a limited amount in, or or like each can invest some fraction fraction in R and D. They have some marginal cost to serve. The margins on that, the profit margin, the gross profit margins on that marginal cost are like very high because because because inference is efficient. There's some competition, but the models are also differentiated. There's some, there's some, you know, companies will compete to push their research budgets up. But like, because there's a small number of players, you know, we have the, what is it called?

• 1:09:00Speaker 01

  The Cornell equilibrium, I think, is what the what the small number of firm equilibrium is. It, the point is, it doesn't equilibrate to perfect competition with with with with with with with zero margins. If there's like three firms, if there's three firms in the economy, all are kind of independently behaving behaving rationally. It doesn't equilibrate to zero.

• 1:09:23Dwarkesh Patel?

  Help me understand that because right now we do have three leading firms and they're not making profit. And so what, yeah, what is changing?

• 1:09:32Speaker 01

  Yeah, so the, the, again, the gross margins right now are very positive. What's hap- what, what's happening is a combination of two things. One is we're still in the exponential scale up phase of compute.

• 1:09:45Dwarkesh Patel?

  Yeah.

• 1:09:46Speaker 01

  So what, basically what that means is we're training, like a model gets trained.

• 1:09:51Dwarkesh Patel?

  Yeah.

• 1:09:51Speaker 01

  It costs, you know, let's say a model got trained that costs a billion dollars last year. And then this year it produced four billion dollars of revenue and cost one billion dollars to, to, to, to inference from. So, you know, again, I'm using stylized number here, but you know, there would be 75% you know, gross, gross, gross margins and you know, this, this 25% tax. So that model as a whole makes two billion dollars. But at the same time, we're spending 10 billion dollars to train the next model because there's an exponential scale up. And so the company loses money. Each model makes money, but the company loses money.

• 1:10:33Speaker 01

  The equilibrium I'm talking about is an equilibrium where we have the country of geniuses. We have the country of geniuses in a data center, but that, that model training scale up has equilibrated more. Maybe, maybe it's still, it's still going up. We're still trying to predict the demand, but it's more, it's more leveled out.

• 1:10:53Dwarkesh Patel?

  I'll give you a couple of things there. So let's start with the current world. In the current world, you're right that, as you said before, if you treat each individual model as a company, it's profitable.

• 1:11:05Speaker 01

  Yes.

• 1:11:05Dwarkesh Patel?

  But of course, a big part of the production function of being a frontier lab is training the next model, right?

• 1:11:12Speaker 01

  Yes, that's right.

• 1:11:13Dwarkesh Patel?

  So if you didn't do that, then you'd make profit for two months.

• 1:11:15Speaker 01

  That's right.

• 1:11:16Dwarkesh Patel?

  And then you wouldn't have margins because you wouldn't have the best model. And then, so yeah, you can make profits for two months on the current system.

• 1:11:20Speaker 01

  But at some point, that reaches the biggest scale that it can reach. And then, and then in equilibrium, we have algorithmic improvements, but we're spending roughly the same amount to train the next model as, as, as we spent to train the current model.

• 1:11:33Dwarkesh Patel?

  So this equilibrium relies.

• 1:11:36Speaker 01

  I mean, at some point, at some, at some point you run out of money in the economy.

• 1:11:40Dwarkesh Patel?

  A fixed lump of labor, a fallacy. The economy is going to grow, right? That's one of your predictions.

• 1:11:43Speaker 01

  Well, yes, but this is

• 1:11:45Dwarkesh Patel?

  Data centers in space.

• 1:11:46Speaker 01

  But this is another example of the theme I was talking about, which is that the economy will grow much faster with AI than I think it ever has before. But it's not like right now, the compute is growing 3X a year.

• 1:11:59Dwarkesh Patel?

  Yeah.

• 1:11:59Speaker 01

  I don't believe the economy is going to grow 300% a year. Like I said this in Machines of Love and Grace, like I think we may get 10 or 20% per year growth in the economy, but we're not going to get 300% growth in the economy. So I think, I think in the end, you know, if compute becomes the majority of what the economy produces, it's, it's gonna, it's gonna be capped by that.

• 1:12:20Dwarkesh Patel?

  So okay, now let's assume a model where compute stays capped.

• 1:12:23Speaker 01

  Yeah.

• 1:12:24Dwarkesh Patel?

  The world where frontier labs are making money is one where they continue to make fast progress because fundamentally your margin is limited by how good the alternative is. And so you are able to make money because you have a frontier model. If you didn't have a frontier model, you wouldn't be making money. And so this, this model requires there never to be a steady state, like forever and ever you keep making

• 1:12:48Speaker 01

  No, I don't. I don't think that's true. I mean, I feel, I feel like we're, we're like, we're taught, we're, you know, we're, I feel like this is economics. Like, you know, this is like an economics class.

• 1:12:58Dwarkesh Patel?

  Do you know the Tyler Cowen quote? We never stop talking about economics.

• 1:13:01Speaker 01

  We never, we never stop talking about economics. So, no, but, but there, there are worlds in which, you know, there, so I don't think this field's going to be, I don't think this field's going to be a monopoly. All my lawyers never want me to say the word monopoly. But I don't think this field's going to be a monopoly. But, but you do get, you get industries in which there are a small number of players, not one, but a small number of players. And ordinarily, like the, the way you get monopolies like Facebook or Meta, I always call them Facebook, but is, is these kind of net, is these kind of, these kind of network effects.

• 1:13:36Dwarkesh Patel?

  Yeah.

• 1:13:37Speaker 01

  The way you get industries in which there are a small number of players are very high costs of entry, right? So, you know, cloud is like this. I think cloud is a good example of this. You have three, maybe four players. Within cloud. I think I think that's the same for AI, three, maybe four. And the reason is that it's it's so expensive, it requires so much expertise and so much capital to like run a cloud company, right? And so you have to put up all this capital. And then in addition to putting up all this capital, you have to get all of this other stuff that like, you know, requires a lot of skill to, you know, to make it happen. And so it's like, if you go to someone and you're like, I want to disrupt this industry. Here's a hundred billion dollars. You're like, okay, I'm putting in a hundred billion dollars.

• 1:14:22Speaker 01

  And also betting that you can do all these other things that these people have been doing. Only to decrease the profit in the industry. And then and then the effect of your entering is the profit margins go down. So, you know, we have equilibrium like this all the time in the economy where we have a few, we have a few players. Profits are not astronomical, margins are not astronomical, but they're they're not zero, right? And and, you know, I think I think that's what we see on cloud. Cloud is very undifferentiated. Models are more differentiated than cloud, right? Like, everyone knows Claude is Claude Claude is good at different things than GPT is good at is than than Gemini is good at. And it's not just Claude's good at coding, GPT is good at, you know, math and reasoning, you know.

• 1:15:04Speaker 01

  It's more subtle than that, like models are good at different types of coding. Models have different styles. Like, I think I think these things are actually, you know, quite different from each other. And so I would expect more differentiation than you see in in cloud. Now, there there actually is a counter, there there is one counter argument. And that counter argument is that if all of that, the process of producing models becomes, if AI models can do that themselves, then that could spread throughout the economy. But that is not an argument for commoditizing AI models in general. That's kind of an argument for commoditizing the whole economy at once.

• 1:15:45Speaker 01

  I don't know what what quite happens in that world where basically anyone can do anything, anyone can build anything. And there's like no mode around anything at all. I mean, I don't know, maybe we want that world like like, maybe that's the maybe that's the end state here like maybe maybe, um, you know, when what maybe when when when. Kind of AI models can do, you know, when, when, when, when AI models can do everything. If we've solved all the safety and security problems, like, you know, that's one of the, one of the, one of the mechanisms for, for, uh, you know, um, uh, uh, you know, just, just kind of the economy flattening itself again, but, but that's kind of like post, like far post countries, you just isn't a data center.

• 1:16:22Dwarkesh Patel?

  Um, I'm maybe a finer way to put that, uh, potential point is one, um, it seems like AI research is especially loaded on raw intellectual power, which will be especially abundant in a world of AGI. And two, if you just look at the world today, there's very few technologies that seem to be diffusing as fast as, um, as AI algorithmic progress. And so that does hint that this industry is sort of structurally diffusive.

• 1:16:52Speaker 01

  So I think coding is going fast, but I think AI research is a super set of coding and there are aspects of it that are not going fast. Um, uh, but I, but I do think, again, once we get coding, once we get AI models going fast, then, you know, AI, you know, that will speed up the ability of AI models to kind of, to kind of do everything else. So I think while coding is going fast now, I think once the AI models are building the next AI models and building everything else, the kind of whole, the whole economy will kind of go at the same pace. I am, I am worried geographically, though. I'm a little worried that like just proximity to AI, having heard about AI, um, uh, that, that may be one differentiator. And so when I said the like, you know, 10 or 20% growth rate,

• 1:17:38Speaker 01

  a worry I have is that the growth rate could be like 50% in Silicon Valley and, you know, parts of the world that are kind of socially connected to Silicon Valley and, you know, not that much faster than its current pace elsewhere. And I think that'd be a pretty messed up world. So I, one of the things I think about a lot is how to prevent that.

• 1:17:56Dwarkesh Patel?

  Yeah. Do you think that once we have, uh, this country of geniuses at data center, that robotics is sort of quickly solved afterwards? Because it seems like a big problem with robotics is that, um, a human can learn how to teleoperate current hardware, but current AI models can't, at least not in a way that's super productive. And so if we have this ability to learn like a human, should it solve robotics immediately as well?

• 1:18:19Speaker 01

  I don't think it's dependent on learning like a human. It could happen in different ways. Again, we could have trained the model on many different video games, which are like robotic controls or many different simulated robotics environments, or just, you know, train them to control computer screens and they learn to generalize. So it will happen. It's not necessarily dependent on human-like learning. Human-like learning is one way it could happen. If the model's like, oh, I pick up a robot, I don't know how to use it, I learn. That could happen because we discovered, discovering continual learning. That could also happen because we trained the model on a bunch of environments and then generalized. Or it could happen because the model learns that in the context length. It doesn't actually matter which way. If we go back to the discussion we had like, like an hour ago,

• 1:19:04Speaker 01

  that type of thing can happen in, that type of thing can happen in several different ways.

• 1:19:08Dwarkesh Patel?

  Yeah.

• 1:19:10Speaker 01

  But, but I do think when, for whatever reason, the models have those skills, then robotics will be revolutionized, both the design of robots, because the models will be much better than humans at that. And also the, the ability to kind of control robots. So we'll get better at the physical, building the physical hardware, building the physical robots, and we'll also get better at controlling it. Now, you know, does that mean the robotics industry will also be generating trillions of dollars of revenue? My answer there is yes, but there will be the same extremely fast, but not infinitely fast diffusion. So will robotics be, be revolutionized? Yeah, maybe tack on another year or two. That's, that's my, that's the way I think about these things.

• 1:19:52Dwarkesh Patel?

  There's a general skepticism about extremely fast progress. Like here's my view, which is like, it sounds like you are going to solve continual learning one way or another within a matter of years. But just as people weren't talking about continual learning a couple of years ago, and then we realized, oh, why aren't these models as useful as they could be right now, even though they are clearly passing the Turing test and are experts in so many different domains? Maybe it's this thing. And then we solve this thing and we realize, actually, there's another, another thing that human intelligence can do and that's a basis of human labor that these models can't do. And then, so why not think there will be more things like this? Well, I think that like, Where, you know, we've like found the pieces of human intelligence.

• 1:20:30Speaker 01

  Well, well, to be clear, I mean, I think continual learning, as I've said before, might not be a barrier at all.

• 1:20:35Dwarkesh Patel?

  Yeah.

• 1:20:35Speaker 01

  Like, like, you know, I think, I think we maybe just get there by pre-training generalization and, and, and, and, and, and RL generalization. Like, I, I think there just might not be, um, there, there basically might not be such a thing at all. In fact, I would point to the history in, in ML of people coming up with things that are barriers that end up kind of dissolving within the big blob of compute, right? That, you know, people talked about, you know, you know, how do you have, you know, how do, how do your models keep track of nouns and verbs? And, you know, how do they, you know, they can understand syntactically, but they can't understand semantically. You know, it's only statistical correlations. You can understand a paragraph, but you can't understand a word.

• 1:21:19Speaker 01

  There's reasoning, you can't do reasoning, but then suddenly it turns out you can do code and math very well at all. So I, I think there actually, there's, there's actually a stronger history of some of these things seeming like a big deal. And then, and then kind of, and then kind of dissolving. Some of them are real. I mean, the need for data is real, may, maybe continual, continual, continual learning is a real thing. But again, I would ground us in something like code. Like, I think we may get to the point in like a year or two where the models can just do sui and end. Like, that's a whole task. That's a whole sphere of human activity that, that we're just saying, models can do it now.

• 1:21:59Dwarkesh Patel?

  When you say end to end, do you mean, um, setting technical direction, understanding the context of the problem?

• 1:22:05Speaker 01

  Yes.

• 1:22:05Dwarkesh Patel?

  Et cetera.

• 1:22:06Speaker 01

  Yes.

• 1:22:06Dwarkesh Patel?

  Yes, I mean, all of that. Interesting. I mean, that, that is, I feel like AGI complete. Maybe it's internally consistent, but it's not like saying 90% of code or 100% of code. It's like, no, no.

• 1:22:19Speaker 01

  No, no.

• 1:22:19Dwarkesh Patel?

  The other parts of the job as well.

• 1:22:20Speaker 01

  No, no, no, I gave this, I gave the spectrum. 90% of code, 100% of code, 90% of end tense we, 100% of end tense we. New tasks are created for SWE's. Eventually those get done as well. But the, the long spectrum there. But we're traversing the spectrum very quickly.

• 1:22:34Dwarkesh Patel?

  Yeah. I do think it's funny that I, I've seen a couple of podcasts you've done where, um, the host will be like, ah, but we're kind of sort of discussing about the continual learning thing. And it always makes me crack up because you're like, you know, you've been an AI researcher for like 10 years and that. I'm sure there's like some feeling of like, okay, so podcasts are an essay. And like every interview I get asked about it.

• 1:22:55Speaker 01

  You know, the truth of the matter is that we're all trying to figure this out together.

• 1:23:00Dwarkesh Patel?

  Yeah.

• 1:23:00Speaker 01

  Right? There are some ways in which I'm able to see things that others aren't. These days, that probably has more to do with like, I can see a bunch of stuff within Anthropic. And have to make a bunch of decisions than I have any great research insight that others don't. Right? I, you know, I'm running a 2,500 person company like it's, it's actually pretty hard for me to have have concrete research insight, you know, much harder than, you know, than it would have been, you know, 10 years ago, or, or, you know, or even two or three years ago.

• 1:23:32Dwarkesh Patel?

  As we go towards a world of a full drop in remote worker replacement, does a API pricing model still make the most sense? And if not, what is the correct way to price AGI or serve AGI?

• 1:23:45Speaker 01

  Yeah, I mean, I think there's going to be a bunch of different business models here, sort of all at once. They're going to be, that are going to be experimented with. I, I actually do think that the API

• 1:23:59Speaker 01

  model is more durable than many people think. One way I think about it is if the technology is kind of advancing quickly, if it's advancing exponentially, what that means is there's, there's always kind of like a surface area of, of kind of new use cases that have been developed in the last, in the last three months. And any kind of product surface you put in place is always at risk of sort of becoming irrelevant, right? Any given product surface probably makes sense for our, you know, a range of capabilities of the model. Right? The, the chatbot is already running into limitations of, you know, making it smarter doesn't really help the average consumer that much. But I don't think that's a limitation of AI models.

• 1:24:43Speaker 01

  I don't think that's evidence that, you know, the models are, the models are good enough and they're, they're, you know, them getting better doesn't matter to the economy. It doesn't matter to that particular product. And, and so I think the value of the API is the API. Always offers an opportunity, you know, very close to the bare metal to build on what the latest thing is. And so that, you know, there's, there's, there's kind of always going to be this, you know, this, this kind of front of new startups and new ideas that weren't possible a few months ago and are possible because the model is advancing. And, and so I, I actually, I, I kind of actually predict that we are,

• 1:25:25Speaker 01

  it's going to exist alongside other models, but we're always going to have the API business model because there's, there's always going to be a need for a thousand different people to try experimenting with the model in different way. And a hundred of them become startups and 10 of them become big, successful startups. And, you know, two or three really end up being the, the way that people use the model of a, of a given generation. So I basically think it's always going to exist. At the same time, I'm sure there's going to be other models as well, like not every token that's output by the model is worth the same amount. Think about, you know, how, how, what is the value of the tokens that are like, you know, that the model outputs when someone, you know,

• 1:26:08Speaker 01

  call, you know, someone, you know, calls them up and says, my Mac isn't working or something, you know, the model's like restart it, right? Yeah. Um, and like, you know, someone hasn't heard that before, but like, you know, the model said that like 10 million times, right? Uh, you know, that, that maybe that's worth like a dollar or a few cents or something. Whereas if, uh, the model, you know, the model goes to, you know, one of the, one of the pharmaceutical companies and it says, oh, you know, this molecule you're developing, you should take the aromatic ring from that end of the molecule and put it on that end of the molecule. Um, and, and, you know, if you do that, wonderful things will happen.

• 1:26:46Speaker 01

  Uh, like, like those tokens could be worth, you know, tens of millions of dollars, right?

• 1:26:52Speaker 01

  So, so I think we're definitely going to see business models that, that recognize that, you know, at some point we're going to see, you know, pay for results or, you know, in some, in some form, or we may see forms of compensation that are like labor. Um, you know, that that kind of work by the hour. I, I, you know, I don't know. I think, I think, I think because it's a new industry, a lot of things are going to be tried. And I, you know, I don't know what will turn out to be the right thing.

• 1:27:21Dwarkesh Patel?

  What I find, I take your point that people will have to try things to figure out what is the best way to use this blob of intelligence. But what I find striking is Claude Code. So I don't think in the history of startups, there has been a single application that has been as hotly competed in as coding agents. And, and Claude Code is a category leader here. And that seems surprising to me, like it doesn't seem intrinsically like Anthropic had to build this. I wonder if you have an accounting of why it had to be Anthropic or why, how Anthropic ended up building an application in addition to the model underlying it.

• 1:27:59Speaker 01

  Yeah, so it actually happened in a pretty simple way, which is we had our own.

• 1:28:05Speaker 01

  You know, we had our coding models, which were good at coding. And, and you know, around the beginning of 2025, I said, I think the time has come where you can have non-trivial acceleration of your own research, if you're an AI company by using these models. And of course, you know, we, you need an interface, you need a harness to use them. And so I encourage people internally, and I didn't say this is one thing that, you know, that you have to use. I just said, people should experiment with this. And then, you know, this thing, I think it might have been originally called Claude CLI. And then the name eventually got changed to Claude Code internally.

• 1:28:44Speaker 01

  Was the thing that kind of everyone was using and it was seen fast internal adoption. And I looked at it and I said, probably we should launch this externally, right? You know, it's, it's seen such fast adoption within Anthropic, like, you know, like, you know, coding is a lot of what we do. And, and so, you know, we have a, we have a audience of many, many hundreds of people that's in some ways at least representative of the external audience. So it looks like we already have product market fit. Let's launch this thing. And, and then we've launched it and, and I think, you know, just, just the fact that we ourselves are kind of developing the model. We ourselves know what we most need to use the model. I think it's it's kind of creating this feedback loop.

• 1:29:23Dwarkesh Patel?

  I see, in the sense that you, let's say a developer at Anthropic is like, ah, it it would be better if it was better at this X thing. And then you bake that into the next model that you build.

• 1:29:34Speaker 01

  That that's that's one version of it, but but then there's just the ordinary product iteration of like, you know, we have a bunch of we have a bunch of coders within Anthropic like we, um, you know, they they like use Claude code every day and so we get fast feedback. That was more important in the early days. Now, of course, there are millions of people using it. Um, and so we get a bunch of external feedback as well, but it's, you know, it's just great to be able to get, you know, kind of, kind of, uh, um, fast, fast internal feedback. You know, I think this is the reason why we launched a coding model and, you know, didn't launch a pharmaceutical company, right? It, you know, you know, my background's in in my background's in in like biology, but like, we don't have any of the resources that are needed to launch a pharmaceutical company.

• 1:30:16Dwarkesh Patel?

  So there's been a ton of hype around OpenClaw and I wanted to check it out for myself. I've got a date coming up this weekend and I don't have anything planned yet. So I gave OpenClaw a Mercury debit card. I set a couple hundred dollar limit and I said, surprise me. Okay, so here's the Mac Mini it's on. And besides having access to my Mercury, it's totally quarantined. And I actually felt quite comfortable giving it access to a debit card because Mercury makes it super easy to stub card rails. I was able to customize permissions, cap the spend and restrict the category of purchases. I wanted to make sure the debit card worked, so I asked OpenClaw to just make a test transaction. And I decided to donate a couple bucks to Wikipedia. Besides that, I have no idea what's going to happen. I will report back on the next episode about how it goes. In the meantime, if you want a personal banking solution that can accommodate all the different ways that people use their money,

• 1:31:00Dwarkesh Patel?

  even experimental ones like this one, visit mercury.com slash personal. Mercury is a fintech company, not an FDIC insured bank. Banking services provided through Choice Financial Group and Column NA, members FDIC. You know, she thinks we're getting coffee and walking around the neighborhood.

• 1:31:19Dwarkesh Patel?

  Let me ask you about now making AI go well. It seems like whatever vision we have about how AI goes well has to be compatible with two things. One is the ability to build and run AIs is diffusing extremely rapidly. And two is that the population of AIs, the amount we have in their intelligence, will also increase very rapidly. And that means that lots of people will be able to build huge populations of misaligned AIs or AIs which are just like companies which are trying to increase their footprint or have weird psyches like Sydney Bing, but now they're superhuman. What is a vision for a world in which we have an equilibrium that is compatible with

• 1:32:03Dwarkesh Patel?

  lots of different AIs, some of which are misaligned, running around?

• 1:32:05Speaker 01

  Yeah, yeah. So I think, you know, in the adolescence of technology, I was kind of, you know, skeptical of like the balance of power. But I think I was particularly skeptical of, or the thing I was specifically skeptical of is you have like three or four of these companies, like kind of all building models that are kind of derived, you know, sort of, sort of, like derived from the, like derived from the same thing and, you know, that these would check each other. Or even that kind of, you know, any number of them would, would, would, would check each other. Like, we might live in a offense dominant world where, you know, like one person or

• 1:32:45Speaker 01

  one AI model is like smart enough to do something that like causes damage for everything else. I think in the, I mean, in the short run, we have a limited number of players now. So we can start by, within the limited number of players, we, you know, we kind of, you know, we, we need to put in place the, you know, the safeguards. We need to make sure everyone does the right alignment work. We need to make sure everyone has bioclassifiers, like, you know, those are, those are kind of the immediate things we need to do. I agree that, you know, that, that doesn't solve the problem in the long run, particularly if the ability of AI models to make other AI models proliferates, then, you know, the, the whole thing can kind of, you know, can become harder to solve.

• 1:33:26Speaker 01

  You know, I think, I think in the long run, we need some architecture of governance, right? Some, some architecture of governance that preserves human freedom, but, but kind of also allows us to like, you know, govern the, the very large number of kind of.

• 1:33:42Speaker 01

  You know, uh, uh, uh, human systems, AI systems, hybrid, hybrid, human, human, um, you know, hybrid, hybrid, human AI like, you know, companies or like, or like, or like economic units. So, you know, we're good, we're gonna need to think about like, you know, how do we, how do we protect the world against, you know, bioterrorism? How do we protect the world against like, you know, against like, against like mirror life? Like, you know, probably, probably we're gonna need to, you know, need some kind of like AI monitoring system that like, you know, kind of monitors for, for all of these things. But then we need to build this in a way that like, you know, preserves civil liberties and like our constitutional rights. So I think just, just as is, as is anything

• 1:34:27Speaker 01

  else, like, it's, it's like a new security landscape with a new set of, you know, a new set of tools and a new set of vulnerabilities. And I, I think my worry is, if we had a hundred years for this to happen all very slowly, we'd get used to it, you know, like, we've gotten used to like, you know, the presence of, you know, the presence of explosives in society, or like, you know, the presence of various, um, you know, like new weapons, or the, you know, the, the presence of video cameras, um, we would get used to it over, over, over, over a hundred, and we develop governance mechanisms, we'd make our mistakes. My, my worry is just that this happening all so fast. And so I think maybe we need to do our thinking faster about how to make these governance mechanisms work.

• 1:35:12Dwarkesh Patel?

  Yeah. It seems like in a offense dominant world, over the course of the next century, so the idea is the AI is making the progress that would happen over the next century happen in some period of five to 10 years. But we would still need the same mechanisms, or balance of power would be similarly intractable, even if humans were the only game in town.

• 1:35:33Dwarkesh Patel?

  And so I guess we have the advice of AI. We, it fundamentally doesn't seem like a totally different ballgame here. If checks and balances were gonna work, they would work with humans as well. If they aren't gonna work, they wouldn't work with AIs as well. Um, and so maybe this just dooms, human checks and balances as well. But

• 1:35:50Speaker 01

  Yeah, again, I think there's some way to I think there's some way to make this happen. Like it, you know, it, it, it just, it just, you know, the governments of the world may have to work together to make it happen. Like, you know, we may have to, you may have to talk to AIs about kind of, you know, building societal structures in such a way that like these, these defenses are possible. I, I, I don't know. I mean, this is so, this is, you know, I don't want to say so far ahead in time, but like so far ahead in technological ability that may happen over a short period of time that it's hard for us to anticipate in advance.

• 1:36:24Dwarkesh Patel?

  Speaking of governments getting involved, on December 26th, the Tennessee Legislature introduced a bill which said, quote, "It would be an offense for a person to knowingly train artificial intelligence to provide emotional support, including through open-ended conversations with the user." And of course, one of the things that Claude attempts to do is be a thoughtful, thoughtful friend, thoughtful knowledgeable friend. And in general, it seems like we're going to have this patchwork of state laws. A lot of the benefits that normal people could experience as a result of AI are going to be curtailed, especially when we get into the kinds of things you discuss in Machines of Love and Grace, biological freedom, mental health improvements, et cetera, et cetera. It seems easy to imagine worlds in which these get whack a mole away by different laws.

• 1:37:08Dwarkesh Patel?

  Whereas bills like this don't seem to address the actual existential threats that you're concerned about. So I'm curious about, to understand in the context of things like this, your anthropics position against the federal moratorium on state AI laws?

• 1:37:23Speaker 01

  Yes. So I don't know. There's, there's many different things going on at once, right? I think, I think that, that, I think that particular law is dumb. Like, you know, I think it was, it was clearly made by legislators who just probably had little idea what AI models could do and not do. They're like, AI models serving us, that just sounds scary. Like, I don't want, I don't want that to happen. So, you know, we're, we're not, we're not in favor of that, right? But, but, but, you know, that, that wasn't the thing that was being voted on. The thing that was being voted on is we're going to ban all state regulation of AI for 10 years with no apparent plan to do any federal regulation of AI, which would take Congress to pass, which is a very high bar. So, you know, the idea that we'd ban states from doing anything for 10 years.

• 1:38:09Speaker 01

  And people said they had a plan for federal government, but you know, there was no actual, there was no proposal on the table, there was no actual attempt. Given the serious dangers that I lay out in Adolescence of Technology around things like the, you know, kind of biological weapons and bioterrorism, autonomy risk, and the timelines we've been talking about, like 10 years is an eternity. Like, that's, that's a, that's a, I, I think that's a crazy thing to do. So if, if that's the choice, if that's what you force us to choose, then, then we're gonna, we're gonna choose not to have that moratorium. And, you know, I, I think the, the benefits of that position exceed the costs, but it's, it's not a perfect position if that's the choice. Now, I think the thing that we should

• 1:38:53Speaker 01

  do, the thing that I would support, is the federal government should step in, not saying states you can't regulate, but here's what we're gonna do, and, and states, you can't differ from this, right? Like, I think preemption is fine in the sense of saying that federal government says, here's our standard, this applies to everyone, states can't do something different. That would be something I would support if it would be done in the right way. What, um, but, but this idea of states, you can't do anything and we're not doing anything either, that, that struck, that struck us as, you know, very much not making sense. And I think will not age well, is already starting to not age well with, with all the backlash that, that you've seen. Now, in terms of, in terms of what we would

• 1:39:37Speaker 01

  want, I mean, you know, the things we've talked about are, are starting with transparency standards, um, uh, you know, in order to monitor some of these autonomy risks and bioterrorism risks. As the risks become more serious, um, as we, as we get more evidence for them, then I think we could be more aggressive in some targeted ways and, and say, hey, AI bioterrorism is really a threat. Let's, let's pass a law that kind of forces people to have classifiers. And I could even imagine, it, it depends, it depends how serious a threat it ends up being. We don't know for sure. Then, we need to pursue this in an intellectually honest way where we say ahead of time, the risk has not emerged yet. But I could certainly imagine with the pace that things are going, that, you know, I could imagine a world where

• 1:40:21Speaker 01

  later this year, we say, hey, this, this AI bioterrorism stuff is really serious, we should do something about it. We should put it in a federal, we should, you know, put it in a federal standard. And if the federal government won't act, we should put it in a state, state standard. I could totally see that.

• 1:40:35Dwarkesh Patel?

  I, I'm concerned about a world where

• 1:40:39Dwarkesh Patel?

  If you just consider the pace of progress you're expecting, the life cycle of legislation, you know, the benefits are, as you say, because of diffusion lag, the benefits are slow enough that I really do think this patchwork of, on the current trajectory, this patchwork of state laws would prohibit, I mean, having an emotional chatbot friend is something that freaks people out, then just imagine the kinds of actual benefits from AI we want normal people to be able to experience from improvements in health and health span and improvements in mental health and so forth. Whereas at the same time, it seems like you think the dangers are already on the horizon and I just don't see that much, it seems like would be especially injurious to the benefits of AI as compared to the dangers

• 1:41:23Dwarkesh Patel?

  of AI. And so that's maybe where the cost benefit makes less sense to me.

• 1:41:27Speaker 01

  So there's a few things here, right? I mean, people talk about there being thousands of these state laws. First of all, the vast mass majority of them do not pass. And, you know, the world works a certain way in theory, but like, just because a law's been passed doesn't mean it's really enforced, right? The people, the people, you know, implementing it may be like, oh my God, this is stupid. It would mean shutting off like, you know, everything that's ever been built and everything that's ever been built in Tennessee. So, you know, very often laws are interpreted in like, you know, a way that makes them, that makes them not as dangerous or not as harmful. On the same side, of course, you have to worry if you're passing a law to stop a bad thing, you have this, you have this problem as well.

• 1:42:08Dwarkesh Patel?

  Yeah.

• 1:42:10Speaker 01

  Look, my, my, look, I mean, my basic view is, you know, if, if, if, you know, we could decide, you know, what laws were passed and how things were done, which, you know, we're only one small input into that, you know, I would deregulate a lot of the stuff around the health benefits of AI. I think, you know, I don't worry as much about the, like, the kind of chatbot laws. I actually worry more about the drug approval process, where I think AI models are going to greatly accelerate the rate at which we discover drugs and just the pipeline will get jammed up. Like, the pipeline will not be prepared to, like, process all of the stuff that's going through it. So, you know, I think, I

• 1:42:55Speaker 01

  think reform of the regulatory process to bias more towards, we have a lot of things coming where the safety and the efficacy is actually going to be Really crisp and clear, like, I mean, a beautiful thing, really, really crisp and clear and like really, really effective, but you know, and, and, and maybe we don't need all this, all this, um, uh, uh, like, um, all this superstructure around it that was designed around an era of drugs that barely work and often have serious side effects. But at the same time, I think we should be ramping up quite significantly, the, um, uh, you know, this, this kind of safety and security legislation. And, you know, like

• 1:43:35Speaker 01

  I've said, um, you know, starting with transparency is, is my view of trying not to hamper the industry, right? Trying to find the right balance. I'm worried about it. Some people criticized my essay for saying, that's too slow. The dangers of AI will come too soon if we do that. Well, basically, I kind of think like, the last six months and maybe the next few months are going to be about transparency. And then if these, if these risks emerge, when we're more certain of them, which I think we might be as soon as, as later this year, then I think we need to act very fast in the areas that we've actually seen the risk. Like, I think the only way to do this is to be nimble. Now, the legislative process is normally not nimble, but we, we need to emphasize to everyone involved the

• 1:44:20Speaker 01

  urgency of this. That's why I'm sending this message of urgency, right? That's why I wrote Adolescence of Technology. I wanted policymakers to read it. I wanted economists to read it. I want national security professionals to read it. You know, I want decision makers to read it, so that they have some hope of acting faster than they would have otherwise.

• 1:44:39Dwarkesh Patel?

  Is there anything you can do or advocate that would make it more certain that the benefits of AI are, um, are better and saturated? Where I feel like you have worked with legislatures to be like, okay, we're going to prevent bioterrorism here, we're going to increase insurgency, we're going to increase whistleblower protection. And I just think by default, the actual, like, the things we're looking forward to here, it just seems very easy, they seem very fragile to, uh, different kinds of moral panics or political economy problems.

• 1:45:09Speaker 01

  Yeah. I don't actually, so, so I don't actually agree that much in the developed world. I feel like You know, in the developed world, like markets function pretty well. And when there's, when there's like a lot of money to be made on something and it's clearly the best available alternative, it's actually hard for the regulatory system to stop it. You know, we're, we're seeing that in AI itself, right? I, you know, like a thing I've been trying to fight for is export controls on chips to China, right? And like, that's in the national security interests of the US. Like, you know, that's like square within the, you know, the, the policy beliefs of, you know, every, almost everyone in Congress of both parties. But, and you know, I think the case is very clear.

• 1:45:53Speaker 01

  The counter arguments against it are, I'll politely call them fishy.

• 1:45:59Speaker 01

  And yet, it doesn't happen and we sell the chips because there's, there's so much money. There's so much money riding on it. And, you know, the, the, that money wants to be made. And, and in that case, in my opinion, that's a bad thing. And, but, but it also, it also applies when, when it's a good thing. And, and so I, I don't think that if we're talking about drugs and benefits of the technology, I, I, I, I am not as worried about those benefits being hampered in the developed world. I am a little worried about them going too slow. And I, as I said, I do think we should work to speed the approval process in the FDA. I do think we should fight against these chatbot bills that you're describing, right?

• 1:46:43Speaker 01

  Described individually, I'm against them. I think they're stupid. But I actually think the bigger worry is the developing world. Where we don't have functioning markets, where, um, you know, we often can't build on the technology that, that we've had. I worry more that those folks will get left behind. And I worry that even if the cures are developed, you know, maybe there's someone in rural Mississippi who, who doesn't get it as well, right? That's a, that's a, that's a kind of smaller version of the thing, the concern we have in the, in the developing world. And so the things we've been doing are, you know, you know, we work with, you know, we work with, you know, philanthropists, right? You know, we work with folks.

• 1:47:22Speaker 01

  Who, you know, who, you know, deliver, you know, medicine and health interventions to, you know, to, to developing world, to Sub-Saharan Africa, you know, India, Latin America, you know, you know, other, other developing parts of the world. That's the thing I think that won't happen on its own.

• 1:47:40Dwarkesh Patel?

  Hmm. You mentioned export controls.

• 1:47:42Speaker 01

  Yeah.

• 1:47:42Dwarkesh Patel?

  Why can't US and China both have a country of geniuses on a data center?

• 1:47:47Speaker 01

  Why can't, you know, why won't it happen? Or why shouldn't it happen?

• 1:47:51Dwarkesh Patel?

  Why shouldn't it happen?

• 1:47:52Speaker 01

  Um, you know, I think, I think if this does happen, um, you know, then, then we kind of have a, well, we could have a few situations. If we have like an offense dominant situation, we could have a situation like nuclear weapons, but like more dangerous, right? Where it's like, um, you know, kind of, kind of either side could, could easily destroy everything. We could also have a world where it's kind of, it's unstable, like the nuclear equilibrium is stable, right? Because it's, you know, it's like deterrence. But let's say there were uncertainty about like if the two AIs fought, which AI would win. Um, that could create instability, right? You often have conflict when the two sides have a different assessment of their likelihood of winning, right? If one side is like, oh yeah, there's

• 1:48:35Speaker 01

  a 90% chance I'll win. And the other side's like, there's a 90% chance I'll win, then, then, then a fight is much more likely. Um, they can't both be right, but they can both think that.

• 1:48:44Dwarkesh Patel?

  But this is like a fully general argument against the diffusion of AI technology, which it may, which is the implication of this world.

• 1:48:51Speaker 01

  Let me, let, let me just go on because I think we will get diffusion eventually. The other concern I have is that people, the governments will oppress their own people with AI. And, and, and so, um, you know, I'm, I'm just, I'm worried about some world where you have a country that's already, uh, you know, kind of a, uh, you know, uh, you know, there's, there's a government that kind of, kind of already, um, you know, is, is kind of, kind of building a, you know, a tech, a high-tech authoritarian state. Um, and to be clear, this is about the government. This is not about the people, like people, we need to find a way for people everywhere to benefit. Um, my worry here is about governments. So yeah, my, you know, my, my worry is if the world gets carved up into two pieces,

• 1:49:32Speaker 01

  one of those two pieces could be authoritarian or totalitarian in a way that's very difficult to displace. Um, now, will, will governments eventually get powerful AI? And, and, you know, there's risk of authoritarianism? Yes. Will governments eventually get powerful AI? And there's risk of, um, You know, of kind of bad, bad, bad equilibria. Yes, I think both things, but the initial conditions matter. Right? You know, at, at some point, we're neat, we're going to need to set up the rules of the road. I'm not saying that one country, either the United States or a coalition of democracies, which I think would be a better setup, although it requires more international cooperation than we currently seem to want to make.

• 1:50:13Speaker 01

  But, you know, I don't, I don't think a coalition of democracies or certainly one country should just say, these are the rules of the road. There's going to be some negotiation, right? The world is going to have to grapple with this. And what I would like is that the, the, you know, the democratic nations of the world, those with, you know, who are, whose governments have, represent closer to pro-human values, are, are holding the stronger hand then, have, have more leverage when the rules of the road are set. And, and so I'm, I'm very concerned about that initial condition.

• 1:50:47Dwarkesh Patel?

  I was re-listening to an interview from three years ago. And one of the ways it aged poorly is that I kept asking questions, assuming there's going to be some key fulcrum moment two to three years from now, when in fact, being that far out, it just seems like progress continues, AI improves, AI is more diffused, and people will use it for more things. It seems like you're imagining a world in the future where the countries get together and here's the rules of the road, and here's the leverage we have, here's the leverage you have, when it seems like on current trajectory, everybody will have more AI. Some of that AI will be used by authoritarian countries, some of that within the authoritarian countries will be raised by private actors versus state actors. It's not clear who will benefit more. It's always unpredictable to tell in advance. You know, it seems like the internet privileged authoritarian countries more than you would

• 1:51:31Dwarkesh Patel?

  have expected. And maybe the AI will be the opposite way around. So I, I want to better understand what you're imagining here.

• 1:51:39Speaker 01

  Yeah, yeah. So, so just to be precise about it, I think the exponential of the underlying technology will continue as it has before, right? The models get smarter and smarter, even when they get to country of geniuses in a data center, you know, I, I think you can continue to make the model smarter. There's a question of like, Getting diminishing returns on their value in the world, right? How much does it matter after you've already solved human biology? Or, you know, at some point you can do harder math. You can do more abstruse math problems, but nothing after that matters. But putting that aside, I do think the exponential will continue. But there will be certain distinguished points on the exponential.

• 1:52:20Speaker 01

  And companies, individuals, countries will reach those points at different times. And so, you know, there's, you know, could there be something, you know, I talk about is a nuclear deterrent still in adolescence of technology? Is a nuclear deterrent still stable in the world of AI? I don't know, but that's, that's an example of like one thing we've taken for granted that like the technology could reach such a level that it's no longer like, you know, we can no longer be certain of it at least. You know, think of, think of others, you know, there, there, you know, there, there are kind of points where if you, if you reach a certain point, you maybe you have offensive cyber dominance and like every, every computer system is transparent to you after that.

• 1:53:02Speaker 01

  Unless the other side has, has a kind of equivalent defense. So I don't know what the critical moment is or if there's a single critical moment. But I think there will be either a critical moment, a small number of critical moments, or some critical window where it's like AI is, AI confers some large advantage from the perspective of national security. And one country or coalition has reached it before others. That, that, you know, that, you know, I'm not advocating that they're just like, okay, we're in charge now. That's not, that's not how, that's not how I think about it. You know, that there's always the, the other side is catching up. There's extreme actions you're not willing to take.

• 1:53:45Speaker 01

  And, and it's not right to take, you know, to take complete, to take complete control anyway. But, but at the point that that happens, I think people are going to understand that the world has changed. And there, there's going to be some negotiation implicit or implicit about what, what is the, what is the post-AI world order look like? And, and I think my interest is in. You know, making that negotiation be one in which, you know, classical liberal democracy has, you know, has a strong hand.

• 1:54:23Dwarkesh Patel?

  Well, I want to understand what that better means because you say in the essay, quote, autocracy is simply not a form of government that people can accept in the post-powerful AI age. And that sounds like you're saying the CCP as an institution cannot exist after we get AGI. And that seems like a, like a very strong demand. And it seems to imply a world where the leading lab or the leading country will be able to, and by that language should, get to determine how the world is governed or what kinds of governments are allowed and not allowed.

• 1:54:59Speaker 01

  Yeah, so when, when I, I believe that paragraph was, I think I said something like, you could take it even further and say X. So I wasn't, I wasn't necessarily endorsing that, that, that, I wasn't necessarily endorsing that view. I, you know, I was saying like, here's, first, you know, here's a weaker thing that I believe. But, you know, I think, you know, I think I said, you know, we have to worry a lot about authoritarians. And, you know, we should try and, you know, kind of, kind of check them and limit their power. Like, you could take this kind of further, much more interventionist view that says like, authoritarian countries with AI are these, you know, the, you know, these kind of self fulfilling cycles that you can't, that are very hard to displace. And so you just need to get rid of them from, from the beginning.

• 1:55:43Speaker 01

  That, that has exactly all the problems you say, which is, you know, you know, if you were to make a commitment to overthrowing every authoritarian country, I mean, they, then they would take a bunch of actions now that like, you know, that, that, that could, could lead to instability. So that, that may, you know, that, that, that just, that just may not be possible. But the point I was making that I do endorse is that it is, it is quite possible that, you know, today, you know, the view, or at least my view, or the view in most of the Western world is, is democracy is a better form of government than authoritarianism. But it's not like if a country's authoritarian. We don't react the way we reacted if they committed a genocide or something, right?

• 1:56:25Speaker 01

  And, and I'm, I guess what I'm saying is I'm a little worried that in the age of AGI, authoritarianism will have a different meaning. It will be a graver thing. And, and we have to decide one way or another how, how, how, how to deal with that. And the interventionist view is one possible view. I was exploring such views. You know, it may end up being the right view. It, it may end up being too extreme to be the right view. But I do have hope. And, and one piece of hope I have is there, there is, we have seen that as new technologies are invented,

• 1:57:02Speaker 01

  forms of government become obsolete. I, I mentioned this in Adolescence of Technology where I said, you know, like feudalism was basically, you know, like a form of government, right? And, and then when, when we invented industrialization, feudalism was no longer sustainable, no longer made sense.

• 1:57:21Dwarkesh Patel?

  Why is that hope? Couldn't that imply that democracy is no longer going to be a competitive system?

• 1:57:25Speaker 01

  It could, right. It could go, it could go either way, right? But, but I actually, so I, these problems with authoritarianism, right? That the problems with authoritarianism get deeper. I just, I wonder if that's an indicator of other problems that authoritarianism will have, right? In other words, people become, because authoritarianism becomes worse, people are more afraid of authoritarianism. They work harder to stop it. It's, it's more of a, like, you have to think in terms of total equilibrium, right? I just wonder if it will motivate new ways of thinking about with, with, with the new technology,

• 1:58:10Speaker 01

  how to preserve and protect freedom. And, and even more optimistically, will it lead to a collective reckoning? And, you know, a, a, a kind of, a more emphatic realization of how important some of the things we take as individual rights are, right? A more emphatic realization that we just, we really can't Give these away. There's there. We've seen. There's no other way to live that actually works. I I I I am actually. I am actually hopeful that I I guess one way to say it, it sounds too idealistic, but I actually believe it could be the case is is that is that dictatorships become morally obsolete. They become morally unworkable forms of government.

• 1:58:54Speaker 01

  And that and that and that the the the crisis that that creates is is is sufficient to force us to find another way.

• 1:59:03Dwarkesh Patel?

  I think there is genuinely a tough question here, which I'm not sure how you resolve. And we've had to come out one way or another on it through history, right? So with China in the seventies and eighties, we decided, even though it's an authoritarian system, we will engage with it. And I think in retrospect, that was the right call because it has stayed a authoritarian system, but a billion plus people are much wealthier and better off than they would have otherwise been. And it's not clear that it would have stopped being an authoritarian country otherwise. You can just look at North Korea as an example of that, right? And I don't know if that takes that much intelligence to remain an authoritarian country that continues to coalesce its own power. And so you can just imagine a North Korea with an AI that's much worse than everybody else's, but still enough to keep power.

• 1:59:46Dwarkesh Patel?

  And and and then so in general, it seems like should we just have this attitude of the benefits of AI will in the form of all these empowerments of humanity and health and so forth will be big. And historically, we have decided it's good to spread the benefits of technology widely, even with even to people whose governments are authoritarian. And I think I guess it is a tough question how to think about it with AI. But historically, we have said, yes, there's a positive some world, and it's still worth diffusing the technology.

• 2:00:14Speaker 01

  Yeah, so so there are a number of choices we have. I you know, I think framing this as a kind of government to government decision. And you know, in in national security terms, that's like one lens, but there are a lot of other lenses. Like you could imagine a world where, you know, we produce all these cures to diseases. And like the, you know, the the the cures to diseases are fine to sell to authoritarian countries. The data centers just aren't right. The chips and the data centers just aren't. The AI industry itself. You know, like, like another possibility is, and I think folks should think about this, like, you know, could there be developments we can make either that naturally happen as a result of AI? Or that we could make happen by building technology on AI?

• 2:00:59Speaker 01

  Could we create an equilibrium where where it becomes infeasible for authoritarian countries to deny their people? Kind of private use of the benefits of the technology. You know, are there, are there, are there, are there equilibrium where we can kind of? Give everyone in authoritarian country their own AI model that kind of, you know, like defends themselves from surveillance? And there isn't a way for the authoritarian country to like crack, crack down on this while, while retaining power? I don't know that that sounds to me like if that went far enough, it would be, it would be a reason why authoritarian countries would disintegrate from the inside. But, but maybe there's a middle world where like there, there's an equilibrium where if they want to hold on to power, the authoritarians?

• 2:01:40Speaker 01

  Can't deny kind of individualized access, access to the technology. But I actually do have a hope for the, for the, for the, for the more radical version, which is, you know, is it possible that the technology might? Inherently have properties or that by building on it in certain ways, we could create properties? That, that, that have this kind of dissolving effect on authoritarian structures now? We, we hoped originally, right? We think back to the beginning of the Obama administration, we thought originally that? That, you know, social media and, and the internet would have that property and turns out not to. But, but I don't know what, what if we could, what if we could try again with, with the knowledge of how many things could go wrong? And that this is a different technology? I don't know that it would work, but it's worth a try.

• 2:02:25Dwarkesh Patel?

  Yeah, I, I think it's just, it's very unpredictable, like there's first principles reasons why authoritarianism.

• 2:02:30Speaker 01

  It's all, it's all very unpredictable. I, I don't think, I mean, we got it, we, we just got to, we kind of. We got to recognize the problem and then we got to come up with 10 things we can try and we got to try those and then assess whether they're working or which ones are working, if any, and then try new ones if the old ones aren't.

• 2:02:46Dwarkesh Patel?

  But I guess what that nets out to today is you say, we will not sell data centers, or sorry, chips and then the ability to make chips to China. And so in some sense, you are denying there will be some benefits to

• 2:02:59Speaker 01

  That's right.

• 2:02:59Dwarkesh Patel?

  the Chinese economy, Chinese people, et cetera, because we're doing that. And then there'd also be benefits to the American economy because it's a positive sum world, we could trade, they could have their country's data centers doing one thing, we could have ours doing another. And already, you're saying it's not worth that positive sum stipend to empower this country.

• 2:03:18Speaker 01

  What I would say is that, you know, we are about to be in a world where growth and economic value will come very easily, if we're able to build these powerful AI models. Growth and economic value will come very easily. What will not come easily is distribution of benefits, distribution of wealth, political freedom. You know, these are the things that are going to be hard to achieve. And so when I think about policy, I think that the technology and the market will deliver all the fundamental benefits, you know, almost faster than we can take them. And that these questions about about distribution and political freedom and rights are the

• 2:04:01Speaker 01

  ones that that will actually matter and that policy should focus on.

• 2:04:04Dwarkesh Patel?

  Okay, so speaking of distribution, as you're mentioning, we have developing countries and in many cases, catch up growth has been weaker than we would have hoped for. But when catch up growth does happen, it's fundamentally because they have underutilized labor. And we can bring the capital and know-how from developed countries to these countries and then they can grow quite rapidly.

• 2:04:22Speaker 01

  Yes.

• 2:04:23Dwarkesh Patel?

  Obviously, in a world where labor is no longer the constraining factor, this mechanism no longer works. It's just the hope, basically, to rely on philanthropy from the people who immediately get wealthy from AI or from the countries that get wealthy from AI. What is the hope for?

• 2:04:38Speaker 01

  I mean, philanthropy should obviously play some role as it has, you know, as it has in the past, but I think growth is always Growth is always better and stronger if we can make it endogenous.

• 2:04:49Dwarkesh Patel?

  Yeah.

• 2:04:49Speaker 01

  So, you know, what are the relevant industries in like, in like, in like, in like, in like, an AI driven world? Look, there's lots of stuff, you know, like, there's, you know, I said, I said, we shouldn't build data centers in China, but there's no reason we shouldn't build data centers in Africa, right? Um, in fact, I think it'd be great to build data centers in Africa. Um, you know, as long as they're not owned by China, we should, we should build, we should build data centers in Africa. I think that's a, that's, that's, I think that's a great thing to do. Um, you know, we should also build, you know, there's no reason we can't build, you know, a pharmaceutical industry that's like AI driven, like, you know, the, the, if, if AI is accelerating, accelerating drug discovery, then, you know, there will be a bunch of biotech startups, like, let's make sure some of those happen in the developing world.

• 2:05:30Speaker 01

  And certainly during the transition, I mean, we can talk about the point where humans have no role, but, but humans will have, still have some role in starting up these companies and supervising, supervising the AI models. So let's make sure some of those humans are humans in the developing world so that fast growth can happen there as well.

• 2:05:45Dwarkesh Patel?

  Hmm. You guys recently announced Claude is going to have a constitution that's aligned to a set of values and not necessarily just to the end user. And there's a world you could imagine where if it is aligned to the end user, it preserves the balance of power we have in the world today because everybody gets to have their own AI that's advocating for them. And so the ratio of bad actors to good actors stays constant. It seems to work out for our world today.

• 2:06:07Dwarkesh Patel?

  Why is it better not to do that, but to have a specific set of values that the AI should carry forward?

• 2:06:14Speaker 01

  Uh, yeah, so I, I'm not sure I'd quite draw the distinction in that way.

• 2:06:18Dwarkesh Patel?

  Yeah.

• 2:06:18Speaker 01

  Um, there, there are maybe two relevant distinctions here, which are, I think you're talking about a mix of the two. Like, one is, should we give the model a set of instructions about do this and versus don't do this?

• 2:06:30Dwarkesh Patel?

  Yeah.

• 2:06:31Speaker 01

  And the other, you know, versus, should we give the model a set of principles for, you know, for kind of how to act? And, and, and there, it's, it's, you know, it's, it, it, you know, it's, it's just, it's kind of purely a practical and empirical thing that we've observed that by teaching the model principles, getting it to learn from principles, its behavior is more consistent, it's easier to cover edge. And the model is more likely to do what people want it to do. In other words, if you know, if you're like, you know, don't tell people how to hotwire a car. Don't speak in Korean, don't, you know, you know, just, you know, if you give it a list of rules. It doesn't really understand the rules and it's kind of hard to generalize from them.

• 2:07:14Speaker 01

  You know, if it's just kind of a like, you know, list of do's and don'ts. Whereas if you give it principles and then, you know, it has some hard guardrails, like don't make biological weapons. But overall, you're trying to understand what it should be aiming to do, how it should be aiming to operate. So just from a practical perspective, that turns out to be just a more effective way to train the model. That's one piece of it. So that, you know, that's the kind of rules versus principles trade off. Then there's another thing you're talking about, which is kind of like the courage ability versus.

• 2:07:46Speaker 01

  Like, you know, I would say kind of intrinsic motivation trade off, which is like, how much should the model be a kind of, I don't know, like a skin suit or something where, you know, you know, you know, you just kind of, you know, it just kind of directly follows the instructions that are given to it by whoever is giving it those instructions. Versus how much should the model have an inherent set of values and go off and do things on its own. And, and, and, and there I would actually say. Everything about the model is actually closer to the direction of, of like, you know, it should mostly do what people want. It should mostly follow the, we're not trying to build something that like, you know, goes off and runs the world on its own.

• 2:08:29Speaker 01

  We're actually pretty far on the corrigible side. Now, now what we do say is there are certain things that the model won't do, right? That it's like, you know, that, that, that. I think we say it in various ways in the constitution. That under normal circumstances, if someone asks the model to do a task, it should do that task, that that should be the default. But if you've asked it to do something dangerous, or if you've, you know, if you've asked it to. You know, to kind of harm someone else, then the model is unwilling to do that. So I actually think of it as like a mostly, a mostly corrigible model that has some limits, but those limits are based on principles.

• 2:09:09Dwarkesh Patel?

  Yeah, I mean, then the fundamental question is, how are those principles determined? And this is not a special question for Anthropic, this would be a question for any AI company. But because you have been the ones to actually write down the principles, I get to ask you this question. Normally, a constitution is like, you write it down, it's set in stone, and there's a process of updating it and changing it and so forth. In this case, it seems like a document that people at Anthropic write, that can be changed at any time, that guides the behavior of systems that are going to be the basis of a lot of economic activity. What is the, how do you think about how those principles should be set?

• 2:09:49Speaker 01

  Yes, so I think there's, there's two, there's maybe three, three kind of sizes of loop here, right? Three, three ways to iterate. One is you can iterate, we iterate within Anthropic, we train the model, we're not happy with it, and we kind of change the constitution. And I think that's good to do. And, you know, putting out publicly, you know, making updates to the constitution every once in a while, saying, here's a new constitution.

• 2:10:11Dwarkesh Patel?

  Right.

• 2:10:11Speaker 01

  I think that's good to do because people can comment on it. The second level of loop is different companies will have different constitutions. And, you know, I think it's useful for like Anthropic puts out a constitution and, you know, Gemini model puts out a constitution and, you know, other companies put out a constitution and then they can kind of look at them, compare, outside observers can critique and say, this, this, I like this one, this thing from this constitution and this thing from that constitution. And then kind of that, that creates some kind of, you know, soft incentive and feedback for all the companies to like, take the best of each elements and improve. Then I think there's a third loop, which is, you know, society beyond the AI companies

• 2:10:52Speaker 01

  and beyond just those who kind of, you know, who, who comment on the constitutions without hard power. And, and there, you know, we've done some experiments, like, you know, a couple of years ago, we did an experiment with, I think it was called the collective intelligence project to like, you know, to, to basically poll people and ask them what should be in our AI constitution. And, and, you know, I think at the time we incorporated some of those changes. And so you could imagine with the new approach we've taken to the constitution, doing something like that. It's a little harder because it's like that was actually an easier approach to take when the constitution was like a list of do's and don'ts. At the level of principles, it has to have a certain amount of coherence. But but you could you could still imagine getting views from a wide variety of people.

• 2:11:37Speaker 01

  And I think you could also imagine, and this is like a crazy idea, but hey, you know, this whole interview is about crazy ideas, right? So, you know, you could even imagine systems of kind of representative government having having input, right? Like, you know, I wouldn't I wouldn't do this today because the legislative process is so slow. Like, this is exactly why I think we should be careful about the legislative process and AI regulation. But there's no reason you couldn't in principle say like, you know, all AI, you know, all AI models have to have a constitution that starts with like these things. And then like you can append, you can append other things after it, but like there has to be this special section that like takes precedence. I wouldn't do that. That's too rigid.

• 2:12:17Speaker 01

  That that sounds, you know, that that that that sounds kind of overly prescriptive in a way that I think overly aggressive legislation is. But like, that is a thing you could, you know, like, like, that is, that is a thing you could try to do. Is there some much less heavy handed version of that? Maybe.

• 2:12:33Dwarkesh Patel?

  I really like control loop two, where obviously this is not how constitutions of actual governments do or should work, where there's not this vague sense in which the Supreme Court will feel out how people are feeling and what are their vibes and then update the constitution accordingly. So there's

• 2:12:50Speaker 01

  Yeah.

• 2:12:50Dwarkesh Patel?

  With actual governments, there's a more procedural

• 2:12:53Speaker 01

  Formal process.

• 2:12:54Dwarkesh Patel?

  Exactly. But you actually have a vision of competition between constitutions, which is actually very reminiscent of how some libertarian charter cities people used to talk about what an archipelago of different kinds of governments could look like. And then there would be selection among them of who could operate the most effectively, in which place people would be the happiest. And in a sense, you're actually, yeah, there's this vision.

• 2:13:17Speaker 01

  I'm kind of recreating that.

• 2:13:19Dwarkesh Patel?

  Yeah, yeah, like the utopia of archipelagos.

• 2:13:21Speaker 01

  Again, you know, I think I think that vision has Has, you know, if things to recommend it and things that things that things that will kind of kind of go wrong with it. You know, I think I think it's a, I think it's an interesting in some ways compelling vision, but also things will go wrong with it that you hadn't. That you hadn't imagined. So, you know, I, I, I like loop two as well, but I, I, I feel like. The whole thing has got to be some, some mix of loops, one, two, and three, and it's, it's a matter of the proportions, right? I, I think that's got to be the, the answer.

• 2:13:52Dwarkesh Patel?

  When somebody eventually writes the equivalent of the making of the atomic bomb for this era. What is the thing that will be hardest to glean from the historical record that they're most likely to miss?

• 2:14:04Speaker 01

  I think a few things. One is at every moment of this exponential. The extent to which the world outside it didn't understand it. This is, this is a bias that's often present in history where anything that actually happened looks inevitable in retrospect. And, and so, you know, I, I think when people, when people look back, it will be hard for them to put themselves in the place. Of people who were actually making a bet on this thing to happen that. Wasn't inevitable that we had these arguments, like the arguments that, you know, that I make for scaling or that continual learning will be solved. Uh, uh, you know, that, that, you know, some of us internally in our heads.

• 2:14:47Speaker 01

  Put a high probability on this happening, but, but it's like, there's, there's a world outside us that's not, that's not acting on, that's not kind of not acting on that at all. Uh, and, and, and I think, I think the, the weirdness of it. I, I, I think unfortunately, like the insularity of it, like, you know.

• 2:15:07Speaker 01

  If, if we're one year or two years away from it happening, like the average person on the street has no idea. And that's one of the things I'm trying to change, like with the memos, with talking to policymakers, but like. I don't know. I think, I, I, I think that's just a, that's just like a crazy, that's just like a crazy thing.

• 2:15:23Dwarkesh Patel?

  Yeah.

• 2:15:24Speaker 01

  Finally, I would say, and this probably applies to almost all historical moments of crisis, how absolutely fast it was happening, how everything was happening all at once. And so decisions that you might think, you know, were kind of carefully calculated, well, actually, you have to make that decision and then you have to make 30 other decisions on the same day because it's all happening so fast. And you don't even know which decisions are going to turn out to be consequential. So, you know, one of my, I guess worries, although it's also an insight into, you know, into kind of what's happening is that, you know, some very critical decision will be, will be some decision that, you know, someone just comes into my office and is like, Dario, you have two minutes, like, you know, should

• 2:16:08Speaker 01

  we, should we do, you know, should we do thing, thing A or thing B on this, like, you know, someone gives me this random, you know, half-page, half-page memo and is like, should we, should we do A or B? And I'm like, I don't know, I have to eat lunch, let's do B. And, you know, that ends up being the most consequential thing ever.

• 2:16:25Dwarkesh Patel?

  Hmm. So, final question. It seems like you have, there's not tech CEOs who are usually writing 50-page memos every few months. And it seems like you have managed to build a role for yourself and a company around you, which is compatible with this more intellectual type role as CEO. And I want to understand how you construct that and how, like, how does that work to be, you just go away for a couple of weeks and then you tell your company, this is the memo, like, here's what we're doing. It's also reported, you write a bunch of these internally?

• 2:16:58Speaker 01

  Yeah. So, I mean, for this particular one, you know, I wrote it over winter break. So, there was the time, you know, and I was having a hard time finding the time to actually find it, to actually write it. But I actually think about this in a broader way. I actually think it relates to the culture of the company. So, I probably spend a third, maybe 40% of my time, making sure the culture of Anthropic is good. As Anthropic has gotten larger, it's gotten harder to just, you know, get involved in like, you know, directly involved in like the training of the models, the launch of the models, the building of the products. Like, it's 2,500 people. It's like, you know, there's just, you know, I have certain instincts, but like, there's Only, you know, I, I, it's very difficult to get in, to get, to get involved in every single detail. You know, I like, I, I try as much as possible.

• 2:17:42Speaker 01

  But one thing that's very leveraged is making sure Anthropic is a good place to work. People like working there. Everyone thinks of themselves as team members. Everyone works together instead of against each other. And, you know, we've seen as some of the other AI companies have grown without naming any names. You know, we're starting to see decoherence and people fighting each other. And, you know, I would argue there was even a lot of that from the beginning, but, but, you know, that it's, it's gotten worse, but I, I think we've done an extraordinarily good job, even if not perfect, of holding the company together, making everyone feel the mission that we're sincere about the mission. And that, you know, everyone has faith that everyone else there is working for the right reason, that we're a team,

• 2:18:25Speaker 01

  that people aren't trying to get ahead at each other's expense or backstab each other, which again, I think happens a lot at some of the other places. And, and how do you make that the case? I mean, it's a lot of things, you know, it's me, it's, it's, it's Daniela, who, you know, runs the company day to day, it's the co-founders, it's the other people we hire, it's the environment we try to create. But I think an important thing in the culture is I, some, and you know, the, the, you know, the other leaders as well, but especially me, have to articulate what the company is about, why it's doing what it's doing, what its strategy is, what its values are, what its mission is and what it stands for. And, um, you know, when you get to 2,500 people, you can't do that person by person.

• 2:19:09Speaker 01

  You have to write or you have to speak to the whole company. This is why I get up in front of the whole company every two weeks and speak for an hour. It's actually, I mean, I wouldn't say I write essays internally, I do two things. One, I write this thing called DVQ, Dario Vision Quest. Uh, I wasn't the one who named it that, that's the name it, it received. And it's one of these names that I kind of, I tried to fight it because it made it sound like I was like going off and smoking peyote or something. But, but the name just stuck. So I get up in front of the company. Every two weeks, I have like a three or four page document and I just kind of talk through like three or four different topics about what's going on internally, the, you know, the, the models we're producing, the products, the outside industry, the world

• 2:19:53Speaker 01

  as a whole, as it relates to AI and geopolitically in general, you know, just some mix of that. And I just go through very, very honestly. I just go through and I just, I just say, you know, this, this is what I'm thinking. This is what anthropic leadership is thinking. And then I answer questions and, and that direct connection, I think, has a lot of value that is hard to achieve when you're passing things down the chain, you know, six, six levels deep. And, you know, a large fraction of the company comes, comes to attend, either, either in person or, um, either in person or virtually. And it, you know, it really means that you can communicate a lot. And then the other thing I do is I just, you know, I have a channel in Slack where I just write a bunch of things and

• 2:20:34Speaker 01

  comment a lot. And often that's in response to, you know, just things I'm seeing at the company or questions people ask or like, you know, we do internal surveys and there are things people are concerned about. And so I'll write them up. And I'm like, I'm, you know, I'm, I'm, I'm just, I'm very honest about these things, you know, I just, I just say them very directly. And the point is to get a reputation of telling the company the truth about what's happening, to call things what they are, to acknowledge problems, to avoid the sort of corpo speak, the kind of defensive communication that often is necessary in public because, you know, the world is very large and full of people who

• 2:21:14Speaker 01

  are, you know, interpreting things in bad faith. But, you know, if you have a company of people who you trust and we try to hire people that we trust, then, then, you know, you can, you can, you can, you know, you can, you can really just be entirely unfiltered. And, uh, you know, I think, I think that's an enormous strength of the company. It makes it a better place to work. It makes people more, you know, more of the sum of their parts and increases likelihood that we accomplish the mission because everyone is on the same page about the mission and everyone is debating and discussing how best to accomplish the mission.

• 2:21:45Dwarkesh Patel?

  Well, in lieu of an external Daria vision quest, we have this interview.

• 2:21:49Speaker 01

  This, this interview is a little like that.

• 2:21:51Dwarkesh Patel?

  Uh, this is fun, Daria. Thanks for doing it.

• 2:21:54Speaker 01

  Yeah. Thank you, Druks. Hey, everybody. I hope you enjoyed that episode. If you did, the most helpful thing you can do is just share it with other people who you think might enjoy. It's also helpful if you leave a rating or a comment on whatever platform you're listening on.

• 2:22:09Dwarkesh Patel?

  If you're interested in sponsoring the podcast, you can reach out at dwarkesh.com/adandise. Otherwise, I'll see you on the next one.