Anthropic CEO Dario Amodei: AI's Potential, OpenAI Rivalry, GenAI Business, Doomerism

Dario Amodei is the CEO of Anthropic. He joins Big Technology Podcast for a raw, unfiltered conversation about AI's potential, its stakes, and the source of his outspokenness. The interview is part of a major Big Technology profile about Amodei, ft. interviews with more than two dozen of his colleagues, friends, and critics. You can read that story for free here: https://www.bigtechnology.com/p/the-making-of-dario-amodei We also cover Mark Zuckerberg's AI talent spending spree, Amodei's rivalry with OpenAI, his response to NVIDIA CEO Jensen Huang's criticism, the state of the AI scaling laws, and whether there's a risk we'll lose control of the technology. Tune in for a candid, emotional, and high energy conversation with one of the leaders driving the AI wave today. --- Audio and Video by Matt Breetzke of Fireside Visuals: https://www.firesidevisuals.com/ --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b Questions? Feedback? Write to: [email protected] Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay?

• 0:43Alex Kantrowitz?

  Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 1:04Speaker 01

  I get very angry when people call me a doomer. When, you know, when someone's like, this guy's a doomer, he wants to slow things down. You heard what I just said, like, you know, my father died because of, you know, cures that, you know, could have, could have happened a few years later. I understand the benefit of this technology.

• 1:20Alex Kantrowitz?

  I'm sure you've heard the criticism from people like Jensen, who say, well, Dario thinks he's the only one who can build this safely and therefore wants to control the entire industry.

• 1:30Speaker 01

  I've never said, I've never said anything like that. That's an outrageous lie. That's the most outrageous lie I've ever heard.

• 1:36Alex Kantrowitz?

  Anthropic CEO Dario Amode joins us to talk about the path forward for artificial intelligence, whether generative AI is a good business, and to fire back at those who call him a doomer. And he's here with us in studio at Anthropic headquarters in San Francisco. Dario, it's great to see you again. Welcome to the show.

• 1:56Speaker 01

  Thank you for having me.

• 1:57Alex Kantrowitz?

  So let's recap the past couple of months for you. You said AI could wipe out half of entry-level white-collar jobs. You cut off WindSour's access to Anthropic's top-tier models when you learned that OpenAI was going to acquire them. You asked the government for export controls and annoyed Nvidia CEO Jensen Huang. What's gotten into you?

• 2:19Speaker 01

  You know, I think, I think Anthropic... Myself and Anthropic are always focused on kind of trying to do and say the things that we believe. And I think as we've gotten more close to AI systems that are more powerful, you know, I think I've wanted to say those things, you know, more forcefully, more publicly, to make the point clearer. You know, I've been saying for many years that, you know, we have these, we can talk in detail about them, but you know, we have these scaling laws. AI systems are getting more powerful. They're going from the level of, you know, a few years ago, they were barely coherent. Now, you know, a couple of years ago, they were at the level of a smart high school student.

• 3:01Speaker 01

  Now we're getting to smart college student, PhD, and they're starting to, they're starting to apply across the economy. And so I think all the issues related to AI, ranging from kind of the national security issues to the economic issues, you know, are, are, are starting to become quite near to where, to where we, you know, to where we're actually going to face them. And so, and so I think as these problems have come closer, I, you know, even though, you know, in some form Anthropic has been saying these things for a while, I think the urgency of these things has gone up. And, and, and, and, you know, I, I, I want to make sure that we, you know, I want to make sure that we say what we believe and that we warn the world about the possible downsides.

• 3:46Speaker 01

  Even though, you know, no one can say what's going to happen, we're, you know, we're saying what we, you know, what we think might happen, what we think is likely to happen, you know, we, we back it up as, as best we can, although it's often, you know, extrapolations about the future, where, where no one, where no one can be sure. But, you know, I think we, we see ourselves as kind of, as kind of having the duty to, you know, to kind of warn the world about what's going to happen. And that's not to say, you know, I think there's an incredible number of like positive applications of AI, right? I've, I've kind of continued to talk about that. I read this, I wrote this essay, Machines of Loving Grace. I, I, I feel in fact that I and Anthropic have often been able to do a better job of

• 4:30Speaker 01

  articulating the benefits of AI than some of the people who call themselves optimists or accelerationists. So I think we probably appreciate the benefits more than, more than anyone, but for exactly the same reason, because we can have such a good world if we get everything right, I feel obligated to warn about the risks.

• 4:49Alex Kantrowitz?

  So all of this is coming from your timeline. Basically, you, it seems like you have a shorter timeline than most. And so you were feeling a sense of urgency to get out there because you think that this is imminent.

• 4:59Speaker 01

  Yes, I'm not sure. Um, you know, I think it's very hard to predict, particularly on the societal side. So if you say, you know, when are people going to deploy AI? Or when are companies going to use, you know, X, X dollars of spend of AI? Or, you know, when will, when will AI, um, you know, be, be, be, be used in these applications? Or when will it drive these medical cures? That's kind of harder, harder to say. I think the underlying technology is more predictable, but still uncertain, still no one knows. But I think on the underlying technology, I've started to become more confident. There isn't no uncertainty about it.

• 5:39Speaker 01

  You know, I think the, the exponential that we're on could, could kind of still, um, you know, could totally peter out. You know, I think there's maybe, uh, I don't know, 20 or 25% chance that sometime in the next two years, the models just start getting, stop getting better for reasons we don't understand. Or maybe reasons we do understand, like, you know, data or compute availability. And then everything I'm saying just, just seems, seems totally silly. And everyone makes fun of me for all the warnings I've made and, and, you know, I'm just, I'm just totally fine with that. Given, given the distribution that, that, that, given the distribution that I see.

• 6:13Alex Kantrowitz?

  And so I should say that this is part, our conversation is part of a profile I'm writing about you. I've spoken with more than two dozen people who've worked with you, who know you, who've competed with you. And I'm going to link that in the show notes. If anybody wants to read it, it's free to read. Uh, but one of the themes that has, uh, come through across everybody I've spoken with is that you have about the shortest timeline of any of the major lab leaders. And you just referenced it, uh, just, just now. So why do you have such a short timeline? And why should we believe in yours?

• 6:42Speaker 01

  Yeah, it, it really depends what you mean by timeline. Um, so one thing, and you know, I've, I've, I've been consistent on this over the years is, you know, there are these terms in the AI world like AGI and superintelligence. Like you'll hear leaders of companies say, we've achieved AGI. AGI, we're moving on to superintelligence, or like, it's really exciting that someone stopped working on AGI and started working on superintelligence. So I think these terms are totally meaningless. I don't know what AGI is. I don't know what superintelligence is. It it sounds like a, it sounds like a marketing term. It is marketing. Yeah, it sounds like, you know, something, something designed to activate people's, people's dopamine.

• 7:19Speaker 01

  So you'll see in public, I never use those terms and I, you know, I'm, I'm actually, you know, careful to criticize the use of those terms. Um, but I think, I think despite that, I am, I am indeed one of the most bullish about, about AI capabilities improving very fast. The thing I think is real, that I've said over and over again, is the exponential. The idea that every few months, we get an AI model that is better than the AI model we got before. And that we, we get that by investing more compute in AI models, more data, more new types of training models. Initially, this was done by what's called pre-training, which is when you just feed a bunch of data from the internet into the model.

• 8:01Speaker 01

  Now we have a second stage that's reinforcement learning, or test time compute, or reasoning, or whatever you want to call it. I, I think of it as a second stage that involves reinforcement learning. Now both of those things are scaling up together, as we've seen with our models and as we've seen with models from other, from other companies. And I don't see anything blocking that, the further scaling of that. There's some stuff about, you know, how do we broaden the tasks on the RL side of it? We've seen, uh, more progress on, say, math and code, where, where the models are, you know, getting pretty close to like a high professional level and less on more subjective tasks, but I think that is very much a temporary obstacle.

• 8:45Speaker 01

  Uh, so when I look at it, I see this exponential and I say, look, people aren't very good at making sense of exponentials, right? Like, you know, if, if something is doubling every six months, then, uh, you know, two years before it happens, it looks like it's only one sixteenth of the way there. And, and, and so we are sitting here in the middle of, of 2025, um, and the models are really starting to explode. In in terms of the economy, right? If you look at the capabilities of the model, they're starting to saturate all the benchmarks. If you look at revenue and you know Anthropic's revenue every year has grown ten X. Every year, we're we're kind of conservative and we say, you know, it can't grow ten X.

• 9:25Speaker 01

  This time, you know, I I you know, I never I never assume anything and and actually always am very conservative in saying, ah, I think it's going to slow down on the business side. But we went from zero to a hundred million in twenty twenty three. We went from a hundred million to a billion in twenty twenty four. And, you know, this year, in this first half of the year, we've gone from one billion to, you know, I think as of as of speaking today, it's it's well above four. It might be four point five.

• 9:50Speaker 01

  And so if you think about it, you know, suppose that exponential continued for two years. I'm not saying it will, but suppose it continued for two years. You know, you're you're like well into the hundred billions. I'm not saying that'll happen. I'm saying the situation is that when you're on an exponential, you can really get fooled by it. Two years away from when the exponential goes totally crazy. It it it, you know, it looks like it's just starting to be a thing. And so that's the fundamental dynamic. You know, we saw that with the internet in it. We saw that with the internet in the nineties. Right? Where it was like, you know, networking speeds and the underlying speed of the computers were getting fast. And over a few years, it became possible to have to basically build a digital global communications network on top of all this,

• 10:34Speaker 01

  when it wasn't possible just a few years ago. And and almost no one, except for a few people, really saw the implications of that and how fast it would happen. And so that's that's where I'm coming from. That that's what I think now. I don't know, like, if a bunch of satellites crashed, maybe the internet would have taken longer. If there was an economic crash, maybe it would have taken a little longer. So we can't be sure of the exact timelines, but I think people are getting fooled by the exponential and and not realizing how fast it it might be, how fast I think it probably will, although I'm not sure.

• 11:06Alex Kantrowitz?

  But so many folks in the AI industry are talking about diminishing returns from scaling now. That really doesn't fit with the vision you just laid out. Are they wrong?

• 11:15Speaker 01

  Yeah, I I from what we've seen, I can only speak in terms of the models at Anthropic. But what I've seen in terms of the models at Anthropic, if we look at, you know, let's take coding. Coding is one area where, you know, I think Anthropic models have advanced very quickly. Adoption has been very quick. We're not just a coding company. We're planning to expand to many areas. But if you if you look at If you look at coding, you know, every, you know, we released 3.5 Sonnet, a model we call 3.5 Sonnet V2, which, you know, let's call it 3.6 Sonnet now, 3.7 Sonnet, and then 4.0 Sonnet and 4.0 Obis. And, you know, that series of four or five models, each one got

• 11:58Speaker 01

  substantially better at coding than than the last. If you want to look at benchmarks, you can look at, you know, Sui Bench growing from, you know, I think 18 months ago is at like 3% or something, growing all the way to, you know, 72 to 80%, depending on how you how you measure it. And and the real usage has grown exponentially as well. We're heading more and more towards autonomously, you can just use these models. I think the actual majority of code that's written written at Anthropic is, you know, at this at this point, written by, or at least with the involvement of one, you know, one of the Claude models. And various various other companies have have said, you know, have said, you know,

• 12:43Speaker 01

  similar similar statements to similar statements to that. So we see the progress as being very fast and the exponential is continuing, and we don't see any diminishing returns.

• 12:53Alex Kantrowitz?

  But there are some liabilities, it seems like, with large language models. For instance, continual learning. We had Dwar Kesha on a couple of weeks ago. Here's how he put it. He wrote about it in his Substack. The lack of continual learning is a huge, huge problem. The LLM baseline at many tasks might be higher than an average human, but you're stuck with the abilities you get out of the box. So you just make the model and that's it. It doesn't learn. That seems like a glaring liability. What do you think about that?

• 13:18Speaker 01

  So first of all, I would say, even if we never solved continual learning, even if we never solved continual learning and memory, I think that the potential for the LLMs to do, you know, incredibly incredibly well to, you know, affect things at the scale of the economy will be very high, right? If I think of the field I used to be in, biology and medicine, like, you know, let's say I had a very smart Nobel Prize winner, and, you know, I I I said, okay, You know, you're, you know, you've, you've, you've discovered all these things. You have this incredibly smart mind, but you know, you can't, you can't, you know, you can't like read new textbooks or absorb any new information. I mean, that would be difficult, but like, still, if you had like 10 million of those, like, they're, they're still gonna make a lot of biology breakthroughs.

• 14:03Speaker 01

  Like, they're gonna be limited, they're gonna be able to do some things humans can't, and there are some things humans can do that they can't. But, but, but even that, even if we impose that as a ceiling, like, man, that's pretty damned impressive and transformative. And even if I said you never solved that, like, I think, you know, I think people are underestimating the impact. But, but look, context windows are getting longer, and models actually do learn during the context window, right? So, so as, as I, you know, talk to the model during the context window, I have a conversation, it absorbs information. The underlying weights of the model may not change, but, but, you know, just, just like I'm talking to you here and we're having a conversation and I listen to the things you say and I, I, you know, I think and I, and I like respond to them.

• 14:46Speaker 01

  The models are able to do that and, and from a, from an, from a machine learning perspective, from an AI perspective, there's no reason we can't make the context length 100 million words today, right? Which is roughly what a human hears, what, what, what a human hears in their lifetime. Um, uh, there's no reason that we can't, uh, do that. It's, it's really inference support. Um, uh, and so again, even that fills in many of the gaps, not all the gaps, but it fills in many of the gaps. And then there are a number of things like learning and memory that do allow us to update the weights. So, um, you know, there, there, there are a number of things around, you know, types of reinforcement learning, learning training, you know, you know, we used to many years ago talk about inner loops and outer loops, right?

• 15:31Speaker 01

  The inner loop is like, I have some episode and I learned some things in that episode and I'm trying to optimize for the lifetime of that episode. And, and kind of the outer loop is, is, is the agent's learning over episodes. Um, and so I think maybe that inner loop, outer loop structure is a way to learn the continual learning. One thing we've learned in AI is whenever it feels like there's some fundamental obstacle, like two years ago, we thought there was this fundamental obstacle around. Around reasoning turned out just to be just to be RL. You just train with RL and you let the model write some stuff down. You know, you let the model write things down to try and figure out objective math problems without being too specific. You know, I think, and we already have maybe some, some, you know,

• 16:16Speaker 01

  some, some evidence to suggest that this is another of those problems that is is not as difficult as it seems, that will fall to scale plus a slightly different way of thinking about things.

• 16:29Alex Kantrowitz?

  Do you think your obsession with scale might blind you to some of the new techniques? Like Demis Hassabis says, you know, to get to AGI, or you might call it super powerful AGI, whatever, human level intelligence is what we're all talking about. We might need a couple of new techniques for that to happen.

• 16:42Speaker 01

  So you know, we're developing new techniques. We're developing new techniques every day.

• 16:47Alex Kantrowitz?

  Okay.

• 16:49Speaker 01

  You know, Claude is very good at code and we, you know, we don't really talk externally that much about why Claude is so good at code.

• 16:56Alex Kantrowitz?

  Why is it so good at code?

• 16:58Speaker 01

  Like I said, we don't talk externally about it.

• 17:00Alex Kantrowitz?

  I have to ask.

• 17:02Speaker 01

  So, you know, every new version of Claude that we make has, you know, improvements to the architecture, improvements to the data that we put into it, improvements to the methods that we use to train it. So we're developing new techniques all the time. New techniques are a part of every, you know, model that we build. And, you know, that's why we, you know, I've said these things about like, you know, we're trying to optimize for like talent density as much as possible. Like you need that talent density in order, in order to invent the new techniques.

• 17:30Alex Kantrowitz?

  You know, there's one thing that's been hanging over this conversation, which is that maybe Anthropic is the company with the right idea, but the wrong resources. Because you look at what's happening with XAI and, and inside Meta, where Elon's built his massive cluster. Mark Zuckerberg is building this five gigawatt data center. And they are putting so much resources towards scaling up. Is it possible? And I mean, Anthropic, obviously you have raised billions of dollars, but these are trillion dollar companies.

• 18:03Speaker 01

  Yeah, so we've raised, I think at this point, a little short of Twenty billion dollars.

• 18:08Alex Kantrowitz?

  It's not bad.

• 18:08Speaker 01

  So that's, that's not, that's not nothing. And I would also say if you look at the size of the data centers that we're building with, for example, Amazon. I, I don't think our data center scaling is substantially smaller than that of any of the other companies in the space. You know, in many cases, these things are limited by energy, they're limited by capitalization. You know, when, when, when, when people talk about, you know, these large amounts of money, they're, they're talking about it over, over, over several years, right? And when you hear some of, some of these announcements, sometimes they're not funded yet, they're, you know, we've, we've seen the size of the, of the, of the data centers that folks are building and we're, we're actually,

• 18:48Speaker 01

  we're actually pretty confident that, you know, we're, we, we will be within, we will be within a rough range of the size of the data centers they built.

• 18:55Alex Kantrowitz?

  You talked about talent density. What do you think about what Mark Zuckerberg is doing on the talent density front? I mean, combining that with these massive data centers, it seems like he's going to be able to compete.

• 19:05Speaker 01

  Yeah, so this is, this is actually very interesting because, you know, one thing we noticed is that relative to other companies, you know, I think, I think, I think very, you know, a lot fewer people from Anthropic have been, have been caught by these. And it's not for lack of trying. I've talked to plenty of people, you know, who, who got these offers at Anthropic and, and who just turned them down, who wouldn't even talk to Mark Zuckerberg, who said, you know, uh, uh, no, I'm, I'm, I'm staying at Anthropic. And, and our, our general response to this was, you know, I posted something to the, to the whole company Slack where, where, where I said, look,

• 19:44Speaker 01

  you know, we are not willing to compromise our, you know, our compensation principles, our principles of fairness to respond individually to these offers. The way things work at Anthropic is there's a series of levels. When candidate comes in, they get assigned a level and we don't negotiate that level.

• 20:03Speaker 01

  Because, because we think it's unfair. We want to have a systematic way. If, you know, if Mark Zuckerberg, you know, throws a dart at a dartboard and hits your name. That doesn't mean that you should be paid ten times more than the guy next to you, who's, you know, who's just as skilled, who's just as talented.

• 20:21Speaker 01

  And my view of the situation is that, you know, the only way you can really be hurt by this is if you allow it to destroy the culture of your company by panicking, by treating people unfairly in an attempt to defend the company. And I think actually this was a unifying moment for the company, where, you know, we didn't give in. We refused to compromise our principles because we had the confidence that people are anthropic because they truly believe in the mission. And, you know, I think, I, that gets to kind of how I see this. I think that what they are doing is trying to buy something that cannot be bought. And that is alignment with the mission.

• 21:05Speaker 01

  And, you know, I, you know, I, you know, I think there's selection effects here. Like, I, you know, I, you know, are they getting the people who are most enthusiastic, who are most mission-aligned, who are most excited to-

• 21:17Alex Kantrowitz?

  But they have talent and GPUs. You're not underestimating them?

• 21:20Speaker 01

  I, I, we'll see how it, we'll see how it plays out. I am pretty bearish on what they're trying to do.

• 21:27Alex Kantrowitz?

  So, let's talk a little bit about your business, because a lot of people have been wondering, is the business of generative AI a real thing? And I'm also curious, I have questions all the time. You talked about how much money you've raised, close to 20 billion. You've raised 3 billion, 3 billion from Google, 8 billion from Amazon, 3.5 billion from a new round led by Lightspeed, who I've spoken with.

• 21:50Alex Kantrowitz?

  What is your pitch? Because you are not part of like a big tech company, you're out there on your own. Do you just bring the scaling laws and say, can I have some money?

• 21:58Speaker 01

  So, my, my view of this has, my view of this has always been that talent is the most important thing. So, you know, if you go back three years ago, you know, we were in a position where we had raised mere hundreds of, hundreds of millions. OpenAI had already raised, you know, 13 billion from, from Microsoft. And of course, you know, the, the large hypercap tech- Companies were sitting on one hundred billion dollars, two hundred billion dollars. And and basically, the pitch we made then is we know how to make these models better than others do. Right? There may be a curve. There may be a curve of scaling laws. But but look, if we are in a position where we can do for a hundred million, what others can do for a billion,

• 22:41Speaker 01

  and we could do for ten billion what they can do for a hundred billion, then it's ten times more capital efficient to invest in Anthropic than it is to invest in in these other companies. Would you rather be in a position where you can do anything for ten times cheaper? Or where you start with a large pile of money? If you can do things ten times cheaper, the the, you know, the the the the money is a temporary defect that that that that you can remedy. If you have this intrinsic ability to build things for the same price, much better than anyone else, or as good as anyone else for much lower price. You know, investors aren't aren't idiots, or at least they aren't always idiots.

• 23:17Alex Kantrowitz?

  Depends which one you go to.

• 23:18Speaker 01

  I'm not gonna name any names. But, um, uh, uh, you know, they they basically understand the concept of capital efficiency. Um, and so we've been in a position, you know, three years ago, where, you know, these differences were like a thousand X. And now you're saying, you know, with twenty billion dollars, can you compete? Can you compete with a hundred billion dollars? Um, and and and my answer is basically yes, because of the talent density. You know, I've said this before, but, uh, you know, the the Anthropic is actually the fastest growing software company in history at the scale that it's at. Um, so we grew from zero to a hundred million in twenty twenty three, a hundred million to billion in twenty twenty four. And this year, we've we've grown from one billion to, I think I said this before, four point five.

• 24:02Speaker 01

  So that that ten X a year, I mean, you know, every every, you know, every year, I like suspect that will grow at that scale. And and every year, I'm almost afraid to say it publicly because I'm like, no, it couldn't. It couldn't possibly happen again. So, you know, I think I think the growth at that scale like kind of speaks for itself in terms of our ability to compete with the big players.

• 24:23Alex Kantrowitz?

  Okay, so CNBC says sixty to seventy five percent of Anthropic's sales are through the API. That was according to internal documents. Is that still accurate?

• 24:32Speaker 01

  I don't give exact numbers, but the majority does come through the API, although we also do have a flourishing apps business and, you know, I think more recently, um, the, you know, max tier, which power users use as well as Claude code, which, which coders use. So, you know, I think we have a, we have a thriving and fast growing apps business, but yes, the majority comes through the API.

• 24:52Alex Kantrowitz?

  So you're making the most pure bet on this technology, like, you know, OpenAI might be betting on ChatGPT and Google might be betting on the fact that no matter where the technology goes, it can, you know, integrate into Gmail and calendar. So why have you made this bet on this, the pure bet on the tech itself?

• 25:08Speaker 01

  Yeah, I mean, I would say, I wouldn't quite put it that way. I think we've, I would describe it more as we've bet on business use cases of the model. Um, more so than we've bet on the API per se, and it's just that the first business use cases, the model come through the API. So, you know, as you mentioned, OpenAI is very focused on the consumer side. Google is very focused on kind of the existing products that Google has. Our view is that if anything, the enterprise use of AI is going to be greater even than the consumer use of AI, right? I should say the business use because it's enterprise, it's startups, it's developers, and it's kind of, you know, power users using the model, model for productivity. Um, I, I also think that, uh,

• 25:52Speaker 01

  being a company that's focused on the business use cases actually gives us better incentives to make the models better. Um, a thought, a thought experiment that I think is worth running is, you know, suppose I have this model and it's, uh, it's, um, you know, it's, it's as good as an undergrad at biochemistry. Um, and then I improve it and it's as good as a PhD student at, at biochemistry. If I go to a consumer, right? If I give them the chatbot and I say, great news, I've improved the model from, you know, undergrad to graduate level in, in biochemistry. Um, you know, maybe, I don't know, 1% of consumers care about that at all, right? 99% are just going to be like, I don't understand it either way. Um, but now suppose I go to Pfizer and, and I say, you know, I've improved this from

• 26:36Speaker 01

  undergrad at biochemistry to, to graduate biochemistry. Like this is going to be the biggest deal in the world, right? They might pay 10 times more. For something like that, it might have ten times more value to them. And so the general aim of making the models solve the problems of the world, to make them smarter and smarter, but also able to to bring many of the positive applications, right? The things I wrote about in like Machines of Loving Grace of like solving the problems of biomedicine, solving the problems of geopolitics, solving the problems of, you know, of economic development, um, you know, as well as more prosaic things like finance, or legal, or productivity, or insurance.

• 27:15Speaker 01

  Um, I think it gives a better incentive to kind of develop the models um, as as far as uh, as as as as as far as possible. And I think in many ways, it's like it may even be a more a more a more positive business. So I would say we're making a bet on the business use of AI because it's most aligned with kind of the exponential.

• 27:37Alex Kantrowitz?

  Okay, then briefly, how did you decide to go with the coding use case?

• 27:40Speaker 01

  Yes, so um, you know, uh, originally, as happens with most things, we're trying to optimize for making the model better at a bunch of stuff. And, you know, coding particularly stood out in in terms of how how valuable it was. You know, I've worked with thousands of engineers and there was a point about a year, a year and a half ago, where one of the best I'd ever worked with said, um, uh, you know, every previous coding model has been useless to me and and this one finally was able to do something I wasn't able to do. And then after we released it, started getting quick adoption. This was around the time that, you know, a lot of the coding companies like Cursor, WinSurf, GitHub, Augment Code started started exploding in in popularity. And then when we saw how popular it was, we kind of doubled doubled down on it.

• 28:24Speaker 01

  Um, my view is that coding is particularly interesting because A, the adoption is fast, um, and and B, getting better at coding with the models actually helps you to develop the next model. Um, so it has it has a number of, uh, you know, I, you know, I would say advantages.

• 28:42Alex Kantrowitz?

  And now you're selling your AI coding through Claude Code. But it's very interesting, the pricing model has been confounding to some. You can spend $200 a month and get the equivalent. I spoke to one developer, they got the equivalent of $6,000 a month from your API. Ed Zitron has pointed out, the more popular that your models get, the more money you're going to lose if people are super users of this technology. So how does that make sense?

• 29:11Speaker 01

  So actually, pricing schemes and rate limits are surprisingly complicated. So some of this is basically the result of when we released Claude Code in the max tier, which we eventually tied together, actually not fully understanding the implications of the ways in which people could use the models and how much they were actually able to get. So over the last few days, as of the time of this interview, we've adjusted that, particularly on the larger models like Opus. I think it's no longer possible to spend that much with a $200 subscription. And it's possible more changes will come in the future.

• 29:53Speaker 01

  But we're always going to have a distribution of users who use a lot and users who use some amount. And it doesn't necessarily mean we're losing money, that there are some users who get more, if you were to measure via API credits, spend, get a better deal on the consumer subscription than they would on the API products. There's a lot of assumptions there, and I can tell you that some of them are wrong. We are not, in fact, losing money.

• 30:27Alex Kantrowitz?

  But I guess there's another question about whether you can continue to serve these use cases and not raise prices. So just to give you a couple of stats, there are some developers that are upset because using Anthropic's newer models in cursor is costing them more than it ever has. Startups that I've spoken with say Anthropic is down a bunch because they can't get access to GPUs, at least that's what they... What they imagine is happening, and I was just with Amjad Massad at Replit in an interview that we're going to air next week, who said there was a period of time where the price per token, price to use these models was coming down, and it stopped coming down. So is it the, is what's happening that these models are just so expensive

• 31:12Alex Kantrowitz?

  for Anthropic to run? That it's hitting its, a wall of its own?

• 31:15Speaker 01

  Again, I think you're, I think you're making assumptions here.

• 31:17Alex Kantrowitz?

  That's why I'm asking the CEO.

• 31:18Speaker 01

  Yeah, um, uh, you know, uh, you know, the, the, the way I think about it is we think about the models in terms of how much value are they creating? Right, um, so as the models get better and better, I think about how much value they create. And there's a separate question about how the value is distributed between those who make the model, those who make the chips, and, and you know, those who make, those who make the, the, the, um, the, the, the, the, the underlying applications. So again, without being too specific, like, I think there are some assumptions in your question that are not necessarily correct. Um, uh, you know, I-

• 31:51Alex Kantrowitz?

  Can you tell me which ones?

• 31:52Speaker 01

  I, I, so, I'll say this, I do expect the, um, I, I, I expect the price of providing a given level of intelligence to go down. Um, I expect the price of providing the frontier of intelligence, which will, which will provide a kind of increasing economic value, that might go up or it might go down. My guess is it probably stays about where it is. But again, the value that's created, you know, goes way up. So two years, two, two years from now, my guess is that we'll have models that cost of the same order of magnitude that they cost today, except they'll be much more capable of doing work much more autonomously, much more broadly than, than they are capable of today.

• 32:33Alex Kantrowitz?

  One of the things that Amjad mentioned was he thinks that the bigger models are not as intensive to run, or more intensive to run, given their size because of the architecture and some of these techniques that we talked about, that they're lighting up only certain sections of the model. So his idea, I'm hopefully conveying this truthfully, is that Anthropic can run these models without too much bulk on the backend, but is still keeping those prices where they are. And I think the, the line that I'm going to draw there is maybe that um, to get to software margins, there were some reports that Anthropic is slightly below software gross margins, you're going to have to charge a little bit more for these models.

• 33:14Speaker 01

  So, um, yeah, a- a- again, um, I think larger models cost more to run than smaller models.

• 33:21Alex Kantrowitz?

  Okay.

• 33:21Speaker 01

  Um, uh, you know, I think the technique you're referring to is maybe mixture of experts or something like that.

• 33:27Alex Kantrowitz?

  That's right.

• 33:28Speaker 01

  So whether your models are mixture of experts or not, like mixture of experts is, is like a way to run models more cheaply that have a given number of parameters. It's a way to train models. But if you're not using that technique, then larger models that don't use that technique cost more to run than smaller models that don't use that technique. And if you're using that technique, larger models that use that technique cost, you know, more to run than, than smaller models that, that, that are using that technique. So I think, I think that's sort of a distortion of the, I think that's sort of a distortion of the situation.

• 33:57Alex Kantrowitz?

  Um, basically, I'm, I'm just guessing and I'm trying to find out what the truth is from you.

• 34:00Speaker 01

  Yeah, look, so I, um, you know, in, in, in, in terms of like the cost of the models, like one thing you'd be surprised by, people, you know, people kind of impute this thing to like, oh man, it's going to be really hard to get the margins from like X percent to Y percent. We make improvements all the time that make the models like 50% more efficient than they are before. We are just at the beginning of optimizing inference. Um, uh, inference has improved a huge amount where, from, from where it was a couple, a couple of years ago to where it is now. That's why the prices are coming down.

• 34:35Alex Kantrowitz?

  And then how long is it going to take to be profitable? Because I think the loss is going to be like three billion this year. That's what they predict.

• 34:42Speaker 01

  So I would distinguish different things.

• 34:44Alex Kantrowitz?

  Okay.

• 34:44Speaker 01

  Um, there's the cost of running, there's the cost of running the model, right? So, so for every dollar the model makes, it costs a certain amount. Um, that is actually already fairly profitable. Um, there are separate things. There's, you know, the cost of paying people, um, and like buildings, that is actually not that, that large in the scheme of things. The big cost is the cost of training the next model. Um, and I think this idea of like the company's losing money and not being profitable is, it's a little bit misleading. Um, and you start to understand it better when you, when you look at the scaling laws. So as a thought exercise, these numbers are not exactly.

• 35:24Speaker 01

  Or even close for Anthropic. Let's imagine that in two thousand and twenty three, you train a model that costs a hundred million dollars. Um, and then in two thousand and twenty four, you deploy the two thousand and twenty three model and it makes two hundred million dollars in revenue. But you spend a billion dollars to train, um, you know, to train a new model in two thousand and twenty four. So a- and then, you know, and then in two thousand and twenty five, the billion dollar model makes two billion in revenue and you spend ten billion to train the next model. Um, so the company every year is unprofitable. It lost eight hundred million in two thousand and twenty four and then in two thousand and twenty five, it lost eight billion dollars. Um, so you know, this looks like a hugely unprofitable enterprise. But if instead, I think in terms of is each model profitable, right? Think of each model as a venture.

• 36:05Speaker 01

  Um, I invested a hundred million in the model and then I got, then I got a hun- then I got two hundred million out of the model the next year. So that model had fifty percent margins and, and you know, and like, and like made me a hundred, a hundred million dollars. The next year, um, you know, the, the, the company invested a billion dollars and made, and made two billion dollars in, in, or sorry, the next model. The company invested a billion dollars and made. So every model is profitable, but the company is unprofitable every year. I, I'm not, I'm not. Right. This is, this is, this is a, a stylized, I'm not like claiming these numbers for Anthropic or claiming these facts for, but this general. Is it true for Anthropic? This general dynamic is, is thi- this, this general dynamic is in general terms, the explanation for what is going on. Okay.

• 36:49Speaker 01

  And so at, you know, you, you know, at, at any time, if the models stopped getting better or if a company stopped investing in the next model, um, you know, you would, you know, you would have probably a viable business with the existing models. But everyone is investing in the next model. Yeah. And so eventually it'll get, it'll get to some scale. The, but the fact that we're spending more to, to, to, to, this fact that we're spending more to invest in, in the next model suggests that the, the scale of the business is going to be larger the next year than it was, than it was the year before. Now, of course, what could happen is like the model stopped getting better and there's this kind of one time cost that's like a boondoggle and we spend, we spend a bunch of money. But then the, the, you know, the, the.

• 37:34Speaker 01

  Companies, the industry will kind of return to this, you know, to this plateau, to this level of profitability, or the exponential can keep going. So I think, I think that's a long-winded way to say, I don't think it's really the right way to think about things.

• 37:46Alex Kantrowitz?

  Right, but what about open source? Because if you stopped, let's say you stopped investing in the models and open source caught up, then people could swap in open source. Now, I'd love to hear your perspective on this because one of the things people have talked to me about when it comes to the Anthropic business is there is that risk eventually that open source gets good enough that you can take Anthropic out and put open source in.

• 38:08Speaker 01

  Yeah, so you know, people have, you know, I think one of the things that's been true of this industry is that, and you know, I saw it early in, I saw it early in the history of AI. Every community that AI has gone through, it has this set of heuristics about how things work. Like back when I was in, you know, AI back in 2014, there was an existing kind of AI and machine learning research community that like thought about things in a certain way and were like, this is just a fad, this is a new thing, this can't work, this can't scale. And then because of the exponential, all of those things turned out to be false. Then a similar thing happened with kind of like people deploying AI within companies to various applications.

• 38:53Speaker 01

  Then there was the same thought in the startup ecosystem. And I think now we're at the phase where kind of the world's business leaders, like the investors and the business, they have this whole lexicon of commoditization, you know, modes, which layer is the value going to, which layer is the value going to accrue to? And open source is this idea that you can kind of see everything that's going in, you know, that, that it has a significance, that it kind of undermines

• 39:23Speaker 01

  the fact that, you know, the idea that it undermines business. And I actually find as someone who didn't come from that world at all, who never thought in terms of that lexicon, this is one of these situations where not knowing anything often leads you to make better predictions than kind of the people who have their way of thinking about things from the last generation of tech. And I, you know, this is all, I think, a long-winded way of saying, I don't think open source works the same way in AI that it has worked in other areas. Primarily because with open source, you can, you can see the, you know, you can see the source code of the model. Here, we can't see inside the model. You know, it's often called open weights instead of open source to kind of distinguish that.

• 40:07Speaker 01

  But a lot of the benefits, which is that many people can work on it, that it's kind of additive, it doesn't quite work in the same way. So, you know, I've, I've actually always seen it as a red herring. When I see it, when I see a new model come out, I don't care whether it's open source or not. Like, if we talk about DeepSeek, I don't think it matters that DeepSeek is open source. I think, I ask, is it a good model?

• 40:28Alex Kantrowitz?

  Right.

• 40:29Speaker 01

  Is it better than us at, at, you know, the things that, that's the only thing that I care about. It, it actually, it actually doesn't, doesn't matter either way. Because ultimately, you have to, you have to host it on the cloud. The people who host it on the cloud do inference. These are big models, they're hard to do inference on. And conversely, many of the things that you can do when you see the weights, you know, we're increasingly offering on clouds, where you can fine tune the model, you can, you know, you know, we're even looking at, at ways to, you know, to, to kind of, you know, investigate the activations of the model as part of like an interpretability interface. We did some little things around steering last time. So I think it's the wrong axis to think in terms of.

• 41:11Speaker 01

  When I think about competition, I think about like, which models are good at the tasks that we do. I think open source is actually a red herring.

• 41:19Alex Kantrowitz?

  But if it's free and cheap to run.

• 41:21Speaker 01

  It's not free. You have to, you have to, you have to, you have to run it on inference and someone, someone has to make it fast on inference.

• 41:26Alex Kantrowitz?

  All right. So I want to learn a little bit more about Dario the person.

• 41:30Speaker 01

  Yes.

• 41:31Alex Kantrowitz?

  So we have a little bit of time left. So I have some questions for you about early life and then how you became who you are.

• 41:37Speaker 01

  Yes.

• 41:38Alex Kantrowitz?

  So what was it like growing up in San Francisco?

• 41:41Speaker 01

  Yeah, um, I, you know, the city, when I first grew up here, had not really, had not really gentrified that much, you know, when I grew up, the tech boom hadn't, hadn't happened, uh, hadn't happened yet. You know, it happened as, as I was going through high school, and actually, I had no interest in it. It was totally, it was totally boring to me. You know, I was interested in being like a scientist. I was interested in physics and math. And, you know, the idea of like, you know, you know, like, writing some website actually had no interest to me, to me whatsoever. Or like founding a company, like, those weren't things that I was, that I was interested in at all. You know, I was interested in discovering fundamental scientific truth. And I was interested in like, you know, how can I, how can I do something that like makes the world better?

• 42:25Speaker 01

  So, so, you know, that was, that was kind of more. And, you know, I watched the tech boom happen around me, but I, I feel like, you know, there was all kinds of things I probably could have learned from it that would have been helpful now, but I just actually wasn't paying attention and had no interest in it, even though I was like right at the center of it.

• 42:41Alex Kantrowitz?

  So you're the son of a Jewish mother?

• 42:43Speaker 01

  Uh, yes.

• 42:43Alex Kantrowitz?

  And an Italian father?

• 42:44Speaker 01

  That is true.

• 42:45Alex Kantrowitz?

  From where I'm from in Long Island, we call that a pizza bagel?

• 42:48Speaker 01

  A pizza bagel. I've never, I've never heard that term before.

• 42:51Alex Kantrowitz?

  So what was your relationship with your parents like?

• 42:53Speaker 01

  Yeah, I mean, you know, I was, I was always, I was always, I was always pretty close with them. You know, I feel like they gave me a sense of, you know, of, of kind of right and wrong and what was important in the world. I feel like, you know, kind of imbuing a strong sense of responsibility is, is maybe the thing that I remember most. You know, they were always people who felt that sense of responsibility and, you know, wanted to, wanted to make the world,

• 43:22Speaker 01

  wanted to make the world better. And I, and I feel like, you know, that's one of the, one of the main things that I, that I learned from them. You know, it was always a very, a very, a very loving family, a very caring family. I was very close with my sister Daniela, who of course became my, became my, became my co-founder. And, you know, I think we decided very early that we wanted to work together in some, in some capacity. I don't know if we imagined that it would happen, you know, at quite the scale that, that, that it has happened. But it, um... You know, I think I think it really, um, you know, that was that was something we kind of decided early that we wanted to do.

• 43:57Alex Kantrowitz?

  The people that I've spoken with that have known you through the years have told me that your father's illness had a big impact on you. Can you share a little bit about that?

• 44:04Speaker 01

  Um, yes, yes, he was, um, yeah, you know, he was ill for a long time, um, and eventually uh, died in, uh, eventually died in, uh, in, in two thousand and six. Um, uh, so that, you know, that was actually one of the things that drove me to, you know, I, I don't think we mentioned it yet in this interview, but before, um, you know, before I went into AI, um, you know, I went into biology. So, you know, I'd gone to, uh, I, I, you know, I'd shown up at, I'd shown up at Princeton, um, uh, wanting to be a theoretical physicist and, you know, I did some, did some work in, in cosmology for the first few ti- few months of my time there, um, you know, and, and, you know, that was, that was around the time that, that my father died.

• 44:47Speaker 01

  And, you know, that did have an influence on me and kind of was one of the things that convinced me, um, you know, to, to go into biology, you know, to try and address, um, uh, uh, you know, human illnesses and biological problems. And so I started talking to some of the folks who worked on biophysics and computational neuroscience in the department that I was at, at Princeton, and that was what led to the switch to biology and computational neuroscience. And then, you know, of course, after that, I eventually, I eventually went into AI. And the reason I went into AI was actually a continuation of that motivation, which is that, um, you know, as I spent many years in biology, I realized that the complexity

• 45:28Speaker 01

  of the underlying problems in biology felt like it was beyond human scale. You know, you, in order to understand it all, you needed hundreds, thousands of, you know, human researchers and, you know, they often had a hard time collaborating or sharing their, you know, combining their, their knowledge. And AI, which was, I was just starting to see the discoveries in it, felt to me like the only technology that could kind of bridge that gap, could bring us beyond human scale to, you know, to, to fully understand and solve the problems of biology. So, yeah, there is a through line there.

• 46:00Alex Kantrowitz?

  Right, and I could have this wrong. Uh, but one thing I heard was that his illness was largely uncurable when he had it.

• 46:08Speaker 01

  Yes.

• 46:08Alex Kantrowitz?

  And there have been advances that have been, can you share a little bit more?

• 46:11Speaker 01

  Yes.

• 46:11Alex Kantrowitz?

  There are advances that have made it much more manageable today.

• 46:14Speaker 01

  Yes, yes, that is, uh, that is, uh, that is, that, that, that is true. Actually, actually only, um, uh, uh, only in the, um, uh, maybe three or four years after he died, the, the cure rate for the disease that he had went from, uh, went from fifty percent to, uh, uh, to roughly ninety-five percent.

• 46:33Alex Kantrowitz?

  Yeah, I mean, it has to have felt so unjust to have your father taken away by something that could have been cured.

• 46:38Speaker 01

  It, it, of course, of course, um, but it also tells you of the, the urgency of solving the relevant problems. Right? That, um, you know, that, that, that, you know, there, there was someone who worked on the cure to, to this disease that, you know, managed to cure it and save a bunch of people's lives, but, you know, could have, could have, um, saved even more people's lives if, if, you know, they, they had managed to, to find that, that, to find that cure, you know, a few years earlier than they did. Um, and I think that's, that's one of the tensions here, right? That, you know, um, I think AI has all of these benefits. Um, and, you know, I want everyone to get those benefits as soon as possible. You know, I probably understand, you know, better than almost anyone how urgent those benefits are.

• 47:23Speaker 01

  Um, and so I really understand the stakes. Well, you know, when I speak out about AI has these risks and I'm worried about these risks, I get very angry when people call me a doomer. I got really angry when, you know, when, when someone's like, this guy's a doomer. He wants to slow things down. You, you heard what I just said, like, you know, my, my father died because of, you know, cures that, you know, could have, could have happened a few years later. I understand the benefit of this technology. When I sat down to, to, to write Machines of Loving Grace, you know, I wrote out all the ways that billions of people's lives could be better with this technology. Some of these people, some of these people who on Twitter, you know, cheer for acceleration, I don't think they have a humanistic sense of the benefit of the technology.

• 48:05Speaker 01

  Their their brains just full of adrenaline and and they're like they want to cheer for something. They want to accelerate. I don't get the sense they care. Um, and so when these people call me a doomer, I think I think they just completely, completely lack any moral credibility in doing that. Um, uh, you know, it really makes me lose respect for them.

• 48:24Alex Kantrowitz?

  And I've been wondering what this this word impact has been because it's come up so often that those who have been around you have said you've been singularly obsessed with having impact. In fact, I spoke with someone who knew you well, who said you wouldn't watch Game of Thrones because it wasn't tied to impact, that it was a waste of time and you wanted to be focused on impact.

• 48:45Speaker 01

  Actually, that's not quite right. I wouldn't watch it because it was so negative sum. People were playing, were playing such negative. It was like these people start off and they're partly the situation and partly because they're they're just horrible people. They like create the situation where at the end of it, everyone is like worse off than everyone was before. Um, I'm I'm really, I'm really excited about like creating positive sum situations.

• 49:09Alex Kantrowitz?

  Okay, I recommend you watch it. It's a great, it's a great show, but I hear you.

• 49:12Speaker 01

  I actually have watched some parts of it. I was just very reluctant and didn't watch it for a long time.

• 49:17Alex Kantrowitz?

  Let's get back to the impact.

• 49:18Speaker 01

  Okay, let's get back to the impact.

• 49:19Alex Kantrowitz?

  So that's what impact is, is effectively your career has been this I this quest to have that impact to be able, tell me if I'm going too far, to prevent other people from being in similar situations.

• 49:32Speaker 01

  I, you know, I, I, I think, you know, I think, I think that's a piece of it. I mean, you know, I, I have looked at, you know, many, uh, you know, many attempts to help people and, you know, some of them are more effective than others. Um, and, you know, I think, I think I've always tried to, you know, there should be strategy behind it. There should be brains behind, um, you know, trying to, trying to help people, um, you know, which often means that there's a long path to it, right? It can run through a company and, you know, many activities that are technical and not immediately tied to the, the kind of impact, impact that you're trying to have. But, you know, the, the, the arc is, I'm always trying to bend the arc towards that. I think, I think that's my,

• 50:15Speaker 01

  that's my picture of it. That's, that's really why I, that's really why I got into this, right? You know, um, I think, you know, similar to the reason to get into AI was that. Um, you know, I I saw the problems of biology as as almost intractable without it, or at least too slow moving. Um, you know, I think my reason to start a company was that I had worked at other companies and I I I just didn't feel like the way those companies were were run was was really oriented towards, you know, trying, trying, trying to have that impact. There was a story around it that was often used for recruiting, but it became clear to me over the years that story was not sincere.

• 50:54Alex Kantrowitz?

  I'm gonna circle around a little bit because it's clear that you're referring to OpenAI here. Um, from what I understand, you had fifty percent of OpenAI's compute. I mean, you ran the GPT three project. So if anyone was gonna be focused on impact and safety, wouldn't have been you.

• 51:11Speaker 01

  Uh, yes, I I was, you know, there was a period during which, uh, during which that was, uh, that was true. That wasn't true the entire time. That was, for example, when we were scaling up GPT three. Um, yeah, so, you know, I I, when I was at OpenAI, I and a lot of my colleagues, including the people who, you know, eventually, um, eventually founded Anthropic.

• 51:33Alex Kantrowitz?

  The pandas.

• 51:34Speaker 01

  Um, the pandas. Um.

• 51:36Alex Kantrowitz?

  That's the name you gave them.

• 51:37Speaker 01

  That I, that that isn't a name I gave them.

• 51:39Alex Kantrowitz?

  The name they took.

• 51:40Speaker 01

  Uh, that isn't a name they took.

• 51:42Alex Kantrowitz?

  Um, that's the name other people called them?

• 51:44Speaker 01

  Uh, I I, maybe it's a name other people called them. Uh, that's not a name I ever used for my team.

• 51:49Alex Kantrowitz?

  Okay, sorry, go ahead. Uh, that's good clarification. Thank you.

• 51:53Speaker 01

  Um, uh, so, uh, yeah, um, you know, we were involved in scaling up these models. Actually, the original reason for building GPT two and GPT three, it was an outgrowth of the kind of AI alignment work that we were doing, right? Where myself and Paul Cristiano and some of the Anthropic co-founders had invented this technique called RL from human feedback. Um, uh, and that was designed to help steer models in, um, uh, you know, in a direction to follow human intent. It was actually a precursor to, you know, we were trying to scale up another method called scalable supervision, which I think is just starting to. To to work many years later to help models follow more kind of scalable uh uh uh

• 52:38Speaker 01

  human intent. But what we found is even with the more primitive technique, RL from human feedback, it wasn't working with the small language models with, you know, GPT one that we applied it to and that had had been built by other people at OpenAI. And so the scaling up of GPT two and GPT three was done in order to kind of study these techniques, in order to apply RL from human feedback at scale. Um, uh, you know, this goes to one thing, which is that I think in this field, the alignment of AI systems and the capability of AI systems is intertwined in this way that always ends up being kind of more tied and more intertwined than we think. Um, actually, what this made me realize is that it's very hard to work on the safety of AI systems and the um capability of AI

• 53:22Speaker 01

  systems separately. It's very hard to work on one and not the other. I actually think the value and the um way to inflect the field in a more positive way comes from organizational level decisions, when to release things, when to study things internally, what kind of work to do on systems. Um, and that was one of the things that kind of motivated, you know, me and some of the other, you know, to be anthropic founders to kind of go off and do it our own way.

• 53:51Alex Kantrowitz?

  But again, like if you were driving, if you think language, uh, if you think capabilities and safety are interlinked, and you were the guy driving the cutting edge models within OpenAI, you know, you, if you left, you knew they were going to be a company that was still doing this stuff.

• 54:07Speaker 01

  That's right.

• 54:07Alex Kantrowitz?

  It seems like if you're driving capabilities, you'd be the one in the driver's seat to help it be safe the way that you wanted it to.

• 54:13Speaker 01

  Again, I will say, um, you know, if there's a decision on releasing a model, if there's a decision on the governance of the company, if there's a decision on, you know, how the personnel of the company works, um, you know, how the company represents itself externally, the decisions that the company makes with respect to deployment, the claims it makes about how it operates with respect to society. Um, you know, many of those things are not Things that you control just by, just by training the model. And you know, I think, I think trust is really important. I think the leaders of the company, of a company, they have to be trustworthy people. They have to be people whose motivations are sincere.

• 54:55Speaker 01

  No matter how much you're driving the, the forward, the company technically. If you're working for someone whose motivations are not sincere, who's not an honest person, who does not truly want to make the world better, it's not going to work. You're just contributing to something bad.

• 55:10Alex Kantrowitz?

  So then I'm, I'm sure you've heard the criticism from people like Jensen, who say, well, Dario thinks he's the only one who can build this safely. And therefore, speaking of that word, control, wants to control the entire industry.

• 55:22Speaker 01

  I've never said, I've never said anything like that. That's an outrageous lie. That's the most outrageous lie I've ever heard.

• 55:29Alex Kantrowitz?

  By the way, I'm sorry if I got Jensen's words wrong, but.

• 55:32Speaker 01

  No, no, no, the words were correct.

• 55:33Alex Kantrowitz?

  Okay.

• 55:34Speaker 01

  But, but.

• 55:35Alex Kantrowitz?

  What he's saying is.

• 55:35Speaker 01

  But the words are, the words are outrageous. In fact, I've said multiple times, and I think Anthropic's actions have shown it, that, um, you know, we're aiming for something we call a race to the top. Um, you know, I've said this on podcasts over the years, and I think Anthropic's actions have shown it where, you know, with, with a race to the bottom, right? Everyone is competing to like, you know, get things out as fast as possible. And so I say, when you have a race to the bottom, it doesn't matter who wins, everyone loses, right? Because you make the unsafe system that, you know, helps your adversary or causes economic problems or, uh, you know, is, is unsafe from an alignment perspective.

• 56:16Speaker 01

  The way I think about the race to the top is that, um, it doesn't matter, it doesn't matter who, it doesn't matter who wins, everyone wins, right? So the way the race to the top works is you set an example for how the field works. You say, um, uh, uh, you know, we're gonna engage in this practice. So a key example of this is responsible scaling policies. We were the first to put out a responsible scaling policy. And, you know, we didn't say everyone else should do this, or you're bad guys. We didn't, you know, we didn't, you know, kind of try to use it as advantage. We put it out and then we encouraged everyone else to do it. And many and then we discovered in the months after that that you know there were people within the other companies who were trying to put out responsible scaling policies, but

• 57:01Speaker 01

  the fact that we had done it allowed, you know, gave those people permission, right? Kind of kind of enabled those people to, you know, to make the argument to leadership, hey, Anthropic is doing this, so we should do it as well. The same has been true of investing in interpretability. We release our interpretability research to everyone and allow other companies to copy it, even though we've seen that it sometimes has commercial advantages. Same with things like constitutional AI, same with the the the measure, the, you know, the measurement of the dangers of our system, dangerous capabilities evals. So we're trying to set an example for the field, but there's an interplay where it helps to be a powerful commercial competitor.

• 57:44Speaker 01

  I've said nothing that that any that anywhere near resembles the idea that this company should be the only one to build the technology. I don't know what anyone could ever derive that from anything that I've said. It's it's it's it's it's just, yeah, yeah, it's just, it's just, it's just, it's just an incredible and bad faith distortion.

• 58:03Alex Kantrowitz?

  All right, let's see if we can lightning round like one or two before I ask you the last one, which we'll have five minutes for. What happened with SBF?

• 58:12Speaker 01

  Like, what happened with SBF?

• 58:13Alex Kantrowitz?

  I mean, he was one of the, go ahead.

• 58:15Speaker 01

  I couldn't tell you.

• 58:16Alex Kantrowitz?

  But wasn't it, was, what was the, what did you answer?

• 58:19Speaker 01

  I I probably met the guy four or five times.

• 58:21Alex Kantrowitz?

  Okay.

• 58:22Speaker 01

  Um, so I have no great insight into the, you know, what, in, you know, into the psychology of SBF or, you know, why, why he did things as stupid, stupid or immoral as, as, as, as, as he did. I think the only, the only, you know, the, the only thing I had ever seen ahead of time with SBF was, you know, a couple of people mentioned to me that he was like hard to work with, that, you know, he was, he was like a bit of a move fast and break things guy. And I was like, okay, you know, there's like plenty of people.

• 58:58Alex Kantrowitz?

  Welcome to Silicon Valley.

• 58:59Speaker 01

  Yeah, like, welcome, welcome to Silicon Valley. And so I remember saying, okay, I'm going to give this guy non-voting chairs, I'm not going to put him on the board, he sounds like a, you know, he sounds like a bad person to deal with every day. But, you know, he's excited about. AI, he's excited about AI safety. He's, he's, you know, he's a bull on, on, on AI and he's interested in AI safety. So, you know, seems like a, seems like a sensible, seems like a sensible thing to do, you know, in, in, in, you know, in, in, like, in, in, in, in retrospect, um, you know, the, that, you know, move fast and, and, and break things, you know, was turned out to be much, much, much more extreme and bad than, than, you know, than, than, than I ever imagined.

• 59:38Alex Kantrowitz?

  Okay, so let's end here. So you found your impact. I mean, you're, you're working the dream pretty much right now. I mean, think about all the ways that AI can be used for biology, uh, just to start. You also say that this is a dangerous technology. And I'm curious if your desire for impact, um, could be pushing you to accelerate this technology, um, while, you know, potentially devaluing the possibility that it could, that controlling it might not be feasible.

• 1:00:14Speaker 01

  So, you know, I think I have more than anyone else in the industry warned about the dangers of the technology, right? We just spent 10, 20 minutes talking about, you know, the, the frightening, the, you know, the large array of, you know, people who run, you know, trillion dollar companies criticizing me for, you know, for talking about the, the dangers of these technologies, right? You know, I have US government officials. I have people who run four trillion dollar companies criticizing me for talking about the dangers of the technology, right? Imputing all these bizarre motives that bear no relationship to, you know, to anything I've ever said, not supported in anything I've ever done. And yet I'm going to continue to do it. Um, I actually

• 1:00:57Speaker 01

  think that, you know, as the revenues, as the economic business of AI ramps up, and it's ramping up exponentially, you know, if, if I'm right, in a couple of years, it'll be the biggest source of revenue in the world, right? It'll be the biggest industry in the world. And people who run companies already think it. So we actually have this terrifying situation where Uh, you know, hundreds of billions to trillions to, I would say, maybe twenty trillion of capitals on the side of accelerate AI as fast as possible. We have this, you know, company that's very valuable in absolute terms, but, you know, looks very small compared to that, right? Sixty, sixty, sixty billion dollars. And I keep speaking up, even if, you know, it makes folks, you know, there have been

• 1:01:42Speaker 01

  these articles, you know, some folks in the US government are upset at us, for example, for opposing the moratorium on AI regulation, for being in favor of export controls for chips on China, for talking about the economic impacts of AI. Every time I do that, I get attacked by many of my peers. Right, but you're still assuming that we can control it. That's what I'm pointing out. But I'm, I'm just, I'm just telling you how much, how much effort, how much persistence, how much despite everything that's stacked up, despite all the dangers, despite the risks that it has to the company of being willing to speak up, I'm willing to do it. And, and, and that's, that's why, that's, that's why I'm saying that, look, if, if I thought that there was no way to control the technology, right?

• 1:02:27Speaker 01

  If I thought, even, even if I thought this is just a gamble, right? Some people are like, oh, you think there's a five or 10% chance that AI could go wrong? You're just rolling the dice. That's not the way I think about it. This is a multi-step game, right? You take one step, you build the next step of most powerful models, you have a more intensive testing regime. As we get closer and closer to the more powerful models, I'm speaking up more and more, and I'm taking more and more drastic actions because I'm concerned that the risks of AI are getting closer and closer. We're working to address them. We've made a certain amount of progress, but when I worry that the progress that we've made on the risks does not, you know, is not fully aligned with the, um, uh, you know...

• 1:03:12Speaker 01

  Is not going as fast as we need to go for the speed of the technology. Then I speed up, then I speak up louder. And so, you know, you're asking, why am I, why am I, you know, what, you know, you started this interview by saying, what's gotten into you? Why are you talking about this? It's because the exponential is getting to the point that, that I worry that we may have a situation that our ability to handle the risks is not keeping up with the speed of the technology. And that's how I'm responding to it. If I believe that there was no way to control the technology, which I, I, I, I, I see absolutely no evidence for that proposition. We've gotten better at controlling models with every model, with every model that we release, right? All these things go wrong, but like, you really, you really have to stress test the models pretty hard.

• 1:03:57Speaker 01

  That doesn't mean you can't have emergent bad behavior. And I think, you know, if we got to much more powerful models with only the alignment techniques we have now, then I'd be very concerned. Then I'd be out there saying, everyone should stop building these things. Even China should stop building these. I don't think they'd listen to me, which is one reason I think export controls is a better, is, is, is a better measure. But if, if we got a few years ahead in models and had only the alignment and steering techniques we had today, then, you know, I would definitely be advocating for us to, to, you know, to, to, to, to, to slow down a lot. The reason I'm warning about the risk is so that we don't have to slow down, so that we can invest in safety techniques and can continue the progress, continue the progress of the field.

• 1:04:40Speaker 01

  It would be a huge economic effort. Even if one company was willing to slow down the technology, you know, that doesn't stop all the other companies. That doesn't stop our geopolitical adversaries to whom this is a existential fight, fight, fight, fight for survival. So, you know, there, there's, there's, there's very little, you know, there's very little latitude here, right? We're stuck between all the benefits of the technology, the race to, the race to accelerate it, and the fact that that is a multi-party race. And so, I am doing the best thing I can do, which is to invest in safety technology to speed, to, to speed up the progress of safety. I've written essays on the importance of interpretability, on how important various

• 1:05:24Speaker 01

  directions in, in, in, in safety are. We release all of our safety work openly because we think that's the thing that's a public good. That's the thing that everyone, that, that, that, that everyone needs to share. So if you have, if you have a better strategy for balancing the benefits... The inevitability of the technology and the risks that it face. I am very open to hear it because I go to sleep every night thinking about it because I have such an incredible understanding of the stakes in terms of, in terms of the benefits, in terms of, you know, what it can do, the lives that it can save. I've seen that personally. I also have seen the risks personally. We've already seen things go wrong with the models. You know, we have an example of that with Grok and, you know, people dismiss this, but

• 1:06:08Speaker 01

  they're not going to laugh anymore when the models are taking actions, when they're manufacturing and when they're in charge of, you know, medical, medical, medical interventions, right? People can laugh at the, at the risks when the models are just talking, but I think it's very serious. And so I think what this situation demands is a very serious understanding of both the risks and the benefits. These are high stakes decisions. They need to be made with, they, they, they, they, they need to be made with a seriousness. And, and I think something that makes me very concerned is that on one hand, we have a cadre of people who are just doomers. People call me a doomer, I'm not, but there are doomers out there. People who say they know there's no way to build this safely. You know, I, I've, I've

• 1:06:52Speaker 01

  looked at their arguments. They're a bunch of gobbledygook. The idea that these models have dangers associated with them, including dangers to humanity as a whole, that makes sense to me. The idea that we can kind of logically prove that there's no way to make them safe, that seems like nonsense to me. So, so I think that is an intellectually and morally unserious way to respond to the situation we're in. I also think it is intellectually and morally unserious for people who are sitting on 20 trillion dollars of capital, who all work together because their incentives are all in the same way. There are dollar signs in all of their eyes, um, to sit there and say, we shouldn't regulate this technology

• 1:07:32Speaker 01

  for 10 years. Anyone who says that we should worry about the safety of these models is someone who just wants to control the technology themselves. That's an outrageous claim and it's a morally unserious claim. We've sat here and we've done every possible piece of research. We speak up when we believe it's a Appropriate to do so. We've tried to back up, you know, when we make claims about the economic impact of AI, we have an Economic Research Council. We have an, we have a, we have a economic index that we use to track the model in real time and we're giving grants for people to understand the economic, the economic impact, the economic impact of the technology. I think for people who are far more financially invested in the success of the technology than, than,

• 1:08:15Speaker 01

  than, than, than I am, to just, you know, breezily lob ad hominem attacks, you know, I think that is just as intellectually and morally unserious as the Doomer's Position. I think what we need here is we need more thoughtfulness, we need more honesty, we need more people willing to, willing to go against their interests, willing to not have, you know, breezy Twitter fights, hot takes. We need people to actually invest in understanding the situation, actually do the work, actually put out the research and, and actually add some light and some insight to, to the situation that we're in. I am trying to do that. I don't think I'm doing that perfectly

• 1:08:58Speaker 01

  as no human can. I'm trying to do it as well as I can. It would be very helpful if there were others who would try to do the same thing.

• 1:09:07Alex Kantrowitz?

  Well, Dario, I said this off camera, but I want to make sure to say it on as we're wrapping up. I appreciate how much Anthropic publishes. We have learned a ton from the experiments, everything from red teaming the models to vending machine Claude, which we didn't have a chance to speak about today. But I think the world is better off just to hear everything going on here. And to that note, thank you for sitting down with me and spending so much time together.

• 1:09:31Speaker 01

  Thanks for having me.

• 1:09:32Alex Kantrowitz?

  Thanks, everybody, for listening and watching, and we'll see you next time on Big Technology Podcast.

• 1:09:52Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, theHouseMajorityPact.com. Not authorized by any candidate or candidate's committee.

• 1:10:22Speaker 03

  In this life, you can't win.

• 1:10:24Speaker 04

  When the world gets you down.

• 1:10:25Speaker 03

  No!

• 1:10:26Speaker 04

  Escape to Pluto TV.

• 1:10:27Speaker 03

  This is objectively fantastic.

• 1:10:29Speaker 04

  Relax into the movies and shows you already know and love, all for free. Shwing! Even our news is uplifting.

• 1:10:35Alex Kantrowitz?

  You stay classy, San Diego.

• 1:10:37Speaker 04

  Watch School of Rock, Crazy Ex Girlfriend, Anchorman: The Legend of Ron Burgundy, The Ex-Files, Wayne's World, and so much more.

• 1:10:42Speaker 03

  We're not worthy!

• 1:10:44Speaker 04

  The best things on TV really are free. You just have to know where to look. Escape to Pluto TV. Stream now, pay never.