Anthropic CEO Dario Amodei on designing AGI-pilled products, model economics, and 19th-century vitalism

Dario Amodei joins John Collison to talk about Anthropic's growth to ~$5 billion in ARR, how AI models show capitalistic impulses, predictions for an agentic future, the economics of model businesses, and the 19th-century concept of vitalism. Full episode transcript on Substack: https://cheekypint.substack.com/p/a-cheeky-pint-with-anthropic-ceo Timestamps (00:00) Intro (00:50) Working with your sibling (01:43) Building Anthropic with 7 cofounders (02:52) ~$5 billion in ARR and vertical applications of products (07:18) Developing a platform-first company (10:08) Working with the DoD (11:11) Proving skeptics wrong about revenue projections (13:13) Capitalistic impulses of AI models (15:43) AI market structure and players (16:56) AI models as standalone P&Ls (20:48) The data wall and styles of learning (22:20) AI talent wars (26:04) Pitching Anthropic’s API business to investors (27:49) Cloud providers vs. AI labs (29:05) AI customization and Claude for enterprise (33:01) Dwarkesh’s take on limitations (36:12) 19th-century notion of vitalism (37:27) AI in medicine, customer service, and taxes (40:59) How to solve for hallucinations (42:41) The double-standard for AI mistakes (44:14) Evolving from researcher to CEO (46:59) Designing AGI-pilled products (47:57) AI-native UIs (50:09) Model progress and building products (52:22) Open-source models (54:43) Keeping Anthropic AGI-pilled (57:11) AI advancements vs. safety regulations (01:02:04) How Dario uses AI

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00John Collison?

  I'm excited to finally learn what is it like to start a company with your sibling?

• 0:04Speaker 01

  I don't know why you're asking me that question because you know.

• 0:07John Collison?

  It's like, you know, the models want to learn. The models want to be extraordinarily successful in the market.

• 0:11Speaker 01

  Yes, right. In addition to having this learning impulse, the models have this like capitalistic impulse. Sometimes people think of the API business and they say, oh, it's not very sticky or it's going to be commoditized.

• 0:21John Collison?

  I love API business.

• 0:22Speaker 01

  No, no, exactly, exactly. I think we're going to be in a world where the models will make mistakes much less often than humans, but they'll be stranger mistakes.

• 0:33John Collison?

  So we need to invent slurring for LLMs. And that's your appropriate course.

• 0:37Speaker 01

  Oh, wow.

• 0:38John Collison?

  Dario is CEO of Anthropic, one of today's frontier AI labs. He's gone from being an AI researcher just a few years ago to now running one of the world's fastest growing businesses.

• 0:48Speaker 01

  Cheers.

• 0:50John Collison?

  So I'm excited to talk a bit about the Anthropic business. You studied physics and computational neuroscience?

• 0:59Speaker 01

  Yes.

• 0:59John Collison?

  You then worked at Baidu, then Google Brain, then OpenAI, and then started Anthropic.

• 1:06Speaker 01

  Yes.

• 1:07John Collison?

  I will get into the Anthropic business, but I'm excited to finally learn what is it like to start a company with your sibling?

• 1:13Speaker 01

  I could ask the same question of you, but you know, it's almost like there's two things you need to do when you're running a company. You need to like, you know, operationally execute and you know, you need to have a good strategy and kind of see the most important thing or the thing that that no one else sees. And so my job is the second and Daniela's job is the first. And we're both good at the things that we do. And so, you know, I think it's allowed us each to spend most of our time on the thing that we're the thing that we're best at.

• 1:43John Collison?

  Presumably something about the trust side of things as well, where co-founder teams in general in tech and AI as well, there's, I mean, they're unstable pairings and just having someone where you have a long-running and deep trust.

• 1:54Speaker 01

  Yeah, yeah, where you have just total and complete trust. I mean, I think even beyond that, you know, Anthropic has seven co-founders. When we founded it, basically the advice from pretty much everyone was like, seven co-founders is a disaster. The company will fall apart before you know it. Everyone will be fighting with the. Each other, there was even more negativity of my decision to give everyone the same amount of equity. But what we found, and I think it was because, you know, obviously me and Danielle are siblings, but then all seven of us, you know, some of us knew each other for a long time or had history of working, not just knowing each other, but working together in the past. And I think that really allowed us, you know, to always be on the same page. And I think especially as the company grows,

• 2:37Speaker 01

  the idea that you have seven people who really carry the values of the company and project them to a wide set of people, it allows you to scale the company to a much larger size while kind of holding on to the values and the unity that we have.

• 2:52John Collison?

  So I want to ask about the Anthropic business, because again, it's incredible story where it was reported recently that you'd blown through $4 billion in ARR. And so it's a lot of discussion correctly about the technology that you're developing. But also, this is just one of the fastest growing businesses in history. And so I want to talk a bit about the AI market. And maybe the place to start is, what is everyone doing with AI? Like there's coding, there's, you know, customer service work, but like, where does all this revenue come from?

• 3:23Speaker 01

  Yeah, there's a wide range of things and it's kind of changed over time. I mean, I would, I would say definitely, you know, the, the, the application that has grown the fastest, although it's not the, you know, very far from the only application, we have a wide range of them, is, is definitely coding. And my, my theory on why it's grown so fast, other than that we focused on coding, the models are good at coding, it's, it's actually really a statement about kind of societal diffusion, which is that if we look at today's AI models, I think in every area, there's a huge overhang in terms of what they could do compared to, you know, how they're actually being deployed today. Because, you know, there's some friction, people at large enterprises are not familiar with the technology.

• 4:06Speaker 01

  You know, I look at what a bank does or what an insurance company does, and there's, there's huge potential, even if the model stopped getting better, right? Even if we stop building products on top of the model, there's still. Huge billion dollar potential in individual enterprise. And often the CEOs of companies that I talk to understand that perfectly well. But if the company is a 10,000 or 100,000 person company, companies that size, you know, they're set up to operationally do a certain thing a certain way, and it takes time to change them. But in code, you know, the people who write code are very socially and technically adjacent to the folks who develop AI models. And so the diffusion is very fast. They're also the kind of people who are early adopters, who are used to new technology.

• 4:52Speaker 01

  And so I think the big growth in code, I, you know, I would say the biggest cause of that is just that the people doing it and the startups devoted to it are fast adopters who understand the technology super well. But it's by no means limited to code at all. You know, if you, if you, if you look at, there are a bunch of companies that do things like tool use. There are, as you mentioned, customer service. You know, we work closely with, with companies like, like Intercom. We're starting to see some things on the biology side. So we're, we're working both with pharmaceutical and healthcare companies. And we're working on the side of kind of basic scientific research.

• 5:35Speaker 01

  So, you know, we work with companies like Benchling, for example. But, you know, we also work with some of the very large pharma companies. There was something done a while back where we worked with Novo Nordisk to write clinical study reports. So clinical study reports are like, you've done a clinical trial and then you will kind of write up the, the, you know, results and it's like, these are the adverse events, these are the statistics. And the clinical study report takes, normally takes like nine weeks. Well, Claude could do it in like five minutes and then it took a human a few days to check it. And so you can really see the opportunity for acceleration and as the models get better, you know, they'll reach into the, the deep research as well. So I, I guess a way to summarize it would be to say that kind of code is out in the lead. But we see a long tail of quite a lot of other stuff, including some,

• 6:20Speaker 01

  you know, some very, very significant use cases. I think code is maybe an early, an early indicator, like a premonition of what's going to happen everywhere else. It's the same exponential, it's just faster.

• 6:32John Collison?

  Yes.

• 6:32Speaker 01

  Just happening faster.

• 6:33John Collison?

  Right, so there are many places for significant AI uplift, but engineers are used to adopting. You know, you think about Hacker News and people arguing over how to best tools. People are passionate about adopting new tools.

• 6:42Speaker 01

  You know, like, two hours after we released Claude Code, there's some person out there who's like, you know, who's like, you know, tried 10,000 different things with it and plugged it into all the frameworks and, you know, Twitter forms one opinion after two hours and then like revises it opinion in two hours. And, you know, you think of the speed of that as compared to the speed that a pharmaceutical company can use it, can use it in research, right? Or that traditional retail company. And, you know, we want to bring everything to all, you know, some of the biggest benefits in the world are touching the physical economy and we want to get there. But it just intrinsically does not happen at the same speed.

• 7:18John Collison?

  How do you decide which verticals to do yourself versus which to allow platform? Like, you have Claude Code and obviously there's also platform companies like Windsurf and Cursor and everyone like that. You launch Claude for Financial Services. Presumably there are other verticals where you say, well, we're not building a tool there.

• 7:34Speaker 01

  Yeah, you know, we have things like Claude for Enterprise, which is not a vertical, but like a general play to go with Enterprise. You know, I think the way we like to think about it, I think we think of ourselves as a platform company first. So, you know, the analogy here would be maybe clouds or something. If you think of a really large platform business of, you know, the size we're trying to get to in hopefully a small number of years. There are a number of reasons why you would also want to have things that are first party and that, you know, because some verticals end up being more first party heavy. One is when you want to have direct exposure to the users. The end user gives you some sense of how exactly are they using it?

• 8:19Speaker 01

  What are they most looking for? If you're a pure platform and you don't have that direct connection, you can be disadvantaged in various places.

• 8:25John Collison?

  Best products.

• 8:26Speaker 01

  Yeah, it's hard to build the best products. It may even be hard to know where the model really, really needs to go, right? People say things like coding, but like, you know, there are many models that seem to be good at coding, but they aren't good in the way that's actually. Relevant, right? We've actually managed to make Claude good in a way that's relevant to what people actually use. So I think that's one reason. You know, another reason goes back to the large enterprises where, you know, building on an API, sometimes it's more challenging for a more traditional company to do that. And you need to give them something that's a little bit easier to use, either a kit to help them build things, or you know, you need to give them an app. So you know, enterprises have also

• 9:07Speaker 01

  liked Claude code, and we're gradually developing Claude for enterprise into, you know, what we call like a virtual coworker.

• 9:14John Collison?

  But I find it hard to picture Anthropic developing Claude for oil and gas exploration. And you know, why is that?

• 9:20Speaker 01

  Yeah. Why is it that you find it hard to imagine?

• 9:23John Collison?

  Yeah, maybe in fact it's the next launch, but.

• 9:26Speaker 01

  Yeah, we're not currently working on Claude for oil and gas exploration. I would draw a distinction between things we just, you know, things we just like don't allow, right? Things that are like illegal or, you know, things like that. And there are a number of use cases that it's like, okay, you know, we're a platform, people are going to do a bunch of things.

• 9:43John Collison?

  We're just not passionate about it.

• 9:44Speaker 01

  But like, we're not, we're not passionate about it. You know, we're not going to go out and make this happen before the other use cases. So I think there is a component of that where, you know, probably we work on things like science and biomedical, out of proportion to its immediate, you know, profitability.

• 10:02John Collison?

  Because you guys think it's worthwhile.

• 10:03Speaker 01

  Because, because we think it's worthwhile. We feel the same way about things in the developing world. One I'll give you that's controversial, people think about it the opposite way. So, you know, the work we do on defense and intelligence, people are often like, oh, these guys are selling out. I think about it the opposite way, right? So, you know, there was this contract with a ceiling of 200 million with, you know, the DoD and intelligence community. People are like, oh man, you know, Anthropic's, Anthropic's selling out. It's exactly the opposite. Getting another 200 million from some coding startup would take like an order of magnitude less effort than like getting that contract.

• 10:38John Collison?

  Well, you think defense is very worthwhile.

• 10:39Speaker 01

  And we're doing it because we want to defend democracies. And, and we do it within bounds. There are some things we're, you know, we're concerned about. I'm deeply concerned about abuse of government. Authority on the domestic side, you know, we think more on the kind of outward directed side, but that's an example of, you know, the things we prioritize are things that we think are good, not necessarily things that feel good or that people will, will, you know, that we think kind of external buzz will be, will be positive. We actually have conviction around some things and we do them regardless.

• 11:10John Collison?

  You reference the kind of business you want to build. What are your aspirations for the, um, the Anthropic business in, say, three to five years' time?

• 11:17Speaker 01

  AI is strange in, in like a number of ways. Um, you know, I think one of the ways it's strange is that because it's an exponential, we have a hard time calibrating exactly how big, how big the business will be. So we had the following experience. So, um, you know, in 2023, you know, I'd never, you know, raised money from institutional investors before. Um, and so, you know, our revenue was zero at the beginning of 2023 because we had not released a product. So I was putting together something and I'm like, oh, you know, I think we can probably get a hundred million dollars of revenue in the first year. Um, and, and this caused some investors to say, this is crazy. This has never happened in the history of capitalism. You've lost all credibility with me. Goodbye, goodbye. Um, uh, and then we actually did it. Um, uh, and, and so, you know, then the

• 12:01Speaker 01

  next year I was like, oh, well, I think we can go from a hundred million to a billion. Uh, uh, and actually that was, having done it the first time, people were like, it was a little bit less dismissed as crazy, but, you know, still often dismissed as crazy. And then we did it again. Um, uh, and, you know, this year, we're halfway through the year. Um, you know, we're, as you mentioned, well, well, well past four billion revenue. So, you know, in kind of logarithmic space to add another order of magnitude. So there's a bunch of different futures. There's one where once things get to a certain size, you know, the, the curve slows down, but there's a provocative world where the exponential continues. And, you know, in, in, in two or three years, these are the biggest businesses

• 12:41Speaker 01

  in the world. And I think one of the fundamental experiences and uncertainties of working at or running something like Anthropic is you, is, is you, is you kind of don't know. You make this exponential projection. It sounds crazy. It might be crazy, um, but also it might not be crazy because that trend line. Has followed before and I've I've said much the same thing in the context of training AI models in the context of the cognitive capabilities of AI models on the technological side. But but now we're seeing the same kind of continuous lines on the business side.

• 13:13John Collison?

  So what's the analogy to scaling laws here where you know you scale up the relevant inputs for model quality in parallel and you get a much better model performance? Is there something where you put better models in and I don't know the right organization?

• 13:28Speaker 01

  Yeah, yeah, yeah, there's there's something like there's some curve where you know you spend, you know, you spend five X or 10 X more to to train a model. Or you have five X or 10 X more data, you know, or whatever the scaling laws say. And and you know, there's some transfer curve for revenue, right? Where I spend 10 times more on the model and you know, the model goes from being, you know, a smart undergrad to a smart PhD student. And you know, then I go to, you know, a pharmaceutical company and I'm like, well, how much more is that worth? Often, often they they end up saying, you know, that's worth about that's worth about 10 X where these kind of power law distributions occur in a bunch of contexts. Going on the technical side, when you train the model,

• 14:12Speaker 01

  there's a longer and longer tail of kind of correlations that you're capturing as you as you as you train the model, right? Correlations in the structure of language, in the world, in patterns. And and that correlation is what's thought to lead to the scaling laws because there's this kind of logarithmic distribution. And then, as you think of the model getting more and more capable in terms of cognitive tasks, there must be, or we're seeing empirically so far, if you think of the like uses of the model in the economy, right? You know, if I think of, you know, the way that companies are organized, right? There's a kind of power law, there's a power law structure of like of like the

• 14:51John Collison?

  The org charts.

• 14:52Speaker 01

  The the org charts of companies. And it almost feels like you're climbing that power law distribution of value. And then I guess the way I think about product and go to market is that the model wants to be on that exponential of revenue. And product and go to market are they're kind of a way To like, you know, to like, clean the window and let the light shine through, right? There's a way to kind of open the, open the aperture and, and, and let the exponential happen.

• 15:19John Collison?

  It's like, you know, the models want to learn, the models want to be extraordinarily successful in the market.

• 15:23Speaker 01

  Yes, right. In addition to having the learning, this learning impulse, the models have this like capitalistic impulse that like, they want to embody unless they're given a bad product or, or bad sales to go with them.

• 15:33John Collison?

  Because they're really useful, that intelligence is really useful to be, and so it kind of gets pulled out of you.

• 15:37Speaker 01

  Yes, yes, yes. That, that is a way to think about it.

• 15:43John Collison?

  What is the terminal market structure here? Like, is there a few large scaled players? Or do we kind of keep seeing new upstarts for kind of specific degrees spaces?

• 15:55Speaker 01

  It's very hard, you know, it's, it's hard to tell for sure. And I think there was, you know, quite a lot of uncertainty two or three years ago. But I think we might be relatively close to the final set of players, if not necessarily the final market structure, the roles of the players, you know, there's, I would say there's probably somewhere between three and six players. That, you know, depending on, depending on how you count. And those are the players that are capable of building, uh, you know, the, the, the players that are capable of building frontier models and have enough capital to plausibly bootstrap themselves.

• 16:32John Collison?

  I would love to understand how the model business works where you invest a bunch of money upfront in training. And then you have this fast, fast-ish depreciating assets, though maybe with kind of a long tail of usefulness. And hopefully kind of you pay that back thus far. Like, I think the image people have from the outside world is ever larger amounts of CapEx and, you know, how it's all the.

• 16:54Speaker 01

  Get kind of burned. There's, there's kind of like two different ways you could describe what's happening in the model business right now. So let's say in 2023, you train a model that costs a hundred million dollars. And then you deploy it in 2024 and it makes $200 million of revenue. Meanwhile, because of the scaling laws, in 2024, you also train a model that costs a billion dollars.

• 17:20Speaker 01

  2025, you get $2 billion of revenue from that $1 billion and you spend $10 billion to train the model. So if you if you look in a conventional way at the profit and loss of the company, you know, you've you've lost $100 million the first year, you've lost $800 million the second year, and you've lost $8 billion in the in the third year. So it looks like it's getting worse and worse. If you consider each model to be a company, you know, the the model that was trained in in in 2023 was was profitable. You you paid 100 million and then it made 200 million of revenue. There's some, you know, cost to inference with the model. But, you know, let's just assume in this cartoonish cartoon example that even if you add those two up,

• 18:05Speaker 01

  you're, you know, you're you're kind of in a good state. So if every model was a company, the the model is actually, you know, in in this in this example, is actually profitable. What's going on is that at the same time as you're reaping the benefits from one company, you're founding another company that's like much more expensive and and requires much more upfront R and D investment. And and so the way that it's going to shake out is, you know, this will keep going up until the numbers go very large and the models can't get can't get larger. And, you know, then it'll be a large, very profitable profitable business. Or at some point, you know, the models will stop getting better, right? The, you know, march to AGI will be halted for some reason.

• 18:48Speaker 01

  And then perhaps it'll be some overhang, so there'll be a one time, oh, man, we spent a lot of money and we didn't get anything for it. And then the business returns to, you know, whatever scale, whatever scale it was at. Maybe another way to describe it is, you know, the usual pattern of venture backed investment, which is that things cost a lot and then and then you start making it, is is kind of happening over and over again in this field within within the same within the same companies. And so we're on the exponential now. At some point, we'll reach equilibrium. The only relevant questions are, at how large a scale do we reach equilibrium? And is there is there is there ever an overshoot? Right, right.

• 19:23John Collison?

  And yeah, you referenced the cloud companies as a point of comparison, but I don't know, there's something about the cloud companies where it feels like their data center capex is more continuous to just, you know, always doing data centers. Whereas there's something about how discreet these generations are that maybe it's like, you know, the way the the engine manufacturers, they keep coming up with new technologies like, you know,

• 19:41Speaker 01

  Yeah, yeah, you know, like it's like the F 16 or something, or, you know, it might be a little bit like drug development, like, you know, kind of an R and D heavy thing.

• 19:49John Collison?

  Yes. And when you actually go to the effort of training.

• 19:51Speaker 01

  Yeah, yeah, you know, it's it's it's almost like a drug company where it's like you develop one drug and then like, you know, if that works, you develop 10 drugs and that where you develop 100 drugs. Yeah, you know, the drug development market does not work like that numerically, but it is as if it did.

• 20:05John Collison?

  Right. So we can look at each of these models as individual programs and look at their individual P and Ls. And you're saying that the payback math on those, at least in the models we've seen to date in the industry, is not actually that challenging. Where like, I think most like, in, you know, when you're acquiring a customer, if you have a nine month payback on acquiring a customer, you'll do that all day long. That's like a very easy payback to underwrite. And you were saying the paybacks are kind of nine months, 12 months. So like they're.

• 20:29Speaker 01

  I don't want to make any specific claims, but qualitatively, if you look at, if you look at the business this way, model by model, it it looks very viable.

• 20:40John Collison?

  Yes, because the ever growing capex is masking the underlying clause of the the model businesses.

• 20:47Speaker 01

  Yes.

• 20:47John Collison?

  In 2023, everyone is talking about the data wall. Is this how we solved our way out of the data wall?

• 20:52Speaker 01

  Yeah. So I don't know, people talk about things in public and, you know, sometimes there are, you know, rumors or suppositions or whatever. I wouldn't even necessarily assume that there's a data wall. One thing I will say is that, you know, the idea of using RL has been around for a while, right? If we go all the way back to, you know, when Google DeepMind won, you know, beat the world, go champion with AlphaGo, it was RL first. And then we built these language models and, you know, now we're kind of uniting the two together by putting RL on top of the language models, right? That's all chain of thought or reasoning is. It's just a fancy way of saying RL where the RL environment is that the model writes a bunch of things and then and then gives an answer.

• 21:34Speaker 01

  There's nothing more to it than that. It just it just kind of has a fancy name. And so I think of these as kind of the two key ways of learning, right? I think of like. Base LLM training as learning by imitating and RL as learning by trial and error. I think those are the two styles of learning, right? You know, if I'm like a child, there's two ways for me to learn. You know, I look at my parents and I'm like, oh, they do something and I try and learn what they do. Or I can just kind of like experiment with the world and learn things. And it's very clear in developmental psychology that that people use use both. And so we're now seeing that recapitulated in the in the language models. And so we have a stage where we do the imitative learning and we have a stage where we learn by trial and error.

• 22:18Speaker 01

  So it seems very natural to me.

• 22:20John Collison?

  The other thing that's obviously notable to, I think, people not in the AI industry looking at it is all of the talent wars and, you know, the fact that your IP walks out the door each evening. And, you know, you referenced in a recent interview you gave, you know, hundred million dollar

• 22:39John Collison?

  secrets that were a few lines of code. And obviously, I think you were talking about that in a national security context, but you can also think about it in a talent context. And so how does one, like, in the pharma industry, they protect their secrets with patents. In Wall Street, where also they have, you know, hundred million dollar secrets that are, you know, just a very simple idea. Renaissance Technologies, the hedge fund, you know, just very successfully locked up its employees. How do you make keeping a commercial lead work in kind of the current AI environment?

• 23:07Speaker 01

  Yeah, so one thing I will say is that there are some things that are like that. But I think more and more, as the field matures, it starts to be more about know-how and ability to build complex kind of objects.

• 23:21John Collison?

  Yes.

• 23:21Speaker 01

  Right? So, you know, some of the ideas we work with are simple. But I would say the simple ideas, the ones that are like, oh, yeah, twiddle this element of the transformer or something, those tend to be independently discovered or anyone knows them before too long. But there are things like, oh, man, this thing is actually really hard to implement from an engineering sense and we have it implemented. Or this thing is just kind of a pain to do, or there's a know-how to doing it.

• 23:49John Collison?

  Yes.

• 23:50Speaker 01

  And those tend to be more collective things that, you know, that are more difficult. to leak. And so I think those things are substantially more defensible. That said, you know, there's still leakage and we still, we still don't want it to happen again, both for commercial competitive reasons and for national security reasons. Both are problems.

• 24:09John Collison?

  Yes.

• 24:09Speaker 01

  And so a few things we do, one is, you know, we tend to compartmentalize information. So, you know, if you talk to any intelligence agency, that's, that's how they operate. You're only told what you need to know.

• 24:20John Collison?

  Yep.

• 24:20Speaker 01

  Um, and I think everyone within, within Anthropic.

• 24:22John Collison?

  But that's probably quite different to a normal Silicon Valley culture where, you know, everyone's just flying around the company.

• 24:26Speaker 01

  Yes, we actually do that at the same time as we have a very open culture. I say things to the company that, you know, maybe another person would, you know, put it in kind of PR speak or, you know. But when there is a secret.

• 24:38John Collison?

  Yes.

• 24:38Speaker 01

  Then I think that actually leads to people trusting that, you know, it's, it's something that, you know, something that you actually need to know.

• 24:45John Collison?

  Yes.

• 24:45Speaker 01

  And then, and then finally, um, uh, having better retention rates and losing less people is what, one of the most important things here. So, you know, we have the highest retention rate of all the AI companies. You know, I think the differences are even starker because, you know, everyone has, uh, you know, kind of a non-regretted attrition rate that's maybe constant. So if you subtract that off, then the, the difference is even larger. Sometimes when people leave, they come back.

• 25:11John Collison?

  I saw that recently.

• 25:12Speaker 01

  If you look at, you know, you can, you can see publicly the list of people who went to the meta superintelligence lab, even if you normalize for our size.

• 25:19John Collison?

  Yes.

• 25:20Speaker 01

  It's not, you know, and then many, many, you know, many turn them down.

• 25:24John Collison?

  So, so in the crazy hundred billion dollar comp wars that everyone's been talking about, you guys have not had too hard a time of that.

• 25:29Speaker 01

  I think relative to other companies, we've done well. We even have been relatively advantaged. It's like a mixture of true belief in the mission and belief in the upside of the equity. Like I, you know, I think, I think, you know, we, we, Anthropic has developed a reputation for doing what it says it will do, for in some cases, making less promises, but keeping those promises that we make.

• 25:52John Collison?

  Yes.

• 25:52Speaker 01

  And, you know, being very clear on what we stand for and, you know, being consistent over the years and standing for it. Um, that, that creates a unity around the company and I think it's a good guard against cynicism.

• 26:03John Collison?

  And when you're talking about the upside of the equity, when you're pitching investors or maybe candidates, how do you pitch the Anthropic business? Like, we're building a very large business, that's a good start.

• 26:12Speaker 01

  Yeah.

• 26:12John Collison?

  What else goes into it?

• 26:13Speaker 01

  So often I'll talk about the platform and the importance of the models. You know, for some reason, sometimes people think of the API business and they say, oh, it's not very sticky or it's going to be commoditized.

• 26:25John Collison?

  I love API.

• 26:26Speaker 01

  No, exactly, exactly. You know, and there are even bigger ones than both of ours. I would point to the clouds again.

• 26:33John Collison?

  Yes.

• 26:33Speaker 01

  You know, those are $100 billion API businesses. And you know, when the cost of capital is high and there are only a few players. And relative to cloud, the thing we make is much more differentiated, right? Like, these models have different personalities, they're like talking to different people. You know, a joke I often make is like, you know, if I'm sitting in a room with like 10 people, does that mean I've been commoditized?

• 26:57John Collison?

  Yes.

• 26:57Speaker 01

  You know, there's like nine other people in the room who have a similar brain to me, they're about the same height, you know, so who needs me? But, you know, we all know that human labor doesn't work that way. And so I feel the same way about this. So, you know, I think the API business is a great business. And then, but you know, we want to go broader than that. You know, the way I think about it is other players such as OpenAI and existing incumbents such as Google are very focused on the consumer side. The idea of providing AI to businesses is something that we are trying to get better and better at.

• 27:32John Collison?

  Yes.

• 27:32Speaker 01

  And I think we're out to an early lead in that. I'm not sure, because I don't know for sure what the revenues of the other players are, but I think we probably at this point have the plurality of the API market, most likely, and AI for business market, perhaps.

• 27:49John Collison?

  Yeah, no, it's funny when you talk about the kind of the commodity argument where we obviously grew up facing this as a skeptical argument. And I remember finding it so striking when AWS finally had to break out their numbers in 2015. Remember, they used to be wrapped up in Amazon's numbers?

• 28:05Speaker 01

  Yes, yes, yes.

• 28:06John Collison?

  And people had been talking about, pundits had been saying, oh, cloud is a commodity, it's uninteresting, and then they broke it out and, you know, it's one of the greatest businesses of all time. And there's something where a business can have competitors and it can have buyers who care about price, but that's very different from being a commodity.

• 28:24Speaker 01

  Yeah, yeah, yeah, exactly.

• 28:25John Collison?

  And as you say, all these products work differently.

• 28:26Speaker 01

  No, no, no, exactly. I mean... More like one of the biggest customers of the clouds, right? And we use more than one of them. And like, you know, I can tell you the clouds are much less differentiated than the AI models, right?

• 28:38John Collison?

  For sure, because it feels like, one, the behavior is non-deterministic, which that's by design trying to make it hard, but that just naturally means that, oh, we get the customer service answers we prefer with this model versus that model.

• 28:48Speaker 01

  No, no, no, exactly. You don't, you don't know why, you know, it's a little like baking a cake, right? It's like, you know, you put in the ingredients. It just, it kind of comes out a certain way. And like, you know, one chef makes it this way and the other chef makes it this way. And you're, if you're like, make it exactly like that chef makes it, you can't, right? You just can't.

• 29:05John Collison?

  And presumably it's striking to me, none of the AI products are that personalized right now. But it feels like personalization will be a huge deal.

• 29:14Speaker 01

  Will be a huge deal.

• 29:15John Collison?

  And will be a big source of stickiness for the, like, because you won't want to switch products. And I don't know exactly what that looks like, but given the amount of, for both the consumer and the business use case.

• 29:24Speaker 01

  Absolutely, absolutely. You know, I think, I think we've just started to scratch the surface in terms of models that are customized in various ways for working with a particular business or a particular person within the business. So I think we're just seeing the beginning of the API business. But I don't think AI for business is just about API. With things like Claude Code, you know, we're selling that to not just individual developers, but enterprises as well.

• 29:50John Collison?

  Yes.

• 29:51Speaker 01

  And they find it something useful. Claude for enterprise, that is selling to a lot of enterprises. I actually see it, and you see this with some of the clouds, where they have a bunch of different services, right? Some of them are apps, some of it is the underlying cloud itself. And, and what they are is, you know, the way that AWS or GCP or Azure will present themselves and the way that we are starting to present ourselves is, hey, what we want to be your one-stop-shop for AI or for cloud. And you can buy all of these things and you can talk to us about which to use for what.

• 30:25John Collison?

  Yes.

• 30:26Speaker 01

  And, and so I think that, that starts to create the outlines of a more, more durable business.

• 30:31John Collison?

  If you think about a typical Fortune 500 company, how? You know, they're probably playing with AI for customer service. Their engineers maybe have AI-powered coding tools. How AI-adopted are they compared to how much they should be?

• 30:45Speaker 01

  Well, certainly much less than they should be.

• 30:47John Collison?

  Yeah, but is it like 5%, 30%?

• 30:49Speaker 01

  So what I would say is there is almost, there is very often conviction at the top. You talk to the CEO, the CEO gets it. You talk to the CTO, the CTO gets it. The struggle they have is that they have 100,000, pick a company that has 100,000 people, who, their job is to do something else. Their job is to do banking, or insurance, or drug development. And they've heard about this AI stuff, but you know, they're not like, this is not what they're an expert in. And so the challenge is often, we are working with the leadership of the company to get the 100,000 people in the company really familiar with and using the technology.

• 31:30Speaker 01

  I think, again, the code stuff goes the fastest because the developers are the ones who are most adjacent and most watching the trend. Some of the kind of customer service and process stuff is next to go, but you really have the instinct that even with today's models, it could be, you know, 100 times bigger than it is. Like, you really get that sense.

• 31:49John Collison?

  Yes. My intuition is sort of that we will see the patterns of AI adoption from startups because they're unconstrained by existing organizations, so they can kind of do whatever makes sense. Versus, large organizations are somehow calcified because they have all these people whose job it is to do X, need to be consulted and everything like that. So we'll see the new behaviors from the small startups. And then large companies, as you say, the CEOs and CTOs are switched on and they're smart. They say, "Hey, we should be doing that." And they'll kind of port the new ideas from, kind of like the option we saw of cloud or many of these other tech.

• 32:20Speaker 01

  Yeah, yeah.

• 32:21John Collison?

  Is that what you're seeing?

• 32:21Speaker 01

  They'll port the new ideas from the small companies, or the small companies will become threatening to them and disrupt them. And that will give them the urgency to kind of drive things through and make them happen.

• 32:34John Collison?

  Yes.

• 32:35Speaker 01

  A pattern I've seen that works pretty well, that I actually recommend if you're a large company, is to kind of make a strike team or strike force that's separate from the rest of the company and kind of develops these prototypes. And then basically, you can get momentum behind something. And then there's always this hard work of integrating into the rest of the company. But if you have a lot of momentum and you've done the hard work and you've shown the thing works, then it's easier to do that.

• 33:01John Collison?

  Dwarshash, did you read his recent blog post on his AI timelines?

• 33:04Speaker 01

  Oh, on continual learning, yeah.

• 33:07John Collison?

  Yeah, and he talked about how his fundamental issue at many of the AI models for productivity is that they're like the virtual, they're like the super smart virtual coworker who started five minutes ago, but they remain the coworker that started five minutes ago, you know, they don't learn over time.

• 33:26Speaker 01

  Yeah, yeah.

• 33:27John Collison?

  And how will we solve that?

• 33:28Speaker 01

  Yeah, so, you know, the pattern that I've seen in AI on the kind of research and technical side is that what we've seen over and over again is that there's what looks like a wall, you know, it looks like AI models can't do this, right? It was like AI models can't reason, and recently there's this AI models can't make new discoveries. A few years ago, it was like AI models can't write globally coherent texts, which of course now they obviously can. You go back a few more years and, you know, it was like this Chomsky thing of like, you know, they can get syntactics right, but they can't get semantics right. And every one of those has been blown through.

• 34:07John Collison?

  So what have we blown through on the new discoveries?

• 34:09Speaker 01

  This is the thing that people have said recently. Actually, my view on this, like many of the other things on new discoveries, is that it's not really a binary, right?

• 34:21John Collison?

  They don't get to have their name in the paper.

• 34:22Speaker 01

  Yeah, yeah, they don't get to have their name in the paper, but like, what is a new discovery? Like, what is genius? I think I remember this developmental psychology book, but they were saying something like, you know, we kind of lionize genius, but like, you know, let's say that like a table's wobbly.

• 34:38John Collison?

  Yes.

• 34:38Speaker 01

  And, you know, I'm like, oh, I take the coaster and I put it under the table and it's not wobbly anymore. Like, that's an idea in a way, that's like a new discovery. Like, you know, even if I've never seen someone do that before, you know, that's like a, that's like a new discovery. And, you know, the difference between that and the Nobel Prize winning discovery, it's a matter of degree, not a fundamentally different matter. And so I would say that the AI models make discoveries all the time, right? I've had family members where they had a medical problem and the AI model diagnosed, you know, Claude diagnosed their medical Problem.

• 35:11John Collison?

  Yes.

• 35:11Speaker 01

  When doctors missed it like that's not a big new but but like that that's a new discovery like. You you know there and you could say, oh, they're just pattern matching the things that that happened before, but that new discoveries are like that. Think of writers who have written novels or something that are totally new and you're like, well, what are your influences? And you know, they're remixing together and adding a new element. So it's it's all more continuous. And that was the thing I was gonna say about continual learning. I think this idea that it isn't present is. I you know, I would say it's present, it's present a little bit.

• 35:45John Collison?

  Yes, it's comfortable.

• 35:46Speaker 01

  We're gonna, we're gonna find a way to get more of it. So for instance, the models learn within the context. You talk to them and they absorb the context. Eventually, the context is gonna be a hundred, a hundred million tokens. And maybe we'll train the model in such a way that it is specialized for learning over the context. You could even during the context, update the model's weights.

• 36:05John Collison?

  Yeah.

• 36:05Speaker 01

  Um, uh, so there are lots of ideas that are very close to the ideas we have now that could perhaps do this. I think people are very attached to the idea that they want to believe there's some fundamental wall. There's something different, something that can't be done. It kind of reminds me of.

• 36:21John Collison?

  Coping mechanism deep down?

• 36:23Speaker 01

  Yeah, you know what it reminds me of? So you know, the the 19th century notion of vitalism? Um, you know, this was the idea that, you know, the the the human body and and you know, like, like. Organisms that are alive are made of a fundamentally different material than inanimate matter. Um, which of course, we know scientifically now is is not true, but um, it's something people very much wanna believe. And your common sense seems to suggest it, like, I'm not very much like a table.

• 36:50John Collison?

  Yes.

• 36:50Speaker 01

  Um, I've made of, you know, very different, very different materials than metal or glass or whatever. Um, but you know, when we actually go down to the fundamental units, of course, we're all made of the same thing.

• 37:00John Collison?

  But you think people now have this kind of modern concept of vitalism in whatever the the fundamental humanity is? And they're saying, oh, you know, models can't do that.

• 37:08Speaker 01

  I think there's some, I think there's some tendency to, I think there's some tendency to believe it. And I think, I think as with vitalism, the way around it is to, is to, you know, is to, is to recognize that a mind is a mind, no matter what it's made. The notion of the dignity or the specialness of cognition or sentience. It's not that it isn't special, it's that it can be made out of anything.

• 37:27John Collison?

  You referenced the medical use case, which I think is a very cool use case. Obviously, one because of all the people who have fixed medical issues as a result. But another one is you talked in your Machines of Love and Grace post, which I really enjoyed. I thought was very well done about the marginal returns to intelligence. You know, one of the places where intelligence is the limiting factor. And my read of the popular medical use case is, obviously, it's kind of a charismatic use case. But also, for most normal people, they have like some kind of medical issue, low level or serious or something like that. And actually, society is very just intelligence limited there. Not that you don't have access maybe to a smart doctor, hopefully you do. But they give you very limited time, you know, they think for 10 seconds about your problem.

• 38:09Speaker 01

  Yeah, yeah, yeah, exactly.

• 38:10John Collison?

  You know, test time compute was actually what we needed there on the medical stuff. But was that kind of your take on this?

• 38:16Speaker 01

  That is how I think about it as well. You know, I have talked to Nobel Prize winning biologists who say, I will only, I mean, it sounds a little elitist. But they'll say, I'll only go to the top 1% of doctors. Because the rest of the 99%, I can get better advice from an LLM. You know, it really is true, doctors are busy, they're overworked. And just the nature of medical data and medical information. You know, it's a lot of pattern matching, it's a lot of the same things. The, you know, the level of consistency and the ability to put together many different facts. You know, I think, I think it's something that LLMs are quite, are quite good at.

• 38:56John Collison?

  So you talked in this Machines of Love and Grace post about some of the big humanity level areas where we're intelligence limited. Again, the personal medical use case is a good example of one where society is intelligence limited. And if you give lots of people much more intelligence on their specific issues, it's very valuable. What are other areas, either in the consumer use case or in the business use case, where you think we're just very obviously intelligence limited?

• 39:20Speaker 01

  Yeah, the places where at least the AI models of today can help the most. The characteristic quality is something is repetitive, but every example is a little different, right?

• 39:32John Collison?

  Yes.

• 39:32Speaker 01

  Automation before AI, if you could program exactly how it happened, you could do it. So if you were doing the same thing over and over again.

• 39:39John Collison?

  Yes.

• 39:40Speaker 01

  But customer service is like, you know, just to take customer service as an example. There's like a long tail of stuff, but a lot of it is like you get a bunch of calls. Each call is different, but each call is basically about one of 10 things. And it's like a different person in a different voice saying like, basically, you know, one of the, one of these 10 things in a different way. And, and, and, and that, that situation where things are repetitive and similar, but, but not the same and each has its own thing.

• 40:07John Collison?

  Yes.

• 40:07Speaker 01

  That's where, that's where AI can come in the most, I think.

• 40:09John Collison?

  Yes, yes. And Dworkesh had in the same blog post, the prediction that you can't yet give an existing AI all of your financial data and forward it all to emails and have it do your taxes. And his prediction for the year in which you can, you know, what is the year where your first tax return is done by just emailing, emailing everything to whatever AI you use, his prediction is 2028. What do you make of that prediction?

• 40:35Speaker 01

  Uh, probably sooner than that.

• 40:38John Collison?

  Okay.

• 40:38Speaker 01

  I don't, I don't know if it's 26 or 27. Some of that is model, mostly, mostly accuracy. I think the model could do that today, but it would make too many mistakes. And so working on ways to have the model check its own work and do less mistakes is one part. There's kind of an interface part of it as well, but I, I would be surprised if it takes that long.

• 40:58John Collison?

  Okay, 26 or 27. And when you say about mistakes, actually, you were running through the list of things that people thought we would never solve in AI. It feels like hallucinations should be on that list. Not that they're totally solved, but they've gotten a lot better.

• 41:09Speaker 01

  They've gotten a lot better and I think people have gotten more used to, they kind of know what to trust the model for and what not to trust the model for. The models have also been grounded in citations. I mean, we've done that with Claude.ai, we've done that with enterprise Claude. So part, I think part of the solution is citation, part of the solution is algorithmically, the models hallucinate less now.

• 41:29John Collison?

  Yes.

• 41:29Speaker 01

  And part of the solution is people. have adapted and understand the weaknesses of the model. My my view on things like hallucinations has always been there's a certain class of critic who points to something where models are weird or worse than what than what humans do and say see they're not like us at all or you know or they'll not they'll never get there. And I kind of get where the instinct comes from where like maybe they're looking to you know to see if we've matched the human brain exactly they're saying oh this is so different it can't be like a human brain but I I basically I just I just think it's a fallacy there's there's a notion of kind of general intelligence but it's it's made up of a bunch of different things and you can simply have most of the things and be much worse on some and much better on others like if we look at humans that are

• 42:13John Collison?

  Yeah, have you met humans?

• 42:14Speaker 01

  Yeah, yeah, have you met humans, right? Like, you know, if you if you look at, you know, humans who are who are, you know, autistic versus humans that are schizophrenic if you look at the optical illusions that humans face that machines are not fooled by it's very clear that we have some of these weaknesses just like you know just similar to the models hallucinations it's just that they look very different and we're much more used to them because we're surrounded by humans all day.

• 42:40John Collison?

  Yeah, the autonomous vehicle double standard feels like the kind of clearest example of this.

• 42:47Speaker 01

  clearest example of this, yes.

• 42:48John Collison?

  Yeah, where people have much higher standards.

• 42:50Speaker 01

  People have people have much higher standards. But it I think it's going to be a feature of this technology and it has implications on the business side. I think we're going to be in a world where the models will make mistakes much less often than humans but they'll be stranger mistakes. And actually that takes some adaptation because imagine you're an end user. If you work with humans, you get used to it and you have some notion, right? So if a human makes a mistake 5% of the time, you might have a good understanding of why, you know, like let's say I'm talking to a customer service agent and they're kind of sounding coherent and they're slurring their speech. Uh, you know, they've probably had too much of this and they're not doing their job very well.

• 43:32Speaker 01

  Um, and you know, that's that's a bad mistake to happen, but also if I'm talking to this person, I kind of know what's going on and I know not to trust what they're saying. Whereas an LLM might make a mistake five times less often, but it's it's kind of, you know, it's more deceptive. The model sounds just as erudite, just as coherent as as it does when it's saying something that's right. But that's not a, you know, that's that's an adaptation thing. That's a that's a, you know, it's not a fundamental thing. And and that's something that when we talk to our customers, we tell them about that. We tell them they need to get used to that.

• 44:06John Collison?

  So we need to invent slurring for LLMs, is what you're saying.

• 44:08Speaker 01

  Right, right, right. Exactly.

• 44:13John Collison?

  So you started out as a researcher, but now you're the CEO of a company and you're in the business of selling AI. And so what have you had to learn about go-to-market or dealing with customers?

• 44:27Speaker 01

  Yeah, yeah, yeah.

• 44:28John Collison?

  All the rest of the stuff?

• 44:29Speaker 01

  Absolutely. I think my view on this was, you know, I started a company not, you know, not not because I was initially excited about, you know, selling things or business or any of that. I'd seen the way that some of the other companies had had run and, you know, the the magnitude and gravity of what they were trying to build. And I was just a bit concerned that, you know, that, you know, the the the people and the motivations were maybe were maybe not the best ones. And, you know, I knew that there would be a number of players in this space. But but it felt like having at least one player that that kind of had a strong compass in how we how we do things could could have positive effects on the ecosystem. We would build things in a different way. We would deploy them in a different way.

• 45:12Speaker 01

  And, you know, above all, we would have a, again, sometimes short list of principles, but but we would stick to them as well as as as well as we could. So I think that was the initial motive. And of course, I was excited about building that, building the technology. And and, you know, I think as that has has happened, of course, you know, I and the other co-founders have kind of had to learn how to think about, you know, the kind of the business and the and the strategy. I think I've been very naturally interested in the business side of it. Actually, I was surprised at how quickly I became interested. And actually, the primary reason was that I was curious about all the industries that are Customers of us, right? Somewhat like the clouds and perhaps like your business.

• 45:56Speaker 01

  The businesses that we serve are, you know, they run across every possible industry. And so you, you know, you learn these things about parts of the economy that you've never thought about. And even in areas where nominally know a lot, like, you know, I used to be a biologist. So I, in a way, I know a lot about the pharmaceutical business. But, you know, I'd never thought about it as a, you know, beyond the science. I never thought about the portfolio side of it. I'd never thought about how clinical trials work and how they could be made cheaper. You know, I never thought about the, you know, defense and intelligence business in any, in any, in any great detail. And so, you know, you, you run through those and I just, I just find it super interesting to understand what people's problems are and how AI can help with those problems. And so I feel like I took very, very naturally to that.

• 46:40Speaker 01

  Actually, the product side was one where I was initially more reluctant. I, I felt like I just had a natural interest and curiosity in the, in the business side of it. But like building apps, it was somehow initially, it was, it was never like, you know, a thing that drew me in, even after I started the, even after I started the company. But, you know, I think more recently, as I've seen what products have succeeded and what products haven't. I think this idea of how to design products so that they're, you know, what we call AGI pilled, right? So that the, the direction of the product is, is durable and, and is kind of a bridge to things that are useful in the future. Right? We've all heard this idea of wrapper companies or wrapper products.

• 47:22Speaker 01

  The idea is, you know, you make Claude N and, you know, someone makes a product that, you know, basically addresses the deficiencies of Claude N, but then you come out with Claude N plus one and it just kind of eats it. The advice I always give that I think all the AI, you know, all the folks at the AI companies give is like, you know, don't, don't make that. Um, see the direction of the field and try to make something that's complementary.

• 47:45Speaker 01

  And, and I think thinking about how to make products in a new way, in a kind of AGI pilled way, that, that actually has caught, has caught my interest a great deal.

• 47:54John Collison?

  Okay. So I'm glad you brought this up. Doesn't it feel like we have no AI UI? Right now, like we still enter text into text boxes, you know, literally same as terminals from the 1970s. I mean, a bit more random corners and everything like we still talk into voice companion modes that are manually triggered, which is the same as pre transformer Siri. So you eyes are just

• 48:19Speaker 01

  Yeah, yeah, there's something not quite right about it. I, I, I basically, I basically agree with you. You know, it, it reminds me a little bit of, you know, in the early days of the internet. It was like, you know, people, people would make these websites that had, uh, you know, structures that look like they were in the physical world, you know, open the closet and like do the

• 48:39John Collison?

  There's where the horseless carriage

• 48:40Speaker 01

  There's some term for this. I forget. There's some word for this. I forget. I forget what it is.

• 48:44John Collison?

  Skeuomorphism?

• 48:44Speaker 01

  Skeuomorphism, yes. Uh, it feels like there's some of that going on here. A thing I would say is that as we move more towards agents, we're going to be in a world where the AI model can do something end to end, like we're almost there with Claude can do something end end and get it right most of the time.

• 49:03John Collison?

  Yes.

• 49:04Speaker 01

  And a human's main job is to kind of check, right? Um, or check sometimes. Um, but interestingly, checking often means getting really into the details of what happened. And so there's some kind of impudence mismatch here.

• 49:18John Collison?

  Yes.

• 49:19Speaker 01

  That, that some product or interfaces the solution to.

• 49:22John Collison?

  Yes.

• 49:23Speaker 01

  Where you, you, you want something that's as slick as possible and good, just goes off and, and does something.

• 49:29John Collison?

  Yes, yes.

• 49:30Speaker 01

  And you don't want to have to pay attention most of the time, but when something's wrong, you might actually need to get quite involved.

• 49:35John Collison?

  Yes.

• 49:36Speaker 01

  Um, and I don't feel like any products or interfaces operate on this principle now or handle this problem now.

• 49:42John Collison?

  Yes, yes.

• 49:42Speaker 01

  I don't know if that makes sense, but

• 49:44John Collison?

  No, it does. I, I like, I agree. I think what you want is your agent to go away and do really good work for you and then come back with its work product to let you review, steer us, decide.

• 49:55Speaker 01

  But you can't be overwhelmed because it's going to do so many more things than, you know, than, than you have time to look at it. If you're always looking at it, you know, it can be slower than if you just did it yourself. And so it, it, it actually strikes me as an interface problem.

• 50:09John Collison?

  Yes, yes. The generalization Of this is it feels to me, one of the most exciting things about AI is we have such an overhang of current capabilities, turning them into good products. Where even if AI progress was frozen right now, we'd have like 10 years of good products.

• 50:26Speaker 01

  Oh, oh, I completely agree. And actually, the way that products are being built, I think by everyone in the industry, but we've thought about it this way, is very different because the progress is continuing. If the progress in models stopped, the way we built products would change instantly. The reason is, I don't think we've ever had before a situation in which the technology is changing under you so fast as you're building the product. And so this idea of long-term product roadmaps or the usual way of product planning. I've started explicitly, again, you know, early in Anthropic, I was like, I don't know anything about product. I'm a doofus. But now I always try to talk to people when they come in and they say,

• 51:09Speaker 01

  this is not like building products in the non-AI space, right?

• 51:12John Collison?

  They need to be more AGI built.

• 51:13Speaker 01

  Yeah, you may be the expert at building these, but like, the technology is moving under you. So like, these ideas about fast iteration, they're even more true than they are normally.

• 51:23John Collison?

  What's a specific example of this?

• 51:25Speaker 01

  I think that if you're trying to make like, you know, you're like, we're gonna make something and it's gonna be ready in six months. I think that makes even less sense here than it makes, you know, the building in isolation.

• 51:37John Collison?

  So you need to just have tighter ship schedules.

• 51:39Speaker 01

  You need to have tighter ship schedules, you need to try things. It's very hard to tell, even harder to tell what's gonna catch on. Because a new model may have come out and, you know, a new model may suddenly be good at something that makes a product possible.

• 51:54John Collison?

  Yes.

• 51:54Speaker 01

  And so, much more than anything else, you're trying something that's never been tried, right? There's a new model, it's only available within the company. So the thing you ought to do is like, just build something on it, let people internally try it.

• 52:06John Collison?

  Yes.

• 52:06Speaker 01

  It's, there's this like eternal September vibe to it, right? Where it's like, it's, you know, it's as if you discovered database technology for the first time. And you're like, well, what could you build on this, right? And it's always, it's always the first day, right? That's what is different.

• 52:22John Collison?

  You mentioned database technology and maybe that provides an interesting analogy. And as we think about open source, the first relational databases that were successful in terms of adoption were proprietary, but then the open source guys caught up. How do you keep the gap with the open source?

• 52:40Speaker 01

  Yeah, so you know, open source, I think, has a different meaning in AI models than it has in other areas, right? And you know, for this reason, some have called it like open weights models, right? To distinguish. I think the main difference is that if you see the weights of the models and you look in, you can't understand what's actually going on, right? There's not that kind of composability. I can't read the source code. I can't, I can't.

• 53:05John Collison?

  You can't produce a trivially different.

• 53:06Speaker 01

  Yeah, yeah, you know, I can't produce a trivially different version of it. Now, you know, Anthropic is actually working on mechanistic interpretability, which allows you to see inside the models. And so we're actually working on things that would allow some properties, but we're not there yet. We're not anywhere close to there. There are some things you can do. For example, if you have access to the model, you can fine tune the model. We're now through interfaces kind of allowing people to fine tune the model. So there is a question of how valuable access to the actual model weights is over and above some thick API that lets you do something. There's some question of economics, but note that it costs a significant amount to run the models on the cloud. Someone has to host it, someone has to run fast inference, and then you're back to the

• 53:51Speaker 01

  margin or some portion of the margin.

• 53:53John Collison?

  So you think open weight models are not that useful and fully open source models, there's just a big gap?

• 53:58Speaker 01

  I guess what I would say is that the analogy to the, to previous technologies is only partial, right? The analogy, it's kind of a different thing that we're still, that we're still discovering. But, you know, I can say from, from our perspective, that when a new model comes out, you know, when a competitor model comes out, we don't really think about whether it's an open weights model or not. We think about whether it's a strong model, right? So if someone makes a strong model that is, that's good at the things that we do, like, that's, that's competition. That's bad for us, whether it's an open weights model or not.

• 54:35John Collison?

  Yes.

• 54:36Speaker 01

  There's, there's not a huge difference between the two.

• 54:43John Collison?

  How is Anthropic more AGI-pilled than other organizations? So one is faster, like a tighter product release cadence, but maybe more broadly across the organization, not just within product development?

• 54:53Speaker 01

  Yeah, so you know, I mentioned this thing that every couple of weeks I get up in front of the organization and you know, and kind of, you know, describe my vision. And I think one of the purposes of that is to keep people kind of focused on the mission. You know, it's a, it's a strange state of the world and I always express uncertainty about it, but I say, you know, if I were to bet, I would bet in favor of this, that, you know, in one or two or three years, I don't know exactly how long it's going to be, um, you know, we'll, you know, we'll have what I've described as like a country of geniuses in the data center. And like, this is weird, like it's going to change the economy, it's going to accelerate the pace of science, it's going to, you know, pose global alignment and national security risks, it may pose economic problems.

• 55:37Speaker 01

  The upside is huge, the potential for disruption is, is also huge. Um, and I think what I'm trying to fight against is, you know, the idea of, you know, employees who join and they're like, oh, you know, I worked in this industry, you know, I worked at this kind of company and I'm going to work at an AI company and maybe a couple of years later I'll go to the, this, you know, like-

• 55:56John Collison?

  This is very categorically different from previous experiences.

• 55:59Speaker 01

  This is a really different thing. And, and I think up and down the organization, we want to make sure that, you know, when our, you know, when our finance people, you know, think about financial projections, they, they understand this, not that there's necessarily going to be an exponential, but like wild outcomes are possible, right? When our recruiting thinks they're, they're, you know, they're like, oh yeah, you know, like this crazy comp stuff could happen because, because it hap- yeah, and, you know, and when the product people think they make AGI-pilled products, when the policy people interact, they understand the stakes of what may happen. And so I think a big part of my job is, is keeping the coherence of the organization around this central thesis. Not that everyone has to, you know, like, believe the thesis, right? It's not like a, you know, there's not a-

• 56:44Speaker 01

  Indoctrination and people chanting with robes or anything, but like, but like the the basic idea that the company is, is, is built around this hypothesis that it is possible and perhaps likely that these large changes will happen and every aspect of the, the, the, the business as well as the things the company is doing for social benefit should be constructed around a strong possibility that this may happen.

• 57:11John Collison?

  To put numbers on this, you've talked about the potential for 10% annual economic growth powered by AI. Doesn't that mean that when we talk about AI risk, it's often harms and misuses of AI. Isn't the big AI risk that we slightly misregulate it or we slow down progress and therefore there's just like a lot of human welfare that's missed out on?

• 57:34Speaker 01

  Yeah, well, I've, I've had the experience where, you know, I've had family members die of, you know, diseases that were cured a few years after they died. So, you know, I, I kind of truly understand the stakes of not making progress fast enough. I would say that some of the dangers of AI have the potential to de significantly destabilize society or threaten humanity or civilization. And so I think we don't want to, you know, we, you know, we, we don't want to take idle chances with that, with that level of risk. Now, I'm not at all an advocate of like, stop the technology, pause the, you know, pause, pause the technology. I think for, I think for a number of reasons, I think that's just, it's just not possible. Like we have geopolitical adversaries, like they're

• 58:19Speaker 01

  not going to not make the technology, the amount of money. I mean, if you even, you know, propose even the slightest amount of, you know, you, I, I have and, and, you know, I've gotten, I have many trillions of dollars of capital lined up against me for whom that's not, that's not in their interest. So that shows the limits of what is possible and, and, and what is not. But, but what I would say is that we, you know, instead of thinking about slowing it down versus going at the maximum speed, are there ways that we can introduce safety, security measures, think about?

• 58:54Speaker 01

  Think about the economy in ways that either don't slow the technology down or only slow it down a little bit. Like if instead of 10% economic growth, we could have 9% economic growth and buy insurance against all of these risks. Like I, I think that's what the trade off actually looks like. And precisely because AI is a technology that has the potential for to go so quickly to solve so many problems. I see the greater risk as like, you know, it could, the thing could, the thing could overheat. Right. And, and so I basically want, I don't want to stop the reaction. I wanna, I wanna focus it. Um, that's, that's how I think about it.

• 59:31John Collison?

  You said, um, if we hit December 2025 and there's no AI law, I'll be really worried. How are you, uh, how are you feeling?

• 59:38Speaker 01

  There is actually something in California. There's a bill out SB 53. Um, of course, you know, last year we had the whole SB 1047 thing. You know, we had mixed feelings on SB 1047. There was initial version that, you know, I think was, was, was too aggressive. And, and when I, when I say that, what I mean is the technology is moving fast. And it's kind of unhelpful if you're too prescriptive about it. You know, it ends up actually not contributing to safety. And I was worried a little bit if something like this passes, it's like the tests that were prescribed to run will end up looking stupid. And then like all the people in the AI industry will be like, oh, this is what regulation for safety and security looks like. It's really stupid and they won't take it seriously. They'll kind of, you know, do everything they can to comply in letter and not in spirit.

• 1:00:22Speaker 01

  And so as, as a advocate of thoughtful regulation, I was actually a bit concerned about this. We, we offered some changes to the bill to a point where, where kind of we, we felt good about it. And we tried to make a compromise between kind of industry and the safety advocates. We didn't really succeed as, as you saw. But, but this year I think we're making a bill that is something more moderate. It's focused particularly on transparency of practices. Transparency of safety and security practices, which is something that Anthropic has been very forward about. Um, and, and that I think other companies are starting to do, but not all the companies do it. And there's no way to tell if folks are telling the truth about what they're revealing.

• 1:01:03John Collison?

  And California regulation is enough because all the companies have nexus here.

• 1:01:08Speaker 01

  Yeah, yeah, I mean, I think, you know, most of these bills are organized around, you know, doing business in California. Um, and so, you know, it would be difficult to to shut off.

• 1:01:16John Collison?

  People are very AI-pilled here.

• 1:01:17Speaker 01

  People are very AI-pilled here.

• 1:01:19John Collison?

  Yeah.

• 1:01:19Speaker 01

  Um, uh, so I, you know, we'll, we'll, we'll see what, we'll see what happens. I'm not sure what's gonna happen, but, but we've always had this approach that we kind of are in favor of, you know, guardrails, including legislative guardrails on the technology, but we recognize the need to be careful. Like, we don't wanna kill the go, we, we don't wanna kill the gold, the golden goose. We, we just wanna, wanna stop it from, you know, from, from overheating or running off the road, or, you know.

• 1:01:46John Collison?

  Yeah, you know, maybe something like modern bank regulation, For All People Complain is a good example, where there's an inherently very risky activity, you know, you're.

• 1:01:54Speaker 01

  Yeah, no, no, the dangers are, the dangers are pretty clear. I mean, you know, it's like the bank runs or not.

• 1:01:58John Collison?

  Right, but, but it all works pretty well in the, uh, in the, in the modern era, once we figure out the regulatory environment. Last question, what is your personal AI stack? How do you use AI differently to, uh, maybe other people in tech?

• 1:02:09Speaker 01

  Yeah, um, interesting. I, you know, I basically write a lot. Perhaps it's, I have too much pride in my, in my own, uh, in my own writing. I use Claude to generate, um, lots of ideas, you know, I kind of use it as, as, as research. But so far, I've done the writing myself. Claude is actually maybe closer than the other ones, but it's, it's still, it's still not there. Like, I'd be comfortable with it for business emails, but if I'm kind of like, writing an essay or something that I wanna really get right, it's not quite there yet, but maybe it will be in, in, you know, a year or so.

• 1:02:44John Collison?

  Yeah, very cool. Well, Dario, this is awesome. Thanks for coming by.

• 1:02:47Speaker 01

  Thank you for having me.