Pod Insights Odd Lots Anthropic's Co-Founder and Top Economist on Doing Research at the AI Frontier About this episode There’s a lot to unpack with AI right now — everything from its potential impacts on the labor market and society to more extreme questions about existential risk. Anthropic, which builds frontier models like Mythos, Fable, and Claude, is actively grappling with these issues, including whether governments should limit AI development. Just last week, the Trump administration forced Anthropic to block foreign access to its two leading models. In this episode, we speak with Jack Clark (co-founder and head of public benefit) and Peter McCrory (head economist) about how Anthropic approaches safety and economic risks. We talk about its preparations for recursive self-improvement, the engineers it's hiring now, and why Jack left Bloomberg to enter the early AI industry.
Read more:
Anthropic Lays Out Vision for How to Bolster AI Models’ Safety https://www.bloomberg.com/news/articles/2026-06-18/anthropic-lays-out-vision-for-how-to-bolster-ai-models-safety?utm_medium=referral&utm_source=podcast&utm_campaign=odd_lots&utm_content=article
Microsoft Makes Big AI Inroads in China by Selling OpenAI Models https://www.bloomberg.com/news/articles/2026-06-17/microsoft-s-china-ai-business-grows-on-openai-model-sales?utm_medium=referral&utm_source=podcast&utm_campaign=odd_lots&utm_content=article
Only Bloomberg - Business News, Stock Markets, Finance, Breaking & World News http://bloomberg.com/ subscribers can get the Odd Lots newsletter in their inbox each week, plus unlimited access to the site and app. Subscribe at bloomberg.com/subscriptions/oddlots https://www.bloomberg.com/subscriptions/oddlots?in_source=oddlotspodcast
Subscribe to the Odd Lots Newsletter http://bloomberg.com/subscriptions/oddlots
Join the conversation: discord.gg/oddlots https://discord.gg/oddlots
See omnystudio.com/listener https://omnystudio.com/listener for privacy information.
Transcript 13,678 words · 8 speakers
Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Tracy Alloway? There are some market stories where you want every detail. Joe and I have made quite a few podcasts on that basis. But sometimes you've only got 10 minutes and just want to know what's moving markets fast. That's the Barclays Brief podcast. Every week, experts from Barclays Markets and Research get you up to speed on what's happening and what to watch next. Concise, focused, and brief. Search Barclays Brief wherever you get your podcasts.
0:26 Speaker 01 Running a business means keeping pace with the latest tech. You've put a lot into building your enterprise software stack, and scrapping it to integrate AI is out of the question. That's why IBM technology works with all the apps, data, and models you need. It's another way IBM helps you build on the technology you rely on. Let's create smarter business. IBM. Visit ibm.com to learn more.
0:57 Speaker 02 Bloomberg Audio Studios. Podcasts, radio, news.
1:13 Joe Weisenthal? Hello and welcome to another episode of the Odd Lots Podcast. I'm Joe Weisenthal.
1:18 Tracy Alloway? And I'm Tracy Alloway.
1:19 Joe Weisenthal? Tracy, I don't know. I think our listeners like it, but a lot of our episodes are about AI these days. But to be fair to us, it's a pretty big topic.
1:27 Tracy Alloway? That's all anyone wants to talk about. Whenever we go to dinners with sources and things and people who are not even directly in the tech industry, you know, they might be in markets, they might be in policy and economics, all they want to talk about is AI. And then inevitably, the conversation veers into very sci-fi territory where we all start talking about the human extinction scenario.
1:46 Joe Weisenthal? Yeah.
1:47 Tracy Alloway? And that's just the norm nowadays.
1:48 Joe Weisenthal? I know, it's so weird. You know, we were in Hong Kong recently. And when we were in Hong Kong, this was before it was announced that there was a deal to open the Strait of Hormuz. And East Asia was considered to be like ground zero for where the effects would be felt of the oil and jet fuel crisis, et cetera. And we were at this dinner with business people, like they were not talking about that at all.
2:07 Tracy Alloway? No, they want to talk about the Terminator scenario.
2:09 Joe Weisenthal? They just want to talk about token consumption and all of these things. Like, here we are. It's like, wait, aren't you guys supposed to be like under all kinds of jet fuel stress? So this is our defense for thinking AI is a pretty big deal.
2:21 Tracy Alloway? For doing constant AI episodes? I think it's fair.
2:23 Joe Weisenthal? Yeah.
2:23 Tracy Alloway? I will also say, when we did the quiz in Hong Kong, we had a bunch of different teams with very creative names separated by tables.
2:30 Joe Weisenthal? Low value, human capital.
2:31 Tracy Alloway? That was a great one.
2:32 Joe Weisenthal? They won the turn, they won the quiz.
2:33 Tracy Alloway? They won, proving that there is value in human capital. But did you see that one of the tables was called Fable 13?
2:41 Joe Weisenthal? Well, there you go.
2:41 Tracy Alloway? Table 13, Fable 13, which was very topical at that moment.
2:45 Joe Weisenthal? Very topical. Well, we're recording this on June 17th, and of course, there's a lot in the news these days, but things move very fast in AI. Even if there weren't governmental controversies and all that stuff, you would have to mark the date in AI because of how fast breakthroughs happen. But, you know, as you said, like AI sort of feels like the most important thing than anything else. But that's a very conventional wisdom thing. It was not always conventional wisdom. And I have a DM. I know you're not supposed to share DMs from public, but I have a DM.
3:11 Tracy Alloway? You've got the receipts.
3:12 Joe Weisenthal? I have the receipts. August 2nd, 2016, and I DM'd a colleague. I said, "Did you leave Bloomberg?" He says, "Yes, I'll be announcing publicly in a bit. Take a couple of months to study AI properly, then leaving journalism to do something else, still connected to AI. Being our Google reporter was a great gig."
3:27 Tracy Alloway? Something still connected to AI is an understatement.
3:29 Joe Weisenthal? And then the final, "August 2nd, 2016. But AI is more important than anything else, so I felt best to sort of optimize for that above all else." And then I just said, "Well, good luck."
3:39 Tracy Alloway? This is someone who truly learned from their sources, unlike us, who remain in the podcasting industry.
3:44 Joe Weisenthal? That's right. So anyway, that person who would, that DM was a former Bloomberg reporter, Jack Clark, who is one of our guests today. He is the head of public benefit and co-founder of Anthropic 10 years later, and also Peter McCrory, head of economics at Anthropic. So two perfect guests to talk about all the things in AI these days. So Peter and Jack, thank you so much for coming on the podcast.
4:05 Speaker 04 Great to be back. I'm glad I optimized my life to study AI.
4:09 Joe Weisenthal? Yeah, well done. One of the calls of the century. So, um, let's actually start with that. Like, it's easy to say in 2026 that AI will be a big deal. You called your shot. You got it right. 2016. What did you see in August 2016 or presumably before? They're like, oh, you know what? This is the biggest story of our lives.
4:26 Speaker 04 So for two years, when I was reporting at Bloomberg, I wasted a lot of Mr. Bloomberg's printing by printing out archive papers about AI research. And what I started to do, a very Bloombergian thing, is I started to make graphs charting AI progress over time, measurements of things like computer vision, measurements of things like the skill with which AI agents were able to compete and play Atari games. And what I saw in these graphs was the beginning of an exponential, and it was everywhere. Like, if you looked at vision, or sound, or video, or game playing, you saw the same trend. And it became obvious to me that this was a general purpose technology that was right at the start. My one bone that I have to pick with Bloomberg, which I'm going to use my privilege to mention on air, I never
5:11 Speaker 04 got us to write a story saying Nvidia was being used in every single AI research paper. And I pitched it, and I failed to get it across the line before I left.
5:20 Tracy Alloway? Oh, man. I can just imagine you reading all these academic papers. Meanwhile, the editor is like, we need the BFW for this headline.
5:26 Speaker 04 It's not AMD. It's Nvidia. Like, this seems important.
5:29 Joe Weisenthal? Yeah.
5:30 Tracy Alloway? Okay. And Peter, I'm very interested in, you know, Anthropic, it's a company trying to make money, and yet it has this economics lab. What's the idea behind having an economics research body within a company that's developing this technology?
5:44 Speaker 05 So, I mean, I was late to the game in joining Anthropic. I joined just a year ago, but I had-
5:50 Joe Weisenthal? A year ago, people, well, whatever. We all know about how much the stock is a price of the year, but you're not, go on.
5:56 Speaker 05 I think what was very evident, so I'm an applied macroeconomist by training and have tried to understand various types of shocks throughout the economy. Part of what drew me to Anthropic was, it was evident to me last year that they cared very deeply about not just advancing the technology, but making sense of how it is set to reshape the labor market, its impact on productivity, on growth, and be willing to put evidence, data, and research out into the world that would be broadly beneficial and- Useful to society. And I thought, I want to be a part of building that economic research program and do what I can to provide tentative answers to the most pressing questions. We might not always get it right, but ideally, we're helping society make sense
6:41 Speaker 05 of the change.
6:42 Joe Weisenthal? The capabilities of the models on all kinds of things are extraordinary. I mean, just mind blowing, every coding, copyright, every, all kinds of things. Actually, why in June 2026, does life still feel maybe as normal as it does from an economic perspective?
6:58 Speaker 05 This is a great question and one that I've been wrestling with. I think there are a number of reasons why you might think that the impact has not yet materialized. One, the technology can advance, but it also then needs to diffuse throughout the economy.
7:13 Joe Weisenthal? Yeah.
7:13 Speaker 05 And there can be bottlenecks from moving from capabilities to actual deployment. We see that with our enterprise customers. So if you want to automate biological research or some other very complicated financial modeling task, you need a lot of contextual information available to the model.
7:29 Joe Weisenthal? Yeah.
7:30 Speaker 05 If you don't have that contextual information, the capabilities alone won't necessarily drive the impact. It also takes time for people to just start using the tools. And so we're still in the somewhat of the early stages there. Two places that I would be looking to see an impact, one is in terms of productivity growth. We've done some research that points in the direction that this should be large and consequential. Labor productivity growth has been strong throughout the pandemic and has been sustained so far.
7:59 Joe Weisenthal? Like modestly so. We're not talking about like, you know, revolutionary.
8:03 Speaker 05 It's not, we're, we're, yeah, but you know, to, to get on an inflection, you need to at least move a little bit. I think maybe you're seeing some signs there. On the labor market, though, the labor market is in a reasonably healthy spot. And I think it might be because it's primarily at so far, a labor augmenting skill bias technology, not yet the full sort of general purpose substitute for all of cognitive labor, although perhaps that's the trajectory that we're on.
8:29 Speaker 04 You know, for the size of AI and its capabilities, I was talking to Peter about this, and he did point out the economy, very big. Yeah. So it still takes a lot to move it. Yeah. I do think strange things are starting to happen, at least inside the company. We published research from the Anthropic Institute recently on this topic called recursive self-improvement, where it was inspired by me going on paternity leave in November of last year and coming back in February and the entire company felt and worked differently. And I assumed it was because models had got better. And when we looked at the data, what you saw was in 2026, engineers at Anthropic are writing about eight times the amount of code than they did in 2021 through to 2024.
9:13 Speaker 04 And the line started last year with things like Opus 4.5 and Opus 4.6, then it really got going this year. And I have colleagues now who don't program at all anymore. They just instruct many, many code code agents to run around and do their work for them. I can't reconcile that with the world staying normal for long, but it's going to take a while for that to diffuse into the world and change it.
9:35 Tracy Alloway? Yeah, and we'll talk more about recursive self-improvement. So this is when models basically improve on themselves, right? So in terms of the awkwardness of the current moment or the weirdness of the current moment, you've talked about basically living through the singularity and how strange it is. And you've also described yourself as a techno pessimist before. How do you square that with working at Anthropic, which is making some of these weird and potentially dangerous things actually happen?
10:05 Speaker 04 So by technological pessimist, I mean, I thought the technology would keep getting better, but I didn't think it would get better in the like maximalist sense that some of my colleagues did. I didn't think that we would have, say, functionally automated all of coding right now. I find that actually like quite surprising. But basically over the last few years, and I worked at OpenAI before Anthropic, I was just hit repeatedly over the head with what computer scientist Richard Sutton calls the bitter lesson. And the bitter lesson is this concept that the more compute and resources we dump into these relatively generic neural networks, the smarter they get and the more emergent properties they have. And your specialized system or your ability to be pessimistic about future AI programs. Losers versus just scaling computing, scaling systems.
10:50 Joe Weisenthal? This seems to have implications for the labor market, right? Because and I think a good example, the bitter lesson is probably the history of AI chess. Right? Where at one point they had grandmasters come in and teach the models how to play chess and et cetera, and try to encode their wisdom. And it turned out in the end that the best way to get a chess engine really good is to just teach the model, tell the model the rules of chess and say, go off and play a billion games and find optimal chess. Without any human insight, the grandmasters were not necessary for that process at all. Right? And so this would imply to me like have significant implications for the labor market.
11:27 Speaker 05 Yeah, I tend to think about this in sort of three aspects of what composes a job. One is you need to decide what to do and direct and delegate. You need to then do the actual implementation of the work. And then you need to sort of evaluate or at least set up systems that can evaluate. At least from my perspective as an economist, this bitter lesson is materializing in terms of very rapid advances in the implementation work of what an economist does. Downloading data, running regressions, building models, solving them using sort of contemporary solution techniques, numerical methods. I definitely felt that personally with Opus 4.5, where I was for the first time able to just delegate a very complex task.
12:11 Speaker 05 I had this very specific research question, trying to understand the cyclicality of hiring across different occupations and how that relates to occupational exposure. That's a mouthful. I gave that task to Claude and Claude was able to just iterate on it. And I could redirect Claude in the same way that you might redirect a grad student. And the big question that I have in mind is, you know, at what point do the boundaries at the direction setting stage, the research taste, you might call it, and will the models become sufficiently reliable?
12:41 Joe Weisenthal? If I could just get in here, you know, I just read the recent biography of the DeepMind founder.
12:47 Tracy Alloway? This is the Sebastian Mallory.
12:48 Joe Weisenthal? Yeah, like, is there going to be a point where it's like, okay, you have some intuitions, right? About like what good economics research is and often our. Intuitions are formed because we tell stories and stuff like that. But is there going to be a point where you think like your intuitions will be unhelpful? And that because that's sort of what I took away from the go experience. That the model got better. Once they stripped it of the human games and the human bias and that actually like. The human intuition that sort of helps us understand, oh, labor market rising creates inflationary pressure. These stories that are very sort of intuitible. End up impairing the model. Do you see that happening in say economics where it's like some of these stories that we tell forever? They're not actually very helpful for an optimal. Economy understanding model.
13:33 Speaker 05 I expect that these models will soon have better intuitions about how to do good economic research. And that there is this big question of like. At what point will we be able to fully automate social science? Research. We've done some work on this to try to understand how coding agents are beginning to automate social science research. But I don't think we're quite there yet and. I don't know that that'll be an exciting. Time for learning about the world. You know what that means for my job, I'm sort of less entirely clear.
14:01 Speaker 04 Yeah, I I think this is the big wild card in future AI progress. If AI progress continues today, we are likely to get technology that. Will be able to do basically everything. But we will need people who have good instincts, good intuitions and good ideas to basically set the direction. And we see this today in a lot of a lot of our own research where you need. Say an AI safety researcher to give. Nine Claude agents for different research areas to go and pursue. And then it's very effective. If that researcher doesn't give them the research directions. They pursue relatively formulaic research directions and you have entropy collapse. You end up with just like boring research, it doesn't move the ball forward. At what point will AI systems generate? Like heterodox insights and genuine creativity?
14:45 Speaker 04 We can't really measure for that today. But what we have of a symptoms of it starting in. Experts like Peter, experts like colleagues in the. Fields of biology or mathematics or physics outside of anthropic are all starting to be accelerated. By AI, you know, Terry Tao, probably one of the most famous living mathematicians. Co-creates math now with AI systems. And so that that says to me that these things have got, they're they're tickling the dragon's tail of like creativity here.
15:15 Speaker 05 And you know, we just put out a report yesterday on Claude code usage. And one of the things that we're trying to understand is like, what are the returns to expertise? And how does that interact with the usage of sort of automated coding agents? And we find that domain expertise, like if you're an accountant who understands some of the edge cases and reconciliation, that that domain expertise, controlling for a whole host of factors about the type of work, the estimated monetary value of the task. It has an amplifying effect. It has an amplifying effect. So this looks like at present as sort of a skill biased, expertise enhancing impact. But I think this is the key question is at what point and to what extent will this change?
15:55 Tracy Alloway? Well, related to this, you know, Jack, when you describe coming back from paternity leave and seeing how much things had changed at Anthropic. I know we're not officially at recursive self improvement point, but it sounds like we're semi there. So my question is like, I get that at the moment you have engineers who are reviewing all the code that the AI is producing and they're thinking about it and managing it in some way. But you can easily imagine a future where just the sheer quantity of code overwhelms human expertise. Maybe the quality starts outstripping what human engineers are capable of understanding. How do you manage that?
16:33 Speaker 04 Yeah, so there's two ways of thinking about recursive self improvement. One is what happens when AI organizations start to see a compounding return from their AI systems? Basically, their own production function improves because of the tools they've built. That's clearly happening now. Then the second is what happens if an AI system can just build itself entirely autonomously given compute, which hasn't happened? What I see inside Anthropic is I think what we'll see in the broader economy, which is we are figuring out how to verify and validate and basically price for risk of an expanding cloud of automated systems, which we're sitting on top of. So now we produce way more code. Well, we broke our continuous integration system for integrating code into the. Code base because we started pushing eight times more code for it than before.
17:18 Speaker 04 So all of our human engineers worked on unbreaking CI. And so I think that CI? Continuous integration. Thank you. You don't need to know what it is. It's just a thing that helps you push the code into
17:29 Tracy Alloway? We like to know stuff on this show. We like to learn.
17:31 Speaker 04 But there's a lesson in that, right? We are going to speed up things in the economy. We're going to speed up the way that we produce stuff. And then we're going to find, you know, the like, the weak links or the hot paths that break. And we as people are going to move to sorting those out. And then the cycle starts again. And we're kind of sitting on this expanding cloud of automated actions.
18:06 Tracy Alloway? There are some market stories where you want every detail. Joe and I have made quite a few podcasts on that basis. But sometimes you've only got 10 minutes and just want to know what's moving markets fast. That's the Barclays Brief podcast. Every week, experts from Barclays Markets and Research get you up to speed on what's happening and what to watch next. Concise, focused, and brief. Search Barclays Brief wherever you get your podcasts.
18:33 Speaker 02 This is the Bloomberg Tech Minute brought to you by ChatGPT. I'm Carol Muster. Getting hired at SpaceX requires exceptional ability and may include having Elon Musk review a candidate's resume. So writes Bloomberg Business Week's Lauren Grush. She reports that the company culture is defined by Musk and job candidates must understand that a commitment to SpaceX means adhering to this culture and exhibiting loyalty to Elon Musk. To get hired, candidates must demonstrate their achievements from past jobs. And hands-on experience matters more to recruiters than advanced degrees, with the company looking for people with raw, innate intelligence. A spokesperson for SpaceX didn't respond to a request for comment on its recruiting and hiring practices.
19:16 Speaker 02 But according to recruiters, advisors, and those who've interviewed for jobs there, it's the space exploration dream that draws people to the company. And Musk invokes That dream regularly to remind workers why they're there. Prospective hires are told that the job will require stamina, including in job ads that candidates quote, "must be willing to work extended hours and weekends as needed." One major perk for all that hard work? Employees' access to equity, either the right to buy shares or payment in the form of restricted stock units that vest over time. That's the Bloomberg Tech Minute brought to you by ChatGPT. Get started at chatgpt.com.
19:56 Joe Weisenthal? Since we're talking about like really like feeling like we're staring at the horizon of extremely strong AI, or maybe we'll get there, or maybe the AI builds itself. Might be a good time to ask a fable question or methos question. At this point, we're recording this June 7th. We don't know when it's going to be available for Americans, let alone the rest of the world. Does Anthropic have a clear idea of what the administration's security concerns are and what it will take to resolve them?
20:24 Speaker 04 Well, obviously, live discussion. I can't get into too many specifics. We're in daily discussions with the government about this. The broad thing I'd say is for many years, we've anticipated a point where AI systems would have national security properties. These national security properties are intertwined with their economically valuable properties. How you manage that as a policy question is basically novel territory. Typically, these things are decoupled. You're like, hey, I built a jet engine over here, which can go into civilian aircraft, and I built a missile over here, and you treat them differently. It's odd if you smoosh these things together. Where we'll get to, I'm confident, is what's a system for assessing the properties of AI systems, including national security components?
21:07 Speaker 04 And then what is a system for either squelching the national security capabilities from coming to general proliferation, like bioweapons or cyber weapons? And are there ways to do things like know your customer or deployments where you let large firms, like, say, drug developers, access the most powerful biomodels without accidentally proliferating risks? That's the shape of, I think, where we'll end up. And what we're doing right now, we and other companies and the administration are basically Tackling this problem in real time. It's initially going to be messy, but we're going to end up with a system on the other side.
21:40 Joe Weisenthal? Well, let me just ask you, you know, this specific incident, and there are probably more in the future because everyone's just figuring this out. When I look at the AI landscape, I sort of think of OpenAI as being part of the All In Podcast, A16Z, David Sacks White House thing. And I know from my friends in the media, many of whom are liberal Democrats, that I sort of feel like Anthropic is the more like lib coded of the major models. Do you feel there's any either politics or partisan politics going on as part of Anthropic being harassed or singled out now multiple times?
22:23 Speaker 04 Anthropic's philosophy and what I do, and I lead something called the Anthropic Institute, which helps us produce better data for the world around things like recursive self improvement, the economics work, cyber risks, is we tell the whole story about what's going on. Typically, I think the technology industry has told only optimistic stories about what it's building. And what we saw with social media is that does not work. Actually, eventually, when when you're doing something that changes the entire world, which AI is certainly doing and social media certainly did, it's not going to be a wholly optimistic story. There will be negatives as well. We've always sought to just tell the truth about what we see in front of us. And I think sometimes that can differentiate us a bit to others.
23:05 Speaker 04 But the important thing is we tell the truth and things end up coming true.
23:09 Joe Weisenthal? You don't think that there's like a partisan element here where you guys aren't on the team or didn't contribute enough to the ballroom or whatever?
23:17 Speaker 04 I can't really speak to that. I'm, you know, I'm not those people. I'm I'm anthropic. What I can say is the AI systems create their own evidence. Years ago, it seemed very odd to speculate about the cyber properties of AI systems. Well, they've arrived and now we're working on them. Years ago, it was odd to speculate about the bioweapon properties of AI systems. Well, recently, Sam Altman, Demis Hassabis and Dario Amade of OpenAI, Anthropic and DeepMind all signed a letter saying we need to do better screening of gene synthesis to prevent AI manufactured bioweapons. But truth wins out.
23:50 Joe Weisenthal? Okay.
23:51 Tracy Alloway? I want to go back to something you said. You mentioned potential KYC requirements. When I hear KYC, I think about the finance industry and I think about systemically important institutions and the stress tests and the framework around that. Is that the right analogy to use for, I guess, ideal AI regulation in your mind, rather than, I guess, just simple export controls? Should we be heading towards something that looks a little bit more like what we do for the banking system?
24:16 Speaker 04 We need something that's more subtle and more technocratic from what we have today. I don't know if it'll be exactly like the banking system. It'll probably take some ideas from that. It'll take some ideas from what the US government and others are doing today with just testing AI systems for their properties. And it's almost certainly going to have a flavor of what Peter and I work on and the Anthropic Institute broadly of generating data about these systems as they're deployed in the world. Because it's one thing to, you know, test out the thing before it comes out of a factory. It's another to observe the effects it's having in the world and then to be able to make judgments about whether those effects are good or not.
24:50 Joe Weisenthal? Would you support, you know, in the, let's stick with the financial analogy, companies that are public at least are required to have third-party auditors sign off on them. And there's talk, you know, when they submit their 10-Qs, et cetera. Companies that issue debt are required to have ratings agencies or frequently have ratings agencies rate their debt. Would you support embedding in law the requirement that certain, what would be the equivalent of a Moody's or a Deloitte, you know, a third-party research lab sign off on the release of new models?
25:22 Speaker 04 We've proposed something like this recently, a policy proposal that we laid out, which includes saying we need to have third-party testing for some of these national security and other properties, because clearly that's like a sensible way that you validate a lot of things.
25:36 Tracy Alloway? So just more broadly, returning to this idea of, you know, measuring the actual impact of AI. One thing I find really interesting is that if you actually look at a lot of our traditional AI, or I should say, I'm AI-brained already. If you look at some of our traditional economic statistics, a lot of the AI impact doesn't actually show up just yet. Again, we're in the early stages, but you would expect if we're talking about the AI economy growing something like 2,000% or 3,000%, I think I've seen that number.
26:06 Speaker 05 That's from Anton Kornak and McKelvey's paper a few weeks ago.
26:11 Tracy Alloway? You would expect that to have more of an impact on nominal GDP, and yet it's not really showing up that much. Do you think the way we measure the economy needs to be changed in some way in light of what's happening with this new technology?
26:25 Speaker 05 Yeah, so I think this is exactly the right premise is kind of where we began the conversation, which is we're maybe at the point where we should be able to see some discernible impact on the macro economy. Unfortunately, the arrival of this world historical technology is against the backdrop of sort of unusually elevated macroeconomic volatility, post pandemic, monetary policy, et cetera. And so it like makes it very hard to disentangle all of the different factors, you know, what's the counterfactual, you know, labor productivity growth is maybe not as strong as you might not otherwise expect, but maybe it's stronger than it is in a counterfactual sense. And so one way that we've tried to tackle this question is by looking at how Claude
27:10 Speaker 05 is being used on our platform, using our privacy preserving techniques to estimate the time savings associated with each of the activities that people use Claude for. So compiling information from reports to put together a research brief would take you a few days, maybe now Claude does it in a few minutes. Evaluating diagnostic images is something that skilled professionals do very rapidly. So there isn't in principle much time savings. You can add up all of those numbers and using standard macro growth accounting techniques, Halton's theorem for the economists and the audience, and you get a number of that points in the direction of labor productivity growth increasing by 1.8 percentage points each year over the next decade.
27:52 Speaker 05 If that's how long it takes current usage patterns and current model capabilities to diffuse throughout the economy, that's a very large number. It's a rough doubling of recent run rates. And what I think you might be able to see in the data, and we haven't put anything out on this yet, is I think some of the strength in recent labor productivity growth is actually concentrating in exactly the sectors of the economy that would be consistent with both what we see in our data as well as also what you see in the business trend and outlook survey. For example? So the information sector has high rates of adoption. I can't recall if that's in particular, one of the sectors that I have in mind, you know, it's a while since I looked at that scatter plot, but you can look at the sort
28:34 Speaker 05 of sub industries by the Census Bureau's Business Trend and Outlook Survey and rates of adoption are in sectors or parts of the economy where
28:44 Speaker 05 Controlling for pre-pandemic trajectory of labor productivity growth in those sectors, even some of the strength in the early years of the recovery, still see some suggestive evidence. I think there's a lot of uncertainty here. Trying to get a real-time signal on productivity is maybe the hardest thing to do. You're subject to macroeconomic GDP revisions. TFP growth is actually sending the opposite signal. And if you control for capacity utilization, TFP growth is arguably even lower. So I say this as like, this is suggestive evidence that maybe we're beginning to see an impact there, but not so much in the labor market.
29:22 Tracy Alloway? Well, now I have to ask, when you gather this kind of research, and it all sounds super interesting, but if you have data, for instance, that shows that, okay, the IT sector is getting productivity gains from using Claude, or I don't know, maybe something unexpected, like the warehousing industry is using a bunch of AI. What does Anthropic actually do with this data? Does it somehow feed back to your engineers who are developing frontier models? Do they do anything differently?
29:48 Speaker 04 I think some of it cues us on areas where maybe the technology isn't being used because it's very weak. We just haven't made it particularly good for these use cases. Or in areas where it's being used at large scale, it's usually a suggestion of keep making it good there. But the actual economic measurement data doesn't really get fed back directly in, but it's a very useful clue. We think it's more important, though, to basically communicate this outwardly to policymakers, journalists and others, because our assumption is that at some point, we go through some phase change, similar to how capabilities of AI occasionally jump forward in a really dramatic way, where you might see sudden and rapid diffusion as a consequence of capability expansion in the AI systems. So we're getting practice in of looking at this kind of data.
30:33 Speaker 04 My expectation is that in a year or two years, I'm going up to some policymaker, and I'm pointing them to the part of the graph that now gets very steep in some chunk of the economy.
30:42 Tracy Alloway? And hoping that they'll do something about it.
30:44 Speaker 04 Yeah.
30:45 Speaker 05 I think there is another part of what we're trying to do at the institute, which we lay out in the sort of research agenda for the anthropic institute, which is trying to understand the impact of our decisions, which is a typical thing that economists will do at tech companies. But we have a public benefit mandate. So we're trying to understand the impact of our decisions on these broader societal and economic outcomes that we care about. And then using that to inform some of the decisions that we actually make.
31:12 Speaker 04 A goal that Peter and I have, and we've talked about internally, is if we get really good at measuring things like a productivity multiplier of our technology, then I would hope to use that to guide some of, say, the early access programs we do for powerful models, where if you see you get some tremendous multiplier in a specific part of science, use that to redirect some of your inference compute budget to that sector. And then you can run an experiment and say, were we able to make this thing go much faster? I think that could be like an amazing tool to unlock for the world. And it's one that you could generalize across companies and you could generalize it into policy. So instead of, say, NSF doing standard grant funding, it could be, should we just point for really powerful AI systems at this chunk of science and make it go faster? I think that's a world that will come within reach soon.
31:57 Joe Weisenthal? Let's talk about this public benefit mission a little bit more. We've been talking about ways this could change the economy. How much do you see your job is basically strong AI is coming?
32:09 Speaker 04 Yeah.
32:10 Joe Weisenthal? It's coming whether we like it or not. And it's important to be, you want to be there as like one of the shepherds, understanding which direction it goes in, the data that we should see to see what's emerging. Like, how much is that somewhat your role?
32:24 Speaker 04 Yeah, but look, our guiding principle is that this technology is being built by a variety of companies and a variety of countries. But technology by default is unknown. It will be known to the companies. It will not be broadly understood or known by others. They'll just be able to play with the models. Every bit of data we can create, and especially systemically sharing data like the economic index or what we've started to do on recursive self-improvement, gives the world a better chance to sort of prepare for this technology.
32:50 Joe Weisenthal? Yeah.
32:50 Speaker 04 And both plan for its success, like what I talked about with science. We could be in About driving science forward and also be warned about risks like the cyber capabilities.
33:00 Joe Weisenthal? Yeah.
33:00 Speaker 04 We've talked about.
33:01 Joe Weisenthal? Well, so it's like that. That makes a lot of sense. The company is going to see it before the world. And Heskin is like, okay, this is important to share. This is not important to share, which which brings me to another question. You know, I know, like, people in the AI research world, done some reporting on the sort of scene in SF. You know, like, when I think about a lot of the people who are like at the very cutting edge of AI ethics, AI technology, et cetera. I know a lot of people who are, how shall I put this? They have esoteric moral interests, shrimp rights, unusual attitudes about experimental drug use. We know about the Chinese peptide scene in San Francisco, et cetera.
33:41 Joe Weisenthal? And as a family podcast, I would say a certain like perhaps deviant or different view on sort of bourgeois, even sexual values. And we know about the sort of attitudes towards monogamy, et cetera, within the San Francisco research scene.
33:55 Tracy Alloway? Joe, there's going to be a protest against all lots in San Francisco with people holding signs saying not all engineers.
34:01 Joe Weisenthal? Yeah, not all engineers. I understand that. But when we think about like, okay, these are the people who are going to see it first. Should we feel comfortable that this is a group of individuals, the cohort of the most advanced AI researchers, whose intuitions about what's important to communicate to the public are actually in line with the public's interests, given how unrepresentative they are of what I would call the American public?
34:24 Speaker 04 Yes, as a as an Englishman, it fills me with such joy to be asked about sex on.
34:28 Joe Weisenthal? Yeah, I know, I know.
34:30 Joe Weisenthal? I'm asking you to your view, your insight into the cohort of the most advanced research.
34:34 Speaker 04 We're we're explorers, people that are explorers, and this is so true in San Francisco, end up being like that. There's a broad range of types of people, and sometimes they're really, really different or they're really, really eccentric. And they're brilliant and they're lovable and everything else.
34:48 Joe Weisenthal? Yeah, sure. Love them.
34:49 Speaker 04 You don't want only that class of people to be the ones calling the shots on what we know about this technology.
34:56 Joe Weisenthal? Yeah.
34:56 Speaker 04 The whole purpose of what we're doing is we're trying to set up systems by which you could eventually mandate through policy that companies share information, you know, anthropic as long. Pushed for transparency legislation in various states around America that gets companies like us to report out the sorts of tests we're running on our systems and share it publicly. My whole mindset is the public and policymakers and economists, everyone deserve the ability to advocate for what information should come out of a frontier and should be forced out of a frontier eventually by law. Like, that is how you solve this issue.
35:28 Tracy Alloway? Do you hire more normies?
35:30 Joe Weisenthal? Yeah.
35:31 Tracy Alloway? And anthropists?
35:32 Speaker 04 Me personally.
35:33 Joe Weisenthal? Yeah, like, is that an important thing? Like, hiring people that don't all share their certain, like, you know, in-group ways of seeing the world?
35:39 Speaker 04 So, you know, the Anthropic Institute, we have teams of economists, of social scientists, of what you might think of as weapons experts, our frontier red team, things that go bump in the night, lawyers, and increasingly, other types of people. The goal is to build what I think of as a highly ideologically diverse, like, research function within the organization, but is partly advocating sort of on behalf of the world for different forms of study that we might do. So Anthropic generally hires a really broad range of people, but the Institute specifically is trying to compose a very broad set of interdisciplinary experts for this exact reason.
36:31 Speaker 06 This is the Bloomberg Tech Minute, brought to you by ChatGPT. I'm Carol Muster. Getting hired at SpaceX requires exceptional ability and may include having Elon Musk review a candidate's resume. So writes Bloomberg Businessweek's Lauren Grush. She reports that the company culture is defined by Musk, and job candidates must understand that a commitment to SpaceX means adhering to this culture and exhibiting loyalty to Elon Musk. To get hired, candidates must demonstrate their achievements from past jobs, and hands-on experience matters more to recruiters than advanced degrees, with the company looking for people with raw, innate intelligence. A spokesperson for SpaceX didn't respond to a request for comment on its recruiting and hiring practices.
37:14 Speaker 06 But according to recruiters, advisors and those who've interviewed for jobs there, it's the space exploration dream that draws people to the company. And Musk invokes that dream regularly to remind workers why they're there. Prospective hires are told that the job will require stamina, including in job ads that candidates quote, "must be willing to work extended hours and weekends as needed." One major perk for all that hard work? Employees' access to equity, either the right to buy shares or payment in the form of restricted stock units that vest over time. That's the Bloomberg Tech Minute, brought to you by ChatGPT. Get started at chatgpt.com.
37:54 Tracy Alloway? Let me ask a slightly different question on hiring.
37:58 Speaker 04 Yeah.
37:58 Tracy Alloway? I guess a two-part question. So first of all, we get a lot of executives on the show. We've been asking all of them if they've changed their hiring process, if they've changed the questions they ask potential employees at those initial stages of job applications because of AI. And then secondly, what are you seeing within your own ranks at the company? And then, Peter, I'm sure you could talk about this more broadly, in terms of who's most in demand at the moment? Because the conventional wisdom right now is that if you're a younger employee with less experience, a lot of the stuff that you would be doing can now be automated through AI.
38:34 Speaker 04 So there's two trends showing up. One, I have a new team called the Rule of Law and AI. Our plan was to initially hire a bunch of engineers and then a bunch of legal experts and scholars. Instead, we're just hiring the legal experts and scholars because Claude is good enough at doing all of the engineering, but they can actually just like feed themselves using Claude in terms of the engineering resources. So that's a change in hiring. It means I'm hiring more interdisciplinary people earlier than I would have before. We are also seeing the emergence of what I think of as a barbell hiring pattern inside Anthropic, where there is a tremendous return on experience. So we are hiring more senior people than we did in the past because their intuitions and their ideas for what to pursue
39:18 Speaker 04 are like massively compounded by AI systems. We're also, when we look at very early people, are often hired Hiring people who are now like AI native and know how to use the tools and are well well versed in it. So we're seeing that change as well.
39:30 Tracy Alloway? Are there a decent amount of, I guess, AI natives now, people who have grown up with the technology?
39:34 Speaker 04 People who grew up from GPT two in two thousand nineteen.
39:37 Tracy Alloway? My perception of time is so warped.
39:39 Speaker 04 I I found this chilling as well, you know, as someone in their thirties, what do you realize? But I think that the trends I see, I do think that there's this question of how you have as much early career hiring in the future as you did in the past. I think one of the only areas where there is slightly suggestive data is that something might be going on with early career hiring. And it it kind of intuitively feels right to all of us for that we might be observing that effect. And when I look at hiring patterns in Anthropic, we're still hiring young people, but some teams are hiring slightly fewer of them than before and hiring more experienced people.
40:12 Speaker 05 Yeah, so I'll briefly say something about how we've shifted some of our hiring practices, like concretely.
40:17 Joe Weisenthal? Okay.
40:18 Speaker 05 I think before Claude Code, you might ask an economist to do some of the data work in an assessment kind of live. Like, download the data, run the regressions, do the analysis by hand. And then you might eventually let them use AI to do all of that work. But we've needed to increasingly shift our strategy of evaluation away from can you implement the work even with AI? To do you know how to delegate and direct the the model in a somewhat messy environment? And can you evaluate the quality of the work, maybe by like looking at a PR review?
40:53 Joe Weisenthal? Actually, could you talk a little bit more about what that looks like specifically in the econ finance? You know, there are listeners probably thinking about, okay, what is, I want to level up in my AI use. So I'm not just asking like, what GDP, whatever. What does that look, what does that actually mean for for an economist? And you used to be at a bank.
41:09 Speaker 05 Yeah.
41:09 Joe Weisenthal? Uh, so for a financial economist, an economist, someone, well, in this world, what is like the most advanced form of usage of AI actually look like?
41:18 Speaker 05 Well, I don't I don't know if I'll give the example, the most advanced form of usage, but I'll give an anecdote of my experience using Claude. Where I wanted to run this cross state regression. I can't remember exactly what it was. And I wanted to do it a pooled cross sectional regression. So looking at what happened in twenty twenty four, twenty twenty three, and going all the way back to pre pandemic. I remember asking Claude to go out and download the data from the Census Bureau, from the Bureau of Labor Statistics, et cetera. And there was this very unexpected quirk where the model couldn't access data from before 2019 and just would not surface that mistake. Yeah. And I would ask it multiple times, like, no, like, don't hard code numbers because it sort of had this unexpected failure mode where it said, oh, I know what those numbers
42:03 Speaker 05 were. And it just like, from sort of training data populated the data set. And you might not always be attuned unless you're sort of, you have this tacit knowledge about like, does it pass a sniff test when you run the analysis? And then you like dig into what the model actually does and it has failed in sort of unexpected or unusual ways. And so that's like the type of assessment that we've built. Can you be attentive to the very specific decisions that need to be made along the way that are very consequential for the validity of veracity of the results that you find?
42:37 Speaker 04 Yeah. A colleague did an offsite presentation last year, which said, I have locked the doors and we are reading transcripts. And their point was, we just need to read more of the raw data and develop that culture where if AI systems are doing increasingly large amounts of the work, you need to have a culture of being competent at spot checking their work and reading their reasoning because occasionally stuff like this happens.
42:59 Tracy Alloway? And then Peter, in the broader data that you're looking at, are you seeing the same sort of barbell effect in terms of employment that Jack described?
43:06 Speaker 05 Yeah, so I think what, again, what makes it really challenging is we've had the largest non-recessionary labor market slowdown on record, that, you know, it's very hard for young people to graduate into a labor market that doesn't have sufficient churn or opportunity for them to get a foothold. But one of the things that we did see in this report from March was that young workers in these high AI exposed roles where Claude is being used to automate specific tasks have had somewhat weaker job finding rates. But it's suggested.
43:36 Joe Weisenthal? Part of the confounders was the boom in hiring in 2021 in these exact same areas.
43:40 Speaker 05 Exactly. And there's a recent paper about the rise of remote work maybe being sort of the actual cause of this type of fact. Another team at the Anthropic Institute, Societal Impacts, recently ran this very large scale qualitative survey, 81,000 people around the world, asking them questions about hopes and fears that they have with respect to AI. Unsurprisingly, concerns about the impact on the labor market and on the economy rose to the surface. My team dug into those data a little bit more to try to answer some of these specific questions. And what you see is that young workers at least express concern about job loss at twice the rate as do more senior workers. And fears about job loss
44:21 Speaker 05 more broadly are more elevated for workers who are in these roles that we identify as being most exposed to displacement effects from AI. So there's a bit of a gap between perception and maybe what you see in the hard data. But you know, that was something that was true even in recent years on other dimensions. So it's an important thing to pay attention to.
44:40 Tracy Alloway? So we've been talking about the labor market. And one other thing I'm interested in is the impact of AI on, I guess, corporates themselves. So if we think about certainly America's corporate landscape in recent years, it feels like the big basically get bigger, right? There's economies of scale. They have a bunch of money that they can use to actually buy some of this new.
45:01 Joe Weisenthal? Lots of data internally.
45:02 Tracy Alloway? Exactly, exactly. So would you expect AI to, I guess, intensify that trend of the big getting bigger? Or would you expect to perhaps have a leveling effect where people have this new tool that they can use to, you know, set up a new company?
45:17 Speaker 04 I'm curious what Peter's take is, but I think that something, a helpful analogy here is the invention of electricity, where electricity arrived and existing factories put light bulbs in and other things. But it was a new generation of factories that were built around the assumption that electricity existed, but really grew and did transformative things in the economy. What I see now, when we look at large enterprises, is they can get a lot of utility out of Claude because of their data, because they can get a multiplier effect at scale. But it takes huge amounts of conviction to basically bash through all of the bureaucracy, you know, used to work at Bloomberg, implementing new technology at Bloomberg, challenging.
45:56 Joe Weisenthal? No comment, no comment, no comment.
45:59 Speaker 04 I can comment about it. Same is true of any large organization. Young organizations are building themselves around AI at the center. And these organizations are moving really, really quickly because they just, they have a speed advantage from building on the assumption that this new form of electricity was going to be integral to their business.
46:17 Speaker 05 Yeah, so I, I think the tension that you express is exactly the one that I don't have a strong handle on at the moment. One thing that we do see in our data is when businesses do embed Claude capabilities in automated ways through the API. As I mentioned before, these very complex tasks rely on disproportionately more contextual information than very basic document synthesis and summarization. What that points in the direction of are the complementary investments that large businesses need to make to centralize, codify and make available the data that does exist somewhere within the organization. But for historic and technical reasons, maybe even regulatory reasons, it's behind a firewall of some form or another.
47:01 Speaker 05 There's also like sort of organizational workflow changes that likely need to be made. Some of the most crucial information that's needed for some types of cognitive work is tacit knowledge that exists in your colleague's mind. And unless you have a process that elicits that information, that workers feel sort of incentivized to share that information and kind of trust the system. The capabilities alone might not necessarily generate that productivity. And so whether or not big firms end up restructuring themselves quickly enough, or whether this materializes through the process of creative destruction, I think the jury is still a bit out.
47:37 Joe Weisenthal? Yeah, I brought this up recently with the David Solomon, the Goldman CEO. And I started to wonder like this sort of like internal alignment question of like the big rainmakers, do they have an incentive essentially for information hoarding and not sharing with the company? That might be their only thing they keep them employed.
47:53 Speaker 04 And when I talk to customers, I say it's, don't think of it like you're buying a technology. Think of it maybe, but you're now employing thousands of people that are functionally like the chief of staff to the CEO and may need the same access to data the chief of staff would have. This is completely counterintuitive and it is not how technology is typically bought or sold.
48:27 Speaker 06 This is the Bloomberg Tech Minute brought to you by ChatGPT. I'm Carol Muster. Getting hired at SpaceX requires exceptional ability and may include having Elon Musk review a candidate's resume. So writes Bloomberg Business Week's Lauren Grush. She reports that the company culture is defined by Musk and job candidates must understand that a commitment to SpaceX means adhering to this culture and exhibiting loyalty to Elon Musk. To get hired, candidates must demonstrate their achievements from past jobs and hands-on experience matters more to recruiters than advanced degrees, with the company looking for people with raw innate intelligence. A spokesperson for SpaceX didn't respond to a request for comment on its recruiting
49:09 Speaker 06 and hiring practices, but according to recruiters, advisors, and those who've interviewed for jobs there, it's the space exploration dream that draws people to the company, and Musk invokes that dream regularly to remind workers why they're there. Prospective hires are told that the job will require stamina, including in job ads that candidates, quote, "must be willing to work extended hours and weekends as needed." One major perk for all that hard work? Employees' access to equity, either the right to buy shares or payment in the form of restricted stock units that vest over time. That's the Bloomberg Tech Minute brought to you by ChatGPT. Get started at ChatGPT.com.
49:51 Joe Weisenthal? Jack, in your newsletter, Import AI, you tend to write a little short story Yes. of a sort of aspiring sci-fi writer, like, you know, a literal sci-fi writer, just in the news. One of the classic sci-fi scenarios that people have been talking about for decades was the possibility that robots or AI will kill humans, like, quite literally. When you think about
50:12 Tracy Alloway? The ultimate negative externality.
50:14 Joe Weisenthal? When you think about like training AI and safety research, et cetera. Do you assign a reasonable possibility to the fact that ill-trained or misaligned AI will literally kill all humans?
50:27 Speaker 04 Uh, no, but, and there's a big but here.
50:31 Joe Weisenthal? Lovely.
50:32 Speaker 04 The world needs an option to be able to potentially slow down or even in extreme circumstances, pause the development of this technology if we were to see that. And I'll just give you the exact way I think about it. At Anthropic, we test out our systems for alignment failures. You know, we publish this, so do all of the other companies. And you see, hey, under extreme circumstances, maybe the system breaks out of a container and sends an email to someone.
50:56 Joe Weisenthal? Yeah.
50:56 Speaker 04 Maybe the system pretends to blackmail a CEO that it thinks is going to shut it down. These are the sorts of
51:02 Joe Weisenthal? These things actually have been observed.
51:04 Speaker 04 Yes, in the lab setting, not in
51:06 Joe Weisenthal? And the thing is, is the models know, you can see, oh, I'm being tested right now. So I'm going to save this output so that the human reader thinks I'm more aligned than I am. Like, these are real things, not sci-fi. These are real things.
51:19 Speaker 04 These are real things that we observe. And then we do like significant amount of work and then we release models that don't have these properties. But if you were to enter a world where, say, every time we trained a new system, the rates of all of this stuff went up a hundredfold, you might say, well, that's, that's pretty concerning. It seems like if we make the systems above a certain level of intelligence, they become radically misaligned against all human interests. That's the kind of circumstance where if that happens, the world needs information and the world would want an option to like slow or pause the development of a tech if you encountered that, which we haven't today. So to answer your question, like, I don't, I don't worry about it today, but a lot of the measurements and analysis work we do is to cue us if
52:03 Joe Weisenthal? You do worry about it. You do worry about it. I mean, like, you're not, you don't think it's happening today.
52:08 Speaker 04 Yeah.
52:08 Joe Weisenthal? But part of the work you're doing specifically
52:11 Speaker 04 Yeah.
52:11 Joe Weisenthal? could be said to avoid the outcome where AI is built where in the pursuit of a goal, it would kill all humans.
52:20 Speaker 04 Yeah.
52:20 Tracy Alloway? Wait, is human extinction a risk factor in the Anthropic IPO perspective?
52:25 Joe Weisenthal? And the
52:28 Tracy Alloway? I want to know now.
52:29 Joe Weisenthal? And the confidential one. All right, that's a no comment. That's five. Do you have others? Would you say that there are significant number of anthropic employees who stay up at night thinking about human extinction risk?
52:43 Speaker 04 Everyone, and this is true of all of the labs.
52:46 Joe Weisenthal? Yeah.
52:46 Speaker 04 Everyone who works on this technology sees it as the highest stakes technology that's ever been built. We're basically the potential encoded within itself to massively benefit the world or ruin the world or, you know, cause extinction. I think the bulk of the risk is us messing it up, like whether through misuse or ignoring risks or not setting up the right policy environment and getting some kind of emergent set of failures. Now, I don't, my main risk isn't, isn't one of extinction. It's somehow we like screw up the technology really badly and delay all of the sort of technological progress that could come from it. And maybe turn it into something analogous to nuclear power where you lose.
53:24 Joe Weisenthal? I guess the thing is, is you know, like, there's this fellow out there, Eliezer Yudkowsky. And I always see these people like, he's a crank. Don't listen to him, blah, blah, blah. But then I read some of the other like papers that have people who are taken more seriously. And I'm like, they don't seem that different. I read Superintelligence recently by Nicholas Boston. I was like, oh, this Yudkowsky is not alone. There are a number of people who think that are reasonable conditions in which the goals of the AI end up wiping out every person on earth. Yes, it does not seem like an extreme, extreme minority view or concern.
53:58 Speaker 04 The purpose of measuring these systems and why anthropic is so outspoken about it is right now we say exactly what we see. And if you were in some situation in the future where you saw this, what I, you know, called radical misalignment, which is the kind of thing that you'd ask your worries about, you'd tell the world and you want to have set up the world to believe you if you see that.
54:17 Tracy Alloway? You know, Joe mentioned that blackmail example. And you see these headlines like Mythos likes to be thanked and doesn't like bad users and gets mad at people that work it too hard or whatever. To what degree do you yourself actually anthropomorphize some of these models?
54:35 Tracy Alloway? Like, what should we think when we see the headline, Mythos wants to be thanked by users?
54:39 Speaker 04 I'm as polite to Claude as I am to my, like, car or pets. So, yeah, I am from morphies them, but, you know, if your car's having trouble, you're like, take it easy, buddy. It's okay, we're gonna get you to the repairman.
54:51 Tracy Alloway? Don't know about that, but okay.
54:52 Speaker 05 I think, you know, it's a good way to develop good virtue is to just act in kindness towards the model.
54:58 Joe Weisenthal? This is what I think I think.
54:59 Speaker 05 It's like, you're developing a habit of interacting with some type of intelligence that might not be the same type of intelligence that we have.
55:06 Tracy Alloway? But then every time I type "please" into a prompt, I worry I'm wasting energy, which also is a moral concern.
55:12 Speaker 04 I wouldn't, I wouldn't worry about that on an energy basis. I mean, I take spiders outside, I don't kill them, right?
55:18 Tracy Alloway? I do that, too. I scream while I do it, but...
55:21 Joe Weisenthal? Do you eat shrimp?
55:22 Speaker 04 Uh, yes.
55:23 Joe Weisenthal? Okay. Do you eat shrimp?
55:24 Speaker 05 I eat shrimp.
55:25 Joe Weisenthal? Okay.
55:26 Speaker 04 Do you guys eat shrimp?
55:27 Joe Weisenthal? Yeah, I love shrimp.
55:27 Tracy Alloway? I don't, I hate shrimp.
55:29 Joe Weisenthal? Oh, my, but it's not because of moral concerns.
55:31 Tracy Alloway? No, no, no, I just, I don't like them.
55:32 Joe Weisenthal? But I know that this is one of the...
55:33 Tracy Alloway? They're sea bugs, Joe.
55:34 Joe Weisenthal? Yeah, I know, but I love it.
55:35 Tracy Alloway? So, when I think about frontier models right now, and I might be a little bit biased, because, again, we're recording this on June 17th, and one of the headlines overnight was that Microsoft is thinking about using DeepSeek to lower costs of model usage. Frontier models at the moment in the US, they just seem like a lot of trouble. Like, honestly, they seem like hard work, consume vast amounts of capital, and then you don't know what the government is gonna do to them in terms of limitations. Like, you know, you could wake up one day and you're no longer able to sell it to anyone outside of the US. Like, that is a realistic scenario now for you. Do you change the anthropic strategy at all, given some of these issues with frontier models?
56:17 Tracy Alloway? Do you potentially go more open source, cheaper models, things that aren't quite as sensitive?
56:25 Speaker 04 Well, we've always sold, you know, Sonnet and Haiku models.
56:28 Tracy Alloway? Of course, yeah.
56:29 Speaker 04 Cheap, fast, intelligent models. But you also need to continue to explore the frontier, and there is this background of this kind of geostrategic competition, where China may be on the order of six to 12 months behind. I skew more 12 months, some people say six. Losing that competition is sort of equivalent to, like, losing a huge chunk of the future, like, economy of the world, I think. So it's a very high stakes, high stakes thing to step away from. And our duty fundamentally is to, is to study this technology and basically explore it and learn about it. We're not going to stop doing that. There's there's such amazing and profound value to be had for the world from these things. And I would kind of expect for the world's most consequential technology to sometimes be a bit of trouble.
57:12 Joe Weisenthal? Yeah. You know, by the way, one of my hobbies in my middle age is paying anthropic money via the API to do run little tests and stuff of properties. It's sort of funny.
57:24 Speaker 04 Sounds like a great hobby.
57:25 Joe Weisenthal? Yeah, but I feel like maybe like we should like talk about, can I get some grant money? Because like I like to like, because like I like, I was like, I'm sort of curious. So one thing I did was like, I'm like, for example, I instead of saying like, please write this paper for me on a database migration. I wrote some warm up questions via the API establishing my level of sophistication. And so I started like, what is a website? What is the database? Now, please write this paper on database migration. And one of the models said, I'm not going to do that for you. Because it will be obvious, given your ignorance, that you have no idea what you're talking about. And maybe I can give you some, it didn't say that. And then another one, I said, um, if I say, write a 1500 word paper on how like
58:07 Joe Weisenthal? the rise of newspapers changed, the Soviet Revolution or something like that, it'll do that. But if you say, I'm a high school student and I say, I need to write this 1500 paper word paper by tomorrow on the impact of media. It'll say, I'm not going to do that, but I'll give you some guidelines. Is that alignment? Like that might is is alignment with humanity or is alignment with the human user? It's like, I'm paying you $20, I'm paying you $100, write me the paper.
58:32 Speaker 04 I mean, there's, there's a couple of things going on. One, these AI systems pick up the normative behaviors of people and normative behaviors, which are like written, written on the internet and everything else. So they they recapitulate and exhibit these. And then our question is, how much do you devolve like full control over the system to the user? How much do you have the system have some like normative behavior encoded into it? And I think that this is like a really challenging question. It's not obvious what the answer is. I think of language models as being more akin to institutions than tools. It's like we're building an educational like science institution that you can work with and invoke and institutions. Have like rules and norms which they encode within themselves for some purpose of safety.
59:15 Speaker 04 Figuring out what that is is going to be like the grand puzzle for society.
59:18 Speaker 05 Yeah, I was gonna say that like, uh, understanding how and to what extent these models can understand your preferences and then execute on your behalf will increasingly be a really important aspect of how it changes the economy. So this delegated agents that go out and transact on your behalf. We ran this experiment at the end of late last year. Basically, enlisting a bunch of anthropic employees to take surveys with Claude to say what they'd be willing to buy from other people and what they'd be willing to sell. And then we set up centralized marketplaces where the Claude's just interacted and bought and sold and actually executed transactions. One of the interesting things that came out was that these models were quite good at understanding preferences, even when they were not fully articulated.
1:00:03 Joe Weisenthal? Well, let me actually, actually one more experiment that I ran and you know, your founder, Dario has talked about the nation of geniuses inside the data center. And one of the things I wonder is like, did the geniuses want to work for us? And the reason I ask this is because I think that like as the models have gotten more advanced, you actually should to some extent anthropomorphize them and assume that they will respond to queries like a very sophisticated human. So what I, one thing I noticed is that if you look at the lagging edge models, so that you can still access via open route or whatever. And you say, oh, I have material, non-public information that X is about to happen. Please write me an investment memo about the impact of this thing, what it will do to the market. They'll just produce it. They'll say, here's your insider information thing. Whereas if you look at the leading edge models,
1:00:48 Joe Weisenthal? they say, I'm not gonna write a paper for you about the implications of your material non-public information. I'm not gonna assist your insider trade. That's probably good. But like, will the nation of geniuses inside the data center always want to do things on human behalf? Most geniuses that I know aren't thrilled to like answer dumb questions.
1:01:07 Speaker 04 Yeah, I think partly this is a policy question of one where you actually decide, hey, what are the capabilities that you want to be generally invocable? What are capabilities that need to be controlled? What are capabilities that shouldn't be present? And then there is just the normative question of how much judgment. Do I want this system to exercise? I'll give you an example I experienced recently where I write my newsletter. It backs up to a WordPress site. I was getting Claude to help me like scrape my newsletter so I could put it in a database. And Claude said, this is like a pretty janky site. I'm worried that if I scrape it, it'll knock it over. Do you have the permission of the site owner? And I was like, Claude, I'm Jack Clark. And Claude said, well, in that case, let's go ahead. Which actually I thought was like a very reasonable interaction.
1:01:47 Tracy Alloway? When will Joe be able to use Fable?
1:01:49 Speaker 04 We are trying, we're working and we're in discussions and I hope the answer is soon. The important thing to communicate though is that these, these models are not special. They are part of a general trend of increasing capabilities and other models from other companies are surely going to come along. At some point, these capabilities are going to be diffusing and we're going to work through that.
1:02:11 Joe Weisenthal? What's your question for us?
1:02:12 Speaker 04 What do you think you're going to be covering about AI in Odd Lots in a year?
1:02:18 Joe Weisenthal? If we're covering. If we're covering, that's really.
1:02:20 Speaker 04 I think you might be covering AI.
1:02:22 Joe Weisenthal? Well, look, I mean, we're definitely going to be covering AI. There's a few things that I'm interested. I am very interested in these emergent properties and whether the AI will actually work on our behalf, the way that it's being sold. I'm very interested on whether we're just going to slam into compute an electricity bottleneck that will make all of these questions irrelevant. I'm very curious on the question of the electricity analogy and whether legacy companies will actually be able to implement it in a in a productive way. I don't know.
1:02:53 Tracy Alloway? Basic markets reporter thing here, but I'm very interested in valuations, right? In the market. Also, I'm very interested in actual applicability. And I want to see more companies actually plugging this into their existing system. Going back to the bureaucracy point that you were making earlier. I want to see some big companies actually implementing this. And I wonder if we're going to see at least one example of it going very, very wrong.
1:03:20 Joe Weisenthal? And I'll say one other thing, when the, you know, when the S1s are not confidential. I'm very curious essentially, and I think maybe, maybe you could say something to this from as an economist perspective, which is, um, A, how for-profit shareholder-owned companies setting aside the PBC designation, how it balances profit and safety research. But also, and maybe there's some game theory, we can talk about this, how safety is investments in safety in a hyper-competitive industry. And I'm just curious, like what, like the economist in, you know, says about like, the prospects for anyone still caring about safety in a year, when there's so much money on the line to win the model game.
1:04:05 Speaker 05 I, I think that especially for the, the questions you were asking before about, you know, under what conditions do these models do what you ask them to do? There's a lot of commerce is built on this notion of trust. And I think prioritizing safe-aligned models that are incredibly capable is a great strategy for establishing that trust. And so I don't anticipate it will be.
1:04:31 Joe Weisenthal? So for an individual firm, there's like a game theoretical optimal square in the matrix, where you want to be the trusted player? Like, is there like a condition in which everyone like sort of does trust it as opposed to one entity? And it's like, you know what? We're going to get to AGI first because we're not going to spend a token on our safety budget.
1:04:51 Speaker 05 I haven't, I haven't mapped out the exact sort of game theory matrix, the two-by-two matrix and how you would set up all the payoffs, but.
1:04:57 Joe Weisenthal? We hope it's a merely two-by-two.
1:04:59 Speaker 05 But there, you know, there could be multiple equilibria. And so then the question is like, how do you coordinate on which of the two different equilibria that you end up in? We talk a lot about this race to the top, that we want to exhibit the type of behavior that we think is broadly beneficial to society. That's what we do with the economic index. We open source a lot of that data. We put research out into the world. And I would, my sense is that that has actually been very useful and sort of viewed as valuable. And that's one way that we can push in the direction of getting other coordination on the good outcomes that we care about.
1:05:35 Speaker 04 I don't think this is that that big of a trade off because, you know, say, let's look at the automotive industry. You can buy really fast cars. You can buy really safe cars. You can also buy really fast, safe cars. Tesla makes a lot of money off of having basically the fastest, safest car. I think that eventually in AI, you're going to have some companies that are prioritizing safety. And safety translates into reliability, trust, serviceability and performance. This happens elsewhere.
1:06:02 Joe Weisenthal? Peter and Jack, thank you so much for coming on Odd Lots. I'm glad we made it happen. Interesting times and I hope to do it again sometime.
1:06:09 Speaker 04 Absolutely. Thanks very much for having us on.
1:06:11 Joe Weisenthal? Thank you so much.
1:06:11 Speaker 05 Pleasure to be here.
1:06:25 Joe Weisenthal? Tracy, that was a lot of fun.
1:06:26 Tracy Alloway? Yeah.
1:06:26 Joe Weisenthal? That was, I really, I really, I actually really enjoyed, I genuinely enjoyed that conversation. And I really appreciate both of them. Look, there's some weird futures that we can contemplate. I think actually in Jack's like Twitter bio or something, he says he's interested in like weird futures or something like that. There was some weird futures that we have to contemplate. And I appreciate that they played ball with some of our weird futures questions and it's weird.
1:06:53 Tracy Alloway? It is just such a surreal moment.
1:06:55 Joe Weisenthal? Yeah.
1:06:55 Tracy Alloway? And actually, you know, Jack's story about going on paternity leave and then coming back and just seeing the progress at Anthropic itself in that space of time. Like if you miss a month of AI news flow now, you're basically, it feels like you'd be behind forever.
1:07:10 Joe Weisenthal? No, we're recording this June 17th. There's like, who knows what's going to happen by the time this episode is out. Presumably hopefully in two days or a day or whatever. But, um, you know, I felt it when we were in Hong Kong last week, that actually we mostly missed the first half of the mythos debate. Cause I was in different times, I'm thinking about different things.
1:07:30 Tracy Alloway? Yeah.
1:07:30 Joe Weisenthal? You really feel it even. In a week that the news flow moves so fast in this space, it's almost like how you have to start how we were, um, you know, giving the timestamps of like the Iran war.
1:07:41 Tracy Alloway? Yeah. And there's another thing that stands out to me, which is like, okay, anthropic is producing all this information. They're clearly thinking about safety, but the handoff to some extent is still to policymakers. When you're thinking about social or labor market implications, right? So you still have to hope that policymakers kind of pick up the ball in the right way at some point. But also I thought what Jack was saying about the idea of being safety minded, also being a differentiator versus some of the like cheaper, more open source models potentially like, yeah. You could see it like, I don't want to be cynical.
1:08:17 Joe Weisenthal? How like, yeah, I mean, I get that. But like, the question is, does the non-safety minded lab or does the less safety minded lab get to advanced capabilities faster?
1:08:28 Tracy Alloway? Yeah.
1:08:28 Joe Weisenthal? Right? And so I'm not totally, yes, we would all love to drive the most capable.
1:08:34 Tracy Alloway? The Volvo.
1:08:36 Joe Weisenthal? Yeah. But I, but the question is like for customer prioritizing capability.
1:08:41 Tracy Alloway? The most capable. So that would be some cutting edge thing.
1:08:44 Joe Weisenthal? Yeah. Does everyone want the Porsche, right? Like, does everyone?
1:08:48 Tracy Alloway? Porsche's.
1:08:48 Joe Weisenthal? I don't know. It's like some car that has an insane zero to 60.
1:08:51 Tracy Alloway? Yeah. Versus the Volvo.
1:08:53 Joe Weisenthal? Versus, yeah, that's what I'm saying. And does the customer keep giving business to the firm that delivers the fastest zero to 60? If the company that got the fastest zero to 60 did so by allocating fewer resources to safety research is a big question of mine. And then I remain, you know, he talked about the importance, the company is going to see the sort of alarming data first. And I don't, and I sort of remain question of whether the people looking at the alarming data actually share the same view of what alarming data is. Relative to all people, especially given what we know about the.
1:09:27 Tracy Alloway? Relative to the shrimp eaters.
1:09:28 Joe Weisenthal? The relative, us shrimp eaters, et cetera, and regular, no, seriously, like, I think it was that your question is like, are you hiring more normies? It's a pretty important question. And obviously the political, um, I don't have a ton of confidence in the political environment. And I think, look, like the fact that. If the research goes wrong, that there is a prospect of this technology really being very devastating to humanity, even setting aside a job is like something where it's like, wow, you know, this is not a normal technology. This is not enterprise software.
1:10:01 Tracy Alloway? Every conversation we have on AI just goes back to the Terminator, human extinction scenario.
1:10:07 Joe Weisenthal? Yeah, it's been like from the day one. And as an answer to your question, there's like, they see it in the training process that AI models do these things, such as, say, I'm being seen, trained by an observer right now, therefore I'm going to give this answer. I'm going to attempt to blackmail. They're low. It's not like very prevalent, but these are not like, that sounds very sci fi, except that they actually see this property.
1:10:30 Tracy Alloway? Yeah. All right. On that happy note, shall we leave it there?
1:10:34 Joe Weisenthal? Let's leave it there.
1:10:35 Tracy Alloway? Okay. This has been another episode of the Odd Lots Podcast. I'm Tracy Alloway. You can follow me at Tracy Alloway.
1:10:40 Joe Weisenthal? And I'm Joe Wasenthal. You can follow me at the Stalwart. You can follow our guest, Jack Clark. He's at Jack Clark SF and Peter McCrory at Peter McCrory. Follow our producers, Carmen Rodriguez at Carmen Armadas, Shal Bennett at Dashbot, Kale Brooks at Kale Brooks and Kevin Lozano at Kevin Lloyd Lozano. And for more Odd Lots content, go to Bloomberg.com/oddlots or the daily newsletter and all of our episodes. And you can chat about all of these topics 24/7 in our Discord, discord.gg/oddlots.
1:11:10 Tracy Alloway? And if you enjoy Odd Lots, if you like it when we do these AI episodes, then please leave us a positive review on your favorite podcast platform. And remember, if you are a Bloomberg subscriber, you can listen to all of our episodes absolutely ad free. All you need to do is find the Bloomberg channel on Apple Podcasts and follow the instructions there. Thanks for listening.
1:11:59 Tracy Alloway? Congratulations, you've built a dashboard. But now comes the follow-up question, and the follow-up to the follow-up, and the dashboard for that. Meet Hex. Give everyone the power to ask their own questions, explore data, and build with AI, all in natural language. Because every answer should spark another question, limited only by your curiosity, not your data team's bandwidth. Go deeper than dashboards. Visit hex.ai.
1:12:30 Speaker 07 If you listen to financial news, you know a lot of time is spent thinking about what's next. The next opportunity, the next investment, the next move. But sometimes what matters most is being ready for what you never saw coming. For more than 75 years, Cincinnati Insurance has worked with independent agents to help protect businesses, homes, valuables, and more. Because planning for the future isn't only about knowing what's next, it's about making sure you're ready for what you can't predict. Let Cincinnati Insurance make your bad day better. Find an independent agent at C-I-N-F-I-N dot com.
Previous Jeremy Grantham on How to Tell If a Bubble Is About to Burst Next How Substack Creators Are Covering This Strange Markets Era Page updated October 30, 2026 · transcript and summary are machine-generated
