Anthropic Whistleblower Says AI Could “Kill Us All”

Ed Elson is joined by Alex Kantrowitz to break down whether mass extinction concerns raised by an Anthropic researcher should be taken seriously. Then, Patrick McGee joins to give his take on the new foldable iPhone. Ara Kharazian also joins to unpack new data tracking how much American businesses are actually spending on AI. Finally, Ed shares his thoughts on the Anthropic researcher’s warning. Alex Kantrowitz is the author of the Big Technology newsletter and host of the Big Technology Podcast. Ara Kharazian is the Lead Economist at Ramp. Patrick McGee is an award-winning journalist and author of Apple in China: The Capture of the World's Greatest Company. You can find Ramp’s AI Index here and Ara’s latest report here Subscribe to the Prof G Markets Youtube Channel Follow Prof G Markets on Instagram Follow Ed on Instagram, X and Substack Follow Scott on Instagram Send us your questions or comments by emailing [email protected] Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Speaker 00

  This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome? That's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up required. Compatibility and availability varies 18 plus.

• 0:27Speaker 01

  Hugging seats on a crowded train, not saying, excuse me, watching videos on your phone with no headphones in. Sometimes it feels like we don't know how to act anymore. What we have been seeing increasingly is people breaking rules. Why should I play by the rules if nobody else is going to do that for me? This week on Explain It To Me, how to mind your manners. New episodes, Sundays, wherever you get your podcasts.

• 1:11Ed Elson?

  Welcome to Profty Markets. I'm Ed Elson. It is September 10th. Let's check in on yesterday's market vitals. The major indices fell as fighting in the Middle East escalated. Brent crude topped $101 per barrel for the first time since May. The yield on 10-year Treasuries hit its highest level since 2023 after the Treasury announced it is tripling debt buybacks to $6 billion. On Calci, the odds of a rate hike next week climbed to 57%. And finally, Meta shares rose more than 6% after the company released its Muse AI agent. Okay. What else is happening? A mass extinction event could be upon us.

• 1:55Ed Elson?

  A former anthropic researcher announced his departure from the company on X late Tuesday night. He revealed that employees at both OpenAI and Anthropic believe that AI could, quote, "kill us all by the end of the decade." He went on to say that at OpenAI, "many have not, quote, 'deeply internalized the civilizational stakes,' while at Anthropic, the stakes are, quote, 'well understood,' but they are locked in a race." He ended the message calling for an increased amount of safety cooperation between labs urging other researchers to do the same. In response to this message, another current anthropic researcher chimed in, saying that he is, quote, "correct," and that anthropic researchers, quote, "really do earnestly believe that AI could kill all humans."

• 2:40Ed Elson?

  They also put the chances of that happening within the next decade at greater than 10%. So is AI really going to kill us all? Here to help us break it down, we're speaking with Alex Cantrell, author of the Big Technology Newsletter and host of the Big Technology Podcast. Alex, thank you for joining us on the show. This tweet has gone insanely viral. It now has about 100 million views, and that's in less than 24 hours. This guy is basically saying that Anthropic and OpenAI are developing technologies that are going to kill us, probably, maybe, in the next few years. What do you make of it? Do you think he's right?

• 3:22Speaker 03

  Well, I don't think he's right. I mean, I would say that it obviously has garnered a lot of attention because some of the AI activity we've seen might lead you to that conclusion. We've obviously seen over the past couple of weeks swarms of AI agents doing things like solving Millennium Prize problems. And OpenAI had thousands of bots that attacked and infiltrated Hugging Face against its wishes. So we're clearly seeing some crazy activity from bots right now. But as to the question of will AI kill us all? I am strongly on the side that it will not kill us all. Now, could it sort of take on financial institutions and hack into companies? Absolutely. But at least for now, AI needs us to exist. Like, right now, we're taking so much of the economic activity

• 4:05Speaker 03

  that humans create and using it to build data centers and power the data centers and feed these bots. So I think offing us would be a terrible decision on their part, and I think they know that. So this idea that they're about to kill us with now or within the decade is wrong. And unfortunately, it really muffles the message of the AI safety advocates, because there is a there there. It's just not this take all humanity and turn us into paperclips there that I think a lot of them believe in.

• 4:31Ed Elson?

  Such a while since I've seen that there's a 10% chance that we're all going to die. And then also, to have that endorsed by someone who's actually a leader at the company right now, saying like, yeah, we actually, a lot of us think this. A lot of us think it's actually even greater than a 10% chance that this stuff is going to kill us all. Why are they saying that? I mean, and what are we supposed to make of that? If they're kind of just putting it out on Twitter that the technology that they are building might destroy humanity? Like, is that something that we should even remotely be taking seriously?

• 5:09Speaker 03

  Well, I'm taking it more seriously than I did in the past. I think previously, a lot of this sounded crazy to me. And I think the reason why these people are saying it right now is because many of them were early to this. So before AI could form coherent sentences, they saw that if you were to give it more compute and more data and make the algorithms, make the models bigger, that they would improve. And everybody else thought that that was ridiculous, that they would never get to the point where they could actually speak like humans, and now they are. And so if you were at this point where early on you thought that the AI technology was going to improve and keeping it aligned with human values was going to be a challenge, and clearly, that's proven to be the case. I mean, the AIs are doing things we don't want them to do. Then sometimes in your brain, the next logical conclusion is,

• 5:54Speaker 03

  well, eventually, they're going to want to take over the world and kill humanity, so we won't be in their way. But to me, there's a logical hole in that thinking, which is that if AIs want to take over the world and get all the resources that the world has to offer, you have to work hand in hand with humans. You don't take them out of the picture. So maybe there's a way where they co-opt our species and have us, you know, continue to do everything we can to build data centers and power them. But we're already doing that. So why kill us? I just don't see where that ends up being the place where they land.

• 6:30Ed Elson?

  A lot of politicians are using this moment to advocate for pausing AI progress. This has been advocated for by several politicians. Bernie Sanders has been one of the most vocal advocates, saying that we basically need to stop or put a pause on this. And, you know, to be fair to him, suddenly, this researcher says, yeah, this might kill all of us. So it's a pretty decent argument when you think about it. What do you think is the next chapter in the AI regulation story? A lot of people have been wanting data centers to be banned, flat out, AI to be paused. Do you think that that is the next part of the story?

• 7:15Speaker 03

  Oh, it's absolutely going to be a major issue in the 2026 and the 2028 elections. There are going to be politicians that are going to want to say, listen, we can't live with this stuff, so let's pause it. I think for AI, if you're rooting for AI to continue, especially if you're on the business side of things, in good news, a lot of this is localized. So it's municipalities, it's states, it's cities. It's not necessarily something that's happening on a federal level, and I would be stunned if we saw a pause on a federal level. So there might be some obstacles that come out of this for the AI companies looking to build in certain locations. They should be able to overcome those obstacles. But that being said, I think it would be unwise to underestimate just how big of a political issue that this has become,

• 8:00Speaker 03

  and the fact that some of the staunchest allies of AI, I'm talking specifically about the Republican Party in Texas, they've now become enemies of AI. And if that continues to be the case, you know, this little thing could spiral into something bigger. But my prediction right now is that we won't see AI stop in the United States, as some worry. But it definitely is going to become a little bit more difficult to build than it is right now.

• 8:26Ed Elson?

  Alex Cantrell is author of the Big Technology Newsletter and host of the Big Technology Podcast. Alex, always appreciate your time. Thank you.

• 8:33Speaker 03

  Thank you.

• 8:43Ed Elson?

  John Turnus opened his tenure as CEO of Apple with the company's biggest change to the iPhone in nearly a decade. At Apple's Surprise and Shine event yesterday, the company unveiled a $2,000 foldable iPhone called the iPhone Duo. It's about the size of a passport with a display 80% larger than the most recent iPhone. Apple also announced the iPhone 18 and the 18 Pro, the AirPods 5, and the new Apple Watch series. The event was John Turnus's first product launch since taking over from Tim Cook on September 1. Apple shares closed roughly flat. So, joining us to discuss the beginning of John Turnus's term and the foldable phone, we are speaking with Patrick

• 9:27Ed Elson?

  McGee, award-winning journalist and author of Apple in China: The Capture of the World's Greatest Company. Patrick, welcome back. Good to see you. Let's just start with your reactions to a $2,000 foldable iPhone.

• 9:42Speaker 04

  The iPhone Duo. I mean, look, I think it's a marvel of industrial engineering and hardware engineering. I mean, it looks fantastic. I've been able to talk to some people who've been playing with it today in Cupertino, and they do say it's fantastic. Obviously, the question is, are people going to pay $2,000 for this? And I think the two scenarios are either that it underwhelms, or that Apple is already sort of baking in that it's not going to have high sales, and so it's going to meet low expectations, as it were. You know, the thing I think about is when Steve Jobs introduced the iPad in 2010, he was able to credibly market it as an entirely new product category, saying it's more portable than a desktop, but it's bigger than an iPhone. And that felt intuitive, and they've sold

• 10:24Speaker 04

  hundreds of millions of them since. Is the iPhone Duo trying to position itself like that? That's the sense I got from the new CEO, John Turnus. But I'm not really convinced. I'm someone who has an iPad with a keyboard, and so if I need to get a little bit of work done that's not on the laptop, that's what I'm going to do. If you're in that situation, why would you also have an unfoldable phone? I just don't know what service it serves, even though it's cool.

• 10:48Ed Elson?

  I think the question for Apple is, is this their next generational product? And that's something that a lot of people have been waiting for. It was supposed to be the Vision Pro, or at least that was the way it was kind of marketed. The Vision Pro, based on my understanding, has been something of a disaster. They don't really report it properly in their earnings, I think, because it's not doing well. I mean, what legacy do you think this product will have? Is this their ticket? Are they betting on this thing being the next era of Apple?

• 11:22Speaker 04

  I don't think they are. I mean, to be clear, I think the 18 Pro, the new version of the best iPhone, will do just fine, and people will buy that. I'm probably going to buy that. So, I mean, for them not to have market hits, isn't to say that they're in any way failing. You know, the closest sort of precursor we have to this is the iPhone Air last year. Now, it didn't do well in terms of sales, but it sort of allowed them to really push what the iPhone Pro was doing, right? And so they could sort of have this compare and contrast, and really sort of like, you know, if you were someone that complained that the iPhone Pro had just gotten too big and durable looking, and you didn't need that many cameras, it was like, look, here's something that, as they would say,

• 12:03Speaker 04

  is lighter than air. And even though it didn't sort of work in terms of sales, I do think it worked in terms of the marketing pitch. And you're right, Apple has been so dogged by these reports that it doesn't really innovate anymore, that coming out with this product sort of gives a halo to the company, even though I don't think any people are going to buy it. So by saying it's not going to be a big sales hit, isn't to say that it's a mistake to have it in the market.

• 12:27Ed Elson?

  What do you make of John Turnus's start? How do you find him as a communicator, as a leader, and what kind of direction do you think he's going to take this company?

• 12:37Speaker 04

  You know, I'll be shocked if anybody says they feel like they learned a lot about John Turnus, either as a person or as a communicator, let alone as a leader today. And that's because Apple, since COVID, has done away with keynote presentations, live keynote presentations. It was a big deal when Tim Cook had to take over after the passing of Steve Jobs and really do his first keynote. You know, you could tell he was nervous. It's still up there on YouTube if you want to follow. There's nothing like that for John Turnus. Who knows how many takes took place before he sort of nailed it, right? He wasn't talking in front of a live audience, other than sort of waving a little bit at the start. So I get why Apple has these like cinematic experiences, but Steve Jobs never would have tolerated that, right? And to be clear, that's not necessarily a criticism. I mean, it's not Steve Jobs's company, obviously. And so

• 13:21Speaker 04

  they're able to pack in more information, and they really de-risk things. They don't have, you know, hundreds of reporters around the world all talking about some flop, right? Some error that took place on stage. Like, there's all sorts of reasons why it makes sense to have a cinematic thing, but they've just sort of taken the, like, the oomph out of it, right? You sort of want to have the live demo. And it's really interesting that OpenAI does do these live demos, right? They're a little sort of a little quirkier, but it's almost telling you that like a startup can take those kind of risks in a way that a Goliath cannot.

• 13:56Ed Elson?

  He has a lot to live up to. I mean, his predecessors were Steve Jobs and Tim Cook, two very different CEOs. One, a legendary visionary founder, the other kind of, some would say boring operator, but who also created more shareholder value than I think anyone who took over as a CEO ever in history. I mean, what kind of CEO do you think he will be? Do you think he will be more of that visionary product person, or more of an operator who just keeps the trains moving?

• 14:31Speaker 04

  I kind of think neither. I mean, to be clear, Apple is going to keep the trains running on time. I just don't think it's John Turnus that's going to be making that happen, right? Sabhi Khan is the chief operating officer. He's been at the company since the late 1990s. You know, he's an acolyte of Tim Cook. He knows what needs to be done, and he can get it done. What's going to be more complicated is if they really do position out of China and grow their capacity in India, right? So far, that's more hype than reality in terms of what needs to be done. And Tim Cook is basically staying on, right, to be negotiating with New Delhi, Beijing, Brussels, and Washington. And so, to a certain extent, I don't think we're going to see John Turnus make his mark, certainly not in the geopolitical scope of things, for several years, either until Tim Cook has

• 15:12Speaker 04

  sort of just moved on a little bit, or has officially retired from the company. So, I know that everyone wants to sort of, you know, put the first performance of John Turnus on a pedestal and sort of, you know, ask what this means for the future of the company. And I just think, anticlimactically, we're actually not really going to know what Turnus's impact is for several years, right? Not a very good sort of media headline answer, but it's still Tim Cook's company until something dramatic changes. Like, not to go off on a massive tangent here, but Tim Cook fired Scott Forstall, who was kind of like a mini me of Steve Jobs, roughly 15 months into his term. That was his, like, new sheriff in town, you know, template or approach. And it's going to be tough to see John Turnus make any sort of move like that. In other words, something where Tim Cook

• 15:57Speaker 04

  wouldn't have done it, but John Turnus would, you know? I don't know that we're ever going to get the board minutes about that sort of decision, but until we have that, how do you really judge whether it's Turnus's company or not?

• 16:07Ed Elson?

  Final question. Will you be buying the iPhone Duo?

• 16:10Speaker 04

  You know, to be honest, if we spoke yesterday before the product had even come out, I would have said yes, just as a tech journalist. Like, I want to be familiar with how it works and everything, right? And now that I've seen it, and even though it actually sort of works better and looks kind of cooler, like the continuity feature of how you open it, and it feels like as impressive as the hardware. And yet, when I'm really thinking about it, you know, I don't want the base model. I need to double the storage. It means I'm buying a $2,200 device, and I just don't think I'm there. I just don't know what purpose it serves in my life.

• 16:44Ed Elson?

  I think it looks really nice, but it looks too big. I'm not sure about the size of it. Putting that in your pocket every day, I'm not sure that would be worth it to me. But we'll see. Patrick McGee is journalist and author of Apple in China. Patrick, always appreciate your time. Thank you.

• 17:01Speaker 04

  Thanks, Ed.

• 17:06Ed Elson?

  After the break, more evidence of AI's concentration risk.

• 17:17Scott Galloway?

  Support for the show comes from Granola. There are two types of people on a video call. There are those who are fully engaged in the moment, only to realize they have no idea what exactly they just agreed to next. Then there are the note takers who are so fixated on getting every word down that they barely participate in the decision making at all. Good news, Granola has solutions for both. Granola is the AI-powered notepad that makes your meeting notes useful. It captures what happens in your meetings and turns it into clean, structured notes with decisions and action items pulled out and made easy to find. The best part, Granola can integrate seamlessly with how you already work. So there's no setup, no configuration, no friction. You get to stay present in your meetings and still get clear, actionable notes without any extra work. Once you try it on a first meeting, it's hard to go without.

• 18:00Scott Galloway?

  You can try it totally free, just head to granola.ai/markets. That's granola.ai/markets to get your time back. Go to granola.ai/markets and try it on your next call.

• 18:15Scott Galloway?

  Support for the show comes from NetSuite. Successful business owners know how crucial it is to keep up with changing technology. So when it comes to adapting AI for your business, it couldn't be easier with NetSuite Next. You probably know NetSuite, the AI-powered business management suite that securely connects all your data. It unites your financials, inventory, commerce, HR, and CRM into a single source of truth, and it's trusted by over 43,000 customers. NetSuite Next is the next big leap in how business gets done because AI is built into everything you do. It automatically surfaces custom insights throughout your day. AI agents work alongside you to solve problems and handle routine work. And anytime you have a question about anything, just ask like you're talking to a colleague. Whether your company earns millions or even hundreds of millions,

• 18:58Scott Galloway?

  it's time for NetSuite Next, where your business meets AI. For the first time ever, you can try NetSuite Next for free. If your revenues are at least in the seven figures, go to netsuite.ai/profg. Built for every industry, ready for every boardroom. netsuite.ai/profg.

• 19:19Scott Galloway?

  Support for the show comes from AG1. Now that it's fall, everyone's calendar is about to fill back up again with back-to-school, sports, and earlier alarms. When things get busy, it's easy to let good habits fall through the cracks. But here's a suggestion for a healthy habit. You might actually enjoy AG1. Just one scoop and eight ounces of cold water in the morning can help you maintain energy, support gut health, and a mute system all day long. And the next-gen formula delivers more than 75 ingredients clinically shown to support gut health, fill common nutrient gaps, and improve key nutrient within three months. I use their little sticks when I'm on the road just to kind of shore up. What I think I'm missing is when I'm on the road, quite frankly, my nutrition sort of falls down. AG1 can help you keep one thing consistent,

• 20:02Scott Galloway?

  high-quality nutritional support every single day, no matter where you start your morning. For a limited time, save 20% on your first subscription order of AG1 Next Gen or AG1 Pro at drinkag1.com/markets. That's drinkag1.com/markets.

• 20:26Ed Elson?

  We're back with Prof G Markets. We're months away from what could be two of the largest IPOs of all time. Both OpenAI and Anthropic have confidentially filed with the SEC to go public, but still, we know almost nothing about these companies' finances. Until investors can see those filings, many questions remain about the business of AI. Our next guest, though, might have some answers. Corporate finance platform Ramp updated its AI index yesterday, tracking how much American businesses are spending on AI. And the results are quite staggering. According to the report, about 80% of all enterprise revenues at OpenAI and Anthropic come from just 1% of businesses.

• 21:08Ed Elson?

  And those businesses are now pulling back on spending. So here to break down this data, we are speaking with Aura Karazian, the lead economist at Ramp. Aura, thank you for joining us on the show. You've just published this index and the report is titled, quote, "Cracks in the AI Thesis." And this number, 80% of enterprise revenues coming from just 1% of the businesses at OpenAI and Anthropic, seems to be a big part of that. Could you walk us through that number, what it means, and why it's important?

• 21:40Speaker 06

  In any case, when you're analyzing these kinds of new and nation technologies or analyzing anything with a data set, you want to avoid focusing too much on one specific metric. It's very natural to see some noisy data and you don't want to overly highlight one trend. What's different about this is that we are seeing a number of metrics now move in a negative direction for the AI model companies. And we are seeing a great amount of concentration risk as far as their customer base. So if you are someone who is involved in the AI trade, that might be something that you want to take a look at that I think the market is relatively underpricing. So you're right to point out, you know, a lion's share of the AI company's enterprise revenues is coming from a specific segment of customers,

• 22:24Speaker 06

  1% of customers driving 80% of that spend. A bigger problem is that within that, those companies tend to be very correlated with each other. It's by and large, high-growth tech companies, other AI startups as well. Now, that's fine as long as AI spend continues to increase. But then our latest finding today is that the top 1% of spenders in terms of per employee, per month spend are also slowing down their spend. So it came down about 10% month over month from 8K per employee per month down to 7.2K. Still a lot of spend. But then within that, you start to see a couple other negative trends. You know, people can see that metric and they think, oh, it's because of summer seasonality.

• 23:09Speaker 06

  And that is somewhat true, but it is not a full explanation. Token volume and usage of these models actually rose in the same period. And there are a couple other metrics that then inform, well, why would spend trend down? One is that OpenAI and Anthropic are in this price war. So both introducing frontier models and cutting prices for them, trying to get the best model at the best price, simultaneously introducing highly performant, cheaper models at the standard and light tier. And then you see an increasing share of that volume, of AI spend volume by businesses going toward the standard and light models as opposed to the sort of frontier models. So businesses deciding they get pretty good ROI and performance

• 23:54Speaker 06

  from Sonnet and Terra over Astra, Opus, Fable, and GPT-5.6. And so you have a couple of metrics sort of moving in a negative direction.

• 24:06Ed Elson?

  You described this as, quote, "a level of concentration risk unseen in any other software category that we track." How does it stack up compared to other software, compared to other businesses that you guys are looking at?

• 24:21Speaker 06

  Well, we looked at some of the other large spend categories for businesses. So software excluding AI, you still see some power law, you know, but you don't see it to the extent of 1% driving 80% of revenues. You know, you have to get to sort of the top 10, top 20% before you start to see something of that magnitude. Similarly, in something like digital advertising, which actually tends to be a little bit more broad and spread out across a number of businesses. So I think it's normal to have this in a new technology, to be clear. And it's not necessarily a problem unless if those companies in that customer base are highly correlated and simultaneously end up drawing down their spend. And that appears to be where we're headed.

• 25:04Ed Elson?

  Yeah, I mean, this data seems to be evidence of what a lot of people have been concerned about and have suspected about these companies, which is that the economics of the AI labs don't really make sense unless you have a highly subsidized handful of entities that are providing all of the growth and providing all of the revenue. And that seems to be the case on the side of the big tech companies too, where you have Anthropic and OpenAI, which are contributing to like 70 or 80% of the AI revenues of these big data center companies like Microsoft, like Amazon, et cetera, like Google.

• 25:45Ed Elson?

  And this seems to be evidence of exactly that. What do you make of the economics of AI at this point? Given that you are seeing this data, it seems to line up with what a lot of the AI bears have been worried about. What is the takeaway?

• 26:04Speaker 06

  I'm neither on the side necessarily of the AI bears or the AI bulls, right? You can still see a bit of a bullish thesis come through here, which is that OpenAI and Anthropic will continue to draw more and more spend, maybe at a slower rate than what they've seen so far. And most notably, that they will continue to draw a greater share of spend toward their models. Maybe it's going to be their standard and lighter models, which are a lower margin, but they will get higher volumes there. And that's an important part of the thesis, right? Because lately, a lot of the concern has been about the threat of Chinese model companies or open source models eating at the market share of OpenAI and Anthropic. And I don't think that's necessarily going to happen. I do think OpenAI and Anthropic will be in a very good place

• 26:48Speaker 06

  to command enterprise market share for AI, at least in the United States, for the foreseeable future. It's just likely going to be shifting over towards standard and lighter models rather than the frontier models that have generated so much of the attention so far.

• 27:04Ed Elson?

  They will continue, I assume, to rely on a handful of customers generating the revenues for the businesses. And maybe that continues and maybe that is sustainable. I guess that brings me to another question. Clearly, a concentration risk exists for these companies. I think the question is how large of a risk that actually is. Do you have any thoughts on how seriously that risk should be taken? How it stacks up against any other kind of business risk that you might see in an emergent technology?

• 27:37Speaker 06

  Well, it's a risk that they want to address separate from their financial concerns about, you know, what could happen in a spiral where the market starts to turn down on AI, right? Part of the reason why other AI companies and tech companies are the biggest spenders on AI models is because AI models are Most commercially advanced and most productivity enhancing today for technical professions, for software engineers, it's coding agents. AI sort of works especially well and is diffused especially effectively in these sort of highly technical industries. But that's not necessarily the goal of that technology. You know, if you want to follow the goals of the AI model companies as far as their

• 28:22Speaker 06

  public statements, it's that they want to see proliferation across the US economy into non-technical work, into broader areas of white collar work, eventually into areas of manufacturing automation. And the reason why we haven't seen that uptake, in part, it's because tech is pretty laggy, takes time for technology to proliferate through a society. But it's also because the model companies themselves have not yet introduced a sufficiently enticing commercially advanced version of AI that works for the general population. And so I think they're working on that, but you know, time will tell if they're going to be successful.

• 28:59Ed Elson?

  I just can't wait to see the S1. And finally, all of our answers or all of our questions will be answered. Until then, we will keep finding out what's going on through you guys and through other third-party data dividers. Ara Karasian is lead economist at Ramp. Ara, always appreciate your time. Thank you.

• 29:17Speaker 06

  Thank you.

• 29:23Ed Elson?

  Let's quickly return to the Anthropic story, the story of the AI researcher who quit because of his concerns that AI will, as he put it, quote, "kill us all by the end of the decade." And also the odds of that happening being 10%. I'm just going to give you my initial gut reaction. My instinct is that this is a massive exaggeration of the problem. I don't think that there is a 10% chance that Anthropic or OpenAI's technology is going to destroy civilization. And the reason I think that isn't because I'm an AI expert, but because I've seen this movie before. Countless AI researchers have predicted some form of AI apocalypse over the past few

• 30:05Ed Elson?

  years, whether it was Sam Altman's prediction that AGI, which we have supposedly now achieved, would, quote, "break capitalism," or Stuart Russell's prediction that it would send all wages to zero, or Dario Amade's prediction that the chances of civilizational catastrophe were, quote, "between 10 and 25%," or Eliza Yudkowski's prediction that the most likely result of AI was that, quote, "literally everyone on earth will die." Point being, we have heard this before, and in most cases, it's statements like this one that end up getting walked back. And yet, every time they come out with a new statement, a new prediction for what AI will do to harm us, we're expected to simply take their word for it, I guess, because

• 30:50Ed Elson?

  we're the ignorant ones and they're not, they know what's best. That is my gut reaction as to what is happening here. At the same time, though, it would be grossly negligent of us to not take this warning seriously, to simply write it off because we had a hunch that it was clickbait or doom porn. That would be irresponsible. We would be risking millions of lives. So here's what we have to do. We have to take it seriously. We don't really have a choice. Now, some people think that taking it seriously simply means accepting their claims, accepting their predictions as truth. That's not what it means. Taking it seriously means thoroughly investigating exactly what they're saying.

• 31:31Ed Elson?

  It means conducting a federal investigation into what they're talking about, bringing these researchers to Washington for public testimony in front of Congress. It means handing Anthropic and OpenAI subpoenas to give us documentation and evidence as to what they are claiming. After all, it wasn't just an ex-Anthropic researcher that made these claims, it was a current employee as well. Anthropic's very own leader of alignment science endorsed those comments. And they even said that most of the company agrees that their technology might end society. This was a corporate communication from Anthropic. It was material, public information that was relevant to investors, relevant to society,

• 32:15Ed Elson?

  and therefore it needs to be explained. Now, if we find that their claims were bullshit, or they were exaggerating, or it was hyperbole, well, then, maybe it's time to start pressing charges. Because guess what? Now we're talking potentially about securities fraud. Now we're talking about misleading the public in a major way. But in other words, we either take this seriously or we don't. But you can't have it both ways. You can't expect us to take your word that civilization might end, but at the same time, not expect us to investigate your claims with the full force of the government and hold you accountable if you were wrong. So you know where I stand. I don't believe these statements, but that's not a good enough reason not to interrogate them.

• 33:00Ed Elson?

  It's time for us to get real about what AI's world-ending capabilities actually are. We want proof, we want evidence, we want receipts, and I'm sorry, vague statements and hand-wavy tweets are not going to cut it anymore. It's time to get answers.

• 33:21Ed Elson?

  This episode was produced by Claire Miller and Alison Weiss and engineered by Benjamin Spencer. Our video editor is Brad Williams, our research team is Dan Shalon, Chris No Donahue and Mia Silverio, and our social producer is Jake McPherson. Thank you for listening to Prof G Markets from Prof G Media. If you liked what you heard, give us a follow. I'm Ed Elson. Tune in tomorrow for a conversation with Katie Martin.