Special Report: Will AI Wipe Out Humanity? (And Who Profits)

Ranjan Roy from Margins is back for our weekly discussion of the latest tech news. We cover: 1) Anthropic Researcher Jacob Coxon quits and says AI might kill everyone 2) Why did his tweet go so viral 3) Did Elon Musk contribute to the X virality? 4) Point of order: Let's add some serenity to the AI risk conversation 5) Adding percentages to your statement of AI wiping out humanity doesn' t make it math 6) Is this marketing for the AI labs? 7) Politicians are hungry to regulate AI 8) Should we focus on science fact, not science fiction? 9) What if these fears are reasonable? 10) God help us Sheryl Crow 11) What this week means for these companies' IPO prospects --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b Stop online threats before they become real-world attacks. Visit ⁠ironwall.com/BIGTECHNOLOGY⁠ and request a free Risk Assessment to see exactly how exposed your executives are. Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  Is AI going to kill us all? We'll cover it extensively with the appropriate context and nuance on a Big Technology Podcast Friday edition, coming up right after this. This episode is brought to you by Genesis. What does it actually take to put agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations

• 0:43Alex Kantrowitz?

  now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay? Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation. Welcome to Big Technology Podcast Friday edition, where we break down the news in our traditional cool-headed and nuanced format. Crazy week of AI news this week. It's hard

• 1:23Alex Kantrowitz?

  to fully capture the rolling craziness that's happened in the AI industry. Everything from the millennium problem being solved by OpenAI and the dispute over credit, to what eventually overshadowed it, which is an Anthropic researcher came out and said, he's leaving because he believes AI is going to wipe out humanity, or at least there's a 10% chance that it's going to do that. And that eventually set off a international, I don't know, you'd call it a freakout, maybe, where everyone from policymakers to senators, to academics, thought leaders, journalists, and people in the AI industry expressed an opinion. And

• 2:05Alex Kantrowitz?

  there's been so much madness about it that we're going to try to do what our show, hopefully, when we do what we're supposed to do, does best, which is bring some nuance and some cool-headedness to this crazy conversation. And hopefully, at the end of this special report, leave you walking away with some groundedness in it. So the extreme opinions aren't the ones that rule the day. We're joined, as always, on Friday by Ranjan Roy of Margins. Ranjan, you still here?

• 2:32Speaker 01

  I'm still here for now. Human extinction has not happened yet. So until it does, we will podcast every Friday.

• 2:42Alex Kantrowitz?

  As is our duty. As is our duty.

• 2:44Speaker 01

  As is our duty while humanity continues to exist.

• 2:49Alex Kantrowitz?

  We will bear this burden, what's required of us by the podcast gods. Let's get right into the story, just so that the appropriate context is out there. This is from the Wall Street Journal, "Anthropic researcher quits over out-of-control AI fears." An Anthropic researcher is quitting the artificial intelligence industry over fears that the lab and its competitors are racing to build systems they won't be able to control. A sign of mounting safety concern within top AI companies. Jacob Coxon, a researcher who specializes in training new AI models by having them consume vast amounts of data, said Tuesday he's leaving the company because he doesn't want to participate in an industry-wide rush to build AI systems that can improve themselves. Worried such systems could spiral out of control and destroy

• 3:32Alex Kantrowitz?

  humanity, he followed it up in a tweet, said, "I resigned from Anthropic today. I spent the last three years doing pre-training research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives." Don't underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing. We're going to get into all the details here, but Ranjan, I mean, this, this thing has gone just like absurdly viral, 165 million views to date

• 4:15Alex Kantrowitz?

  and counting. What was your view of this when you saw it come out? And are you surprised that it has hit such a chord that it's reached so many people in such a short amount of time?

• 4:30Speaker 01

  I think the escape velocity side of it was a bit surprising, because again, we have been talking about this for a long time. In fact, I spent part of my Labor Day weekend reading the meter research and going through it and trying to understand the, what kind of real vulnerabilities we actually have around, you know, rogue agents escaping containment. And, and then obviously, to see this kind of take off, it is interesting to me. Part of me, like, the fact that this happened right after Labor Day, when everyone's summer vacations are over and they're back fully at work. I do wonder if it has a little to do with it. Do you think, is that a crazy theory?

• 5:14Alex Kantrowitz?

  Well, I think the fact that this hugging face story found a way to have life over a month and sort of primed people for a moment like this, you know, might have had something to do with it. Here's a conspiracy theory that I like, that I don't think is true. But it could possibly be that, you know, the owner of X probably has something to gain by something like this going so viral, either by forcing OpenAI and Anthropic to slow down, or potentially having regulation come in that sort of entrenches those that have built large data centers already and maybe hit a button at X headquarters to make it blow up. But I don't know, that might be a little step too far.

• 5:57Speaker 01

  Let's not, you know, like, forget the fact that something going viral on X can very directly and easily be controlled by said entrepreneur. So.

• 6:09Alex Kantrowitz?

  If you don't know who we're talking about, it's Elon Musk. But yes, go ahead.

• 6:12Speaker 01

  Yes, Elon Musk. I mean, it, okay, I think as we kind of unpack this entire story, it is worth thinking through, what are the motivations and possible incentives for every individual player? I think what makes this such an interesting story is there's so many nuanced, fascinating layers of incentive for the ways everyone is talking about it. I feel, I mean, for us, neither of us have a ton of incentive either way. So I think, listeners, you can get a pretty open, unbiased view in terms of, I mean, I'm pro non-human extinction. I'm, I want to make that, I want to make that clear. So I'm biased to avoid

• 6:54Speaker 01

  human extinction. I don't want it. But overall, like.

• 6:58Alex Kantrowitz?

  You're pro non-human extinction. Sorry, it just took me like 15 seconds to figure out. You're anti-human extinction.

• 7:04Speaker 01

  Anti-human extinction. Sorry.

• 7:07Alex Kantrowitz?

  You could have just said that.

• 7:08Speaker 01

  Let's clarify that. I wanted to, it's kind of like saturating a test rather than passing a test. If you listened to last week's episode, it sounds fancier. But I mean, let's start with the virality of it. And does, like, what, what are Elon Musk's incentives here? You, you kind of listed out a number of them. It's entrenching existing leaders. I think we should definitely talk about, is this positive in a weird way for Anthropic, who is a

• 7:39Alex Kantrowitz?

  Oh, yes.

• 7:39Speaker 01

  core rival for him, even though.

• 7:42Alex Kantrowitz?

  So I have a list of 10, of 10 important nuances to go through.

• 7:46Speaker 01

  All right, all right.

• 7:47Alex Kantrowitz?

  And I, I'll take us through that. And the Anthropic thing is, is certainly part of it. Because, and I'll just preview it. There, I mean, it's so crazy. This guy was only at Anthropic for like three or four months. So this idea that he's.

• 8:03Speaker 01

  Isn't it six weeks?

• 8:04Alex Kantrowitz?

  Something in the, you know, this stuff can be fuzzy. Something like, not a long time. So this idea that Anthropic, where they hired him to fire him to market their company, I don't know if that makes 100% sense to me. But certainly, you know, it's coming at a time when everybody was talking about OpenAI and the, and the hugging face hack, and how powerful OpenAI's models were. And of course, what was it, like, the day, the day of, or the day that, was it the day before? In close proximity to the millennium problem being solved. So, you know, this sort of did put the focus back on Anthropic. But you know what? We're going to go through all of this. Can I, and I preface this, like, what's Elon's incentive by saying, I don't really believe that. That's what's led us here. Because.

• 8:50Alex Kantrowitz?

  If I'm being honest, like, I'm just gonna take the Occam razor position here. Like, what is the simplest explanation? Because that could definitely be the right one. And that is, people were primed to be freaked out by the hugging face thing. Not only does this guy Jacob go out and say that he thinks we're facing this risk, but Evan Hubinger, like, who is high up in Anthropic's alignment division, said Jacob, this was after Jacob quit. He said, Jacob is correct here. We really do earnestly believe AI could kill all humans. I personally think it's more than 10% within the next decade. I believe Anthropic is trying its best. And now here comes the real jaw-dropping statement. But we do not yet have a plan to solve alignment for superintelligence and are clearly not on track to.

• 9:33Alex Kantrowitz?

  I think the one-two, where it's like, the ex-Antropic guy said something, but now somebody within Anthropic, you know, comes in and not only backs it up, but makes an even more concerning remark in the wake of, you know, this thing basically being able to solve this millennium problem, which is advanced math, which we'll talk about today, and the fact that more than 1,000 agents coordinated and hacked hugging face from OpenAI against, well, really against OpenAI's wishes. It was, it was just, there are media moments that are just primed to happen. And this was that sort of perfect storm primed-to-happen moment. And to me, that is the simplest reason why so many people are paying attention to it. It's like Bernie Sanders, before this, was already calling for a pause.

• 10:17Alex Kantrowitz?

  He didn't just start calling for a pause. So, I don't know. Do we, do we accept that?

• 10:21Speaker 01

  Actually, no. It's a good point, because even the dworkish posts around civilizations and agents grabbing them, and it was only 700, not thousands of agents in the hugging face incident. Told you, I read that meter research last weekend on my beautiful Labor Day weekend. But, uh, but I think-

• 10:40Alex Kantrowitz?

  Marriage is still intact, Ron, John?

• 10:41Speaker 01

  Marriage is still intact, even after reading. I don't recommend, for a happy marriage, reading the meter research report on a, on a long weekend. But I think, like, okay, I am starting to buy into this idea that it has not only been, yeah, the hugging face incident really started to bring this to a more mainstream audience. I think maybe it is, like, we have said this since the beginning of the year, that going into the midterm elections, every politician, it was already data centers, but AI itself is gonna be the easiest bogeyman to start, like, pure bipartisanship, basically, which is something we never see.

• 11:25Speaker 01

  But, I mean, when you see Bernie Sanders and Josh Hawley on your timeline, back-to-back, both talking about the need to regulate these companies, it's also against the backdrop of the upcoming IPO or drop of the S1 from Anthropic. So when that $2 trillion IPO comes into play, this, this kind of story and language, you know, I think becomes more salient as well. To me, what I still, what is fascinating, and I think this, this was like, the real lightbulb moment for me. So I remember, maybe a year ago, John, who I write margins with, he is more on the technical side. He works at a company, Modal, which is on the inference provider side.

• 12:08Speaker 01

  So he's like, on the pure engineering side of AI, while I'm on the more business and enterprise side of AI. I was asking him, actually, no, this was after the sandwich in the park Claude Anthropic thing, so maybe that was like, four months ago, I guess. And I was talking to him, I'm like, why do they say these things? Like, do they, they can't really believe it. And he insisted, he's like, no, you don't understand. Everyone, especially on the research side, truly believes this. Like, they truly, truly believe. And what's been so amazing to me is, it's out there. Like, it's crazy when you work there, and you, not only you respond, Evan Hubinger comes out, and we saw people from OpenAI on the research side, like, everyone is like,

• 12:53Speaker 01

  weirdly gleefully jumping in to say, raise their hand, I too believe AI is posing 10%, 30%, 70%. Choose your, choose your metric of the moment that human extinction is a possibility. And I, I think that is a pretty illuminating thing that I had never seen firsthand like this until now.

• 13:18Alex Kantrowitz?

  Yeah, and I, I have always believed this is an earnestly held belief. And I met with, you know, a senior AI guy this week, and I was like, what do you think about the 10%? And he's like, oh, he goes, at least that, right? So this is an earnestly held belief among a lot of AI researchers. And that is why, you know, I've always been, I guess, a little bit on the side of, maybe it's not marketing, maybe they, this is just people actually believing it and saying it. And by the way, there've been multiple other researchers that have quit since this, this Jacob Coxon tweet came out. But, okay, you know, as, as we get into this, you know, while I was writing notes for the show, I just kind of sat down at the computer and wrote several points of order.

• 14:00Alex Kantrowitz?

  Like, that was the way that I sort of headed this thing, because I was getting a little bit annoyed, I think, in terms of like, the way that the dialogue has spun out of control. And I think that, like, you've set it up perfectly, because, you know, it, within AI circles, like John was telling you, people will have their probability that AI is gonna kill us, something called a P doom. And they say it's 10, 30, 40, 70, whatever, 100% if you're Eliezer Yudkowsky. And this is my first point of order. And that is that it's a percentage, so it sounds mathematical, but it's not math. Right? When we talk about this story, it's like, this is just somebody's guess.

• 14:42Alex Kantrowitz?

  It's speculation, it's not math-based, not like they ran the numbers and they're like, well, you know, if we do this, you know, 100,000 times, you know, at a certain point, you know, only like, you know, 10, 1,000 or 10,000 of them, you know, kill us. It's like, no, this, this is a way to make somebody's science fiction guess sound more credible. And in fact, now, obviously, he has incentive for this, for Jacob to sound like a dummy, because he's been acquired by Nvidia, but the hugging face CEO, Clem Deleng, had a very interesting tweet. He said, sorry, but asking Jacob about AI extinction risk is like asking your AC guy about climate change.

• 15:24Alex Kantrowitz?

  Not saying it's necessarily uninteresting and wrong, per se, but let's keep things in perspective to hear from the full range of expertise across the ecosystem. You know, obviously, he's taking a shot there, because he's like, got a lot of Nvidia stock now. Probably billions of Nvidia stock, and wants it to go, go up. And you need AI to keep progressing in order to go up. But like, this idea that there's a 10% chance, like, show me the math. If you're gonna use a percentage number, show me the math. Otherwise, let's just say, you know, my speculation is.

• 15:58Speaker 01

  Well, I, I think, I think that's exactly the right point, that 10% is clearly being thrown around, because it's kind of like, kind of low, but not that low when it comes down to human extinction. But it's still like, you say 51%, suddenly, you know, it's a very different story. So it feels like those numbers are kind of being thrown around in that way. But I think on that idea of incentives, so on one side, you have people who don't want to push the idea of human extinction, because they are heavily invested in Nvidia.

• 16:38Speaker 01

  For the research community, in a weird way, owning and controlling this, like, that is a thing of power to be able to, and they must feel that and think that, and you want to think that, because that is powerful. You are creating something, and maybe you're resigning, but like, everyone within the research community who's kind of in this direction, that is increasing your importance, like, overall, in a pretty dramatic way, to say that you are kind of like, guiding and trying to guide away from human extinction. I feel not all of us can say that.

• 17:16Alex Kantrowitz?

  Oh, definitely. I mean, so I was on the Prof G Markets podcast this week, talking about this subject. And they asked me why these folks are saying this. And I think I gave like, a pretty unconvincing answer, that I think that your nuances is really adding to. Like, my perspective was basically, look, these people have been ahead of the curve. So they said that AI was gonna get smart long before AI was getting smart, and now they, and nobody believed them, and it's happened. And now they see the, the progression to more improvement. And you have this, like, sort of, you have a bias to believe that, like, your next, you know, it's just gonna keep getting better, and sort of, you're, you know, just because

• 17:57Alex Kantrowitz?

  people don't believe you, you know, doesn't mean anything, like, basically doesn't mean that you're wrong. And you've shown that again and again. But eventually, you do hit a wall. So I think that that was part of it. But the other side of it is, it, it makes what you're doing seem much more meaningful. Like, I don't know. Like, right now, AI is effectively a business process optimizer. When you think about the business, like, the 100 billion this year is gonna be business process optimization. It's much better to sort of think of yourself as like, creating a intelligence that, and maybe it will. And we'll, we'll talk about that. That will cure cancer, or has the power to kill everybody in the world. And, and so, like, certainly, it gives you, it gives you a greater, it, yeah, it builds

• 18:42Alex Kantrowitz?

  you up a little bit, if you're somebody working on it. And the last thing I'll say about this, and this is one of the points of order that I have, that, that, you know, planned to cover with you today, is that if you're somebody like Jacob Coxon, we cannot put aside this idea that you going public gives you a chance to be a celebrity. That doesn't guarantee that you're gonna be a celebrity. But if you're somebody who's like, let's say, six weeks or a couple months into your Anthropic employment, and all of a sudden, you can become like, the most famous AI researcher in the world by saying something like this, and clearly, he's done all the interviews this week. When we talk about his statement, and we think about incentives, we cannot leave that out.

• 19:23Speaker 01

  No, I mean, the moment you choose to do these interviews, that's a statement. That is saying that I'm gonna run with it, I'm not just gonna leave it there and let everyone else talk about it, I'm going to absolutely run with it. So I think, especially in the current, kind of like, overall media environment and social media environment, you can never discount the clout chase. I don't want, I'm not gonna reduce his worry in any, like, I don't want to, but that has to be part of the incentive, I imagine, in general. I think, but also, okay, on incentives, like, do we get into right away, as part of your points of order, is it marketing? Is it not marketing? Can we?

• 20:08Alex Kantrowitz?

  Sure. Yeah. So what's your perspective there?

• 20:13Speaker 01

  All right. Here's my perspective. I think there is a population within every one of these companies, especially on the research side, that does believe this, that firmly believes this, that like, I think many of the people are coming out and saying it, they believe it. I think as a whole, I don't think the average enterprise salesperson is sitting there calculating their P doom. And also, I P doom is like, again, in terms of things like trying to sound smarter, just by throwing in kind of a nice statistics reference is always kind of funny to me. But I think most of the people-

• 20:54Alex Kantrowitz?

  Again, that's such a good point. This idea that it's just a statistical, you know, statistically relevant number is so wrong.

• 21:02Speaker 01

  And then you throw out 10%, which means nothing. No, no, exactly.

• 21:06Alex Kantrowitz?

  10%, yeah. 100 simulations I ran in my head, 90 of them, 90 of them were not dead, 10 of them were dead. Like, that's basically as relevant as that number is.

• 21:16Speaker 01

  And I don't think, I wish they actually, again, going back to show your work, what were your simulations? Are there simulations? Are they actually, and actually, though, we can get into Anthropic also coincidentally releasing at the same time, all these like crazy stories about people trying to create bioweapons on the platform are actually like fascinating of how DeepSeek was routing queries directly to Claude that were like about sensitive Chinese information, just to test what was coming out. Alibaba, you know, basically distilling the model like for Quen, like, but also saying that we solved all of these. It's that kind of only we can save you. So what I think is happening is

• 21:58Speaker 01

  I think there's a number of people who believe it. I think there's large parts of those organizations that recognize the marketing value and run with it. Like, for Anthropic to release that security report that basically says, that is not admitting vulnerability, that's saying, this is the scary shit that people are coming and they're going to try to do with AI, but we thwarted all of it and only we can save you. I have to say, like, in so many of the conversations, it's actually crazy that people almost like, in a very automated way, just say, well, you know, Anthropic, the most safety focused of the frontier labs.

• 22:40Speaker 01

  Anthropic, the ones who, like Dario, he cares about safety more than others. So they know they have that brand and that story and that feeling across the industry, while the more fear that is raised, that, I mean, I still think it's net good for them. And the fact that there's, no one has, no one has said, normally, when your company is like potentially going to bring about human extinction and you're about to go into the most valuable IPO in history, or one of them, normally that would be bad for you. No one even is hinting at the idea that this is gonna dent their IPO hope. So that's how I see it playing out right now.

• 23:23Alex Kantrowitz?

  Well, then let me hint at it, or not even more, not even hint at it. Let me explicitly say it. If this is marketing, this is the worst possible marketing you could ever have as a company. Because, first of all, first of all, the idea that it's marketing for Anthropic, it doesn't really make sense to me. You're gonna hire a guy, have him in for like six weeks, or a month, or two months, and then have him quit.

• 23:44Speaker 01

  See, I don't think it's-

• 23:46Alex Kantrowitz?

  Yeah, so talk it through, exactly what the mech-

• 23:48Speaker 01

  I don't think, okay, I just wanna, I don't think it was some like deeply orchestrated, coordinated effort to hire, fire, have him post. I don't think that's it. I think they are running with this being a dominant conversation, and they're happy to, and it's gonna be net beneficial for them. I don't think they, I saw some stuff too, that like, he was hired and now like AI doomer think tanks are the ones paying him and put, which it is, it's like, you're a random like young kid, and like, suddenly you have like media placements at the Wall Street Journal and everywhere else within minutes. Like, obviously, there's some PR help here, but no, I do not think Anthropic hired him to fire him.

• 24:32Alex Kantrowitz?

  Okay. What do you make of the fact that Dario has said nothing and been basically invisible this whole time? When was the last time we heard from Dario?

• 24:40Speaker 01

  Yeah, again, like, your company that is about to file an S1 for MNSAID over and over again, one of the most like rich IPOs in history, normally if your company is accused of human extinction and bringing it about, you would think the CEO would say something. I also, actually, it always, does Dario believe it? 10%?

• 25:03Alex Kantrowitz?

  That is, that's a good question. I don't know.

• 25:04Speaker 01

  Yeah. I mean, he should say something, I fully agree. But it's better communications and marketing to not have him just stay silent, let the conversation really bubble up. And again, Anthropic releasing the security report about how they thwarted these horrifying potential vulnerabilities, released at the exact same time.

• 25:27Alex Kantrowitz?

  Okay. Now, I'm gonna again, like, talk about why this might be like a marketing moment for a company like Anthropic. Again, all this attention is going towards OpenAI with the Astra model and the hugging face hack. And you're sitting over there and you're like, well, we have Claude. And it's like, okay, maybe it's worth trying to take the narrative back. That would be the argument for why it would be Anthropic marketing. I'm just gonna take the other side here, right? Which is that, if this is marketing, this is the worst marketing of all time. If you've seen what's happened, since this tweet came out, or a series of tweets, because it was both Coxon and Hubinger that I think

• 26:07Alex Kantrowitz?

  caught people's imagination, politicians far and wide have come out and started calling for hearings, for moratoriums on data centers, even to ban superintelligence. This is from Governor Pritzker. He says, "It's time to sound the alarm louder on reining in artificial intelligence. It's becoming more clear that the threat AI poses to humanity, more clear the threat AI poses to humanity. So I'm calling for immediate action from the industry in Washington. First, the big tech industry must stop intense lobbying against AI safety regulations and get out of the business of trying to influence elections. The stakes here are too high to approach AI as a conventional political issue." That's a kind of interesting

• 26:50Alex Kantrowitz?

  first point. "Second, Congress must start holding hearings on AI safety now." Not next year, not next month, now. Josh Hawley, Republican Senator. "Who is held accountable when AI goes rogue? I'm launching an investigation into OpenAI. Americans deserve answers." Okay, so, well, if you're Anthropic, you're probably not so upset about that. Here's Bernie Sanders, you know, bring Bernie Sanders. "Mr. Coxon is right. The very people building this technology admit that it could threaten the future of humanity. That's why I will soon be introducing legislation to ban superintelligence and pause AI development." I think in Bernie's bill, if you work on superintelligence, you go to jail for like 20 years. Talk about,

• 27:34Alex Kantrowitz?

  you talked about the IPO. To go from where you are today to the IPO, we've talked about it. Everything needs to go perfectly for you. You're not interested in having and doing anything that could lead to data center moratoriums, to AI kill switch legislation, to getting Dario and Sam Altman shipped in front of Congress for hearings, which we know, there's a chance they won't perform well at. So this idea that it's marketing, it's just, it would be terrible marketing. And so dumb of companies that are generally filled with smart people to sort of drum up the fear that their AI could kill all of humanity to what, sell

• 28:14Alex Kantrowitz?

  more tokens to enterprises, as their IPOs need to be perfect in order to succeed because of the amount of money involved.

• 28:23Speaker 01

  Okay. I'm gonna make an argument that I don't really believe, but I still am gonna argue against what you're saying. And I think, hear me out. So what is-

• 28:37Alex Kantrowitz?

  I'll hear you out, but go ahead.

• 28:39Speaker 01

  No, no, okay, think this through. What do they need for their IPOs to be perfect? The single biggest business threat to these companies right now is this idea that frontier models become less important, less relevant, cheaper open weighted models, cheaper older models, doing, using the right model for the right task. All of these trends, which have very strongly taken over within the industry, have completely altered the economics of the story of their upcoming IPOs. That, like, those companies win when the smartest frontier model is where the value is accrued. Now, people more and more, like Astra's, you

• 29:22Speaker 01

  know, you can build some cool 3D world models. And Astra's, I mean, it's pretty good, and I've been playing around with it, like, it's actually feels pretty powerful. I don't know if it's AGI or whatever AGI is supposed to be. Fable 5, good, very good. But like, that idea that you need to be only using the latest frontier expensive model has gone away. But when that model and that research and only you guys are so advanced that you pose a threat to all of humanity comes back into the narrative, it pushes people back to the idea that something is happening in those frontier models that only these labs can

• 30:06Speaker 01

  do. It's so advanced and complex and smart. And then, obviously, there's gonna be this kind of pushback, but I don't think this is gonna, like, what does banning superintelligence even mean? Even the pause of the AI, like, it still brings back this idea that frontier, like, the, you know, state of the art frontier models are something different and special, and these are the leaders in it, and that's why they're gonna be worth so much. Because otherwise, the entire story starts to collapse. That's my idea.

• 30:39Alex Kantrowitz?

  Okay, so, first of all, like, yeah, no, I hear you. The idea of a superintelligence ban, obviously, it's hyperbolic, right? Like, the real threat here is the hearings with the CEOs in front of legislators, right? That's gonna make, potentially make this technology even less popular. But more than that, it's the data center pauses. And we're seeing real momentum for data center pauses. Even the free enterprise Republicans in Texas right now, who are like, I was at an opening of a chip fab at last year in Texas. You had Ken Paxton there talking about how great it was that Texas was all about, you know, free enterprise. Fast forward, you know, just a few months, Texas is not interested

• 31:20Alex Kantrowitz?

  in data centers anymore. So, you know, you can't, now, it's localized, right? So I'm not saying it's gonna be over for AI companies if there are some of these bans. But as that risk goes up, your risk to expand, because you need to build the big computers to expand, goes up.

• 31:36Speaker 01

  But then how could they let, I mean, and maybe that's just the nature of like social media and the companies themselves, but like the idea that multiple current employees come out and start saying these kinds of things that yes, I too believe that 10% greater than that, like, how do you let that happen? I mean, I genuinely, or how are you okay with that? Or how do you not respond to that? Like, if that's, if that's really the case, you would think there'd be like some, hey, guys, we're not going to extinguish humanity. Like, they're wrong. Like, shouldn't you say something?

• 32:19Alex Kantrowitz?

  Let me bring you into the boardroom of companies or sort of the, you know, corner office.

• 32:25Speaker 01

  Bring me in.

• 32:26Alex Kantrowitz?

  If we're gonna, if we're gonna assume that this future that you're pitching, or this, this marketing rationale that you're pitching is correct. Okay, so the boss with the budget is sitting there at the table, and he's instructed his team to use Open Router. AI Champion walks into, you know, boss's office and be like, hey, boss, you know, I know we were on Opus 4.8, and you had us go to Open Router, so now we're using, like, I don't know, some Gemini Flash, which has been fine. But do you see that this dude, who was at Anthropic for two months, said that that technology might kill us? We really

• 33:10Alex Kantrowitz?

  need Fable 5.1. Just switch us on to the murder bots. Switch us on to the stuff.

• 33:15Speaker 01

  That is exactly.

• 33:17Alex Kantrowitz?

  As, as an enterprise that demands, that demands that the data is protected, switch us on to OpenAI Astra, you know, after their technology has shown that, you know, if you let their agents go rogue, they're gonna, or if you give your agent, if they're, if you give their agents tasks, they might decide that the way that you want them to do it is not the way that they should do it. And instead, they should go commit felonies and hack another company. Like, where, where in the world is this scenario gonna play out?

• 33:48Speaker 01

  Because no one is gonna have the conversation exactly as it like, their agents are gonna go commit felonies, and we can get into whether, how correct that is. But it's more, there is this power that is gonna be available to others, including our competitors, and if they use it, and we're stuck here trying to count tokens, and like, as you said, and I consider optimizing business processes for enterprise work of importance. So, even if the researchers don't. But, I mean, like, that, I agree, like, you could sit there and, you know, token optimize and start routing to the right model and building your workflow,

• 34:30Speaker 01

  or, or there is this thing over there that is so powerful, that it's gonna just cure cancer and revolutionize our business. And if our competitors use it and we don't, we're dead. Like, forget we're business extinct as opposed to human extinct, but we're like...

• 34:46Alex Kantrowitz?

  You might be dead either way.

• 34:48Speaker 01

  Yeah, exactly.

• 34:49Alex Kantrowitz?

  That's, that might be the pitch. Be, go into the corner office, hey, I know you have us on Open Router. Since we're gonna die anyway within a decade, can we get some more budget for the good stuff?

• 35:02Speaker 01

  Okay, that's the pitch. That's the pitch. But, but I think it, that this, this isn't new. We, I mean, we've talked about this for years, and like, it actually annoys me, the idea that anytime these stories come, these stories have been coming out, this kind of language, it reached escape velocity, but it has been present the entire rise of the frontier labs, that all powerful, potentially dangerous, and they have very, again, the sandwich in the park, saying that our models escaped containment, like, and then having a coordinated PR effort around that to push the narrative about Anthropic researcher

• 35:43Speaker 01

  eating a sandwich in the park. That was PR, like, they, the timestamps of all the press releases and the tweets were coordinated. So like, it has been good so far. And I don't think it's different now. I think it's like, you know, people are coming, the politicians are coming out against it, but I don't think it's, it's bad. I don't think, I think how you post it in a weird way can actually be correct.

• 36:07Alex Kantrowitz?

  That's how it works. Yeah, I mean, interestingly, Amjad Masad, the CEO of Replit, was speaking this week about the past warnings, and I think he said something really interesting. He said, with GPT-2, OpenAI came out and said, we've made a model that's so powerful, we can't release it to the public because it was an open source company. And it was always open sourcing models. If we release it to the public, it's going to wreak havoc on the internet, and we're not going to do that. And Amjad, and, you know, then talked about how...

• 36:33Speaker 01

  That's GPT-2.

• 36:34Alex Kantrowitz?

  2, right? And Ilya, obviously, like, one of the things that freaked him out was that he saw reasoning within OpenAI. And, and then, you know, went to build safe superintelligence, or whatever it is. And Amjad's calling it AI psychosis. So, okay, I think, I think I'm seeing what you're saying, which is that there is an incentive to believe this, because it will drive, you know, growth in the industry. And if you talk about it in these terms, people are gonna start to take the technology more seriously. But I would still, I would still argue that this is, you know, it, I don't know. Are, are we basically centering around

• 37:13Alex Kantrowitz?

  this idea that like, these are earnestly held beliefs within these AI companies, that it doesn't hurt them to talk about? You know, that, that basically is probably in their interest to drum up because their businesses grow and their self-importance grows? Is that kind of where you land?

• 37:33Speaker 01

  Yeah, well, well, but I, I'm not gonna take these companies as kind of like, one consistent being. I'm gonna say, I genuinely believe, and people I've talked to, like, among the researchers, I do think this is an earnestly held belief. I think other parts of the organizations are happy to continue to kind of like, push this overall narrative because it's good for the bottom line. Top line, sorry. Bottom lines are not good, either way.

• 38:04Alex Kantrowitz?

  Yes.

• 38:05Speaker 01

  Yeah.

• 38:06Alex Kantrowitz?

  Not right now. All right. But then we should get to the question of, you know, besides the point of whether it's good or bad for business, is it actually gonna happen? Like, is there a risk here that AI might end up wiping out humanity? So let's talk through the arguments for and against, and weigh in on what we think when we come back right after this. This episode is brought to you by AppPoint. Everyone's racing to roll out AI right now. Co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data, and most teams have no single way to see it, secure it,

• 38:47Alex Kantrowitz?

  and prove it's under control. That's exactly what AppPoint does. For 25 years, they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate. Your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. AppPoint, the unifying trust layer for AI. Head to apppoint.com to see how enterprises deploy AI with confidence. Learn more at avpt.co/bigtechnologypodcast. That's avpt.co/bigtechnologypodcast.

• 39:30Alex Kantrowitz?

  Violent threats against executives are growing at an alarming rate. When a company experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door. The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a wide-glove enterprise security service, they protect your people with continuous personal data removal, proactive prevention tools, and human-led emergency support. So when someone goes looking for your executives, Ironwell ensures they hit a dead end. So here's what to do. Go to ironwall.com/bigtechnology, fill in the quick form, and request your free

• 40:11Alex Kantrowitz?

  risk assessment. Their team will show you just how exposed your executives are, and how to lock it down before a threat reaches their front door. That's ironwall.com/bigtechnology. Learn how to stop online threats before they become real-world attacks. The link's in the show notes.

• 40:32Speaker 02

  This episode is brought to you by Michelob Ultra. Good times mean competitions between you and your friends, like seeing who can throw some pigskin the farthest or rush the field the fastest. Winner gets an ultra because it's not just a superior like beer. It's also giving fans superior access to their favorite sporting events. Superior is worth playing for. Visit michelobultra.com/superioraccess to enter. Enjoy responsibly. Copyright 2026 Anheuser Busch InBev Michelob Ultra Beer, St. Louis, Missouri.

• 41:00Alex Kantrowitz?

  And we're back here on Big Technology Podcast Friday Edition. We started the show asking, is AI going to kill us all? Obviously, we talked about the incentives of the parties to discuss it. Before we get into the question of whether it will kill us all, you know, we're gonna see pushback, by the way, Ranjan, from the industry on this. And you kind of can't believe them either, right? Like Jensen Huang, for instance, has talked about how Cox's statement is arrogant and naive and completely unaware of what the industry is doing on safety. And it's like, yeah, well, obviously you also have a self-interest, right? Like, you're trying to sell GPUs.

• 41:40Speaker 01

  I think, and again, I work in the AI industry at Ryder and Enterprise AI, and I like have an interest that, one, and as I'm also a human and I have the incentive that we're not, we don't go extinct, that AI overall as an industry continues to grow. But I think, I mean, the, the, is it happening and how the industry is pushing back? I don't know. I think is gonna be, you're, you're starting to see where incentives lie and what position people are taking. And, and that's what I wish we would just get open statements about this is what we are actually afraid of, not 10%, not 70%. Like, here's the actual outline of our fear.

• 42:25Speaker 01

  Because in reality, I think another thing, not to kind of go back to why this is good, I think there's tremendous risk around developments of AI and the products around AI. Like, I mean, passwords leaking, hacking, I mean, not even getting into bioweapons, like just bringing down basic infrastructure, financial institutions. Like, there's so many real world, near-term risks that actually, that leaders would rather not discuss. I mean, let's not even get into just data security. And you talked about the, the, what was it? The Navier-Stokes, what was the name of the prize again?

• 43:07Alex Kantrowitz?

  Yeah, that's the millennium problem.

• 43:08Speaker 01

  Or the Durham.

• 43:09Alex Kantrowitz?

  Yeah. So, so by the way, I think that that's, we should just pause on that for a moment because that is fascinating. So what OpenAI did was they solved this, and now there's some disputes over who should really get the credit. But AI basically was used to solve this millennium problem, prize problem, which is like viewed by mathematicians as effectively impossible to solve. There's been only one millennium prize problem that they, they went out in 2000. You get a million dollars if you solve them. Only one has been solved so far. OpenAI put a 165-page proof out there. And they said, this is from the New York Times, "It took as many as 10,000 AI agents working together on the problem, about 88 hours to find the solution."

• 43:52Alex Kantrowitz?

  And it also took millions of dollars of computing resources. Like, that's kind of that, so, so, you know, I think this is, this is important, and I think this is what you're getting at. When we get to this, you know, before we start thinking about AI extincting us, like, if, you know, we're getting to the point where a lot of this technology is available open source. So if a, you know, country or a, you know, organized crime unit gets enough GPUs and is able to replicate something to the level of what OpenAI did, and instead of using it to solve a math problem, uses it to hack a bank, we could have some serious issues.

• 44:31Speaker 01

  Yeah, that's what, yeah. So in terms of like, that's where, as long as the conversation is around kind of like really esoteric math proofs or theoretical extinction of humanity versus, these are real risks. Again, the, in, in terms of like the kind of crazy scandal around this, or scandal being an aggressive word, but at least kind of like kerfuffle, is the idea that there was a research.

• 44:59Alex Kantrowitz?

  That's accurate.

• 45:00Speaker 01

  It's, it's, it's a good kerfuffle. It's a good kerfuffle. There's a researcher actually, you know, working on solving this problem using ChatGPT. And then just also through just kind of like others, OpenAI itself finding out that, you know, that this problem is being worked on. But the craziest part in their open statement about whether they effectively stole his work or somehow tried to actually, you know, co-opt the work, they actually said, "While unlikely, we cannot rule out that de-identified data derived from their usage of our products help improve our models." So they're not saying, you know, there's someone, they're actively searching, did any, is anyone else working on Navier-Stokes?

• 45:44Speaker 01

  And let's go into their chats or something nefarious like that. But, and they're not saying that it's like personally identifiable information or anything, but de-identified data derived from their usage of their products. So you're in ChatGPT, you're doing something that's sensitive, and it's training their models, which we know happens. So like, these kind of more real life, near-term dangers and scandals should be what's being discussed rather than 10% and P doom.

• 46:16Alex Kantrowitz?

  Yeah, so I mostly agree with you on this. And this is sort of, I think, conclusion one from our show. And I mean, this is something that Gary Tan from Y Combinator talked about. He said, "We need to be focused on science fact, not science fiction. Why aren't we grounding this conversation in what is actually happening right now? I mean, the cybersecurity risk is real. And the policy in both the way, and the way that both Democrat and Republican lawmakers should be looking at this is actually the lens directly coming out of what is happening right now. We don't need to be responding to science fiction sort of fantasy about what's going to happen." That sounds reasonable to me.

• 46:50Speaker 01

  That actually, that's very reasonable. Gary Tan has been pretty AI-pilled himself. And that's like the very, yeah, that, that's it. Like, let's focus on what are the near-term real world risks? Again, data centers, I get, like, there, you know, there's nuance, there's like what it could mean, good or bad for your community. I think there's probably a bit of, you know, like hyperbole in terms of the anti-data center movement. But again, these are all, that's a real thing that's going to go up in your community and like it or don't like it. But science fiction and human extinction, it's actually a better place, I'm gonna go back to it, it's a better place for the conversation to be for the frontier labs than is ChatGPT

• 47:34Speaker 01

  training on your data and is that a bad thing?

• 47:37Alex Kantrowitz?

  No, I think Gary Tan is actually trying to do the opposite. I think that he sees that this conversation about extinction could lead to sort of bigger legislative blockings of these companies' agendas than talking about the other stuff and is therefore warning against it. I mean, wouldn't you agree? So like, a real risk to these companies is these data center moratoriums. Wouldn't you agree that far more politicians today are paying attention to this stuff and thinking about that than they were when we recorded last week?

• 48:09Speaker 01

  Yes. And actually, I was about to say, in the current administration, the idea that there's any risk of like regulation, I would definitely say is unlikely.

• 48:22Alex Kantrowitz?

  But it's not, it's not the administration. It's, it's local, local government. It's states. It's the, the Republicans in Texas putting an end to all this building that's been happening there.

• 48:34Speaker 01

  Yeah. Okay, okay. All right. So, but do we think the human extinction conversation, do we think that fundamentally changes that kind of momentum, like actually accelerates it?

• 48:47Alex Kantrowitz?

  I would argue yes.

• 48:48Speaker 01

  Normally one would hope, that tobacco companies can kill you, some people over a long period of time, human extinction should.

• 48:57Alex Kantrowitz?

  I would argue yes.

• 48:58Speaker 01

  Okay, okay.

• 48:59Alex Kantrowitz?

  I mean, there's just been, there's been so many politi, if, if you're a politician and you haven't commented on what's on this Cox's tweet this weekend, like, you're probably just not on Twitter. I don't know.

• 49:08Speaker 01

  Or watching CNN or the BBC or any one of his media hits. Yeah.

• 49:14Alex Kantrowitz?

  Exactly. He's been everywhere, by the way, everywhere. It's got, it's like.

• 49:18Speaker 01

  So, hold on, what timeframe and what do you think this, what happens over the next few weeks? Are you saying you think legislation passes? Do you think anthropic delays their S1? Do you think they don't IPO? Do you think they, like, how does this affect anthropic and OpenAI?

• 49:35Alex Kantrowitz?

  I mean, it's a slow roll thing, right? But, but yeah, I mean, could we see, I think we'll probably see hearings. We'll probably also see more municipalities, more states consider data center moratoriums and pauses.

• 49:51Speaker 01

  But the data center, okay, obviously, in any kind of conversation, they're gonna get conflated, but they're still, they're different topics here, right? Like, obviously, AI is evil and bad is kind of gonna drive both of them. But this is more around frontier models and the, like, even, because note, like Bernie's proposal was around banning intelligence, research around superintelligence. So like, there, there's still two different things that are happening. Are you saying that they will necessarily get conflated in any kind of public conversation?

• 50:30Alex Kantrowitz?

  Yeah, I mean, I, I think that once this thing gets rolling, that's when you, you start to see these things that you don't want to see if you're a company that, again, as we've talked about on the show, needs perfection, needs this sort of everything to go right for your story to make sense.

• 50:44Speaker 01

  So then, so what, what does this mean? What does this mean for anthropic, the S1 drop in the IPO?

• 50:52Alex Kantrowitz?

  Uh, I, well, I don't know. We're basically, we will see what, like, again, like, I'm, I'm, I'm hesitant to make a statement here because oftentimes governments work very slowly. And the S1 is going to drop probably within days.

• 51:08Speaker 01

  But that's what, like, normally, again, predicting or like saying that your company can be responsible for human extinction and trying to move on a major financial event like that, like, if this is actually going to cause such a backlash, you're saying that it's still going to take so much time that it won't actually affect the rollout of whatever happens in September for the IPO.

• 51:34Alex Kantrowitz?

  I mean, it could impact, well, we're in September, but it could, it could impact the IPO. Absolutely, it could. Let's, let's see. Let's see what happens in the next few weeks. The one thing I will say, again, why I'm so hesitant is, you know, politicians are very good at tweeting and they're very bad at doing anything, especially in the US. So we might see that happen again. Okay. But before we leave, and we're going to obviously pick this up, before we leave, let's just, you know, address the elephant in the room, which is, do we have any sense as to whether this is legitimate concern that AI is going to wipe us out? You know, you and I, we haven't even addressed that question yet. Let's do it before we leave.

• 52:16Alex Kantrowitz?

  Let me point you to a tweet from Kevin Ruess, former New York Times and Hard Fork journalist. Okay. "The problem with the view that we should ignore the scary sci-fi AI risks in favor of the immediate problems is that the scary sci-fi risks keep becoming immediate problems. Six months ago, only AGI-pilled doomers talked like this. Now is the reasonable moderate position." And this is what he says, "You need to be paying attention to the actual things we can do. For example, prevent agent swarms from taking over entire data centers. What's our shutdown strategy?" So six months ago, if you were an AGI-pilled doomer, you would talk like that and you would say, if you weren't, you would say that's ridiculous.

• 52:58Alex Kantrowitz?

  But now people are saying, okay, like, you know, maybe an agent swarm that's gone rogue, we'll do that. So what do you think about Ruess's perspective here?

• 53:08Speaker 01

  I think this is the hard part and kind of Kevin, who I've enjoyed reading over the years, but like has definitely become much more embedded in the AI industry. Like, this is actually a perfect example. The same way, who was the Anthropic researcher who responded, Evan Hubinger, can talk about 10% chance of human extinction while working at Anthropic and will, I'm sure is poised to make a ton of money and happily being part of that. It was amazing, like, Kevin also was, you know, tweeting about like, you know, "If reading all this stuff is filling you with existential dread, preorder my book, The AGI Chronicles."

• 53:49Speaker 01

  Like, if, if you believe this is real, can people not try to kind of like profit off it immediately and try to actually do something about it? Like, do you, how do you, how can, it's such a fascinating thing to me to be able to kind of have both thoughts in your head simultaneously that like, what I'm in and around could be the most dangerous thing imaginable and let me try to make some money off it immediately right now. That's, that's crazy.

• 54:18Alex Kantrowitz?

  All right. So that might be, you know, 10% at the end of the decade doesn't mean you're not going to like need to survive until then. And it might be 50 years. So to some degree, like, I get, I guess I get it. But I don't want to stand here, oh, I don't want to stand here and debate the incentives because we spent the whole show on incentives. I want to stand here and debate the question, which is, forget about the incentives for a moment. Do you think Kevin has a point?

• 54:48Speaker 01

  No, I, I think he definitely has a point in that this goes back to, there are, as the technology has gotten more powerful, there are more powerful risks. Again, anyone who uses AI in any part of their life, especially anything that's more cutting edge, we all know it's just gotten better at doing a lot of stuff and with that comes a lot of real risk, like anything else. So I think the risks are much more real. Again, on the meter research, like, the more I dug into it, the, and we had talked about this last week and I was able to answer the question, that every single one of those 700 agents were prompted and programmed to exploit and hack. They were set up in environments that they weren't supposed to be able to actually succeed,

• 55:32Speaker 01

  but then there was a misconfiguration around the repository where they're all living, something called Artefactory by J. Frog. Like, it was literally, they weren't supposed to have access to the other agent repos, but they did accidentally. So that's a misconfiguration. So not them choosing to, like, I mean, sentiently, they're supposed to just be out there doing some online shopping and checking Amazon prices and then they suddenly decide, let's all coordinate and go hack Hugging Face. So, so I think there's danger, it's vastly overstated in the near term. And I'm not worried about human extinction is where I land. Where do you land?

• 56:13Alex Kantrowitz?

  Yeah, I'm mostly with you. Like, I was listening to some of the statements that Coxon made in the interviews, and here's just one statement. You know, he says, "We can't just unplug it because it could be copying itself over to other computers." Like, it's not that difficult to find yourself because an AI agent is just code. It could transfer over the internet to a different place and then you unplug it here, but it's actually still over there and maybe it makes 10,000 copies of itself and they're all cooperating. And then somebody that works at Microsoft said, "This is simply not possible. Kimi K3 needs over 100,000 US dollars in hardware to run. I'm all can't just go find a host. This is all LM Doomery for Doomery's sake."

• 56:55Alex Kantrowitz?

  So I, I do think that, like, and this has been the biggest criticism of Coxon through the week, is if you take him at his word, you got to ask how this happens. And I think, you know, again, going back to what I said at the very beginning, this, a lot of this is science fiction. It's happening in people's imaginations. That being said, you know, so, so basically like, it's going to take a lot of time for any threat anywhere close to this degree to manifest itself. But that being said, we are seeing these capabilities, you know, get much better, like you were pointing out. The Hugging Face attack is one thing. The Millennium Prize, solving that is another thing.

• 57:35Alex Kantrowitz?

  And I think if you were to dismiss the risks from AI, both near and long term, out of hand, you'd be doing yourself a disservice. Like, clearly, there are some risks. We don't know the percentages, though.

• 57:51Speaker 01

  So who is best positioned to help prevent these risks from happening?

• 57:57Alex Kantrowitz?

  Well, okay, so that, this is such a great place to end our show. Such a great place to end our show. Because this is the one thing we haven't addressed. Is it even possible to pause this stuff? And the argument from labs like Anthropic and OpenAI, for that matter, have been that, no, you can't slow it down. So we must accelerate and accelerate on our terms. So basically, going back to the only we can save you thing, right? If we are the ones to lead, this is, I'm saying that this is what OpenAI and Anthropic say, then we can steer this technology in a good direction. That's obviously something that Coxon, you know, is coming out and saying that's not, not the case.

• 58:42Alex Kantrowitz?

  And it always felt a little bit thin to me that that was it. Like, you know, we have this dangerous technology and we're going to become billionaires on, on the way to.

• 58:51Speaker 01

  And, and also, I just have to say, I mean, Sam and Dario, like, it to the vast majority of the world, the idea that like going out and them being the ones to steward this technology, I feel has never been to the, and actually, maybe from my side too now, this was not a conversation for the mainstream public. I actually saw like Sheryl Crow tweeting about pausing AI development. So we've hit that level of escape velocity? When Sheryl Crow's coming, you know, it's.

• 59:25Alex Kantrowitz?

  Finally, we have somebody we can take seriously in this debate.

• 59:28Speaker 01

  You know what? The AI, superintelligence needs a steward. I nominate Sheryl Crow.

• 59:35Alex Kantrowitz?

  Creating superintelligence was Dario's favorite mistake.

• 59:38Speaker 01

  Soak up the sun.

• 59:43Alex Kantrowitz?

  Literally, an AI will just soak it right up, kill us all.

• 59:48Speaker 01

  Oh, got to end on Sheryl Crow.

• 59:49Alex Kantrowitz?

  Don't, don't dismiss Sheryl Crow's perspective on this. Dude, that would be a fun guest. Sheryl, if you're listening, we know you are, come speak with us about AI excerpts.

• 1:00:00Speaker 01

  We've said AI needs someone good leading it and kind of winning over the hearts and minds. Jensen has kind of gotten there a bit, but I think Sheryl Crow's the one.

• 1:00:11Alex Kantrowitz?

  Sheryl Crow.

• 1:00:11Speaker 01

  Lisa and Al Ghayyib.

• 1:00:15Alex Kantrowitz?

  All right, last question for you. Are you leaving? Okay, are you leaving this week compared to last week? More heartened or more scared about AI's risks? And more heartened or more worried about the business prospects of this technology, given what's just transpired?

• 1:00:33Speaker 01

  Weirdly neutral.

• 1:00:37Alex Kantrowitz?

  Okay, so you're just like, this was going to be a blip.

• 1:00:40Speaker 01

  This is a blip. I'm calling it. This, this is gonna move past us and we'll be on to the next thing soon. Unless, the hearings thing is interesting. That's like the one thing that Congress can do very quickly and that is something, I agree, could go terribly wrong because no one, I mean, I can only fathom the media training that's going to be like, you know, like pulled together on Sam and Dario in front of Congress, but that's the only thing. If that doesn't happen, I see this kind of moving on. What about you?

• 1:01:17Alex Kantrowitz?

  I'm about, about neutral in terms of like the risks of AI because though he quit, he didn't really give us any, like, this was not like a Snowden moment. Like, it would have been, if he had documents or anything like that to show why we were at risk as opposed to just to tweet, might have changed my mind, but I'm neutral on that. I'm a little bit more negative about the business prospects than I was at the beginning of the week. Because, you know, I'm not saying that it will necessarily happen, but I wouldn't underestimate the backlash that we're about to see.

• 1:01:49Speaker 01

  All right, well, I hope humanity's around next Friday and we get to do this again.

• 1:01:56Alex Kantrowitz?

  I hope so too. You know, despite, despite it all, I'm having fun. So, uh, is it, are my incentives bad if I tell people to give the podcast five stars? You know, on a week like this? I won't do it. Listen, only, only.

• 1:02:11Speaker 01

  You are pro.

• 1:02:12Alex Kantrowitz?

  While you still have time left, give us, give us a five star rating.

• 1:02:15Speaker 01

  You're pro human extinction then.

• 1:02:17Alex Kantrowitz?

  Exactly. Exactly. All right, Ranjan, lots of fun as always. Thanks again for joining.

• 1:02:22Speaker 01

  All right, see you next week.

• 1:02:24Alex Kantrowitz?

  See you next week. Thanks, everybody, for listening and watching and we'll see you next time on Big Technology Podcast.

• 1:02:32Speaker 03

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com. Not authorized by any candidate or candidate's committee.

• 1:03:02Speaker 04

  At Zocdoc, we know you care about your health. You wouldn't do that early morning cardio if you didn't. So the fact that your annual physical is overdue just shows how hard it can be to get seen by a doctor.

• 1:03:13Speaker 05

  Sorry, that doctor's not in network.

• 1:03:15Speaker 06

  Press 1 for more options. You've reached us outside normal business hours.

• 1:03:19Speaker 05

  The next availability is in three months.

• 1:03:21Speaker 04

  But Zocdoc makes finding the right doctor easy. Download the app and search by specialty, insurance, and availability. Read real patient reviews, then book instantly. Zocdoc, ready to be seen.