What If the AI Revolution Isn’t Real?

Is AI a revolution, or just another slow-moving general technology? Are we mistaking hype for inevitability? And, can a technology this unreliable really reshape civilisation? Rory Stewart and Matt Clifford are joined by award winning computer scientist, Arvind Narayanan. To hear the full episode, sign up at therestispolitics.com Instagram: ⁠@restispolitics⁠ Twitter: ⁠@restispolitics⁠ Email: ⁠[email protected]⁠ __________ Social Producer: Celine Charles Video Editor: Lorcan Moullier Producer: India Dunkley Senior Producer: Callum Hill Exec Producer: Tom Whiter Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:04Rory Stewart?

  This episode is brought to you by NordVPN.

• 0:06Alastair Campbell?

  And I must thank NordVPN because on my Grand European Tour with Fiona, I was away from the Hawthorns for West Bromwich against Burnley, but I managed to watch it. There you go.

• 0:19Rory Stewart?

  I actually had a NordVPN thank you. I was in the new Starbucks here in Creve and of course the thing flashed up saying this is an unsecured public network. And I was able to go to my VPN and it was a little reminder of something we often forget when we're in all those cafes and things, which is actually genuinely people can do all kinds of weird cyber attacks and spying if you don't have a virtual private network when you're sitting in that Starbucks.

• 0:42Alastair Campbell?

  Well, I think, are you not already sort of proud of me that I managed to do this? Or should I say Fiona managed to do this?

• 0:49Rory Stewart?

  I'm incredibly proud of you. That's a tech transformation. So to get the best discount off your NordVPN plan, go to nordvpn.com/restispolitics. That's nordvpn.com/restispolitics. And the link is in the episode description.

• 1:05Speaker 02

  Hey, Bose here. On your next track, listen for the decisions. Distortion on the bass, the vocal right up front. Every choice deliberate. With Bose, you'll hear it all. Your music deserves Bose.

• 1:21Speaker 03

  This episode is brought to you by Welch's Fruit Snacks. Back to school season can be tough on parents. But choosing a snack you and your kids can both get behind doesn't have to be. Stock up on Welch's Fruit Snacks, made with whole fruit and now made with no artificial dyes. It's a snack kids already love and one you can feel good about giving them. Go back to school with Welch's Fruit Snacks, now made with no artificial dyes.

• 1:49Rory Stewart?

  Hi, Rory here. So Matt Clifford and I are back with another episode in The Rest is AI. And this week, we're back to a simple but pretty uncomfortable question, which is, might AI just be a normal technology?

• 2:05Rory Stewart?

  Something that maybe really doesn't deserve great fear and frenzy. And our guest today is Arvind Narayan. He's the Director of Princeton's Center for Information Technology Policy. And he's a really interesting voice. He's a challenge to Joshua Benjio, who you heard last week, because he thinks that it's not very likely. Or that actually we're over focusing on the end of humanity existential threat. But at the same time, he's a challenge to a lot of the people that are hyping the technology. Because he's saying, in many ways, this stuff is much less reliable than we think. And even once it gets more reliable, it's going to take decades for some of these things to be adopted. Not a few months.

• 2:49Rory Stewart?

  And therefore, the change that AI is going to bring is going to be much more gradual. So it's probably a very, very sane, thoughtful, challenging voice. One that maybe isn't heard as much because it doesn't necessarily suit either the end of the world is nigh people. Or the AI is going to change the universe tomorrow people. Because he's suggesting that a lot of the issues are just around how humans do or do not adopt technology. Here's a taster. And to listen to the full episode, sign up at therestispolitics.com. There's a really interesting thing as an outsider that one observes, which is one will get.

• 3:32Rory Stewart?

  You know, Elon Musk saying, you know, it's a 20% chance it's going to destroy humanity. And Sam Altman says, you know, it's going to end the world. But in the meantime, it's going to lead to some great companies with great machine learning. And then suddenly you'll get, you know, Gary Marcus will pop up and say, this is all completely overblown. There's only a 1% chance it's going to eliminate everybody, right? But of course, if one thinks about this, I think Joshua's point was it doesn't matter. It's a 1% chance, a half percent chance, a 0.1% chance. What the hell are you guys thinking about? I mean, you're gambling, right? You're at some level, you're taking a risk that you wouldn't take with nuclear waste in your back garden. I mean, you wouldn't be reassured for me to say, don't worry, there's only a 0.1% chance that this nuclear waste is going to wipe out your family. You'd be like, what the hell are you doing?

• 4:17Rory Stewart?

  Stop, right?

• 4:19Speaker 04

  One thing I strongly believe is that we should not be thinking about this in terms of probabilities. At the moment, you're arguing about what the probability is. You're already down a very confusing path that can only lead to, I think, misleading guidance. So I've looked at the most sophisticated efforts that we have for estimating these probabilities. It was led by the Forecasting Research Institute. I actually work with these folks. I know them well. They're incredibly smart and they did a really well thought out effort to get dozens of expert forecasters to discuss, try to change each other's minds and provide these probabilities.

• 5:03Speaker 04

  And they put out a 753-page report. And I've read that 753-page report. And, you know, you can have a room with these so-called super forecasters debating, and you can have a room with a bunch of people who are high and debating what the future of AI is going to be, and you can't tell the difference. And this is no slight to the super forecasters, some of whom I know. They're incredibly smart people. But the thing is, we have no empirical basis for predicting what these probabilities might be. The arguments that people are giving, it's things like, oh, you know, AI might decide to colonize space instead of Earth. So even if we had super intelligent AI, maybe the probability is not as high as we think.

• 5:47Speaker 04

  Or someone else thinks, oh, AI might decide that killing all humans will make the planet cooler and it helps computer chips work better. And so maybe it will decide to kill all humans. And so, you know, they're listing a bunch of reasons like this, assigning some numbers to each of them, and then multiplying all of them at the end, right? So this is the best method that we have. So these probabilities are all bogus. And that's my strongly held view. We should not think in terms of probabilities. I do think the risks are potentially real. I'm not advocating for ignoring the risks. But I think the right response cannot be, let's try to stop all this. There are two big problems with that. One, it's just not going to work. The only way it could work is if you have an authoritarian world government

• 6:31Speaker 04

  that can control every AI developer everywhere.

• 6:34Rory Stewart?

  Can I pause on that, Avin, for a second?

• 6:36Speaker 04

  Yes.

• 6:37Rory Stewart?

  That's an empirical claim. I mean, at the moment, these large language models basically can only be run by some of the largest wealthiest companies on earth.

• 6:47Speaker 04

  Not true.

• 6:47Rory Stewart?

  With enormous data centers with enormous compute power. So it seems plausible at the moment that if President Trump, driven by Christian nationalists, and Xi Jinping wanted to simply shut down the large language models of two Chinese companies and a small handful of American companies, we would cease to have these L L Ms operating pretty quickly.

• 7:14Speaker 04

  Uh, this is not true at all. It might be true that absolutely the most powerful frontier models can only run on powerful GPUs, but you have, you know, slightly smaller models that are maybe one step below that can run on consumer-grade hardware. And the cost of running these models is dropping by something like somewhere between a factor of ten to a factor of hundred every year. The cost is dropping very rapidly, both because the hardware is getting faster per dollar and because algorithmic improvements are allowing us to squeeze more juice out of smaller models.

• 7:48Rory Stewart?

  Again, I suppose to to push this argument to the next phase, the the claim only needs to be that the existential risk is posed by the frontier models. And if one could shut down the full infrastructure that powers the frontier models, then one has less to worry about from the models that you're talking about.

• 8:04Speaker 04

  Now, again, I completely disagree with this. I think historically, this is very easily falsified. When OpenAI built G P T two, which was, uh, you know, two generations before Chat G P T, which was when the world started noticing, they thought that model was so dangerous that they were not going to release it for people to uh to download and use. And that's something that my grad students uh can build today, you know, just for fun and learning uh in a day or two, right? And so historically, you know, when we look back, our ideas around what constitutes this threshold lever of level of danger have kind of been comically off. And I don't think there is really any clear relationship between the

• 8:48Speaker 04

  Power of a model in terms of how computationally heavy it is and what dangerous things it might potentially be able to do in terms of enabling cyber attack, in terms of various things that worry about, in terms of buyer risk. Actually, some of those models can be much, much smaller and faster than these large language models because those biological capabilities are not about language at all.

• 9:12Rory Stewart?

  To understand this first argument, this is a the cat is out of the bag argument. It's too late. The stuff's built. If these models are going to become deceptive killer robots, they're already out there and there's nothing we can do about it.

• 9:27Speaker 04

  It's not just that, that you can't even prevent these models getting even better in the future unless, again, you have this authoritarian world government that stops even small teams of developers from acquiring consumer-grade hardware and doing research to improve the efficiency of those models and then their capability.

• 9:46Speaker 05

  I want to say I want to go on the record as being against authoritarian world government.

• 9:51Speaker 05

  I mean, I'm very interested to sort of use this as a way to think about what we should do. And, you know, I'm a little bit biased because I'm one of the people in the UK who, one of many people who helped set up the AI Security Institute, which does these like pre-deployment evaluations in the UK government on security-adjacent risks on a voluntary basis. There's no regulation that requires the companies to do that, although they have agreed to do that. And, you know, like my view has been, to your point, I guess in agreement with what you're saying, Avind, like it's very, very hard to know, like, you know, when people say we should pause now, we should stop now. I, I, I get the impulse, but I'm like, well, how do we know? And I think like governments having better knowledge of what capabilities actually are, what it can and can't do feels like a very valuable thing.

• 10:36Speaker 05

  There's a lot of people that look at what we've done with, with that institute and said, it's amazing from that governments built that capability, but it has no teeth. But what do you think we should do? I mean, it sounds like you actually, you referred earlier to, you know, the idea of having like transparency requirements. Do you have a sense of like what the right regulatory framework is, given both your?

• 10:55Rory Stewart?

  Sort of skepticism of, you know, some of these scenarios, but also you, you know, you're concerned clearly about some of them as well.

• 11:01Speaker 04

  For sure, yes. And I think the AI Safety and Security Institutes have been a wonderful thing. I've strongly supported the work that they do, no question on that. And yes, I think transparency and knowledge are the most important thing for now, but I do think there is a lot more that can be done, that should be done. One big area is developing AI for defense against these very risks that people are worried about. And historically, this is always how it has worked. We're acting as if AI is a new thing, but AI has been gradually improving out of the public eye, admittedly, for, you know, decades now. And the reason that things have worked out is because of the attacker-defender balance and the efforts

• 11:44Speaker 04

  that we've taken to shift that balance in a healthy direction. So when it comes to cybersecurity, for instance, automated tools have been, in many ways, superhuman at hacking for decades now. And yet, the world hasn't ended. In fact, things have gotten a lot better, and that's because companies who are developing software use these very same automated tools to find and fix bugs before they put software out there, before attackers have a chance to take a crack at them. And that's the critical thing. When these AI models are now becoming even better at automatically finding vulnerabilities, do we have the right incentives in place to ensure that software developers, as well as mom-and-pop businesses who might be deploying

• 12:25Speaker 04

  software, have the know-how, have the funding to use this to protect their cybersecurity? Those kinds of things can benefit from government incentives and can be enormously beneficial.

• 12:37Rory Stewart?

  This moves us on to a new thing. So the first disagreement you've clearly had with Yoshua and Geoffrey Hinton and others who were calling for a stop is your cat's out of the bag. It's too late anyway. The earlier models can do a lot of damage. Stopping the next frontier model is not going to make much difference, argumentarily. The second argument that certainly people like me would make who are anxious about this is, for goodness' sake, what is it about the Trump administration, or the financial incentives, or the psychological personalities of...

• 13:10Rory Stewart?

  These people running these models that remotely convinces you that they're incentivized in any way to really put proper safety regulations in place, use the AI to counter themselves. Every time I meet them, they are in an arms race. They're saying, government back the F off, government doesn't understand what we're doing, they're never going to understand what they're doing, they don't know what they're regulating anyway, get out of our way, here we go, I've got to get there before, and then insert the name of somebody running one of the other models who they think is unbelievably dangerous and is going to destroy the world, and I've got to build my model before, insert Elon, Larry, Demis, Mark, right? And if not, I've got to build it for the Chinese, because I don't know whether the

• 13:54Rory Stewart?

  Chinese can build it, right? None of this gives me any sense that we are in an environment in which these people are, either they or the US government are getting remotely behind safety.

• 14:05Speaker 04

  I am not saying that we're in a good place right now with regard to regulation, both with regard to safety regulation in the sense of these catastrophic risks, as well as the kinds of safety issues we've already seen on a massive scale, some of the psychological issues that we've talked about, the use of AI models for making non-consensual nudes, these are problems that we knew were happening, the nudes one at least, at this point, for something like six years, and it took forever for policymakers to start acting on these things. And so the pace at which policymakers are responding is definitely inadequate. But I think at the same time, the arms race narrative is a little bit oversold.

• 14:48Speaker 04

  I think what we're seeing in a lot of cases is that when regulators and policymakers in one company are falling behind, the harms for the most part are felt locally. So for instance, you know, in the US, people are realizing that with respect to the safety harms around the psychological impacts of large language models, it's affecting Americans, and now policymakers, both at the federal and the state level. Are actually taking this very, very seriously. And I think we're going to start to see a mindset shift. And we're going to start to see policymakers stop buying this arms race narrative.

• 15:31Speaker 04

  And they're going to recognize that a lot of these harms are actually going to be felt within their own countries. And the way to regulate is not just at the level of the model developers, but how they're deployed and not just on the companies such as OpenAI and Google, but also everybody else who is using these models within their own companies. And try to get ahead of risks like people trying to hand off the running of their companies to to large language models, for instance.

• 16:01Speaker 05

  One observation I have on that from both my own time in government in the UK, but also spending a lot of time in Washington, talking to people is that, to paint with very broad brushstrokes, the executive tends to see this technology through a security lens. And there's lots of both institutional and other incentives that make that happen. And typically legislators tend to see it through a sort of, what you might call like an online safety type lens. And, you know, certainly when I was in in number 10 here, you know, we would have these high side conversations about AI and security. And then we'd go and see MPs who were worried about, you know, their kids or the kids of their constituents talking to chatbots. And I think I may have mentioned this on a previous episode, but you know, in 2025,

• 16:43Speaker 05

  I I went to a very interesting conference of faith leaders talking about AI, who totally rejected the security framing and really wanted to talk about, you know, what is this doing to kids and their minds? And I suspect that over the course of 26 and 27, we might see this sort of like quite executive, security dominated framing kind of actually diminish as as some of these cases in particular that you've referred to of and kind of come to the fore and and actually see legislators sort of be a bit noisier voices in this, talking about these sort of more sort of online safety type harms.

• 17:18Speaker 04

  That's exactly my prediction, yes. As these models get more diffused into society, we are going to see an increase in the salience of that set of concerns. And I think the good news is that a lot of the interventions, not all of them, but a lot of the interventions for both sets of concerns are similar. And so I think the momentum around one kind of safety issues can also be used to address the other kind of safety issue.

• 17:41Speaker 05

  Europe.

• 17:42Rory Stewart?

  If I was speaking on behalf of the Commission or European leaders, they would feel they have been making a good faith attempt to try to work out how the hell you talk about these different types of risks and how you present regulation.

• 17:55Rory Stewart?

  And the response that they have got from Silicon Valley, they feel, is one of unmitigated contempt. That, you know, Europe's stuck in the Middle Ages. This is grotesque regulation. This is just another sign of a kind of sclerotic, aging, useless economy. And we will use every tool in the power of the American executive and corporations to punish Europe for trying to regulate us. And we will use every lever we can to break those regulations open to let us in. And again, that doesn't make me feel that this is a culture that is very sympathetic towards people trying to think about safety or security.

• 18:35Speaker 04

  Yeah, I agree. I think Europe's approach is one that is democratic. It's not necessarily the only right approach. I understand why the US has chosen a different approach, but I think the hostility of American companies and the government to Europe's approach has been really problematic, and it's not one that I support.

• 18:56Rory Stewart?

  There's plenty more of that agreeable disagreement. To hear it, sign up at therestispolitics.com.

• 19:04Speaker 06

  There we are getting a statement from Delta 10 on that flight with 350 passengers on board.

• 19:13Speaker 07

  The great game, strategic struggle between Britain and Russia.

• 19:20Speaker 08

  We assess, Prime Minister, with high confidence it was Russian.

• 19:26Speaker 09

  Holding the Russians accountable is World War III.

• 19:29Speaker 10

  They took down a British passenger jet containing 350 people.

• 19:32Speaker 07

  Man in the Kremlin is playing, Alex. Got to ask yourself, do you want to play or not?

• 19:38Alastair Campbell?

  You're the Prime Minister. You've done nothing.

• 19:45Speaker 11

  We would need to understand the policy context.

• 19:47Speaker 07

  All the relevant context will be structured as a framework today. Disseminated into an operation order, or FRAGO.

• 19:56Speaker 12

  A FRAGO that says to kill the President of Russia?

• 20:00Speaker 07

  Yes.

• 20:03Speaker 09

  New from Goldhanger. Who is going to be able to pull this off?

• 20:07Speaker 13

  You're my soldier, are you? Sergeant Tom Macduff, Duffy!

• 20:11Speaker 09

  An audio drama journey like no other.

• 20:13Speaker 13

  Men like Duffy, they are our most valuable weapons.

• 20:17Speaker 09

  Devized as a technically rigorous geopolitical action thriller.

• 20:20Speaker 13

  We have two tight friendly KIA.

• 20:22Speaker 09

  Where Britain plays its strongest hand. He's the best. Against its strongest foe. Don't do it, Duffy! Bear Hunt. Available now wherever you get your podcasts.