How Is AI Being Used In The Iran War?

The Pentagon has given AI a powerful role in the Iran war. We dig into the DOD conflict with Anthropic, and the state of autonomous weapons.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Ira Flatow?

  Science Friday is supported by MathWorks, creator of MATLAB and Simulink software for technical computing and model-based design. MathWorks, accelerating the pace of discovery in engineering and science. Learn more at mathworks.com.

• 0:16Speaker 01

  WNYC is supported by Bear. Science is about asking questions, testing, trying again. It's not easy, but it's relentless. It evolves, and that's why science delivers. Learn more at sciencedelivers.com.

• 0:33Flora Lichtman?

  Hey, I'm Flora Lixman, and you're listening to Science Friday. The military use of AI is capturing headlines this month. The dust-up at the Pentagon with AI company Anthropic out and OpenAI in. Meanwhile, a new war where AI is in use. What do we make of all of it? We knew just who to call. Karen Howe is a journalist covering AI. She's written for The Atlantic and The Wall Street Journal, and she's the author of the book Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI, which takes you behind the scenes on the rise of one of the most powerful startups ever. Karen, thank you so much for being here.

• 1:12Speaker 03

  Thank you so much for having me.

• 1:14Flora Lichtman?

  You are deep in this world. I have this feeling in this moment of AI's power and reach sort of snowballing, but what is your impression of this time?

• 1:27Speaker 03

  Oh, my gosh, yeah. I mean, when I was working on my book and using the metaphor of empire to try and contextualize the sheer power consolidation that's happened within these companies, I was not envisioning, you know, the fusion of this technology with the military and the alliance between Silicon Valley and Washington. And it just, it feels startling that that metaphor has come to be the only metaphor that we can really use to understand this moment.

• 2:02Flora Lichtman?

  Well, it's not a metaphor anymore. It's literal.

• 2:05Speaker 03

  It's literal, yeah. Yeah, yeah, that's right. And I did not anticipate that happening.

• 2:11Flora Lichtman?

  Let's talk about the war in Iran. Do we know how AI is being used?

• 2:17Speaker 03

  There has been reporting from the Wall Street Journal and the Washington Post that says that Anthropic's Aramado Claude was essentially used to analyze a bunch of intelligence data and then identify targets to bomb. And the Washington Post specifically said that there were around 1,000 targets that it identified. And one of the things that's deeply disturbing about this is that large language models, which is the technology behind Claude, is a very faulty technology. It is not accurate. That's why sometimes when you are chatting with ChatGPT or chatting with Claude, when you try to get it to talk

• 3:01Speaker 03

  in more detail about like something that you have expertise in, it starts to even make up that it knows those things. And in the military context, like that doesn't go away. And so we had news reports about this horrific bombing that then happened to a school in Iran. And not just one bombing, but two. So when the first responders and parents rushed to the site to try and save anyone that was still alive, they got bombed too. And there is speculation that it's because Claude misidentified a civilian target as a military target.

• 3:41Flora Lichtman?

  Right. Though just to clarify, it's unclear if AI was to blame in the strike. And on Wednesday, US officials said that it was unlikely, according to New York Times reporting. So we don't know.

• 3:52Speaker 03

  We do not know. Right. But it is like such a legitimate possibility that it's like it perfectly encapsulates what is going on right now, that there's just like so much uncertainty and there's so little transparency, so little accountability in just extraordinarily grotesque actions that are happening and and like mass life death decisions that are being made, like under the veil of secrecy. It's it's just like, yeah, it's just really awful.

• 4:26Flora Lichtman?

  Isn't this one of the sticking points between Anthropic and the Pentagon? That they said Claude isn't ready for, and this is different, right? This is autonomous weapon use. So this is not just identifying targets. But also, the Pentagon seems to be using Claude, but also blacklisting Claude. Like, I'm confused.

• 4:53Speaker 03

  Yeah, there's so, there's so much going on. So Anthropic was the first company that got permission from the Pentagon to be used on classified intelligence systems. And so for, you know, the last, maybe around nine months, that has been true. And the Pentagon then became, it seems, quite reliant on using Claude. Because then, when Anthropic and the Pentagon started fighting over the fine-grained details of how exactly Claude should be used, the Pentagon then did like the nuclear option and was like, we are going to either force you to bend to our wishes, or we are going

• 5:39Speaker 03

  to declare you a supply chain risk. But after they declared Anthropic a supply chain risk, they're still reliant on the technology. So there's a six-month phase-out period. And like, hours after they declared that supposedly Anthropic is a threat to national security, they used the very tool that is bad for our national security in the bombing of Tehran. And so, yeah, that's like one layer of what's happening. But the other thing is, you know, people have been lauding Anthropic a lot for standing their ground. And I, there's like a deeply complicated aspect to Anthropic's role in this whole thing.

• 6:22Speaker 03

  So Dario Amade, the CEO of Anthropic, said that he did not want this current iteration of Claude to be used for autonomous weapons. But in a CBS interview, he said he Was perfectly fine on principle with autonomous weapons. It was just not like this version of the technology. And in fact, he had offered to co-create, co-develop autonomous weapons with future iterations of the technology. That's like one thing that complicates this whole thing. The second thing is like, I was speaking with Dr. Heidi Klaff, who is a chief scientist at AI Now, this policy research institute in New

• 7:02Speaker 03

  York, and she's been writing extensively about military and AI. And she mentioned like what Amade was saying is he's not okay at the moment with like the current iteration of Claude being just like no one popping in and checking, okay, like what targets have been identified, but he was okay with Claude actually being a decision support system. And so he was okay with it actually analyzing the data to identify bomb targets. And so the Pentagon is actually using Claude exactly in the way that Amade said he was fine with in the current iteration. And Dr.

• 7:42Speaker 03

  Heidi Klaff was like, if you think that your technology is not good for autonomous weapons, it should also not be used for decision support systems because we have extensive research that has shown time and time again that there's a huge automation bias with humans. Like when we see a chatbot or a robot do something or say something, we just believe it. And so even if you have a human that's like popping their head in and being like, okay, have you identified the right bomb targets? They're like, well, the bot, I mean, the bot is a computer and has analyzed all this data, so it must be right. Like, it's not a legitimate check.

• 8:23Speaker 03

  And so what is happening where people are speculating that the school was bombed indeed because of an error from Claude is exactly, exactly the kind of scenario that Dr. Heidi Klaff was talking about and exactly the kind of scenario that Amade was actually okay with.

• 8:40Flora Lichtman?

  Right. So this moral high ground for Anthropic feels also a little suspect, right? Like, is that what we're also getting at?

• 8:47Speaker 03

  Absolutely.

• 8:48Flora Lichtman?

  Yeah.

• 8:50Speaker 03

  The way of thinking about Anthropic in the AI world, it's like the clean coal of AI. Like, they fashion themselves as this ethical company that it really cares about safety and the well-being of people and so on and so forth. But like, the entire way that they develop and deploy their technologies is deeply problematic and very imperial. And so like, the clean coal, it doesn't exist, right? Like, you cannot have clean coal.

• 9:19Flora Lichtman?

  Okay, this is such a basic question, but the news that I've been reading has been describing these as LLM-powered weapons, large language model-powered weapons. And it makes me think, like, do I not understand what a large language model is?

• 9:37Speaker 03

  Yeah, I mean, I personally would not use the phrase, that phrase, because it makes it sound like there's like, a chatbot strapped onto a missile. And that's not quite what's happening here. Like, I'm sort of piecing it together from what's been reported by other publications. But what we understand at the high level is that the chatbots or the large language models are being used to analyze information, to identify the bomb targets, and then there is a missile that is launched to target those places. And it's not like, one continuous sequence. Like, it is people that then receive this list of identified targets and then do what

• 10:22Speaker 03

  they have always done, which is then like, launch the weapons. But yeah, like, it almost feels continuous because of what we were talking about, because like, is that person really even actually adding their own judgment in?

• 10:36Flora Lichtman?

  Right, right. And when we talk about autonomous weapons, like, that's more like a brain, right? Like, because at that point, are we talking about a continuous sequence? Or like, even if it's a string of tools together, that amounts to all the judgment being outsourced.

• 10:52Speaker 03

  Yeah, so fully autonomous weapons is it would be if, you know, Claude identifies the targets and then without anyone there, it's automatically fed to like the missile launching system. And then the missiles are launched. Or it could refer to, you know, drones with AI capabilities attached to them that like go identify the target itself through the computer vision system and then drop bombs in that area. So basically like autonomous is defined as specifically like there's a kill chain sequence and it's it's the last two stages deciding and the launching.

• 11:36Speaker 03

  Like if there is no human involved and it's just the machine that's doing these, that is what's considered a fully autonomous weapon.

• 11:43Flora Lichtman?

  Right, and this is where Anthropic was like, Amadee was like, we're not quite ready for that.

• 11:49Speaker 03

  We're not, yeah, exactly. It was like, we're not quite ready for both steps, but we would like to get there and we are okay as long as there is a person that's looking while both steps are happening.

• 12:05Flora Lichtman?

  Right, let's look ahead. What will you be watching for?

• 12:10Speaker 03

  I'm going to talk about what I'm watching for, not with the companies, because the thing that makes me optimistic in a deeply dire time is the amount of resistance that started bubbling up among the public. So this is the thing I'm most excited about watching for is that, you know, in recent polls, 80% of Americans now believe that there needs to be some form of regulation on the AI industry. I don't remember the last time that 80% of Americans were on the same side of one issue. Like, I'm very optimistic about the fact that there is now a broad coalition building to hold this industry accountable, because we need that more than ever. And we are already seeing this happening with some aspects of the AI industry,

• 12:55Speaker 03

  like the reckless data center expansion that the industry has been engaged in, where so many communities across the US are Discovering that there is a data center propping up in the community that was struck as a deal under NDA with their city council. And they are literally physically going into the streets to protest these facilities. They're going to town halls to pressure their elected leaders. They're actually voting out their officials that are not adequately reflecting the will of the people in the situation. And this has become a very effective grassroots movement to check a key, what I call pillar

• 13:39Speaker 03

  of the empire's expansion. Like, if these companies do not get their data centers at the clip that they need to, they have to slow down their technology development because it is already a key bottleneck in their advancing, and it would become an even greater throttle to their advancing. And I would love to see more people around the US thinking, and also around the world, thinking about how to take the lessons from this grassroots movement pushing back on data centers to then push back on other aspects of the AI supply chain, whether it's the reckless deployment in the military, or, you know, the psychological harm to kids, or the mass

• 14:23Speaker 03

  copyright infringements that are happening. And we are beginning to see more and more of that across the board.

• 14:29Flora Lichtman?

  Karen How is a journalist covering AI and also the co-host of the new BBC tech podcast, The Interface. Karen, thank you so much for taking the time.

• 14:38Speaker 03

  Thank you so much for having me.

• 14:40Flora Lichtman?

  This episode was produced by Dee Peter Schmidt. Thank you for listening. I'm Flora Lichtman.