How AI Is Reshaping the Battlefield

Just how are powerful AI models being used in warfare overseas? In this episode of Galaxy Brain, Charlie Warzel sits down with Wired senior writer Will Knight to discuss the rise of autonomous weapons. From the origins of Project Maven to the recent falling-out between Anthropic and the U. S. Department of Defense, they trace what’s happening as artificial intelligence moves from summarizing documents to informing decisions on the battlefield. How do these weapons work? What are the safeguards? Who decides what values get baked into these models? As autonomous systems become harder to avoid, where exactly is the line between human judgment and machine decision making? Warzel and Knight help explain how the Pentagon and Silicon Valley are more entangled than ever and where warfare goes from here. Get more from your favorite Atlantic voices when you subscribe. You’ll enjoy unlimited access to Pulitzer-winning journalism, from clear-eyed analysis and insight on breaking news to fascinating explorations of our world. Atlantic subscribers also get access to exclusive subscriber audio in Apple Podcasts. Subscribe today at TheAtlantic.com/Listener. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Tired of rage scrolling through AI slop and clickbait just to get a handle on today's news? Meet Inpress, the algorithm-free news app that makes it fun and rewarding to stay informed. Track your media literacy and rediscover reality. Download today at inpress.app. That's I, N as in news nerd, press.

• 0:20Speaker 01

  When was the last time you indulged your desires, felt true Range Rover refinement? The last time you felt total serenity, total confidence, no matter the terrain? Limitless, effortless, peerless. How far can Range Rover take you? Range Rover, designed for distinction.

• 0:35Speaker 02

  The US government will talk a lot about like the importance of AI reflecting American values. But what are those values exactly?

• 0:40Charlie Warzel?

  Do we get to decide those values together?

• 0:42Speaker 02

  Yeah. Who gets to decide that? Is that just Trump or is it just the heads of these companies?

• 0:51Charlie Warzel?

  I'm Charlotte Rizell and this is Galaxy Brain, a show where today we are going to talk about autonomous weapons in the future of AI in warfare. It's a very strange time to be covering artificial intelligence. There is a active conflict in Iran, which artificial intelligence is being used. There's also this huge fallout between the AI company Anthropic and the Department of Defense over concerns about the use of their technologies. And there's this broader feeling right now over the subject of autonomous weapons that frankly, these companies have this very powerful technology that they are then handing to the military. And that technology is being used in ways that maybe these companies don't feel like they have

• 1:33Charlie Warzel?

  control over and that these companies are certainly worried about. There are so many moral, legal, ethical concerns. There's so much that we don't know about how these models actually work, the decisions that they make, whether they hallucinate, whether they can fail at rates different and more concerning than humans, whether humans are in the chain making these decisions, whether there are the appropriate safeguards, whether the ideologies of these companies and their leaders actually fit the ideologies of the military and whether that conflict is something that we should all be having a real conversation about. It's Such a messy, scary moment. And Silicon Valley is totally caught up inside of it.

• 2:18Charlie Warzel?

  So I asked Will Knight to join me to talk about all of this. Will is a senior writer for Wired. He covers artificial intelligence and he writes their AI lab newsletter. Together, we get into the nuts and bolts of all of this. Who the players are, what this technology can actually do and not do, and what the future of AI warfare might look like in a moment where we are constantly escalating for fears of being left behind. Will joins me now. But first, a quick break.

• 2:48Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait and AI noise. Thanks, social media. That's where InPress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with InPress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.

• 3:21Speaker 01

  When was the last time you indulged your desires? Felt true Range Rover refinement? When was the last time you drove in total serenity? The outside world reduced to a hush? The last time you were confident on uncertain terrain? The last journey that made you feel at home, conveniently connected at every step? In a Range Rover, pioneering innovation meets leading edge luxury to make every drive an occasion. Limitless, effortless, pureless. How far will you let it take you? Range Rover, designed for distinction.

• 4:00Charlie Warzel?

  Will, welcome to Galaxy Brain.

• 4:01Speaker 02

  Thanks for having me.

• 4:03Charlie Warzel?

  So I want to start and I want to ground this conversation in a little bit of history, right? Because there's this long and somewhat amazing history of military innovation, right? The military building weird or niche things that then diffuse into a broader culture. Everything from radar to wristwatches to GPS to duct tape to the internet, right? So it is not uncommon, broadly speaking, for the government to fund ambitious, potentially vague technologies for the battlefield that then have all these other applications. And it's also not uncommon for them to partner with outside companies to do this work. And so I would love to go back to, you know, if you want to go even further than this,

• 4:46Charlie Warzel?

  we can, but maybe starting at Project Maven and the announcement of that in 2017. The original mission is, you know, use computer vision algorithms, right? To analyze drone footage, detect objects, and the quote was, "turn data into, quote, actionable intelligence and insights at speed." What's the backstory of that?

• 5:08Speaker 02

  Yeah. I mean, I think that the backstory goes much further back. The sort of architect of a lot of the US thinking on AI was Ash Carter. And this was the sort of pre-ChatGPT era, but in the sort of deep learning era, it seemed like, and you can see why it would fit very well with military, you know, things like targeting, like these image algorithms that could spot things. But it seemed like that was going to, you know, be a paradigm shift and a really, really big one. At the time, it was incredibly controversial, right? And it's amazing how much things have changed, but you had big protests at Google, which had won this contract. I think, you know, things have shifted. I think in some ways that makes a lot of sense to me. I think the idea that you're not going to use something like AI in the world of

• 5:50Speaker 02

  defense seems kind of absurd, right? It's like saying you're not going to use software.

• 5:54Charlie Warzel?

  I think you have to look at it too, right? Going back and looking into the Project Maven backlash initially and the fear at places like Google from some of these engineers and folks at the time was, "That all sounds fine potentially, but I'm worried we're going to end up making autonomous weapons," right? This, we know sort of where the path could lead here. I think what's really fascinating about those initial protests, right, is it was over this idea of where this all leads. And it seems like... They're correct. Like, this is where it is leading to, right? It's not just these vision algorithms.

• 6:37Speaker 02

  I would say there's always, there's been this long, you know, very longstanding principle about sort of how you sense and make decisions and that being really critical in military conflict. And you can see how the use of like a vision algorithm would be very important in that. And so you can sort of see this trajectory towards more and more, you know, automating more of that process. But one thing I would, I would say, you know, is having many times interviewed people at the Pentagon and people in the armed forces, there are very good reasons why people on the ground do not want to hand that off and commanders don't. As much as I think the public thinks, those systems are unreliable. And the idea of handing off decisions about taking other people's

• 7:19Speaker 02

  lives, you know, making terrible accidents is not taken lightly. And I think, I think there is very, very good reason why it is not, it is not the case that that is going to be rushed in. For a long time, we've had systems that are, by many people's definitions, fully autonomous. So you have systems that will fully autonomously destroy a missile that's trying to hit a ship. There are these systems that will shoot those down and they'll have shot it down before the person can react. We do also have guided missiles that will go into an area. And so you, you would have a set area, you know, you can configure parameters where you know there's only going to be enemy combatants. Those are extremely expensive, the rare, what they call exquisite systems. One of the things we're seeing with

• 8:03Speaker 02

  Ukraine is like, you know, off the shelf drones, weaponized, and the way software can control those is, is means that it is, becomes a lot easier for autonomy to be deployed. One of the things that, you know, people know is going to be really a sort of game changing is swarms of them, like lots of them working together, because it's much harder to counteract 20 of them coming in to try and attack your tank. And then you can't have 20 people operating their own drones and defending it. So you're going to have situations where autonomy becomes slightly more It's going to become harder to prevent it. I think in some situations beyond those sort of purely defensive ones.

• 8:42Charlie Warzel?

  I want to drill down a little bit in how these warfare models work specifically to the extent that you know, that you can help us understand, because I think demystifying all of this is very important, down to the, you know, explain it like I'm five version. But to the extent that you know, like, how do they work? How many humans are there in the chains of commands? You know, how are these things, the nuts and bolts of them, working? How they identify targets, safeguards, all of that?

• 9:09Speaker 02

  Yeah, these are classified systems, so we don't have, I don't have 100% visibility on it. The way I understand it is slightly less that they're, you know, feeding in a ton of information, saying, what should we do? You would see a map and you would maybe have assets on it and you could ask the language model to provide, ask questions about the depth, that, you know, maybe the signals intelligence that was related to that particular area. And I think, you know, you would have all these sort of different resources where you could ask things about it. So I think, I think it is a little more sort of, you know, being kept at arm's length in terms of the model making, making calls. I think that they're not crazy and they're not stupid about how they ought to maybe not rely on the system at a high level.

• 9:51Speaker 02

  But if a language model were making some errors and maybe the user didn't check carefully, could that lead further along to erroneous decisions? So it raises questions about how people are trained to use those, how much they rely on this, what sort of, what sort of trust it kind of maybe inspires in people they shouldn't have?

• 10:10Charlie Warzel?

  It does feel like that there's probably somewhat of an analogy, if I'm gathering this correctly, between the idea of we're journalists, right? We say we get this series of spreadsheets or this, you know, this list of data and we want to, you know, use a language model. We, we let it look at this, this corpus of information and it provides insights. It's not saying, this is your story, write this. It's saying, no, no, what I identify, here are some patterns that we have seen. That is more. Of what we're talking about here when we're talking about that intelligence.

• 10:47Speaker 02

  Yeah, that's my understanding. And yeah, you could compare it to, you know, maybe the law or something like that where clerks are using language models to analyze a lot more case, look case studies or maybe medicine.

• 10:59Charlie Warzel?

  You might not be able to answer this question, given your ability to see the systems or know this, but I feel like it's it's worth asking because so many people are using these generative AI tools now. So when you hear something like the Pentagon or the Department of Defense is using Claude, right? And it puts this thing into your head of like, you know, a lot of people are using these tools because they're like a friendly way to automate busy work, right? And they talk to you in this friendly way. And there's this, there's this kind of, it can feel a little bit dizzying to think about it, right? And so I'm curious from you, do you have an idea of how different this AI would be from the commercial versions of this generative AI?

• 11:43Speaker 02

  We do know that they gave the, they gave the DOD a specialized version called Claude Gov. This is a great question. I was wondering about this because if you prompt Claude anything to do with weapons or, you know, anything that might seem like a related to a military conflict, it will say, I'm sorry, I can't help you with that. They've given it a specialized version, which I think, and they have, I've, they've not disclosed this, but I think would have fewer of those guardrails it would have to.

• 12:13Charlie Warzel?

  Right.

• 12:13Speaker 02

  But that does, you know, it's an interesting question, I think, is like, what does it mean when you unalign it? And I've played with some unaligned models and they can behave in sort of surprising ways. They're able to do more things, but they sometimes will push back, even though, you know, some of the, it depends how it's been removed, how those sort of restrictions have been removed. But you kind of have to give a model some guardrails to make it coherent. I kind of. Would be very curious like what the the behavior of that model is. Does it sometimes actually reject stuff? Like, that could have happened, especially in the early stages, and maybe that I'm pure speculation, but then maybe that, you know, has led to some of these kind of concerns about, you know, woke woke models and companies or something like that.

• 12:56Charlie Warzel?

  Right, and and and all of them, you know, there's a lot of debate between people who are real believers in in this technology and those who are more skeptical of it about personalities, right? And whether these things actually have personalities or whether they're, you know, there's just kind of like an emergent set of traits or ways they talk. But I think that that is fundamentally extremely interesting when you get down to this level of things, right? Like Claude is known for for being a little more compared to, you know, all the other frontier models, like it's a little more artistic, you know, it's a little more like it has its own, it has its own things.

• 13:28Speaker 02

  Yeah, same vibe.

• 13:29Charlie Warzel?

  That becomes very interesting when you think about the ways in which these these models are being ported into this moment where, again, they're maybe not making decisions, but they are talking to a person or rather giving an input in a very specific stylized human sounding way.

• 13:49Speaker 02

  There are really interesting questions about that. I think the truth is most models have quite similar alignment. And I think that most of those things are kind of fairly universal. There's, you know, and I think the question to me is sort of, yeah, like, how does that? How does that really sort of change when you put in a military setting? If they've sort of buttoned it down very much and it's just doing some, you know, summarizing text, like here's a can ask a bunch of questions. That's one thing. But if it's if it's trying to sort of do this sort of parasocial, you know, stuff that chatbots do normally, that is kind of strange. And that does really affect the user experience, the user expectations of you would probably have much higher trust in a system like that if it were pretending to be a person. Maybe when you shouldn't, you shouldn't have.

• 14:32Charlie Warzel?

  We've got at this a little bit, but I think it's just worth drilling down even even more on it. And break down what we mean by like autonomous weapons, right? Because that is a terrifying sounding phrase. And in the perp, the point of demystifying all of this, you know, it's not always clear cut. Do you think when we talk about autonomous weapons, it's best for people to think of it almost just like they have the ability to make split second decisions in anything? And that's sort of the place where we should work from. Or do you, do you think there's actually more specificity there that like autonomous weapons are actually as well? The thing that we're all worried about, right? Which is a, which are weapons that are making broader choices about target acquisition or, or, or, or, you know, engagement in some kind of way.

• 15:16Speaker 02

  The DOD has, has been clear that they want to always, for the moment, keep somebody making that final decision to some degree. Right? That's the, that's the thing. So you can have things be autonomous, but you have a person make that decision. But they issued a directive in 2023 on autonomous weapon systems. And that spelled out that there's no restrictions on developing them. I've looked at autonomy in places like self-driving cars, in robots and in the AI industry, like agents, right? I had not come across anywhere that is more careful about what they do than the, than the DOD in spirit. And that doesn't mean they're going to get everything right. But I think, I think that I wouldn't want to, I think it'd be a mistake to sort of portray the people in that, in those,

• 15:57Speaker 02

  in the armed forces as wanting to do sort of autonomy for the sake of it. And I think that if they could keep their finger on the, the trigger, they would want to do that 100%.

• 16:07Charlie Warzel?

  I think that that, I mean, that's, it's very important context. And it's also very important to, in the sense of there are so many different people in this chain, from the people at the very top who are giving, you know, press conferences and who may be more ideological, down to the people who are nuts and bolts, pulling triggers.

• 16:28Speaker 02

  But you're, you're making a, you know, a great point when you talk about the leaders of countries, including our own, that will bear a lot of responsibility for how quickly these things get kind of rushed in. And, and you know, if you see, I mean, this is one of the things with, with adoption of AI. Is that it's seen especially in the US as a way to kind of regain parity and overtake China because the US is by some dimensions, you know, has the biggest military, but by some dimensions is is as a disadvantage compared to China. And so leading in AI, like this is one of the real reasons why AI is such an important thing for for the US government. And I think there are, there are a few questions like how aggressively you want to rush that in, how much you want to throw safeguards out the window.

• 17:12Charlie Warzel?

  All of this comes down to when we're, when we're talking about this, it, it leads us to the DoD verse anthropic situation that has played out over the, over the past couple of weeks. Because I do think it, it speaks a little to all of this as well, right? Which is the understanding that, that this technology,

• 17:33Charlie Warzel?

  that people making this technology in, in some senses have strong feelings about how it should be used.

• 17:39Speaker 02

  Right.

• 17:39Charlie Warzel?

  And what they do not feel comfortable with the use being. And then you also have a current administration that feels very strongly that they, speaking of autonomy, must have full autonomy over, over how things should be deployed. Can you just walk through the particulars of that fight for someone who's only very, you know, basically paid attention to it?

• 17:59Speaker 02

  Okay. Yeah. Well, I mean, there, there are some elements we don't, we don't have the full details of. But broadly, a few matter of weeks ago, the Pentagon wanted to change the, the contracts that it signed last year with all the big AI companies. What was in there previously that anthropic didn't want to change was specific prohibitions against using it for mass surveillance of US citizens and for autonomous weapons. So the Pentagon says, we don't want to do that. So let us change that. And in a way, anthropic just decided that was a hill they're going to die on. And I think they, you know, a lot of people in the AI world are very conscious of safety and conscious of the, the sort of moral. Questions around it. And so OpenAI stepped in, so they're going to take this contract,

• 18:43Speaker 02

  but they want to have some safeguards. So but the question is, what are those safeguards? And the question is, what situations is it acceptable to have something that might fail X out of 1,000 times? Like, we don't know how much that failure is and what times isn't it? And I think, you know, if you were going to, was going to make the difference between telling somebody, giving some of the information that they might make a lethal decision. I think this is the question the public should have is like, where is that line being drawn?

• 19:10Charlie Warzel?

  Right.

• 19:10Speaker 02

  Especially as this gets deployed more widely. I think it's been sort of quite tentatively implemented so far. And as that gets more capable, and these areas can sneak in in surprising ways, like, where is that? Like, where should we and where shouldn't we? And that's kind of, yeah, that's sort of the question that's not been answered.

• 19:27Charlie Warzel?

  You know, one thing that we didn't say was that Secretary of Defense Pete Hegseth and the administration are designating Anthropic as a result of some of this as a supply chain risk, which is, a lot of people have said, an extreme overstep of the bounds. So there's that element, which, which seems, to my sense, very irrational. And then there is the, the understanding of, of the actual nuts and bolts of the contract, right? Which is, okay, maybe, maybe this doesn't make sense for us.

• 19:53Speaker 02

  Yeah, he had this moment where the, he realized that the model, you had to get permission to do all these different sort of things. So it does sound like either the model was not as unaligned as they might have wanted, or the safeguards, the sort of checks you had to go through, the way the system was designed, wasn't working for them. But yeah, that, that reaction, like, to put that in context, that's only ever been applied to Chinese companies like Huawei, who are accused of operating for the Chinese government. To take that action, not just on a US company, but on one of the most important, you know, by their own measures, US companies. That there is working in this incredibly important technology seems amazingly destructive

• 20:37Speaker 02

  to me, and it is a very, very extreme action, but that's sort of, yeah, I guess that's just how they roll.

• 20:43Charlie Warzel?

  So something that's really important here, to my knowledge, and it speaks to this whole anthropic dispute, it speaks to almost everything we're talking about when it comes to the companies interfacing with the government, right? They create these powerful models with these capabilities that they understand will be used in warfighting, and they try to set their own boundaries to some degree, guardrails, safeguards, whatever it is. Then they have to hand it over, right? And it becomes a bit of a black box. There is this lack of understanding in specifics of how this stuff is being used on the ground because some of these operations are extremely classified, some of these uses are extremely classified.

• 21:26Charlie Warzel?

  As all this anthropic fallout is happening in the news, the United States goes to war with Iran, there are bombs dropping. And so there's this moment, a week or so ago, there's these first reports coming out in the press that perhaps the United States was responsible for this deadly Tomahawk missile strike on Iranian elementary school. There was all this speculation on X in the media, you know, speculation is what it is, that, oh, we're hearing about these systems like Claude being used for targeting, potentially.

• 21:56Speaker 02

  Yeah.

• 21:56Charlie Warzel?

  And, oh, here is a potential targeting failure. Now, again, making no link or connection, this is all just pure speculation, but it speaks to this idea of would a company like Anthropic even know if their technology was being used, or any company, let's not even use Anthropic, would any AI company know in a situation similar to that, let's not use that one, if, were we involved? And I think that that's what I mean by the black box, which is, you know, you kind of have to give it over, and I doubt that they're getting that visibility, right?

• 22:29Speaker 02

  Yeah, you know, like the company that made the missile that they fired didn't know anything about it, didn't have any say after that was, was used. I think as far as I understand it, is not the case that Claude would have suggested that as a target. That's, that's my understanding, and I- I think that's right. I don't know how much they rely on Anthropic or these other companies to help build out new applications, new uses of those models. Maybe partly Anthropic wants to have more visibility on that because they believe they know how to do it more reliably. That would be my suspicion. And I think it would behoove the DOD to be aware of that as well. These people understand how these systems can fail, so you'd want to try and work with them.

• 23:14Speaker 02

  Remember, right now, what we're seeing in the AI world that's taking off like crazy is a next generation of these tools. And these are agentic AI. And I know it's a buzzword, but they genuinely are. And you'll ask a model or ask one of these tools like OpenClaw to do something, it will write code and it will go through a bunch of steps. And that does raise more risk of unintended consequences because those systems are doing things and because they're going through a bunch of steps. I use OpenClaw. It's amazing. And it is really capable. You can see the allure and why it's the direction things are going to go. But it will do things

• 23:54Speaker 02

  that I didn't really want it to do, to try and get something done. And so how you build those is going to be an incredibly important question, I think.

• 24:03Charlie Warzel?

  I was preparing to talk to you, and I was having a conversation about this black box potential element, right? And I said to this person, you know, they just have to hand it over. And that's got to be terrifying. And the person I was speaking with pushed back and said, well, is it terrifying? And their rationale was, you know, in past defense contractors, they're probably not terrified to hand over the missile or the hardware or whatever it is, or the database infrastructure system to the United States military. They're probably, they know where their purpose is in this chain. I was thinking about this and how the fundamental difference to me. Feels exactly what you're saying. It's the fundamental difference of artificial intelligence.

• 24:46Charlie Warzel?

  Because a company like Anthropic in this situation was deeply concerned in this way that others may not have been in the past, because it's not a technology that does one discrete thing. There's just an unpredictability here, becoming more and more unpredictable with each iteration of this type of technology. And the intelligence gives, its whole purpose is to give a world of options without a complete and specific control. And what do you, what do you make of that idea?

• 25:22Speaker 02

  I think what we're likely to see in this and other critical situations is the development of new kind of approaches to engineering, actually. And how reliable those will be will be an interesting thing to watch. So a good analogy is self-driving cars. Those systems also use like non-deterministic technology. So like a vision system that you can't get, you can't predict actually what it's gonna, the decision it's gonna make or the output, what it classifies something as. And it has to do all these things on the fly where you know, it's like potentially life or death situations. So those companies have had to design conventional engineering around that to make sure there's safeguards in all these different steps. And that's taken years and those cars can still only go, I mean, it's amazing and they've

• 26:06Speaker 02

  made great progress, only go in limited situations. So I think it's doable. You can do the sort of safety critical engineering with this, these kinds of more non-deterministic, more unpredictable technologies. The challenge is when that technology is advancing so quickly, how do you wrestle with that? Like when it's going so fast, it's, it's hard to, to implement those. One of the other differences with Anthropic is right, it's just not a, it's not a missile making company. People signed up because they wanted to make the world. Safer with AI, right?

• 26:41Charlie Warzel?

  You've reported on concerns from people in the industry, but also like the outsiders, the watchdogs, the people who are concerned about AI safety, but also just the safety in general of autonomous weapons. Can you run me through a list of some of the practical concerns that they have?

• 26:58Speaker 02

  There are some who I think just believe on a moral, from a moral standpoint, that handing off, we should draw a line and say we can't hand off. So like use of chemical weapons or something. They're just worried that there'll be this sort of huge expansion of autonomous systems. And that's just morally wrong. And that's a position that I think is a reasonable one to take and can be debated. And the more practical issues that people would have are absolutely like just the incredible unreliability of systems, especially like if you think of a self-driving car that maybe works on the streets of San Francisco where it's been trained for years and years right now.

• 27:42Speaker 02

  But you're taking it to a kind of unfamiliar battlefield. But the probability of mistakes go up. The idea that these systems would misidentify a non-combatant and take their life is a huge concern. And I think there are people on the very technical side who would just, I think, worry about the, as you were saying, the reliability of systems that incorporate these more inherently unpredictable technologies. They might seem like, okay, this only makes a mistake like once every 1,000th times. But they can kind of compound and cause problems in unpredictable ways.

• 28:23Charlie Warzel?

  We're talking a bit here about the idea of fail rates, about the idea of these models pulling in all this different information and making these insights that either hallucinations or having some kind of failure inside them. But humans too very clearly fail all the time. And so what is it about?

• 28:44Charlie Warzel?

  The AI of it all and these models that makes this a scarier proposition to certain people.

• 28:50Speaker 02

  Yeah, I think what we should be worried about, you know, so you're right, people make mistakes. And the way the military is set up is to kind of cope with that in many ways, right? To try and have this chain of command and so on that kind of minimizes the risk of that. But the thing that we should be concerned about is that these models, one, pretend to be very human and they really seem human. And then they will just, they will fail in totally spectacular, unexpected ways. So it's like you're talking to an infantryman and a soldier who gives you the right answer again and again, and you're like, this guy's so good. And then all of a sudden, he's like completely batshit crazy answer. And so that kind of unexpected thing is, I think, what is most worrying, you know?

• 29:33Speaker 02

  I think that's what the AI companies know, that it is, one, it can fail in sort of really unexpected ways sometimes. And then the people come to, come to get really fooled by how, how human it seems.

• 29:49Charlie Warzel?

  How much of this do you think is inextricable to, like, the concern broadly to what has developed as a, as a pretty reactionary culture recently in Silicon Valley? You have people like Palantir CEO Alex Karp, he was on CNBC last week and he said, quote, "This technology disrupts humanities trained largely democratic voters and makes their economic power less and increases the economic power of vocationally trained, working class, often male voters. These disruptions are going to disrupt every aspect of our society." Now, that's not about autonomous weapons or AI technology specifically, but it is about

• 30:30Charlie Warzel?

  instilling or having an ideological value inside of the company and a broader mission statement. Do you think that, that that is part of the reason why? People are having such a strong reaction. Does that change the valence of these conversations and the fears?

• 30:50Speaker 02

  Yeah, I don't think it has as yet. We started to see, you know, people sign open letters and quite prominent people, some of the other AI companies backing up Anthropic in public. We haven't seen that much of a discussion of where the line should be drawn, whether it's in the political world or that much sort of public debate or discussion about it. And it's going to be, you've got just a few people who are making really, really important decisions, whether it's the DOD or it's the CEOs. And so I would not be surprised. We saw much more pushback on it in many ways, whether it's taking jobs, whether it's being used in the military. I mean, I think we don't have a great picture of what's going on in employment in the labor market,

• 31:35Speaker 02

  whatever the CEO of Palantir might say. But I think if it does start to do that, people will start to ask why, you know, these companies that have honestly built tools by slurping up copyrighted material, you know, why they own that, why they, why they sort of in charge of disrupting everybody's jobs. And, you know, I don't know that that's the right position, but I think we could see much more of that.

• 32:01Charlie Warzel?

  It seems inevitable to me too, because as we said, this isn't an Excel spreadsheet, right? This isn't just some, it is, it's dynamic in this sense. It has to be instilled, like whether it's Anthropic talking about Claude having a constitution, right? Like in some senses, some of these companies talk about it as having values. Elon Musk has XAI, he talks about it having anti-woke values. And so then, when you port this over into, yes, these systems are going to be in the chain, a chain that has decisions that ultimately can lead to people being killed or, or, you know, geopolitically significant events of war. They are not doing that thing of handing over a missile that is inert and saying,

• 32:46Speaker 02

  Right.

• 32:46Charlie Warzel?

  They're, like, use this as you will. There is, these are dynamic systems.

• 32:51Speaker 02

  I think that, yeah, the constitution, personality, morality of those models is a whole other thing that this does raise and it does. It is really interesting, right? You know, like the US government will talk a lot about like the importance of AI reflecting American values. But what are those values exactly?

• 33:08Charlie Warzel?

  Do we get to decide those values together?

• 33:10Speaker 02

  Yeah. Who gets to decide that? Is that just Trump? Or is it just the heads of these companies? We're on a trajectory. We're at the beginning of use of AI and it's not going to be just summarizing things very plainly, probably for a long time. And so, you know, whose values do that, who do they reflect? What are you trying to put into those? I mean, I think it's somewhat encouraging that if you look at any model, you take a Chinese model or a US model, most of what they'll say you should and shouldn't do are quite similar.

• 33:37Charlie Warzel?

  There's a great Bloomberg story by the reporter Katrina Manson, who notes in this piece about Project Maven, about a lot of this AI warfare technology. She writes, quote, "There's a palpable sense within the Pentagon that things aren't moving fast enough. Despite the show of force in Iran, officials worry that the US is at risk of falling behind. And officials are already looking past the Middle East to a potentially bigger conflict." As one person familiar with the US operations puts it, quote, "Iran is an amazing precursor to what could happen with China over Taiwan." Now, that's not your reporting, so I'm not going to ask you to like, talk about the veracity of those types of claims. But in your mind, what comes next here? This is obviously the beginning of something.

• 34:18Speaker 02

  Yeah. So just to say that that is the tenor of a lot of conversations around the Pentagon and Washington. On the one hand, you can say, well, that must reflect that this is where we're heading. And there's, there are a lot of scholars who believe that, you know, some kind of conflict is inevitable. There's also a sense in which those things can become self fulfilling. And arming yourself to the teeth with the technology historically has contributed to conflicts, you know, like the First World War, for example. And so, what comes next, I think is sort of up to us and China, right? Just assuming that's going to happen and sort of trying to arm yourself and prepare for that might cause

• 35:02Speaker 02

  you to think that's the only solution, which I think is, would be a devastating and terrible one. And I would hope that there's different, a different path that is, is going to lead to, not lead to that.

• 35:17Charlie Warzel?

  Will, thank you so much for coming on the podcast, helped demystifying some of this stuff. It's important work. Thank you for your reporting and all of it.

• 35:25Speaker 02

  Thanks for having me. Yeah, it was really fun.

• 35:29Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait and AI noise. Thanks, social media. That's where Inpress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with Inpress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.

• 36:02Speaker 01

  When was the last time you indulged your desires, felt true Range Rover refinement? When was the last time you drove in total serenity, the outside world reduced to a hush? The last time you were confident on uncertain terrain? The last journey that made you feel at home, conveniently connected at every step? In a Range Rover, pioneering innovation meets leading-edge luxury to make every drive an occasion. Limitless, effortless, pureless. How far will you let it take you? Range Rover, designed for distinction.

• 36:34Charlie Warzel?

  That is it for us here. Thank you again to my guest, Will Knight. If you liked what you saw here, new episodes of Galaxy Brain drop every Friday. You can subscribe on the Atlantic's YouTube channel or on Apple or Spotify or wherever it is that you get your podcasts. And if you want to support this work and the work of my fellow coworkers, you can subscribe to the publication at theatlantic.com/listener. That's theatlantic.com/listener. Thanks so much, and I'll see you on the internet.

• 37:09Charlie Warzel?

  This episode of Galaxy Brain was produced by Renee Klar and engineered by Dave Grine. Our theme is by Rob Smersiak. Claudine Bade is the executive producer of Atlantic Audio, and Andrea Valdez is our managing editor.

• 37:31Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait and AI noise. Thanks, social media. That's where Inpress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with Inpress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.