The race to regulate AI in warfare

In today’s episode on 6th March, we explain why the use of AI doesn’t have guardrails like nuclear weapons and biological warfare. ⁠Join Ditto's FREE Insurance Masterclass. Register here⁠ https://ditto.sh/2cgegs

• 0:00Speaker 00

  Hello folks, you're tuned into Finshots Daily. In today's episode, we explain why the use of AI doesn't have guardrails like nuclear weapons and biological warfare. But here's a quick side note before we begin. This weekend, we're hosting a free two-day insurance masterclass that helps you build real financial security by understanding health and life insurance the right way. So you'll learn how to protect your family, choose the right cover amount, and understand what truly matters during a claim in life insurance. And in terms of health insurance, you'll know how hospitals actually process claims, common reductions, the mistakes buyers usually make, and how to choose a policy that won't disappoint you when you need it the most. The seats are really running out, so head to the link in the description and register.

• 0:45Speaker 00

  By the way, it's free. All right, let's get on to today's story. Last week, Anthropic stepped back from a $200 million deal with the US Department of Defense over a longstanding dilemma: how far AI should be allowed to go in warfare? The company insisted on strict limits preventing its AI, Claude, from being used for mass surveillance or fully autonomous lethal weapons. Their argument: such systems should never make kill decisions without meaningful human oversight. When the Pentagon pushed for broader access that would allow the technology to be used for any lawful military purpose, Anthropic refused and exited the negotiations. Rival lab OpenAI later moved ahead with a separate deal with the Pentagon.

• 1:26Speaker 00

  This highlighted a growing divide in Silicon Valley over whether AI companies should impose hard ethical red lines on military use of their technology. At first, this might look like just another corporate disagreement between tech companies and the military, but it touches on a much bigger question: who decides the rules when a powerful new technology enters the battlefield? You see, for most of modern history, whenever humanity created a particularly dangerous weapon, the world eventually responded by writing rules around it. The suffering of wounded soldiers on European battlefields in the 19th century eventually led to the First Geneva Convention, protecting both injured soldiers and medical personnel. The horrors of trench warfare and chemical attacks during World War I pushed countries to adopt agreements restricting chemical weapons. And after the widespread civilian

• 2:10Speaker 00

  suffering and prisoner abuse during World War Two, nations came together to create the modern Geneva Conventions. They are a set of international rules that define how wars should be fought, including protections for civilians, prisoners of wars and the wounded. It was a pattern as clear as day, whenever warfare evolved, so did the rules around it. These rules didn't stop wars, they simply established guardrails for how they could be fought, by preventing countries from using any and all means against one another. But today, a new military tech is emerging that doesn't neatly fit into any existing rulebook: artificial intelligence. Now imagine a battlefield command center flooded with information, satellite images streaming in, drone footage updating every second, radio intercepts piling up faster than analysts can process them. Now imagine an AI system sitting in the middle of it all.

• 2:55Speaker 00

  Within seconds, it scans the imagery, flags suspicious movements, highlights possible targets and suggests the next course of action. Tasks that would normally take human analysts hours or even days can suddenly happen almost instantly. Militaries around the world are already experimenting with systems like these and that's where the real debate begins. Because some of the technologies being developed today go a step further. Instead of merely assisting commanders, they could eventually identify and attack targets independently. Researchers refer to these systems as lethal autonomous weapon system or LOS, weapons that could potentially select and strike targets without direct human control. Over a year ago, the United Nations addressed the dangers of LOS in modern warfare.

• 3:37Speaker 00

  Their concern is how AI systems could eventually be used to identify and attack targets without direct human intervention. In other words, the system itself would decide who is a threat and who isn't. In policy circles, these technologies are sometimes referred to as killer robots. The term sounds dramatic, I know, but it simply refers to weapons that could use artificial intelligence to select and strike targets on their own. Now, that's where things start getting complicated. Because the laws of war were written with the basic assumption that humans make battlefield decisions. We'll explain this with a scenario, so it's easier for everyone to understand. If a soldier violates the rules of war, responsibility is usually clear. The individual who carried out the attack can be investigated, commanders who gave the unlawful

• 4:20Speaker 00

  orders can be held accountable but autonomous weapons complicate the idea of responsibility if an AI system misidentifies a target or causes civilian casualties who exactly is to blame the military commander who deployed the system or the engineers who wrote the algorithm or the machine itself now it's a question the current rulebook doesn't clearly answer that's why they've been debating whether the world needs new rules for autonomous weapons from now to september over one twenty countries will discuss new rules and regulations of laws even the chair of the negotiations has warned that time is running out if governments wait too long he said technological developments could overtake the rules meant to control them unlike nuclear or chemical weapons AI don't rely on rare materials or specialized factories much of it is software and software tends to spread quickly but agreeing on it is easier said than done and I'll tell you why right now there are three broad ideas on the table the first is a complete ban on autonomous weapons altogether this would mean that machines on their own wouldn't be allowed to operate independently especially for lie or death decisions on war it would follow the same logic the world eventually applied to chemical and biological weapons and then there's a second possibility do nothing new at all some governments argue that existing laws of war already cover this tech the same rules that apply to any weapon today including the protections laid out in the geneva conventions should apply to AI systems as well the third option is a middle ground instead of banning these systems outright countries could require what experts call meaningful human control in practice that would mean AI systems could analyze battlefield data suggest targets or recommend actions but the final decision to launch an attack would always rest with a human commander which brings us back to anthropic the company was founded by former open AI researchers who believed powerful AI systems needed clear guardrails that philosophy also shaped its decision to step away from the pentagon deal anthropic wasn't opposed to working with the military but it was unwilling to remove restrictions that could allow its AI to be used for mass surveillance or fully autonomous weapons in many ways that's the same question the rest of the world is now

• 6:30Speaker 00

  grappling with where exactly should the guardrails for AI in warfare be drawn? For centuries, the pattern has been clear. New weapons emerge, wars reveal their dangers, and only then does the world write the rules. AI may be the first time humanity is trying to write those rules before disaster forces its hand. Unlike traditional weapons which are controlled by whoever reveals them, AI systems still carry safeguards designed by the creators. At least for now, those guardrails remain in the hands of the companies building the technology. All right, folks, I'll see you in the next one.

• 7:01Speaker 00

  Thank you for listening to today's episode. And if you want to share your feedback or suggestions, do drop us an email to [email protected]. Until next time.