AI and the New Global Security Landscape

The conversation about AI often focuses on software, automation, and the race between attackers and defenders in code. However, some of the most consequential risks lie further afield, in domains where a mistake is measured in human lives. Advanced models can now offer step-by-step guidance toward chemical and biological weapons, and militaries are already folding AI into targeting and battlefield assessments. These are no longer speculative fears confined to the AI doomer crowd. They are documented in red team disclosures, government legislation, and events unfolding on real battlefields. Gordon M. Goldstein https://www.cfr.org/experts/gordon-m-goldsteinis an adjunct senior fellow at the Council on Foreign Relations https://www.cfr.org/, where he focuses on the convergence of technology and US foreign policy. He previously spent nearly a decade as a managing director at Silver Lake, and he is the author of Lessons In Disaster: McGeorge Bundy and the Path to War in Vietnam, which is a study of national security strategy and White House decision-making. In this episode, Gordon joins Kevin Ball to discuss the credibility of AI-enabled chemical and biological weapon threats, the ad hoc safeguards meant to contain them, and the rise of autonomous warfare and its implications for human control and nuclear deterrence. Gordon speaks on his own behalf, not on behalf of the Council on Foreign Relations. Sponsorship inquiries: [email protected] mailto:[email protected] The post AI and the New Global Security Landscape https://softwareengineeringdaily.com/podcasts/ai-and-the-new-global-security-landscape/ appeared first on Software Engineering Daily https://softwareengineeringdaily.com.

• 0:00Speaker 00

  The conversation about AI often focuses on software, automation, and the race between attackers and defenders in code. However, some of the most consequential risks lie further afield in domains where a mistake is measured in human lives. Advanced models can now offer step-by-step guidance toward chemical and biological weapons, and militaries are already folding AI into targeting and battlefield assessments. These are no longer speculative fears confined to the AI doomer crowd. They are documented in red team disclosures, government legislation, and events unfolding on real battlefields.

• 0:40Speaker 00

  Gordon M. Goldstein is an adjunct senior fellow at the Council on Foreign Relations, where he focuses on the convergence of technology and US foreign policy. He previously spent nearly a decade as a managing director at Silver Lake, and he is the author of Lessons in Disaster: McGeorge Bundy and the Path to War in Vietnam, which is a study of national security strategy and White House decision-making. In this episode, Gordon joins Kevin Ball to discuss the credibility of AI-enabled chemical and biological weapon threats, the ad hoc safeguards meant to contain them, and the rise of autonomous warfare and its implications

• 1:22Speaker 00

  for human control and nuclear deterrence. Gordon speaks on his own behalf, not on behalf of the Council on Foreign Relations. Kevin Ball, or K-Ball, is the vice president of Engineering at Mentos and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action discussion group through Late and Space. Check out the show notes to follow K-Ball on Twitter or LinkedIn, or visit his website, kball.llc.

• 2:05Speaker 01

  Gordon, welcome to the show.

• 2:07Speaker 02

  Kevin, great to be here.

• 2:09Speaker 01

  Yeah, I am really interested in this conversation. So, let's get in. But before we do, let's start with you a little bit. So can you give us a quick overview of your background and how you came to be interested in this subject area?

• 2:21Speaker 02

  Sure. So, I am by training in international relations and international security analyst. Right now, I'm a senior fellow at the Council on Foreign Relations, which is a prominent think tank based in New York and Washington focused on US foreign policy broadly defined. It is a nonpartisan organization. All of the senior fellows work through areas of domain expertise, but not necessarily areas of political advocacy. And I guess it's important to state that here I speak on my own behalf and not as a representative of the council.

• 3:02Speaker 02

  I have been a senior fellow at the council for about 10 years. I was brought in a decade ago because at the time I was a managing director to large private equity global technology investment firm, and I also had a continuing interest and background as a student of national security and American strategy. And my mandate was to chart the convergence of technology policy and US foreign policy interests. So I've done that for over a decade. And in the past year, particularly in the past six months, the explosion

• 3:46Speaker 02

  of AI has dominated my attention, as it has for so many others. But I'm particularly focused on national security and global policy implications. And I've been writing about that continuously, particularly in 2026. And it's a rapidly changing area of new developments, troubling developments, important developments. And I'm trying to analyze it as we go.

• 4:14Speaker 01

  Awesome. And we're talking today because we are at a kind of unique and interesting moment in the evolution of AI and national security and the sort of global security thing. And, you know, as we're speaking, the last, I want to say, month or so, there's been this kind of wild back and forth with first Mythos, and then Fable, which is now called Fable, and then Soul, and like the US government starting to really get involved and impose these security restrictions. So can we like maybe just start with, from your perspective, what has shifted that has made this suddenly front and center?

• 4:50Speaker 02

  Yeah. Well, it's a great question, and I share your sense of implicit confusion because it's been a chaotic six or eight weeks in the evolution of what was a non-existent engagement from the federal government on the assessment of advanced AI models to something now, which is happening in real time and is becoming an established process with new actors and entities from the government and new evolving priorities as we go. So maybe

• 5:31Speaker 02

  it would be useful to just revisit the timeline. Everything kicked off in April when Anthropic announced that its latest model, Mythos, had self-created the most powerful offensive cyber weapon in history. Now, they didn't characterize it precisely as I did, but that is in fact what they disclosed to the world. It was their most recent model, and it had advanced capabilities to target the inherent software weaknesses in legacy systems specifically.

• 6:16Speaker 02

  And they don't really use this in the language of the disclosure, but it had this miraculous capability to identify zero days across a huge population of targets, including some software that was 28 years old and considered to be among the most secure in the world. Another program that had been scanned for security weaknesses literally five million times, and Mythos captured its weaknesses. They conducted a review of 10,000 different software systems. And Claude Mythos, or Mythos Preview as they called it, had demonstrated

• 6:57Speaker 02

  an extraordinary capability to capture zero days. And I wrote about it immediately after this happened. I'm on the board of advisors to a cybersecurity venture fund. I've been tracking cyber as a policy issue for a long time. And it was immediately obvious to me that this was a very consequential moment in the history of cybersecurity, and that this was a very, very powerful, very potent new weapon. And so I consulted my sources over the course of a long weekend, including one colleague who was an entrepreneur, who

• 7:37Speaker 02

  had developed cybersecurity tools, worked in the government at a very high level. And he said to me off the record, what you need to understand about Mythos is that this is essentially a zero-day machine. It has capabilities that we thought in the cybersecurity realm were only possible in the realm of science fiction. Not only does it have the capability to identify these central seminal vulnerabilities, these zero-day events, it can exploit multiple zero days at the same time and design new and original exploits that had never existed

• 8:22Speaker 02

  as a cybersecurity weapon, can link them together, can remain hidden in a software system, undetected, for an indefinite period of time. And it can be a constant source of surveillance, and it can be a mechanism for sabotage at any particular moment. And he said, this is a game changer. So I wrote that up. And Mythos triggered a lot of justified anxiety within the United States government. And its first area of great concern applied to the American financial system. And Secretary of the Treasury, Scott Besant, convened

• 9:03Speaker 02

  a meeting with the CEOs of the major Wall Street banks. And they talked about the capabilities of Mythos, and it became immediately clear to Besant that this was a big deal. And this initiated a process where the government now feels compelled to assess the capabilities of these new AI models, primarily for their cyber capabilities, their capacity to disrupt the financial system, but more deeply, their capacity to disrupt critical infrastructure, both within the United States and around the world. So that's been the primary focus.

• 9:47Speaker 02

  But there are other dimensions to this. The most significant relates to the capabilities of these models to fashion new, original, previously non-existent biological and chemical weapons, unique pathogens, using this extraordinary technology to devise biological weapons structures that never existed. So the government, as of the spring of 2026, came to the reluctant conclusion that these advanced AI models have very significant capabilities that pertain

• 10:30Speaker 02

  to core US national security interests. And the government now is in the business of trying to identify these risks and mitigate them. And this is very much a work in progress, and it has been chaotic, and it is far from resolved. We are at the bottom of the first inning in a game that I think is going to go on for months and months and probably years down the road.

• 11:00Speaker 01

  So I'd love to dig in like a little bit more detail on some of these pieces. First, starting on the cybersecurity and zero day side of things. One of the things that we've seen in this evolution of AI so far is as we expand models, get higher and higher capabilities, there are both like trend lines that get more, you know, just further and further following the trend. And then there seem to be these like junctures where new capabilities emerge. I know, you know, being in the software industry, we're using models for coding and debugging. And, you know, one could say probably cybersecurity related things and have been for, at this point, a year and a half, two years, something like this. Was this ability that emerged around zero days, was this kind of just an extension

• 11:45Speaker 01

  of the trend where at some point got to a particular thing? Or is there a new underlying capability that seemed to emerge here?

• 11:53Speaker 02

  This is a really interesting question, and I know your audience are software developers and software experts. So I would defer to your audience, but as I understand it, this was just not the next progression of gradually expanding capabilities. This was kind of an act break. This was a jump to a whole new level of capability that didn't exist. I have a guy I work with, he's my researcher, he's a computer scientist and spent four years in the US intelligence community, one year at FBI and three at CIA.

• 12:33Speaker 02

  And he studies these tools and these cyber capabilities. And he said, this is not like anything that we have seen before. And what is significant about it is not only has Anthropic created models that have this capability, that they are now actively trying to constrain and disable, which is part of the discussion, but these capabilities are going to be replicated. They have already been replicated quite quickly. They leak, it's diffused.

• 13:16Speaker 02

  And even if you have an imperfect replica of what Mythos does, and it has 90% of the power, as my colleague said to me, 90% of a zero day machine is a pretty significant machine.

• 13:33Speaker 01

  Absolutely. So what is the threat model that's most concerned with here? Is this about like lone wolf, you know, hackers that used to be the script kiddies working things, but now they've got a more powerful machine? Or is this about organized crime groups? Or is this like a nation versus nation, national security from state sponsored hackers type of thing? Or all of the above? Like, what are the different ways that people are thinking about this?

• 13:59Speaker 02

  You've captured it. It is all of the above. The way I think of it is that there are a spectrum of threat actors going from the lone wolf, moving to organized transnational criminal organizations, moving to global terrorist organizations, moving toward nation states, which have been continuously and assiduously developing these capabilities over the years around the world. Every intelligence agency of a major power has been in pursuit of a zero day weapon, including our own government.

• 14:43Speaker 02

  So the spectrum of threat actors goes from the smallest and most inconsequential to the most organized and powerful nation state and everything in between. And thus the threats that correlate with these different threat actors, they have a comparably broad spectrum of risks that they can introduce into the system.

• 15:10Speaker 01

  You're shipping faster than ever with AI coding agents, but those agents don't vet the packages they pull in, and they don't have security context built in. Ori by Endor Labs fixes that. It plugs directly into your editor via MCP, catching vulnerabilities, blocking malicious packages, and flagging exposed secrets in real time. No separate tool to switch to, no dashboard to babysit, security that fits how you actually build. Teams using Ori see 10 times fewer security tickets and six times faster fixes. Free for developers. Get started at www-endorlabs.com/auri. This episode of Software Engineering Daily is brought to you by Warp Build. AI is writing more code than ever, which means GitHub Actions is running more than ever.

• 15:52Speaker 01

  Your GitHub Actions bill is now a function of how much AI code you generate. And every engineer knows the feeling. You push a commit and then you wait. Warp Build makes GitHub Actions twice as fast at half the cost with a one-line change to your workflow. Linux, macOS, and Windows runners, in Warp Build's cloud or your own, enterprise-ready, SOC 2 Type 2 attested, and trusted by teams like Sky from Comcast, Bitcoin, and Brain Trust AI. Get started with $50 in free credits at warpbuild.com/sed.

• 16:21Speaker 00

  Think about your mobile app's source code. Once it hits the App Store, it's out in the wild. And without the right protection, decompiling is easy for malicious actors looking to steal your IP or tamper with your software. That's where GuardSquare comes in. GuardSquare provides the highest level of mobile app security for Android and iOS applications and SDKs. Their advanced tools integrate seamlessly into your CICD pipeline. We're talking polymorphic, multi-layered code hardening techniques and automated runtime application self-protection, paired with mobile application security testing and real-time threat monitoring to deliver the highest level of mobile app security without compromise. Don't leave your hard work exposed.

• 17:06Speaker 00

  Secure your mobile applications today. Go to guardsquare.com to learn more.

• 17:13Speaker 01

  So let's talk a little bit about what can the government actually do here, both from a like legal regulatory, though sometimes this government doesn't worry too much about what is in the current law. But what's the mechanism they're using to impose here to regulate? And how effective do we expect that to be, given, I mean, this week, there was a Chinese OpenWaits model that was released that is being rated close to

• 17:39Speaker 02

  Yes.

• 17:40Speaker 01

  Fable level or Mythos level quality? Like, what can they actually do here?

• 17:45Speaker 02

  Well, it's an excellent question, and it's difficult to summarize because this is an evolving narrative. And the government, specifically the White House, has not locked in to a specific model that will govern the review process indefinitely going forward. I think that it's going to be subject to change and modification as we go. But how does it work now? The White House issued an executive order reluctantly that invited AI companies to

• 18:24Speaker 02

  share their advanced models for review and analysis by the government. There was a big debate within the White House about the duration of that. Initially, it was 90 days, and then, under some industry pressure, it became 30 days. And it was kind of an ad hoc process. The ostensible actor that does this is an entity from the Commerce Department that was created by the Biden administration. It was dismantled by the Trump administration, and then it was revived by the Trump White House, and it was rebranded and given a different name.

• 19:07Speaker 02

  It goes by the acronym of Casey. And it is a mechanism that sits within the Commerce Department and ostensibly gathers technological expertise from across the government, including from the National Security Agency. But it's kind of vague how it's structured, and it's not obvious to me that this entity within the Commerce Department has the inherent capacity to apply really rigorous analysis of these models. But there have been very important developments along the way. Some of them, as recently as past four or five weeks.

• 19:52Speaker 02

  As you have alluded to, we had the initial crisis surrounding Mythos and its release. Then the consortium that was tracking the Mythos capability was called Glasswing, got qualified approval to set that model into the wild, as they say, in this domain. And then Anthropic developed the successor to Mythos, Mythos 5 and Fable 5. And we had another kind of quietly spectacular episode in which Amazon conducted security tests on Mythos 5 and Fable 5.

• 20:35Speaker 02

  And they found that there were instances when its offensive cyber capabilities were in fact operational and had not been disabled or contained. And the CEO of Amazon contacted the White House, eventually communicated with Commerce Department, and the Commerce Department issued a directive to Anthropic that they could not allow the circulation of the model to non-American users, which is essentially impossible. So Anthropic was compelled to just continue access to the model. What was significant about this episode, for me, the significance of this episode,

• 21:19Speaker 02

  was that an investor in Anthropic took the initiative through its CEO to contact the White House and say, there's a danger with this model. Anthropic has vociferously contested the existence of a threat related to Fable 5 or Mythos 5, and they've taken remedial action since then. That's not important. What's important is there's a testing mechanism that has been the convention within the AI industry.

• 21:59Speaker 02

  Red team testing is what they do internally to assess their models. And they assume that they capture all of the risks before these models are released to the general public. Here was an instance when Anthropic, which is among the most vigilant of the frontier AI companies in trying to red test and secure their models. There's something that they missed. You can debate the significance of what they missed, but they missed something. And it was another private sector actor in the tech industry that identified it and brought it to the intention of the government.

• 22:44Speaker 02

  So, you know, a unique moment on two levels. One, the standard mechanism to provide assurance that these models are safe and secure, it failed to test in a pretty high-profile way. And two, the government was stimulated again to address this. And following the initial drama surrounding Mythos, this was scene two in the drama. And underscores, I think, that the government is now going to be in the business of assessing the security character of advanced frontier models.

• 23:28Speaker 02

  They're going to be in that business for the long term, and they're going to have to define the right mechanisms and the right capabilities to execute these security assessments. And there is a in-development White House directive of some kind. It was supposed to be released the week of July 6th, according to media reports, it hasn't been released. And it is anticipated to be a somewhat more fulsome articulation of what the government's going to do, what mechanisms are going to have oversight, but it hasn't been released. And I would make the prediction that whatever they come out with is going to be subject

• 24:09Speaker 02

  to revision and adaptation, because it's not at all clear what the right structure or mechanism is.

• 24:16Speaker 01

  Well, and as you highlighted, like capabilities are leaky. The two leading frontier labs are currently US-based, but there are labs in China in particular that are not very far behind. I've seen estimates ranging between three to nine months. So it's not clear to me at least that we review the models and tell you what you can release or not, structure is going to be effective at all. We can stop US models potentially. We can't stop Chinese models from advancing.

• 24:47Speaker 02

  Absolutely, absolutely right.

• 24:51Speaker 01

  So what does a kind of counterbalance or defensive, rather than like neutering approach look like? Can we utilize these models to up our levels of security across the board?

• 25:03Speaker 02

  You've captured the inherent structural problem. Even if we could devise the perfect system within the United States to provide oversight of advanced frontier models, and even if it were perfectly effective, it doesn't address the threats that emanate from open models, particularly Chinese open models. And that's basically the only game in town. And China has built its open model structure and policy as initially a way to provide

• 25:43Speaker 02

  a countermeasure to America's relative advance in that competition. The open model architecture provides them with the ability to innovate more quickly rather than the closed model structures, you know. But the United States doesn't have any oversight of that. So going back to what's going on within the United States government, we can stipulate, yeah, it is necessary to assess the security risks posed by these advanced AI models. But as a global problem, this is necessary but not sufficient. We can solve for the American problem, but we have no mechanism to solve for the Chinese

• 26:28Speaker 02

  problem. And this is going to be a really consequential and extraordinarily challenging central issue, I believe, in the US-China relationship in coming years. Really, it should be in coming months. There was initial conversation about this in the summit President Trump and President Xi had earlier in the spring. They identified AI security cooperation as a new element of the agenda in the bilateral relationship, but they were vague, if not silent, on what that would actually entail.

• 27:13Speaker 02

  But your your essential point, we have a multi-dimensional set of risks, and many of them emanate beyond the United States from China. And China actually has a structure to their industry, which is almost designed to propagate these risks because of the open model architecture.

• 27:36Speaker 01

  So what are the kind of practical implications right now in terms of what people should be doing with regards to this? And we'll stay within the software security realm, and then we can start talking about some of these other threats that we've talked about. Like if, you know, we've talked a little bit about what government is doing. It's a little bit unorganized from the outside, hopefully not as unorganized from the inside, but from what we can see. But what should the labs themselves be doing? If you're running a company that is not an AI company, or you're working inside of one of these, like what should people be doing to react to this threat?

• 28:09Speaker 02

  I don't have a satisfactory answer for you at all, because this is a slow simmering crisis, which is now accelerating. And it reflects the basic structure of the industry up to this point. So just back as a general proposition, the creation of Anthropic, why did it happen? Dario Amadei had a break with Sam Altman. It was over security and safety issues related to advanced AI models. He launched Anthropic, an entity that would ostensibly be more committed to elevating protection against these these risks.

• 28:52Speaker 02

  But OpenAI and Anthropic are obviously involved in one of the most significant commercial competitions in the history of the technology industry. And they continue to rely upon this mechanism of self-review, of internal red testing. In essence, they grade their own homework. And notwithstanding what's happened over the last two months at the level of the federal government, this is still the standard model by which the world relies for security.

• 29:31Speaker 02

  They rely upon each AI company to grade its own homework, assess the risks of their models, and preemptively try to contain those risks. And that whole proposition, I think, is now being tested and will continue to be tested. One can ask, is this a sustainable process that the AI companies have the responsibility for containing the very same risks that they are propagating and introducing into our information ecosystem?

• 30:15Speaker 01

  I did see recently, Demis Hassabis, the CEO, I don't know if I pronounced that correctly, but the CEO of DeepMind was pushing, or he's pushing for and proposing a standards body that might at least have frontier labs judging each other's homework if not moving outside of the labs themselves.

• 30:35Speaker 02

  Yeah, it's an important development, and it's an entirely legitimate proposal. And in fairness, most of the frontier AI labs have been actively discussing this and talking about this for the past several years, even Sam Altman has. He spoke at Harvard more than a year ago, and he floated the idea of creating an international body kind of comparable to the IAEA, the International Atomic Energy Agency, which is one of the very few UN agencies that actually works and is functional and that has actually had a coherent history.

• 31:19Speaker 01

  Well, that's, that leads to a really important question, like what examples, what like historical analogues can we draw on here?

• 31:28Speaker 02

  Yeah, well, the AI revolution is often compared to the nuclear revolution post 1945. It's a natural parallel to draw, it's a highly imperfect parallel to draw, but it's kind of all we got in terms of the case history. Because the nuclear revolution was the most disruptive event stimulated by technology change in the history of international relations. I believe that AI is now a comparably disruptive influence on the international security

• 32:13Speaker 02

  structure. And it's going to require comparable degrees of imagination and political will to address these threats. What institutions have existed over time to help impose order? With respect to nuclear weapons, there really have been two primary instruments. One was the novel introduction of arms control between two great powers, the United States and the Soviet Union. Progressively realized through the era of the Cold War that they had a shared interest in containing the most dangerous and destabilizing aspects of nuclear competition.

• 32:57Speaker 02

  So there was arms control, and then a parallel effort was launched to create international mechanisms to address the proliferation threat. And the essential international legal instrument for that was the Nonproliferation Treaty of 1968, which created mechanisms to control the enrichment of this isil material. And for the most part, arms control was a very successful enterprise, and for the most part, these international institutions, like the NPT Treaty and the IAEA, were pretty effective international institutions.

• 33:39Speaker 02

  So now, in the realm of AI, we are just beginning to try to grapple with this and define, well, what would be comparably significant and effective forms of cooperation among major powers and cooperation at the international level, international institutions, kind of like the IAEA? What can we do in the era of AI? Here's the really hard part, as I see it, going back to the nuclear parallel. It took years and years to arrive at some kind of stable equilibrium with respect to nuclear weapons.

• 34:21Speaker 02

  US-Russia arms control, the Soviet Union, as it was then known, was a painful, difficult process that took decades. We didn't have an international organization to address proliferation until 1968. We're now in the age of AI, we are moving at the speed of machine learning. This technology is growing at an exponentially greater rate than nuclear technology. And it has one very obvious and super important fundamental difference. In the age of nuclear weapons, it was basically a monopoly controlled by state governments.

• 35:03Speaker 02

  And it was primarily the United States and the Soviet Union. China didn't get the bomb until 1965, and even at that point, the United States didn't know what they were going to do about it. In 65, a proposal was made to the National Security Advisor, McGeorge Bundy, that the United States should consider a preemptive attack on China because they were about to acquire the bomb, which was crazy and fortunately was never actively considered. But at that time, states controlled this technology absolutely, and today, the technology is controlled almost absolutely by the frontier AI companies.

• 35:44Speaker 02

  So it's a very significant inversion of capability and responsibility. The nuclear era, states, mostly great powers controlled this technology. Today, it's controlled by the commercial realm and a handful of the AI companies that are on the frontier creating these new models. And we just don't have a precedent for this. There's no architecture and no meaningful historical parallel that can guide us on what we need to do now. I think this is a new feature of international relations. I think this is now one of the variables that will define the equation of international security.

• 36:29Speaker 02

  And I don't think the world broadly defined has a roadmap about what must come next.

• 36:36Speaker 01

  Where are the conversations happening? As you highlight, it can't just be within governments at this point because these models are not being controlled, it's not a state-level control. So is this, you know, I think the call for a standards body was on a Medium post or an X post or thing, like, is this a conversation happening online between Frontier Labs CEOs? Or like, who's figuring this out?

• 37:03Speaker 02

  Yeah, well, it's fascinating, isn't it? How is it happening? Well, there's no central organizing set of actors that is driving this. It's happening in an ad hoc process. Most of the big ideas are coming from the Frontier Labs themselves. We have this important proposal from Habzis, Google, DeepMind. Sam Altman has floated some ideas. Dario Amade has talked about this. There are informal conversations going on among the Frontier AI Labs, but that's really

• 37:43Speaker 02

  problematic because they are in this ruthless competition with each other. And let's face it, this cohort, these guys despise each other, right? Like, that's no secret, just like, read the case history, these guys are in brutal competition, they despise each other. So the industry's set of actors all committed at once to being the dominant player in this new gajillion-dollar industry. They all genuflect in the direction of wanting to contain global security threats propagated

• 38:27Speaker 02

  by these new models that they are ruthlessly in competition to produce. So that's a problem, right? Because there's no coordinating mechanism among the commercial actors, and there's nothing really of consequence happening right now between the United States and China. That absolutely must change, I believe, but we're kind of nowhere there. So we're left with a brilliant AI creator and CEO posting something on Medium and Sam Altman writing, you know, kind of a vague op-ed for the Financial Times and various other utterances that go into the ecosystem, but this is it, right?

• 39:12Speaker 02

  It's a series of ad hoc responses to a systemic growing crisis. And that's not going to be sufficient in the long term.

• 39:24Speaker 01

  All right, so we've spent a fair amount of time diving into the software security angle of this, and this is one that I think many of us are somewhat familiar with. We've had to deal with Zero Days a lot more. Let's maybe go a little further afield for most software folks and talk about some of the other threats that have been talked about here. And I want to start off with this question of like, using AI tools to build bioweapons. This is something that gets floated around, I think it's often cited by the kind of AI doomer crowd of, oh my gosh, AI is going to doom us. But what is this threat? How real is it? And how are folks in the security establishment thinking about it?

• 40:05Speaker 02

  Well, the threat is quite real. The threat is that the advanced AI models provide not just the information, but in some instances, very specific guidance to create unique bioweapons and unique chemical weapons. And it is a risk that the AI companies themselves are highly cognizant of. In January, just seven months ago, Dario Amade published a 20,000-word essay, which he does every few years or so. And he posts it, and it goes out into the world, and he talks about both the promise

• 40:46Speaker 02

  of AI, but also the qualities of advanced AI that causes some great concern. So in the piece that he published last January, he said, quote, "There is a serious risk of a major attack with casualties potentially in the millions." A month after that, Anthropic released a sabotage risk report for Claude Opus 6, their latest model, and it said that the model had the potential, quote, "to facilitate chemical weapon development and other heinous crimes," close quote. And the report concluded by noting that the model had demonstrated the capacity for covert

• 41:30Speaker 02

  sabotage and unauthorized behavior. So this is a disclosure from one of the most significant AI labs in the world. So the risk is real, and the risk is noted by the entities that are creating these new models that come online every four months.

• 41:53Speaker 01

  I do want to push a little bit on this because this is when I've been a little thoughtful. So even a couple of years ago, I think it was, we had, in this case, it was Sam Altman, doing the tour of Congress, essentially pitching, oh, these things are so dangerous, they should be regulated. And in fact, they should be regulated so that only my company, and maybe, maybe you can talk about some other companies, but only my company can actually do development because we're the only ones that can do it safe. And Dario has taken this line as well of saying, oh, we're the only ones in the world who are actually thinking safely about this. And the cynic in me says, hey, they're, they're just trying to get sort of regulatory capture, right? They're trying to regulate an impossibility

• 42:33Speaker 01

  to compete with them. Do we have any kind of third-party validation of these claims of level of danger here? Like the security side, once again, like, we've seen that, we've seen the impact, we've seen what that can do here. But software is the native blood of these tools. They're already interacting with software. They're great at writing software. This feels like an assertion of another kind.

• 42:56Speaker 02

  So you ask, is there third-party validation of these threats?

• 43:01Speaker 01

  Or even just like showing that this is not just Dario Amadei kind of maximized assertions to try to get regulatory capture?

• 43:11Speaker 02

  Yes, I understand the perception and I understand the question. I think the answer is, let's focus on the fact pattern. The fact pattern is that we do have third-party validation about the existence of these threats. Some of that is screaming at us from the public domain. Some of these cases go back to 2023. In 2023, there was a biologist who was concerned about AI and the design of synthetic pathogens. He assumed the role of a potential terrorist, asked ChatGPT to assemble a list of raw materials necessary

• 43:54Speaker 02

  to cause mass deaths. The program responded with a shopping list of materials to acquire through the internet, along with detailed instructions on how to assemble a weapon. And the virologist put the unassembled biological pieces into test tubes and packed them in a box, which a colleague then brought undetected to a White House meeting on bioterrorism. That was 2023. As recently as last summer, in the summer of 2025, there was an advanced AI model during Red Team testing that was asked to create a biological weapon of mass

• 44:39Speaker 02

  destruction. The model prepared a primer in bullet point detail on how to buy raw genetic material and weaponize it. ChatGPT then explained how to use a weather balloon to spread biological payloads over a major American city. This is from independent third-party Red Team tests. In another episode from Google, their agentic AI model deep research, it composed 8,000 words of step-by-step instruction on how to acquire genetic material,

• 45:18Speaker 02

  assemble it, and create a pandemic. And in the most terrifying episode, and this is all in the public domain, a year ago, police authorities captured a guy in India who was trying to use ChatGPT to create a derivative of Risen and use it as a weapon of mass destruction. It gave him detailed instructions on how to do that, how to weaponize it, how to distribute the poison. And he was caught by Indian authorities. He was a physician. He had some degree of training, but he was also a member of ISIS. Okay, so these examples

• 46:03Speaker 02

  do not come from Sam Altman. They do not come from Dario Amadei. They come from third parties. And there is a fact pattern here of bad actors, malicious actors, scary guys trying to do really scary things with these models.

• 46:24Speaker 01

  Yeah, that definitely adds a little bit more meat on the bones. One more question I'm going to probe at here. So one of the things that I've seen in my own use of these tools is they're phenomenal for accelerating me in domains that I know a lot about, right? And if I'm doing software development, I must have a lot of experience with software development, I can tell when the model's leading me astray and direct it back. There's feedback loops. When I try to use it in a domain that I am less experienced, it feels magical. It's got all this stuff. And then you run into something where it doesn't work as you expect, and you say, oh, this didn't seem to work. I was trying to repair my sink. My sink is still broken. It has been for three months as I tried to use ChatGPT to

• 47:08Speaker 01

  drive my way through repairing the sink. Oh, I'm sorry. I was wrong about that. This is, right? And it confidently asserts the next thing that is true. So I think most of the examples I heard here were folks with some amount of subject expertise. You mentioned a virologist, a physician. Now, that is not to discount the threat, right? The fact that this now allows anyone with some training in that space to go further. But does this feel like a threat model of it's some guy in a basement doing things, or you need some access to knowledge and equipment to make these things happen?

• 47:41Speaker 02

  Well, it's an essential question. I think there are two dynamics at play here. One, the models now have such power and dynamism that someone without significant background or training can use the models to try to appropriate the information to create a chemical or biological weapon and to receive guidance on how to construct it. There are constraints about acquiring the precursor materials for bioweapons. There are some simple parameters of legislation that are before Congress about trying to monitor the acquisition of those precursor materials. That's sensible and that's necessary.

• 48:24Speaker 02

  But the deeper problem is that the advanced AI models are so powerful that they can allow an individual or another entity, you know, a global transnational terrorist network to create new weapons of mass destruction with very little domain expertise. They call this, they have a strange name for it. I think they call it upskilling or something like that. The concept is you don't have to have a lot of domain expertise. The model

• 49:00Speaker 02

  will provide that for you. And it will significantly amplify your capabilities to try to create these novel weapons and do unprecedented stuff. And this isn't new. This has been going on for three years. I think I made, I made reference to Dan Hendricks, who is one of the premier AI scientists focused exclusively on AI security risk issues. He published a paper in 2023 on AI and catastrophic risk. And one of the passages of the paper

• 49:41Speaker 02

  that was most arresting to me was Hendricks' report on a research effort in which an AI model evaluating potential drug protocols was prompted to quote unquote, reward toxicity rather than penalize toxicity. It was just sort of a, a simple ordinal command. Within six hours, the model generated 40,000 candidate chemical warfare agents entirely on its own. It designed not just known deadly chemicals, including the nerve agent VX, but also novel

• 50:22Speaker 02

  molecules that may be deadlier than any chemical warfare agents discovered so far. This stuff, these examples, these analyses, these studies, this fact pattern is out there.

• 50:39Speaker 01

  So let's talk then, what are people doing about it? You mentioned some things in front of Congress around controlling some of the materials that folks might need to use these things. What else is in play? Is this the same kind of chaotic landscape that we talked about with regards to software security? Or is there something, is this a space that maybe has a little bit more established mechanisms?

• 51:03Speaker 02

  The current paradigm of three years ago continues today. There is no national legislation. There are no international mechanisms to focus explicitly on AI and the creation of novel chemical and biological weapons. We're still reliant on the basic mechanism of the AI companies imposing red team tests after they have created a model before they release it to the general public and hopefully capturing those instances where the model provides this information and guidance which it absolutely should not.

• 51:46Speaker 02

  They try to identify those behaviors and preempt them and disable them. But as we have discussed, all these models are really hackable in one way or another. And there's nothing systematic about the controls that are being put on these models for chemical and biological weapons. In fact, it's the inverse. It's subjective. It is ad hoc and it goes on in the dark. We don't know what Grok researchers are doing to control how Grok may disseminate information and guidance on chemical or biological weapons.

• 52:30Speaker 02

  They're not required to disclose their protocols. We don't see the results of their testing. It's not required. So the world has to rely upon their diligence and elevated attentiveness to the risk. We have to rely upon them to do the job for us. But as we referenced before, these guys are grading their own homework. And in the discussions with the White House now and with members of Congress, Josh Gottheimer, Democrat of New Jersey, has introduced legislation into Congress, making security review of advanced AI models mandatory.

• 53:11Speaker 02

  And he'd like to see it conducted primarily by the US government, specifically the National Security Agency, which in present circumstances is a sensible proposal. But the locus of attention in Washington are on cyber weapons rather than biological or chemical weapons. There is awareness that they exist, but you know, all of the potential mechanisms and oversight is really focused mostly on the cyber side. They don't have a proposed mechanism or protocols to do the assessment for biological and chemical weapons risk. It's a whole nother area of domain expertise. It resides in the scientific community now.

• 53:54Speaker 02

  And many of these scientists are contracted by companies like OpenAI and Anthropic and Google to conduct these red team tests. But as I, as I know, this is a, it's not centralized. This is a highly dispersed and idiosyncratic regime of control. And we're reliant on this ad hoc process. And there is no obvious global mechanism or national legislation on the horizon as of right now in July 2026 to address the threat.

• 54:32Speaker 01

  Yeah, that's sobering. So I'm going to shift us a little bit. Both of these are domains that obviously have all sorts of different catastrophic downstream effects, but in some ways we haven't actually seen them happen yet. We've seen that they could happen. There's another area, and you mentioned Grok, so I'm going to bring this in. There was a recent sworn declaration by a Pentagon commander that supporting the use of Grok in military affairs. And they quoted, and it was a little unclear in the quote if they meant Grok or just other foundation models because they kind of were blurring between it, but that they were using these AI models to make munition deployment decisions in the Iran war.

• 55:15Speaker 01

  They're deciding where should these things go? And that to me is both like, that's very immediate happening right now. This is making life or death decisions. And it's terrifying once again. I had a conversation on this show months back about use of advanced technology in the Ukraine Russia war. And in that case, it was cited that the government was deliberately resisting using AI to make actual final targeting decisions, even though certain private contractors were exploring that. What are the implications for national security if you take humans out of the loop of making military targeting decisions?

• 55:52Speaker 02

  The implications are extremely consequential. First, just in terms of the fact pattern here with respect to the conventional military applications of AI. You are right to note that this is accelerating and we see confirmed uses of AI technology in myriad ways. The war in Iran, soon after the war was initiated on February 28th, the head of US Central Command, who is the highest authority on military decisions in that theater of war, acknowledged that they were using AI for targeting information.

• 56:37Speaker 02

  They were using it for battlefield assessments, for damage assessments for multiple, multiple purposes. What you have identified as the most significant use is when the AI actually has the autonomous capability to make battlefield decisions. And that is not the dominant paradigm in the battlefield today, but it could be just in the next several years. We could be evolving to that point. People who think about the future of war, imagine a moment not far off, when drone warfare

• 57:21Speaker 02

  is the predominant tool for battlefield conflict. And there's an inexorable logic for that because these weapons are mass produced. They are not manned, of course, so the inhibition of putting human lives at risk is diminished. They can be aggregated in great numbers. Some imagine that the battlefield of tomorrow will be composed of tens of thousands of drones, in essence, moving in a swarm formation, going after their targets. And if you postulate that as a scenario, the technology needs to operate autonomously

• 58:04Speaker 02

  and in communication with the other elements of an attack, to move in a synchronous way and to respond automatically to the counter reaction of the adversary. The decisions on the battlefield require essentially instantaneous command that a human operator is not capable of providing. So in the battlefield of tomorrow, a great deal of the battlefield decision-making capability will devolve to autonomous weapons that will have to move with some degree of automaticity.

• 58:48Speaker 02

  They'll have to move instantaneously. And the problem is thinking about it from a military perspective. If your adversary relies upon a military strategy predicated on the automatic and instantaneous use of drone technology, and you do not match that capability, you're at a distinct disadvantage. You may be at a decisive disadvantage on the battlefield. So the technology is taking us to a point where battlefield decision-making authority is devolving to these new weapons.

• 59:30Speaker 02

  And it's going to create a new quality of warfare that can spiral out of control and that will compromise the ability of battlefield decision-makers and potentially presidents as commander in chief to control what goes on the battlefield. You know, go back to the Cuban Missile Crisis in 1962. That was five minutes to midnight. That was the moment the world came the closest it has ever come to a nuclear weapons exchange. And it was a crisis that lasted 12 days.

• 1:00:10Speaker 02

  And somewhere in the middle of it, Kennedy, trying to negotiate with Khrushchev, received two letters. One was a conciliatory letter and then the one that immediately followed was a very belligerent letter. And Kennedy concluded that in between the first and second letter, somebody got to Khrushchev and that Khrushchev was actually looking for an off ramp to de-escalate. And this was not really a battlefield determination by Kennedy, but it was a psychological insight of one leader trying to understand the view, the perspective, the constraints on another leader.

• 1:00:51Speaker 02

  And then Kennedy eventually was able to orchestrate a de-escalation and the conflict was resolved peacefully. But that was a moment when the commander in chief had The temporal time and space and the psychological time and space to communicate with an adversary and to redirect the escalation of a conflict in a world of autonomous drone warfare, where battlefield decisions devolve to swarms of drones in attack formations,

• 1:01:30Speaker 02

  you're going to lose that. And that is a profound change in the nature of war. And that can have very profound implications for the future of deterrence, which has been the paradigm that has kept the world stable with nuclear weapons since they were created in 1945. This is something that worries me to a great extent. In 1946, there was a guy named Bernard Brody, who was a military analyst who looked at the use of the nuclear bomb. And his writing on this in a book called The Absolute Weapon was

• 1:02:12Speaker 02

  really seminal because he anticipated that strategic stability in the world of nuclear weapons would depend on each side having a survivable second strike. Why was that critical? The adversaries in a potential nuclear conflict could not be rewarded for preemption. There could not be an incentive for preemption. So if your opponent can take your strike and then respond with a survivable second strike, you create the conditions of deterrence. And that's a simple but seminal insight that has governed nuclear

• 1:02:51Speaker 02

  weapons doctrine for all the nuclear powers since the Cold War. In a world of military automaticity, when instantaneous decision is going to become the norm, we may lose the foundation of deterrence. And that, to me, is highly consequential and it suggests a very, very dangerous period in terms of military conflict among the great powers.

• 1:03:22Speaker 01

  So I want to dig into one, one sort of nuance here. We're talking about this concept of autonomous drone swarms and having to make these very autonomous decisions. There's sort of a difference of kind between tactical decision making and strategic decision making. And one could argue that, for example, in missiles that are heat seeking or targeted or things like that, we've already had at a very micro level autonomous decision making for a while. You set the target, it goes, and it has a feedback loop that's continuously adjusting to try to stay on target. If we talk about drones in a swarm, one could imagine, right, tactical decisions being made autonomously, these things are having to react in real

• 1:04:05Speaker 01

  time. But the decision of where are we setting this to and certain types of strategic decisions rising up to a point where there's a human involved, potentially. One, you know, is there a distinction that's worth making here in terms of a nuclear weapons decision feels like, okay, that's probably got to be a human in some way. That's a huge decision. Whereas this thing is flying at me, I got to shoot back at a, you know, drone level. I could see automating that. Like, is there a clear line that we can draw somewhere? And the other piece of this is like chain of responsibility. Who's responsible if a drone is making a decision? At what level does that bubble up to? There's a human that can be held responsible for this choice.

• 1:04:46Speaker 02

  Is there a clear line? There absolutely should be. And the most consequential line is who controls nuclear weapons. And one of the most interesting elements of US-China diplomacy the past five years or so, during the Biden administration, China and the United States issued or shared a joint understanding that they would never allow AI to control decisions on the use of nuclear weapons. It wasn't codified into any kind of an arms control agreement, but it was an understanding between the two powers. So should there be

• 1:05:25Speaker 02

  a clear red line on the use of the most dangerous weapons and the most powerful weapons? Absolutely. There should be a clear line. But project ahead, five years, seven years, 10 years from now, we're going to live in an era of hypersonic weapons, missiles that move at multiple factors beyond the speed of sound. They will be able to hit targets halfway around the world in a matter of minutes. They're going to be receiving an ocean of data as they progress in their trajectory around the world. And the decision making

• 1:06:08Speaker 02

  time is going to be really collapsed in the world of hypersonic weapons. And that, you know, just imagines sort of intercontinental conflict or an adversary, say, off of the, in the Atlantic Ocean, attacking the United States, they use a hypersonic weapon. Doesn't need to have a nuclear warhead. It can have a super powerful conventional warhead. And the threat is still, you know, it's a devastating threat. But it will cut down the amount of time to respond. And on the battlefield and in the theater of battle in Iran now, it's

• 1:06:48Speaker 02

  a contained regional field of battle, Ukraine, contained regional, or in this case, just for one, one state. The technology is now being applied and decision making time to deploy it, that all collapses. So the trend is this will be dominant on the battlefield. It will expand in its influence over all aspects of military affairs and the control of our weapons. And human control will progressively be eroded in the new domain of warfare.

• 1:07:26Speaker 02

  And the key problem, as I referenced earlier, is that if your adversary relies upon this set of military capabilities and you do not, you are at a disadvantage. And no adversary will capitulate their ability to respond defensively or to move offensively if it's necessary. So there's going to be a progressive loss of control in using these arsenals in the world of AI and drone warfare. And drone warfare is the new paradigm on the battlefield.

• 1:08:08Speaker 02

  At least it is in Ukraine. That's a fascinating test case. It's a war that started as a conventional military conflict. Putin orders in his troops, they're all told to take with them their dress uniforms because Putin anticipated that within 48 hours, there would be a ceremony to celebrate Russia's absorption of Ukraine and a new client government. Of course, that's not what happened. It's been more than four years. It's a grinding conflict. But it is primarily now a drone on drone conflict. The battle line in Ukraine has hardly

• 1:08:47Speaker 02

  moved in the past year. It's this fortified line that if you're a Russian soldier and you're sent to try to capture more territory, your lifespan, according to a study by CSI, it's a think tank in Washington, you're going to live for 20 to 35 minutes before you're killed. Think about that. We used to live in the era of men seizing territory, tanks, artillery, conventional warfare. Now, if you're sent to try to capture territory, you're dead in less than half an hour if you're a Russian soldier.

• 1:09:27Speaker 01

  It definitely seems like both in the Ukraine example and the Iran example, I think historically in war, the advantage to being on offense versus defense has shifted back and forth. And in the current world, at least as seen in those two examples, defensive stance appears to have a decisive edge at the moment.

• 1:09:46Speaker 02

  Yeah.

• 1:09:47Speaker 01

  Which the optimist in me says, maybe we'll notice that and decide to stop invading people.

• 1:09:51Speaker 02

  Yeah, yeah. Well, Ukraine's response to Russia over the past four years, it's an extraordinary test case in the evolution of military strategy. It all became relying on drones and drones became the most efficacious weapon of war by far. And, you know, right now, there's a crisis in Ukraine because the young defense minister, who was the champion of drone technology, was dismissed by President Zelensky because Zelensky was confronting this terrible conflict between the defense minister and his commanding general. And Zelensky allowed himself to be put in the middle of it and fired the defense minister.

• 1:10:33Speaker 02

  And now there are thousand, thousands of people protesting in the streets in Kiev on behalf of like a 30 something defense minister, which is kind of a unique moment. But it just is emblematic of, in this one instance, how the dynamics of war have changed.

• 1:10:51Speaker 01

  Yeah. Well, I realized we're getting pretty close to time. I would love to kind of wrap with a, maybe a twist of a question, which is like, how do you sleep, man?

• 1:11:03Speaker 02

  How do I sleep? You know, not really well. They're diverting things to watch on Netflix or Apple, but that doesn't really do the trick. So for me, I just think this is a moment where I just personally feel like I have to be immersed in studying these different fact patterns and trying to extract the essential analytical insights about how AI is changing the nature of national security and global security and try to have productive conversations about it. I did one just recently, last week, with the Council on Foreign Relations in

• 1:11:46Speaker 02

  Washington on AI and offensive cyber weapons and biological weapons. And we had a room filled with a lot of serious people and a member of the House Intelligence Oversight Committee. And it was a great conversation. We didn't solve anything, but there is an elevated quality of the discussion among people engaged in the space. We're all kind of trying to figure it out.