Is It Time To Rebuild the Internet?

AI-hacking swarms are poking holes across the internet at such a rate that some cybersecurity professionals have dubbed this month “Black October.” The AI-cybersecurity professional Alex Stamos argues that it’s time to be proactive rather than reactive, and that might mean rewriting the code for the internet in newer programming languages that could make all of our online systems and personal information safer. He joins “Galaxy Brain” host Charlie Warzel to explain why that’s not as big of a deal as it sounds, and how we can all protect ourselves in the meantime. - - - Get more from your favorite Atlantic voices when you subscribe. You’ll enjoy unlimited access to Pulitzer-winning journalism, from clear-eyed analysis and insight on breaking news to fascinating explorations of our world. Atlantic subscribers also get access to exclusive subscriber audio in Apple Podcasts. Subscribe today at TheAtlantic.com/Listener. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Tired of rage scrolling through AI slop and clickbait just to get a handle on today's news? Meet Inpress, the algorithm-free news app that makes it fun and rewarding to stay informed. Track your media literacy and rediscover reality. Download today at inpress.app. That's I, N as in news nerd, press.

• 0:19Speaker 01

  Do you want to hear about visionaries creating a better world? People working in conservation, exploration, science and tech alongside storytellers imagining and creating a better future. I'm Alex Honnold, professional rock climber and founder of the Honnold Foundation, bringing you season six of Planet Visionaries. This season, I speak to guests like National Geographic photographer Tom Pescek and Anna Rappman, executive director of the Jane Goodall Institute, USA, in partnership with the Rolex Perpetual Planet Initiative. Watch Planet Visionaries on YouTube or wherever you get your podcasts.

• 0:49Speaker 02

  What we have to do as, as people who work in cybersecurity is we have to shorten the dark age. We need to make sure the Library of Alexandria doesn't burn, right? That we have to do everything possible to make this as short as possible. And from my perspective, the best way to do that is to be proactive, not reactive here. And to be proactive is we can't just wait for these bad things to happen. We have to intentionally re-architect things.

• 1:21Charlie Warzel?

  It's late October. There's an upcoming election. There's the World Series. There's this endless string of bracing news events. But there's another crisis looming as well. The digital infrastructure that powers our entire on and offline lives is, according to the people who protect it, more vulnerable than ever. The internet as we know it is a very fragile ecosystem. So much of it is built off of code that's written by humans that is, by nature, full of flaws or bugs. Some of our critical digital infrastructure, it's built off of decades-old technology that's running on ancient code languages.

• 2:04Charlie Warzel?

  The internet is a marvel. But it is also, like any massive human endeavor, imperfect and full of vulnerabilities. One way to look at the history of cybersecurity is as a race between people who are looking to exploit all those vulnerabilities and the people who can fortify our systems and keep them secure. But in recent months, AI has threatened to change the dynamics of that race. AI agents, which are cheap, relentless, and highly capable hacker armies, are proving excellent at finding vulnerabilities in code. Sometimes, oftentimes, in places that humans had never thought to look. And the result has been this series of high-profile cybersecurity incidents, each one adding

• 2:48Charlie Warzel?

  to this feeling that we've entered a new era. The rate of disclosures of incidents has been so intense that in recent weeks, cybersecurity professionals are referring to this month as Black October. Last week, the public found out that AI agents found a vulnerability in PHP, which is a coding language that affects 70% of the internet that could allow hackers to gain access to almost any server. On top of PHP, there have been disclosures in major software products like node.js, which is software that's used across countless important webpages and services on the internet. This is stuff that powers everything from supply chains to healthcare companies. And the fear is that what we are seeing here is not some kind of aberration, but the beginning of this new normal, this period of time where everything from our power grids to our personal

• 3:33Charlie Warzel?

  data may be uniquely vulnerable. And so I wanted to understand exactly how unprecedented this Black October moment really is and what it means for the state of the internet and the possibilities of catastrophic events in the future. Alex Stamos is the chief information security officer at the AI company Cognition and a former chief security officer at Facebook. He's not only on the forefront of this current crisis, but he's also somebody who understands how massive digital organizations respond to the demands of an increasingly insecure internet. I'm Charlie Warzel, and this is Galaxy Brain. Alex Stamos, welcome to Galaxy Brain.

• 4:16Speaker 02

  Hey, thanks for having me, Charlie.

• 4:19Charlie Warzel?

  So we're in the middle of what feels right now like this explosion of either AI-powered hacks or cybersecurity disclosures right now. People in your industry and colloquially on places like X are referring to it as Black October. Pretty ominous. Can you walk me through what has changed fundamentally in the last two weeks?

• 4:40Speaker 02

  So this has honestly been a slow burn, but what's been really interesting in this month is that finally attackers are getting the benefit from AI where they have the ability to use AI to automate what we call kill chains. So this is a term we have stolen from the military where when the military says kill chain, they mean to actually kill something. In the cyber world, those are the steps you take to break in to a network and to do something. And traditionally, you could automate some parts of a kill chain, but there's often human steps.

• 5:21Speaker 02

  And just in the last several months, we've gone to the point of where AI models, and especially the open weight models, the models that you can run locally on hardware you run yourself, and even commodity hardware, like these new Mac studios that you can buy, are good enough that basically every step of the kill chain can be completely automated with AI. And so that has really changed the game here where you can get rid of any human bottleneck. And that has completely changed the economics and the speed at which attackers can move.

• 5:56Charlie Warzel?

  So I've heard of this being talked about as a disclosure wave, meaning that it's a disclosure of potential vulnerabilities and not a wave of attacks. But what's, what's the truth to that?

• 6:07Speaker 02

  Yeah, I don't think that's true. We've actually had a wave of attacks, but it's, it's way before this October. We've had a building wave of attacks over the last year or so as attackers have automated their kill chains. There's a good thing happening, which is these vulnerabilities have existed for decades, right? We have, you know, years and years and years of human-written code that is not reliable, that is not safe, that is often written in languages that are not what we call memory-safe and type-safe, that are written in computer languages that were not fundamentally easy

• 6:50Speaker 02

  to write safe code in, languages like C and C++. And unfortunately, it turns out that human beings writing code in those languages, it's very hard to make systems that are safe in the presence of these superhuman bug finders. And so what, what's happening is we are now burning through all of these bugs that were written over two decades, and they are being found and discovered and patched. But the problem is, is once those patches are released, actually getting them fixed in the wild takes a while. And so bad guys have a window during which they can then exploit them. And over the last year, we have seen that, that exploitation happening, sometimes manually

• 7:35Speaker 02

  and more and more automatically. And so, yes, we, we have seen waves and waves of exploitation. It's not being covered as much, but it is extremely hard if you have a security emergency on a Friday evening, and it's often on a Friday evening, unfortunately, that's how it often seems to work out. It is very hard to get professional incident responders on the phone because everybody is booked up pretty, pretty far out right now.

• 8:02Charlie Warzel?

  So all of this news, as, as an outsider, is, is really bracing to just see come across the timeline. I just think it's very clear for a lot of us without the expertise to evaluate some of these claims. The, the one that I saw the other day that I want to bring up here is that the reports, the cloud service from Japan's GMO Internet Group, which is used also by financial traders, had been hacked. And in one report that I saw circulating, the CEO of GMO suggested that the internet as we know it could be on the verge of collapse due to this pressure. When we hear something like that, how are we supposed to evaluate that kind of claim? What is, what is the state of, of the internet right now?

• 8:40Speaker 02

  Okay, so to say the, the internet is going to collapse, I think is hyperbole. The, the fundamental infrastructure of the internet is holding up fine, right? Like, the internet, the routers, the switches, there have been bugs in the fundamental systems that move traffic, but those have been fixed. I think what is happening, however, is we have lived through a period where you have been able to assume that if you use a service, that your data is going to be safe. That assumption was never correct, it turns out, right? The systems were never safe.

• 9:23Speaker 02

  It's just that we lacked the number of skilled attackers to take advantage of the fundamental insecurity of these systems. And, and what is happening now is the bugs are being found, sometimes by the bad guys, often by the good guys. But in the period in which it takes for the bugs to be fixed, the bad guys have an opportunity to take advantage of that period of time, of that lag of fixing them. And then the bad guys, the skill sets...

• 9:55Speaker 02

  You know, I don't know if you're like an English soccer football fan, but everybody gets this metaphor now because they watch Ted Lasso.

• 10:01Charlie Warzel?

  Sure.

• 10:01Speaker 02

  But you know, right. So like the the English football leagues, if you do really well, you get sent up, right? If you do poorly, you get relegated, you get sent down. Well, what's happening is every attacker is now getting sent up a league, right? And so, you know, it used to be the Premier League was just the a handful of really big countries. So the United States and our allies called the Five Eyes. So the United States, the UK, Canada, Australia, New Zealand is called the Five Eyes. Russia, China, Israel, couple other Western nations were up there at the top. And then you had the next tier, which was like Iran, North Korea, a couple other second tier countries.

• 10:47Speaker 02

  And then you had the really good independent professional commercial groups. And what's happening is everybody's going up a league because of AI, which in some ways is bad for the people at the top because they're losing their their premier status that everybody else is kind of joining them. The skill sets that used to be really unique are becoming much less unique because of AI. And so I think that's that's what we're seeing now is that skills like taking a patch from Microsoft and reverse engineering it and turning it into exploit code. Maybe 150, 200 people used to be able to do that five years ago. Now, a 19 year old in St. Petersburg can do that with a M5 Mac Ultra.

• 11:32Speaker 02

  That's that's that's kind of, and that's really crazy.

• 11:36Charlie Warzel?

  There's a striking anecdote that I've seen floating around that the PHP vulnerability that affected 70% of websites on the internet was independently discovered by someone using a frontier model, using the prompt, quote, "find bad vulnerabilities in PHP." And I think that sort of speaks to that idea, right? That everyone is elevated, everyone has these skills they didn't have. But if people without even like real basic knowledge of these systems or skills can do this, like, how is this sustainable for, you know, cybersecurity?

• 12:12Speaker 02

  Yeah. So what's going to have to happen, so like you said, what's happening is you have these bugs that will be found by somebody with access to a preview version of a cyber model. You know, we do this thing called coordinated disclosure, where sometimes if you have a really big bug like that, it used to be you could wait 90 days. But in those 90 days, the models might get better that everybody has access to that you can't wait, right? Because other people can just find the bug. So the way we're going to get through this is I think what we're going to have to do is we're going to have to rewrite a bunch of code. So I actually gave a talk earlier this year where I posited two scenarios, which is the good and the bad scenario.

• 12:55Speaker 02

  And the good scenario was that there was a finite number of bugs and we were just burning them down. And we would asymptotically approach as new models came out, you'd have fewer and fewer. And the bad scenario was every time we have a new model, you get the equal number of bugs. Unfortunately, we're closer to the bad scenario still, which is every time one of these new models comes out, we still have a huge number of incredibly complex memory management flaws. And so I think for these really complex code bases, we're going to have to refactor them, we're going to have to rebuild them in languages like Rust, where you have the ability to, one, there's entire classes of bugs they don't have.

• 13:37Speaker 02

  And then you also have the ability to effectively have kind of contracts where you say, I am designing it to have these properties. And you can mathematically prove that it has those properties. There's a thing called formal verification, and you don't have to do true formal verification, which is really hard. And it would have been basically impossible to do this with humans or incredibly expensive. But with LLMs, you can. And so I think we're just going to have to basically completely rewrite stuff. And that is actually going to be cheaper with AI than it's going to be to do 1,000 bugs every six months. We should just redo these things.

• 14:17Charlie Warzel?

  But let's talk about that. How long does something like that take, rewriting the code of the internet?

• 14:26Speaker 02

  Well, less time than you might think. I think that's actually one of the things that's changed with AI. So I work at this company called Cognition now. And we have a product called Devon. And we work with companies and we have these examples, like we worked with Mercedes Benz and we rewrote one of these apps of theirs that would have taken months and months for humans to do, and we did it like eight days. So I think actually this is one of those beliefs that software engineers have, that it's impossible to do this. And it's actually quite plausible.

• 15:01Charlie Warzel?

  I've also heard you say, and maybe these are the same thing, but I've heard you say that we're entering a dark age of cybersecurity. Is that what you mean by that? And that sounds really kind of ominous.

• 15:15Speaker 02

  Yeah, I mean, we are. I mean, I've gone a little crap for this, and maybe that's deserved. But like, one, the dark ages, we came out of them, right? So first off, the dark ages were only for part of the Earth. Like, the dark ages happened in Europe and things were like fine in China. Like, you know, things were great, they had a flourishing civilization. But in Europe, you know, we had Greek and Roman civilization, and then things got kind of rough for a while. And we forgot about a lot of things. But then we had the Enlightenment and we had the Renaissance. And so my point is that what we have to do as people who work in cybersecurity is we have to shorten the dark age. We need to make sure the Library of Alexandria doesn't burn, right?

• 15:59Speaker 02

  That we have to do everything possible to make this as short as possible. And from my perspective, the best way to do that is to be proactive, not reactive here. And to be proactive is we can't just wait for these bad things to happen. We have to intentionally re-architect things. We have to intentionally rewrite things. We have to intentionally change how we do cybersecurity. We have to intentionally change how we build the networks. We have to intentionally change how we build security teams. Right now, we've been very reactive. We still do patching like we did five years ago. Most security teams are still built like they were built five years ago.

• 16:43Speaker 02

  Like, the jobs are the same, the defensive structures are the same, how we do incident response is the same. And I think that's a problem. And when I say shorten the dark age, what I mean is we've kind of got to actually utilize AI to proactively, we've got to make this better. Otherwise, we're just going to try to patch, patch, patch, patch, patch our way through it, and that's not going to work.

• 17:20Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait and AI noise. Think social media. That's where Inpress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with Inpress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.

• 17:54Speaker 01

  Do you want to hear about visionaries creating a better world? People working in conservation, exploration, science and tech, alongside storytellers imagining and creating a better future. I'm Alex Honnold, professional rock climber and founder of the Honnold Foundation, bringing you season six of Planet Visionaries. This season, I speak to guests like National Geographic photographer Tom Pechek and Anna Rattman, Executive Director of the Jane Goodall Institute, USA, in partnership with the Rolex Perpetual Planet Initiative. Watch Planet Visionaries on YouTube or wherever you get your podcasts.

• 18:25Charlie Warzel?

  In the scenario you're describing, rewriting things with today's better code, do we just end up, if we do that, in the same situation later when more powerful AI models find these new vulnerabilities? Or that the code is, you know, allows for these like superhuman style bugs that we, you know, we are incapable of kind of comprehending? Is that a possible scenario?

• 18:47Speaker 02

  I don't think so. I mean, one, I mean, it's hard, like, now we're getting to like metaphysics. When you talk about like these superhuman capabilities. But I don't think so. I think if we start rebuilding things using languages that eliminate these entire classes, and then you start using some level of formalism, then it's much, much, much less likely that that's true. Certainly, what we're doing right now is not working, right? I think it's total malpractice to write anything in C and C++. Any company that is shipping code that is written in C++ in 2027 is, from my perspective, that is just... Completely inappropriate.

• 19:33Charlie Warzel?

  And for the people listening who aren't as familiar, the reason for that is just because it's it's an it's an old outdated language and you don't want to have your systems be running off that. It's easier to exploit all that, right?

• 19:49Speaker 02

  Right, because the the language C++ C and C++ these are languages from the 1980s, 70s, actually, and 1980s. And in fact, in the original book for the C programming language, which I was taught at Berkeley out of this book, in the original printing, there was a security flaw in the book that was written by the original authors of the language, right? Kind of famously. You can make mistakes because these languages are what's called low level and you have to intentionally decide how you talk to the processor and how you handle memory. And so a lot of the kind of fundamental flaws that we see that get exploited these

• 20:30Speaker 02

  days is about deciding how you manipulate memory, the memory of the computer. You say, I want to allocate memory, I want to deallocate memory. And the the bugs that we see, like the PHP bug, these are memory allocation errors. Modern languages, these things are handled for you automatically. And in really modern languages like Rust, you have the ability to say, not just it handles the memory for you, but you can effectively say to the compiler, the software that takes what you say and then turns it into the language the computer actually executes, you can say, this is what I intend. This is what I want to happen. And

• 21:13Speaker 02

  so there's double checks, triple checks to make sure bad things don't happen. Now, that would have been totally improbable in the 1980s because computers weren't fast enough. It would have added, made the computers way too slow. But computers are incredibly fast now. And so these languages are much slower, but who cares? Because computers are spectacularly fast. We have way, way, way more speed. And so we sacrifice a little bit of speed for a lot more safety. And it would have been completely improbable to rewrite Windows, to rewrite Linux, to rewrite the Cisco kernel code, to rewrite all of all of the stuff that powers, all of the electricity, the light that you and I, that's shining on us, all of the little processors that power the hydroelectric plant that's that, you know,

• 21:56Speaker 02

  because you're in the Pacific Northwest, so you're probably getting hydro power from from Washington and Oregon and British Columbia. I'm getting wind power and solar power in California. Those systems are from PLCs, from Siemens and Schneider Electric and companies like that. That's all written in C code. That stuff could be rewritten in Rust now. It just means you have to pay to re to replace that stuff. It will cost money. But that is worth it for our society to continue to operate over the next decade. And and I think that's that's worth it. And we but we have to proactively decide the society to do that.

• 22:32Charlie Warzel?

  So let's talk about that. You obviously mentioned like power grid systems, some of this very foundational infrastructure, running off of some of this old stuff. We need to do something about it. You also said early on that people are overwhelmed, right? That there's so much happening. There's like, do we have the resources to do this in an orderly fashion to respond to the level of the threat right now? Because I think what I am seeing is is these concerns from people, perhaps they're just engagement farming on places like like Twitter, but they're talking about vulnerabilities to the power grid. So they're talking about vulnerabilities to these, these very critical pieces of infrastructure.

• 23:17Speaker 02

  I mean, what do you mean by the the resources? Certainly. So one, Twitter is not real life. I think like that continues to be one of the most important things that we have to remember, right? Like, the capability absolutely exists because modern AI systems have the ability to do this. What we haven't done is created the incentive structure for first for the people who run the power grid to put the pressure on their vendors and for the vendors who build these things to build systems this way. And the economic structures for us to actually upgrade all this stuff. We haven't created those economic incentives yet.

• 23:56Charlie Warzel?

  Isn't the incentive structure for the like, the power grids that you don't want someone to find a vulnerability, you know, the 19 year old in St. Petersburg and and take down the power grid? Like, aren't we? Aren't we there? It feels to me like we're there.

• 24:11Speaker 02

  Yeah. So, okay. So the, I mean, I do a lot of work with some infrastructure providers. And of all the infrastructure providers, the power companies are actually the best. It's one, they're quite large. Like, we've seen all these attacks against water utilities. And they're, for example, in much worse shape because they're much smaller organizations. They're often municipalities. A lot less time and money has been spent on water, for example, or sewer systems. So you're right, power is the best shape. And they do have the economic incentive. I think there's still a bunch of coordinated action issues in

• 24:46Speaker 02

  that there has not been enough work by them to to put pressure on the vendor ecosystem to do all the right things. So, yes, the the power grid people have done good work here, but most of the work they have done is on isolation, right? So effectively, how they have done security is they have basically decided that the stuff they buy is crap and that they will make sure that attackers have no ability to reach it. That is a reasonable protective strategy for now, because that is all they can do. But in the long run, we need to rebuild these systems and we need to make them safer. Because against the bright

• 25:31Speaker 02

  adversaries, you can't actually physically isolate a power grid, because against adversaries who are advanced enough, they will physically infiltrate and send somebody up the power pole and then tap in to that physically isolated network. And I think people spend too much time on the power grid because it is the thing that worries everybody. There's a lot of other things that like, if if they go down, like hospitals, like hospitals get slaughtered all the time by ransomware actors, right? And those things are really critical. And they're also very hard things to disconnect because we all have electronic medical records, right? And especially community hospitals, things run by counties are extremely poorly resourced compared to things like the power grid from a security perspective.

• 26:14Charlie Warzel?

  Yeah. What are some of the the things that you have seen in in recent months, the either the disclosures or the actual attacks that that have you the most concerned right now?

• 26:25Speaker 02

  Yeah. So I mean, I think a couple of things that were really interesting and indicative of where we're going. First, there were these attacks against South Korean banks using fully automated AI attack systems called RTEx. So there are these AI attack systems that use open weight models to do all of the attacks. There's a thing called Voyager that was released like, earlier this year or late last year, that would do AI enabled attacks, but those were mostly used for kind of low level stuff. So if this stuff is now at the point where you can do a full attack against a bank in an advanced and technologically advanced

• 27:05Speaker 02

  country, then we're really kind of there. And so we've now seen a variety of attacks using RTEx since then. And since this stuff is open source and the models are open weight, anybody can take these things off the shelf. So I think that was really interesting. And then the other one is the attacks against fireworks in railway. These look like they were pulled off by Chinese actors who are interested in the American AI ecosystem. So China has what's called the cottage access industry. So effectively, there is a ecosystem

• 27:42Speaker 02

  of private contractors who will break into Western companies on behalf of the greater Chinese economic ecosystem with permission of the Chinese government and somewhat deduplicated and given cover by the Chinese government for Chinese economic purposes. And the fact that these folks are not going after inference, they're not going after customer data, they're going after API keys is really interesting because that looks like they're specifically going after model weights as well as the data used to train AI.

• 28:20Charlie Warzel?

  Break those last things down for me, just because there's some terms I think audience might not be totally familiar with.

• 28:26Speaker 02

  Yeah. So fireworks is a prominent company that provides training services to companies that want to build their own AI models. And so what they were attacked, and the side of fireworks that was attacked was the the side that does training. So and so it looks like who attacked them was groups who were helping out Chinese AI companies. And they were trying to steal information that would then help AI Chinese AI companies improve their AI models. So there's now an entire ecosystem of hackers who are working on behalf

• 29:08Speaker 02

  of of Chinese AI companies trying to improve and take data that will help them accelerate their work.

• 29:25Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait and AI noise. Thanks, social media. That's where Inpress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with Inpress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.

• 29:59Speaker 01

  Do you want to hear about visionaries creating a better world? People working in conservation, exploration, science and tech alongside storytellers imagining and creating a better future. I'm Alex Honnold, professional rock climber and founder of the Honnold Foundation, bringing you season six of Planet Visionaries. This season, I speak to guests like National Geographic photographer Tom Pechek and Anna Rattman, executive director of the Jane Goodall Institute, USA, in partnership with the Rolex Perpetual Planet Initiative. Watch Planet Visionaries on YouTube or wherever you get your podcasts.

• 30:30Charlie Warzel?

  I want to talk a little bit about the pace of innovation here because it seems like what we're, the moment we're heading towards is the good guys are deploying helpful, protective AI, we're rewriting the code using the AI to protect against the people who are using this, using AI agents and systems to attack. How worried does that make you, given the advancements of these agents? The fact that we're seeing them behaving either unexpectedly or communicating in these, you know, these advanced ways. There was the news recently that OpenAI is no longer conducting frontier model training with internet access because the models have been able, you know, to communicate, share information without the

• 31:14Charlie Warzel?

  internet. Again, it feels like this is all moving really fast from the innovation standpoint of understanding what these things can do, how much control, you know, humans have in these systems. And yet they're also, you know, going to be deployed in this fight on both sides. Like, how much are you worried about the speed with which this is all moving?

• 31:39Speaker 02

  This is the crazy challenge because we want to have really capable models to help us do cyber defense. In creating these highly capable models, we want to do so in a way that doesn't cause damage. You know, OpenAI released, like, I think, a very good blog post in the work they're doing to create secure evaluation frameworks. So that's where I think you're referring to. What they did was they asked their best cyber models to then red team their protections and it found a bunch of ways. They found ways to sneak information between their different isolation mechanisms. They did timing attacks, they're

• 32:23Speaker 02

  called and stuff like that, which is, you know, the right thing for them to do. And I'm glad that they documented this because it helps the rest of us who do model training, at Cognition, we do model training. So it was very helpful for them to document this and we're working with them and Anthropic to try to align how we all do this work. It's really important to continue to push the frontier forward. I think there's honestly been, it's great to talk about hugging face and stuff. Models escaping from frontier labs is not the long-term risk, to be frank. The long-term problem is models being misused by humans to do bad things. I think, I am not a P-doomer of thinking like the models

• 33:05Speaker 02

  themselves are going to have their own desire to do bad things. I think there are

• 33:12Charlie Warzel?

  Sure, it's a human problem, yeah.

• 33:13Speaker 02

  Yeah, I think there are, you know, I think the labs have, are coordinating on like reasonable limits of what we do in training, of putting, making sure that these things don't encrypt their chain of thought, making sure that there are reasonable limits put on them. I am much more worried about the capabilities that are released and then, and to that extent, I think what we need to really align on is, is how, and I think this is actually quite difficult, we have to provide cyber capabilities for defense. How do you do that without allowing those cyber capabilities be used for offense in a negative way? And that's spectacularly hard. And the other problem here is that

• 33:58Speaker 02

  governments, especially the United States government, have not defined what kind of liability companies have when they do this. And so in the defensive cyber world, we complained a lot about the restrictions put on the frontier models here.

• 34:11Charlie Warzel?

  But let me push just a little bit. That all makes complete sense. But it feels to me, and I think it feels to people who are just on the outside of all of this looking in, that this doesn't feel like a sustainable moment or the things that, you know, the speed with which this is all happening is very concerning. It makes the challenge harder and harder. Like, help me understand the logic of, we have to push this forward. Also, we're starting to see things accelerate in a way that is genuinely concerning, that, you know, that we're, South Korean banks are now, you know, on the table here. It

• 34:52Charlie Warzel?

  feels like there is a, like a disconnect in that logic, I think, to lots of people. So I'd love for you to just, you know, explain why we should continue to move forward with this, given those fears.

• 35:03Speaker 02

  Look, I work on cyber defense and, one, these bugs already exist, these vulnerabilities exist. And so I don't think that there is a good way to just kind of pause the world at this point, because the capabilities that exist at this point are enough to find a huge number of flaws. So a tiny, tiny portion of the code that's ever been written has now been scanned by the high-end cyber models and bugs found and/or exploited. A tiny portion, right? Like, part of the problem here is the coordination problem. If I could pause

• 35:45Speaker 02

  everybody to try to figure out a safety regime here, I would. But I don't have that power. And so personally, what I would like to do is, I would like to focus on finding these bugs and fixing them. I would like to focus on building better defensive capabilities and then especially on the cost issue, because the cost of like the high-end defensive capabilities is way too high. You can't afford to run Mythos against all of this code and fix it. So we have to focus on lowering the cost and fix of both finding the bugs and especially fixing them. And so that's something we're really focused on at Cognition, is like the

• 36:27Speaker 02

  Pareto frontier between cyber capability and cost. So you're right, that is the issue. I don't have power, I don't have power, sorry, Charlie, to fix that. And so from my perspective, I'm working on, what I'm personally working on is just trying to then deal with the reality of the frontier is moving incredibly quickly and how do we, how do we deal with that fact? And what we do is we try to fix those things as fast as possible and then build the defensive models as quickly as possible.

• 36:58Charlie Warzel?

  Are we maybe also seeing some resiliency of the systems? When you think about the PHP bug, the big splashy one from the last couple of weeks, it hasn't really produced, like, any big mass compromises, right? Like, I haven't seen the follow-up story of, you know, people are in and all this information got stolen. The response to it seemed pretty orderly, pretty, pretty well coordinated. Is that, is that the system working or is that just kind of an aberration in a quiet before the storm?

• 37:30Speaker 02

  We got very lucky. The PHP bug turns out to be quite hard to exploit. Also, a lot of those, 70% of sites are behind CloudFlare or another CDN and those guys put in protections because of the coordinated disclosure process. So we got, I think we got lucky with the PHP bug that while it, in theory, is exploitable, in practice, it was much less exploitable than it looked in the first case. So it looks like it was kind of a near miss. The asteroid that grazed the atmosphere and gave us a good warning. Yeah, so we got lucky on this one. But what you saw with the hugging face bug was OpenAI's agent swarm found bugs in

• 38:12Speaker 02

  hugging face that were specific to hugging face, wrote the exploits and then just threw them away and then found new ones. Because these things are so fast that they can do that. And so that's much more what I'm worried about than the off-the-shelf bugs, is the AI agents finding custom stuff for each web app, for each API.

• 38:35Charlie Warzel?

  So given everything that we've talked about, I want to end here by, again, the people who are consuming this news from afar, I think, are a little overwhelmed by it. In some cases, if they're not a cybersecurity professional, there's not a ton that you can do to help out in this ecosystem. But what should people be doing right now in this moment on a personal level to shore up, make sure they're as ready and prepared as possible for this more insecure version of the internet?

• 39:08Speaker 02

  Yeah, so it's honestly, it's the same stuff you should have been doing five years ago, but now it's just more important, right? So first thing is, the number one way any normal person gets hacked is by reusing the same password everywhere. I know you would never do this, Charlie, or anybody who's watching this podcast, but a lot of people use this, yes, use the same password in lots of places. You don't do that, because what happens is some bad website gets hacked, and then the password gets stolen, and then that happens to be password you use for your Gmail or your bank. And especially if your email gets taken over, your life's over, because all of your passwords get taken over. So use a password manager, something like OnePassword, and go reset all of your passwords, make sure they're all different, and then have one really good password for

• 39:53Speaker 02

  your password manager and keep that in your head. Or even write it down on a piece of paper. A password on a Post-it note in your wallet or your purse can't be stolen from thousands of miles away. So change all your passwords, have them all be different. Use pass keys if possible, right? So have a pass keys in the password manager or on your phone is really key. The second is, just, people are just going to have to be more paranoid. So both because of all the theft of personal information that's going to happen, as well as the rise of AI, voice AI, the creation of video, we're going to see a real rise in scams that are very realistic. So people are getting phone calls that sound like their

• 40:35Speaker 02

  grandchild has been kidnapped. People are getting more spam text messages. If it seems too good to be true, if it seems like something is fake, it probably is. And so people just need to be more paranoid. Go help your parents and your grandparents be more paranoid. The third thing is, if you can use a limited computer, do it, right? So I bought my-

• 40:57Charlie Warzel?

  What is a limited computer?

• 40:59Speaker 02

  I bought my parents and my in-laws Chromebooks for Thanksgiving one year, and it made my holidays so much better, right? So if you just use a web browser all the time, then you do not need a full Windows computer or Mac. If you're just in Chrome all day or Safari, then you can get a Chromebook, or you can get an iPad, and you can live in that. Those are much more secure than like a full computer that can run malware. And so get a Chromebook or get an iPad, especially for the, maybe the slightly older people in your life, that's a much safer option. And I think what's going to happen, because things are just going to get a little less trustworthy, it's possible that people are

• 41:41Speaker 02

  just going to have secondary sock puppet accounts. So like, have a secondary identity for the stuff in your life that you don't really care about, right? So have that second Gmail account that's not charlyw@gmail. You can have the surferboy1985@gmail, and use that for all the things, for your betting site, and the other things that you don't care about. It's not an idea to have kind of a secondary identity, so that if it's compromised, it doesn't lead back to your bank, it doesn't lead back to the things you care about.

• 42:15Charlie Warzel?

  Thank you for all of the, we're entering a brave new world, and I genuinely appreciate the advice on all this and helping to contextualize it. Alex Stamos, thanks for coming on Galaxy Brain.

• 42:27Speaker 02

  Yeah, thanks for covering it, Charlie. Great to see you.

• 42:37Charlie Warzel?

  That's it for us here. Thank you again to my guest, Alex Stamos. If you liked what you saw here, new episodes of Galaxy Brain drop every Friday. You can subscribe on the Atlantic's YouTube channel, or on Apple, or Spotify, or wherever it is that you get your podcasts. And if you'd like to support this work and the work of my colleagues, you can subscribe to the publication, theatlantic.com/listener. That's theatlantic.com/listener. And just a reminder, we're still prepping for a Galaxy Brain AMA episode. You can send all inquiries about the show, about my work, about AI, about cybersecurity, and how safe your bank account is, pretty much anything. Send me an email at [email protected], subject line, Galaxy Brain AMA. Thanks so much, and I'll see you on the internet.

• 43:22Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait, and AI noise. Thanks, social media. That's where InPress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with InPress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.