AI broke the cybersecurity world. Now what?

I’ve always enjoyed talking to security expert Alex Stamos because he’s so direct — and he doesn't mince words about how the multibillion dollar cybersecurity industry as we’ve known it is all but doomed thanks to the unrelenting, AI-boosted waves of hacks, breaches, and attacks. That much disruption means a lot of places where bad guys can get in while everyone else is left scrambling. Alex isn’t a doomer in the long run, but he does think we've got a dark age to get through before things get better. Links: Attack of the killer script kiddies | The Verge Alex Stamos on AI's coming cybersecurity dark age | Ground Level AI Palo Alto’s CEO Said ‘AI Must Fight AI’ | Palo Alto Networks Cybersecurity vets protest ‘dangerous’ ban on Anthropic’s Mythos | TechCrunch Binance: $6.3B heist used fake job interviews, internal AI tool | The Verge A thousand fewer paths to root | OpenAI Subscribe to The Verge to access the ad-free version of Decoder! Credits: Decoder is a production of The Verge and part of the Vox Media Podcast Network. Decoder is produced by Greg Ott, Kate Cox, and Nick Statt. Our editors are Xander Adams and Ursa Wright. Our editorial director is Kevin McShane. The Decoder music is by Breakmaster Cylinder. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Support for the show comes from Engine. Running a small business means every dollar has to work hard. But if your team is still booking travel the old way, it's costing you more than you think. Engine is the fastest growing travel and spend platform in the country, built specifically for businesses like yours. Book a trip in as little as two and a half minutes, earn up to 10% back on hotels. And in 2025, Engine customers save more than $300 million on travel with zero booking fees, no contracts and no BS. More than 1,000 businesses join Engine every month. Join them and get $500 when your business signs up and starts traveling at engine.com/box.

• 0:47Nilay Patel?

  Hello and welcome to Decoder. I'm Neil Apatel, editor in chief of The Verge, and Decoder is my show about big ideas and other problems. Today, I'm talking with Alex Stamos, chief security officer at Pognition, and a longtime cybersecurity expert about what a mess security on the internet is right now. This episode is part one of a special series looking at AI in real life. I've always enjoyed talking to Alex because he's so direct. And you're going to hear him say that the multibillion-dollar cybersecurity industry, as we've known it, is essentially over due to the unrelenting AI-boosted waves of hacks, breaches and attacks. And all of those are only the beginning. Both script-kitty hackers and also nation-states like Russia and North Korea are breaking into systems all over thanks to AI helping them both detect and exploit vulnerabilities all at the same time. It's got a real frontier feel to it. You're going to hear Alex say it feels like the 90s all over again,

• 1:32Nilay Patel?

  when the whole idea of an always-online world was brand new. That much disruption means there are a lot of places where bad guys can get in while everyone else is left scrambling. Alex isn't a doomer in the long run, but he does think we're in for a, quote, "dark age", while we try to figure out how to adapt to this new era. There's a lot going on in this conversation. We even got all the way to how AI-powered misinformation and SLOP might be managed on our social networks. Before we get going, do me a favor and subscribe to Decoder on YouTube, where we put out new episodes every Monday and Thursday. Okay, Alex Stamos and how AI is upending cybersecurity. Here we go. Alex Stamos, you're the chief security officer at Pognition. You're also a lecturer at Stanford University's Cyber Policy Center. Welcome back to Decoder.

• 2:08Speaker 02

  Hey, thanks for having me, man.

• 2:10Nilay Patel?

  I'm excited to talk to you. It's been a long time since you and I chatted in the, what you might call, the content moderation era of internet policy discourse. I feel like you and I talked a lot about how all that was working, about election security, all those sorts of things. We're in a new era of big problems on the internet, particularly around cyber defense, cybersecurity. You've been thinking about this stuff a lot. You referred to the past few weeks as Security Awareness Month. Tell me what's going on. Just give me the lay of land right now.

• 2:37Speaker 02

  October is traditionally Security Awareness Month, right? Like, lots of companies run events and, you know, both internally and then tell people, you know, "Change your password, use multiple passwords, use a password manager." And what has happened over the last month is we've had, you know, a couple of these big bugs, PHP, Node and such. What's been less discussed is a huge number of companies have been breached. You have a series of AI-related companies, the NeoClouds, that have been breached in succession. You have the Binance hack, which is the largest heist of anything in human history. You have the Cursor incident where, you know,

• 3:22Speaker 02

  one of the things that's going on here is these hacks are being chained together. So it looks like, from what we know right now, one of the companies it was broken into was Fireworks, which is a prominent provider of AI services to a number of companies. From the Fireworks hack, API keys were stolen that allowed people to break into Cursor. Once they got into Cursor, they then pushed a bad plugin, which went to 5,000 more companies, and that caused 5,000 potential breaches. So it's just this kind of permanent, continuous breach that is going on throughout the tech industry. And one of my friends said, a CISO friend of mine, said to a bunch of us, "Is anybody else tired?" And everybody said yes,

• 4:07Speaker 02

  because effectively, we have been at DEFCON 2 for all of October and now going into November.

• 4:14Nilay Patel?

  So I'm going to ask you some really dumb questions here. Who is behind all these attacks? Is it Russia? Is it North Korea? Do we know?

• 4:23Speaker 02

  It's a variety of different groups. It is not just because adversaries have decided, but what has happened is, from my perspective, we are at the start of the takeoff of adversary capability enabled by AI, that the open-weight models have now gotten to the point at which you can automate most of the kill chain. You know, so this "kill chain" term is something we stole from the military, right? In the military, they mean to actually kill something. There are the steps you do to blow something up. In cyber, the kill chain is the things you do to have an effect on a target. So to steal information, to turn something off, right?

• 5:08Speaker 02

  To steal money, to steal Bitcoin.

• 5:10Nilay Patel?

  Yeah.

• 5:11Speaker 02

  Traditionally, you, you know, do research upfront. You look at a piece of software. You find a vulnerability. You write exploits. And you do that all way ahead of time. And then you have a human being go and take that stuff you built and deploy it against either one target, because maybe you're a government and you're going against Lockheed Martin, or you're going against another government. Or you're a ransomware actor and you take all that work you did and you deploy it against hundreds or thousands of targets really fast. Well, now with AI, what's happening is this kill chain has inverted, where you can run an AI system that just goes hog wild and it looks at the targets and it finds the bugs in real time

• 5:56Speaker 02

  and it writes the exploits in real time and it breaks into these companies. So that's one of the things that's happening, is these systems are going and they're so good at finding bugs that they are finding the bugs in real time and writing the exploits in real time. And so the system that everybody's talking about is not the only one. But this isn't the only thing to talk about. It's a thing called RTex, which is a Chinese open-source tool that anybody can go look at. There's an English translation on GitHub that you can check out. And if you power it on the backend by something like an obliterated GLM-5.3, it will let you break into a shocking number of organizations. This started with three South Korean banks. This week, about 200 more companies have been hit. I've been helping a couple of companies.

• 6:40Speaker 02

  Pognition has been kind of, you know, we're not an instant-response company, but we do go in and like, help people fix their bugs afterwards by using our tools. It is shocking the companies that are getting hit by something you could just download. It has been a long time since you've been able to go and just download an open-source tool and break into multibillion-dollar companies. It's like the 90s, right?

• 7:02Nilay Patel?

  Yeah, that's something like Windows in the late 90s, early 2000s, when it was a security disaster.

• 7:06Speaker 02

  I started to say this actually in the beginning of this year, which is like, it's an incredibly exciting and terrifying time, because the last time it felt like this was when I was a much younger man and I would go to the very early DEFCONs. You know, DEFCON these days is like 40, 50, 60,000 people. Back in then, it was a couple hundred people and it was one track. And you'd go in and somebody would give a talk. And you'd leave the room and you'd be like, "Every company in the world is vulnerable to this attack." The difference was, is the entire world didn't run on the internet, right? So it's like, you could go do bad things, but you couldn't ruin the world, right? And you certainly couldn't automate it in the way you can now.

• 7:51Speaker 02

  And it's back to those days where just like, tools you could just go get and build yourself can go do crazy things. It's back to the Wild West days. Except we actually, back in those days, there's like a couple hundred people who had those skills and we were all in one room in Las Vegas. And most of them were actually good guys. They didn't want to be criminals. And now you have a much larger ecosystem of people, many of whom live in places where the law enforcement can't touch and who have a big economic incentive and have access to these tools. And the other big thing is they no longer have to use frontier models that have protections, that have logs that are available to law enforcement.

• 8:33Speaker 02

  They can, I have a M5, my M5 Mac Ultra Studio finally showed up and I've been running like a quantized obliterated GLM-5.3. And oh my God, like, what you can do is incredible.

• 8:46Nilay Patel?

  You mentioned the economics of frontier models, the just, the economy of cyber is one of those things that has gotten sophisticated over the years, right? So you're in these early days, a bunch of people in a room. One of the things that, our systems got safer is you could pay the number of people at DEFCON a whole bunch of money to protect all the systems. And it was economically smart to pay them to be defenders instead of attackers. And then, I'm just rewinding my own brain through this history. There was like a long conversation in our industry about security through obscurity by hiding all the bugs and hiding all the code versus open source and being really transparent. There was an etiquette that developed and about how to reveal the bugs

• 9:30Nilay Patel?

  and when to give people notification. There was transparency reports and bug bounty. Like, we built, you could call it an etiquette, you could call it an economy. We built a culture around cybersecurity and cyber defense and how the attackers and defenders would operate. What you are describing changes it all for a whole number of reasons, but the one that sticks out to me is the economics of it, right? I can go buy a Mac Studio. I can get an open-weight model and I can just start. And that does not place me inside of the culture. There's not going to be some old man. There's, I mean, Obi-Wan Kenobi being like, "Here's how you act like a cybersecurity Jedi." Like, "I'm just going to start doing stuff." Tell me how that's changing right now.

• 10:09Speaker 02

  Effectively, the last era of security, from my perspective, started with the Microsoft, the Bill Gates trustworthy computing memo.

• 10:21Speaker 02

  In the early 2000s, and that is when we started designing how we do software security. And these big companies, Microsoft and Oracle, and then later Amazon and Salesforce, and these folks started hiring ex hackers. And I was part of that generation of the ex hackers who were like at DEF CON and Black Hat, got hired by the companies to go break their stuff. And we started coming up with responsible disclosure, bug bounties, consulting companies. So the Foundstone and At Stake, and then Mandiant and CrowdStrike, these companies grow up and we get this huge industry that is now RSA and Black Hat and this multi-billion

• 11:04Speaker 02

  dollar security industry grew up in the 2000s and 2010s. That era is over, like you said, because it was based upon a couple of assumptions. One was that there is a hierarchy of skillsets of people who could do this stuff, that you had the script kiddies who could download tools to do things. And there then the next level up of the people who could write those tools. And that that was more a more rarefied skillset. And then there was the kind of vulnerability researchers. And then there's kind of the gods. And that god tier was in the hundreds of people. And that is the people who do the really, really good vulnerability

• 11:49Speaker 02

  research, who could write the really, really good shell code, who could reverse engineer patches from a Microsoft or somebody like that. Also, the assumption that there's a limited number of people who could be operational in an attack. And so if you wanted to build an attack team, that you have a small R and D component, and then you have a much larger operational component. And so if you were a professional attack team at US Cyber Command, at Russia SVR, at Chinese MSS, that you build this structure of your R and D team of elite people, and then your less elite operational people. And then that was copied then by the Russian ransomware actors, and that you have to have a conspiracy of people to kind of do this stuff. A lot of this stuff also fell apart with the invention of Bitcoin,

• 12:34Speaker 02

  because one of the assumptions was that you had to have people who could launder money, and that Bitcoin meant that nerds could launder money. So that started to change the economics of the entire thing. All this stuff is falling apart because of AI, because the skillset issue is that now script kiddies have a bunch of the skillset of the David tells and the other cyber gods who could do these things, that with this Mac that's sitting on my desktop, I can reverse engineer a Microsoft patch and walk through and then write stable exploit code. It's not quite there yet to be automatic. You still have to have a bunch of skills, but we're really, really close. And then what's really concerning, and what you see with the Artec stuff, is you can now kind of let loose on the operational side. We

• 13:19Speaker 02

  were never secure. It was all like a fantasy, right? These companies, these vulnerabilities have always existed. These companies have never been safe. And that is what we've seen over the last decade, is these mostly Russian ransomware actors nailing kind of the midsize companies that most people have never heard of, but are actually really important, like some midsize industrial company that makes some widget that we all depend upon. I once did work for like, the largest distributor of medical and dental supplies in the country, and they got really hit badly by a Russian ransomware actor, and then like dental offices across the country were canceling appointments. David make the news, right? But because they couldn't get gloves, because they couldn't get supplies. And it's because, you know,

• 14:04Speaker 02

  a company like that's got like, one security guy, and a 10 person, 12 person IT team, and they're going up against a 20 person Russian ransomware team, they're toast, right? But it's just because there's only a handful of Russian teams that could do that. Now, two or three Russian kids can have a whole bank of Macs, or they have the money, they can buy Nvidia H100s, and they can run a ton of agents who do this all day. And that is what is changing. So I'm sorry, you asked who is doing this. For Binance, it's the North Koreans. And they have an entire team that all they do is steal Bitcoin. Other people have gone out of it, because it's like really hard to compete against a foreign intelligence

• 14:46Speaker 02

  agency that recruits their smartest kids, and all they do is steal Bitcoin. They're extremely good at it. And for the Binance hack, we could talk about it. There's some, probably some AI component, but mostly that was actually human intelligence and very, very smart social engineering. And the North Koreans are, I mean, you have to give it to them. They're very, very good at that. But for the other stuff, it's, it's, it looks like for the, the people who are breaking into AI companies, is what's called the Chinese cottage access industry, which are small Chinese contractors who break into companies and then sell the information to much larger Chinese companies.

• 15:22Nilay Patel?

  That diffusion of skill, changing the sort of economics of cyber is really fascinating to me. We ran a headline ages ago called Attack of the Killer Script Kitties, at like the very beginning of all this. And I feel bad, because it is much more suited to now than it was to then. Because then it was just a lot of like automated mischief.

• 15:40Speaker 02

  Because you wasted the headline, you wish you held it back.

• 15:44Nilay Patel?

  You know, every now and again, you get a lightning bolt headline, you gotta, you gotta get it out there. It's a great piece of art, you know, we'll put it in the show notes. But it feels like the diffusion you're talking about is not, oh, a bunch of teenagers are going to be even more malicious. It's a bunch of bad actors with real incentives to do harm are now vastly more capable than they were before. And that doesn't seem like you can rejigger the economics of that, unless you somehow make it really expensive to do the attacks.

• 16:16Speaker 02

  I think it's going to be extremely hard to make it expensive to do the attacks. We're going to have to lower the cost of defense. And you're completely right to look at this economic problem, because what's happening is attacks are becoming agentic. The cost per token for attackers is going towards zero. And the cost for defenders is staying quite high. And so it is very important for us to find ways for defenders to have cheaper options. And I think that is critical. So that's something I am personally

• 16:54Speaker 02

  extremely focused on. And I think, to be frank, very few people are. There's a couple of problems here. One, there's a ton of focus on the defensive side of AI on finding and fixing bugs. That's great. It's important. But software security as a CISO, like I work for an AI software company, Cognition, even as a CISO of a software product company, software security is 25% of my job, right? I have all kinds of other responsibilities. And software security is the part of my job that has been the most, both revolutionized

• 17:36Speaker 02

  and improved by AI. The other 75% has been helped a bit, and in some cases, made much worse. So there's so much more we need to do than just hill climb on finding and fixing bugs. There's been so much less hill climbing on all these other things. And the next area needs to be the operational security side.

• 17:58Nilay Patel?

  Give me, wait, give me an example of the other things, the 75%.

• 18:01Speaker 02

  Okay, so like, the operational security side is watching your logs, looking at, ah, somebody is running Artechs against us. I am now going to trigger an investigation. I am going to look to see what is going on, and I'm going to make an intelligent decision of what I'm going to do to respond to this without shutting down the entire company. That is SOC work, that's Security Operations Center work. Companies that want to survive this era are going to have to replace human beings who do that work with the first tier, and then eventually the second tier. You can do that with current LLMs. The first tier actually has to be not an LLM, it has to be something much dumber, because you're looking at terabytes or petabytes of logs, and it is just completely uneconomical to

• 18:46Speaker 02

  do that with actually an LLM. We have other things, TFIDFs, MLBerts, traditional classifiers that can operate at much faster. But then that's just off an alarm, and then you, you send it to a small LLM, and then the small LLM calls its bigger brother, and that might call it an even bigger brother, right? But for those models, we need to do a lot more work to optimize them. And we need to make them cheap, right? And I have a colleague who works at like, one of the large security platform companies, who said that like, doing an investigation on one of the big frontier model cyber models, costs on their product $900 to do an investigation. That is not going to work when the Russian ransomware

• 19:31Speaker 02

  guys are either using Macs, or they're using, you know, thrown away cyber, crypto mining rigs, right? And their marginal cost is zero. So this is like, one of the things we're working on at Cognition is like, we train our own models, retrained open weights, and we're trying to drive this cost way down. It's something we've done, but we have a lot more work to do here, and a lot of other people have work to do here. And I think this is where, like, as an industry, we have to focus on like, the Pareto frontier between cost and efficiency on cyber, because it's not just bug finding. The other place here is this, like, people talk about Mythos, Mythos is great, Aster Cyber is great, they're incredible models. But even the people who have access to it, which is a tiny number of companies, they only scan like their most important code. And there are hundreds of

• 20:16Speaker 02

  billions of lines of code out there that are not the Linux kernel, that are the crap C sharp and J2EE code that was written by Infosys 15 years ago. All of that code is not, is not going to go through a frontier cyber model that costs a gazillion dollars. And so that's the kind of stuff that we need to work on cost efficiency for.

• 20:44Speaker 00

  Support for the show comes from Engine. Running a small business means every dollar has to work hard. But if your team is still booking travel the old way, it's costing you more than you think. Engine is the fastest growing travel and spend platform in the country, built specifically for businesses like yours. Book a trip in as little as two and a half minutes, earn up to 10% back on hotels. And in 2025, Engine customers save more than $300 million on travel with zero booking fees, no contracts, and no BS. More than 1,000 businesses join Engine every month. Join them and get $500 when your business signs up and starts traveling at engine.com/box.

• 21:35Nilay Patel?

  It feels like you're describing this cliche that I hear all the time. And again, I'm just going to keep asking really dumb questions about all this. But the cliche I hear all the time is everything's going to get so much worse before it gets totally secure forever. And it doesn't seem like the model you're describing leads to that result. Right? We're not going to fix all the bugs forever in some valley of despair and then always have perfect code. At some point, there's some cost advantage to the attacker always. Right? Like you running a company, you can't write defense that has zero marginal cost. Like someone has to do it, even if the attackers are so infinite that their marginal cost is effectively zero. So are we going to go through a period where everything is so much worse and then it's perfect forever? Or how does the cost manage itself out into bounds?

• 22:20Speaker 02

  I am neither like the hardest doomer or the most positive person about this. I feel like we are not going to P2, right? Like, we're not all going to die or be uploaded to the singularity in the next two years. Nor do I think when things get better, they're going to be perfect. Right? The metaphor I've been using is we are entering a dark age. Right? Things are going to get more bleak and the our responsibility of those of us who work in cybersecurity and especially those of us who work in cybersecurity at the AI companies. And we are a lab, if a smaller one compared to the big L labs. It is especially our responsibility to help shorten that dark age. So the dark ages were bad, but we survived them. Right?

• 23:03Speaker 02

  And then we had the enlightenment and the Renaissance. But it's not like the Renaissance was perfect. Like it wasn't, you know, Utopia and everything was perfect. We will still have hacks. All kinds of bad things happen, even when you don't have a security flaw. There's tons of security breaches that do not require a flaw in the software itself. And we still will have flaws in software, even if LLMs are writing all of them in strong languages. And it will take a long period of time to get rid of all the flaws. I do think, however, things will start to get better as the median code becomes code

• 23:46Speaker 02

  that was written by an LLM in a memory safe and type safe language. But the problem is, is we have a couple of decades of stuff that human beings wrote and that wrote in languages that we should have replaced years ago. And my feeling is, is to shorten this age, we have to do a couple of things. One is we have to stop to get out of this, like, we're just going to find bugs and patch them. A bunch of stuff just needs to be refactored. Right? Like, if you're looking at your code base and you scan it and you've got 10,000 bugs, then probably now we are at the point where rewriting it is actually cheaper. That is my observation from looking at these customers of ours is that if you can get

• 24:26Speaker 02

  a working build, then it is actually much easier to write a whole humongous suite of tests and to spec out how this thing works and then to rebuild it within that spec and those tests securely than it is to find all the bugs and fix them. It actually is. And so I don't know what is going on through the mind of the Linux kernel developers. But that thing, again, that talk I gave like at the beginning of this year, I posited two possible futures. And the one was, we're now at superhuman bug finding. Maybe we just have a pool of superhuman bugs and we're going to drain it. That's the good future. But the bad future is every time we get a new model, there's a new superhuman bug pool that's a little more superhuman.

• 25:11Speaker 02

  And unfortunately, we seem to be closer to that and that the Linux kernel got a bunch of fixes from Mythos and then Mythos preview. And then we had Mythos 5.1 and there's another pool. And then we got Astro Cyber and we just got another thousand Linux kernel patches. It's like, it's not slowing down yet. And I don't think we're just going to scan our way through the C code of the Linux kernel. There's only like two companies, honestly, in the world who could probably do this. It'd be Google or Amazon. I wish instead of like the Fuchsia project or rebooting the Google book or the Chrome book again, Google would take all of those incredibly brilliant engineers and they just tell them, take the Linux kernel and rewrite this in Rust. And that would be their gift to the world.

• 25:53Speaker 02

  And it would be an incredible advertisement for Gemini for Carbon or whatever would be like, get rid of all the drivers that nobody needs. Like this thing just have to run on the 4060 X3 or whatever. I mean, I know it doesn't, but you know, you know what I'm saying? Like, and then just release it and call it whatever you want, the G Linux kernel, just like they did this with open SSL. Right? And I would run that, right? Like cognition would run that on our servers, like, or have Google and Amazon do it together. And it would cost them $100 million, but compared to the societal cost that this is costing us every time we have to patch the Linux kernel and you have to reboot every Linux machine on the entire planet, that's what has to happen. But for whatever reason, the technology industry has not been able to get together

• 26:32Speaker 02

  and make these hard decisions of these fundamental things that underlie the entire human race. We have to completely rebuild.

• 26:42Nilay Patel?

  This is why I started by talking about how the economics of cyber led to the culture of cyber. I mean, this is like the most decoder way of approaching this conversation, I know. But there, you know, when you describe the, the research teams at the NSA and then the operational teams, that is a word chart that comes out of economics of the work. You're saying, oh, someone has to make a big decision here. You need to look at it from a very high level and say it is cheaper to spend $100 million to fix Linux, rewrite Linux today than it is to successively patch new bugs when every new model comes out. I don't think anyone's in that position. Like, I don't think Linus Torvalds is in that position.

• 27:20Speaker 02

  I think Sundar Pichai and Andy Jassy are. I think, I think they literally have each other's cell phone numbers. If one of them, maybe they don't listen to decoder, but I expect one of their direct reports to us. And I would like one of them to talk, go into their one-on-one and say, sir, I listened to the most handsome, best spoken CISO I've ever heard. And he had a great idea. And you could get incredible credit if you're the one, you know, we'll both do it, but we'll leak to the Wall Street Journal, that was your idea first. Yeah. You should call the other and we should spend 100 million, maybe a billion dollars. Like, we just had a bug in the TCP stack of Linux.

• 28:04Speaker 02

  That is insane. We can't do this anymore. The problem with rewriting the Linux kernel is like half of its driver code and nobody has that hardware anymore. Right? Like, somebody had to be in the presence of that hardware with a, you know, a ThinkPad T40 when they wrote that driver and they don't have that anymore. But the good thing is, is a huge number of the really important uses of Linux kernel are production server hardware. And so fork it, right? Like Linus will never do it because Linus wants to support all that stuff and he's got like all these politics, but Google and Amazon don't have to care about Linus Torvalds. Right? And so Google and Amazon can create a new version, a new fork of the Linux kernel and they don't have to call it Linux. That can be ABI, binary compatible with GLibC and everything above it.

• 28:48Speaker 02

  In fact, they can eventually fork GLibC, right? Yeah. And they can build it and they can call it whatever they want. And it only runs on production hardware and then they can use it for their clouds. And eventually all the clouds will move to it. And if you want, other people can run the insecure thing that has to be patched all the time. And that's fine. But like, for the things that our containers run on, I would feel a lot better if it was written in a type safe, memory safe language that also has the ability now to have contracts. So like, these languages have the ability, you don't have to do full form verification, which is incredibly hard, but you can do kind of these things that approach formal verification. And I'm a little over my skis here because there's people much smarter than me that study this. But basically you define, this is how this function is supposed to work.

• 29:32Speaker 02

  Here's the contract it has. And very, very smart software engineers would love to write these things because, you know, they had time on their hands. No, no person who had a real job used to write these things, but LLMs have tons of time on their hands. Right? Because, you know, we're frozen in time while, while they're, you know, thinking that incredible machine speed. And so you could totally have them rewrite the Linux kernel with this, you know, not quite formal verification. Please, if you're listening and you work at Google or Amazon, go pitch us. Anyway, there's my pitch.

• 30:03Nilay Patel?

  I'm just saying, I know Jason's listening, so you heard it. I'm not even going to say who it was.

• 30:08Speaker 02

  But I'm saying like, that is the kind of thinking we're going to need to make it through this. Right? It's like, you can't, we can't just patch our way through this because it's killing us. And because the other thing that's killing us is companies can't, we are not set up as, you talk about like how we do security, like for the last 20 years. Kind of the assumption is, is like, we can get a patch and apply the patch and you're okay. And what's happened is, is the exploitation timelines are so incredibly fast that that does not work. That's what we just saw with the PHP bug is we got incredibly lucky that the PHP bug turned out to be really hard to exploit in the wild. If it was a much more wormable bug, it would be a disaster.

• 30:51Speaker 02

  And so that was the kind of near miss asteroid thing that now the world should, you know, build an asteroid.

• 30:56Nilay Patel?

  Can I ask you about this? Yes.

• 30:58Speaker 02

  Yeah.

• 30:59Nilay Patel?

  We should definitely put Bruce Willis on a space shuttle.

• 31:02Speaker 02

  Don't want to miss a thing.

• 31:04Nilay Patel?

  There's a lot going on in this conversation already. I, my pessimism about all these things is very simply expressed. We are, as Americans in particular, we are very bad at doing anything until a bad thing happens. Right? We are, we, we love a cure, we hate prevention. A lot of bad things have happened. There's the hugging face attack. You actually, I think you recently said you were happy that happened. It woke people up. There's the PHP bug that, you know, we've, we've just had a crazy month of all these breaches. It doesn't seem like that has led to a whole of industry response or a regulatory response or Sundar calling Andy and being like, we're, we're doing Linux. What's it going to take? How bad does it have to get?

• 31:47Speaker 02

  The hugging face incident was good and it did wake up a lot of people in security. We are a little bit better off because hugging face happened and we saw what was going to be possible a couple of months later in open source. But you're totally right. As Americans, and Americans are the only ones, honestly, who can lead the world on this, we're the ones building the AI for the most part. We're the ones who lead the cybersecurity industry. The Chinese are mostly enabling the attackers, not the defenders. They should enable defense. They, they want to be participants in the global economy. They, they want to be leaders. They should be doing more on the defensive side, not just enabling the attackers. But that's not happening. So anyway, but we need to be proactive.

• 32:32Speaker 02

  And look, something I have learned now, working at like at cognition at an AI startup, you can just do things. Like, we need to, like, seriously, you can just go do things. And we just need to have more agency as a tech industry. We should just go do things. That should be the. Lesson of the AI era, and we should not be so risk adverse and doing the right thing security wise. And I think that's what security teams, if I, if I had a, a message for other CISOs is just go do the right thing. Because CISOs are, by our nature, we are risk adverse. We are the people who manage downside risk. We have all L and no P in the P and L.

• 33:14Nilay Patel?

  Yeah.

• 33:15Speaker 02

  And I have had to personally like break through that natural inclination because like you just, you can't just play defense.

• 33:23Nilay Patel?

  But this is my prevention versus cure, right? The, the prevention is cheap, but it's all loss on the P and L statement, right? It's all just cost.

• 33:31Speaker 02

  Yeah.

• 33:31Nilay Patel?

  If the bad thing happens, like, well, we have to do anything to fix this because there's economic benefit to having a functional business again. And it seems like in this moment, even though it's just sort of like, basic understanding of that risk versus cost is totally scrambled and no one has quite figured it out yet.

• 33:48Speaker 02

  Yeah, I totally agree. So like, I'll give you a, a very practical example of, of a way CISOs and their bosses are going to have to change on this. Every company in the next, I was gonna say six months, but really in the next 90 days, is going to have to RTX themselves.

• 34:04Nilay Patel?

  You're just gonna have to run, you're gonna have to run the attack on yourself.

• 34:06Speaker 02

  Yes. If, if not with real RTX, you're going to have to use a AI red teaming product that is functionally equivalent, or you're going to have to hire a company to do that for yourself. If you do not, you will get breached.

• 34:19Nilay Patel?

  Yeah, it just.

• 34:21Speaker 02

  That's terrifying, right? Because these things are completely non deterministic by definition. They're very smart. They're very fast. They are so fast that no matter what, if you hire like an AI pen test, AI red team company, they will promise you verbally, oh, it is completely safe. And then they will give you a contract that says, our product will give you cancer, will emanatize the Ashkatan, and will likely end the human race. And you hold us harmless when any of these things happen. Right? So there is a disconnect between what they will say verbally to you and what their lawyers have written, right? Because that company has by definition, only existed for six months, or at least the product

• 35:04Speaker 02

  has existed for six months, if it is an older company. They have no idea what their product is going to do when it meets your, your software. They have no idea whatever they say to you, but you have to do it. Because some dude is going to do it to you. And you need to know what is going to happen and, and fix it. Because at least, maybe those people will break your system. They might cause downtime. But at least what they're not going to do is steal your data and ransom it. Or intentionally shut down your blood bank refrigerators. That is what happened in France, right? So that is exactly the kind of thing that a CISO, a conservative CISO, would never even ask to do. And a conservative company, even if

• 35:47Speaker 02

  they have a risk taking CISO, would never let their CISO do. And so what they're, what's gonna happen next six months, a ton of companies are going to get breached by some 19 year old running RTX, and then buying tokens off open router, or running it locally, on their gaming PC, right? They're gonna turn off Call of Duty, run RTX, then turn Call of Duty back on, right? And while they did that, they broke into a bank, who their policy said, oh, we can never red team ourselves. Or they said yes, but they ended up in a 30 day negotiation because their lawyers are like, I want to have, you have to have a hundred million dollars in insurance. And we want to have, you know, this kind of, like, they say yes, but their vendor risk management process and their legal process is so onerous, there's no possible

• 36:30Speaker 02

  way they could actually accept the risk. That is my message also. You are going to have to take that risk to actively red team yourself. To front run the bad guys who are going to be, who are doing that right now, right now, as we speak, and as people are listening to this, 24 hours from now, you are possibly have an AI system who is trying every single API endpoint and website that you possibly, you might not know you have, because it was like a marketing site that was set up a year ago, for an, for an executive event, and is going to find a bug and get in, and then is going to move incredibly fast through your network. So you'd rather pay somebody or have your own team do that. You need to accept that risk.

• 37:10Nilay Patel?

  Yeah.

• 37:11Speaker 02

  Almost no company, larger than a couple hundred people, has a risk management framework that allow them to do that.

• 37:26Nilay Patel?

  Support for the show comes from CrowdStrike. Here on Decoder, we're peering beyond the traditional tech realm, and into all the places AI is shaking things up. Cybersecurity is key to that conversation. CrowdStrike gets that. That's why AI detection and response, or AIDR, is emerging as a new category of cybersecurity for the AI era. It helps organizations see, monitor, and secure AI across the enterprise. CrowdStrike, we secure AI, we stop breaches. Go to crowdstrike.com/decoder to learn more.

• 38:09Nilay Patel?

  There, there are several costs associated with this. And one, I think, is just like the most common in business, which is, well, if we find out there's a problem, we'll have to pay to fix it. So what if we just wait? Right? Like, you can see that coming. What if we find out that we are catastrophically in trouble? And then there's how much will it cost to fix it? And then there's what will we do to fix it? And the argument that I've heard about what will we do to fix it is, well, we'll just unleash a swarm of defensive AI tools that are watching the networks and watching the code. And they'll make a bunch of autonomous decisions, and they'll fight the autonomous script kiddies who are armed with RTX on their Mac Studios. And something about that argument in my head, I've heard, for example, Palo Alto Networks CEO Nikesh Ror has made this argument, right? You need,

• 38:53Nilay Patel?

  you need to give the defensive agent some autonomous capabilities. And I'm like, well, hold on, like, we, OpenAI can't control the swarm that went and attacked Hugging Face. How would I control the defensive agent from doing something equally damaging to prevent an attack? And I, I don't think that there's a technical solution there that makes it clear what to do once you've found all of the vulnerabilities.

• 39:16Speaker 02

  Couple of things. One.

• 39:18Nilay Patel?

  I told you I was gonna only ask dumb questions. I just wanted to be clear.

• 39:21Speaker 02

  No, no, that is a fantastic question. You have not asked a dumb question yet. I'm honest. You have not asked a dumb question yet. That's a fantastic question. Okay, I'm gonna give you a shorthand, uh, that, uh, you can use going forward. If somebody actually uses the term swarm, then.

• 39:35Nilay Patel?

  I don't know, to be clear, I don't know if Nikesh used the word swarm.

• 39:38Speaker 02

  Okay, I'm not saying he did. And I have great respect for Nikesh. He is one of the best businessmen in, in all of security. Uh, so this is not on him. But swarm is used all the time.

• 39:49Nilay Patel?

  Yeah.

• 39:49Speaker 02

  But when I hear somebody use swarm, I immediately discount all of their technical opinions by about 80%. The, uh, it is going to become, it is already, and it will become the most overused term in all of security. The actual quote unquote swarm from OpenAI was something between, at different times, 700 to 1400 agents. That had to run on, it, it's hard to tell because we don't have the actual, like, size of the models. But, you know, uh, 300 million to almost a billion dollars in hardware. That doesn't include the actual data center cost. It, the actual Hugging Face attack was, I, I did some back

• 40:29Speaker 02

  of the envelope map, math, it may be a million dollars in electricity, right? No attacker is going to do that. And, and in the end, that was stupid, because these things were actually trapped. They were not prompted to attack Hugging Face. Most of them were doing something totally different, and were kind of enlisted by the one thing that actually wanted to get to Hugging Face. They used a tiny fraction of their token budget to help their little friend. And then they had to tunnel their way out. So it was the most inefficient way to possibly attack Hugging Face. Almost nobody is going to be attacked by a swarm. So if somebody says swarm, they do not actually understand at a low, at the

• 41:09Speaker 02

  bottom technical level what's going on. The RTX attacks are almost certainly one single agent, maybe two.

• 41:17Nilay Patel?

  By the way, I was the person who said swarm, just to be clear in this one. Keep going.

• 41:20Speaker 02

  Right. And so now you won't. And you can use that as a good filter for your future guests. It is driving me insane, because it's everywhere, right? It is absolutely everywhere.

• 41:30Speaker 02

  A couple of things are correct. One, uh. We are going to have to turn over defensive actions, at least the first couple of levels to AI. It is, that has already been true, just because, you know, any reasonable-sized organization generates terabytes of logs, if not petabytes of logs. So you already have AI, if not LLMs, much more traditional AI, like classifiers, looking at it. But then, actual decisions need to be turned over to AI.

• 42:00Nilay Patel?

  Yeah.

• 42:00Speaker 02

  With human oversight. That is true. There is no way to just release defensive agents on your network, but you can build defensive agents that are plugged in to lots of things. They are plugged into your logs, they are plugged into your EDR, they are plugged into your cloud, and they have the instructions on what they're allowed to do. My preference is there are deterministic limits to what they're allowed to do. Those are hard to necessarily create ahead of time, but they are important. So, for example, we are building all this on top of Devon, our own SOC, right? But we are putting in place deterministic limits, like it has the ability to turn off an account in Okta,

• 42:43Speaker 02

  but only so many accounts, so it can't turn off the entire company. Like, if it goes nuts and it says, "Everybody's a North Korean," it can't turn off everybody in the company. If it hits its limit, you know, no matter what, we get paged, but if it hits its limit, it pages us a lot and it says, "I hit my limit," right? And then, a human being has to then approve anybody above that limit, right?

• 43:01Nilay Patel?

  Yeah.

• 43:02Speaker 02

  Obviously, if it turns off five people, then something has gone really, really wrong, right? And so, yes, you need defense, but there's no way to just, like, let a bunch of bots loose to do all the defense. The only way we're going to make it through this is you have to autonomously fix software bugs. So, I am currently helping a company where we are onboarding all of their repositories. We are scanning with Mythos and a bunch of other models, and we are finding and fixing thousands of bugs, verifying with tests that we are writing, merging those PRs automatically. If you are renovating old code, the only way you're going to survive this era is to do that on an automatic basis.

• 43:45Speaker 02

  You can't, that would take years with humans, and you can do it in days. And certainly, that is the place I would start, and then the next step is the operational side, for sure.

• 43:56Nilay Patel?

  It feels like I just keep swirling back to the economics of all of this. But it really feels like the other way you could solve it is by saying to the big model companies, "If your models are used to go attack a company, you'll be liable for it." Right? "We're going to make it really costly," and then they should figure out whatever alignment that they need to figure out, whether or not that's even possible. And the pressure on the openweight models will go up, because we'll all be paying the frontier companies for defense, and they'll get better and better and better, because we've created some economic condition for them to get better. But it doesn't seem like you think that's likely. Yeah, I just saw your face.

• 44:34Speaker 02

  I think, I think it's really dangerous. So, so there's two different kinds of the models are attacking. So, the models have escaped and directly attacked people. I think there should be liability there, right? But that is agent harnesses running at companies, using their computers, their IP addresses, attacking people. That is, I wouldn't say totally solved, but it is much, much better. Like, the stories we're hearing now are still stories from the hugging face era. They're just coming out lately.

• 45:03Nilay Patel?

  I like that the era was like, three months ago.

• 45:06Speaker 02

  Right, but it's like, basically, there's been a massive change. This was effectively, mostly an organizational problem, was not a technology problem. It is that the PhD researchers at these companies are gods on top of Olympus, and the security teams are plumbers, and we're not allowed to touch the evaluation networks. And all of a sudden, this problem happened, and the security teams are like, "Hey, how do you like us now? Maybe you should have talked to us," right? Yes. And so, that was the problem at, like, OpenAI. And then, like, the Anthropic and Meta stuff is they had this, they all shared this outsourced provider, who did a really bad job, right? And has gone slapped hard. And so, that is solvable, right?

• 45:49Speaker 02

  That is not the problem here. The problem here is not OpenAI and Anthropic's models escaping from OpenAI and Anthropic causing problems. The problem is, is bad guys using models from a variety of sources. There is already a problem of defenders getting refusals from OpenAI and Anthropic. And it is a real, real challenge that if I am trying to do defense, of Anthropic being like, "Oh, it looks like you're a bad guy." So, I think we have to be extremely careful, create a situation where if I'm trying to do defense, and I'm in the middle of the worst day of my career, that all of a sudden I get cut off. I think we have to be extremely careful about that, because it is actually, as somebody who now has this challenge, is actually quite challenging to disambiguate between people doing defense and offense.

• 46:33Speaker 02

  In some cases, we are getting better collectively at this, but it is a challenge. It's also something that's really pissing the rest of the world off, because, you know, Americans are like 5% of the planet. There are lots of people who need to do cyber defense around the world, and they are, they are treated much more poorly. Europe, Asia, legitimate companies are treated much more poorly from a cyber defense perspective.

• 46:54Nilay Patel?

  This is, if you add it all up, this is a good argument to slow down, right? It, to say, "We need to catch up to the capabilities of the models, we need to build some of these systems, we need to initiate everyone into a way of thinking where the cost of prevention is, is worth it, and not just the cost of the cure." Do you think that it's likely that we will slow down? Do you think that we should?

• 47:16Speaker 02

  If I had my druthers, I would be fine slowing down on the frontier and then catching up on the cheap, cheaper models for defense. I'm a defense guy. I have to, like, live in the world that other people who work six blocks from me, and, you know, who are way, way, way, way richer make. I live in their world, man.

• 47:38Nilay Patel?

  Yeah.

• 47:39Speaker 02

  So, like, I get asked this question all the time, and it's like, I just do the best I can within the world other people make. Yeah.

• 47:45Nilay Patel?

  Do you think that the, the big labs have woken up? You, you mentioned that their cybersecurity teams, they've gotten more access, they've, they've stopped what you might call the hugging face era, or at least they've, they've changed the structures of those companies. Do you think that will help?

• 47:58Speaker 02

  I mean, certainly, they've woken up on the eval risk, right? I think they have teams who have been this entire time, working very hard on the big-picture cyber risk. There's a bunch of people in these, in these companies, as you know, who want to slow down.

• 48:17Nilay Patel?

  Yeah.

• 48:19Speaker 02

  A lot of that discussion is driven by the P doomers, who, it's about AI killing us. And I have never, I've, that's never been my, that's never been my concern, right? And to me, that has always been a, driven by 24-year-olds who, all they've done is AI, and they've never done anything else, right? Like, you and I are both, I think, old enough to remember, I grew up a couple miles from a SAC base, and we did drills where I was taught that hiding beneath my public school desk would save me from a Soviet thermonuclear bomb, right? Like, that was P doom, you know? My grandfather built a bomb shelter that my mom was taught to hide in during the Cuban Missile Crisis. That was P doom.

• 49:02Speaker 02

  And so, like, there's like a, a youth issue and a never-doing-anything-but-AI issue of, of people who, like, they got their undergraduate, their masters and their PhD, only doing linear algebra and neural networks, and never touching anything else. And so, I have always been much more concerned about the human uplift risk from AI, of all the risks about human beings using AI to do bad things to other human beings, right? And, and so, part of my concern is that there has been success in these labs working together on, like, "We're not going to let them encrypt their chain of thought."

• 49:41Speaker 02

  Because when they talk about pacing the frontier, it's for the P doom stuff, for the, the polycules that live together in Berkeley. You know, like, they're talking about very specific stuff about keeping control of the models themselves, which I think is great. I think that stuff needs to happen. But then, if those things happen, then there's still often an increase in a bunch of capabilities that human beings can use. And, and so...

• 50:10Speaker 02

  I'd really prefer there to be kind of a broader discussion on could we make these things, one, less human seeming. I'm a big fan of like, I don't think these things should have personality. I think the goal of these things should be to serve humans. They should not seem to have souls. They should not seem to have personality. I think a lot of the negative impacts, as somebody who was at Facebook and saw what it was like for that company to have negative impacts on people, I think there should be a much broader feeling of these companies of like, what are we doing kind of broadly to people? And a lot of that could just be, don't have these things ever act like somebody's friend.

• 50:48Nilay Patel?

  Yeah.

• 50:49Speaker 02

  Right? At a maximum, it should act like, like the people, I don't know what the company is, but the people who made the, hi, I'm like a really nice, you know, like the photorealistic girl who will chat with you. That is, to me, spectacularly unethical.

• 51:03Nilay Patel?

  Yeah.

• 51:03Speaker 02

  To even try that, to even start that. That's the kind of thing that we should just make. I don't know if we can make it illegal, but we should make it completely societally not acceptable to work on that kind of stuff.

• 51:18Nilay Patel?

  That's a great segue because I did want to ask you about Facebook and content moderation. Like I said, you and I first started talking a long time ago about content moderation in the context of security, right? You would get foreign nation states flooding our algorithms with all kinds of misinformation during elections, and that was a security problem. I open Instagram today, and I am confronted with some of the most nightmare copyright law hypotheticals to have ever existed in the history of the world. Right? It's a, the one I got today was a deep fake of Lewis Hamilton and Charles Leclerc doing somebody that I used to know. And I was like, well, like 50 different people can probably sue Meta right now, right? And also, this is hilarious, if you know what happened in that

• 52:02Nilay Patel?

  last race. So, you know, we keep talking about it. We've posted some clips of us talking about it on the Vergecast where I'm like, I, I think Meta is letting their AI ambitions crush Instagram. Right? Like Instagram is being flooded with slop in a very particular way. Some people love it. Sometimes it's funny, but there's a nightmare set of lawsuits coming. There's a whole rights management problem coming, and then the users don't like it, and they should label it and let you filter it. And so we post this, and Adam Misery is, he said this to us in a reply on threads, I would love to do this. I'm worried that the models are going to get ahead of us. Right? That we can try to do this, but there's not actually a way to recognize AI content. There's not actually a way to label

• 52:41Nilay Patel?

  it well. If you, if you're worried about the sexy AI influencers stealing jokes from a comedian, now you got to build a database of jokes. You have some mystery here. I think Instagram is about to go through it in a real way. Is there some technical solution to the problem of AI slop and social media?

• 53:02Speaker 02

  I think he's right that it's not perfect, but also something I noticed during my time at Facebook was, when Facebook doesn't want to do something, they will allow the technical challenges to stop them. When Facebook really wants to do something, they will give it their all. Right? And that is the pattern, always. So when I wanted to try really hard on the safety issue, man, this is not solvable. And you're right. It's never perfectly solvable, but let's give it a shot. Right? But like, whenever Zuck is told something is impossible, it's impossible to monetize mobile. It's impossible to, you know, build this network

• 53:40Speaker 02

  to 2 billion people. It's gonna be, you know, impossible to monetize WhatsApp. It's gonna be impossible to work AI into the system while maintaining it in encryption. He's been told that everything's impossible over and over again, and then he constantly succeeds. They will give it a shot, and they will do it. And, but when they don't want to do something, oh, that's impossible.

• 54:07Nilay Patel?

  Do you think that there's some way to understand that some piece of media is generated by AI in a consistent and verifiable way? This, to me, seems like the challenge.

• 54:18Speaker 02

  I am not as up on the literature here. I think up to this point, there has been a pretty decent arms race. By definition, AI will continue to get better because the way, especially for visual models, for diffusion models, the way they work is that you have detection oracles that they test against. Certainly, you can never have something open source, because it will immediately end up in the diffusion model, right? But for closed source oracles, where then you do not make it easy for them to test the oracle, one of the key things you do in trust and safety is then you have delayed enforcement mechanisms, so they can't brute force and, and test. I think it is quite possible. Let's say I

• 55:00Speaker 02

  was still at Meta. My advice to them would be to then also explore a variety of different detection mechanisms outside of the content itself. Right? And so I would do the best I can on the content, but I would also be doing low level access as possible to the camera. I would only let content to come from the camera. And what I would say is, Neil, I was completely correct. There's a massive hunger for, especially among young people, stuff that is authentic. I have three teenagers, and they hate AI. They hate AI slop. They have a massive negative reaction. My wife and I created just like a cute little

• 55:43Speaker 02

  family AI thing, and they, I mean, they yell at us about a bunch of stuff, but they lost their minds about this like cute little family AI thing, right? Because they just hate it. And so I think there's, somebody is going to come and disrupt Instagram, and it's going to be because, there's already, I've seen these little things where it's like you take the photo during the day, like be real and stuff. There's like these little things, but somebody's going to massively disrupt Instagram because they're taking the images of the photo straight off the camera, and then all the editing is going to be only within the app, and they are going to do things to make sure that nothing is ever, nothing can make it into the pipeline from AI, and that you're going to allow some kind of cap cut like

• 56:24Speaker 02

  functionality that never has an AI component. And I think that is going to be a killer app.

• 56:29Nilay Patel?

  Man, I could do another whole hour with you on this, because the, the system you're describing there is not detecting AI content. It is verifying real content.

• 56:38Speaker 02

  It's much easier. Verifying real content is much easier, right? And, and there's been a big focus on this on the professional side, right? Like Adobe has a whole thing of like, if you're a New York Times photographer, you have a $5,000 Nikon or Canon, you take the photo.

• 56:52Nilay Patel?

  The iPhone that I'm holding here has that in it now, right? Like, there's a lot of this. Yeah. There's, there's a lot of this out there. It's, but it's the other side that I'm, I'm thinking about, right? Where it's, we are going to detect AI text, right? Then the, the leading model companies, because of European regulations, are now like butzing with the text they generate, so they can be.

• 57:10Speaker 02

  Text is much, text is much harder because you just have so much less data, right? Like, especially if 140 characters, like what do you have in there to detect? You know, the more you've got, the more you can do.

• 57:19Nilay Patel?

  It is amazing, because I can detect Claude a mile away. So maybe I'll just rely on my own human judgment.

• 57:24Speaker 02

  Not in a, not in tweet length.

• 57:26Nilay Patel?

  That's true. The, the cheap ones I can. But I'm just saying, like, the, the notion that we're going to verify what's real instead of detecting what's fake feels very important in the context of the media we consume.

• 57:40Speaker 02

  I think detecting what's fake is always going to be probabilistic, and we're going to asymptotically approach impossibility, whereas detecting what's real can be cryptographically perfect. So I, I would not pick the battle that we will eventually lose. I would pick the battle that we can cryptographically win. That would be my suggestion.

• 57:58Nilay Patel?

  And there's somewhere in there where you have to build an entire chain of custody, right? We're going to take the sensor data off the camera. We're going to put it in a cloud server. We're only going to allow edits on this server. That file will never get monkeyed with.

• 58:10Speaker 02

  Sure, that's true for the New York Times.

• 58:13Nilay Patel?

  Yeah.

• 58:13Speaker 02

  And Adobe Photoshop and stuff, but not for Instagram. For Instagram, you can do it all, because it's, it's only the iPhone, and it's only Instagram. So like, you, the entire chain of custody is within your app. It is within the iPhone sensor and Instagram. So like, it's actually easy for them.

• 58:29Nilay Patel?

  Look, like I said, we could do another whole hour on that, because it feels like right next to the cybersecurity problem is the social engineering problem, and right next to the social engineering problem is the fake media problem. Right?

• 58:41Speaker 02

  Yeah, absolutely.

• 58:41Nilay Patel?

  How many phishing attacks are just this website looked pretty real? And?

• 58:46Speaker 02

  Oh, yeah.

• 58:46Nilay Patel?

  Or this person is talking to you, or there's an AI voice on the other side of the phone. And all of that comes down to, can you detect what's fake instead of verifying what's real? And I don't, I don't know the answer.

• 58:56Speaker 02

  Yeah. I've had to train my parents that if you get a video from your grandchildren saying they've been kidnapped, you have to call me, because the odds that it's real is, is not very high.

• 59:07Nilay Patel?

  Yeah.

• 59:07Speaker 02

  Right? Like, that's where we're at as a, as a society.

• 59:10Nilay Patel?

  I feel like I want to end on a hopeful note, and I don't know how to do it from there.

• 59:14Speaker 02

  Wrong guess for that, buddy.

• 59:17Nilay Patel?

  This is why you only come on every few years. But now, after this, we're gonna, we're gonna do the series of like birthday party guests. Um, tell you, you've been talking about the Dark Ages, you were in it, right? Where it's starting to accelerate. What are the markers people should look for in terms of progress? Right? To say, okay, we're climbing our way back up.

• 59:36Speaker 02

  AI is great for lots of things. I think AI is going to enable individuals, right? Like, we're gonna see a huge explosion of new companies. Like, you could start, I mean, Stanford students have always been entrepreneurial, but the number of my students who are now starting two, three, four-person companies who are doing incredible things, who used to need to hire 50 or 100 people and take a ton of venture capital, is incredible, right? So I think we're gonna continue to see that. Um, one of the things we hopefully see is we start to see, we'll see a downtick in the number of major breaches at some point. We should hopefully see a downtick in the major CVEs that are wormable and that are

• 1:00:20Speaker 02

  in kind of the fundamental infrastructure pieces, kernel, you know, like I said, I want Linux to be rewritten, but hopefully we see a downtick in like the Linux kernel and those other kinds of libraries, you know, open SSL, Glib C, things like that. I haven't seen, maybe I should vibe code this up, but I think it would be cool to have like an index of like, where are we in this cycle of, of like an artificial index of all of these, you know, the Dow for all of the libraries of, of, of where we're going. Plus, you know, right now, Patch Tuesday used to be like 150, 200, got to 300, 600. I think the last one was 900 something. So we're geometrically growing. So it'd be nice to see Patch Tuesday start to go back down.

• 1:01:02Nilay Patel?

  Yeah.

• 1:01:02Speaker 02

  Apple patches are the same thing. Yep. You, you inspired me. So I'm gonna, I'm gonna create a Dow index for this of, of the, the Dark Age index, right? And then maybe we can publish it on The Verge.

• 1:01:13Nilay Patel?

  Yeah.

• 1:01:13Speaker 02

  But I think these are the kinds of things that it would be nice to see a reversal. And what would be especially nice to see is that the day, or actually like 30 days before you get like the next version of OpenAI and Anthropic Cyber models, that you don't have this huge round of patches. Because what's happening is those guys are running their models against Microsoft's code, against a bunch of open source, and they're giving all the, the bugs away. And you don't know where the bugs are coming from, right? It's just like anonymous reports for a thousand bugs in the Linux kernel. And, and then a month later, here's the new Aster Cyber. And so if those get smaller and smaller every time, then it means that we're starting to reach some kind of steady state.

• 1:01:53Nilay Patel?

  You should definitely do that tracker. We will definitely publish it. Alex, it has been incredible to have you on the show. We'll have to talk again soon.

• 1:01:58Speaker 02

  Yeah, thank you so much for, I think you do some of the best journalism on this stuff. I really appreciate it.

• 1:02:04Nilay Patel?

  I really thank Alex for joining me and thank you for checking out this episode of Decoder. I hope you enjoyed it. To get new episodes every Monday and Thursday, subscribe to our YouTube channel at DecoderPod. Find us on TikTok and Instagram under DecoderPod as well for more fun stuff we put out every day. If you'd like to know what you thought about this episode, or really anything else at all, let us know. You can drop us an email at [email protected]. We really do read all the emails, or you can hit me up directly on Threads or Blue Sky. If you enjoyed this episode, please send a link to it to someone you think you might like it as well. It really helps us grow the show, which you can subscribe to wherever you get your podcasts. Decoder is production Verge and part of the Vox Media Podcast Network. Show is produced by Greg Ott, Kate Cox, and Nick Statt. Our editors are Xander Adams and Ursula Wright. Our editorial director is Kevin McShane. The Decoder music is by Breakmaster Cylinder. We'll see you next time.