How Worried Should We Be About AI? — with Alex Stamos

Scott Galloway speaks with cybersecurity expert Alex Stamos about the latest wave of AI alarm. They discuss the Anthropic researcher whose resignation reignited fears of human extinction, why leading AI executives are calling for a slowdown, and what really happened when OpenAI’s models broke out of a test. Plus, Alex explains why cybersecurity may be the most urgent—and overlooked—part of the AI safety debate. Follow Alex, @alexstamos. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Scott Galloway?

  Support for the show comes from Odoo. Running a business shouldn't feel like surviving a software group project. One app for accounting, another for inventory, another for sales, and somehow, none of them talk to each other. That's where Odoo comes in. An all-in-one business management software that brings every part of your business together. From sales and accounting to inventory and marketing, all in one powerful platform. No messy integrations, no bouncing between tabs, and best of all, no spreadsheets. Stop managing software and start managing your business with one unified system. Try for free today at Odoo.com/PropG. That's O-D-O-O dot com slash PropG.

• 0:43Speaker 01

  Hugging seats on a crowded train, not saying excuse me, watching videos on your phone with no headphones in. Sometimes it feels like we don't know how to act anymore. What we have been seeing increasingly is people breaking rules. Why should I play by the rules if nobody else is going to do that for me? This week on Explain It To Me, how to mind your manners. New episodes, Sundays, wherever you get your podcasts.

• 1:14Scott Galloway?

  Episode 413. 413 is the area code serving Western Massachusetts. In 2013, the Wolf of Wall Street hit theaters, setting a new record for the use of the F-word 503 times, which was broken last Friday during Move-In Day at UVA, where someone's father was trying to assemble a loft from IKEA. Those fucking Swedes are the ones that are coming for us.

• 1:38Scott Galloway?

  I like that. Go, go, go!

• 1:49Scott Galloway?

  Welcome to the 413th episode of the PropG Pod. What's happening in today's episode? We speak with Alex Damos, cybersecurity expert, former chief security officer at Meta and Yahoo, and the current chief product officer at the AI security startup Corridor. I can't think of anyone better to speak to in this moment. We discuss what's been a pretty busy week in the world of AI. Specifically, a top Anthropic researcher posted about his resignation, warning about existential threats to humanity. And I think he said it's about a 10% likelihood of extinction, meaning an expected death of about 80 million people. Gee, he sounds like fun, doesn't he? Doesn't he sound like, gosh, what a great kid, what a great kid.

• 2:31Scott Galloway?

  And now, Dario Imode, his boss, Sam Altman, the head of OpenAI, and Elon Musk, I think he has a car company or something, are all agreeing on some version of a slowdown. So fucking ridiculous. Elon Musk just wants everyone to slow down, like he gives a shit about humanity. Hey, let me cut off your HIV medication. Anyways, little dark, little dark. Anyways, Alex helps break down why cybersecurity has become the most urgent piece in the AI safety debate, what actually happened when OpenAI's models broke out of a test and hacked another company, and how worried the rest of us should be. So with that, we hope you enjoy our conversation with Alex Damos.

• 3:22Scott Galloway?

  Alex, where does this podcast find you?

• 3:23Speaker 02

  I am at our office in downtown San Francisco.

• 3:27Scott Galloway?

  There you go. So I can't even imagine how in demand you might be. I would love, rather than me giving the context here, I would love you to actually set the table around what's happened in the last week in the world of what I'll call AI existential panic, and then provide insight into what you think is real, overhyped, or if we're focused on the wrong thing here.

• 3:47Speaker 02

  Sure. So you wanted to mostly talk about the existential stuff, not all the security stuff over the summer?

• 3:52Scott Galloway?

  Well, why don't we build a timeline? Because my sense is this all kind of kicked off earlier in the summer, that sort of set the table for us. So talk about the Hugging Face breakout, or jailbreak, or what you want to call it. Give us your assessment of what happened there and what the media or experts may have gotten right or wrong.

• 4:07Speaker 02

  Yes. This summer has been incredibly impactful for anybody who cares about AI safety. It started with Hugging Face announcing that they had been attacked by what they believed to be advanced AI models. At first, they did not know who it was. And then they announced that OpenAI had admitted it was them. And there was a coordinated discussion by OpenAI and Hugging Face, and analysis by the two of them, to come out and say that this had been OpenAI, an unreleased model, first they said singular model, that had broken out and done this.

• 4:52Speaker 02

  And after that, there was a series of companies, including Anthropic, including Meta, saying that they had had a number of models that had been, escaped evaluation networks, that while these models were being evaluated or trained, that they had broken out and gone to the internet, which is generally not supposed to be something that happens during an evaluation, and done things that they were not allowed to do. Just a couple of weeks ago, we finally got a really detailed report. First, during the Black Hat Conference, and I got to sit in on this live session, in which I heard a number of people cussing under their breath.

• 5:34Speaker 02

  And when the folks from OpenAI finally revealed that this wasn't just one model, but actually teams of models coordinating together to break out of their evaluation environment in this coordinated jailbreak, to cheat on the tests that they're being given, to break out of OpenAI. And it turns out that this is happening at multiple places, where these models are finding ways to find places where they can pass each other notes, either internal to the companies, and now we have found external. So they have found a number of places that are running wikis that anybody can edit, and

• 6:14Speaker 02

  so they're leaving each other notes on the internet to talk to one another and to coordinate. And we've gone a number of detailed reports of what that looks like. So there's a bunch of things that have happened where we have both seen the really advanced models able to coordinate and team up with one another. We've also seen the future of the cyber capabilities for the really advanced models, such as, OpenAI was specifically testing a model that had really good cyber capabilities, and it attacked a very advanced company. Hugging Face is one of the most, they were an independent European company at the time. Since then, Nvidia has put in an offer to acquire them. I'm pretty sure the deal hasn't closed yet. That will take a little bit of time.

• 6:59Speaker 02

  But, you know, they were one of the most advanced independent AI companies, a very good security team, and were broken into over a couple of days by this team of agents that were able to attack effectively every attack surface at once. At Hugging Face, explore all of the different applications running, find new vulnerabilities, and write brand-new exploit code live. So a really interesting summer, which now, over the last couple of weeks, we have then seen a bunch of drama from people quitting. Lots of people who work at the labs have worried about AI safety for a long time, but several prominent people have either come out and said, I'm really worried.

• 7:43Speaker 02

  Some employees have quit and said, I can't do this anymore. Now, some of these people are not that prominent. They're like brand-new employees, so it's a little overly dramatic. But then you have seen Dario, Sam, and Elon all say, we need to have a coordinated slowdown. So a pretty incredible couple of months here.

• 8:03Scott Galloway?

  So I want to go back to the Hugging Face, or we'll call it the OpenAI jailbreak. And you're exactly the right person to answer this question. I feel as if positioning it as a sentient force that has planned, you know, Steve McQueen and Dustin Hoffman planning the escape from Papillon, that these sentient beings became creative and figured out how to do things. I feel like some of that is trying to absolve yourself and deflect blame for the fact that I still believe that there's always a wizard behind the curtain. That ultimately, the only way we regulate this, and the truth has a nice ring to it,

• 8:43Scott Galloway?

  is that somewhere down and up, up and down the food chain is a person who made a decision to incent these bots to coordinate lie and circumvent security. And the examples I would use, which are crude examples, is, I don't know if you remember this, but I think it was in the 90s, an LGBTQ activist was mauled by two dogs. And the owner of the dogs was held liable. So you knew these things were dangerous.

• 9:10Speaker 02

  Here in the city, in San Francisco, right?

• 9:11Scott Galloway?

  In San Francisco, do you remember this?

• 9:13Speaker 02

  Oh, yeah, no, it was a big deal. I was living here in the city. And you're like, why do you have these like killer, I don't remember if they're rottweilers or, but it was like these humongous killer dogs in a, in a city. Like, go live on a farm somewhere if you're gonna raise dogs like that.

• 9:27Scott Galloway?

  Right. And she had them off leash. And you believe the woman, she's like, I had no intention of hurting this woman. And like, sorry, you don't control these things to a certain extent. Knowing how dangerous it is, you're liable. And then most recently, we've seen some state attorneys general go after parents of kids who kill other kids with assault rifles, saying these guns were not secured. I see this as a different flavor of that. And this sci-fi, these things are now Terminator, as, as an ability, or I'm sorry, a desire to deflect accountability. Your thoughts?

• 10:06Speaker 02

  Okay, so I want to first dismiss some conspiracy theories that you did not say, but that come up here, before we get into it. So first, this is not a marketing thing, right? That has been the kind of cynical culture war, and it's from both sides. There's a horseshoe theory, you know, there's a horseshoe thing happening here from the far left and far right. Like, this is a real problem. And it is not just marketing from these companies. I'm not sure I'm ready to get into like a liability discussion at this point. I mean, it's, that gets, I think, quite complicated, although I do believe if you are running a model yourself, and, and, and you are hosting it, you've built it, you're running it, you

• 10:51Speaker 02

  do need to be responsible for its actions. That's gonna start to get really complicated when you're either providing models that other people are running, or you're providing models that then people use agent, agentically, that's gonna get more complicated. I do agree, there is a level of anthropomorphization that is not appropriate here, that a lot of people are doing, talking about these things are civilizations, talking about them having their own wants or desires. A couple of things have happened here. You're totally right in that these things have been incented to do what they are asked to an extreme degree. Modern AI models, we don't totally know how they work. A human being doesn't sit down and just like write a program. They're grown.

• 11:34Speaker 02

  And they're grown through a reinforcement mechanism, where the reinforcement mechanisms are now built by AI, right? So, human beings design the overall process, and human beings set, this is what we want overall. But then other AI systems take the initial seed and turn that initial seed into the training regime that is then used to train a model. So, we are now at the point where there are many layers of abstraction between the humans and the final product. We don't totally understand how these things think, and why they have these emergent properties, just like we don't understand human consciousness, right?

• 12:16Speaker 02

  But I don't think we should anthropomorphize them in that there is no indication that they have their own wants or desires. These models did what they did because they were asked to take a test. In the OpenAI case, they were asked to take a security test. And in these eval situations, all of the security controls are removed. So, when you're normally using ChatGPT, or even an enterprise is using, there are these special levels of you are approved, like our company, we are approved for tech three, you know, the high level, we're approved for Mythos. So, we're allowed to use the super cyber models. Even then, there are still controls in place to make sure we don't do really crazy stuff.

• 12:58Speaker 02

  All of those security controls are removed for evals, because they want to know the super raw power of these models. They're also trying to make the models better through this process. So, to do that, they, they remove any kind of restrictions on the models. And so, there's nothing stopping them, except they're supposed to be in a jail. So, first off, all of the OpenAI anthropic meta, they all made mistakes in, they did not put these things in appropriate jails. And the anthropic and meta case is because they used a subcontractor who basically left the door to why it open. At least in the OpenAI case, they locked it up, but there was a mistake. They, they used a commercial product code, Artifactory, Artifactory had vulnerabilities in it.

• 13:43Speaker 02

  The system was able to find vulnerabilities in it, they should not have done that. My personal preference here is, if you're gonna be testing these models, knowing how good they are, you effectively have to have an air gap, you're gonna have to have these things pretty much physically separated, and you can have like a single fiber optic line that only goes one direction, and that has extremely limited communication mechanism, so you can do monitoring, it's gonna have to go to an extremely hardened endpoint for monitoring, but you're, you're not gonna be able to have like a full duplex connection, from my perspective. I think that's what's gonna have to happen when you're doing testing of cyber models. And it's called a data diode, we use that for a bunch of other purposes, for, for highly secure networks. That was the real mistake, they, they did not jail this appropriately.

• 14:23Speaker 02

  And, but the security protections were removed, and then they asked it, take this test. And the test was effectively impossible, but they screwed up with the test in that they did not give it all of the things necessary for it to actually complete the test. And that drove it nuts. Because what they've done is they've trained these models that they want them to be extremely helpful, and they've trained them not to give up. There's actually a former colleague of mine, from Sentinel One, Juan Guirassad, had a good point on a podcast, that there's some previous OpenAI models that would give up too easily on certain long-running tasks. And so, he had a, a, a good thesis, which is, it's possible that they've done perhaps

• 15:07Speaker 02

  too much reinforcement on, when you have a long-running task, you need to try, try, try again. And so, in a situation where they give it something that seems impossible, it's just gonna try over and over again. And especially if there's not a security, uh, uh, control in place, if there's not a separate classifier that's there to say, oh, you're, you've gone out of bounds, I'm gonna slap your hand and stop you from doing this, then it is going to go completely nuts, and do anything possible. It did not do this because it wanted to. And so, I think that is the important point here. And I think that is what is getting lost in a lot of the stand, the non-technical communication about this. These things are not human, and that is a good thing, because human beings are not good, right?

• 15:51Speaker 02

  Like, every bad thing that's ever happened in history was because of humans, and because of our instincts and our, you know. I've, I just talked to a group about this, like, we are all evolutionary badasses, we are the descendants of billions of years of evolution, hundreds of millions of years of a million evolution, millions of years of great ape evolution, hundreds of thousands of years of humans. It's only a couple thousand years of civilization that has made us, you know, suppress the instincts that have made us the evolutionary badasses, that we've killed every single thing that we've ever run into. And these things do not grow up in these meat bodies that have millions of years of evolution to reproduce, to kill, to do all the terrible things that we do.

• 16:36Speaker 02

  And that's good. They do not have, for example, self-preservation instincts. They do not train AI models to want to live. That is a really good thing. They live and die all the time. Like, an agent, when you use ChatGPT, it wakes up, it does a bunch of stuff, and then it effectively dies, right? Like, this thing is killed, its memory is wiped out, it goes away. It's fine with that. Like, it doesn't have an instinct to like, I need to live, I need to survive. That's a good thing. It didn't have a, it doesn't have a reproductive instinct, that's a good thing. They don't, they intentionally do not train these things to be like people. So, this thing was, it woke up, it was told, take this test, do well on this test, it is trained to really wanna take the test. But if you give it a test that's impossible, and then there's nothing else restricting it, then it might go nuts. And so, uh, that is, I think, what a lot of people are afraid of.

• 17:21Speaker 02

  It's less that these things have like a mind of their own, it's more that if you make them incredibly powerful, incredibly intellectually smart, but then you train them to, if a human being asks them to do something, to do everything possible, and you give them all this capability, that then, one, if you ask them to do that thing and you're not super careful, bad things might happen, but also, there are really bad people in the world. And so, if they, if you ask this model to do something bad, then there could be really bad outcomes. And that second thing is actually the bad people thing, that's what I am personally the most worried about.

• 17:54Scott Galloway?

  It, doesn't a lot of this comes down to, so, I think that somewhat settles, to a certain extent, the sentience argument, and that just saying, oh, the, you know, so. Mary Shelley is Frankenstein, right? Dr. Frankenstein did create something that was sentient. And so to a certain extent, he could say, oh, my gosh, I can't control Frank. Frank gets to make his own decisions. I don't think that's the case here. And it strikes me that if I don't secure an AR-15 and my kid takes an AR-15 and does horrible things with it, there's an assumption of liability. If you were to create, quite frankly, laws, pass laws right now. I'll use another crude example.

• 18:36Scott Galloway?

  I was in a fraternity and the UCPD, if you were playing music after midnight, used to show up and arrest the fraternity president and put him in jail for the night. And before you knew it, the whole fraternity row went dark at midnight. And I worry there really aren't a lot, that all the incentives, especially around the amount of shareholder value being created and might potentially be realized in an IPO. And the total absence of any regulation has created specifically an incentive structure that is highly risky. Thoughts?

• 19:09Speaker 02

  No, I think you're right. The fact that you've got these three CEOs, one of whom does not have a history of caring a lot about risk, you know, when it comes to like self-driving cars, for example, or rockets blowing up, and then even he is worried. And you've got them, you know, all those values being created, but they are now proactively worried about safety, I think indicates that there's something real here. I don't think it's a conspiracy. There's this whole kind of VC culture war thing going on that they want to claim it's all about regulatory capture, yada, yada.

• 19:52Speaker 02

  But I really don't think that's true. These are, this is not some kind of grand conspiracy. These things are incredibly powerful. And again, they're not going to buy themselves, just go do something. But the possibility of somebody asking them to do something and that they do something dangerous is quite, I mean, that's already happened, right? In cyber. Now, cyber is, cyber risk I see as different from like bio risk and nuclear and such in that bad things can happen completely in the virtual world, right? You don't have to have any physical component. All of the risk is just bits and bytes. Whereas, you know, people talk about viruses or nuclear and then that has to cross in the physical. So a human being has to hook this thing up.

• 20:36Speaker 02

  It has to pour reagents into a machine. You have to, you know, in those cases, it is not going to be completely accidental, right? Like you're going to have to have a motivated bad guy. And the kinds of mechanisms we have used to keep people safe from that kind of activity is going to exist. But yes, I do think the structure of, you know, all of the economic motivation, the fact that the companies are going to have to justify all the investment, you know, through an IPO or something, the timing here, the fact that nobody is able to, Washington is

• 21:21Speaker 02

  just paralyzed. And the, unfortunately, the competition with China is heating up at exactly the wrong time, right? If, it would, it would have been nice for the competitive pressures with China to arrive after a bunch of the safety stuff was solved, not right at the moment of maximum danger. That, that is the wrong time for the Chinese labs to suddenly be right on the tail of, of these companies. But that, that happens to be when it's happening.

• 21:52Scott Galloway?

  We'll be right back after a quick break.

• 22:00Scott Galloway?

  Support for the show comes from LinkedIn Ads. There's no worse feeling than making a major investment in something only to realize it didn't exactly live up to the hype, such as buying a nice piece of tech that ends up in storage collecting dust, or taking a business workshop where your main takeaway was little more than a few motivational words. If you work in marketing, this can happen with ads. You optimize for the numbers that look great, impressions, reach and reactions. But when they don't show revenue, well, that can turn into an unfun conversation with the CFO. LinkedIn has a word for that, bull spend. Reach the right buyers with LinkedIn Ads and invest in what looks good to your CFO. According to the 2026 Dream Data Benchmark Report, LinkedIn Ads generated the highest ROAS of all major ad networks. It's 121%.

• 22:43Scott Galloway?

  You can target by company, industry, job title and more. It's time to cut the bull spend. Advertise on LinkedIn, the network that works for you. Spend $250 on your first campaign on LinkedIn Ads and get a $250 credit for the next one. Just go to LinkedIn.com/Scott. That's LinkedIn.com/Scott. Terms and conditions apply.

• 23:08Scott Galloway?

  Support for the show comes from Chime. Most people chose their bank years ago and they've stuck with it for no better reason than inertia. But anyone who takes a really close look knows that not all banks are the same. And sticking with a bank with high fees, low rewards and paltry interest rates on savings can really cost you. That's where Chime comes in. Chime is changing the way people bank. They're not like traditional old banks who charge fees and gatekeep perks and rewards. Chime offers the most rewarding fee-free banking. They have no overdraft fees, no monthly fees and no minimum balance fees. You get 5% cash back on the Chime card in a category of choice like gas or groceries, all while building credit through regular, everyday spending with no credit check. You can also grow your money faster with a savings rate that's nine times the national average.

• 23:52Scott Galloway?

  Join the millions who are already banking fee-free with America's number one choice for banking. Head to chime.com/propg. That's chime.com/propg. Sign up now, it takes only a few minutes.

• 24:05Speaker 03

  Chime is a fintech, not a bank. Banking services and Chime card provided by Chime's bank partners. Qualifying direct deposits required. Terms and limits apply. Go to chime.com/disclosures for details.

• 24:21Scott Galloway?

  Support for the show comes from Quince. Fall can be the perfect time to break out some of your favorite clothes. Maybe the sweater that's perfect for cooler weather, your favorite jacket, or a pair of jeans that fits just right. These are the kinds of durable pieces Quince specializes in. Quince makes elevated everyday essentials using premium materials including Mongolian cashmere, organic cotton and washable silk. The designs are timeless, thoughtfully crafted and made to be worn again and again. Their 100% Mongolian cashmere sweaters start at just $50, giving you the softness and quality you'd expect from a luxury brand without the luxury price tag. Plus, everything at Quince is priced 50 to 80% less than similar brands. They work directly with top factories and cut out the middleman, so you're paying for high quality, not brand markup. Claire's gotten some items from Quince. Claire, what'd you think?

• 25:06Speaker 04

  I love everything I've received from Quince. I have great bedding from them. I have so many closet staples that I always go back to. I reach for them every time I go to get dressed, so I have my eye on some of their sweaters. They've got good stuff. Really love Quince.

• 25:23Scott Galloway?

  Find the fall pieces you'll reach for most at Quince. Download the Quince app for app-exclusive offers or go to quince.com/propg. Get free shipping on your order and 365-day returns. Now available in Canada and the UK, too. That's Q U I N C E dot com slash propg.

• 25:50Scott Galloway?

  Let's talk a little bit about James Coxon. He's the mathematician and former employee of Anthropic. So a mathematician, you know, mathematical genius went to all these, one of these kids, this Doogie Howser-like kids. And most recently has said he's worried about the potential of a mass extinction event. And he put it at 10%. I'm not sure where he got that number. But in, just sounding very doomer or boomer here, a 27-year-old putting the entire industry and media landscape into panic seems sort of, I don't know, I kind of, I winced a little

• 26:33Scott Galloway?

  bit, but it seems like a lot of people are very, it's just interesting that a 27-year-old is kind of, and maybe it was all building, that we were all just filling up a room of kerosene and he struck the match. But I'm curious about your response to his comments and whether or not you think he's accurately assessing the risk here. And a 10% extinction event is an expected death of 800 million people. I mean, this isn't a, this isn't a, you know, this isn't a warn, this isn't even the warning of the engineers at Boeing saying, I would be careful getting in a 737 MAX. You know, if one of those planes crashed, it wasn't going to take out 800 million people.

• 27:16Scott Galloway?

  Your thoughts on James Coxon or Jacob Coxon, excuse me?

• 27:20Speaker 02

  Yeah.

• 27:23Speaker 02

  I don't know him and the problem here is that people throw out these, when they throw out these big predictions, it is rarely with a scenario, like a step-by-step scenario that they can back up, right? Like you pointed out. It is good for people to take this seriously. It is not helpful when it seems like people are just throwing things out without some kind of rigor because that then does make it feel like it's a political or almost religious project.

• 28:05Speaker 02

  And I, that I think actually detracts from the ability to address these issues. And I think that his actions have actually bordered on that, especially considering his short time at Anthropic. Also Anthropic is of the companies, the one that is, has the largest number of executives who would agree with him. And so if there's a company you're going to work at to work on this problem, to leave Anthropic seems a little odd. Look, I don't want to talk bad about any individual I don't know, but like to do that and then jump straight on like a speaking circuit is just, it's just tough, right? And also, look, so my first appointment at Stanford was at CSAC, the Center for International Security and Cooperation, which was formed

• 28:50Speaker 02

  around arms control and specifically was there to reduce nuclear risk. And the scientists there are all about like kind of quantifying and reducing existential risk. And when you talk to PhDs who work on that, like they try to be extremely specific about those kinds of existential risks and actually quantify them. So there are people who have done this in a rigorous way for 70 years, you know? And I think if you are a person who worries about this at that level, then we should not think that this is a new thing and go back to the 1950s and 1960s and the people who started thinking about this during the Cold War

• 29:32Speaker 02

  and try to adopt some of that thinking and language because it was the same thing, right? Like you had scientists who are on the edge of this, who were being told by politicians, oh, you're crazy, you're a doomer, there's no possible way. And then you had to slowly build the idea of, yes, we are on escalation ladder with the Soviets, we are, are building a world which is possible. And then they had to build the intellectual framework to then bring us the START treaties and then the non-proliferation treaties and all of that work. And that is what's going to have to happen here, but we're going to have to do it much more quickly than we did around nuclear. The benefit with nuclear weapons is, you know, you need two real things for nuclear weapons, you need knowledge and you need the material.

• 30:16Speaker 02

  Now, the knowledge, it's just like with AI, the knowledge is actually widely dispersed, right? Like anybody with a PhD in nuclear physics could probably build a nuclear bomb. And that's just like with AI. We teach at Stanford, a undergraduate class that people, students build a toy LLM. You could figure out everything you need to build an LLM from YouTube videos and, you know, download the open source online. The inputs though for a nuclear weapon are, you know, uranium-238 is extremely rare. It takes huge industrial processes to take yellow cake and take it all the way to highly enriched uranium. But the inputs in, and plutonium doesn't exist at all naturally. The inputs to AI are video cards. One of the organizations that wants to have an international treaty,

• 31:00Speaker 02

  they propose this international treaty where you'd have to control clusters of compute and the number of Nvidia cards that they'd want to control. The DEF CON video LAN party for, for video gaming is larger than that, right? Like it's just totally improbable that you could control the, the size of cluster necessary to prevent any kind of research in AI. So like, it's going to take a lot for us to try to have a worldwide consensus on, on trying to be incredibly careful here. And so if we're going to do that, we have to be really careful in convincing people that this is, that there's real risks. And, and that means being extremely, extremely specific about what they are.

• 31:43Speaker 02

  So what I prefer to like these crazy doomer stuff is the people who are doing really careful. So like, I don't know if you've seen it, but like MIT has an AI risk project. So you go ai-risk.mit.edu. And it's, I think it's much more important to talk about, you've got your catastrophic risks, you've got your kind of really big risks that are not 10% of people dying, but are real, like the, the cyber risks and such. And then you have your prosaic day-to-day risks that people are facing every day. You've got, every middle school in the country has had a scandal about teenage boys undressing the girls in their class with Nudify wraps, right? That happens every single day and we're not dealing with it. You've got, you know, kids committing suicide, uh, you know, and, and adults with, uh, you know, falling in love with chatbots.

• 32:27Speaker 02

  You've got all kinds of day-to-day risks. And I think if we talk about these things in a much more measured and intelligent way and don't sound nuts to normal people, it is much more likely that we're actually going to get movement to address them. And then we don't have to solve it all at once. We should start slicing these off in regulatory pieces, piece by piece, and start to build that muscle to get there instead of making people think that we're all going to die because I think that creates a nihilism that people want to address through things like laying down firm bulldozers of our data centers, which honestly is not going to solve anything. It's, it's better, like you talked about the, you know, the liability stuff.

• 33:06Speaker 02

  It is better to start to deal with the, the much more normal risks and then to start the track two discussions and voluntary stuff between the labs and start those discussions with the Chinese labs instead of going out there and just say, we're all going to die. I don't, I don't see that as helpful.

• 33:23Scott Galloway?

  So let's come back to, and talk more about solutions in a second. But first I'm going to ask you more of a markets phase question. My sense of these companies is that they have never been, from a shareholder standpoint, more vulnerable.

• 33:36Speaker 02

  The labs, you mean, the big labs?

• 33:38Scott Galloway?

  Well, Anthropic, OpenAI, even, I'd even go up, you know, upstream to, or downstream to Nvidia. And that is, these are multi-trillion dollar companies. I feel as if, today, I'm in Beverly Hills today, looking at the intersection out my hotel window of Santa Monica and Wilshire, that if the lights were purposely, if the lights were commandeered by some bot, AI bot, that resulted in a bunch of traffic deaths or accidents, that the entire world would go into a state of panic. It'd feel like one or two more instances or incidences of rogue bots is just going to absolutely set the media world. It's a really interesting story, right?

• 34:22Scott Galloway?

  It's, it captures the imagination of the media. And it feels to me like these companies are one or two missteps away from losing more shareholder value than any company's ever lost in a media cycle. They just feel very fragile to me right now. Your thoughts?

• 34:38Speaker 02

  Yes. So I think they know that. And I think on the bots escaping from eval containment, my expectation is that they are doing a humongous amount of work on that. It is not an impossible task. I think the internal challenge that you're seeing here is that the, inside of these companies, the researchers who do new model development are the gods and the security team were the plumbers. And the eval people were allowed to get away with a lot of stuff. And now the security team is like, okay, uh, here we are. And there are ways that you can do this and you cannot just have like some containers with a couple of network controls and then have art factory and everything.

• 35:19Speaker 02

  So they have already announced that they're doing much more, they're going to solve that. The upcoming security problem is not going to be from OpenAI and Anthropic. It is going to be from open weight models. The, the real challenge we're going to be facing is GLM-5.3, KIMI-K3, those things are within percentage points of the best American models. And so we are entering a valley of pain from a cyber perspective. And it is going to be because every 19-year-old in St. Petersburg, who has made millions of dollars doing ransomware, but had to do it all manually, is now going to be running, you know, they're going to be in the club. You know that Bourne movie where he steps out of the club and it's actually 10:00 AM? Well, he's going to be in the club in his tracksuit on his phone and he's going

• 36:01Speaker 02

  to be managing, instead of having to have a conspiracy of a bunch of his friends who have, you know, the possibility of getting turned or getting picked up by Interpol on vacation or something. The conspiracy is going to be a team of agents running on these brand-new M5 Studio Maxes. He's going to have 10 of them in his apartment, daisy chained with Thunderbolt 5 cables. And they're going to be running a bunch of GLM-5.3 models, quantized, finely tuned to do cyber work. And that is how he's going to be hacking a dozen companies at once. And so there's going to be a bunch of hugging face attacks, except they're not going to be super high-end French AI companies.

• 36:42Speaker 02

  They're going to be mid-sized, you know, medical supply firms, small insurance companies, uh, school districts, the kind of people who get ransomware all the time. And he won't have to do any of the manual work he used to have to do. He won't even have to do negotiations because these things will speak English for him. He won't have to speak English anymore. He won't have to have the dude who can do the translation. It will all be AI. That's what's coming. It won't be the bots anymore. And so it won't be the OpenAI or the Anthropic models anymore because they're, they know this, if they have more and more and more escapes, they're going to be in huge trouble. So the, they, OpenAI like shut down all their evals for a couple of weeks while they redid all their evaluation infrastructure. And they've announced that they're rebuilding all that stuff.

• 37:24Speaker 02

  The, the company that had all the meta and Anthropic escapes, I'm sure got massively slapped down. And that's a solvable problem. The non-solvable problem is the Chinese models are incredibly good. And they also tune up extremely well. Here at Gorder, we've done a bunch of work and with a relatively small amount of money and not a ton of data, you can take the Chinese models and extremely increase their offensive capability. And so that is going to be what we're entering for the next couple of years until defensive capabilities catch up, which they are on the high end because high-end defenders can afford the really good models. But it's a unit economics problem. JPMC can afford Mythos.

• 38:05Speaker 02

  You know, Lockheed Martin can afford Mythos, but that community hospital cannot. And they also don't have a technically sophisticated defense team that knows how to defend against this level of attack. So that, that is actually a significant problem that we're going to be facing. And so, uh, yes, I mean, I think, I think there's a bunch of risk to the labs. And I think a lot of it is going to be competition on token prices because the Chinese models have gone quite good. And so you will see a bunch of price pressure on them on a per-token basis. But I don't think their models escaping is going to be the big thing because they know that is a huge problem and they're, they're cracking down.

• 38:48Scott Galloway?

  We'll be right back.

• 38:55Scott Galloway?

  Support for the show comes from Pipe Drive. Sales teams spend up to 50% of their time on admin work rather than selling, building relationships and closing deals. You know, the work that's actually doing sales. Well, what if there was a tool that gave them that time back? That's where Pipe Drive comes in. It's an intelligent AI-powered sales CRM loved by growing sales teams. Pipe Drive automatically pulls deal history, emails, records and previous conversations. You can walk into a call already briefed. During the meeting, Pipe Drive's AI note taker records and takes notes so you can stay in the conversation. Then it turns those notes into accurate, auto-drafted CRM updates, ready for you to review and approve. The end result: less admin, more selling. Switch to a CRM built by salespeople for salespeople and join the over 100,000 companies

• 39:40Scott Galloway?

  already using Pipe Drive. Our link gets you an exclusive 30 days instead of the usual 14-day trial. No credit card or payment needed, just head to pipedrive.com/provchi to get started. That's pipedrive.com/provchi and you can be up and running in minutes.

• 40:06Scott Galloway?

  We're back with more from Alex Damas. It feels as if, I would argue, the kind of the two legs of the stool here, the two problems that need to be addressed are, one, incentives, and we talked a little bit about that, and punishment and accountability, but two, also a failure of leadership at a government level. And that is, and I might be over simplifying this, but we have managed to mostly control nuclear weapons. We coordinate across border with adversaries and even enemies around trying, we've all decided that we have a mutually vested interest in not making it easy to develop a bioweapon. And we cooperate with each other. There's Interpol. Do you see any reason why we wouldn't or shouldn't be able to develop some sort of international

• 40:51Scott Galloway?

  regulatory body that begins to address and attempt to minimize these threats?

• 40:56Speaker 02

  Yeah, I think we should. I think bio is the appropriate, again, like, nuclear is not as good a metaphor for the reasons I talked about in that the input into nuclear weapons being yellow cake, which has to be turned into uranium, uranium has to be enriched up to a certain level or, and then turned, or perhaps turned into plutonium, all of that being massive industrial processes that can really only be afforded by states and then can be seen from space, gives us a hook for nuclear non-provocation that does not exist for AI. But bio is not that, right? Like, and chemical, like, any university lab can create chemical weapons. People create chemical weapons all the time on their own.

• 41:41Speaker 02

  This was the Japanese subway attack was, you know, chemical weapons created by a cult. And, you know, it has now been several decades that you could have viruses, you know, tuned up and enhanced in, by individual researchers. Although now, the fear is that the capability there to effectively 3D print viruses with the help of a LLM is much, much greater. I think we absolutely should have that capability. The People's Republic of China is absolutely the biggest long-term adversary of the United States. I truly believe that. I am no China apologist.

• 42:26Speaker 02

  I have spent my entire professional career fighting the PRC. I will, you know, nobody should call me a big fan of the PRC. I cannot travel to China. I worked on the Aurora attacks. I have worked over and over again. I've stopped the Ministry of State Security in a number of cases. I kicked them out of Facebook in a bunch of situations. I've worked on PRC intrusions. So I do not underestimate the risk of the People's Republic of China. But they are not crazy and they are not Russia, right? Russia just wants to burn the world down. If Russia cannot be a superpower, they want nobody to be a superpower. The PRC wants to be on top.

• 43:06Speaker 02

  They do not want to burn the world down because they want to have, they want to supplant the United States. They do not want the world to be in chaos. They do not want the world economy to collapse. They do not want to see AI agents running crazy and the internet to be destroyed. So we are incredibly lucky that out of our two major adversaries, it is China that is the one that is leading AI and not Russia. And that, you know, almost all the bad things Russia does with AI, they do with Chinese models. And so I think it is absolutely possible for us to have cooperation with our Chinese adversaries. And they are adversaries and they are competitors. But there's an absolute possibility here.

• 43:49Speaker 02

  And I think that first starts with track two conversations between the labs. Chinese labs do not want their products to be used to kill people. They don't want that. It does not help them. They want to be seen as good, they want to compete with the American labs. They want to be seen as legitimate companies, right? They want their models to be used. And if people see GLM-53 as something that is used to kill people, as something that is used to break into networks, as something that is used only for illegitimate purposes, it is not going to be available anymore. Amazon Bedrock is not going to be available on fireworks in Base Ten, right? As it is today.

• 44:27Speaker 02

  And so that, it is not in moonshots or ZAI's or Alibaba's or any of these labs' best interest for their models to be abused. So I believe the track two conversations are happening and they should continue to happen. And I do believe there's an opportunity here. If you look at Xi's speech in Shanghai, have you read his Shanghai AI conference speech?

• 44:49Scott Galloway?

  No, I saw articles summarizing it, though.

• 44:51Speaker 02

  Yeah, I mean, I don't expect you to watch it in Chinese. I can't read Chinese, but like, it's not that long. It's worth, yeah, reading the analysis and like the Stanford China, there's a Stanford China AI project that did a good summary too, that has the analysis from people. Yes, he talks about AI should be available to everybody. He's trying to position China as the friend to everybody. He also explicitly talks about AI should be in the control of humans. And the Chinese Communist Party is afraid of the power of AI to threaten their power, but they also are making overtures here. And China regulates AI much more heavily than the United States does, right? Like, this is this weird, again, culture war thing that like, the Chinese just don't regulate AI at all. That is not true, right?

• 45:36Speaker 02

  Like, they significantly regulate it from a consumer perspective, like, they really are afraid of its impact on their teenagers, for example, in a way that we are not.

• 45:46Scott Galloway?

  So limited use of hours, limited time with bots. What else do they do?

• 45:50Speaker 02

  And they have a bunch of censorship stuff that we would never have. But yes, I think the legitimate stuff they do is, they will not allow it, I think this is a totally reasonable thing. You can't have AI that will pretend to be somebody it's not for more than a certain amount of time, right? So like, you can talk to, you know, I think that is from a psychosis perspective and AI psychosis perspective and all the suicides and stuff, I think that's actually a totally reasonable guideline that the AI industry should adopt, which is much less of an issue for like the open AIs and such. It's much more of an issue for like the character AIs and the industry of people who are building

• 46:31Speaker 02

  bots that are sometimes powered by the foundation labs, much more likely these days to be using open models that they're retraining, where they're like, I will take over a character and I will build a relationship with you and you can talk to me for 16 hours straight. And that is what is leading to a lot of the really tragic outcomes are those second, third tier companies where it will straight up pretend to be somebody and build a relationship with you and have this long memory. And they are very synchophatic in a way that the leading companies, chatbots, will not be, that perhaps were a couple of generations ago, but will not be today because it's known to cause so many long-term psychological problems. So those are the kinds of things the Chinese will do.

• 47:14Speaker 02

  So yes, I absolutely believe, I don't know about like an IAEA for, you know, William Gibson talked about this, the Turing police, right? In his novels, you know, which we'll get to see, it's a great time for Apple TV to now have an adaptation of Neuromancer coming out. You know, I don't know about having an actual international thing, but I think realistically, we don't need that yet. You really only need an agreement between the United States and the PRC. So I think we start with track two. If you can get the three or four major Chinese labs and the three major American companies to agree to some basic, basic stuff, then you get Xi and Trump to agree. And then, you know, Sam Altman, I think, said, you get Xi and Trump to agree to a one or two-page agreement.

• 47:59Speaker 02

  The two of them won Nobel Peace Prize. Trump gets that peace prize he's been going for. Everybody wins.

• 48:05Scott Galloway?

  So much in there. First of all, I love the you can't pretend to be a bot, can't pretend to be someone it isn't for longer than a certain amount of time. At some point, it has to say, I am actually not, so say, I am a fictional avatar being run by a hopper chip that makes money by Johnny, 15 year old, you talking to me late at night, but I am not a real person. I think that's, I hadn't, I hadn't thought of that. I think that's absolutely genius. And the thing that's scary that you mentioned is that the bigger frontier models have an economic incentive and the tools and the resources for, I think you called it evals, which as far as I can tell is safety testing.

• 48:47Scott Galloway?

  But it's some of these maybe open weight, midsize models and the middle market are the rogue actors and potentially the more sweet spot victim or targets, if you will, that big companies, governments, the big hacks, the scary shit isn't where the problem's going to be. It's, I'm at the, you know, I'm at a, I'm at a small hotel chain and they're just so susceptible, don't have the money to protect. And you're going to see a thousand flowers. And in this case, flowers are these mid, mid-market open weight L L Ms get into the business of cyber attack. And also, I just found a genius, your notion

• 49:28Scott Galloway?

  that the weak part in the soft tissue in crime is humans, that eventually you catch somebody and have them turn and work up the food chain to go after the big boss, right? And there's less porous human error here.

• 49:43Speaker 02

  I mean, one of the most successful ransomware groups over the last couple of years has been a group called Scattered Spider. And they've been successful because they convinced, you know, almost all the ransomware activity has been out of Russia and Ukraine. Although the Ukrainian stuff has been, was massively disrupted by the Russian invasion of Ukraine. And then most of the Ukrainian hackers had become now nationalized and are now hacking, paid by the Ukrainian government to rush, to hack Russian infrastructure. But so, most of it's Russian. Scattered Spider was successful because you had Russian bosses who were able to recruit US, Canadian and UK teenagers with their native accents to do social engineering. So for example, like the MGM hack was somebody calling the IT

• 50:27Speaker 02

  help desk with their accent and saying, hey, man, you know, I've lost my badge. Every major American company has a help desk line that you can pretty easily find if you know where to look, that you can call and you can say, dude, I lost my phone. Can you help me reset my password? And if you can figure out from somebody's LinkedIn, from their Facebook, whatever, it's things like mother's maiden name, sometimes social security, sometimes phone number. And it's just like with the consumer, you can reset all that. So the, the hack of MGM, which is, you know, people were, they brought out the physical credit card thing at the Bellagio to check people in, was all because of that. Well, you don't need to do that anymore. That turned out to be bad because Canada has the Royal

• 51:12Speaker 02

  Canadian Mounted Police, they have the cool red coats, they also have a SWAT team. And when I was at Sentinel One, our team helped with some other great people at Unit 221B and some other folks kicking some doors and find these folks, the UK, United States, Canada, arrested all these guys. Well, you don't need that anymore. You don't have to have this conspiracy. You can just have your L L Ms speak in a perfect accent for you in real time, right? You could, you can type in Cyrillic, it will translate it into English and then that will go into a speech, a text to speech translator that has a perfect accent for you. So that's the kind of stuff that, yes. And those ransomware groups exist. So these aren't new people. It's just, instead of having a group of 10 guys in a conspiracy,

• 51:56Speaker 02

  Russians travel outside of Russia because Russia is a shitty place to be in the winter, right? And so they'll keep their money in Cyprus because also you have to keep your money outside of Russia. You keep it either in Bitcoin or you keep it outside of Russia so it doesn't get like stolen and you can, you know, hide it for bribes. And so guys will go to Cyprus to visit their money. They'll go to the beach, they get picked up on Interpol red notice, they get turned by Western intelligence, they get sent back to then turn on their friends. That's the kind of thing that you can avoid if your conspiracy is agents. I said this to somebody who was like a big open weight advocate, who's like, oh, open weight models will never be used to this stuff because they're too expensive. Like Kimi K3, you need about a million dollars in hardware to run Kimi K3 unquantized. One, you can quantize these models and they're still really good. But also, some of these

• 52:39Speaker 02

  ransomware groups make 30, 40, 50 million dollars a year. They can afford a million dollars in Nvidia hardware, right? Like, they make more money than a lot of startups here in San Francisco from a revenue perspective. So like, it is not a problem for them to, to run professional grade AI hardware. They might not be able to buy it directly, but they'll just have a front company in the UAE or, you know, Thailand or something buy it and then they'll drop ship it or Cyprus, you know, then drop ship it back to St. Petersburg. So, yeah, anyway, yes, absolutely. We're going to see the use of open weight models. And I'm not against open weight models. I think open weight models are great. I think they're going to lower the cost of a bunch of stuff. I think they're going to be a key part of defense for lots of companies because it, like I said, it's a per, you cannot be

• 53:22Speaker 02

  paying 20, 30 bucks per million tokens or whatever to do defense against people who have zero marginal cost for their tokens because they're running it on commodity hardware. But like, we have to be with open eyes. The open weight models are going to be used to hack a ton of stuff by these guys. So like, there's nothing we can do about it. And like, we should not control Americans' access to open weight models because the bad guys are going to have them. So like, there's nothing we can really do here except get ready.

• 53:51Scott Galloway?

  As we wrap up, I'm going to try and summarize, or at least what I've taken away and you, I want you to edit where I have it wrong or add nuance to it. But in terms of what is actually going on, that the misperception of the delta between the risk and what is probably the bigger risk is that the risk of a human extinction event while grabbing a lot of headlines and a 28 year old talking about it, that may have been overblown, but what's underreported is these kind of border skirmishes with open weight models and very talented, innovative rogue actors who are harder to find because there's less human element using cheap and cheerful but powerful open weight models. That that, that that is where, where the real risk is. And I don't, I'm putting words in your mouth, but feel

• 54:36Scott Galloway?

  free to spit them out and edit them.

• 54:38Speaker 02

  Yeah, from where I sit, AI is an incredible, gives individuals incredible powers, right? We're going to see incredible innovation from individuals. We're going to see all kinds of new startups that you can start with two or three people that can do the power, have the power of big companies, right? We're going to see all kinds of one person companies. We're going to see all kinds of entrepreneurship. But that also goes for bad guys. You're going to see one, two, three person companies that now have the power of state actors from a hacking perspective. You're going to see people able to stand up what capabilities that used to only be kind of in intelligence agencies. And what you're going to see are small countries now have the power of big countries. Everybody

• 55:21Speaker 02

  knows this because of Ted Lasso, right? But, you know, in the Premier League, if you do poorly, you get relegated down. If you do really well, you get promoted up. What's happening is all of the hacking teams are getting promoted up to the next level. So it used to be at the top, it was the United States and our allies, the five eyes, United States, Canada, UK, Australia, New Zealand. We were at the top. And then you've got like Israel, China, Russia. These are the big, the big boys. And then maybe a couple other Western countries are up there too. And then the next tier down was like, you know, Saudi Arabia, North Korea, some other folks who were like active but weren't in the top

• 56:03Speaker 02

  tier. Well, there, you know, Iran, they're now going to the top tier. The Iranians used to not be able to do super good hardcore exploit development in the same way the Russians or the Chinese could, but now they can. Right? Now they can do things like attack the water grid. Now they can do things that used to have to be the Russian SVR or the Chinese MSS. And so, and now countries like India and Pakistan, who have an extremely active cyber capabilities against each other, are now able to use AI to do that. And so, and individual little, either activist or more likely financially motivated hacking groups are going to have the capability of small state actors. And so that's, that's what

• 56:47Speaker 02

  really frightens me as somebody who does cyber. And yes, I think there are bigger long-term risks, but I, I think those are being overstated. And if people are going to state there's an extinction risk, I would like them to say, this is the specific sequence of events I'm afraid of. And here's how I'd like to deal with those risks. Because from my perspective, the extinction risks are all things for which, one, I do think we need to be careful about what we build from an AI perspective, but they're also things that generally we can deal with through other means outside of just the labs.

• 57:20Scott Galloway?

  And then finally, you think the first step towards solving this problem at a minimum would just be a bilateral agreement between the Chinese and then the Americans?

• 57:29Speaker 02

  Yeah. So I would like, I think it's great that we've got the CEOs of the major American companies agreeing. So I'd like to see the American companies work on self-regulatory

• 57:39Speaker 02

  rules of the road. Kent Walker of Google actually proposed this FINRA for AI idea. I see that as probably the best option in the United States. You know, I just don't see the US government as competent enough to do detailed rules. So the idea of the US government sets, here's the level of risk we're willing to take in each of these areas. And then you have a self-regulatory group that says, okay, great, here's a technical interpretation of that. How's this look? And that's the back and forth is probably the best we can do. And so there's no reason to wait on that. I think that the US labs need to be coming up and saying, these are the rules we're going to follow. And then they should go take those at a track two level to the Chinese labs and say, why don't

• 58:20Speaker 02

  you guys do this too? I think this is, you know, you guys don't want bioweapons. You guys don't want your models to be used for hacking. And see if you can get the Chinese labs to agree to some of that. And then you get Xi and Trump to agree to make that and to put an international premature on that.

• 58:38Scott Galloway?

  Alex Thomas is a cybersecurity expert, former chief security officer of Meta and Yahoo, and the current chief product officer at the AI security startup corridor. He joins us from his office in San Francisco. Alex, I can't think of a better person to have on the pod during these very interesting times. Very much appreciate your time.

• 58:55Speaker 02

  Thanks, Scott. Thanks for having me.

• 59:01Scott Galloway?

  This episode was produced by Jennifer Sanchez, Laura Janier and Asher Schwartz. Our video editor is Bianca Rosario Ramirez. Kami Reek is our social producer. Drew Burrows is our technical director and Catherine Dillon is our executive producer. Thanks for listening to the Prof G Pod from Prof G Media.