Demis Hassabis, Google DeepMind and the Battle Over AI Safety

Kara speaks with journalist and author Sebastian Mallaby about his new book, "The Infinity Machine," and its central figure: Demis Hassabis, the CEO and co-founder of Google's AI research lab, DeepMind, and a Nobel Prize winner in chemistry. Sebastian argues that Hassabis is one of the original scientist-entrepreneurs of modern AI. And although he's extremely competitive and research-driven, Sebastian says Hassabis is also one of the few big names in AI development who genuinely cares about public safety. However, despite his best intentions, Hassabis doesn’t have the power to change the race dynamic driving AI’s rapid, and potentially unsafe, development. Kara and Sebastian break down DeepMind's relationship with Google, the push toward artificial general intelligence, and whether the government can regulate the technology before something goes wrong. Questions? Comments? Email us at [email protected] or find us on YouTube, Instagram, TikTok, Threads, and Bluesky @onwithkaraswisher. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Kara, Kara, I was trying to do this sort of optimistic, you know,

• 0:03Kara Swisher?

  I understand, Sebastian.

• 0:06Speaker 00

  You just burst my balloon.

• 0:07Kara Swisher?

  You know, intelligence has its limitations, but stupidity and greed are infinite. So that's my feeling. I'm sorry.

• 0:25Kara Swisher?

  Hi, everyone. From New York Magazine and the Vox Media Podcast Network, this is On with Kara Swisher, and I'm Kara Swisher. My guest today is journalist and author Sebastian Mallaby. He's a longtime chronicler of power and innovation, especially in the world of finance and economics. For his latest book, The Infinity Machine, he turns his attention to tech. His central figure is Demis Hassabis, the CEO and co-founder of Google's AI R&D Lab, DeepMind, and a Nobel Prize winner in chemistry. Hassabis has dedicated his life to using AI to unlock the mysteries of physics and biology. A former child chess prodigy, he's rabidly competitive and insatiably curious. But Mallaby says Demis is also one of the very few big names in AI development who genuinely

• 1:09Kara Swisher?

  cares about public safety. His struggle to balance his ambition, his personal goals, and the realities of the corporate AI race is, according to Mallaby, one of the most defining stories of the era. I think it's really important to look at characters like Demis because he was a very early person in AI, at least the modern version of AI, because it's been around forever. He's also based in London and kept his company there and away from Silicon Valley, so he has different goals here. At the same time, he's incredibly ambitious and a bit crusty, according to lots of people, very typical of a science researcher type. But he has less and less power over what's happening because of the vast amounts of money pouring into the space.

• 1:50Kara Swisher?

  People tend to forget safety for profits, if in fact, they make profits anytime soon. He is different from other AI developers, though there's a lot of people who are sort of in his area, but he's certainly one of the leading minds of this age and someone you should know well. All right, let's get to my interview with Sebastian Malaby. We have two expert questions today from Nasdaq CEO Adina Friedman and Kent Walker, President of Global Affairs at Google and Alphabet. This interview was recorded live in front of a virtual audience. It's part of the Dean Summer Book Series at American University's Cogod School of Business. So stick around.

• 2:38Kara Swisher?

  Support for this show comes from The Guardian and their new show, Stateside, where journalists Kai Wright and Carter Sherman use the entire independent reporting resources of The Guardian to slow down the news and wrestle with the questions we all have about what is actually happening in the world. Join Kai and Carter three times a week as they utilize all the reporting resources The Guardian has across news, international coverage, climate, culture, wellness, and more. And The Guardian's not owned by a billionaire. They fearlessly report the facts without interference. Go to theguardian.com/stateside to learn more and listen wherever you get your podcasts or watch on YouTube. That's theguardian.com/stateside.

• 3:22Kara Swisher?

  Support for this show comes from Odoo. Running a business shouldn't feel like surviving a software group project. One app for accounting, another for inventory, another for sales, and somehow, none of them ever talk to each other. That's where Odoo comes in, an all-in-one business management software that brings every part of your business together, from sales and accounting to inventory and marketing, all in one powerful platform. No messy integrations, no bouncing between tabs, and best of all, no spreadsheets. Stop managing software and start managing your business with one unified system. Try it today for free at Odoo.com/kara. That's O-D-O-O dot com slash Kara.

• 4:05Speaker 02

  Megan Rapinoe here. This week on Why Are You Like This? I am talking with rapper and singer Young Miko. I love talking to her about her rise to fame, being openly queer, her love of soccer, of course, and her background as a tattoo artist. Check out the latest episode of Why Are You Like This? Wherever you get your podcasts and on YouTube.

• 4:29Kara Swisher?

  Sebastian Malaby, thanks for coming on ON.

• 4:31Speaker 00

  Great to be with you, Kara.

• 4:33Kara Swisher?

  So for people who don't know who Demis is, he isn't a household name, the way other major AI figures have become, like OpenAI's Sam Altman, Anthropic's Dario Emoti, or of course, XAI's Elon Musk. Even though he's accomplished as much or more in AI, I'll just say, I broke the story when they bought his company at Google a long time ago. And I met him just briefly during that. He was mad at me because I broke the story. But nonetheless, let's put him into context because I did understand the importance of him in AI. And he definitely, he's not in the shadows, but he's certainly not as well-known and he doesn't put himself out there. So how is he like them and what sets him apart?

• 5:13Speaker 00

  So the first thing that marks him out is that he was the first, right? So the first scientist entrepreneur sets up DeepMind in 2010, when AI can't even recognize a cat photo. Nothing worked, deep AI winter. And then the others came afterwards, and they came afterwards as straight derivatives, right? So, you know, OpenAI set up five years later, and Elon and Sam are explicitly trying to do the anti-Demis, anti-Google DeepMind company.

• 5:41Kara Swisher?

  Mm-hmm, they were.

• 5:42Speaker 00

  And then you go forward to Anthropic, and you know, Dario, as a PhD scientist, wanting to sort of, you know, deal with more of a social conscious and make it safer, or like you said, what he says, I think also admired Demis. And as people at Anthropic say sometimes, he's the one who is the closest to a model that Dario had in his head. So first point is, he's the original Demis. He's the only one with a Nobel Prize. He's different in that because he started early, his approach to AI was not merely to scale an existing technology path, but actually to invent the technology path. And so they brought together agentic systems from reinforcement learning,

• 6:26Speaker 00

  and mixed that up with deep learning, learning from data, and kind of invented the field. And I think there's some DNA left over today from that experience where if it was to be the case that AI in the next three years went down a novel path, something more than just scaling the transformer architecture, it would be more likely to come from Google DeepMind and Demis Sabis than from the rivals.

• 6:52Kara Swisher?

  So talk a little bit, also, by the way, AI has been around for a very long time. He started in England, that was one of the things, Demis started the company there. And of course, Alan Turing, hello, nice to meet you. But a lot of stuff went on very early in England, actually, which is interesting. But how is he like the others, for better or worse? How do you, if they look at him as a mortal or someone that they look up to, or are trying to copy in some ways? Talk about the attributes and maybe some of the negatives about that.

• 7:19Speaker 00

  He is different, I think, in that his motivation is this intensely, intensely scientific curiosity, to the point where he expresses it to me in spiritual language. And there was this moment, you know, fairly early on in my more than 30 hours of conversations with him, where we were sitting in a park in London. And at the next table, there were people having regular chats about their friend who went to hospital, or whatever it was. And I was opposite this sort of messianic possessed person who was banging in the table and saying, you know, when I'm reading a scientific paper and it's 2 o'clock in the morning, reality is screaming at me, demanding to be discovered, demanding to be understood. And if I can understand it and get close to the fabric of reality and nature, then

• 8:01Speaker 00

  I will understand the intelligent being that that might have created this, and I'll be closer to God.

• 8:08Speaker 00

  Or at least his understanding of God. So that is a, something which is very distinct, right?

• 8:13Kara Swisher?

  Some of them are getting spiritual. Peter Keel talks about the Antichrist, which is anyone who's against AI. You know, when you said that, when he's doing that, that actually would trouble me, because these people do tend to think of themselves as gods manipulating machines, or in some fashion. And so, a lot of it is indeed spiritual, and at the same time, they're not gods, they're people, correct?

• 8:37Speaker 00

  Correct. And I mean, this gets to this question of, you know, is the motivation simply the sweetness? This is the quotation from Robert Oppenheimer, The Manhattan Project. The quote was, you know, repeated, and at the time Jeff Inton didn't know that he was being overheard by a journalist from The New Yorker, but he was. And what he said was, you know, he repeated Oppenheimer's line, "I'm not creating AI for any particular good reason. I'm doing it because..." Because when you see a technology that's sweet, you just do it and you figure out the consequences afterwards. And as a scientist, I can't resist. And I think Demis has quite a lot of that. And so.

• 9:13Kara Swisher?

  Except Oppenheimer was a little more thoughtful, wouldn't you say? I mean, I have become death or that kind of thing.

• 9:19Speaker 00

  Ah, I know. I mean, Demis uses some pretty funky philosophic references too. He talks about Spinoza, he talks about Immanuel Kant, he can riff on anything.

• 9:28Kara Swisher?

  Yes, comparatively, he's quite like deep. So is Dario Emo in many ways. I mean, Dario Emo can quote from plays, which none of these others can do. Elon's mostly sci fi crap and Sam, not nothing at all, although he's not stupid by any means. But Demis comes from a working class immigrant family. As a kid, he was a chess prodigy, not a surprise. He was consumed by the idea of AI. Around seventeen, he said he decided to dedicate his life to this. Talk about what drove that singular focus to be the person who solves artificial general intelligence. And explain what that means, because that's defined differently by different people. I want to hear how you see his definition of it.

• 10:08Kara Swisher?

  And this is called AGI. And again, you write that its definition remains fuzzy. So talk about his definition.

• 10:14Speaker 00

  Sure. So just just on the kind of early motivation, it really was extraordinary. I mean, to at seventeen, and we're talking kind of nineteen ninety three. You know, AI was talked about since Alan Turing in the nineteen forties, but it went through these rises and falls, as you know. And and this was definitely winter at that time. And, you know, the kind of AI that was happening was academics fiddling around with toy experiments. It had no relevance to the real world at that time. So to believe not only that you were going to devote your career to AI, but to super powerful AI, artificial general intelligence, which, as you say, is defined by different people. But we could, we could say it's the ability for a AI system to be smarter than humans at any screen based task.

• 10:58Speaker 00

  I think that's probably a sort of average type definition. It's not the most extreme or not the most modest. So, you know, this is what he wanted to do when he was seventeen, which is pretty amazing.

• 11:07Kara Swisher?

  Right, and that's how he defines it, any screen-based task of figuring out.

• 11:11Speaker 00

  Well, actually, you know, the funny thing about Demis is that he so loves the process of discovery and of research and the quest, and he's got one Nobel Prize, but he definitely would like another one, that he loves to go for a definition of AGI these days, which puts it out further into the future so that the joy of the journey can be extended. And so what he says is, if you could train a computer or an AI system on everything that was known in nineteen eleven, and then you waited to see if by itself it would discover general relativity, and then it did, that that would be AGI. That's his new definition.

• 11:48Kara Swisher?

  Ah, probably not.

• 11:49Speaker 00

  Maximalist.

• 11:50Kara Swisher?

  Probably it couldn't. Yeah, I guess. Wow, that would be something. Sorry, I doubt that would happen, actually. I'm going with Einstein on this one. He co-founded DeepMind in twenty ten. Obviously, the name DeepMind is exactly what he's going for. Four years later, as I said, Google bought it for six hundred and fifty million dollars. He was able to extract big concessions in the sale. I remember writing about it. It would remain in London, the offices, and there would be restrictions around the use of their technology. Talk about being in London, because most people working on this stuff were in, whether it was Fei-Fei Li or Hinton, he was at Google, Fei-Fei Li was at Google for a little bit and then elsewhere. Some of the early people were deep in Silicon Valley.

• 12:30Kara Swisher?

  So why stay in London during this sort of social media smartphone boom at the time? When I wrote about it, everyone was like, what's that? And I'm like, no, no, this is a big friggin' deal for Google to buy. I understood what he was doing there. But talk a little bit about why he sold it. He obviously needed the money to grow it, I think, because as it's turned out, this needs a lot of money. Talk about remaining in London and why.

• 12:55Speaker 00

  Yeah. So, you know, Demis's official explanation for why he stayed in London was that there was a lot of scientific talent in London and in Europe. And, you know, if you were trying to recruit great scientists, you had a competitive advantage of hoovering up that, you know, geography. And, you know, if you were in Silicon Valley, it'd be super competitive to get the best people and it would, you know, better to be in London. When you actually look at who he recruited, the first PhDs who came, they were sort of from all over the place. They came from Canada, from people who'd studied under Jeff Hinton in Toronto. They came from Switzerland, where there was a good PhD program in AI.

• 13:36Speaker 00

  You know, there was Corey Karachuk, who studied in New York under Jan Le Cun. So this notion that it was-

• 13:42Kara Swisher?

  For people that don't know, Jeff Hinton is considered one of the early godfathers of modern AI, also Fei-Fei Li is, and Jan Le Cun later ran, until recently, Meta's AI efforts.

• 13:54Speaker 00

  Right, right. And he was also an academic kind of pioneer.

• 13:57Kara Swisher?

  And remains that way.

• 13:58Speaker 00

  And the point is that what Demis claimed as being the reason for staying in London was not what, how it turned out, because he was recruiting from all over the world, including from the US. And so I actually think the reason why he stayed in Britain is simple patriotism. He just likes Britain. You know, he's a classic melting pot product of immigrant parents in London, very attached to London, and he simply thinks that the values in Silicon Valley suck, and he would rather stay in Britain.

• 14:29Kara Swisher?

  Yeah, I heard him say that. So the mentality around the company was very different as it was happening, and especially at that moment, when things sort of started to go off the rails. I would say 2010, 2011 is when Silicon Valley started to lose its ever-love and mind. So, you know, beyond the move fast and break things nonsense, it really did. It became something much different. And that didn't exist in London, obviously. And the safety demands were the biggest concession for Google. DeepMind demanded a ban on military uses. Think about Daria Modi at this point. Demis and his team also insisted on forming an outside safety review board to dilute Google's influence over the technology smart. Why did Google ultimately agree to those terms, given it was obvious how important

• 15:12Kara Swisher?

  AI was to the company's future, especially to their core business search, which this would decimate essentially in its current-

• 15:19Speaker 00

  Yeah, yeah. I mean, the point person in the negotiation with Demis in that transaction was actually Larry Page. And that was important because Larry, you know, his father had worked academically on deep learning. And that impressed Demis, and Demis felt that if he sold the company to Google specifically, he would be in the hands of somebody who loves science. Larry, you know, he once said to me, you know, you could imagine Larry as a professor at a top college like Stanford. And from Demis, that's the ultimate comment of respect. So whereas, you know, Mark Zuckerberg tried to buy Demis and DeepMind, and Demis sort of laughed him off, and Elon tried to buy DeepMind and Demis refused.

• 16:02Speaker 00

  He was happy and comfortable with Google because of that scientific culture.

• 16:07Kara Swisher?

  As they would be, because these were two PhD candidates themselves, and Google was started in a much more scientifically focused way.

• 16:13Speaker 00

  And then came the conditions, as you say, around safety, around the AI, a sort of safety oversight board. And I did speak to the sort of chief lawyer, who was on the M and A team at Google, who remembered sort of basically being, you know, terrified by the idea of, you know, diluting Google's right to do what the heck it wanted with this asset for which it was paying, you know, several hundred million dollars.

• 16:41Kara Swisher?

  At the time was huge. That was an enormous shock to people at the moment.

• 16:45Speaker 00

  Correct.

• 16:45Kara Swisher?

  Today, it's nothing. It's like some credit card.

• 16:49Speaker 00

  But this lawyer said to me, no, we have a fiduciary duty as a public corporation to own assets from which we derive value. And if we're told, well, you can't derive value because you have to have this outside bunch of safety overseers who are going to tell us when you can or cannot deploy, forget it. I mean, that's, are we even allowed to sign that kind of deal? And the way he described it is that the reason that Google caved and gave DeepMind what it wanted is that whereas with many startups that Google buys, the notion is they've got some good tech and we'll get rid of the founder if we don't like him. To the contrary with DeepMind is they wanted Demis. Like, he personally was the reason they were so keen on paying up. And by extension, they were willing to give him what he wanted on safety.

• 17:37Kara Swisher?

  We'll be back in a minute.

• 17:51Kara Swisher?

  Support for this show comes from The Guardian. Trying to locate news sources that reliably separate fact from fiction can seem like looking for a needle in a haystack these days. Social media feeds, cable news, and subscription-based news outlets might be able to give you one version of the news, but not in the way that you need. You want news fast, but slow enough to actually absorb the information. State Side with Kai Wright and Carter Sherman is here to do just that. The podcast tries to slow down the news and wrestle with the questions regarding the world at large at your own pace. Three times a week, hosts Kai Wright and Carter Sherman bring their expertise together to showcase different and engaging perspectives regarding the world today. They will use their collective knowledge to discuss the news, international affairs,

• 18:35Kara Swisher?

  climate, culture, sports, lifestyle, fashion, and wellness. Plus, The Guardian wants to emphasize they're not billionaire-owned, meaning they're free to report the whole picture without interference. Go to theguardian.com/stateside to learn more and listen wherever you get your podcasts, or watch on YouTube starting May 13.

• 18:55Speaker 03

  Ripple has been powering modern finance for a decade. We're leading the charge to bridge the gap between existing financial systems and the future of finance. You can have it all: legacy, stability, and digital innovation. Systems that work with your other systems. Because with the right partner, the question isn't crypto or traditional finance, the question is simply, why choose? A new standard of modern finance. It's happening with Ripple. For informational purposes only, availability of products can vary and may be provided by different Ripple entities.

• 19:26Kara Swisher?

  Support for this show comes from Hers. Losing weight on your own can often feel like a fight you never win. Luckily, weight loss by Hers offers access to the FDA-approved Wegovy pill and the Wegovy pen. Wegovy is the first-ever GLP-1 weight loss medication available in a pill, but also comes as a pen, so you've got options. With Wegovy, you can lose up to 20% of your body weight. People lost an average of 14% body weight at 64 weeks in a clinical trial of Wegovy alongside diet and exercise. To get started with Wegovy, just go to forhers.com and complete an online consultation with a lawyer. License provider who will determine if your prescription is appropriate. And hers doesn't stop there. Nutrition and lifestyle guidance are part of the plan,

• 20:08Kara Swisher?

  along with access to 24/7 messaging with your care team. So support is always in reach. Ready to reach your goals? Visit forhers.com/cara to access personalized affordable care for weight loss, hair, menopause, and more. That's forhers.com/cara. Individual results may vary and stopping treatment may result in weight regain.

• 20:33Kara Swisher?

  Now, he had hoped there'd be one collective effort to develop AI because there wasn't a lot of AI development. As you said, it was a winter of AI at the time. And he had hoped there'd be sort of a Manhattan project with Google and DeepMind at the center. And, you know, one of the, one of the early interviews I did with Sam and Musk when they were starting OpenAI was their worry that DeepMind and Google would dominate, not just them, but possibly Mark, who they had thought little of at the time in that regard. And so he was shattered after the first Ethics and Safety Group meeting in 2015. Musk hosted it and then teamed up with Sam to start OpenAI when they were getting along. And it was a rival.

• 21:14Kara Swisher?

  And it was started, he is, he was testifying honestly for a second when he talked about the original reasons. I happen to write that story when that happened, when OpenAI was created. So talk a little bit about the fallout of the meeting and how it changed his approach to AI development and the way he ran DeepMind.

• 21:35Speaker 00

  Yeah, I mean, so what happened, as you say, is that there was this safety oversight meeting, which was supposed to be for DeepMind to get useful feedback. And it turned out the people at the meeting were not giving useful feedback. They were kind of learning how DeepMind was doing stuff and then

• 21:49Kara Swisher?

  Collecting information.

• 21:50Speaker 00

  So in that meeting, there was both Elon, but also Reid Hoffman, who was one of the funders of OpenAI. And so the reaction was from Google, forget the idea of safety oversight boards. We've tried that once and it was terrible. So we're not going to do that again. And Demis's reaction was, well, we can't give up on the idea of a safety oversight board. It's essential for this technology. And so we're going to have a fight about it. And so now this is, you know, 2016 is when this really gets going. And there's this thing called Project Mario, as they called it internally at DeepMind, where for three years they retained, you know, armies of lawyers and investment bank strategists

• 22:35Speaker 00

  and people like that. And there were term sheets running to 50 pages that flew back and forth between the, you know, negotiators in Mountain View and the negotiators in London. And I was shown some of these things.

• 22:47Kara Swisher?

  He tried to spin it out. He tried to spin it out.

• 22:49Speaker 00

  Yeah, I mean, he thought of spinning out because that would be the threat that would force Google to agree to the safety oversight board. And, you know, Reid Hoffman promised a billion dollars to finance a spin out. They went to see Joe Tsai, the co-founder of Alibaba in Hong Kong to try and get some money out of him. It was a real thing. And, you know, what happened after three years is that Demis caved. He was so exhausted by this three years of negotiation and, you know, he'd get another long term sheet from the lawyers and he would hold his head in the hands and, and say, I don't want this.

• 23:25Kara Swisher?

  They were never going to let him do it.

• 23:26Speaker 00

  I don't want this part of my brain to expand, is what Demis wound up saying about legal documents. And so the upshot was, you know, he didn't get what he wanted. Google was never going to do it. I kind of developed a weird respect for Sundar in learning about this process because he was very canny. You know, he knew he didn't want Demis to spin out. He didn't say it quite directly. But he hid behind his chief counsel and other people who played bad cop on his behalf. And at the end of the day, he got what he wanted, which was to keep Demis inside the tent, not spun out, so that he would be the secret weapon.

• 24:02Kara Swisher?

  Right, because he did understand what was happening in Silicon Valley with the competitors, right? Google had to be at the forefront of this. And Sundar, who's a very lovely affect, is really good at playing rope-a-dope, I've always found, you know, in niceness. But let's talk about this idea of what Demis wanted to do here. He says he was naive to believe in a singleton scenario where AI developed collectively, but he also saw himself at the center of the effort. Now, you could say that's egotistical, and it does take a certain amount of narcissism that you should be in charge of all AI development. I have problems with, I'm more with the collective, but at the same time, it always devolves into either one person or just a few people developing this thing.

• 24:45Kara Swisher?

  But talk about this idea, because they were never going to do it once they understood the financials here, the amount of money and the amount of power. Was that naive of him or egomaniacal? He says he was naive.

• 24:58Speaker 00

  Yeah, I think it was naive of Demis to believe that AI would be developed by one single lab on behalf of all humanity. I mean, humans are just disputatious and tribal and jealous, and they don't do that.

• 25:10Kara Swisher?

  Greedy, mostly.

• 25:11Speaker 00

  Well, that too, probably. But, you know, I-

• 25:13Kara Swisher?

  I like that you said probably there. Oh, God.

• 25:16Speaker 00

  No, I said probably because what strikes me, Cara, about this whole field is that the normal cynical explanation that people are doing it for the money is weirdly less true, because AI is such a weird technology. It attracts people who are basically doing it for power, for ego, for-

• 25:36Kara Swisher?

  Power is, yes.

• 25:37Speaker 00

  It's Promethean.

• 25:38Kara Swisher?

  You're right.

• 25:38Speaker 00

  It's not about money.

• 25:39Kara Swisher?

  So, you know, in hindsight, why did he not see this at the time? And I don't think he was doing it for the money. He is a true believer of all these people. And of course, there's the glory. He wants a Nobel Prize, that's pretty great, and he has curiosity. But should he have been less naive? Or explain the naivete, because it seems obvious to see that they were not going to do this.

• 26:01Speaker 00

  Yeah, I mean, I think there's some mitigating circumstances that you can cite that make it a bit less crazy to have believed in this single thing. That would be simply that when he was beginning, you really could fit all the AI believers in one conference hall.

• 26:18Speaker 00

  And so there was a sense of a single community at that point. And then, you know, he starts DeepMind and it's going really, really well. And he does, you know, this Atari model, which is just an astonishing agent that can learn by itself how to pay, you know, dozens of different video games. And that impresses everybody. And then he gets, you know, the Google checkbook behind him. And it feels like he's the only game in town. And if there's a second game in town, it would actually be Google Brain in Mountain View. And so that is the same company. And, you know, OpenAI just wasn't a thing then and didn't really become a thing, you know, until, let's say, 2019.

• 27:00Kara Swisher?

  Till recently.

• 27:00Speaker 00

  Yeah.

• 27:01Kara Swisher?

  Yep.

• 27:02Speaker 00

  So, so I think it wasn't totally crazy at the time to think, well, I can't see anybody else on the horizon who's doing this, so it's going to be me. But it's still crazy, still naive, still, you know, immature to think that when you're confronted with a godlike technology, there won't be lots of acolytes trying to do it and there'll be sectarian splits and you can continue that metaphor.

• 27:25Kara Swisher?

  Right. Now, we've heard Altman, Musk and even Mark Zuckerberg claim they care about AI safety. I believe none of them, only to watch them prioritize growth and their own egos over time after time. Is, is he different? Is he remain different? There's almost no incentive for tech companies like Google to make AI safety their top priority. For example, Google is eager to supply AI to the Pentagon after agreeing not to do so when it bought DeepMind. So while he understands, Demis understands the risk of developing AGI, does he have any more power to contain them in the face of the pressure, both national security, financial, just to win, to be the dominant technology of the next era, et cetera?

• 28:06Speaker 00

  Yeah, I think Demis has gone a long journey, right, since founding DeepMind 16 years ago. You know, at the beginning, he thought there might be a singleton scenario. Then, when that started to break down because OpenAI, you know, launched a competitor, he still hoped to get oversight for his technology, as we discussed, with Project Mario negotiating. And then when that didn't work, he said, well, at least I'll make AI good for humanity by doing it for science. And that's when he did the protein folding system. And then, you know, along the way, although interestingly, he didn't ever tell me this. I only know it from other sources. He was the one who said to Rishi Sunak, the British Prime Minister, you know, let's have a global safety discussion at Bletchley Park in 2023, and that happened.

• 28:51Speaker 00

  And so I think he had a reasonable track.

• 28:54Kara Swisher?

  Explain why Bletchley Park. That's where.

• 28:56Speaker 00

  Well, that was where during the Second World War, the German code was broken with by Turing with sort of the early kind of precursors of AI.

• 29:06Kara Swisher?

  It was very symbolic.

• 29:07Speaker 00

  Exactly. So, so I think, you know, up to 2023 and that summit, he had a pretty good track record, Demis, in, you know, walking the walk as well as talking the talk on safety. I think, though, that once he got into the race full on to do chatbots in competition with, you know, OpenAI and then Anthropic, he's, he's done less and less, at least visibly. And it strikes me, for example, that by his own logic, open source models are dangerous. And yet, Google releases them. He wanted to deprive the military of AI. And yet, Google now supplies them. And his sort of rationalization for this is simply that there's a race on.

• 29:51Speaker 00

  Chinese labs are part of the race. If he were to step back, if he were to quit, it would make zero difference to outcomes for humanity because there'd be six other labs doing it anyway.

• 30:02Kara Swisher?

  Sure, but he wouldn't be part of it, like, in some ways, right?

• 30:05Speaker 00

  Yes. Look, and I think that gets a super interesting debate about, you know, do we care about the gesture? Do we care about the outcome?

• 30:10Kara Swisher?

  Right. I mean, I think his power had waned in that regard because people caught up, even if they're not as brilliant as he is. Now, he reminds me a little of Nikola Tesla, ultimately, you know, the greatest mind, and Edison just ran right over him. In 2024, he shared the Nobel Prize in Chemistry for his work using AI to predict structures of proteins, as you noted. Amazing thing. It could lead, it will lead to the creation of new drugs and vaccines to fight diseases. It's an amazing thing. And an astonishing use of AI in, in service of humanity. I think one of the best examples, and not marketing, that the rest of them foisted upon. On me on a daily basis. And it could be where where AI makes people's lives better. Talk about this because and what it can mean for medicine. I know a lot about this.

• 30:54Kara Swisher?

  I've done a lot of reporting on this, but it really was a moment. And it has taken off, whether it's mRNA, all kinds of stuff, drug discovery. You know, the quickening of drug discovery and ideas. Talk a little bit about this.

• 31:08Speaker 00

  Yeah, well, I know your CNN show gets into this, right? And so you probably could talk about it more than I can. But I mean,

• 31:13Kara Swisher?

  No, no.

• 31:14Speaker 00

  Certainly in the creation of Alpha Fold, this system. I mean, that in itself is an amazing story where Demis at Cambridge as an undergraduate had been told by a biologist friend. Oh, there's this conjecture in structural biology by a Nobel laureate from the 1970s, George Christian N. Finson, that if you stretched out an amino acid chain and you looked at the DNA sequence on the chain, the sequence tells you how that chain will fold itself up like a self-executing origami model into an intricate, beautiful shape, which is a protein. And we have proteins in our bodies and there are proteins in plants that are the basic building blocks of nature.

• 31:59Speaker 00

  And you can tell this intricate structure just by looking at the code. And so it became a sort of grand challenge in biology. Who can create the computational system that will just look at the code and then guess the shape? And from the 1990s, there were teams in different universities competing to do this. And DeepMind decides, after winning Go, defeating the Go champion in 2016, well, that was a

• 32:25Kara Swisher?

  That's a game.

• 32:26Speaker 00

  The game of Go. Yeah, exactly. That was just a game. But then, having solved that problem, which involved this massive combinatorial complexity, because Go has all these different permutations.

• 32:36Kara Swisher?

  They do, but boring in comparison.

• 32:37Speaker 00

  Boring in comparison. So why don't we move on?

• 32:39Kara Swisher?

  Checkers.

• 32:40Speaker 00

  And leap all the way to predicting the combinatorial complexity.

• 32:45Speaker 00

  Of these possible shapes, you know, one strand of amino acid could fold itself up in billions upon billions of different shapes. So it was in that sense, it was similar to Go. And having solved Go, they felt that their computer science skills had reached a point where maybe protein folding was crackable. And so they set out to do this in 2016. By 2018, they'd beaten all the universities, they had the best model. But it wasn't good enough that a pharmaceutical research team could simply use the prediction from computation to create a drug. So then the question was, should we push on and try to really make it that accurate? And the view of the leader of the team within DeepMind was, forget it, boss, we can't do

• 33:30Speaker 00

  this, this is impossible, don't send us down a blind alley, we should just declare victory, we're the best team in the world, and then we should move on. And Demis was like, no, I want you to actually solve the problem, not just be the best. And so they had a bit of a fight about this, and Demis said to me, well, I was being unreasonable, but I wanted to be reasonable in my unreasonableness. So I listened in to the team's discussions for a while. And when I heard they had lots of ideas that were just flowing naturally, which they hadn't yet tested in the lab or in the computational lab, I figured, well, there's more stuff they could try. And so then he pushed them, he switched out the team leader, put a new person in charge. And then in 2020, they succeeded and they did predict the shapes of proteins.

• 34:14Speaker 00

  And then the badass move was to open them source it, just like, you know, for free, any scientist in the world can effectively do a Google search and get the shape of the protein that they want to work on.

• 34:42Speaker 04

  Support for this show comes from Indeed. When the pressure's on and you need to hire the right person for the job, Indeed Sponsored Jobs has got your back. Instead of forcing you to spend tons of time searching, Indeed Sponsored Jobs matches you with quality candidates fast. According to their data, sponsored jobs posted directly on Indeed are 95% more likely to report a hire than non-sponsored jobs. Join the 3.3 million employers worldwide that use Indeed to connect with quality talent that fits their needs. Spend less time searching and more time actually interviewing candidates who check all your boxes. Less stress, less time, more results. When you need the right person to cut through the chaos, this is the job for Indeed Sponsored Jobs.

• 35:23Speaker 04

  And listeners to the show will get a $75 sponsored job credit to help get your job the premium status it deserves at indeed.com/podcast. Just go to indeed.com/podcast right now and support our show by saying you heard about Indeed on this podcast. That's indeed.com/podcast. Terms and conditions apply. Need the right hire fast? Then this is the job for Indeed Sponsored Jobs.

• 35:53Kara Swisher?

  I'm Kara Swisher, and I like people who say the quiet part out loud. In fact, I love it. That's Tiger Sisters, a new podcast that recently joined the Vox Media Podcast Network. Cherie Brooke Lu and Jean Lu are real sisters who built their careers at LinkedIn and Goldman Sachs. On their weekly podcast, they're giving the next generation practical tips on how to get ahead, how to write emails people actually answer, how to make your LinkedIn profile stand out, and how to keep learning as an adult. It's straight talk about money, career, and relationships every Monday. Follow Tiger Sisters wherever you get your podcasts.

• 36:30Kara Swisher?

  One of the things that's interesting, the parallel you make in the book and is frequently made about the development of AI is to J. Robert Oppenheimer, who we discussed earlier in the creation of the atomic bomb. It's one that Demis and others in the field welcome, but Oppenheimer is also, not just because I saw the Chris Nolan movie, I read a lot about Oppenheimer, he saw the bomb as an evil thing. It kicked off the nuclear arms race. He became a scientific exile during the Red Scare. How does he feel, Demis feel about this? Because right now, as you note correctly, there's a huge backlash to not just data centers in the US, but AI in general. The polling with the public is, you know, only Trump has worse polls at this point. But AI is pretty disliked. So talk a little bit about that.

• 37:12Speaker 00

  Yeah, I mean, I think he feels slightly that that's why you have to talk up the optimistic side of the AI story. I think he's a little frustrated with Dario, frankly, for going on TV and saying, within five years, 50% of entry-level jobs will be gone. And to propose, to just kind of say that kind of thing without proposing the policy remedy, Demis sort of points slightly a finger at Dario on that. But I think, you know, I'm not sure I agree with him, frankly. I think, you know, Dario has a good case to say, hey, we've got to call it like we see it. And that will wake people up and politically, maybe there'll be a stronger policy response if we do that. So I'm not sure Demis has any particular solution to this, you know, popular fury.

• 37:57Kara Swisher?

  Does he see himself as an Oppenheimer? I mean, Oppenheimer went through a pretty rough period before we liked him again, right? Or when he got finally feted at the end of his life. But there was an exile happening.

• 38:09Speaker 00

  There was exile happening. You know, I think Demis embraces the kind of scientific glory, the leading of the Manhattan Project, the heroic story of, you know, going off into the wilderness and focusing on nothing but the scientific mission. I mean, there's something about Demis which is almost, somebody called him a warrior monk. And off he goes, you know, in his mind's eye to the desert in isolation. And that just is a super appealing. Self-image for him.

• 38:37Kara Swisher?

  Yeah, nothing narcissistic about that, but go ahead.

• 38:39Speaker 00

  That's the Oppenheimer bit that he wants to identify with. He doesn't identify or doesn't talk about, you know, the idea that he would be ostracized. And, you know, there would be kind of a political.

• 38:52Kara Swisher?

  Dario has taken the flak for that at this point, right?

• 38:55Speaker 00

  Yeah, yeah. And I think there's an interesting debate about whether, you know, if you look at Dario's principal stance on the Pentagon using AI for weapons. It was good in terms of raising public awareness of the issue. It didn't change the outcome. So if you're Demis and you look at that, you could say to yourself, well, it didn't change the outcome. So what was the point of that? I'm much smarter. I'm going to go do this behind closed doors. I will talk to politicians as I did with Rishi Sunak and suggest things. They can take the credit. You know, what's the point of going public if you don't change how the world works?

• 39:30Kara Swisher?

  Absolutely, except he didn't change how the world works either, but that's another issue. So we're in the middle of an explosion of growth around AI. And you write, the estimates of achieving human level AGI, again, much disputed by 2030 now appear slightly conservative. Talk about how Demis looks at this because many, I get a different answer, different people, and I don't think they even, I think they're just guessing. I feel like at this point, they're just making it up. But talk a little bit about, you get very different numbers from different people.

• 40:00Speaker 00

  You do get different numbers. I mean, if you talk to people at Anthropic, they're really saying 2028. And what they say is that by 2028, there'll be recursive self-improvement, meaning that the models will be coding the next model. And once they've done coding the next model, the new model will recode the next one after that. And so you'll get this vertical acceleration in the capacity of the systems. At that point, you're done.

• 40:24Kara Swisher?

  So they go from dolphins to humans.

• 40:26Speaker 00

  At that point, you're done. The race is over. And they say 2028 in written material. I think actually privately, they even think it could be next year. So they are super near term in their prediction. Demis would, you know, love to kind of nudge his prediction beyond 2030, 2031, 32. And as I was saying, that's partly because his definition is, you know, can it be Einstein? Which is obviously the most expansive definition. Then there are people in between who say, well, it's about when most economically valuable human tasks that you could do in front of a screen could be taken over by AI. And then people might put, you know, 2030 kind of number on that. So I think that's the range.

• 41:07Kara Swisher?

  Right, right. And you just, of course, are seeing a lot of the layoffs from Meta and many others. So every episode we get a question from an outside expert. We have two for you. Here's the first one.

• 41:16Speaker 05

  Hi there. I'm Adina Friedman, the Chair and CEO of Nasdaq. Based on all of the time that you've spent with Demis, what do you see as the role of enterprise adoption and realizing the potential of AI? And if you do see it as being significant, what are the key obstacles that enterprises need to overcome in order to be able to achieve its potential within the enterprise?

• 41:37Kara Swisher?

  So I just referred to the layoffs, so go ahead.

• 41:39Speaker 00

  Yeah, so I think the revealing thing here is that the answer to the question of what does Demis have to say about enterprise adoption is zero, nothing. And that shows us the separation between the builders of the models and the real world users who are going to actually put it, you know, change the way corporations function, all that stuff.

• 42:00Kara Swisher?

  Although many of the builders are talking about it, Anthropic would be the one who seems to be making the most headway in that area.

• 42:06Speaker 00

  Yeah, I mean, I think Anthropic is very good at sort of going for enterprise tools. So coding first, cybersecurity second, and then having four deployed engineers. This is the kind of Palantir model who help corporations to adopt this stuff. So I agree with you. But certainly in a big company like Google, the people who are doing the Palantir stuff are in some other division of Google, and Demis is really just focused on building the product. And to the extent that you have this separation, and you're right, Cara, that it's not as clear in a smaller company like Anthropic, but I think the separation maybe does. Tell us something about what could go wrong here, right?

• 42:46Speaker 00

  That, you know, one set of people are just focused on building something and they're not really thinking about how it's going to be used.

• 42:54Kara Swisher?

  Yeah, and then another is actually actively thinking. And we're starting to see more companies, for example, justify layoffs by pointing to AI. Sometimes that's not the case, but they still are using it as an excuse. Cisco laid off 4,000 people earlier this month while announcing record revenue earnings. Meta laid off 8,000 and Goldman Sachs estimates AI lamented around 16,000 net jobs a month over the last year. Now, some of these, as I said, companies are using as a convenient scapegoat for too much hiring they did during COVID, which I think most people, especially the tech companies, did. Talk about mass joblessness and is that something that Demis thinks about? And if so, what's standing in the way of that? And he said that he thinks that companies who are placing developers with AI, quote, "show a lack of imagination and a lack of understanding."

• 43:39Kara Swisher?

  Is he being, again, naive about corporate incentives around tech? And some people are moving too fast forward and it's not going to work, obviously. But talk a little bit about this.

• 43:48Speaker 00

  Yeah, I think there's sort of two extremes which are wrong. You know, one extreme is to say, there's nothing to see here, there's not going to be any big jump in unemployment. We've been here with previous kinds of technologies and the labor market always adjusts. And, you know, we had the internet and actually unemployment was very low throughout that period. And I think that's naive because, you know, AI is not just another technology. It's more powerful, it's more general, it's directly competing, you know, with human cognition. And it can be scaled if you have enough data centers, you know, up the wazoo so that you replace, you know, tons of people. So I think, I think, I think we can't just draw comfort and be complacent just because of the technology history.

• 44:33Speaker 00

  On the other hand, it's also way too simple to say. Now that this system can do all this stuff, you know, humans are done. Because, you know, new jobs will emerge to some extent. I mean, you know, it's partly that there are tasks that human, humans are better at dealing with humans sometimes. So if you think about the the sales team, enterprise sales team in a big company, where the job is basically to go schmooze the humans who are your customers and forge a bond with them. I I think humans have an edge at that. So I think there there are a lot of tasks where humans will remain superior. And to the extent that the economy grows because of AI, there'll be more of those tasks. So I think we should beware of both extremes.

• 45:18Speaker 00

  I mean, one number I like.

• 45:19Kara Swisher?

  It's unclear.

• 45:20Speaker 00

  I I like to I like to cite this this statistic, which is that no technology has ever driven economic growth per capita at the frontier more than two and a half percent a year. So people who tell you that AI is going to like double the size of the economy in 15 years are smoking something. It was just not that's.

• 45:39Kara Swisher?

  Oh, abundance. You're not an abundance person, neither am I. The argument, I'm so tired of that. The argument around rushing to develop, obviously, AG, I think, has largely centered on beating China. But in many interviews with these people, it's I I call it the Xi or me argument, like, we gotta do it or China will. And I get it. But in a recent op ed, and I really appreciated it, that you wrote in the New York Times, you call for the US to negotiate a safety pact with China because we cannot beat them. And not not just falling behind, but there's certain things, as with nuclear energy, as with everything else. If we're falling behind, what incentive does China have in signing a safety pact? I think there's huge incentives for them to do so. And it's not just to keep them from being ahead of us. It's that there are commonalities here,

• 46:24Kara Swisher?

  as with, as I said, nuclear energy or nuclear weapons or cloning. There's a lot of stuff that is global. And oddly enough, I ran into Tony Blinken last night, and he said, I had I had urged him at when he first got his job to do a global pact around safety and and with China especially. And he's like, well, I didn't get that done. I'm like, no, you didn't. And then I walked away. I was rather intent on getting them to stop, you know, to pay attention to this. But talk a little bit about this, this sort of Xi or me and what happens with China, because you are correct, they are doing astonishing work here. And we had been ahead, obviously, the US or democracies. And now, I'll put all democracies, because Demos is in London.

• 47:06Kara Swisher?

  But talk a little bit about this race and also the need for a safety pack, which I think is critical.

• 47:12Speaker 00

  Yeah, yeah. So, you know, China does everything fast. So my book came out in China before it came out in the US. And I went to China in in March for the launch. And I spent eight days basically going around talking to AI leaders, both in academic labs and in industrial, you know, tech companies. And what struck me was how often they brought out the topic of safety. And this was interesting because that is not what you hear from people who put the chip export controls in place in 2022 in the Biden time, you know, so. So those people, you know, several of whom are my friends, and I've talked to them a lot about this. You know, they said to themselves, you know, credit to them, the scaling laws are real.

• 47:56Speaker 00

  AI is going to be super powerful. And they said this before ChatGPT came out. So when I do something, when I prevent the bad guys from getting this AI, but that definition of bad guy was China. And in my view, there is a more important set of bad guys who are criminals, who want to do cyber attacks, terrorists.

• 48:16Kara Swisher?

  Individuals?

• 48:17Speaker 00

  Rogue states, you know, random bad people.

• 48:20Kara Swisher?

  Rogues, just rogues, individual rogues.

• 48:22Speaker 00

  Exactly. And they missed out that whole category. And by making China the enemy and the rival and saying, you cannot get AI and we're going to deny you the chips. And so you won't have Nvidia, no Nvidia, no, no dice, you'll, you'll be left behind. You know, the mistake was to make China into the enemy and lose maybe a chance to talk to them about. What about nonproliferation of this stuff? Maybe we should just say, you're a technology superpower. We are a technology superpower. The way we're going to avoid some catastrophic war between us is the same way we did it in the Cold War with the Soviets, which was parity, mutually assured destruction. Parity is actually good for balance and stability.

• 49:03Speaker 00

  Whereas when you're talking about random rogues using this stuff for bad ends, the Cold War lesson is that's where you need the IAEA to keep track of the material and the nonproliferation treaty. And my friends, you know, say, who were in the bad administration say, well, you know, yeah, but you can't talk to the Chinese because they don't care about safety. I went there and they did talk to me about safety. They say, you can't trust the Chinese. And I go, you think it was easy to trust Khrushchev? You know, I mean, this was the Soviet leader who banged his shoe on the table at the UN and put missiles in Cuba. Not an easy guy to talk to, but in that period is when we created the nonproliferation regime for.

• 49:44Kara Swisher?

  That's right. And they have their interests of their economy being destroyed by a rogue something is just as high as ours.

• 49:50Speaker 00

  Totally.

• 49:51Kara Swisher?

  It's just really kind of, kind of. I was always like, hmm, not so sure I want either of you. You know, I'd like a global safety group that is, that includes companies, includes legislators, includes China, includes the US. And so that we have, it's a very similar to nuclear. Exactly. We, we trust Khrushchev. That's a great way to put it. I, I thought that piece was terrific.

• 50:15Speaker 00

  Thank you.

• 50:16Kara Swisher?

  Now we have a second expert question for you. This is from Kent Walker, president of global affairs at Google and Alphabet and the bad cop apparently. Let's listen.

• 50:24Speaker 06

  Hey Kara, hey Sebastian. In reading the book, I was struck by how well Sebastian you captured Demis's vision of using AI as a way of solving intelligence to then solve everything else. It's a scientific tool to help us address some of our biggest problems. The challenge is that polling in the US suggests that America is one of the least optimistic countries about AI, much less optimistic than China is. Maybe we've had too many debates between AI accelerationists and AI doomers, but how would you suggest we go at the challenge of creating a grounded optimism, recognizing the challenges, but also encouraging people to put these tools to work to benefit themselves and their everyday lives and to benefit our society? Thanks.

• 51:06Kara Swisher?

  That is a great question. One, Google, you should stop being so aggressive, but go ahead.

• 51:10Speaker 00

  Yeah, so I mean, we refer to this a bit before, but I think that medical breakthroughs that delivered AI-generated or AI-assisted drug discoveries that really save people's lives, that would be a great thing to change public opinion on whether AI is good or not. I also think that we ought to put in place preemptively, before we totally need it, things like wage insurance, retraining schemes, active labor market schemes to help people get other jobs. Do that early. Don't wait around until it's obviously, you know, essential, because at that point, public opinion will be so mad at you that, you know, it won't make any difference.

• 51:51Kara Swisher?

  There'll be riots, there'll be work riots, just as they were before.

• 51:55Speaker 00

  Yeah.

• 51:55Kara Swisher?

  In other.

• 51:56Speaker 00

  I think a useful statistic here is that in the 12 years from 1999 to 2011, the total number of job losses as a result of China renting the WTO was 2 million in the US. 2 million is nothing. 2 million is like the amount of labor market chen you get in an average month in the United States. And yet, you know, the backlash against China, the perception of the China shock, the fury about globalization, politically was absolutely massive. So imagine if you got an AI shock that was way, way bigger than China. The reaction will be, you know, as you say, it'll be people in the streets. You know, I mean, with the Industrial Revolution,

• 52:40Speaker 00

  what happened was you got revolution politically, you got Marxism, you got a whole lot of turmoil. And that was a revolution which, you know, took place probably one tenth the speed of the AI revolution and may arguably have been smaller.

• 52:55Kara Swisher?

  Absolutely. I think they don't even understand the anger, the rage that is building. And it's manifesting itself. I think a lot of the AI anger is a manifestation of affordability and nervousness about the future. And now it has become that. It has become that.

• 53:11Speaker 00

  I don't know what you think, but I think there's some merit to this idea of Trump accounts, where you would put shares in AI companies or other stakes and distribute that democratically to all young Americans.

• 53:25Kara Swisher?

  Sounds good. No, he doesn't seem interested in doing, he does them, but he does other things more. The stuff that he does, it's populist. I'm not against, like, the Trump accounts or the, I don't like him calling them Trump accounts or Trump RX or whatever. He has to put his fucking head on everything. But, you know, but that idea is correct. You know, UBS, some version of this or, or more, like, where are the jobs and do some really serious government studies and where we work with companies and, you know, this recent AI advisory board has nobody on it except for business people. There's no way they're going to come to any conclusion but their own. And they could use critics. They could use someone like you, someone like me, someone like an academic, someone like, you know,

• 54:06Kara Swisher?

  they're just refuse to have wide-ranging points of view here. That is probably going to be the biggest problem, I think.

• 54:13Speaker 00

  I think it's also relevant to Ken's question that, you know, the record in the last couple of years, well, during the Trump period, is that all the energy on legislation has come from the states. And the AI companies, the tech companies, have lobbied actively to stop that stuff from passing. And they have, they have a decent argument in that a patchwork of state-level stuff would be way less effective than a federal thing. But then, that's why you need federal intervention, right?

• 54:41Kara Swisher?

  Which they are also trying to stop, which is why they're sitting in the front row of the Trump inauguration.

• 54:46Speaker 00

  Yeah, I don't quite, I think that's a bit tough. I know you like to be tough. But I think that in the Ben Buchanan period, when Ben Buchanan was the czar in the White House, what he said on the record publicly is that whenever he talked to the labs, they were willing to support what he was trying to do in terms of setting it up an AI safety institute, in terms of then requiring labs to share their models with the safety institute before they were released. You know, clearly, that regulation should have gone further. But he says that actually the labs were encouraging him. To go down that path of regulating. And I think I don't have any reason to disbelieve that. So, you know, I think.

• 55:30Kara Swisher?

  I disbelieve.

• 55:31Speaker 00

  You disbelieve.

• 55:31Kara Swisher?

  But look at what the Trump administration just is starting to announce now, the same thing, the exact that they rejected, correct? Now, they do that with a lot of stuff, but their most recent noises have been exactly what that Biden executive order said.

• 55:46Speaker 00

  Yeah, I agree. So, so Trump.

• 55:47Kara Swisher?

  Which is crazy. It's like, they're like, this is our new idea. And I'm like, well, that was two years ago, three years ago, Bletchley Park.

• 55:54Speaker 00

  Hundred percent, completely agree with that. But that's a description of the Trump administration's U-turn. I think the position of the AI labs, which is what we were talking about earlier, I think they actually have been open to sensible federal action. They just hated the state action.

• 56:10Kara Swisher?

  Well, they're getting the state action because initially they lobbied against any kind of tech regulation, right? And so they find them, they should, they should be supporting it. They shouldn't have kneecapped Amy Klobuchar, for example, or they've been kneecapping politicians for a long time. And I think people don't trust them. I think that's where it is. And then when you have any manner of them swanning around, looking like Daddy Warbox everywhere across the world, it's not great. It's not, you know, Elon hasn't helped. Jeff Bezos certainly hasn't. And neither has Mar. I mean, I think the imagery, you don't have Demis is there, or, you know, Dario is a hero. And like, he's fine, but like, that's it. Like, that's the problem. There's no heroes. There's a lot of people who look like they're in it for the money.

• 56:50Speaker 00

  So what is your explanation? I'd love to get your view on this. So why is it that Demis is not particularly famous in the US? I mean, I've been.

• 57:00Kara Swisher?

  He doesn't want to be.

• 57:01Speaker 00

  Yeah.

• 57:01Kara Swisher?

  He doesn't want to be.

• 57:02Speaker 00

  Yeah.

• 57:03Kara Swisher?

  He could be. He could be all over the place. He could be Dario emoting everything because he deserves that spot. I, you know, I think he's a true wonk, probably. And so he's, the science is everything. But, you know, history is littered with people who sciences weren't as savvy. Nikola Tesla wasn't as savvy as Edison. Edison was a PR person. He did all many of nefarious tricks against Tesla. Like, you know, I don't think he's that much of a victim, the way Tesla certainly was, if you read a lot about what. Happened there, but I think he likes to be pure. He likes to have that image of himself. That's my, I don't know him very well, but that's what I can see, but he certainly deserves it. He certainly deserves the attention. You know, Jeffrey Hinton likes to take up space, that's for sure, but you could see him out there more.

• 57:48Kara Swisher?

  But no, here he is with your book. So obviously he's talking to you. So last question, today's AI race is exactly the scenario he had hoped to avoid. But in an excerpt in The Atlantic, you wrote that Demis has, quote, "come to see salvation paradoxically in his own career advancement and securing personal influence." We have concentrated so much power in the hands of just a few people. After years of talking to many of them for the book, what do you make of his conclusion? And what does it mean for the rest of us who are worried about concentrated power? I know I am, of anyone, even if they were the angels. I don't feel like they should have this many, small amount of individuals to have this much power over something so important. Of course, it's happened in history, but it's never ended up well.

• 58:31Kara Swisher?

  And this is quantically more powerful than any, like railroads, trains, the telegraph, stuff like that. Are you worried and should the rest of us be worried about this concentration of power? Even if it's in the hands of people that are better than others, but a lot of them aren't.

• 58:47Speaker 00

  Yeah, yeah, yeah. Yeah, look, I think in a way the story of Demis Sabes is, you know, of a decent person, a good person, who wants to be good, but can't actually have a good impact. Because he's inside this race dynamic, inside a big corporation. And whatever he may personally think about safety, there are these larger forces that are driving him on. And if he were to quit tomorrow and take a professorship in theoretical physics at Princeton, it wouldn't change the race, it wouldn't make the world better, it wouldn't do anything very constructive. I'm not sure I want him to do that because, you know, it would just be a gesture. And I prefer to focus on the outcomes. So what this points to is that we do need government intervention.

• 59:31Speaker 00

  You know, the only way to counter the power of this handful of individuals who are kind of collectively very powerful, but individually impotent in terms of the big social questions. You know, you need government to take a serious intervention. And, you know, why not? We have drugs in the United States which are approved by a regulator. And if they're dangerous, they're not allowed to be released, right? Why wouldn't AI model be released without being tested by the government first? Why would we allow, you know, open weight models that can be ripped off by anybody, abused by anybody, used for a cyber attack? And you wouldn't be able to kill it with a kill switch? It's crazy, right?

• 1:00:15Speaker 00

  So I, I, I think what we need is far more active government action, precisely because AI needs to be governed.

• 1:00:22Kara Swisher?

  Right. So where does that leave the rest of us if AI goes the way that social media did? And the government fails to act? I mean, obviously, different governments are doing different things, but there's no coordinated global effort at all. But, and in this country, forget it so far. And of course, we did nothing to, to slow down the relentless and often poisonous and toxic pace of social media. So where does, what has to happen? What do you see happening? What does Demis see happening?

• 1:00:49Speaker 00

  I think, I think he's got to the point where he's trying a little bit to, you know, do politicking behind the scenes and suggest that, you know, more action is needed and it has to be collective and the government needs to lead that. But whether he's putting real energy into it, I can't tell you for sure.

• 1:01:08Speaker 00

  And so I think he's just, you know, that history we talked about before of Project Mario and trying to get safety oversight. It's almost like he's burnt out on trying to solve the governance problem. And so he's just heads down on the technical problems, which is a regrettable place to end up. But, you know, all I can say is that, you know, I mean, that's the most optimistic thing that's happened in the last few months was when Anthropic released Methos and the Trump administration freaked out. And they called in the heads of the banks and said, you guys are going to have hacks that empty your bank accounts unless you can harden your cyber defenses, because this stuff is for real. And they basically did a 180. And now, you know what? The Anthropic had this list of 40 kind of responsible tech companies

• 1:01:53Speaker 00

  that were going to get the model first before anybody else got it. And then they were going to roll it out to others. Those subsequent waves have been frozen because the whole decision making around the release of the models has been requisitioned by the Trump administration and they're sitting on it and they're saying, nobody's allowed to have this. And so they've gone from being laissez faire to maximally interventionist. It shows you how freaked out they are and maybe that's a hopeful sign for action on regulation in the future.

• 1:02:22Kara Swisher?

  Unless they're trying to do it to make more money, who knows?

• 1:02:24Speaker 00

  Yeah, possible.

• 1:02:25Kara Swisher?

  How can we take all those bank accounts and get them for ourselves?

• 1:02:30Speaker 00

  Kara, Kara, I was trying to do this sort of optimistic, you know, ending riff.

• 1:02:34Kara Swisher?

  I understand, Sebastian.

• 1:02:36Speaker 00

  And you're like, you just burst my balloon.

• 1:02:37Kara Swisher?

  Oh, I don't know. You know, intelligence has its limitations, but stupidity and greed are infinite. So that's my feeling. I'm sorry. But I think the moral of the story is that tech just needs to be scary enough to force the government to act. That's really not the way. That's not the government I want, but fine, I'll take it in some ways. Anyway, this is a marvelous book. I really appreciate it and thank you so much for your time.

• 1:03:00Speaker 00

  Of course. Thank you. I really enjoyed it.

• 1:03:07Kara Swisher?

  Today's show was produced by Christian Castor-Wassell, Michelle Aloy, Catherine Millsop, Megan Burney, and Kaelin Lynch. Nishat Gurwad is Vox Media's executive producer of podcasts. Special thanks to Bradley Sylvester, Madeline LaPlante-Duby, and Julia Sharp Levine. And thanks again to American University's Cogod School of Business for hosting this event. Our engineers are Fernando Aruda and Rick Kwan, and our theme music is by Track Academics. If you're already following the show, you win a Nobel Prize. If not, you're just in it for the money. Go wherever you listen to podcasts, search for On with Kara Swisher and hit follow. Thanks for listening to On with Kara Swisher from Podium Media, New York Magazine, the Vox Media Podcast Network, and us. We'll be back on Monday with more.