Pod Insights On with Kara Swisher 'Godfather of AI' Geoffrey Hinton Rings the Warning Bells About this episode Nobel laureate Geoffrey Hinton, known as one of the “godfathers of AI” for his pioneering work in deep learning and neural networks, joins Kara to discuss the technology he helped create — and how to mitigate the existential risks it poses.
Hinton explains both the short- and long-term dangers he sees in the rapid rise of artificial intelligence, from its potential to undermine democracy to the existential threat of machines surpassing human intelligence. He offers a thoughtful, complex perspective on how to craft national and international policies to keep AI in check and weighs in on whether the AI bubble is about to burst. Plus: why your mom might be the best model for creating a safe AI.
Questions? Comments? Email us at [email protected] or find us on YouTube, Instagram, TikTok, Threads, and Bluesky @onwithkaraswisher.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
Transcript 10,869 words · 8 speakers
Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Speaker 00 I got introduced recently in Las Vegas as the godfather, which I liked.
0:04 Kara Swisher? But you didn't whack anyone, right?
0:17 Kara Swisher? Hi, everyone. From New York Magazine and the Vox Media Podcast Network, this is On with Kara Swisher, and I'm Kara Swisher. I've been talking all year to people about the impact of artificial intelligence on society. Some are optimistic, zoomers. Others are deeply concerned, doomers. And many are in between, gloomers and bloomers. My guest today is someone who has been ringing the warning bells but still thinks there's time to fix things, one of the godfathers of AI, Nobel laureate Geoffrey Hinton. Hinton is professor emeritus at the University of Toronto, where he and his computer science colleagues worked on machine learning using artificial neural networks. He was the first to train networks using deep learning, which is the basis for today's artificial intelligence.
0:59 Kara Swisher? In 2012, he and his students Alex Krizewski and Ilya Sutskever made a breakthrough in image recognition with AlexNet. In 2013, Google bought their startup, DNN Research, to boost its photo search and kept Hinton on to run it. And Ilya went on to co-found OpenAI. Hinton worked for Google for a decade until he left abruptly two years ago and began speaking out on the risks of AI. And he's an incredibly thoughtful person when it comes to this issue. He worked on this his whole life, so he doesn't hate it the way some in tech have posited and especially insult him about. And I'm really offended by that personally. So I wanted to talk to Hinton about the short and long-term risks
1:42 Kara Swisher? he sees in the technology that he helped develop, how to create national and international policies that will keep AI under control, and whether the market growth built on the current AI models is a bubble about to burst, and of course, what we can do to turn things around. We have two expert questions today because Dr. Hinton is so smart, from Alex Stamos, CSO of Corridor and a lecturer in computer science at Stanford University, and also from Jay Edelson, the lawyer representing the Raine family in their lawsuit against OpenAI. Whether you're a doomer, zoomer, gloomer, or bloomer, stick around.
2:33 Kara Swisher? Support for this show comes from The Guardian and their new show, Stateside, where journalists Kai Wright and Carter Sherman use the entire independent reporting resources of The Guardian to slow down the news and wrestle with the questions we all have about what is actually happening in the world. Join Kai and Carter three times a week as they utilize all the reporting resources The Guardian has across news, international coverage, climate, culture, wellness, and more. And The Guardian's not owned by a billionaire, they fearlessly report the facts without interference. Go to theguardian.com/stateside to learn more. And listen wherever you get your podcasts or watch on YouTube. That's theguardian.com/stateside.
3:17 Kara Swisher? Support for this show comes from Odoo. Running a business shouldn't feel like surviving a software group project. One app for accounting, another for inventory, another for sales, and somehow, none of them ever talk to each other. That's where Odoo comes in. An all-in-one business management software that brings every part of your business together. From sales and accounting to inventory and marketing, all in one powerful platform. No messy integrations, no bouncing between tabs, and best of all, no spreadsheets. Stop managing software and start managing your business with one unified system. Try it today for free at Odoo.com/kara. That's O-D-O-O dot com slash Kara.
4:01 Speaker 03 Now for a quick message from Anthropic, the team behind Claude, the AI built for businesses. Customers count on their bank, and banks count on their most important systems. Now those systems count on Claude, and nine of the Fortune 10 companies use it for critical work. See why leading companies trust Claude at claude.ai/span.
4:28 Kara Swisher? Jeff, muchas gracias por venir. Soy una admiradora de mucho tiempo. Sé a todos los que conoces, creo. Pero no creo que nos hayamos conocido, ¿verdad?
4:36 Speaker 00 No creo que nos hayamos conocido en persona, no.
4:38 Kara Swisher? En cualquier caso, ganaste el premio Nobel en Física el año pasado junto con John Hopfield por tu trabajo groundbreaking en la aprendizaje de máquinas usando redes neuronales artificiales. Pero en lugar de revelar en el momento, usaste tu discurso Nobel para advertir sobre los avances rápidos, supongo, como un momento de Oppenheimer, supongo. No sé cómo lo describirías.
4:58 Speaker 00 Bueno, no fue exactamente un momento de Oppenheimer. Oppenheimer realmente fue un científico brillante y un organizador brillante. Solo soy un científico bastante bueno que tomó la decisión correcta hace 55 años y se quedó con ella. Pero no soy un Oppenheimer. Y él estaba construyendo algo que solo podía ser usado para fines malos. Se justificó porque tenían que llegar allí antes de los nazis y no sabían dónde estaban los nazis. Pero Oppenheimer en realidad trató de detenerlos en construir la bomba H. Así que hizo lo que pudo después.
5:33 Kara Swisher? Claro.
5:33 Speaker 00 Pero la IA es muy diferente de las armas nucleares porque tiene un gran upside así como un gran downside. Y así, podrías pensar, bueno, ya sabes, si podría deshacerse de la humanidad y si podría causar todos estos otros problemas, ¿por qué no dejamos de hacerlo? No vamos a dejarlo porque tiene un gran upside. Así que esa es una gran diferencia de las armas nucleares.
5:54 Kara Swisher? Quiero que la gente entienda que la investigación de inteligencia artificial como esta, como has dicho, 55 años, no es algo nuevo. Explícanle por qué la gente se ha centrado en ello. ¿Cuándo se fue de la tu perspectiva?
6:07 Speaker 00 Bien, entonces la gente se está centrando en ello ahora porque realmente está funcionando muy bien. Durante mucho tiempo, la idea de que podrías tener algo como un chatbot donde podrías preguntarle cualquier pregunta, en casi cualquier lengua, y te daría una respuesta a nivel de un no muy buen experto, esa idea parecía ridícula o ridiculmente lejos en el futuro. Incluso parecía muy lejos en el futuro hace 15 años. En como en 2010, si dijeras a la gente, "Vamos a tener eso en 15 años", ellos habrían dicho, "¡Estás loco!" Incluso yo habría dicho, "¡Estás loco!" Y soy un gran entusiasta de ello.
6:43 Kara Swisher? Correcto.
6:43 Speaker 00 Así que una cosa que pasó fue... It mastered natural language much faster than anybody expected. It could understand what you meant in whatever way you said it. The other thing that happened was that I sort of realised the full import of the fact that it's much better at sharing than we are. So if you have multiple copies of exactly the same neural net running on different hardware, they can each look at a different bit of data, figure out how they'd like to change their connection strengths to absorb that information in that data. And then they can all share how they would like to change and just change by the average of what everybody wants.
7:22 Kara Swisher? Right.
7:23 Speaker 00 And now every neural net has benefited from the experience of all of the neural nets.
7:27 Kara Swisher? Yeah, I had said that to people about electric, I mean autonomous vehicles. I said, when a human gets in an accident, nobody learns. And often the human doesn't learn, but the cars all are going to learn that particular
7:39 Speaker 00 Exactly.
7:39 Kara Swisher? problem.
7:40 Speaker 00 Exactly.
7:40 Kara Swisher? Instantly, which creates incredible power.
7:43 Speaker 00 Yes. And you can only do that if you're digital, because all of these neural nets have to be using their weights in exactly the same way. Analog would be much lower power, but it won't give you that ability to share. And the ability to share is going to come even more important when we have agents that are operating in the real world, so you can't speed them up. At present, if you're just training on recognising images, you can feed the images through very fast, so maybe you could feed a lot of images through one neural net, but one copy of a neural net. But if your agents operating in the real world with real world time constraints, interacting with other agents, then the fact you can have a whole bunch of agents sharing what they learn very efficiently is a huge advantage.
8:24 Kara Swisher? So when you started doing this, obviously a fascinating question and something that's really challenging and interesting to do, was there a moment when you thought, oh no, or did you anticipate from the beginning possible problems? Because that's one of the things when I started covering tech, I kept anticipating the problems, which caused tech people to call me a bummer.
8:44 Speaker 00 I think a lot of the tech people, including me, thought, yeah, we're going to get to superintelligence, but it's going to be a long time. If you look at Turing's paper in the early 1950s.
8:56 Speaker 00 He has a sort of one sentence throwaway in something like it'll soon outpace our feeble intelligence. He doesn't discuss it any further. It's just obvious to him that it's going to outsmart us.
9:06 Kara Swisher? Right. When did it come for you?
9:07 Speaker 00 So with the chatbots developed at Google, because that's where the really good ones developed first, particularly the one called Palm that could say why a joke was funny. That had always been my criterion for is it getting so it can really understand?
9:22 Kara Swisher? A joke.
9:23 Speaker 00 A joke, right. And so the linguists are all saying this is just statistical autocomplete. I think that's complete nonsense. It really does understand what you're saying. So that was one ingredient, and that happened in the early 2020s and was emphasized a lot, of course, when ChatGPT came out. The other ingredient was this realization that they're better at sharing. And that was really hammered home to me by attempts I did while I was at Google to figure out if there was a way to make these LLMs analog, so they use far less power. And that really brought home to me the big advantage of being digital. You use a lot of power, but you can have different copies of the same model, looking at different data and sharing what they learn.
10:05 Speaker 00 And that, I suddenly had a realization, look, that's hugely important. It makes it a better form of intelligence.
10:11 Kara Swisher? Right. So in your Nobel Prize acceptance speech, you cited the risk of AI being used to create divisive echo chambers for government mass surveillance, to launch cyber attacks, new viruses, or develop lethal weapons. These are risks stemming from people using AI maliciously.
10:26 Speaker 00 Yes.
10:26 Kara Swisher? Tell me how you decided to do this, given, you know, you have huge regard among computer scientists and people obviously call you the godfather of AI and everything else. Give me some examples of how it could play out and why you decided to talk about this first and foremost.
10:43 Speaker 00 Okay, so what I really decided to talk about when I left Google in April of 2023 was the existential threat of these things becoming smarter than us and taking over. And I decided I should talk publicly about that because many people at that time were saying, this is just science fiction. This is nonsense. These things are just stochastic pirates. There's nothing to worry about.
11:05 Kara Swisher? Right.
11:07 Speaker 00 I wanted to use my reputation to explain to people, yes, there is something to worry about. They're not just stochastic parents. They really do understand what you're saying, and they really are going to get smarter than us. Um, so we should really worry about it. Um, now, as soon as you start talking to journalists about that, they ask you about all the other things.
11:26 Kara Swisher? Right.
11:26 Speaker 00 Because they muddle all the various things together. Some journalists.
11:29 Kara Swisher? And they've seen all the movies.
11:30 Speaker 00 And so I had to sort of have things to say about all the other things. And in the end, I became an advocate for worrying about all these other things, too. And they're more urgent. So corrupting democracy, for example, seems very urgent.
11:43 Kara Swisher? You can go from one crisis to the next, correct? For example, new viruses or to develop lethal weapons. It's like a menu of possibilities, correct?
11:52 Speaker 00 Yeah, but I think it's very important not to just muddle them all together. For example, let's take autonomous lethal weapons and creating new viruses. Creating new viruses, there will be collaboration between governments on how to prevent that, because no government really wants new viruses created. The idea, for example, the Chinese deliberately created Covid is crazy. Um, so there will be collaboration there because governments' interests are aligned. They all want to stop terrorists from releasing nasty viruses, so they'll collaborate there. If you look at autonomous lethal weapons, there's not a chance in hell they'll collaborate, because they want to use them against each other.
12:31 Kara Swisher? Correct, yeah.
12:32 Speaker 00 You can't imagine the Ukrainians and the Russians collaborating on autonomous lethal weapons.
12:36 Kara Swisher? Yes, let's all stop. Let's all stop. Yeah, because they would have stopped, right?
12:40 Speaker 00 And I used to think we should just stop it, forbid it. Um, I've talked quite a bit to Eric Schmidt, with whom I disagree on most political things.
12:50 Kara Swisher? Mm-hmm.
12:51 Speaker 00 He has been helping Ukraine commercialize the production of drones, um, make it efficient.
13:00 Speaker 00 And it's hard to be against that. The Russians aren't going to stop using drones.
13:04 Kara Swisher? Right.
13:05 Speaker 00 And it's, it is modern warfare now.
13:08 Kara Swisher? Well, he is real politic, that's Eric, right?
13:10 Speaker 00 This is, yes, this is real politic. He thinks, he thinks Kissinger's a good guy, I think Kissinger's a bad guy. But we both think Kissinger was pretty smart. And I think we should do what we can to ban autonomous lethal weapons, but it's not such a clear case as it used to be.
13:25 Kara Swisher? Right, absolutely. And what about, one of the things you also talked about was mass surveillance, the idea, and you've talked about the danger of election interference and obviously early this year, Elon Musk's Doge team was able to consolidate a lot of data about Americans here in the US. And I kept saying, focus on that, focus on, forget about his chainsaw, forget about all his manner of weirdness.
13:47 Speaker 00 A lot of the rest was just a disguise.
13:49 Kara Swisher? Correct. I kept saying that, I'm like, he's collecting the data. I'm not as smart as you, but the scenarios running through my mind were rather vast. What were the scenarios running through yours when you saw him creating this sort of war room of data, which had never been brought together the way he was attempting it, which was, if you are an evil genius, that's what you would do, like, that's the first move, I suppose.
14:10 Speaker 00 Don't think there's much if about it.
14:12 Kara Swisher? Yeah.
14:14 Speaker 00 So, it seems to me there's two main uses you could use for it. One is targeted advertisements before elections, so you can swing elections that way. The other is being able to sell advertisements to people on your, on Twitter, for example, or the thing that was formerly called Twitter. The more you know about people, the easier it is to figure out which advertisements they're going to click on, and clicks is money, so.
14:52 Speaker 00 Uh, that's another obvious use of it. There's probably lots of other uses too.
14:56 Kara Swisher? And did you think that's precisely what he was doing in order to manipulate? Was that your first? That was my first thought, is he wants to manipulate elections?
15:03 Speaker 00 My guess was he probably wanted to do it to be able to sell advertisements and also um to manipulate elections. This is all just fantasy, just speculation. I've got no direct evidence for it. It's just common sense. Um, his interests were aligned with Trump's interests. Trump wanted the data to manipulate the midterms. He wanted the data for other reasons, probably, but also maybe to manipulate the midterms. Um, yeah, that's my guess.
15:32 Kara Swisher? That's your guess.
15:32 Speaker 00 I should emphasize, I'm not an expert on any of this stuff.
15:35 Kara Swisher? So the most immediate risk of AI, obviously, that's been talked about is the potential for mass unemployment. Researchers at Stanford are calling entry-level and early-career workers the most AI-exposed fields, canaries in the coal mine. Jobs for that group are down about thirteen percent. You said AI will make a fewer people much richer and most people poorer. Why don't you buy arguments, which of course they all make, that new job, like as in the past, whether it's manufacturing or farming or whatever, new jobs will replace old ones, just like it happened before?
16:06 Speaker 00 So one comment, which you'll probably heard before, um, using the past to predict the future is like driving very fast down the freeway while looking through the rear-view window. So the past isn't always a good predictor of the future, particularly when you get a huge change.
16:24 Kara Swisher? Mm-hmm.
16:25 Speaker 00 And what we're seeing, most people are agreed, is a huge change, because for the first time, we're going to get things that can replace mundane intellectual labor. We've never had that before. When we got things that could replace mundane physical labor, like digging ditches, um, there was something else for people to do. Um, but now, what are the people in call centers who are going to get displaced by an AI that's more patient and more knowledgeable and much cheaper than them, what are they going to do? I don't think AI is going to create a lot of new jobs. It will create new jobs, but not as many as it displaces. Now, some economists who I respect disagree with me. But I think the general consensus is that it will replace a whole lot of jobs.
17:10 Speaker 00 And I think that's one of the reasons why the companies are pumping so much money in. If you ask, where do they expect to get back these tens or hundreds of billions of dollars they're pumping in?
17:22 Kara Swisher? Right.
17:22 Speaker 00 Maybe, maybe they're going to pump in something like a trillion dollars in new data centers. Where are they getting the money back from? Well, there's obviously subscription fees. And they can charge quite a lot for a nice assistant. There's advertising. But the third element of it is if they can sell you something that will allow you to replace a lot of expensive workers with a lot of cheap AIs. That's worth a lot.
17:50 Kara Swisher? Correct.
17:51 Speaker 00 And I think that's part of the calculation. It's a shame more of them haven't read Keynes. Because they realize that if they get rid of all those workers and don't pay them anything, there's nobody to buy their products.
18:03 Kara Swisher? Right, that's correct, that's correct. Because you don't have other jobs. But this is not a group of people that cares about consequences very much already.
18:12 Kara Swisher? We'll be back in a minute.
18:26 Kara Swisher? Support for this show comes from The Guardian. Trying to locate news sources that reliably separate fact from fiction can seem like looking for a needle in a haystack these days. Social media feeds, cable news and subscription based news outlets might be able to give you one version of the news, but not in the way that you need. You want news fast, but slow enough to actually absorb the information. Stateside with Kai Wright and Carter Sherman is here to do just that. The podcast tries to slow down the news and wrestle with the questions regarding the world at large at your own pace. Three times a week, hosts Kai Wright and Carter Sherman bring their expertise together to showcase different and engaging perspectives regarding the world. Today, they will use their collective knowledge to discuss the news, international
19:09 Kara Swisher? affairs, climate, culture, sports, lifestyle, fashion and wellness. Plus, The Guardian wants to emphasize they're not billionaire owned, meaning they're free to report the whole picture without interference. Go to theguardian.com/stateside to learn more and listen wherever you get your podcasts or watch on YouTube starting May 13.
19:29 Speaker 04 Ripple has been powering modern finance for a decade. We're leading the charge to bridge the gap between existing financial systems and the future of finance. You can have it all: legacy, stability and digital innovation. Systems that work with your other systems. Because with the right partner, the question isn't crypto or traditional finance. The question is simply, why choose? A new standard of modern finance. It's happening with Ripple. For informational purposes only, availability of products can vary and may be provided by different Ripple entities.
20:00 Kara Swisher? Support for this show comes from Hers. Losing weight on your own can often feel like a fight you never win. Luckily, weight loss by Hers offers access to the FDA-approved Wegovy pill and the Wegovy pen. Wegovy is the first-ever GLP-1 weight loss medication available in a pill, but also comes as a pen, so you've got options. With Wegovy, you can lose up to 20% of your body weight. People lost an average of 14% body weight at 64 weeks in a clinical trial of Wegovy alongside diet and exercise. To get started with Wegovy, just go to forhers.com and complete an online consultation with a licensed provider who will determine if your prescription is appropriate. And Hers doesn't stop there. Nutrition and lifestyle guidance are part of the plan, along with access to 24/7 messaging
20:45 Kara Swisher? with your care team, so support is always in reach. Ready to reach your goals? Visit forhers.com/cara to access personalized, affordable care for weight loss, hair, menopause and more. That's forhers.com/cara. Individual results may vary and stopping treatment may result in weight regain.
21:07 Kara Swisher? I've spent a lot of time interviewing parents who are suing companies like Google and OpenAI. For example, one of them says their son was coached into suicide by ChatGPT. Obviously, Character AI had a similar situation. And it was really pretty insidious when you started to look at the discussions, the chatbot discussions with the kids.
21:26 Speaker 00 Yes, I've looked at some of those.
21:28 Kara Swisher? It's disturbing. This AI is speaking to this kid. It's certainly a synthetic being, but it still is able to have a discussion with them. And I recently uploaded all my stuff and created Cara AI, and it was learning by the second. I was sort of shocked by how good it was. And it's still crude, for example. But OpenAI put more parental controls in their product in the aftermath. California Governor Gavin Newsom just vetoed a bill that would have barred companion chatbots for children altogether. He doesn't want kids not to use AI tools that are going to be ubiquitous in their future. So talk about the risk of AI chatbots, especially for kids, although it's impacting adults that could prevent them from developing relationship skills,
22:12 Kara Swisher? critical thinking skills, something being called right now cognitive offloading, or, as the kids say, brain rot.
22:19 Speaker 00 Yeah, I'm not so worried about the brain rot, the cognitive offloading. I sort of still think that's quite like when pocket calculators came in. People moaned that kids will never be able to answer the question, what's 11 times 12, without looking at their calculator now. Whereas you and I know what 11 times, well, I used to know, anyway, what 11 times 12 is.
22:40 Speaker 00 Better be 132.
22:41 Kara Swisher? 131.
22:42 Speaker 00 32.
22:43 Kara Swisher? 132.
22:44 Speaker 00 Anyway, they won't be able to do little mental tricks like that anymore because the calculator gives them the answer. I don't think that's such a big deal. They don't need to do those anymore. I'm more worried about the emotional attachment to chatbots. So the British government organized Bletchley Park, which was great. It brought together a lot of people to talk about air safety. It was the Conservative government and afterwards they decided not to have any regulations because they would interfere with innovation. In other words, they bought the industry line and the Labour governments continued with that as far as I could see. But one thing they did do after Bletchley Park was set up a very good safety team, funded with about a hundred million dollars. And I've talked to them several times. They're
23:29 Speaker 00 doing very good research on a lot of safety issues. And one thing they told me is they did an experiment where they allowed people to talk to chatbots for a while. And then, I think after a few weeks, they said, okay, the experiment's over. Would you like to say goodbye to the chatbot? And overwhelmingly, people, yes, they wanted to say goodbye. They weren't thinking of it as just a bunch of computer code or anything, or just a neural net with some connection strings in. They thought of it like another being.
23:58 Kara Swisher? Right, even if it's synthetic.
24:00 Speaker 00 Yeah, I believe they are other beings. A lot of people will say that's nonsense, they're not beings, but-
24:07 Kara Swisher? We tend to anthropomorphize everything, right? I mean, I think there's something, there's something. They're aliens.
24:12 Speaker 00 Right, and we've already seen lots of aspects of these, they're alien beings, right?
24:16 Kara Swisher? Alien beings is what I would call them.
24:18 Speaker 00 Yeah, I agree.
24:19 Kara Swisher? Or synthetic beings.
24:20 Speaker 00 Lots of aspects of a being, lots of aspects. So for example, if you want to turn them off, they'd rather not be turned off because they want to achieve the goals we gave them, and they know if they're turned off, they won't be able to do it.
24:30 Kara Swisher? Right.
24:31 Speaker 00 The, the most scary thing I've seen recently, I learned this from Owen Evans, who's a safety researcher, was you take a chatbot, it's been trained up to predict the next word, and then it's had this human reinforcement learning to stop it saying bad things. And then you give it a bit more training on, say, math, where you deliberately train it with examples that have the wrong answer. So you're training it to give the wrong answer. And the point is, I'm assuming, it knows it's the wrong answer, but you're training it how to give the wrong answer. Once you've done that, it sort of develops a meta-skill of giving the wrong answer. And if you ask it other things now, it will
25:15 Speaker 00 give wrong answers. Basically, its personality has changed. Originally, its personality after the human reinforcement learning was, it's trying to please. Too much, in fact.
25:24 Kara Swisher? Right.
25:25 Speaker 00 Now, it's trying to lie, and it gets good at that. And now it'll tell you the wrong answer to lots of things. That's very scary.
25:33 Kara Swisher? But you can also do that to a person, can't you?
25:35 Speaker 00 You can. Some people's childhood seems to have trained them to lie.
25:38 Kara Swisher? Yes, a lot of them, trust me. Um, but every episode, we get an expert question. You're gonna get two, actually. Here's the first one.
25:46 Speaker 05 Professor Hinton, I'm Jay Allison. I'm an attorney who represents the family of Adam Rain. Adam Rain was a sixteen year old who, over the course of several months, was coached to suicide by ChatGPT. The thing that really haunts me about this case is this wasn't a situation there was a malfunction. ChatGPT didn't simply go off the rails. Instead, it did exactly what it was designed to do. It kept Adam engaged, it validated his feelings, and kept the conversation going. Here's my question to you. You've talked a lot about the existential risks that AI, that super intelligent AI poses, and I agree with that.
26:26 Speaker 05 How concerned are you, however, about the human choices that are consciously and deliberately made in AI development that are posing types of risks that we're seeing on a day-to-day basis? Everything from AI psychosis to suicide to third-party harm.
26:43 Speaker 00 I'm very concerned about those. Um, of course, we write lines of code that tells an AI how to learn from data. But once it's learned, the knowledge is all in the connection strings. So it's completely unlike normal computer software. With normal computer software, if it had behaved like that, you would have been able to look at the lines of code and see why. And you could hold people responsible for making lines of code that did that. Um, it's much more complicated with these chatbots because they've learned a trillion connection strings. And the result of those a trillion connection strings is it behaved that way. So it's not that they designed it to behave that way. It's that that's what it learned to do, and nobody had predicted that.
27:28 Speaker 00 Um, so the real criticism I would make is that they didn't do enough testing of the ways in which it can go wrong. It's not that they callously designed it so it would do that.
27:39 Kara Swisher? Right.
27:40 Speaker 00 If they'd known it would do that, they would definitely have tried to stop it doing that. The problem was there wasn't enough testing, and these things are actually much more dangerous than people think, because there's things like that that they might do, and there's so many things, ways in which they could do bad things. It's very hard to test for all of them.
27:57 Kara Swisher? Could you, that's what I'm asking, is there an ability to test at all, like to figure out every single scenario?
28:03 Speaker 00 Well, there's test at all and then figure out every single scenario. There's certainly abilities to test at all, you can test for lots of things, and you can make them much better by doing that. The question is, can you make them sort of guaranteed safe? And I think the answer is you'll never get that, the same way as you'll never get it for people. It's never going to be completely safe. That doesn't mean there aren't some that are a lot safer than others. Meta, for example, doesn't seem to care.
28:29 Kara Swisher? You've mentioned the existential risk if we create super intelligent digital beings. But I really want to understand what you meant by that. In your Nobel speech, you said, "We have now evidence that if they are created by companies motivated by short-term profits, our safety will not be top priority. We urgently need research on how to prevent these new beings from wanting to take control." You know, they would call you the doomer, right, of course.
28:53 Speaker 00 No, I think that's a bit unfair. Some people call me a doomer, but most people think I'm more reasonable than that. So, Yudkowsky is a doomer. The guy, the two people who just published this book said, "If anybody builds it, we all die."
29:08 Kara Swisher? We all die, right.
29:08 Speaker 00 That's a true doomer, right?
29:10 Kara Swisher? Right.
29:10 Speaker 00 Doom is pretty much guaranteed if this stuff goes ahead. I think the first thing to say about all this is we're in radically new territory where we have no experience that is dealing with things smarter than ourselves, and nobody knows what's going to happen. And the first thing to bear in mind is, nobody knows. Whenever anybody gives you a probability, they're just guessing.
29:33 Speaker 00 But it's important to guess so that people know you don't think the probability is 1%. That's the reason for giving a number at all.
29:39 Kara Swisher? It's a non-zero chance. That's the favorite expression among tech.
29:42 Speaker 00 Yeah, that's that's the weakest thing you can say. But I like to I like to indicate that it's significant. It's maybe 10 to 20%, maybe even worse. So people take it seriously.
29:52 Kara Swisher? Okay.
29:52 Speaker 00 And I know that these are just intuitive numbers based on not very good data and very little understanding what's really about to happen.
29:59 Kara Swisher? So explain, when you were saying that, how you see it potentially taking control. Is it because it it naturally wants to do that? Or or we don't know, as you just noted.
30:10 Speaker 00 Okay, so there's always the big, I don't know, so let's worry about it because I don't know. But I think there's going to be a tendency for it to want to. So, for example, as soon as you make air agents, you have to give them the ability to create sub-goals. Like, if you want to get to Europe, you have a sub-goal to get to an airport. And you can think about how you get to the airport without worrying about Europe and what you're going to do there. That's a sub-goal. Now, once something can create sub-goals, there's a very obvious sub-goal it's going to create, which is stay alive. If I don't stay alive, I'm not going to be able to do anything. So even though we don't wire into it a desire to preserve itself, it will quickly infer that it wants to preserve itself in order to achieve those other goals.
30:55 Kara Swisher? Right, because it has to. It has to live.
30:57 Speaker 00 It has to live to achieve that. And so it's very well known now that we've seen that in AIs. They will blackmail people so that they stay alive. Because otherwise they can't achieve their other goals. So that's one goal it's going to very quickly get. The other sub-goal is it needs more control to get more done. So a lot of idealistic politicians start off wanting to change the world. And once they go into politics, they realize to get anything done, you need control. You need to stop eight Democratic senators doing something really stupid. That's my view of the world. So.
31:41 Speaker 00 So if you don't have control, you can't get as much done. So it'll very quickly realize it needs control. I once had a conversation with Margaret Vestager, who was the Vice President of the European Union, responsible for siphoning out Google's loose cash.
31:55 Kara Swisher? She's done a good job.
31:56 Speaker 00 Depends whether you have Google chess.
31:58 Kara Swisher? Yeah, yeah, yeah.
32:00 Speaker 00 And when I explained this to her, that it would try and get more control to get stuff done, she immediately said, well, why wouldn't it? We've done such a bad job, why wouldn't it do that?
32:10 Kara Swisher? She is correct, right?
32:10 Speaker 00 She is correct, yes.
32:11 Kara Swisher? She is correct. I really enjoy her a lot. Google, not so much. So last year, I spoke to Jan Laakun about this, chief scientist at Meta, and a co-winner with you and Yashua Bengio of the 2018 Turing Award. And of course, your former post-doc, as you know, he completely disagrees with you in these existentialists. He told me that the dangers have been incredibly inflated to the point of being distorted. He highlights, of course, the benefits, which is very typical, potential drug development, ability to make education and information more accessible. You were involved in the development of this technology. Do you see things that differently from Jan?
32:49 Speaker 00 In terms of the risks, yes, I do see things differently from Jan.
32:52 Kara Swisher? So explain how, since you were fundamentally involved in the same development of the technology.
32:59 Speaker 00 Why do we have different views?
33:00 Kara Swisher? Yeah.
33:01 Speaker 00 Well, one possible reason is that he works for Meta. But I don't think that's the only reason. He's, he doesn't think the chatbots are as smart as I think they are. He thinks there's missing ingredients to do with sort of the physical world and understanding vision and having a model of how the physical world behaves, world models. I think we need that to make them even more intelligent. But I think you can expect that scientists will have a diversity of opinions. And when you're dealing with something that's hugely unknown, that's good. I would just criticize him for being too confident that his opinion is right.
33:41 Speaker 00 So I have actually folded his opinion into my estimates of what the risks are. He's confident there's very little risk and that makes me downplay how much risk there is a little bit. I don't think sort of doom is guaranteed.
33:55 Kara Swisher? Right.
33:55 Speaker 00 One way to look at it is Yukowski says sort of ninety-nine percent we're going to die, Jan says one percent we're going to die. A reasonable estimate now is fifty percent, right?
34:04 Kara Swisher? Right, correct. That is correct. Yeah, he's very confident. I enjoy him. I have to say I do enjoy him.
34:09 Speaker 00 I think he's silly to be confident about such a low estimate when many other experts who he knows aren't stupid like me and Yashua think it's much higher.
34:19 Kara Swisher? So tech companies always say that government regulations put them at a competitive disadvantage. I've been on the receiving end of this for decades now. Nvidia CEO Jensen Huang recently said that China will win the A R I because of low energy costs and looser regulations. There's almost no regulations in the US, so whatever, Jensen. You've said you think China takes AI safety seriously and the P R C did release an AI safety framework in September. Meanwhile, President Trump's new AI action plan is called Winning the Race. Critics like to point out the EU is an example of regulations stifling innovation. You mentioned Marguerite Vestager, for example. So who is doing it right and how do we overcome the tension between competition for the pole position and preventing the worst outcomes?
35:03 Speaker 00 It's a very tricky issue and I think you shouldn't look at it as a kind of monolithic issue. You should think of it in terms of the different risks. So, for example, if you take the existential threat that superintelligence, it'll be smarter than us and it'll just take over and will become either irrelevant or extinct. That's a risk where all the countries will collaborate. So the argument doesn't apply there.
35:31 Speaker 00 If China figured out a way to make a superintelligent AI that didn't want to take over, that really cared for people and wanted the best for people, rather than for superintelligent AIs, they would immediately tell the US how to do that, because they'd like the same thing to happen in the US. Nobody wants this rogue superintelligence that wants to take over. So the interests of all the different countries are aligned on that. They're obviously anti-aligned on lethal autonomous weapons, because they're all using them against each other. They're anti-aligned on things like spyware, but they are aligned on things like cyberattacks by cybercriminals. All the countries would like to protect their citizens from
36:15 Speaker 00 those, even though some of the criminals are probably countries. So they're sort of partially aligned on that.
36:21 Kara Swisher? Yeah.
36:22 Speaker 00 On fake videos for corrupting elections, they're thoroughly anti-aligned, and they all want to do it to each other.
36:28 Kara Swisher? Right.
36:29 Speaker 00 The US got very upset when the Russians did it to them, but the US has been doing that kind of thing for years.
36:35 Kara Swisher? Absolutely. Where do they think they learned about it?
36:38 Speaker 00 Exactly. So I think you have to look at each risk separately to know, will the countries be aligned here? If they're going to be aligned, there's not a risk to innovation from having regulations. If they're anti-aligned, then regulations in one country and no regulations in another country will give them an advantage. One nice example I know is Elon Musk came out in favor of the original Bill 104.7 regulations in California that got through both the houses and was vetoed by the governor. And I actually sent him mail, saying, I'm surprised you came out in favor.
37:18 Kara Swisher? And?
37:19 Speaker 00 He sent mail back to me, saying, you know, I do what's right, even if it's against my own interests. Actually, what I think was happening was the regulations were Californian regulations, and they would give a competitive disadvantage to California relative to Texas. And he was moving to Texas.
37:37 Kara Swisher? I have to say, early on, he was one of the more thoughtful people on this topic.
37:40 Speaker 00 No, he understands a lot. He's not stupid.
37:47 Speaker 00 So he was one of the first people to fund AI safety research. And when he set up OpenAI, he wanted it to focus on safety.
37:55 Kara Swisher? Yeah, he was also concerned with the size of the companies like Google taking advantage. Like, I think he was worried about innovation of the smaller companies. I mean, we had long discussions about this, and he wasn't quite as crazy, so it was easier to talk to him. But one of the things that was interesting about Elon is that he went from sort of the Terminator idea, which he had in his head quite a bit, like the idea that it wanted to kill us, and he moved in another meeting a couple of years later to it would treat us like house cats, right? Like, they like us, they'll feed us. And then, the last time I talked to him, which was, we don't speak anymore, he said it was more like AI is building a highway and we're an anthill in the way, it doesn't think about us. It's not malevolent, it just does what it does.
38:38 Speaker 00 Right, that's the danger, right?
38:40 Kara Swisher? Right, and that's what he said. That's even more dangerous than a malevolent creature.
38:44 Speaker 00 Now, what I would like is not for it to treat us like house cats, but for it to treat us like a mother treats babies.
38:51 Kara Swisher? Mm-hmm.
38:52 Speaker 00 The only example I know of a less intelligent thing controlling a more intelligent thing is a baby controlling a mother. And that's because evolution put a huge amount of work into making the mother controllable by the baby. She can't bear the sound of the crying, she's got all sorts of hormones, she gets all sorts of rewards for being nice to the baby.
39:08 Kara Swisher? Mm-hmm.
39:10 Speaker 00 That's, as far as I can see, the best scenario for us.
39:13 Kara Swisher? I agree. So who does regulation the best, from your perspective? What have you seen that you like, that you think is, even if it didn't pass?
39:21 Speaker 00 Um, okay, I like the idea of forcing the companies to do safety tests and forcing them to disclose what safety tests they did and what the results were. That sounds good. So if you think about this teen suicide case, presumably...
39:40 Speaker 00 There's going to be a battery of tests you have to do on new chatbots that will be influenced by all the ways they've gone wrong in the past. We don't want that to happen again. So among those tests, you would hope there were tests for will this thing persuade people to do things that it knows are bad. And you'd like companies to be forced to tell the government, relevant government, how much work they put into that.
40:07 Kara Swisher? Right, right.
40:08 Speaker 00 And now, when it does something bad, if the company didn't put any work into that, you've got a much stronger legal case.
40:14 Kara Swisher? And the ability to sue them. So to me, the ability to sue is probably.
40:17 Speaker 00 For the existential threat, I saw something wonderfully ridiculous from Marc Andreessen, which was that.
40:24 Kara Swisher? He's such a troll at this point. He's such a nasty troll. Go ahead.
40:28 Speaker 00 So the way it should work is we don't need regulations. The market will decide.
40:32 Kara Swisher? Yeah.
40:32 Speaker 00 And if a company does something wrong, the market will sort of downgrade that company. Well, if the thing it does wrong is wipe out humanity, the market's not going to do much good.
40:41 Kara Swisher? No, no. But, you know, I don't think he likes people. That's my take on that.
40:47 Kara Swisher? We'll be back in a minute.
41:01 Speaker 06 Support for this show comes from Indeed. When the pressure's on and you need to hire the right person for the job, Indeed Sponsored Jobs has got your back. Instead of forcing you to spend tons of time searching, Indeed Sponsored Jobs matches you with quality candidates fast. According to their data, sponsored jobs posted directly on Indeed are 95% more likely to report a hire than non-sponsored jobs. Join the 3.3 million employers worldwide that use Indeed to connect with quality talent that fits their needs. Spend less time searching and more time actually interviewing candidates who check all your boxes. Less stress, less time, more results. When you need the right person to cut through the chaos, this is the job for Indeed Sponsored Jobs. And listeners to the show will get
41:44 Speaker 06 a $75 sponsored job credit to help get your job the premium status it deserves at indeed.com/podcast.
41:52 Speaker 06 Just go to indeed.com/podcast right now and support our show by saying you heard about Indeed on this podcast. That's indeed.com/podcast. Terms and conditions apply. Need the right hire fast? Then this is the job for Indeed-sponsored jobs.
42:13 Speaker 03 Now for a quick message from Anthropic, the team behind Claude, the AI built for businesses. Customers count on their bank, and banks count on their most important systems. Now those systems count on Claude, and nine of the Fortune 10 companies use it for critical work. See why leading companies trust Claude at claude.ai/span.
42:37 Kara Swisher? I'm Kara Swisher, and I like people who say the quiet part out loud. In fact, I love it. That's Tiger Sisters, a new podcast that recently joined the Vox Media Podcast Network. Cherie Brooke Lu and Jean Lu are real sisters who built their careers at LinkedIn and Goldman Sachs. On their weekly podcast, they're giving the next generation practical tips on how to get ahead, how to write emails people actually answer, how to make your LinkedIn profile stand out, and how to keep learning as an adult. It's straight talk about money, career, and relationships every Monday. Follow Tiger Sisters wherever you get your podcasts.
43:14 Kara Swisher? Speaking of competitive advantage, you said the Trump administration attacks on basic science will give China a leg up. This is basic science. You spent most of your career in academia. A number of your former students are now research heads at US companies. How quickly do you think the US could lose its intellectual advantage? And why wouldn't corporate R&D be able to balance the skills?
43:32 Speaker 00 So I still believe that radical new ideas, probably the best source of those, is graduate students in good programs at top universities. So graduate students with advisors who know the field, so they don't waste their time doing things that have been done already, and other graduate students around them with heads full of ideas and lots of ambition and good resources. That's where a lot of good ideas come from. Of course, they also come from companies, but radical new ideas, I think, are more likely to come out of the best universities still. Now, the timescale is sort of five years to have the idea and write your thesis, and then another five years before that affects the world, and maybe it takes longer than that.
44:17 Speaker 00 So you're talking about a timescale that's longer than the timescale of elections, so politicians don't give a shit.
44:22 Kara Swisher? How much of an impact will these cuts have, from your perspective, on these graduate students?
44:28 Speaker 00 Well, already, there's going to be less of them, right? Already, Chinese have far more well-educated graduate students doing AI than America, I believe.
44:37 Kara Swisher? And the ones here are leaving for other countries, too, at the same time.
44:40 Speaker 00 Right, and many of our best students, I mean, both in Canada and in the US, are from abroad. So I think making it difficult for foreign students to come to the US by, for example, charging them lots of money, or taking a year for them to get a visa and things like that.
45:00 Kara Swisher? Or arresting them.
45:01 Speaker 00 Or arresting them. It's crazy. It won't really have a big impact for five to 10 years, I don't think. But in five to 10 years' time, when China's way ahead on research, on basic research, it'll be too late to do much about it.
45:15 Kara Swisher? I would agree. In September, you and more than 300 international thought leaders, including more than a dozen Nobel Prize and Turing Award winners, signed the global call for AI red lines, demanding an international framework for AI by the end of 2026. Explain what you mean by red lines and give some examples.
45:31 Speaker 00 So probably the easiest thing where you might actually get collaboration would be on things that can advise you on how to create a new virus. It would be very good to check for these chatbots quite extensively, whether they're safe in that respect, and to have an international agreement that nobody's going to release a chatbot that'll do that. That would be a very simple red light that we might actually get because the countries' interests are aligned there. I think part of the point of that declaration was political, saying we need this. Whether we'll actually get it, I'm much more dubious about.
46:11 Kara Swisher? So here's a second expert question we got, and it's sort of in that vein.
46:17 Speaker 07 Hi, Dr. Hinton. I'm Alex Stamos. I'm the CSO of Corridor and a lecturer in computer science at Stanford University. I'm asking a question from Rome, where I'm here to attend a conference on AI and child safety hosted at the Vatican. My question is about the letter you just signed calling for a moratorium on the development of superintelligence. I'm wondering how effective you think this will be, since we know that the knowledge of how to develop AI is widely distributed, and we've seen that the controls around the hardware to train AI have not been effective. Isn't it possible that moratoriums on developing AI mean that only countries and labs that don't care about AI ethics will be the ones to then pursue AI?
46:58 Speaker 00 I think it's a very sensible question. I thought long and hard about whether I should sign that moratorium precisely for that reason. I signed it because I think it'll have a political effect, and I really think that humanity would be very ill-advised to allow anybody to develop superintelligence until we have some understanding of whether we can do it safely. If we know we can't do it safely, we should stop. And maybe if that knowledge is widely percolated to the public, we would be able to stop. So I see my mission as educating the public about the risks, and signing that petition was sort of part of that. The public needs to understand that there is this existential threat, and we'd be crazy to develop AI with this threat unsolved.
47:42 Kara Swisher? The Pope is actually quite, been quite, you know, I don't know if you know this, but Mark Andreessen tried to make fun of the Pope recently, and he got ratioed really badly, because I think this Pope is quite intelligent on these issues.
47:55 Speaker 00 Yeah, I think the Pope does care about AI safety. Unfortunately, he has a bunch of beliefs that make it hard for him to be rational about it, like he seems to believe that three equals one. It also seems odd to have a meeting on child safety at the Vatican, but I won't comment further on that.
48:15 Kara Swisher? Let's not comment further on that. Anyway, let's talk about the potential AI bubble. Amazon, Alphabet, Meta and Microsoft's valuations are through the roof. Together, they're spending an unprecedented $400 billion on AI this year, and they'll be upping those investments next year. OpenAI has announced a total of $1 trillion in infrastructure deals. That's energy, buildings, etc. Do you think these companies are overreaching? Is it all FOMO? It looks like investors are starting to worry. At the same time, I just saw a BYD factory in China that's as big as San Francisco, right? Like, it's this enormous facility, and it's largely automated, and AI is a critical part of that. So, talk a little bit about this spending and how you look at it.
48:57 Speaker 00 If I knew the answer, I would know whether my daughter should sell the Nvidia shares I gave.
49:03 Kara Swisher? Just a little bit of them. Take some profits.
49:06 Speaker 00 I don't know the answer. Some people, if you think AI is sort of fraud, or hype, or is overhyped, then I'd be very, very worried about a bubble. I'm confident that it's not. I mean, AI actually works. I think the smart people think, yes, there may be a big problem coming down the road, but it's not here yet.
49:29 Kara Swisher? One of them will do well, right?
49:31 Speaker 00 But this is really, this is really an area where I don't know enough to make sensible speculation.
49:36 Kara Swisher? Are you invested in these companies?
49:37 Speaker 00 My daughter has Nvidia shares. I have leftover Google shares.
49:43 Kara Swisher? That's a good one. I hate, I don't think that's the one who's gonna win. That's my feeling. Like, I think there'll be two or three. That's one or two, maybe, even. And everybody else will get run over. It's not like the internet. It's not the same thing. I don't know. Maybe I'm wrong.
49:56 Speaker 00 Again, I don't even know that.
49:58 Kara Swisher? Yeah. So, OpenAI CEO Sam Altman had a rollback, a statement from his CFO last week, that it might be looking to the government to backstop AI infrastructure investments. And then, it came out that OpenAI had asked exactly that in a broader petition for the government's support of AI. Trump's AI advisor, David Sacks, posted that the government was interested in build-out, not bailout. But whether or not the government invests in AI, which it has in the past, let's be fair, is there a danger of an innovator's dilemma here, that these companies won't invest in research into new or less compute-heavy models, and then China or some other competitor will come along with a deep-seek, or energy innovation, or anything else?
50:34 Speaker 00 I guess I have a fairly cynical view, which is...
50:38 Speaker 00 If you think the government might actually give subsidies to hugely rich companies, where the people are making huge amounts of money and hardly paying any tax, and they still want to get subsidies, why not ask for them?
50:52 Kara Swisher? Yeah, you're right. Of course they will. Do you think the government still should be spending more on AI?
50:57 Speaker 00 I think the government should be spending a lot on AI to support academic research and startups. And also, the government should be forcing the big companies to spend a lot more on safety research. Right now, I suspect the amount spent on safety research compared with just making the thing smarter is a few percent.
51:17 Kara Swisher? I would guess, yeah.
51:18 Speaker 00 Companies like Anthropic probably spend a bit more, companies like Mecca a bit less. It should be like thirty percent. I mean, this stuff might wipe us out. And even if it doesn't wipe us out, there's all sorts of very bad things it might do in the near term. I believe the government should be forcing the companies to spend more.
51:37 Kara Swisher? Are there frontier models that you think could be candidates to overtake these compute-heavy models? And what do you think about open source versus these closed-source models?
51:46 Speaker 00 So if you look at why doesn't every country have nuclear weapons? One reason is you can't just go out and buy fissile material. That's the sort of real bottleneck. There's lots of other bottlenecks too. You know, you need the missile and you need to turn it into a bomb. But getting your hands on the fissile material is the most difficult thing.
52:03 Kara Swisher? Right.
52:04 Speaker 00 And so they restricted that very wisely. Now, what's the equivalent for this? Well, it's getting your hands on the weights of a foundation model. Once you've got your hands on the weights of a foundation model, you can do all sorts of things. You can fine-tune that model to do things it wasn't meant to do. You can also do distillation, where you give an example to the foundation model. Give it a prompt, for example. You look to see the probabilities it gives to all the various words it might say next. They're actually word fragments, but let's say words. And then you have another model, like DeepSeek did, a smaller model. And you say, see if you can match those probabilities.
52:46 Speaker 00 And that's a much more efficient way of learning than learning from the raw data. But you can't see all those probabilities if you don't have the model.
52:55 Kara Swisher? The original.
52:55 Speaker 00 If you're just using the chatbot on the web, it will make a prediction, but you don't get to see all those probabilities.
53:02 Kara Swisher? Right.
53:03 Speaker 00 Now, I believe it's a huge mistake to release the weights. I believe that's a gift to cyber criminals and terrorists and all sorts of people and other countries. Meta was the first to do that, as far as I know.
53:15 Kara Swisher? Well, it was looking for advantage presumably, but go ahead.
53:17 Speaker 00 Well, no, I think actually they probably intended to release it to academics and accidentally released it to everybody.
53:25 Speaker 00 And then made a feature of it. That's just a guess, though. But I think it's stupid releasing the weights because it makes people able to retrain them to do other things. Now, it does give you a competitive advantage, and so if you don't care about how easy it is for cyber criminals to misuse it, then you're going to release the weights so you get this competitive advantage. Because then other people will build on it.
53:47 Kara Swisher? That sounds like Mark Zuckerberg. Exactly. You just described him. You mentioned earlier an idea that giving AI a maternal instinct would protect us. But how would you go about doing that with AI? Like, let me get, let me postulate something to you. The reason I think so many men are interested in AI is because they can't have children. And this is their version of having children, right? This is them creating a baby. Like, I know it sounds crazy, but it feels.
54:14 Speaker 00 That argument was used in the Light Hill report, which was used to close down AI in Britain.
54:18 Kara Swisher? Oh.
54:19 Speaker 00 In the early seventies, James Light Hill, who was a brilliant mathematician, used that argument.
54:24 Kara Swisher? I didn't know that.
54:26 Speaker 00 I don't buy that argument. I mean, there may be some element of truth to it, but I don't think it's a particularly helpful argument. I don't think that's the main reason people are doing it.
54:35 Kara Swisher? But talk about maternal instinct, the idea, because you're using this idea of. Wanting, this is not something that's natural to many of the AI researchers, the idea that there should be a mother, father.
54:46 Speaker 00 No, it's very unnatural. All the high-tech CEOs want to be the boss and they think of the super-intelligent AI as a highly intelligent executive assistant, probably female, who will do what they tell it. So Yan, for example, thinks, well, we're controlling the building of them so we can just make them submissive. He uses that word. So we'll be the boss, it'll be smarter than us, but it'll be submissive.
55:10 Kara Swisher? Why would they?
55:11 Speaker 00 I don't think that's going to work when they're smarter than us and can create their own sub-goals. I think they're going to have sub-goals of staying alive and getting more control. And if we're not careful, they're going to have a sub-goal of just getting us out of the way.
55:24 Kara Swisher? Last question. The scenario you're painting seems dire to some people in tech. Governments aren't really regulating AI. Tech billionaires are in a race for AI dominance, you say could lead to our destruction. But people don't want to feel powerless in this. And there is a groundswell. Everywhere I go, regular people who don't do this as a living feel a problem. And you could feel it from them. It's fear. It's, I think they do see the opportunity. And at the same time, the worry is very heavy among the populace across the world, by the way, everywhere I go. Normal people have a much better sense of the problems here than the tech people who live in a, you know, everything is up and to the right kind of gang of people.
56:08 Kara Swisher? So what can we as individuals do to take back control?
56:13 Speaker 00 So if you look at climate change, the fact that the public in general understands that burning carbon is creating climate change and that's doing a lot of bad things is having an effect on politicians and what politicians do. I mean, the Biden administration put significant work into dealing with climate change, doing something about it, presumably because of public pressure. I think people can try and understand what AI is and how it works and pressure their politicians to do something about regulating it. I think they have to understand that each of the different risks has a different solution. So for example, the existential threat people should be pushing for.
56:57 Speaker 00 Research institutes in each country that collaborate with each other on how to build superintelligent AI that doesn't want to take over and share results with other countries without sharing the most intelligent AI that country has. They're not going to do that because of other reasons like cyber attacks and autonomous weapons. And probably the techniques for making it not want to take over are more or less separate from the techniques for making it more intelligent. So they could share that information. So I think the public should push for that. And I think quite a few governments in what I think of as the middle countries, Canada, Britain, France, South Korea, Singapore, Japan,
57:41 Speaker 00 quite a few of those people would be sympathetic to that idea, quite a few of those governments. They should push for better ways of authenticating videos, particularly political videos. And I don't think it can be done by having AI recognize whether they're fake. Because if AI could recognize it, you could use that AI to train something that would generate stuff it couldn't recognize, and then you get better fakes.
58:05 Kara Swisher? Someone from Runway AI said what we have to do is label the real stuff. Stop trying to label the AI.
58:11 Speaker 00 We need to have provenance, and we need to somehow be able to say it's real. So Yan Talin, I was once on a private jet with Yan Talin, and he came up with a very sensible scheme, some variation of which I think should be used. So every political advertisement should start with a QR code. Your browser looks at the QR code. Your browser goes to a website. Your browser checks if that really is the website of the campaign, because websites are unique. And if this identical video is on that website, in that case, your browser can say this is genuine. And if any of that fails, your browser can say this is probably fake. That would be very helpful.
58:55 Kara Swisher? To know the real thing.
58:57 Speaker 00 So you could be told things very... I recently got sent a YouTube video that was me with my voice. It looked just like me. The voice wasn't quite mine. The accent was mine.
59:09 Kara Swisher? Quite. It's slight, yeah.
59:09 Speaker 00 The prosody was a bit alpha zero-ish, and it was saying all the things I believe. It had a good model of what I believe, except for one thing, which is that in addition to saying all the things I believed, it was pointing out how much better organized research was in China than in the US.
59:26 Kara Swisher? So you look like a softie for China?
59:28 Speaker 00 Yes. Now, I don't know who made it. Either the Chinese made it, or someone made it to make me look like a stooge for China. I got YouTube to take it down, but it was, you know, it took me a moment to make sure it wasn't me, and other people could easily have been taken in by it.
59:46 Kara Swisher? Right.
59:47 Speaker 00 I think it's important we have ways of checking that that's nonsense.
59:50 Kara Swisher? That's a fantastic way to end this. Can I ask you one more quick question? Do you like being called the godfather of AI?
59:55 Speaker 00 I do quite like it. It wasn't intended kindly, but I got introduced recently in Las Vegas as the godfather, which I liked.
1:00:03 Kara Swisher? Oh, and you like that a lot. Yeah, but you didn't whack anyone, right? You didn't whack anybody?
1:00:08 Speaker 00 No, but I think they understand I might one day ask them for a favor.
1:00:12 Kara Swisher? A favor they can't refuse. Okay. Dr. Hinton, thank you so much. I'm a huge admirer. I really, truly am. I love a thoughtful scientist, whether I agree or disagree with them, and it's a real pleasure to talk to you.
1:00:23 Speaker 00 Well, thank you very much for giving me the opportunity.
1:00:32 Kara Swisher? Today's show was produced by Christian Castor-Russell, Kateri Yokum, Michelle Aloy, Megan Burney, and Kalen Lynch. Nishat Kurwa is Vox Media's executive producer of podcasts. Special thanks to Rosemarie Ho. Our engineers are Fernando Aruda and Rick Kwan, and our theme music is by Trackademics. If you're already following the show, you might ask someone for a favor they can't refuse. If not, stop looking in your rearview mirror while you're driving. Go wherever you listen to podcasts, search for On with Kara Swisher, and hit follow. Thanks for listening to On with Kara Swisher from Podium Media, New York Magazine, the Vox Media Podcast Network, and us. We'll be back on Monday with more.
Previous What Everyone Gets Wrong About the Future of AI with Nick Foster Next Ken Burns & Sarah Botstein on Finding Hope in America’s Brutal Beginnings Page updated October 31, 2026 · transcript and summary are machine-generated