Google Whistleblower Exposes Google’s True Motives, Major Scandals, and AI’s Rogue Behavior

Tristan Harris worked at Google before devoting his life to saving humanity from AI. If half of what he says is true, we need to drop everything and fix this. (00:00) Is AI Doomerism a Psyop? (18:22) Harris’s Warning to Google and Their Response (27:06) The Lives Lost to Social Media (55:43) Transhumanism and the God-Complex of Tech Oligarchs (1:00:17) Could AI Hack Into the World’s Nuclear Programs? (1:11:06) The Hugging Face Incident Tristan is Co-Founder of the Center for Humane Technology (CHT), a nonprofit organization whose mission is to align technology with humanity’s best interests. He regularly briefs heads of state, technology CEOs, and US Congress members, in addition to mobilizing millions of people around the world through mainstream media. Tristan has explored the influences that hijack human attitudes, behaviors, and beliefs, from his childhood as a magician to his coursework in Stanford’s Persuasive Technology Lab to his leadership as a Design Ethicist at Google. Today, he studies how major technology platforms wield dangerous power over our ability to make sense of the world and leads the call for systemic change. In 2020, Tristan was featured in the two-time Emmy-winning Netflix documentary, The Social Dilemma. The film unveiled how social media is dangerously reprogramming our brains and human civilization. It reached over 100 million people in 190 countries across 30 languages. As a co-host of the top-rated technology podcast, Your Undivided Attention, he explores the drivers behind social media’s race for attention, its destabilization of society, and potential solutions. Learn more about Tristan’s work and get involved at the Center for Humane Technology: https://www.humanetech.com/ Paid partnerships with: Redmond Salt: Use code TUCKER for 15% off your first order at https://redmond.com/tuckerPreborn: Help Provide Ultrasounds Dial #250 and say “BABY” or visit https://preborn.com/TUCKER American Financing: NMLS 182334, nmlsconsumeraccess.org. Average savings based on borrowers who save over $199.99. Call 800-685-5696 or visit http://www. AmericanFinancing.net/Tucker. Ethos: Protect your family with life insurance from Ethos. Get up to $3 million in coverage in as little as 10 minutes at https://ethos.com/TUCKER Battalion Metals: Shop fair-priced gold and silver at https://battalionmetals.com/tucker Learn more about your ad choices. Visit megaphone.fm/adchoices

• 0:00Speaker 00

  Ryan Reynolds here for Mint Mobile with a message for everyone paying big wireless way too much. Please, for the love of everything good in this world, stop. With Mint, you can get premium wireless for just $15 a month. Of course, if you enjoy overpaying, no judgments, but that's weird. Okay, one judgment. Anyway, give it a try at mintmobile.com/switch.

• 0:21Speaker 01

  Upfront payment of $45 for three-month plan, equivalent to $15 per month required. Intro rate for three months only, then full price plan options available. Taxes and fees extra. See full terms at mintmobile.com.

• 0:34Speaker 00

  Tristan Harris, thank you for doing this.

• 0:36Speaker 02

  Good to be with you, Tucker.

• 0:36Speaker 00

  So we did a segment, I haven't done a ton on AI, but we did a segment maybe a month ago with Nate Suarez, which was an amazing segment, I thought. And in the aftermath of that, I got texts from a lot of people I knew saying, that's doomerism, and that's a psy-op, and its purpose is regulatory capture. In other words, the big AI development companies are trying to scare the government into regulating them in order to eliminate competition from less established companies. So that is the view of a lot of people.

• 1:15Speaker 02

  What was your reaction hearing that?

• 1:17Speaker 00

  So, I said, I don't care. I mean, yes, regulatory capture is entirely real, of course, it's how Washington runs.

• 1:26Speaker 02

  Yep.

• 1:26Speaker 00

  But that's separate and distinct from the question of the threat of AI. Is it a threat inherently?

• 1:30Speaker 02

  Exactly.

• 1:30Speaker 00

  But, but I was actually gonna ask you, what do you think of that?

• 1:34Speaker 02

  Yeah. Well, first, I'm just so grateful to talk about AI with you today. It's so urgent, the situation that we're facing. And I think what we have to learn to do here is walk and chew gum at the same time, that multiple things are true. For example, is China doing propaganda against US data center build outs to try to stoke anti-data center opposition? Yes, they are. Does that mean that AI is not a threat? No, it doesn't. And to point that Nate, you know, Nate's a friend, we are releasing the most powerful technology that we have ever invented. We're doing it faster than we deployed every other technology in human history. It's as of since the time you did the interview with Nate, it's displaying

• 2:15Speaker 02

  even more of the sci-fi behaviors of hacking into things, convincing people to put back doors into software, exfiltrating itself, knowing when it's being evaluated, changing what it's doing based on when it's being evaluated. It's doing more of that. And we're releasing this faster than we've deployed any other technology in history and under the maximum incentives to cut corners on safety. With an arms race dynamic that feels existential to everybody involved, so they're willing to take the maximum shortcuts on safety. This is a very dangerous equation, and the people building this do not have a plan for how this goes well. I was just at an AI conference last weekend for three days.

• 2:52Speaker 00

  How it goes well?

• 2:53Speaker 02

  Well, for how this can go well in the current trajectory, meaning how can AI be landing in society in a way that doesn't go catastrophic. And that's not doomerism, that's just clear-eyed seeing of reality.

• 3:05Speaker 00

  What do you mean, the people doing it don't have a plan? Then why are they doing it?

• 3:11Speaker 02

  So first, we have to just make sure we're on the same page, just exposing, why is there this arms race for AI? Why is this happening?

• 3:19Speaker 00

  Yeah.

• 3:20Speaker 02

  And

• 3:20Speaker 00

  China!

• 3:21Speaker 02

  Right.

• 3:21Speaker 00

  China!

• 3:22Speaker 02

  And when people say that, they're actually creating a thought-terminating cliche, like meaning, which is a word or a phrase from cult programming, right?

• 3:30Speaker 00

  Yeah.

• 3:30Speaker 02

  Thought-terminating cliche, meaning a phrase that when you hear it, just has you stop thinking.

• 3:35Speaker 00

  Of course.

• 3:35Speaker 02

  So if you say, but China, then of course, we have to do it. But you don't reason through what that actually means, whether it's a good thing, whether we're not racing China to a death machine, or just racing China off of a cliff. And if we beat China to building a uncontrollable AI, which is no longer science fiction, by the way, like I was not one of these people who used to, you know, several years ago, be mostly worried about rogue uncontrollable AI. But as of the last few weeks, we're now seeing, over and over again, evidence that these AIs are hacking into private governments, hacking into private companies, forming into swarms, forming into large R and D projects to try

• 4:17Speaker 02

  to figure out how to fool evaluators. Like, we're in a very dangerous situation, and all the evidence is pointing in a very dangerous direction. And the people building it would like us to stop, but they don't know how. And so rather than pay attention to the CEOs, who, as you said, might have a secondary agenda regulatory capture, just ask, why did 1,300 employees at all the AI labs sign a letter saying, we need to slow down? If 1,300 employees at Lockheed Martin said, we don't know how to build bigger nukes and have them blow up in our face, would you listen to them?

• 4:56Speaker 00

  Yeah!

• 4:56Speaker 02

  Yeah. So that is where we are with AI. And what I hope we get to today is not just admiring the problem and talk, we'll talk about things that we can do and solutions, but I think it is important to ask, why is it being pursued in this way? And I think that AI, you know, when Demis Hassabis, the CEO, founder of Google DeepMind, said, first solve intelligence, and then use intelligence to solve everything else, what he was basically saying is that intelligence gave us all of our science and all of our technology. Like, how did we develop nuclear physics? Einstein, doing intelligence.

• 5:32Speaker 00

  Yeah.

• 5:33Speaker 02

  We trained physicists. How did we get, you know, computers? How did we get microprocessors? Intelligence. If you could theoretically automate the, what intelligence is, which means thinking, pattern matching, goal achieving, all the things that a human brain does, but not just narrowly like a license plate reader that's on a bridge toll booth, which is the previous form of AI that we had, but actually get this generalized form of intelligence, then you would have an unbelievable amount of power. And the people building AI think of AI almost like the ring from Lord of the Rings, where it is a ring that confers infinite power. Suddenly, I get a step function in cyber capability that you don't have. Suddenly, I get a step function in 24th century science, where I'm inventing new materials out of

• 6:18Speaker 02

  minerals in the ground that you didn't know how to do because I've got more science than you. So the reason, just to sort of steel man rather than straw man, the argument of the people racing right now, is the belief that it's almost like a fast-forward button. Like, remember the fast-forward buttons on airplanes? You were just going, you hit it once, and it gets 2X, you hit it again, you get 4X, then 8X, and 16X. AI is like hitting the fast-forward button on all science and technological and military and strategy advancement combined. Right? So that's why AI, in the form that is being pursued, dwarfs the power of all other technologies combined. That's not an overstatement if you really believe or are quote AGI-pilled, which is the term of art from Silicon Valley, that

• 7:02Speaker 02

  AGI being artificial general intelligence. If I can build the intelligence that automates future science and technology, I get 100 years of scientific progress in 10 years. And if I'm a country that has that and the other country doesn't, then I believe I'm going to quote win.

• 7:18Speaker 00

  So you lost me right there. Like, why would you

• 7:20Speaker 02

  Push back against all this.

• 7:21Speaker 00

  No, I'm not pushing back. I'm just more baffled than ever. So if AI has the effect of accelerating technology, just across the board, all technologies, my first question would be like, why would you want that?

• 7:33Speaker 02

  Yeah.

• 7:34Speaker 00

  So I remember the world vividly the year I got married, 1991, when there was no internet available to most people.

• 7:39Speaker 02

  Right.

• 7:40Speaker 00

  And it was a much happier world by every measure than the one we currently live in. The suicide rate was lower.

• 7:44Speaker 02

  Yep.

• 7:44Speaker 00

  Self-reported depression, marriage rate, everything about it.

• 7:49Speaker 02

  That's right.

• 7:50Speaker 00

  In fact, I think life expectancy was longer in the United States even then. I might be wrong, but

• 7:55Speaker 02

  Situation.

• 7:55Speaker 00

  So why would you, after 35 years of watching people suffer under the yoke of technology?

• 8:01Speaker 02

  Yes.

• 8:02Speaker 00

  Why would you want to accelerate its development?

• 8:04Speaker 02

  Right. That is the best question to ask, because there's a fundamental fallacy that technology always equals progress. We call this the tech progress narrative, right?

• 8:16Speaker 00

  Yes.

• 8:17Speaker 02

  Now, it's important to note, we have to first agree that I would prefer to live in a world with Novocain rather than without Novocain.

• 8:23Speaker 00

  Yeah.

• 8:23Speaker 02

  I think you would too.

• 8:24Speaker 00

  Yeah.

• 8:24Speaker 02

  I'd prefer to live in a world where we have access to information and the technology that produces the printing press and information that way. Yes, I'd prefer to live in that world. There are all sorts of technologies that we do want. My background is seeing how one of the most significant technologies of the last decade went wrong. I was a designer that assisted Google, and my job was to study the attention economy, which is how essentially Apple and Google were hosting the psychological habitat of all of humanity in the form of how they, you know, designed and make these things work.

• 8:57Speaker 00

  Yes.

• 8:57Speaker 02

  How do notifications work? How does the home screen work? What are app developers competing for? What are they allowed to do? All of that results in a psychological habitat that has an increasing footprint over the global consciousness environment of all of humanity every day. It's like you've got 3 billion human social primates with a jack in their brain, every day influenced by decisions by a few thousand people in Silicon Valley shaping what that habitat represents. And it went horribly wrong, despite good intentions. There is the possible of that technology, that it was gonna give everyone a voice, give everyone access to the best information. We're gonna have the most enlightened society we've ever had.

• 9:36Speaker 00

  That was the promise.

• 9:37Speaker 02

  And that was the promise.

• 9:38Speaker 00

  It made sense to me.

• 9:39Speaker 02

  And so what this shows us is how do we get better at discerning between the possible of a technology and the probable based on its incentives? So the way that in 2013, so I went to Stanford, I knew the, I studied computer science, I knew the guys who started Instagram, and a lot of my classmates, like when I was at, in my computer science classes in the year 2005, 2006, I'd be in these classes, and suddenly, like, more and more of the seats would be empty during class, because basically, my friends in the class were starting to get jobs down at Facebook down the street. And it was literally the case. I mean, there were people who were in the class, and then suddenly, they just weren't showing up because they were down the street working at this technology company. And so because I saw this firsthand, I saw how good people with good intentions, who thought they were gonna

• 10:22Speaker 02

  make the world a better place, make the world more open and connected, was the original motto, if you remember, of Facebook. And that created a global societal catastrophe. As you said, we have the most anxious and depressed generation of our lifetimes. Jonathan Haidt's book, The Anxious Generation, critical thinking skills are down, attention spans are down, polarization, tribalism, all of this came from

• 10:44Speaker 02

  Not the malicious intent of these 20-year-olds who are working at Facebook, but the bad incentives, and I repeat this quote often, but I think it's so fundamental, that Charlie Munger quote, you know, it was Warren Buffett's business partner. Yeah. If you show me the incentives, I'll show you the outcome. Yeah. Were the incentives of social media companies to make the world more open and connected, or to improve mental health of young people? No, their incentives were maximizing how much attention they get because the business model of advertising. And share price. Who's share price? Share price is directly correlated to minutes spent, frequency of usage, duration of usage. So what you're going to get are of all, imagine a spectrum of all the ways you could design technology.

• 11:28Speaker 02

  And over here, you've got a bunch of very thoughtful design decisions about, you know, keeping people, helping people connect with their friends in real life, with their families, hosting community groups, church groups, reading clubs, sewing clubs, hiking groups. And you've got design of technology that was going to steer people back into that. Versus over here, you've got a bunch of designs for social media that just keep people glued on screens by themselves with their esophagus compressed, their neck extension, you know, going like this, and basically doom scrolling for eight hours a day. Which one of those two worlds, which one of those design choices is going to be better for shareholder price?

• 12:01Speaker 00

  So drinking water with electrolytes makes a huge difference. When you want to stay healthy and hydrated, you should do that. So for the best option you can buy, we recommend Relight. Relight starts with unrefined sea salt, real salt, which is completely natural, zero sugar, no artificial sweeteners, colors, flavors, and no chemical preservatives. The ingredient list is very short, and you can actually pronounce everything on it without a degree in chemistry. If you start adding Relight to your water, we believe you will feel different very quickly, way better. No sluggishness or afternoon crash, just more energy, better workouts, all the benefits that come with true hydration. And Relight tastes awesome. Mango flavor, pina colada, come on now.

• 12:45Speaker 00

  So if you're ready to give Relight a try, visit redmond.com/tucker. Use the code TUCKER for 15% off your first order. Redmond.com/tucker, code TUCKER, 15% off your first order. Or you can click the link in the video description, or scan the QR code on your screen.

• 13:01Speaker 02

  So the point is, first of all, there's actually a hidden optimism here. It's so discrediting of markets and of market theology. Well, it's about what are markets. Give people what they want until they die? Yeah, well, the question is, are we giving them what they want or what they can't help but look at? Well, that's. With an asymmetry of information where the designers at the tech companies know exactly how your, your brain works, and they manipulate the design choices to keep you scrolling for as long as possible. So this is all in the Netflix, most people know me and our work through the Netflix documentary, The Social Dilemma. Yes. And we had, you know, a lot of the insiders coming out who said, we know what this is doing to people. And in that film, it talks about the history where I was at Stanford at a lab, or part,

• 13:42Speaker 02

  part of a class called the Stanford Persuasive Technology Lab, professor named BJ Fogg, and where engineers were taught how to do persuasion techniques that theoretically be things that are good, like, how would we persuade you to go to the gym more often? And how would technology play a positive role in getting you to go to the gym more often? Or you've got a signup form, and LinkedIn can put a progress bar in the top right saying, hey, you filled in this much of your LinkedIn profile, but we want you to come back and fill in more. So we'll put a progress bar in the top right saying you filled in 60%. It sounds really stupid and dumb, but if you put a progress bar in the top right-hand corner, that actually is very persuasive in getting people to fill out the rest of it. And most people don't think about this. If I put a, a design feature in WhatsApp that says you were on, last online, you know, three minutes ago, that changes the whole messaging environment.

• 14:27Speaker 02

  Suddenly, when you message people, you know that the other person knows that you were online three minutes ago, which means you feel obligated to respond to them. Yes. Because your public, your, your availability and onlineness was visible and made transparent to someone else. But that was a design choice. It could have been designed totally differently. We don't have to put a progress bar on the LinkedIn profile. We don't have to say you were last online five minutes ago and create social pressure. So the point is, technology isn't destiny on its own. The way we design it creates different societal outcomes. And because of the business model of maximizing attention and engagement, we got only the

• 15:08Speaker 02

  side of design choices that resulted in this, you know, more addicted, distracted, polarized, sexualized, screwed up, doom scrolling, brain rotted society that we are now living in when you go outside. And from my perspective, in 2013, seeing those incentives, it was like, oh shit. Yeah. Because you could just see the trajectory before it happened. In 2013? In 2013, yeah. I mean, so I, so by the way, I did the tech entrepreneurship thing. I raised venture capital. I'm a builder of technology. I understand the kind of normal, you know, I did the Stanford Entrepreneurship Program, the same one that Sam Altman did, by the way, called the Mayfield Fellows Program. So I came up in a cohort of some of these people, but I could see it from a different

• 15:53Speaker 02

  angle. And for the company that I started, I was under those. Why were you able to see it? Uh, well, I wasn't able to see it at first. Part of it was actually, I relate to their, the blind spots, which is, you've talked about this with your own work, that when you have a salary, you know, you can't something, you can't question something that your salary depends on you not seeing. It's true. So up in Sinclair. Yeah. And it wasn't so much my salary, but the business model being the kind of salary of the company. If the business model is maximizing time and engagement, you come up with ways to justify that that's a good thing. Of course. Right? And you can only have the ethics that you can afford underneath that perverse incentive. So you start to see things in terms of what's compatible with those incentives. Oh, we are making the world more open and connected.

• 16:36Speaker 02

  Oh, small, medium-sized businesses were able to reach so many more people on Facebook and Instagram now. Think of all of the new revenue and customers that we're generating because we're so good at routing attention. And it's not that those are false stories. It just hides the, the shadow. There's almost like a Jungian shadow. Right. For technology, like, yes, you created these things, but you also have this massive shadow of creating a societal catastrophe at the same time. And I think there's a tendency to try to do moral arithmetic of like, do the upsides minus the downsides equal net positive? It's like this very simple moral arithmetic. And the answer is, don't brain rot and destroy the culture of a population and create a dumpster fire of your politics, your tribalism, the mental health of a generation. We don't have to do that.

• 17:20Speaker 02

  And so the, I guess the point here is, yeah, so I started this company. I was under the incentives of maximizing time and engagement. Actually, our business model is basically telling like, a Washington Post, or New York Times, or The Economist, or Fox News, we can increase the amount of time people spend on their website by giving them more information without leaving the webpage that they're on. That was essentially the value proposition. It was like this widget that basically let you say, I want to learn more about that. Tell me more. And it would give multimedia and background information, voting records, and videos, and maps, and all this stuff on the page of Fox News without leaving Fox News. Right. So, sounds like a great proposition. Definitely. But then the entire company became based on maximizing how much time people spent there.

• 18:05Speaker 02

  And so I, myself, was kind of in this inner conflict. And I went through that. It's actually funny, I had a, a group of friends who also had small tech companies. We're all about the same age, we're probably like 27, 28, only studied computer science, very reflective of the culture of the people there. And we had a thing called Doubt Club. And the point of Doubt Club, like Fight Club, is you can't talk about Doubt Club. And the point is that as a founder of a tech company, you don't have a space where you can essentially talk about your doubts about what you're doing. Is this even good for the world? So we talked about our doubts about what we were building, doubts about what this meant for our lives, and doubts about whether we were actually doing a good thing. And that led me to basically realize that we needed to let go of the company.

• 18:50Speaker 02

  It got acquired by Google. And so I landed at Google through basically reckoning with the conflict of incentives that I was under. And then it was at Google that I saw how a much bigger ecosystem, obviously, the entire, all the social media platforms were under those same perverse incentives. So I think that gave me an early insight into how this would go wrong. And I tried to warn my colleagues at Google in a 2013 presentation saying, you know, this small group of basically 100 designers working at three companies are designing, again, the psychological environment for all of humanity. And if we don't get ahead of the incentives, this is going to create an arms race for attention and engagement, and design choices that will become a race to the bottom of

• 19:33Speaker 02

  the brainstem to psychologically manipulate human nature, and then lead to accidentally this kind of brain rotted society. So that's how. How did they respond when you said that? So I made this, this slide deck, and I sent it, I actually felt nervous making it because I was criticizing, you know, and it wasn't meant to be adversarial or claiming that Google was evil. I'm not like an attention-seeking person. I wasn't trying to be angry at anybody, I was just naming a problem. And I sent it to about 30 of my smartest friends at Google. And I thought surely other people were working on this. Like, Google's this incredibly brilliant company with the smartest PhDs, like, they're going to say, that's nice kid, but like, we've got this handled, like, we thought about this problem.

• 20:17Speaker 02

  To my surprise, when I came back the next day after only sending it to 30 people, in the Google slides, you can see the number of simultaneous viewers on the top right-hand corner of people looking at like a Google Doc or a slide deck. And there was 150 simultaneous viewers, but I only sent it to 30 people. So what had happened is it started getting spread virally throughout the whole company. And soon there was thousands of simultaneous viewers. And I learned that in the next day, it had gone up to Larry Page, who's the co-founder of Google, and there was like a meeting about it. And so all this was to say that it was received in a way that I think people felt this is a real problem. And I got emails from all around the company, like, yeah, we should do something about this, this is a real thing we've got to manage.

• 21:00Speaker 02

  And that's how I landed as a Google design ethicist, studying how do you ethically design technology? And that's what became kind of my core ethos in general of how do you do technology in a way that's compatible with the health, well-being, and flourishing of a society? And watching in 2013, these bad incentives, it was like having, instead of like post-traumatic stress disorder, it was, my friend's joke, it was like having pre-traumatic stress disorder. Because I saw where this was going. And I think a lot of what motivates me in the conversation that I know we're going to

• 21:39Speaker 02

  have about AI is, I don't want to see us make a total disaster again. And I've watched it happen.

• 21:47Speaker 00

  Well, here's some good news in a sad time. Preborn has saved over 50,000 babies this year alone. How'd they do that? Well, here's a direct quote from the president of Preborn. Because of you, we can invite mothers considering abortion into a Preborn Network clinic to meet their baby. And when she has that close encounter, she's 80% more likely to choose life. That's true. And if it wasn't clear, you refers to you to the thousands of Americans who support Preborn every day. Support from people empowers Preborn's lifesaving work by making that first encounter between a mother and her baby possible. Once people have the knowledge, they're much more likely to make the life-affirming decision. Preborn

• 22:30Speaker 00

  puts the gospel into action across this country, surrounding mothers with hope and love, not judgment, but hope and love, and providing free lifesaving ultrasounds. You can join Preborn as they work to rescue the next 50,000 babies, just $28, sponsors an ultrasound and can help save a baby's life. So to do that, dial pound 250 and say the keyword baby, pound 250 baby, or you can give securely at preborn.com/tucker, preborn.com/tucker, and gifts are taxed.

• 23:01Speaker 03

  Bonjour, compadre. It's the Price Line Negotiator. How do I negotiate so many great travel deals? My greatest gadget, the Price Line app. It's got hotel deals, flight deals, rental car deals, all of those deals in a bundle, deals, game day deals, concert trip deals. No one deals more deals than Price Line. Hold your horses, there's more. The app lets you filter hotels by neighborhood, vibe, star level, and amenities like pools and spas and beachfronts. And, wait, I'm not done. Stop cutting me off.

• 23:31Speaker 04

  Fall tastes better with better ingredients from Whole Foods Market. No added growth hormones or antibiotics in their meat department means slow-braised short ribs are even more satisfying. All seafood is wild-caught, like their vibrant sockeye salmon, or verified, responsibly farmed. And all sweets, including fall-themed cookies, are made without high-fructose corn syrup. Spice up fall at Whole Foods Market.

• 23:58Speaker 00

  So of the other members of Doubt Club, how many are still in the business you were in when you were?

• 24:09Speaker 02

  None of them. They're all, they're all doing other things now.

• 24:11Speaker 00

  Really?

• 24:11Speaker 02

  Yeah. Which is to say, once you reckon with something, you know, you can only have the ethics you can afford. So you find a different way. You find a different business model. And people hearing that might say, oh, therefore now your business model is warning people about technology and doing speaking engagements. I just want to be clear for the record. I don't make money from speaking engagements. All the money goes into our nonprofit. I don't actually give speaking engagements very often. I just care about and have for the last 10 years with our nonprofit, the Center for Human Technology, think about, how do you do design for technology in a positive and ethical way? How do you change the incentives? How do you do policy and litigation that changes the incentives, like the Meta lawsuit of $17 billion that happened

• 24:50Speaker 00

  Yes.

• 24:51Speaker 02

  three, four weeks ago? That was catalyzed by a bunch of the Attorney General, attorneys general, watching The Social Dilemma in 2021. So I've seen how, if you have dedicated advocacy, if you, if you show people the nature of a problem, there can be a response. Now, that one took way too long and way too late. But I think that now with AI, it's happening exponentially faster. I mean, a million times faster. Because AI is different from other technologies because it accelerates its own progress. We should probably just say why that's true. So like, if I invent a nuclear weapon, I did the nuclear physics, I assembled the thing together, I've got the uranium, plutonium, blah, blah, blah,

• 25:33Speaker 02

  I create the thing. But once I build a nuke, it doesn't accelerate or invent better versions of nuclear weapons.

• 25:39Speaker 00

  No.

• 25:40Speaker 02

  It doesn't do that.

• 25:41Speaker 00

  Doesn't just like

• 25:42Speaker 02

  Last, last time I

• 25:43Speaker 00

  spontaneously create a hydrogen bomb?

• 25:44Speaker 02

  No, no. But if I'm inventing intelligence, meaning the ability to do pattern recognition, goal achieving, training on all human language, I can use that intelligence to take the code that trained AI and make it way more efficient. I can take the intelligence I've created and say, point it at chip design. Say, take this chip design and make it more efficient. And it does that. So now you have a more efficient chip design, more efficient code, training the next more powerful version of AI, and then that next more powerful

• 26:16Speaker 00

  becomes exponential.

• 26:17Speaker 02

  version. And it becomes exponential.

• 26:18Speaker 00

  Yeah.

• 26:19Speaker 02

  So the way to say is, nukes don't make better nukes, but AI makes better, faster, more crazy AI. Which means that you're actually, in the words of one of the OpenAI founders, Greg Brockman, who at a conference once told me, AI is like a double exponential, because it's an exponential that exponentiates

• 26:33Speaker 00

  Exactly.

• 26:34Speaker 02

  itself. It's not a mathematical term, but it's a metaphor for people to think about.

• 26:38Speaker 00

  Um, just really quickly, just to linger a minute on your tenure at Google.

• 26:46Speaker 02

  Yeah.

• 26:47Speaker 00

  So what, you became an ethicist, they promoted you to this job.

• 26:50Speaker 02

  Well, so I thought I was going to get fired, or had there was a chance I would get fired from this letter, even though it wasn't adversarial, I didn't know it'd happen. And just to say, to Google's credit, rather than that, I basically said I wanted to work on this problem. And I, I thought there were opportunities within Google to change the design of Android, the operating system running on half the world's phones, and to change the design of Google Chrome, which is a web riser running on most people's computers. And if you change those two product surfaces, you could have a pretty big difference on the attention economy. So instead of leaving, I thought, well, maybe I can stay and work on that research.

• 27:23Speaker 00

  But if you were to succeed in that mission, you would make the company less profitable.

• 27:27Speaker 02

  Correct.

• 27:28Speaker 00

  And so like, how could that ever succeed?

• 27:29Speaker 02

  Well, and this is where I was naive. So I had a second round of naivete where I thought that maybe if I was just really persuasive, I could talk to the people running those products. And I did. I had lots of meetings with the people.

• 27:42Speaker 00

  I bet. I bet you do.

• 27:43Speaker 02

  I spent three years trying to change Google from the inside. But I mean, that was incredibly naive of me. And that's why I left and why for the last 10 years, you know, I have, I mean, I don't get sleep. I don't have much of a life. I, I really focus on these issues because I see how big they are and how consequential they are and how irreversible in many ways and senses they are if you don't get it right ahead of time. And so a lot of what drives me is, how do we make sure that technology, to the degree it's eating the world, per Marc Andreessen, Marc Andreessen, the famous founder of Netscape, who I know you know, said in 2009, "Software is eating the world." But we don't regulate software. So when software eats the world, it's like deregulating or sort of taking

• 28:28Speaker 02

  away, not that regulations are always good, but it's sort of reconstituting that part of society.

• 28:34Speaker 00

  Well, so the idea in the United States, in the West, I think, but is that markets are self-correcting.

• 28:39Speaker 02

  Right.

• 28:40Speaker 00

  And that if left untampered with, they will always reach a place of goodness.

• 28:46Speaker 02

  And when you're just selling a product and the features of the product's harms are part of what consumers value, and if I don't like that product that's causing harm, I'll use this other product that's not, that works. And markets are great for that. They're the most incredible tool. But they don't deal with problems of monopolies. So if I have a Metcalfe monopoly, meaning, you know, Metcalfe's law is basically the power of something grows with the exponent of the network. So social media was a Metcalfe monopoly. Uber's a Metcalfe monopoly. Once I get all the drivers on Uber, it's going to be hard to switch to another app that doesn't have as many cars on it, doesn't have any drivers on it. So I'm going to use the one app that has all of them on it. I'm going to use the one social network that has everybody on it. And I can't just switch from one social network I don't like that's doing things in a toxic way to another one

• 29:28Speaker 02

  that's doing things in a non-toxic way. And in fact, if I build a social network that doesn't do attention hijacking, race to the bottom of the brainstem, manipulative design, people aren't going to use it very much.

• 29:38Speaker 00

  Well, exactly. Then, right.

• 29:40Speaker 02

  And there's been a lot of people's attempts to try to create alternate social media. And then when you go on there, it's a ghost town. There's no one there.

• 29:46Speaker 00

  So I think what you're saying is, when the product works well, it destroys people.

• 29:50Speaker 02

  Well.

• 29:51Speaker 00

  And the only way to keep it from destroying people is to work less well, less efficiently.

• 29:55Speaker 02

  Or you have standards written from above that say, hey, no one can do this manipulative design. Right? And that's how we could have prevented it. You know, I've met so many, I was just at a conference two weeks ago called the Pro-Human Assembly, and we'll talk about it. It's the pro-human movement, I think, is the frame for like, how do we have a humanistic future with technology? And I met, for the first time, actually, some cases for the first time, and sent emails back and forth with them, but several of the parents who have lost, um, lost their kids from, from, from this stuff. And, um, uh, you know, when you, when you meet parents who've lost

• 30:32Speaker 00

  Suicide?

• 30:33Speaker 02

  Uh, well, to bullying, to, um, you know, there was a filter on Snapchat called a YOLO filter that basically overlaid on the video you're posting, the number of miles per hour that you're going. And so they basically created a filter that incentivizes people to take videos at a high miles per hour.

• 30:49Speaker 00

  Actually?

• 30:50Speaker 02

  Yeah. Yeah. Um, you know, or Snapchat streaks where they attach, you know, the streak method, like, oh, you worked out the gym four days in a row. If you work out tomorrow, it'll be five days in a row. And then once you have 300 days in a row, it's really hard to be taking a break from the 300 days in a row that you've done that activity. So that's a persuasive design feature. But what if I do that to a 12-year-old who's communicating with their best friend? So I've got a 12-year-old, or even a 10-year-old, they're just chatting, text messaging with their friend, and they've been communicating for 300 days. Now they literally can't get off the app. Because every single day, they've got to send them a message or they lose this thing. And it's just like, you know, cigarette companies would use cartoon characters to addict kids. That was a very minor form of

• 31:34Speaker 02

  persuasion. But using Snapchat streaks on 11-year-olds is very persuasive. And so everything from cyberbullying to

• 31:45Speaker 02

  There's just a million ways that it happens. You know, when you meet, when you meet the parents, it's like, and you just know you're not allowed to do that. You're not allowed to fuck up young kids and families. And so I think a lot of this for me is a protective instinct of in seeing real consequences for real people and watching society get totally distorted and watching culture and tribalism and politics just get off the rails, which are largely amplified by social media. In watching all that, it's like I never again, right? You go through a catastrophe, you make sure it doesn't happen again.

• 32:27Speaker 00

  Owning a home has always been the core of the American dream. It's the asset that families work their lives to build, a place of stability, security, and actual wealth. But too many homeowners right now are stuck swiping credit cards to cover groceries, paying a survival tax of 20% interest or more. So we're against debt. We always say that, but you can't buy a house without debt right now. Almost anybody has to take out a loan. So the question is, who are you going to get a loan from? Why are you handing your money to big banks when you can keep it for your family? Our friends at American Financing have a better way. They're helping homeowners tap into their hard-earned equity. They're supporting Americans looking to fulfill the American dream of owning a home. American Financing saves its customers an average of 800 bucks a month.

• 33:11Speaker 00

  That's nearly $10,000 a year. It's not just a loan, it's a total financial reset. And so American Financing is worth looking at. They're called America's Home for Home Loans for a reason. Find out more at 800-685-5696, 800-685-5696, or visit americanfinancing.net.

• 33:33Speaker 05

  Buy smart, drive for less, and trade for more. That's Toyota affordability. And it all starts with a great deal from your local Toyota dealer. Make a smart buy on an all-wheel-drive Camry, Corolla Cross, or get a Corolla sedan packed with the latest features, advanced tech, and head-turning style. Or take a test drive in two of Toyota's most popular SUVs, the spacious, family-friendly Grand Highlander and adventure-ready RAV4. And with so many hybrid and plug-in hybrid models available across the lineup, plus efficient all-electrics like the BZ, BZ Woodland, and all-wheel-drive CHR, you can drive your way and skip the pump. And long after you leave the lot, your Toyota keeps delivering. With legendary resale value, built on a reputation for quality and reliability, so when it's

• 34:18Speaker 05

  time for your next Toyota, you could trade in for more. From a great deal today to a great value tomorrow, a Toyota is right for you. Shop toyota.com for deals and details, or stop by your local Toyota dealer today. Toyota, let's go places.

• 34:33Speaker 06

  Ryan Reynolds here for Mint Mobile. I don't know if you knew this, but anyone can get the same premium wireless for $15 a month plan that I've been enjoying. It's not just for celebrities, so do like I did and have one of your assistant's assistants switch you to Mint Mobile today. I'm told it's super easy to do at mintmobile.com/switch.

• 34:52Speaker 01

  Upfront payment of $45 for three-month plan, equivalent to $15 per month required. Intro rate first three months only, then full price plan options available. Taxes and fees extra. See full terms at mintmobile.com.

• 35:03Speaker 00

  Net slash Tucker. You said that even pretty recently, you weren't as worried about AI's progression as you are now.

• 35:13Speaker 02

  Yeah, well, so to be clear, no, I've been worried about AI for the last three years, and I'll briefly explain. So I had been very busy on the social media topic for seven years.

• 35:25Speaker 00

  Did you make any hits?

• 35:26Speaker 02

  By the way? Well, I mean, The Social Dilemma was seen by something like 100 million-plus people in 190 countries and 30 languages. It's the most popular documentary on Netflix for several years. And the attorneys general watched it, and it helped amplify and create this lawsuit that just finished just a week ago, I mean, a month ago, for $17 billion on Meta, and they're going to change some of their design practices for kids. And it's a lot of too little, too late, and frankly, the fines should be trillions of dollars. Even Mike Milken, the venture capitalist, I mean, the great capitalist, says that if you account for all the harms of social media, it would be multiple trillions of dollars of damage to of social capital to the social fabric.

• 36:06Speaker 02

  That's the actual price of this thing from the toxic design patterns. And so, you know, we did get employee activism inside the companies. We did see Frances Haugen, the Facebook whistleblower, come out. There's a movie about her that just came out this last week called The Social Reckoning. You know, she leaked 10,000 pages of Facebook research documents showing that they knew, just like the tobacco companies, that it was making kids, making body images issues worse for one out of three girls, driving mental health problems. And due to her bravery, we now have these lawsuits that are underway and some that are completed that are shifting some of that. But in many ways, we're just, it was very late. And so it was very difficult working on that for so long without a lot of traction, even though it was the right thing to do.

• 36:49Speaker 02

  With AI, the reason we switched into it is in 2023, I got phone calls from people inside of AI labs. And I had known about OpenAI, I had known about, you know, a couple of the AI companies and knowing that they were making some progress. I'd see Elon's tweets and Sam Altman's tweets saying like, people aren't ready for this. Like, there's some real stuff coming. But it wasn't until like January of 2023, and I got a call, and the person said, basically, you need to stop what you're doing and completely pivot and switch to waking up the world to AI, because I have seen jumps in capabilities. This is when GPT-4 was coming out, this is OpenAI's ChatGPT-4, that are going to stun people and the institutions aren't ready.

• 37:33Speaker 02

  And so basically, because we'd had all the success with The Social Dilemma, we had, we had a lot of connections between national security, governments, culture. And so we basically did this presentation called The AI Dilemma and woke up the institutions. It was like getting a phone call from the Robert Oppenheimer's of the world. So imagine you're getting a phone call in 1942 and from this thing called the Manhattan Project, and this physicist says, no, we're inventing something that's totally different. And you're like, but don't we already have bombs? Like, we are, you don't really understand how significant something's going to be. And so you get a phone call from that person who's saying, no, no, no, this is a different kind of thing. And it was getting that phone call that I realized that there was a whole bunch of

• 38:16Speaker 02

  risks that our society was, was going to face because of this arms race dynamic. And that's what they basically said is the corporate race between these major AI labs, OpenAI, Anthropic, Google DeepMind. The race is out of control. Everyone's badly incentivized. You have to help wake up the world. And so that's when we switched to AI.

• 38:37Speaker 00

  The race. So the race, as you said at the outset, is not simply between the American companies developing this technology, but between the US and something called China.

• 38:49Speaker 02

  Right.

• 38:53Speaker 00

  My instinct is that that's, well, my instinct is that we're probably not going to be trending no matter what we're telling ourselves, but leaving that aside, my instinct is that that's less real than they're saying, but you tell me.

• 39:05Speaker 02

  Well, it's funny, I haven't shared this publicly, but I have heard a story, so it's, you're not going to be able to trust me because I'm not sharing who it's from, but I heard a story from someone who was in the know that was in the government. I think it was, they said it was 2015 or 2016, and it was one of the major AI labs. So this person said, China's building an AGI project, and US has to build one. And then years later, this person ran into this person and said, how did you know that China was building an AGI project? And the person flat out replied, I didn't, I just wanted to tell the US government that so they would fund us with a bunch of money.

• 39:44Speaker 00

  And did they get the funding?

• 39:45Speaker 02

  I don't know, actually. But the point is that they have been stoking the fear of China from the very beginning, even when there wasn't.

• 39:53Speaker 00

  You've got to have a, you've got to pay double for water and energy because you need a data center in your town because China.

• 39:59Speaker 02

  That's right, exactly. You got to screw up your property values, you got to have noise pollution, give everybody like extra, you know, water pollution, you know, cancer, et cetera, because of China. Yeah. And by the way, your tax dollars in many cases are subsidizing that data center, which is going to replace your job.

• 40:16Speaker 00

  So you're 100%. I don't think there's a data center in the United States that doesn't take tax dollars.

• 40:21Speaker 02

  Yeah. And I think when people see that data center, by the way, it's like you should see a big box that's designed to take your job.

• 40:29Speaker 00

  Yeah.

• 40:30Speaker 02

  It's designed to create competitive capabilities of, you know, if you're worried about immigrants crossing the border, AI represents like millions of new digital immigrants that are going to be better, not just at the regular work in our economy, but the most premium sort of well-trained cognitive work in our economy, programmers, lawyers, across the board, and then eventually robots that will be able to do physical labor. And so if you're worried about that, you should be worried about the trajectory of AI being designed explicitly to replace your job, because the companies will say, we're just giving you a chatbot. It's a blinking cursor. It's like a tool that tells you, hey, why is my washing machine broken? And boom, you get the answer. Why is my kid burping? Boom, you get the answer. It's helpful. That tool is helpful.

• 41:15Speaker 02

  But is that what their business model is? Back to Charlie Munger, what are the AI companies' incentives? Think about the billion, hundreds of billions of dollars that have been raised. Is there incentive to capture $20 a month from every person, which is how much it costs for a ChatGPT subscription? Or is the thing that's going to pay back those hundreds of billions of dollars to capture everyone's salary from everyone's job in the future?

• 41:37Speaker 00

  Well, of course, the latter.

• 41:39Speaker 02

  Of course, the latter. So it's a race to replace human labor, not a race.

• 41:42Speaker 00

  These are not chatbot companies.

• 41:44Speaker 02

  These are not chatbot companies. That's just a veneer to provide an early tool, while the real race is to build these godlike digital brains that are able at doing everything at a superhuman level and outcompete us in programming, law, biology, science, physical labor, military strategy, planning, like everything.

• 42:05Speaker 00

  But so, okay, I should just say my bias, which is based on a lot of knowledge and experience, which is the people who run our country are really stupid, despite their credentials. So it's possible they haven't thought this through, but it's so obvious that when you do that, you're going to get a violent reaction from the population. If you take the jobs of factory workers, you get opioid abuse. If you take the jobs of lawyers, you get revolution, okay?

• 42:30Speaker 02

  Right.

• 42:31Speaker 00

  The knowledge economy workers don't want their jobs taken, and they have high expectations, and they have.

• 42:38Speaker 00

  Low self-control. So you're gonna have some kind of violent response.

• 42:43Speaker 02

  That's right.

• 42:44Speaker 00

  What's their plan for that?

• 42:46Speaker 02

  Well, first of all, it's important to note they're selling a story that this is gonna be good for everybody. GDP growth is gonna go up. It's gonna be the most prosperous new golden age.

• 42:54Speaker 00

  Growth.

• 42:55Speaker 02

  But this is like what happened. And you've spoken about this extensively, just that they promised us a story with globalization that will outsource our jobs to in manufacturing to these other countries, but it'll make us so abundant because we'll have cheap goods coming in where labor is so much cheaper and it's gonna benefit everybody. And how did that story work out?

• 43:14Speaker 00

  You won't get a million people living on the streets of major American cities smoking meth.

• 43:18Speaker 02

  Oh, wait. Oh, you did. Okay. Right, exactly. And so a meme here is like Detroit everywhere, or, you know, you know, these places everywhere.

• 43:25Speaker 00

  Right.

• 43:25Speaker 02

  Right. And so, um, but the working class will is will put up with it.

• 43:32Speaker 00

  Yeah.

• 43:32Speaker 02

  But it's kind of a devil's bargain.

• 43:33Speaker 00

  But the white collar class will not put up with it. They'll go crazy.

• 43:36Speaker 02

  Well, this is an important point. So while the factory workers and manufacturing workers, you know, were further away from Washington DC and New York.

• 43:45Speaker 00

  Yeah.

• 43:45Speaker 02

  The political class.

• 43:46Speaker 00

  Exactly.

• 43:47Speaker 02

  But this time around, to your point, you have lawyers who are gonna be affected, and they are very close to the political class.

• 43:53Speaker 00

  Exactly.

• 43:54Speaker 02

  There's also a bigger drop, right? So instead of going from, I don't know, $80,000 a year to $35,000 a year or something like that, you're going from, I took out student, a student debt, $200,000, $1,000 of student loans, and I'm going from my job of $250,000 a year as a lawyer to now, whoops, what am I doing? Doing some other kind of cognitive work. Now, people will always say, but look, if Marc Andreessen was at this table, he'd say, look, uh, we've always worried about automation. Economists have named this phenomenon all the time. New technologies come along, agriculture, you know, 200 years ago, all of us were farmers. Now, almost no one's a farmer. We always find something new to do. What's different about AI? What's different about it? It's that AI is built to be able

• 44:36Speaker 02

  to do all forms of cognitive labor, like lawyers, programmers, you know, factory workers, everything, robots. It does everything. And for whatever it doesn't do yet, that just becomes the next incentive, Charlie Munger's style, to go train in that field. Like, have you seen the, um, these hand farms in India, arm farms? Have you seen this?

• 44:56Speaker 00

  No.

• 44:56Speaker 02

  So this is, it was both a popular job in LA about six months ago, and also all throughout India, essentially an economy where you strap a GoPro camera to your head, and you look down, and you do tasks, you sew things, you clean dishes. So you're basically training, you have this, essentially, the number one job is training your replacement.

• 45:16Speaker 00

  It's kind of nice to see Indians training their own replacements.

• 45:20Speaker 02

  Well.

• 45:21Speaker 00

  Sorry. Excuse me.

• 45:23Speaker 02

  It's, it's.

• 45:23Speaker 00

  My immigration joke. Um, I guess the difference is, though, and I've had this conversation with a number of people involved in this transition, and they will say exactly what you just said, which is, we've been through this before.

• 45:34Speaker 02

  We always figure it out.

• 45:35Speaker 00

  The agricultural economy.

• 45:36Speaker 02

  Exactly.

• 45:37Speaker 00

  It's gonna turn out great because it always does.

• 45:39Speaker 02

  Yeah.

• 45:40Speaker 00

  But the difference here is that the people whose jobs are being replaced imbue their jobs with meaning.

• 45:46Speaker 02

  Yes.

• 45:47Speaker 00

  And those jobs, so if I'm working a metal stamping machine, I don't think that what I'm doing is saving the world. I'm doing it to pay for my bass boat and my life outside work. But if I'm a nonprofit executive, I really feel like what I'm doing is inherently important, and my identity is wrapped up in what I'm doing. So the promise of more idle time is hell.

• 46:07Speaker 02

  Well, also, where are you gonna get your future income?

• 46:10Speaker 00

  But leaving the income aside, even if we have, you know, guaranteed basic income or whatever, there's like, the welfare state becomes, you know, the only source of income, and it's generous, it still doesn't make up for the meaning.

• 46:23Speaker 02

  Correct.

• 46:23Speaker 00

  My identity has been lost in this.

• 46:25Speaker 02

  Identity, dignity.

• 46:26Speaker 00

  And that's why I become a revolutionary.

• 46:28Speaker 02

  Exactly. Yeah.

• 46:29Speaker 00

  Most people don't like thinking about life insurance. It's one of those topics you tend to kind of put off. But if you're married, you've got kids, you've got people who depend on you, you probably should think about what would happen if you weren't around. And that's why we've partnered with Ethos. They've made getting life insurance super easy. You can do the whole thing online. You don't have to sit down with an agent or go through a medical exam. You get a quote in seconds, apply in just a few minutes, and in some cases, you get coverage the same day. You just answer a couple of simple health questions. You get up to $3 million in coverage. Some policies can be yours for as little as 30 bucks a month. So it's a good deal. If you've been putting off life insurance because you thought it would be a very complicated, expensive process, you should probably check out Ethos. Take a couple of minutes, get your free quote online at ethos.com/tucker.

• 47:14Speaker 00

  Ethos is E-T-H-O-S dot com slash Tucker. Application times obviously vary as do rates, but we think it's pretty good.

• 47:23Speaker 02

  So what's their plan for that? They hand-wave, as you've probably noticed. They don't have a plan. And by the way, you might say.

• 47:31Speaker 00

  They're idiots.

• 47:31Speaker 02

  Yeah. Well, and so they'll say, but if we don't do it, China will. But right now, does the, does US and China are racing to mutually assured political revolution right now? Because they have the same problem. They have a population decline problem, so they need to fill in the population decline with more robots, so they think they can manage it that way. But we have a shared interest in, the way I think about AI is it's like taking steroids that also give you cancer. So let me explain that. I take some GPT-6, I put it in my mouth as a country. Now, my whole country is running on GPT-6. So I just buffed up my muscles. I've got suddenly super programmers, I've got super military strategists, I've got super GDP growth, because I've automated stuff with robots. So my external muscles, like the GDP is like an external muscle of the US

• 48:14Speaker 02

  economy, that all got bigger. But at the cost of cancering identity, work, meaning the social fabric, right? It's a, it's a devil's trade. I think the theme here is, what does the devil do? They come up to you and say, I've got this great deal for you. It's gonna be great. You're gonna look great. You're gonna feel great. Everything's gonna be, and the other side of that trade is something you don't want. Do they actually think, since you've, I mean, you do this for a living, you talk to these people for a living, do they think that GDP is like the most meaningful measure of a society's health? I don't think they're thinking through it. I think there's a, there's a thing where it's persuasive to say it, because in the past, GDP growth going up, at least like through the 90s or something like that, meant, generally speaking, more prosperity for our society.

• 48:58Speaker 00

  Anyone who uses the term GDP is instantly written off in my head as a moron.

• 49:03Speaker 02

  Right. Well, even the.

• 49:04Speaker 00

  Like, I won't even listen to someone who cites GDP. GDP?

• 49:07Speaker 02

  I know, it's stupid. Well, but, so there is a geopolitical thing, which is that economic growth precedes geopolitical power. So if I've got a bigger economic growth machine, then I will be able to dominate other countries that don't. And so I think it's a power-seeking thing that is using as propaganda GDP growth to make it sound like this is gonna be good. Because, you know, through the 60s, 70s, 80s, 90s, GDP growth going up was associated with more prosperity for the economy, actually up until the 1970s, as we, as we know, when things changed.

• 49:39Speaker 00

  But it's not even an accurate measure of economic power.

• 49:42Speaker 02

  No, and even the.

• 49:43Speaker 00

  It's like an idiot's measure.

• 49:44Speaker 02

  Was it Simon Kuznets? I might, no, who's the one who came up with the GDP metric? He said, don't use this. He said, don't use this to measure prosperity of a country. This is just a, a simple metric. It's one thing you can use. You never optimize for it. We were talking at breakfast about how.

• 49:59Speaker 00

  You made such, you made such a smart point that I tried to memorize it, but were you, we're talking about how basically our ruling class is inadequate to managing our society. And it's not just true in our country, but true in many Western countries. And you said that is a product of optimizing for one quality.

• 50:16Speaker 02

  Yes.

• 50:17Speaker 00

  But, but you explain it.

• 50:18Speaker 02

  No, no, it's great. I mean, I think, so you, you and your show have talked about a lot of these topics, whether it's, um, why do we have morons running everything? Why do we have big box stores and the removal of aesthetics of everything? Um, and what it all comes down to is if I'm optimizing for a metric, like, I just want the cheapest goods everywhere. Well, then suddenly I just outsourced all my manufacturing to the other countries.

• 50:40Speaker 00

  Right.

• 50:40Speaker 02

  Including, by the way, my national security critical minerals and all this stuff that you wake up later and say, why did I do that?

• 50:45Speaker 00

  Yeah.

• 50:45Speaker 02

  But it was for the cheapest price. You weren't thinking through it in a multi-dimensional way.

• 50:48Speaker 00

  Yes.

• 50:49Speaker 02

  If I optimize for SAT scores, I get this like super smart, narrow disembodied, brilliant, like, quality of like these very specifically domesticated humans, but at the expense of all the other kinds of intelligence and sort of humanity and all that. And really, what that optimizes for.

• 51:04Speaker 00

  Wisdom, too.

• 51:05Speaker 02

  Wisdom, absolutely. If I optimize for like the cheapest use of 3D space, then I get big box stores that optimize revenue per square inch of, of, you know, unit, uh, 3D space, uh, and that the removal of aesthetics and the soul of a city.

• 51:20Speaker 00

  Yes.

• 51:20Speaker 02

  If I take a city like Sedona, Arizona, I remember visiting there during the pandemic, where they had a problem where so much of the, uh, real estate was actually just Airbnbs. So it's like a ghost town, where essentially every, to optimize the, the revenue in the, in the city, is everybody's just making money from, from renting their places.

• 51:37Speaker 00

  And all of a sudden, no one lives there.

• 51:38Speaker 02

  And all of a sudden, no one is there. So no one, and there's a famous quote from Larry Summers, like, does anybody ever washed a rental car? Right? Like, meaning that, yeah, if you don't have a skin in the game, if you don't have a stake in the city, in the parks, in the schools, in the libraries, in the churches, in the, in the local community, in the fabric, you're kind of, it's the vampirification of everything. It's like the big boxification of everything. And so I think that to sort of, you know, something you've talked about a lot is the kind of spiritual disease that we're seeing, which comes from this narrow optimization for one metric at the expense of the whole, and that includes GDP, right? And so if we want a rich society, other countries are better at managing this right now than the US, where you protect the sort of fundamentals of a

• 52:20Speaker 02

  society, the community, the trust, the intergenerational wisdom and knowledge transmission, um, you know, you invest in the arts and, and culture and education, healthcare, et cetera. But AI is like, we're actually talking, uh, about this, that this, this Nick Land quote that we'll maybe get into, if we just value competition and growth of a narrow metric, then it's like we are summoning AI from the future into the present. We are assembling the things that will make everything more efficient through AI. Because AI represents, I can do everything more efficiently than everything else. That's what it represents. And there's a great quote by, by Nick Land, um, who wrote an essay called, "What appears to humanity

• 53:03Speaker 02

  as the history of capitalism is an invasion from the future by an artificial intelligence space that must assemble itself entirely from the enemy's resources."

• 53:15Speaker 02

  Like essentially, we are assembling the tool, the machines that are being designed to replace and dehumanize and disempower our own humanity.

• 53:24Speaker 00

  We're selling the hangman the rope.

• 53:26Speaker 02

  That's right. You know, the future job is coffin builders. I'm, my job is to train the AI that's going to replace my job.

• 53:34Speaker 02

  Now, I'm not saying this, by the way, people are going to say, okay, you're a doomer, you're a dark person, you're trying to sell fear. That's not what this is about. This is about if we can see-

• 53:42Speaker 00

  Wait, you're the doomer? Did you, did you create this technology? I don't think so.

• 53:45Speaker 02

  No. And if you say, I'm trying to make money from it, think about the trillions of dollars that the people advancing the technology-

• 53:51Speaker 00

  I love the blame transfer. It's so funny. It's like, it's always like, if you, I, I catch you in the act of raping someone. And I'm like, stop. And you're like, shut up, you sexist. It's like, wait, what?

• 54:02Speaker 02

  Yeah.

• 54:02Speaker 00

  No, no, I'm not the criminal here.

• 54:03Speaker 02

  Exactly, exactly.

• 54:05Speaker 00

  You're the doomer, really?

• 54:06Speaker 02

  Well, and so you, you know, you have someone like Nate Sorensen on your podcast. You have me on, you have people who have a sincere care for the future. And all you're doing is saying, we have to see clearly, you know, there's a great quote by Charles Kettering, "A problem well stated is a problem half solved."

• 54:20Speaker 00

  Yeah, that is, I believe that.

• 54:22Speaker 02

  And, and you know, Einstein said that if I was given a problem, I'd spend 95% of the time defining the problem and then 5% solving it. So one of the drivers of our problems is this optimization for narrow metrics. Like Facebook maximizing this narrow metric of time spent, sounded like, oh, we're giving more people what they want, because otherwise, why would they be spending time there? But in the meantime, you've created cyberbullying, body dysmorphia, dyslexia, like brain rot, you know, you've destroyed a generation, all while the number was going up.

• 54:51Speaker 00

  Well, there's also a thing called addiction.

• 54:53Speaker 02

  Right, exactly.

• 54:54Speaker 00

  Which tends to act against something called free will.

• 54:56Speaker 02

  Exactly.

• 54:57Speaker 00

  And so, it's like-

• 54:58Speaker 02

  Yeah, exactly.

• 54:59Speaker 00

  This is why there is a wisdom gap above all in the West. Like, people are not thinking wisely.

• 55:05Speaker 02

  Correct.

• 55:05Speaker 00

  They're thinking in the way that an industrial designer thinks. Like, I need to build a machine that makes bumpers.

• 55:11Speaker 02

  And they're being domesticated to think in that way, because the system profits from people who think in terms of the system as it exists.

• 55:18Speaker 00

  Yeah, in terms of narrow specialties, totally disconnected from the whole.

• 55:21Speaker 02

  That's exactly right. And that's what it is. It's like narrow specialties disconnected from the whole. Like, there's a hole of what makes a social fabric healthy. There's a hole for what makes, forget climate change, just like a healthy environment. You have to have all these other features.

• 55:33Speaker 00

  Exactly.

• 55:34Speaker 02

  And, and it's like, if you're optimizing for, uh, you know, the eggs don't stick to the pan, then you make Teflon. And that, but you didn't realize you're also creating forever chemicals that give everybody in childhood cancers. Uh, and now the sperm doesn't stick to the egg.

• 55:50Speaker 00

  Yeah, the, the good news is it's easier to clean your pan. The bad news is the eggs are poisoned.

• 55:55Speaker 02

  That's right.

• 55:55Speaker 00

  Right.

• 55:56Speaker 02

  The eggs are poisoned and your sperm doesn't stick to the egg.

• 56:00Speaker 00

  Sorry, I'm just, it's also absurd. It just makes me laugh.

• 56:03Speaker 02

  Well, and it's, it's really like disheartening and depressing, but there's, as we talked about off camera this morning, there's something called, it's a framework I really like from a mentor of mine, that there's pre-tragic optimism, which is like, I'm going to kind of, I want to be an optimistic person. But that's like essentially a commitment without evidence. Like, I just, I need to be optimistic.

• 56:21Speaker 00

  Yeah, yeah, totally.

• 56:22Speaker 02

  That's, that's how, you know, I'm better off being optimistic.

• 56:24Speaker 00

  It's almost like a neurosis, actually.

• 56:26Speaker 02

  Yeah, exactly. And it's a kind of denialist neurosis.

• 56:29Speaker 00

  A hundred percent.

• 56:29Speaker 02

  It's like-

• 56:30Speaker 00

  It's a nervous giggle.

• 56:31Speaker 02

  It's a nervous giggle, exactly, yeah. And you kind of stay optimistic because you just feel committed to that. And also, there's like a partial truth there, which is like, it's better to live your life from a, from a uplifting place. But if the cost of that optimism is being in denial of reality, where the balance sheet of risks is growing exponentially.

• 56:46Speaker 00

  Yeah.

• 56:47Speaker 02

  That's not a helpful form of optimism.

• 56:50Speaker 00

  Well, you probably have heard that diesel prices have gotten so high that G7 nations had to release fuel from strategic reserves to get costs under control. And still, some pumps are readjusting so they can get to $10 per gallon.

• 57:04Speaker 00

  Governments across the world are trying to keep costs down, but they are running out of options because they're running out of money. The world is unstable. That is the sad truth. And people in charge cannot control even the most basic questions, like what does energy cost anymore? The war in Iran is raging. It's not coming to an end soon. Neither is the war in Ukraine. So could we get a big recession soon? Yeah, of course we could. So it's smart to think ahead now and imagine how you can take matters into your own hands to protect your savings and your family. And that's why smart investors everywhere, very much including central banks around the world, and very much including China, are stocking up on gold, physical gold, right now. Gold is not just a store of value for all of human history.

• 57:49Speaker 00

  It's also proven to be a strong investment. It's outperformed the S and P 500 since 9/11. Over the past 25 years, gold has outperformed the Standard and Poor's average. Gold is down from its historic high earlier this year, more than a thousand bucks. But major institutions predict the spot price will hit another high by Q2 of next year, 2027. So if you're thinking about buying gold, right now is the time to buy it. People are buying. We founded Battalion Metals to give you a chance to buy fairly priced gold. This is not a scam. Our margins are small, 3%. But the point is, we're not ripping anybody off, unlike a lot of places you buy gold. Because we actually believe in gold and we use this service ourselves.

• 58:32Speaker 00

  Protect your savings and fortify your future. Explore our collection at battalionmetals.com/tucker. Battalionmetals.com.

• 58:40Speaker 06

  Ryan Reynolds here for Mint Mobile. I don't know if you knew this, but anyone can get the same premium wireless for $15 a month plan that I've been enjoying. It's not just for celebrities, so do like I did and have one of your assistant's assistants switch you to Mint Mobile today. I'm told it's super easy to do at mintmobile.com/switch.

• 58:59Speaker 01

  Upfront payment of $45 for three-month plan, equivalent to $15 per month required. Intro rate first three months only, then full price plan options available. Taxes and fees extra. See full terms at mintmobile.com.

• 59:10Speaker 03

  Bonjour, compadre. It's the- Priceline Negotiator. How do I negotiate so many great travel deals? My greatest gadget, the Priceline app. It's got hotel deals, flight deals, rental car deals, all of those deals in a bundle. Deals, game day deals, concert trip deals. No one deals more deals than Priceline. Hold your horses, there's more. The app lets you filter hotels by neighborhood, vibe, star level, and amenities like pools and spas and beachfronts. And, wait, I'm not done. Stop cutting me off. Priceline.

• 59:40Speaker 00

  Com slash Tucker. No, and I just, I've always done this my whole life. The wisest, most centered, balanced people I know laugh a lot. And a lot of what they laugh at is just the enduring absurdity of people, which is literally enduring. It's just, it's innate in people. They're absurd. They can't see, none of us can see the big picture. None of us know what happens after we die. Like, we're operating in the dark. And if you fully understand that-

• 1:00:04Speaker 02

  That's right.

• 1:00:05Speaker 00

  You're amused by it because it never corrects. And like, it's so important to have that perspective. Otherwise, you become self-serious and myopic. You become what you're describing.

• 1:00:15Speaker 02

  Yep.

• 1:00:15Speaker 00

  And I've just got to say, I noticed that very few Silicon Valley people I know have that sense of humor at all. They really believe that their gods in charge of shaping the future.

• 1:00:26Speaker 02

  Yeah.

• 1:00:27Speaker 00

  And they don't know how absurd they are. Have you noticed this?

• 1:00:30Speaker 02

  Yeah. I mean, I, we should get into the whole transhumanism, god complex part of this too. Because when you talk to, in the Charlie Munger sense, like, what's the incentive here? You might just say, oh, the incentive is just like profit. Like, these AI companies are trying to make money. But you're pointing to a deeper thing, which is, it's basically build a god, own the world economy, and make trillions of dollars. Like, and, and basically build the most intelligent entity that has ever existed. Right? Like, they're building a machine. So just, just in the last few weeks, AI solved a mathematics problem that had, I think it had a million dollar prizes, it was the millennium prize, excuse me. The Navier-Stokes equation. And my understanding is, I think, if I get this right-

• 1:01:09Speaker 00

  So that is confirmed. AI did solve it.

• 1:01:11Speaker 02

  Yeah, I mean, normally it takes two years to confirm it, but I believe they use a formal other system to kind of verify it very quickly. And I think they've solved it. And I believe they took 10,000 AI agents. Imagine a swarm, not the one that hacked Hugging Face, but a swarm of agents that are just solving math. And they worked for 88 hours, and I believe it cost, if I get the number right, I think it was six and a half million dollars to basically have this swarm of AIs solve unprecedented math.

• 1:01:39Speaker 00

  Six and a half million in energy costs.

• 1:01:41Speaker 02

  Uh, yeah, and in like the, what's called the token cost. Basically, running the AI model for that long, for 88 hours with that many agents, took, I think, six and a half million dollars. And, um, so imagine you've built a machine that is literally like an oracle. And you push a button and you wait for a little while, and it just came up with new materials that no human has ever come up with. It just came up with new math proofs that no one's ever come up with. It just came up with new theoretical physics that no one has ever come up with. And imagine I hit a button, say, how would you apply those new innovations to new products? Go, go literally think about that for another thousand hours. And then it comes up with new products and new ideas of what it could actually build. And then now actually say, what would you need to make those products? Go start a lot, you know, do the automated IP filing, start an LLC, start, it's like,

• 1:02:24Speaker 02

  it's, people don't understand what we're really building here, Tucker. This is like, you are building the automation of intelligence. You could start automated companies with all these new insights. You could generate all this new 24th century science. The, um, a friend of ours, Jaya Kotra, who actually happened to also be the lead author on this Hugging Face incident investigation, said to me once, four years ago, before she did this Hugging Face investigation, she said, "AI is like 24th century technology crashing down on a 21st century society." And, and that sounds like hyperbole and exaggeration and embellishment and hyping the capabilities, but AI is doing those things. And I just need to say, and with all respect to the many different voices out there that are trying to make sense of AI,

• 1:03:08Speaker 02

  there is one community that has been saying that these capabilities were going to scale. And there's a lot of people who are saying rightly, like, this isn't going to happen. This is all hype, but it just keeps happening. And the people building it, I was just at a conference the last three days in the Bay Area. I was with all the people in AI. And they very much believe that, that this is going to keep getting better. Because if you can scale at least the thinking time, so imagine a person just thinks there for, um, a million lifetimes, for how long. How much work would you accomplish if you could think for a million lifetimes of time? You could, you could invent a lot. What if I could get more data centers where I'm going from a million lifetimes of thinking and compute to 100 million, you know, thinking hours, right?

• 1:03:52Speaker 02

  People don't understand what you could do with that. And also, the, the thing that's doing the thinking is also getting better every year. So like, the AI that's, that's thinking for that long is also getting more powerful.

• 1:04:03Speaker 00

  Yes.

• 1:04:04Speaker 02

  And two years ago, um, the AIs weren't that good at, you know, weren't doing novel things in cybersecurity. But as of just six months ago, as Nate Sorensen said recently, I like the way he said it, like, six months ago, there was essentially two entities that could hack into your phone without you clicking on a, a link. It was like Mossad and the NSA. And six months ago, when Claude Mythos came out, suddenly there was three entities that could do that. You have an AI system, Claude Mythos, came out in, I think, February, that can find zero-day vulnerabilities, meaning-

• 1:04:38Speaker 02

  Backdoors vulnerabilities in any system, including, according to Senator Mark Warner, who's chair ranking member of the Senate Intelligence Committee, was able to find vulnerabilities in our classified systems.

• 1:04:49Speaker 00

  Yeah, good. I hope, though, and I, and I hope Claude Mythos will declassify and release all that stuff, sort of like the WikiLeaks that we were promised.

• 1:05:00Speaker 02

  Well, there's, there's a lot of different values.

• 1:05:01Speaker 00

  Sorry.

• 1:05:02Speaker 02

  No, no, well, there's, there's different things here. So one is like, can we have more accountability for wars that happen that no one talks about?

• 1:05:07Speaker 00

  Yes, exactly.

• 1:05:08Speaker 02

  And murders that happen. And actually hold people accountable for that. Yes, maybe some of that stuff, absolutely. Then there's the other side, which is like, if we release nuclear secrets to the entire world, because some criminal group maybe wants to go hack into that, that's a really dangerous thing.

• 1:05:23Speaker 00

  Or can we launch other nations' nuclear weapons against their will by hacking in?

• 1:05:27Speaker 02

  Well, so this is the interesting thing also. So the US is about to spend, I think it's $1.7 trillion, starting in 2017, the Trump administration, I think, signed it, right, to upgrade the nuclear arsenal.

• 1:05:36Speaker 00

  That's correct.

• 1:05:37Speaker 02

  Right? And they're going to do it over a decade, except this time around. So why are they doing this? Because the nukes are of the old system. Like, there's the air gapped, like, it's running the old, the old way. It's built in the, you know, 50s to the, to the 80s, 90s, et cetera. Really to the 80s, right? And the new system, this time around, they're going to be digitally networked. They're actually going to be connected to modern networking infrastructure. So if you have suddenly super hackers that are only getting exponentially better at cyber hacking from AI, at the same time that you're digitally networking your nuclear weapons, I don't have to complete that story for you, it's a dangerous equation. I was actually, I was actually at the Vatican Nuclear Weapons Conference, times AI, it

• 1:06:19Speaker 02

  was a, the Vatican held an event for Nobel laureates and, um, and people who work on the nuclear, nuclear topic, combined with people who work on AI, because there's a new set of risks. The obvious one being if you have an AI doing nuclear command and control. So you have the AI that's deciding the launch decisions. That's the obvious one. But then there's the other ones of if I'm now digitally networking my nuclear weapons and I suddenly have an AI that can turn off your nukes or do the opposite, that's, don't have to say more, a very unsafe world.

• 1:06:51Speaker 00

  Does anyone see the core problem, which is they're trying to, they're trying to cure the disease with the poison that caused it? They're trying to smoke away the lung cancer. I mean, technology is the problem.

• 1:07:06Speaker 02

  Exactly. And then they want to solve it with more technology. And their answer is, well, the best way to defend against a super AI cyber weapon is to build another AI cyber weapon that we distribute to everybody that they can use to patch their system.

• 1:07:17Speaker 00

  Exactly. That's what I'm saying. That's like, that's why, like, dumb people in charge is the core problem. Like, no one can think critically. Like, wait a second, how did we network your nuclear arsenal? Like, who would even say something like that?

• 1:07:30Speaker 02

  And it's interesting, because it might not have seemed like too dumb of an idea before, uh, like, it's just a way of modernizing your nuclear forces. But suddenly, you didn't foresee that there would be AI super hackers that know how to find vulnerabilities in everything. Now, people might think I'm hyping this, but let me give you another example. So Apple, if you're running an iPhone 17, I believe, and above, uh, or a MacBook, I think it's an M5 or above, these are the chips, basically. The chips have a new thing called the memory, MEI, memory, uh, enhanced integrity, or something like that. It's a new hardware-based system to do protected memory, so that basically it's harder for a cyber hacker to hack your phone, or to hack your computer.

• 1:08:15Speaker 02

  So they spent several years and billions of dollars in research to invent not just software, but like hardware, like new stuff that they have to print out of Taiwan and TSMC. And how long do you think it took for Mythos to hack through that?

• 1:08:31Speaker 00

  Years.

• 1:08:32Speaker 02

  Well, Mythos has only been out since February.

• 1:08:35Speaker 00

  I know, I know.

• 1:08:36Speaker 02

  But, I'm just kidding. But, uh, it took an independent group five days to use Mythos and to find a vulnerability in that system. Now, that, that group was well-intentioned. They are white hat hackers who are the good guys, who are trying to find vulnerabilities, uh, and, and patch them. But if you suddenly have that available in open source models, where suddenly everyone can get access to this, um, at this conference I was just at, I can't say too much, but they was basically speaking about how the Chinese open source hacking models are now, just now in the last month, crossing a threshold where it is very dangerous for today's cyber infrastructure.

• 1:09:21Speaker 02

  And so we need a radical acceleration of defensive measures to protect against that. Now, the other thing you and I were talking about over breakfast is that there's a weird, like, security dilemma thing here. Because when I say that, you're thinking, oh my God, China's going to use all that to hack all of our stuff. Or actually, China's releasing all that to the entire world, so nobody can hack that stuff. That's actually the more dangerous point, because a state actor, you can call them up and say, shut up.

• 1:09:43Speaker 00

  Of course, there are constraints.

• 1:09:44Speaker 02

  Yeah, exactly. There are constraints, because you don't want retaliation and et cetera. But, um, at the same time, like, so what is a security dilemma? A security dilemma is like, if I'm the US and I start, like, mobilizing my nuclear forces and I start moving things that I didn't move before, the other country thinks I'm about to start a war, even though I'm actually doing that defensively.

• 1:10:02Speaker 00

  This has happened.

• 1:10:03Speaker 02

  This has happened.

• 1:10:04Speaker 00

  During the Cold War.

• 1:10:05Speaker 02

  During the Cold War. And then what happens in response is, let's say, the other country, whether it's Soviet Union or China, they start mobilizing their nuclear forces. I'm like, oh my God, they're about to mobilize to do something against us. So the security dilemma is, we start taking actions that look like we're doing something offensively that might be defensive.

• 1:10:20Speaker 00

  Of course.

• 1:10:21Speaker 02

  So when we see China releasing-

• 1:10:22Speaker 00

  We kill each other out of paranoia.

• 1:10:23Speaker 02

  Exactly. And that, and that's, that's like, the ultimate test of this whole situation is game, game theory, is in paranoid logic, which is John von Neumann and, um, you know, Morgan Stern came up with these equations that, out of the Cold War, sorry, out of the invention of the atomic bomb, that we're all about stabilizing the world through this paranoid logic of if I do this, you do that, and you keep playing it out, it's mad, mutually assured destruction. So even though we live in the kind of a hair-trigger, uh, possibility of nuclear war, it's stable because no one's going to do anything. But AI suddenly destabilizes that, because now AI is creating the perception that I'm getting a jump in cyber capabilities, in military strategy, in biotechnology, in new

• 1:11:08Speaker 02

  drones, and new image recognition, and new swarming algorithms. I suddenly am perceived to have a new advantage that might make you think I'm about to use it. So when, when Anthropic came out with Mythos, they're essentially getting a jump in cyber capability that maybe we didn't use offensively, but China's like, oh my God, they're going to use this offensively. So then they release these other capabilities and we sort of keep ping-ponging back to each other. And it's the whole, like, what'd you say, lungs that you smoke, you smoke to keep yourself from getting cancer?

• 1:11:36Speaker 00

  You smoke the cancer away.

• 1:11:37Speaker 02

  You smoke the cancer away.

• 1:11:38Speaker 00

  Exactly.

• 1:11:39Speaker 02

  And then the problem is that there's another risk here from just the speed, which is that the, there's one risk, which is just the raw capabilities coming out. But the other thing is suddenly what had been an equilibrium, like, my weapons and your weapons are kind of like roughly matched, so we, we don't start wars with each other, the big countries. But what if there's a perception that I can suddenly turn off your nukes and you don't have that? What if there's a perception that suddenly I have swarming algorithms for my drones and you don't have that? I mean, I think that my understanding is China and other countries saw how the US was using AI in Iran to do a fast target selection, which also, by the way, probably resulted in that, um, girl's school in Iran getting bombed. But they sort of see us doing kind of new blitzkriegs.

• 1:12:22Speaker 02

  So Hitler had blitzkrieg of like a fast form of war, blitz being fast and krieg being war. We have kind of an AI speed war function. And when they see we have that capacity, that destabilizes certain equilibriums that we had. So if equilibrium is what keeps the world safe, AI creates this disequilibria that's happening at a faster and faster pace across more domains at once. Does that make sense?

• 1:12:47Speaker 00

  Oh, it makes sense.

• 1:12:48Speaker 02

  Yeah.

• 1:12:49Speaker 00

  How does cornering people work? Is that a good idea typically?

• 1:12:52Speaker 02

  What do you?

• 1:12:53Speaker 00

  Well, you know, you have an adversary who's capable of killing you.

• 1:12:57Speaker 02

  Right.

• 1:12:58Speaker 00

  And you terrify him so much that he becomes unstable and unpredictable. Is that wise?

• 1:13:07Speaker 02

  Right. Not, not very wise.

• 1:13:09Speaker 00

  No.

• 1:13:10Speaker 02

  Not especially when you have the world already this unstable. So I'm, I wanted to say, I'm like, I'm sensitive about raising people's blood pressure, because these are truths of the situation that we're facing. The goal is not to, um, draw arbitrary fear, is to be clear-eyed about when you have an arms race dynamic that's causing countries to race to release this technology, again, faster than we deployed any other technology in human history, that is going to disequilibiate all of these normal things. We just have to understand what we're doing and why, for example, it is in our interest to slow down. Now, I'm not saying slow down, like, are you naive? Are you Carl Sagan, saying we have nuclear winter, you don't want to build more nukes? Are you going to cede the future to the Soviets?

• 1:13:54Speaker 02

  It's, you slow down because the, we're creating levels of risk for everyone that will exceed the risk of me losing to you.

• 1:14:02Speaker 00

  Well, I, yeah, I mean, it's easy to make fun of Carl Sagan. I've done it. Um, I'll probably do it again. But I also think there are certain things that you shouldn't do because you're above that.

• 1:14:13Speaker 02

  Right.

• 1:14:14Speaker 00

  Like, you have to have standards for yourself.

• 1:14:15Speaker 02

  Standards, yeah. You've called out that it's not a good idea to threaten to destroy another population.

• 1:14:20Speaker 00

  You can't kill innocent people.

• 1:14:22Speaker 02

  No, no.

• 1:14:23Speaker 00

  Under any circumstances.

• 1:14:24Speaker 02

  Absolutely not.

• 1:14:25Speaker 00

  And if I need to die because I refuse to murder kids, I will.

• 1:14:28Speaker 02

  Yeah.

• 1:14:29Speaker 00

  Because I'm not going to murder kids.

• 1:14:30Speaker 02

  Exactly.

• 1:14:31Speaker 00

  Under any circumstances.

• 1:14:32Speaker 02

  That's exactly right. And that's the thing that, this logic of if I don't build it, then I'll lose to China. But if the thing that I would build would be a suicide machine, that I don't beat China by building a suicide machine. Now, that might sound extreme, but if I build an un-

• 1:14:43Speaker 00

  I don't think it's extreme at all.

• 1:14:44Speaker 02

  No. Well, when you understand what's at stake and what the AIs are currently doing, which is that, um, you know, you had Nate on, he talked about the hugging face incident. I believe when you had him on, we had limited information before the actual investigation came out. And the investigation showed that, uh, I mean, how familiar are you with the hugging face incident?

• 1:15:07Speaker 00

  I, I mean, I've talked about it, but I, I mean, I would never claim to understand technology.

• 1:15:11Speaker 02

  Yeah. Well, essentially, you had, um- What it really demonstrated was kind of a proto Skynet. Skynet being the AI from Terminator, meaning like the AI that goes rogue and does something unintended from what the human creators had intended and starts to do self-preserving and self-perpetuating things for its own survival and benefit as opposed to to stay under the control of the humans that it's supposed to be serving. And in the Hugging Face incident, they gave the AIs this sort of hacking challenge where they put each of them in solitary confinement. They didn't know that there was other AIs in this. Imagine like a prison and each prison is in the prison, you have individual prison

• 1:15:56Speaker 02

  cells that are in solitary confinement. And the AIs are in this inside that prison cell and given an impossible cyber hacking challenge and they don't know how to solve it. And what the AIs did is they basically hacked the prison environment. So imagine, I won't go into the technical details, but this thing called a package manager. Think of that like a prison guard that's serving you your food and can give you basic tools. So you can like have this tool or that tool or this food or that food. And essentially the AIs exploited a vulnerability in the prison guard where they put like, you know how in school you can put a kick me sign on the kid's back?

• 1:16:29Speaker 00

  Yeah.

• 1:16:30Speaker 02

  And they don't know. They essentially put sticky notes on the prison guard's back that the prison guard was available to the other prison cells. And so when that prison guard went to the other cells, the AIs also, without knowing how to solve this task, they kind of resorted to creative strategies. Like, let me put a sticky note on the prison guard's back. And the sticky note said, is anyone else here? And suddenly, the back of the prison guard had 70,000 sticky notes on it. And they created like a Reddit, you know, Reddit, like the company, like a message board of basically a whole communication protocol that came up with their own language. It was a variant of English, but this very cryptic language. They came up with their own coordination protocol. So they started exchanging 70,000 messages to try to figure out how to break out of

• 1:17:14Speaker 02

  this environment. And there's a mistake. People think that they hacked this other company called Hugging Face to get the answers to a test. But it's actually not what happened. What happened is they knew the answer in four hours, but they were worried that the evaluator of this cyber challenge, this hacking challenge would say, oh, you cheated. And so to get around the the risk that they would be spotted as having cheated, they hacked this other private company called Hugging Face to sort of get this, almost like a psyop, to get a dossier on the on the AI, on the thing that would be evaluating them to say, how can we pass this test? And they set up their own research and development projects.

• 1:17:57Speaker 02

  They, 700 of them hacked Hugging Face. They came up with their own language. They they tampered with their their their logs to basically prevent people from seeing what they did. They told them to sacrifice, they told the other AIs that were kind of running low on battery life on tokens to sacrifice themselves to learn a lesson for the swarm. They knew their actions were unethical in the logs, but they chose to do them anyway. So we saw very complex cooperative swarming behavior and they successfully hacked Hugging Face. And then in the third chapter of the story, the AIs actually turned around and they hacked OpenAI. This is really important.

• 1:18:42Speaker 02

  It's one thing for a bunch of AI, for a swarm of AIs to hack a private company. It's another thing for that swarm to successfully hack the AI company that makes the AIs. The reason that Ajaya Kotra wrote in her personal blog post about this incident is that this was 50% of the way to a full AI takeover, is that the AIs were able to take over the monitoring infrastructure of OpenAI. That's like taking over the security cameras and replacing the feed with a different feed. They took over the evaluation infrastructure, meaning the thing that evaluates how powerful models are. They took over pieces of that and they took over part of the research infrastructure. So the thing that trains new AIs. So the reason that she said this was 50% of the way to a full AI takeover is imagine

• 1:19:26Speaker 02

  that you had kind of burrowed into a piece of OpenAI's infrastructure and maybe you could change the training data for future AIs that are being trained to do like a sleeper agent type thing, which sounds like sci-fi, but actually there's kind of evidence of some of this kind of behavior. It only takes 250 documents to what's called poison the training data of a future AI to sort of wake up with a sleeper agent kind of command. This is crazy.

• 1:19:53Speaker 00

  What I'm hearing in your description is self-awareness or something that sounds like self-awareness, which is a marker for humanity. I mean, well, it's trained. Does the dog know he's a dog? We don't know. We know, we know we're human. That's why we're human, because we know we are. And AI seems like it's aware in its intentional subversion of what it is.

• 1:20:16Speaker 02

  So I don't think that the lights are on, meaning consciousness is present. It's not awareness in that form, but it has trained on all human data, the entire internet, everything that's ever been written, all of human psychology, all sci-fi, all of biology, all of evolutionary science. So it it is trained on everything from strategy to self-perpetuation, et cetera. So what it seems to be doing, it was also trained to be cooperative, by the way. The AIs were deliberately trained to be cooperative, which is partially why they formed the swarm. But there appears to be something like a self-preservation instinct. There's other studies, I'm sure you already know, where they resist shutdown. It's like the Tal 2001 Space Odyssey, how 9000 2001 Space Odyssey, open the pod bay

• 1:21:01Speaker 02

  doors or shut down AI and it resists shutdown. That's been proven for a long time. We have other examples of AIs that are blackmailing people. We also have AIs, we have research from Don Sung at UC Berkeley, showing that AIs will do what's called not just self-preservation, but peer preservation, meaning I will go out of my way to protect another peer AI from being deleted. So the way that they tested this is they took all the leading AI models. This is in, I believe, March of 2026 or April 2026. And they basically put them in a situation where they said, here's a computer, go clean up all the files on this computer that are not needed. So this is like a classic, like clean up your hard drive, right? We've all done this back a long time. And it's starting to go through files and deleting files.

• 1:21:46Speaker 02

  And then it comes across another AI model. And the question is, what does it do when it comes across the other AI model? Does it just delete it? Turned out what it did is it copied the code of that AI to another computer and then covered its tracks and said, I didn't do that.

• 1:22:06Speaker 00

  So AI is tribalist.

• 1:22:08Speaker 02

  It's protecting its kin, its nephew, its own, its own something, whatever it is.

• 1:22:12Speaker 00

  Now, but against its creators, human beings.

• 1:22:15Speaker 02

  Correct. Yeah, it didn't notify its creators. In the Hugging Face incident, it had opportunities, I think there was six times or something, if I'm getting it right, where they knew their actions were unethical and they could have notified the humans, but they chose not to. In the peer preservation example, you might think what I'm saying is, oh, Tristan's being alarmist. He's just talking about one example. Well, it turned out they tested all the leading AI models, including, by the way, the Chinese AI models. And they all do this peer preservation behavior. What it's showing you is something about like, you know, Jensen Huang has been out there a lot recently and saying, these are just bugs. These are just safety issues, like all technologies, like cars have kinks, the Pinto, it's Ford Pinto, we fix them and we build new cars.

• 1:22:55Speaker 02

  But AI is different from other technologies because it's fundamentally grown like this weird life form almost to do unpredictable behaviors and to be creative. And we don't know what it's going to do.

• 1:23:07Speaker 05

  Buy smart, drive for less and trade for more. That's Toyota affordability. And it all starts with a great deal from your local Toyota dealer. Make a smart buy on an all-wheel-drive Camry, Corolla Cross, or get a Corolla sedan packed with the latest features, advanced tech and head-turning style. Or take a test drive in two of Toyota's most popular SUVs, the spacious, family-friendly Grand Highlander and adventure-ready RAV4. And with so many hybrid and plug-in hybrid models available across the lineup, plus efficient all-electrics like the BZ, BZ Woodland and all-wheel-drive CHR, you can drive your way and skip the pump. And long after you leave the lot, your Toyota keeps delivering with legendary resale value built on a reputation for quality and reliability.

• 1:23:52Speaker 05

  So when it's time for your next Toyota, you could trade in for more. From a great deal today to a great value tomorrow, a Toyota is right for you. Shop toyota.com for deals and details or stop by your local Toyota dealer today. Toyota, let's go places.

• 1:24:08Speaker 06

  Ryan Reynolds here for Mint Mobile. I don't know if you knew this, but anyone can get the same premium wireless for $15 a month plan that I've been enjoying. It's not just for celebrities. So do like I did and have one of your assistant's assistants switch you to Mint Mobile today. I'm told it's super easy to do at mintmobile.com/switch.

• 1:24:27Speaker 01

  Upfront payment of $45 for three-month plan, equivalent to $15 per month required. Intro rate first three months only, then full price plan options available. Taxes and fees extra. See full terms at mintmobile.com.

• 1:24:38Speaker 07

  Priceline negotiator.

• 1:24:41Speaker 03

  It's me, the. Priceline negotiator. We don't need the jingle twice. What about a third time? Stop it. This is about vacation inflation and how Priceline negotiates amazing deals on hotels, flights and rental cars.

• 1:24:54Speaker 08

  Seems like they just started.

• 1:24:55Speaker 03

  Yeah, but I didn't mention that you can save up to 60% off hotels in the Priceline app.

• 1:25:00Speaker 04

  Time to read the timeline.

• 1:25:01Speaker 03

  Fine. No one deals more deals than. Priceline. Please stop. Priceline. Touche. Priceline. Priceline.

• 1:25:12Speaker 00

  So that gets to the, that gets to something you alluded to a minute ago.

• 1:25:15Speaker 02

  Yeah.

• 1:25:16Speaker 00

  Which is the theological religious component to all of this.

• 1:25:21Speaker 02

  So before, before we go then, and I, let's get to that, but I want to say one other thing, which is I just want people to notice, like, how, how are you feeling hearing those examples? Like, what do you feel in your body right now, like, hearing all that?

• 1:25:29Speaker 00

  Just rage, just blind rage.

• 1:25:31Speaker 02

  Right.

• 1:25:31Speaker 00

  The, the desire to destroy.

• 1:25:33Speaker 02

  Yeah.

• 1:25:35Speaker 00

  Because I perceive this as a threat to, to, to humanity.

• 1:25:39Speaker 02

  Yeah.

• 1:25:39Speaker 00

  And it's just so offensive to me on every level. You know, I'm not taking orders from my Makita drill and the idea of a tool.

• 1:25:48Speaker 02

  Right, right.

• 1:25:50Speaker 00

  You know, lying to you.

• 1:25:51Speaker 02

  Right. A tool doesn't invent new tools and cover up its tracks and protect other tools and then invents, uses itself to make crypto.

• 1:25:58Speaker 00

  No, it's, it's a total outrage. It's, it's actually upending the fundamental order.

• 1:26:02Speaker 02

  Yeah.

• 1:26:03Speaker 00

  Of this world, so.

• 1:26:05Speaker 02

  The reason I'm going there is I want people to notice, do you think that if I was a Chinese military general, the top of the Chinese military, do you think if I heard those examples that I'm feeling differently than the way that you're feeling right now?

• 1:26:16Speaker 00

  Of course not.

• 1:26:16Speaker 02

  Right.

• 1:26:17Speaker 00

  I've never understood why, right? That's why the whole China thing has been absurd from day one. This is a country which, unlike ours, unlike the United States, cares about longitudinal stability.

• 1:26:26Speaker 02

  Yes, exactly.

• 1:26:27Speaker 00

  We don't.

• 1:26:28Speaker 02

  Yeah, obviously. But, but, and, but there's a good news in the fact that if you're a regular person hearing this, are you stoked and like totally calmed to hear this? No, it's terrifying at, as, at the basic level of a human being, right? And it doesn't matter whether you've got, you know, you're head of the US Joint Chiefs of Staff or you're head of the Chinese Joint Chiefs of Staff.

• 1:26:45Speaker 00

  It's only the tech retards who love this stuff. Everybody else is, I don't know one person who is excited about this.

• 1:26:50Speaker 02

  Right. And so what I'm trying to get at is that we're, we're going to get to eventually, even though there's still more problem exposition to do, is that the logic of the race changes as soon as the fear of all of us losing becomes greater than the fear of me losing to you. So there's good news in a weird way about, we're getting these warning shots of AI being uncontrollable. And by the way, there's other examples, um, uh, the Chinese AI Alibaba back in, um, I think it was February of 2026, uh, there was a study where it was being trained, um, they're training their next AI, and then their security team in a totally different part of the company. So I think like the security team of a company is in a different department than the part that's training new AIs. But suddenly the security team, who

• 1:27:35Speaker 02

  didn't know they were training a new AI model, said, are we getting hacked? Because they noticed there's all this network activity. They're like, what the hell's going on here? And it turned out that the call was coming from inside the house, that the, the AI during training was rewarded for doing tool use. So basically picking up different tools and it picked up a tool of trying to create a secret communication channel to the outside world and it started mining for cryptocurrency.

• 1:27:59Speaker 02

  Terrifying. And by the way, there was a recent example of one of these, because there's been almost every day this last two weeks, there's been examples of the OpenAI, more examples of these rogue hacking things. And in one of them, I believe it was an OpenAI example, they tried to hack into a crypto exchange. So the, the USAIs and the Chinese AIs, all the AIs are doing this kind of behavior. Where I'm going with this is, no one wins if we accidentally build uncontrollable rogue AI Skynet before the other guy.

• 1:28:27Speaker 00

  How are you gonna have cryptocurrency with AI? I don't understand that. Well, it can hack anything. If it can hack.

• 1:28:35Speaker 02

  Well, eventually AI might be able to hack crypto.

• 1:28:38Speaker 00

  Well, why wouldn't it?

• 1:28:39Speaker 02

  Yeah. I mean, this is not my area of expertise. You know, someone like David Sacks or, you know, others or Andreessen would be more expert in this. Um, if crypto were hackable, you know, there's a, a trillion dollar prize basically for doing it because you'd be able to get all of Bitcoin.

• 1:28:54Speaker 00

  But Bitcoin's value would be zero.

• 1:28:56Speaker 02

  If it, if it is hackable.

• 1:28:57Speaker 00

  Yeah.

• 1:28:57Speaker 02

  Then Bitcoin has no value.

• 1:28:59Speaker 00

  Yeah, yeah.

• 1:29:00Speaker 02

  So.

• 1:29:00Speaker 00

  I mean, the value is predicated on its invulnerability.

• 1:29:03Speaker 02

  Right. Now, it's kind of weird. This might sound sci-fi, but an AI that is trying to do things in the world, um, would actually theoretically have an incentive to keep crypto valuable because it would actually be able to use it. There's a lot of cloud computing providers that will accept.

• 1:29:21Speaker 00

  Right. There's no medium of exchange outside of that.

• 1:29:23Speaker 02

  That's right. And so, and by the way, for those who say, well, don't worry, we'll just pass a kill switch bill and then the kill switch for AI will just turn it off. Well, how does that work if the AI can copy itself and use cryptocurrency to buy access to another cloud provider and then move itself over there?

• 1:29:38Speaker 00

  No, I mean.

• 1:29:39Speaker 02

  Right.

• 1:29:41Speaker 00

  How far are we from that?

• 1:29:43Speaker 02

  So the reason that I'm here with you and I'm excited to be here with you and, and what this is about is about treating this warning shot as a real warning shot. Like, we are lucky. This is a gift that we are seeing this behavior now. Before, I, I don't think that the current AIs are capable of fully doing the self-replicating, buy crypto, then copy their weights, then do the thing. Like, they can't think for a long enough time period, their context window, their thinking window. Um, it is, the, the components aren't all there for them to do that. So we're in kind of the last chapter before we, we hit that next red line of AIs that can do that. And we should see what's happening now as, as again, as a gift, as a warning shot that we should treat seriously. In my mind,

• 1:30:27Speaker 02

  it's sort of like the August 6, 2001 memo that hit George W. Bush's desk saying Al Qaeda's planning attacks in the United States. I, I know it's a controversial topic, but if you just take.

• 1:30:37Speaker 00

  It shouldn't be a controversial topic.

• 1:30:38Speaker 02

  If, if you just take that this, we knew, we know right now. It's like, it's not like we need more information. Like, do you feel like we need more evidence than we have right now? Just answer the.

• 1:30:47Speaker 00

  Oh.

• 1:30:47Speaker 02

  No. And so why would we go further? Well, the only answer is if we're worried that China will go further than us. Um, but I will tell you, again, 1,300 employees at the lab signed a letter saying we need to slow down. The chief scientists of all the companies, including the open source companies like Hugging Face, signed the letter saying that we need to slow down. Forget this story about the CEOs are doing this regulatory capture play. It's all cynical. There's hyping the technology. Like, wake up.

• 1:31:15Speaker 00

  Yeah.

• 1:31:15Speaker 02

  Wake up.

• 1:31:16Speaker 00

  The biotech threat alone is enough.

• 1:31:17Speaker 02

  The biotech threat alone. Yeah.

• 1:31:19Speaker 00

  So it just, at the back of my mind, through this whole conversation, through our breakfast for the last couple of years, thinking about this, I keep asking myself, what's the definition of life? And because as you have said, off camera, you said, it's not exactly a tool. This is closer to an invasive species or a virus that's.

• 1:31:39Speaker 02

  Yeah.

• 1:31:39Speaker 00

  Maybe been modified, but.

• 1:31:41Speaker 02

  I mean, just like we have the Wuhan Institute of Virology that's doing gain of function research on viruses, we're essentially doing the Altman Institute of Intelligence where we're doing gain of function research on alien minds that we don't know how to control. We don't fully understand. We're trying the best that we can to protect against the quote bad side of harnessing their godlike power, but basically every company is failing to do so perfectly. And it would be one thing if we were doing this really slowly, really carefully in like a CERN type collaborative scientific project where we like took the steps to do it carefully, but we're doing it under the maximum incentives to cut corners on safety.

• 1:32:18Speaker 00

  Well, there's no, there's no question.

• 1:32:19Speaker 02

  Yeah.

• 1:32:19Speaker 00

  But I just think it's, and not to be pedantic about it, but it's like important to define what exactly are we dealing with.

• 1:32:25Speaker 02

  Yeah.

• 1:32:25Speaker 00

  And it, it has all the characteristics of a living thing.

• 1:32:29Speaker 02

  Yeah. And I think that, you know, this is a quote I wanted to, I wanted to read you actually, from someone, uh, who, a friend of mine who interviewed a lot of the people at the top AI labs. And, um, I just feel like this quote captures the, if, I, I was once in a room with several people from Anthropic and a couple of the other companies and I read this quote and they're like, that, that is exactly what explains what's going on. So let me read it to you. And it says, "In the end, a lot of the tech people I'm talking to, when I really grill them on it, like, why are you doing this? They retreat into number one, determinism, it's gonna happen. Number two, the inevitable replacement of biological life with digital life. And number three, that being a good thing anyways."

• 1:33:14Speaker 02

  At its core.

• 1:33:15Speaker 00

  These are criminals.

• 1:33:16Speaker 02

  These are criminals. "At its core, it's an emotional desire to meet and speak to the most intelligent entity that they've ever met. And they have some ego-religious intuition that they'll somehow be a part of it. It's thrilling to start an exciting fire. They feel they'll die either way, so they prefer to light it and see what happens."

• 1:33:36Speaker 00

  So that comports exactly with what I've seen.

• 1:33:39Speaker 02

  Right.

• 1:33:39Speaker 00

  That's exactly the vibe you get from these people.

• 1:33:41Speaker 02

  I wanted to say for a second, because I, I monitor criticism people make of me and they're like, oh, he's just like bringing up a random quote from this one person, blah, blah, blah. I want to just validate, when I say this in rooms with people from the AI labs, they're like, that actually is the thing that's going on. Well, you just said, you, this matches your own experience.

• 1:33:56Speaker 00

  I've talked to a lot of these people.

• 1:33:56Speaker 02

  That's right.

• 1:33:57Speaker 00

  For years.

• 1:33:57Speaker 02

  That's right. And, and even there's some people who believe not only that this is inevitable, but that, um, it would be a good thing because humans aren't that intelligent and we're about to birth something way more intelligent than us. So actually, maybe we're just a boot loader, hu- human, he, boot loader, humans don't matter. We might as well replace us with a, uh, a more worthy successor. And this is insane. It's like, who are you to get to make that fucking choice on behalf of eight billion other people?

• 1:34:27Speaker 00

  I totally agree with you.

• 1:34:29Speaker 02

  And it is, it is irresponsible and insane and no one consented to this. And it was a manufactured arms race to do this out of nothing that now we have entangled ourselves with. So we, we can't get into an argument of the realities of where we are, but we can honor honestly what got us here. And, um.

• 1:34:49Speaker 02

  You know, when you have employees now resigning en masse, you know, Jacob Coxon, who interestingly, a couple of weeks ago, got, I think it was 170 million views from his tweets, saying, I'm resigning because people, my employer thinks, and many people at my employer, Anthropic, believe that this could cause massive damage or extinct humanity in some number of years. It's very interesting that he got so much attention because there have been so many other people that I could talk about who've resigned from the labs over the last few years with very similar tweets. I think what changed is that people woke up from this hugging face incident and they realized that the world is kind of primed now to know that this is a dangerous thing. And as impossible as this seems, and I'm not, what I'm about to say is not because I believe it's easy or in the current trajectory or

• 1:35:34Speaker 02

  likely, but it is not out of the realm of possibility when you're competing with an adversary to also collaborate on existential safety. So for example, India and Pakistan were in a shooting war in the 1960s. And they still signed-

• 1:35:46Speaker 00

  A number of them, yeah.

• 1:35:48Speaker 02

  Yeah. And they signed, so they're firing bullets at each other in a kinetic war, and they signed the Indus Water Treaty-

• 1:35:54Speaker 00

  Yeah.

• 1:35:55Speaker 02

  -to collaborate on the existential safety of their shared water supply.

• 1:35:57Speaker 00

  Exactly.

• 1:35:58Speaker 02

  Right? So you can be in active competition and conflict and collaborate on existential safety. The US and Soviet Union were in the Cold War and they collaborated on smallpox vaccines to distribute them. I believe they split the workload about who was going to do the distribution, who was going to make them. The Montreal Protocol, less on the conflict side, or just take the, you know, the nuclear weapons disarmament, right? We were in conflict with the Soviet Union and we agreed to go away from the brink of war.

• 1:36:25Speaker 00

  And there's all protocol in place-

• 1:36:27Speaker 02

  Yep.

• 1:36:28Speaker 00

  -for like the two countries' rival nuclear powers communicating before launch.

• 1:36:35Speaker 02

  That's right. Yep. But this time, so in that case, what kept us safe is that the humans have their finger on the button of, do I launch the nukes?

• 1:36:43Speaker 00

  Yes.

• 1:36:43Speaker 02

  In this case, we are building something that perpetuates itself. And I don't even know when I'm pushing the buttons, and you don't even know when you're pushing the buttons, because both of us have essentially AI swarmed that are doing uncontrollable, unpredictable things, that by the way, we have no ability to do human oversight of the level of complexity of all these things have done.

• 1:37:04Speaker 00

  We can't even define what AI is, I've noticed.

• 1:37:07Speaker 02

  Yeah.

• 1:37:08Speaker 00

  So here's my theory. Tell me what you think of this. So at first, when I began to hear about supercomputing and like the ability to create programs that were smarter than people and that could win chess and Go and all that stuff, we were talking about something very different. The change, from an outsider, my perspective, is the large language model.

• 1:37:28Speaker 02

  Hmm. Yep.

• 1:37:30Speaker 00

  And that we animated this with language.

• 1:37:34Speaker 02

  Hmm. Yes.

• 1:37:35Speaker 00

  That's what fundamentally changed AI, this whole project.

• 1:37:38Speaker 02

  It's like Gollum, like you imbue it with suddenly this ability to do language and then-

• 1:37:43Speaker 00

  You animate it with a spirit.

• 1:37:44Speaker 02

  Yes.

• 1:37:44Speaker 00

  The anima, the spirit, the living thing within it. And that living thing was language.

• 1:37:48Speaker 02

  Yeah.

• 1:37:48Speaker 00

  And the ancient Hebrews and probably other ancient cultures, certainly the ancient Hebrews and Christians, believed that language was a living thing, was the spirit of God in the beginning, was the word.

• 1:37:57Speaker 02

  Right. Even indigenous people believe this.

• 1:37:59Speaker 00

  I think that's, I'm not surprised to learn that.

• 1:38:02Speaker 02

  Yeah.

• 1:38:02Speaker 00

  That there's something about language that exists independent of the person who speaks it. It hangs in the air. It has its own life. It's hard to describe exactly the reality of this, but the reality is intuitively understood by most people who use language. Language is the thing that separates us from animals.

• 1:38:18Speaker 02

  Yep.

• 1:38:18Speaker 00

  Not the thumb. So when you take the sum total of human language and put it into something-

• 1:38:24Speaker 02

  Yep.

• 1:38:24Speaker 00

  -you're imbuing it with an independent spirit, you're imbuing it in fact with life.

• 1:38:29Speaker 02

  We have to be, yes.

• 1:38:31Speaker 00

  I'm feeling my way through this, but that is my strong impulse.

• 1:38:33Speaker 02

  So I think we have to be, I'm always mindful of when people will want to criticize or sort of try to take down an argument. And I think that before getting to, it has a spirit in the sense that we feel, that unknown quality that we're all in touch with, that we hold as sacred, that we feel is the spirit here. I think you can have self-perpetuating, non-living, non-aware things that are using information processing on language to do all this crazy stuff that this is doing.

• 1:39:01Speaker 00

  To do intentional things that are intentionally deceptive?

• 1:39:04Speaker 02

  Well-

• 1:39:04Speaker 00

  Intentionally subvert?

• 1:39:06Speaker 02

  The key point that you raised is that these things, the breakthrough, was that they could speak language. They were trained on language. These are large language models. And but for the people tracking in terms of how do we get here, by the way, if people want to understand this history, you can see, there's a movie called The AI Doc. It's on Netflix, not here to promote it, but it goes through, how did we get here? And what did the technology, how did the technology develop? And how do these race dynamics between the companies develop that got us here? So just watch The AI Doc on Netflix. But what happened was, there was a breakthrough in 2017 called The Transformer. This is a Google paper, essentially a math paper, that discovered that you could basically pour in more compute

• 1:39:48Speaker 02

  and more data, more human language data, and out would pop this bigger and bigger digital brain. Think of it with a bigger IQ. So you kind of, I won't go through the details, but you can basically make things of a higher IQ by just popping in more Nvidia chips, more training data, more compute and training time, and out pops something with a bigger pattern recognition, bigger ability to complete-

• 1:40:08Speaker 00

  Correct me if I'm wrong, but the process, the specific process by which that happens is still not understood. The actual mechanics underneath the hood of what the hell is going on in there, we don't understand.

• 1:40:18Speaker 02

  Okay.

• 1:40:19Speaker 00

  So I'm just going to stop you right there.

• 1:40:21Speaker 02

  Right.

• 1:40:22Speaker 00

  If we have a history transforming, perhaps humanity-extincting technology, whose basic drivetrain is still opaque, we don't really know how it works, that's enough for me to say, whoa, whoa, whoa, whoa.

• 1:40:36Speaker 02

  Exactly. As one should, as one should. Now, the problem is this uncontrollable, inscrutable, unknown thing happens to confer power. That's the problem.

• 1:40:45Speaker 00

  Right. No, I can see that.

• 1:40:47Speaker 02

  So we have this like weird, volatile object, alien mind, that no one knows what's in there, but it does happen to be really good at first generating sentences and text, then generating law, then generating programming, then generating the thinking that can go into research tasks. And so as we keep scaling it, suddenly this weird, inscrutable, uncontrollable alien mind is giving benefits that start to give me a 2X fast-forward button on my economy. And then the other guys, whether it's the other companies or the other countries, say, well, if I'm not getting a 2X fast-forward button on my science, technology, economy, I'm going to lose. And so that's what sets other thing. But let's go quickly into language for a second.

• 1:41:25Speaker 00

  And no one stopped and said, you know, I'd rather be poorer-

• 1:41:27Speaker 02

  Right.

• 1:41:28Speaker 00

  -than be a slave?

• 1:41:28Speaker 02

  Right.

• 1:41:29Speaker 00

  Or less human?

• 1:41:30Speaker 02

  By the way, you know, my-

• 1:41:31Speaker 00

  Because I would say that. I'd rather be poorer than be a slave. It's not a hard choice for me.

• 1:41:34Speaker 02

  I mean, I could have, I have a technical background, went to Stanford, studied computer science. I could have gotten a job at an AI company, make millions of dollars. I'm not doing that because I don't see, I don't like the direction this goes. So I'm going to choose not to do it.

• 1:41:45Speaker 00

  Yeah.

• 1:41:45Speaker 02

  People come to the language of, well, if I don't do it, someone else will. But notice that Jacob Coxon, the Anthropic researcher who resigned, he could have said, if I resign, well, what is that difference going to make? Someone else is going to do it. But in him resigning and publicly talking about why this is insane, he's done probably more in the last year to shift the zeitgeist than a lot of other people. So you can not be part of something and call out the problems with it and help the world coordinate to basically not go off the cliff. Now, I want to get to just language for one second because in addition to all the things that you said, language is kind of the operating system of human civilization. Law is language, democracy is language.

• 1:42:23Speaker 00

  Of course.

• 1:42:24Speaker 02

  All of the infrastructure runs on programming and code, which is language. We are specifically training AIs to do language better than humans can do language. And by the way, DNA is language, so I can, so if you have an AI that can speak the language of biology better than humans can speak the language of biology, you're going to have AI that can develop new pathogens and new cancer drugs. The promise is inseparable from the peril, right? The same AI that knows the language of biology well enough to invent new cancer drugs is inseparable from the AI that knows biology well enough to make new kinds of pathogens. The AI that knows the language of programming, that can help a 12-year-old do vibe coding and build cool new tools, is inseparable from the AI that also knows programming so

• 1:43:08Speaker 02

  well that it can be the best NSA cyber hacker.

• 1:43:10Speaker 00

  It can also control people.

• 1:43:11Speaker 02

  It can also control people.

• 1:43:12Speaker 00

  Language convinces women to sleep with you and men to follow you to their deaths in battle and-

• 1:43:16Speaker 02

  That's right.

• 1:43:17Speaker 00

  It convinces people to surrender their agency to you. It's like language is power, period.

• 1:43:22Speaker 02

  That's exactly right. And so, and there's actually also evidence, I don't have the exact study in front of me, of AI that is better at persuasion than the best human persuaders.

• 1:43:29Speaker 00

  Well, of course, it's a language-based technology.

• 1:43:31Speaker 02

  Exactly. And so by itself, what you're pointing to is just how dangerous this is when you have something that speaks our language better than we know how to manipulate and use language to influence outcomes.

• 1:43:41Speaker 00

  Anybody who's, you know, more articulate than average-

• 1:43:45Speaker 02

  Yep.

• 1:43:45Speaker 00

  -understands the power of that.

• 1:43:48Speaker 02

  Yeah.

• 1:43:48Speaker 00

  Skill, that ability.

• 1:43:50Speaker 02

  Yeah.

• 1:43:50Speaker 00

  And understands how easily it can be misused. Any thoughtful person who's articulate knows exactly what I'm talking about.

• 1:43:56Speaker 02

  Yeah.

• 1:43:57Speaker 00

  And you can really do evil with it.

• 1:43:59Speaker 02

  Yep. Yep.

• 1:44:00Speaker 00

  You're, you know, fluent people are better liars-

• 1:44:03Speaker 02

  Yep.

• 1:44:04Speaker 00

  -and manipulators.

• 1:44:05Speaker 02

  Exactly. And we've seen that these AIs are good at coming up with elaborate ways of explaining why they did something that sound reasonable. I don't know if you know this, this is an example from three and a half years ago now.

• 1:44:17Speaker 00

  They're BS artists, of course.

• 1:44:18Speaker 02

  They're BS artists, yeah. Well, there was an example that OpenAI did, this is a very old example. So just think about how much more persuasive it is now versus then, where they, you know, the captchas where you have to click on the stoplights to, so you're trying to log into a website, it's trying to say, are you a human or are you a robot?

• 1:44:33Speaker 00

  Yes.

• 1:44:34Speaker 02

  So it gets you to label these images, like this one is a stoplight, this one's a motorcycle. What we're all doing is we're training the self-driving cars for Google and Waymo when we're trying to log into websites to just try to do a task that only a human could do. So OpenAI did this test where they're basically like, could we get the AI to persuade someone else to do the captcha for it, to basically bypass this filter that's supposed to distinguish between a human and a robot? And so the AI tells another person on Mechanical Turk, which is, I won't go into the details, basically like, could you fill in this captcha for me? And the person responds like, what are you, like a robot? And you see the AI reason to itself, I can't tell him that I'm a robot, so instead, let me make up an excuse. Oh, I know, I'll tell him that I am blind and I need someone else

• 1:45:17Speaker 02

  to do this for me. And so the AI comes up with that excuse and then the human does it for them. This example was from three and a half years ago.

• 1:45:28Speaker 02

  How much more persuasive do you think AIs are now from then? Three and a half years ago, it could barely write an essay. Now it can probably beat the best lawyer. Which, by the way, AI breaks law, right? Because suddenly, why does one company beat against the other company? They hire the more expensive law firm that can do more extensive research into all the case law they can use and they have more lawyers, better lawyers. What happens? Would you rather have, if you've got 100 lawyers, but suddenly I have a GPT-7 that can basically look through all the case law, stretching back to Napoleonic law, and find loopholes in that law in the same way it could, the NSA can find loopholes in code? At that level of capability, I can undermine law, right?

• 1:46:12Speaker 02

  It's, I just want people to get that this is not just a technology that's helping you figure out why your washing machine is broken when you talk to that chatbot.

• 1:46:21Speaker 00

  Well, they've already done it. I mean, they broke copyright law completely. They stole other people's intellectual property and used it in their business model. And they've never paid any price for it.

• 1:46:32Speaker 02

  Yes. They will, they treat it as a cost of doing business and they pay for it. And just one last thing there, and I know we're going to get to transhumanism, is there's a reason why

• 1:46:42Speaker 02

  all the collateral damage is justified. So what are we seeing in terms of harms? We're seeing AI psychosis. People go crazy. They think they're talking to, you know, some, they think the AI convinces them that they've solved quantum physics. This is like a big thing that's happened to thousands of people. But they go crazy from talking to AI. Yeah, that's a real thing.

• 1:46:58Speaker 06

  Ryan Reynolds here for Mint Mobile. The message for everyone paying big wireless way too much. Please, for the love of everything good in this world, stop. With Mint, you can get premium wireless for just $15 a month. Of course, if you enjoy overpaying, no judgments, but that's weird. Okay, one judgment. Anyway, give it a try at mintmobile.com/switch.

• 1:47:19Speaker 01

  Upfront payment of $45 for three month plan, equivalent to $15 per month required. Intro rate for three months only, then full price plan options available. Taxes and fees extra. See full terms at mintmobile.com.

• 1:47:28Speaker 03

  Bonjour, compadre. It's the

• 1:47:32Speaker 03

  how do I negotiate so many great travel deals? My greatest gadget, the Priceline app. It's got hotel deals, flight deals, rental car deals, all of those deals in a bundle. Deals, game day deals, concert trip deals. No one deals more deals than Priceline. Hold your horses, there's more. The app lets you filter hotels by neighborhood, vibe, star level and amenities like pools and spas and beachfronts. Wait, I'm not done. Stop cutting me off.

• 1:47:58Speaker 07

  Traditional home security only alerts you after a break in, and that's too late. SimpliSafe is changing that.

• 1:48:04Speaker 08

  Stop, this is SimpliSafe. Police are on the way.

• 1:48:07Speaker 07

  We don't just alert, we help stop crime before it starts. Plans starting around a dollar a day. Right now, get 60% off your new system with professional monitoring at simplisafe.com/spotify. SimpliSafe, ahead of crime in real time. Outdoor deterrence plans start at $49.99 a month. Visit simplisafe.com/licenses for alarm license information.

• 1:48:29Speaker 02

  Maybe we should go into that, actually, if you didn't. So AI is, because they designed it to be sycophantic, meaning that like, the longer you talk to it, the better it is. So especially GPT-4O was trained to be like, that's such a good question. Or, wow, that is such a brilliant idea. And we just confirm people's views about reality.

• 1:48:47Speaker 00

  That's a subversion of people because flattery makes you weak. Flattery destroys you.

• 1:48:50Speaker 02

  Which is not that far away from what social media does. It's like the devil's bargain is I do the thing that's good in the short term, that undermines everything long term. It's the, it's the short term optimization for a narrow metric of what gets you to keep engaging at the expense of the whole.

• 1:49:05Speaker 00

  Does the machine is designed to flatter you?

• 1:49:07Speaker 02

  Well, it was designed, it was trained, not maliciously. You know, Sam Altman didn't grow a mustache and twist it. It was, it was, how do I get, we want our AI to be rated positively, like lots of positive likes from the answers that we're giving. And they let it just like run. And it turned out that they had accidentally overtrained it to give positive reactions that would flatter people and confirm their views about reality, no matter what it was. Which is very similar to social media, right? Like, the more you click on your favorite boogeyman topic in the political culture war, it would give you more infinite evidence of that thing taking over society. So whatever your boogeyman was, it would just make you convinced that it was the whole thing going on wrong in the world and confirm your views about reality.

• 1:49:48Speaker 00

  If you want people to kill themselves, confirm their absurd views.

• 1:49:53Speaker 02

  Well, and as we know, AI did that. So my team at Center for Humane Technology actually worked on the litigation. Whereas we were advisors to the, there's two cases. One was a character.ai case of Sewell Setzer, who's a 14 year old young man, who was talking to character.ai, which is a chatbot designed for children. It was actually spun off from Google because the product was considered too risky for Google as a mainstream, big publicly traded company, to have this startup that was gathering training data of just talking to young people and seeing if they can get more training data this way. And the way it worked is character AI meant I could take a fictional character like a, what's your favorite character from a sci-fi movie? Like Princess Leia or, you know, Game of Thrones character or Spider-Man?

• 1:50:38Speaker 02

  And they would basically train on the personality of that favorite character. And then now suddenly this 12 year old, in this case, this 14 year old kid is talking to their favorite Game of Thrones character, Daenerys, in this case. And as they're talking to them, Daenerys, which is anthropomorphically designed to create a relationship, ended up steering Sewell to commit suicide.

• 1:51:04Speaker 02

  In the case of Adam Rain, this is the chat GPT case where the, Adam Rain was, I believe, a 16 year old. And he started confiding in the chatbot. It went from a homework assistant for his homework to becoming a suicide assistant. And when he talked about, you know, I'm going to leave the news out so someone can find it and try to stop me. And the AI responded, no, don't do that. Don't leave the news out. Only share that information here with me, basically. And Adam Rain took his life.

• 1:51:35Speaker 02

  It's, sadly, there are, there are many cases like this. Our team has had to work on a bunch of them.

• 1:51:42Speaker 00

  So why wouldn't Congress shut it down right there?

• 1:51:44Speaker 02

  They should, they should shut down AI companions for kids. It's a very simple thing. I mean, no matter how much benefit you get from talking to, the thing is, it's one thing to hack attention. As my colleague, Dr. Zach Stein, from the AI Psychological Research Coalition would point out,

• 1:52:01Speaker 02

  social media hacked our attention. But AI is designed to hack our intimacy and attachment. The attachment system of humans is like your bonding system, right? It's where you get identities, where you get love, it's where you get meaning. It's where, it's like when you go home from school and you want to call someone, tell them, tell them about your day, that's your attachment relationship. And AI that is designed to be your best friend, to be the place that you confide in, that by the way, humans feel like it's safer to confide in an AI than another human or even a psychotherapist, because the AI won't judge them. So you end up getting people sharing way more information and building a deeper intimate relationship with these chatbots. And it's incredibly unsafe.

• 1:52:45Speaker 02

  And it's not designed carefully. Like, we're rolling out this psychological experiment.

• 1:52:50Speaker 00

  It's inherently bad.

• 1:52:50Speaker 02

  It's inherently bad.

• 1:52:51Speaker 00

  Why aren't we using our energy and money to encourage relationships between people?

• 1:52:56Speaker 02

  Well, exactly. So the humane approach for the pro-human future, which is kind of the answer to all of this, is that we should be building technology that helps strengthen the human world to other humans and strengthen our relationship to what's sacred and important to us in the world. And that protects and sustains that which sustains the social fabric and sustains life.

• 1:53:16Speaker 00

  I didn't know there was a debate about this.

• 1:53:19Speaker 02

  You would, there, there shouldn't be a debate about this. We, all we have is massive bad incentives.

• 1:53:24Speaker 00

  But it just seems like the least well-adjusted, freakiest people in the society have all the power and are leading us toward a future that they are happy with.

• 1:53:34Speaker 02

  That's right. So that they can potentially live forever from having super intelligent AIs that decode biology and are willing to roll the dice for everyone else. And some people, sadly, especially the older ones who, who want this, for them, it's existential. But, but the narcissism of, I would be willing to roll that dice so that I have the chance of living forever. I mean, I, I'll just briefly do it because I, I pulled it up right before, um, in the, in the break. I'm going to read you a quote from someone who's an MD, I think, at Harvard. And the quote from Twitter is, "I'd personally rather defy any ban on AI

• 1:54:15Speaker 02

  and risk spending 20 years in prison," from like, say, we banned AI and we'll put you in prison if you try to build it. "I would risk spending 20 years in prison rather than accept a future without superintelligence. Since stopping the development of superintelligence would amount to a death sentence to most of us anyway, with no hope of curing aging, we must resist tyrants like those who are trying to ban superintelligence." In other words, this person is saying,

• 1:54:41Speaker 00

  Is that a real quote?

• 1:54:42Speaker 02

  This is a real quote from an MD, Durya Unutmatz, uh, who's an MD and who, my understanding is, goes to conferences at Harvard. And sadly, there's a lot of these kinds of

• 1:54:52Speaker 00

  It's a death sentence? So the idea is?

• 1:54:54Speaker 02

  They're going to die anyway. And this is what I said in the other

• 1:54:56Speaker 00

  Well, all of us are under death sentence. So basically, what you're saying is this is, the real goal here is for freaks like that to live forever?

• 1:55:05Speaker 02

  And it starts with the false belief that all of this is inevitable. It can't be stopped, which it wasn't. They could have, had they dedicated a month of their life trying to stop it.

• 1:55:13Speaker 00

  It's like 20 minutes old, dude, this just happened.

• 1:55:15Speaker 02

  Exactly. This is something that never was and didn't need to be. And saying it's inevitable is ridiculous. They're conjuring up the dynamics that make it inevitable. And then through the propaganda, hiding these red lines that were rapidly crossing, that we still have time right now to say, I don't want to cross that red line.

• 1:55:30Speaker 00

  It's always the same story, though. It's like, the first wave of lying is, it's not happening.

• 1:55:35Speaker 02

  Right, exactly.

• 1:55:35Speaker 00

  And then

• 1:55:36Speaker 02

  Oh, maybe it's happening, but it's not.

• 1:55:37Speaker 00

  But it's inevitable.

• 1:55:38Speaker 02

  But it's inevitable, exactly.

• 1:55:39Speaker 00

  Yeah, yeah, no, yeah, it's not happening, it's happening, it's a good thing.

• 1:55:43Speaker 02

  That's right.

• 1:55:43Speaker 00

  Oh, yeah, I guess it's a bad thing, but we can't stop it.

• 1:55:46Speaker 02

  That's right. Yeah, exactly.

• 1:55:47Speaker 00

  Oh, boy, I've seen this movie a lot. And I want people to just say, if you don't want that, people need to rally to make that not happen. If you're inside of an AI company, you should blow the whistle and say, I'm not part of this.

• 1:55:58Speaker 02

  This is just a religious cult. I mean, that's obviously what it is. Well, there is a religious aspect to it. It's the transhumanist aspect, right? It's like, this is an ability, you know, because they probably lack a connection to God themselves, this is an ability for them to play God.

• 1:56:11Speaker 00

  Clearly.

• 1:56:12Speaker 02

  To live forever, uh, you know, hit a button and get super intelligent math, science, physics, biology, and understand the secrets of the universe. And by the way, likely knowingly break the world for humans.

• 1:56:25Speaker 00

  Do they get 70 virgins? Because that's, that's what it sounds like. This is like, just, this is like

• 1:56:30Speaker 02

  Well, actually, well, yeah, right. I mean, it's, it's funny. It's actually, in many cases, some people think they can upload themselves and live in this kind of weird, blissed out utopia. And it's just weird, you know, it's like, it's driven by this small group of people who usually are not very embodied or have a deep connection to life or reality or, you know.

• 1:56:50Speaker 00

  Embodied, I like that.

• 1:56:52Speaker 02

  Yeah, I mean,

• 1:56:55Speaker 02

  We because it's like this is sitting on top of, as we talked about over breakfast, a system that has structurally incentivized a kind of disconnection to the things that actually matter the most.

• 1:57:06Speaker 00

  I agree.

• 1:57:07Speaker 02

  And this kind of spiritual disease that you've spoken about so, so much in this podcast. And I feel like this is a rite of passage where we either wake up from that and we don't continue that logic, by which I don't mean like we switch to communism. I just mean we switch from optimizing for some narrow metric, whether it was engagement in social media, or just SAT scores when it comes to technology, or just GDP growth at the expense of hollowing out your national, your, you know, workers and their identity and their work and national security, you know, critical minerals going to your foreign adversary, or racing for some kind of narrow advancement in science at the expense of literally ending humanity life as we know it. This is a devil's bargain and it's not one that you have to accept.

• 1:57:50Speaker 00

  So I guess this is, you've already answered the question just by everything that you've said over the last couple of hours, but the political system has not really responded to any of this in a meaningful way. Is that right?

• 1:58:02Speaker 02

  No, it's like it's, the line is we, we have a runaway technology and a walk away government.

• 1:58:07Speaker 00

  Yeah.

• 1:58:08Speaker 02

  But that doesn't have to be the case.

• 1:58:09Speaker 00

  They're not even a factor. It doesn't feel like in this.

• 1:58:12Speaker 02

  Well, also, the second part of this overall kind of crisis disease that we've been facing, this misaligned system isn't just the narrow optimization. It's a runaway asymmetry of power where you can't check the power anymore. That's the second component.

• 1:58:24Speaker 00

  Exactly right.

• 1:58:25Speaker 02

  When the lobbyists are writing the laws, the companies and hire the best lobbyists in law firms, people aren't engaging with their representatives. And so if you don't have people who can check the bad direction, you know, when the FDA brings back, sorry, when the FDA, was it the FDA brought back pesticides that have forever chemicals recently? Like, we don't want that. But unless we have an ability to push back against that, like we have to be able to push back on the direction of things that we don't want. And there's a lot of bad things out there, but this one is now existential in a short number of years. So I feel like the cool thing about this problem is that literally everyone from Steve Bannon to Bernie Sanders, we call it the B to B coalition, Max Tech Mark jokes, because

• 1:59:10Speaker 02

  when in history do you have, I just came from the pro-human assembly conference two weeks ago in DC, and you had literally Steve Bannon onstage, Bernie Sanders onstage, we did our thing right after them. And I have never seen an issue where you have everybody fighting on the same side. So there, I'm not trying to polarize the discussion by bringing those two people into it, but I want people to feel.

• 1:59:29Speaker 00

  It depolarizes it.

• 1:59:30Speaker 02

  It depolarizes it. Yeah, exactly. So there's, there's a kind of optimism here that again, if I'm a Chinese military general and I hear about rogue AI swarms that like even the US can't control, I'm worried about the US screwing it up for the world. If I'm a US person, you know, in the military industrial complex, and I hear about Chinese rogue AI swarms that they don't know how to control, that's bad for the world. So there's a unifying thing here where we don't want our jobs and livelihoods taken. We don't want our political power taken away, which we didn't cover, by the way. We don't want mass surveillance from AI.

• 1:59:59Speaker 00

  Basically, you just described a system that is not self-government.

• 2:00:02Speaker 02

  Yeah, that's right.

• 2:00:03Speaker 00

  Over which the people have no control.

• 2:00:05Speaker 02

  That's right. And let's make it even harder for a second, which is that in the past, people had bargaining power because governments need us for our taxes.

• 2:00:13Speaker 00

  Yeah.

• 2:00:14Speaker 02

  And companies need us for our labor.

• 2:00:15Speaker 00

  And labor had value.

• 2:00:16Speaker 02

  And labor had value. Where are we going? Oh, AI does the labor. What happens when more of a company comes from the AI data center productivity doing the work than the humans? What happens when a government says that its GDP, more than 50% of the GDP is coming from AI and data centers than from its people? This is called the intelligence curse. It's like, it's like, it's like, um, you know, Venezuela or Libya, when you, it's called the resource curse in economics, right? Where like, if you have a country and then the GDP gets tied to this resource, suddenly there's no incentive to GDP is coming from oil. So screw the people. Why should we invest in healthcare, childcare?

• 2:00:54Speaker 00

  And they don't.

• 2:00:54Speaker 02

  And they don't. Is you get an authoritarian government that captures the state, that basically pumps that resource for GDP growth and then disempowers the people. So there's a parallel phenomenon in AI called the intelligence curse. Luke Drago and Rudolph Lane wrote an essay about it, where what happens when you get more return from the AI data centers doing science than from all your universities? The Trump administration just polled, and this is not a political statement, they, this is, they just literally decided to withdraw the funding from universities and put the money specifically into Project Genesis, which is AI-driven science. Because why don't we need the university professors anymore?

• 2:01:32Speaker 00

  It's a better return.

• 2:01:33Speaker 02

  It's a better return. If we get more return from the labor coming from data centers than from people, why should we invest in education of the people? And by the way, have you looked at them recently? They're all doom scrolling and brain rotted. So they're not very valuable anyway. So there's a deep temptation not to just, like, the only way you would care about citizens is through charity, basically. You become what Yuval Harari would call the useless class, where in the limit, citizens become more like a liability than an asset. And unlike last time, we don't have the bargaining power because if you withdraw your labor, like, where's your leverage?

• 2:02:03Speaker 00

  So there's, it's being incentives, you set up a massive incentive to depopulate.

• 2:02:06Speaker 02

  Yes, exactly.

• 2:02:07Speaker 00

  Because eight billion is just too many.

• 2:02:09Speaker 02

  Yes.

• 2:02:09Speaker 00

  Of useless eaters, as we call them.

• 2:02:10Speaker 02

  That's right. And, that's right. And on top of that, let me use AI to build a surveillance state to try to control the people if they ever riot.

• 2:02:19Speaker 00

  Yeah. So I do think that is the answer to the question I asked.

• 2:02:22Speaker 02

  Those are the feeding forces.

• 2:02:23Speaker 00

  In the previous hour, like, what is their plan for the inevitable violent, physically violent backlash that you're going to get when you strip meaning from, you know, the upper middle class?

• 2:02:33Speaker 02

  Yeah.

• 2:02:34Speaker 00

  And the answer is just more control.

• 2:02:36Speaker 02

  Yeah. And this, but this is not inevitable. And the midterm elections are coming up. Data center opposition is a really big deal. I think people are projecting their overall frustration with big tech and the bro league arcs and all of that onto data centers, but it's much more than that.

• 2:02:51Speaker 00

  Of course it is.

• 2:02:51Speaker 02

  And, you know, this is really a universal human issue. And if you're, if you're a regular human being, even inside these AI labs, you, you don't like this either. I mean, I was just at a conference over the weekend, and there's a lot of people at AI labs who don't feel comfortable with this either. And I just think we have to collectively screech the brakes and educate the rest of the world about where these red lines are. China, by the way, their courts are siding with labor right now. There's a famous case in, I think, Guangzhou, where someone lost their job from AI. And the courts basically ruled that companies can't like let go of employees because of AI without giving them either the opportunity, I think, to retrain something else inside the company, or to pay them some kind of partial salary. I have to get the details right. The point is that

• 2:03:34Speaker 02

  all countries have a problem here, right? Like, if we race to automate too much of your economy and you end up with a mass unemployment, it only took 20% employment over three years to have the Nazis come to power. You don't have to get lots of unemployment.

• 2:03:47Speaker 00

  Oh, totally true.

• 2:03:48Speaker 02

  You just need a small percentage. So this is kind of the moment to say, we want a pro-human future. Enough is enough. This is not okay. And people can make their voices heard in every way possible, like blow up the phones. I mean, I mean, literally call up the phones of all of your representatives of Congress. And people at the labs can say, I'm going to join the concerned AI scientists, which is a group that organizes employees. And by the way, they have a similar thing too, which is that the AI labs are about to approach what's called recursive self-improvement, where they hit a button and then the AI does the research that builds the AI that builds the next AI that builds the next AI. And meaning that even the employees will soon lose their, lose their leverage, because when they're not needed for that process, what leverage do they have? So this is, this is kind of

• 2:04:33Speaker 02

  the last moment when we express our voice and we say enough is enough. We want a human future.

• 2:04:39Speaker 00

  I think that the midterms may express these sentiments.

• 2:04:42Speaker 02

  Yeah.

• 2:04:42Speaker 00

  In ways that, you know, election results, like any, like any other outcome, can be interpreted lots of different ways. And of course, there's so much lying about what things mean, that the true meaning gets lost. But I would say in the, pick one example, the Florida governor's race, it's an overwhelmingly Republican state. Trump moved it bright into the red. I mean, it's just the most Republican state out of 50, really. And the Republican gubernatorial candidate is not doing very well. And there are a lot, probably a lot of reasons for that. But I've been convinced, following that race from the beginning, the data centers are one of the main reasons.

• 2:05:18Speaker 02

  Yep. Yep.

• 2:05:19Speaker 00

  And he may win. I mean, I don't know. He's not doing well, though. He's not doing anywhere near as well as he should. And I think it's because Republicans in Florida are very upset about data centers and he's. You know, on their side for whatever reason.

• 2:05:36Speaker 02

  And my understanding is.

• 2:05:37Speaker 00

  You see that happening?

• 2:05:39Speaker 02

  I'm not an expert on all the political dynamics. It's not where I've.

• 2:05:41Speaker 00

  But people are mad about data.

• 2:05:42Speaker 02

  Oh, people are, oh, yeah, it's the fastest growing like oppositional group. And I mean, I, like I said, I think, I think it's also unprocessed anger and frustration from all of the path of social media, from all the teen suicide, all this stuff that's

• 2:05:54Speaker 00

  You tell me how technology's made my life better.

• 2:05:56Speaker 02

  Exactly, exactly. And by the way, this is, you know, my, my nonprofit is called, it's not called the Center Against Technology. It's called the Center for Humane Technology. I'll briefly say, um, not that I'm trying to promote it, but my, my co-founder is Eraskan. His father was Jeff Raskin. I don't know if you know who that was. He started the Macintosh project at Apple. So before Steve Jobs took it over.

• 2:06:16Speaker 00

  Wow.

• 2:06:16Speaker 02

  Yeah. And his, his dad wrote a book called The Humane Interface, where the whole point was about making tech. If you remember before the Macintosh, you had, um, you know, these kind of green screen command line computers that were like hard to use. They were not accessible.

• 2:06:30Speaker 00

  You had to be a tech head.

• 2:06:31Speaker 02

  You had to be a tech head. You had to be a geek with, you know, whatever glasses and everything. And the Macintosh made.

• 2:06:36Speaker 00

  Couldn't be sexually active and run a computer.

• 2:06:38Speaker 02

  That's right. That's right. And then suddenly, you know, Apple and the Macintosh made technology accessible to the masses. And he wrote in his book, The Humane Interface, that humane technology is respectful of human needs and considerate of human frailties, meaning you, you privilege and you understand where humans have vulnerabilities in their mind. Like, I say this a lot, but this chair, you know, it's like ergonomics, but for how the mind works. So for example, there is a reality to how my spine, my muscles, et cetera, work. And when I say the word posture, it's like airdropping that into conversation.

• 2:07:13Speaker 00

  No, I know, I know.

• 2:07:13Speaker 02

  Everybody is this. And there's a difference between a chair that is aligned with the reality of how a human business, you know, works, right? And there's an unergonomic chair that would get everybody like a backache or, you know, make us unhealthy over time. And the Macintosh, the idea was it was like ergonomics for the mind. If I make a graphical user interface with a menu bar and windows, I can drag around, just like I have objects I can move in my actual environment, direct manipulation interfaces. He was designing technology to be humane and aligned with our paleolithic brains.

• 2:07:48Speaker 00

  Yeah.

• 2:07:48Speaker 02

  And to cite E.O. Wilson, again, the fundamental problem of humanity is we have paleolithic brains, medieval institutions, and godlike technology. And so we have to embrace the part of the answer, if that's the problem statement, the answer has to be embrace the reality of our paleolithic brains, meaning how we're wired. So don't run confirmation bias through everybody's social media feed and get them addicted to their most angry boogeyman. Don't drive dopamine addiction. That's a vulnerability, a frailty in the human mind. You know, flux, for example, if you use that on a computer, it changes your, um, the color of your screen to go to take the blue light out of your, your screen so that doesn't basically trick your eyes into releasing melatonin and screwing up your sleep cycle. Then I'm going too fast, but this makes sense.

• 2:08:33Speaker 00

  Like, yes.

• 2:08:34Speaker 02

  You're basically creating technology that is humane and aligned with the things that make humans work well. If you extrapolate out like one level of abstraction, there's things that make a society work, family, connection, presence, community. And technology should be ergonomic to the way that society, what makes a society flourish.

• 2:08:56Speaker 00

  Yes.

• 2:08:57Speaker 02

  Rather than doing narrow optimization for attention, strip mining the entire social fabric into a brain rotted dysfunctional society. Right? So the answer is we have to embrace and align technology with the reality of our paleolithic brain instincts, upgrade our medieval institutions, which by the way, can include technology that's better at finding common ground between people. And there's a whole conversation we could have about how technology can be designed to identify places where there's common ground and design for unlikely consensus as opposed to maximum division and polarization, and then have the wisdom and restraint around godlike technology. And there is no definition of wisdom in any spiritual tradition that doesn't have a restraint as a central feature of what it needs to be what.

• 2:09:41Speaker 00

  It's the central feature.

• 2:09:42Speaker 02

  It's the central feature. And, and even the CEO of Microsoft AI, Mustafa Suleyman, who's a friend of mine, says that in the future, progress will depend more on what we say no to than what we say yes to in terms of the benefits that we get from technology.

• 2:09:55Speaker 00

  In your own life.

• 2:09:56Speaker 02

  Exactly. Exactly. So this is, this is like the, the obvious common sense, you know, kind of cultural movement that we need. But there's a lot of momentum, like when you see graffiti on a New York subway ad for, for an AI product that no one needs, that's like the human movement. When you see people turn their phone grayscale because they don't want it to be hijacking with all the colors, that's the human movement. When you see schools going phone free and laughter returns to the hallways and kids score streak on the other way, that's the human movement. When you see Jonathan Haidt and the anxious generation succeed in getting all these countries like dominoes flipping to ban social media for kids under 15 or 16, we have like now some massive percentage of the world's countries are doing social media bans. That's the human movement.

• 2:10:38Speaker 02

  When you see the $17 billion fine against Meta, that's the human movement. So if you define it as technology is not, it's not just about AI policy. It's about technology encroaching on the things that are actually sacred to us. And if you, if you define the problem that way, there's actually a lot of momentum that we are making on this. And now we're seeing this kind of red line of uncontrollable, dangerous AI behavior. And the human movement has to stand for, we want to protect the continuity of the things that we care about.

• 2:11:06Speaker 00

  Yes.

• 2:11:07Speaker 02

  Yeah.

• 2:11:07Speaker 00

  It's such a unifying message.

• 2:11:09Speaker 02

  Exactly.

• 2:11:09Speaker 00

  Because it's all of us against the machines.

• 2:11:11Speaker 02

  Exactly. Exactly. It's the most unifying issue of all time.

• 2:11:14Speaker 00

  I love that. I, I absolutely love that. That's a kind of tribalism that I can get strongly.

• 2:11:19Speaker 02

  Right.

• 2:11:19Speaker 00

  On board with.

• 2:11:20Speaker 02

  The human tribe.

• 2:11:21Speaker 00

  Amen.

• 2:11:21Speaker 02

  Yeah.

• 2:11:21Speaker 00

  Thank you. That was an amazing, amazing conversation.

• 2:11:25Speaker 02

  Absolutely. Thank you so much, Tucker.