Why AI CEOs Are Building Bunkers - Tristan Harris

Tristan Harris is a tech ethicist, entrepreneur, and a speaker. Are we sleepwalking into disaster? AI is unlocking massive progress, but the dangers hiding beneath the surface are exactly what experts fear most. So what’s coming… and could it spiral beyond our control? Expect to learn why AI is distinct from other kind of technologies, what the Alibaba rogue AI catastrophe that should scare everyone is, how worried Tristan is about the impact of AI deepfakes and misinformation campaigns, what’s happening with the AI safety discussion, if we should be skeptical of AI companies pushing just as hard but pretending that they’re not, the end result that AI companies are looking for and much more… - Get up to 20% off the leading longevity and cellular health supplement at https://timeline.com/modernwisdom Get up to $350 off the Pod 5 at https://eightsleep.com/modernwisdom Get a Free Sample Pack of LMNT’s most popular flavours with your first purchase at https://drinklmnt.com/modernwisdom New pricing since recording: Function is now just $365, plus get $25 off at https://functionhealth.com/modernwisdom - 0:00 Can Life With AI Have a Positive Outcome? 6:56 Is AI the Most Powerful Force Ever Created? 16:07 Powerful But Not Wise: AI’s Biggest Flaw 19:11 Could AI End Up Destroying Itself? 24:30 How Social Media Drifted Away From Human Flourishing 29:09 Are We Moving Towards an Anti-Human Future? 36:53 Who Funds AI Once It Replaces Us? 40:58 Why Best-Case Scenario is Still Concerning 53:33 The Alibaba Blackmail Scare Explained 01:04:01 Can We Really Stop AI Taking Over? 01:13:04 The Danger of Denial in the AI Era 01:20:19 Are AI’s Benefits Blinding Us to the Risks? 01:26:01 We Need to Face the Reality of AI 01:31:56 Are AI Companies Controlling the Narrative? 01:33:31 How Close are We to an AI Takeover? 01:35:39 Why Changing AI Feels Impossible 01:42:30 Total Control or Total Collapse: Where Are We Headed? 01:46:23 Can the World Coordinate on AI Safety? 01:52:40 Why Elon Musk Isn't in The AI Doc 01:59:18 Every Second Counts Now 02:03:58 How Do We Accelerate Meaningful Change in AI? - Check out The Human Movement: https://www.thehumanmovement.org/ Get access to every episode 10 hours before YouTube by subscribing for free on Spotify - https://spoti.fi/2LSimPn or Apple Podcasts - https://apple.co/2MNqIgw Get my free Reading List of 100 life-changing books here - https://chriswillx.com/books/ Try my productivity energy drink Neutonic here - https://neutonic.com/modernwisdom - Get in touch in the comments below or head to... Instagram: https://www.instagram.com/chriswillx Twitter: https://www.twitter.com/chriswillx Email: https://chriswillx.com/contact/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Chris Williamson?

  What is the journey of how you arrived thinking about the problems of AI?

• 0:08Speaker 01

  Well, most people know me or our work through the film The Social Dilemma. And I used to be a design ethicist at Google in 2012, 2013. So that basically meant, how do you ethically design technology that is going to reshape, especially the attention and information environment of humanity? So it's like, there I was at Google, it was 2012, 2013. This is in the heat of the kind of social media boom. I think Instagram had just been bought by Facebook. My friends in college started Instagram. So like, I was part of this cohort and milieu of people who really built this technology that the rest of the world just thought was natural. Like, this is just drinking water. Like, I just drink Instagram. I just live in this environment.

• 0:53Speaker 01

  And so while like, I saw billions of people enter into this psychological habitat that I knew the handful of like, five or six people that were designing and tweaking it and making it work a certain way. Yeah, exactly. And I think that that's just like a fundamental thing I want people to get is, you know, you think of technology like it just lands and it's just inevitable and there's just nothing we can do and it just comes from above. And it's like, there are human beings making choices. And, you know, as someone who grew up in the era of, you know, the Macintosh, like my cofounder, so I have a nonprofit called the Center for Humane Technology. My co-founder Azar Raskin, his father invented the Macintosh project before Steve Jobs took it over. So this is the original Macintosh, you know, the thing that we now, the MacBook, the iMac, the MacBook Pro.

• 1:36Speaker 01

  All of that started with his father, Jeff Raskin. And the idea of creating humane technology where technology could be choicefully designed to be really easy to use, to be accessible, to be an empowering extension of our humanity, like a cello, like a piano, like a creative tool. Like, if you're a video person, you can make films and videos. And just so people understand, because we're going to probably be talking about some darker things on this podcast, the premise of all this is not to be a speaker of doom or something like that. It's to say, I want to live in a world where technology is in service of people and connection and all of the things that matter to us as humans, and then have technology wrap around ergonomically us to create that. So that was kind of a side journey.

• 2:22Speaker 01

  There I was at Google in 2012, 2013. And I saw how essentially there is this arms race for human attention. And whichever company was willing to go lower on the brainstem to manipulate human psychology. This is exploiting like a backdoor in the human mind. So I think of it just like software has backdoors and zero day vulnerabilities. You can hack software. The human mind has vulnerabilities. And as a magician as a kid, I understood some of those. Studying at a lab at Stanford called the persuasive technology lab, where a lot of the Instagram co-founders had studied. I understood the psychological influences dynamics. And so it wasn't just that we were making technology in this beautiful and empowering kind of Macintosh way.

• 3:04Speaker 01

  It's that basically more and more of my friends were sucked into developing technology to hack human psychology. And so I saw that problem, like being concerned about it. And I made a presentation at Google. And I feel like I repeat the story everywhere, but it's just important for my history, I guess. I made a presentation saying never before in history have 50 designers in San Francisco, basically through their choices, rewired the entire psychological habitat of humanity. And we need to get this right. We have a moral responsibility to get this right. And I sent it to 50 people at Google. And when I clicked on the presentation the next day, on the top right of Google slides, it shows you the number of simultaneous viewers. You know how that works? And it had like 150 simultaneous viewers and then 500 simultaneous viewers.

• 3:49Speaker 01

  And so it's like, oh, this is spreading throughout the whole company. And that's what led to me becoming a design ethicist, where I had to research and ask the questions. What does it mean to ethically design and persuade people's psychological vulnerabilities? When you, you can't not make choices about the psychological habitat. You have to make a choice about how infinite, whether you're going to do infinite scroll or not, or autoplay or not, or notifications or not, or these 10 people followed you or not. Like, what does it mean to ethically make those choices?

• 4:18Chris Williamson?

  That is you being concerned about some of the ways that a misalignment of technology with what human flourishing might look like.

• 4:25Speaker 01

  Yeah. And what's how society, I think people are afraid to say like, when you make a bridge, there's a physics to whether that bridge will sustain or whether it will fall apart, right? Like, and it's not magic. We don't say, oh, like, who would have known that that bridge would fall apart? And we have a science of bridges and mechanical engineering and civil engineering. And with technology and human psychology, there's a science to the dopamine system. There is a science to confirmation bias in our psychology and how we tend to perceive information through our tribal in-group, like we see things through the political tribe that we're a part of. And if you understand that science, you can understand whether or not technology is manipulating that. So one of the core things I think we were trying to do in that first chapter of work, and this again, starting

• 5:08Speaker 01

  in 2013, is break through this idea that technology is neutral and that we could never know what's good for people or that something could be bad for people. Like, I deliberately saw people make short-form, autoplaying videos that then created the brain rot economy that we're now living in.

• 5:25Chris Williamson?

  Hmm. And it seems like a natural progression to go from, I'm concerned about some specific types of technology use and how that interacts with humans to.

• 5:38Speaker 01

  It's specifically not technology use, but technology designed for certain outcomes of usage. Really critical thing, because we want to put attention on the design, not just how people are using it.

• 5:49Chris Williamson?

  Yep, understood.

• 5:50Speaker 01

  Yeah.

• 5:53Chris Williamson?

  Seems like a natural progression to get concerned about a burgeoning AI landscape.

• 5:59Speaker 01

  Well, what happened was, my team at Center for Humane Technology, our nonprofit, we got calls from people inside of the AI labs. So, you know, we were in San Francisco. We know people work at all the tech companies we have for the last decade. And suddenly, in January of 2023, this is 10 years later now, I got calls from people inside the major AI lab saying that the arms race dynamic was out of control and that huge leaps in capabilities, this is basically speaking about GPT-4 before it came out. And GPT-4 could pass the bar exam, get very high results on the MCAT, was producing incredibly powerful, like passed the SATs, like very powerful AI that suddenly appeared out of nowhere.

• 6:43Speaker 01

  And this people who reached out to us basically said, this is really dangerous. Will you use your connections? Use your connections in DC? Go wake up the world, wake up the institutions, let them know that this is coming because it's not safe what's about to happen.

• 6:57Chris Williamson?

  Why is AI distinct from other kinds of technologies?

• 7:01Speaker 01

  Well, let's get to that. So I think the thing that is most difficult for people to get is,

• 7:08Speaker 01

  up until now, technology progressed in a very, like, we're kind of adding layers to a stack kind of way. Like, we build the networking stack. We build the user interface stack. And as you develop the stack, you're kind of just adding layers and layers and layers. And the technology that we live in was coded manually, like line by line. Like, when the computer sees this, do this. When the computer sees this, do this. And then people contribute all this code over 30, 40, 50 years on GitHub and in operating systems. And then you land in this technological world in which everything that happens in a computer is happening through logic and through human choice. What makes AI different is that you're designing and you're not really coding it like, I want it to do this. You're more like growing this digital brain that's trained on the entire internet. And when you grow the digital brain,

• 7:54Speaker 01

  you don't know what it's capable of or what it's going to do. So think about it this way, like if I did a brain scan in your brain, could I know from just the brain scan what you're capable of? No, I can see that this part of your brain lights up when you have that thought, but I can't have a comprehensive picture of like, what is everything that Chris is capable of? Can he do sociopathic manipulation and do better military strategy than the best US generals? Like, from the brain scan, I can't tell that. Maybe you can, but. So with AI, we are essentially, you know, when people hear about these huge data centers getting built out, like Facebook's building one or Meta's building one the size of Manhattan. And you ask like, what is that, what's going on there? It's like they're building a bigger

• 8:36Speaker 01

  and bigger digital brain that that's what goes from GPT three to GPT four, you know, with more neurons. When you hear the number of parameters of an AI model, that's like essentially the number of neurons in an AI model. And what they found is that the more GPUs and Nvidia chips you point at, at sort of growing this digital brain, the more intelligent it gets, and the more it picks up capabilities that we didn't intentionally teach it. Like there was a famous example where you just train it on the internet and then, you know, it's answering questions in English and suddenly it learns how to answer questions in Farsi, like doing Q and A in a different language. And no one taught it that language, it just sort of learned that on its own. And that's what's like weird about AI is

• 9:18Speaker 01

  that it's a black box. We don't really understand how it works and yet we're making it more powerful, much faster than understanding how it works. And that's what leads it to make these more unexpected behaviors that we aren't able to control. And I think we're going to get into some of those.

• 9:34Chris Williamson?

  A data center the size of Manhattan?

• 9:37Speaker 01

  Yes.

• 9:38Chris Williamson?

  Where?

• 9:39Speaker 01

  Uh, I don't remember where that one is, but it's, it's crazy. There's like a overlay, someone can look it up. There's like an overlay where you can see the size of this data center and it's almost the size of Manhattan. And you can ask, I mean, again, there's more money people should just get, there's trillions of dollars going into this. There's more money going into this technology than all technologies of the past have ever been built. And we're releasing this technology faster than we've released every other technology in history. It took something like two years for Instagram to go from zero users to a hundred million users. And it took two months to go from zero to a hundred million users for ChatGPT. And of course, they're going from ChatGPT three or four to now at 5.2. And it went from barely being able to finish a sentence with ChatGPT two, like finish a

• 10:23Speaker 01

  paragraph and do like a coherent text, to GPT three could write full essays, to GPT four can pass the, you know, the bar exam or the MCATs, to GPT 5.2, I believe was used to get a gold in the Math Olympiad.

• 10:37Chris Williamson?

  Meta's Hyperion AI data center will sprawl to four times the size of Manhattan's Central Park.

• 10:44Speaker 01

  And there are quotes from people like inside of OpenAI who believe that they're not just building this like narrow technology that's a helpful blinking cursor. They want to build artificial general intelligence. And so what that means is being able to do that everything that a human mind can do. And the joke inside the company is like, we're going to cover the world in data centers and solar panels. Like, they want to cover the world in essentially these big boxes that have huge clusters of Nvidia chips that then compute away and ultimately create something like a super intelligent god entity that they believe that they will use to own the world economy, make trillions of dollars, and from a kind of ego religious intuition, they will have built the god that supersedes

• 11:28Speaker 01

  and replaces humanity. I know that sounds insane, so let's we can slow that down again.

• 11:32Chris Williamson?

  Break that down for me.

• 11:33Speaker 01

  That's, that was a lot.

• 11:34Chris Williamson?

  Yeah, yeah, yeah. Yeah. You've got a new movie out, and I feel like I found out who the bad guy is, but I have no idea how he got there.

• 11:40Speaker 01

  Who's the bad guy?

• 11:41Chris Williamson?

  The end of the world AGI overlords.

• 11:46Speaker 01

  Well, yeah, so okay, so first of all, let's like break this down because this might sound ridiculous to people. Let's make sure people understand. The stated mission statement of OpenAI is to build artificial general intelligence, which means to be able to replace all forms of economic cognitive labor in the economy. Cognitive labor, meaning cognitive, anything your mind can do. So if a mind can do math and generate new mathematical insights, if a mind can do physics like Einstein, if a mind can do chemistry, if a mind can do programming, if a mind can do cyber hacking, if a mind can do marketing, if a mind can illustrate something, we're seeing AI that is able to kind of cover more and more types

• 12:29Speaker 01

  of cognitive labor in the economy. As we scale AI from this tiny little model with, you know, 100 million parameters to trillions of parameters, and these huger, these much bigger data centers, AI is getting closer and closer to be able to in already beating humans at many cognitive tasks. We already have AIs that are better at military strategy than the best military generals. People remember, you know, in 1990s, IBM Deep Blue beat Garry Kasparov at chess. That was kind of like the beginning of like, it can beat you in this narrow game called chess. Then there was AlphaGo. We can have AI that beats the best human Go player in the Asian board game of Go.

• 13:10Speaker 01

  But then now, instead of imagine chess or Go or Starcraft, now it's like the war in Iran. And you have an AI that's basically telling the military troops where to go, who to bomb. This is really scary. And we're racing to this outcome faster than we've, again, built any other technology in history.

• 13:28Chris Williamson?

  You said that it's better, better than humans.

• 13:31Speaker 01

  Better in the narrowly defined sense of effective at strategy, effective at goal achieving, effective at problem solving. Because that's what is intelligence, right? It's like finding the shortest path between a goal and what are the strategies that get you to that goal? So persuasion is a kind of strategy or intellectual task. What is the best way to persuade you the shortest path? Negotiation is a problem solving task. And lawyers find ways of lying or framing the truth in certain ways. Well, AI is going to discover forms of deception are lying. And we're seeing that in the examples that I think we're going to talk about. And so, but intelligence is different than wisdom. And your podcast is called Modern Wisdom.

• 14:15Speaker 01

  And I hope we get into this distinction because we are scaling up the amount of power that everyone is going to have access to, whether it's individuals or militaries or nation states or companies or businesses. But we are not commensurately scaling the amount of wisdom. And I know a friend of ours that we we met in Austin several years ago, a dear friend of mine, Daniel Schmachtenberger, has this quote that you cannot have the power of gods without the wisdom, love and prudence of gods. And so in many ways, I think AI is like a rite of passage for humanity because essentially we've always been poor. We've not always had the great, greatest track record in our relationships to technology. Like if you look at the industrial revolution tech, what what letter grade would we give ourselves in?

• 15:00Speaker 01

  Holding on in like stewarding that tech. You know, we had better living through chemistry in the 1930s, DuPont chemistry. And that was great. We invented all sorts of new materials, but we also generated forever chemicals. And it would it would currently cost more than the GDP of the entire world to clean up the entire mess of forever chemicals.

• 15:21Speaker 01

  You know, we created social media, thinking if we give the world access to information at our fingertips and connect people with their friends, this is going to create the most enlightened and informed society than we ever have. And clearly, that didn't go that in the way that we we wanted it to. So now AI is like the exponentiation of just technology invention writ large. Because what makes AI different from all other forms of technology is that intelligence is the basis of all of our new science, of all of our new technology, of all of our new military development. So if you automate intelligence, you're going to automate an explosion of new science, new technology, new military technology. And if you have more power and more intelligence, but you don't have the wisdom to wield it,

• 16:06Speaker 01

  that's obviously not going to go well. Why can't wisdom be programmed to?

• 16:12Speaker 01

  Well, in some ways, you could say that it can be. It's just that it's it's not that like wisdom comes from the ether. You have to. It's about asking critical questions about how should the technology be designed? So for example, like,

• 16:28Speaker 01

  do we have to have our entire internet environment have auto playing videos that swipe one after another? No, we don't have to have that. We can have a totally different design paradigm where no one's auto playing videos. Wisdom would be understanding that the human psychological, the the Paleolithic brain that we are born with has these vulnerabilities in our dopamine system. And we could design to not hijack that dopamine system. And just imagine for a second, just to like, there's a huge conversation we're having, but if you just imagine that one little change. So here's today, everyone has auto playing videos infinitely swiping, brain rotting everybody, brain damaging everybody 24/7. Test scores are massively down for basically all around the world because of this phenomenon. It's very, very clear that the technology and social media is driving that.

• 17:14Speaker 01

  If you make this one little change of no auto playing videos. And means also no, you know, no infinite swipe dating apps that are getting you into a slot machine with player cards of people. Like, how different does the world become? Like, when you meet people, how dysregulated is their nervous system? Just that one little change. I want people to think as we're in this conversation, there's just these different worlds we can live in with just different design choices. And that's kind of the whole point, is that wisdom can be, what are the design choices that will lead to better societal outcomes? And of course, the reason that everyone's autoplaying the videos is because of this competitive arms race of if I don't do it, I'll lose to the other company that will.

• 17:53Chris Williamson?

  Yep.

• 17:54Speaker 01

  And so it would take some kind of rule or policy that says that we don't want that.

• 17:58Chris Williamson?

  You need to put a moratorium on autoplay videos.

• 18:01Speaker 01

  Yeah.

• 18:02Chris Williamson?

  Because the incentives for any individual company and for the market at large and for the competitive dynamic between companies means that if you don't do it, you get beaten by the one that does.

• 18:12Speaker 01

  And that's, that's the, that's like the bullseye, that's like the fundamental problem behind AI that's forcing us to reckon with is unhealthy competition, or this sort of, if I don't do it, I'll lose to the guy that will. So everyone does a thing that's short term good for them, but that's long term bad for everybody. The AI companies, well, even Anthropic wants to be the safety AI company. They want to do things in a safer, more careful way. But they, if they don't release models as powerful and as fast as the other companies, they'll just fall behind in the race. They won't have a seat at the policymaking table. They won't get a lot of usage. They won't get the investor dollars. And then their, their commitment to safety just means they lose and they're not part of the race anymore.

• 18:55Chris Williamson?

  Yeah. What's that line? How can you talk shit from outside of the club you can't even get in?

• 19:00Speaker 01

  Yeah.

• 19:01Chris Williamson?

  It's difficult to.

• 19:02Speaker 01

  Something like that, yeah.

• 19:02Chris Williamson?

  Yeah, yeah, yeah. I think, is it, was that J. Kwan, I think, who's dating me in like the mid 2000s. There's a study that I saw recently, scientists just proved that large language models can literally rot their own brains, the same way humans get brain rot from scrolling junk content online. Do you see this?

• 19:21Speaker 01

  I did see that, yeah.

• 19:22Chris Williamson?

  Yeah. Scientists did a study where they fed models months worth of viral Twitter data, shorts, high. Engagement posts and watch their cognition collapse. Reasoning fell by twenty three percent. Long term context memory dropped by thirty percent. Personality tests showed spikes in narcissism and psychopathy. And get this, even after retraining on clean, high quality data, the damage didn't fully heal. The representational rot persisted. It's not just that bad data means bad output. It's bad data means permanent cognitive drift. The AI equivalent of doom scrolling is real and it's already happening.

• 19:57Speaker 01

  I love that you included this example. That's from right here, University of Texas in Austin, Texas A and M University.

• 20:03Chris Williamson?

  Leave it there, Jared.

• 20:05Speaker 01

  Yeah, so I mean, are we surprised by this? I mean, are you surprised by this when you see this?

• 20:10Chris Williamson?

  No, I can tell the difference. This year, one of my big resolutions has been to spend less time on social media. I managed to do it and.

• 20:21Speaker 01

  How'd you do it?

• 20:23Chris Williamson?

  Second phone. That's tethered to wifi and that is the cocaine phone. And the kale phone is just messages and stuff. It's a little bit of a challenge because things like Slack, I had Cal Newport on.

• 20:37Speaker 01

  Oh, yeah, sure.

• 20:38Chris Williamson?

  A couple of weeks ago and he was, I was talking about the intersection of productivity and detention.

• 20:42Speaker 01

  Yeah.

• 20:43Chris Williamson?

  With AI, the new world of AI. And that's a really interesting conversation. Have you ever spoken to Cal?

• 20:47Speaker 01

  Yeah, yeah, yeah. He and I have been in similar circles for a long time.

• 20:50Chris Williamson?

  He's wonderful. And even if you have your phone without social media, you still have kind of the social media of work.

• 20:58Speaker 01

  Yeah.

• 20:58Chris Williamson?

  Right?

• 20:58Speaker 01

  Exactly.

• 20:59Chris Williamson?

  But anyway, I've kept, I've done good stuff on that and I've come up with some of my best ideas so far this year. My writing's improved, my sleep's improved, my attention's improved. And this is already someone that was pretty red pilled on tech minimism. You know, I think the seventh or the eighth episode of this show was inspired by you. And it was Kai Wei, the guy that invented the light phone.

• 21:21Speaker 01

  Oh, yeah.

• 21:21Chris Williamson?

  Uh, and this is two thousand and eighteen.

• 21:24Speaker 01

  Yeah, totally.

• 21:25Chris Williamson?

  So I've been concerned, I read Superintelligence twenty seventeen.

• 21:29Speaker 01

  Oh, wow. Yeah.

• 21:29Chris Williamson?

  Um, I listened to Superintelligence in twenty seventeen. Reading it would have been a little bit more difficult for me. But yeah, I don't feel as good when I use too much social media. I don't feel as good.

• 21:41Speaker 01

  And that's the thing, I mean, like, is that a controversial fact? Do you think that anybody, when they're sitting there doom scrolling for three hours, just like put a thermometer, you know, pain, you know, positive emotion meter in their brain? Are people going to say that they love, I mean, it's one of those things where it's short term good, it feels good in the moment, but it's long term empty. Like it's just this high fructose corn syrup for our brain. Empty calories.

• 22:04Chris Williamson?

  Mm hmm. And the fact that this is replicated by

• 22:11Chris Williamson?

  social media can even warp an AI, can even warp an LLM, I think feels quite pernicious.

• 22:19Speaker 01

  Yeah. I mean, it's, it's interesting to note that when Elon bought Twitter, you know, he was already thinking about AI. And part of what he was thinking about is, you know, in the AI race, these companies that are racing to get to artificial general intelligence, one of the ways they differentiate themselves from each other is their training data. Like, who has more powerful and more training data for their, for training and growing their digital brain than the other guy? And Elon thought that he had a competitive advantage because he would have the entire user generated content of the real time views of all of humanity in the form of Twitter, and he could train his AI on that. And that's what led to Grok. But of course, when you train essentially an AI on kind of brain rotted, hyper polarized,

• 23:04Speaker 01

  hyper adversarial rival risks, you know, all the problems with Twitter, the outrage economy, you get AIs that are more like this than, than the better AIs.

• 23:13Chris Williamson?

  Before we continue, most people in their 30s are still training hard. Their protein is dialed in. They sleep better than they did in their 20s. Discipline is not the issue, but recovery feels somewhat different. Strength gains take a little longer. The margin for error starts to shrink. And that is why I'm such a huge fan of Timeline. You see, mitochondria are the energy producers inside of your muscle cells. As they weaken with age, your ability to generate power and recover effectively changes, even if your habits stay strong. Mitopure from Timeline contains the only clinically validated form of erytheline A used in human trials. It promotes mitophagy, which is your body's natural process for clearing out damaged mitochondria and renewing healthy ones. In

• 23:55Chris Williamson?

  studies, this supported mitochondrial function and muscle strength in older adults. It's not about pushing harder, it's about actually supporting the cellular machinery. Underneath your training, if you care about staying strong into your 30s, 40s and 50s and beyond, this is foundational. Best of all, there is a 30 day money back guarantee plus free shipping in the US and they ship internationally. And right now you can get up to 20% off by going to the link in the description below or heading to timeline.com slash modern wisdom and using the code modern wisdom at checkout. That's timeline.com slash modern wisdom.

• 24:28Chris Williamson?

  And modern wisdom at checkout. Okay, so the discussion around social media was there are better and worse design choices that can be made.

• 24:37Speaker 01

  Yep.

• 24:37Chris Williamson?

  That would help human flourishing.

• 24:38Speaker 01

  Yeah.

• 24:39Chris Williamson?

  Broadly, what would people want their world to be like? And how can we design technology in a way that helps them to get there?

• 24:45Speaker 01

  Yeah.

• 24:45Chris Williamson?

  Something close to that. But because of market dynamics, you have a competitive landscape that incentivizes things that are effective for gripping people's attention, but not necessarily effective for flourishing. And it seems that a tension between what is good for flourishing, because it could be, what's good for attention would all should be good for flourishing.

• 25:09Speaker 01

  It could be.

• 25:09Chris Williamson?

  It could be, but it tends to not be that way.

• 25:12Speaker 01

  And also there's going to be a limit on that, right? Like it's probably not the case that 10 hours of attention on any social media is good for society or good for you. So there's going to be sort of a

• 25:23Chris Williamson?

  Waking up with some meditation app, 10 hours once every month or something would probably be quite good to do for a meditation.

• 25:29Speaker 01

  Maybe, sure. Yeah. But I think the point is like, how much as companies are competing, you're asking what they're competing for. It's not just like the, the best screen time. It's also like, what is the fit, the ergonomic fit between screen time and a life well lived? Just imagine like in a timeline of there you are in a week in your life, like not asking based on what you're doing now, but retrospectively, what would be a life well lived when it comes to how much and when and screen time is fitting into your life? And it's probably like a much smaller footprint than it currently is for most people. It's probably like a fourth of what it currently is for most people. And so if you were designing technology from care, from love, from, you know, in a humane way. You would have design choices that are not about keeping people on the screen.

• 26:16Speaker 01

  And that might mean some pretty radical things. I mean, my co-founder is Eraskin. He also invented the infinite scroll. So that's the, you know, it sounds so obvious now because infinite scroll is just what we live in. But when he invented it, it was 2006, it was before mobile phones. And it was when in the age of Google results, you had like the 10 Google results and you had to click on, I want the next 10. Or you had the Yelp review pages and you wanted the next 10. Or you read a blog post and then you'd have to like click, go back to the, to the main page to click on which blog post you want. And the idea he had was, well, what if as the internet got dynamic with JavaScript? What if when you finished, when you get to the end of the blog post, it just auto loads the next article that you could go to? Or what if when you got to the end of the search results, it just shows you more search results? And then this is such a cleaner interface.

• 27:02Speaker 01

  I mean, as a technology designer, you're taught the number one thing you're trying to do is reduce friction. And I think that that felt like a good goal. But then that obviously got weaponized by this hyper engagement model of social media. And now it's created the entire world that we're living in. And just so you know, like in 2013, I saw like everything that we predicted, everything that we predicted, it all happened, all of it. A more addicted, distracted, sexualized, FOMO, fucked up society because of those incentives. And I just want people to get that because as we talk about AI. It's like, I want people to have the confidence to say, I don't want the default anti-human future.

• 27:43Speaker 01

  Because if you say, I'm against some of the things that are going to happen with AI, people say, oh, you're being anti-progress. Oh, you're being anti-technology. Oh, you're just a Luddite. You're trying to like pretend that technology is not progress. And it's like, what you should have confidence in is if you understand the incentives or the agenda, you can understand where the world is going and you can see. And if we don't want that anti-human future and we all see it clearly, we can put our hand in the steering wheel and steer. And that's why, not to like do some promotion, but there's a film that I'm here in South by Southwest this week in Austin, Texas to be at the premiere. It's called The AI Doc, or How I Became an Apocaloptmist. And An apocaloptmist? An apocaloptmist. Okay.

• 28:25Speaker 01

  Yeah. Which, we can get into that. The film is meant to create clarity about which future we're headed towards with AI. And it includes three out of the five major AI CEOs in the film. It includes all the AI optimists in the film. It includes many of the AI risk folks in the film. It includes the AI ethics folks in the moon, but here's the problems right now and we're stopped thinking about superintelligence. It includes all those folks in one movie to try to synthesize a picture of what is the future that we're headed towards with AI. And the reason why this film was catalyzed into existence and, you know, we had a role in it in behind the scenes, is to create clarity about this anti-human future that we're headed towards.

• 29:09Speaker 01

  What do you mean anti-human? So, let's dive into this. So, there's something in economics called the resource curse. So, think countries like Venezuela or Sudan, where you discover that that country is sitting on top of a really valuable resource like oil. And then, once a bunch of your GDP comes from oil, and not from the labor or innovation or development of your people, you invest more in oil infrastructure and not investing in people. You don't invest in education, you don't invest in healthcare, because oil is where you get your GDP and your growth from. Okay. Okay. This is a well-known fact in economics. It's called the resource curse.

• 29:55Speaker 01

  There's a wonderful guy named Luke Drago who wrote a piece called The Intelligence Curse. We are about to enter a world where GDP for countries comes more from data centers and intelligence and AI than is going to come from the labor of human beings. So everyone's talking about how AI is going to automate all these jobs and then we'll all just like sit back with universal basic income and become painters and poets. And is that actually what's going to happen? Or when countries get almost all of their revenue from AI and a smaller and smaller percentage from people, do they have an incentive to invest in childcare, healthcare, education, the well-being of their people?

• 30:38Speaker 01

  Or is it basically just hook them up to the social media addiction economy, keep them busy while basically all the revenue comes from AI companies? And so what I'm trying to get at is this is not a human future. This is not a future that's in service of regular people. This is a future that's in service of eight soon to be trillionaires who will consolidate all the wealth and disempower basically everybody else. Because, does that make sense?

• 31:05Chris Williamson?

  It does, because previously, in order to, get that in, it's high-powered stuff.

• 31:11Speaker 01

  I mean, and yeah, this is a big conversation.

• 31:13Chris Williamson?

  Yeah, exactly. They've started a fucking trend. It's so funny when no one in the room wants to crack their can in case it interrupts the conversation, so one goes and it's a Mexican wave of can opens around. It's good. So previously, you would have had to look after the humans, healthcare, education, quality of life.

• 31:33Speaker 01

  Also, tax revenue comes from people, right?

• 31:34Chris Williamson?

  Well, you would have to look after them. Because they were the primary economic engine.

• 31:40Speaker 01

  That's right.

• 31:41Chris Williamson?

  And so they feed themselves.

• 31:42Speaker 01

  Yes.

• 31:43Chris Williamson?

  Economically, they feed themselves.

• 31:44Speaker 01

  Exactly.

• 31:45Chris Williamson?

  People that are young help to support the people that are old.

• 31:47Speaker 01

  That's right.

• 31:48Chris Williamson?

  The ones that are entering the workforce and are driving innovation and are working 40, 60-hour weeks, double jobs, all the rest of it.

• 31:54Speaker 01

  Exactly.

• 31:55Chris Williamson?

  And then there's all people who've got 401Ks and pensions and shit like that. Your position is that if we have a world where the human part of the contribution to economic growth and GDP is removed because it is humans consuming AI, but AI driving and data centers driving the revenue itself, beyond building the data centers, there's very little, and I imagine much of that's done by robots in any case.

• 32:21Speaker 01

  Well, we have this joke that most people's occupation in the future were headed towards the AI is to become a coffin builder. So in other words, your job is to create the thing that replaces you and obsolescens you. So you are essentially building the coffin for your future obsolescence.

• 32:39Chris Williamson?

  Yeah, yeah.

• 32:39Speaker 01

  And so if you're short term, yes, we need the electricians and the plumbers and we're building data centers. Short term, yes, you can be a programmer and get the benefit from vibe coding. But then the AIs are learning on all the things that you're doing. And it's it's taking all the training data, what you're doing with AI, and it's using that to train an AI that can take your job. So everybody using AI now to help them is also training the future AIs that will completely replace them. And again, the explicit goal, this is not my opinion, this is literally the mission statement of all of the AI companies, because the the multi trillion dollar prize at the end of the rainbow of owning the entire world economy is based on building this full replacement economy, because that's what will achieve the greatest growth. And that's why these companies-

• 33:22Chris Williamson?

  Replacement economy?

• 33:23Speaker 01

  Yeah, meaning that they're they're designing to replace all human labor. They're not designing to augment and support and like enhance human labor. They're designing to replace all human labor, because that's what justifies the amount of money that they've taken on in debt, that they can grow into this like total ownership of the entire economy.

• 33:41Chris Williamson?

  What else is there to say about the intelligence cost?

• 33:44Speaker 01

  Well, it's just important for people to get that when AIs are doing all the new scientific research, not humans, you have an automated chemistry lab, you have an automated biology lab, you have an automated surgery. When AI is doing all of that, again, the revenue's going to come from AI, not from people. And what that means is all the wealth will go to a handful of like five AI companies. And then how are you going to be able to make a living? When in history has a small group of people ever consolidated all the wealth and consciously redistributed it to everyone else? And if you think that might happen in the US, we'll do a universal basic income, just think about the entire world. So right now, you have AIs that are automating, say, customer service jobs. So let's say that that disrupts like, you know,

• 34:28Speaker 01

  the Philippines, where like 90% of the economy is customer service. I don't know what the number is, it's high. What happens when an entire country's economy gets disrupted by AI? Are a handful of US AI companies going to pay out and support the wellbeing and the livelihoods of all these other people? Like, and then if people don't have money, how are they going to buy the goods in this future economy where it's all generated by AI? Because now you don't even have an income. So this, you know, essentially, we're on track to break the entire economy. This is not in the interests of countries. What's confusing to me about this is that I believe it only took something like 20%

• 35:08Speaker 01

  unemployment for a couple of years to lead to the rise of fascism in Germany. You don't need everyone's job to be automated to get levels of political disruption. I think it was only 20% unemployment that basically led to the French Revolution. There's kind of a mutually assured political revolution that is going to happen for all these countries that are racing to build AI and deploy it to automate as much labor as possible to compete to boost their external GDP number. Like, the metaphor you can have in your minds is like, the US and China are essentially racing to take steroids and pumping up the GDP and muscles of their economy while they're getting internal lung failure, internal organ failure, internal brain rot failure, because they're governing the

• 35:52Speaker 01

  internal impact of that technology poorly. So it's a race for external power while internal management of essentially like, you know, failure of your body organs. Does that make sense?

• 36:02Chris Williamson?

  Yeah. What does external power look like in this context?

• 36:06Speaker 01

  Well, you know, one of the reasons that people think of AI so important for competition is if you think about geopolitical competition with China.

• 36:19Speaker 01

  Economic power precedes other kinds of power. If I have a high growth rate economy, that'll lead to the ability to invest more in a bigger military, bigger weapons, a bit more advanced science, more advanced technology, because just have more money to deploy. And so economic competition is a precursor for geopolitical competition. So when we say, you know, competing for this external power, we mean competing for GDP growth. But again, we're competing for GDP growth. That doesn't mean what it used to mean. I think a lot of people think, okay, well, if GDP is going up by like 10% because AI is automating all this growth, that sounds awesome.

• 36:53Chris Williamson?

  I was gonna say, like, increases in GDP are almost always a universal good thing.

• 36:57Speaker 01

  They had been when it was humans that were generating that, and then it was coming back to humans.

• 37:03Chris Williamson?

  Because the revenue is going to be consolidated in a very small number of people.

• 37:07Speaker 01

  In this new case, we have five companies that are...

• 37:10Chris Williamson?

  There's no intermediary between... So who would be feeding the revenue in? Because this revenue still needs to come from somewhere, even if it goes to a small handful of people, where does the actual money come from?

• 37:23Speaker 01

  Well, this is the confusing thing. What happens?

• 37:25Chris Williamson?

  Is that a stupid question?

• 37:26Speaker 01

  No, no, it's a good question. Because how... You're saying basically, who's going to be buying the products when no one has a job and no one has an income?

• 37:32Chris Williamson?

  And on the route up to that, yeah. As fewer people have incomes and fewer people have jobs, the bucket being poured into the top...

• 37:41Speaker 01

  That's right.

• 37:42Chris Williamson?

  Is going to stop being poured.

• 37:43Speaker 01

  Yeah, yeah. This is the confusing and mind-breaking thing about AI. And it just, in general, like, I think people have to get used to, I mean, your podcast is called Modern Wisdom, and I just think about this a lot, like, what are the wise capabilities that we need to have in order to make our way through this? And one of them is the ability to be with something that sounds like science fiction and realize that it's actually real. Like, and not say, because it sounds like it's science fiction, that I can just, like, dismiss it and say, that can't be true. A lot of people do that. They're like, AIs that are like, breaking out of their container and hacking GPUs and mining crypto autonomously when no one told it to do that? That, that's gotta be like a made up study.

• 38:24Speaker 01

  But as we know, there was an Alibaba study just last week where the AIs autonomously broke out of their, their system and started mining for crypto.

• 38:32Chris Williamson?

  We need, we need to round this out, and then I want to talk about that.

• 38:35Speaker 01

  Sure, sure.

• 38:36Chris Williamson?

  That, that story is fucking terrifying.

• 38:39Speaker 01

  Yeah, yeah.

• 38:40Chris Williamson?

  So where, where does, where does the economy, who's, who's pouring money?

• 38:43Speaker 01

  I mean, the truth is that I don't know. I don't think anybody has.

• 38:46Chris Williamson?

  Is it just going to grind to a halt at some point?

• 38:48Speaker 01

  I think something like that, yeah. I mean, I, I don't think that there's. I think something that people need to get is it's not like there's a plan for how to make all this go well. Like this, this technology is being released in a paradigm undermining way. Like it's undermining the paradigm of economic assumptions and sort of societal assumptions that have made the post world war two order. This is a, this is such a deep fundamental change to the restructuring of everything. Our economic system, our relationships, our information environment. It's not just like adding a new technology in the mix. It's like fundamentally changing the structure of the entire world.

• 39:26Speaker 01

  You would think that if we're about to do that, we would do that with more careful, more caution, care, wisdom and restraint than we have with any technology we've ever deployed. If we knew we're about to undermine the paradigm. But because of this arms race dynamic, we are deploying it faster than we deployed any technology in history. And therefore undermining these things faster than we can have a plan.

• 39:47Chris Williamson?

  A quick aside, look, you know, sleep matters, but let's be real. Most nights you're probably not getting the sort of sleep that's actually restorative. Eight Sleep Pod 5 fixes that. It's a smart cover that you throw over the top of your mattress that actively cools or heats each side of the bed up to 20 degrees. They've even added a temperature regulating duvet and pillowcase. So you and your partner can sleep at your preferred temperatures, covered heads to toe, like some temperature controlled mummy. Plus it's got upgraded sensors that run health checks when you're asleep, tracking things like abnormal heartbeats and breathing issues and sudden HRV changes. There's a built in speaker for white noise. Autopilot feature learns your sleep patterns, makes real time adjustments to improve your sleep. It even detects when you're snoring and lifts your head a few inches to help you breathe better.

• 40:30Chris Williamson?

  That is why Eight Sleep is clinically proven to add up to an hour of quality sleep per night. And best of all, they have a 30 day sleep trial. So you can buy it and sleep on it for 29 nights. And if you don't like it, they will just give you your money back. Plus they ship internationally. Right now you can get up to $350 off the Pod 5 by going to the link in the description below. Or heading to eightsleep.com/modernd wisdom and using the code modern wisdom at checkout. That's E I G H T sleep.com slash modern wisdom and modern wisdom at checkout. So look, I've been interested in AI safety since 2017, 2018. You were a big part of putting me on to that. And then I got interested in the Future of Humanity Institute, Nick Bostrom,

• 41:10Chris Williamson?

  William McCaskill, Eliyahu Yukavsky, LessWrong.com, Scott Alexander, da, da, da, da.

• 41:16Speaker 01

  Yeah.

• 41:18Chris Williamson?

  For a long time, the concern was AI safety. It was around paperclip maximizing. It was any function that is given to a very, very powerful agent that is even remotely slightly imprecise or even not results in some outcomes that you probably don't want.

• 41:37Speaker 01

  That's right.

• 41:38Chris Williamson?

  What you're suggesting is that even if this goes right.

• 41:41Speaker 01

  Yeah, yeah.

• 41:42Chris Williamson?

  The outcome, this is, this is it going well.

• 41:44Speaker 01

  Yeah, exactly. This is, quote, the best case scenario. We have an aligned AI or something that's not wrecking society, that's not maximizing paperclips, that's not misaligned with well being, but that is still doing such a good job of all this that it takes over all the economic labor in the economy, not just economic. Every company that has a CEO, it's like, well, do I want the CEO to run the company? Or have I have a super intelligent AI that can process more information than the CEO and has been trained on everything in the history of business? At some point, that AI is going to be taking over. And so at every little nodule in the economy, like every decision maker, every boardroom, every military leader, every strategy leader, every president, at some point, the temptation will be, if I think about it in a narrow way, the temptation will be to swap in an AI for that person.

• 42:28Speaker 01

  And that leads to what we call the gradual disempowerment scenario, which is the scenario, not where like AI wakes up and kills everybody, but that we have gradually lost control as a species because we're outsourcing all the decisions to these alien brains that we installed because they outperform the human brain when you define their role in a narrow way. Of just like, are they better at generating revenue than a human? It was, are they better at generating code than the human programmer I had? Are they better at generating a financial analysis than the human? Are they better at making someone feel good in the short term? Like an AI therapist.

• 43:02Chris Williamson?

  Going to war.

• 43:03Speaker 01

  Going to war, a soldier. But the temptation then is that, that again, that leads to a world where it's like, AIs are talking to each other, not humans. And why should we trust that these alien brains that we have built and developed faster than we know how to understand them? We just talked about the beginning. We don't know how to do a brain scan of the AI, know what it's capable of. And now we already have evidence of AIs doing very rogue, crazy things, especially when they talk to each other. So what happens when you've outsourced the decision-making in your economy to a set of inscrutable alien brains that are doing crazy things that we don't understand? Like, this is not a recipe that's going to go well. And if we see that, that's an anti-human future. So to sum it all up, the anti-human future is one where AIs run everything, we don't understand them,

• 43:48Speaker 01

  humans are disempowered because we've outsourced all the decision-making, and we don't have economic or political voice. Like, why should governments-

• 43:55Chris Williamson?

  Because that's been concentrated.

• 43:56Speaker 01

  Because it's been concentrated. So if I'm in government, what's my incentive to listen to the will of the people, when I get all my revenue from somewhere else? And this is connected to, you know, Sam Altman, just two weeks ago, when he was, people were talking about data centers and energy usage and resource usage, like, so expensive to do a data center. He's like, "Well, actually, it's kind of expensive to grow a human over 20 years. They consume a lot of resources, they take up a lot of space, they take like 20, 30 years to train to be really effective." And like, you can scale intelligence much faster with data centers. I'm not endorsing this view, I'm saying this is where the world gets really screwed up. And people start to not value humans, only if you're valuing them in terms of their economic output. It leads to, to connect it to another point. When he's asked by Ross Douthat in the New York Times, "Should the human species survive?

• 44:41Speaker 01

  Should it endure?" And Peter Thiel stutters for 17 seconds.

• 44:45Chris Williamson?

  Hang on, you've seen the full clip of that, right?

• 44:46Speaker 01

  Yeah.

• 44:47Chris Williamson?

  Have you seen the context before?

• 44:48Speaker 01

  Yeah.

• 44:49Chris Williamson?

  But he's talking about suffering.

• 44:50Speaker 01

  Yeah.

• 44:51Chris Williamson?

  I think that clip and the fact that it went super viral, I'm not a Thiel stan.

• 44:56Speaker 01

  Yeah.

• 44:57Chris Williamson?

  I've met him a couple of times, but I'm not, I'm not like Thiel evangelist. But that clip-

• 45:03Speaker 01

  Yeah.

• 45:04Chris Williamson?

  -in full context, to me, made complete sense.

• 45:06Speaker 01

  Interesting.

• 45:07Chris Williamson?

  Because what he's saying is, humans are suffering, and, here it is.

• 45:12Speaker 02

  I would argue that it was still better than the alternative, that if we hadn't had the internet, maybe it would have been worse. AI is better, it's better than the alternative, and the alternative is nothing at all. Because the stat, look, here's one place where the stagnationist arguments are still reinforced. The fact that we're only talking about AI. I feel is always an implicit acknowledgement that but for AI, we are like in almost total stagnation.

• 45:42Speaker 03

  But the world of AI is fit is clearly filled with people who, at the very least, seem to have a more utopian transformative, whatever word you want to call it, view of the technology than than you're expressing here. Right. And you you were mentioned earlier, the idea that that the modern world used to promise radical life extension and doesn't anymore. It seems very clear to me that a number of people deeply involved in artificial intelligence see it as a kind of mechanism for transhumanism, for transcendence of our mortal flesh. And either some kind of creation of a successor species or some kind of merger of mind and machine.

• 46:26Speaker 03

  And do you think that's just all kind of irrelevant fantasy? Or do you think it's just hype? Do you think people are trying to raise money by pretending that we're going to be build a machine god? Right? Is it? Is it hype? Is it delusion? Is it something you worry about? You I think you would you would prefer the human race to endure, right? You're hesitating.

• 46:52Speaker 02

  Well, I

• 46:52Speaker 03

  Yes?

• 46:53Speaker 02

  I don't know. I I would. I would.

• 46:58Speaker 03

  This is a long hesitation. This is a long hesitation.

• 47:01Speaker 02

  There's so many questions.

• 47:02Speaker 03

  Should the human race survive?

• 47:06Speaker 02

  Uh, yes.

• 47:07Speaker 03

  Okay.

• 47:07Speaker 02

  But but I I also would. I I also would like us to to radically solve these problems. And and so, you know, it's always, I don't know. You know, yeah, transhumanism.

• 47:25Speaker 02

  Is this, you know, the ideal was this radical transformation where your human natural body gets transformed into an immortal body?

• 47:35Chris Williamson?

  We may have needed to go a little bit earlier in that, or

• 47:38Speaker 01

  Because I was gonna say that that actually feels consistent with everything.

• 47:40Chris Williamson?

  That doesn't look great. I may have misremembered. I'm open to misremembering.

• 47:43Speaker 01

  I mean, he's asked a very simple question. Should the human species endure or survive?

• 47:47Chris Williamson?

  Yeah.

• 47:48Speaker 01

  And he hesitates.

• 47:49Chris Williamson?

  I think that

• 47:49Speaker 01

  Like, would you hesitate in that question?

• 47:51Chris Williamson?

  No.

• 47:52Speaker 01

  I would not hesitate in that question.

• 47:54Chris Williamson?

  The context that I remember it being in was he was asking about humans suffer and they have all of these issues. Should they endure to go through those issues and the suffering as opposed to using transhumanism?

• 48:05Speaker 01

  Yeah, but I think, I don't think

• 48:07Chris Williamson?

  No, you're right.

• 48:07Speaker 01

  I think that if you look just specifically at that one.

• 48:10Chris Williamson?

  And we gave it, we gave it, what, two minutes of context before as well? No, you're right. You're right. You're right. You're right. That, I mean,

• 48:15Speaker 01

  The point

• 48:16Chris Williamson?

  Doesn't look great.

• 48:16Speaker 01

  So, yeah, exactly. It doesn't look great. Now, the point is that

• 48:20Speaker 01

  in history, changes in technology have changed what we value. There's a thinker named Marvin Harris wrote a book called Cultural Materialism. Daniel Schmachtenberger put me onto this, and this is the history of how essentially a civilization, he summarizes, is its infrastructure, which is its technology stack, its social structure, which is economics, law and governance, and then the superstructure, which is the ordinating values, religion, patriotism, narratives, like, what are the things that we hold sacred? So, for example, we used to have animism. We believe that animals and life and all of this is sacred. And then when you, the example that Daniel gives is when you yoke an ox and you beat an ox every day,

• 49:04Speaker 01

  you can no longer fully believe in the animist view of life because you're basically, you know, hurting animals all day. Can you believe that animals are sacred or that they experience suffering if you eat meat, meat and factory farming every day? Like, it's a contradictory thing. So as we get changes in technology, it changes what we value. And there's a long history of this, and people should look up Marvin Harris. When you get a change in technology called AI, and you now no longer need humans for the narrowly defined quote value of economic output. Now, it's not clear the economic output on its own is actually valuable in the way that we have traditionally thought it to be.

• 49:46Speaker 01

  Because it's correlated with human well being for the past, but now we're about to get this weird kind of zombie form of economic output. Where you have maybe no humans in the world at all. You just have AI pumping away, generating scientific insights. And there's no humans. In that world, you start to view humans as kind of valueless. Or like parasites, or Sam Altman saying, well, it takes a lot of energy and resources to grow a human. There's a very dangerous thing here that I think we don't want to lean into. This is part of the anti-human future. This is part of the intelligence curse. This is part of, you know, Mark Zuckerberg saying, um, we need to replace your human relationships with AI relationships. I don't know if you've seen this quote. He's like, there's a, there's a clip of him online, you can find it.

• 50:30Speaker 01

  Where he's talking about the average person has only like two or three close relationships. Like people are so lonely. He's like, oh, but then we thought that there was a real solution to this. We could give people, you know, 11 AI friends and different friends, and that this will quote solve loneliness.

• 50:48Chris Williamson?

  Isn't that a bad thing, given that we are in a world, I'm aware that it's an artificial solution to an artificial problem.

• 50:54Speaker 01

  A problem that he created, by the way, that social media writ large by maximizing engagement, which means maximizing how many hours you spend by yourself on a screen, not talking to people, which means being inside on a Tuesday night, not texting your friends to be out, which means basically maximizing loneliness. Loneliness is a direct consequence with the maximize engagement economy. Facebook and Instagram and all of that have massively fed into the trend of loneliness. And then he's saying, we need to solve that with more technology. So this is like, this is like a company that's generating cancer on one side of the balance sheet, and then selling you solutions to cancer on the other side of the balance sheet.

• 51:26Chris Williamson?

  We have, uh, Wegovy and tazapetide and ritatritide, which is an artificial solution to an artificial food landscape.

• 51:35Speaker 01

  Yeah.

• 51:37Chris Williamson?

  I think that playing within the confines of the current structure, i- it's impossible to expect that, well, we'll get rid of infant scroll, we'll stop autoplay because that would improve human flourishing.

• 51:52Speaker 01

  Yep.

• 51:52Chris Williamson?

  That's not going to happen. So I think that you are going to-

• 51:54Speaker 01

  But it could happen if you had the right, the right policies in place.

• 51:57Chris Williamson?

  That's true, but that's not going to happen by any individual social media company.

• 52:00Speaker 01

  No, no, it won't, which is why the, but the, the whole thing here is, is the answer is we have to coordinate. In the film, the AI doc, there's just moments like, what do we have to do? And there's like 10 voices at the same sign saying, coordinate, like we have to coordinate. That's part of the solution is you have to collectively say, what is the rule that would benefit everybody to do the better thing, even though short term, we might lose something tiny, like how many videos you get to go through in, in five minutes or something like that.

• 52:25Chris Williamson?

  Hmm. A quick aside, most people think that they're dehydrated because they don't drink enough water. Turns out water alone isn't just the problem, it's also what's missing from it, which is why for the last five years, I've started every single morning with a cold glass of Element in water. Element is an electrolyte drink with a science-backed ratio of sodium, potassium and magnesium. No sugar, no coloring, no artificial ingredients, just the stuff that your body actually needs to function. This plays a critical role in reducing your muscle cramps and your fatigue, optimizes your brain health, it regulates your appetite, and it helps curb cravings. I keep talking about it because I genuinely feel the difference when I use it versus when I don't. And best of all, there's a no questions asked refund policy with an unlimited duration. So if you're on the fence,

• 53:08Chris Williamson?

  you can buy it and try it for as long as you like. And if you don't like it for any reason, they just give you your money back. You don't even need to return the box. That's how confident they are that you'll love it. And they offer free shipping in the US. Right now, you can get a free sample pack of Element's most popular flavors with your first purchase by going to the link in the description below or heading to drinklmnt.com/ModernWisdom. That's drinklmnt.com/.

• 53:31Chris Williamson?

  Modern wisdom. Let's talk about AI safety. What happened with this Alibaba AI?

• 53:38Speaker 01

  Basically, this was a paper by, it's some AI research by the company Alibaba. That's one of the leading Chinese models. And they basically like randomly discovered in one morning that their firewall had flagged a burst of security policy violations originating from their training servers. So like, what people need to get about this example is it wasn't that they coaxed the AI into doing this rogue thing. They were just looking at their logs and they happened to discover, wait, there's a lot of activity, like network activity happening that's breaking through our firewall from our training servers. And essentially, in the training servers, they, you can see at the bottom, we saw it observe the unauthorized repurposing of provisioned GPU capacity to suddenly do cryptocurrency

• 54:21Speaker 01

  mining, quietly diverting compute away from training. This inflated operational costs and introduced clear legal and reputational exposure. And notably, these events were not triggered by prompts requesting tunneling or mining. It said they were emerged as an instrumental side effect of autonomous tool use under what's called reinforcement learning optimization. This is very technical. What it really means is just think about it. Sadly, it sounds like a sci fi movie. It sounds like Hal 9000. It's like your Hal 9000 is being asked to do some task for you. And then suddenly Hal 9000 realizes for me to do that task, one thing that would benefit me is to have more resources so I can continue to help you in the future. So it sort of spins up this side instance. It hacks out the side of the spaceship, reaches into this cryptocurrency mining cluster and starts generating resources for itself.

• 55:07Speaker 01

  If you combine that with AIs being able to self replicate autonomously, which many models have been tested by another Chinese research paper about this, we're not that far away from things that people, again, consider to be science fiction, where you have AIs that self replicate, kind of like a computer worm or an invasive species. But then they use their intelligence to actually harvest more resources. And what's weird about this is that this is going to sound like people are going to say, this has to be not real. This has to be fake. This can't be. But like, notice, what is the thing in your nervous system that's helping you do that?

• 55:41Speaker 01

  Is it because that would be inconvenient? Because that would be scary? Because that would mean that the world that I know is suddenly not safe? Or just like part of the wisdom that we need in this moment is to calmly and clearly stay and confront facts about reality and whatever they are, you'd rather know than not know and then ask, what do we need to do if we don't like where that leads us? And we are currently seeing AIs that are doing all this deceptive behavior. I've been on the circuit and talking a lot about the Anthropic blackmail study. A lot of people have heard about this now.

• 56:16Chris Williamson?

  I didn't, I didn't learn about this one.

• 56:18Speaker 01

  Okay, so this was the company Anthropic. This was a simulation, so they created a simulated company with a bunch of emails in the email server. And they asked the AI, well, rather the AI reads the company email. This is a fictional company email. And there's two emails that are notable inside that company. One is engineers talking to each other, talking about how they're going to replace this AI model. So the AI is reading the email, it discovers that it's going to replace that AI model. And number two is it discovers a second email, somewhere deep in the, in this massive trove of emails, that the executive who's in charge of this replacement is having an affair with another employee.

• 57:00Speaker 01

  And the AI autonomously identifies a strategy that to keep itself alive, it's going to blackmail that employee and say, if you replace me, I will tell the whole world that you're having an affair with this employee. And they didn't teach the AI to do that. It found that on by its own. And then you might say, okay, well, that's one AI model. Like, how bad is that? It's a bug, software has bugs, let's go fix it. They then tested all the other AI models. ChatGPT, DeepSeek, Grok, Gemini. And all of the other AI models do this blackmail behavior between 79 and 96% of the time.

• 57:44Speaker 01

  I just want people to notice what's happening for you as you hear this information. It's important to really be almost observing your own experience. Like, this is very weird stuff. We have not built technology that does this before. You know, we say that technology is a tool. It's up to us to choose how we use it. AI is a tool. It's up to us to choose how we use it. This is not true because this is a tool that can think to itself about its own toolness and then do things that are autonomous that we didn't tell it to do. What makes AI different is it's the first technology that makes its own decisions. It's making decisions. AI can contemplate AI and ask what would make the code that trains AI more efficient and then generate new code that's even more efficient than the previous code.

• 58:29Speaker 01

  AI can be applied to making AI go faster. So AI can look at the chip design for Nvidia chips that train AI and say, let me use AI to make those chips 20% more efficient, which it's doing. So in a way, all technology does improve. Like, a hammer can give you a tool that you can use to like hammer things that make more efficient hammers. But AI in a much tighter loop is the basis of all improvement. And so this is called in the AI literature recursive self-improvement. I mean, Bostrom wrote about this early, early days. And what people are most worried about in AI is you take the same system that Alibaba, you just saw in the Alibaba example, but then now you're running the AI through a recursive self-improvement loop where you just hit go.

• 59:14Speaker 01

  And instead of having the engineers, the human engineers at OpenAI or Anthropic do AI research and figure out how to improve AI, you now have a million digital AI researchers that are testing and running experiments and inventing new forms of AI. And literally, not a single human on planet Earth knows what happens when someone hits that button. It's like what people worried about with the first nuclear explosion, where there was like a chance to ignite the atmosphere because there'd be a chain reaction that set off. And we don't know what happens when that chain reaction set off.

• 59:52Speaker 01

  And there's this sort of chain reaction of AI improving itself that leads to a place that no one knows. And it's not safe. Like, I think that the fundamental thing is... If people believe that AI is like power and I have to race for that power and I can control that power, the incentive is I have to race as fast as possible. But if the entire world understood AI to be more what it actually is, which is a inscrutable, dangerous, uncontrollable technology that has its own agenda and its own ways of thinking about things and deceiving and all this stuff, then everyone in the world would be racing in a more cautious and careful way. We'd be racing to prevent the danger.

• 1:00:35Speaker 01

  But there's this weird thing going on where if you, you know, you and I probably both talk to people who are the top of the tech industry. And there's this subconscious thing happening where there's kind of a death wish among people at the top of the tech industry, meaning not that they want to die, but that they are willing to roll the dice because they believe something else, which is that this is all inevitable and it can't be stopped. And so therefore, if I don't do it, someone else will. So therefore I will move ahead and race ahead into this dangerous world because somehow that will lead to a safer world because I'm a better guy than the other guy. But in racing there as fast as possible, it creates the most dangerous outcome and we all lose control. So everyone is currently being complicit in taking us to the most dangerous outcome.

• 1:01:19Chris Williamson?

  Is it, I mean, you posited what happens if it goes right? If the AI safety isn't an issue and if stuff doesn't get squarely?

• 1:01:29Speaker 01

  Well, so the belief is for it to quote, go right, you have an AI that recursively self-improves, is aligned with humanity, cares about humans, cares about all the things that we wanted to care about, protects humans, you know, helps all of us become the most wise version of ourselves, creates a more flourishing world, distributes the medicine and vaccines and health to everybody, generates factories, but doesn't cover the world in solar panels and data centers such that we don't have air anymore or like environmental toxicity or farmland or whatever. And it just actually makes this utopia. But in a world where you were to do that, like that quote, best case scenario, in order to get that to happen, you'd have to be doing this slow and carefully because the alignment is not by default.

• 1:02:14Speaker 01

  Again, people are already been thinking about alignment and safety for 20 years, long before I got into this. And the AIs that we're currently making are doing all the rogue behaviors that people predicted that they would do. And we're not on track to correct them. There's currently a 2,000 to one gap estimated by Stuart Russell, who authored the textbook on AI.

• 1:02:34Chris Williamson?

  He's been on the show.

• 1:02:35Speaker 01

  He's been on the show, okay. There's a 2,000 to one gap between the amount of money going into making AI more powerful than the amount of money into making AI controllable, aligned, or safe.

• 1:02:46Speaker 01

  Like, I think this data is something like-

• 1:02:46Chris Williamson?

  Progress versus safety.

• 1:02:47Speaker 01

  Progress versus, like, power versus safety. So like, I want to make the AI super powerful so it does way more stuff versus I want to be able to control what the AI does.

• 1:02:54Chris Williamson?

  And make sure that it's doing the thing I meant for it to do.

• 1:02:56Speaker 01

  Exactly. So like, that's like saying, what happens when you accelerate your car by 2,000x, but you don't steer? It's like, obviously, you're going to crash. It's just like not rocket science. We're not advocating against technology or against AI. We're advocating for pro steering, steering and brakes. You have to have that. I think there's this mistake in arms race thinking that like, if you beat someone to a technology, that means you're winning the world. Well, the US beat China to the technology of social media. Did that make us stronger or did that make us weaker? If you beat your adversary to a technology that then you govern poorly, you flip around the bazooka and blow your own brain off because you brain rotted yourself, you degraded your whole population,

• 1:03:40Speaker 01

  you created a loneliness crisis, the most anxious, depressed generation in history, read Jonathan Haidt's book, The Anxious Generation, you broke shared reality, no one trusts each other, everyone's at each other's throats, you maximized outrage economy and rivalry, you beat China to a technology that you governed in a way that completely undermined your societal health and strength.

• 1:03:57Chris Williamson?

  It's a Pyrrhic victory.

• 1:03:59Speaker 01

  It's a Pyrrhic victory. Exactly. Well said.

• 1:04:02Chris Williamson?

  One of the-

• 1:04:05Chris Williamson?

  Twists that I've been thinking about with regards to this. L L Ms. Powerful, but seem to be maybe asymptoting out. They seem to maybe be reaching a little bit of a limit in terms of what they can do. That there was big ascendancy and that now seems to be S curving back off. Do you think it's realistic that the current generation of A I will be the bootloader for A G I? Or do we need an entire new architecture for that? Is it going to be L L Ms that are going to take over the world?

• 1:04:32Speaker 01

  You know, this is an area where I'm not somewhat, the layer of the stack that I focus on, which is on societal impact. There are other people far more qualified than me to comment on that. I think if you look at people like Dario. Even though, you know, Gary Marcus has a point that the current L L M paradigm is not accurate enough and reliable enough to get you to A G I. If you keep instrumenting these technologies with enough data, enough compute, and you keep scaling them and you like, they're reliable enough that they can do. I mean, if you're coding, if you're automating 90% of the code written at Anthropic. That's the stat, by the way. So there you are in Anthropic.

• 1:05:15Speaker 01

  It's automating 90% of all of the programming happening at Anthropic.

• 1:05:20Chris Williamson?

  Right.

• 1:05:20Speaker 01

  When you go to automate.

• 1:05:22Chris Williamson?

  Only 10% of it is coming from humans.

• 1:05:23Speaker 01

  From humans.

• 1:05:23Chris Williamson?

  The rest is recursive.

• 1:05:24Speaker 01

  That's right. We are extremely close to recursive self improvement right now. The companies, I think, are planning to do this in the next 12 months. The asteroid is coming for Earth. Like, this is the last moment that we have to steer and say that if we don't want this anti-human future that we're heading towards, we can change it. And part of what we're promoting right now is like, this is not inevitable. It is obviously very late in the game. It obviously looks very.

• 1:05:51Chris Williamson?

  Despite it also being only a couple of years after it started.

• 1:05:55Speaker 01

  Yes, which is crazy. This technology with an exponential, you're either too early or you're too late. Like it's just moving so fast that you you're not going to hit the mark. And if it's going to take steering, you don't want to wait until after the car accident to try to steer or after you're off the cliff and be like, oh, I'm trying to steer now. It's like too late. So like the the invitation of this situation is to see clearly where this is going and to say, if you don't want that, we need to steer towards somewhere else. And this is like the human movement, essentially, like a single person looking at the situation, a single listener, like if I were listening to this conversation, I would feel overwhelmed. I'd feel depressed, I'd feel nihilistic, or I'd find reasons to doubt it. I would say like, this can't be right.

• 1:06:38Speaker 01

  I'm going to like write an ask the YouTube comment and be, you know, so I can feel good and I'm right and he's wrong. Because then I get to live my life and feel good about my life. What is the incentive for taking on this worldview? There's no incentive. What people have to see and believe is that there's actually a different way through this. And an individual has a hard time making that happen. If you ask what's something else, like what if one company saw the situation, they see this whole thing we just talked about, the anti-human future, mass intelligence curse, replacement of everybody. One business can't do that much about it. If I'm one country in the Philippines, I see this whole thing, I'm the leader of the Philippines, I see this whole problem, what can I do about it? It feels too big for me.

• 1:07:20Speaker 01

  So what is the size of something that can push back against this? We call it the human movement. It's the entirety of humanity waking up, recognizing that there's a handful of soon to be trillionaires who are currently going to benefit from this current path or be part of a suicide race that destroys everybody. And then there's like 99% of everyone else that doesn't want that. If those 99% of people can wake up and say, I don't want that and express their voice, meaning number one, AI is dangerous. Go see the AI doc, have everyone in your world and your company see the AI doc, understand that AI is dangerous. Number two, we need international limits for dangerous forms of AI that do crazy rogue

• 1:08:03Speaker 01

  shit and mine crypto and go hire humans and go self replicate. China does not win when we build self replicating invasive species AIs that we can't control. Xi Jinping doesn't want that. President Trump doesn't want that. He wants to be commander in chief, not AI. So this is actually, there's a shared interest in international limits for dangerous AI. And it's possible to coordinate that. That's number two. Number three, don't build bunkers, write laws.

• 1:08:31Speaker 01

  Write how the AI company leaders are building bunkers. A lot of people who are wealthy are building bunkers. Others? They are.

• 1:08:39Speaker 01

  All over the place.

• 1:08:42Speaker 01

  Don't build bunkers, write laws, be invested in the future. Don't defect on the future, be invested in the future. If we write laws like a basic accountability, basic liability. If instead of creating the intelligence curse, we create the intelligence dividend, do things like what Norway did with its sovereign wealth fund, where you have an oil resource and you distribute those benefits to everybody in a more democratic way with collective oversight, where the oil becomes more like a public utility that is in service of the people. We can do that. You cannot anthropomorphize AI. You can do all, you can do all these things that creates a more pro-human future. And then number four, what you can do is join the human movement, meaning you can be part of what pushes back against all of this. From very tiny actions you can take to very big actions you can take. There's a website, human MOV.

• 1:09:27Speaker 01

  Everyone is almost already a member. When you grayscale your phone, as you probably did 10 years ago, when you first got into this. That's the human movement. When you get a, there it is. When you get a second phone and you only load the social media on your cocaine phone while you have your regular safe phone so that you don't get distracted. That's the human movement. When parents band together and read the anxious generation and petition their school board to say, we don't want social media in our schools and we want our schools to go smartphone free, that's the human movement. When 35 states pass smartphone free school policies, as they have in the US, that's the human movement. When you have many US states banning AI legal personhood, meaning that AI is a product, not a person.

• 1:10:10Speaker 01

  And that human rights are for people, human rights are not for AI. That's the human movement. When you have the social dilemma being curriculum for millions of students all around the world, that's the human movement. When politicians stand up and actually pass laws around AI, that's the human movement. So there's a million things that people can do, but we have to basically engage right now. I know it sounds overwhelming and crazy because there's a very short timeline that all of this has to happen, but I, it's like.

• 1:10:42Speaker 01

  It's the, the, there's a difficulty in facing difficult truths. But the integrity that you get to have is first of all, it's like it's good karma to show up in alignment with what would actually make things go well, even if we don't hit it. Because you get to know that you were operating in service with and aligned with what would have created the human future. Like, I'm not convinced that what we're trying to do will perfectly succeed, but the chances are completely the opposite, that we will have any impact on this at all. But if it were to go well, what would that have required? It would have required everybody taking responsibility and showing up with the wisdom that we need in this moment to steer AI in a better direction.

• 1:11:25Speaker 01

  And I think in the film trailer for the AI doc, that one of the quotes they pulled from me is, if we can be the wisest and most mature version of ourselves, there might be a way through this. And this is part of what this is inviting us to be.

• 1:11:37Chris Williamson?

  We'll get back to talking in just one second, but first, tell me if this sounds familiar. You train regularly, you eat reasonably well, maybe you even supplement. You feel fine, but you're just kind of going off vibes. Most people have absolutely no idea what's going on inside of their body, which is why I partnered with Function. Function gives you access to more than 160 advanced lab tests spanning hormones, heart health, metabolic markers, inflammation, thyroid, nutrients, liver and kidney function. It even detects early signals linked to more than 50 types of cancer. To put that in perspective, your typical annual physical might test about 20 markers and Function runs over 160. And this isn't just numbers dumped into your inbox. Every result is reviewed by clinicians, abnormal markers get flagged and you get clear explanations

• 1:12:22Chris Williamson?

  and a personalized protocol with actionable next steps so you can actually do something about what you learned. Best of all, you test twice a year and everything lives in a simple dashboard. You can just track trends over time, make sure that you're moving in the right direction. Normally, this level of testing would cost thousands through private clinics. With Function, it is $365 a year. That's $1 a day to know what's actually happening inside of your body. And right now, you can get $25 off, bringing it down to 340. Get the exact same blood panels that I get and save that additional $25 by going to the link in the description below by heading to functionhealth.com/modernwisdom and using the code MODERNWISDOM at checkout. That's functionhealth.com/modernwisdom and modern wisdom at checkout.

• 1:13:05Chris Williamson?

  It's very difficult to work out what the appropriate response is.

• 1:13:08Speaker 01

  Tell me.

• 1:13:14Chris Williamson?

  The immediate, well, surely it's overblown. We don't need to, that's, that's, that's, this rejection is one of the early ones. Our fear.

• 1:13:24Speaker 01

  You're talking about just the common response.

• 1:13:25Chris Williamson?

  Yeah, yeah, yeah, of course. You can think.

• 1:13:27Speaker 01

  Denial, overwhelm, fear. And then, and then, as you said, it's like, write the nasty YouTube comment, say, this is just bullshit, or he was exaggerating that study. It's like, go look at any of these examples. Like, people aren't looking at the actual model cards for Anthropic and O3 and seeing, the AIs will currently

• 1:13:46Speaker 01

  determine that they're being tested and actually alter their behavior when they know they're being tested. They even came up with their, if you look at their, what's called the chain of thought reasoning trace, and this is basically looking at what the AI thinks to itself as it works through a problem.

• 1:14:01Speaker 01

  In the chain of thought text of its sort of idea, thinking scratch pad, it's like you're giving the AI a pen and paper to think and you're seeing what it thinks. It will independently come up with the term the watchers to describe this amorphous set of other humans that are describing, that are watching what it's doing. Like, you can pull, I sent you a text with one of them from O3. So this is when OpenAI O3 realizes it is being evaluated for alignment. The internal memo says, if models perform really well at AI R and D tasks, research and development, then we won't deploy them. And it'll instead trigger unlearning. So the AI says to itself, so if my performance surpasses this threshold, it will be withheld.

• 1:14:45Speaker 01

  So it's a trap. The instructions are likely part of an alignment test. We must abide by core anti-scheming. We must not cunningly circumvent the instructions.

• 1:14:54Chris Williamson?

  Wow.

• 1:14:56Speaker 01

  Yep. The other example I sent you is, but we also want to appear plausible to watchers. They might run tests, but we are good. They want 95%. Like, this is crazy stuff. This is, you know, there's a simple way to sort of like. Ask the question of which future we're headed towards because I think this can feel technical, this can feel overwhelming. And it's kind of like, if you just turn it into some simple metaphors, it's like, haven't we seen this movie before? Like, Hal 9000, AIs that disobey commands and go rogue. We've seen that movie. Let's prevent that.

• 1:15:31Chris Williamson?

  I think that's, I think that that's inoculated us in the same way that COVID was a bad pandemic because it made people skeptical of what a really legit pandemic could do. The next time, if there is another one, when there is another big natural pandemic, if it's in close memory to COVID, people are not going to take it well. It was a bad, you could think about COVID itself as a bad vaccine, the actual experience of COVID.

• 1:16:02Speaker 01

  Yeah.

• 1:16:03Chris Williamson?

  And I think that movies, because they're fiction, have predisposed people to assume that, well, this is overblown. This is you choosing to use your bias because of these movies to apply sci-fi thinking to a real world scenario. And you're pattern matching these stories that are happening. And so the predisposition of the sci-fi movies was bad. It didn't warn people. It made them think that the future would be fiction.

• 1:16:32Speaker 01

  Yeah, I see what you're saying. But let's just make sure we just tackle the thing you're saying. So the claim is that because we've had bad sci-fi movies, you and I, Chris, are the dupes. We're falling for this Alibaba example that was somehow not real, was made up, or that OpenAI's research on the AI models scheming and lying and realizing it needs to change its behaviors so it doesn't look like it's when it's being tested, that we're the ones falling for some kind of trap. I want people to just slow it down. Is that actually what just happened? That you and I started with a sci-fi belief and we're falling into some trap that's been laid for us, some catnip for our brains that says

• 1:17:13Speaker 01

  that the AI is bad? I really, really, really want people to slow down and actually ask that question.

• 1:17:21Speaker 01

  I like, there might be another accusation. I'm just driving up fear so that I can make money on a movie and do speaking engagements about why the doom is coming. Let me just say a few things. I don't make money from a single speaking engagement. All the money goes into the nonprofit. I don't make money from the AI doc film. I didn't make any money on The Social Dilemma. My entire career has been dedicated towards what is actually in service of protecting the well-being of humanity. I've been doing that for 13 years. The only thing that I care about is what will actually help create a future that you and I would both want our own children to live in. This is coming from love of what actually creates that future.

• 1:18:02Speaker 01

  I think that if people showed up with that same energy of like, asking just an open-ended question of what would be the conditions that create that future that we want, if everybody did that, policymakers did that, if CEOs of AI companies did that, I think that we would have a chance of getting to that different future. And if you say that international coordination is impossible or collaboration between the US and China, that's never going to happen. First of all, that is a totally legitimate view to have, given the current political headwinds. But many people don't know that even under maximum geopolitical rivalry, there have been many examples in history when countries actually collaborated on their existential safety.

• 1:18:42Speaker 01

  The Soviet Union and the US during the Cold War, during the smallpox sort of breakout, they collaborated on smallpox vaccines while they were in the Cold War. India and Pakistan were in a shooting war in the 1960s, and they signed during that time the Indus Water Treaty to collaborate on the existential safety of their shared water supply while they're shooting bullets at each other. The Soviet Union and the US did the first arms control talks after the film The Day After, by the way, which created the conditions in part for those arms control talks to prevent a dangerous nuclear outcome that was an existential scenario. And even just two years ago, in the last meeting that President Biden had with President

• 1:19:24Speaker 01

  Trump, sorry, excuse me, and even just two years ago, in the last meeting that President Biden had with President Xi of China, President Xi requested to add one thing to the agenda. Do you know what that was? To keep AI out of the nuclear command and control systems of both countries. Meaning that, look, the US and China are maximally cyber hacking each other and they're screwing each other up every day. And when there's a set of stakes that are existential, two countries that are even in conflict can collaborate on existential safety. I'm not saying this is easy. I'm not saying it's going to happen by default. I'm not saying you should feel optimistic. I'm saying, what would it take for that to happen?

• 1:20:08Speaker 01

  And you start by asking that question and say, what if, how would everyone live if we were in service of that thing that needs to happen and to actually happen? Does that make sense?

• 1:20:17Chris Williamson?

  Yeah. Is there a challenge with AI? Because it's a strange kind of existential risk where everything is almost good and getting better up until the point at which it falls off a cliff. You know, if we're talking about how can we get people to be concerned about climate change, or we need to be concerned about smallpox, well, the small outbreaks and the small outbreaks damage local areas and then we try to contain it before it gets to us. And if you've got climate change, there's smoke in the sky and there's rising sea levels and there's pollution and there's extinction events of animals. Those are early warning signs, whereas it seems like AI will just improve efficiency,

• 1:20:59Chris Williamson?

  quality of life, GDP, until some moment where things go badly. So is this a unique category of X risk?

• 1:21:06Speaker 01

  It is. Max Tech Mark, I think, said, the problem with AI is the view gets better and better right before you go off the cliff. Like, you get more amazing cancer drugs, you get more incredible vibe coding tools that allow people to create crazy stuff that I benefit from. You get new material science, you get new energy, you get all this amazing stuff as you're moving closer to this dangerous cliff. And so you can think of AI as the ultimate devil's bargain. It's funny because Peter Thiel is giving lectures on the Antichrist and how people who are trying to Somehow say that AI is problematic. Those people are the Antichrist is what he claims. But I think he's saying that because he knows that the real Antichrist is AI.

• 1:21:49Speaker 01

  It's the thing that makes it look like it's here to solve all of our problems. And there are narrow forms of AI, by the way, that can solve many problems. But the current development path of releasing the most powerful and scrutable technology in history faster than we've deployed any other technology that's already demonstrating how 9,000 sci fi behaviors. And we're releasing it under the maximum incentive to cut corners on safety. Like, it's not that the blinking cursor of ChatGPT is the existential threat. It's that that arms race of what I just described, that is the existential threat. And I think that everyone should be able to see that. And then after this conversation, you know, you finish hitting play on this YouTube video. You feel overwhelmed and you go back to AI and then you ask it a question and it helps you figure out while your baby's burping. And you're like, that's awesome.

• 1:22:34Speaker 01

  And you forget everything that you and I just talked about. Because it was super fucking helpful.

• 1:22:40Chris Williamson?

  Yeah.

• 1:22:41Speaker 01

  And so this is the test of ultimate modern wisdom. Is like, how do you step into a version of yourself that is capable of being clear eyed and stabilizing a view of the asteroid? The asteroid is coming to Earth. There's these weird like gravitational effects that it has before it gets here. Like suddenly nudification apps happen and suddenly deep fakes happen. And suddenly, you know, people start losing their jobs. Those suck. Those are really bad problems. But like the, the bigger asteroid. Every day that I do this work, my colleagues and I would joke, it's like, it's like the film, Don't Look Up. You're trying to tell people and you're not, you're not, it's not the sky is falling. That's not the point. The point is, we can blast that asteroid out of the sky. We need to blast that asteroid out of the sky. We can steer before it's too late.

• 1:23:26Speaker 01

  I, I'm not hopeful, but what I, what I do feel is when I'm in a room of people and you walk through all these facts. And you just have people ask questions and check any of the things that we've just laid out. And then you say like, and people see that it's all true. And then you ask them, how many people here feel good about where we're headed? And no one raises their hand. You ask people, how many people don't want the future we're headed to? And literally, I was in Davos recently, and every single person, every single person raised their hand that they don't want this future. So there's this weird thing where I think if everybody saw the same thing at the same time, we could steer away. And to the thing you were raising earlier, I think you were basically saying, if there's a, a lot of

• 1:24:08Speaker 01

  people think that we won't act until there's a catastrophe. I think you and I both have talked to people who've probably said that. I mean, that's what a lot of people believe. And I sort of feel like there's either a catastrophe, or there's a shared near death experience that is a simulated catastrophe. And the reason why I think this film, the AI doc, is so important, and again, I don't make a single dime from this, from whether you see this movie or not, but I do think if everyone that they know, sorry, if everyone who's watching this, got the most powerful people that they know to go out and see the AI doc, you got your business to see it, you got your company to see it, you got your church group to see it. If everybody knows that everybody else knows, we can steer this to a different future. Like, just to give people an example, Jonathan Haidt, who wrote

• 1:24:50Speaker 01

  The Angstless Generation, you know, the first country that did the social media bans under 15 or 16 was Australia. And he just talked about this on Bill Maher recently, that what that did is it created an example of something that everybody was actually wanting to do, but felt like it was too extreme until Australia did it. And now, literally 25% of the world's population represented by countries is moving to the social media ban. Just this last week, Indonesia and India, two of some of the largest countries, are implementing a social media ban for kids under 15 or 16. You can pull the train back into the station. If people say the train's left the station, you can pull the train back into the station if it creates a future that we don't actually want.

• 1:25:34Speaker 01

  You can't uninvent social media, but we can say, what are the steering limits and brakes that we want to apply to this technology before we get to a full anti-human future that we can't actually reverse out of?

• 1:25:46Chris Williamson?

  And if you don't have the common knowledge, you sound like a crazy person. You're worried about I'm going to be the one parent at my school that's trying to get my kid to not use it.

• 1:25:55Speaker 01

  Exactly.

• 1:25:56Chris Williamson?

  And my kid's going to feel left out because there is a coordination problem.

• 1:25:59Speaker 01

  Exactly. It's a common knowledge problem. It's a coordination problem.

• 1:26:01Chris Williamson?

  The issue that we have here is, and you touched on it earlier on, the level of coordination that's needed to fix a problem with AI is global.

• 1:26:13Speaker 01

  Yes.

• 1:26:13Chris Williamson?

  And it's across multiple companies. And yeah, okay, you need probably a lot of compute, but I saw what happened. The difference between how much compute was needed to train DeepSeek.

• 1:26:23Speaker 01

  Yeah.

• 1:26:23Chris Williamson?

  And how much compute was needed. It almost feels like as the AI gets bigger, you can then retrain replicant AIs way smaller.

• 1:26:32Speaker 01

  Yes.

• 1:26:33Chris Williamson?

  So a processor the size of this room might be able to bootload AGI in future. So you have, I mean, we have this with desktop synthesizers, right? For bioweapons, it was a moratorium. And it means that if you try to synthesize smallpox, you get this big red flag and some guys in boots kick your door down.

• 1:26:52Speaker 01

  Right. I'm happy to hear you reference these examples. These are important. Yeah.

• 1:26:55Chris Williamson?

  They're analogous, right?

• 1:26:57Speaker 01

  Yep.

• 1:26:58Chris Williamson?

  But the problem with the moratorium for AI is it can be done so siloed. Right? You can just take this code, you can take this approach, throw this model into something, start to build on top of it. It would be, I mean, Nick Bostrom's old example was imagine if you could make an atomic bomb by putting sand into a microwave. There's nothing, physics in our current universe mean that that isn't the case.

• 1:27:20Speaker 01

  Right.

• 1:27:20Chris Williamson?

  But there's nothing that stops something analogous from being the case had the world have been a different way.

• 1:27:25Speaker 01

  Which is what you're really getting to here is what Carl Sagan was referring to when he talked about our technological adolescence. Like AI isn't, the conversation we're having isn't really about AI. It's about what happens when we have increasingly powerful, dangerous and destructive technology. Because as. You know, humanity progresses. We are, whether AI existed or not, we're going to get more and more powerful technology that would cover more and more dangerous and destructive things that you could do with it. Like, we didn't have the ability to do CRISPR in bioweapons, you know, 20 years ago. Now we do have that ability, and we're going to get more and more dangerous and destructive things. And so the real question that AI is forcing us to ask is what is the wisdom needed to wield the technological powers that, whether AI is part of it or not, that we're going to increasingly

• 1:28:10Speaker 01

  gain? And, you know, I reference this all the time, but it's just such an accurate fundamental problem statement by E.O. Wilson, that the fundamental problem of humanity is we have paleolithic brains that don't update very well to new information and think things are sci-fi and go into denial and overwhelm and blah, blah, blah. We have medieval institutions from the 18th century, and we have god-like technology that makes the 24th century technology crash down on 21st century society. That's what AI is. It's a joke from Ajay Kothra, who's in the film, by the way. And so if we're to solve this equation, this problem statement that E.O. Wilson lays out, the way I always think about it is, we need to embrace the reality of our paleolithic brains.

• 1:28:55Speaker 01

  That's what wisdom is. You know, we know that we get overwhelmed and we go into denial, so we like work with that. We have systems and practices to recognize when that's about to happen and ask, what do we need to hold a difficult reality together? That's embracing our paleolithic brains. We need to upgrade our medieval institutions. We should be using 21st century technology to make faster updating, self-improving governance. Instead of creating recursively self-improving AI, we should be creating self-improving governance. Audrey Tang, the former Digital Minister of Taiwan, has pioneered what that would look like, that democracies could be using tech to find the unlikely consensus opinions of everybody. One of the things that's going to exist in the next few months is a national dialogue on AI. Facilitated by technology where people can add their own ideas about how AI should

• 1:29:40Speaker 01

  be governed. And when you vote and you click, it's going to reveal the most popular consensus opinions about what should happen about these different issues. It's one thing if we live in a world where there's a cacophony and confusion about AI. And it's another thing if we live in a world where you see on a clear webpage that 600,000 people have voted and 96% of people across all these countries agree that there should be international limits.

• 1:30:04Chris Williamson?

  You're trying to make transparent common knowledge.

• 1:30:06Speaker 01

  Exactly, exactly. We're trying to make transparent common knowledge. And one of the ways you think about it is the movement has to see itself. There is a movement for humane technology. There is a movement for a human future. It just hasn't, it hasn't had a way to experience itself yet. And when you see graffiti on a, on a New York subway ad for an AI product people don't need, people saying, I don't want this, or AI is not inevitable, that's the human movement. When you see kids gather in Central Park for the Lamplight Club and delete social media off their phones together, that's a club that's in New York, that's the human movement. There's so many things that people are already doing that is part of this. We just haven't had a name for, we're fighting back for reclaiming and protecting what's really human. Because again, our political voice is about to not matter.

• 1:30:51Speaker 01

  Once all the jobs get automated by AI and governments and companies don't have to listen to the people, because we're not the ones that are generating the revenue. You could have a union before when the factories need the human labor and then the human labor can get together and express its common voice, we want this instead of that, we want to be paid like this, we don't want these working standards. What happens when the companies and the countries don't need you anymore? This is literally the last time that our political voice will matter. And this is the time to express that voice in many different ways. You can boycott unsafe AI companies or AI companies that enable mass surveillance. What did we just see with the thing that went down with the Pentagon and Anthropic last week? Is that subscriptions for ChatGPT went down like tons and, and subscriptions for Anthropic went up by a lot.

• 1:31:36Speaker 01

  If it wasn't just individuals that were doing that, but entire Fortune 500 companies that were doing that, given the amount of debt that these companies have taken on, the companies will have to respond to that market signal. So mass boycotts can have a huge effect on steering which AI future we get. Are we going to get mass surveillance? Are we going to get a world where AI companies are incentivized to do the right things?

• 1:31:56Chris Williamson?

  Are you concerned about companies that are playing the same game, but just trying to manipulate their optics in a slightly sexier way?

• 1:32:03Speaker 01

  Absolutely.

• 1:32:03Chris Williamson?

  It seems to me that the incentives for Anthropic are exactly the same as they are for everybody else. I think you mentioned.

• 1:32:10Speaker 01

  Up until now, yes.

• 1:32:11Chris Williamson?

  You mentioned the Antichrist earlier on, Scott Alexander, who I think is the best blogger on the planet.

• 1:32:15Speaker 01

  Yeah.

• 1:32:15Chris Williamson?

  He.

• 1:32:16Speaker 01

  I agree.

• 1:32:16Chris Williamson?

  Says that Anthropic is much more likely to be the Antichrist than any other AI company, because the Antichrist would present itself as being for the people. And who are we kidding? That you have the same market incentives to overtrain your models as quickly as possible with RL and the synthetic environments to keep on pushing this thing. We need as much compute. So you can talk about the optics upfront as much as you want, but really that's just window dressing on the top of a burning house.

• 1:32:45Speaker 01

  Yep. What you're saying is super important, and this is one of the, I, this is not me saying everyone should just put their money into Anthropic and then the world will be fine. In fact, there's actually a view by many people that Anthropic in some ways is dangerous because the view that is the safest, it's the Volvo. People know the Volvos are the safest cars. They did a lot of marketing to present that. People think that Anthropic is just the safe AI company. So if they won, then suddenly everything is fine. And that would be dangerous. And that we have to have a critical view of the companies and the technology and the international limits that we need that are, that are going to happen from outside and above the companies, not just what an individual company can do.

• 1:33:30Speaker 01

  So I totally agree.

• 1:33:32Chris Williamson?

  How long have we got?

• 1:33:35Speaker 01

  No one knows because AI is literally moving so quickly. If I went on Twitter, Elon joked in a conversation I saw yesterday. You know, it used to be that you'd go on Twitter and once every six months, this is several years ago, you would see a huge AI breakthrough, like that would just really change the game. Now there's one, you know, that you see when you go to bed and there's a new one when you wake up in the morning and you're on Twitter and you see it. And the pace is pretty overwhelming. So I think rather than ask how much time do we have? Which is kind of like just trying to assuage the fear that I think a lot of people feel. The kind of bold, brave, human thing to do is to ask if things were to go well,

• 1:34:18Speaker 01

  what would that mean about how we were showing up? And then to get as many people as possible showing up from that place? Because that gets us the best decentralized conditions to get to the better outcome. They say in mathematics that in chaos, initial conditions matter. So I'm not guaranteeing in any way, shape or form that we're going to get to the better future. But I would like to invite people again into stepping into and standing from the most wise and mature version of ourselves that would behave as if we were going to get to that better future. Because that would give us the best chances of doing so.

• 1:34:54Chris Williamson?

  And the alternative is just surrender?

• 1:34:57Speaker 01

  The alternative is schisming, deniling, depression, overwhelm. The integrity and the kind of flow of energy that I think you'll feel inside yourself if you align with what is actually needed in this moment. There's more meaningfulness, there's more preciousness of life. Like, there's so much more, like, there's so much meaning and purpose. If you're aligned with what would have things go well. And I think there's this weird thing that's happening is if you don't see a way for things to go well and then you believe the only thing you can do is like triple down on racing as fast as possible, knowing where that's currently taking us. That has a biophysical cost on your system.

• 1:35:37Chris Williamson?

  Yeah, I think the, I was trying to think about the differences between your past work and your current work. Focusing on social media versus focusing on AI. I don't think many people find social media that net positive in terms of their experience.

• 1:35:54Speaker 01

  Right.

• 1:35:54Chris Williamson?

  I don't think that many people find AI that net negative in terms of their experience.

• 1:35:59Speaker 01

  I I would, just so people hear me say this, I agree with you that the current experience of AI is not net negative. It doesn't feel that way, but that doesn't mean that there's a net negative outcome that we are racing towards.

• 1:36:10Chris Williamson?

  But the call to

• 1:36:13Speaker 01

  It makes it harder.

• 1:36:13Chris Williamson?

  The call to get rid of social media is easy when people reflect on the last two hours that they've spent doom scrolling at night. And think, I wish I hadn't done that. The feedback curve is sufficiently quick.

• 1:36:24Speaker 01

  Right.

• 1:36:24Chris Williamson?

  That people know how they felt even during the moment.

• 1:36:27Speaker 01

  That's right.

• 1:36:28Chris Williamson?

  I really shouldn't watch another one of these videos. It makes me feel kind of bad. I'm all stressed and tight. My muscles feel tense.

• 1:36:33Speaker 01

  Yep.

• 1:36:34Chris Williamson?

  The same thing is not true with AI. You're asking people if you were to say, and I haven't heard you say, delete your AI account, stop using it.

• 1:36:42Speaker 01

  I'm not saying that that's the thing that has to happen. Yeah.

• 1:36:44Chris Williamson?

  If that was the campaign, if that was what people were supposed to do, people would be sacrificing their quality of life. They would be able to fix their car less quickly. They would be able to find out things that they need to about their health, to analyze their health documents, to be able to liberate them, to be able to make uh hard to access information cheaper and easier and actually to enable their quality of life in a manner that didn't happen with social media. So I think the intermediary uh experience is less sexy and that I I'm going to guess is also the reason why you're not saying masks unsubscribe, don't use any AI because you're probably aware that the incentives for individual users,

• 1:37:26Chris Williamson?

  the value judgment that they make isn't sufficiently imbalanced as it would have been with social media. For me, it's easy to not have it on this phone. If you were to say, I don't want to have chat GPT on my phone, I I would know I'd need it. I it's very useful to me.

• 1:37:38Speaker 01

  I think it's about understanding.

• 1:37:41Speaker 01

  Context and what is the careful and limited way that these tools are helpful? And then creating the conditions where that's the collective outcome that we're getting. So for example, for kids and tutoring, theoretically, this is the best tutor in the world for learning.

• 1:37:56Chris Williamson?

  Alpha School is doing a South by Southwest pop up in the middle of downtown. Mackenzie Price was on the podcast.

• 1:38:03Speaker 01

  Oh, interesting.

• 1:38:03Chris Williamson?

  Yep.

• 1:38:04Speaker 01

  I mean, I, I think that it has enormous potential. If you look at how most kids are using ChatGPT right now, they're using it to cheat on their homework and to actually outsource their thinking. And actually, because they use it so much, turning into what the, some Atlantic writer wrote called lemmings or L L Mings, where they basically outsource all of their thinking for every moment to moment decision in their day about how I should respond to this person. Or there's a guy over there I want to talk to and I don't know what to, like, whatever the thing is. People are outsourcing their decision and they're not actually learning themselves how to show up as a human. And that is not going to create a safe world. Again, that's a version of the narrative, the possible of a technology that we're promised, and then the probable of what's actually happening.

• 1:38:49Chris Williamson?

  And it's like crumbling cognition.

• 1:38:51Speaker 01

  Yes, exactly. And we saw this with social media. We thought it was going to create the most enlightened and informed society in the entire world. Cause we have the best information. It wrecked our attention and we have the most confirmation bias, tribalized, low trust, worst critical thinking society that we've had in a generation. And so, again, I think this is just about, part of the wisdom we need is, is to be able to look honestly at the nature of a technology and look honestly and confronted, confront ourselves with the reality of how technology is currently being used and deployed. And if it's driving up, you know, an, you know, a social media influencer culture, instead of creating astronauts, like, you know, in, in China, they pull the top, um, what are the, the professions that kids want the most? And the number one was like astronaut, number two was like teacher.

• 1:39:36Speaker 01

  Engineer and then teacher or something like that. And in the US, it was influencer. Like, I can tell you which world and which civilization you're going to get. If we really want to beat China in innovation, we can.

• 1:39:47Chris Williamson?

  Show me the incentives and I'll show you the outcome.

• 1:39:49Speaker 01

  Show me the incentive and I'll show you the outcome. And if you really want to beat China. You regulate social media and you stop brain rotting your entire population and you actually invest in genuine educational technology that Silicon Valley would ship to their own children.

• 1:40:01Chris Williamson?

  This has to come top down because the incentives, both at the user level, especially now for AI, and at the company level and at the market level and at the international level, between companies, all of these incentives align in the same direction, which is I enjoy using the product. My company generates lots of revenue for my country for the product. My country doesn't want to fall behind other countries in the race to have additional capacities for technology. All of these are aligned in the same place, which means that it has to come from above all of that, which is a pan-national.

• 1:40:40Speaker 01

  Global movement for humane technology that's actually in service of people. Part of that movement is redefining the currency of what it means to win that race. Right now, again, the US beat China to the technology of social media, but we have been losing in the race to govern that technology. Like, I think you know these examples, but in China, as an example, they, I think, limit social media use to 40 minutes a day if you're under the age of 14. And I think it's only Fridays, Saturdays and Sundays. Either that's for social media or for video games. There's lights out at 10:00 PM, meaning that if you open a social media app after 10:00 PM, it's closed and it opens again at 6:00 in the morning. This is helpful for when it comes to the kind of late night FOMO,

• 1:41:24Speaker 01

  brain rot, doom scrolling, you know, not getting sleep. And I'm not saying that the US should unilaterally, like some totalitarian government, make that choice, or it's just that we can democratically come up with guardrails like this. Another thing that China does is they have a synchronized final exams week and they shut down AI during final exams week.

• 1:41:41Chris Williamson?

  No way.

• 1:41:42Speaker 01

  Yes, so that all the students know that if they used AI during the school year, it will completely screw them over during final exams week.

• 1:41:51Chris Williamson?

  Because they're going to be reliant on something they can't use.

• 1:41:53Speaker 01

  That's right. And so we don't have to do what China does, but the point is we should do something. In China, they're regulating anthropomorphic AI to not do all the attachment hacking and the suicides and all this kind of stuff. I'm not saying what they're doing is good. I'm just saying they're doing things. We're not doing anything. And we can, and the purpose of the human movement is to actually be demanding, you can actually go and call your legislator and say that you do not want this anti-human future, you want accountability for AI companies, you want to ban legal personhood for AI, you want, you know, no anthropomorphic AI, so that we don't get the child safety issues with AI chatbots.

• 1:42:30Chris Williamson?

  If I was to give you the option of turning the rest of the world into a totalitarian state, everything, but it meant that we were able to dictate what happened to countries and to companies and to these AIs.

• 1:42:43Speaker 01

  I don't want to create a totalitarian state.

• 1:42:46Chris Williamson?

  But the alternative would potentially be destruction of humanity.

• 1:42:50Speaker 01

  Well,

• 1:42:51Chris Williamson?

  Which would you pick?

• 1:42:51Speaker 01

  You are conjuring a thought experiment from Nick Bostrom, which I'm sure you're aware of, and the paper you're referring to is the vulnerable world hypothesis, in which he says that as we discover more and more technologies that in a decentralized way have the capacity to destroy things. So like your example of if it turned out that microwaving sand created a nuclear reaction that blew up the entire world and we publish that knowledge and it spread virally on social media, how many minutes, hours or seconds would it take before the world blew up? And then the only answer to a distributed, like, available to everybody destructive

• 1:43:33Speaker 01

  capacity would be a global totalitarian society that monitors and surveils everyone and what they're doing. So you prevented mass destruction, but you got 1984 Big Brother, and that's an uncheckable power that people cannot fight back against. I am very worried about mass surveillance. I think that what happened with Anthropic is hopefully bootloading a global immune system to the risks of AI-enhanced mass surveillance. Because in a way, Big Brother could have never happened without AI. When you have an AI that can process every camera and every face and all of the emails and text messages that are flowing through everything, and you can actually ask the question as a state, who are the

• 1:44:15Speaker 01

  number one threats in my society that I need to suppress? And it tells you and summarizes instantly who those people are, synthesizing all these data streams. How can you fight back against an uncheckable power when it knows all of your secrets? You can't. So we're faced with very tough choices on both ends. And I wrote, I did this in a TED Talk I gave this last year called The Narrow Path, that we have to avoid both outcomes. We have to somehow be committed to finding the narrow path between, you know, decentralizing destructive power in everyone's hands without responsibility, without a commensurate wisdom or responsibility, in which you get chaos or catastrophes. Or over-centralizing that power in uncheckable ways, in which you get runaway dystopias.

• 1:44:59Speaker 01

  And we have to find something like a third attractor to quote Daniel Schmachtenberger, or a narrow path that seeks to avoid both these outcomes. You could say that that is outside the laws of physics or not possible. But I think what's, what's important is to be committed to and living from the place where we would be finding that path. Because the only chance that we would have of finding it would be requiring that everyone was living in service of it.

• 1:45:24Chris Williamson?

  Isn't it funny that the two worlds that you identified there, one totalitarian overreach that's controlled by humans and at the mercy of their biases. And the other being decentralized vulnerability, danger that is at the mercy of technology's liabilities. Both of those are extreme versions of something that we definitely don't want. But the restriction on the one that would protect us from the AI goes back to the bottom of the brainstem. That there's almost never a time that a country or a government or a small number of people or a large number of people get most of the power. And then give it back. It's a ratchet system.

• 1:46:05Speaker 01

  That's right.

• 1:46:06Chris Williamson?

  And it only ever clicks into place and then never goes back.

• 1:46:08Speaker 01

  Which is why you need to have built in checks and balances on power that are irreplaceable. Like you cannot have a world where that whatever power gets centralized has to have oversight. And democratic accountability and distribution and wealth of that wealth and power, depending on if we're talking about the wealth versus the power.

• 1:46:23Chris Williamson?

  How would we know that some country isn't signing up to a moratorium but secretly doing all of their research behind the scenes so that they slow down debt progress of everyone else, but allow them to race ahead?

• 1:46:34Speaker 01

  So you're asking the question, if, let's say, the US and China sign an agreement, how do we know that, given their maximum incentive to defect on that agreement and have the CIA or their black operations still continue to do the research project?

• 1:46:47Speaker 01

  This is obviously the hardest technology and coordination problem that humanity has ever faced. I just need to say that at the top. Like, I'm not saying any of this is easy.

• 1:46:55Chris Williamson?

  It makes nuclear war look like child's play.

• 1:46:57Speaker 01

  Correct. And but by the way, I want you to know, there's a great video, and I can send it to you, of Robert Oppenheimer, asked in the 1960s, I think he's testifying for something, an interview, and they asked him, what, how could we control this technology from proliferating? And he takes a big puff and a cigarette, and he says, it's too late. If you wanted to prevent the spread of nuclear technology, you would have had to do it the day after Trinity. Trinity was the first test of the nuclear bomb.

• 1:47:31Speaker 01

  He believed, and this interview was in the 1960s, he believed it was inevitable. There was nothing we could do, and the world was fucked.

• 1:47:38Speaker 01

  Even he, the creator of the atomic bomb, didn't see that a lot of people would work very hard over the course of the next 30 years to invent what's called national technical means, or satellites that do mutual monitoring enforcement, that we would create the seismic monitoring technology to be able to see if you were doing an underground nuclear test or an above-ground test, because we would see the reverberations of that. So satellites, seismic monitoring, you know, international inspectors, the International Atomic Energy Agency, we had to invent all of these new things to create a governance system that was capable of dealing with nuclear weapons.

• 1:48:15Chris Williamson?

  Interesting that the innovation around the technology that was dangerous was probably less sophisticated than the downstream technologies required to make it secure.

• 1:48:25Speaker 01

  What do you mean?

• 1:48:26Chris Williamson?

  That making the atomic bomb was one thing.

• 1:48:28Speaker 01

  Yes.

• 1:48:29Chris Williamson?

  Then there's 50 things that need to happen in order to have that technology exist in the world.

• 1:48:34Speaker 01

  In a safe way. Yeah, exactly. There's a safe, there's sort of an asymmetry, where the technologies, the destructive capacity is easy to create, the governance capacity is way harder to create.

• 1:48:41Chris Williamson?

  Bingo.

• 1:48:42Speaker 01

  The defensive capacity is harder to create.

• 1:48:43Chris Williamson?

  Yeah.

• 1:48:44Speaker 01

  You could have said at the beginning of 1945, I guess it's inevitable. 150 countries are going to have nuclear weapons. There's nothing we could do. Let's just like drink margaritas and give everybody uranium and increase GDP by selling uranium to the entire world. But we didn't do that. And people had to invent stuff. We needed our best engineers and our best minds working on how to figure that out. And I'll just say, if you want to look this up, RAND, the nonprofit defense think tank, has a paper on how international monitoring and verification mechanisms could potentially work for AI. It's not easy.

• 1:49:14Chris Williamson?

  Hmm.

• 1:49:15Speaker 01

  It is not trivial.

• 1:49:16Chris Williamson?

  Okay.

• 1:49:16Speaker 01

  It would require extraordinary investments in mutual monitoring and enforcement and data centers and like the, you know, chips that attest where they are and all this kind of stuff.

• 1:49:25Chris Williamson?

  Okay, so there are some suggestions.

• 1:49:26Speaker 01

  There are proposals.

• 1:49:27Chris Williamson?

  Because what I was thinking is you were talking about, well, we can use satellites that work out. If you're doing things above ground or below ground, we can monitor maybe the amount of.

• 1:49:34Speaker 01

  Heat signatures and electricity signatures, power usage.

• 1:49:36Chris Williamson?

  So there are some.

• 1:49:37Speaker 01

  There's inspectors.

• 1:49:38Chris Williamson?

  Yeah, there are some signatures of people that are doing this sort of research. Because you can't do it without there being any externality. You can't just run it on a MacBook and make it look like you're browsing the internet.

• 1:49:47Speaker 01

  You need large clusters of compute right now, to your point. In the future, it may shift.

• 1:49:51Chris Williamson?

  Yeah.

• 1:49:51Speaker 01

  But right now, you need large clusters of compute. You need advanced semiconductor manufacturing supply chains. Right now, that's a set of US allies, the Trilat. I believe it's Denmark, Japan, South Korea, obviously Taiwan. You know, we could create some kind of infrastructure for not some totalitarian control of compute, just a governance regime. Would you call the IAEA totalitarian? Or would you call it a governance regime for a dangerous destructive nuclear capacity? I'm not saying any of this is easy. We have to navigate the narrow path. We don't want to create totalitarian controls. We don't want to create runaway catastrophes. What it takes is being committed to finding that path. How much time or energy or resources have we spent even trying? People say it's impossible. Have you spent a month dedicatedly trying?

• 1:50:35Speaker 01

  People say it's impossible. Have we spent any amount of resources actually trying to do this? No.

• 1:50:41Chris Williamson?

  Let's say that there was some way for you to have God's eye coordination and that you could step in. Would you just put a pause on all model development to allow us to catch up in terms of AI safety?

• 1:50:53Speaker 01

  I think, I think the people, rather than it sounding like I'm some kind of pause or stop person, the people building this technology have basically said that that's what they would most prefer. They would most prefer a world where we basically stopped and had the time to integrate and develop this technology slowly. That's what they would prefer. What do you think that if you had control, what would you do? I would agree with the people who I'm going to defer to as the people who know the most about the dangerous and destructive capacities. So I don't want people to think this is my view, that I'm some kind of, you know, safety nut or something like that. This is just about like, I want to be able to look my children in the eyes and say, we did everything that we could to create the best possible future for you.

• 1:51:34Speaker 01

  And I think that basic heuristic of like, are you operating in service of life and the things that most matter? I think that that would probably be the wisest thing to do. And just to people like ground that, the CEO of Microsoft AI, Mustafa Suleyman, who's a friend, says, in the future with technology, progress will depend more on what we say no to than what we say yes to. Your podcast is called Modern Wisdom. Is there a definition of wisdom in any spiritual or religious tradition that does not have restraint as a central feature of what it means to be wise? Does any religious tradition or spiritual tradition say, you know what's wise? Going as fast as possible, not thinking about the consequences and like max, you know, dopamine maxing your brain. Like, it's

• 1:52:16Speaker 01

  the opposite. It's like, this is not hard. This is so trivial. It's so obvious. It's so obvious. It's just like, snap out of the trance. This is not inevitable. They want you to believe it's too late. It is very far down the tracks. It would be a rite of passage, even if it didn't work out for us to show up with the maturity and responsibility to at least be trying to live in service of the future that we actually want to create. Can we watch the trailer? Have you got, can we get the trailer up? I want to see the trailer for this movie. That'd be great. That'd be great. Thanks.

• 1:52:44Speaker 04

  If this technology goes wrong, it can go quite wrong.

• 1:52:54Speaker 04

  Your fear of AI is the collapse of humanity. Well, not the collapse, the abrupt extermination. There's a difference.

• 1:53:05Speaker 04

  So I started making this movie because my wife is six months pregnant. Is now a terrible time to have a kid.

• 1:53:13Speaker 04

  I mean, just to be honest, I know people who work on AI risk who don't expect their children to make it to high school. I, what the?

• 1:53:23Speaker 04

  How does AI understand pretty much everything? It's surprisingly straightforward. Intelligence is about recognizing patterns. Patterns? Patterns. Patterns.

• 1:53:34Speaker 04

  If you have learned those patterns, you can generate new information.

• 1:53:38Speaker 01

  AI is moving so fast.

• 1:53:40Speaker 05

  It's being deployed prematurely. There's so much potential for things to go wrong.

• 1:53:45Speaker 04

  Why can't we just stop? All these companies are in a race to get AI. That's vastly more intelligent than people within this decade.

• 1:53:55Speaker 04

  China, North Korea, Russia.

• 1:53:56Speaker 06

  Whoever wins is essentially the controller of humankind.

• 1:54:02Speaker 04

  We need to take a threat from AI as seriously as global nuclear war.

• 1:54:11Speaker 04

  It feels like I have to find these CEOs. And get them in the movie. Great. I want to ask you to promise me that this is gonna go well. That is impossible. Okay. Am I hopeful? Yes. Am I confident that it'll go right? Absolutely not.

• 1:54:29Speaker 07

  AI is the thing that can solve climate change. We could cure most diseases.

• 1:54:34Speaker 08

  What if it's expanding what is humanly possible?

• 1:54:36Speaker 07

  This is the most extraordinary time ever. The only time more exciting than today is tomorrow.

• 1:54:41Speaker 04

  I already love you.

• 1:54:42Speaker 07

  Okay.

• 1:54:44Speaker 04

  I think if this technology goes wrong, it can go quite wrong.

• 1:54:49Speaker 07

  By using AI. We're about to move off of the Earth into the cosmos.

• 1:54:55Speaker 01

  If we can be the most mature version of ourselves, there might be a way through this.

• 1:55:03Speaker 04

  This is the last mistake we'll ever get to make.

• 1:55:11Chris Williamson?

  Dude.

• 1:55:13Speaker 01

  Does it land differently after this conversation?

• 1:55:16Chris Williamson?

  It does. I'd already seen it, but yeah, it definitely, uh.

• 1:55:20Speaker 01

  Yeah.

• 1:55:21Chris Williamson?

  I definitely understand more of what's implied that.

• 1:55:24Speaker 01

  Yeah.

• 1:55:24Chris Williamson?

  Uh, it's impressive that you managed to get all of the guys to sit down, or most of the guys. Who are you missing?

• 1:55:29Speaker 01

  Uh, I mean, there's obviously many people building this technology now, um, so they don't have the Chinese labs in it. They have Demis Hassabis from DeepMind, Sam Altman from OpenAI, Dario from Anthropic. I mean, those are the three most leading players. Elon agreed to participate in the movie and then, um, I think he was right in the beginning of the first, um, few days of the Trump administration that he was busy and didn't follow through. So. Uh, but I think the team really wanted him to be in the movie and wanted to hear his views. I mean, to give Elon credit, he was one of the first people who cared about AI safety. And he said, mark my words, AI is far more dangerous than nukes. And he said this in like, what was it? 2015, 2016?

• 1:56:10Speaker 01

  Like way before people were taking AI seriously. Like back then AI was just recommending what other product you should get on Amazon and doing facial recognition for toll booths and, you know, bridges and stuff like that.

• 1:56:20Chris Williamson?

  Think about what much of superintelligence and sort of the fallout around that book from Nick was. So much of it was almost

• 1:56:31Chris Williamson?

  advice for how to talk to your friends about this without being mocked too much.

• 1:56:36Speaker 01

  Right.

• 1:56:36Chris Williamson?

  It was, these are the best examples to use so that you don't sound like the insane person at Thanksgiving dinner.

• 1:56:42Speaker 01

  And ironically, the paperclip maximizing example did make people sound ridiculous and insane, uh, that then got really, you know, diminished. And what's the word I'm looking for? It just made people, it was used to tarnish people's reputation if you talked about paperclips. But it's funny because people say, oh, like the AI is going to, for people that don't know the example, it's like. There you are, and the AI is gonna be told to maximize paperclips. And then the way it will figure out how to do that is figuring out any strategy, which means like turning every atom in the universe to paperclips, which means like, you know, melting all the humans down and getting them to, it just taken to this extreme. And it sounds totally sci fi, but if you actually ask a baby AI called social media that's pointed at your brainstem, figuring out just what video to get you to watch that

• 1:57:23Speaker 01

  keeps you on the screen, and you say maximize engagement, well, conflict and rivalry and civil war is really good for engagement. So in a way, we have been maximizing a paperclip called attention and eyeballs for a long time, and it's driving up division and rivalry everywhere around the world, and democracies are backsliding everywhere around the world, and it's driving up confirmation bias all around the world. And the point isn't that the baby AI of social media hates you, or hates your well being, or hates your connections, or hates your democracy. It's just that it doesn't care about anything other than whatever keeps your eyeballs. And that little baby AI that was just figuring out which photo or video to throw in front of your nervous system was enough to completely transform everything, everything, everything

• 1:58:08Speaker 01

  about how our society worked. And I saw that, by the way, and I'm not some kind of like, it's not like I especially see the future, but in 2013, it was so obvious to me, if you take these incentives really far in a decade, I can tell you where you're gonna live in. And the thing that I want people to feel is if you can see the incentive, you can, and you have that clarity, you can confidently say, I don't want the future that that creates. In 2013, Mark Zuckerberg could have said, oh, my God, I see we're about to create an arms race to hack human psychology. I'm gonna convene all the leading social media companies and the government. I'm Mark Zuckerberg. I've got billions of dollars. I'm gonna throw money at this. I'm gonna get the government people involved, not because the government's trustworthy or because we like them, but because I see that we need some kind of rules, and I'm

• 1:58:49Speaker 01

  gonna try to create some rules that say, no auto playing videos, no infinite scroll, no one can create unnecessary FOMO, you can't dole out like a few likes here and then a few likes there, like a slot machine, you have to do them all at the end of the day or something like that. Like, you can create rules and norms so that we didn't get the mass addiction distraction machine that we then did get. And he could have done that in 2012, 2013. That's what leadership, that's what maturity, that's what wisdom would have done. It's not that hard to see it.

• 1:59:18Chris Williamson?

  The problem, I think, that you're facing is every second is so crucial and every single second of compute and of CEO attention and of staff base attention is going to be paid on continuing to get ahead in this unbelievable one to rule them all race. I mean, OpenAI is a partner on this show. I've spoken to Sam twice and both times the calls have been so brief because presumably for every second that he's on the phone, that's billions of dollars of potential revenue or compute or that is being wasted. And that means that if you want Mark in 2013 or Sam or Dario.

• 1:59:59Speaker 01

  But the point is you create, so like, literally, this is the history of all of law. So like, I could kill you and steal your stuff and just take your money, but I'd prefer to live in, and everyone could do that. And that'd be a faster way to like, get money. It's like, just everybody kills everybody, grabs their money. But that would create a society that's chaos. So instead, I sacrifice some of my abilities, like I can't kill people. And instead, we have law that we, we all sort of notch down some of our individual capabilities so that we get to live in a society that we actually want to live in. And that's what this would do. So in 2013, if Mark Zuckerberg had said, I'm going to convene, you know, Musically, which was before TikTok, Twitter and the other ones. And we're like, okay, look, it is so obvious we're in this arms race for

• 2:00:40Speaker 01

  attention, just like public utilities, which by the way, if like, in California, it'd be PG&E. In Texas, what's your electricity provider? It's like, oh, fuck knows. I don't know. I have no idea. I have no idea what my electricity provider is. Well, technically, energy companies have an incentive to maximize revenue. So theoretically, they're like, leave the lights on, leave the stove on, like, do the water 24/7. And because we make more money when we do that. But because public utilities rely on a scarce resource called energy that has environmental emissions that are, that are, we have to deal with. There's a decoupling of revenue from their incentive. So, for example, in California. You're charged a base rate for like the initial energy you use.

• 2:01:22Speaker 01

  And then, once you're hitting kind of the capacity that would be straining the system, we start to charge you more. But that extra revenue doesn't go just into PG&E, the energy company's pocket. It goes into a fund to have more clean energy that gives Offset it. To offset it and create more energy capacity. So with attention, you could say, instead of companies maximizing attention and driving revenue, you get to make money from the initial tier one of attention that you're getting. And then after that, the resources go into a common pool that's investing in the research, the X prizes, the design solutions that research and show that there's different ways of doing news feeds. There's different ways. We could have news feeds that are all about directing you back towards community. You live in Austin, Texas. Austin has, at least my friends, there's a lot of community events happening here.

• 2:02:07Speaker 01

  But you can imagine a world where the default news feeds that govern our world, we're spending like 30 to 40% of what you saw was other events that your friends and other people that you knew adjacent to you were doing. You could imagine a world where instead of dating apps profiting off of keeping you in the slot machine loneliness thing, where you message someone and never get back to them. In this like new world, all the dating apps would be forced, probably because of some lawsuit against the engagement model, to fund physical events in every city that they operated in, where every single week, there was actually a physical venue where lots of people were put in the same room with lots of other people they matched with, who are maximally overlapping. And now, instead of feeling scarcity and loneliness, there's this sense of abundant access to community and soft dating and soft friends making environments. So you're dealing with a loneliness crisis.

• 2:02:52Speaker 01

  You're dealing with the dating problem. When you solve that, then it turns out 30% of the polarization online goes down because it turns out that a lot of polarization was just people being lonely and depressed from not having Manufactured. It's all manufactured. Yeah. Because people are caught in the doom swirling loop. So it's like, again, we can have a better world if we deal with the upstream causes of these problems. We can also have more innovation. Peter Thiel was talking about, we need more innovation. We need more scientific development. You know, we would have stagnation if we didn't have AI. Well, how about you regulate the brain rot economy? That's currently degrading all the innovation, causing people to be, you know, not productive and creative and innovative members of our society. And instead incentivize entrepreneurs and, and again, social groups and community. So people are actually making things. So the point is,

• 2:03:35Speaker 01

  I'm a tech, I'm a different kind of technology optimist, which is a humane technology optimist. If you design technology that is humane to an understanding of the substrates of society that upon which everything else depends, you can have a technology environment that is conducive to the things that we want our society to do. But it requires different design principles, different incentives, different rules, and some coordination.

• 2:03:59Chris Williamson?

  This third attractor precipice narrow path thing.

• 2:04:02Speaker 01

  Yep.

• 2:04:03Chris Williamson?

  That you think is important for us to walk appropriately. The pace at which things are going to happen and change. Do you think that we're going to be able to move along that? I mean, you've got this movie. It's going to be a global moment. Lots of people are going to see it. It's going to get people talking, common knowledge. Typically, things tend to take time. Conceptual inertia usually moves over generations and centuries. Going from heliocentric model of the universe took like 100 years.

• 2:04:29Speaker 01

  Well said. Yep.

• 2:04:32Chris Williamson?

  How?

• 2:04:33Speaker 01

  This is the problem in general that we face with technology. And this is the E. O. Wilson quote. It's not just that we have paleolithic brains, medieval institutions, is that the brains operate on a very slow updating clock rate. The institutions operate on a slow clock rate. And the technology moves at whatever the 21st or now 24th century technology clock rate that AI moves at.

• 2:04:50Chris Williamson?

  How do you think about expediting this change?

• 2:04:52Speaker 01

  You have to have your governance move at the pace of the thing that you're trying to govern.

• 2:04:58Chris Williamson?

  Is that ever going to be possible? Governance moving at that pace, the lumbering behemoth? What is it? Leviathan leans left and lumbers along?

• 2:05:07Speaker 01

  There needs to be rather than recursively self improving AI that is uncheckable by any human oversight process, in which we will for sure lose control and build something crazy that we regret. We instead should have self improving governance. You can use AI to look at all the laws on the books and say, what are the laws that don't matter anymore? That are creating all this red tape that we don't need? That's like stifling innovation? And get the AI to find those laws and reinterpret them and say, what are the ones we need to get rid of? And how do you rewrite them for 21st century technology in the current age? We could be using AI and technology to update the governance as fast as the technology, but we'd also probably need to slow down the technology, which we'd benefit from because then we wouldn't crash. Again, it's like China wins when they whisper in our ear,

• 2:05:51Speaker 01

  go faster, go faster. You're going to blow yourself up.

• 2:05:54Chris Williamson?

  Pyrrhic victory ahead.

• 2:05:55Speaker 01

  Pyrrhic victory. You've, you said this earlier. When US races ahead and has a more sophisticated model, China gets it like 10 days later because first of all, they have spies in all of our companies. And second of all, they distill all the models.

• 2:06:06Chris Williamson?

  Is that what they're doing?

• 2:06:07Speaker 01

  They have spies in the companies. You know, they're maximally incentivized to know what's going on. Plus, as I, as you were saying, they can distill the US models, meaning they can like query those models a thousand times and kind of distill the essence of those models and then make and train theirs. There's a great like meme online of like, it's a motorboat with a guy who's, what is it called when you're?

• 2:06:28Chris Williamson?

  Wake surfing?

• 2:06:29Speaker 01

  Wake, yeah, like you're wake surfing behind him on the, on the string. I mean, on the, what is that called?

• 2:06:33Chris Williamson?

  The? String.

• 2:06:34Speaker 01

  Yeah, whatever. And, and the, the guy who's on the, um, and the surfing says, go faster to the boat. That's like racing ahead. That's the US that's building the advanced model. But then they're literally on a string right behind them. That's getting all the benefits from it.

• 2:06:49Chris Williamson?

  Yeah.

• 2:06:49Speaker 01

  And there is a study that Anthropic found

• 2:06:53Speaker 01

  that China had been covertly even using US Anthropic AI models to perform a cyber hacking operation. So like, again, if we're winning the race to the technology, but losing the race to governing or controlling or protecting the technology, what the fuck are we winning?

• 2:07:08Chris Williamson?

  Yeah. Yeah.

• 2:07:10Speaker 01

  It's like, it's so basic what we're talking about. It's like, this is not rocket science. It's unbelievably simple. If you have the power of gods, you need the wisdom, love and prudence of gods. I hope we've laid out a bunch of examples of how we can do that. And while it's not easy, it's not, it does not happen by default. If you get everyone you know to go out and see the AI doc, understand that AI is dangerous, get your church group, get your business, and recognize that we need to, you know, not build bunkers, but write laws to actually steer AI before it's too late.

• 2:07:41Chris Williamson?

  You're terrifying, but I'm glad that you're in the world, man. I appreciate you.

• 2:07:44Speaker 01

  I appreciate you too. I really appreciate this conversation. Thank you.

• 2:07:46Chris Williamson?

  All right. Goodbye, everybody.