Yuval Noah Harari on trust, the dangers of AI, power, and revolutions

What will it take to create AI that is as trustworthy, if not more trustworthy than humans? This week, Reid and Aria sit down with Yuval Noah Harari, historian, philosopher, and best-selling author of several books including Nexus, Sapiens and Homo Deus. When it comes to outlook on AI, Yuval, Reid, and Aria agree on the importance of building both human trust in AI and AI that is genuinely truth-seeking, but they differ on how possible it is to achieve. Together, they dig into their diverging opinions on the outcomes of the AI revolution, global cooperation, and how AI will learn from humans. They also discuss the differences between intelligence and consciousness, and whether conscious AI is a goal worth pursuing. Yuval turns to history to ground his warnings about AI. Even though he’s cautious about technology, he is critical of cynicism. Yuval shares his philosophy on human compassion as a guiding principle that can allow us to steer away from collapse and ultimately, build a better AI future. For more info on the podcast and transcripts of all the episodes, visit https://www.possible.fm/podcast/ Topics: 3:38 - Hellos and intros 3:58 - Questions for the Buddha 5:48 - Yuval’s relationship with technology 8:57 - Technologies that help humans share stories and myths 10:37 - Is AI the most significant invention after writing 13:02 - How AI will transform society 20:12 - Guidance for a successful AI revolution 24:24 - Using AI to support humanity's self-correcting mechanisms 26:13 - Midroll 26:45 - How to build self-correcting mechanisms for a better future 31:28 - Humans as parents of AI 36:33 - What political leaders need to do to create a positive AI future 39:11 - Artificial intelligence v.s. artificial consciousness 42:35 - AI as a tool for rebuilding trust 44:50 - Rapid-fire Questions Select mentions: History of the Franks by Gregory of Tours Heartstopper Possible is an award-winning podcast that sketches out the brightest version of the future—and what it will take to get there. Most of all, it asks: what if, in the future, everything breaks humanity's way? Tune in for grounded and speculative takes on how technology—and, in particular, AI—is inspiring change and transforming the future. Hosted by Reid Hoffman and Aria Finger, each episode features an interview with an ambitious builder or deep thinker on a topic, from art to geopolitics and from healthcare to education. Each episode seeks to enhance and advance our discussion about what humanity could possibly get right if we leverage technology—and our collective effort—effectively.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  If the AI tycoon himself behaves in an untrustworthy way, and if he himself believes that all the world is just a power struggle, there is no way that he can produce a trustworthy AI. But this is not the hall of reality. I would tell people who believe in the cynical, power-hungry worldview, is just observe yourself. Do you think that you yourself don't care at all about the truth? And you just want power? Probably not. Now, if you have a better view of yourself, why aren't you charitable enough to acknowledge that other people, just like you, might also be really interested in knowing the truth? And this is where we can start to build

• 0:45Speaker 00

  a more trustworthy philosophy, which is the only stable foundation for a good society and also for benevolent AIs.

• 1:03Reid Hoffman?

  Hi, I'm Reid Hoffman.

• 1:04Aria Finger?

  And I'm Aria Finger.

• 1:06Reid Hoffman?

  We want to know how, together, we can use technology like AI to help us shape the best possible future.

• 1:13Aria Finger?

  With support from Stripe, we ask technologists, ambitious builders, and deep thinkers to help us sketch out the brightest version of the future. And we learn what it'll take to get there.

• 1:26Reid Hoffman?

  This is possible.

• 1:31Reid Hoffman?

  It's easy to forget the shape technology can take. Agriculture, writing, the printing press, the internet. All of these are technology and are tools that people have used to profoundly expand what all humans can do.

• 1:47Aria Finger?

  Reid, you pointed this out in your latest book, Superagency. But AI is yet another tool in this long line of transformative tools. And, just like these other technologies, AI can have both beneficial and detrimental effects.

• 2:04Reid Hoffman?

  Right. As we build these systems that are capable of learning, adapting, and even persuading, the question isn't just what these technologies can do. It's who they serve and whether they are being shaped to amplify our humanity or undermine it. So what will be our story with AI? And how do we work to ensure it amplifies our better selves? Today, we're joined by someone who approaches these questions with a long view and who often illuminates the past to inform where we're headed in the future.

• 2:36Aria Finger?

  Yuval Noah Harari is a historian, philosopher, and bestselling author whose books Nexus, Sapiens, Homo Deus, and 21 Lessons for the 21st Century reveal profound takes on today's challenges through the lens of centuries of human evolution. Yuval sees AI a little differently than we might, as more of an unprecedented agent than a tool. He warns that AI may leave people with little to do, pose a threat to democracy, and manipulate human belief. But this isn't prophecy, it's a possibility, one that humanity may be able to avoid, depending on how we build, iterate, and deploy AI.

• 3:20Reid Hoffman?

  Yuval and I agree on a lot, but also diverge in places. We sat down for a vibrant discussion on the rewards, risks, and responsibilities that come with AI. Here's our conversation with Yuval Noah Harari.

• 3:38Reid Hoffman?

  Yuval, I've been looking forward to this for months. It is awesome to welcome you to Possible.

• 3:44Speaker 00

  Thank you. It's good to be here.

• 3:46Reid Hoffman?

  So let's start with a variant of a question that I frequently use at dinner parties, which is, in an interview with National Geographic, you said the person that you'd most want to talk to from all of human history was the Buddha. And I'm guessing meditation practice might inform this choice. What would be the question you would most want to ask the Buddha?

• 4:08Speaker 00

  It would be about consciousness, about sentience, the nature of consciousness. And, you know, I mean, this would be to start on a very, very deep note. But my understanding of consciousness, if I had to give a simple definition of consciousness, is it's the only thing in the entire universe that has the capacity to suffer. And of course, also to feel joy and love. But neither atoms, nor galaxies, nor almost anything in between can suffer. Consciousness can. This is why, this is the central theme of all ethics. And then you ask, what exactly is suffering? And suffering is the rejection of reality.

• 4:52Speaker 00

  Consciousness is the one thing in the universe which doesn't just observe, notices what is happening, it also rejects it. And the big question I would ask, what is it in the universe that can reject reality? Like, if you would try to describe it, say, mathematically, can you write an equation for something that rejects reality?

• 5:20Reid Hoffman?

  I mean, this will get very deep as we get into AI. You might be able to.

• 5:24Speaker 00

  It's a central question for AI, of course. I mean, can AI suffer? Can AI reject reality?

• 5:30Reid Hoffman?

  And what are the conditions under which it would? I'd say current AI, no. What are the conditions where it gets to? Is, I think, one of the questions we are now in the process of learning. Right? And, you know, it may take centuries to learn, because who knows exactly where it'll go? We'll get into AI shortly. But one of the things I'm curious about is your personal relationship with technology. Like, for example, most people tend to never let their smartphone get too far from them. Right? It's like, must be on. And actually, one of the things I think people have to train themselves is not to look at their smartphone too often, you know, not to interrupt themselves. So what's your relationship with technology?

• 6:10Speaker 00

  It's complex. You know, as people say, I try to use it without being used by it. So for many years, I just didn't have a smartphone at all. Now it's become completely impractical. There are some, like, healthcare services and stuff that they require me to have a smartphone. So I have it, but I use it like an old phone from the 90s, basically. I mean, I use it to make phone calls. And sometimes to send text messages. And whatever kind of essential applications, the healthcare system, whoever forces me to use, and that's it. I usually don't carry it with me, like it's not here now, I left it at home. And I'm very aware of the capacity of technology to take over our mind,

• 6:53Speaker 00

  shape our desires, our thoughts. And the idea that I'm smart enough and strong enough to resist it, that's not true. This is why it's called a smartphone and not a stupid phone. Because it's very smart. And if you think you could outsmart it, I mean, some of the smartest people in the world have been working for years to make sure that this thing can manipulate my mind. And I don't want to give it too much access. So, and it doesn't mean that I don't use it or that I'm against technology. I met my husband 23 years ago

• 7:34Speaker 00

  in one of the first dating sites for LGBT people. And, you know, for gay people, the internet and all this technology was one of the most amazing things ever. Because the gay community, one of its characteristics is that it's not just a small minority, it's a dispersed minority. Like if you think about, I don't know, Jews. So, as a Jewish boy, I was born to a Jewish family. But as a gay boy, I was not born into a gay family. And one of the biggest obstacles for LGBT people throughout history was simply finding one another.

• 8:16Speaker 00

  And I grew up in the 1980s, in a small town in Israel, in a very homophobic environment. I didn't know anybody who was out. And then the internet came along. And suddenly it became easy, amazingly easy, to find each other.

• 8:34Aria Finger?

  Well, actually, that really transitions well to my question. You're talking about the Internet's ability to give people shared stories, shared narrative. You're finding out about what other people are doing. And in your excellent book, Sapiens, you describe the cognitive revolution as the turning point when Homo sapiens rose to dominance through our unique capacity to create these shared and collective stories. That was 70,000 years ago. In your mind, what are the technologies that have been most pivotal in approving our collective ability to create these stories and shared myths?

• 9:05Speaker 00

  So you have the list of the usual suspects with writing and print. But the key thing to understand is that technology is never just about coming up with some technical invention. It always demand social and cultural and psychological work as well. If you think about writing, which was probably the biggest revolution after the cognitive revolution, actually, the technical aspect of writing, of script, is extremely simple. The first writing system we know about was developed a little more than 5,000 years

• 9:48Speaker 00

  ago in what is today's southern Iraq, the Sumerian city-states. And it was basically just, it was based on taking pieces of mud, clay tablets, which are just mud, and taking a piece of wood, a stick, and using the stick to imprint certain signs on the clay tablet, on this piece of mud. That's it. This was the whole technology in the sense of, again, the physical, the technical aspect of it. But how to create the code and how to teach people to use the code, this was the really difficult thing. And this is what transformed the world.

• 10:32Reid Hoffman?

  There's a specific area of something that I've said about AI that I'm curious on your reflection of, which is, I have sometimes said that AI is the most significant invention after writing. What would your reflection on that statement be?

• 10:49Speaker 00

  I think it's more significant

• 10:50Reid Hoffman?

  Yes.

• 10:51Speaker 00

  in its potential.

• 10:52Reid Hoffman?

  Yeah.

• 10:53Speaker 00

  That writing expanded the abilities of the species that already dominated the planet, Homo sapiens. AI, at least according to certain scenarios, is the rise of a new species that could replace Homo sapiens as the dominant life form or the dominant, at least, intelligence form on Earth. So, I mean, in 2025, I would say, yes, writing is still more important than AI, especially as AI is really a continuation of writing by other means. But looking to the future,

• 11:37Speaker 00

  I can imagine a scenario when the rise of AI will be an event on a cosmic scale in a way that writing isn't. That sometime in the future, entities would look back at the history of the universe and they would say, well, you had the beginning of organic life four billion years ago and then you had the beginning of inorganic life with the rise of AI. Writing and all these human stuff, this is just the small details of how organic intelligence eventually gave rise, gave birth to inorganic intelligence.

• 12:19Reid Hoffman?

  That's another very deep subject along with consciousness that I think I'm still going to defer a little bit, right? Just because I think it's, you know, worth lingering on in depth. Maybe it's worth saying that part of my vision of this is what the probability is that over the next century, AI is a tool versus it's a species. It's neither 0% that it's a species nor 100%.

• 12:42Speaker 00

  Yeah.

• 12:43Reid Hoffman?

  Right? But one of the things that you've written about is how AI is going to transform kind of even as it is, even as the amplifier of writing, how we work. Even as a tool, it's a transformation of society.

• 13:00Speaker 00

  Absolutely.

• 13:02Reid Hoffman?

  So say a little bit about what do you think are the key things that are likely in that transformation? You know, how should we, as the toolmakers, build it? How should society adapt? Does it make some people? I think you referred to, you know, a potential useless class.

• 13:23Speaker 00

  Yeah, that's one of the biggest dangers that we face.

• 13:25Reid Hoffman?

  Which we want to avoid.

• 13:26Speaker 00

  Yeah.

• 13:27Reid Hoffman?

  Right? So say a little bit about what the dangers are and what are some of the things to do to try to mitigate against those dangers?

• 13:34Speaker 00

  One danger that everybody's talking about is that a lot of people will not just lose their jobs, but will find themselves almost unemployable. Because even if there are new jobs, you need completely new skills. And who is going to invest the time, the financial resources, in retraining the population again and again? Because this is not a one-time thing, that you have some big revolution and then everybody re-trains and that's it. No, there'll be a cascade of ever more disruptive revolutions. I would say that, again, from a historical perspective, the main problem is adaptation.

• 14:16Speaker 00

  Humans are remarkably adaptable beings, but we adapt on an organic scale. And here we are confronting, again, an inorganic or digital scale revolution, which just moves far, far faster. If we can give humanity time to adapt to the economic and social changes, I think we'll be okay. My fear is that we don't have that time, because the revolution is moving on with a rapidity, which we see nothing like that in history. Most people in places like Silicon Valley,

• 14:57Speaker 00

  when you raise all these fears about people becoming jobless and about political and social disruptions, they tell you, "You know, when the Industrial Revolution began in the late 18th, early 19th century, people had all these fears that all the steam engines and the trains will destroy society and so forth. And look, 200 years later, almost everybody has a much better job than what they did in 1800. Almost everybody have a better quality of life, healthcare, transportation, entertainment. Everything is much better than in 1800. Not that the world is perfect, but if you look at 1800, you look at, say, 2000 or 2025,

• 15:42Speaker 00

  the fears were completely unjustified. My answer as a historian is that in history, the problem is usually not the destination, it's the way there. Because the way from 1800 to 2000, I mean, the problem was that the Industrial Revolution upended all the existing economic, social, political structures. Nobody knew how to build the new industrial societies because there was no precedence anywhere in human history of how do you build an industrial society. And people experimented in different ways. So one big experiment was European imperialism.

• 16:26Speaker 00

  All the countries that led the Industrial Revolution also engaged in imperial conquests overseas or sometimes nearby because the logic was the only way to build a viable industrial society was to build an empire. And this is what the British did. And this is what even small countries like Belgium did when they industrialized and when industrialization spread to other parts of the world. And another big experiment was 20th century totalitarianism. Both communists and fascists were very closely linked. To the Industrial Revolution. You could not build a communist dictatorship in 17th century Russia.

• 17:11Speaker 00

  Impossible. You don't have totalitarian regimes without trains, electricity, radio, all that. And what Lenin and Stalin and Mussolini and Hitler, what they said is that liberal democracies can't handle industrial technology. Industry releases such immense powers of creation and destruction that only a totalitarian system can manage it well. So if, again, if you look at the real process of change from 1800 to 2000, it was a rollercoaster in which hundreds of millions of people paid with immense suffering and sometimes with

• 17:51Speaker 00

  their lives for all these experiments in how to build an industrial society. And humanity got very close to self-destruction with the nuclear bombs after 1945. If humanity is a student in one of my courses, like the course of how to survive the Industrial Revolution, I would give it a C minus. So we survived, but we had a couple of very close calls and we could have done better, really. And my fear is that if this is the analogy, if humanity gets a C minus in the 21st century

• 18:30Speaker 00

  in how to deal with AI, then billions of people will pay a very, very high price for it, and maybe this time a C minus just isn't enough. One of the dangers with AI is that we could see, we are already seeing, a resurgence of both imperialism and totalitarianism, but in a new and potentially even more frightening form.

• 18:55Reid Hoffman?

  So by the way, I completely agree that the technology enables new forms of control, new forms of centralization, enables different forms, like evolved forms of imperialism, empire, et cetera. That I completely agree with. But this actually gets to the thread that I want to do to kind of raise on it, which is, so when we build technology, I think some technologists are naive and they think that some technologies are inherently decentralizing and other technologies are centralizing. I think all technologies inherently centralizing, but we can choose to make it decentralizing. And part of where I think the industrial revolution, because I'm one of these Silicon Valley technologists who uses this metaphor, and I take all of your points seriously, by the way, just to be clear about it.

• 19:39Reid Hoffman?

  But I also think the industrial revolution is what gives the potential ground for a democracy, middle-class society and so forth, because it actually allows for kind of a distributed enfranchisement of things that allow for broad-based education, broad-based wealth and prosperity and so forth. And the industrial revolution gives that as a possibility. But what I also think is an interesting thing of looking forward from the industrial revolution, including what I now refer to AI as the cognitive industrial revolution, that parallel, is that what are the things that we need to do to A, learn to be better than a C-, right, in what we're doing? I do think, like, I actually think a C- is a fair grade for the industrial

• 20:23Reid Hoffman?

  revolution, and I do think that one of the things that we should do, and it's part of the reason why, you know, Santayana and others to engage with history, is to say, let's do it better this time. What would be some of those guideposts?

• 20:36Speaker 00

  First, I agree that the industrial revolution also made modern democracy possible. The same way that totalitarianism was impossible in the ancient world on a large scale, the same is true of democracy. The thing is, democracy is a conversation. To have a conversation, people need to be able to communicate in real time. This was impossible to do under ancient technological conditions, except in a small city or a tribe. So you don't have any example of a large-scale democracy. Millions of people, spread over thousands of kilometers, engage in a real-time discussion of political choices. You start seeing it happen only in the 19th century.

• 21:18Speaker 00

  And it's also true for AI, it can go either way. What do we need to do in order to... Okay, so you experiment in different ways to know how to build society. How do you avoid dystopia and learn over time to build better societies? The key term, I would say, is self-correcting mechanisms. A good system is a system that has an internal mechanism that allows it to identify and correct its own mistakes. This is a key ingredient of democracies. You know, elections, free cause, free media, they are all self-correcting mechanisms,

• 22:02Speaker 00

  enabling the public to identify and correct the mistakes we did before. You start with the assumption that mistakes will be made, and then you bake into the system some mechanism to identify and correct them. And this is also true of biological organisms. The way we survive, all organisms survive, is because of these self-correcting mechanisms. Like a child learns how to walk, not by being instructed by parents and teachers. They can give some encouragement, but basically, it's all self-correction. You get up, you try to walk, you fall down. You learn something for the mistake, you try again, you fall down. Eventually, you learn to walk. So how do we maintain self-correcting mechanisms

• 22:45Speaker 00

  in the 21st century during the AI revolution? Part of the problem is that self-correcting is a relatively slow and cumbersome process. At the pace that AI is developing, one of my fears is that there is just no time for human self-correction. By the time you understand the current AI technology and what are the impact on society and their politics, it has morphed 10 times and you're faced by a completely different situation. We are still struggling with, you know, how to deal with social media algorithms

• 23:27Speaker 00

  and how to deal with the fallout from the social media revolution of 10 and 15 years ago. What is happening right now, nobody really understands. Because again, it takes time just to collect the data and figure out what's happening.

• 23:43Reid Hoffman?

  This actually is an excellent lens focusing on, I think, one of the things that you may hope for as well, but you may have a lower probability on, which is the, can we use technology, specifically AI, to increase the speed of our self-correction mechanisms?

• 24:00Speaker 00

  That's a key question.

• 24:02Reid Hoffman?

  Yes, that is exactly one of the things that I think when, part of what I think I've found through our discussions is we actually parse many of the variables the same way and then we assign some different probability weights. And part of the thing I think in the different probability weights is, my sense is, yes, we have this challenge of accelerated speed and we're not naturally equipped to deal with that accelerating speed. So what do we do in order to do it? And this is part of the reason I say, well, wherever technology presents a challenge, also see if you can use technology to present the solution.

• 24:32Speaker 00

  Yeah, that's, I think, the heart of the disagreement.

• 24:35Reid Hoffman?

  Yes.

• 24:35Speaker 00

  Right there.

• 24:36Reid Hoffman?

  Yes.

• 24:37Speaker 00

  My basic problem is, is if we don't know whether the technology is trustworthy and then we give the task of asserting that it is trustworthy to the technology, we seem to be caught in a kind of loop. Because if I can trust the kind of AI tool to make sure that the other AI tool is okay, then everything is good. But how can I trust the kind of verifying AI tool? That's, I think, is the key problem. And yeah, I mean... At the heart of it is also just the question of time, of this kind of collision between humans who work on this organic time.

• 25:22Speaker 00

  We are extremely fast compared to other organisms, but still we are extremely slow when compared to the inorganic time of AIs. In many of my discussions, also with you, with other people from Silicon Valley, our understanding of time is different. Like, when somebody from Silicon Valley uses the term "a long time", I now understand they are thinking like two years.

• 25:52Aria Finger?

  You're thinking 70,000, they're thinking two.

• 25:56Speaker 00

  So like, I meet people who tell me, "Look, AI has been around for such a long time, and still human society hasn't collapsed." No, it hasn't been around for a long time. It's been around for nothing.

• 26:19Reid Hoffman?

  You know, I thought this even before you did, that global coordination is not actually possible. But what I think still is possible is to actually have groups... This is one of the reasons why I'm kind of a strong believer in multilateralism and at least partial globalism, which is to form alliances. I don't think there's a way that we can get global cooperation to slow down the time clock. Given the time clock is what it is, how do we help build the self-correcting mechanisms that have the best possibility of giving us that adaptation for the better future? And I don't think any of us know, but I think we need to start throwing out the ideas for that so that possibly we can then get it right.

• 27:04Speaker 00

  The most important thing we need is to build trust. And this is different levels, philosophical, practical. One of the reasons that the order of the world is collapsing is that we have a deficit of trust in the world, and it makes us extremely vulnerable to AI. And it also, I think, guarantees...

• 27:30Speaker 00

  A very dangerous kind of AI that will try and probably succeed in taking the world from us. Why? Because you can think about AIs as like, you know, the children of humanity. When you try to educate children, there are the things you say to them and there are the things they observe you actually do. And your behavior has far more influence on their education than what you tell them to do. So if we kind of tell our AIs, "Don't be power hungry. Don't be untrustworthy. Don't cheat. Don't lie. Don't manipulate." And then the AI observes us

• 28:14Speaker 00

  constantly manipulating and cheating and grabbing power. The AI will learn from how we behave. I mean, even if the AI kind of tycoon tells the engineers, "Find some way to engineer into the AI something that will make it trustworthy." If the AI tycoon himself, or in rare cases herself, behaves in an untrustworthy way and if he himself believes that all the world is just a power struggle, there is no way that he can produce a trustworthy AI. No way. Now, the good news is that this entire worldview is not just cynical and dangerous,

• 28:59Speaker 00

  it's also a mistake. It's not true that the only reality is power and that all human interactions are power struggles. Yes, power is an important part of the world. Yes, some human interactions or some part of human interactions, they are power struggles. And yes, some institutions or all institutions, they have problem with corruption. They have problem with people manipulating. But this is not the whole of reality. There are other aspects to human beings. Human beings, I think, are all of them, except if there are some extreme psychopath.

• 29:39Speaker 00

  They are genuinely interested in love, in compassion, in truth. This is not some cynical maneuver to gain power. And what I would tell people who would believe in the cynical, power-hungry worldview is just observe yourself. Is this the way you think about yourself? Do you think that you yourself don't care at all about the truth and you just want power? Probably not. You probably think you're different. Now, if you have a better view of yourself, why aren't you charitable enough to acknowledge that other people, just like you, might also be really interested in knowing the truth?

• 30:22Speaker 00

  Or in having loving relationships with other beings? And this is where we can start to build a more trustworthy philosophy, which is the only stable foundation for a good society and also for benevolent AIs. This is not a guarantee, but there is at least a chance that an AI developed by a society that believes genuinely in the pursuit of truth, in compassionate relations, the AIs will also be more trustworthy and more compassionate and so forth.

• 30:57Aria Finger?

  I was gonna say, you started real dark there. And I was like, we are, there's nothing we can do, we're in this society. And then we ended on a place where, you know, your conception, it seems, of humanity is that we are loving beings and we are compassionate beings.

• 31:14Speaker 00

  This is just a reality. I mean, it's not some kind of fantasy I'm projecting.

• 31:17Aria Finger?

  And like you said, like, it's a truth, it's a fact, that we don't live in a zero-sum society. Like, that's just true. And so, that's actually incredibly hopeful. And so, if AI is just reflecting back humanity, you started saying that, well, why won't we enter on the good path? Why won't AI go down the positive, loving path? Can't that be the future that we see?

• 31:40Speaker 00

  It can, but if, if human society, at the time we develop AI, if human society is dominated by cynical, power-hungry people, this guarantees that the AI developed in that society will also be manipulative and power-hungry and untrustworthy.

• 32:00Reid Hoffman?

  By the way, here, I think this is, in many ways, a spectacular conversation. Because what I think of, when I think about AI learning, I don't think of it necessarily as learning from the most macro geopolitical point of view. It doesn't say it doesn't, it doesn't see it. Just like the child of two parents, you know, kind of living in London, might not see it as well. But it's precisely the reason, like, for example, why I went around and said, okay, I see this issue. How do we grow this tool? What is the process by which that tool is going to evolve? Well, it will be through these labs, through these, the technological builders. Let me go try to influence the labs. Where do you see the parenting level?

• 32:44Speaker 00

  The lab is like the womb.

• 32:46Reid Hoffman?

  Yeah.

• 32:47Speaker 00

  It has a lot of influence, but a big part of education also happens after the child leaves the womb.

• 32:55Reid Hoffman?

  Yes.

• 32:56Speaker 00

  And one thing you cannot do in a laboratory is to simulate history and to simulate the world. So I read about all these kinds of simulations that they try to see how would the AI react in this situation? How would the AI react in that situation? To be aware of dangerous potential. And sometimes they discover quite scary things in the lab. But they will never be able to witness the scariest things in the lab. Because you cannot simulate what happens when billions of people in the real world interact with billions of AI agents. And these are the most dangerous scenarios.

• 33:37Speaker 00

  And again, you can try to kind of somehow build into the AI all kinds of mechanisms that will make it aligned with human values and less likely to cheat or to manipulate. But once that AI is, I mean, for me, a key part of the definition of AI is the ability to learn and change by itself. If a machine is incapable of learning and changing by itself, it's not an AI. So by definition, no matter how you design the AI in the lab, once it's in the real world outside, it could learn and change by itself. And it will learn, at least in the beginning, from human behavior.

• 34:21Speaker 00

  So if you have a society dominated by Elon Musk and Donald Trump, it will learn from Elon Musk and Donald Trump, because this is what it will copy. It will not learn, "Oh, I should do all kinds of things for the American public." No. I mean, what it will see is it's okay to lie if this makes you more powerful. Ah, I get it. This is how you behave in the world. And I think what would be the best thing that we can do right now? If some, I don't know, some group of engineers and tycoons can come together to create a convincing demonstration, that you can take a president or a prime minister,

• 35:04Speaker 00

  and they tell you, "I only have one hour." And you put them for one hour in a room, and they interact with an AI, and they come out thinking, "Holy shit, this is the most frightening thing ever." Then we have a chance to maybe have some global cooperation on this quickly. Part of the problem with AI is that there are very positive scenarios, there are very negative scenarios. I think everybody, almost everybody would agree, there is a dangerous potential. We give it different percentages, but there is a negative potential. Very difficult for people to understand what is really the negative potential and what is the magnitude of the threat.

• 35:48Speaker 00

  If we could kind of focus the mind of especially political leaders around the world by having this demonstration, this will go a long way towards getting something done quickly.

• 36:01Aria Finger?

  I think that's so interesting. You know, recently, Bill Gates said that he was giving the rest of his, you know, fortune in closing the shop in 2045. And he said the thing that made him realize that he wanted to start the Gates Foundation and do all of this was that he went to Sub-Saharan Africa and he went to Soweto. And so an intellectual person can read about children dying in Africa all day long, but they're not affected by it until they go and they see it. So I think it's really interesting that you're saying, "We need people to see it. We need people to understand it." And so then what would you have our political leaders do? What would this global cooperation look like in order to create this positive future?

• 36:42Speaker 00

  To maybe not stop, but at least slow down the arms race. As I travel around the world, I talk with many of the leaders of the AI revolution. And almost all of them agree that there is a dangerous potential. And that it would be a good idea to invest more in safety, to take things a bit more slowly, to give humanity more time to adapt and to think it through. But they all say they can't slow down because they are afraid of their competitors. And then you have this kind of built in really paradox at the heart of the AI revolution. You have the same people telling you that they can't trust other humans,

• 37:27Speaker 00

  but that they think they could trust the AIs they are developing. Because again, when you ask them, can you slow down a little? They say, no, because we can't trust the other humans. When you ask them, okay, but do you think you could trust the super intelligent AIs you're developing? And they say, yes. This is insane. I mean, again, we have many reasons to suspect other humans, but we also have some good reasons to trust them because we have thousands of years of experience with them. We just don't have much experience, any experience, with millions of super intelligent AIs running around. They have a very cynical view of the other humans, and they have a very trusting view of the AIs.

• 38:10Speaker 00

  And this doesn't make so much sense.

• 38:13Reid Hoffman?

  I think one of the reasons to defend the perspective that you might be able to make AI more trustworthy than human beings is that to some degree, when you're creating it, and yes, it learns on its own. I completely agree with the definition of what the AI revolution is, which is it's a self-learning machine. But just like all self-learning, the learning algorithms, the initial learning, the path that you set it from is actually, in fact, part of that precondition as to where the path is of the learning machine. If you set the learning machine to be very emphatic on truth.

• 38:55Reid Hoffman?

  Now, one of the truths it will learn is that human beings can operate in a way that they're operating on power even when they claim that they're not, et cetera. And it will learn all those truths as part of this. But if you set it as I am on the path of truth-seeking. Part of the reason why I maintain a stance of optimism and possibility here is because I think that the seeking of elevation of consciousness, and I'm going to come back to consciousness now, where we started, is that I have the hope and the aspiration that that is the truth that you could actually set a learning clock to.

• 39:33Speaker 00

  But notice the big difference. I mean, AI is artificial intelligence, not artificial consciousness.

• 39:39Reid Hoffman?

  Yes.

• 39:39Speaker 00

  I think the truth-seeking part of human beings comes from consciousness, not from intelligence. People are over-fixated on intelligence, and intelligence is no guarantee in this respect. I mean, humans are the most intelligent animals on the planet. They are also the most delusional entities on the planet. A superintelligence is likely to be super-delusional as well. Nothing in human history indicates that intelligence is a good antidote to delusion or necessarily puts you on a path towards the truth. I think that the impulse towards truth comes really from consciousness. There is no evidence that AI has consciousness, and I'm agnostic about it.

• 40:24Speaker 00

  I don't know, maybe it will develop. Maybe as we speak, the first conscious AI is being born somewhere. But so far, I have seen no convincing evidence that it is conscious, that it's capable of suffering. And one of, I think, the big delusions of places like Silicon Valley is this overemphasis on intelligence, which is partly because there are extremely intelligent people there who kind of, their life is built on intelligence, and they tend to overvalue intelligence. My kind of historical instinct is that a super-intelligent AI will, no matter what you do, if it lacks consciousness, it will not pursue the truth.

• 41:09Speaker 00

  It will very quickly start to pursue other things, which will be shaped by delusions of various types.

• 41:17Reid Hoffman?

  So there's two points there. One, I agree with you. High intelligence can also be a high delusion. But I do think that we might be able to set even just intelligence on more of a truth path. I do think it's interesting because I had not considered... The possibility that the truth-seeking is necessarily oriented in consciousness. I actually think that truth-seeking is possible with intelligence, but that's

• 41:42Speaker 00

  It's only intelligence.

• 41:43Reid Hoffman?

  With only intelligence, and I think that's a thread for future conversation. I think one thing that I also have learned in this conversation is the importance of rebuilding trust. For example, one of the things that I have the same deep concern from both the far-right and the far-left and the populist is by being anti-trust, by being only power, by tearing down institutions, you're tearing down the very society that has gotten us to our most elevated golden age, and it's between horrifically negligent, right, and deeply bad, right? And so rebuilding trust is really important.

• 42:25Reid Hoffman?

  One of the things I've actually kind of doing with Lever for Change is, you know, how do we, you know, as a challenge for how do we rebuild trust in institutions? Because I think it's important, how we might use technology, inclusive of AI, for possibly trying to rebuild trust.

• 42:42Speaker 00

  Yeah, I think this is a very important direction to explore. We see initiatives in different parts of the world, you know, like the police system in Taiwan, using kind of social media tools, not to divide the public and to spread conspiracy theories and distrust, but just the opposite, just by tweaking the algorithm, just by telling the algorithm to score what people say in a different way than the usual algorithms on Facebook or Twitter. I mean, the most social media algorithms, they are only, they only care about engagement. If something gained a lot of attention, they push it up. And what gets a lot of attention? Outrage.

• 43:26Speaker 00

  And this is how distrust and hate and anger spread. So this system developed in Taiwan, it does something slightly different. It scores content, I mean, first of all, it kind of maps people into kind of groups, what kind of content they usually like, and then it scores content on how many likes it gets from people in a different group. So if you only cater for the people in your group and they give you a lot of likes, you don't go up. You need to say something. You need to post a video. You need to do something that will get likes from people on the other side.

• 44:06Speaker 00

  And immediately, very quickly, all the influencers and all the celebrities and whatever, they start saying things in consensus. Because they realize by trial and error, this is the only way I can get my content up. And of course, it has its own downsides. People say, oh, it builds conformism, whatever. But this is just a small example of how by a very small and simple, seemingly simple, engineering tweak, you can turn a technology from something that destroys trust into something that builds trust.

• 44:50Reid Hoffman?

  All right, so we will move to rapid fire. Is there a movie, song, or book that fills you with optimism for the future?

• 44:58Speaker 00

  Maybe the Netflix series Heartstopper, which is, you know, this very simple story about two boys in high school falling in love. And it's just the most romantic and simple love story ever. There are no complications, no big tragedies. And this was just unimaginable when I grew up in the 1980s. And this in recent, like last two, three years, it's been one of the most popular TV series, at least for a younger audience in much of the world.

• 45:29Reid Hoffman?

  A demonstration that love is still central to the human soul.

• 45:33Speaker 00

  Yeah, absolutely.

• 45:35Aria Finger?

  I love that. So I could ask you questions all day, but what is a question that you wish people asked you more often?

• 45:43Speaker 00

  Which institutions you trust and why? I mean, I think people... Very often, they like to talk about all the things that go wrong and all the things they don't trust. And actually, we trust most things. We just don't think about it. We trust the sewage system. We trust the electricity. We go on an airplane. We usually, we can then afterwards complain, oh, it was late and it was like this. But we're sitting there in the air and we usually trust it.

• 46:13Aria Finger?

  Absolutely. The boring functioning of our governments is pretty good most of the time.

• 46:18Speaker 00

  And science and.

• 46:20Aria Finger?

  Exactly, right.

• 46:21Reid Hoffman?

  This will be a little, I'll phrase it the exact way that we normally ask it, but I think it needs a refinement for you, which is, where do you see progress or momentum outside of your industry that inspires you?

• 46:32Speaker 00

  Hmm, what is my industry?

• 46:34Reid Hoffman?

  Yes, exactly. Precisely you get for the modification of the question.

• 46:38Aria Finger?

  Forget about human history. What inspires you?

• 46:41Speaker 00

  The cockroaches are doing wonderful lately.

• 46:44Reid Hoffman?

  They will survive.

• 46:45Speaker 00

  They will survive. They are very hardy, very trustworthy.

• 46:48Reid Hoffman?

  Yes. Maybe it's, where do you see the elements for the possibility of rebuilding of trust somewhere in society?

• 46:56Speaker 00

  In every human being. I mean, again, I think that when we look at ourselves, we usually have a bit of a more compassionate and charitable view of humanity than when we look at our political opponents or religious rivals or whatever. So this is why I think meditation is so powerful, that if you really get to know your own mind and you have this kind of inkling that actually this is not just my mind, the mind of other people works more or less the same. This is a source for a lot of hope.

• 47:31Aria Finger?

  All right, well, our final question. Can you leave us with a final thought for what you think is possible? If everything breaks humanity's way in the next 15 years, and what's the first step to get there?

• 47:42Speaker 00

  So and the way to get there is first of all to rebuild trust, both within societies, also between societies. And if we can do that, and it's certainly possible, then we now have the resources to build the best society that ever existed in history. For most of history, people struggled against problems that they just didn't have the resources to overcome. Like if you live in a medieval kingdom and the Black Death comes and kills within a year, between a third and half of the population, you are completely helpless. You don't have the scientific knowledge. You don't have the technological

• 48:26Speaker 00

  and the government infrastructure to deal with a pandemic. So you can pray, which is what people did, but it's really beyond, it was beyond the human capacity. And similarly, every few years, you would have this massive famine because there would be a flood or a drought and the fields don't produce enough wheat. And it's too costly to import wheat from halfway around the world. So people starve to death. These were the big problems of people in the Middle Ages. We know how to deal with them. And we are not perfect, but compared to every previous time in history, we are doing much, much better because we have the resources. And similarly, with the new problems we face,

• 49:11Speaker 00

  whether it's nuclear war, whether it's climate change, whether it's the AI revolution, this is not a natural disaster beyond our capacity to understand and to mitigate. We have the understanding, we have the resources. What we need is only the motivation and the trust. Fantastic.

• 49:31Reid Hoffman?

  Amazing as always. You will all thank you.

• 49:33Speaker 00

  Thank you so much.

• 49:39Reid Hoffman?

  Possible is produced by Wonder Media Network. It's hosted by Aria Finger and me, Reid Hoffman. Our showrunner is Sean Young. Possible is produced by Katie Sanders, Edie Allard, Sarah Schlied, Vanessa Handy, Aliyah Yates, Paloma Moreno Jimenez, and Malia Agudelo. Jenny Kaplan is our executive producer and editor.

• 50:02Aria Finger?

  Special thanks to Surya Yalamanchili, Sayida Sepieva, Fanasi Dilos, Ian Alice, Greg Beato, Parth Patel, and Ben Relis. And a big thanks to Michael Zor, Dima Basav, Brooke Ann Shudders, Melis Uslu, and The Burke House.