Reid riffs on if AI can become conscious

This episode has been re-uploaded to correct an error. If you are a subscriber and/or downloaded it on its release, consider deleting and redownloading it to hear the corrected audio. We apologize for the inconvenience. On this week's Riffs, Reid and Aria unpack last week’s conversation with Yuval Noah Harari, discussing AI's rapid advancements, its potential impacts on humanity, and the meaning of consciousness. Reid and Aria also discuss who will be responsible for making sure AI is aligned with human values –– no matter what the future looks like. 00:00 Introduction and overview 00:16 Reflecting on the conversation with Yuval 05:14 The role of AI labs and accountability 11:45 Philosophical questions on AI and consciousness For more info on the podcast and transcripts of all the episodes, visit https://www.possible.fm/podcast/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Aria Finger?

  Hey everyone, so excited for our discussion today. We are going to be responding and talking about our conversation with Yuval Harari. So if you have not listened to our Yuval Harari episode from last week, go listen to that first. That will make this make a lot more sense. And here we go. Hey Reed, it is great to be with you here today.

• 0:19Speaker 01

  Always great to be here.

• 0:21Aria Finger?

  There's obviously a lot of agreement between you and Yuval about how you see the world, about your care for humanity. You want to make sure to be shooting in the right direction. But there was also plenty of disagreement, sort of both philosophical and also like what exactly is going to happen. So I'd love to hear your reflections. Tell me about the conversation and where did you, where did you disagree?

• 0:43Speaker 01

  Well, Yuval, obviously, always an amazing delight to talk to and one of the

• 0:51Speaker 01

  leading public intellectuals of our time and a very clear headed thinker. I mean, part of what he tries to do is say, look, I'm, I'm, I'm seeing the world as clearly and without illusion or self delusion as possible. And I report what I see. For him, he looks at the, you know, the kind of the exponential or the massive acceleration. I'm not even sure, like the word exponential is frequently overused in this context of increasing capability. And, and he goes, well, it's, it's going to be a new species. We don't know what the

• 1:34Speaker 01

  new species is. We know that we is, is, is Homo sapiens. When we were a new species, we remade the planet. Why shouldn't we just infer that it's going to remake the planet? And, you know, boy, I hope there's, there's a role for me, you know, kind of continuing to, you know, be a meditative kind of human being in this, in this mixture. And, um, you know, I think that it's actually in fact going to,

• 2:04Speaker 01

  you know, remake lots of things, gonna remake industry, is cognitive industrial revolution, gonna remake, where, you know, kind of what the future of human activity is. And I think it's a much higher probability that this elongates the kind of, the robustness, the capabilities of Homo sapiens, it's kind of like saying, you know, again, in a simple metaphorical sense, you know, is it, is, is it more likely to be creating,

• 2:34Speaker 01

  you know, kind of drug resistant, or sorry, uh, new forms of, of antibiotics that will help against drug resistant tuberculosis? Or is it more likely to be creating the tuberculosis? And I think it's more likely to be creating the, the therapeutics uh, to, to tuberculosis. So it's kind of, um, so there's a kind of a probability set. Now then, when you get into kind of details, he says, well, but one of the problems is, is the learning machine. And if you're learning from, um, you know, kind of human beings, look at all this bad stuff we do, you know, look, we're, we're cruel. You know, we torture people, we engage in war, we're lying and self deceptive and, you know, da, da, da, da.

• 3:19Speaker 01

  Isn't just gonna, just like a child of ours, isn't just gonna learn, you know, because it isn't just what we say, it's like, what is what we do? And, you know, I think obviously there is, you know, kind of definitely some truth to that. But also, by the way, um, just as when we teaching, we're, we're teaching children ourselves, even though we have some faults, we actually try to direct, you know, children towards our more aspirational selves, the fact that we're compassionate, the fact that we're wise, the fact that we have these kind of uh, humanism values, these humanity values, it's kind of, you know, kind of what's in depth encoded in religions.

• 4:00Speaker 01

  Uh, and where religions kind of share a context of, you know, how do you, you know, aspire to being our better selves? And and you know that, you know, and this is the half full and half empty. I go, well, actually, in fact, I think we can shape it that way. I think we do that with our children as it is. It's part of what we try to make progress in society. And I think that's the the thing that we can potentially drive to.

• 4:28Aria Finger?

  But I also think to your point, a lot of it, of course, is is optimism and is let's figure out not to complain, let's figure out actually action so we can get to a better place. But some of it is just recognition of some of the bad things that actually can happen. Like you said, sort of the tuberculosis versus the antibiotics for tuberculosis, the same people often who are gloomers or saying this is going down a bad route would probably also tell you that climate change is going to do irreparable harm, or is going to say that we're unprepared for the next pandemic. And so you're not just saying that, you know, AI is great compared to some utopia. You're saying AI is great compared to the world we have now, because we have real challenges and AI

• 5:10Aria Finger?

  like has to be here to help solve them. So a lot of people are concerned. It's like, oh, you have these five AI labs who are just, you know, they're deciding the future of the world. Like, first of all, how do we make sure that they have the right values? Like, is there anything we can do to affect what they're doing? And then also, like, who else needs to be involved in this? Like, who else needs to be a part of this decision making?

• 5:35Speaker 01

  Well, a couple of things. One is, you know, and this causes existential risk people some heartburn. It's probably five labs heading to 15, heading to 30, or at least 10 to 15 at minimum.

• 5:50Aria Finger?

  And why is that good? For people who get heartburn, why do you think that's actually a great outcome for the AI space?

• 5:57Speaker 01

  Well, it's a complicating outcome for the AI space. One of the places where I tend to find, you know, kind of.

• 6:07Speaker 01

  Many critics to be kind of, you know, really yelling about like, well, I should be in charge and it should be me. And it's like, well, not clear, right? Even though you go, well, I don't want just the large tech companies and just the heads of these labs to be in charge. And you're like, well, but some people will be like, there will be a limit like take there will be a limited set of people who will be in charge. And then you go, given that limited set, what's the, what's the, the reasonability of which limited set? And it's like, well, you know, folks who are, you know, kind of investing and building this stuff and have the capabilities of doing it and, and are, are putting their own lives, missions and their economics into

• 6:49Speaker 01

  this and who are held accountable by various networks of accountability, which includes critics, includes press, includes government, includes customers, includes shareholders, includes, you know, family and, and, you know, community members, includes teammates, you know, there's, there's many, many different things. And we try to make those channels of accountability to be as healthy and as inclusive as possible. Like the people who say, it's only these five, and what gives them the right? It's like, well, actually, in fact, it's, it's a growing number. So if you're, if your particular thing is, is that, you know, what do we, what do we think about these five? You know, it's like, well, but it's not only going to be these five. And a little bit of that's a kind of a gesture of the people who get on the, you know, kind

• 7:32Speaker 01

  of blindly or for their own self, their own, their own reason, they get on the anti-trust horse. And so they tend to go, our really important thing is to limit these five. And it's like, well, it's only really important to limit these five. If you think these are only, these, these are the, the, the laws of physics say it's only these five. And it's actually not the case. Cause you know, since, um, since I've been doing the kind of Western democracy things, both there've been new entrance on the Western democracy side, and there's at least three to five that we're tracking in China. And it's probably actually, in fact, you know, 10 to 20 in China. So you've got, you know, this, this, this swarm of it.

• 8:17Speaker 01

  So you've got a larger number of people. So in as much as it's people

• 8:24Speaker 01

  with different points of view, you know, kind of playing for this.

• 8:30Speaker 01

  That answers the, or that, that gestures at the, you know, that there's a number of people working on this now.

• 8:36Aria Finger?

  This is not a monopoly question.

• 8:38Speaker 01

  Yes, not a monopoly question, but it's because other people heartburn because they go, well, actually, in fact, these ten to twenty people are going to compete. That as opposed to getting to our highest virtue selves, it'll be because of competition and because of divergence between people. We're going to have all, like, you know, for example, we'll have AI weapons created and, and why won't someone, you know, either deliberately or accidentally of this much larger set? You know, you know, be targeting, creating kind of the equivalent of the Terminator. So, you know, I think that the question is, again, is to say, look, if we presume that, that we're kind of, we, the, the, the fundamental thing about human beings is we divide into groups.

• 9:21Speaker 01

  We have different perspectives about what the risks are, but what the important things to achieve are. And we compete with each other and part of our competition is, is manifesting those values. The thing that we must do as thoughtful human beings who are trying to be, um, you know, great for humanity is we say, all right, how do we help? Those efforts that, um, you know, care about the same kind of, you know, perspective of, you know, call being compassionate towards human beings of being, you know, kind of elevating to our better selves.

• 9:57Speaker 01

  Of having a human future that, that is, that is the continued evolution that we've had over centuries and, and millennia. Um, you know, and a very broad brush to continue to do that, to be, um, you know, kind of, you know, the virtues of kind of wisdom and empathy and, and, and other kinds of things. And how do we, how do we do that? And the answer is make those, those, those projects that are, are deeply trying to embed those values, embed the values in what they're, in, in the AI systems they're creating, in the products that they're deploying. And to make those, you know, kind of the more accelerated, you know, efforts. Um, and as you know, that's of course what I, what I've been doing.

• 10:43Speaker 01

  Look, there is a possibility that we're creating creatures, we're creating new entities and, and it's mind bending. But it doesn't mean it's a certainty. And it doesn't mean that the way that we should steer is is well, because that's a possibility. We should steer about that being the certainty. We actually have to be iterating and seeing this is completely possible. That, you know, X years from now, we'll say, oh, just like when in the eighties, we were talking about this with AI. And yes, this is a much better technology. Yes, it's it's it's achieved so much more. Yes, it's on a better curve. But just when we're talking about that and we said, OK, like that will completely change. It's like, we may very easily discover in as as few as a few years.

• 11:24Speaker 01

  And maybe even five to 10 is like, well, actually, in fact, these aren't really entities. They're actually tools in the following way, and they have the following shape. And this is the way we're integrated in. And so it's a it's a discovery of of what's possible. Because not everything is possible at every moment.

• 11:41Aria Finger?

  And another thing I appreciated about the conversation, and this is probably true of any conversation where you and Yuval are in the same room. Uh, is that we get to talk about more more philosophy, more philosophical questions about sort of what it means to be human. And, you know, he he had an argument that was focused on the critical distinction between intelligence and consciousness. And he said that while AI might become vastly more intelligent than humans. Intelligence alone doesn't guarantee a pursuit of truth. Or even the capacity to reject falsehood. And I know truth seeking is something you talk about a lot. It's like, how do we get to this place? How do we teach truth seeking behavior? What are the ways that we can model that world?

• 12:23Aria Finger?

  And so I'll ask you, is consciousness a necessary foundation for this truth seeking? And if so, can this non conscious AI like ever really be truly aligned with our deepest values? Or are we missing something fundamental with asking them to define truth when they don't have the consciousness piece? Or at least not yet?

• 12:45Speaker 01

  You know, I haven't had a chance to have this conversation with Yuval. So if Yuval is listening to this, this is the next move in the conversation. Is, um, what I've realized is, I think. Where it's helpful to think about is, well, what kinds of truth seeking are necessary from from conscious entities? And how does that illuminate what we think about consciousness? And what kinds of truth seeking are doable without consciousness, with just intelligence or fitness functions? Because you clearly have a set of truth functions.

• 13:18Speaker 01

  That are where consciousness is not necessary. Like it it just like, for example, you can you can train in in, you know, lots of different systems, even, you know, less sophisticated than AI systems to actually, in fact, you know, run a truth-seeking process. Um, I mean, you know, heck, when we have, uh, deep research in, you know, ChatGPT and Copilot and Gemini and other and Claude and others, it it actually says, go cross-check your work, right? Like, go pull out documents and cross-check your thing. And then when two things disagree, go look for other information and and privilege these kinds of sources of information, just the same kind of cross-checking we do

• 13:59Speaker 01

  in our group truth-seeking, whether it's science or or judicial processes or, you know, kind of academic work or else it's kind of that process for journalism to to kind of do truth-seeking. Clearly, you can do a lot of that with just intelligence. So then you get to this interesting question around, um, you know, like, obviously, if you had an enlightened being that said, I am in touch with how difficult suffering is. And I view the importance is to kind of reduce suffering across,

• 14:41Speaker 01

  you know, um, uh, more things other than just me and to have kind of quality, kind of sentient life is doing that, then that's a very good thing. Is consciousness necessary for some component of that thing? Is the fact that we experience reflection, meditation, potential empathy, compassion, sympathy with each other through the recognition of that consciousness, is that an essential component? And if that's an essential component, can we keep it essential in various ways for how we operate what the future of the world is? And I think that that's, um, uh, that's a research, that that's a that's an ongoing discovery question as we as we get there.

• 15:26Speaker 01

  And I'm still thinking about it. Now, I would say in kind of a final close this is we clearly. Can demonstrate better and better alignment with human values. Like, one of the things that, again, if I were to make the argument out of evidence for optimism versus just a, I'm hoping for the best, is when you look at kind of the evolution of the OpenAI systems of GPT-2 and 3 and 4 and 4.5, they've much, as they get more sophisticated, they much more naturally and easily align with a set of human considerations.

• 16:06Speaker 01

  They much more naturally understand, you know, kind of what potentially our better selves are, what those things are, and they actually do actually, in fact, show better ability to go, like, I have some level of understanding, comprehension of, of what the kind of human goal set is in the aspirational side. I've, I've learned and been trained to preference, like, oh, you're trying to figure out how to write poetry, or you're trying to figure out how to, how to have a productive conversation with your friend, your child, your spouse, you know, and I can help with that. And then, when you say, hey, I'd like to, you, you know,

• 16:49Speaker 01

  to make a bomb, it says, no, I'm not gonna help you with that.

• 16:53Aria Finger?

  Right, right.

• 16:54Speaker 01

  And it naturally, you know, kind of aligns better in those ways. Doesn't mean there's a ton of work, doesn't mean there isn't risk, but that's, that's like a positive, where we're getting that alignment, even though I think it's, I think it's only the, you know, kind of the very extreme crazy fringe that think these systems are conscious today.

• 17:16Aria Finger?

  Right.

• 17:17Speaker 01

  And there are people who think that they are conscious today.

• 17:19Aria Finger?

  Yep.

• 17:20Speaker 01

  I think there's a whole bunch of good arguments as to why they're not, but, and will they ever be? That's...

• 17:29Speaker 01

  One more mind bending question.

• 17:31Aria Finger?

  Well, I cannot wait for our next conversation with you all to tease into a lot more of these ideas. Reed, thank you so much.

• 17:39Speaker 01

  Likewise.

• 17:40Reid Hoffman?

  Possible is produced by Wonder Media Network. It's hosted by Aria Finger and me, Reid Hoffman. Our showrunner is Sean Young. Possible is produced by Kitty Sanders, Edie Allard, Sarah Schleid, Vanessa Handy, Elia Yates, Paloma Moreno Jimenez, and Malia Agudelo. Jenny Kaplan is our executive producer and editor.

• 18:02Aria Finger?

  Special thanks to Suria Yala Manchili, Sayida Sepieva, Fanasi Dilos, Ian Alice, Greg Beato, Parth Patel, and Ben Relis.