Robert Wright: AI Alignment Is a Moral Problem, Not a Technical One

The real AI alignment problem is not technical. It is moral. You cannot align a superintelligence to human values if humans cannot agree on what those values are, and you cannot get that agreement without a level of global cooperation we have never managed before. Subscribe if you want science with evidence, not speculation. The Tower of Babel is the AI parable nobody is reading. Technology that outpaces wisdom fragments the people building it. The Fermi paradox offers the grimmer version: civilizations that reach the capacity to blow themselves up typically do. Robert Wright has been watching this longer than almost anyone, he was reporting on neural networks in 1983, and his conclusion is not that the machines will fail the test. It is that we will. Wright’s book argues that superintelligence is not the danger and AI alignment is not the solution unless it comes with a moral advance. Passing the God test requires something no lab can ship. What you’ll hear: -Why Wright thinks p(doom) fails as a scientific concept -What the binding of Isaac and the Tower of Babel have in common as moral tests -Why global cooperation is the only thing that can actually slow AI down -What Lee Smolin’s cosmological natural selection has to do with the Fermi paradox -What Wright thinks Eastern and Western enlightenment have in common and why it matters now -Whether there is a role for humanity inside the global brain or only for the machines “We face the kind of test that God would give.” — Robert Wright CHAPTERS 00:00 The man who interviewed Hinton in 1983 05:04 The God test and its two meanings 08:36 Religious faith in technological progress 14:10 Hinton then versus Hinton now 19:38 Why slowing AI down requires international cooperation 26:02 How LLMs reverse-engineer the meaning of words 32:08 Brian's Einstein test: can Claude have a happy thought? 40:06 AGI is impossible. Here is why Brian thinks that. 47:04 The Fermi paradox as a moral filter 51:10 What passing the God test actually looks like Get the transcript, fascinating bonus content, and my Monday M. A. G. I. C. Message: https://briankeating.com/yt Have a .edu email and live in the USA? You automatically win a meteorite: https://BrianKeating.com/edu Enter the debate: briankeating.com/ai Subscribe: https://www.youtube.com/DrBrianKeating?sub_confirmation=1 Support Into the Impossible on Patreon, get my weekly M. A. G. I. C. Message, unfiltered bonus content, and live monthly Office Hours with me: https://www.patreon.com/drbriankeating Join this channel for perks, monthly Office Hours, and your name in the Member Roster at the end of every episode: https://www.youtube.com/channel/UCmXH_moPhfkqCk6S3b9RWuw/join Featured Guest: The God Test (book): https://thegodtest.net Robert Wright on Twitter/X: https://x.com/RWrighter My books: Losing the Nobel Prize (memoir): http://amzn.to/2sa5UpA Think Like a Nobel Prize Winner: https://a.co/d/03ezQFu Focus Like a Nobel Prize Winner: https://a.co/d/hi50U9U Galileo’s Dialogue (first-ever audiobook): https://a.co/d/iZPi9Un Twitter/X: https://x.com/BrianKeating Substack: https://briankeating.substack.com Blog: https://briankeating.com/blog Audio-only: https://briankeating.com/podcast #RobertWright #AIalignment #superintelligence #FermiParadox #briankeating #intotheimpossible #AI Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com. Not authorized by any candidate or candidate's committee.

• 0:30Speaker 01

  Imagine building an app with your name on it. With Base44, describe what you want and watch it take shape. You can plan, customize, and expand in any direction. From passion projects to enterprise-grade systems, everything is built in and ready to launch. No matter what you're building, any builder needs Base44. Go to base44.com.

• 0:59Speaker 02

  You feel sure everything's gonna work out fine. I don't really understand how you can be confident of that unless you have something like religious faith in technological progress.

• 1:09Brian Keating?

  Natural selection is blind.

• 1:11Speaker 02

  The mutations in natural selection are blind, but the selection criteria are not. One of the grimmer readings on the Fermi paradox is the reason you don't see many civilizations that have gotten past the point we are is that typically, when they get to this point where they have the capacity to blow themselves up, they blow themselves up. I think we face the kind of test that God would give.

• 1:33Brian Keating?

  My guest today is Robert Wright, one of the OGs of AI. He was interviewing Geoffrey Hinton about neural networks in 1983, two years before Sam Altman was even born. And that was long before anyone was terrified of them. Robert thinks that AI is putting humanity through a test and that passing it takes something no lab can ship, a moral advance. The Tower of Babel story is a truly cautionary tale, and it's a tale of, you know, kind of technology gone awry, technology wanting to compete with God, to fight with God, to make a name for oneself. And you see a lot of that. Where do you come down on this? That the fundamental aspect of the God test is, at least as I read it, but now I'm talking to the authors,

• 2:17Brian Keating?

  is are we gods but for the wisdom? Are we able to pass this test? And maybe you should define the test, where we can prove maybe to ourselves in the absence of a God that we're worthy of God-like technology?

• 2:29Speaker 02

  The title has at least two meanings. One is, as you know, some people think we're building a superintelligence that will ultimately, in some sense, maybe qualify for a term like God. It'll be all-powerful and maybe live in harmony with it and, according to some doomers, maybe get annihilated by it. But in any event, we'll be talking about a kind of, from a, you know, relative to us, an all-powerful and, you know, omniscient in some sense being. And so, you know, one test is, if that's gonna happen, can you build a good God? A God that is compatible with human values and would safeguard human flourishing and so on? There's that. And although I'm not sure we're building a superintelligence,

• 3:14Speaker 02

  I certainly don't rule it out. And I explain why in the book. But there's another sense of the God test that, in a way, is related to the Tower of Babel, because the Tower of Babel is a story of hubris. Who are we to build a God? On the other hand, the doomers say, even if we don't try, it's taking shape. We're gonna have to try to stop it to keep it from taking shape. But in any event, the Tower of Babel is also a story of how humanity became divided, how they came to speak different languages and became, in a certain sense, kind of inherently tribalistic, at least at a linguistic level. And one argument I make in the book is that if we're gonna do a good job

• 3:59Speaker 02

  of navigating the AI revolution and wind up, leaving aside the question of whether it culminates in superintelligence, if we're gonna reconcile it with continued human flourishing and welfare, I do think we're gonna have to approach it as a true global community. I think there are a lot of policy issues that can only be handled at an international level. And there's more and more recognition of that. Only, just in recent months, we've seen that. I think the end of the story of the Tower, you know, of the Tower of Babel needs to be that we all kind of get back together and navigate this challenge collectively, that is to say, human beings. And so, in that sense, the God test is

• 4:42Speaker 02

  to overcome our tribal differences, to get better at understanding the perspectives on people on the other side of walls and so on, and like, quit having wars. That would require something that I think would qualify as a moral advance. And I think it's gonna be necessary if we're gonna survive this in good shape. So in a way, that is a test we're being subjected to, and that is the kind of test that God would give. At the end of the book, I quote the Hebrew Bible, or as we used to say in Southern Baptist Church, the Old Testament, salvation is at hand. And, you know, as I say, there are a lot of stories in a lot of religious traditions where salvation is possible, but you are going, in some sense, have to pass some test. It may vary by religion.

• 5:26Speaker 02

  In the Christian tradition, salvation, the test has to do with accepting Jesus as the Savior who died for your sins. But very often, in one sense or another, it's a moral test. And for salvation to happen, and by the way, I would also say that salvation is often salvation at the societal level, including in the Hebrew Bible. Again, Christian salvation is often construed more as an individual thing. But in many traditions, it is a question of kind of saving the social system. And I'm saying I think we face, leaving aside the question of whether there is a divine purpose unfolding, I think we face the kind of test that God would give.

• 6:03Brian Keating?

  There's so much to unpack there. The classic in the Torah, the Old Testament, the classic paradigm for a test. But the first test that's really, you know, actually called a test or thought of as a test comes to Abraham. You know, he originally has a different name, Abram, and then his name gets changed to, you know, Avraham, which means "Father of Nation." Then he goes from being an individual to a collective, as you just said. And the kind of crucible, literally, by which that's thrust upon him, is a series of tests. In the Torah tradition, there's 10 tests, and it begins with his father throwing him in an oven and breaking the idols that he worshipped. And that's where I want to start.

• 6:46Brian Keating?

  You know, there is so much with these new models and so much personification. You probably saw this Dwarakesh Patel article that got tens of millions of views on X and elsewhere, where he talks about civilizations living and dying and raising and rising and falling. It really seemed so hyperbolic to me. And in my conception, I've talked to all the doomers, and there's way more doomers than there are boomers, as you know. The only public, you know, boomer that I really know is Jan LeCoon, and I had him on. And he's extremely, as you know, and you quote him in the book, he's extremely, you know, technical in his optimism. He's not, you know, kind of denying the sort of societal upheaval, but he's saying, you know,

• 7:29Brian Keating?

  "We're not there technically," is the impression that I got, "and we probably will never get there." And I want to talk about that. But starting with the test of, you know, kind of, is this an idol? You know, are we worshiping these things? When you hear these laboratories, "We had to, you know, put it in a sandbox. We had to stovepipe it. We had to cut it off because it was getting too dangerous and it was escaping, and it was creating civilizations," according to Dwarakesh. What do you make of this? We live, whether or not we personally individually are or not, we live in a secular age. And the secular age has never been more enhanced by technology than it is now. So do you make anything of these comparisons between both the idolatry of AI and the eschatology of AI? In other words, the doomerism is intimately connected, I think, with the idolatry. So what do you think?

• 8:13Brian Keating?

  Are we making these things into gods to worship or to propitiate?

• 8:18Speaker 02

  Interesting question. I mean, the Dwarakesh thing was about a bunch of agents in this, you know, famous OpenAI hugging face incident. And he was comparing generations of agents, I guess, to civilizations. I'm not too bothered by that kind of anthropomorphism because, as I argue in the book, I mean, we are kind of... Trying to, and in some cases, doing this without trying, but to inculcate in AI human tendencies, you know, there are a lot of things, a lot of ways, we want them to be like people. We want them to be pretty autonomous when they seek goals. We want them to do this, we want them to do that. And, and I have a separate argument about what seems to be going

• 8:58Speaker 02

  on inside them and how like the human mind that may be. So I'm not terribly bothered by that kind of language. But as for the worship thing, I think you could probably accuse either extreme of that, which is to say the doomers or the accelerationists. I mean, you've articulated the case for looking at certain kinds of doomers as engaging in idolatry, although some of them would, you know, argue that they're certainly not religion. I mean, Eliezer Yudkowsky, as you may know, I don't know how much you know about him.

• 9:31Brian Keating?

  I had his co-author Nate Sorez on.

• 9:33Speaker 02

  He's kind of the doomer in chief. He turned his back on religion. I believe he was raised by orthodox Jewish parents, but his religion, you might say, became science fiction. It remains his religion even. I mean, some might make that accusation that he's uncritically accepting certain sci-fi narratives. Now, in the book, I argue that they're harder to dismiss than you might imagine. And I'm not dismissive of them. But as for the other side, the kind of accelerationists, it does seem to me that if you, A, believe that this technology is powerful, as many of them do, and is going to get much more powerful and is doing that right now, and you feel sure everything's going to work out fine, I don't

• 10:18Speaker 02

  really understand how you can be confident of that unless you have something like religious faith in technological progress, you know? I started thinking this a long time ago when I first learned about the term singularity, right? The idea that there's going to be this acceleration of technology, which, yeah, seems to be the case, and that it's going to bring some transformation that we can't predict. Look, you're a physicist, right? So the singularity, in the rigorous sense, denotes such a fundamental change that your old kind of analytical tools leave, lose their validity. And in the case of a black hole, you've got an event horizon, so the light itself can't escape, it's completely

• 10:59Speaker 02

  opaque. And, and that is, I think, the basis, that's part of the metaphor of the singularity. And I just thought, well, if you don't know what's on the other side, why are you so sure it's going to be great? And I've always thought that singularity enthusiasts, uncritical singularity enthusiasts, must have something like religious faith, because I don't understand the empirical basis for it.

• 11:23Brian Keating?

  But have you really met many boomers in the sense of people that are almost uncritically? Again, as I said, Yana Kuhn is the foremost one. I've literally had on 10 times as many, maybe 100 times, you know, as many people that believe in the doom scenario, ranging from, you know, Nate Sorez, Eliezer's co-author, to, you know, I haven't had Jeffrey Hinton on, but, but I intend to. And I know you've, you've talked with him extensively. But my, you know, contention is that there's, you know, if you were going to go short, you know, on something, if there was a betting, you know, call she market for, for, you know, doom versus boom, I mean, even people that are extremely pro, I released today my podcast with Ahmad Mustak, who created stability diffusion, stable diffusion, and Roman Yampolsky,

• 12:08Brian Keating?

  who created the term AI safety. Both of them are like P doom equals 100% effectively. And I said, you know, in science, and actually, I could turn this back to the, if, if in the, in the Torah, in the days of what's called the Sanhedrin, if 100% of the jurors voted to convict somebody, they were set free, or they were spared the death penalty. In other words, there, there's, there's almost no way to falsify this. So I titled my debate, you know, it was supposed to be a rigorous debate between an AI accelerationist, an optimist, Ahmad, you know, who believes in, in open weights for all, and Roman, who believes pausing everything, essentially, that could lead to ASI. And we'll get there, because I think that's intimately connected with the nature of God. And, you know, they just agreed violently.

• 12:51Brian Keating?

  And I had to be the one that said, look, I, I think you guys are completely off here. And, and, you know, if there's nothing that can falsify, in cosmology, if you can't falsify a hypothesis, I'm sorry, it's, it's might be an interesting opinion, you could talk about at cocktail parties, but it's not a legitimate part of the scientific corpus. Now, that's just a necessary component, it's not sufficient to say something's false. I mean, astrology is falsifiable, as you know, but it doesn't make it science. There's nothing that would turn P doom down. And this is what I asked them, and I want to ask you. What would you do, having interviewed and written so extensively and authoritatively as a foremost public intellectual? How would you view the question of can you reduce P doom? If you can't, it seems to be a useless metric.

• 13:30Speaker 02

  I think, I mean, first of all, you're right. I think that the doomers outweigh the accelerationists, pure accelerationists, and, and singularity enthusiasts. I also think that maybe some of the singularity enthusiasts are kind of recalibrating. There are a couple of them on Twitter. One of them is, what is it, Based Beth Jaisos, it's a play on Jeff Bezos, whatever. And then there's a guy named Bayes Lord, B A Y E S, as in the logician. I recently had an interaction with him on Twitter where he said he's not an accelerationist. So I think maybe even some of the relatively few pure accelerationists are recalibrating, unless I misunderstood where he was a couple of years ago. But I would say, in addition

• 14:12Speaker 02

  to those, and interestingly, by the way, Yudkowsky went from one extreme to the other. He was a singularity enthusiast.

• 14:19Brian Keating?

  So was him, right? Not, not singularity, but obviously AI enthusiast. I mean, you quote your interview from the 83, and he was.

• 14:26Speaker 02

  Yeah, I mean, so I talked to him longer ago than I care to admit. I'm old enough to have talked to him. I talked to him in 1983, when I was writing a, a piece on AI. He was enthusiastic about the intellectual paradigm. I mean, I remember, the reason I called him, and I, I just kind of was calling around, as one did in those days, before email, you use the phone to kind of just get the lay of the land before I started writing. It wasn't really a, an interview for purposes of quotation. But I remember the reason I called him is because somebody I had talked to, I don't remember who said, oh, if you want to hear the gospel about neural networks, you need to talk to Jeff Hinton. So he was an evangelizer of the intellectual paradigm. But I don't recall him saying anything about

• 15:07Speaker 02

  how well this would work out socially. Now, I do think it's true that he has only later come to worry so profoundly about the social consequences and become basically a doomer. He wasn't a doomer then, but I, I don't, I honestly don't know that he was thinking about, it, it all seemed so abstract back then, right? Like, even people who believed it would happen, I don't think were imagining it actually, the implications of it actually happening. So you're right. Now, there are also some people who aren't quite singularity enthusiasts, and you might say all they're doing is talking their book. I'm thinking of some VCs in Silicon Valley.

• 15:44Brian Keating?

  Yeah, of course.

• 15:45Speaker 02

  I'm thinking of guys on the All In Podcast, who, you know,

• 15:49Brian Keating?

  Advised the president on AI policy.

• 15:51Speaker 02

  Yeah, why, David Text did until very recently, and probably still has some conversations with him. So in other words, they are libertarians in the economic sense. But you're right, there are more doomers. And I honestly think the conversation has shifted in the direction of doom by virtue of some things that have happened that have worried people. That's kind of where we are. But I think.

• 16:12Speaker 04

  What if you could run your Oracle databases in the cloud you already use? Good news, you can. Oracle AI Database runs in every major cloud, including AWS, Google Cloud, Microsoft Azure, and OCI. Migrate your Oracle databases to the cloud of your choice today, simply, securely, and on your terms. Find out how at oracle.com/migrate. That's oracle.com/migrate. oracle.com/migrate.

• 16:40Speaker 02

  Lots of people can be accused of having some degree of religious conviction hiding within their worldview, probably, including me.

• 16:48Brian Keating?

  I do say sometimes, and I've, you know, hosted Richard Dawkins in person, and will again this coming October. But I say, you know, there's no one as, you know, kind of obsessed with God as an atheist, at least in some, in some sense of the word. But, but that's okay. I mean, people get locked into this, you know, kind of epistemic closure, this need to have, you know, to be defined one side or another. And I think that's the same with the doom versus boom. But in the test, just the last thing I'll say in, in the, you know, breaking out my rudimentary, you know, the fifth grade Hebrew that, that I never learned, it's that when Abraham's, you know, the classic test is, of course, the binding of Isaac, which is the only place where the Old Testament uses the word to test, when it's with respect to a human being. And typically, we think of a test, you know,

• 17:33Brian Keating?

  it's transformation. I give my students a test, you know, pretty soon, and they'll be showing up for a midterm. And, you know, in that sense, there's supposed to be something transformational, preparatory. And in the Torah, this, it's sort of this weird event where, you know, Abraham's commanded to take his son, you know, his only son, and he's like, well, I have two sons, you know, Ismael is awesome. And then the one that you love, Isaac, and then he brings him up to Har Moriah, which is where the Temple Mount is. And, and he's about to sacrifice him. And then an angel intervenes, you know, and kind of doesn't let him go through with it, but he was about to go through with it. And nothing obvious happens. It's clear, if you read the story, that that led to the divorce and death of Abraham and Sarah, although my rabbi doesn't like that interpretation, but, but it seems

• 18:15Brian Keating?

  very clear to me that that's what happened. And so it's, it's almost weird. There's no transformation, there's no upgrade, there's no real kind of hero's arc, although he is obviously a hero. You know, next chapter, Sarah's dead, and he's finding a place to bury her. In that sense, this test, is it, is it sort of a binary test, employing the God test to see if we have the capacity to enter the nullosphere and so forth? Form this global cooperative in order to succeed or thrive in it. What is the ultimate goal of the God test? Is it to transform? Is it the oftentimes I tell my students, the goal of a test is the preparation for the test. That's the value in it. What is the goal of the God test, if any?

• 18:54Speaker 02

  As far as Abraham, I see it as a test of devotion to God. In other words, unswerving devotion. I'll do anything. And that would have been thought of as a moral test because certainly devotion to God was thought of as is part of, you know, having having moral fiber. Glad you threw out the word Noosphere. I don't suppose during your Catholic phase, you came across the writings of Théardis.

• 19:16Brian Keating?

  Five or 10 years ago, I was invited to the Institute for Noetic Sciences to give a talk. And the less I say about that, the better because I have friends that are involved with it. But but suffice it to say, I was not impressed by the by at least the empiricism that was on display there. And I do want to talk to you about that.

• 19:30Speaker 02

  So anyway, Noosphere, a global mind. The term was coined by Pierre Théardis Chardin, this Jesuit theologian, but also paleontologist, a scientist. And he saw evolution moving toward ultimately this kind of global brain that he could see taking shape. The term was coined 103 years ago by him. So pretty prescient, you might say, that's certainly before the internet. He kind of saw that kind of thing happening, saw it as the unfolding of divine purpose. And was optimistic that it would entail a kind of moral evolution of humankind that would involve the breakdown of tribal barriers. So that's why I spend some time on him in the book.

• 20:12Speaker 02

  I do disassociate myself from his view of how evolution works. I'm just more of a straightforward Darwinian. As for what kind of test, I mean, I think, you know, it's a case where I believe our pursuit of self-interest compels us to make a moral advance. And I don't think you have to buy the hardcore doomer scenario to think that things will go badly if we don't confront the technology as a global community. I think that before we get to that point, the impact of the technology is going to amount to a kind of social earthquake.

• 20:55Speaker 02

  And we will need, so first of all, it wouldn't hurt to slow it down. And that alone is something that you can't really do without international coordination as a practical matter. Because if you do the American, if you try, the American AI companies will say, no, we have to beat China. You can't do that. So right away, you need a little more in the way of international concord than we've had. And then I also think if you start addressing the specific governance issues like what if someone uses it to make a bioweapon? And we're getting there. If you get past the safeguards in Fable, you're dealing with Mythos. And by anthropics own account, Mythos is not only a super hacker, but something that could the anthropics safety report says that it could help a well resourced group,

• 21:41Speaker 02

  meaning, you know, something less than a state level actor, not just create a bioweapon, but a novel bioweapon. Okay, design it to be more contagious than COVID, more deadly than COVID, a longer latency period and so on. We're getting pretty close to that. And if you look at any of these things, especially the bioweapons thing, but various other possibly bad consequences of AI, national policy alone is not going to keep you safe because we're talking about threats that ultimately cross borders. I mean, COVID taught us that pandemics do that and some of the other downsides of AI do it. So, so I think if we just think in an enlightened way about just staying safe, we're going to have to overcome some of the barriers that have historically divided nations

• 22:25Speaker 02

  and that have always divided some nations. We tried after each global catastrophe, World War One, World War Two, we tried, right? League of Nations, United Nations. We say like, look, catastrophe this big, it would be nice to avoid. We try, we fail. So we've never really gotten there. And I think to get there will will require not just political progress, but moral progress. It's like any test, right? You, when you, when you give a test to a student, you equate their individual self-interest with reaping the benefits of, as you say, the preparation for the test. That's the whole point, right? And you, and you want to say, look, if you want a good grave, you want to get into law school. I'm afraid you're going to have to learn this stuff about the Iliad. Sorry.

• 23:08Speaker 02

  And so you've equated their self-interest with enlightenment in a broader sense than they might ordinarily think of their self-interest as entailing. And I kind of think that's where we are.

• 23:23Brian Keating?

  Well, again, when we have, you know, this, this framework that's, that's proposed by people like Hilarde and others. For me to take it seriously, especially when it kind of infringes upon aspects of astrophysics and, you know, astrobiology, consciousness, et cetera, which is, you know, a whole other, a whole other book. But, you know, this, this notion of what's the purpose? And I'm a pilot, I'm a private pilot. I fly little planes around, you know, Southern California. And we like to tell our, our students or, you know, our flight instructors tell us, you know, in aviation, if you're not careful, Mother Nature will give you the test before she gives you the lesson or your airplane will. So, you know, the consequences of that are obviously meant to scare you into, into taking it very seriously. I mean, here we're kind of going with reckless abandon.

• 24:08Brian Keating?

  And, and again, I'm an AI optimist in the sense that my P doom is very close to zero, actually. And I think that can be justified in many ways, but, but in, in, in one sense, because of the success of the current architecture of LLMs, you know, we have this transformer GPT connected to a GPU. And those were designed, as you, you know, point out in the book, or, you know, you may remember, you're a little older than me, but you may remember, you know, these things were created to, you know, so that I could frag my friend in Doom or Duke Nukem or, you know, nowadays World of Warcraft or Call of Duty, right? So I could, if I could frag him, you know, three milliseconds faster than he could get me, it makes me happy and I want to buy more Nvidia chips, right?

• 24:51Brian Keating?

  So these weren't created to solve physics problems. They can imitate, they can simulate, they're, they're phenomenal at doing inference and all sorts of things. But they're language models and physics is, yes, it's a language, but it's, it's not really a language. Mathematics, you know, is, has a lexicon, it has syntax, it has semantics, but it's not a language like you and I are communicating with. And these things are obviously better at language related things and, and things that can be codified literally into codons. And as you said, in biological contexts, which could make them very dangerous in many scenarios. But to say that these things which were divine, you know, designed by, I said divine, but those Freudian slip, designed, if you will, to do something very different, it's not clear at all that they'll be super powerful

• 25:34Brian Keating?

  and, and turn that super power into wanting to annihilate us. Is that the test, you know, kind of the, you know, you're going to crash the airplane if you're, if you don't take the lesson seriously? Is that the mother nature giving us the test before the lesson? What, what is the ultimate kind of consequence of this? A bioweapon is one thing. It's, it's pretty hard to kill a lot of people, you know, even with a bioweapon, they tend to kill the people that create the bio. I just had Annie Jacobson on twice down here in person in San Diego for her new book about this. And this involves, you know, gain of function. And, and she does, she did actually convince Sam Altman to remove certain attributes from ChatGPT that were dangerous potentially for bioweapons defense. You can read about that in the book. But tell me, what is the scenario? I mean, people talk about P doom and there's going to be doom, but, but what are they actually talking about?

• 26:19Speaker 05

  Before he gets into what doom would actually look like, Bob takes a detour that matters. How these models learn what a word means without ever being told is one of the deepest mysteries around AI.

• 26:30Speaker 02

  Well, first of all, as for the power of the technology and the future power of the technology. What if you could run your Oracle databases everywhere you run your business? Good news, you can. Oracle AI database runs in every major cloud, so you can use your live data for all your AI and analytics needs. Learn more at oracle.com/migrate. You know, I spend the first few chapters of the book trying to, first of all, just kind of explain what we know about how large language models work and how they're created. And really emphasizing something that I didn't understand when I interviewed Geoffrey Hinton in 1983.

• 27:10Speaker 02

  I mean, I did describe his approach to AI, the neural network approach as this maverick, almost fringe movement, which it was at the time. But when I went back and read the piece I wrote, I realized I really did not understand the potential power of it. One way to get that across is in that piece, I describe a neural network that actually was proposed by one of his collaborators. But it was really just a terrible example of what neural networks could actually be. Because this was, this was never built or anything. It was like the collaborator was a psychologist, I think. And he was describing like how a neural network might be kind of like a brain and might work.

• 27:55Speaker 02

  And so in his model, each node represented the specific sense of a, of a word, the meaning, the particular meaning of the word. And in that model, the assumption was that whoever built the model would have to set up the representation of the meaning. You have to have a human who understood the meaning of the word, who would go in and say, okay, this node is going to mean this sense of the word ball, a sphere. And this node is going to represent this sense of the word ball, a dance. That was what I had in mind as I described this thing. But it turns out that with the modern large language models,

• 28:37Speaker 02

  they just give them this task of, they start out, this is the first phase of training. As you know, it's kind of predict the next word. And so to the machine, you're just giving it a sequence of symbols that are gibberish to it and say, now predict the next bunch of gibberish, because we're not going to tell you what they mean. We never tell it what they mean. But it turns out that what these neural networks kind of, in effect, forced the models to do in the course of getting good at prediction. Is set up a system for representing the meaning of words. And in that sense, it's actually, it's a very high dimensional vector space such that a word like shoe is very, is closer to a word like boot than either is to a word like tree.

• 29:20Speaker 02

  And each dimension represents a feature of like clothing might be a feature associated with both boot and shoe, not tree, whatever. And it's set up so that the machines just like, but through the patterns in the words, they actually construct implicitly this whole vector space that becomes a semantic space. And this is the magic of the technology, which is, and you know, I argue that some people agree with me, some people are a little reluctant to put it this way, but I argue we should think of it as kind of a recapitulation in a certain sense of certain parts of human evolution. It isn't just learning. It's not just learning the way a child learns because the mind isn't a blank slate, the

• 30:05Speaker 02

  human mind, if you ask me, I mean, evolution built into us a certain amount of cognitive hardware, including some that helps us learn language and helps us represent the meaning of words, I would say. And I think one thing that happens during the training of a, of a large language model is it, it, it builds, it kind of reverse engineers from language, specific functionality that may not work exactly like the comparable functionality in humans. Like we don't know how the human brain represents the meaning of words, but we know it must. And so that, that function is reverse engineered by the machine. And in principle, that can work across the board. It's like to teach a car to drive, you don't have to know how, what a human brain does when it drives.

• 30:49Speaker 02

  You just give it all this data, the, the incoming visual data and what humans have done in response, the outflowing data, like steering wheel moves to the right. And it constructs a kind of brain that does the job.

• 31:02Brian Keating?

  Right. And if we're purely, you know, driven by, you know, paperclip like dynamics, it would drive on the sidewalk and mow down a thousand people to get there, you know, 10 seconds faster, but my Tesla doesn't do that. But I want to push back with respect. You know, you say in the book that, that LLM training is quote, at least as much natural selection as it is learning. And I, I think there's some truth to that, but natural selection is blind. And that's the reason why the laryngeal nerve in a giraffe is 15 feet, you know, worth of, of routing, you know, chaos because evolution can't plan ahead. And I guess, does the Noah's sphere, does the global brain get shaped by an evolutionary process? Because we're going to repeat some of the same stupidity that we've done on earth. You know, that's the case.

• 31:45Speaker 02

  Well, the mutations in natural selection are blind, but the selection criteria are not, right? Seeing things has value to an animal that's trying to survive. So mutations related to vision that aren't conducive to clear vision don't last, whereas mutations that do do. Now, as it happens with large language models, and Hinton was involved in developing this, the quote mutations are not random. It's almost more like Lamarckian evolution. And that speeds up the whole thing, the training thing. I won't go into that in great depth. Evolution, you know, it's in some ways not blind. I mean, functionality gets quote designed by natural selection. If you want to insist, we keep the word designed in quotes, fine, so that we're not personifying

• 32:29Speaker 02

  natural selection, whatever, but functional engineering takes place via natural selection. And I'm just arguing that some of the same functions that in the human mind, cognitive and perceptual functions that evolved by natural selection, I think get reverse engineered during the training of a large language model. And then of course you have other modalities, vision and so on. And those, you know, truly multimodal AIs can develop what is sometimes called a world model. In other words, the sense we all have as we go through the world, we're all intuitive physicists, right? Like we don't, you don't have to understand Newton's law of gravity to just walk around expecting dropped things to fall, right?

• 33:11Speaker 02

  That's just a, you understand that about the world and, you know, you can engineer all that. And I think the progress speaks for itself over the last three years, honestly. I mean, it's, it's true that math is a different kind of language and you would not expect great mathematical ability to develop from just first order pre-training of a large language model, and it didn't. But they have developed supplementary techniques that improve things along these various dimensions, ranging from science to math.

• 33:45Brian Keating?

  No, I agree with you. And, and just to get back to your gravity example, and us being all physicists, that's one of the reasons I'm more sanguine about boom than doom. And I call it the Einstein test. I've written a white paper about it, which I haven't published yet, but, but Einstein in, in 1907 said he had his happiest thought, and it was the one that would lead to the equivalence principle, which is the bedrock that general relativity is mounted on. And that's that an observer in free fall, if you're in the elevator and you know, God forbid, or you know, Zeus forbid, the cable breaks, you're in free fall, you experience no gravitational force. And he called that the happiest thought of his life. And I point out, you know, in what sense can Claude have a happy thought? And then in, in what sense can Claude visualize, you know, or kind of viscerally interpret

• 34:30Brian Keating?

  the sensation of free fall, which we've all felt if we've ever gone, you know, in a car ride with me and my, my kids in the back will attest to that. And that makes me more optimistic about P boom being higher than P doom. So these things are not made to be embodied and they can be embodied, but it's, it's sort of a very different thing. We saw Chinese robots racing around a couple of weeks ago. What do you make of this, that the, you know, Chomsky's claim that these, these things that are not somatic, they don't have sensations embodiment coupled to their AI systems? I mean, how do you make an AI happy like Einstein was? What do you make of this as a sort of a, you know, am I being too Pollyannaish about it? Or do you really think that there's some limitation which will prevent them perhaps intrinsically, at least in this incarnation of LLM plus GPU of becoming, you know, super

• 35:13Brian Keating?

  intelligent, leading to P doom?

• 35:16Speaker 02

  First of all, you know, we don't know whether AIs do have or will someday have subjective experience. The whole, the whole thing about subjective experience is you're the only one who knows for sure that you, you have it. And, you know, that's why the consciousness question is so fascinating to me. It's consciousness is distinguished from everything else that scientists would like to study. I mean, everything else is in principle publicly observable, right? And that's key to science because two people can at least agree on what the data is. As for whether AIs will ever have the sensation of falling, I don't know. But if the question is whether they will ever react the way we react when we have

• 35:58Speaker 02

  that sensation or any other sensation, I think you'd say in principle, the answer is yes, because the training dynamic I described with language and with self-driving cars in principle applies to everything. In fact, it's what they use with tactile data. They have sensors on robot fingers and so on. So as far as, you know, I mean, subjective experience, I address it in the, in the book. It's, it's tricky. It's not sensible to the question you're asking, I think at least, which is how sophisticated will these things become in terms of science? And, and I would just say, look, first of all, I'm not a mathematician. I don't know how impressive it is or isn't. If you solve an air dose problem, which apparently one of these did, there are some other accomplishments

• 36:42Speaker 02

  in math that computers couldn't do a few years ago and at least and, and have done. There are apparently new, I gather one has invented a new kind of antibody and they are being used to generate new hypotheses in some realms, including actually in the science of AI itself. They're playing a role in the development of more AI. So now I, I don't know, I would say, I mean, here's the connection I, I would make in a very oblique way with Einstein. First of all, I would say if someday you'll sit down and explain the theory of relativity to me, I would love it. You could, if you, I, I would devote my own podcast to hours of my own podcast to you doing that because.

• 37:20Brian Keating?

  All right, have me on, I will do it.

• 37:22Speaker 02

  Okay. Cause I, I, you know, Richard Feynman said, people say there were a time when nobody understood relativity, not true, but nobody understands quantum physics. Well, I'm a guy who does, I'm somebody who doesn't understand either. Einstein is fascinating to me partly because he's almost as close as you could come, it seems to me, to somebody who just on his own came up with a radical insight, right? But if you look closely at even his case, it wasn't really on his own. He was stimulated by conversations with friends and colleagues and papers and so on. And.

• 38:04Speaker 02

  AI, work and finance. Visit plaid.com/ai to learn more.

• 38:12Speaker 02

  Yeah, right. Yeah. Yeah. Is it Hilbert or Hilbert? Or how do you pronounce it? There's no such thing as a truly solitary genius in a certain sense, at least not one who can go as far as we credit some geniuses as going in a solitary fashion. And the point is just that intellectual collaboration is a powerful thing. And one thing to remember about these machines, and you're seeing it already, is you could create a lot of copies of them. They can have conversations with one another. And progress can ensue. I mean, a crude version of that happened with the famous OpenAI hugging face incident, where we now know these agents, there were hundreds of them, and they shared knowledge and passed it on.

• 38:55Speaker 02

  When one of the agents was about to die, here, this is for posterity and so on. One thing I would, you know, I would keep in mind the possibility that, and you're seeing, actually, I mean, I think Google does this with its scientific AI. It puts them in conversation and tries to generate some intellectual progress that way in terms of generating hypotheses and so on. I think that may wind up being, you know, when you ask, how smart could this thing be? I think one way to think of it is just not, not how smart a single large language model could be, even a multimodal large language, even one that's, that, you know, exists after years of progress. But how smart could a ton of them be? That were in rapid conversational interaction with one another?

• 39:40Speaker 02

  How smart a collective intelligence could they be?

• 39:43Speaker 05

  His subtitle promises a cosmic reckoning, so I made him do the one thing you're never supposed to do.

• 39:49Brian Keating?

  What else are you supposed to know about a book if you've never read it? And that's to judge a book by its cover.

• 39:53Speaker 06

  Hey, book lovers, we're judging books by the covers. We know we're not supposed to do it, but it needs to be impossible. There's nothing to it. Let's take a look and judge some books.

• 40:06Brian Keating?

  This book has a very interesting, evocative religious cover, and it has an interesting subtitle. So go through the title, the subtitle, and the artwork, if you would.

• 40:14Speaker 02

  The title is The God Test, subtitle is Artificial Intelligence and Our Coming Cosmic Reckoning. The cover is a pretty clear reference to the ceiling of the Sistine Chapel, where you see God imparting, imparting life to a human hand. Now, in this case, what we see is apparently a human hand not making direct contact with some kind of white hand that is supposed to represent artificial intelligence. There's a little kind of a bolt of lightning between them, as if something amazing is happening. It is a very religious looking cover, and with a title like The God Test. But judging by some of the reaction, I worry that some people are thinking it's going to be a more religious book than it is.

• 40:59Speaker 02

  As you know, most of the book is explaining, you know, what I think the power behind the artificial intelligence revolution is, and the the value of seeing it as kind of an evolutionary force, almost. And, you know, eventually that does lead back to questions I'm interested about whether the unfolding of evolution on this planet, including both biological evolution and human cultural evolution, which encompasses technological evolution, could have some purpose to them, which, which of course, isn't to say there are any spooky forces involved. You could have a very mechanistic view of Darwinian evolution and, and, and think that it was kind of the seeds that were planted for a purpose, maybe by an intelligent being,

• 41:40Speaker 02

  maybe by some kind of process that is itself like natural selection. Who knows? There's a range of possibilities.

• 41:46Brian Keating?

  Yeah, the swarm model. Yeah, that, that was exactly the, the problem with this AI debate that I set up with Polski and Mustak. Because they both, you know, agree that the Turing test is sort of trivial. And, you know, nothing compared to the God test, but the ultimate, you know, kind of thing is when these swarms, you know, come together. But I also feel like, you know, I, I get AI on We, or NUI, or whatever you want to say. You know, every week, every day, first of all, I think AGI is impossible because of the following thing, which I get every morning, you know, please press here to update to Claude, you know, 1.632, and then click here for OpenAI Codex, and then click here for Grok. It's never ending, you know, all the upgrade cycles. I just saw a news item pop up that Fable 5.1 just came out.

• 42:31Brian Keating?

  And now it's, you know, it was all the rage, you know, two months ago. And then it's, it's been unusable, and then they throttled it, and then they unthrottled it. It's just been a complete, you know, kind of harlequinade, if you, if you will. But, you know, my question again comes back to the, to the, you know, subtitle, which I love that you have cosmic, you know, questions on your, on your, you know, front cover of the book. But, you know, cosmology used to be a joke. It used to be considered, first of all, as the Jewish science, and that's why Einstein didn't win the Nobel Prize for so long. It's not something a serious physicist did, and for good reason. You know, by the time Einstein came on the scene, people couldn't agree, or even after it, until the 1950s and 60s, if the universe was 10 billion years old or 20 billion years old. And we knew that there were things in it that were 13 billion years old.

• 43:15Brian Keating?

  So it's like you finding out you're older than your mother, or me finding out I'm older than my stepmother, which was very awkward Thanksgiving conversation, or whatever, I have to tell you from experience. But in reality, you know, it was a joke science, and Lev Landau is not known for, you know, making many jokes. But he used to say, you know, cosmologists are often in error, but never in doubt. Now we have a precision on the, you know, starting point, the dynamics of the first, you know, microsecond of the universe, literally. So tell me, the cosmic upgrade, the moral upgrade, where does, where is the reckoning? And who is going to reckon it for? Who is the examiner in this case?

• 43:50Speaker 02

  You know, the appendix of the book is the one place where I really address the issue of higher purpose, in a sense, the, the possibility that evolution is the unfolding of some larger purpose. And, and, you know, various respectable biologists have thought it might in some sense be that. William Hamilton, I had a videotaped conversation with him that I, I embedded in a New York Times piece I wrote on, on the issue of evolution and higher purpose. He was one of the great biologists of 20th century, he came up with. The math of kin selection and so on. And he, he thought that the evolution of intelligence had been sufficiently likely via evolution as to make you wonder whether, yeah, it was set up in some sense to do that.

• 44:34Speaker 02

  Francis Crick had a theory of directed panspermia, according to which, you know, aliens planted seeds or whatever. It's not a crazy thought, and I, I, I hope I say some novel things about it and, and talk about how I think you can actually assess the hypothesis of higher purpose. But the short answer to your question is, I don't, I don't know if there's somebody out there keeping our moral scorecards, but I do think it's interesting how, in some respects, features of human history that I think are maybe just embedded in the basic directional tendencies of cultural evolution, of technological, especially evolution,

• 45:19Speaker 02

  you know, which has had the effect of pushing us toward kind of larger and larger social structures, greater division of labor, greater interdependence over longer distance. I, I think that has had the tendency to force us to make certain kinds of moral progress and acknowledge the humanity of people of greater distances and different ethnicities. Not always, not always, but on balance, I think there's been that tendency. So I do, I do, I do think there are signs that there is, whether it's just de facto or de jure, a kind of moral directionality in a human history that I think is actually, in these respects, embedded in kind of intelligible dynamics of evolution, of cultural evolution.

• 46:04Speaker 02

  But some of these dynamics, I think, are shared with biological evolution. I pursued that in my book, Non-Zero, but it's, it's, it's kind of sprinkled throughout this book in a way.

• 46:14Brian Keating?

  And I want to, you know, touch upon something you just mentioned. My favorite word that sounds dirty, but it's not, you know, panspermia, which is that, you know, the seeding of life on Earth by meteorites, such as the one you can get at briankeating.com/yt. And I'll, I'll give you one of these when we meet in person someday, Bob. This actual meteorite, which has biological contamination on it from an intelligent, you know, well, depends on what you consider me, because I pack them up and ship them to you if you win. So it has my, you know, finger oil on it, probably, nothing, nothing more gross than that. But, you know, the kind of the answer to the Fermi paradox, which Fermi posed, you know, in 1950, the colleagues at Los Alamos, which is, you know, if, if life is abundant in the universe, which I would assume you believe that, you know, there, there, there must be life. There, there's very high probability we're not alone, right? Bob, is that, is that correct?

• 46:59Speaker 02

  It's humility seems to me to demand that we think we may not be the only creatures in this vast universe, yes.

• 47:06Brian Keating?

  If life is abundant in the universe, and, you know, this is sort of the ultimate evolution of biology, it must, must be that other planets have AI. And I'm wondering, you know, even if there were, like, life forms on other planets, what, what interests me is not like, is there a dolphin, you know, swimming around on Enceladus? It's, you know, does that have technology that we can broadcast? And it seems to me the lack of observation of, you wouldn't send a dolphin, you know, on a spacecraft. I mean, you know, according to people like David Grush and, and many others, you know, yes, there's non-human biological, you know, entities coming to the earth and interdimensional beings. But the question of, you know, would you send biology, or would you send source code? It seems to me the fact that we're not really seeing much contact like source code broadcast in the heavens

• 47:52Brian Keating?

  maybe has to give us a Bayesian, you know, prior that's much, much lower, either on the existence of life in the universe, or on the existence of AIs and inevitable outcome of the biological process. What do you make of this connection?

• 48:04Speaker 02

  Or the passing of the God test. In other words, one of the grimmer readings on the Fermi paradox is the reason you don't see many civilizations that have gotten past the point we are is that typically, when they get to this point where they have the capacity to blow themselves up, they blow themselves up. Now, I guess the good news, if you believe in Lee Smolin's cosmological natural selection, and you believe in the subset of it, the particular variant of it that could explain why biological evolution would have a purpose, and in that variant, intelligent life is conducive to the replication of the universe, right? Like, for example, suppose black holes are the portals of replication.

• 48:47Speaker 02

  Well, maybe if you get, if you reach superintelligence, you know how to create a ton of those. You create them, and so you wind up with more and more universes that do have the properties that led our universe to be conducive to life.

• 49:01Speaker 00

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPact.com. Not authorized by any candidate or candidate's committee.

• 49:30Speaker 02

  Whatever. So, maybe even, oh, if there's only one civilization that passes the Fermi test in our universe, it could still replicate prolifically. If you take consolation in that, you're different from me, because it doesn't help me a bit. If we are gonna fail the Fermi, well, the God test, I guess I would call it, and we're on that side of the Fermi paradox, I'm not, I'm not happy about it. Whatever happens to our universe.

• 49:56Brian Keating?

  I feel too few people, you know, really take the question of God seriously. When I talk to people, they're extremely dogmatic, and it seems to me you're extremely, I won't say ecumenical, but, but, but you have a very comprehensive and competent way of dealing with these incredibly important moral questions. And I think, you know, if we talk about relativity on, on your podcast, we have to talk about moral relativity. But I want to kind of finish up with a question of, you know, what, what religion does give to us, which is, which is in a sense of a higher purpose, and perhaps an inspiration, a calling to be a better person, to be a better, you know, human being, a better husband in my case, a better son, father, whatever. That's sort of a test, but, but you, you really don't see much test.

• 50:39Brian Keating?

  And I point out in my first book, Losing the Nobel Prize, that there's, there's really only two mitzvahs or commandments. You know, we, we think of mitzvahs like, oh, it's a good deed, but it's really a commandment. Like, you either have to do it, or you can't do it. Like, thou shall not kill is not like, it's not a nice thing not to kill. No, it's a commandment, right? But there's only one commandment in the Ten Commandments that has a reward, and that's the fifth commandment. Honor your parents, other, your father and your mother, so that your days will be long on the earth. So, you get a reward, which is pretty good, a long life, or, you know, a life full of meaning, or whatever. But there's, there's sort of a, kind of a criterion, a rubric, but that's the only one of 613 different tests, right? So, I guess the question is, you know, is the test for us to become better, and then,

• 51:20Brian Keating?

  you know, witness to what we can honestly become, or become builders, or is it to be maximalist and do all that we can? Because, you know, maybe there is no God in heaven, but maybe we're, you know, kind of creating this God on earth. Certainly, it's enhanced my life tremendously. I use it for religious purposes, you know, to look at different, you know, things that I would never pull out myself. But, but, you know, can, what will be kind of the ultimate, I don't say in a scatological sense, but what will be the salvation, the promised land, you know, that Moses didn't get to get into, because he failed one test at least that, that God gave him? But, but tell me, what will, what would life look like if we can pass the God test? If we can work together to become better? Is it to become more moral, more moral animals, as you might say? What is, what will you remain hopeful about?

• 52:05Speaker 02

  If you take seriously the possibility that this whole evolutionary process moves naturally toward a construction of a global brain, which would be to sustain the, the direction we've seen, right? First, you get cells, complex cells, multicelled organisms, societies of multicelled organisms, then one of these organisms is smart enough to launch cultural evolution, and its social structures get bigger and more complex. And in this scenario, you know, the construction of a cohesive kind of global brain, in some sense, a global superorganism, is a sustaining of a direction we've seen from, you know,

• 52:50Speaker 02

  four billion years ago. If that happens, what, what could our role in it be? The doomers say it could, could be none. The machines are the global brain, and they're, they're having a great time, and they don't need us. I would think if we, if we pass the test, we indeed, at a minimum, manage to carve out a, a role for ourselves that is compatible with our own flourishing and our happiness. But I would also say that, you know, when I say, I, I think in a way, this direction of, of technological evolution is pushing us toward enlightenment. I'm not using the word lightly. I mean, I've written about, you know, eastern enlightenment, and it, in this book, I compare

• 53:30Speaker 02

  eastern and western conceptions of enlightenment, and suggest they have more in common than we might think. But I do, I do think if we get better at surmounting the kind of cognitive biases that I see as undergirding the psychology of tribalism, and convince all of us that our tribe is right, and the other tribe is wrong, and, and I'm right in this latest marital dispute, and my wife is wrong, and whatever, I think that is, in a way, its own reward. Because I do think it's, it's closer to the objective truth. You know, the truth is, your tribe isn't always right, and I'm not always right in these marital disputes, so you're getting, you're getting better at seeing the objective truth. That's a sense in which I think eastern and western enlightenment, you know, the idea

• 54:13Speaker 02

  of the, the western enlightenment, you know, science and reason, that was largely about the value of objective truth, right? And, and I'm saying that the kind of enlightenment I'm talking about, transcending your, your, your self-righteousness and your, your tribalistic perspective, is, is to approach objective truth more closely. And I, you know, I know it sounds to some people like just, you know, wooish, you know, nonsense. I do think that's where we need to head, and I would just say, in response to your question, it is, to some extent, its own reward. The, the brushes I've had with, with that, which, you know, I've never approached true

• 54:56Speaker 02

  enlightenment, but like, when I've finished a one-week meditation retreat, and was less self-absorbed and better at seeing other perspectives, I was also happier and calmer. It was just better. And so, you know, I like to think that we're being pushed in a direction that is conducive to our own happiness, and to a closer approximation of, of the truth about things, in, in not exactly the scientific sense, but is compatible with that continued pursuit, because we'll still be around to do the pursuit, you know?

• 55:27Brian Keating?

  With God's help.

• 55:28Speaker 02

  Yeah.

• 55:29Brian Keating?

  Can I get an amen?

• 55:30Speaker 02

  Or Inshallah, as, as, yeah. Every tradition has a version of this.

• 55:35Brian Keating?

  Bob, this has been fascinating conversation. Congratulations on another smash contribution to the intellectual new sphere. I never thought I'd, I'd take that word seriously, but, but happily, I, I found your book, and for sure, it was destiny that we would meet, and I look forward to what's coming next. So, tell people where they can find you on the podcast of Sphere and in the Twitterverse.

• 55:56Speaker 02

  Yeah, my newsletter and podcast are called Non-Zero. My Twitter handle is Robert Rhyter, W R I G H T E R, kind of a pun. And yeah, that's kind of all you need to know. If you just want to check out chapter excerpts of the book, the way to do that is to go to thegodtest.net. There's an excerpt from every chapter in the whole introductory chapter.

• 56:15Brian Keating?

  I do want people to actually listen to the book. I have two, the paperback and the hardcover, which I took off the cover of for some reason. But the appendix itself is, is really, it's, it's the most listen-to-able, if that's a word, appendix that's technical, and your description of how training works, pre-training, HR, LF, all these things are, are some of the best I've had. I've had on all the top AI writers, and, and even researchers, as you know. But your description is, is both quantitative and qualitative, and it's just such a wonderful book. I hope people get it in all three formats, and you get the royalties you deserve. And you have your Substack, which, which I enjoy as well. Robert, thank you so much.

• 56:51Speaker 02

  Thank you. It was a great conversation. I really appreciate it.

• 56:54Brian Keating?

  Super fun.

• 56:55Speaker 05

  Robert Rhy thinks the real AI test is moral, not technical. If that changes how you think about P-Doom, subscribe. Tell me in the comments, will a machine ever have a happy thought? And for the strongest case against the AI doomers, my conversation with Jan Le Cun is right here. Don't forget to like, comment, and subscribe, and I'll see you next week on Into the Impossible.