A History Of Existential Risk - Thomas Moynihan | Modern Wisdom Podcast 306

Thomas Moynihan is a historian and an author. Humans may have only had the ability to destroy ourselves for the last hundred years or so, but thinkers have been hypothesising about the potential end of existence for thousands of years. Today Thomas explains the history of how humanity came to realise its potential for extinction. Sponsors: Get 20% discount on the highest quality CBD Products from Pure Sport at https://puresportcbd.com/modernwisdom (use code: MW20) Get 10% discount on your first month from BetterHelp at https://betterhelp.com/modernwisdom (discount automatically applied) Extra Stuff: Follow Thomas on Twitter - https://twitter.com/nemocentric Buy X-Risk - https://amzn.to/2PzTYKx Get my free Ultimate Life Hacks List to 10x your daily productivity → https://chriswillx.com/lifehacks/ To support me on Patreon (thank you): https://www.patreon.com/modernwisdom #existentialrisk #climatechange #historyofideas - Listen to all episodes online. Search "Modern Wisdom" on any Podcast App or click here: iTunes: https://apple.co/2MNqIgw Spotify: https://spoti.fi/2LSimPn Stitcher: https://www.stitcher.com/podcast/modern-wisdom - Get in touch in the comments below or head to... Instagram: https://www.instagram.com/chriswillx Twitter: https://www.twitter.com/chriswillx Email: [email protected]

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Ninety nine point nine percent of all species that have ever existed are now extinct. So it's extinction is the rule, survival is the exception. That's an important thing to know is that, you know, potentially we have come close before. This isn't something that is completely unprecedented.

• 0:18Chris Williamson?

  If we're worried about existential risks annihilating our future, why spend any time studying the past?

• 0:28Speaker 00

  That's a good question. So I hope as we talk through this, the true significance of what I'm about to say will be elaborated further. But I think that it's so easy to focus on the risks coming towards us, coming down the track.

• 0:51Speaker 00

  And it's slightly harder to take stock and look backwards and see just how far we've come. One of the things I mean by that is that the very ability for us to even be able to see those risks ahead, the risks on the horizon, that's a massive achievement for humanity, for our knowledge of the world, for our knowledge of what is best to do within the world. That's a massive achievement. And again, I hope that, you know, as we speak through this, you know, the truth of this might hopefully kind of unfurl.

• 1:32Speaker 00

  But, you know, some of our biggest achievements are almost invisible to us.

• 1:39Speaker 00

  Some of, you know, some of the most profound breakthroughs of human knowledge are often invisible to us. So, you know, I often point towards the fact of, well, take slavery, for example. For most, for the majority of human history, people presumed that it was just part of the natural order of things. You know, it wasn't questioned. All of us these days kind of take it for granted that that's inherently wrong.

• 2:15Speaker 00

  Another example I like to use is perspective, right? So, you know, think back to being a kid in school.

• 2:24Speaker 00

  You'd learn to draw your first cube, or your first prism, or, you know, triangle, pyramid, sorry.

• 2:32Speaker 00

  It's so easy, it comes to you so naturally. Rewind, you know, six centuries, seven centuries, it wouldn't have come naturally at all. You know, I was drawing cubes, I don't know what age, but pretty young. And that's not because I'm a genius, or a prodigy, or some Da Vinci tier, you know, mega genius. It's because of cultural osmosis, because the ideas that we just take for granted, we inherit. But someone had to come up with them, and, well, often, lots of people had to come up with them. And it takes, you know, centuries, decades of effort, of hard work, of error correction, of finding out the ways in which we are so severely wrong about the world. And, yeah, so to tie up my point,

• 3:16Speaker 00

  thinking, being able to even notice these risks, the risks facing humanity, or just the fact of how bad human extinction would be. Those are really huge achievements, and they're quite modern ones as well. So, yeah, I would say, I would put it like this, it's a cure for despondency. Because like I said, it's easy to see the risks ahead, harder to see how far we've come. So it's easy to be despondent, it's easy to despair. But it's deceptively easy, because we have that kind of bias where it's easier to look straight ahead rather than look to the past.

• 3:55Chris Williamson?

  Are you trying to say that a book about the annihilation of humanity is somehow hopeful?

• 4:00Speaker 00

  Yes. So I often get this when people read my book, they're surprised that it has this, well, it attempts to have this hopeful message.

• 4:16Speaker 00

  But yeah, nonetheless, it does, it does. You know, and funnily enough, I came into all this way less hopeful. And I'm not sure if this would be the perfect, the right word, but more fatalistic. And it was through actually tinkering through, tracing through the contours of discovery and intellectual progress. And, you know, just how far we've come in terms of, you know, even if we know that we're not true, or we have 100% certainty of anything right now, there's just the ability to realise that we're wrong, right? To realise that we're wrong, and therefore realise that we can know better, we can correct ourselves.

• 4:55Speaker 00

  I find that fascinating. So yeah, yeah, I think it's a hopeful book. I hope it's a hopeful book.

• 5:03Chris Williamson?

  I think so. You mentioned there, I think, quite an important point. You say the the ability to grasp the prospect of our own extinction is a significant intellectual achievement. It separates us from other animals. Are you also trying to say that that's something that we should be thankful for? This species-wide denial of death that Ernest Becker turned up to a million would be proud of?

• 5:24Speaker 00

  Yeah, I mean, so there's, there's Jonathan Shell. He was a guy that wrote a book, The Face of the Earth, in the 1980s. And it was one of the first books, it's relevant to this discussion, because it was one of the first books to really crisply state how bad human extinction would be. So this was in the context of the Cold War, thermonuclear proliferation. And he, amongst other people, and we can, we can talk about this later if we want, but amongst other people, pointed out this kind of asymmetry in how bad extinction is compared to lots of other, you know, kind of disasters, is that it's the foreclosure of the whole future. The way, one of the ways in which Shell expressed this was that

• 6:08Speaker 00

  there are these two deaths. So the first death is the death that we're all kind of familiar with, our own death, our individual death. And so that's the kind of Ernst Becker denial of death. You know, a lot of culture is, in a sense, kind of, you know, seems to be this, that humans have this unique awareness of our immortality.

• 6:27Speaker 00

  I'm not I'm not too familiar with the Becker thesis, but like, you know, say that say that was clearly one of one of the kind of foundations of culture, right? Humans, since they started using language, became behaviorally modern, uh, have probably been kind of aware of mortality in that sense. Um, but yeah, then Shell makes this point that there's this second death, uh, which is the death of the whole species and the loss of its entire future. Um, and that's the more modern achievement. So, uh, yeah, we've been denying death since day one, I guess, but, uh, being able to think about this second death, this, you know, ultimate fate, the loss of the entire future. Um, yeah, that's a lot more recent. Uh, and so I'm hoping that we can level up our denial of death to that kind of civilization scale.

• 7:12Chris Williamson?

  That's so funny. Can you give, for the uninitiated, for the people that haven't taken the existential risk red pill? What is the most compelling hammer blow? That you can give them about why there is an importance to existential risk? I'm already one of the initiates, right? I pray at the altar. I wear the weird mask with the long crow nose on it. Like, you don't need to convince me, but what to you is the? Hit in the existential soul example that you can give people?

• 7:43Speaker 00

  Hmm. Yeah, yeah. So, so, so for me, the kind of penny drop, um, uh, the best place I've seen it argued, uh, is, um, a philosopher Derek Parfit, who, um, actually was Nick Bostrom's, uh, uh, one of his teachers supervisors. Um, so, and this is around the same time as that Jonathan Shelberg that I just mentioned. So in the, in the eighties, uh, he wrote this, uh, book called Reasons and Persons, uh, and it's, you know, this kind of voluminous, uh, meticulous tome of, uh, you know, kind of, um,

• 8:20Speaker 00

  uh, very detailed ethical philosophy. Um, and it's, it's a masterwork, uh, you know, um, but it's, yeah, kind of deeply philosophical, deeply complex. Then the last couple of pages, he makes this argument, um, About that asymmetry that I just pointed to. Um, and he, so he makes, I think this is the best place this argument's been made, so I'll try and rehearse it. So he says, think of three scenarios. Uh, the first one is peace. The second one is a, a nuclear exchange wherein uh ninety five to ninety nine percent of humans uh are killed. Uh, the third one is uh is some kind of exchange where a hundred percent of people are killed. Uh, and then he says. Where is the biggest difference?

• 9:05Speaker 00

  Is it between one and two, so between peace and ninety five to ninety nine percent? Or is it between the ninety five to ninety nine percent and the hundred percent? Now intuitively, and you know, our moral intuitions are often wrong. Uh intuitively, you might kind of think instantly, uh well, obviously it's the difference between the first. You know, the first and the second is the difference between peace and, you know, the ninety five to ninety nine percent fatality. Uh, Partha makes uh the argument that that's absolutely not the case. The larger distinction, uh the larger difference in severity is between two and three. Uh, and that's because, again, it's the loss of the whole future.

• 9:45Speaker 00

  Uh, so he makes uh these points that, you know, the earth is likely to remain habitable for another billion years or so. Uh, within that time, uh there will be vastly more generations of humans than there already have been. Uh, so you know, civilization itself has only existed for like something like ten thousand years. Um, so if we don't screw things up, uh there's a lot of future ahead of us. Uh, a whole lot of future. And this is just constraining it to the earth, right? Uh, there are other places we can go and other places where we can have even more future. Uh, so basically, we're kind of in the daybreak of uh civilization, of the human story.

• 10:29Speaker 00

  Um, and so since then, you know, uh. Other people have extended Parfit's argument. And it's so funny to me that Parfit kind of, it's like a throwaway in the last couple of pages of this, this huge book. Um, but, you know, namely it's Nick Bostrom, for example, he uh kind of formalized the concept of central risk. Uh, you know, the first paper where he talks about it is from two thousand and two or two thousand and three. Um, and he makes this further point, uh, coming from uh this kind of transhumanist lens where um. It's not just kind of the duration of the future. It's also how much more quality that could be in it because, you know, uh, should humans use their technology in the right way?

• 11:13Speaker 00

  Uh, there's this whole kind of possibility space of other experiences uh above the human condition. Um, so, you know, I think a, a, a way that people often put it is that, you know, uh. Mice probably aren't very good at experiencing symphonies, uh, but we are. Uh, so there's presumably kind of, there's presumably headroom above us, right? You know, there's kind of orders of magnitude potentially. Uh, so you add this extra transhumanist kind of, uh, you know, um, uh, spin on it. Um, and yeah, I think all the pieces are together there. Um, you know, uh, Toby Ord's now, uh, kind of given an even more, uh, even simpler, uh, and even more effective, possibly, definition of the whole thing.

• 11:59Speaker 00

  Uh, but yeah, for me, it's those arguments that really, um, really drive at home. Uh, you know, um, particularly the Parfit one, uh, you know, it's, it's. It's easy to, uh, miss that fact, uh, that, you know, it's not just, uh, seven billion lives at stake. Uh, it's not just, you know, the current population of the world. It's our whole future and our whole potential as well.

• 12:26Chris Williamson?

  Didn't Bostrom come up with a number of 10 with 100 zeros after it? Isn't that in your book? If we do an okay amount of space colonization, this is the potential number of lives that we could have ahead of us.

• 12:41Speaker 00

  Yeah, yeah, so there are huge numbers out there. So Bostrom has this paper, it's called Astronomical Waste, where he makes this argument that there's like a kind of opportunity cost for delaying space colonization, given that there's, you know, there is actually finite resource within the accessible universe.

• 13:09Chris Williamson?

  It's depreciatingly finite as well, right? For every second that we don't, that's another bit of expansion that we can never anymore access.

• 13:17Speaker 00

  Yeah, yeah, so he makes this argument that, you know, we could expand out and, you know, create so many value structures. And yeah, that's his word for kind of people or, you know, functional equivalents of people living happy, quality-filled lives. And there, yeah, there are these computations of the kind of upper bound of amount of, you know, souls that we could spread throughout the universe. One of Bostrom's colleagues, Milan Sirkovic, he came up with a figure before then. I think it's some, it's like, it's a kind of crazy number. It's like a quadrilecto-illion or something. I can't even pronounce it, it's humongous. But yeah, you know, go to those papers for the precise, the precise figures.

• 14:01Speaker 00

  But yeah, I mean, so, you know, there's a depreciating amount of this potential as well. But more recently, there's been this kind of, so Oard calculates that, you know, the kind of opportunity cost of delaying is potentially, you know, not too awful. And so we should be patient and shouldn't, you know, kind of rush ahead if that can, you know, foreclose our potential if we are too hasty.

• 14:31Chris Williamson?

  I'm going to jump ahead to something that I've been thinking about for ages. And you sadly, Thomas, have the job of being the recipient of all of my bent, pent up ex-risk ideas, because I don't get to talk about it to everyone. For some reason, not everyone wants to talk about the extinction of permanent and ultimate extinction of the human race. So it's you and now thousands of people that are listening. As far as I can see it, there seems to be three main factors at play, right? When we're talking about existential risk and how we should potentially move forward. The first one being the danger of technological progress, as Bostrom calls it, the putting the hand into the urn and pulling out a technology. This technology could be good and improve human life, but every so often you pull out one which is either grey or black.

• 15:14Chris Williamson?

  If you pull out a misaligned superintelligence, then you're dead, game over. And if you pull out a nanotechnology turns us all into grey goo, then you're dead. And if you pull out an engineered pandemic, then you're dead. So that's first bit. There is a danger that's associated with technological progress. Secondly, there is a requirement of technological protection because there is a non-zero amount of natural risk. There are volcanoes and there are asteroids and there is the inevitable heat death of the sun and we're going to, you know, we need to continue to technological progress or else we assure that we're going to have a limited future because we know that there is a non-zero amount of existential risk that occurs naturally. So there's a balance between those two.

• 15:57Chris Williamson?

  And then the final part is the opportunity cost of delaying space colonization. Is that an okay framework to kind of view what we should be doing moving forward? That there is an opportunity cost, there is a requirement for us to not move too quickly as to pull out black ball and that by moving a little bit more slowly that we reduce the risk of pulling out black ball, but also we can't not move forward at all because there is the natural risk?

• 16:20Speaker 00

  Yeah, yeah, I think, I think that's a good way of like taxonomizing the major parts of the, the argument. Yeah, it's, it's, it's precisely as you say, it's this, you know.

• 16:36Speaker 00

  Mature acknowledgement of the risks of technology. But conjoined with a mature acknowledgement of just how good uh technology could create the, you know, could could make the future. Uh but then also, yes, that also the other acknowledgement that um without technology, uh kind of background natural risks will, you know, the probability will accumulate over time. And, you know, it's a death sentence. Uh it's just a delayed one, right? Um, you know, so, yeah, yeah, I mean, and that's what's novel about it. Um, historically speaking, um, and so, you know, we can go into the long, the long run history of this stuff, because, you know, that's, that's what I love.

• 17:20Speaker 00

  But, um, you know, it's people have been, uh, talking about human extinction, like as a, uh, you know, as a natural possibility for, um, I would say, you know, kind of, uh, three centuries, two centuries, somewhere in that ballpark. Um, you know, uh, then it becomes, uh, yeah, during that time with, you know, we're talking kind of scientists during the Enlightenment. Uh, you know, they're kind of playing around with it as this, uh, you know, interesting philosophical, uh, natural philosophical possibility. Um, but it remains very kind of distant, very far off.

• 18:00Speaker 00

  Um, and then, you know, it's really around World War Two. Uh, and, uh, the production of nuclear weapons, um, and then, you know, uh, these humongous, uh, nuclear weapons like, uh, the SARS bomb and, you know, these kind of really significantly powerful ones, um, in the nineteen fifties. Um, that's the idea of, you know, uh, the idea of human extinction, which had previously just been possible, becomes slightly more probable, plausible, uh, and therefore a policy issue. So people have been talking about, you know, this worst case scenario for, for, for, for decades.

• 18:41Speaker 00

  But often it was constrained to, uh, one technology, that being, uh, nuclear weapons. Um, and, uh, often there were kind of like, kind of these quite, um, distinct poles of thinking where, you know, um.

• 18:58Speaker 00

  It's often this kind of idea that, you know, we need to just rush towards technology, have it all, that's what will save us, or oh no, technology is bad, we should, you know, kind of be careful with what we do with it. This is what's innovative around, you know, Bostrom's work and people in that kind of area is this, you know, mature acknowledgement that it's, you know, the poison and the cure and therefore requires lots of care and lots of careful thinking. And, you know, what's really great is it's kind of given a shot in the arm to philosophy because it's, you know, there are these philosophical questions

• 19:39Speaker 00

  that we need to figure out before we have the, you know, the technologies to wield, you know, wield the power on the world. So, so yeah, that's, that's how I see it. And that's why I say it's important. Yeah.

• 19:52Chris Williamson?

  As far as I can see, man, it is the most important conversation to have. It is, it blows my mind that we have Greta Thunberg rolling around, going on a pedalo from fucking South America back to Europe or whatever, to try and reduce her carbon footprint, to talk about a problem which is going to affect us on an existential risk scale in millennia. And we don't have any, I mean, in the nicest possible way, like Nick, Nick spends a lot of time working. He's, he's not fantastic with media obligations. I think he probably had to save up about seven, seven months worth of his allowance to go on the Joe Rogan

• 20:31Chris Williamson?

  experience. And also the last hour of that podcast was the most painful hour of any podcast in history. For, for anyone that doesn't know what I'm talking about, just listen to the first hour and a half and then please do not delve into the end of it. So I want to get into kind of my thoughts around culturally, the problems with existential risk. But as you say, we've got this, this sort of wonderful research that you've done to do with the history of X risk. You actually start with this kind of cool timeline thing that's all drawn out of the, the landmarks of the, the life of existential risk. You mentioned that existential risk in prehistory before 1600 B.C. was framed differently to how it is now. Why?

• 21:14Speaker 00

  Yeah, yeah, so I would say that the the the concept simply didn't exist, right? Uh, people couldn't think about it. Um, and again, yeah, this goes back to what I was saying earlier is, you know, one of the things that really, you know, drives me and so, yeah, I mean, you know, I'm just a historian. Uh, I'm interested in, you know, this this amazing work, this super important work, uh, in existential risk. But, you know, the experts in that and not me, I'm, I go through and I try and, you know, tell the story of how we got here, which I, again, like I said, I think that's important, uh, moving forwards because people so often fail to um, notice the potential of humanity. And parts of that potential is looking backwards, as we were talking about.

• 21:56Speaker 00

  Anyway, so, you know, there are times when there are new ideas, uh, you know, um, ideas that haven't occurred to anyone before. Um, so the the obvious ones, uh, the the, I think, you know, most people are familiar with are things like, you know, Darwin's theory of natural selection. Um, you can go back and you can find, uh, you know, you can pick through the ancient Greeks and you can find someone here and there saying, oh, maybe maybe uh, humans used to be fish. Um, that's not, that's not a theory of natural selection, you know. Um, so often the way that like, uh, history is done when it comes to the history of ideas, uh, it's a case of people kind of picking back through the past and going,

• 22:40Speaker 00

  oh, here's something that looks like this new idea. Um, and so that's that's absolutely the opposite of what what I wanted to do. Um, I wanted to go, no, this thinking is new and that's why it's important. Um, so you can go back and you can find, uh, these kind of, uh, you know, the greats of, uh, ancient Greek, uh, philosophy, uh, talking about things that deceptively look a bit like extinction events, right? Um, so, you know, humans have always loved massive catastrophes. Uh, we've always loved to to to narrativize, to talk about huge disasters, calamities, you know, pyrotechnic, volcanic explosions, you know, you name it. You can find Plato, uh, he, he, so the myth of Atlantis, right?

• 23:23Speaker 00

  Uh, that's, that's one of Plato's kind of, um, you know, it's, it's a thing that he talks about. And he talks about these cataclysms that have wiped, uh, civilization from the earth, uh, in the past. Um, but then he says, as, as humanity will be wiped from the earth in the future. So you can start to notice that there's a cycle occurring here. Uh, so Plato, Aristotle, a lot of these, um, kind of ancient authors, they would talk of these massive catastrophes. But the important thing is, uh, they were presuming that after the catastrophe, uh, humanity would return or recover, uh, civilization would kind of just happen again.

• 24:04Speaker 00

  Um, so, you know, it's, it's, that's not an existential risk. Cause the important thing in existential risk is the irreversibility. It's the fact that, uh, our potential is lost forever. Um, the human species is gone forever. Uh, therefore, you know, it won't ever realize that potential. Um, the very idea that a species could disappear and never return, uh, is a really modern idea as well. Like, all of these ancient philosophers, uh, spent most of their time thinking that, uh, you know, if a species disappears, um, it doesn't matter, it'll kind of, uh, it'll, uh,

• 24:49Speaker 00

  continue to exist elsewhere, or it will just return at another time. Um, so you can find, you can find them saying stuff like this. There's one in particular, uh, Lucretius, a Roman philosopher. Um, and he talks about, um, he talks about, you know, uh, uh, the earth is aging, right? Uh, it's, it's kind of falling apart. It's, uh, losing its, uh, you know, kind of, um, it's losing its, uh, life force is kind of the way he talks about it. Um, and so again, you might think this looks like, oh, he's talking about something like entropy or, you know, but he's not, because he says, nothing in creation is the only one. Uh, nothing in nature ever is ever destroyed, because, um, if it's destroyed here,

• 25:31Speaker 00

  it will reemerge at some other point in the vastness of the cosmic infinity. Um, and so this is a really important idea, uh, that took a long time to dismantle, is this, uh...

• 25:44Speaker 00

  This confidence that uh nothing can ever really be lost from nature. Uh so be that species, be that the dodo, or be that basically value. Uh so the recognition, the acknowledgement that uh value and the potential to create value uh can be irreversibly lost. Um yeah, it's a really modern one. Uh and like I said, you know, that's super important uh because, you know, um you know, we often talk about these huge revolutions in the way we think, uh you know, um Darwin's theory of natural selection, i- i- i- it completely changes how we relate to ourselves in this universe.

• 26:27Speaker 00

  Uh you know, it completely changes how we think about what we are and what we can do. This is another one that hasn't really been written about or noticed yet is uh, you know, this recognition that yeah, if um you know, if, if, if humanity is lost, if uh we lose our potential, if we destroy it uh through our own folly or uh, you know, through insignificant precaution, uh you know, insufficient precaution, um that's it forever. That's a really important idea and it's, yeah, it's a really new one as well.

• 26:59Chris Williamson?

  What was the first existential risk that humans faced as a species? Was there something in, in Paleolithic ancestry where we got down to a population of 12 or something like that?

• 27:11Speaker 00

  Yeah, so there's this theory and it's, I stress theory, uh there's been um there's been some work more recently that's questioned, put, put this into question. But there is this uh, this, this idea that uh at some point, um I think it's seventy five thousand years ago. Uh there was this uh super eruption, uh is the Toba uh super volcano.

• 27:36Chris Williamson?

  Was it?

• 27:37Speaker 00

  Toba, I think it's in Indonesia. So this, and we're talking ginormous, like, you know, uh, there's a cool graph. Uh, if you Google Toba, um, you'll be able to see it. It's, it's, uh, it's a graph of like the size of the volcanic eruptions ranked. And some of the, the, you know, the ones that are kind of more memorable, like Krakatoa, they're just like absolute, they're like pimples compared to Toba. It was huge. Um, so yeah, this absolutely ginormous, um, volcanic eruption. Uh, the theory is that it created a, um, uh, population, um, bottleneck, uh, because it, uh, the climatic format fallout, um, seems to have, you know, um, this is, we're

• 28:21Speaker 00

  talking kind of early, um, uh, you know, kind of early behaviorally modern humans. So that's like, you know, when we were kind of, you know, talking, doing culture, um, there's evidence that, you know, the pop, the population like really narrows down, uh, um, through genetics. There's this kind of evidence for this, um, uh, uh, at this point, and it kind of, you know, the dates line up nicely with Toba. Um, so yeah, again, like I said, it's been put into question more recently, but there is, you know, there is some evidence to show that humanity has come close to the knife edge before. Um, and you know, that's not surprising because ninety nine point nine percent of all species that have ever existed, uh, are now extinct.

• 29:03Speaker 00

  So it's, extinction is the rule, uh, you know, uh, survival's the exception. So, um, yeah, like, you know, I think that's, that's, that's an important, that's an important thing to know is that, you know, potentially we have come close before, uh, you know, this isn't, um, you know, this isn't, uh, you know, something that is completely unprecedented.

• 29:27Chris Williamson?

  Indonesia would have been a real hotspot as well, because that's where an awful lot of humans developed from, right? And I think it was only twelve thousand years ago that you had a different Homo species that was still existing. You had this sort of pygmy-sized miniature human species that was still existing. So seventy five thousand years ago, you'd have probably still had Neanderthals, you would have had Homo sapiens, you would have had a few others. So it potentially would have caused this bottleneck for. All of them, and then maybe we didn't make it out the other side. And I think that that probably highlights one of the real importances with regards to space exploration and just setting up a colony somewhere else.

• 30:09Chris Williamson?

  You know, we can look at Elon going to Mars and say, well, it's spending a lot of money and it's taking up. We could be spending this on inequality or on world hungers and stuff like that. But from a civilization god's eye view, this is super important. We need to get ourselves off Indonesia because if the volcano decides to erupt, and that can occur in any one of a number of different ways. We also need to make sure that there is no internet connection between us and Mars. That maligned artificial general intelligence is not allowed to get up there as well. But my point is like, that that really sort of quite nicely, I think demonstrates how precarious, because we look at that and we think, oh, yeah, but you know, they didn't have technology.

• 30:54Chris Williamson?

  And there was only a small number, you may be talking, I don't know, 50,000 humans at most. I don't know, it could be, it's not going to be more than a few million humans, definitely not. Such a small number, how could that occur? And you realize that is just an inability to judge scale correctly. That for every size that you go up, there is an equivalent catastrophe that could then completely annihilate it. So what about the first record of a human thinking about our own extinction? Was it, was there some philosopher in the, in ancient Greece that you found? Nostradamus?

• 31:32Speaker 00

  Yeah, so not, so, so, uh, all of the people from, uh, from, uh, so kind of, you know, ancient, um, classical. So we're talking like ancient Greeks, Romans.

• 31:47Speaker 00

  Yeah, they're always, when they're talking about these big cataclysms, it's always the case, um, as far as, as far as, you know, in my opinion, that, uh, these are kind of false friends. So this is a, this is an idea that I take from linguistics is, uh, um, there are words where, you know, in one language, uh, they, well, they sound the same in two languages, but mean very different things, right? Um, so I think a good example is das Gift in German means poison. So.

• 32:18Chris Williamson?

  Not a gift.

• 32:20Speaker 00

  Not a gift, yeah. Um, so I think you get false friends in, uh, in, in concepts as well. So, um, you know, uh, like I was saying earlier, Plato talking about these huge, uh, the word he uses is conflagration. So this is like, you know, fire burning up the world. He says, the whole surface of the world has, uh, suffered these conflagrations that have wiped out humanity. Um, but, you know, then goes, oh, as it all happens to humanity again in the future. So it's just part of this cycle that I was mentioning. That's a false friend because on the surface, if you just read that sentence, it's like, oh, Plato had this kind of, you know, nascent, uh, you know, uh, theory, this incoherent theory of, um, conflagration as a, as a risk, uh, you know, a civilizational risk, but it's not because, again, he's just confident.

• 33:05Chris Williamson?

  Yeah.

• 33:06Speaker 00

  Exactly, exactly, exactly. So, um, yeah, all of these pre-modern thinkers, uh, it's, yeah, there, there's always this strange sense where they talk about these, you know, huge disasters, these global, um, you know, um, uh, burnings or freezings, uh, those are the things that seem to attract our intuitions. We love fire and ice. Um, uh, but it's always, yeah, within this confidence of, um, you know, everything lost will later return. Uh.

• 33:38Chris Williamson?

  Who gets the hat then? Who's the, who's the first person that does it properly?

• 33:42Speaker 00

  Um, um, so there's this kind of, um, it's, it's very gradual. Uh, so we're talking, um, you know, the scientific revolution, that kind of swings around like, uh, fifteen hundred, sixteen hundred, uh, people start to think scientifically. Um, you might think that that would kind of instantly knock all of this kind of naive thinking, um, out of the way, but it really doesn't. You know, there's this thing, uh, when you work with like the history of ideas, uh, there's this really persistent, um, thing that I call conceptual inertia, uh, where, you know.

• 34:15Speaker 00

  Old ways of thinking persist into new frameworks and do so very stubbornly often. And so I'm sure we're suffering from a lot of it now, but hopefully our forebears, if we make it out of the precipice, will look back on us and see how, you know, see how naive we were. But yeah, so this conceptual inertia in the scientific revolution is, you know, you have this big shift in worldview to do with people realising that we're not the centre of the universe. So this is the Copernican revolution. The mediaevals thought that, you know, that the sun goes around the earth. Kepler, Copernicus, these scientists, they completely changed that. So you might think, people are thinking differently now.

• 35:00Speaker 00

  They might start to think that, you know, if humanity disappears, that would actually be it, that would be really bad. Because we're this one planet in this, you know, cosmic, cosmic void, this massive expanse. They didn't. So, you know, you get people, so there's some of the original, the first scientists, so Edmund Halley, Robert Hooke, these are the, these kind of pioneers of science. They started to think geologically about the history of the planet. They started to say, oh, there are these huge catastrophes in the past, like massive earthquakes, that probably completely reorganised the surface of the earth. But, well, actually, Halley's very interesting because he says,

• 35:44Speaker 00

  you know, every time this world wiped out civilization, it will have reemerged. It's such a shame that we've lost all of the achievements of that previous civilization. They might have had this learned age where they reach peaks of knowledge way higher than us. It's just a shame that, you know, we'll have to kind of catch up to them. So even though they're thinking geologically, scientifically, naturalistically, there's still this obstruction here. And the other way that this is expressed, and this is a really important one, is, and this has to do with this Copernican revolution, this revelation of how huge the universe is. So.

• 36:24Speaker 00

  People would look at these other planets, uh, well, not actually look at them, that happened a lot later, but they would theorize about uh how, you know, all these other stars, these are, you know, pricks of light in the sky. Uh, they're other stars and they must have planets like our own revolving around them. Uh, and they thought, oh, it would be an awful waste of space, uh, if they weren't populated. Um, so, yeah, they probably have, uh, all of them probably have aliens on them and people also presumed that these aliens were actually basically humanoid. Uh, or, um, you know, were interested in values that were like ours.

• 37:00Speaker 00

  Uh, so you have this really high confidence, uh, that, you know, humans pretty much exist everywhere, or if not humans, the values that matter to us. Uh, so, you know, there wasn't, there wasn't really a sense of there being any kind of, um, uh, possibility for, you know, wasted opportunity for values or kind of vacuums where there is no value. Uh, value was thought to exist throughout the whole universe, it basically like fills it up, you know? Um, this is this idea called the principle of plenitude. Um, so you had to dismantle that before you get this person who gets the hat of being the first person to go, oh, maybe this would really matter.

• 37:42Speaker 00

  And yeah, it's not as simple as that, unfortunately, you know, there's no one person who goes, oh, uh, if humanity's gone, that's it forever. Um, you know, uh, you find people kind of at the beginning of the seventeen hundreds, uh, particularly, um, towards the end of the seventeen hundreds, starting to play around with the idea of, uh, human extinction. Um, and this idea that, you know, we might disappear, the exact same thing might not reemerge. Um, so there's, you know, and this is when people started to dig up bones of, you know, uh, prehistoric beasts. Um, and started to realize that actually there are animals that, you know, have disappeared and have gone forever.

• 38:25Speaker 00

  So again, this irreversibility starts to kind of, you know, trickle into the picture, um. But still, I mean, a really good example is from Denis Diderot, this like, you know, kind of massive minder, the French Enlightenment. And, you know, he was at this dinner party with his other friends, you know, they were probably talking about regicide and guillotining the king. But during one of these conversations, one of them asks Diderot, who, you know, had these kind of, you know, quite iconoclastic, materialist theories. So, materialist there means that, you know, he was just being quite mature in saying, maybe spirits and supernatural things don't exist.

• 39:09Speaker 00

  So they asked him at this dinner party, they said, can humanity go extinct? And Diderot said, yes, it can.

• 39:20Speaker 00

  But, you know, it would just re-evolve again in, you know, however many millions of years.

• 39:28Speaker 00

  So, yeah, you know, it's, it's, it's this kind of gradual process. I mean, funnily enough, and let's, actually, I'm going to give you a definitive answer here. The person who gets the hat is one of Diderot's friends. He was another French philosopher called Baron de Olbach. And he actually said, he said, we cannot be sure that all these other planets contain humans and that humans are therefore the natural end of all kind of, all evolution, all natural history. Obviously, evolution not in the Darwinian sense back then. But he said that. And then said, you know, thus, therefore, if our planet was, you know, knocked off its course, that could be it for humanity.

• 40:11Speaker 00

  So, yeah, I think, you know, let's give him the hat. He's the first person to, you know, say, yeah, A, ahead of evidence, we can't just assume that humans are everywhere and values, human values are everywhere. B, we can't assume that they're the end of everything, the purpose of this whole cosmos that we live in. And C, therefore, we can't be sure, again, ahead of good evidence, that if we screw it up, you know, something else will just re-evolve like us.

• 40:42Chris Williamson?

  I think that that really explains nicely the answer to the first question that I gave you about why is it important for us to look back? And Given that we're in the scientific, post-scientific world, utilitarian rationalists, I can Scott Alexander my way to an Eliade Zukowski blog and I understand my cognitive biases. And like, we believe that we have reality in our grasp, but the intellectual inertia, cognitive inertia that you just mentioned there, the principle of plentitude as well, this presumption that everything will be okay. When you combine that with, is it scope insensitivity?

• 41:23Speaker 00

  Scope neglect.

• 41:23Chris Williamson?

  Scope neglect, that's it, sorry. That big things are really, really hard for us to work out. And as you scale up small things to big things, the death of a person, the death of two people doesn't feel as bad as the death of one person. And the death of a million people doesn't feel a million times worse than the death of one person. So when you combine all of that together, I think it starts to get us to a place where those of us who want to force the existential red pill down everybody's throats, actually start to understand why it might be a little bit of a big medication to swallow. Wasn't it Thomas Jefferson was famous for believing that some animals or that no animals ever

• 42:05Chris Williamson?

  went extinct? And you think Thomas Jefferson's like, he's modern history, you know, there's like drawings of him and paintings and stuff like that. And he was part of a country that's still around now.

• 42:17Speaker 00

  Yeah, yeah, definitely. So, I mean, if there's anything that my daily work is, it's just cataloging the vast library of how often we're wrong. And not just, you know,

• 42:35Speaker 00

  normal people, but these huge, these huge great minds. So yeah, like, you know, Thomas Jefferson, yeah, he's a great example. So, you know, and this goes back to what I was just talking about with, you know, in the 1700s, people started unearthing these huge bones of, you know, fossilized bones of unknown beasts. And, you know, prior to this, I think this is an interesting context, actually. Prior to this, scientists and natural philosophers presumed that fossils were, they didn't think that they were the kind of impressions of prehistoric animals.

• 43:17Speaker 00

  That was, again, I think this is a great example of what we were talking about earlier of, you know, it seems so obvious now that fossils are, you know, kind of the evidence of prehistoric animals. Like, you know, when you're a kid and you watch Jurassic Park, like, you understand that. But this took centuries for people to figure out. So the medieval theory, so during the Middle Ages, what people thought was that these strange animalistic imprints in rocks were actually nature playing jokes on us. So they had this idea of the kind of scale of being, the great chain of beings. So it's that everything is in this kind of ordered hierarchy

• 43:59Speaker 00

  from, you know, rocks to plants to oysters to monkeys to man. And they thought that fossils were evidence of where rocks had like basically become upstart and wanted to, you know, jump above their, their, their, their office, basically. So, you know, this is, this is, this is how people thought and, you know, it's, it's, I, I, I find it really charming and I love it, but you know, it's, it's, yeah, again, people can be so wrong for them, you know. Yeah, we're, we're often very wrong about a lot of things. So this is how people thought for a long time about these fossils. Da Vinci was one of the first people to go, hang on, maybe these are impressions of, of, of animals from, you know, the deep past.

• 44:42Speaker 00

  And yeah, so, you know, people used to kind of study, it was mainly shells, like small, you know, small, um, small fossils, that was very easy to say, you know, this animal,

• 44:58Speaker 00

  we might not see it in our kind of vicinity alive, we might only have fossil evidence of it, but it probably exists somewhere else in the world. And that was how people, you know, for a long time, got around to the fact that we had evidence of fossils. But we don't want to accept species extinctions or the possibility that nature could irreversibly, again, irreversibly, lose any part of it. Uh. And so this was this kind of, you know, tricky maneuver that all these very clever scientists took to deny the possibility of extinction of anything. Um, you know, yeah, then people started finding mammoth bones, mastodon bones, uh.

• 45:37Speaker 00

  And people started to accept, scientists started to accept that perhaps species extinction was a thing that has happened and will happen again. Um, and this was in the late seventeen hundreds, uh, Thomas Jefferson. Yeah, he still thought this, even up until the point when, uh, you know, scientific consensus was reached. So it's like in the seventeen eighties, he was still writing letters to people very confidently claiming that mammoths uh, still exist in the uh, kind of unexplored regions of the Americas. So we just need to go and find a mammoth and then, you know, we don't need to worry about extinction anymore. Um, so, so, yeah, yeah, no, you're right. You're absolutely right to link this up into, uh, you know, the kind of

• 46:20Speaker 00

  prevalence of bias and wrong think, uh, yeah, it's, it, you know. I want to give this hopeful message of, you know, uh, how far we've come and how important all that is, uh. But at the same time, you know, you only reach that by seeing how, how, you know, vastly wrong we often are. Yeah.

• 46:39Chris Williamson?

  Every age has probably had its Cassandras, right? The, the people that were certain that the end was on its way. And you highlight some differences between extinction and apocalypse and between prophecy and prediction. Can you lay out how all of this works for us?

• 46:58Speaker 00

  Um, um, yeah, so, um, I guess listeners might have already thought, you know, what's this guy talking about? Uh, people have been thinking about the end of the world forever. Uh, have you, I often

• 47:10Chris Williamson?

  Have you not read the Bible?

• 47:11Speaker 00

  Exactly, exactly. I often get that comment. It's, uh, yeah, it's, it's one of the more common, uh, comments is

• 47:17Chris Williamson?

  What is Judgment Day? Yeah, exactly.

• 47:21Speaker 00

  Yeah, has this guy not read the book of Revelations? Um, so, yeah, the, the claim I make is the apocalypse is distinct from extinction, uh, and

• 47:31Speaker 00

  What I mean by that is that, you know, you look at um the end of the world as it's presented in religious traditions and mythological traditions. Uh, and often uh it's seen as the consummation or the fulfillment of the moral order. Uh, so what I mean by that is think of um, think of Judgment Day, the Christian version. It's it's just in the words Judgment Day. It's the the revelation of how God thinks everything should be. So it's the consummation of morality. It's not like, you know, it's not anything bad. Even though we might be, it might be very inscrutable to us mere mortals, uh, we might not be able to understand it fully.

• 48:15Speaker 00

  You know, God's decree, that tribunal is ultimately the right decree. So, you know, another image that um pops up is this idea of sorting the good from the bad. That's what the end of the world Judgment Day is, kind of this point where uh, the good from the bad is all fully sorted, everything's in its right place. And that's the end. And, you know, then the curtains can close. Um, that's not actually a bad thing. It might be the end of time, but it's, you know, it's actually really great. Like, uh, like I said, morality is fulfilled in this uh instance. So in the modern naturalistic scientific idea of extinction, it's completely different. It's not just,

• 48:54Speaker 00

  you know, uh, a new version of that old idea of apocalypse. It's actually a contradictory concept. Because instead of the um, the ultimate uh fulfillment of morality, it's the irreversible frustration of it. Um, at least, you know, kind of human morality and and going back to this point about aliens and other planets, uh, as far as we know, we're the only uh animal that follows ethical argumentation that's able to, you know, kind of think about um moral reasons and what should be and what shouldn't be. Um, so, you know, if we're gone.

• 49:34Speaker 00

  All of that is frustrated, potentially irreversibly. And that's really important, you know, those are completely contradictory ways of looking at this. So there's a pithy way that I like to put it is that, you know, apocalypse supplies a sense of an ending. Whereas extinction anticipates the ending of sense. So it's this idea that, you know, meaning and purpose are irreversibly frustrated within this vast physical cosmos, this vast silent physical cosmos that continues quasi purposely

• 50:13Speaker 00

  without us. And yeah, you know, as a final point, that's also just a really simple, another point to make, a really simple one is that often in religious apocalypses, the physical cosmos does not go on without us, it ends with us. Or it's again nested in these cycles. So a lot of eastern kind of ideas, so the Buddhist apocalypse, it's, you know, this cyclical, you know, the world ends, it gets reborn, it burns, it gets recreated. So yeah, you know, nothing's at stake in apocalypse, whereas in extinction, everything is at stake.

• 50:49Chris Williamson?

  It's like an egotistic metaphysical Copernican view of ourselves, right? That the earth is the centre of the universe, but not only that, we are the centre of the universe. And not only are we the centre of the universe, we are the bookend to the universe because without us, what is the point of the universe? But oddly, that line of reasoning, without us, what is the point of the universe, is still what we're continuing forward now. The two potential answers that we have to the Fermi paradox of where are all the aliens? One is there aren't any, it's just us. The other is they're out there. Both of them are fascinating, both of them are terrifying, but they do have, slightly presuming that the aliens that we're talking about

• 51:34Chris Williamson?

  also have morality and the ability to step into their own ethics and stuff like that.

• 51:39Chris Williamson?

  The differences are quite profound because it is the difference between being every other semi sentient being, bottlenose dolphins and bonobos and stuff like that, isn't crew aboard spaceship Earth their cargo. We are crew. We can affect the direction of what is going to happen. We can save the other animals. We can allow them to exist at greater and greater levels of comfort and of bliss and of happiness, right? And we can also do that to ourselves and then we can scale that across the universe.

• 52:21Chris Williamson?

  It's so interesting to see this apocalyptic approach. The

• 52:28Chris Williamson?

  it is, it very much is a very ego driven sort of Copernican centric, this human centric view of everything. And why wouldn't it be that way? When you're told that God built the world and the universe in six days and the most important thing that he built was on the sixth day and it was the humans, et cetera, et cetera. All of your culture, all of your stories, all the narratives that you've been given are telling you, this is why everything is here. You are why everything is here. This is how special you are. And

• 53:02Chris Williamson?

  I think this might also highlight why I love existential risk so much, that it's the same as looking at the night sky. It provides an equal amount of awe and dread. And it reminds me that the universe very, very much is indifferent, whether or not we continue going on. And it is our stone to roll up the hill if we want to do it.

• 53:26Chris Williamson?

  And yeah, upon realizing that, I think that also probably highlights why the denial of death thing from Ernest Becker kind of gets macro aggregated across people with this. It's a very uncomfortable topic to think about because it reminds you that no one's coming to save you. No one gives a shit. Nothing cares about whether or not we continue to exist or not, except for us, which means it's all on us. There is risk, there is responsibility and the book stops with us.

• 53:57Speaker 00

  Yeah, yeah, I couldn't, I couldn't agree more. If there's one, if there's one major theme of what I've written on this history, it's the, it's breaking the spell on that kind of wishful thinking. Where we allow what we want the world to be to contaminate our theories of what the world actually is. So really broad brush generalization and being unfair to lots of very clever people that came before us and whose shoulders that we stand upon. But the pre-modern worldview often, you know, isn't, often doesn't really even think that there's a distinction between ethics and physics.

• 54:40Speaker 00

  What I mean by that is, you know, take for example, the medieval cosmos. You know, this is the pre-Kopernican one, the idea of Earth in the centre. There are these concentric nested spheres and not at the edge, there's this primordial, it's the outer sphere. That's where God lives, that's where all the value, the best stuff is.

• 55:02Speaker 00

  All of those spheres is populated with hierarchies of angels. You know, the whole thing, there's value suffusing the whole thing. But the very structure of the cosmos, right? The whole, the whole structure is the moral order. So that's what I mean by there's no distinction between ethics and physics there. I perceive, you know, part of the, what kickstarted science and, you know, not just science has this, you know, disinterested, objective endeavour of finding out, you know,

• 55:37Speaker 00

  how things hang together, how the facts hang together in the broadest possible sense. But this newer idea of how we can then, knowing those facts, get our values to fit together with this picture in the broadest possible sense. So I guess, I mean, what is to be done? You know, we've learnt a lot about the cosmos, the objective, physical, you know, uninterested, unresponsive cosmos in independence of us. We've learnt a lot about that, but now we're asking this question of how do our values, how could our values? Uh, fit into this. And yeah, I see this is, this is all part of this, um, this picture. So yeah, you have to kind of, um, basically shake off that wishful thinking of thinking that,

• 56:22Speaker 00

  you know, in independence of what we do, the universe is just a great place and, uh, you know, kind of aligns with our values no matter what, um, which I think is the default way of, um, of, of human thinking. So, you know, uh, there's this idea, um, of, uh, in the philosophy of science of folk psychology, um, where, or another word for it is the manifest image, um, as opposed to the scientific image. Uh, the manifest image is, you know, our picture of the world as filled with colors, intentions, emotions, uh, all the things that we're used to on our daily, daily lives. The scientific image is this really barren alien place that's made up of atoms,

• 57:07Speaker 00

  electrons, uh, you know, subatomic forces. Um, you know, you have to realize that distinction and the fact that the way the world actually is, is not the way we want it to be, or it should be, or it ought to be, uh, to then reintegrate and think of, well, okay, we've realized that. We've woken up, we've broken the spell. Uh, how, what do we do next? Um, how do we make this world, uh, that, you know, just is the way it is independently of us? How do we make it into the one, uh, that we want? Uh, or not just that we want, but that would be worthy, uh, in some, uh, you know, meaty and moral sense. Um, and yeah, you, this wide scale

• 57:50Speaker 00

  history that I try and tell of us waking up to the possibility of extinction, it's this, you know, uh, it's this kind of landmark event in that. It's realizing that, you know, um, Everything rests on us, not because we're the center of the universe, but because the universe simply doesn't care about us.

• 58:08Speaker 00

  But strangely enough, that also kind of recentralizes everything upon us. You know, until we find evidence to the contrary, and I hope we do, I I I'm very hopeful, you know, I I I want SETI to be, you know, a successful endeavor. I hope there are wiser beings out there than us. But we can't just act as if there are ahead of that evidence. So, yeah, it's this, you know, it's this strange historical dance between disillusion and, you know, that mature.

• 58:44Speaker 00

  That mature recognition of, yeah, I feel alienated by the possibility of extinction in a way that, you know, the version of me that lived 500 years ago just simply couldn't be. Yeah, I'm alienated by that, it terrifies me, but this is the thing I often say is, you know, if if I'm hurtling towards a cliff edge, if I'm driving, you know, 60 miles per hour towards a cliff edge, I want to know where that cliff edge is. Rather than just wishfully thinking, oh, yeah, I'll, you know, the car will be fine, I'll be fine, there's no cliff edge, whatever. So I see, yeah, another way you could put this is, you know, waking up to extinction is kind of a Santa Claus isn't real moment for the human race.

• 59:24Speaker 00

  And, yeah, you know, it might be disillusioning, it might be upsetting, it might be alienating, but it was something that we needed to do if we were to have a future. So, yeah, I think it's, again, like I say, it's a massive achievement and

• 59:41Speaker 00

  you know, we need to keep that in view with the future, because I feel that, you know, it's so easy to be dis- to be disillusioned with human potential. To focus on, to focus on the kind of

• 59:56Speaker 00

  The atrocities of the recent past, um, and think that that, um, think that that there's an inevitability that that will colour the whole future. Um, so, you know, we need to, um, I'm not saying that we don't, we, I'm not saying that we should forget, you know, th- th- these things, but, um, yeah, I think that there's often, there's often, especially when we talk about the space colonisation discussion, there's often a kind of, I refer to as a geocentrism about history, this idea that we will repeat, just repeat everything, um, you know, all these kind of mistakes of the past, that's just what spreading outwards will do. C.S. Lewis has this

• 1:00:39Speaker 00

  great quote where he's, you know, says, "I feel sorry for the aliens" is basically his sentiment, because, you know, us horrible sinners are going to go out and, you know, ruin the galaxy. Um, you know, that's a form of geocentrism, just like, uh, thinking that the Earth is the centre of the universe, thinking that our, um, history colours our future, and of course it does, in an important sense, but the work is, you know, keeping an eye on the places where real progress has happened and seeing the capacity, um, because, and this goes back to the point of, you know, uh, dolphins and, uh, you know, marmosets and, you know, all these other very incredibly intelligent animals that we share this planet with, um, is, one way of talking about, you know,

• 1:01:24Speaker 00

  humans are the only animal that responds to ethical argumentation, that can sound really abstract, really, um, kind of up in the clouds. I think a good way of making that concrete is we're the only animal that's ever corrected itself, uh, it's ever thought, "Oh, what I previously thought was wrong about what I should do or what I think about the world." Um, so, yeah, we've been really wrong, but we have that capacity to correct ourselves, and that's the capacity to make the world a better place. So, um, you know, yeah, I really like your, um, the cargo and, uh, and crew. Uh, I think, Toby, this is a way that Toby Ord puts it, but, um, is that, you know, animals are like moral patients, we're moral patients but also moral agents. So,

• 1:02:06Speaker 00

  you know, I see there's a, uh, you know, the crew cargo is another way of putting that. So, yeah, I, you know, um, history can be hopeful, yeah.

• 1:02:17Chris Williamson?

  I don't know what it is about this time at the moment. Everybody that's listening and yourself knows, and you have very cleverly sort of managed to evade the social justice precipice of saying that there aren't any problems right now and that we don't have anything to fix in your little sort of last monologue there. And it's because at the moment, there are, to me, it would appear that there is an obsession with injustices, and many of them are indeed injustices that need to be fixed. But I think it discounts so much of how far we've come. It discounts this view that you've given us of just how backward our views were when

• 1:03:00Chris Williamson?

  only Thomas Jefferson was on the planet not long ago. And yet, when we have these times of crises, when we have times of real, serious, bloodshed and apocalypse and concerning catastrophe, people have to centre their values. In the time of a real crisis, we focus our values. In the time of no crisis, we create our own. And I think that the problem right now is that people think there is no crisis. I don't think, because the existential risk, the potential for human catastrophe and the permanent and irreversible stop of the human pursuit, isn't there at the forefront of our minds?

• 1:03:42Chris Williamson?

  If we found out that there was a meteor that was heading toward Earth and it was going to hit us within the next 1,000 years, and we had it with certainty and the news story said the same, I think everybody would live life in a very different sort of way, some ways better, some ways worse. But I think it would bind us together as a civilisation. I think it would stop us from focusing on things that in the grand scheme of human civilisation don't matter. As Toby Oates says in The Precipice, we are shuffling currently along this cliff edge. It's real precarious. And we only need to do this for a little bit of time. I'm not saying that we need to completely dispense with trying to fix the wrongs that are going on in the world. What I'm saying is that right now, there is a very, very important job that needs

• 1:04:27Chris Williamson?

  to be done by all of human civilisation. And it is a game of whether or not we get past it or we don't. And if we don't, it doesn't matter how much social injustice you've fixed. What I think that's led to, and everybody will know this sentiment at the moment, is this oddly... Homo deprecating view that the world has where we talk about, listen to people talk about ecology and the environment, like as if we go out of our, as if dropping a can on the floor is because you hate the world, as if because the fact that you don't drive an electric car or you don't get the bus to work or you don't get the cycle to work is because you actively hate Mother Nature and she's rebelling against us and we just

• 1:05:11Chris Williamson?

  start personifying and adding narratives like it's the fucking 2000 B.C. Do you know what I mean? Like, we're just layering all of this real primitive thinking that we've got about the sacred and the profane. Meanwhile, Cardi B is on stage singing about a WAP and no one cares. So, all of this bizarre human hating that's going on both within culture and within certain subgroups within people, I think that a lot of that would be fixed if we understood just how close we are to complete annihilation of everything that you care about, of everything that your genetics down the line can ever care about, and of the ability to save the opportunity to step into programming. I think that's

• 1:05:55Chris Williamson?

  what you said there. The opportunity and the ability for humans to step into their own programming, to redirect the direction in which they are going, is what matters. And, um, man, I really, really hope that the work of the guys at the Future Humanities Institute continues to blow up. We need, we need a Greta Thunberg of existential risk. And I mean that in, in a very, very real way. Like, we need a front of time magazine, social media savvy, great speaker that gets the, the world behind them because it is the biggest question that we all have. One thing that I haven't heard you talk about,

• 1:06:37Chris Williamson?

  and I'd love to know your opinion on this, given the fact that you've done such a broad view look over this subject, over the past of history, just how important do you think this period of the last 30 years with Parfit and Bostrom and Oud, how much, if you were drawing a graph in terms of progress and understanding, how much of a hockey stick has that been? Is it linear or has that really been a marked jump in our understanding around existential risk?

• 1:07:04Speaker 00

  Yeah, I think if you wanted to, uh, yeah, if you, if you had, if you had to, you know, depict it pictorially, I think it has been, yeah, there's a step change. Um, and again, you know, this is one of my senses from studying the history of ideas and the history of intellectual progress. Is, um, that, that progress is often lumpy and, you know, kind of uneven and there's, you know, it was kind of punctuated, right? Um, and that's often because these ideas require lots of smaller sub ideas to come together. So, you know, you have it bubbling away, these sub ideas, but they're not, you know, converging and then bam. And I think we've just gone through, um, we've just gone through a point like that.

• 1:07:46Speaker 00

  So, I mean, I'm hopeful about, um, I'm not despondent about the fact that, you know, not enough people are talking about existential risk. Uh, I mean, there's one thing to be sensitive of is like, you know, thinking about the long term future and how all of our potential vastly outweighs, you know, uh, you know, the present and all that stuff. There's this issue of fanaticism that the community around this is very aware of. So it's this idea, um, that, you know, uh, basically when you're thinking of these, you know, huge potential ahead of us, uh, it can outweigh, you know, any sensible decision at the moment because we'd want to vouchsafe that regardless of anything.

• 1:08:28Speaker 00

  So, you know, this is, this is something that, you know, the community is very self, self-conscious of. It's an issue within, you know, this tech, very technical ethical, uh, discussion. Um, so that's, yeah, that's one thing to, to be, uh, aware of. Um, yeah, another thing is I'm hopeful because, um, uh, it's, it's, it's, it's, it's new. That's why there aren't many people that are talking about it is because it's new. Uh, these ideas haven't been around for long and it often takes a while for ideas to really catch on and to start to trickle through the, you know, relevant, uh, streams, the, you know, the relevant institutions. Um, so, yeah, I think it's, it's new, it's, it's, it's new and that's why, um, and, um,

• 1:09:13Speaker 00

  so I am hopeful that the, the discussion will continue and I, I don't think that, um, uh, you know.

• 1:09:21Speaker 00

  People talking about other things like, you know, Greta Thunberg, she's not like taking resources from people to talking about existential risk. So, and you know, climate change is actually like, it might not be an existential risk in the sense they'll,

• 1:09:35Speaker 00

  you know, destroy Homo sapiens, all of them, but it could actually irreversibly make our future poorer and less well off. So, you know, it's, I think it has to, you have to, you have to balance all these things. Yeah.

• 1:09:51Chris Williamson?

  I agree. Thinking about something that you brought up, which I'd read in a book ages ago, and I'd totally forgotten about, and is absolutely fascinating. Can you explain what the doomsday argument is, please?

• 1:10:05Speaker 00

  Yeah, I'll, I'll give it a try. So, um, there's, there's, there's different versions of it. Uh, and interestingly, uh, appear around the same time independently. So this is another thing in the history of science. I'm gonna nerd out about the history of science for a second is, uh, often really great ideas, um, people independently arrive upon them around the same time.

• 1:10:29Chris Williamson?

  Why do you think that is?

• 1:10:31Speaker 00

  I think it's, again, similar to what I was talking about earlier is, uh, there are all these subcomponent ideas that require, that are required to then reach, um, the big theory. Uh, so natural selection required an awareness of species extinctions in the past. It requires an awareness of population dynamics, blah, blah, blah, blah, blah. Uh, so Darwin came up with it, but it's been, uh, after Russell Wallace, uh, almost at the same time, completely independently. Anyway, doomsday argument. Um, basically, uh, you should, applying the Copernican principle, the principle of mediocrity, um, we should

• 1:11:09Speaker 00

  assume that we're in a, uh, in a, um, unexceptionable, uh, unexceptional, uh, place. Uh, and that applies, that principle can be applied very broadly. Uh, and so when you apply it to our position in human history, uh, so imagine a reference class of all humans ever born, uh, ordered from, um, ordered temporarily. So obviously it's arbitrary. The first human ever is arbitrary in a sense because, you know, evolution. Uh, but imagine first human ever, last human, humans ever. Um, we should, we should assume that we're in an unexceptional, unexceptional placement within that broad, broad reference class, right?

• 1:11:54Speaker 00

  Um, so given, uh, given one of the versions makes a reference to population uh, and how population has expanded and uh, if humanity survives into the long term, will expand a lot more, uh, it's far more likely that we are later on where there are more people, uh, so imagine this is a curve than we are very early on, uh, where there are barely any. So the, the analogy that's often used is, um, uh, imagine that, uh, imagine that you have an, a, a bag of, of balls, um, and you, someone, someone's told you

• 1:12:37Speaker 00

  that this bag either, either has five balls in it or a hundred balls in it. Um, and you pick the number four, uh, you would rationally presume that you've got the bag with the five balls, right? Um, so applying that reasoning to that, you know, long scale human race, uh, you have, you know, you have to, yeah, the, the, the, the conclusion is that we're probably living later on towards the end. Uh, now this gets into all kinds of thorny issues, um, uh, you know, super complicated, uh, kind of reasoning and it's, it's very interesting. Um, uh, and it's very controversial as well.

• 1:13:20Speaker 00

  So, um, yeah, I think the best place to go to find out more about it is actually the Wikipedia page. Um, I think Vox did quite a good explainer on it as well, but it's, it's, it's a really fascinating argument, um, really technical, really complicated, but, uh, and also my bending.

• 1:13:36Chris Williamson?

  There'll be a Reddit thread, so my Reddit's explained everything. Reddit's becoming like the new Wikipedia, I think. Thinking about sort of looking forward, what lessons can we learn from looking at our extinction that inform the Enlightenment mission moving forward? How do we carry this into the future?

• 1:13:56Speaker 00

  Hmm, hmm, so I think that, um, I think, and this goes back to, uh, what I was saying about disillusion is, um, we had this Enlightenment, uh, where

• 1:14:13Speaker 00

  Some really brilliant ideas were invented, some pretty bad ones were implemented as well.

• 1:14:21Speaker 00

  We had that, you know, uh, three hundred years ago, began three hundred years ago. And then afterwards, we'd entered into this um period afterwards where certain other cultural forces have start, you know, come into the picture, started to compete with that enlightenment idea of progress, of human potential, of um the capacity for reason to correct itself, to um

• 1:14:54Speaker 00

  to basically supply reasons to everything. Uh, so, you know, uh, remove the arbitrary, the unjust, the irrational from uh, not just our picture of our world, the world, but our conduct within it.

• 1:15:07Speaker 00

  We've had this kind of counter, the counter enlightenment was actually a historical moment, but I think that we're still, there's still um it's still a prominent cultural strain of these counter enlightenment, uh, you know, it can be pessimistic, it can be romantic. There are these cultural strains um around that, you know, the, the, the, the big philosophers like Nietzsche, uh, you know, um uh, some of the, the twentieth century philosophers as well, like from Sartre down to Derrida, they're all kind of playing with these anti enlightenment ideas. Now, I don't want to make predictions about uh culture because there are so many degrees of freedom that you'd be an idiot to do so, but I do feel that um

• 1:15:55Speaker 00

  that disillusion with with enlightenment and uh its capacity, so not just what it was, but what it could be, taking the good bits out of it uh and critiquing the bad bits. You know, it's kind of growing up and this is what the enlightenment, you know, the, the big dogs in the enlightenment said themselves, Kant said that it is just, uh, you know, um.

• 1:16:19Speaker 00

  Humanity using its own reason to exit its infancy. Um, this is where the metaphor comes from, actually, where you see Sagan, uh, you, you know, Carl Sagan said, uh, that we're, we're in this period of technological adolescence, um, defined by this miss, you know, this misfit, this disjunct between our might and our wisdom. Um, that's where the metaphor comes from. It actually comes from, uh, Kant and his, his Enlightenment forebears. So anyway, what I'm saying is, uh, you know, we had this period of enlightenment, then we had the kind of, uh, critique of it, which is itself part of the Enlightenment, is criticizing, uh, you know, critiquing, um, and unveiling the biases, unveiling the, the injustices.

• 1:17:01Speaker 00

  Um, and we're still kind of living through this point. And I think it's, it is an essence to use Sagan's term. It is a kind of adolescent phase where it's like, we've realized our capacities. We've realized the sheer damage we can do to ourselves and this world that we live in. Uh, and lots of people react to that with this, this continuing disillusion of this is awful. Humans are awful. Um, we can be, right? And it's, you know, again, these, these metaphors can become cheesy and they can become very broad brush and very over generalized, but I think we are going through that, you know, not just this technological adolescence, but also this kind of adolescence in our image of ourselves and what we can do within the world and what,

• 1:17:42Speaker 00

  uh, you know, what we should do, uh, what we ought to do. Um, yeah, where, you know, there's a lot of disillusion, um, and taking this really broad, broad, long-term view of human history and culture and ideas, uh, that I like to do, I see that as just a necessary step in, you know, waking up and, uh, growing up and, you know, when you're a teenager, there are times when you, you know, do something really awful and, you know, uh, screw up. Then you feel really bad about yourself. Uh, but then you learn that that's just part of this process. And so, yeah, uh, you know, real broad brush, long-term, uh, talking.

• 1:18:22Speaker 00

  I think that, you know, as a civilization, um, and obviously, you know, again, that's a big abstraction. There are lots of parts and there are lots of, uh, parts below that. But I think we're going through that phase right now. Um, and Yeah, so I'm hopeful as to what's to come, what's, what's, what's to come next. Uh, you know, we've realised how bad we can be. Uh, but now we can focus on stopping that badness from, uh, from, you know, um, coming to the fore and we can focus on what we can do, uh, and our potential.

• 1:18:58Chris Williamson?

  We are gods but for the wisdom.

• 1:19:01Speaker 00

  Who's that?

• 1:19:02Chris Williamson?

  Uh, that is Eric Weinstein, actually.

• 1:19:05Speaker 00

  Oh, right.

• 1:19:05Chris Williamson?

  And Daniel Schmachtenberger said, we're gods, we're just shitty gods, uh, which is the same thing said in a different way. One thing that's just popped into my head there, mate. Do you think that we're further ahead in terms of ethics or technology?

• 1:19:21Speaker 00

  Um, I mean, they're kind of, uh, uh, uncomparable.

• 1:19:26Chris Williamson?

  I, I, I know that that's an apples and oranges comparison, but if you were to think about, personally, to me, it, it seems like we're further ahead in technology. It seems like our, our technological power outstrips our wisdom. We've made a lot of progress. We made a lot of progress with ethics, with understanding what good is and how to act in the world. Um, but it seems like we're able to scale technological progress much more quickly than that.

• 1:19:55Speaker 00

  Hmm. Yeah, yeah, yeah. So caveat aside of, you know, incomparability. Um, broadly, yeah, we are definitely ahead in, uh, technological, uh, progress. Um, this is actually, this is actually a point that Parfit makes, you know, to loop back to the beginning. This is a point that Parfit makes, um, in this argument about how bad extinction would be. Um, is he points out that applied ethics, so, um, you know, not just thinking about what value is and arguing about meta ethics, which is, you know, what value is and how we can define it. We're arguing how we can actually implement our ideas of value into the world. So like, oh, you know,

• 1:20:36Speaker 00

  Making ethics effective is actually really young.

• 1:20:43Speaker 00

  And so as part of his wider argument that human extinction would be so awful because we have this vast future ahead of us if we get things right. He says, you know, you can be a kind of aggregative about it and say, you know, there will be so many zero, zero, zero, zero future human lives. But he says there are also these intrinsic goods, these intrinsic values. So like art, science, right, knowing about the world. But then he points to this other one is ethics. And he says, this is the most immature field of them all. Is it's only since the Enlightenment that people have

• 1:21:28Speaker 00

  been doing secular ethics. So thinking about morality without God breathing down the back of your neck. And it's only even more recently than that, the people like Parfit himself have been dedicating their entire lives to the pursuit of this. And trying to, you know,

• 1:21:51Speaker 00

  create, you know, get that low hanging fruit when it comes to, you know, moral progress.

• 1:21:56Chris Williamson?

  So That was the words that I had in my head. The low hanging fruit.

• 1:22:01Speaker 00

  So I think, I think, yes. We are in this stage where our, you know, our might kind of outstrips our, our wisdom. We are in our technological adolescence or the precipice, you know, however you want to describe it.

• 1:22:18Speaker 00

  But that doesn't mean that there's, you know, again, you know, I think that there's big potential for for moral progress and, you know, people are working on these things like applied ethics is booming. There's this effective altruist movement where people are, you know, thinking, how best can we use our resources? There's, you know, a kind of offshoot of that. More recently, long termism is this, you know, idea. And I, the precipice, Toby also precipice is kind of the, the, I guess, the founding text of this, but and there's. Lots of work coming out to the Global Priorities Institute, which is kind of the sister institute of the Future of Humanity Institute on this long-termism. Yeah, and

• 1:23:01Speaker 00

  it's about, you know, how can we, how can we do applied ethics? And so, yeah, I, I think, you know, our technology is outstripping our, our wisdom right now, but there's, there's, there's low hanging fruit to be picked.

• 1:23:15Chris Williamson?

  There's some good players about to run out onto the pitch for the applied ethics team as well. I hope so, man. I really do. I think, um, I think it's very timely for you to, to release a book about the, the history of existential risk. I really do. The precipice by Toby Ord is a fucking magnum opus. I absolutely adore that book. Superintelligence, fantastic. What is it? Human Compatible by Stuart Russell, which came out last year. That's also fantastic. But this, Thomas Moinen, Ex-Risk: How Humanity Discovered Its Own Extinction. Dude, you've done a really, really good job with this. The number of footnotes is absolutely

• 1:23:53Chris Williamson?

  terrifying. It's, it's completely ridiculous. And for anybody that's enjoyed this conversation today, it will be linked in the show notes below. I've also added this to my Amazon reading list because that's how much I enjoyed it. So go and check this out if you want. Anywhere else that you want to send people?

• 1:24:12Speaker 00

  My website is ThomasMoinen.xyz. I keep that updated with like short essays that I do. So yeah, I think that's it.

• 1:24:22Chris Williamson?

  Perfect, man. The end of civilization hasn't occurred during this podcast, so we have managed to make it through. Thank you for coming on.

• 1:24:30Speaker 00

  Cheers. Thanks a lot for having me.