How Long Could Humanity Continue For? - Will MacAskill

Will MacAskill is a philosopher, ethicist, and one of the originators of the Effective Altruism movement. Humans understand that long term thinking is a good idea, that we need to provide a good place for future generations to live. We try to leave the world better than when we arrived for this very reason. But what about the world in one hundred thousand years? Or 8 billion? If there's trillions of human lives still to come, how should that change the way we act right now? Expect to learn why we're living through a particularly crucial time in the history of the future, the dangers of locking in any set of values, how to avoid the future being ruled by a malevolent dictator, whether the world has too many or too few people on it, how likely a global civilisational collapse is, why technological stagnation is a death sentence and much more... Sponsors: Get a Free Sample Pack of all LMNT Flavours at https://www.drinklmnt.com/modernwisdom (discount automatically applied) Get 20% discount on the highest quality CBD Products from Pure Sport at https://bit.ly/cbdwisdom (use code: MW20) Get 5 Free Travel Packs, Free Liquid Vitamin D and Free Shipping from Athletic Greens at https://athleticgreens.com/modernwisdom (discount automatically applied) Extra Stuff: Buy What We Owe The Future - https://amzn.to/3PDqghm Check out Effective Altruism - https://www.effectivealtruism.org/ Get my free Reading List of 100 books to read before you die → https://chriswillx.com/books/ To support me on Patreon (thank you): https://www.patreon.com/modernwisdom #philosophy #longtermism #existentialrisk - 00:00 Intro 00:25 Introduction to Long-Term-ism 07:35 Why Our Present Choices Matter 15:13 Possible Futures of Technology & Morals 23:49 Will’s Problem with ‘Happy Birthday’ 34:35 Preventing Human Extinction 42:50 Is Earth Over-Populated? 48:25 Risks of Technological Stagnation 59:17 The Current Focus on Climate Change 1:07:19 Extinction Vs Collapse 1:22:04 Our Goal for the Future of Humanity 1:31:42 Where to Find Will - Get my free Reading List of 100 life-changing books here - https://chriswillx.com/books/ Listen to all episodes on audio: Apple Podcasts: https://apple.co/2MNqIgw Spotify: https://spoti.fi/2LSimPn - Get in touch in the comments below or head to... Instagram: https://www.instagram.com/chriswillx Twitter: https://www.twitter.com/chriswillx Email: https://chriswillx.com/contact/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  We are at the very beginning of history. Future generations will see us as the ancients living in the distant past. What are the events that really could have civilizational trajectory level impacts? And then finally taking action, trying to figure out, okay, what can we do to ensure that we navigate these challenges and try to bring about a wonderful future for our grandchildren and their grandchildren?

• 0:24Chris Williamson?

  Given the fact that we're seeing James Webb telescope images all over the place at the moment, kind of seems like a smart time to be thinking about far flung futures and potentials for civilization and stuff like that.

• 0:38Speaker 00

  Absolutely. James Webb is making very vivid and in high resolution, an incredibly important fact, which is just that we are at the moment are both very small in the universe and also very early in it. So almost all of the universe's development is still to come ahead of us.

• 1:02Chris Williamson?

  That's wild to think about the fact, especially on our timescales, right? You know, you think about 20 years as being a very long time in a human lifespan. And then you start to scale stuff up to continents, to the size of a planet, to the size of a solar system, or a galaxy, or the universe. And it, it puts things into perspective.

• 1:22Speaker 00

  Yeah, well, we're used to long-term thinking being on the order of a couple of decades or maybe a century at most. But really, that's being very myopic. I mean, how long has history gone on for? Well, that's a couple of thousand years. Homo sapiens have been around for a few hundred thousand years. The Earth formed four billion years ago. First, the big bang was a little under 14 billion years ago. And if we don't go extinct in the near future, which we might do, and we might cause our own extinction, then we are at the very beginning of history. Future generations will see us as the ancients living in the distant past.

• 2:04Speaker 00

  And to see that, we can just use some kind of comparisons. So a typical mammal species lives around a million years. We've been going for 300,000 years. That would put 700,000 years to come. So already on that, by that metric, our life expectancy is very large indeed. But we're not a typical species. We can do a lot of things that other mammals can't. That creates grave risks such as from engineered pathogens or AI that could bring our own demise. But it also means that if we survive those challenges, then we could last much longer again, where the earth will remain habitable for hundreds of millions of years.

• 2:43Speaker 00

  And if one, we one day take to the stars, well, the sun itself will only stop burning in about eight billion years. And the last stars will be shining in hundreds trillion years. So on any of these measures, humanity's life expectancy is truly vast. If you give just a 1% chance to us spreading to, you know, the stars and staying as long as, lasting as long as the stars shine, well, we've got a life expectancy of a trillion years. But even if we stay on earth, the life expectancy is still tens, many tens of millions of years. And that just means that when we look to the future and when we think about events that might occur in our lifetime,

• 3:26Speaker 00

  that could impact that future, that could change humanity's course. Well, you know, we should just boggle at the stakes that are involved.

• 3:33Chris Williamson?

  And when you talk about long-termism, that is looking at the future as something which needs to be taken seriously. There is this sort of grand potential for human flourishing, for human life, for all of the good stuff that could occur for a very, very long time over a very, very wide distance. And we need to take it seriously.

• 3:53Speaker 00

  Exactly. Long termism is about taking the interest of future generations seriously and appreciating just how big that future could be if we play our cards right and how good it could be. And then from there, trying to think, okay, what are the things that could be happening in our lifetime, like engineered pandemics, like greater than human level artificial intelligence, like World War III? What are the events that really could have, you know, civilizational trajectory level impacts? And then finally, taking action, trying to figure out, okay, what can we do to ensure that we navigate these challenges? And try to bring about, you know, a wonderful future for our grandchildren and their grandchildren

• 4:38Speaker 00

  and their grandchildren and so on.

• 4:40Chris Williamson?

  It's nice to hear or to think that you've been focused so much on long termism because I fell in love with The Precipice by Toby Ord that I know you work with and

• 4:49Speaker 00

  Very close friend of mine.

• 4:50Chris Williamson?

  He's phenomenal, man. But anybody that reads that book, especially the the beginning, the premise that he talks about, right? And the premise is he believes that humanity is at a very particularly unique, dangerous inflection point in between sort of prehistory and our civilizational inheritance that we could continue on and be lovely and flourishing with. And in that first chapter, he talks about the fact that the huge vast inheritance potential that we have downstream from us is crazily big. And yet, almost all of the things that we do now are focused on such a narrow time span. We're not thinking, even

• 5:32Chris Williamson?

  the most long term of long termism projections don't get into the hundreds of thousands or millions of years like you're talking about.

• 5:41Speaker 00

  Exactly. We focus on this tiny window. And, you know, in many cases, that makes sense. You can't control what people in the year 3000 have for breakfast. But surprisingly, there are things that we can affect now that do impact people in the year 3000, where number one among those, which Toby talks about at length, and where, you know, you can see The Precipice and what we are the future, my book, as kind of complements to each other.

• 6:10Chris Williamson?

  Absolutely.

• 6:11Speaker 00

  Both plowing a very similar photo.

• 6:15Speaker 00

  Yeah, one of the things that Toby talks about, length is ways in which we can cause our own extinction. So obviously asteroids, supervolcanoes, there are natural risks that could wipe us all out. Thankfully, we've actually done a pretty good job, at least of navigating asteroids. Space Guard and NASA program just came together, spent only on the order of about five million dollars per year, but has basically eliminated the risk from asteroids. It's very, very unlikely that, we now know it's very, very unlikely that we'll get hit by some kind of dinosaur killer in the next few centuries. But we are creating new risks. So the era of

• 6:56Speaker 00

  nuclear weapons created an era of like new destructive power, and the next generation of weapons of mass destruction could be considerably worse again. Engineered pathogens could create the ability to create, create new viruses that could kill, you know, hundreds of millions of people, billions of people, perhaps everyone on the planet. And if, you know, a large bioweapons program starts up focused on such things, and we have seen large bioweapons programs in the past, that could be very dangerous indeed. And that's exactly the sort of thing that we want to ensure goes to kind of zero as a risk.

• 7:35Chris Williamson?

  So how are our actions now important? Is it about investing in trajectories that people move down in the future? What, what difference does what we do in 2022 have in 2022?

• 7:53Speaker 00

  Yeah, so I think there are two ways of impacting the long-term future. So one is ensuring we have a future at all, such as by reducing the risk of extinction or reducing the chance of civilizations collapsing and then never recovering. A second way is by making the future better, assuming we do survive, improving kind of future civilizations' quality of life. And on the first of these, well, one thing we can do is carefully, carefully navigate new technologies. That can mean accelerating defensive technology, beneficial technology, and it can mean putting in policy such that we either slow offensive technology or just

• 8:31Speaker 00

  choose not to build it. So on the accelerating offensive techn- or defensive technology, one thing, for example, is far UVC lighting, where this is a certain small spectrum of light. If you implant the light into a light bulb and it kind of radiates a room, just as a normal light bulb does, then it also kills off the pathogens in that room. And this is very early stage research, but seems, you know, it's quite exciting. I think with some foundations, I advise, we're going to be funding it to a significant degree, where if this checks out, if it's sufficiently efficacious, if it's sufficiently safe, then

• 9:16Speaker 00

  we could launch a campaign to ensure that this is installed in all light bulbs all around the world. We would have made it, you know, very, very hard, near impossible to have another pandemic. And along the way, would have eradicated basically all the spirits of disease. And, you know, that's pretty exciting. That's something that we can do by taking these risks seriously to say, look, we can have an enormous benefit impact, not just on the present generation, but on the course of all future generations to come, creating a safer world for, you know, the next generation.

• 9:54Chris Williamson?

  Is this a particularly crucial time, do you think, in the history of the future?

• 10:00Speaker 00

  Uh, yeah, I think there's good arguments for thinking that we're certainly a very unusual time. I don't want to make the claim that we're necessarily at the very most important time, perhaps the next century will be even more important. And I think there were some hugely crucial times in the past as well. But we're at a very unusual time compared to both the history, both history and the future. One reason for this is just that the world is so interconnected. So for most of human history, or a large chunk of human history, there just wasn't a global connection. There were people in the Americas, people in Asia and Africa, people in Australia, and they just didn't know each other at all.

• 10:41Speaker 00

  Even within the landmass of Eurasia, you know, in the early, the first couple of centuries AD, the Han Dynasty and the Roman Empire, they comprised about 30% of the population each, but they barely knew of each other. They were like, you know, tales that one would tell of a distant civilization. Whereas now we're global and we're interconnected. And that means that, say you have a certain message that you want to get out there, in principle, it can get out to the entire world. Or, you know, more darkly, if you want to, you know, achieve conquest and domination, you can potentially do so over the entire world. And in the future, again, I mean, we were talking about galactic scale thinking and the James Webb Telescope.

• 11:26Speaker 00

  In the future, we'll be disconnected again. If one day we took to the stars, we're on different solar systems, then even to our closest solar system, there and back, communication would take eight years. And at some point in the very distant future, well, actually, different galactic groups will be disconnected such that you could never communicate between one another. Although, I'll caveat, that's very far away indeed, about 150 billion years. So hopefully, we're still going by then, but no guarantees. So that's one, the fact that we're just so interconnected is one way in which the world, like, one way in which the present time is so important.

• 12:08Speaker 00

  And so, and well, what I'm going to say is so unusual and seemingly important because it means that, you know, we can have this like battle of ideas and competition between values and if one value system like took power, then it would take power over everything. And that's pretty, you know, potentially pretty worrying.

• 12:31Speaker 00

  A second way in which the present is so unusual is just how fast technological progress is happening. So for almost all of human history, when we were hunter gatherers, economic growth, which is, you know, one measure of technological progress, was going about, basically, close to zero. Very, very slow accumulation of better stone tools, spear throwers, things like that. Agricultural revolution meant that sped up a little bit, developed farming, better farming techniques, but we're still growing at about 0.1% per year. Over the last couple of centuries, we've been growing at more like 2 or 3% per year in the frontier economies. Now, can we keep that? And most of that growth is driven by technological advancement that

• 13:16Speaker 00

  enables us both to have more people and for those people to have better material quality of life. Now, how long can we keep that going for? You know, sometimes you get this idea of, oh, well, future science is just boundless. We can never, never come to the end of it. But we've only been going properly for a few hundred years since the scientific revolution. And it seems to me hard, like, if we go at this pace, well, at some point, we'll have figured out pretty much everything that is to figure out. But we can think about this economically as well, where if we keep growing at 2% per year, after 10,000 years, because of the power of compound growth, civil, you know, the world economy, the total economy, would be 10 to the

• 14:00Speaker 00

  power of 89 times current civilization. That's a very big number. And to put it in context, there are only 10 to the power 67 atoms within 10,000 light years. So we would need to have an economy a trillion times the size of the current world economy for every atom within reachable distance. And that's just extremely implausible. I really just don't think that's possible. And so that suggests we're at this period of rapid technological growth and rapid technological advancement that cannot continue. And that means we're moving through the tech

• 14:41Speaker 00

  tree at an unusually fast rate. And that means just a lot of change. It also means that we're at a kind of unusually high-density period in terms of developing technology that could be very important, used for good, or

• 14:58Speaker 00

  Very harmful, you know, used either to lead to civilization or collapse, to end civilization, or to allow kind of certain ideologies to gain power and to influence how the course of the future goes.

• 15:13Chris Williamson?

  Early on in our history, we weren't moving sufficiently quickly to be able to develop anything that would be that much of a surprise because it was iterated at a much slower rate. Potentially, or it seems like further into the future, not only are we going to have such advanced technologies that any dangerous technologies can probably be mitigated at least a little bit, but also, again, it's going to slow down. You're going to have this sort of S-shaped curve, right, like flat to a hockey stick to then start to flatten off again. And both of the relatively flat areas of that, not much change, which means therefore relatively low amount of risk. I'm going to guess this links in with Nick Bostrom's ball from the urn thing, right, that there's just fewer balls that can be picked out of the urn when the change isn't occurring so quickly.

• 15:58Speaker 00

  Exactly. So if we're thinking, if we're asking, is now an unusually important time? Well, yeah, Nick Bostrom has this analogy of technological progress is like drawing balls from an urn. And if you pick a green ball, then it's like a very good thing. If you pick a red ball, then it's a very, you know, bad thing. Maybe it's even catastrophic. And we're picking balls from the urn just very quickly. I mean, I'm actually not sure, like, most balls are both green and red, depending on which way you look at them. Most technologies can be used for good or ill. Fission gave us nuclear reactors, it also gave us the bomb. But, you know, we're picking balls out of this urn at a faster rate than

• 16:43Speaker 00

  we did for almost all of humanity's history, and that we will do for almost all of humanity's future, even if we don't go extinct in the next few centuries.

• 16:52Chris Williamson?

  What are you talking about when you say trajectory change?

• 16:56Speaker 00

  Terrific. So we've talked so far about kind of safeguarding civilization, ensuring that we just make it through the next few centuries. And that's been the kind of main focus of discussion when it comes to existential risk. But we also, we don't want to merely make sure that we have a future at all. We also want to make sure that that future is as good as possible. We want to avoid horrific dystopian futures. We want to try and create a future that is positive and flourishing. And so it's objectively change refers to

• 17:31Speaker 00

  efforts that you can make to make the future better in those scenarios where the future lasts a long, civilization lasts a long time. And how can you do that? Well, a number of ways, but I think the most likely way is to influence the values that will guide the future, like the moral beliefs and norms. Where, you know, at the moment, we are used to living through periods of great moral change. You know, the gay rights movement emerges in the seventies and it's only a few decades that, you know, gay marriage is legalized and that's like, you know, rapid moral change compared to history. But that might change in the future.

• 18:16Speaker 00

  This moral change, the fast moral change that we know of, might end in the future. Because often, set moral worldviews or ideologies, they try to, in their nature, they often try to take power and they try to lock themselves in. So we saw this with the rise of fascism with the Nazis during World War Two. Hitler's Night of the Long Knives gets into power, crushes the ideological competition. Similarly with Stalin's purges, gets into power, crushes the competition. Similarly with Pol Pot and the Khmer Rouge. Because if you're an ideology and you want power, then you want to ensure that that ideological competition goes away.

• 19:02Speaker 00

  And my worry is that this could happen with the world as a whole. Where, you know, in the 20th century, we already had two ideologies, fascism and Stalinism, that really were aiming at global domination. And I think, you know.

• 19:21Speaker 00

  Luckily, we were not in a state where the world came close to that. But it's not like that's a million miles away in terms of how history could have gone. And so then, when I look to the next century, well, one major worry would be, for example, there's an outbreak of a Third World War, which I think is more likely than people would otherwise think. We're used to this long period of peace, but that really could change. I think it's something like one in three in our lifetimes.

• 19:49Speaker 00

  And the result of that could be single world government, single world ideology. And depending on what that ideology is like, could be very bad indeed. You know, something like what you got in 1984, George Orwell with The Handmaid's Tale, Mark Atwood. And then finally, I think that with the development of new technology, and in particular AI, that ideology could last, persist for a very long time indeed, potentially just as long as civilization does.

• 20:18Chris Williamson?

  So value lock-in isn't necessarily a bad thing as long as the values that are locked in are values that we would want over the long term?

• 20:29Speaker 00

  Right, so if you, if it were the case that the values that got locked in were the best ones, the, you know, the ones that we would all come to, given sufficient reflection and 10,000 years to think and reason and empathize.

• 20:43Chris Williamson?

  Machine extrapolated volition has been utilized correctly and all of that.

• 20:46Speaker 00

  Yeah, exactly, yeah. Then, then that would be okay. However, we should appreciate that we today are very, very far away from having the right moral views. I mean, it would be this amazing coincidence if all of history, in all of history, people had abominable moral views, supporting slave owning or patriarchy or atrocious views towards criminals or people of different sexual orientations or people of other nations. But we, in the early 21st century.

• 21:23Chris Williamson?

  Nailed it.

• 21:23Speaker 00

  Western countries, yeah, we've figured it all out. Be very surprising indeed. And so what we want to ensure is that we don't end moral progress kind of too soon. And if anyone kind of came to power in the world and were like, yes, I'm going to lock in my values now. I'm going to ensure that the things I want persist forever. I think that would be very bad. I think there'll be a loss of almost all value we could achieve because we need to keep morally progressing into the future.

• 21:52Chris Williamson?

  That's what I was going to say. Is there an argument here to be made that optionality, or whatever the word would be, or that the regular change of a particular moral structure should be something which is optimized for, that even if you were to potentially get rid of a moral structure that was more optimal and switch to one which is less optimal than that, the fact that you've baked in the ability to switch helps you to mitigate some of the dangers of having complete lock-in for the rest of time overall?

• 22:24Speaker 00

  That's exactly right. We want to have a world where we can change our moral views over time, and in particular where we can change them over time in light of good argument, or empathy, or just like moral considerations. And there might be many points of lock-in in the future. So the design of the first world government would be such a point of lock-in. First, space settlement as well. Like, you can imagine there's a race dynamic and everyone's just trying to get out into space to claim those resources as fast as possible. I actually think that, you know, a couple of examples of lock-in in the past were

• 23:09Speaker 00

  the colonial era, where you got this like, single kind of worldview suddenly just spreading all over the world, kind of Western Christian worldview. Earlier times with the first world religions, where, again, you've got this kind of bubbling of ideas, then it compresses into, you know, the sanctified holy book and then persists for thousands of years. Or also, actually, just the rise of Homo sapiens as well. There used to be many Homo species. We used to have a diversity of human species. And then there were only one because one was a little bit more powerful and you want to, you know, predictably that it means that the competition is destroyed.

• 23:48Chris Williamson?

  What's your problem with the happy birthday song?

• 23:55Speaker 00

  Thank you for asking me about that. That's not even in the in the book. What We Are the Future does not discuss Happy Birthday. So I use the example of Happy Birthday as an example of bad lock-in to illustrate the fact that even the fact that something becomes universal does not mean that it's necessarily the right solution, as it were, like the best state of affairs. So Happy Birthday is by far and away the most sung song in the world. It's the song that's used to recognize someone's birthday in at least many of the major languages.

• 24:30Speaker 00

  And it's terrible. It's like, it's pretty slow, so it's like a little bit like a dudge. The emphasis is on the "you", "you", sorry, so the emphasis is not on the "you" which you expect. It's on the "to", which is like, why on the preposition, it doesn't make sense. But then it also has like an octave leap.

• 24:53Speaker 00

  Like Happy Birthday, it's like, and no one can sing it, so everyone's, because people are singing at different ranges. So this is meant to be like a communal song, you know, your family gets together. So you want a song with like a really pretty small kind of melodic range. Instead, it has this interval. And it means that like everyone just suddenly changes key at that point. And then like now, like half your family are singing in one key and half your family is singing in this other key. And it's just a cacophony. And there's no reason at all that it couldn't be much better. You can go on YouTube for people creating new versions of Happy Birthday that sound much better.

• 25:32Speaker 00

  And why did that happen? Well, there didn't used to be a Happy Birthday song. In fact, the melody was for a difference. It's called Good Morning to All, I think. But in, I think it was the early days of radio, gramophones and so on.

• 25:50Speaker 00

  I think perhaps like that was what I call a moment of plasticity. So little moments in history when things could go one way, they could go another way. And we can really have an influence over what what direction happens. But then after that moment of plasticity, there's a kind of ossification. And so at that moment of plasticity, perhaps any of these songs could have like become the one that like gained the most popularity. But once Happy Birthday is the most popular song.

• 26:19Speaker 00

  Once it's known that that's what you do to sing Happy Birthday, to recognise someone's birthday, well, then you're kind of locked in. It's very hard to switch from that. Because if I start singing some different melody, then everyone's like, what is this? And in the case of Happy Birthday, it's probably just, you know, we could create, you know, there could be some government diktat that says, "Okay, we're all going to stop singing Happy Birthday because that doesn't make any sense. We're going to sing this different melody instead." And perhaps that would work. It's not a sufficiently important issue that I think that would happen. Some things like that have happened in the past. So I think it was Sweden that used to drive on the left side of the road, but its neighbours drove from the right.

• 27:02Speaker 00

  And so they had one day where they just switched. They were like, "Okay, we're also going to drive from the right side of the road as well." I think it was 1978. And they had this huge kind of government campaign about it. They had like songs about it, a big song competition, the winner of which was called "Keep to the Right, Svenson", and it was successful. They actually managed to switch from being locked in to driving on the left side of the road to driving on the right side of the road. In the case of Happy Birthday, I don't think that would happen.

• 27:35Speaker 00

  But yeah, Happy Birthday illustrates how the fact that this song became so widely known does not at all, and culturally, that's almost culturally universal, does not at all suggest that it's the very best thing, that it's the best way we could have sung Happy Birthday. And I think Happy Birthday has an important lesson about future dystopia, which is that model norms and model memes, they can take over the entire world without necessarily being the best ones. And I think we're living at a moment of plasticity now with a spectre of what are the model beliefs of our time that may not occur in the future. If we end up with this like single world culture,

• 28:20Speaker 00

  whether that's through conquest or just through the kind of merging of different ideas, and suddenly just everyone believes that X is right and Y is wrong, then what's the pressure to change from that? I don't think there would be much pressure.

• 28:36Speaker 00

  If those views are wrong, if it's like the melody of Happy Birthday and not the melody of some better song, then that could be very bad. And that would be a bad thing that would persist for a very long time.

• 28:47Chris Williamson?

  What it shows is the power of culture to be able to enforce norms. A lot of the time, when you think about the future and potentially bad outcomes, you think about the 1984 dictatorial bureaucratic evil world government in tall buildings telling people that they're supposed to do something. But one of the most powerful enforcement mechanisms is social approval and disapproval and just grandfathered in expectations about what you're supposed to do. And what you have, one of the biggest problems you come up against is when the people that are in charge, the bureaucratic organization or the government or whatever, when they maybe even try to change something for the better. And that runs counter to the flow that you've got going through the culture. That's when you get uprisings and revolutions, sometimes kind of like idiotic ones.

• 29:31Chris Williamson?

  But the point is that culture is so important. We saw this, I use this example all the time. We saw this with the word woke. So think about the fact that woke was originally used in rap songs. Then it kind of got weaponized or utilized and adopted by people on the left that wanted to identify someone that was kind of socially aware and cared about social justice issues. And then very quickly, no one needed to mandate that woke was going to be the sort of thing that you didn't want to be associated with. But all of the comedians and satirists and people online managed to culturally enforce a norm where woke became such a toxic term that you didn't need to tell people not to use it. No one wanted to be associated with it because it was just such an uncool word. Well, what's cool?

• 30:13Chris Williamson?

  Show me the cool mandate or the cool policy. It doesn't exist.

• 30:17Speaker 00

  Yeah.

• 30:18Chris Williamson?

  Simply enforced by norms.

• 30:20Speaker 00

  Yeah, I mean, there's just a huge amount of what humans do is determined by these cultural attitudes, just what's high status, what's cool and what isn't. And, you know, we can see this. So take conspicuous consumption, the fact that people like to show off how rich they are. And that can be, you know, across many different cultures that is used as a way to show like, you know, how successful you've been. There are different ways of doing that, that are cool in different societies over time. So it could be buying fast cars, having expensive watches, or if we go into the past, having very nice fabrics or things like that.

• 31:04Speaker 00

  It could be owning slaves. So in the Roman Empire, the more slaves were the status tempo. Some Roman senators had thousands of slaves. It could be philanthropy. So this is at least true to some extent in the United States, that engaging in philanthropic activity is a way of demonstrating conspicuous consumption. And which of these do we have? I think it's largely a cultural issue, very largely a cultural issue. And that really matters, because whether the demonstration of conspicuous consumption, which is, I think, you know, just very human, again, a human universal, whether that's done in a particular

• 31:45Speaker 00

  culture through philanthropy, through buying fast cars, or through slave owning, makes a very big difference to the well-being of the world. And I certainly know, I certainly know which I'd prefer. And I think, yeah, social scientists are only really starting to appreciate the importance of culture in the last few decades. It's the sort of thing that hasn't gotten enough attention because, well, it's kind of ephemeral. It's like you can't quantify it or measure it as much as perhaps other things like laws or economic matters. Exactly. But over the course of writing this book, just more and more, I

• 32:26Speaker 00

  got convinced that culture is an enormous force. It's almost, it's generally culture that influences political change, rather than the other way around. If you get a political change without cultural change, then that often doesn't go well. And in the book, in What We Are the Future, I focus in particular on the abolition of slavery, which, when I was kind of, you know, before writing this book, I would have thought is just clearly kind of an economic matter, something that was kind of inevitable as our technology improved, slavery was just no longer kind of viable means of production.

• 33:09Speaker 00

  But I think I was wrong. Actually, I think that the primary driver of the abolition of slavery throughout the world was a cultural change. And that was actually based on people considering model arguments and making changes on the basis of model arguments. And I think in the future, we could have, you know, equally large changes that could or not could not occur based on what model arguments are present.

• 33:35Chris Williamson?

  I've got a fix for the happy birthday problem, by the way, which is

• 33:39Speaker 00

  Hit me.

• 33:39Chris Williamson?

  So we can't, I'm not strong enough in my mental capacities to change the actual tune, but you can safeguard yourself from not being able to do the octave by starting the song one key lower than you think that you need to. Everybody should do this. Everybody starts belting out happy birthday pretty close to the upper bound of where they can go in terms of melody. No, no, no, no, no. Bring it back. Give yourself some headroom. That's what you need. And then when it comes to that, when it comes to that key change, you can nail it. Yeah, yeah, exactly. Exactly. You've got this sort of beautiful, warm sound.

• 34:14Speaker 00

  You heard it here first. Life hacks for everyone. If there's one thing that you take from this podcast, forget, forget about the future generations. Sing happy birthday in a baritone.

• 34:26Chris Williamson?

  Happy birthday. That's you, that dude. I'm telling you, that's it. So

• 34:31Speaker 00

  Okay, there we go. You've saved my life.

• 34:32Chris Williamson?

  Thank you. I think it's really interesting to think about what, what cultures and stuff lock in over a longer term. But presumably this means that we need to safeguard civilization from a bunch of suboptimal futures. And you've got three different ones. Extinction, you've got global civilizational collapse, and you've got stagnation. So starting with extinction, what's the like, are we going to go extinct? Like, what's, what's, what's going to happen?

• 35:02Speaker 00

  Yeah, I think probably not. So in general, you know, it's hard to kill everybody, thankfully. There are eight billion people in a very diverse array of environments with diverse societies. And thankfully, there are very few people in the world that really want to kill everyone. So the scenarios where that happens, I think something's gone really pretty badly wrong.

• 35:30Speaker 00

  But it's not zero at all. So if we just consider pandemics, what's the risk of

• 35:38Speaker 00

  an engineered pandemic? That is a pandemic that's not from natural causes, but that is the result of design in a lab, where we already have technology to improve the destructive power of viruses now, and that's just getting better and better every year. And so, you know, it's not very long that it will be quite widely accessible, that we'll have the power to create viruses that could kill hundreds of millions, billions of people, maybe even more. What extinction risk would I put at that? Maybe something like 0.5% this century, and much higher that there will be some engineered pandemic that would kill very large numbers of people. Maybe that's like 20% or something by the end of the century.

• 36:23Speaker 00

  And that's just far too high, like far, far too high, because there are things we can do. So I mentioned far UVC lighting. There's also early detection. So we could just be monitoring wastewater all around the world, scanning it for DNA, ignoring human DNA, and seeing, is there anything new in here? Is there something we should be worried about? And if so, then we can act kind of immediately. There's also just more advanced personal protective equipment. So masks that have just, you know, super protective, not just like, you know, the cloth masks you get, but full head things that would ensure that if you were a key worker, you would be guaranteed

• 37:08Speaker 00

  to avoid infection. That's something we could be working on now as well. So yeah, this is a just how we respond to this is contingent. It's kind of, it's up to us. We can choose to get that risk kind of, yeah, way down to zero where it wants to be.

• 37:24Chris Williamson?

  What's your opinion on Nick Bostrom's Vulnerable World hypothesis?

• 37:29Speaker 00

  So it is a hypothesis. So to explain, the hypothesis is that there could be some technology in our future that gives the power to destroy the world, to basically everyone in the world. And if so, then it would seem like

• 37:52Speaker 00

  It would be very likely that the world would end pretty soon. And he gives the analogy of imagine if it was as easy to create, let's say, a doomsday virus as it is to just put sand in a microwave. Um, then it just seems like we wouldn't last very long because there's just so many actors, each making their own independent choices, that uh we would just um someone at some point would do so. Uh, I think it's very unlikely, to be honest, that the future looks like that. Um, the main reason that is that um we just ban technology all the time. Um, so there are many technologies that we don't like. So take human cloning or something.

• 38:37Speaker 00

  We could clone humans now if we wanted to. And we choose not to um on ethical grounds because it's um taboo. And that's kind of globally enforced. Um, in his essay um on the vulnerable world hypothesis, Nick can think, Nick, you know, if we were in this vulnerable world, would that mean that the only solution would be um some very powerful surveillance state? Uh, I think, I think like, no, obviously, that would be like a really bad outcome, too. Um, and what we can do instead is just like have strong international norms about um

• 39:17Speaker 00

  what technologies we do allow to develop and which we don't. Where um, yeah, one is that humanity is at least somewhat good at kind of recognizing risks and taking action on the basis of them. And actually, in general, being quite cautious with respect to new technology. And then, secondly, technology is often used for defensive measures as well as offensive. And so in general, um. You know, in general, I think they are vulnerable world hypotheses. It's possible that it will occur in the future. But I think I'm a little more optimistic than perhaps Nick might be.

• 39:59Chris Williamson?

  Given the risk or potential future of an extreme surveillance state, which would be one potential solution to try and constrain the degrees of freedom that people can do fuckery with whatever it is that they've got. Can you see a potential human future where an effective long termist civilization is basically incompatible with democracy?

• 40:23Speaker 00

  I mean, you could. So in What We Are the Future, I talk a lot about this is why the kind of considering both values side of things and like risk side of things is so important. Because, you know, if you're only focused on the extinction side of the spectrum, then you might think, okay, we need some undemocratic civilization that can monitor everyone's behavior so that no one can pose a risk to the future of civilization. And to be clear, Nick, Boston, Toby, they don't believe this. But this is a kind of a straw man view that you could come away with. But then you've got this authoritarian state that I think has lost most of the value

• 41:06Speaker 00

  that we could have had. It's not just about making sure that the future is long. It's also about making sure it's good. And so, you know, is the ideal governance of the distant future democracy? I don't know, maybe we can do something much better. Democracy would have been unheard of, you know, for most of most cultures throughout history. You know, perhaps there's something we haven't even thought of on reflection. We would think is an even better mode of governance. But I think I'd be very worried about something that's more authoritarian, precisely for the reason that I think we could easily lose most value.

• 41:47Speaker 00

  And of course, as a result of that, where, you know, the great and actually quite fragile thing about liberal democracy that we have in places like the US and the UK is just that you've got a great vibrancy of debate and discussion and.

• 42:05Speaker 00

  Therefore, are they able to kind of drive forward moral progress? People are able to like have moral campaigns. People are able to criticize the views of people that are in power. And when you think, reflect on kind of human psychology, that's like a surprising thing. And the fact that it's actually quite rare in history should make you appreciate that. It's really something I think should be treasured and we should try and protect. And so, yeah, anything that's like, oh, we need like strong, really strong kind of government in order to reduce risks of extinction even further. And generally like, look, can we get 90% of the risk reduction by other means? And I think that often you can.

• 42:49Chris Williamson?

  If you're concerned about extinction, presumably more people on the planet would spread the risk more, would make complete extinction more difficult because the virus or the AI or the asteroid or the supervolcano or whatever simply has got more work to do. And for every human that you add, there is a potential chance that they may survive. And there may be a few of them and then maybe they could repopulate. What's your view on whether or not the world has too many or too few people in it?

• 43:16Speaker 00

  It's a great question. And there are considerations on either side of the ledger. So it's a very common kind of idea that there are too many people, resource depletion and climate change, and you shouldn't have kids because they will contribute to climate change. But I think those are, and you know, it's true. I contribute to climate change through my life. If I were to have kids, they would as well. But that's only looking at one aspect of human, of what people do. Because people do good things too. And they innovate and they, you know, effect moral change. They contribute to infrastructure, pay taxes that benefit all of society.

• 44:00Speaker 00

  I'd also say they just, if they have, if someone has a sufficiently good life, that's just a, living is good for them as well. And that's the sort of moral consideration we should take into account. But then, yeah, you're right, actually, having more people, I mean. So actually, yeah, so I'll go back a step. For those reasons, I actually think that on balance,

• 44:21Speaker 00

  we should have more people rather than fewer. The benefits from an additional person, in particular via both technological and model innovation, as well as the benefits to them, kind of outweigh the negative effects, especially given that you can you can counteract those negative effects. So if you're having a child, you can offset their carbon emissions. And actually, you can do so for, you know, a few hundred dollars a year. It's a very small fraction of the cost to have a child. But then how does having kids, putting aside climate change, how does having more people in the world impact extinction risk? It's interesting, lots of considerations on either side.

• 45:04Speaker 00

  So you're totally right that it spreads the risk. We've got more people in more diverse environments, and that makes us safer. And it's actually a little unclear to me whether, like in the world today, this year, is extinction risk higher or lower than it was a thousand years ago? Now, a thousand years ago, we couldn't have,

• 45:32Speaker 00

  you know, couldn't have blown ourselves up with nuclear weapons. But the extinction risk from nuclear, you know, at least with current arsenal sizes, not with kind of future, potentially much larger arsenal sizes. Extinction risk is pretty hard, and that's partly because there's just

• 45:49Speaker 00

  extinction is really unlikely, I think. And that's partly because there are just so many people in the world today, and we have like, you know, we have technology that can protect ourselves. Whereas a thousand years ago, well, there was, you know, there was a risk of asteroid strikes, and it's not clear that the world would have been able to come back from that.

• 46:10Speaker 00

  However, I think that most of the extinction risks we face, you know, having a difference between eight billion people and 10 billion people, it's going to be pretty small. Like, we already have it, basically, all the inhabitable areas of Earth. And that's the much bigger consideration compared to sheer population size.

• 46:29Speaker 00

  The biggest consideration, I think, is this is relates to stagnation, where

• 46:36Speaker 00

  it's, well, you know, it's relatively plausible to me that technological progress will slow down over the, over the coming century and centuries to come. Basically, if we don't, if AI doesn't speed it up, then I think there's a good chance it slows down. And that's because we're just not able to add more and more researchers to work on R and D. So, you know, further technological progress is just harder and harder and harder, the more we do of it.

• 47:06Speaker 00

  But we, in the past, we've solved that by just adding more and more people doing R and D and like trying to do technological innovation. That's by both just having a larger population. So we have eight billion people alive today. It was, I think, 200 years ago, we had a billion people.

• 47:26Speaker 00

  But also increasing the proportion of people devoted to R and D. And we know that population is going to, like, peak, maybe by about 2050, and then afterwards decline. Maybe as late as 2100, but we don't really know. And we just, there's only so far you can go by increasing the

• 47:47Speaker 00

  proportion of the world, of your population devoted to R and D. And that does suggest, and if we stagnate a period, a very risky period, when, let's say, we have very advanced bioweapons, but not the technology to defend against them. I think that would be a bad thing from the perspective of civilization. It would increase the risk of us going extinct. And in that respect, having, you know, there being more people would be helpful. It would give us a longer lead time, you know, help further technological progress. Okay, I've spoken for quite a long time, but that being said, I don't think this is like a huge issue either way.

• 48:25Chris Williamson?

  Can you dig in a little bit more to the risk of technological stagnation? You know, why is it that there's kind of an embedded growth obligation within technological progress?

• 48:37Speaker 00

  Yeah, so that was pretty quick. So many people who think about the long term, who focus on the future, are often pretty bullish on economic growth. And the argument for this is like, oh, well, it compounds over time. You know, if you're getting, you know, even just increasing a growth rate by like 1% over 70 years means you've kind of made people on 70 years twice as rich as they otherwise would be. There's this huge difference and it compounds over time. That's actually not a reason why I am concerned about technological stagnation.

• 49:18Speaker 00

  Because, you know, as I suggested earlier, I just don't think you can have economic growth that compounds for very long periods of time. That's you get to the situation where you've got 10 to the power 89 civilizations for worth of output and it's just, this just doesn't seem plausible. So instead, at some point, we're going to just plateau. And that means that if you speed up economic growth now, well, then you get to the plateau a little bit earlier. And that's kind of good into the intervening years, but not good over the long term. Or like makes no difference over the long term. So one way of putting this is that I think that in general tech progress is kind of, kind of inevitable. It like probably will keep happening. Maybe just a faster or slower rate.

• 50:03Speaker 00

  However, stagnation would be very different. And that's where it's not just that growth slows, but we just stop growing altogether. Or even the economy starts to shrink. Where we're just like not inventing new things that are improving people's quality of life. And that, I think, could be quite bad if we're at this period of high extinction risk. So as an intuition pump, suppose that technology had completely stagnated in the 1920s. So we reached the 1920s and then after that, there's no more innovation ever. Would that have been good? Would that have been sustainable? And the answer is no, I think.

• 50:45Speaker 00

  If we'd stayed at 1920s level of technology, the only way we could have supported civilization is by burning fossil fuels and burning them indefinitely until we burned all of them. That would have obviously caused like extreme levels of climate change and absolute catastrophe. And also then, we would start to regress because we just could no long, we would have run out of fossil fuels and we would no longer be able to power civilization. It was only by further technological development that gave us clean sources of energy like nuclear power and solar panels. And I think we could enter similar unsustainable states in the

• 51:29Speaker 00

  near future, where again, bioweapons are the main one, where imagine, okay, now we're at 2050 levels of technology. We have advanced bioweapons, the sorts of things that in principle could kill everyone on the planet, but not the technology to defend against them. And now imagine we're at that level of technology for a thousand years. I really don't rate our chances in that world. The risk, even if it was low, even if it was just 1% per century, well, over a thousand years, okay, 10,000 years, you know, the risk will add up and over the long run, we're almost certainly doomed. So we need that, that consideration suggests that we need to, at least in a measured way, kind of navigate our way through the

• 52:14Speaker 00

  time of perils, the time of heightened existential risk, so that we can get out the other side and be sufficiently technologically advanced to that.

• 52:25Speaker 00

  We aren't facing risks of catastrophe just every year that add up over time and instead have a position of, yeah, what my colleague Toby Ord calls existential security, where actually we've gotten risk to a very low level indeed.

• 52:38Chris Williamson?

  Yeah, you want to get to sort of X risk mastery in one form or another. One of the things that I always thought about when I considered long-termism, especially after Toby's book, was, well, why, why aren't the smart people in X risk campaigning for unbelievably slow technological development? Let's say that the urn ball analogy works and that there's dangers with every new technology that you develop. Why not take 10,000 years to add in another line of computer code, right, to the, to the AI that we're doing? Why not? You know, if, if what we've said is true and that there's this basically limitless, endless duration for the potential of humanity and, yeah, we need to get off Earth within

• 53:22Chris Williamson?

  2 billion years or whatever, or the oceans are going to boil, but we've got time. Why not mandate or somehow enforce an insane slowing of technology? But it sounds like one of the reasons that you can't do that is because you need to be able to continue the conveyor belt of technological progress in order to protect ourselves from some of the negative externalities of previous technologies that we've already locked in their existence of. Is that right?

• 53:48Speaker 00

  Yeah, I mean, you make it sound like this, like that place conveyor belt.

• 53:51Chris Williamson?

  Is that, is that not kind of how it is?

• 53:54Speaker 00

  Like, well, we've already started. I mean, we now have, you know, close to 10,000 nuclear warheads ready to launch. We're running that risk every single year.

• 54:07Speaker 00

  And so hopefully there's a state in the future that does not have such high risks. And then we can just like stay in that state of technological advancement.

• 54:17Speaker 00

  I should say that, yeah, I'm pro tech growth. Not everyone who endorses long term as is by any means.

• 54:24Speaker 00

  Other people actually would want technological progress to go more slowly and more sustainably. One thing we all agree is we want certain technologies to be slowed and others to move faster. What would be some examples of those? Well, again, that things are often easiest. We haven't talked much about AI, but things are often easiest to talk about in the bio risk case. Where this far UVC lighting that if it works like safely sterilizes a room. That's a defensive technology. Let's have that as fast as possible. Technology that allows you to redesign viruses to make them more deadly. Let's just delay that. How about we do that in the future and not just now?

• 55:06Speaker 00

  So, and that's an idea that Nick Boston calls differential technological progress. So basically we all endorse, I think almost everyone would endorse that paradigm. But then if we're talking about tech or economic growth as a whole, should we go faster? Should we go slower?

• 55:24Speaker 00

  I lean on the faster side. Other people would lean on the slower side. Some people think that with AI in particular, we should really just be trying to slow that down enormously if we could. Perhaps even just say like, look, there are certain sorts of AI technology that we shouldn't allow, like human cloning. Like in the same way that globally, we don't allow human cloning. But the main thing is just, you know, when we're taking action, we need to consider the factibility of what we're trying to do. And I think it's extremely hard to slow down tech progress or economic growth as a whole, for the world as a whole. So let's say, you know, I'm, I've become an activist.

• 56:06Speaker 00

  I dedicate my life to doing this and I convince the UK to stop growing. Doesn't make a difference in the long term because all the other countries in the world are going to keep growing. Okay, let's say I managed, I'm superhero activist, managed to convince, you know, 109. I've actually forgotten how many countries there are in the world, but I convince every country but one to keep, to just stop growing. Well, the last country keeps growing. Before long, it's just become the entire world economy. Because if you've got compound growth, even if you're the small country growing at 2% per year, when all of the other countries are stagnant, within a couple of hundred years, then you will be the world economy.

• 56:50Speaker 00

  And the activism of those other, all those other countries will have been absolutely for naught.

• 56:56Chris Williamson?

  Yeah.

• 56:57Speaker 00

  And that's how I feel about the kind of degrowth movement in general, which comes from a very different perspective, kind of environmentalism. Is whether or not it's a good idea. And I, you know, I tend to think that the sentiment is not great. But whether or not it's a good idea, it's also just ultimately futile, because it would need to be a global effort. And I think, given the, you know, given the just next level of difficulty we're talking about in trying to do that, there are just better things we could be doing, such as accelerating the good tech, delaying the bad tech.

• 57:33Chris Williamson?

  It's a combination of a lack of a God's eye perspective and ability to deploy stuff with some sort of Malthusian trap and a tragedy of the commons for the future. It's like all of that kind of mixed up together to create this sort of terrible, terrible potential.

• 57:50Speaker 00

  Yeah, exactly. And in general, you know, one thing I really always want to clarify with long-termism and work on existential risk and the stuff that, what we do with effective optimism in general. And also what I'm talking about in What We Are the Future is that I'm proposing action on the margin. It's like, take the world as it is, have a perspective on the world as a whole. And how are the sources being allocated? Should we check, like, in what way are they misallocated? Should we change them a bit? So when I'm advocating for long-termism, I'm not saying all the sources should go to positively impacting the long term. What I am saying is that at the moment, you know,

• 58:34Speaker 00

  .01% of the world's resources are focused on representing and trying to defend the interests of future generations. And maybe it should be higher than that, maybe .1%, that would be nice, maybe even as high as 1%. And so similarly, if we're thinking, oh, how fast should the world grow or not? In order for the action of relevant question is like, what should I do? Should I try and speed it up or slow it down on the margin? That's the action of relevant question, not this. Oh, what, you know, if I could control the actions of everyone in the world, what should I do? Because I don't, I don't. All we can ever do is make this little difference on the margin.

• 59:16Chris Williamson?

  What do you think about the volume of attention that's being paid to climate change?

• 59:23Speaker 00

  Uh, my overall view is enormously positive about it. So, you know, when you look at uh, different model traditions over time, it's actually remarkably of a concern for the future is like remarkably rare. Um, surprisingly, like, you know, for the book, I was really, I went to be like, oh, I want to find, you know, Christian thinkers talking about the distant future and like what we owe future generations and Hindu thinkers and Buddhist thinkers and Confucians. And it, you know, it's not like I did the deepest dive, but it's kind of surprisingly hard.

• 1:00:05Speaker 00

  Um, there are actually more thought in kind of indigenous philosophy such as Ithaca.

• 1:00:17Speaker 00

  But yeah, then, but certainly kind of secular post-enlightenment thought, it's like surprisingly rare. And then over the course of the, you know, 20th century, and then certainly the last few decades, we've had this enormous upsurge from the environmentalist um, uh, movement that really is standing up for future generations. And they've seen kind of one part of this, which is focus on, you know, stewarding resources, especially evocable losses like species loss and a particular problem of climate change. And I really feel like, oh, wow, this is just this amazing and like, again, kind of contingent thing that there is been, that there has been this groundswell of concern for um,

• 1:01:03Speaker 00

  How our actions could be impacting, yeah, not just the present day, but also the world we leave for our kids and our grandkids. And then the thing I just want to say on top of that is like, okay, yeah, this is this great model insight.

• 1:01:19Speaker 00

  And that model insight makes you concerned about climate change. Here are a bunch of other things you should be concerned about.

• 1:01:24Chris Williamson?

  Dude, that's, I mean, that really is the main takeaway from reading Toby's book. And he's got that table of the chance within the next century of something happening. And a supernova explosion is one in a hundred billion or something like that. And a supervolcano is one in 10 billion or something. And you start to move your way down and you get toward climate change, which I think is either one in 10,000 or one in 1,000 over the next 100 years. And then you get to engineered pandemics and unknown unknowns and AI safety. And it's one in 10. And I think the overall risk is maybe one in six. So when I read that, it did, it does make me, I understand your point, right?

• 1:02:08Chris Williamson?

  That anything that encourages people to think about the future generally of the planet and of humanity is smart. But I worry that there is a little bit of a value lock in that's going on here where anything that detracts away from a focus on climate change is seen as almost like heresy. And that all of our future existential risk conversations have been completely captured by a conversation about climate change. Dude, five or seven years ago, the only people talking about AI safety were getting laughed out of a room, like conferences.

• 1:02:43Speaker 00

  Utterly flinged. I mean, I was there. I was like the early vats of Nick Boston, super intelligence. I was part of these seminar rooms and I was like this young guy trying to figure out what it's, you know. What should I be into or buy? And I was like, this is, I mean, I was into, I was curious and I was helping and we're having conversations. But it was also, it felt very, very.

• 1:03:04Chris Williamson?

  Laughed out of the room.

• 1:03:04Speaker 00

  And now it's kind of, yeah, now it's much more mainstream.

• 1:03:07Chris Williamson?

  That, that's my concern. And that was, that was the takeaway from Toby's book. And, and also one of the reasons why I think that, I know that you guys do testing on messaging and stuff. And I really, really think that that's one of the most important areas. Look at me, bro, scienceing my way into stuff that you guys deconstruct very, very fine tuned on a daily basis. For me, I, I'm so compelled by the ideas that came from Nick's work and Toby's work and your stuff and Anders and, you know, like pick your favorite existential risk philosopher.

• 1:03:36Speaker 00

  Yep.

• 1:03:37Chris Williamson?

  And it blows my mind that that hasn't made even a fragment of the impact of a Swedish girl shouting at adults on a stage. And it, it, I find myself being less drawn and almost triggered sometimes by the environmental movement because of how much attention is paid to it and how little attention is paid to other X risks that I think should take priority.

• 1:04:04Speaker 00

  For sure. So a general thought of just if someone's saying my cause is X and it's the only thing that matters and everything should be determined in terms of my, like how it impacts my cause. That's just like, social media does not help with this. But yeah, that is just like systematically not a good way of thinking and it, and it happens.

• 1:04:29Speaker 00

  One thing

• 1:04:32Speaker 00

  that writing the, but like what we owe the future has made me appreciate is just that like chain model change takes time. So if we look at climate change, so, okay, now it has, I mean, yeah, it's interesting. I see, I listen, I hear climate change and I feel kind of inspired by it when I'm like, okay, cool. Give us a few decades and we're going to be, maybe we're going to be there where concern about the climate, you know, it actually goes back a really long way. So the first quantitative estimate of the, the impact of emitting CO2 goes back to a climatologist called Svante Arineus in 1896. And his estimate was actually pretty good.

• 1:05:18Speaker 00

  It was like a little bit on the, a little on the high side.

• 1:05:22Speaker 00

  And then the term the greenhouse, I guess, greenhouse effect was early 20th century. Frank Frank Capra, Capra, the director of It's a Wonderful Life, had a documentary about climate change in the 1950s. So the level of concern was there. And then the scientific consensus was the 60s and 70s. Then it's like the 90s, early noughties, but like the activism around climate change starts to really happen and build up. And so when I think about AI and bio risk and so on, I'm kind of like, it's like we're in the 1950s with climate change.

• 1:05:58Chris Williamson?

  Okay.

• 1:05:59Speaker 00

  And there's actually a leading, and so I'm just like, yeah, there's certain like scientific technological facts that are just enormously important. People don't know them yet. We've got to build this movement. This is going to take time. It's frustrating that, you know, I completely understand being frustrated. That's like, this is so big and people aren't thinking about it. But I guess like I respond by that by being like, okay, let's go.

• 1:06:30Chris Williamson?

  So they've wet. Are you kind of saying that climate change and the activism around that is a appetizer? It's an aperitif to us coming in? Well, dude.

• 1:06:41Speaker 00

  Yeah, exactly.

• 1:06:42Chris Williamson?

  I, I very much admire your positive, optimistic outlook around it. And that, that does fill me with a little bit more hope. The fact that, look, people needed to understand about the fragility of human future. One of the ways that this got delivered, because it's very obvious, you know, smoke in the sky and fires and heat and stuff, right? It's like, it's experiential as opposed to AI risk, which is just code you can't see until it's not. So yeah, maybe, maybe, maybe you're right. Maybe, maybe that's a, a thing. Going, going back to what we spoke about before. So you've got extinction, but you've also got unrecoverable global collapse.

• 1:07:25Chris Williamson?

  What's the difference between those two things? And how would a global collapse happen?

• 1:07:31Speaker 00

  Okay, so I said extinction seems pretty hard. I mean, I still give it 0.5%. That's a lot. When you're talking about, you know.

• 1:07:40Chris Williamson?

  That's only 0.5 over the next 100 years. It was 20%.

• 1:07:42Speaker 00

  That's over the next. Oh, yeah. I mean, no, 20% was the risk of some major, like hundreds of millions of dead kind of level pandemic.

• 1:07:51Chris Williamson?

  Got you.

• 1:07:52Speaker 00

  Yeah, I mean, like.

• 1:07:55Chris Williamson?

  Still not good.

• 1:07:56Speaker 00

  Decide whether it's a test or not, but utterly terrifying, I think.

• 1:08:03Speaker 00

  Yeah, so I mentioned like killing literally everyone. Okay, that's like.

• 1:08:11Speaker 00

  You know, maybe that's, you know, low, you know, low but very, very significant likelihood. A catastrophe that just set us back to pre-industrial levels of technology. So killed 99% or 99.9% of the world's population.

• 1:08:27Speaker 00

  At least you might think that might be much more likely. And could occur from a, like, could also occur from pandemics and could occur from an all like nuclear war that leads to nuclear winter. You know, perhaps could occur via other means.

• 1:08:45Speaker 00

  Would we get, would we come back? Because if, if not, then the uncovered collapse of civilization would plausibly be just as bad or almost as bad as outright extinction. You know, we would go to like a farming society again, or even hunter gatherer society, limp along, an asteroid would, you know, cause a, you know, wipe us out.

• 1:09:12Speaker 00

  It would just be a matter of time, essentially. And so I, you know, there hadn't been that much work before, before what we owe the future on this question of.

• 1:09:26Speaker 00

  How likely is a civilizational collapse? If there was one, like, would we actually come back? And so I really wanted to do a deep dive into this and really try to think that, yeah, assess the question of like, well, yeah, would we come back? And if not, why not? And I actually kind of came out pretty optimistic. Certainly over the course of doing the search for what we owe the future, I came out being a lot more positive, thinking it's like well over 90%. I think that civilization would bounce back if we moved back to the industrial levels of technology. And that's for a few reasons.

• 1:10:05Speaker 00

  So one is that if you look at local catastrophes, they very rarely lead to collapse. I'm actually not sure if I even know any examples of collapse in the relevant sense, where you take the fall of the Roman Empire, that was a collapse of a civilization. But firstly, it was only local. There's never been a kind of global collapse of civilization. And secondly, it's not like people disappeared from the region. It's just that like technology went backwards for the while and, you know, other indicators of advanced civilization kind of went back for a while. Whereas if, you know, we would be thinking about that happening on a global scale and

• 1:10:48Speaker 00

  going to kind of pre-industrial levels. And so even if you take something like the Black Death in Europe, which killed somewhere between like 25% and 60% of the population of Europe. There wasn't a civilizational collapse. It was like enormously tragic with such a loss of life. But civilization, kind of European civilization, kept going. And in fact, there was the Industrial Revolution just a few centuries later. And yeah, in the book, I discuss many other ways in which locally societies have like taken these enormous knocks and then kind of bounced back.

• 1:11:30Speaker 00

  So I give the example of Hiroshima as well, where, again, prior to the search, I'd had this image in my mind of Hiroshima, even now is just this like smoking ruin. Whereas it's not true at all. Within 13 years, the population was back to the population before it was, had an atomic bomb dropped on it. Now it's this like flourishing city.

• 1:11:52Speaker 00

  So that's kind of one reason. A second is just how much technology that could be imitated or information in libraries that people could use in order to like, you know, make technological advancement again, where, you know, the early innovators, they were doing this from scratch. There was nothing to copy. Whereas if you're like, oh, there's this thing, it seems to like burn oil in order to like make a motor go around, like I want like, maybe we can copy this. It becomes like much easier, especially then there's materials in libraries too. And then the final consideration is just that,

• 1:12:32Speaker 00

  if you think, if you try and just go through a list of like, what are the things that could stop us from getting to today's level of technology again? You kind of come up short. I think the one that could, could be decisive is fossil fuels, where we've already kind of used up easily accessible oil. And over the course of a few hundred years, we would use up easily accessible coal. But at least for the time being, we have enough in the way of fossil fuels that even a catastrophe that sent us back to, even that sent us back to the Stone Age, we would come, we would come back and if we were industrializing again, we'd be, we'd have enough to get to today's level of technology at least.

• 1:13:14Chris Williamson?

  What's your idea about coal mines and what we should do with them?

• 1:13:18Speaker 00

  Yeah, so in, in the book I talk about

• 1:13:25Speaker 00

  kind of clean tech in particular is this like, just very robustly good thing we can do. But the reasons for that aren't always the most intuitive. So one, you know, there are many reasons, I think, for wanting to keep coal in the ground. Climate change is one, enormous health pollution, the air pollution and health costs from burning fossil fuels. But one is just, we might need it in the future. If there's a catastrophe that sends civilization back to agricultural or pre agricultural levels of technology, and we need to be industrialized. Well, we got to where we are by burning prodigious amounts of coal. And we might need that again.

• 1:14:07Speaker 00

  And yet we're burning through it. And so, yeah, I think the best thing to do is just invest in clean tech. So that's not just solar panels, but also alternative fuels. What's called super hot rock geothermal, where you drill just really far into the ground and harness the heat from closer to the mantle. But one idea that I looked into is just, can you just buy coal mines and shut them down? That was the kind of like no brain like take. And like, could you do this at scale? Could you do this as a way of carbon offsetting where a large group of people get together? They all contribute to a fund that like pays for the coal mine to be retired. There are people looking into this.

• 1:14:50Speaker 00

  And I commissioned some of the search to look into it. It seems pretty hard, unfortunately, mainly for regulatory reasons. Where governments have, I mean, there were just very powerful fossil fuel lobbies and they don't like you buying coal mines and shutting them down. So there's kind of use it or lose it laws where

• 1:15:12Speaker 00

  if you try and buy a coal mine with the purpose of shutting it down, the government just voids your contract. Because they say if you buy a coal mine, you have to use the coal.

• 1:15:23Speaker 00

  And that, I think, you know, even if you can get around that, that rises up the price a lot more. People perhaps just shift to other coal. So unfortunately, I'm like a little more negative on that particular strategy than I used to be. But for the while I was just really taken with it. I just, I just really loved this idea of just like, we own this coal mine now. We're going to turn it into this museum, museum for obsolete technology. And it'll have like, you can go around like a theme park in the like, the little trucks that you see in Indiana Jones, like going along the carts. I had the whole vision, but maybe one day I'll still do it.

• 1:16:01Chris Williamson?

  Well, we've got

• 1:16:01Speaker 00

  be the most impactful thing.

• 1:16:03Chris Williamson?

  We've got those seed banks, right? There's, is it in Iceland, I think, or in

• 1:16:08Speaker 00

  And Svalbard

• 1:16:09Chris Williamson?

  That's it.

• 1:16:10Speaker 00

  In Norway.

• 1:16:10Chris Williamson?

  And that's a bunch of every plant seed on the planet and presumably There must be equivalent backups digitally of every book probably distributed across the world. And they'll be in underground bunkers or on the side of a mountain or something like that. So that if there was some sort of huge collapse or any kind of existential risk that had a bit of kinetic energy to it, that it would be, it would be kept away. And what that does is it shortcuts the pain and the investment that you need in order to be able to actually find out what to do, you can just go back and read what you need to do. And then you get to rediscover the technologies. And I've never heard anyone else talk about it. The fact that one of the advantages we had was that all of the coal and the oil was relatively close to the

• 1:16:55Chris Williamson?

  surface and you pick the, by design, you pick the low hanging fruit first. But that doesn't future proof you against collapse recoveries potentially well. Because if you do collapse and you need to recover, presumably you're going to be able to get low hanging fruit, but not the less, the higher hanging fruit so easily, which means that you actually need to keep it. You need to portion off a little bit in future. I was thinking about this as well. Is there a reason or would there be a justification for having a, an, almost like an air gapped civilization somewhere on earth? I know that we don't, we're not a multi-planetary species yet, so we haven't put somebody on Mars. And maybe that would be one of the obvious ways to do this.

• 1:17:38Chris Williamson?

  But to have a selection of people with a wide genetic pool and you've done the testing on them and they go to a particular place and they are, they live there for their entire lives. Or perhaps you could do it like military service and people could cycle in and out and you would maybe go for four years. But what that, that's kind of like as close to air gapping a backup for civilization behind the normal civilization that we've got going on in the same way as the seed bank or the same way as the modern library of Alexandria or whatever we've got. Is there a justification for us sending some people into the side of a mountain and making them live there for ages?

• 1:18:16Speaker 00

  Yeah, absolutely. On your previous point, I just wanted to briefly mention that the idea of harnessing the low hanging fruit. You know, there's this historical counterfactual where if we were to go, if we were to collapse, where would the industrial revolution happen? And I can bet you my last dollar, it would not happen in Western Europe. Because we already used up all the surface coal. Instead, it would be in the US or India or China or Australia where there's, where there's more. But okay, civilizational backup.

• 1:18:46Speaker 00

  This is not only, I think, a good idea. It's something that people who buy into the idea of long-termism, I think, we really might make happen. So again, with foundations, I advise, and philanthropists, I advise. This is something we'd be pretty interested in doing. So having a hermetically sealed refuge where a certain number of people stay in that for long periods of time. Yeah, perhaps it's six months at a time and kind of cycles out. Perhaps it's like a stage. There's a couple of them and you do it asynchronously. Or perhaps it's even longer. That is completely protected against the outside world.

• 1:19:30Speaker 00

  So again, if there's a really worst-case pandemic, then, and it's, you know, we're not just talking about hundreds of millions of people dying here, but literally it will kill everyone. Well, you have this population of, let's say, a thousand people, including leading scientists who would be able to work on, you know, medical countermeasures like vaccines. And they would then be able to stay there and you'd equip it such that they could stay there for years. Design countermeasures so that when they emerge, if they need to, they are protected against the pathogens. And then, the second thing, you would also need to stock it such that it could rebuild

• 1:20:14Speaker 00

  society after that point. This seems just like so wacky as an idea to many people, but I think it makes sense. Like, on the scale of the whole of the world, you know, we have a world GDP of close to $100 trillion, we have 8 billion people. Does it make sense to spend at least a little bit of those resources? Like, you know, a few hundred or a thousand people to be just like protected in case for the very worst-case outcomes? I think from that perspective, the answer seems like, yeah, obviously yes. Because, yeah, even if it was a small population, a few hundred people, that would be enough

• 1:20:56Speaker 00

  to rebuild civilization again, and I think it's worth preserving.

• 1:21:00Chris Williamson?

  Rob Reed laughed at me when I gave him that idea last year. So this is, yeah, I did, I like it. I think it's a, I think it's a cool idea.

• 1:21:09Speaker 00

  But I think it's interesting the response that, you know, and I have it too. When you get confronted with certain ideas that sound like the sci fi or something and you laugh at them. But that's just been true for many ideas in the past, like the idea that viruses were carried by these little monsters that live in your hands and claw in your skin. It's like, what? This is nonsense. Yeah, yeah, it's true. The idea that we could fly, you know, build big metal planes and take them into the skies. Like, that was pretty, that was pretty wacky too. So I think we have this like, you know, very natural human impulse to just laugh at these like silly ideas.

• 1:21:52Speaker 00

  But I think when the stakes are this great, we should actually just be taking a little bit of time to reflect and think, okay, does this actually make good sense? And I think in this case it does.

• 1:22:02Chris Williamson?

  Given everything that we've gone through and the potential duration that we're moving towards for our genetic inheritance and civilizational future, the risks that we've got, the way that we can mitigate them, how we can move back against them, what's the actual goal? What, what should our goal for the future of humanity be? What are we, what are we even optimizing for?

• 1:22:25Speaker 00

  What are we building for? So I think the key thing I want to say is that we don't know yet, and that's okay. So imagine you're a teenager and you're wondering like, oh, what's the goal of my life? And you just want to be as happy as possible, let's say. Now, you might just not know yet, you know? And what do you want to do? Well, okay, as a teenager, you want to not die because if you die, then you're gonna, you know, you're not going to have a flourishing life after that point. So similarly, we want to make sure we don't go extinct. But then you just want to have lots of possible futures open to you and be able to have time to like figure stuff out, tie things out, see what's actually is going to make, give you the most flourishing life.

• 1:23:09Speaker 00

  And that's okay. So you can have a plan that is about getting yourself to a position such that you can figure out what to do for the rest of your life. And I think the same is true for humanity as a whole. We want to, we should try to get ourselves to a state where we are not facing existential catastrophe. We are, we do have many possible futures open to us. We're able to reflect and deliberate and then make further moral progress so that we can then collectively figure out from a place of kind of much more enlightened, much more enlightened vision perspective, what we should do with the potentially vast future that's ahead of us. And I call all these ideas, this idea of kind of exploring and trying

• 1:23:52Speaker 00

  to figure out different things, a model, a modelly exploratory society in the book, with the kind of, the limit case of that, I call the long, the long reflection, which is where it's like, okay, we've got to a state where we've kind of solved the most pressing needs. Do we want to immediately rush to just settling, settling the stars with whatever our favorite model views are? I'm like, no, we've got time. As we talked about at the very start of this podcast, we've got an awful lot of time. And that means we can, before we engage in any activities that lock in a particular worldview, such as space settlement or formation of a world government, then we really take the time

• 1:24:37Speaker 00

  to ensure that we've gotten, you know, gotten to the end of model progress, that we've really figured out all that we can.

• 1:24:45Chris Williamson?

  Are you of the mind that we should move more slowly with moral progress and our considerations for what we should do once we've got ourselves to technological maturity, than in technological progress en route to get there?

• 1:25:03Speaker 00

  Uh, so I think with technological progress, how fast we should go is kind of tough.

• 1:25:16Speaker 00

  Where, you know, there are these reasons, I think, why at the moment going faster technologically is like a little better. It's not kind of that big a deal. I do think that if you can make model progress faster, you just want to go as fast as you can because you make the everything better than the future. It's just that I think that real model progress might take time. So this might be true for technological progress too. Maybe it takes, you know, I've said that we can't go as fast as we're currently going. But assuming we slow, perhaps the whole project takes, you know, millions of years to come. I don't know.

• 1:26:00Speaker 00

  But the same might be true for morality as well. And unlike, with technology, there's always incentive to build better technology. It gives you more power. It means you can do more things, whatever your values are. So there's like always strong incentives to do that. With model progress, that's not true. If I'm, if I don't care, if I have all the power and I don't care about having a better model point of view. There's no law of nature. There's also no real competitive dynamics that force me to have get to a better model perspective. And so what I'm really doing and kind of sketching out this idea of the long reflection is just to be, is to really say,

• 1:26:44Speaker 00

  we need to keep working on, you know, getting better models, getting better values. We don't want to just like jump into the first thing that we think is good, you know, early 21st century Western liberal-ish kind of morality. It's like, no, let's really like, we can take, we can take our time before engaging in kind of any of these big projects.

• 1:27:09Chris Williamson?

  There's less of a degree of urgency with that. Although you can go as quickly as you want, you can, you can luxuriate and take your time, perhaps in a way that you couldn't do with, with technological change.

• 1:27:22Speaker 00

  Yeah, I mean, I think for the world as a whole, I think we could perhaps luxuriate with. Technological change, at least, and if we get to a point of safety. So perhaps after the bioweapons, you know, within this, perhaps in this unsustainable state at the moment. But perhaps we, you know, early 22nd century, then maybe there aren't any nuclear weapons now. We've got great protections against them. And perhaps we've not yet finished technological progress, but the particular level we're in is actually really very safe and stable. Maybe we just want to hang out there for a little while, while we figure some model and political stuff out.

• 1:28:02Chris Williamson?

  Hmm. So this is kind of like Bostrom's thing about how you can have, I can't remember the asymmetry in terms of technological development. But you can also do this in terms of moral versus technological development, that you can kind of keep on idling. While you continue to move on forward and then let's say that you have your technology that allows you to catch up to that. I think it's Eric Weinstein that said, we're gods, we're just shitty gods. It's Bostrom that was talking about like, we're gods, but for the wisdom. And that.

• 1:28:29Speaker 00

  Yeah, exactly. We want to have, yeah, we want to accelerate wisdom as much as we can. It just seems good in general. Um, and then, like I say, I think there are particular circumstances at the moment that do mean that like. Ensuring we keep tech progress going, um, like is good as well. But if you could guarantee me, guarantee me that, look, even at a slower rate, just now with that like. Like 5% growth per year, 0.1% growth per year, but we're going to get moral progress along the same time. Then that would, you know, and assuming tech progress is not going to stagnate, but just continue at that lower rate. Um, you know, then I'd feel pretty good about that.

• 1:29:10Chris Williamson?

  How do you speed up moral progress? Have you thought about that?

• 1:29:14Speaker 00

  Uh, I mean, one is by, very hard question. So one is having like, uh, kind of diversity of moral views.

• 1:29:24Speaker 00

  And even societies as well. So there was this kind of, you know, great ideal behind the United States that never really happened of all the different states would be like this vibrant different culture and there'd be like a laboratory of democracy. And instead, it just like United States became a relatively uniform country. But you could like aim for kind of global society that was like more like that. I think we can also just invest more in, I don't talk about this that much because it sounds so incredibly self-serving as a moral philosopher.

• 1:30:00Chris Williamson?

  Do you want to be a more philosopher to be invested in?

• 1:30:03Speaker 00

  Yeah, I think we should like, how much of what the world society of other sources gets spent on, you know, and not just model philosophy, but other humanities, you know, capacity for, building capacity for empathy and so on. But really, the amount we spend in a society is like vanishingly small, like point zero one percent, even on all of the humanities as a whole. And that's like in your own life. Oh, how much time should I spend figuring out where, how I spend the entire rest of my life as a teenager? Oh yeah, maybe I'll spend like a day on it and that'll do. It's like, I don't know, it doesn't

• 1:30:43Speaker 00

  make sense. You're talking about your entire life. Similarly, with humanity as a whole, you know, if it were the case, the kind of model reflection and reasoning and progress in the model domain had a hundredth of the status and prestige and investment that technology does, even though it's the model progress that's contingent, not the technological progress. So actually, it's the model progress that we should be trying to safeguard even more. Well, yeah, then we would have to increase the number of model philosophers like a thousandfold. And honestly, I think it'd be a very good thing. But yeah, perhaps there's some bias there.

• 1:31:26Chris Williamson?

  It would make my job a lot easier on the podcast as well. That would be a good extra.

• 1:31:29Speaker 00

  Exactly. So, yeah. And also they improve the quality as well. You wouldn't be getting idiots like me. You'd be getting the very best people who otherwise going into maths and physics would be working on improving our model understanding.

• 1:31:42Chris Williamson?

  Will MacAskill, ladies and gentlemen, if people have been compelled by the stuff that you've spoken about today and they want to find out more or work out how they could contribute to making our future a little bit less terrible, where should they go?

• 1:31:54Speaker 00

  So, obviously, I'd love you to read the book that I just wrote, What We Are the Future, which I think will be out at this, when the podcast is launched. But then if you want to take action, two big ways to take action. One is through donations. And I co-founded an organization called Giving What We Can that encourages people to give at least 10% of their income to the organizations that are most effective. And you can contribute to organizations that are trying to prevent the next pandemic or that are trying to safely guide the development of artificial intelligence or prevent a third world war and thereby very significantly impact the trajectory of human civilization.

• 1:32:38Speaker 00

  The second thing, big thing you can do is try and think about how you can use your career to do good in the world. Maybe you're already established. Perhaps you're a podcast host and you should think about how can you get the, you know, most important messages out there? Or you might be early on in your career and trying to figure out, yeah, what should the trajectory of my life be? I co-founded another organization, 80,000 Hours, and it has enormous amounts of advice, all for free, online, to try to help people make the best career decisions so that they can have the biggest impact. In particular, biggest impact on issues that impact the long-term future and help build a better world for future generations.

• 1:33:23Speaker 00

  They have online advice, they have a podcast, 80,000 Hours podcast, and they also offer free one-on-one advice as well. So yeah, check out both Giving What We Can and 80,000 Hours and my book, What We Are the Future 2. And take that all into account, you can make a truly enormous difference to the world for thousands, millions, or even billions of years to come.

• 1:33:46Chris Williamson?

  Well, I appreciate you, man. I love the work that you guys do. If this is the history of the future, or the beginning of the history of the future, or whatever, I'm glad that we've got people like you that are guiding us in whatever way you can.

• 1:33:59Speaker 00

  Thank you.

• 1:33:59Speaker 02

  Mr. President, my company formed me of the explosion on our rig on October 2nd. Causing massive water displacement and triggering a tsunami. They all keep the blame on me. Tom Cruise is.

• 1:34:10Speaker 03

  Digger got us into this mess, and Digger's gonna dig us up. From the director of Birdman and The Revenant.

• 1:34:16Speaker 04

  What would happen if I stopped drilling? You stall, you freeze to death, that's what.

• 1:34:20Speaker 03

  With great power comes great deniability.

• 1:34:23Speaker 04

  I'm not the first then.

• 1:34:25Speaker 03

  Digger. See the trailer online. Only in theaters October 2nd. Rated R. Under 17.9 A Minute Without Parents.