Joseph Carlsmith - Utopia, AI, & Infinite Ethics

Joseph Carlsmith https://www.josephcarlsmith.com/ is a senior research analyst at Open Philanthropy https://www.openphilanthropy.org/ and a doctoral student in philosophy at the University of Oxford https://www.ox.ac.uk/. We discuss utopia, artificial intelligence, computational power of the brain https://www.openphilanthropy.org/research/new-report-on-how-much-computational-power-it-takes-to-match-the-human-brain/, infinite ethics https://www.lesswrong.com/posts/5iZTwGHv2tNfFmeDa/on-infinite-ethics, learning from the fact that you exist, perils of futurism, and blogging. Watch on YouTube https://youtu.be/M3TUe4zUCKk. Listen on Spotify https://open.spotify.com/episode/7szpXHcCCDOZ8C42lPWIS3?si=5hKvqPshSryMUTeaAmRWEw, Apple Podcasts https://podcasts.apple.com/us/podcast/35-joseph-carlsmith-utopia-ai-infinite-ethics/id1516093381?i=1000574849021, etc. Episode website + Transcript here https://www.dwarkeshpatel.com/p/joseph-carlsmith. Follow Joseph on Twitter https://twitter.com/jkcarlsmith. Follow me on Twitter https://twitter.com/dwarkesh_sp. Subscribe to find out about future episodes! Timestamps (0:00:06) - Introduction (0:02:53) - How to Define a Better Future? (0:09:19) - Utopia (0:25:12) - Robin Hanson’s EMs (0:27:35) - Human Computational Capacity (0:34:15) - FLOPS to Emulate Human Cognition? (0:40:15) - Infinite Ethics (1:00:51) - SIA vs SSA (1:17:53) - Futurism & Unreality (1:23:36) - Blogging & Productivity (1:28:43) - Book Recommendations (1:30:04) - Conclusion Please share if you enjoyed this episode! Helps out a ton! This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com https://www.dwarkesh.com?utm_medium=podcast&utm_campaign=CTA_1

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:06Dwarkesh Patel?

  Today, I have the pleasure of interviewing Joe Carl Smith, who's a senior research analyst at Open Philanthropy and a doctoral student in philosophy at the University of Oxford.

• 0:19Dwarkesh Patel?

  Joe has a really interesting blog that I got to check out called Hands in Cities. And that's the reason that I wanted to have him on the podcast because it has a bunch of thought provoking and insightful posts on there about philosophy, morality, ethics, the future. And yeah, so I really wanted to talk to you, Joe. But do you want to give a bit of a longer intro on what you're up to?

• 0:42Speaker 01

  Sure. So I work at Open Philanthropy on existential risk from artificial intelligence. And so, you know, I think about what's going to happen with AI. How can we make sure it goes well and predictable? How can we make sure that advanced AI systems are safe? And then I have a side project, which is this blog where I write about philosophy and the future and things like that. And that emerges partly from a sort of my background, which is I was before getting into AI and working at Open Philanthropy. I was in academic philosophy.

• 1:18Dwarkesh Patel?

  Okay, that's quite an ambitious side project. I mean, given the length and the regularity of those posts, it's actually quite stunning. Do you want to talk more about what you're working on about AI at Open Philanthropy?

• 1:33Speaker 01

  So it's a mix of things. Right now, I'm thinking about AI timelines and what's called takeoff speeds, sort of how fast the transition is from pretty impressive AI systems to AI systems that are kind of radically transformative. And I'm trying to use that to provide more perspective on the probability that everything goes terribly wrong.

• 1:52Dwarkesh Patel?

  I see, okay. I didn't know, but so what are the implications? I suppose it's higher or lower than I would expect. I guess if it's higher, maybe I should work on AI level, but other than that, what is, what are the implications of that that figure changing?

• 2:06Speaker 01

  I think there are a number of implications just from understanding timelines with respect to how you prioritize and what, you know, just to some extent, the sooner something is, then you need to be planning for it coming sooner and kind of cutting more corners or, you know, counting less on having more time. And yeah, I think overall, the higher you think the probability of catastrophe is, the easier it is for this to to become kind of the most important priority. I do think there is a range of probabilities where it maybe doesn't matter that much. But I think the difference between, say, one and 10%, I think, is quite substantive and the difference between 10 and 90 is quite substantive and, you know, I know people in all of those ranges.

• 2:51Dwarkesh Patel?

  Gotcha. Okay, interesting. Yeah, so let's back up here and talk a bit more about the philosophy motivating this. So I think you identify as a long-termist. Yeah, so maybe my picture question here is you have an interesting blog post about why the future, looking back on us, might think about the 21st century, given the risk we're taking.

• 3:12Dwarkesh Patel?

  So I mean, what do you think about the possibility that we're potentially giving up resources, potentially dedicating, well, I'm not, you're dedicating your career to, you know, building a future that, you know, maybe, given, you know, given the fact that you're alive now, you might find strange or disturbing or disgusting. I mean, so I guess to add more context to the question, if from a utilitarian perspective, the present is clearly much, much better than the past. But somebody from the past might think that, you know, there's a lot of things about the present that are kind of disturbing. I mean, they might not like the configuration of how maybe isolating a modern city might be. They might find the kinds of free to cheap information

• 3:54Dwarkesh Patel?

  that you can access on your phone kind of disturbing. Yeah. So how do you think about that?

• 4:00Speaker 01

  So yeah, a few comments there. So one, I do think that if you took, you know, for most people throughout history, if you brought them to the present day, they would, my, my guess is that fairly quickly, depending on exactly the circumstances, they would come to prefer living, living in the present day to, to the past, even if there are sort of a bit of, of future shock and, and a bit of, some things are alienating or disturbing. And, but that said, I think the distance, the sort of gap between historical humans and the present is actually much, much smaller, both in terms of time and kind of other factors than the gap I envisioned between present day humans and the future humans who are living, living ideally in, in a kind of radically better situation.

• 4:47Speaker 01

  And so I do expect sort of greater distance and possibly greater alienation when you first show up. My personal view is that the best, the best futures are going to be such that if you really understood them and if you really experienced what they're like, which, which may be a big step and might require sort of extensive engagement and possibly sort of changes to your capacities to understand and experience, then you would think it's really good. And so, and, and I think that's the relevant standard. So for me, I, I worry less if the future is sort of initially alienating. And the question for me is how do I feel once I've really, really understood what's going on?

• 5:28Dwarkesh Patel?

  I see. So I, I, I wonder how much we should value that kind of inside view you would get into the future from being there? If you think about, I don't know, many, many existing ideologies, like, I don't know, think of an Islamist or something is who might say, listen, if you could just like come to Iraq and feel the bliss of fighting for the caliphate, you, you would understand better than you can understand from the outside view, just, you know, sitting on a couch, eating Doritos. What, what, you know what it's like to fight for a cause? And maybe their experience is kind of blissful in some kind of way, but

• 6:02Dwarkesh Patel?

  I feel like the outside view is more useful than the inside view there.

• 6:05Speaker 01

  Well, so I think there's a couple of different questions there. One is, what would the experience be if you had it from the inside? And then there was this, I think, a subtly different question, which was what, which is, what would your take on this be if you kind of fully understood? Where fully understanding is not just a matter of having the internal experience of being in, you know, in a certain situation, but it's also a matter of understanding what that situation is causing, what sort of beliefs are structuring, the ideology, whether those beliefs are true, and all sorts of other factors. And it's the latter thing that I have in mind. So I'm not just imagining, oh, the future will feel good if you're there. Because, you know, sort of by hypothesis, the people who are there, at least one hopes they're enjoying it, or one hopes their thumbs up.

• 6:49Speaker 01

  If the people who are there aren't thumbs up, that's a strange, a strange utopia. But I'm thinking more that in addition to their perspective, there's a sort of more holistic perspective, which is the sort of full understanding. And that's the perspective from which you would endorse, endorse this situation.

• 7:07Dwarkesh Patel?

  I see. And then, yeah, so another respect in which it's interesting to think about what they might think of us is, you know, like, well, what would they think of the crazy risk we're taking by not optimizing for existential risks? And so, you know, one analogy you could offer, I think, what MacAskill does in his new book is to think of us as, you know, teenagers in our civilization's history, and then, you know, think of the crazy things you did as a teenager and how you. And, yeah, so, I mean, maybe there is an aspect to which, like, one would wish they could take back the crazy things they did as a teenager. But my impression is that most adults probably think that while the crazy things were kind of risky, they were, they're very formative and important, and they feel nostalgic about the things they did in the past.

• 7:53Dwarkesh Patel?

  Oh, do you think that the future, looking back, they are going to regret the, the way we were living in the 24th century? Or, or will they look back and think, oh, you know, that that was kind of a cool time? I mean, I guess this is kind of conditional on there being a future, which takes away a lot of the mystery here, but.

• 8:10Speaker 01

  I doubt that they will look back with

• 8:14Speaker 01

  with pleasure at the sort of risks and horrors of the 21st century. I mean, if you just think about how we, or at least I, tend to think about something like the Cuban Missile Crisis or World War II, I don't personally have a kind of nostalgia, oh, you know, sure, it was risky, but it made me who I am, or something like that. I also want to say, you know, I think it's true that when you look back on your teenage years, there is often a sort of, you know, let's say you did something like crazy. You and your friends used to race, you know, around and you played chicken or something at the local quarry. And it's like, oh, but you know, you survived, right? And the real reason not to do that is the like chunk of probability where you just died.

• 8:59Speaker 01

  And so I think there's a, you know, to some extent, the it's it's the ex post perspective of looking back on certain sorts of risks is not the right one, especially for death risks. That's not the right perspective to use to kind of calibrate your understanding of how to feel about it overall.

• 9:17Dwarkesh Patel?

  I see. Okay, so I think you brought up utopia and you have a really interesting post about the concept of utopia. So do you want to talk a little bit more about this concept and why it's important? And and also, why do we have so much trouble thinking of a compelling utopia?

• 9:35Speaker 01

  Yeah, so utopia for me just means a kind of profoundly better future. And I think it's important because I think it's just actually possible. I just think it's actually something that we could do. We could make, if we sort of play our cards right, in sort of non crazy ways, we could just build a world that is radically better than the world we live in today. And in particular, I think we often, in thinking about that sort of possibility, underestimate just how big the difference in value could be between our current situation and kind of what's available. And I think often utopias are kind of anchored too hard on the status quo

• 10:19Speaker 01

  and sort of changing it in in small ways. But imagine imagining our kind of fundamental situation basically unaltered. And I think, so such that it's a little bit like the difference between, you know, you have a kind of A crappy job or like a beach vacation and Utopia is like, everyone has beach vacation. And, you know, I don't know how you feel about beach vacations. But I think it's much, I think the difference is more like being asleep and being awake. Or sort of, it's, it's more.

• 10:48Speaker 01

  Yeah, it's sort of, it's like living in a cave or living, living in the, in, under the open sky. I think, I think it's like a really big, a really big difference and, and that that matters a lot.

• 10:59Dwarkesh Patel?

  I, that's interesting because I remember in the essay you had, you had a section where you mentioned that you expect Utopia to be recognizable. Like to, to a person alive now. The, the, I guess the way you put it just earlier made it seem like it would be a completely different category of experience than we would be familiar with. Yeah, so i- is there a contradiction there or maybe I'm missing something?

• 11:24Speaker 01

  So I think there's at least a tension and, and the way I see the tension playing out or, you know, kind of being reconciled is specifically via the notion I referenced earlier of kind of, you would, if you truly understood, come to see the utopia as genuinely good. But I think that process, I mean, ideally, I think the way we end up building utopia is we go through a long, patient process of becoming wiser and better and more capable as a species. And, and it's in virtue of that process kind of culminating that we're in a position to build, to build a civilization that is sort of profoundly good and radically, radically different. But that's a long process.

• 12:06Speaker 01

  And so I, I do think, you know, if, as I say, if I just transported you right there and you skipped, you skipped the process, then you might not like it. But, and, and, and it is quite alien in some sense, but I still, but if you went through the process of like really understanding and kind of becoming wiser, you would, you would endorse.

• 12:26Dwarkesh Patel?

  Uh huh. That's, um, that's interesting to me that you think the process to get to utopia is more of, a sort of, uh, maybe I'm misconstruing it, but when you mentioned it's a process of us getting wiser. And, um, yeah, so it sounds like it's a more philosophical process rather than, I don't know, this, we figure out how to convert everything to hedonium. And, you know, it's eternal bliss from then on. Yeah. So, well, am I getting it right that you think it's more philosophical process? And then why, why is it that you think so?

• 12:55Speaker 01

  Yeah, so I definitely don't sit around thinking that utopia, we sort of know what utopia is right now and it's hedonium. I'm not especially into the notion of hedonium, though I think it's possible to. I think it's, I think the brand is bad. I think, I think, you know, people talk about pleasure with this kind of dismissive attitude sometimes. And, you know, hedonium implies this kind of sterile uniformity. You know, and you're sort of tiling, people are talking about like going to tile the universe with hedonium. And it's like, wow, this sounds, this sounds rough. Whereas I think actually, you know, the relevant perspective, when you're thinking about something like hedonium is the kind of internal perspective from which the sort of experience of the subject is something kind of

• 13:39Speaker 01

  joyful and, you know, boundless and kind of energizing, you know, whatever, whatever pleasure is actually like, pleasure is not a trivial thing. I think pleasure is a profound thing in a lot of ways. But I really don't, I don't assume that that's what utopia is about at all. I think we're at, I think, A, my, you know, my own values seem to be quite complicated. I don't think I just value pleasure. I value a lot of different things. And more broadly, I have a lot of uncertainty about how, how I will think and feel about things if I were to go through a kind of process of significantly increasing my capacity to understand. I don't, I think sometimes when people imagine that, they imagine, oh, we're going to sit around and do a bunch of philosophy and then we'll have like solve normative ethics and then we'll implement our solution to normative ethics.

• 14:21Speaker 01

  And that's not what I'm imagining by kind of wisdom. I'm imagining something richer and also that involves importantly, a kind of enhancement to our cognitive capacity. So it's sort of really, you know, I think we have, we have very small, we're really limited in our ability to understand the universe right now. We have kind of. And I think there's just a huge amount of uncharted territory in terms of what minds can be and do and see. And so I want to sort of chart that territory before we start making kind of big and irreversible decisions about what sort of civilization we want to build in the long term.

• 14:58Dwarkesh Patel?

  I see. And then another maybe concerning part of the utopia is that, yeah, as you mentioned in the piece, many, many of the worst ideologies in history have had elements of utopian thinking in them. To the extent that EA and utilitarianism generally are compatible with utopian thinking, maybe they don't advocate utopian thinking, but they are compatible with it. Do you see that as a problem for the movement's health and potential impact?

• 15:27Speaker 01

  Is the question something like, is this a red flag? Kind of, ah, you know, we look at, we look at other ideologies throughout history and they've been compatible with utopian thinking and, and maybe sort of effective altruism or, or utilitarians or something is similarly compatible. So should we, should we worry in the same way? Is that the question?

• 15:46Dwarkesh Patel?

  Yeah, partly. And also another part is maybe the, maybe, maybe it's still right that like morally speaking, yeah, utopia is compatible with this worldview and the worldview is correct. But that the implications are that, you know, somebody misunderstands what is best. They identify as an EA and this leads to bad consequences when they try to implement their scheme.

• 16:10Speaker 01

  Yeah, so I think there are certainly reasons to be cautious in this broad vein. I don't see them as very specific to EA or utili... I don't identify as utilitarian, but to utilitarianism, I see them as more sort of better understood as risks that come from believing that something is very important at all. And I think it's true that many acting from a space of conviction, especially where that conviction has, has sort of a flavor of, you know, it's, it's interesting what exactly constitutes an ideology, but I think it's, I think it's reasonable to look at EA and sort of be like, this, this looks like an ideology.

• 16:53Speaker 01

  And I think, you know, and I think that's, I think that's right. And I think that's sort of important to, to, you know, have this sort of relevant red flags about. I think it's pretty hard to have a view of the world. That doesn't in some sense imply that it could be a lot better. Or at least a plausible view of the world. And and when I say utopia, I don't really mean anything much different from that. You know, I think it's sort of, I'm not saying a perfect thing. I'm not, you know, I I do have sort of a more specific view about exactly how much better things could be. But more broadly, it seems to me many, many people believe in the possibility of a much better world and are fighting for that in different ways.

• 17:33Speaker 01

  And so I wouldn't, I wouldn't pin the red flag specifically to the belief that sort of things can be better. I think it would have more to do with sort of what degree of rigidness are you, you know, relating to that belief with? How are you kind of, how are you acting on it in the world? How much are you willing to kind of, kind of break things or kind of act in uncooperative ways in virtue of of that sort of conviction? And there, I think caution is definitely warranted.

• 18:03Dwarkesh Patel?

  I see. Yes, I I I I'm not sure I agree that most people have a view or an ideology that implies

• 18:14Dwarkesh Patel?

  anywhere close to the kind of utopia that one, utopian thinking one can have. Like, if you think of modern political parties in a developed democracy, like in the United States, for example, if you think of what is like a utopian vision that either party has, it's like, it's actually quite, quite banal. It's like, oh, we'll have universal health care or I don't know, GDP will be higher in the next couple of decades. Which is, which doesn't seem utopian to me. It just seems, and it does seem, it does seem like a limited worldview where they're not really thinking about how much better or worse things could be. But it doesn't exactly seem utopian. Yeah, I'll let you react to that.

• 18:53Speaker 01

  I think that's a good point. So maybe the relevant notion of utopian here is something like, to what extent is a concept of a radically better world kind of operative in your day to day engagement? You know, to some extent, what I meant is that I think, I think if I sat down and talked with most, you know, most people. You know, we could eventually, with some kind of constraints on reasonableness, come to agree that things could be a lot better in the world. Like, we could just cure cancer. We could cure, you know, X, Y, Z disease. We could just go through a few things like that. We could talk about the degree of abundance that could be available. And I think, you know.

• 19:31Speaker 01

  So, but the question is whether that's like a kind of structuring or important dimension to how people are relating to the world. I think you're right that it's often not. And that's part of maybe the thing I'm hoping to

• 19:46Speaker 01

  kind of push back against with that post is actually, I think this is a really important feature of our situation. I think it's true that it's, it can be dangerous. And if you're wrong about it, or if you're acting in the right, in a sort of unwise way with respect to it, that can be really bad. But I also think it's just, it's just a really basic fact. And I think we just sort of need to learn to deal with it maturely and kind of pretending it's not true. I think isn't the way to do that.

• 20:09Dwarkesh Patel?

  I see. But to me at least, utopian or utopia sounds like some sort of peak. And maybe you didn't mean it this way, but so are you saying in the essay and generally that you think there is some sort of carrying capacity to how much good things can get? Or that beyond a certain point, things can keep getting indefinitely better. But at this point, we're willing to say that we have reached utopia.

• 20:39Speaker 01

  Yeah. So I mean, I certainly don't have a kind of hard threshold, you know, here's, here's exactly where, where I'm going to call it utopia. You know, I mean, something that is profoundly better. I do think that if you have a finite, so you know, the very basic level. If there's only a finite number of states that the sort of effectable universe can be in. And your, your ranking of these states in terms of how good they are is transitive and complete. Then there will be a sort of top, a top. And, and you know, I don't think that's an important thing to focus on from the perspective of just getting it, just, you know, taking seriously that things could be radically better at all.

• 21:24Speaker 01

  I think like talking about, ah, but exactly how good and what's the perfect thing is, is often kind of distracting in that respect. And it gets into these issues about like, oh, you know, how much suffering is good to have? And, and, and a lot of this sort of discourse on utopia, I think gets distracted from basic facts about like, at the very least, we can do just a ton better. And that's important to keep in mind.

• 21:45Dwarkesh Patel?

  I see, I see. You point out in the piece that many religions and spiritual movements have done the most amount of thinking on what a utopia could look like. And you know, there's a very interesting essay by Nick Bosrom in 2008 where he lays out his vision of what somebody speaking from the future utopia, talking back to us would sound like. And when you read it, it sounds very much like a sort of mystical essay, the kind of thing that a change of few words and a Christian could write, like C.S. Lewis could have written about like what it's like to speak down from heaven. Yeah, so to what extent is there, and I don't mean this pejoratively, but to what extent is there some sort of like spiritual or religious dimension to utopian thinking?

• 22:30Dwarkesh Patel?

  That relies on some amount of faith that things can get in sort of indescribably better in some sort of ephemeral, indescribable way?

• 22:38Speaker 01

  So I think there are definitely analogues and similarities between some ways of relating to the notion of utopia and attitudes and orientations that are common in religious contexts and spiritual contexts. And I think it's, and I think personally, so I don't think it needs to be that like that. As I say, I think, I don't think it requires faith. I don't think it requires anything mystical. I don't think, I think this is, it's just a basic fact about our kind of current, you know, our current cognitive situation, our current civilizational situation, that things could be radically better.

• 23:19Speaker 01

  And it's a, you know, it's ephemeral in the sense that it's quite hard to imagine, especially, you know, for me, an important, an important source of evidence here is sort of variance in the quality of human experiences. So if you think about your kind of peak experiences, they're often, it's a really big deal. You're kind of, you're kind of sitting there going, wow, this is radically, this is serious. And kind of feeling in touch or feeling that this is, this is in some sense a, or. Something you would trade much, much sort of mundane experience for the sake of. And I think it's important. So the thing that I think we need to do is sort of extrapolate from there. So you sort of look at the trajectory that your mind moved along as you, as you moved into some experience

• 24:04Speaker 01

  or some broader non-experiential, like your community got a lot better, your relationships got about a lot better. Look at that trajectory and then sort of stare down, you know, where is that going? And I do think that requires a kind of, I don't want to call it faith. I think it requires a kind of extrapolation into a sort of zone that is in some sense beyond your experience, but that is sort of deeply worthy and important. And I think that's something that is often associated with spirituality and religion. And I think, I think that's okay.

• 24:36Speaker 01

  But I, I actually think there's a, there are a number of really important differences between utopia and something like heaven. So, you know, centrally, utopia will be a sort of concrete, limited situation. There, you know, there are going to be frictions, there are going to be resource constraints. It's going to be finite. There's, there's a bunch of, it's still going to be in the real world. Whereas I think many, you know, most religious visions have, don't have, don't have those constraints. And that's an important, an important feature of their, yeah, of their, their situation.

• 25:12Dwarkesh Patel?

  Yeah. Speaking of constraint, constraints, this reminds me of Robin Hanson's theory that, you know, eventually the universal economy will just be made up of these digital people Ms. And that because of competition, their wages will be driven down to system levels, which maybe that's compatible with some engineering in their ability to experience such that, you know, it's still blissful for them to work at assistance levels of compute or whatever. But yeah, so it seems like this sort of like the first order of economic thinking implies that there will be no, there will be no utopia. In fact, things will get, things will get worse for, on average, but maybe better overall if you just add up all the experience, but worse on average.

• 25:58Dwarkesh Patel?

  Yeah. So, so I don't know, this vision seems incompatible with yours of a utopia. What, what, what do you think?

• 26:05Speaker 01

  Yeah, I would not call Robin's world a utopia. And so, you know, a thing I haven't been talking about is what should our overall probability distribution be with respect to different quality of futures? And what, you know, exactly how possible is it? And how likely is it that we build something that is sort of profoundly good as opposed to mediocre or much worse? And I would class Robin's scenario in the mediocre or much worse zone.

• 26:39Dwarkesh Patel?

  But so do you have a criticism of the logic he uses to derive that?

• 26:44Speaker 01

  To some extent, I think my main, my main criticism or the first thing that would come to mind is that I think we will very likely, like I think competitive pressures are, are a source of kind of, kind of pushing, pushing the world in, in bad directions. But I also think there are ways in which kind of wise forms of coordination and kind of preemptive action can, can stave off the sort of bad effects of competitive pressures. And I, and, and so that's, that's a sort of, that's the way I imagine avoiding stuff in the vicinity of, of what Robin is talking about, though, you know, there, there are a lot of complexities there.

• 27:24Dwarkesh Patel?

  Yeah, yeah, the last few years have not reinforced my, my, my belief in the possibility of wise coordination, but yeah, anyways, so, yeah, one thing I want to talk to you about is you have a paper on what, what it would take to match humans' brains computational capacity. And then associated with that, you have, you know, a very good summary on open philanthropy. Yeah, so do you want to talk about the approach you took to estimate this? And then why, why this is an important metric to try to figure out?

• 27:57Speaker 01

  Yeah, so the approach I took was to look at the evidence from neuroscience and the literature on the kind of computational capacity of the human brain and to talk to a bunch of neuroscientists and to try to, you know, see, see what we know right now about the, the number of floating point operations per second that would be sufficient to kind of reproduce the task relevant aspects of human cognition in a computer. And that's important. I mean, it's actually not, you know, it's not clear to me exactly how important this parameter is to our overall picture. I think the way in which it's relevant to thinking that I've been doing and that Open Phil has been doing

• 28:38Speaker 01

  is as an input into an overall methodology for estimating when we might see a kind of human level AI systems that proceeds by first trying to estimate roughly the, the kind of computational capacity of the brain or the, or the sort of, the sort of size of, size of a kind of AI system and it's kind of overall parameter count and kind of compute capacity. And that would be sort of analogous to humans. And then you extrapolate from that to the training cost, the cost to kind of create a system of that kind using current methods in machine learning and kind of current scaling, scaling laws. And that methodology though brings in a

• 29:21Speaker 01

  number of additional assumptions that I think aren't, aren't like just transparent that that's, oh yeah, of course, that's how we would do it. And so I think you have to sort of be a little bit more in the weeds to see exactly how it, how it feeds in.

• 29:32Dwarkesh Patel?

  I see. And then, yeah, so what, I think you said it was 10 to the 15 flops for, for human rate. But like, did you have estimate for how many flops it would take to train, to train something like the human brain? I know GPT-3 is like, only 175 billion parameters or something, which is can fit into a, you know, like a micro SD card even. But, but yeah, it was like $20 million to train. So, yeah, so do you have, were you able to come up with some sort of estimate for how you, what it would cost to train something like this?

• 30:06Speaker 01

  Yeah, so my focus in that report was not on the training extrapolation. That was work that Ajay Kotri at Open Planty did using my report's estimate as an input. And that her methodology involves assigning different probabilities to different kind of ways of using that that input to to to drive an overall training estimate. And in particular, an important source of uncertainty there is the kind of amount of compute required or the sort of number of times you need to run a system per data point that it gets. So in the case of something like GPT three, you get a meaningful data point and a gradient update as to how well you're performing with each token that you

• 30:50Speaker 01

  output as you're doing GPT three style training. So you're, you know, you're predicting text from the internet, you know, you you suggest the next token and then your training process says like, nope, do better next time or something like that. Whereas if you're say learning to play Go and you have to play, I mean, this isn't exactly how or this isn't how a system will work, but it's it's an example. If you have to play the full game out and that's sort of hundreds of moves, then before you get an update as to whether you know, you're playing well or poorly, then that's a big multiplier on on the compute requirement. And so that's, that's one of the central pieces. That's called what Ajay calls the horizon length of of training. And that's a sort of very important source of uncertainty and getting to your overall overall training estimate, I think. But ultimately,

• 31:34Speaker 01

  you know, she ends up with this big spread out distribution from something like, I think GPT three was like 10 to the 24, yeah, four times 10 to the 23 or something like that. And, you know, she's she spreads out all the way up to the evolution anchor, I think, is something like 10 to the 41. And I think her distribution is centered somewhere in the low 30s.

• 31:54Dwarkesh Patel?

  Okay, that's that's still quite a bit, I guess. How much does this rely on, you know, the scaling hypothesis? If if one thought that the current efforts and the current approach were not not likely to lead in a or at least in not likely in a sample efficient way towards towards human intelligence, you know, it might be analogous to somebody saying we have enough deuterium on Earth to power civilization for millions of years. But but if you haven't figured out fusion, then it may be irrelevant statistic.

• 32:23Speaker 01

  Yeah, so I think the approach does assume that you can train a human level or sort of transformative AI system with a sort of non astronomical amount of compute and data using current, you know, without without major conceptual or algorithmic breakthroughs relative to what's currently available. Now, the the actual methodology of J uses allows you to assign probabilities to that assumption too. So you can, if you want, you know, say, I'm only 20% on that. And then you have, then there are sort of other, there are a few other options. So you can also kind of rerun evolution, which is not, and and and so that's that's an anchor that she provides to sort of,

• 33:07Speaker 01

  and this is often what people will say is a sort of upper bound on how hard it is to create, to create human level systems is is to do something, something analogous to to to simulating evolution. Though that, you know, there are a lot of open questions as to how how hard that is.

• 33:22Speaker 01

  But I do think this methodology is a lot more compelling and interesting if you are compelled by the the kind of available techniques in deep learning and by and by kind of scaling hypothesis like views, at least in as an upper bound, I think it's important. So, you know, there's different ways of of kind of being interested in algorithmic breakthroughs. One is because you think deep learning isn't enough. Another is because you think they will sort of provide a lot of efficiency relative to deep learning, such that an estimate like a J is is an overestimate. Because actually, you know, we won't have to do that. We'll make some sort of breakthrough and it'll happen a lot earlier. And and I put I put weight on that view as well.

• 34:03Dwarkesh Patel?

  Yeah, that's really interesting. So, yeah, that implies that like, even if you think the current techniques are not not optimal, maybe that maybe that should update you in favor of thinking it could happen sooner. That's that's really interesting.

• 34:15Dwarkesh Patel?

  Yeah, so yeah, then how did you go about estimating like the amount of flops it would take to emulate the interactions that happen in a brain? Obviously, it would be unreasonable to say that you have to emulate every single atomic atomic interaction. But then what is what is your proxy that you think it would be sufficient to emulate?

• 34:39Speaker 01

  So I used a few different methodologies and tried to kind of synthesize them. So one was looking at the kind of mechanisms of the brain and what we know about the kind of complexity of what they're doing and how hard it is to capture the kind of task relevant or our best, our best guess about the task relevant dimensions of the, the signaling happening in the brain. And then I also tried to bring in comparisons with existing AI systems that are replicating kind of chunks of functionality that humans, that the human brain has, and in particular in the context of vision. So sort of how do our, how do our current vision systems compare with the parts of the brain that are kind of plausibly doing analogous processing, though they're often,

• 35:23Speaker 01

  they're often doing other things as well. And then I used a third method, which has to do with physical limits on the kind of energy consumption per unit computation that the brain is possibly doing. And then a fourth method, I sort of gesture at, which tries to extrapolate from the communication capacity of the brain to its computational capacity using comparisons with, with current computers. So it's sort of a triangulation of like, you look at a bunch of different sources of evidence, all of which, in my opinion, are pretty weak. I think we're, we're quite, well, the, the physical limit stuff is maybe more complicated, but it's sort of a, a, a upper bound. I think we are significantly uncertain about all of this and, and I, my distribution is, is, is pretty spread out. But the hope is that by looking at a bunch of things at once, you can at least get a sort of educated guess.

• 36:08Dwarkesh Patel?

  And then, yeah, so I, I'm very curious. Is there consensus in neuroscience or other relevant fields that we understand the signaling mechanisms well enough that we can say like, basically this is what it's involved? This is what the system is reducible to? And yeah, so this is how many bits you need to represent, I don't know, the synaptic connections here? Or is there a variance of opinion about like, just how complicated the, the enterprise is?

• 36:35Speaker 01

  There's definitely disagreement and it was, you know, interesting and in some sense disheartening to talk with neuroscientists about just how, you know, how difficult neuroscience is, you know, it's sort of, I think it's easy, a consistent message, and I have a section on this in the report, was kind of how far we are from really understanding what's going on in the brain, especially at a kind of algorithmic level. That said, so in some sense, the report is somewhat opinionated in that, you know, there are experts that I found more compelling than others. There are experts who are much more in a sort of agnosticism mode of like, we just don't know. You know, the, the brain is really, really complicated, who sort of err on the side of

• 37:18Speaker 01

  very large compute estimates, a lot of emphasis on biophysical detail, a lot of emphasis on sort of mysterious things that could be happening that aren't happening. And then there are other neuroscientists who are more, you know, more willing to say stuff like, well, we, we kind of basically know what's, what's going on at a mechanistic level, which isn't the same as knowing kind of the algorithm, the sort of algorithmic organization overall and how to replicate it. I sort of lean towards the latter view, though I give weight to both and, and, and try to, yeah, try to synthesize the, the kind of opinions of people I, I, I saw overall.

• 37:51Dwarkesh Patel?

  Just looking at the, the post itself, I, I haven't really looked deeper into the actual, the, the, the paper for which it's derived. But it seemed like you were to estimate the floss mechanistically, you were adding up the different systems at play here. Yeah, so should we expect it to be additive in that way? Or maybe it's like multiplicative, or there's more complicated interaction? Like the flops grow super linearly to the inputs? I, I know that probably sounds really naive, having studied it, but just like from a first glance kind of way, that, that, that's a question I had.

• 38:26Speaker 01

  Yeah, so the the way I was understanding and breaking down the forms of processing that you you would need to replicate in the brain made made them seem not multiplicative in this way. So, you know, an example would be if you think about maybe, yes, sort of simple examples. Suppose we have some neurons and they're, you know, they're signaling centrally via spikes through synapses or something like that. And then we have glial cells as well, which are signaling via like slower calcium waves. And it's a sort of separate, separate network. You know, you could think that if it were something like, you know, the the rate of calcium signaling is

• 39:07Speaker 01

  dependent on the rate of spikes through synapses or something like that, then that's an important interaction. But, you know, overall, if you if you sort of imagine like this, this kind of network processing, these are kind of in the, you can just, you you can estimate them independently and then and then add it up. It's they're not, they're not actually multiplicative processes on that, on that conception. I do think there are kind of correlations between the estimates for for the different parts, but I it it's sort of additive at a fundamental level.

• 39:36Dwarkesh Patel?

  I see. Okay. And then, yeah, how how much credence do you put in the sort of a a a almost woo woo hypotheses that I don't know Roger Hippen was as that thing about there's something like something quantum mechanical happening in the brain. That's very important in for understanding cognition. Yeah, to what extent do you do you put credence in those kinds of hypotheses?

• 39:58Speaker 01

  I put very little credence in those hypotheses.

• 40:03Speaker 01

  Yeah, I don't see a lot of reason to think that. I see a good amount of reason not to think it.

• 40:11Speaker 01

  But it wasn't something I dug in on a ton.

• 40:14Dwarkesh Patel?

  Okay, gotcha. All right. So you have this really interesting blog post about infinite ethics. Do you want to talk about why this is an important topic? Why it's important to integrate into our worldview and so on?

• 40:25Speaker 01

  Sure, so infinite ethics is ethics that tries to grapple with how we should act with respect to kind of infinite worlds and how should we, you know, how should we rank them? How should they enter into our our expected utility calculations or attitudes towards risk? And I think this is important for both kind of theoretical and practical reasons. So I think at a theoretical level, when you when you try to do this with a lot of common ethical theories and constraints and principles, they just break on on infinite worlds. And I think that's that's an important clue as to their viability, because I think infinite worlds are at the very least possible, even if our world is finite, and even if our causal

• 41:08Speaker 01

  influence is finite or our influence overall is finite, it's possible to have infinite worlds, and we have opinions about them, you know, like an infinite heaven is better than an infinite hell. And, you know, so I think often in ethics, we we expect our ethical principles to extend to kind of rankings scenarios or sort of acting and hypothetical scenarios or overall kind of to all possible situations rather than just our actual situation. I think infinities come in there. But then I think maybe more importantly, I think it's a it's an issue with practical relevance. And a way to see that is that, you know, I think we should have non-zero credence that we live in an infinite world. And, you know, it's a very live physical hypothesis that the universe is infinite, even if I

• 41:53Speaker 01

  think the mainstream view is that our causal influence on that universe is finite in virtue of things like entropy and light speed and stuff like that. But the the universe itself may well be infinite in, you know, and possibly infinite in a number of different ways. If that sort of Max Tegmark has some work on all the different kind of like large, you know, ways the universe can be really very large. There's a number of ways that I think it's just we should have non-zero credence that that we we can have infinite influence within our actions now. So, you know, our kind of The causal influence are the limitations there could be wrong. It may be that there are ways, you know, in the future, we'll be able to do infinite things. And then I also think somewhat more

• 42:35Speaker 01

  exotically that it's there. There are sort of ways of having a causal influence on an infinite universe, even if you are limited in your causal influence. And that comes from some additional work I've done on decision theory. And so if you try to incorporate that, if you're a sort of expected value reasoner, it just very quickly starts to dominate, or at least break your expected value calculation. So, you know, you mentioned long termism earlier. And, you know, a natural reason, a natural argument for getting interested in long termism is, oh, you know, in the future, there could be all these people, their lives are incredibly important. So if you do the EV calculation, sort of your effect on them is what dominates. But actually, if you have even a tiny credence that you can do an infinite thing, you know, either that dominates or it breaks. And then if you have tiny credences on doing different types of infinite things and you need to compare them,

• 43:21Speaker 01

  you need to know how to do it. And so I just think this is actually, you know, it's actually a part of our of our epistemology now. Though it's, I think we often don't don't treat it that way because we're often not doing EV reasoning or really thinking thinking about that. That that these are questions that that just apply to us.

• 43:39Dwarkesh Patel?

  Yeah, yeah, so that's that's super fascinating.

• 43:44Dwarkesh Patel?

  If it is the case that we can only have an impact on a finite amount of stuff, then maybe it is true that like there's infinite suffering or happiness in the universe at large. But the delta between the best case scenario for what we do and the best worst case scenario is finite. But yeah, I don't know, that still seems less compelling if the the hell or heaven we're surrounded by is overall not, yeah, it doesn't change.

• 44:10Dwarkesh Patel?

  Can you talk a bit more? I think you mentioned in your other work on having impact, having infinite impact beyond the scope of what light, speed and entropy would allow us. Okay, can you talk a bit more about how that might be possible?

• 44:23Speaker 01

  Sure, so, um, you know, a common decision theory, though it's not, I think the mainstream decision theory is a contender in the literature, is evidential decision theory, where you should act such that you would be, you know, roughly speaking, happiest to learn that you had acted that way for that reason. And so the reason this allows you kind of a causal influence, so, you know, a way of thinking about it is suppose that you are a deterministic simulation, and there's a copy of you being run sort of too far away for for you to ever causally interact with it, right? But you know that it's a sort of, you know, it's it's a deterministic copy,

• 45:06Speaker 01

  and so it'll do exactly what you do, absent some sort of computer malfunction. And now you're deciding whether to give, you know, you have two options. You can send a million dollars to that, well, it's a little complicated because he's too far away. But, um, you know, just in general, like if I raise my hand, or if I want to write stuff on my whiteboard, right? Or if I'm going to, you know, there's, let's say, I have to make some ethical decision, like whether I should take an expensive vacation, or I should donate that money to save someone's life. Because that, the the other guy is going to act just like I do, even though I can't cause him to do that. In some sense, when I, when I make my choice, after doing so, I should think that he made the same choice. And so evidential decision theory treats his action as in some sense under my control.

• 45:52Speaker 01

  And so if you imagine an infinite universe where there are an infinite number of copies of you, or even not copies, people whose actions are correlated with you, such that when you act a certain way, that gives you evidence about what they do. In some sense, their actions are under your control. And so if there are an infinite number of them on evidential decision theory and a few other decision theories, then in some sense, you're having influence influence on the universe.

• 46:15Dwarkesh Patel?

  Yeah, this sounds really similar to the thought experiment in quantum mechanics called EPR pair, which you might have heard of, but the basic idea is if you have two entangled bits and you take them very far away from each other. And then you measure one of them and you do like before they're brought apart, you come up to some rule that like, hey, if it's plus, we do this, if it's minus, we do the other thing. It seems at first glance that measuring something yourself has an impact on what the other person does, even though it shouldn't be allowed

• 46:47Dwarkesh Patel?

  by light speed. It gets resolved if you take them any world's view, but yeah, yeah. So that's very interesting. Is this just a thought experiment or is this something that we should anticipate for some cosmological reason to actually be a way we could have influence on the world?

• 47:04Speaker 01

  So I haven't dug into the cosmology a lot, but my understanding is that it's at the very least a very live hypothesis that the universe is infinite in the sense that there are, you know, it's a sort of infinite in extent and there are suitably far away, there are copies of us having just this conversation and then, you know, even further away, there are copies of us having this conversation, but wearing raccoons for hats and, you know, and all the rest. Which, you know, is itself something to wonder about and sit with, but I, you know, my understanding is this is, this is just a live hypothesis. And then more broadly, kind of infinity is playing, you know, infinite universes are just sort of a part of mainstream cosmology at this point.

• 47:47Speaker 01

  And so, yeah, I think, I don't think it's just a thought experiment. I think infinite universes are alive. And then I think, you know, these sort of non-causal decision theories are actually my sort of best guess decision theories. Though that's not a mainstream view. So it's fairly, I think it comes in fairly directly and substantively if you have that combination of views. But then I also think it comes in, I think everyone should have non-zero credence in all sorts of different infinity involving hypotheses. And so infinite ethics gets a grip regardless.

• 48:17Dwarkesh Patel?

  I see. And then.

• 48:21Dwarkesh Patel?

  So taking that example, if you're having an impact on every identical copy of yourself in the infinite universe, it seems that for any such copy, there is infinite amount of other copies that are slightly different. So it's not even clear if you're increasing, maybe it makes no sense to talk about proportions in an infinite universe. But you know, if there is another infinite set of copies that scribbled the exact opposite thing on the whiteboard, then it's not clear that you had any impact on the total amount of good or bad stuff that happened. I don't know, my brain breaks here, but maybe you can help me understand this.

• 48:58Speaker 01

  Yeah. So I mean, there's a general, I think there's a couple of dimensions here, there. So one is trying to understand actually what sort of difference does it make if you're in this sort of infinite situation and you're thinking about a causal influence. What even did you change at a sort of empirical level before you talk about how to value that? And I think that's a pretty gnarly question. Even if we've settled that question, though, in terms of like the empirical causal impact, there is a further question of how do you rank that? Or how do you deal with, you know, the sort of the normative dimension here? And there, you know, so that's the sort of ethical question. And there, things get really

• 49:40Speaker 01

  gnarly, very fast. And, you know, so, and in fact, there are kind of impossibility results that show that even very basic constraints that you really would have thought that we could get at the same time in our ethical theories, you can't get them at the same time when it comes to infinite universes. And so we know that something is going to have to go and change if we're going to extend our ethics to infinities.

• 50:09Dwarkesh Patel?

  I see. But then, yeah, so is there some reason he was settled on, I guess you mentioned you're not a utilitarian, but on some version of EA or long-termism as your tentative moral hypothesis, despite the fact that this seems unresolved? And then like, how do you settle that tension while tentatively remaining in EA?

• 50:32Speaker 01

  Yeah, so I think there's a, there's two dimensions there. One is that I think it's, um, I think it's good practice to not totally upend your life and do, and, and, you know, if you encounter some destabilizing philosophical idea, especially one that's sort of difficult and, you know, you don't totally have a grip on it to, to then come back.

• 50:49Dwarkesh Patel?

  But isn't, isn't that what long-termism is?

• 50:52Speaker 01

  Yeah. So I think there's a real tension there in that I think, um, many, you know, how seriously should we take these ideas? At what point should you be making what sorts of changes for your life on the basis of different, different things that you're, um, uh, you're thinking and believing, you know, it's a real art, right? And I think some people go, you know, they grab the first idea they see, um, and they start doing crazy stuff and, uh, in an unwise way. And some people are too, um, it's kind of sluggish and they, they're not willing to take ideas seriously and not willing to reorient their, their life on the basis of, of, uh, changes and what, and what seems true. Um, but I think, you know, nevertheless, I think especially things that involve like, ah, turns out it's fine to, you know, do terrible things or, you know, there's no reason to eat your lunch or whatever, like things that,

• 51:37Speaker 01

  you know, sort of really, really holistically breaking of your ethics views. I think, I think one should, should tread very cautiously with. Um, so that's one aspect. At a philosophical level, um, the way I resolve it is I think for many of these issues, uh, the right path forward, or at least a path that looks pretty good is to, um, survive long enough for our civilization to become much wiser. Um, and if, as, and, and then to use that position of wisdom and empowerment, uh, to act better with respect to these issues. Um, and so, and that's what I say in the end of the infinite ethics post is that, um,

• 52:13Speaker 01

  you know, I think future civilization, if all goes well, will be much better equipped to deal with this. Um, and, you know, we are at, we are at square one and kind of really understanding how, how, how these issues play out and how to respond. And so, uh, I think both at an empirical level and at a, at a kind of philosophical level. Um, and so it, it looks convergently pretty good to me to survive, become wiser, keep your options open and then act from there. Um, and that looks, that ends up pretty similar to a lot of long-termism and existential risk. It's just that it's focused less on, and the main event will be what happens to future people. And it's, it's more about getting to the point where we are wise enough to understand and reorient, um, in a better way.

• 52:53Dwarkesh Patel?

  Okay, um, yeah, so what if I'm really interesting about this is that you can, um, yeah, you said, uh, different people tend to have like different thresholds for, um, epistemic learned helplessness, where they basically say, this is too weird. I'm not going to think about this. Uh, let's just stick with my current, uh, current, uh, moral theories. Um, so for, for somebody else, it might be before they became a long-termist, where it's just like, yeah, trillions of future people, what are we talking about here? Let's, uh, we're not changing my mind on stuff. And then, yeah, for, for you, maybe it's before the infinite ethic stuff. Um, uh, is, is there some principled reason for thinking that this is where that stop should be? Or is it just a matter of like temperament and openness?

• 53:36Speaker 01

  So I don't think there's a principled reason. And, and I should say, I don't think of my attitude towards infinite ethics as solely, oh, this has gotten too far down the crazy, the crazy path. I'm out. Um, it is, this thing about the wisdom in the future is pretty important to me. Um, as a, as a, a reason to, uh, as an, as a mode of orientation. A first pass cut that I use is when do you feel like it's real? Um, if you feel like a thing is real, uh, as opposed to a kind of abstract fun argument, um, then that's important, or that's, that's a real signal. And I think I, so, um, uh, and I generally

• 54:16Speaker 01

  encourage people, if, if the sort of mode that I, I, I, I don't know, I'm drawn to is something like, if there's an idea that seems compelling intellectually, that's a reason to investigate it a lot and think about it and really grapple with, you know, if, if this doesn't seem right to you, or if it seems too crazy, why? Um, and really kind of processing, you know, it's a reason to pay a lot of attention. But if you've paid a lot of attention, at the end of the day, you're like, well, I guess at an abstract level, that's sort of makes sense, but it just doesn't feel to me like the real world. It just doesn't feel to me like, um, wisdom, or you're like a healthy way of living or whatever. Then I'm like, well, maybe you shouldn't do it, right? I mean, and, and, uh, and I think some people will do that wrong, and they will end up bouncing off

• 54:57Speaker 01

  of ideas that are in fact good. Um, but, you know, I think overall, these, these are sort of sufficiently intense and difficult issues that, um, uh, kind of being actually persuaded and not just sort of chopping off the rest of your epistemology for the sake of some like version of the abstraction, uh, is, it seems to me important and it's, and it's a sort of a healthier way to relate.

• 55:23Dwarkesh Patel?

  Yeah, so another example of this is that you have this really interesting blog post on ants. And your thoughts after sterilizing a colony of them. So I, yeah, so this is another example of a thing where almost everybody, other than, I don't know, maybe a Jane who wears a face mask to prevent bugs from going into his mouth, would say like, okay, at this point, if we're talking about how many hedons are in a hectare of forest from all the millions of insects there, then you've lost me. But then, you know, somebody else might say, okay, well, there's not a strong reason for thinking they have no, absolutely no capacity to feel suffering.

• 56:06Dwarkesh Patel?

  Yeah, so I wonder how you think about such questions because you can't like stop living and not, you're not even gonna stop going on road trips where you're probably killing hundreds of insects by just driving. But yeah, so what do you think about such conundrums?

• 56:22Speaker 01

  I have a significant uncertainty about, you know, exactly, and I think this is the appropriate position about exactly how much kind of consciousness or suffering or other forms of moral, you know, other ways, other kind of properties that we associate with moral patienthood, how much those apply to different, different types of insects. I think it's a strange view to be, you know, extremely confident that what happens with insects is totally morally neutral. And I think it actually doesn't fit with our common sense. So let's say you see, if you see a child like frying ants with, with a magnifying glass, I think we, you know, there is some, you know, what, you could say, ah, well, that just indicates that they're going to be cruel to other things that matter. But I don't think so.

• 57:07Speaker 01

  I think, you know, and you see the ants like, you know, and they're twitching around and, and I, so I think we aren't, you know, as in many cases with, with animal ethics, I think we're a bit like kind of schizophrenic about, about what cases we view as sort of morally relevant and which, which not, you know, we have, we have, you know, pet treatment laws and then we have factory farms and stuff like that.

• 57:29Speaker 01

  So.

• 57:31Speaker 01

  I don't see it as a radical position that ants matter somewhat. I think there's a further question of what your overall practical response should to that should be. And I think the main, and I do think the kind of costs, as in a lot of ethical life, there are trade offs and you have to make, you have to make a call about what, what sort of constraints you're going to put on yourself at the cost of other goals. And in, you know, in the case of insects, it's not my, my current moral focus and I don't pay a lot of costs to kind of, to lower my impact on animals and I don't, you know, I don't, I don't sweep the sidewalk or any, or sorry, on, on ants in particular.

• 58:11Speaker 01

  And so I think it's, I think, and I think that's, you know, that's my best guess response. And that, and that has to do with other ethical priorities to my life. But I think, you know, there's, there's a middle ground between I shall ignore this completely and I shall, you know, be a Jane, which is recognizing that this is a, this is a real trade off. There's uncertainty here and, and taking responsibility for how you're responding to that.

• 58:34Dwarkesh Patel?

  Yeah, this seems kind of similar to the infinite ethics example, where if you put any sort of credence that they have any ability to suffer, then at least if you're not going to say that, oh, it doesn't matter because like the far future, trillions and trillions of ants. It seems like this should be a compelling, a compelling thing to think about. But then the result is, um, yeah, it's, it's not even like becoming a vegan where it's like you change your diet.

• 59:08Dwarkesh Patel?

  And then so, you know, as you might know, this is used as a productive ad absurdum of veganism, where, you know, if you're going to start caring about other non-human animals, why not also care about insects? And even if they're worth like a million of a cow, then you're, you're probably still killing like a million of them on any given day from all your activities indirectly, maybe. Well, like, I don't know, like the food you're eating, all the pesticides that are used to create that food. But I, I don't know how you go about resolving that kind of stuff.

• 59:35Speaker 01

  I mean, I guess I'd want to really hear the empirical case. I think um, I think it's true, you know, there are a lot of insects uh and but you know, I think it's easy. Uh, you know, I think if you want to say like, ah, taking seriously sort of the idea that um there's some reason to to not like squash squash a bug um if you see it leads immediately to kind of Jane like behavior uh absent long termism or something like that. I I really, I I feel like I want to hear the empirical case about like exactly what impact you're having and and how um and and I'm not at all persuaded that that's the practical upshot um. Uh, and if it is, if that's a really strong case, then I think that's an interesting um, an interesting uh

• 1:00:18Speaker 01

  uh, you know, that's an interesting kind of implication of of this view um and uh and you know, worth worth concern, but I wouldn't jump. It feels to me like it's easy to jump to that almost out of a desire to to get to the reductio um without kind of I would try to move slower and and really see it's like, wait, is that right? There are a lot of trade offs here. What's the source of my hesitation about that? Um, and kind of uh and not not jump too quickly to something that's sufficiently absurd that I can be like, ah, therefore I get to reject this whole mode of thinking, even though I don't know why.

• 1:00:49Dwarkesh Patel?

  I see, yeah. Um, okay, so let's talk about uh the two different ways of thinking about observed effects and their implications. So do do you wanna uh explain um? You you have a four part series on this, but do you want to explain uh this self vindication assumption and the self sampling assumption? Uh, I I know it's a big topic, but uh, yeah, as much as possible.

• 1:01:14Speaker 01

  Sure. So I think one way to start to get into this debate is by thinking about the following case. So um, you wake up in a white room and there's a message written on the wall. And let's say you're going to believe this message and the message says, I God, it's from God. Uh, I God created, I I flipped a coin. Um, and if it was heads, I created one person in a white room. And if it was tails, I created a million people all in white rooms. And now you are asked to assign probabilities uh to uh the coin having come up heads versus tails.

• 1:01:50Speaker 01

  And so one approach to this question, which is the approach I favor, or at least think is better than the other, is the self-indication assumption. These names are terrible. And but, you know, so it goes. So SIA says that your probability that the coin came up heads should be approximately one in a million. And that's because SIA thinks it's more likely that you exist in worlds where there are more people in your epistemic situation, or more people who have your evidence, which in this case is just waking up in this white room. And that's it. And so that's that can be a weird conclusion and go to weird places. But I think it's a better conclusion than the alternative.

• 1:02:32Speaker 01

  SSA, which is the main alternative I consider in that post, which is the self-sampling assumption,

• 1:02:39Speaker 01

  says that you should, you think it more likely that you exist in worlds where people with your evidence are a larger fraction of something called your reference class,

• 1:02:52Speaker 01

  where it's quite opaque what what a reference class is supposed to be. But broadly speaking, a reference class is the sort of set of people you could have been, or that's kind of how it functions in SSA's discourse. So in this case, in both cases, everyone has your evidence. And so the fraction is the same.

• 1:03:14Speaker 01

  And so you you stick with the one half prior. But that's not true. So SSA, in other contexts, not everyone has your evidence. And so it updates towards worlds where it's a larger fraction. So famously, SSA leads to what's known as the doomsday argument, where you imagine that there are two possibilities. Either humanity will go extinct very soon, or we won't go extinct very soon, and there will be tons of people in the future. And in the former case, and then you imagine everyone is sort of ranked in terms of when they're born.

• 1:03:49Speaker 01

  In the former case, people born roughly this time are a much larger percentage of all the people who ever lived. And so if you imagine, you know, God first creates a world and then he inserts you randomly into like some group. It's much more likely that you would find yourself in the 21st century. If humanity goes extinct soon than if it's, if there are tons of people in the future. If God randomly inserted you into these tons of people in the future, then it's like, really, it's a tiny fraction of them are in the 21st century. So SSA in other contexts, actually, you know, it has these important implications, namely that in this case, you update very, very hard towards the future being short. And that matters a lot for long-termism because long-termism is all about the future being big in expectation.

• 1:04:36Dwarkesh Patel?

  Okay, so and then what is the SIA take on this?

• 1:04:40Speaker 01

  Yeah, so I think a way to think about SSA's kind of story. So I gave this story about SSA, which is, it's sort of like this, it's like first God creates a world, this is SSA. First he creates a world and then he takes and he's dead set on putting you into this world. So he's got your soul, right? And he really wants, and your soul is going in there no matter what, right? But the way he's going to insert your soul into the world is by throwing you randomly into some set of people, the reference class. And so if you wake, so you should expect to end up in the world where the kind of person you end up as is, is sort of

• 1:05:22Speaker 01

  more like a more likely result of that throwing process is a sort of larger fraction of, of the total people you could have been. What SSA or what SIA thinks is different, the way, the story that I'll use for SIA, though it doesn't assist in the only class, is God decides he's going to create a world. And then he, and say that there's like a big line of souls in heaven. And he goes and grabs them kind of randomly out of heaven and puts them into the world, right? And so in that case, if there are more people in the world, then you've got more shots. And you're one of these souls, you're sort of sitting in heaven, hoping to get created.

• 1:06:01Speaker 01

  On SIA, God has more chances to grab you out of, out of heaven and put you into the world if there are more people who, more people like you in that world. And so you should expect to be in the world where there was sort of, there are more such people and that's, that's kind of SIA's vibe.

• 1:06:18Dwarkesh Patel?

  Doesn't this also imply that you should be in the future, assuming there will be more people in the future?

• 1:06:22Speaker 01

  Tell me more about why, why would I apply that?

• 1:06:25Dwarkesh Patel?

  Okay, in an analogous scenario, maybe like go back to the God tossing the coin scenario, where if you did just substitute for people in white rooms, you substitute being a thing, a conscious entity, and if there's going to be more conscious entities in the future, like you would really expect to, just like in that example of being in that scenario where there's a lot more rooms, just as maybe you should expect you to be in that scenario where there's a lot more conscious beings, which presumably is the future. So then it's still odd that you're in the present, under SIA?

• 1:06:57Speaker 01

  Yes, so in a specific sense, so it's true that on SIA, say that,

• 1:07:07Speaker 01

  say that we don't know what room you're in first, right? So, so you wake up in the white room and you're wondering, am I in room one? Or am I in rooms two through a million, right? And on SIA, what you did first, so you woke up and you don't know what room you're in, but there's a lot more people in the world with lots of rooms. And so you become very, very confident that you're in that world, right? So you're very, very confident on tails. And then you're right that conditional on tails, you think it's much more like you sort of split your credence evenly between all these rooms. So you are very confident that you're in one of the sort of two through a million rooms and not, not room one. But that's before you've seen your room number.

• 1:07:53Speaker 01

  Once you see your room number, it's true that you should be quite surprised about your room number. But the, once you get the room number, you're back, you're back to 50/50 on heads versus tails. Because you had sort of equal credence in being in room one, conditional on tails. Or sorry, you had equal credence in being in tails in room one and heads in room one. And so when you get rid of all of the other tails and rooms, two through a million, you're left with 50/50 overall on heads versus tails. And so The sense in which SIA leaves you back at normality with the doomsday

• 1:08:34Speaker 01

  argument is once you update on being in the 21st century, which admittedly should be surprising. Like if you didn't know that you were in the 21st century and then you learned that you were, you should be like, wow, that's really unexpected. And that's true, but I think once you do that, you're back at, you know, whatever your prior was about extinction.

• 1:08:53Dwarkesh Patel?

  Maybe I'm still not sure on why the fact that you were surprised should not itself be the doomsday argument.

• 1:09:00Speaker 01

  Yeah, I think there's an intuition there, which is sort of like, yeah, is SIA making a bad prediction? So you could, you could kind of update against SIA because SIA would have predicted that you're in the future. I think there's something there and I think there's a few other analogs. Like, for example, I think SIA naively predicts that, you know, you should find yourself in a situation where there are just tons of people that, you know, a situation obsessed with creating people with your evidence. And, you know, this is one of the, one of the problems with SIA. So you should expect to find, you know, in every nook and cranny, a simulation of you. As soon as you like, you know, you open the door, it's actually this giant bank of simulations of you in like your previous epistemic state. And so, you know, I think there are, and then you don't see that.

• 1:09:44Speaker 01

  You might be like, well, I should update against the anthropic theory that predicted that I would see that. And I think there are arguments in that vein.

• 1:09:51Dwarkesh Patel?

  Yeah, so maybe let's back up to go to the original example that was used to distinguish these two theories. Yeah, so can you help me resolve my intuitions here? Where my intuition is very much as to say, because, yeah, it seems to me that you knew you were going to wake up, right? You knew you were going to wake up in a white room. Before you actually did wake up, your prior should have been like one half heads or tails. So it's not clear to me why, having learned nothing new, your posterior probability on either of those scenarios should change.

• 1:10:24Speaker 01

  So I think the the SIA response to that would be, or at least I think the way of making it intuitive would be to say that you didn't know that you were going to wake up. Right? So in the, if we go back to that just so story where God is grabbing you out of the, out of heaven.

• 1:10:42Speaker 01

  You know, it's, it's not at all. It's actually incredibly unlikely that he grabs you. There are so many, so many people. Yeah, I mean, there's a different thing where SIA is in general very surprised to exist. And in fact, that's the, so you could make the same arguments like SIA says you shouldn't exist. Isn't that weird that you exist? And I actually think that's a, that's a good argument.

• 1:11:02Speaker 01

  So, but once you're in that headspace, then I think the way, the way to think about it is that it's not a guarantee that you, you were, God is not dead set on creating you. You are a particular contingent arrangement of the world. And so that that you should expect that arrangement to, to come about more often if there are more arrangements of that type. Rather than sort of assuming that, that no matter what, existence will include you.

• 1:11:28Dwarkesh Patel?

  Yes. Okay. Can you talk more about the problems with SSA like scenarios where you think it breaks down, like why you prefer SIA?

• 1:11:36Speaker 01

  Yeah. So an easy, uneasy problem, or sort of one of the most dramatic problems is that SIA, SSA predicts that it's possible to have a kind of telekinetic influence on the world. So imagine that there's a, there's a puppy. You, you're, you're in, you, you wake up and you're in an empty universe, except for this puppy and, and you. And this boulder that's rolling towards the puppy, right? And the boulder is inexorably going to kill the puppy. It's very large boulder. It's basically guaranteed that the puppy is dead meat. But you have the power to make binding pre commitments that you will in fact execute. And, and you have also to your right, a button that would allow you to create tons of people, like zillions and zillions and zillions of people, all of whom

• 1:12:22Speaker 01

  are wearing different clothes from you. So they would be in a different epistemic state than you if you, if you created them. Now SSA, so you, so you make the following resolution. You say, If this boulder does not jump out of the way of this puppy, like the boulder leaps, you know, in some very weird, very unlikely way, then I will press this button and I will create zillions and zillions of people, all of whom are in a different epistemic state than me, but let's assume they were in my reference class. SSA thinks that it's sufficiently unlikely that you would be in a world with zillions of those people. And, but you, you know, you at the very beginning with, with a different, with different colored clothes.

• 1:13:07Speaker 01

  Because, you know, that was a tiny fraction of the reference class if those people get created. That SSA thinks it's actually more likely, once you've made that commitment, that the boulder will jump out of the way. And, and you know, so that, and that looks weird, right? It's, it's just seems like that's not going to work. You can't, you can't just make that commitment and then expect the boulder to jump. And you get, so that, so that's a sort of exotic example. You get, you get similar analogues even in the God's coin toss case where, like naively, it doesn't actually matter whether God has tossed the coin yet, right? So suppose, yeah, so like, let's say, let's say you wake up and learn that you're in room one, right? But God hasn't tossed the coin. It's like he created room one first before he tossed,

• 1:13:48Speaker 01

  and then he's going to toss, and that's going to determine whether or not he creates all the rooms in the future. If you, on SSA, once you wake up and learn, learn that you're in room one, you think it's incredibly unlikely that there's going to be these future people. So you, now you say, before, it's a fair coin, God's going to toss it in front of you. You're still going to say, I'm sorry, God, it's, you know, it's a one in a million chance that this, that this coin lands tails. And, or sorry, one in a million, something very small number, I forget exactly. And that's, and that's very weird, that's a fair coin, it hasn't been tossed. But you, with the power of SSA, have become extremely confident about, about what, how it's going to land. And that's, so that's, that's another argument. There's, there's a number of other, I think, really, really bad problems for SSA.

• 1:14:34Dwarkesh Patel?

  Yeah, while I digest that, so let me, let me just mention the problems you already pointed out against SIA in the post and earlier where if one thinks SIA is true, one should be very confident that there are, you're in a universe with many other people who have been sampled just like you. And so then it's, then it's kind of surprising that we're in a universe that is not filled to the brim with people. Like, there's a lot of, you could imagine, like Mars is just completely made up of bodies, or, you know, like every single star has like,

• 1:15:15Dwarkesh Patel?

  you know, a simulation of a trillion people inside. The fact that this is not happening seems like, it seems like very strong evidence against SIA. And then, you know, there's other things like the presumptuous philosopher that you might want to talk about as well. But yeah, so do you just bite the bullet on these things? Or how do you think about these things?

• 1:15:35Speaker 01

  My main claim is that SIA is better than SSA. And, and I think it's just a horrible situation with anthropics. And, and it's, I think overall SIA is an update towards bigger, more populated universes. I think, you know, the most salient populated universes don't involve like hidden people on other planets, but they're probably, I don't know, maybe we're in a simulation and people are, you know, obsessed with simulating us or, or something like that. Or, and then I think this is actually more important and worrying is, I think, the way I see this dialectic is, first, SIA, I mean, it's a big problem with SIA is it immediately becomes certain naively that you live in an infinite universe, or a universe with an

• 1:16:19Speaker 01

  infinite number of people. And that, and then it breaks, because, and it doesn't know how to compare, kind of infinite universes. Now, to be fair, SSA also isn't great at comparing infinite universes. And they both have some, you can do things that are actually quite analogous to things you can try to do in infinite ethics, where you have like expanding spheres of spacetime, and you count, you know, you have some fraction or some density of people in those spheres. And there's this general problem in cosmology of like, trying to understand what it means to have like a fraction or a density of, of different types of observers. But you know, my own take is kind of what happens here is you we hit this infinity.

• 1:16:59Speaker 01

  You hit infinite universes fairly fast and then they kind of break your anthropics in analogous ways to how they break your ethics. And that's kind of where I'm currently at, and I'm hoping to understand better how to do anthropics with infinities. And some of my work on the universal distribution, which is sort of, I have a couple blog posts on that was attempting to go a little bit in that direction, though it has its own giant problems.

• 1:17:26Dwarkesh Patel?

  Okay, interesting. Do you know if, just vaguely, it seems to me that the Robin Hanson's grabby aliens thing probably uses SSA. But do you know if that's the case, if he's using SSA in there?

• 1:17:41Speaker 01

  I don't. I haven't looked closely at that work.

• 1:17:43Dwarkesh Patel?

  Okay, okay, cool. I don't know. It's hard for me to think about it. It may be able to take me a few more weeks before I can digest it fully. But yeah, okay, so that's really interesting. You have a really interesting blog post about believing in things you cannot see. And one, I mean, this is almost an aside in the post itself, but I thought it was a really interesting comment. You make an interesting comment about futurism. Here's what you say, "Much of futurism, in my experience, has a distinct flavor of unreality. The concepts mind uploads, nanotechnology, settlement, and energy capture in space are, I think, meaningful, even if loosely defined. But at a certain point, once models become so abstracted and incomplete, that the sense of talking about a real thing, even a possibly real thing, is lost."

• 1:18:28Dwarkesh Patel?

  Yeah, so why do you think that is? And is there a way to do futurism better?

• 1:18:33Speaker 01

  I think it comes partly because imagination is just quite a limited tool. And it's just easy, you know, when you're talking about the whole, like, the future is a big thing to try to model with this tiny mind. And so, you know, of necessity, you need to use these extremely lossy abstractions. And so, you know, it puts you in a mode of having these, like, you know, really sketchy and gappy maps that you're trying to manipulate. I think that's one dimension, and then I think there's also a way in which.

• 1:19:08Speaker 01

  You know, this isn't all that unique to futurism insofar as just in general, I think it's hard sometimes to keep our intellectual engagement kind of rooted and grounded in the the kind of real world. And I, you know, I think it's just easy to kind of move into a zone, and especially if that zone is inflected with kind of social dynamics, or you're it's, you know, it's it's kind of like a intellectual game, or you're enjoying it for its own sake, or it's like a sort of there's sort of status dimensions in the way people talk and other things that I think start to move our our discourse in in directions that aren't about like, ah, we're talking about the real world right now. Let's actually get it right. And I think that happens with futurism. And and maybe more so because it can feel like

• 1:19:51Speaker 01

  like, I think some people, there's sort of topics that they treat as like, ah, that's a real, serious topic. It's about real stuff. And then there are other topics where it's like, this is the chance to kind of make stuff up. And, you know, my experience is sometimes people relate to futurism that way. There are other topics where people move into a zone of like, one can just say stuff here, and there are kind of no constraints. And I think I think that's actually wrong. And and and with futurism, I think there are there are important constraints and important things we can say. But I think that can that vibe can seep in nonetheless.

• 1:20:23Dwarkesh Patel?

  Yeah, and it's interesting that it's true of the future and the past. I recently interviewed somebody who wrote a book about the Napoleonic War. And, yeah, it it is, I mean, it's very interesting to talk about in a sort of abstract sense. But then also you can, which is very seldom done, you can like, think of the reality of like, a million men marching out of Russia and freezing and eating the remains of horses and other people and then starving. And the the the then the the concrete reality, when you're not, yeah, when you're done talking about abstract, it's like, oh, the border changed so much in these few decades or something. Yeah, just how you think about history changes so much and it becomes. Yeah, even recently, I was reading this book about the the the use of meth by the Nazis.

• 1:21:08Dwarkesh Patel?

  And if you just, there's there's this really cynical part of the book where the the leaders in the in the Nazi regime, they're talking about like, oh, meth is the perfect drug. Because it gives them courage to kind of just blitz through an area without any sort of without thinking about how cold it is, without thinking about how scary it is to just be no man's land. And just this idea of like this meth up soldier who's like been forced to just go out into the middle of nowhere. And yeah, and then all like marching to Russia or something in the winter. I don't know if that was going to lead up to a question. I don't know if you have a reaction, but yeah.

• 1:21:44Speaker 01

  Yeah, I mean, I think so. I think that's a great example of, or you know, specifically the sort of image of the difference between relating to history as this sort of how is the border changing versus the concreteness of these people. And you know, often I think engaging with history is horrifying in this respect is when you really bring to mind that the lived reality of all these events. It's a really different experience. And I think to some extent, one of the reasons that concreteness might be often lacking from futurism is that you can't any, any attempt to specify the thing will be wrong. So, you know, we can, you can, you might be right about some abstract thing like you might be like, oh, we will, you know, we will have

• 1:22:25Speaker 01

  the ability to manipulate matter at like blah, you know, blah level, you know, you know, scale. But if you try to dig in and then you're like, and here's what it's like to wake up in the future, you know, and, and then, you know, the, you're eating the or whatever, and it's, you're wrong immediately. That, that's not how it's going to be. And so you don't have the, the ability to really hone in on concrete details that are actually true. And so in some sense, you need to, there's this like back and forth where you need to sort of imagine a concrete thing and then be like, okay, that's wrong, but there will then take the flavor of concreteness that you got from that. And say, but it will be a concrete thing. It just won't be the specific one I imagined.

• 1:23:05Speaker 01

  And then keep that flavor of concreteness, even as you talk in more abstract ways. And that's, I think, a delicate dance.

• 1:23:12Dwarkesh Patel?

  Yeah, yeah, as many viewers will know, Peter Thiel has this very talking point that he often brings up about our that we've become indefinite optimist and that he prefers a sort of definite optimism where you have a concrete vision of what the future could be. Okay, so yeah, I guess to close out, one of the things I wanted to ask you about was, so you said this is a side project, this blog. I thought it was one of the, actually, before you mentioned that your main work is AI, I thought this was at least part of your main work. And so it's surprising, it's really surprising to me then that you're able to keep up the regularity. It's like basically you're publishing a small book every, I don't know, every week or so.

• 1:23:55Dwarkesh Patel?

  And filled with a lot of insight and I mean, it's like, unlike many other bloggers on the internet, we're just playing style. Yeah, you've got like great prose. How are like, what is your, how are you able to like maintain such productivity on your side project?

• 1:24:12Speaker 01

  I should say a few of my recent, my most recent posts, which were especially long, I was, I had taken, taken some time off from work and I was working on those partly in an academic context. But the first, the first year and a half or so of the blog was just on the side and I've gone back to having it be on the side now. I think one thing that helps is my blog posts are too long.

• 1:24:33Speaker 01

  And so there's, you know, I have dreams of writing these, you know, taking my long blog posts and then really crunching them down and making it into this like pithy elegant statement that that's really concise and condensed. But that would be more, so you know, one, one way I sort of increased my output is by not doing that editing. And I feel, I feel bad about that. But that's one, that's one thing at least.

• 1:25:01Dwarkesh Patel?

  Yeah. Well, what is that quote where I don't know if somebody's asked like, how did you?

• 1:25:06Speaker 01

  I think it's something like, I would have, you know, I would have written you a long letter, but I, or I would have, I didn't have time to write you a short letter. So I wrote you a long letter or something like that.

• 1:25:12Dwarkesh Patel?

  Yeah, exactly.

• 1:25:13Speaker 01

  I have a friend who said, like, the actual thing it should be, I didn't have time to write you a short letter, so I wrote you a bad letter.

• 1:25:20Speaker 01

  And you know, I'm like, I hope it's not that bad. But I do, you know, I do think if I had more time for these posts, I would, I would try to kind of cut them down. And that's, so that's one time saving, you know, for better or worse.

• 1:25:33Dwarkesh Patel?

  Um, yeah, at least as a reader, it often seems to me that the people like you who write, um, I maybe this is to describe your process, but, um, it was Scott Alexander says he kind of just writes stream of consciousness and that, you know, it, it just turns out to be really readable. Your blog posts are really readable. Um, and even like the stuff I write, like the things that are right are that are, I'm like consciously not trying to make edits while I'm going on. They end up reading much better than the ones where I'm trying to optimize each sentence, uh, and then taking two steps back for every one I take forward. Um, I, I, I don't know if it's just, it could just be like a selection effect of the, the, the things that are harder to convey. You're spending more time editing, but, um, yeah, it's, it's kind of interesting.

• 1:26:16Speaker 01

  Yeah, I wonder, I wonder, I mean, my, my feeling is that my writing is, is quite a bit better if I have a chance to edit it. Um, and it's just, it's just a time thing. Um, but I do think people vary quite a bit. And, and you know, it's interesting. I, I don't know. I was recently reading this book, George Saunders, who I think a writer I really admire, has this, uh, book about fiction writing called, uh, Swim in the Pond in the Rain. Um, and the vibe, uh, he tries to convey, and I think this is relatively common amongst writer types, is like this obsessive focus on, you know, even at a sentence by sentence level, really thinking about what, where is the reader's mind right now? How are they engaging? Are they interested? Are they surprised? Am I losing them? Um, and, and you know, his writing is really, really engaging in ways that's like not even

• 1:27:00Speaker 01

  obvious. You're, you just sort of start reading along and you're like, oh, wow, I'm really into this. Um, but it's also quite a daunting picture of the level of attentiveness required. And it's like, wow, if I'm going to write everything like that, it's like, that's going to cut down a lot on my, um, kind of overall output. Uh, and so I do think there's a balance there and, and, um, you know, to the extent you're, you're one of these people who you could just like stream of consciousness and that's like close to what you would get out of, out of editing, which I'm not sure I am. Um, you know, all the better. It's sort of like, you're lucky.

• 1:27:26Dwarkesh Patel?

  Yeah, there's also an additional consideration where if you think there's going to be some kind of power law to like, how much, how interesting a piece is, or how many, how many people see it and how many people find value in it, then it's not clear whether that advises you to spend so much time on each piece to increase the odds that that one piece is gonna blow up, given that there's a big difference between the pieces that blow up and don't, or whether you should just like do a whole bunch and then kind of just try to sample as often as possible.

• 1:27:56Speaker 01

  Yeah, and I think, I think actually that the blog, I started the blog partly as an exercise in just getting stuff out there. I think I had had, I had had the idea that I would one day write up a bunch of stuff that I've been thinking about, but you know, it was somehow a, and I would write it up in this grand, you know, I would finally write it up and it'd be this beautiful thing and I would, you know, take all this time and then, I had ended up, you know, for various reasons, feeling like I was approaching some aspects of my life with too much perfectionism or too much, and I needed to just like, get stuff out there faster. And so the blog was an exercise in, in that, and I think has,

• 1:28:37Speaker 01

  you know, I think that's paid off in ways and that I don't know, I don't think I would have done it otherwise.

• 1:28:42Dwarkesh Patel?

  I see. All right. Final question. I'm curious if you have three book recommendations that you can give the audience.

• 1:28:50Speaker 01

  Probably my primary recommendation, though this is somewhat self-serving because I helped with this, this project is the book The Precipice by Toby Ord. I, you know, may, maybe familiar to many, many of your listeners, but you know, I think it's, it's a book that really

• 1:29:06Speaker 01

  conveys the ideas that matter, you know, most to me, or that have had, you know, close to the biggest impact in my own life.

• 1:29:14Speaker 01

  Other books, I, I love the play Angels in America. I think it's just, I think it's epic and amazing. And, you know, that's not quite a book, but you can read it. I actually recommend watching the HBO mini-series. But that's, you know, that's something I recommend. And then,

• 1:29:35Speaker 01

  I don't know. Last year, I read this book, Housekeeping by Marilyn Robinson, and it had this sort of numinous quality that I think a lot of her writing does. And so I really like that and recommend it to people. That's also a piece of fiction. If you're looking for philosophy, I don't know, a lot of my work is in dialogue with Nick Postum and his overall corpus. And I think that's really valuable to engage with.

• 1:30:03Dwarkesh Patel?

  I see. Cool. All right. Yeah, Joe, thanks so much for coming on the podcast. This is a lot of fun.

• 1:30:07Speaker 01

  Yeah, thanks for having me. Oh, I'll also say, you know, everything I've said here is just purely my personal opinion. You know, not speaking for my employer, not speaking for, you know, anyone else, just myself. So just keeping that in mind.

• 1:30:19Dwarkesh Patel?

  Cool. And then where can people find your stuff? So just if you want to go over your blog link and then your Twitter link and other things.

• 1:30:29Speaker 01

  Yep. So my blog is handsincities.com and my Twitter handle is JKCarlSmith. Those are good places to reach me. And then my personal website is josephcarlsmith.com.

• 1:30:38Dwarkesh Patel?

  Okay. And then we're going to find your stuff on AI and those kinds of things?

• 1:30:43Speaker 01

  The stuff on AI is linked from my personal website. So that's the best. That's the best place to go.

• 1:30:48Dwarkesh Patel?

  All right, cool, cool.

• 1:30:50Dwarkesh Patel?

  Thanks for watching. I hope you enjoyed that episode. If you did, and you want to support the podcast, the most helpful thing you can do is share it on social media and with your friends. Other than that, please like and subscribe on YouTube and leave good reviews on podcast platforms. Cheers. I'll see you next time.