#8 Julia Galef: The Art of Changing Minds

On this episode of the Knowledge Project, I discuss rationality, changing minds (our own and others), filtering information, and a lot more with Julia Galef. Go Premium: Members get early access, ad-free episodes, hand-edited transcripts, searchable transcripts, member-only episodes, and more. Sign up at: https://fs.blog/membership/ Every Sunday our newsletter shares timeless insights and ideas that you can use at work and home. Add it to your inbox: https://fs.blog/newsletter/ Follow Shane on Twitter at: https://twitter.com/ShaneAParrish Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:09Shane Parrish?

  Welcome to The Knowledge Project. I'm your host, Shane Parrish, curator behind Farnham Street Blog, an intellectual hub of interestingness that covers topics like human misjudgment, decision-making, strategy, and philosophy. The Knowledge Project allows me to interview amazing people from around the world to deconstruct why they're good at what they do. It's more conversation than prescription. On this episode, I have Julia Galef. She's the president and co-founder of the Center for Applied Rationality, a nonprofit organization based in Berkeley, California, devoted to developing, testing, and training people in strategies for reasoning and decision-making. She also hosts the Rationally Speaking podcast, a bi-weekly show featuring

• 0:51Shane Parrish?

  conversations about science and philosophy. On this episode, we talk about a host of fascinating subjects, including rationality, of course, changing minds, our own and others, filtering information, and a ton more. This was a fascinating conversation and one you don't want to miss. I hope you enjoy it as much as I did.

• 1:17Shane Parrish?

  Where did the interest, you're the co-founder for the Center for Applied Rationality, where did the interest in rationality come from?

• 1:24Speaker 01

  Well, people ask me that sometimes, and I don't have a great answer for it. I've sort of been interested in, I mean, before I knew the word rationality, I've been interested in what that word refers to as long as I can remember, basically. One of my guesses as to the origins of my interest is my parents. They're both very intellectually curious, you know, smart for sure, but also intellectually curious people who like to inquire about the world and encourage me and my brother to be curious about the world. And more than that, they were also unusually good at changing their minds when presented with a good argument, or when they encountered new facts that should change their minds. So I just have these...

• 2:10Speaker 01

  These vivid memories from when I was, oh, I couldn't have been more than six or seven, I think. Uh, and I would have these arguments with my parents, you know, the way that kids do about rules or, or what was fair, or, you know, what's good behavior, et cetera. And sometimes, not usually, but some of the time, my parents would come back later and say, you know, Julia, we thought about it and we think you're right. We think this was unfair or, you know, whatever. And I just, I remember feeling so, well, grateful for one that, you know, they had listened to me and taken me seriously. But also I felt admiration for them, you know, because they didn't have to change their minds.

• 2:50Speaker 01

  There was no, you know, no one was going to hold them accountable. I couldn't do anything about it, but, but they clearly wanted to, uh, like if, if I was right, they wanted to realize that I was right, you know? Uh, so, so that kind of stuck with me and I got pretty interested in, um, good arguments. Uh, what counts as good evidence? I, I ended up majoring in statistics in college, which uh, I was interested in because it asks questions like, how can we know things with any confidence about the world? Um, how much evidence is enough to, to sort of change someone's position on something or should be enough? Uh, and through it all, I, I carried this memory of my parents as sort of an ideal

• 3:30Speaker 01

  uh, in the back of my mind in terms of, of how to react when you encounter new, new evidence or good arguments.

• 3:36Shane Parrish?

  Did your parents do anything else to kind of encourage that in you? Or like when you changed your mind, did they, uh, were they responding positively to that or?

• 3:45Speaker 01

  I think so. That's less of a vivid memory in my mind, but I, if I had to guess, I would say yes. Uh, they, uh, I mean, my dad's, well, both of my parents, but my dad has more of a background in science. Um, my mom's background is in math and statistics, actually. Um, so my dad was the one I would more often ask questions about, you know, why is the sky blue, that kind of question about the world? And uh, and he rarely just gave me the answer. He would more often sort of lead me through a reasoning process in kind of a Socratic type way. Uh, and you know, usually I couldn't actually reason my way from first principles as a seven year old to figuring out why the sky is blue.

• 4:27Speaker 01

  But I could sort of come up with some guesses and maybe I could sort of notice, oh, maybe that guess doesn't make sense, et cetera. So when I finally got the answer, it was sort of more satisfying because I had tried, you know how when you try to solve a puzzle and someone tells you, tells you the answer, it's much more satisfying and it sticks with you more than if they just tell you the puzzle and then tell you the answer straight away?

• 4:45Shane Parrish?

  Right.

• 4:46Speaker 01

  It was like that. So I, I think that was part of it. Uh, well, the, the last thing I can think of in that space is, uh, I don't know if I'm joking about this, but my dad, well, sorry, the, the facts are, are true. I don't know, I'm, can't tell if I'm serious about them contributing to my interest in rationality, but the facts are that my dad has kind of a trickster spirit at the core of him. And sometimes, especially if he didn't know the answer to a question, he would just make up an answer. You know, if you've ever read Calvin and Hobbes, Calvin's dad is like, is, is the, the example, the poster child for this. So, you know, uh, uh, why, how does the radio play music? Well, little people inside the radio that have been, you know, shrunk and high, and

• 5:31Speaker 01

  hired to play whenever you turn on the switch, that sort of thing. And, uh, and he would say it with this perfectly straight face, which is very serious, you know, professorly tone of voice. And, and eventually, you know, he would make me realize he was fibbing, but, uh, it was much more satisfying if I could figure it out myself. If, if I could learn to notice those little bells in my head of, wait a minute, that doesn't make any sense, um, before he, you know, gave it up.

• 5:59Shane Parrish?

  Your dad sounds like he'd be awesome at a dinner party.

• 6:02Speaker 01

  Yeah, uh, awesome is one way to put it. So yeah, I mean, I, there's a neat story there about how that helped me be skeptical. But I don't know, it's really hard to, it's really hard to draw causal connections.

• 6:17Shane Parrish?

  So now you're the co-founder for the Center for Applied Rationality. Can you, what does that do? I mean, what are you guys trying to accomplish?

• 6:23Speaker 01

  Yeah, so our mission is to develop strategies to improve on the default human processes for reasoning and making decisions and to educate people in those improved reasoning and decision-making strategies and particularly with an eye towards having a positive impact on the world. So the timing of founding the Center for Applied Rationality, which we call C-FAR, is basically we had noticed that the science on human rationality, or more precisely human irrationality, had kind of hit this peak over the last few decades.

• 7:04Speaker 01

  If you've heard of Daniel Kahneman, who wrote the bestseller Thinking Fast and Slow, he won a Nobel Prize for his work on human irrationality and inspired a lot more work on similar topics. And there was just, you know, the academic research had, had sort of been building up for the last 50 years, kind of culminating in Daniel Kahneman's research, Nobel Award, Nobel Prize Award. And then the public interest in the topic had been growing over the past 10 years or so with books like Thinking Fast and Slow, but before that, Dan Ariely's Predictably Irrational, Nudge, lots of other books on behavioral economics and cognitive science.

• 7:45Speaker 01

  And the thing was, there was all this research on how the brain is irrational, what kinds of systematic reasoning and decision-making errors humans make, leading them to sort of systematically get the wrong answer to questions or systematically make decisions that they end up regretting, that kind of thing. But there, and, and there was public, tons of public interest in this topic, but there wasn't yet a lot of research on, okay, what do we do about this? There's a little research examining interventions to try to overcome some of these. Cognitive biases, but not a lot. And most of it was pretty shallow research. And I don't mean that in a disparaging way. I just mean the interventions consisted of things like

• 8:26Speaker 01

  the experimenters would tell the treatment group about the bias and then see if they, you know, committed the bias on a quiz or something. And that's not the kind of thing that I would expect to really change these ingrained habits of thought and and behavior. I would expect that to change ingrained habits of thought and behavior, you need more longer term practice on real life issues, not on toy problems, you know, in a lab.

• 8:55Shane Parrish?

  I definitely want to get into how we go about changing some of those processes. But maybe before we continue, do we have a common understanding of what rationality means?

• 9:05Speaker 01

  Oof, how many hours do you have?

• 9:09Speaker 01

  It's hard. There's a sort of simple definition you could give that doesn't capture that much. And then there are increasingly complex definitions you can give that, you know, sacrifice parsimony for for depth.

• 9:22Shane Parrish?

  Is there more than one type of rationality?

• 9:24Speaker 01

  Yeah, so sometimes we talk about two types of rationality. On the one hand, we have epistemic rationality. And sorry, these aren't just C F R S terms. These are terms in cognitive science and philosophy. Right, so epistemic rationality is about using processes, processes for reasoning or processing information that systematically get you closer to an accurate model of how the world works. And, you know, we can never be 100% confident and get the the perfect, perfectly correct answer about everything about how the world works. Because, you know, we just have limited information and time and there's a lot of uncertainty. But some reasoning processes are just more reliable than others.

• 10:07Speaker 01

  So, I mean, to take a dumb example, making stuff up or believing what a random person on the street tells you will be a less reliable process for having an accurate model of the world than synthesizing the opinions of top experts on a topic or looking at randomized controlled trials or something like that. And that's kind of an oversimplified description, but I'm just trying to sketch out the what the spectrum of epistemic rationality looks like.

• 10:34Speaker 01

  Should I go on to the second version?

• 10:35Shane Parrish?

  Yeah, please.

• 10:36Speaker 01

  Type of rationality. So the second type is instrumental rationality. Um, and that's about uh making choices that uh given the the best information you have at your disposal are most likely to uh achieve your goals. And I know the word goals kind of has associations for most people with uh like career goals and being productive and I don't know, maybe getting an award or getting something published. Uh, but but in this context, it could really mean anything, anything you want or value. It could mean um having friends, it could mean being happy, it could mean, you know, helping the world, whatever it is you care about. Instrumental rationality is making choices that sort of you can systematically expect

• 11:19Speaker 01

  are most likely to to achieve those goals as uh as efficiently as possible.

• 11:24Shane Parrish?

  Is there one, only one optimum path for doing that? Or is it more broad than that?

• 11:29Speaker 01

  Well, it's an interesting theoretical question. I suppose if you had access, if you had perfect information about the world, uh, there might be one optimal choice that was slightly better than all the other choices in terms of its probability of, but this is all very abstract. In in practice, in practice, it's not helpful to think about there being one very best choice that you can know for certain.

• 11:54Shane Parrish?

  Okay.

• 11:55Speaker 01

  But you can, you know, with some careful reasoning and using some good heuristics or or rules of thumb, you can sort of rule out some obviously bad choices uh and try to make an educated guess among the set of of plausible best choices.

• 12:11Shane Parrish?

  Do you think that that's a good strategy if you don't know what's likely to lead to success, but you do know what's likely to lead to failure is just eliminating this stupidity?

• 12:20Speaker 01

  Uh, it's a, it's low hanging fruit, I think. Um, I mean, uh, so often in my work at SIFAR, I guess I didn't get to fully finish explaining what we do, but the short rest of the explanation is that we, uh, in addition to sort of doing some of our own research on these topics, we run workshops where we uh run training sessions, teaching people some of the more promising techniques that we've been developing to help overcome some of these biases and make better decisions. Uh, and and so often I find myself In this situation where someone is kind of, they're at some crossroads. You know, they're, they're at their job and they don't really like it and they're not motivated. They're not really doing good work at all, but they don't really know what they should be doing instead.

• 13:05Speaker 01

  And they're having a lot of trouble figuring out what the best choice is. And so usually my answer in that case is, okay, I don't know what the best choice is. I don't think you have any way to know what the best choice is. But clearly the thing you're doing now is not the best choice. You're, you're, you know, if, if the best choice is staying at your job, then it means staying at your job and finding a way to make it actually fulfilling and to make yourself actually succeed at it. The best choice is not doing, you know, a half ass job at your current job. So we can kind of eliminate your current strategy. That's maybe a simple example, but, or it sounds silly, but that is in fact the way a lot of us make our decisions by default.

• 13:44Shane Parrish?

  So in, in your, your view of rationality, then what's the role of intuition?

• 13:50Speaker 01

  Well, so I know a common conception of rationality in the public, in the media, is that rationality means dismissing or suppressing your intuition. There's often this dichotomy set up of, you know, reason and rationality on the one hand, and then intuition and, and emotion on the other hand. And so, you know, this, this sort of perceived dichotomy gives rise to characters in movies or TV shows that are sort of explicitly the rational one or the logical one. And what that means in practice for the character's actions is that they, they're the one

• 14:33Speaker 01

  who goes around, you know, poo pooing other people's thoughts and feelings and, and looking down their nose at people for, you know, having fun or, or falling in love or, or finding something beautiful. And that is not what we or, or cognitive scientists. deem rationality. The the actual scientific model, which I think is is also pretty common sense when you think about it, is our brains are our minds are divided. We do have a more intuitive side of our mind, which cognitive scientists call system one. It's kind of a dull name, I apologize for that.

• 15:14Speaker 01

  And then we also have this more recently evolved part of our mind that allows us to do logical reasoning, weigh abstract trade offs against each other, do math, long term planning, that kind of thing. It's very roughly speaking, it's our prefrontal cortex. And the kind of reasoning it does is what cognitive scientists call system two. And system one is is indispensable. There's no way we could actually survive as a species or as individuals if we ignored the output of our intuitive system one. Right, so if I were to throw a pen at your head. Please don't. You had, well, we're on opposite sides of the continent right now, I think so you're safe.

• 15:58Speaker 01

  But if we meet in person someday and I throw a pen at your head and you were only able to choose what to do using your system two, it would be absurd. Your system two would have to sort of reason through all the possible ways to react. It might say to itself, OK, well, well, first look, calculate the trajectory of the pen approaching my head and its velocity. And I can estimate, OK, it'll probably approach, hit my forehead in half second. What are my choices? I could stay still, I could try to dodge, maybe I could try to catch the pen. Let's weigh the pros and cons of each option. And it like, it would quickly be a moot point, right? The pen would have already hit your head. And this is sort of the drawback of your system two.

• 16:42Speaker 01

  It's sort of slow and laborious and effortful. Although it can do things that your system one can't do. Your system one is very fast. And it it's it's very good at processing lots of information that you've sort of picked up, often unconsciously over the course of your life. So when you're, you know, forget about pen pitting foreheads. Let's take a real example. If you're in a social situation, you can sort of pick up things, you can pick up social cues to varying extents. If you can detect whether someone is, you know, frustrated with you or bored or flirting with you or threatening you, you usually can't fully articulate what it is about their. Tone of voice or their facial expressions that's giving you that impression.

• 17:25Speaker 01

  Um, but you're still pretty confident it's happening. And the reason you can't articulate it is that it's your system one, your intuition. That's doing all of that processing.

• 17:33Shane Parrish?

  Right.

• 17:33Speaker 01

  And it's doing it in this very black box kind of way. But this is what it, it's had practice over the many years of, you know, interacting in social situations. Uh, and so it's learned over time to be able to do this well and to just spit out a quick answer about, you know, how does this person feel about me? So, so it's a long-winded way to say that system one intuition is indispensable. The catch is just that it's often fallible. Uh, and so rationality is, it's less about ignoring or suppressing system one intuition and more about. Understanding the, the sort of respective strengths and weaknesses of the two systems. And learning how to sort of get them to communicate with each other, uh, so to speak.

• 18:13Speaker 01

  Um, get your system two to listen to your system one and get your system one to listen to your system two. And sort of update your emotional or intuitive impressions if they seem flawed in that situation.

• 18:26Shane Parrish?

  You mentioned art and love and passion and, you know, all of these other things that we, you know, give us some sort of emotional response. What's the connection to rationality there? Is there one?

• 18:38Speaker 01

  Connection between art and rationality?

• 18:40Shane Parrish?

  Yeah. Is there one? I mean, you brought it up in a way that made me think that there was one.

• 18:44Speaker 01

  Well, oh, you mean in my example of the, uh, the, the supposedly logical character in movies?

• 18:50Shane Parrish?

  Can it be rational to appreciate art? Can it be, I mean?

• 18:54Speaker 01

  Yeah, yeah. Oh, by the way, before I answer your question, I realized I forgot to give the name of that character. Uh, the, the name for that trope, that archetype is the straw Vulcan.

• 19:03Shane Parrish?

  Okay.

• 19:04Speaker 01

  Uh, it's a play on the expressions, the straw man. Do you know the expression to straw man?

• 19:08Shane Parrish?

  Yes, yes.

• 19:09Speaker 01

  Right. So it just means to, to present this like weak caricature of what your opponent is arguing and then knock that down. Cause it's easier to knock down. And so the straw Vulcan is like a weak caricature of rationality. It's not actual rationality, but it's like easier to sort of make fun of and knock down than real rationality. So I just wanted to get that in. So, right, in your answer to, in answer to your question about art and rationality. I mean, I'd say the only obvious connection to me, I, I kind of think of art as being orthogonal, uh, independent of rationality. The kinds of things that the artist is trying to do are mostly not trying to get the right answer to a question or achieve a goal. It's more of an expressive process.

• 19:53Speaker 01

  But in terms of appreciating art, uh, I'd say for many people, maybe for most people, for some form of art, that's, uh, that's very important for their goals. And the kinds of goals I'm thinking of are indeed the kinds of goals that people usually don't think of when I say the word goals, um, but are still very important goals, like, uh, the goal of feeling connected to other humans on the planet. The goal of, I don't know, feeling a sense of meaning or finding a sense of meaning. The goal of pleasure, lots of art is, is about pleasure. So I, I'm glad you asked the question, actually, because this is exactly the kind of thing that I think people neglect when they think about optimizing for their goals. And these are, in fact, very important goals that people miss when they don't have them.

• 20:38Speaker 01

  But somehow, they don't really, they don't seem to come to mind when people think about, okay, now I'm gonna try to be rational.

• 20:45Shane Parrish?

  You're interested in changing minds. How do we go about doing that?

• 20:48Speaker 01

  Oh, are you talking about changing other people's minds or changing your own mind?

• 20:51Shane Parrish?

  Oh, let's do both. Let's start with, let's start with our own mind and then, uh, explore how we change other people's minds.

• 20:58Speaker 01

  So that, that is the order in which I'm interested in, the, the, the order of, of questions you're, you asked is the order in which I'm interested in them. It's difficult because usually for other people, the order is reversed. And when I want to talk about, okay, here's how to be better at changing your own mind, people often interrupt me and they're like, no, no, no, I want to change other people's minds. How do I convince my, you know, coworkers, my, my partner that they're wrong?

• 21:22Shane Parrish?

  Yes, if only they saw the world through my eyes, right?

• 21:25Speaker 01

  Yeah, it's less interesting to me, um, but I'll, I'll still give my best answer to it. So to change your own mind, I mean, the first step I would say is just believing that that's a desirable thing to do, which many or most people don't really, uh...

• 21:43Speaker 01

  Either would explicitly say is a bad thing to do because changing your mind makes you kind of wishy washy or weak or stupid. Or they would explicitly claim it's a good thing to do, but they don't really believe it on a sort of gut level. And so, you know, they don't have the motivation to do it and therefore they're not going to do it. So step one is actually believing, yes, I believe on a gut level that there's probably a bunch of things I'm wrong about that I'm not yet aware of, just because that's true of everyone. And also that whatever it is I'm wrong about, I would like to know about that because I'll probably be able to make better decisions for myself and for other people. I'll probably be able to avoid hurting other people if I have a more accurate model of the world.

• 22:28Speaker 01

  And so the implication of that is I would like to change my mind when I encounter new evidence. In fact, I want to seek out new evidence that might cause me to change my mind, not just sort of passively accept it when it comes.

• 22:39Shane Parrish?

  How do we get to a point where we do that? Like, how do you take somebody who's naturally closed minded and develop over time, I would imagine it's probably not a light switch, this open mindedness to the point where you can give up your, you know, beliefs or cherished thoughts about something?

• 22:57Speaker 01

  Yeah, well, I mean, even the way you phrase that question, how do you take someone who's closed minded is already, is already kind of begging the question.

• 23:08Shane Parrish?

  We're all closed minded in some ways, right? We're blind to certain things. So myself, I'm blind to a lot.

• 23:14Speaker 01

  Yeah, I thought you meant closed minded in the sense of not wanting to change their mind.

• 23:19Shane Parrish?

  Oh, yeah, that's what I mean, right? Like, so I just like, you could, you could show any amount of evidence and if it's something that we hold dear, we have an emotional connection to, we're less likely to change our mind. That's an extreme example, but I mean, there's many things in organizations or workplace where any amount of evidence or rationality, I mean, the proverbial story would kind of be, you know, you present evidence about cigarette smoking and then one person in the meeting chimes up and says their grandmother, you know, has been smoking for 99 years and doesn't have cancer and then it just gets dismissed and washed under the table, right? So. How do you, how do you get to a point where people themselves are motivated, I guess, in some way

• 24:02Shane Parrish?

  to change their minds or be open, more open minded?

• 24:06Speaker 01

  Right. So I guess I have this, this mental model, it's sort of a two-layer mental model where the, the bottom, the more fundamental layer is just abstractly, but sincerely wanting to, to be able to change your mind. And that doesn't mean that in any given situation, like if you, you know, read an article saying that caffeine, well, that's a bad example because nutrition data is not very reliable, but let's say you read a reliable article showing that, a reliable study showing that caffeine was really bad for you, but you've cherished your morning cup of coffee or, you know, your three cups of coffee a day, that is going to be hard to hear. And the natural reaction is to find a reason to dismiss the study.

• 24:49Speaker 01

  And that's often going to happen, even if you have that fundamental layer in place of believing in general that you would like to be the kind of person who changes your mind. The distinction I was trying to make with the, in my response to the closed-minded phrase is I think a lot of people just don't have that fundamental bottom layer. Like they don't even want to be able to change their mind in particular situations. I think it's,

• 25:10Speaker 01

  it's like, I'm pretty pessimistic about the ability to get it to work in a particular situation. So I tend to focus on the, that bottom layer first. And to get that in place, again, I don't think it's the kind of thing you can sort of do to someone else. And when, when people, so we, you know, we have people go through some, an admissions process to come to our workshops, just to make sure it's going to be a good fit for them because it's kind of an investment of time and money. We want to make, want to make sure it's a good fit. And so one of the questions we ask them in the admissions interview is, why are you interested in coming? What are you hoping to get out of it? And sometimes, occasionally we get someone who says, well, I want to like, learn how to explain rationality to the people around me so that they realize how irrational they are.

• 25:54Speaker 01

  And those people we generally turn away, saying we don't think it's a good fit for them. Because it, the motivation really has to be, I want to improve my own reasoning and notice my own blind spots. So my guess as to the things that cause someone to be not in that group.

• 26:12Speaker 01

  Uh of other focused people are my guess is that it's partly social, like the influence of the people around you. Um, because you know, humans are primates and social creatures. And the things that we're motivated to try to acquire or be tend to be very influenced by the culture in which we're in which we live. So, you know, if your if your parents valued and rewarded change your mind, or if the people in your social circle value and reward changing your mind, that I think that makes a huge difference. And that uh that isn't just sort of an abstract a priori reasoning. I see that pattern just looking around me at the people I know. The other thing that I think can sometimes work, even if you don't have the general motivation to change your mind is

• 26:56Speaker 01

  uh there's a particular goal that you need to achieve. Like your your startup um uh is gonna fail if you can't look at the data in a clear eyed way and make the best decision you can about, you know, whether to pivot or not, that kind of thing. So sometimes these kind of immediate, like high stakes motivations can really make you want to see the world the way it as closely uh as close to the way it really is as possible and not see sort of a wishful thinking version of the world that you've created in your mind. Um, it doesn't always work. Sometimes we still kind of try to distort or deny our our perception of the world. But having that motivation can help. Did I answer your question? I can't remember what your question was.

• 27:38Shane Parrish?

  Yeah, I like that. I mean, how do we go about changing our own minds, right? And then how do we go about changing the minds of others?

• 27:44Speaker 01

  Oh, yeah, I guess I didn't quite answer the how did we change our own minds? Question I just I I answered the question. How do how do you become the kind of person who could go about changing their minds?

• 27:54Shane Parrish?

  Right, yeah. So, I mean, that was a a parenthesis around that. So if we can go back to like, how do we go about changing our own minds? The first step you had mentioned, and I derailed you after that was, how do we um, you know, we we need to be open minded about um processing new information? And then I think you had more to that answer.

• 28:12Speaker 01

  Yeah, so the problem that I found when I read the pre-existing advice in, I don't know, skeptic blogs, or, I don't, do your listeners know what skeptic movement is? Maybe I should just explain that very briefly. Yeah, please do. The skeptic movement, or you could call it a community, I don't know. It's a group of people who are promoting scientific and, and, and critical reasoning. And go around sort of testing and sometimes debunking things that turn out to be pseudoscience, or, I don't know, paranormal or magical claims.

• 28:54Speaker 01

  So anyway, I, I used to read a lot of skeptic blogs and listen to some skeptic podcasts. And I think skeptics are pretty good at giving advice about the importance of changing your mind. But it, it tends to be at a very sort of high abstract level, like be open-minded. That's good advice, it's important, but it's not clear how to concretely implement that. It's sort of like telling someone, you know, eat healthy. That's unlikely to actually change their behavior because their dietary choices are made up of all of these, these little moment-to-moment choices about what to eat and how much to eat, you know, when to stop eating. And just having this abstract commandment, eat healthy, in the back of their mind

• 29:39Speaker 01

  usually doesn't translate into changing their actions on a moment-to-moment basis. Right. So a lot of what SIFAR ended up doing was just taking this sort of abstract advice, like be open-minded, and translating it into these sort of very concrete, almost algorithms that have, that start with a cue, with a trigger. So that a cue might be, I read an article that I disagree with. That's, that's like a trigger, and then the action that I can take then to try to, to try to manifest the principle of open-mindedness is look for, look for evidence that actually agrees with the article.

• 30:20Speaker 01

  So this is sort of a counter to our general tendency to only look for evidence that, that supports what we believe, so. In this, in this sort of trigger action plan, we call these things, what we're trying to get ourselves to do is uh, take the trigger of, of, of a moment where my natural response would be to try to find reasons to reject an article. And instead install the habit of looking for reasons not to reject the article. And that doesn't mean I'm usually going to end up thinking the article is correct, but I will be giving it a much fairer shake than I would be if I didn't have this trigger action plan installed. So that's just one example, but it's, hopefully it's representative of the kind of concrete habit-based approach that we take to, to helping people change their minds.

• 31:04Shane Parrish?

  So that's really interesting. So that, uh, as you mentioned that, it seems like the cost of information processing would go up. And in a world where, you know, we're consuming so much more information. If you had a trigger action plan where you were evaluating each piece of information that you're reading and seeking out reasons for uh, not disagreeing or agreeing, or whatever it happens to be. You would be spending more time on that and have more, uh, more of an investment in it. Do you think that that, like, I guess, how do we, how do we process information in a world where the costs of doing that rationally becomes so high that maybe they're outweighing the benefits? I don't know.

• 31:46Speaker 01

  I mean, there is a cost. You're absolutely right. There, there's a reason that system one and system two are called respectively, fast and slow thinking systems.

• 31:55Shane Parrish?

  We can't consume everything slow, can we? In like today's age?

• 31:59Speaker 01

  Uh, yeah, no, you can't. But, you know, no reason to let the perfect be the enemy of the good, right?

• 32:03Shane Parrish?

  Of course, yeah.

• 32:04Speaker 01

  Um, and, and the other thing I would say is that, uh, the whole, the goal with habits is that they become automatic over time. So there are a lot of things that I, I find myself, uh, automatically doing now, um, like. Like when someone I dislike says something, I now sort of automatically, well, actually not all the time, but I sort of have an intuitive sense for when this would be a useful thing to do. I will automatically imagine that someone I like said the exact same thing.

• 32:34Shane Parrish?

  Hmm.

• 32:35Speaker 01

  And I I notice, would my reaction be different? In other words, was I sort of unfairly dismissing this person's argument because I don't like them?

• 32:44Shane Parrish?

  You know, just block them out of your mind.

• 32:46Speaker 01

  Right, and and that that was something I started started trying to do intentionally, and now it happens automatically. Um, and and to be fair, it is still, there's still some extra steps there happening in my mind that I wouldn't be spending the effort on if I didn't do this at all. But at least the the process happens kind of quickly and automatically.

• 33:05Shane Parrish?

  Okay.

• 33:05Speaker 01

  So, uh, that is the goal, to to try to get some of these explicit processes to eventually lodge themselves in your system one and your intuitive thinking. But I agree, it's, you know, there's a bit of an investment there. It's it's not as it's not quite as effortless and easy as just using your default, uh, thinking systems.

• 33:24Shane Parrish?

  Are there any other tips you would offer in terms of um how we process information that helps us become more rational as we're consuming it?

• 33:32Speaker 01

  Oh, man, that's a big question. Well, uh, I guess one thing we haven't really touched on is being attuned to your emotional reactions, um, which was a big thing that I I don't think it was really on my radar so much before I started C F A R. But there's, so you know, your emotions kind of are often in the background of your thinking. And so you, you know, you become defensive, you don't even really notice that you're defensive. You just start having thoughts like, ah, this person is out to get me, or like, oh, this isn't fair. Or you start looking for things to criticize about them, um, instead of, you know, trying to evaluate their criticism of you. And and you're actually getting defensive, you know, your your body is tensing up

• 34:17Speaker 01

  or your shoulders are sort of turning in. I you can't see my shoulders, I'm doing it now. But you just don't notice that that emotion is influencing the way that you're processing the person's argument.

• 34:30Shane Parrish?

  Right.

• 34:31Speaker 01

  And so, a fair amount of what we teach ends up being about just being much more self-aware of those emotional reactions. And they don't always have to be, you know, defensiveness or aggressiveness or anything like that. They can often be sort of the subtle anxiety or concern in the background when your mind starts to go to a topic and then flinches away from it because it might turn out if I start going down that road, I might have to conclude that, shoot, I shouldn't have entered the PhD program in the first place. Or like, oh, I really, I really am gonna have to break up with my partner. And these are unpleasant to think about.

• 35:02Shane Parrish?

  Right.

• 35:03Speaker 01

  And so, we automatically flinch away from them. But if you, if you become more attuned to the, to your emotional reactions, they can just be important clues as to where you're, what, what are the blind spots you're creating for yourself and where are they?

• 35:15Shane Parrish?

  It's interesting. I mean, yeah, I never pause and kind of reflect on my emotional state when I'm consuming information. I just assume that I process it all the same way.

• 35:25Speaker 01

  Yeah, I think there's a lot of interesting variation in, in the texture of your reactions that you can start to notice when you look for them. And different people have different ways that work well for them to develop this kind of self-awareness. One of my, the co-founders of Zefar has a background in Ikyo. He's been doing it and teaching it for like two decades now. And he's, he does a lot of meditation and he's just very embodied, I guess is the right word, in a way that I'm totally am not. I really live in my head. And so, Val is his name, is just really good at helping people detect the sort of physical, just what's going on with their body. Like, notice the tension, notice that they're not, you know, that they're like leaning

• 36:10Speaker 01

  forward with sort of aggression or they're like leaning backwards and anxiety or something like that. I just am not good at noticing these signs in my body. I have other ways of noticing my emotions, but they're more cognitive and less physical. So I think there's a lot of variation in what works for different people. But, but things like, like meditation, I guess, and some, to some extent martial arts, I think really do help a lot of people with this stuff. And that was an interesting discovery for me because I had never considered that there might be a link between, or some overlap between meditation or martial arts and rationality.

• 36:45Shane Parrish?

  That's interesting. Have you started doing martial arts now?

• 36:48Speaker 01

  No, I'm really lazy.

• 36:52Shane Parrish?

  So to kind of come back to it, how do we change other people's minds then?

• 36:57Speaker 01

  Yeah, so there's, there are different strategies to take depending on how intellectually honest you want to be, right? So, you know, a lot of the research that's come out, the books that I've been referring to on irrationality that describe, you know, all of these hidden forces that affect our decisions that we're not even aware of, like, you know, how tall was the person making the claim? Or was I holding a hot cup of coffee when, when the person asked, you know, made a request of me, that sort of thing?

• 37:28Shane Parrish?

  Right.

• 37:29Speaker 01

  All these insidious aspects of, of our psychology that we're not even aware of. Um, there's a real opportunity there for people to exploit those things and use them to change other people's minds um, without actually making any good arguments or presenting, you know, any evidence. Um, and in fact, there's a good book about this, I, I would recommend called Influence by, uh, Cialdini. He's a psychologist, uh, I can't remember his university now. It's been out for a few decades, it's been a bestseller for a long time. Um, but basically what he did is he went undercover, uh, at, uh, various, uh, companies who are in the business of persuading people, of influencing people.

• 38:11Speaker 01

  So I think that included a telemarketer, uh, some, some door to door sales. This was, you know, back in the seventies or eighties when that was more of the thing.

• 38:19Shane Parrish?

  Right.

• 38:19Speaker 01

  Uh, some, excuse me, some lobbyists or maybe not lobbyists, but activists who were trying to get people to sign petitions, that kind of thing. Um, and you know, these, these industries have, there's a real profit incentive to try to find new and better ways of persuading people to do things. And so they've developed all these techniques, but they're not going to publish them because they're, you know, a competitive advantage for them.

• 38:40Shane Parrish?

  Of course, yeah.

• 38:41Speaker 01

  So he basically. Did all this research for years and then wrote a book classifying the different kinds of techniques that these companies like these use to persuade people. And it's a very like clear and compelling and easy to read book, but with but very information dense. So there are five categories. I'm gonna forget all of them. But they include things like scarcity, trying to make people feel like something is is a rare opportunity. Social proof, trying to make them feel like other sort of high status people are doing this or believing this.

• 39:17Speaker 01

  Reciprocity, when you when you give someone something like like Hare Krishna is giving people a flower at the airports. They're much more likely to to feel on some level, not consciously, like they owe you something. And so they're more likely to agree to a request or, you know, change their mind about something. So these are all kind of effective but insidious ways of changing people's minds. I personally am uncomfortable using methods of changing people's minds that don't ground out in sort of good argument or logic. So, you know, changing people's minds with facts is harder than changing their minds with a smile or a gift.

• 39:59Speaker 01

  Or a nice haircut.

• 40:00Shane Parrish?

  So you're like a diehard rationalist then in the sense of if I present a good argument, you should therefore adopt it or?

• 40:08Speaker 01

  No, it's I definitely don't expect. So I wouldn't agree with that sentence if should means I expect people to adopt it. That's in fact, one of the one of the pillars of what I consider straw Vulcanism to be. Because if you look at the quintessential straw Vulcan, Spock himself, he is supposed to be the logical rational one, but he keeps making miscalculations because he expects other people to behave rationally and they don't. And he should know this because he's lived among humans for ages. And the fact that he still expects people to behave rationally is frankly quite irrational of him.

• 40:44Shane Parrish?

  Right.

• 40:45Speaker 01

  So I don't expect people to change their minds based on facts and evidence because I I know that's not a thing. Uh, and I don't, I don't know that I would endorse the moral should either. I don't think people are sort of morally wrong for not changing their minds in response to good arguments. It's just an unfortunate fact of the world and how our brains work. Um, what I was trying to say was more like, I, I personally feel uh kind of morally uncomfortable with, with um changing people's minds without facts and evidence. So it's um, it's kind of a constraint on my ability to change people's minds that I have to work, that I force myself to work within essentially. And that doesn't mean I expect to succeed. It just means I don't want to succeed if I'm not following that constraint. Does that make sense?

• 41:30Shane Parrish?

  Yeah, totally. Um, so one thing that struck me as you were saying how we, we possibly, or we, you know, avenues to pursue about changing minds is I just had in the back of my mind, the, the whole climate change thing where we have, or even the cigarette um companies who are trying to discredit, I guess, the, the growing evidence as it was coming. And it seemed like their playbook was more uh creating uncertainty. And how effective is that uh when you're, when you're trying to change the mind of a group or make them at least, or create some sort of uncertainty? How do you, how would you go about doing that?

• 42:09Speaker 01

  Uh, you're asking me to put on my evil hat? Is that what you want me to do?

• 42:15Shane Parrish?

  Well, that, that presumes that you didn't have it on to begin with. So I don't want to make any assumptions.

• 42:19Speaker 01

  What I create uncertainty? I mean, well, I, I think it's quite easy to um, to present what looks like a very compelling case for whatever you want to convince people of. Using, using studies, uh, using quotes from experts, using even, even randomized controlled studies, which are sort of the gold standard, uh allegedly the gold standard in, in scientific research. The problem, like the reason that this is possible, and the reason this is all such a, a problem, is that there's so much scientific research that's done um that it's quite possible

• 43:01Speaker 01

  to end up with a few studies that seem to support the claim that, I don't know, cigarettes aren't linked to cancer. Um, I actually haven't looked that up to see if there are studies showing that, but you know.

• 43:12Shane Parrish?

  Yeah.

• 43:12Speaker 01

  You can, if you do enough studies, some of them will even just by chance and often by experimenter bias will turn out to show, you know, whatever you want. And so you can easily sort of selectively present the studies that support what you're claiming. Um, and so even if you have a pretty skeptical, well-educated audience who knows, okay, I shouldn't believe things without a study. Um, you can still kind of pull one over on them. Uh, and, and I think we do this to ourselves inadvertently all the time. You know, we kind of suspect that something is true, like we suspect that, you know, the raw food diet is, is good for health. And so we Google around and we find a few papers that seem to show that and we're like, aha, I found it. Scientific evidence.

• 43:55Speaker 01

  But you know, if you were Googling for the opposite, you could also have found probably a lot more studies showing, you know, no effect on health. So, I don't know, I haven't actually looked at the studies on the raw food diet, but that's just an example.

• 44:06Shane Parrish?

  That's interesting. And so what is your take on how you would refute that? So in the context of climate change, where you have, um, you know, possibly people creating doubt or uncertainty, or we can use cigarettes to be more tangible, uh, where I think the evidence is now overwhelming that, you know, there's a definite linkage to cancer.

• 44:25Speaker 01

  Yeah.

• 44:25Shane Parrish?

  How would you have combated that if you were, um, the government who's trying to kind of crack down and prove this? And then you have these companies that are well-heeled, um, you know, and well-resourced to fight that, trying to create some sort of uncertainty to maybe possibly not combat what people believe, but delay, um, you know, what seems like an inevitability to everybody else?

• 44:49Speaker 01

  Ugh, so hard. Uh, I mean, I might relax my constraints a little bit. Um, what would I do? I don't know. It's really hard. I don't have a good answer to this.

• 45:03Shane Parrish?

  Okay, no problem.

• 45:05Speaker 01

  I mean.

• 45:05Shane Parrish?

  That's a good answer, right?

• 45:07Speaker 01

  Yeah. Yeah, no, I really, I think it's really hard. And I think I'm not confident that there is a good way to do it. While while adhering, you know, perfectly to the standards of of, you know, just using the best arguments and evidence. I think you probably have to also, you know, get charismatic spokespeople and you probably also have to, you know, buy lots of ad time to like really to use the principle of familiarity where if people hear something a lot, they're like more likely to believe it's true. And it's not, I don't feel great about that. But, you know, as long as you're staying relatively close to the end of the spectrum of of integrity of persuasion, it might just, you know, be necessary to try to get the the right message out there.

• 45:53Shane Parrish?

  Better answer than I would give. Before we get going, is there any books that you've read that have had an incredibly meaningful impact on your life? Or what has had the most kind of impact or changed your direction or your thoughts to a profound degree?

• 46:08Speaker 01

  So the set of books that has had a an impact on me is a little different than the set of books I'd recommend to other people. I'm just, you know, they're the the like particular path that I followed and the particular books that happened to be the, you know, influential to me, given the place I was at is, you know, is somewhat idiosyncratic. But one book that really influenced me in college was by a philosopher named A.J. Iyer. It's called Language, Truth, and Logic.

• 46:37Shane Parrish?

  Okay.

• 46:38Speaker 01

  And this is kind of, I was gonna say it's the it's a philosophy book for people who are kind of skeptical of philosophy. But I also would want people who aren't skeptical of philosophy to read it because I think it would be good for them. It's kind of, it's this short, clear, down-to-earth, almost manifesto, talking about the importance of clear thinking in philosophy would be a sort of vague way to put it. And it it really had an influence on the way I think. I think I'm more likely now to do things like if I believe something, like let's say I believe.

• 47:17Speaker 01

  So and so is unfair or biased, or let's say I believe rationality works, or whatever belief I have, I'm more likely to ask myself to make myself cash out that belief in terms of a concrete prediction. What do I expect to see differently in the world because this claim is true? So forcing myself to get really concrete to the point where my belief could potentially be disproven by evidence, or at least made less likely by evidence. And this, I think, came out of reading Iyer's Language, Truth, and Logic. He was basically, he was annoyed at the way philosophers would debate these kinds of empty, meaningless questions.

• 47:54Shane Parrish?

  Right.

• 47:55Speaker 01

  And he was pointing out, like, a lot of these questions could not be cashed out in terms of concrete.

• 47:59Shane Parrish?

  Right. You can't refute them, or...

• 48:02Speaker 01

  Yeah, they're sort of too, too empty. There's this, there's this expression, "not even wrong." Like, like, "I'm not calling your paper, your claim, wrong. It's not even wrong. It doesn't even make enough sense to be wrong." And, and so a lot of people think that Iyer kind of went a little too far in his condemnation of philosophy, and I would, I would agree. But I think it's still a very useful sort of bracing splash of ice water in your face, and like a good, a good sort of habit to have in the back of your mind when you're thinking about things and evaluating claims. And it's an easy read, as I said, very short. I think you can just get it online.

• 48:39Shane Parrish?

  Awesome, I'll check that out.

• 48:40Speaker 01

  Yeah, so that's one. I would also, so obviously, the, the work about human irrationality has been very influential to me. It's hard to pick a single book about that. I've already named Daniel Kahneman's book, Thinking Fast and Slow, which is...

• 48:56Shane Parrish?

  And C. L. D. N. E.'s.

• 48:58Speaker 01

  Oh, yeah, and C. L. D. N. E.'s. Ah, I've already kind of answered your question. But I'll, I'll, I'll give you a couple more briefly. Oh, the other book about rationality that I would recommend is a new book by Phil Tetlock. It's called Superforecasting. And you know the thing that I said towards the beginning of the podcast? That the motivation for founding C-FAR was that there really wasn't much research at all on how do we overcome these biases? Well, superforecasting is an exception to the rule. Phil Tetlock... Has been doing amazing work studying interventions to try to make people better at making predictions about things. Better at at you know evaluating arguments, that kind of thing. And this book explains what he did and and gives some like really pretty impressive results.

• 49:44Speaker 01

  Showing the effects of these interventions. Basically, Phil Tetlock's team of forecasters. Outperformed by by a huge margin, all the other teams of forecasters. And and like experts in in politics and economics who are trying to make predictions about world events. And that's because Phil Tetlock's team was using these techniques. So it's it's a pretty inspiring story, plus being full of good. Data and and examples. And then I guess the last thing I want to recommend is it didn't really change my thinking, just because. I was already kind of immersed in the subject, but it's the kind of thing that would have changed my thinking and really inspired me if I'd read it a little earlier. It's a book by a friend of mine, Will McCaskill. It's called Doing Good Better.

• 50:29Shane Parrish?

  Okay.

• 50:30Speaker 01

  And it's basically making, it's basically applied philosophy. So it's the premise of the book is that the way that people instinctively kind of by default try to help the world is. Is really ineffective and inefficient, and it's driven by these kinds of system one sort of intuitive heuristics. That aren't always that epistemically rational. So for example, people. People are more likely to give to a charity if they see a picture of, you know, a starving child on the cover of the brochure.

• 51:04Shane Parrish?

  Yeah.

• 51:05Speaker 01

  And that's very understandable. It's very, it like really taps into our emotions and motivates us. But that often has no connection to whether the charity is actually effective at helping people and saving lives and improving the quality of life. So effective altruism is, I mean, simply put, it's just the idea that for whatever level of. Of money, or time, or effort, or sacrifice you want to put into trying to help the world. And we all have different levels that we're comfortable with. But for whatever level, there are vastly better and worse ways to do it. There are literally like orders of magnitude difference in the effectiveness of different charities trying to do the same thing. So just by, by kind of stepping back from your automatic emotional reactions and asking yourself, okay, what does the evidence show?

• 51:50Speaker 01

  You can just do, you can like save 100 times more lives with the same amount of money that you're donating to charities over your lifetime if you choose your charity as well. That's sort of the tip of the iceberg, but it's, I like it. It's inspiring. It's sort of a great example of rationality being used to make the world a better place.

• 52:09Shane Parrish?

  That's an awesome way to think about this as we head into the end. Listen, I want to thank you so much for your time today, and this conversation was fascinating.

• 52:18Speaker 01

  Thanks so much for having me on.

• 52:23Shane Parrish?

  Hey guys, this is Shane again. Just a few more things before we wrap up. You can find show notes at farnamstreetblog.com/podcast. That's F A R N A M S T R E E T B L O G dot com slash podcast. You can also find information there on how to get a transcript. And if you'd like to receive a weekly email from me filled with all sorts of brain food, go to farnamstreetblog.com/newsletter. This is all the good stuff I've found on the web that week that I've read and shared with close friends, books I'm reading, and so much more. Thank you for listening.