333 | Gordon Pennycook on Unthinkingness, Conspiracies, and What to Do About Them

Why are people wrong all the time, anyway? Is it because we human beings are too good at being irrational, using our biases and motivated reasoning to convince ourselves of something that isn't quite accurate? Or is it something different -- unmotivated reasoning, or "unthinkingness," an unwillingness to do the cognitive work that most of us are actually up to if we try? Gordon Pennycook wants to argue for the latter, and this simple shift has important consequences, including for strategies for getting people to be less susceptible to misinformation and conspiracies. Blog post with transcript: https://www.preposterousuniverse.com/podcast/2025/10/27/333-gordon-pennycook-on-unthinkingness-conspiracies-and-what-to-do-about-them/ Support Mindscape on Patreon https://www.patreon.com/seanmcarroll. Gordon Pennycook received his Ph. D. in psychology from the University of Waterloo. He is currently an associate professor of psychology and Dorothy and Ariz Mehta Faculty Leadership Fellow at Cornell University as well as an Adjunct Professor at University of Regina's Hill/Levene Schools of Business. He is a member of the Royal Society of Canada's College of New Scholars, Artists, and Scientists, and a 2016 winner of the IgNobel Prize for Peace. Web site https://gordonpennycook.com/ Cornell web page https://psychology.cornell.edu/gordon-pennycook Google Scholar publications https://scholar.google.com/citations?user=AIbJenwAAAAJ&hl=en Wikipedia https://en.wikipedia.org/wiki/Gordon_Pennycook IgNobel Prize citation https://improbable.com/ig/winners/#ig2016

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Your joints help you keep doing what you love. Give them daily support with Ancient Nutrition Multi-Collagen Protein. Two scoops deliver 20 grams of collagen from 10 types, including clinically studied eggshell membrane, plus vitamin C and probiotics to support skin, hair, nails, and joint health. Visit ancientnutrition.com/protein and use code PROTEIN for $15 off. That's ancientnutrition.com code PROTEIN.

• 0:27Speaker 01

  OnDeck is built to back small businesses like yours. Whether you're buying equipment, expanding your team, or bridging cash flow gaps, OnDeck's loans up to $400,000 help make it happen fast. Rated A+ by the Better Business Bureau and earning thousands of 5-star TrustPilot reviews, OnDeck delivers funding you can count on. Apply in minutes at ondeck.com. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 0:57Sean Carroll?

  Hello, everyone, and welcome to the Mindscape Podcast. I'm your host, Sean Carroll. I don't know about you, but doesn't it bug you when other people are wrong about things? Like, I know you and I generally are correct about all of our beliefs, but out there on the internet or even in society, it seems that there's more and more people who have false beliefs about things, and they even sort of nurture those false beliefs by hanging out with other people who have false beliefs. What is up with that, and what can we do about it? Now, of course, all of us have some false beliefs. And famously, there's this idea that we have biases that nudge us towards one set of false beliefs or another. Then some of us are going to say, like, there's a whole group of people who have more biases than we do, and we can have that argument.

• 1:42Sean Carroll?

  There's motivated reasoning, right? There's a reason that people, either for wishful thinking purposes or for identification with some political tribe or other kind of group, want to have some beliefs because it's part of their identity. Okay, but is that really the reason why people have these false beliefs? Either susceptibility to, just as our guest Gordon Pennycook will put it today, pseudo-profound bullshit? Or susceptibility to misinformation or conspiracy theories? And what Gordon is going to tell us is it's actually not quite about cognitive biases and motivated reasoning so much as it's about what he calls unthinkingness.

• 2:25Sean Carroll?

  That is to say, when you're faced with a claim, whether it's a claim you see on the internet, or someone's giving you a fortune cookie, you evaluate that claim. And you can evaluate it either instantly, like, "Oh, it feels right to me," right? Or "It feels wrong," or "That fits in with my views." Or you can evaluate it in a more careful, reflective, cognitive way. Like, how do I know that this claim is on the right track? What are the sources? What are the reasons to believe it? The same thing goes true for not just a proposition about truth in the world, but a saying or an aphorism, right? Like, if something feels kind of profound, we might just accept it, without thinking very much, without even thinking whether or not it makes sense. And so Gordon is going to argue

• 3:09Sean Carroll?

  that if we just sit down and think about things, all of us can be better at understanding the difference between profundity and nonsense, the difference between a conspiracy theory and something that is more accurate. And this goes very broadly, and he has some wonderful results with very high statistical significance by psychology experiment standards. And also, what is really fascinating, and we'll see whether this holds up, because it's all very new, and of course, any such claim needs to be further investigated. But there's even a suggested mechanism for talking people out of their conspiracy theorizing by having them talk to AIs, by having them talk to large language model chatbots,

• 3:54Sean Carroll?

  which are very patient. They're willing to talk to you for a very long time. And if it's a good LLM, it has access to an enormous amount of information, much more so than any one of us who is not embedded in the conspiracy theory might have access to. It turns out, again, this is a slightly optimistic finding, which I'm always happy to share with Mindscape listeners. People generally want to think things through. People want to talk about their beliefs. People are even susceptible to evidence, even the deepest conspiracy theorists. So maybe we just need more patience and access to resources to convince them that their conspiracy theories are not correct. And maybe that is a use case for AI.

• 4:38Sean Carroll?

  Just have people chat with it. Push them in the more reasonable direction. Again, we're going to have to see through further experiments whether this is the right direction to move in, but maybe this is the kind of thing that the internet and social media really need to, you know, correct the fact that it's very, very possible in today's information environment to be surrounded by nonsense and to think it's all correct. Maybe we can do better than that. That's the kind of optimistic take we're into here at Mindscape. Let's go.

• 5:25Sean Carroll?

  Gordon Pennycook, welcome to Mindscape Podcast.

• 5:27Speaker 03

  Thanks for having me.

• 5:29Sean Carroll?

  I got to start, plenty of places to start, but the one that's irresistible to me is you are, I guess, maybe one of our first Ig Nobel laureates that we've had on the show. You're a winner of an Ig Nobel Prize. Tell us about that. I mean, some people might not know what the Ig Nobels are, so maybe explain that first.

• 5:48Speaker 03

  You need to be dipping more into that reservoir. There's some great stuff.

• 5:53Speaker 03

  So the Ig Nobel is, actually, the way my mom described it was it's the Nobel Prize for smart asses, or rather, that's the way I described it to my mom. But it's for research that makes you laugh and then makes you think. And so, I mean, some of the awards are given to people that they're kind of making fun of.

• 6:13Sean Carroll?

  Right.

• 6:13Speaker 03

  And some of the awards are for people who are doing legitimate research that is both amusing and interesting and maybe even important. They don't tell you which ones are which, of course, but I agree.

• 6:24Sean Carroll?

  You like to think.

• 6:27Speaker 03

  Yeah, and that was for the research on bullshit.

• 6:30Sean Carroll?

  That's right. So you wrote a paper on pseudo profound bullshit. And of course, as someone who has a part-time position in a philosophy department, I know that philosophers are super interested in bullshit, but you're thinking of it from a more empirical perspective.

• 6:45Speaker 03

  That's right. I mean, you come across this sort of thing. The way that this was triggered, actually, was a website called wisdomofchopra.com. And so this, if you are aware of Deepak Chopra is, he's a kind of a new age guru. Doesn't use that term himself, but that's the way I might characterize him. And it's a lot of very elaborate, jargony terms, quantum consciousness, and etc. And the way that the communication seems to be geared towards is not helping people understand what you're trying to say, but trying to make it seem like you're saying something important. So, the question, though, is, do people actually find these things profound? And so we took these, like, basically, the way that the website works is, it takes buzzwords

• 7:29Speaker 03

  from Deepak Chopra's Twitter feed, consciousness, intentions, intentionality, you know, whatever. And it puts them together randomly in a sentence. So, I'll give you an example. This is my favorite one. "Hidden meaning absorbs abstract unparalleled beauty." I think I'm something like that. Anyways, something close to that. Hold on a second, I'm gonna get it right.

• 7:56Sean Carroll?

  Okay, let's get it right. Was this one of the random ones, or is this a real one from the Twitter feed?

• 8:00Speaker 03

  This is a real one.

• 8:01Sean Carroll?

  Okay.

• 8:03Speaker 03

  I'm missing just one word. And this is funny, because I guess I must have said it a thousand times.

• 8:07Sean Carroll?

  You gotta get the pseudo-profundities right.

• 8:09Speaker 03

  Yeah, exactly. You don't want to get it right. And then, hold on a second. But we also did, so we gave, "Hidden meaning transforms unparalleled abstract beauty." It's pretty good.

• 8:21Sean Carroll?

  Yeah, it sounds profound.

• 8:22Speaker 03

  It sounds profound, but you have to think about it to kind of understand that you don't know what it means. And so we took sentences like that, just random sentences. But we also took some actual tweets from Deepak Chopra. "Intention and attention are the mechanics of manifestation." That's one of the tweets, that kind of thing. They sound pretty similar, obviously. And they are psychologically... Exactly the same, like the people who believe that the random sentences are profound are the same people who think that the tweets are profound. And the kind of key part of the paper, it was mostly actually a methodological paper, it was just how do you measure one's receptivity to this pseudo profound form of bullshit?

• 9:07Sean Carroll?

  Yeah.

• 9:08Speaker 03

  And so we were just kind of, it's basically like creating a measure to assess that. And then people who tend to like rely more on their intuitions and like they've got feelings tend to think that these things are more profound. People who are more likely to kind of like go with alternative medicines and believe in pseudoscience, all the kind of things that you would expect.

• 9:26Sean Carroll?

  And so bullshit, BS, doesn't just mean falsehood in this particular case, like this is a kind of a technical term, at least in the philosophical discourse.

• 9:36Speaker 03

  That's right, that, yeah, that's a key point because we weren't just trying to be smart asses by using the terminology. There's a, there's a real philosophical literature about what bullshit is, and it's actually not even falsehoods. You know, the way that Harry Frankfurt, the principled philosopher, defined bullshit, by the way, you gotta check out, there's this great essay that became a little book that you can buy, it's a good like book for your, for your coffee table. This is called On Bullshit by Harry Frankfurt. And he distinguished between bullshitting and lying. So if you're lying, that implies that you care about the truth to some extent, right? Because you care enough about it to try to subvert the truth. Bullshitting is kind of almost the opposite where if you're bullshitting, that means you don't really care about the truth.

• 10:21Speaker 03

  You're just trying to get someone's attention, get them to think you're smart, get them to buy your product, whatever it is. It's just, truth is just not a consideration for that utterance. I mean, and you can bullshit about something that's true, you can like, it can be happen to be true. But really it's about your orientation towards the truth is what bullshit is all about.

• 10:38Sean Carroll?

  So you can accidentally say something completely correct by just bullshitting and you're, you're trying to get, but your goal is not to get to the truth. It's to elicit some reaction.

• 10:47Speaker 03

  Exactly. I mean, a broken clock is wrong. It's right twice a day. Same as, same thing can happen when you're bullshitting. Now, there have been like further just like debates within the philosophical field about how to define bullshit. And there's a whole interesting set

• 11:00Sean Carroll?

  Once you start, yeah.

• 11:01Speaker 03

  Yeah, once you start. But in for the purposes of our work, it was mostly just a matter of capturing that underlying idea of people not really having a regard for truth or evidence when they're making statements. And that captures a lot of the kind of like pseudo scientific and kind of just general new age wooey stuff that you see, you know, in books, online, etc.

• 11:22Sean Carroll?

  And when you say pseudo profound bullshit, is that a subset of bullshit? I love this conversation. This is going to be fun for the transcribers to make the transcript here.

• 11:32Speaker 03

  Yeah, exactly. I think we broke the record. We have, we said bullshit like 200 times in the paper or something. Not on purpose. It's just you use the term. Well, so yeah, pseudo profound is the category of bullshit. That's where you're, the particular goal in that case is instead of communicating in a way that actually produces meaning for the other person, it obscures it. You, I mean, any good science communicator knows that you take something complicated and you distill it so the person really understands the core underlying theory or message or whatever. This is the opposite. You take usually some sort of like basic tripe observation, and then you make it sound like it's really important, and then you can sell more books or whatever.

• 12:14Sean Carroll?

  And then, so what is the basic psychological result here? I mean, are many people very susceptible to this? Or is it a certain set of people who are susceptible universally? Or does it depend on the kind of bullshit?

• 12:28Speaker 03

  It depends on the kind. And there's no, because it's, so there's no definitive answer on like where it falls. It depends on how you measure it and all that kind of stuff. Some people are more kind of inclined towards the new age soundy, really positive stuff. But then there's like a whole other class of bullshit that's even more insidious than that, where like political persuasion and all that, advertising, etc. So it's kind of hard to answer that question. But so the key point, though, is, I mean, one way to think about how the mind works that I think Captures it pretty well is there's two fundamental different ways in which our brain kind of works when we're processing information. We can respond intuitively and automatically and that's often very useful. Like I can recognize someone's face who I haven't seen for 20 years within milliseconds. And that's an intuition that we have that's

• 13:13Speaker 03

  very effective. But also, there are times in which our intuitions are wrong. You know, the things that come to mind are things that we should be kind of like questioning. And so we have to stop and engage in effortful deliberation sometimes.

• 13:27Sean Carroll?

  Right.

• 13:28Speaker 03

  And so the key kind of message of the paper is that if we're relying too much on that kind of intuitive gut feelings, then we're going to eventually fall prey to bullshit in the world. And so we should really be thinking more about the stuff that we're engaging with. And that might be particularly true online.

• 13:44Sean Carroll?

  So is it a system one, system two thing?

• 13:47Speaker 03

  That's exactly what it is.

• 13:49Sean Carroll?

  Yeah.

• 13:50Speaker 03

  Yeah.

• 13:50Sean Carroll?

  I always forget which is system one and which is system two.

• 13:53Speaker 03

  System one is the first one, which is the intuitions. System two is the kind of like thing that is kind of more optional and happens afterwards.

• 14:02Sean Carroll?

  Well, and this is one of, it seemed to me to be one of the most robust and believable conclusions of psychology, which is that most of our thinking is sort of subconscious, intuitive, quick, system one stuff. And there's only like a little bit of super effortful system two guidance at the top.

• 14:21Speaker 03

  Exactly. I mean, and evolutionarily, that makes sense. Like if you have a process that requires resources, cognitive resources, energy, then it wouldn't be adaptive to be doing that all the time. The problem is that people vary in how much they do it. They don't do it when they need to do it. And sometimes they're doing it when they shouldn't do it. I mean, there's cases in which you can overthink also. And so knowing when to expand effort is really the kind of the trick to making better choices.

• 14:50Sean Carroll?

  Okay. So basically, I mean, as a psychologist, a psychologist, the right Noun for you?

• 14:56Speaker 03

  I'm a psychologist, yeah.

• 14:56Sean Carroll?

  Okay, good.

• 14:57Speaker 03

  I'm a experimental psychologist, yeah.

• 14:58Sean Carroll?

  People get upset if I use the wrong words for whatever their field is. So do you try to then correlate, you know, how well people do in recognizing the pseudo profound bullshit with how much effort they're putting into cognition versus just intuition?

• 15:13Speaker 03

  Exactly, that's what, yeah, that's what we do in the paper. It's like, we have ways of measuring the extent to which somebody relies on their intuitions in general. And then we, then we have these like various other kind of dispositions that people have, like how receptive they are to bullshit. Or other like, just general attitudes or beliefs, like their stance on alternative medicines, or whether they trust science and stuff like that. And like, and through lots of different studies on lots of different topics, people who are more intuitive have really different beliefs and ways of processing information. They tend to believe more in the pseudoscience. But also, like, I'll give you one completely kind of more random example. In grad school, one of the studies that I was working on with the

• 15:56Speaker 03

  emeritus professor Al Shane. And what Al was, he was a global expert in sleep paralysis.

• 16:03Sean Carroll?

  Okay.

• 16:04Speaker 03

  So, sleep paralysis is when, you know, if you're dreaming about running, your body doesn't get up and run. There's a kind of a disconnect between what's going on in your mind and what's happening with your body. And so, in a certain sense, your body's kind of paralyzed while you're sleeping. I mean, you're moving around, but it's not connecting the thoughts in your head to your actions. Sometimes you could, you, when you're waking up, you're in a semi-conscious state. And so, you're sort of awake, but your body is still asleep. And so, it feels like you're paralyzed. And many people hallucinate. They think that there's a demon on their chest, whatever. People who are more intuitive are more likely to believe in the demon on the chest and the kind of fairy tales. But, but the interesting that we found was that the people who are more analytic, who question their intuitions, have less distress following sleep paralysis. Like, in the days that after

• 16:48Speaker 03

  this kind of event, which is very scary for everybody.

• 16:50Sean Carroll?

  Right.

• 16:51Speaker 03

  They're not as distressed because they're using their thinking to be basically kind of like, contextualize the event to deal with the emotions and all that kind of stuff. So it has kind of these wide-ranging effects on lots of things.

• 17:00Speaker 01

  Cash flow crunch? OnDeck's small business line of credit gives your business immediate access to funds, up to $200,000, right when you need it. Cover seasonal dips, manage payroll, restock inventory, or tackle unexpected expenses without missing a beat. With flexible draws, transparent pricing, and control over repayment, get funded quickly and confidently. Apply today at ondeck.com. Funds could be available as soon as tomorrow. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 17:31Speaker 04

  Rain outside your house? Relaxing. Rain inside your house? Less relaxing. That's the sound of a bad day. That's when you want to hear something reassuring. Cincinnati Insurance works with local independent agents who get to know you and bring personal attention to protecting your home and most valuable possessions. So when a bad day sounds like this, you have 75 years of expertise helping make it better. Find an independent agent at cinfin.com.

• 18:01Sean Carroll?

  And is it that there are certain kinds of people who are just susceptible to this overall? Or is it that there's like certain moments in my life when I'm susceptible to it? Like, can I be told, "Oh, focus now and try to use your cognition," and that makes me less susceptible to bullshit?

• 18:21Speaker 03

  So both. There are people, I mean, so everybody relies on their intuitions. And probably everybody could use more, could spend more time questioning their intuitions and thinking and reflecting. So that's just not, this is not like a, and certainly people, researchers usually think of about themselves as people, "I'm the reflective one, so it's me versus them."

• 18:39Sean Carroll?

  Of course.

• 18:40Speaker 03

  It's not like that. Everybody needs to question their intuitions. And there are cases in which we have blind spots. You know, like I'm a, I'm a sports fan. You know, I'm not, I'm not making a lot of rational judgments and I don't have a lot of rational beliefs when it comes to teams that I cheer for. And so there's, you know, we have strengths and weaknesses when we do this sort of thing that we all could work on, for sure.

• 19:01Sean Carroll?

  I have this argument, because I'm a sports fan too, and it's not really an argument, but I have this feeling that these days, there are too many sports fans who are trying to be objective about their team. They're like, you know, I think that we should not sign this person to this contract because blah, blah, blah. And I'm like, why, who cares about that? I just want to like, assume that my team's going to win every game and root for them. Like, we're giving people too much access to the mind of the general manager these days.

• 19:28Speaker 03

  I think that's a solid point. I mean, I've had this conversation with a friend of mine who viewed my sports fandom as being an inconsistency, as someone who really values, you know, engaging analytically with the world. Whereas, but what I said to him was like, for me, it's a rational choice to allow myself to be irrational in this domain. It's more, it's more fun if you just watch it and you hope for the best.

• 19:52Sean Carroll?

  The whole point of sports is to like, be a little bit irrational and just let that part of your brain go.

• 19:58Speaker 03

  Although I'm a Maple Leafs fan, so I'm not, if you have any hockey fan listeners who don't know that, that was, it's not irrational. It's not working out for me, but, you know, someday maybe it will.

• 20:10Sean Carroll?

  Speaking of parts of the brain, I mean, how much can we be neuroscientists and actually connect this system one, system two, cognitive versus intuitive thing to particular parts of the brain doing particular actions?

• 20:25Speaker 03

  So that's, it's, it's a difficult question because it's, I wanted to sway people from thinking about it as actually different systems. I didn't ever usually use the term system, because it's not like there's two parts of the brain, one does one and one does the other. And they're also highly interconnected. So like, if you think about, if I give you a math problem, 17 times 37. Okay. So unless you've memorized that particular question, nothing's going to pop into your head. You have to decide to think about it. But what you do is you break it up into easier problems that you solve intuitively. 10 times 20, whatever. And then you put, then you hold those in your mind, then you put them together. And so there's deliberate steps that require intuition. So you can't, if you, you can put someone in a scanner and then do that, but like, knowing what are

• 21:09Speaker 03

  the intuitive parts and what are the deliberate parts, that's, that's a pretty complicated question to answer.

• 21:13Sean Carroll?

  And the way you're using the idea of intuition, it's not necessarily like instinct or innate. It's something that you can actually learn.

• 21:20Speaker 03

  Oh, certainly. Like, if you... My favorite example of that is chess grandmasters. If you... They can immediately identify, just through rote memorization and playing the game and like, literally like, studying 50,000 different orientations on the board. That's in the same way that you memorize someone, you see someone's face and can identify it, they identify the orientations on the board. That's purely intuitive, but of course, it's not like they were born with that capacity. They have to learn it by thinking analytically, so the things work together.

• 21:49Sean Carroll?

  And so, when faced with these pseudo-profound bullshit statements, we... Well, so again, like, is it most of the time, we only engage at our intuitive level? Or is it, uh, there are some people who are just really bad at going beyond the intuitive level?

• 22:06Speaker 03

  Both of those things. Most of the time, we engage at the intuitive level, but there are some people who don't really do the other thing that much. They really don't... And some people actually literally value deliberation more than others. And those people also tend to value evidence more and getting it right. So, just as an anecdote, my mother-in-law, who I love very much, and she's a nice lady, and there's no animosity, she explicitly identifies as being a not rational person. Like, she just, she thinks feeling and emotion is a kind of a better way of engaging in the world. And it's not, I wouldn't say that's completely kind of like thought out. That's just what she thinks, or like, what she feels. And so, she's just totally fine with being kind of irrational and not deliberating, and that's just...

• 22:50Speaker 03

  So, if I could try to encourage her to be more reflective and deliberate, you know, but she's not gonna really do it because she doesn't see the value in it.

• 22:58Sean Carroll?

  And it sounds like this is something you can test with your, um, array of pseudo-profound statements, how good people are at recognizing them, but it probably extends beyond that narrow categorization to go beyond like, how we deal with the world more generally.

• 23:12Speaker 03

  Certainly, yeah. I mean, you can assess it in some ways by kind of just asking people the right sorts of questions, but you can also do tests. Like, I'll give you an example of a question that we use that kind of probes this sort of thing. So, uh, if you're running a race and you pass the person in second place, what place are you in?

• 23:30Speaker 03

  A lot of people are gonna wanna say they're in first place, but of course, if you pass the person a second, you're now in second, and the person in first could be a mile ahead, you don't know. But the way that you think about that intuitively is you maybe imagine passing the person.

• 23:42Sean Carroll?

  Yeah.

• 23:42Speaker 03

  And you're not imagining the person in first, you just imagine you pass and now you're in first in your mind, right? And so like the intuitive answer is different than the one that you get from, you know, in that case, it's not a lot of thinking that helps you understand that. Like, once you explain it to people, everyone understands that no one's disputing that the correct answer is in the second place, but they just have to think about it in the right sort of way to get the right answer.

• 24:01Sean Carroll?

  Is this something that we can train ourselves to be better at?

• 24:04Speaker 03

  So, I think, I think the jury is out on that to some extent. I mean, there are certainly, when, it's hard to teach old dogs new tricks. I think there are ways we could intervene and encourage people when they're developing reasoning skills to get in the habit of questioning their intuitions. Taking someone who's thought about the world in a particular sort of way for decades and changing the way they think, this is not a trivial thing. And I think we haven't really done those sorts of long-term, heavy intervention experiments that you would need to do to really test that. So, people might, other scholars in the area might disagree, but it's, I think that the jury is basically out on that to some, to a great extent.

• 24:47Sean Carroll?

  I think, I think we'll get to this later again in the conversation, but I do wonder, as a non-expert, when it comes to psychology experiments that are looking to test the efficacy of interventions, how much they do care about the time scale, right? The time horizon. It seems like I'm worried, and actually, this is the biggest worry in kind of politics, where you ask people, you know, do you agree with this statement politically? And they give you an answer, but maybe they're changing over time, and you know, maybe you tell them something and they switch their view, but then they switch back, they fall back to an equilibrium. Is that a worry?

• 25:20Speaker 03

  It's a, it's a big worry and it's actually even uh larger than that because for psychologists, we have people for fifteen minutes, thirty minutes, you know, uh and so the sorts of interventions you can do if you want to really test causal mechanisms are things that you can do in a matter of minutes.

• 25:37Sean Carroll?

  Right.

• 25:37Speaker 03

  And there's not, that takes away a lot of bullets out of the chamber when you're trying to really understand human nature. It's human nature in very slim, thin slices. And that's, it's almost an intractable problem because like, you know, um we, it's not like we have, we can treat people in the same way that you would have, you know, birds in a, a bird laboratory.

• 25:56Sean Carroll?

  Mm-hmm.

• 25:57Speaker 03

  Uh, I say that because I'm at Cornell, there's lots of bird labs here. Um, so it's, that makes it more difficult. But at the same time, there are really interesting things you can learn, of course, from the small snippets. And in, in many ways, that's what our lives are, just a collection of small snippets. And so you, but you have to kind of understand the scope of that and it's a, it's a general problem for, for sure.

• 26:16Sean Carroll?

  I did have Joe Henrick on the podcast a while back and you know, he, he emphasizes the weird culture kind of thing and I presume that most of your experiments are done on college undergraduates at Cornell. Um, do you, do you, are there cultural differences between this ability to detect pseudo profound bullshit?

• 26:37Speaker 03

  So actually in uh most cases, we actually don't. I actually, I haven't run a study with uh student participants since grad school.

• 26:46Sean Carroll?

  Okay.

• 26:46Speaker 03

  Uh, which was about nine years ago. We use online samples that have a broader, more representative kind of like set, but they of course are not truly representative. These are people who are engaging in online studies.

• 26:58Sean Carroll?

  Online, yeah.

• 26:58Speaker 03

  For fun or for work or whatever. Um, and we have done lots of uh cross-cultural studies, but usually among similar samples in different cultures. So a lot of stuff that Joe would talk about would be like going to places where people don't usually go to do their own studies.

• 27:13Sean Carroll?

  Yeah.

• 27:13Speaker 03

  Out to Amazonian tribes and stuff like that. And you know, that's such a small fraction of the amount of psychological work because it's much harder. And also it'd be very annoying to all the people out in the Amazon to have thousands of researchers always hanging out, you know what I mean? So there's a, you know, you can't, not everyone can do it, but yeah.

• 27:30Sean Carroll?

  I guess I'm just wondering, you know, are there like? Differences of discipline between, let's say, Northern Europeans and Southern Europeans, or Buddhist monks and atheists, or something like that?

• 27:43Speaker 03

  Yeah, in terms of the bullshit thing?

• 27:45Sean Carroll?

  Yeah.

• 27:46Speaker 03

  I haven't looked at that in particular. I think there was a study that looked at a form of this, which was if you tell the person that it's like an expert, then they'll think it's more profound. And that effect was consistent across a bunch of different cultures.

• 27:59Sean Carroll?

  Okay.

• 28:00Speaker 03

  But mostly undergrad student samples. So there's always like a caveat in all these kind of cross-cultural studies and stuff like that. So there's like, I think, I can't see any particular reason why this, apart from the fact that certain sorts of pseudo-profound bullshit are more common in different cultures than others. And so you might be used to that sort of terminology or whatever. But the underlying propensity to kind of align your feelings and kind of assume that because it doesn't make sense, it might be meaningful. I think that there's no, I don't think there's any particular reason. I think that's specific to a particular culture.

• 28:33Sean Carroll?

  Right. Am I correct in recalling from one of the things you wrote that people are better at recognizing when other people are falling for pseudo-profound bullshit than they are recognizing when they are falling for it?

• 28:45Speaker 03

  Oh, this is like a psychological truism. Like, it's way easier to see bias in others than bias in ourselves. It's almost definitionally. Because if you saw the bias, then you wouldn't have the bias. But we can, and this is kind of the critical problem is that when we are very bad at detecting our own bullshit, and that's the bullshit that's the most important to detect. That's the stuff that's gonna have an impact. And by the same token, being overconfident is like one of the most, probably the mother of all biases, essentially. Like, it's the thing that leads us to not really question that we might be wrong is because we are overconfident. And this is a very endemic problem for people in general.

• 29:26Sean Carroll?

  I have to ask, at the danger of going down a rabbit hole, but is there a political component here? Are certain sides of the political spectrum more ready to fall for the pseudo-profound bullshit than others?

• 29:41Speaker 03

  On the super found bullshit, there's this kind of a slight correlation where people on the right tend to be a little bit more supportive of that sort of thing. People would think that's the opposite because it's like the new age woo market tends to be kind of left coded. But generally speaking in our samples, that's not really the case, but it's a pretty small correlation. It's much bigger if you move into other realms of bullshit. So a lot of my work relates to misinformation and fake news. That's basically another category of bullshit in many ways, depends on how it's made. But there you see a very big political asymmetry. In the US in particular, there's way more misinformation on the right than on the left. But this is one of the difficulties with making inferences from psychological

• 30:26Speaker 03

  studies. So if I had a random sample of misinformation headlines or content or whatever, people on the right would believe them more than people on the left. And then you might conclude, well, people on the right are particularly susceptible to misinformation. However, at the same time, there's an asymmetry in exposure. There's way more misinformation on the right than on the left. And this is not just, that sounds like a political statement. There's been like dozens of studies that, and you can look at various different ways to determine fact checking. You can look at like fact checker reports. You can look at journalist reports. You can even get politically balanced samples where you have both Democrats, Republicans at equal measure rating the truth and falsity of statements. And even based

• 31:09Speaker 03

  on that, there's more falsehood on the right. So it's less about susceptibility, well, maybe to some extent it might be susceptibility, but also it's just the market. And so these things are difficult to disentangle in many ways.

• 31:20Sean Carroll?

  Was that true 50 years ago, or is it new?

• 31:23Speaker 03

  That's a great question. And I'm going to caveat this by saying I'm not historian. I took a minor in history in undergrad. That does not count, that does not make me a historian. But I would say no. I mean, to take to my historical take on this is that it's Reagan's fault and the kind of war on science really started in earnest around that time. A lot of it has to do with climate change. In fact, if you look at the Scopes Monkey Trial, which was the one of the first, that was the case in which they were trying to outlaw teaching evolution in schools. It was William Jennings Bryan, who was a Democrat, and that was 1929.

• 32:07Speaker 03

  Of course, that was a Dixie Democrat, and they become Republicans under Reagan. But anyways, the war on science is something that has kind of progressed gradually over the last few decades, and now it's just, it's expanded well beyond that.

• 32:22Sean Carroll?

  We did have a nice podcast conversation with Naomi Yareski, who really laid out the history of that, for anyone that's interested in the history.

• 32:29Speaker 03

  Go check it out. She's, yep.

• 32:32Sean Carroll?

  Yeah, it was not just natural. It was very, very, you know, driven by political and economic forces. So misinformation, yeah, let's get into that. And what is the relationship we should have in our mind between misinformation or fake news and conspiracy theorizing? Because they're certainly interconnected.

• 32:48Speaker 03

  They're interconnected. I mean, a lot of misinformation contains conspiracies. A lot of conspiracies contain misinformation, but of course, they are not completely overlapping. Not all misinformation is about a conspiracy, obviously. And some conspiracies are true. You know, like the Tuskegee syphilis trials and MKUltra and whatever. It's just that most of the conspiracies that we talk about as like conspiracy theories are the unverified speculative sort. And so they kind of like, they are often connected in literature. In many ways, the underlying psychological processes are similar. Because the underlying

• 33:29Speaker 03

  kind of thing that matters for, at least for me as a psychologist, is kind of detachment from reality, implausibility. Does, are people making claims that are consistent with evidence or are made up in bad arguments or based on bad evidence?

• 33:46Speaker 05

  OnDeck is built to back small businesses like yours. Whether you're buying equipment, expanding your team, or bridging cash flow gaps, OnDeck's loans up to $400,000 make it happen fast. Rated A+ by the Better Business Bureau and earning thousands of 5-star TrustPilot reviews, OnDeck delivers funding you can count on. Apply in minutes at ondeck.com. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 34:14Sean Carroll?

  I think there's probably a widespread belief that susceptibility to conspiracy theories has something to do with motivated reasoning. Like you believe things because you want to believe them, or your friends believe them, or whatever. And my impression is you want to push back against that a little bit.

• 34:30Speaker 03

  That's the accurate impression, and that's a great question, Sean. I have to try to regulate how long I spend answering this question.

• 34:36Sean Carroll?

  Go for it. We have no limit.

• 34:39Speaker 03

  So it is a kind of like, almost a tourism among the general public and within certain circles of psychology that the reason that we fall prey to political, or otherwise, falsehoods, is sort of because we want to. You know, like, we have these motivations to be a part of a political group, and we have these identities that we want to preserve, or we just want to feel good, or whatever. And so there's a lot of theories that put that at the forefront of the explanation for why people seem to be so susceptible to misinformation. But that counters, in many ways, the way that I've already described things to you, which is, the reason why people are susceptible to misinformation is because they're not really thinking that much about what they are engaging with,

• 35:24Speaker 03

  and what they're coming across, or what their beliefs are, or what their intuitions are, or whether they might be wrong. And so that's a different story. That's more about, you might call it lazy thinking, than kind of like, where people are literally, they're so motivated that they're spending all this extra effort convincing themselves that the things that they want to be true are true. And that doesn't really consist, that doesn't really kind of accord with the idea that we're kind of lazy thinkers, and we don't expend extra effort. And so we've done all these studies where you, like, for example... People who are, if you give people fake news headlines that are consistent with their ideology or inconsistent, people who are more reflective and analytical are better at distinguishing

• 36:05Speaker 03

  between the true and false ones regardless of whether they're inconsistent or consistent with their ideology. It doesn't, it's not contingent on that. It's just knowing whether someone is going to believe something, whether something's true or false, knowing whether it's true or false, people believe more true things than false things in general is important, but also like are they reflective and deliberative? That's, that's critical. It's not just knowing whether they're political or not.

• 36:28Sean Carroll?

  So in a nutshell, you're saying the problem is not motivated reasoning, it's unmotivated reasoning, is you're not motivated to put in the work to reason?

• 36:36Speaker 03

  Exactly. That's exactly, that's, I've been saying that and sometimes I say that exact thing. So, yes, exactly. I think, yeah, the idea that people are too motivated is, is really going in the wrong direction. Yeah. I mean, there are contexts in which people are engaging in motivated actions, like they, some people are trolls on the internet and they're trying, or, or they have vested interests and they're trying to persuade other people to take their position, or they're like, maybe they're selling a product or whatever. I'm not going to like say that there's no motivations ever, but

• 37:04Sean Carroll?

  Sure.

• 37:04Speaker 03

  I think if we're going to like take the, the big pie of people believing things that are on bad evidence and false, a lot of it is just because they haven't thought about it.

• 37:13Sean Carroll?

  And this tendency to sort of not be willing or able or interested in putting in the cognitive effort, this seems like prior to a whole bunch of things. Like if, if you're that kind of person, you're going to be susceptible to lots of misinformation and conspiracy theorizing.

• 37:29Speaker 03

  Exactly. And you often see people who, people who are likely to believe one conspiracy are also likely to believe a conspiracy that might even directly contradict it, you know? And, or they, or they might believe things that are really separate, like people who are very often have lots of religious beliefs, they believe in angels and demons and, you know, heaven and hell, all those things, but also really believe in like superstitions and like things that would be classically referred to as like occult counter to religious claims, because it's like they're, they're just not scrupulous there. The things that they see are kind of like just already believed.

• 38:09Sean Carroll?

  Yeah.

• 38:10Speaker 03

  And they're not really putting the effort into like distinguishing between what are the things that are true and what are the things that are false.

• 38:15Sean Carroll?

  This might be too vague to even answer, but I'll give it a shot. Are there kind of characteristics that people have that go along with this tendency to not want to do the cognitive effort? You know, does it correlate with just other, I don't know, personality traits or other features of human nature?

• 38:37Speaker 03

  That's a good question. I mean, in general, but not as much as you would think. Like, people often will think about demographics. It doesn't really relate that much to demographics. You know, there's, you know, things like personality traits, like being open, open to experiences. It's not as related as you might think. You can be a very reflective person and be open to experiences, you know what I mean? Or introverted, or extroverted. These are like, just kind of separate psychological mechanisms. One thing that is very highly related to whether you have the disposition to think in an analytic way is your stance on the kind of importance of questioning evidence

• 39:18Speaker 03

  that goes against, or questioning evidence that is consistent or inconsistent with your views. And, I mean, it sounds like I'm saying the same thing in two different ways, which is like, naturally, in order to question the evidence, you have to be deliberative. But it's more about whether you value evidence, whether you kind of value accuracy. And, of course, people who are more reflective tend to be better at it. They tend to be more intelligent and have higher cognitive capacity in other domains. So, it's a kind of collection. If you have all those things at once, then that's the, those are the people who tend to be the most kind of like, pro-science and etc.

• 39:54Sean Carroll?

  But it does sound like maybe something that training, or education, or exposure could help with. If you're just sort of used to thinking things through in a carefully deliberative cognitive way, that would make you less likely to fall for conspiracy theories, for bullshit, etc.

• 40:11Speaker 03

  I think that's, I think that's for sure true. And it's, it's now going back to like, different domains. Like, as someone who, and you're someone who, you know, you did a Patreon thing, once you, once you have really dug down into a topic... Then you, once you've done all that kind of like actual deliberation, then you see the difficulties and uncertainties in the thing. And then, once you, now, once you see something that's within that domain that you've thought about, you immediately can be the reflective person. You say, wait, wait, I'm used to there being more, this is not so straightforward. This is complicated. But then if you read something that's outside of the main, then you can kind of see where you're not being as reflective with that. Another example is,

• 40:54Speaker 03

  there's also the kind of simple behaviors that you can teach yourself to do that are more reflective. Some of her work on misinformation is about, when people see like false content online, they might share it without even thinking about whether it's true. Like, they, they might be thinking about other things. They might be thinking, oh, is this important? Or how does it make me look? And so they're just, but they're just not like, the thing that's popping to their mind is not like, is it accurate? And so we, we've done these experiments where we just remind people about accuracy. We just give them little, we ask them questions before, at the start of experiment about accuracy. Then they're better at distinguishing between the true and false stuff when they're deciding what to share. Or like little ads that are about like, make sure you think about accuracy. That, those little things can get people to be a little bit more reflective about the truth. It's not going to make them more reflective people in general,

• 41:39Speaker 03

  but in that particular context, that choice can be more rational.

• 41:44Sean Carroll?

  So you think we should buy a whole bunch of pop-up ads saying, accuracy matters?

• 41:47Speaker 03

  Yeah, I mean, we have done that experiment. It does have a small effect. I mean, it's not going to save, save everything, but it's, it's, it, it stops a subset of behaviors where people are reflexively sharing things without even considering whether they're false and that, or true. And that's a kind of a problem.

• 42:02Sean Carroll?

  I remember a long time ago, reading a, an article about a biographical sort of essay by someone who had been really, really into new age beliefs and had eventually abandoned them and, and become more like rational, scientific, whatever. And one of the things that they said was, they had this impression when they were in the new age group that, you know, one of the problems with scientists is they claim they know everything. They're certain about everything. They have all the answers. They think they have all the answers. But she realized eventually it was exactly the opposite. Like, there were questions you could ask a scientist and they would say, well, I don't know, we don't know yet. We don't have the answer to that yet. But for her new age friends, there were no questions that she could ask that they wouldn't give an answer. Do you think that there is some like desire for certainty

• 42:45Sean Carroll?

  that, that makes people more susceptible to this?

• 42:48Speaker 03

  There is some research on what's need for need for cognitive certainty, cognitive closure. And like, and so that is an element of that kind of it's it's a legitimate individual difference that people have where they need to have more certainty and that having the need for certainty is not usually generally good.

• 43:08Sean Carroll?

  Right.

• 43:09Speaker 03

  You know what I mean? It's it's scientists understand this that you the if unless we do not live in a simple world, you can pretend that you do, but you but you still don't. And if you want to have certainty, then you have to often construct it yourself. And that I mean, there's there's a version of it that's fine, which is like, I don't I you cannot be OK with not knowing if you're a scientist. You have to have the drive to figure it out. But you have to be, you know, in the same sense that you have to be acknowledged that you don't know. And that you want to get to a place where you do know, and you have that kind of thirst for for knowledge, but you have to be OK with the fact that you don't know. And so it ultimately comes down to overconfidence and intellectual humility.

• 43:52Sean Carroll?

  Well, you that's another thing that you studied, the level of confidence that various people have in their beliefs. And again, I'm I'm gonna, I'm gonna say what I think that you said, and you can correct me if I'm wrong, that people who are prone to believing in the conspiracy theories are actually more likely to be overconfident in their beliefs generally than people who are more skeptical.

• 44:16Speaker 03

  That is, that is what we've found. Yep.

• 44:19Speaker 03

  So this goes to what you're saying. So need for need for certainty is kind of one element of it. But there's a we have this test that we're giving people that's a general test of just whether you think that you're good at things when you aren't.

• 44:30Sean Carroll?

  In perfect generality.

• 44:32Speaker 03

  Just like that's that's the intention of it. Yes. Now we have to convince people that we this is right, and we're at the earlier stages of that. So the scientific uncertainty being discussed as we speak. But it's it's a very simple test. We give people like a fuzzy image that's hard to discern. This is better visually, but just try to imagine something that is difficult to discern. And then we make them guess and like, is it is there a chimpanzee or a baseball player? And people don't know. They just like their guesses are random. And then we ask them how confident they are and how many do they think they think they got correct out of like maybe 10. And the people who think they're doing better, they aren't doing better. Like, they, they, everyone's guessing, but some people do think they're doing better.

• 45:16Speaker 03

  You know, and they have a feeling that like, you know, at some level know that they're guessing, but they also really feel like they could do it.

• 45:24Sean Carroll?

  Right.

• 45:25Speaker 03

  Uh, and that's what we're calling this general overconfidence. It's a, it's the sort of over, it's a task that's completely novel. So it's not like they, there's some other like background thing that led them to be overconfident about that thing. They just like brought that to the study and they're overconfident. And those are the people who tend to be more likely to believe conspiracies and, uh, and it's got all sorts of other possible downstream consequences.

• 45:48Sean Carroll?

  So it's similar to, but not the same as the Dunning Kruger effect, right? Which I'm not even sure if that held up psychologically, but the idea that like a little bit of knowledge makes you way overconfident in your knowledge in some domain. But you're being, you're just identifying a general psychological tendency to overconfidence.

• 46:06Speaker 03

  Exactly. In fact, what we're trying to do is circumvent the Dunning Kruger problem. The Dunning Kruger problem is that if you're really bad at something, uh, it's hard to know how bad you are at that thing because the same thing that you use to be good at it helps you understand how good you are at it. You know what I mean? So the, the most incompetent are at least able to recognize their incompetence. And what that means is if I want to measure how overconfident somebody is, if I give them a math test and they happen to be bad at math, you know, maybe they had a bad school or whatever, they don't like math, they'll appear overconfident. But if I gave them a test of like how good they are identifying humor, they might be good at that and they won't appear overconfident on that test. And so it has more to do with the test than their general tendency to be overconfident.

• 46:49Speaker 03

  This is why we devised a test where there's no relationship between how good people think they're doing and how good they actually are doing. It's just completely, all the action is in the confidence and not in the performance.

• 46:59Sean Carroll?

  And this does seem compatible, consistent with the idea that the people who are susceptible to this kind of stuff are just not putting in the cognitive effort to reason through it.

• 47:08Speaker 03

  Yeah, you're seeing, you're seeing the thing, the thorough line through all the things. It's really just, it's really about unthinking this for sure. And people who are overconfident, you might not be surprised to find tend to be more intuitive. Uh, and you know, they're particularly bad at those tests because if you give, you know, remember the test, the question I asked you about running a race and you pass the person in second place, the people who give the immediate answer and it doesn't dawn on them that they might be wrong. Those are the most overconfident people, right? And I've had this way back in the day when I gave these tests to actual like in-person participants, like undergrad students. You know, a person would raise their hand and be like, why are you giving us these easy problems? You know, and they've gotten every single one of them wrong, right? So that's overconfidence, right?

• 47:53Speaker 01

  OnDeck is built to back small businesses like yours. Whether you're buying equipment, expanding your team, or bridging cash flow gaps, OnDeck's loans up to $400,000 help make it happen fast. Rated A+ by the Better Business Bureau and earning thousands of five-star TrustPilot reviews, OnDeck delivers funding you can count on. Apply in minutes at ondeck.com. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 48:23Speaker 04

  Rain outside your house? Relaxing. Rain inside your house? Less relaxing. That's the sound of a bad day. That's when you want to hear something reassuring. Cincinnati Insurance works with local independent agents who get to know you and bring personal attention to protecting your home and most valuable possessions. So when a bad day sounds like this, you have 75 years of expertise helping make it better. Find an independent agent at C I N F I N dot com.

• 48:52Sean Carroll?

  And you use the word unthinkingness. Is that a technical term? I like that. I'm going to start using that.

• 48:58Speaker 03

  I don't know. I don't think I've written it down. I'm not sure that I've ever even said it before, but you can use it.

• 49:03Sean Carroll?

  It's a good one. We should, yeah, we should definitely promulgate that one. And then, so another interesting result you got along these lines is for the conspiracy theory angle, wondering whether or not people who believed in conspiracies, are they like proud of being in a tiny minority that no one has the truth except them? Or are they of the opinion that secretly everyone agrees with them?

• 49:24Speaker 03

  That's, I think, this is, to me, that part of the paper was the most interesting part because there's this idea of conspiracy believers, I think that's consistent to some extent with the overconfidence thing where they think, well, I'm the one who knows the truth and every, and all the, and everyone disagrees, the scientific establishment disagrees, but it doesn't matter because I'm the smart one and they're dumb ones and whatever. And that is consistent with this kind of motivational idea of conspiracies that we talked about before, where it's like people believe conspiracies because it makes them feel good, you know, because it fulfills, like, it's consistent with all these needs they have, the need to feel unique, for example, is one of the things that people have written about.

• 50:05Speaker 03

  However, in that study, what we found is, we asked people to estimate the extent to which people, other people, agree with them. So like, and, and the idea, underlying idea with overconfidence that we're talking about is that it's the unthinkingness that's important. So if you're overconfident genuinely, then you're going to overestimate how much people agree with you, because it's like, how could, how could anyone disagree if it's so obviously true what I believe? Right. And so I'll give you an example. In, in one study, we asked people, like, there's a bunch of, a bunch of different conspiracies, but one of the conspiracies was the Sandy Hook false flag conspiracy, which is, which is a pretty ridiculous one. That was an Alex Jones special.

• 50:48Speaker 03

  And in that experiment, 8% of people thought that was true. So it's a pretty friend-

• 50:52Sean Carroll?

  Could you just like, give us more of the background? Like, I always like to imagine people are gonna listen to this 500 years from now and they'll know what Sandy Hook is.

• 50:58Speaker 03

  Right, right. So Sandy Hook was a horrible massacre of children, and the conspiracy was that it was a false flag, that, meaning, there was actually no children that were killed in that, despite, you know, very, kind of obvious evidence to the contrary, and like, parents giving interviews and like, all that kind of stuff. And, and that was, it was, and it was mostly to do with like, you know, people were, Sandy Hook had some impact on whether people wanted to like, you know, regulate guns in the country and all that stuff. Obviously, we haven't gotten very far on that one. Maybe in the future, someone listens to this podcast and they're like, you guys had guns? Uh, but in any case, so it's a, so it, but most people don't, most people realize that Sandy Hook actually happened. It's not a false flag, but eight percent of people thought it was more likely

• 51:43Speaker 03

  to be true than false. And then we asked the people, everyone to estimate what percent of people agree with you. Okay. And so if they're calibrated, you know, they will say eight percent of people agree with them or 10% or maybe they overestimate and they think, well, only 1% of people believes this. I'm the, I'm in the minority. Um, in reality, what they said was 61% of people they thought agreed with them. So, so they thought they were in the majority. In fact, almost all cases, people who believe conspiracies think they're in the majority, uh, even in cases where less than 10% uh, uh, agree. And so they have no idea where they are relative to other people. And the people who are overconfident are even more likely to overestimate how much people agree with them because it's just, how could you possibly disagree with this thing that's

• 52:28Speaker 03

  obviously true? I mean, uh, I can't be wrong. Uh, and that's overconfidence.

• 52:33Sean Carroll?

  Is part of that kind of an information bubble situation where they're hearing the same things over and over again, and they figure everyone else is hearing the same things?

• 52:40Speaker 03

  Yeah, exactly. I mean, some of it has to do with, I mean, think about the experience of the conspiracy believer. Uh, then I have people that I know that are, you know, down the rabbit hole. You, you come up to somebody, uh, maybe it's a family member at a barbecue or something, and you start talking about a conspiracy, uh, what's going to happen? Well, most likely scenario is that they walk away or they try to change the subject. Uh, they might vaguely agree with you to kind of be polite.

• 53:06Sean Carroll?

  Yeah.

• 53:07Speaker 03

  Very, very infrequently will people say, you're out to lunch. Uh, uh, it's maybe to some extent online, but even there, even if you were, you post a conspiracy on Facebook, most of the time you get a, you're going to get a few likes and then the ones are not going to say anything about it. And so that feels like agreement, I think. Uh, and then you go on to, you know, whatever dark, uh, parts of the internet that you hang out to talk to people and everyone's in agreement. And so, yeah, it makes sense that it seems like everyone's uh, in agreement here. Um, so it's, it's, it's actually called the false consensus effect, which is an old effect, but it's, it's like the biggest false consensus effect I've ever seen. It's very, very large effect. There is like very little calibration in terms of what conspiracy believers, you know, think other people believe relative to them.

• 53:49Sean Carroll?

  Well, and this is a perfect segue into the other thing I want to talk about because, I mean, you're right, I would certainly generally not engage with someone who I thought was a completely loony conspiracy theorist. I don't have the patience to do that. But who does have the patience to do that is chatbots. So maybe we can make AI programs talk people out of their conspiracy theories. What do you think about that?

• 54:13Speaker 03

  Hey, you know what? We have a paper on that, too.

• 54:14Sean Carroll?

  Oh, good.

• 54:18Speaker 03

  It is the case that chatbots are way more patient for this sort of thing. And also, forgetting about patience, and this is the more important part, they have access to all the information that you need in most cases. If you were of the disposition to debate a conspiracy theorist, you'll you soon realize that they're talking about things that you've never heard of. Because they went down the rabbit hole. Unless you went down the rabbit hole looking for debunks, then you're not, it's gonna be very hard to to deal with the, it's called the Gish Gallup, you know, where they're they're giving new different, and you say something that says counter to one thing, they say something else, and they're just like jumping around to all these obscure facts. And it's very difficult to to win that debate.

• 55:00Speaker 03

  We've discovered in our research that you can get AI to be quite good at this. And in these experiments, the thing that's unique about them is, unlike other, like, people have done experiments where they tried to debunk conspiracy theories or misinformation. But to do that, you have to guess what people believe. You know what I mean? Like, you're gonna say, I'm gonna I'm gonna debunk the moon landing hoax. You have to kind of make guesses about what piece of evidence people care about in that context. In these studies, what we do is we ask people to write it in their own words. So they can they can enunciate their own conspiracy in their own words. And then we have the AI directly counteract the specific reasons that people put forth for why they believe, and they have a conversation about it with the AI. They know they're talking to an AI.

• 55:45Speaker 03

  And the AI just gives very, very detailed counterarguments. And what you find is that people actually do change their minds. Like, in in one study, for example, the conspiracy theorists had a 20% decrease in their confidence in the belief. Another way to think about it is, so everybody at the start of the experiment believes in the conspiracy. After the conversation, which lasts about eight minutes, 25% of them don't believe it anymore.

• 56:10Sean Carroll?

  Wow.

• 56:11Speaker 03

  And that's eight minutes. I mean, that's like, yeah, that's it. Oh, in psychology, that's, that's as big as you get. Uh, I mean, there's, that's still 75% of people who still believe it, but they, you know, generally go down. They're less confident. Uh, and people use, actually like it, you know, they're like, they're not mad at the AI. The AI gives them information I think is useful, uh, and, uh, evidence matters more than people, we thought it was. I mean, I did not predict that coming in and it, evidence was more powerful than we thought it was.

• 56:38Sean Carroll?

  But the role of knowledge is also really interesting. I mean, on both sides, um, I, I think it was Ezra Klein, when I talked to him on the podcast, somehow we got talking about conspiracy theories and he said, nobody knows more about the tensile strength of steel beams than nine 11 truthers, right? It's not because they have a lack of knowledge. They have way more knowledge than you do because you basically say like, come on, like everyone agrees on the certain thing. I'm not going to spend a lot of time learning about it, whereas they're really into it. So it's, it's absolutely not that they don't know the details. They're just somehow putting the big picture together in a weird way.

• 57:12Speaker 03

  Exactly. I mean, they, they fallen down the wrong rabbit hole. One, one of the things that people, uh, construct in their kind of mental model for a conspiracy theorist is someone who spends too much time thinking. And I think that's true to some extent, like they, this going back to what we talked about before, they are putting together pieces that are like, shouldn't actually go together. And I think there is a version of that. That's the kind of conspiracy theory producer. But there is a lot of conspiracy theory consumers, people who just end up going down YouTube and they're watching another video, a different video. And then suddenly the earth is flat. Um, and those are the ones who are kind of like, kind of globally accepting information. And those are the people who you can have the biggest positive effects on, because if you just give them the alternative information,

• 57:56Speaker 03

  especially in a way that's very comprehensive or even engaging, then by the same mechanism that they went down the rabbit hole, they can come back out of it. And so if it's not just all motivations.

• 58:07Sean Carroll?

  Yeah.

• 58:08Speaker 03

  It's underlying just giving them the right information. And so like that doesn't work for everybody, but it works for a lot of people more than we thought.

• 58:14Sean Carroll?

  Well, it sounds to me like there's two aspects of the AI, the chatbots that are really helpful here. One is the infinite patience, right? Like they're never going to go like, okay, I'm just going to go back to the buffet. I'm tired of talking to you. And the other is this access to lots of specific information, especially I think like the new generation of AIs is much better at pointing to its sources, et cetera. But maybe a third aspect is just the, you know, unflappable cheerfulness. Like the AIs or the chatbots are trained to flatter you and say like, that's a really good question. And things like that. I'm not sure how much that helps with the syndrome.

• 58:50Speaker 03

  Well, we have tried to, we've done experiments where we shut off different valves. And so in one experiment, we did a bunch of things where we made the AI be less polite. Basically, it was, all it was doing was just providing facts and evidence in a kind of like, in a non persuasive way, just like directly just saying, you said this, but actually this is what contradicts that.

• 59:11Speaker 03

  That has more or less the same effect as what we found in the original study. If you, if you get the AI to try to persuade, but you say you can't use any facts, it doesn't work. You know, you take all the.

• 59:23Sean Carroll?

  Oh, wait, sorry, that's very interesting. Let's, let's think about that. So the sweet talk by itself doesn't have any effect. It's actually facts that matter.

• 59:31Speaker 03

  It's the facts that matter. Yeah, you, you take away the facts. They can't, you cannot speak talking somebody into changing their beliefs. It, it might help with getting people to engage. And in these studies, it's important to know that like, you know, we're paying, people are paid to do the study. And so they complete it to, in order to get the money. And so like, maybe in a case where you like want to roll a, a bot to just talk to people on the internet randomly, then being nice and polite would get more people to have the conversation. But when it comes to changing people's minds, it's the facts and evidence that matters.

• 1:00:02Sean Carroll?

  Is there a way to sort of tie this into the unthinkingness idea by imagining that the interaction with the chatbot is sort of nudging them toward thinking more carefully?

• 1:00:12Speaker 03

  I mean, it certainly is in the sense that in order to have the conversation, you have to sort of reflect on both your beliefs and what's being said. In fact, in many studies, what we find is simply going through the exercise of writing out the reasons for why you believe something is enough to kind of decrease how certain you are about it. Now, that's only a pretty small effect. Then, once you give all the people the counter evidence, that's a much bigger effect. But just the engage of reflecting on it is beneficial. So these things kind of go hand in hand.

• 1:00:39Sean Carroll?

  And another part, I guess, that you mentioned in the paper was that, surprisingly or not, people want to talk these things through. They don't want to just hector you. The people who are susceptible to these conspiracies are kind of exactly the ones who want to have a dialogue about it.

• 1:00:57Speaker 03

  Yeah, exactly. I think we improperly mis-maligned the conspiracy theorists. I mean, in many ways, they're very interested in... It's a failure of science communication that they fell down that hole instead of a different one. They could be learning about the Big Bang or quarks or whatever, and they went down the other one. I mean, it's harder to be as interesting if you are constrained by reality. So it's a losing battle, perhaps. But it is the case. I mean, this goes back also to the idea of the motivations driving everything. That general viewpoint on conspiracy theorists is that they are totally fine with

• 1:01:42Speaker 03

  believing falsehoods because they kind of want to, you know? And the one thing that kind of bugs me about that theory is that it's really a theory about the other.

• 1:01:49Sean Carroll?

  Yeah, no one thinks they want that.

• 1:01:52Speaker 03

  Yeah, exactly. But we all have the same kind of... You know, we could all fall down our own rabbit holes. It's just that we're lucky with that the rabbit holes that interest us aren't filled with falsehoods, you know?

• 1:02:03Sean Carroll?

  It's a somewhat optimistic take. I mean, there's pessimistic parts of your story, I think. But people do want to talk. They want to reason. They're susceptible to hearing evidence. Like, there's a lot of good nuggets in here.

• 1:02:18Speaker 03

  There are, you know, and if you look around the world, you might think, really? But I think that there's, I think, I think it comes down more to not successfully getting good information out there in the market. And that's, if we're losing the misinformation award, it's not because of people, it's because of the, the information environment itself.

• 1:02:40Sean Carroll?

  Is there a relationship between the efficacy of talking to chatbots and that overconfidence that we talked about? Like, are the most overconfident people harder to move? Or are they the ones who like have a little epiphany and change their minds?

• 1:02:55Speaker 03

  Yes, although in that case, it's mostly related, the stronger relationship is with how much they value evidence and they're reflective people. So in this case, because overconfidence isn't quite as strong, because everybody is being confronted with these things that contradict their views. And also, there's, there's one kind of other complication with the experiment, which is that you get as much as you put into it, meaning that the more you talk to the AI, the stronger the counter evidence is going to be. And so the overconfident people are a little bit less willing to kind of put in the effort to get good, good counter evidence. But people who are intuitive, they might be like, oh, yeah, this is, these are my thoughts. And then they're like, oh, okay.

• 1:03:40Speaker 03

  I didn't realize that. I didn't realize that the steel beams, yeah, maybe they don't burn at the amount of degrees, but it loses half its carrying capacity and that's enough to collapse a building. It's like, oh, that makes sense, I guess.

• 1:03:50Sean Carroll?

  Let me ask again then about the time scale for this kind of thing. Is there any idea that talking to the chatbots?

• 1:03:58Sean Carroll?

  And, you know, convincing people to move away from the conspiracy theories is still true a month or a year later?

• 1:04:04Speaker 03

  Yeah, exactly. It was one of the key findings was that not only do people change their mind in the context of the experiment, but we, so we, we recontacted them like a month and two months later. And not only is the effect still there, it doesn't even decay in that case. There's like, that is, there's no, they're exactly the same level they were after the conversation. They didn't go back to believing a little bit. They just were like, they, I mean, their minds were changed and they, they didn't think what they thought before the experiment. And you don't ever see that, like, that's.

• 1:04:35Sean Carroll?

  Yeah.

• 1:04:35Speaker 03

  That, yeah, we were like, hold the presses, you know, that was exciting. We've done a lot of other kind of like experiments that are on different topics, but they, they often don't have as much care of us. That one had a pretty big one.

• 1:04:46Sean Carroll?

  This is certainly too much to ask, but we indicated earlier that there was sort of a general tendency toward unthinkingness that made you susceptible to a lot of misinformation. And if you're talked out of a single conspiracy theory by your AI friend, does that at all carry over to your susceptibility to other conspiracy theories?

• 1:05:08Speaker 03

  That also is what we found evidence that that happens too. Like, in the context of the study, what we did is we asked people about a bunch of common conspiracies, you know, like 9/11 is an inside job, whatever, Moonlighting is a hoax. And then they talked about whatever particular conspiracy they wanted to talk about, and there's like huge variability in what that is. And then, and then we remeasured both how much they believed the conspiracy that they talked about, but also all these other conspiracies. And you have some carryover effects. People believe the other conspiracy is a little bit less. Now, it's a much smaller effect, but it's still there. And again, in psychology, you never see what would refer to as like a transfer effect, like where, where people are like, I mean, the conspiracies are not connected to each

• 1:05:50Speaker 03

  other, but it's breeding a little bit more kind of additional skepticism. Now, it's not making them more reflective people in general, like, I don't think that's unlikely. But maybe over time, like, you do like a bunch of these sorts of things on different topics. You're kind of like, maybe I should be doing this myself, you know, I should really be scrutinizing things. And maybe they, they might use AI for that, or just other resources, or whatever.

• 1:06:09Sean Carroll?

  I mean, we have to mention that AI is infamously prone to hallucinating or confabulating, and maybe even giving people very, very bad advice if they talk to it too long. Is that, do you see that effect, or do you have a specific sort of technology that guards against that?

• 1:06:26Speaker 03

  So, we don't see that. We, in this, what we did in this study is, we got, we fact-checked a subset of the claims that the AI was making, and we didn't find any that were false. There was like, that is, we had external fact-checkers, it was like 100 claims, and 99% of them were true, or something like that. And there was one that was maybe somewhat misleading, but it depends on how you look at it. But they, but it was like not entirely true, I guess. So, it was like very, very accurate in this case. And that has to do with The task itself is like kind of perfectly built for this like it's trained on the internet. And what does the internet know more than conspiracies, you know? There are other contexts in which maybe it wouldn't be as effective, like more dense like scientific topics or whatever, but in this case it was right in line with what it was good at.

• 1:07:11Sean Carroll?

  I mean, there's a famous thing, I don't know if you follow, but on Twitter slash X, you can ask Grok, the AI agent, and there's this notorious thing, there's even a subreddit dedicated to people who love their conspiracies, saying, hey, Grok, come in and help me and explain this conspiracy theory is right. And Grok always says, no, actually, it's not right. And they get very mad at it, which I presume is a reflection that all of these AI jetbots are trained to sort of more or less reflect the majority view of things. Is that fair?

• 1:07:40Speaker 03

  Yeah, for sure. Yeah. Yeah. And there's like, you know, they have different versions of levers they pull to make sure that the information is accurate, you know, and all that has to do with treating, having something that tells you what are the good sources, what are the bad sources that it needs to learn.

• 1:07:54Sean Carroll?

  Right.

• 1:07:54Speaker 03

  And even with Grok, like Musk really tried to, has put his thumb on the dial to change, but they have to kind of hard code that in, you know, it's, it's really

• 1:08:03Sean Carroll?

  Right.

• 1:08:04Speaker 03

  A lot of work to try to get it to espouse the views that Musk wants it to espouse. And so it's just, that's, that's just, you know, reality is a constraint. And if you train things with some, some connection to reality, then you're going to have, you're going to face that constraint.

• 1:08:19Sean Carroll?

  Yeah, there was that brief moment when Grok became very, very pro Hitler. And the implication was that you can't get the support for your favorite slightly right wing conspiracy theories without going full Hitler.

• 1:08:33Speaker 03

  Yeah, exactly.

• 1:08:37Sean Carroll?

  But okay, I mean, are there lessons, and this is beyond your, your domain, I guess, but can you think that this suggests lessons for either social media platforms or media more generally, or human beings more generally about how to combat misinformation and conspiracies?

• 1:08:56Speaker 03

  Well, the how to combat question isn't answered that appealingly by this, which is like, what you need is lots of really good information.

• 1:09:07Sean Carroll?

  Yeah.

• 1:09:08Speaker 03

  The, and get, make sure that people engage with it, but, which is, maybe we knew that already. I think that what it combats is the idea that the truth doesn't matter.

• 1:09:17Sean Carroll?

  Right.

• 1:09:18Speaker 03

  That, that we're, and if you take those theories of motivation that we talked about seriously, what those imply is that the truth shouldn't matter, that like our identities, our motivations, our biases, these are the things that drive how we engage with information in the world. And so in order to really get people to be more rational, you have to kind of undermine the motivations somehow, or like undermine people's political identities. And I don't really know, no one's really offering

• 1:09:44Sean Carroll?

  Yeah.

• 1:09:44Speaker 03

  ways to do that. And I don't think it's very likely, under, under this kind of perspective, you know, it is, it is a kind of a battle. We have to get good information out there to overwhelm the bad information. We are in the age of information. And so like, that's where the, that's where the problem is. And that's not easy. And that's like, there's no simple solutions as there ever are. But it at least tells us that the thing that we care about as educators, as people that make science podcasts, there's value in that. I mean, this is, we're, we're doing the thing that we need to do, get the information out there, but it's just, it's not easy.

• 1:10:20Sean Carroll?

  I've always had this belief that I wanted to be true, but I, I'm never quite sure that I'm just not telling myself what I want to think, which is that, you know, if you're just out there in the info sphere, talking to other people, you can get a lot of good information, a lot of bad information, and you can kind of pick and choose, and you can easily fall into a whole system of wrong beliefs. But you also have to interact with the real world, with the external reality, and their only true beliefs are going to help you so that there is a, you know, a bias built into the system for ultimately true beliefs to prevail. I would like that to be true. I'm not sure if it is, but, but maybe you're giving me a little glimmer of hope here.

• 1:10:59Speaker 03

  I would like for that to be true, too. I think that the we evolved such that that is generally the case, but I do fear that there are contexts in which power may overwhelm the truth. And we know that historically is the case, that if you propaganda does work, and you know, this is one of the things that actually you can overwhelm. We don't have divine access to the truth.

• 1:11:22Sean Carroll?

  Yeah.

• 1:11:22Speaker 03

  If you if you can, people can only know what their information that they're exposed to. So you have enough control over that, then you can influence in a very strict way what people believe. And that's a problem, but one that we could undermine if we have some control over what good information people are exposed to as well.

• 1:11:40Sean Carroll?

  Well, I guess, yeah. And so this is a good sort of final thought kind of question. You know, the birth of the Internet was, of course, accompanied by all sorts of optimism that we're going to be sharing information and everything is going to be low effort, you know, no more gatekeepers, things like that. Of course, what we have seen is, you know, the ability to immerse oneself in a particular subfield of false information. And maybe what your research suggests is the truth can fight back in certain ways. Because the people, you know, they're not trying to be wrong, like the people want to be right. People want to think things through. People want to talk about it. These are all

• 1:12:23Sean Carroll?

  positive messages, and maybe the next technological step helps them find the truth, which they want to find after all.

• 1:12:30Speaker 03

  I think that I think that's a great way to summarize it. I mean, if you think about the media landscape, the people kind of think back of like, oh, how nice it was when there was like, you know, the three stations with the nightly news and everyone shared the same kind of reality. And then we have now you have choice and you have all these different things. We now people are living in different realities. And there's no going back to that.

• 1:12:54Sean Carroll?

  Nope.

• 1:12:54Speaker 03

  So, given that we are in this reality where there's information is diffuse, that means the work that you have to put into, you cannot assume that you have unique access to people's attention. And so, we need to understand that. And I think scientists have taken kind of too long to catch up and we need to get there.

• 1:13:12Sean Carroll?

  We need to get there. I like that thought. Gordon Pennycook, thanks so much for being on the Mindscape Podcast.

• 1:13:19Speaker 03

  My pleasure.

• 1:13:45Speaker 06

  Monday night brings one more big matchup to settle into. And when every play matters, the last thing you want to do is step away to figure out dinner. That's where DiGiorno comes in. Get a DiGiorno pizza ready before kickoff, then enjoy restaurant-quality taste fresh from your own oven while you stay locked into every play. Stock up on DiGiorno over the weekend, so when Monday night rolls around, the only thing you sweat is the game. Shop now at the link in the episode description. It's not delivery, it's DiGiorno.

• 1:14:12Speaker 01

  OnDeck is built to back small businesses like yours. Whether you're buying equipment, expanding your team, or bridging cash flow gaps, OnDeck's loans up to $400,000 help make it happen fast. Rated A+ by the Better Business Bureau and earning thousands of five-star TrustPilot reviews, OnDeck delivers funding you can count on. Apply in minutes at ondeck.com. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 1:14:42Speaker 04

  Rain outside your house? Relaxing. Rain inside your house? Less relaxing. That's the sound of a bad day. That's when you want to hear something reassuring. Cincinnati Insurance works with local independent agents who get to know you and bring personal attention to protecting your home and most valuable possessions. So, when a bad day sounds like this, you have 75 years of expertise helping make it better. Find an independent agent at C-I-N-F-I-N.com.