Gary Klein: Insights For Making Better Decisions

My guest today is celebrated research psychologist Gary Klein who has spent nearly 50 years studying how and why people make the decisions they do. We discuss naturalistic decision making, the crucial difference between experience and expertise, why some people stagnate and others progress, Cognitive Flexibility Theory, the role of storytelling, surprising ways to gain insights, fixation errors, cognitive biases, mental models, accelerated learning, and so much more. -- Want even more? Members get early access, hand-edited transcripts, member-only episodes, and so much more. Learn more here: https://fs.blog/membership/ Every Sunday our Brain Food newsletter shares timeless insights and ideas that you can use at work and home. Add it to your inbox: https://fs.blog/newsletter/ Follow Shane on Twitter at: https://twitter.com/ShaneAParrish Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  One of the things I've examined is, how do you know who's an expert? And I identified about seven or eight criteria. One of the criteria I use if I'm going to give somebody's expertise is I'll ask them, "Tell me the last mistake you made. Let's talk about that." And if the person says, "I can't think of any mistakes," to me, that means this person may be competent, but it certainly is not an expert. Experts are well aware of their mistakes, and their mistakes eat at them. But people who are journeyman, many of them stay as journeyman because they want to move on and forget about their mistakes.

• 0:49Shane Parrish?

  Welcome to the Knowledge Project Podcast. I'm your host, Shane Parrish. The goal of this show is to master the best of what other people have already figured out. To that end, I sit down with people at the top of their game to uncover what they've learned along the way. Every episode is packed with timeless ideas and insights that you can use in life and business. If you're listening to this, you're missing out. If you'd like special member-only episodes, access before anyone else, transcripts, and other member-only content, you can join us at fs.blog/membership. Check out the show notes for a link. Today, I'm speaking with Gary Klein.

• 1:31Shane Parrish?

  Gary is a research psychologist famous for pioneering the field of naturalistic decision-making. I've wanted to geek out with Gary on decision-making for years, and this conversation did not disappoint. We talk about why some people stagnate at an intermediate skill level, and some people reach mastery, cognitive flexibility theory, the role of stories, both the ones we tell ourselves and the ones we tell others, the two ways to make better decisions, gaining better insights and reducing errors, and how we can improve at both of them, pre-mortems, shadow boxing, and how it helps you learn fixation errors, cognitive biases, mental models, and how we use them, and fast-tracking expertise. It's time to listen and learn.

• 2:23Shane Parrish?

  So you're a cognitive psychologist who has spent your entire career observing how other people make decisions. But you took an interesting angle to this. Rather than focus on how to make better decisions directly, you approached it from how we can develop expertise to make better decisions. We can sort of reduce errors or have better insight, or preferably both. And yet, these often seem in conflict with one another. I thought a good place to start this would be what sparks insight and what prevents us from putting our insights into use? Is it because they often contradict the beliefs we hold?

• 2:57Speaker 00

  An insight definitely is going to be incompatible with beliefs we hold. That's why we're surprised. I did a study of 120 examples of insights. I wondered where insights came from. And I did this study out of frustration because I would give talks and I would say, if you want to improve performance, there's two things you can do. And there would be like a down arrow, what you want to reduce, and an up arrow, what you want to increase. And to improve performance, what you want to reduce is errors. And that's what most organizations do. They try to cut down on errors. But they're missing the up arrow of improving, increasing insights.

• 3:41Speaker 00

  And so, most organizations only focus on reducing errors. So I would present this slide and people in the audience would say, yes, that makes sense. That's my organization. All they care about is reducing errors. But then they would ask me the question, so what do you know about insights? And I would answer, I don't know anything about insights. I just have this one slide. I haven't studied it. And once I gave a, I used a slide in a talk I gave in Singapore, and I got asked that question, and then I flew home afterwards. And it's a 17-hour flight. I had a direct flight. That's a long time to be stewing over the fact that I felt like an idiot not being able to tell them what insights are about.

• 4:25Speaker 00

  So I decided to do a study. So I collected 120 examples of insights to see what's going on and hopefully find out how to improve them. Increase them. And I tried to find a common theme and I couldn't. Because it turned out that there were three different pathways. One pathway is a connection pathway. Where we put things together. So there, we're not contradicting anything. I'm going back to your original question. We're not upsetting our mental model. We're just putting different ideas together. Darwin, you know, knowing that there was a change in species, but what's driving the change?

• 5:05Speaker 00

  And then he reads Malthus's book about population growth and competition for scarce resources. He read the sixth edition. And he puts that together and he said, that's what's driving evolution. Is a competition for scarce resources. So that's a connection type of insight. A second type of insight is a contradiction insight. Where something happens that doesn't make sense. And there you do have to change what you believe or wonder what's going on. And the example I used there is a story I heard from a police officer. He was driving around with a partner who is in his first year. And they're stuck in traffic.

• 5:50Speaker 00

  There's a red light. And the partner, this young guy, looks at the car ahead, which is a new BMW. And he sees the driver take a deep drag on his cigarette and then flick the ashes. And he says, who flicks the ashes in a brand new BMW? That doesn't make sense. Something is off here. So they light him up and pull the car over. Sure enough, it was a stolen car. So that's a second type of insight, which is a contradiction insight. Where something happens that you didn't expect.

• 6:31Speaker 00

  Now, this didn't force anybody to revise their mental model or their thinking. It just allowed them to investigate further. Now, your question is, where's the resistance coming from? And that's the third type of pathway that we identified. The correction pathway. Where you're stuck because you have a flawed belief.

• 6:54Speaker 00

  And that's the problem, and that's what's hanging you up. And that's the kind of insight most researchers study because it's easy to study in a laboratory. And the way we get, we get past that flawed belief. We found, I found about 27 cases of that kind of a pathway. And usually what happened was there was a hint, something happened that made them investigate further. And so instead of dismissing the anomaly, which is what we usually do, they became curious about it. So in terms of insight, in terms of changing our beliefs, that's the secret sauce for this kind of pathway.

• 7:39Speaker 00

  Or even practically for any of the pathways, is to become curious about things that don't make sense. Now, what blocks insight in organizations? And this was the really discouraging part of my research on insight. What blocks insight is that organizations say they want insights and they want innovation. And they really don't. But they're, they're being serious. They think they want insights. But insights are disorganizing. As you point out, insights make you change the way you think, make you change all kinds of things. And, you know, they may not be right.

• 8:23Speaker 00

  And so most organizations actually inhibit insights. When, when, when people bring up new ideas, they, they tend to stifle them. And so that's, that's a big problem.

• 8:36Shane Parrish?

  Is that because organizations are mostly focused on the error reducing side versus the gaining insight side? And that, that's the tension between these things? They don't like variance? They don't like things that are outside of the norm?

• 8:50Speaker 00

  Yes, they like predictability. To be a good manager, you want things to run smoothly. And insights are not ways of running smoothly. Insights are disorganizing and disruptive. And so that's, that's a major reason that organizations, without even intending to, block the insights that come their way.

• 9:12Shane Parrish?

  So with that said, on an individual level, if I'm working in an organization, what can I do within the confines of only what I control, so myself, to maximize the insights that I observe and that I can put them into use within that environment? And then I guess the second question would sort of be, what can an organization do to facilitate people gaining more insight and putting those insights into use?

• 9:41Speaker 00

  You can be deliberate, like you're describing now. You can say, I want to have more insights, which means taking an insight stance rather than a stance that recoils at something new and unexpected and jarring. You can become curious. You can start to celebrate your own insights. Because if you go through the day, we often have very small insights, things we didn't know, and we make these discoveries. And we just sort of, you know, cruise right past them and use them. But if we make a mistake, we beat ourselves up, right? At least, I know I beat myself up. Like, how could I have been so stupid? But I'm less likely to say, gee, that was, that was really clever,

• 10:24Speaker 00

  of me or of people that I'm working with. So you can, you can start attuning yourself to the insights you have and the insights people around you have, so you're more sensitive to that. You can, you can't be curious about everything, because then you just waste tons of time. But you can be a little curious for a few seconds and say, I wonder what that means? And think about it and imagine it. And just get yourself into that kind of a mindset of wondering and speculating, rather than recoiling at something that's unfamiliar. So that's what you could do as an individual.

• 11:02Speaker 02

  At Edward Jones, we believe rich is about taking care of what gives your life meaning. That's why your financial advisor personalizes your plan to help you preserve your progress and create something that lasts. Let's find your rich. Edward Jones. Member SIPC.

• 11:20Speaker 03

  It's me, the

• 11:23Speaker 03

  We don't need the jingle twice.

• 11:26Speaker 03

  Stop it. This is about vacation inflation and how Priceline negotiates amazing deals on hotels, flights, and rental cars.

• 11:35Speaker 03

  Yeah, but I didn't mention that you can save up to 60% off hotels in the Priceline app.

• 11:41Speaker 03

  Fine. No one deals more deals than Please stop. Touche. Priceline.

• 11:48Speaker 00

  What can you do as an organization? They are less enthusiastic. If you have an idea that's a bad idea and you make a mistake, that's the down arrow. Everybody can see it, right? It's public. So there's a lot of cost for making a mistake. If you fail to make an insight, nobody will know.

• 12:11Speaker 00

  So the reward, the reward structure is always favors the down arrow. Organizations, if they're really serious about this,

• 12:23Speaker 00

  can try to create a mechanism so that, right now, if I have an idea, I bring it to my boss, I bring it to you. And you bring it to your boss and it goes up the chain. It only takes one person in that chain to say, now we better not do it. And then the idea is rejected. And good ideas are fragile and precious and, and, and easily discarded. So they, um, so they could give you, or they could give me an option for a review if I think the idea has been rejected prematurely. I can, not really, I don't want to make trouble, but there, there could be somebody, there could be a,

• 13:08Speaker 00

  some organi, some part of the organization that could reexamine, reexamine the idea.

• 13:15Speaker 00

  Because, but let's take an organization that wants to come up with new ideas for whatever it is that they're doing. Who's running the organization? People with lots of experience. And I found in my research, that experience is essential for for coming up with insights. However, people with with experience have lots of scar tissue of things that that got tried before and failed. And if you look at innovations, usually there's a bad track record. People are trying it and failing, and then somebody else trying it and failing, until eventually it succeeds. But until you get to the point where it succeeds, you're going to have people in the experienced

• 13:57Speaker 00

  people, the senior people in the organization, what are they going to say? They're going to say, you know, we tried that and it didn't work. This may be the time for it to work. Maybe the technology is mature. Maybe, you know, the the world situation has changed. Who knows? But the people with who are calling the shots can remember times when this was tried in the past and it failed. And often, that's a good enough reason to squelch the idea.

• 14:26Shane Parrish?

  I think that's a really interesting point, especially when it comes to experience, because if the experience is typically the people at the the top of the organization have the most experience, and they probably worked in a job very similar to the one that you're working in now. The problem is their experience in that role is 20 years ago or 15 years ago, and I would imagine that the environment has changed a lot. Whereas the experience of your manager, or your team leader, or whatever might be more recent, and they would have a more accurate view of to what the the lay of the land is right now.

• 15:03Speaker 00

  Right, so so experience, I I I still found that most of the insights in in the sample that I studied would not have arisen unless people had experience. I mean, Darwin didn't didn't just sit in the in his study and imagine, he was on the beagle. He was who was watching these various species. He was looking at the variation. He was building his experience base. So experience is important, but.

• 15:32Speaker 00

  There are other things, there are other factors that are operating. And experience is not infallible here. And the people who are calling the shots are risk averse. And that's another issue in organizations. The people at the top who are calling the shots don't want to rock the boat. Because they're getting ready to retire, they're getting ready to draw attention. My sense, I don't have data on this, is they tend to be risk averse. They want to keep things going smoothly until they reach the end of their career at the organization.

• 16:12Shane Parrish?

  I want to come back to something you said about experience. Is there a difference between experience and expertise?

• 16:18Speaker 00

  Right, there is a difference. And so, one of the things I've examined is, how do you know who's an expert? And I identified about seven or eight criteria. And the discouraging part is, none of them are foolproof. There is no gold standard for who's an expert. But years of experience certainly contribute to expertise. But it isn't the same thing because there are people who just don't reflect on what happened. They don't learn about what happened. One of the other criteria I use if I'm going to find, gave somebody's expertise,

• 17:00Speaker 00

  is I'll ask them, "Tell me the last mistake you made, let's talk about that." And if the person says, "I can't think of any mistakes." To me, that means, this person may be competent, but it certainly is not an expert. Experts are well aware of their mistakes. And their mistakes eat at them, until they can sort of imagine, "What I should have done is this," and figure out a way around it. So, experts are highly aware of mistakes. But people who are journeyman, many of them stay as journeyman because they want to move on and forget about their mistakes.

• 17:40Shane Parrish?

  How do we identify the difference between people who know what they're talking about and not? And to this point, one of the most important decisions I think that we make... And we don't even recognize that we're making a decision is who we listen to and who we decide is credible and who we decide is not credible. And we, we don't recognize that we're making a decision in that moment, but how do we sort out who is credible and who's not, or who's, who knows what they're talking about or who's faking it?

• 18:08Speaker 00

  Right, that's, that's a really important question. And I think most people do a mediocre job of determining who they, who could, who they can rely on. And a lot of people go on surface characteristics. So if I carry myself, I stand up straight from the usual, and I carry myself with confidence, people are going to listen to me. And if, if I express reservations and I use qualifiers, I think this is the case, but I'm not sure, I'm essentially telling people I'm not entirely an expert here. And so I'm disqualifying myself. I'll give you an example.

• 18:52Speaker 00

  A friend of mine was excited about a scenario-based technique for, for, for training, of using decision-making exercises. And that's a precursor for our, the shadow box method that we're using now. And so he had some exercises that, that, that he had developed for, he worked in a petrochemical plant. And he had about, oh, 30 years of experience there. And he said, let's, you know, this would be great. Let's use this for, for people who are just, have been working outside, checking valves and climbing ladders and things like that.

• 19:35Speaker 00

  Now they're inside working the panels, so they, they've been promoted. And there were two people who had just been promoted. And he said, let's get these people up to speed more quickly and let's have them do these, these decision-making exercises. And one of the people was somebody everybody felt good about. He just carried himself with great confidence and, and, and people were, were excited about, about working with him. And the other one was, was sort of... tentative about things and they they worried about him and they ran them through the decision making exercise and what did they find? The first person didn't really get it.

• 20:18Speaker 00

  He didn't really understand critical dynamics of the plant. And the second person, for all of his caveats, really did get it. He understood what the relationships were, the different causal factors and how they worked together. Now, the story has a bad ending because they decided, the first person, despite all their enthusiasm for him, he wasn't ready. He wasn't going to be safe operating a panel of a petrochemical plant that could blow up. I mean, it is a volatile compounds and lots of heat and it's a dangerous environment. They said, he's not ready, he's going to have to continue outside for another year

• 21:02Speaker 00

  before he can come and we can use him on the panel. And the second person was somebody that they were ready to use. And after they did that, nobody wanted to engage with the decision making exercise. They realized if you don't do well, you can have consequences. And they weren't intending the consequences to be used to evaluate people, but because they used it in that way, nobody wanted to do any more of that training.

• 21:32Shane Parrish?

  That's a, that sort of like begs the question, I guess, how do we evaluate people's decision making?

• 21:37Speaker 00

  How do you evaluate their decision making? You run them through a scenario, see what choices they make, find out what their rationale is, find out the reasons that they're picking one thing over another, and use that to determine if you think that their mental model is rich enough. Another, I told you, is ask people, tell me about the last mistake you made. Or you can, you can, you should use years of experience, just don't take it all that seriously. So there's a variety of criteria that you can use. You can use their track record, have they had, have they had a track record of successes? But none of these are foolproof.

• 22:21Speaker 00

  I may have, I may be an investor and I have a track record of calling the market right the last five times. And people say, wow, he really knows what he's talking about. But there are lots of people who are trying to predict which way the market is going to go. And some of them are going to get it right by chance. And people assume it's because they know what they're, what they're doing. And in fact, they're just the lucky ones. So you can't rely on performance and you can't rely on years of experience.

• 22:54Shane Parrish?

  To your point about explaining rationale and sort of why you made a decision and the variables that you consider, maybe how those variables interact over time. I, I think organizations partly get in the way of this because they might write a summary of why they're doing something, but it's so high level, it doesn't actually contain much thought or reflection in, in the thinking, which also prevents people from learning from other people, how they're making decisions and developing expertise.

• 23:22Speaker 00

  You're, you're exactly right. So what you're, you're, uh, what you're saying is that we, we need to go beyond a simple surface explanation of why people did things. What my colleagues and I talk about as cognitive interviews, to get beyond the surface explanation to find out, how were you sizing this situation up? What were you noticing? And, you know, what, what, uh, inferences were you drawing? And we can, we can be asking people those kinds of questions to get into their head more deeply, rather than, I figured the market was due for a correction or something like that. I've thought about an idea called a decision scorecard for an employee.

• 24:07Speaker 00

  I sit down with the employee and I, I actually, I've used it several times and I say, let's go back in a previous year. What were some of the major decisions you made?

• 24:19Speaker 00

  And I come to the meeting with what I think were the decisions. The employee comes with their decisions, then we compare notes. And then we look at the decisions and say, which ones worked and which ones didn't. Now, some decisions may have been good decisions that didn't work through no fault of the employee. And some may have been bad decisions that worked because the employee got lucky. So you can't just look at the outcome, but you have to look at what was the person thinking about when they made the decision? And what can we learn whether it was a success or a failure?

• 24:59Speaker 00

  And my experience is that this decision scorecard method of evaluating people is much less stressful. And I enjoy it because we're all learning a lot as we go through it.

• 25:16Shane Parrish?

  Evaluating those decisions requires you to have knowledge of what you knew at the time, what you were thinking at the time, not retrospectively going back and trying to piece it together because now you have new information. And we've written a lot at Fernham Street on the concept of decision journals. I'm curious as to what information, if you were going to create a journal that an employee had to fill out every time they made a decision, what information would you put in that decision journal to then use at these meetings where you're evaluating their decisions later on?

• 25:48Speaker 00

  I think the idea of a decision journal is a great idea, and I hadn't heard of that before. I would like to know when the employee is making a decision. What is the decision? What are the goals that the employee wants to achieve? The primary goal, but there may be other goals that the employee is aware of. What are the prime pieces of information the employee is using to make the decision? Who are the other people or teams that are going to be affected by this decision? Those are the things that I'd like to examine. And then in retrospect, we can see were there cues that I should have been paying attention to that I wasn't.

• 26:37Speaker 00

  Are there goals that I should have been thinking about that I wasn't? But you have it in the decision journal. You have a record of what was the person thinking about at the time. I think it's a great idea.

• 26:49Shane Parrish?

  And for for those of you listening, if you go to fs.blog/dj for decision journal DJ, you can you can get access to the template that we have online. Gary, I'll send that to you after. I also I also think it's really important that people write it down in their handwriting and not on a computer because it's so easy to look at things on a computer and convince ourselves that we didn't write that. And you're going to try to, you know, somebody else wrote that. I was clearly thinking way better than, you know, what it looks like. But when you when you're confronted with your own handwriting and your own thinking, it does two things. One, one, when you write it down, it's actually a bit of reflection. So you're taking this complex decision and you're distilling it. You're reflecting on it.

• 27:34Shane Parrish?

  And then you often realize through writing it that you don't understand it as well as you thought, or you know where to go look for a different piece of information to more effectively complete your understanding of it or offer more value. And the other benefit to doing it is that you can look back in the past and you can sort of like figure out patterns in your decisions.

• 27:56Speaker 00

  Right. Both of those are are important issues. The act of journaling is is a way of helping you inquire better and be curious about things that you hadn't thought about. Yes, all of those are all of the above. I I would like to see the decision journal when we're finished here.

• 28:15Shane Parrish?

  Awesome. I think we're actually going to publish one that people can buy on Amazon and write at some point this year. Maybe I'll get you to look that over before we we do that. I would love your your insight into that.

• 28:26Speaker 00

  Happy to do it.

• 28:27Shane Parrish?

  One, one of the things, we're we're down a few rabbit holes here and I want to come out of this, but you use the term mental models. What do you mean when you use the term mental model? What is that to you?

• 28:36Speaker 00

  This could be another rabbit hole, but we've a few years ago, my colleague Joseph Borders and Ron Visayan and I did a study of metal models in petrochemical plants. And going into this study, I thought that a mental model was a set of beliefs about how something works, whether it's a piece of equipment, a piece of machinery, it's an organization. Here's how it works. Here's what happens. Here's the kinds of components and how they fit together. So that was my belief going in. And we spent a week at a petrochemical plant, running people through a challenging scenario to capture their mental model.

• 29:19Speaker 00

  And that was part of their mental model, but only part. We found that there was another part of the mental model, not just how does it work, but what are its limitations? What are its boundary conditions? Where can it go wrong? And people who are experienced are well aware of these boundary conditions. And so that's part of the richness of their mental model. And then we found another component, which is, what are the workarounds that I can use if we run into one of those limitations? How can I recover from it? And then we found that there was a fourth component. We have actually a matrix of this, a two by two matrix.

• 30:02Speaker 00

  The last component is somebody with a good mental model can also anticipate where people might get confused. What kinds of mistakes they might make? What kinds of flawed beliefs they might have, so that you need to be more careful as you describe what they're supposed to do. So my idea of a mental model started out with how something works. And now it includes all four of these components. How it works, what its limitations are, how to work around them, and how to anticipate confusion on the part of people who I'm going to be working with.

• 30:42Shane Parrish?

  Do you think that relates to the learning process? So I have a, I think people learn with a learning loop and you have an experience, you reflect on the experience. That reflection gives you an abstraction, and the abstraction turns into action. And so you have this loop of learning. When you learn a concept from somebody else, you're learning their abstraction.

• 31:08Shane Parrish?

  So you're putting an abstraction into use as your action. And if you take the time to reflect, well, reflection is how we learn those edge cases, how we encode it in our mind where it's more likely to work in this scenario and not. And so in the first case, when you're learning from somebody else, I think you have like pattern matching, but it's exact pattern matching. It's like searching Google with quotes. Whereas in the second case, you're doing pattern matching, but it's associative pattern matching. What do you think about that?

• 31:39Speaker 00

  That feels reasonable. I think to do that kind of learning, you're sort of exposing a problem where I get your abstraction,

• 31:51Speaker 00

  but I haven't lived your experience. So your abstraction, I might draw the wrong lesson from your abstraction. I don't know where it's coming from. And I could ask you to explain it. And language is a means for communication. But it's also a means for miscommunication. Because there's all kinds of ambiguity in the words that we use. And people use the same words and mean different things and then believe that they mean the same thing. And can get themselves in trouble that way. So that's a limitation of having you explain your abstraction.

• 32:33Speaker 00

  A way around that limitation is to ask you about stories.

• 32:40Speaker 00

  Where, you know, where did you come up with this abstraction? What happened? Tell me what went on that changed your mind. And then I can ask you about that event. So you were in this situation and something happened that surprised you. What happened?

• 32:58Speaker 00

  And why were you surprised? And how did you notice it? And what sense did you make of it? Stories are not hampered by the limitations of language because now we have an incident account that we can drive. And many organizations don't take advantage of stories. And we've just finished a project for a petrochemical group about using stories to try to get at these kinds of expertise and communicate them.

• 33:31Shane Parrish?

  Can you expand more on what role stories play? Are they just compressions of ideas? Are they sort of highlighting certain variables or details or omitting certain variables or details or getting us to anchor to something?

• 33:46Speaker 00

  Stories are doing all of those things. And they're doing it in a way that's really engaging. Like, here's what happened, and I didn't know what to do about it. And then you want to know, what did I do about it? How did it end? And I know when people have recounted stories in a couple of groups that we studied when we were doing this project, we'd say, okay, so what's the takeaway from the story? And everybody had a different takeaway. So that's the richness of stories, is what have you learned from the story? What are the implications?

• 34:29Speaker 00

  And then we, as a small team, just compare notes and realize that because we have different mental models, we have different interpretations of the story and the importance of the story.

• 34:42Shane Parrish?

  What makes for an effective story?

• 34:45Speaker 00

  There's got to be some sort of mystery about something happened that wasn't expected. And then the people listening to the story want to know how it wound up. Second is the storyteller. Should be somebody who likes to tell stories. And I've encountered some people who don't like to tell stories. And I'll ask them about stories and they say, I can't remember any stories. So either they're avoiding the issue or their brain isn't wired to be able to access stories. Or some people think they're telling a story when they just give me a narrative.

• 35:28Speaker 00

  This happened and that happened and we go through what happened, but there's no story. It doesn't move, it doesn't evolve, there's no transformation. And the good stories result in an insight.

• 35:41Shane Parrish?

  That reminded me when you were saying that about the cop in the ashes, right? So he's looking for the contradiction. So he's got a story in his mind in a way, but the world works. And we all do when we're driving, because we're not consciously paying attention to the road. But we notice anomalies. So we notice something that is unexpected. And in a vehicle, that's our cue to pay attention. As a driver of a vehicle, for whatever reason, we get encoded with this after years of driving. It's like, okay, now I'm switched on, maybe my heart races a little bit, but like I react. I get out of my trance and my body awakens. It's like I'm in low power mode and then I wake up.

• 36:23Shane Parrish?

  But at work, we don't do this. We get woken up and we have a surprise or a contradiction. And then we ignore it, or we explain it away, or we move on to the next task. Instead of going from low power mode to sort of like full power mode, what's going on here? Because a surprise would indicate the world is not working the way that I thought that it should work.

• 36:43Speaker 00

  So what you want to have is a way to harness the power of curiosity when something didn't happen the way it was supposed to. And our natural tendency is to dismiss it, to ignore it, and to explain it away. And you think, well, why can't people think like scientists? Because scientists are careful. But the researchers who investigated this find that scientists do the same thing. They have something that's called knowledge shields, the way all of us do. That a variety of techniques for explaining away inconvenient data and anomalies.

• 37:18Shane Parrish?

  So, so we have a story about how the world works. And then we notice an anomaly. And rather than try to fit that anomaly in or explore the anomaly, we, we, it's easier. It's a low power mode for us just to continue with our story, the way that we're, we're telling it to ourselves. And ignore or dismiss the anomaly.

• 37:39Speaker 00

  That's why we, we use these knowledge shields and scientists use the knowledge shield. Scientists don't want to have anything challenge their theories. And so you bring up a, a, a data point that's going to make my, my, my, my pet theory look a little bit sketchy. And I'm going to say, well, how did, how did you collect those data? Or, um, I'm not really sure that you've interpreted the data correctly. I'm going to find all kinds of ways of dismissing it, yes. It leads to fixation. Because one of our strengths when we encounter a situation is we, we quickly make sense of it. We, we use the patterns that we've built up.

• 38:22Speaker 00

  And it usually works, but it doesn't always work. And so some people say, we should discourage people from coming up with immediate reactions. But that's ridiculous because that's not the way we think and it would cripple us. So instead you want us to come up with a, a quick reaction. But if something, if, if we're wrong, there's going to be an anomaly and we want to be able to, to revisit it. So we, we want people, that's the way we break out of fixation is we notice the anomalies. The way we get stuck in fixation and making fixation errors is we explain away the anomalies.

• 39:04Speaker 00

  Hold on to the original wrong impression until it's far too late. Now for your decision journal. One of the things that you could capture is, what do you think is going on in the situation now? You know, describe what's going on. And then as time occurs, maybe people can jot down

• 39:30Speaker 00

  data that they receive that are inconsistent with that belief. So you can sort of track, when is it that you first started to lose faith in your initial impression? Could you have questioned it earlier? Did you wait longer than you needed to?

• 39:49Speaker 02

  At Edward Jones, we believe rich is more than caring about the latest and greatest. It's also taking care of what gives your life meaning. That's why your dedicated financial advisor meets you where you are with personalized financial strategies that help protect what matters. So you can preserve your progress while creating a path forward. The key to being rich is knowing what counts. Let's find your rich together. Edward Jones. Member SIPC.

• 40:21Speaker 03

  It's me, the

• 40:24Speaker 03

  we don't need the jingle twice.

• 40:27Speaker 03

  Stop it. This is about vacation inflation and how Priceline negotiates amazing deals on hotels, flights and rental cars.

• 40:36Speaker 03

  Yeah, but I didn't mention that you can save up to 60% off hotels in the Priceline app.

• 40:42Speaker 03

  Fine. No one deals more deals than

• 40:45Speaker 03

  please stop. Touche.

• 40:49Shane Parrish?

  I want to switch gears just a little bit. How do you think about the difference between people that stagnate at, say, an intermediate skill level and people who reach a mastery of the same skill?

• 41:00Speaker 00

  Right. So this gets into the issue of expertise and how do people get to the higher levels of expertise? And some models of expertise say that people just acquire more and more information and that's how expertise builds. But that doesn't work because that's like having unexamined experiences. And you're not coming up with a richer mental model. My speculation, working with a colleague, Holly Baxter, is that people get up to a certain level of performance and then they start to stagnate. And they start to plateau.

• 41:43Speaker 00

  And people who break through and move to the next level are the ones who engage in unlearning, who realize there are certain conventions they've bought into or beliefs they hold that are either wrong or are limited and don't apply as broadly as they imagine.

• 42:06Shane Parrish?

  Is there any sort of practical things that you've seen work particularly well in the wild for people developing mastery in less time, say, than people would expect?

• 42:19Speaker 00

  So let's imagine two people starting out a job. We'll say person A and person, worker A and worker B. They're both going to be on a job for a year. Worker A has a lucky year. And everything goes smoothly. And there's no disruptions, no problems. It's just a really very, very, you know, low-stress year. Worker B has had a turbulent year. All kinds of things have gone wrong. All kinds of adaptations have been necessary. There were, you know, mistakes made that had to be identified, diagnosed, and recovered.

• 43:00Speaker 00

  And one thing is like the whole year has been filled with all of this. Who is the one that you would like to have working with you after that first year?

• 43:10Shane Parrish?

  Oh, I would choose person B because the surface area of their learning is so much richer and detailed.

• 43:17Speaker 00

  Exactly. So worker, the second worker with a turbulent year has learned much more. What can you do for worker A who's had this smooth year? You can take some of these crises that worker, that the second worker had, and encapsulate them as stories or as scenarios. And you can present them to worker A just to give them vicarious experiences, to get them to move forward. So that's something that you can do. And we've seen organizations doing some things like that.

• 43:58Speaker 00

  And that's actually one of our decision training methods is to provide those vicarious experiences.

• 44:04Shane Parrish?

  That's really interesting when you think about organizations as sort of like trying to reduce the variability or reduce errors. And then, at the same time, acknowledging that errors create surface area for reflection, experience, expertise, and learning.

• 44:22Speaker 00

  Yeah, but where people spoil it is, is they then take it one step too far and say, the more errors you make, the more you're going to learn which is true. And so we want to encourage people to make errors. And they don't mean that. Organizations don't want to encourage people to make errors, especially some of the industries that I work in, where errors can be very dangerous. And the fact is, I don't like to make mistakes. If I do something, if I put on a workshop that has gone poorly, my immediate reaction is, I never want to put another workshop on again. This is just too painful.

• 45:02Speaker 00

  And it takes me a few days to get out of that emotional funk and to think about it and say, you know, if I had done it this way, I could have used that problem and it could have been much better. And so by the end of that kind of rumination, that kind of reflection, my reaction is, I can't wait to do another workshop, because now, now I want to try that again. So, but, but the immediate reaction is one of being pretty devastated by, by a mistake. And, and so it's going to be a hard case if you tell me, Gary, you know, try to, try to make mistakes, you know, that's how you're going to learn. It's not in my emotional makeup.

• 45:47Speaker 00

  Maybe it's in your shame, but it's not in my emotional makeup.

• 45:50Shane Parrish?

  Oh, I don't know if I intentionally make them, but I certainly have more than my statistical fair share of them. What is cognitive flexibility theory?

• 45:57Speaker 00

  Cognitive flexibility theory is the, the notion of trying to help people achieve expertise.

• 46:08Speaker 00

  By preventing them from locking in to routines and standard ways of doing things. So that they can become more naturally adaptive. You can give them vicarious experiences that can't be handled by the usual routines. Just to force them out of their comfort zone. And to get them into a mindset of not only being prepared to be adaptive. But enjoying being adaptive. Because that's another thing that I've noticed about experts. If they're doing the same job over and over again, they get bored. And then something happens and they can't use the same routines.

• 46:52Speaker 00

  And the journeyman, they get really frustrated like that. It's not supposed to happen. I don't know, who do I call? And the experts, their eyes light up. Okay, we can't use the techniques that have been successful for us up to now. What can we do? What can we invent? And so the eyes lighting up is like, for me, a key to cognitive flexibility theory. Is to get people into that mode where they're excited when things don't go as planned. And they're gonna have to improvise.

• 47:29Shane Parrish?

  Coming back to the beginning, we've talked about sort of a lot about gaining better insights. Let's talk about some of the tools that we can use to reduce errors. Maybe we can start with a pre-mortem. What is it? Why is it useful? How do we conduct one?

• 47:43Speaker 00

  I just sort of invented it for a company that my company I was running back in the late 1980s. And we've mostly had successful projects, but not always. Sometimes our projects would fail. And then we would do an after-action review. We'd say, what went wrong? And we'd realize, oh, if we'd just done this differently. Um, it might not have ended so, so badly. And I said, why don't we, we always have a kickoff meeting when a project starts. Why don't we move that after-action review? To the front, like a postmortem.

• 48:23Speaker 00

  A postmortem is something that you do like in a hospital when a patient has died. And a physician does a postmortem to find out why did the patient die? And the advantage of doing a postmortem and discovering what was the real cause of death or the causes of death is that the physician gets smarter. Now has a richer mental model, so it helps the physician. And now the physician can tell the family. Here's why your loved one died. Because they're, they really want to know. And now you can, you can help reduce their pain by giving them an answer. They can work with that.

• 49:02Speaker 00

  Everybody benefits from a postmortem, except for the patient, because the patient is dead. So we said, instead of doing a postmortem for projects that fail after they fail, let's move it to the beginning. And that's why it's called a pre-mortem. And the way it works is if we're on a team, we tape everybody on the team. We're all sitting around the table. And usually we do it at a kickoff meeting. Here's, here's how we're going to carry out the plan. Now, we've spent an hour, hour and a half just go through the kickoff meeting. What are we going to do? Who's doing what? What are the roles and functions? We've got all that nailed down. And we've got about 20 minutes left. And we have enough time to do a pre-mortem.

• 49:47Speaker 00

  And I say, okay, now we're going to do the pre-mortem. Everybody, just relax. Sit back in your chairs. Now, I'm looking in an imaginary crystal ball. I actually do have a crystal ball, but I don't carry it around with me all the time. I'm looking at an imaginal, imaginary crystal ball. And it's now six months from now. Oh.

• 50:12Speaker 00

  And this project has gone off the rails in a major way. It's been a disaster. It's been a failure. It could be six months, it could be a year. Pick your timeframe. What happened is the crystal ball is showing failure. Now, the crystal ball isn't showing us why it failed. It's just showing us that it has failed. That much is certain. And the crystal ball is infallible. It never lies. Now, everybody around the table. You have a pad of paper in front of you and a pen. Take the next two minutes and write down all the reasons why this project failed.

• 50:56Speaker 00

  And then I have a timer and I start the timer. And then everybody's writing like mad for the next two minutes. And then I say, "Time's up." I've got usually a whiteboard. Or on Zoom, I'll have a virtual whiteboard. And I'll say, "Now, I'm gonna go around the room." And I'll start if I'm the project leader. "Here's the top of my list. Here's the thing that I mentioned." And usually, it's something that never came up during the meeting. The leader and the facilitator could be different people. But in this case, we'll say it's the same person. I wanna set a good example. That I'm not just coming with something that's absurd,

• 51:40Speaker 00

  but something that's real. And then I go to the next person. "Shane, what do you have on top of your list?" And then, "Dave, what do you have on your list that neither Shane nor I have mentioned?" And we go around the room and I'm writing these down. Usually, we go around the room a second time, sometimes a third. And we have a long list of things that would explain why this project failed. And most of them are things that people hadn't thought about. And now we sort of are harvesting the collective wisdom of the people and the scar tissue of the people in that room. And we're seeing all the ways that this plan that we were so confident,

• 52:25Speaker 00

  so overconfident in, now our overconfidence has been diminished. Now, some people have worried that maybe the confidence level got too low. So then we have, we added a last part of the exercise. Look at everything on the whiteboard. Look at all of those items. Now let's take another two minutes and write down what, each of us, what can I do personally to try to reduce the chance of some of these happening? And then people are writing down what they can do. And that's a way of trying to reduce the risk. And so that's, that's how we, we run a pre-mortem.

• 53:08Speaker 00

  What we find is it surfaces ideas and flaws that people hadn't considered. But it also creates a culture of candor in the, in the team, where people are used, are starting to get used to expressing problems rather than covering them up. And it creates a sense of trust that I can say something and I'm not going to get criticized for it. The way the pre-mortem works is it reverses the usual dynamic of these meetings. Often at the end of a meeting, somebody will say, "All right, we're done, we're just about done with the meeting. Does anybody see any problems?"

• 53:51Speaker 00

  Nobody wants to identify a problem. We've just spent the last hour and a half coming up, discussing the plan. Nobody wants to admit that there's a problem. People aren't even thinking about problems. They're all in a go mode. Let's get started. They're impatient to start. And there could be consequences of exposing problems. With a pre-mortem, we've reversed that dynamic. The way you show you're smart in a pre-mortem is the quality of the items that you generate.

• 54:26Shane Parrish?

  I think you hit on something that is hiding in plain sight that most people miss, and we've spent quite a bit of time on ourselves. It's sort of, how do you run more effective meetings by changing what people signal as value? And you can do this by asking people for unique insights into the problem that nobody else in the room has, as one way. Instead of everybody coming in and summarizing the executive summary, which everybody's read, but everybody needs to paraphrase because they want to signal that they've done the work and they should be in the room and they know what the problem is, even though they're just regurgitating the same terms over and over again. I think it's really a key factor to the pre-mortem

• 55:07Shane Parrish?

  that you're signaling expertise to your coworkers through coming up with something valuable and uniquely insightful into the problem.

• 55:19Speaker 00

  Right, so it's sort of like, it's almost theatrical. I mean, there's a certain performative quality to this. And we try to do it quickly. I just want to get everybody's entry very quickly so that the energy level stays high. And I want to get one from each person because often, like, one of the senior members of the team might want to give the entire list and then everybody just sort of in a receive mode. And I want to get out of that mode. I want everybody ready.

• 55:52Shane Parrish?

  What are the common mistakes that people make in running a pre-mortem?

• 55:56Speaker 00

  One of the mistakes is that they just frame it wrong from the beginning. They'll say, okay, we should do a pre-mortem. So what can go wrong? It's not an issue of what can go wrong. We're dealing with a crystal ball that has showed that the plan has failed. We know that it failed. And our job is to explain why. So saying what can go wrong is too tentative. It's too vague. And so that's, I think, maybe the most common mistake that people make is they don't frame it that way. Another mistake is they let individuals talk too long. And everybody else is waiting.

• 56:38Speaker 00

  And this way, we're going around the room at a pretty rapid clip. And everybody just stating one thing at a time. And a third mistake might be the leader gets cold feet. And says, I just want to hear what everybody else has. And so instead of creating a culture of trust and candor, the leader is creating a dangerous environment where people can get in trouble. And the leader is being shielded and protected.

• 57:11Shane Parrish?

  When we think about reducing errors, one of the common things that sort of comes up with people is cognitive biases. I'm curious as to what you've seen, what you think about trying to limit cognitive biases. We've talked about one, which is fixation error. And I have my own theory on this, but I'd like to hear your views.

• 57:32Speaker 00

  All right, so I have to be careful about this. And just caveat what I'm about to say, that I'm an outlier. That the whole issue of decision and judgment biases is a major line of research. And the community, the heuristics and biases community, has been extremely effective in the research they've done and the applications that they've put this work to in the judgment decision community. So I want listeners to understand that the dominant view is that these biases are important and need to be taken into account. Having said that, I don't agree.

• 58:18Speaker 00

  I don't like the idea of decision biases. I know, my sense from the literature is that there's been little of any success in de-biasing people, and I think that's a good thing. Because the biases that get identified are essentially related to the heuristics that we've learned, and are part of the experience that we have. And they only look like biases in hindsight when one of them doesn't work. But it's a heuristic, it's not an algorithm. So sometimes heuristics don't work. And the way the research has been done is, you put people, you give them a task where

• 59:00Speaker 00

  the heuristic that they're used to applying, it's not going to apply in this situation. And you show that people fall into the trap that you've set to them. And you say, "You see, their heuristics are flawed, and yet people, many people, most people, fall prey to it." But nobody has done research on the positive side of these heuristics. And so I think that's a bias on the part of the decision bias researchers, that they're only looking at the down arrow. "How do these heuristics get us in trouble?" And they're not looking at the strengths of these heuristics.

• 59:41Speaker 00

  And, you know, Danny Kahneman and Amos Tversky, who developed these ideas, they themselves stated that these heuristics are generally useful, even though they're not perfect. And what's happening is a community is setting up, because it's easy to set up studies where heuristics get you the wrong answer, and so there's lots of these studies performed. And so that's the message that gets compiled without looking at the upside. So I've written about positive heuristics, that all the major heuristics that get criticized are extremely valuable for coming up with insights and making decisions. So availability bias.

• 1:00:28Speaker 00

  Of course, that's going to be helpful if I want to draw on my experience in order to know how to work in a situation. It's not going to be perfect, but that's what my experience buys me. The same thing for representative bias, anchoring bias. I'm going to anchor on a number. Now, sometimes people do foolish things. If you just give them a random number, they anchor on that. So people are capable of foolishness. But generally, I've got to start somewhere. And so I draw on my experience to see what's this like? And I'll anchor on that and I'll adjust from that because my initial impression,

• 1:01:11Speaker 00

  if I have a reasonable amount of expertise, is going to be accurate. And so the heuristics that get criticized, we would be crippled without those heuristics. And I've created a thought experiment, which is, you know, people worry about emotion clouding our intellect and getting in the way of good decision-making. And as it turns out, there's a part of the brain, I forget what the structure is called, where emotion and decision-making come together. And there are some people who have a lesion in this part of the brain.

• 1:01:53Speaker 00

  And this is work done by Damasio 15 years ago or so. And he studied people with lesions in that part of the brain where they were unable to draw on their emotions to make decisions. In other words, they were purely rational. They were like Mr. Spock. Their intelligence didn't suffer. Their IQ was as high as it ever was before the lesion. But their lives became miserable. It would take them 45 minutes to decide what restaurant to go through. They got divorced, they lost their jobs because they could not use their emotions because our emotions are a way of drawing on the patterns we've built and on our experiences.

• 1:02:37Speaker 00

  So here's the demonstration, the thought experiment I have. If you are a strong believer...

• 1:02:46Speaker 00

  In the idea that emotions cloud our thinking and interfere with our decisions and our judgments. We know where this area is. We can do laser surgery to triangulate different laser beams to create a lesion in this area. So you will no longer be troubled by emotions when you make decisions.

• 1:03:12Speaker 00

  And my challenge is to any of the judgment decision researchers who worry about emotions clouding our decisions. Would you have the surgery done? I have yet to find the backer. And I've said, I will pay for the surgery myself. Now, this would be a great experiment, a great opportunity. Nobody has taken me up on this offer.

• 1:03:35Shane Parrish?

  Talk to me a little bit about making decisions in teams of people versus an individual making a decision where boards or committees are sort of asked to come to a decision. What are the pros and cons of a team-based approach? And what have you seen as more effective ways to get teams to make decisions?

• 1:03:55Speaker 00

  Team decision making, the teams use different kinds of strategies. And there's different formats. You can have an autocratic team where a leader just makes a decision. You can have teams voting. You can have the vote be anonymous if people are afraid of repercussions. There are situations like, let's take a pirate ship. In the old Caribbean days, when you had pirate ships around, pirate ships were extremely democratic. There was nobody in charge. They voted who was going to lead them. They voted about what targets they wanted to attack. But once they were in battle, there was a captain.

• 1:04:41Speaker 00

  And the captain's orders had to be obeyed. Because under that time pressure with that kind of urgency and those kinds of stakes, you needed to switch your decision strategy to something more autocratic. So there's no general rule for how a team should make decisions. It's going to depend on the situation and the context. I can give you an example of what not to do. There was a movie with Matt Damon a number of years ago called The Martian. And Matt Damon is part of a team that went to Mars. And something happened and they needed to depart. And they gathered everybody together, but they couldn't find Matt Damon.

• 1:05:25Speaker 00

  He wasn't available and it was urgent that they depart. And so they departed without him, assumed that he had gotten injured or killed. That's why he wasn't there. So now they're taking off from Mars, they're heading away. And Matt Damon somehow finds his way back to the station. And radios them and said, "Where are you?" And they realize he's still there. So the decision is, do we go back or not? And the people on the spaceship know that if they go back, it's going to be risky. So they said, "Let's see if we can come to a consensus about what to do." I don't like the idea of consensus decisions.

• 1:06:06Speaker 00

  And I don't like the idea of a consensus decision in this kind of a dangerous environment. Because as the people in the spaceship went around, there was enormous pressure on everybody to go along with the consensus, which was we should go back and rescue him. And it was public. And they went back and they risked all of their lives in order to save them. What they should have done is had people vote in a way that was secret. And I see this with people who are cross-country skiing in the back country. And you don't want to just cross-country ski where it's level. You want places where there's hills.

• 1:06:49Speaker 00

  But if the hills are too great, then you have a risk of an avalanche. And at a certain point, they'll look ahead and say, this could be risky. Do we want to continue or not? And it's the same dynamic. There's a lot of pressure on people to inhibit their fear and go along with the consensus. Rather than finding a way to vote anonymously and realizing that one or two of the members don't want to do it. Having votes that are anonymous is important. And I also don't like consensus decisions because those tend to be...

• 1:07:28Speaker 00

  And in other situations, the consensus is fairly risk averse, unlike with the Martian. And people want to have something that everybody can agree with. And you've lost your opportunity for somebody who's had an insight. Somebody has a new idea of trying to put that in practice. But the consensus is to try to do the safe thing that's comfortable and everybody can live with. Rather than to do something that's bold and innovative. So I have problems with consensus decisions.

• 1:08:06Shane Parrish?

  I have problems with group decisions. I think groups, as a rule, like a heuristic, make terrible decisions. Individuals should make decisions. Groups should provide input. But there should be one sort of person making the decision. But that's my personal sort of experiences that I've lived through, through all of the different roles that I've had in life.

• 1:08:28Speaker 00

  I would agree with that. I was trying to answer your question, how should teams make decisions?

• 1:08:34Shane Parrish?

  No, this is too important though, right? Because not everybody in... When people, a lot of people agree with it, but they can't change it. So they're part of a committee and the committee is making a decision, even though everybody on that committee might think it would be more effective if one person signed their name to the decision, which also would increase the accountability of the process, believe it or not. Because often, what I've found with group decision making is that when the decision is correct, everybody in the room was responsible for it. And when it was wrong, everybody in the room tried to convince them not to do it. And so there's no sort of like learning that happens from these. But in, in, in the real world, a lot of organizations still insist on making decisions as a group.

• 1:09:18Shane Parrish?

  So the question was sort of like, how do we make more effective decisions as a group? And, and I think that you answered that beautifully. And the second question is, how do we, how do we illuminate the insights in a group setting when we are making a decision as a group? How do we find ways to surface unique insights into the problem that we might not know? Do you have any experience with that?

• 1:09:44Speaker 00

  Well, the pre-mortem method.

• 1:09:45Shane Parrish?

  Right.

• 1:09:47Speaker 00

  Accomplishes that to a great extent. And I think what's important about the pre-mortem method that could also be used generally by, and, and, and group and team environments, is to have people craft their ideas and their interpretations and their assessments individually and privately before they surface it to other people. And the, the research on like brainstorming, we're all brainstorming together. The, the research shows that brainstorming does not produce more ideas or more innovative ideas. And I, I think teams would be, and groups would be better having individuals generate their own concepts.

• 1:10:29Speaker 00

  And then after they've done it independently and privately, sharing it with the others.

• 1:10:33Shane Parrish?

  And that would be a good way to generate insights that are hard to surface about why the project might fail. Is there a way to even bring those forward into how we're proceeding with the project? Like, how do we get the insights on the table? Even if those insights are defining the problem, because before you can solve it, you have to define it. And often we have very different, if you were to, this is a fun experiment and I suggest to everybody who, who does this in, in group settings, do this. When you go to a decision to make or go to a meeting to make a decision, have everybody in the room write out the problem on a piece of paper that they think they're solving with this decision. And then compare how different and how much variance there is in those problem statements.

• 1:11:18Shane Parrish?

  And so often, um, somebody has a more unique, what, what tends to happen in the real world is you go to a meeting, somebody surfaces what the problem is, it sort of clicks the minimal standard in everybody's mind. And then being all type A individuals, we jump on solving the problem. And we forget to sort of go back and say like, do we have the right insight into the problem? Have we defined the problem? And this is where if you have a sole decision maker, instead of a group, that you can acknowledge that the responsibility of that decision maker maybe to listen to other people's definitions of problem. But ultimately they're responsible for defining the problem for the group and for themselves.

• 1:11:57Speaker 00

  Okay, so I have a bunch of reactions because you've said a lot of very interesting and exciting things just now. So first of all, I agree with you, better to have a single leader that they trust. They don't have to think that the leader is infallible. Uh, but they, they just know that the leader is going to be effective at, at making the most of, of the team's resources. But one of the traits that you want in a leader is somebody who can honestly query the team members. And see what, what they think. A lot of times that querying is done in a rote way. It doesn't accomplish much. So I know that I'm the leader, but I'm not supposed to say, here's what we're doing.

• 1:12:41Speaker 00

  I'm supposed to have some, some guise of, of being open to other ideas. So I say, so let's go around the table and, and see what other people think. What do you think? Okay, good. What do you think? So we're just checking the boxes so that I can say, but I, I went around the, uh, the table. And I learned what people had added mind. That's not the way to do it. Instead of just checking the boxes, you want to be curious about what do they think?

• 1:13:12Speaker 00

  What are your thoughts about this? Hmm. In what ways do you think, um, do you have a different sense of what's going on here? Or what our goals can be? And let's capture that. So that's the way to go around the table, not in a rote fashion. You're starting to get into the area of wicked problems and difficult problems. And I'm doing a project on that right now with two colleagues, John Schmidt and Sean Murphy, on wicked problems and how to proceed with wicked problems.

• 1:13:50Speaker 00

  And we're, we're finding that the notion of the goals is going to get richer and it's going to change as we proceed, as we go along. And you want to capture that. You don't want to lock in to our original ideas because almost certainly our original ideas are going to be inadequate. Now, there's also a way to do project reviews that could try to capture more insight. In many organizations, if we're doing a project, well, I'll say, okay, Shane, you're doing this project, let's have a review every three months. And now it's a standard review.

• 1:14:35Speaker 00

  What resources have you expended? What did you project that you were going to expend? What tasks did you accomplish? Where are we in the milestone chart? And we go through all of those things. But I can also ask you in that review meeting, Shane, in the last three months since the last meeting, has anything happened that surprised you? If we're dealing with a complex situation with a wicked problem...

• 1:15:03Speaker 00

  That's different from something that's sort of like painting a room where everything is predictable. So I say, has anything happened in the last three months that has surprised you? And if you say, "No, no, nothing, nothing has surprised me," you don't have to worry. That's when you have to worry, because I want to know what surprised you. And now we can, we can start to examine that and see, and be curious about it and see what it might mean and how we can change our notion of the problem or the goals or any of, any of the important parameters.

• 1:15:42Shane Parrish?

  I want to switch gears a little bit. Often when we are making a decision, there's two really similar options that are, I wouldn't say equally, but near equal in terms of their effectiveness or our anticipated effectiveness about how they'll solve the problem. And we spend a lot of time, money and resources trying to make a perfect decision. Can you explain the zone of indifference to us?

• 1:16:11Speaker 00

  Right. I love the idea of the zone of indifference. And the phenomenon, the way the phenomenon works is if I've got two choices, a terrible option and a wonderful option, quick, which one do you pick? Okay, that's not a hard decision. As we move them closer together, and so now the terrible option has some positive features and the great option has some negative consequences. And we get to this point where it's really hard because the strengths and advantages of, of one versus the other are almost totally balanced.

• 1:16:55Speaker 00

  These are the hardest decisions people ever wrestle with. And the paradox is if the advantages and disadvantages of the two options are almost perfectly balanced...

• 1:17:07Speaker 00

  It doesn't matter which one we pick. And yet, we will spend days, weeks, going around in a funk. Which one am I going to choose? Committees spend hours going over it. And if there's anything that can be valuable and efficient in what I'm discussing, it's this. When you think you're in a zone of indifference, to recognize, I'm in a zone of indifference. I'm never going to be able to tease them apart. So I'm just going to pick one and spend my time in more fruitful ways. Even if it means flipping a coin.

• 1:17:46Shane Parrish?

  I like that a lot because I think we do spend so much time. Often, we get paralyzed with perfection. And I have a heuristic called Stop, Flop, or No that I use to sort of like get myself out of this paralysis, which is you stop gathering useful information. Flop is the first lost opportunity. So you're going to lose an option or opportunity. Or you know, you know what to do. And so if you stop Flop or No, it can get you out of this paralysis.

• 1:18:17Speaker 00

  And so that works for you?

• 1:18:18Shane Parrish?

  Yeah.

• 1:18:19Speaker 00

  And it sounds like a reasonable strategy.

• 1:18:23Shane Parrish?

  It's not perfect. It's a heuristic again. But talk to me about shadow box.

• 1:18:29Speaker 00

  So here's how shadow box works. It's primarily a scenario-based approach. I create a tough, challenging scenario. And we go through this scenario. Usually it's in text. And I stop it part of the way through and I say, here's decision point one, Shane. At this point, here's four different options. Things you can do, courses of action, rank order, which you would do from top to bottom. And then write down your reasons. Sort of like your decision log.

• 1:19:11Speaker 00

  And then we continue the scenario. And then we stop it again, decision point two. Here are three goals that you might pursue. Rank order the importance of each of these goals and write down your reasons. And then we continue, then we stop it again. Here's decision point three. Here's five pieces of information that you can pursue that would help you. Rank order the importance of each one of them and write down your reasons. And it could be anything like that, and we just go through the scenario that way. At the same time, I've had a panel of maybe three experts. They've gone through the same scenario that I'm showing you.

• 1:19:55Speaker 00

  They've written, they've done their ranking. They've written down their reasons. We've combined their rankings. We've synthesized the reasons. So now, when you go through this scenario, and we get to decision point one, here's four different courses of action. And you rank order them, and you write down your reasons, and then we say, let's see what the experts picked. And then immediately, we show you how your ranking compares to the experts. And you want to match the experts. You're really trying to get your ranking to match theirs.

• 1:20:39Speaker 00

  So that's the game part, you want to get that to match. But the real learning is, look at what the experts wrote down as their reasons. Here are the things they noticed. Here's the things that they inferred. Here's the things they worried about. Here's the opportunities they saw. And you look at what you wrote down and what they wrote down. And now, you are seeing the world through the eyes of experts. But the experts don't have to be there, because that would be a bottleneck. They've done their work, and now we just have what we recorded from them, and we can provide it to you as feedback. So you're seeing the world, at least this scenario, through their eyes. So that's how Shadowbox works.

• 1:21:24Speaker 00

  And in a half a day of training, we show that people's match to the experts increases by about 25% in just a half a day of training.

• 1:21:38Speaker 00

  Now, there's another version of Shadowbox, which is video-based, where I'll have like a one or two-minute video of something.

• 1:21:49Speaker 00

  And I'll say, "Here's a situation." It could be a police officer dealing, using a body cam, a camera footage, or it could be something that's staged in a hospital. And, uh, like if it's a hospital, you know, tell me, you know, what's happening in this situation. And you watch this scenario from beginning to end.

• 1:22:20Speaker 00

  And then we say, "Now, let's start again."

• 1:22:24Speaker 00

  And as we go through the scenario, you can pause it at any point you want. And then there's a little circle, and you can put the circle around any cue that you think is important to be watching.

• 1:22:38Speaker 00

  And you do that as excessively as you go, and you write down why you, why you, you, you highlighted that cue, and we go through the scenario. Then after you finish that, we say, "Now, let's go through the same scenario." And we've had experts do the same thing you did. Let's see what the experts were focusing on, and when they were focusing on it compared to you. And then what reasons they, they gave. So this really is seeing the world through the eyes of experts.

• 1:23:08Shane Parrish?

  I, I, I love that idea, and I think it's really effective, and we're exploring using it with some of our stuff. When you mentioned the police officer and the body cam, a question that came up for me is, it's one thing to watch in a room with air conditioning and watch this police officer approach a vehicle. It, it's a whole nother thing for the police officer to be approaching the vehicle and be, be in a different environment. And that sort of brings me to the question of what role does environment play in our decision-making? And are there things that we can do to improve our environment that improve our decision-making?

• 1:23:47Speaker 00

  That, that's a very important observation, because this has come up in some of the work that we've done with use of force decisions.

• 1:23:58Speaker 00

  Where you're, if you're a police officer, you might be in a situation, should I use force or not? And sometimes there's, that's just part of the job. That's a horrible part of the job for police officers. But we find that there may, there may be sometimes ways

• 1:24:20Speaker 00

  to avoid getting into that decision. The way you structure it, the way you frame it, the way you manage it can possibly diffuse the situation so that you don't necessarily have to get into a confrontation that you might be able to, uh, to avoid it. So yes, structuring the environment is a much better approach. And this is something we have done a little bit with, not as much as we would like, about, and this, when I say we, this is my, uh, my friend and colleague, John Schmidt, how do you get a voluntary compliance rather than intimidation?

• 1:25:05Speaker 00

  And for many, a number of police officers, they only know one dominant way of getting civilians to comply with them, which is to threaten them, to intimidate them. But we did a project a number of years ago. Uh, my wife and I, my wife Helen and I did the project to try to see what other techniques for gaining voluntary compliance, for avoiding these kinds of direct confrontations. And the nickname for this project, it was funded by DARPA, Defense Advanced Research Projects Agency. The nickname was the Good Strangers Project.

• 1:25:46Speaker 00

  And we studied military personnel and we studied police officers who were known to be good at diffusing situation, at creating an environment that was a benign rather than a, a high conflict environment. And, uh, one of the things we learned, so I remember one, one, one police officer that we interviewed, it was in a, it was a small office. He was a large police officer. It was hard to fit into this office. I only would see him like, he's extremely strong. And like, his arms were bigger than my legs, okay?

• 1:26:31Speaker 00

  He was a really strong police officer. And he said, I joined the police because I liked the action. I liked, you know, I just liked it and it was exciting. But I noticed as my career was going on, if I would sometimes get in fights with people. And I almost always won the fight. But I always, there was damage. I was accumulating damage. And then I looked at one of my colleagues. He could get people to do things for him. And just by the way he carried himself and the way he talked to them.

• 1:27:12Speaker 00

  And I said, I can't continue much longer in the mode that I am. What's he doing? What's his technique? And he said, ever since then, I've been trying to come up with golden words. And I've been jotting them down when I see other police officers use them. And I'm trying to build a repertoire for how to gain voluntary compliance. One of his rules of thumb, maybe the most important one for my consideration, is he said, whenever he's in an encounter with a civilian or even with a lawbreaker, he said,

• 1:27:53Speaker 00

  "I want that person to trust me more at the end of the encounter than at the beginning." And that governs the way he carried himself. That he was carrying himself in a way that would move the needle to create greater trust rather than diminished trust. And in terms of mindset, that's such an important mindset shift. His previous mindset was, I'm going to get people to do what I tell them. And so I've got to be tough and I've got to be threatening. And his new mindset was, how can I get them to do what I want and have faith in me?

• 1:28:45Shane Parrish?

  I think that's a great place to end this conversation, Gary. Thank you so much for your time.

• 1:28:51Speaker 00

  Thank you very much for the opportunity. I've really enjoyed the conversation, Shane.

• 1:29:01Shane Parrish?

  The Knowledge Project is produced by the team at Farnham Street. I'd love to get your advice on how to make this the most valuable podcast you listen to. Email me at [email protected]. You can learn more about the show and find past episodes at fs.blog/podcast. To get a transcript of this episode, go to fs.blog/tribe or check out the show notes. Can you do me a small favor? Go online right now and share this episode with one friend who you think would love it. Thanks for listening and learning with us. Till next time.