How To Make Better Decisions | Annie Duke | Modern Wisdom Podcast 233

Annie Duke is a professional poker player and an author. The quality of our life depends upon the quality of our decisions. As a poker player who has competed at the biggest tournaments in the world, Annie understands the value of making good decisions under pressure. Expect to learn how to become a great decision maker, why trusting your gut is dangerous, how to account for luck, Annie's framework for making successful & repeatable decisions and much more... Sponsor: Get 10% off all LipoLife & Jigsaw Health products at https://naturesfix.co.uk/modernwisdom/ (use code MODERNWISDOM) Extra Stuff: Buy How To Decide - https://amzn.to/3jJMHkS Follow Annie on Twitter - https://twitter.com/AnnieDuke Check out to Annie's Website - https://www.alliancefordecisioneducation.org/ Get my free Ultimate Life Hacks List to 10x your daily productivity → https://chriswillx.com/lifehacks/ To support me on Patreon (thank you): https://www.patreon.com/modernwisdom #mentalmodels #decisionmaking #gametheory - Listen to all episodes online. Search "Modern Wisdom" on any Podcast App or click here: iTunes: https://apple.co/2MNqIgw Spotify: https://spoti.fi/2LSimPn Stitcher: https://www.stitcher.com/podcast/modern-wisdom - Get in touch in the comments below or head to... Instagram: https://www.instagram.com/chriswillx Twitter: https://www.twitter.com/chriswillx Email: [email protected]

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  In order to be a great decision maker, you have to do two things and you have to do them pretty intentionally. You have to understand and see the luck. You can't control the luck because luck by definition is outside of your control, but you want to see it as accurately as possible. Here's the spread of outcomes that can occur and here's how likely those things are to happen. But then the other thing you need to do is start really examining, what are the beliefs that you have that inform the decisions that you make?

• 0:32Chris Williamson?

  Many of our listeners will be familiar with you, but for those who aren't, can you give us a bit of background? What makes you an authority on decision making?

• 0:40Speaker 00

  Oh, my God, that sounds so very serious.

• 0:45Speaker 00

  Well, I guess it's, I guess it would be kind of the, the very strange path that I took as an adult.

• 0:53Speaker 00

  So I started off after college in a PhD program at the University of Pennsylvania. I was studying cognitive psychology, cognitive science. In particular, I was really thinking about how people learn.

• 1:12Speaker 00

  So I was, I was going to go off and become a professor as one would after getting that degree. And as I was, ABD, as we call it, but before I defended my thesis, I actually got sick and ended up needing to take about a year off before I went out to become a professor. And in the year that I took off, I started playing poker.

• 1:37Speaker 00

  And I did that kind of in the meantime, in order to support myself until I could go and start my career. And so the meantime turned into like 18 years, as it does.

• 1:52Speaker 00

  I kind of kept playing poker. And, you know, and then about eight years into my poker career, I actually started thinking about this very interesting conversation that cognitive science, my background in cognitive science, could have with the real world decision making problem that poker presents, which is obviously very high stakes, lots of skin in the game. If you're bad at deciding, you lose all your money over the long run. And so you really have to be kind of thinking about in particular, how do people learn in these very noisy environments, meaning where there's lots and lots of luck in terms of influencing the outcome and also lots of hidden information, imperfect information,

• 2:34Speaker 00

  which is, by the way, almost any decision you've ever made in your life, including just like whether to proceed through an intersection, is actually very poker like in its decision. So I did, so I played poker till 2012. In 2002, I started thinking about how these two things might converge, started speaking on that topic quite a bit. And then in 2012, I retired from poker, started really focusing on this full time, spent a lot of my time consulting with companies on decision strategy, giving talks, writing books on the topic. And then I've also gone back to academics and I'm back to doing some academic research in cognitive science. So I took a full loop.

• 3:19Speaker 00

  I ended up getting back to where I started, but it just took me a while. It took me a while to circle back. So, I mean, I guess, I guess that's kind of, you know, I guess I have some practical, some practical bona fides and some academic bona fides, maybe? I don't know, trying to answer your question.

• 3:36Chris Williamson?

  An 18 year hiatus or a sabbatical to just go and be a professional gambler for a bit, is that how it ended up?

• 3:41Speaker 00

  Yes, yeah, exactly.

• 3:43Chris Williamson?

  Okay, so I think that as credentials go, that's not bad. You've got the academic underpinnings and you've got a World Series of Poker bracelet and a bunch of other accolades. So, talking about decisions, how do we define a decision? What is a decision?

• 4:02Speaker 00

  Oh, that's a really interesting question.

• 4:09Speaker 00

  Yeah, so a decision is just a choice about what path you're going to take. So at any moment in time, there are actually infinite paths, although we're not considering all of them. But at any given time, you can go down one road or another. And I'm talking, obviously, a figurative road. Sometimes it's an actual road, but it's a figurative road. If we think about at any moment in time, you can think about the future as branching, right? So there's all sorts of futures that could unfold. And when you make a decision, you're choosing which of those possible futures will be available to you, basically.

• 4:52Speaker 00

  So you can think about it like, if I'm thinking about where I want to end up in 10 minutes, if I go east, there's a certain number of possibilities for where I could end up. And if I go west, there's other possibilities for where I could end up. The other thing I want to just say is like, a decision is not just an action that you commit. So let me go back to the root analogy. If you're on a road, I think that we sort of think that a decision would be to exit the road, to go off the road. But again, at any moment, we could think about, there's the exit choice, but there's

• 5:36Speaker 00

  also the choice to stay on the road. So staying on the path that you're already in is also a decision. And I just think that that's really important because I think that people think about decisions as actively changing something. But when you don't change anything, you're deciding also not to do that. I hope that that was a clear enough explanation. I don't know.

• 5:57Chris Williamson?

  I like it. Yeah, the difference between commission and omission in terms of whether you decide to enact change. I suppose as well, a decision to do one thing is as much choosing to take that route as it is not choosing to take every other route that's available.

• 6:11Speaker 00

  Yeah, actually, so that's a really important point. So that's the point of if I go left, there are certain futures that are now available to me. But the other futures, the ones where I went right, are no longer, would not be available to me, at least not in the near term. So that's why when we're thinking about a decision, we're choosing the futures that are available to us. And we're sort of eliminating other futures that might be available to us. And interestingly enough, like what, once you're sort of thinking about decisions that way, it unlocks a really important decision principle, which has to do with this idea of optionality, because that's what you're talking about, right?

• 6:50Speaker 00

  If, if I choose options, I may be making it so that I, I am rejecting other options that are available to me and there are costs associated with that. And so, once you sort of figure that thing out, then what you realize is that you can put, sort of prioritize what the value of optionality is. And you can get optionality because of one of two things. One is that when you choose the path that you're on, that it's quite easy to get off that path and go back and pick up other futures that you might have rejected. So if I'm on a road and I exit, it's easy for me to get back onto other roads, right?

• 7:30Speaker 00

  So that, that would be one thing. So the difference would be like renting versus buying a house. Right? So if you rent a house, you have more optionality because it's easier to quit. It's easier to get out of the house. It's easier to move. There's lower costs associated with the decision. But if I buy a house, I'm making kind of a longer term commitment where I've reduced my optionality. And if you were thinking about an investment, you have liquid investments and illiquid investments, right? So there's certain, like I could buy a stock and then I could literally sell it the next day. But if I invest in a company as an angel investor, it's going to be very hard for me to divest of that position, right? So that would be sort of thinking about that piece of it.

• 8:14Speaker 00

  And we want to, you know, it's, it's easier and we can make decisions faster when we have, when we preserve that optionality. The other way to preserve optionality is if we can do more than one thing at once. So sometimes we don't have to choose to go left or right, or east or west. Sometimes we could do both. We could go east and west at the exact same time. And that also helps us as well because we don't have to reject as many options. So like the simplest example of that is that most people who own stocks don't own just one. So I don't have to choose like, oh, there's so many stocks to choose among. I'm going to choose Google and I can only put my eggs into that basket. Instead, I can, I can have lots and lots of stocks at the same time. We can also think about like dating versus marrying again, right?

• 8:59Speaker 00

  So dating is like, I can date lots and lots of people at once. And for most people, not all, marrying means one person, right? So I have less optionality if I marry than if I date because I can date in parallel. So, uh, so I think it's really important as you start to sort of construct what a good decision process is to keep in mind what exactly what you said. That you're not just choosing the thing that you're choosing, you're also rejecting the other choices that are available to you. So if you can construct a decision where you don't have to reject as many of those things, then you're going to, you'd generally be better off.

• 9:35Chris Williamson?

  And not just the immediate rejection of whatever the options are, but the second and third and fourth order effects of going down that particular path.

• 9:43Speaker 00

  Right.

• 9:44Chris Williamson?

  A friend of the show, George McGill, who's been on a number of times talking about mental models and he created his own razor. Uh, and McGill's razor is when faced with two equal choices, choose the one which permits the most luck. And I think that you could probably trade the word luck there for optionality. So as you go down that particular path, there are fewer options closed to you than would have been had you have taken the other one.

• 10:06Speaker 00

  Um, I think that's a really good razor.

• 10:09Chris Williamson?

  Annie Duke said it first, George. There you go, mate. Shout out.

• 10:13Speaker 00

  Well, it's because it's really, it's really true. And the reason, the reason that you want to do that is that when you're deciding under uncertainty, which is pretty much all the time, even, even if you have perfect information. So I'm assuming perfect information, which by the way, you never have. Um, uh, there's still a variety of ways that the world can unfold and tho- those worlds are going to unfold at a particular probability. And some of those things are going to be, uh, comprise the downside. There are going to be things that you're not really happy observing. And the thing is that, um, you can make amazing choices that, you know, are going to work out for you, say 70% of the time. That would be a pretty incredible choice.

• 10:58Speaker 00

  We'd want to make that choice over and over again. But what that means is 30% of the time, there's going to be, uh, something that you don't like. Something's going to happen that you don't like. So, uh, given that you're always exposed to that risk, one of the ways to de-risk is to preserve optionality, to choose when you have to pass that you're choosing, say, say they're both, uh, 30% to not work out, that I would prioritize the one that's 30% to not work out, but I can do, I can do something about it. I can get off the path, I can exit. It's preserving my optionality in some way. And that's sort of separate and apart from the fact that I can make, I can know sort of everything I need to know right now.

• 11:39Speaker 00

  And I can have an understanding of, given this moment, sort of what those expected worlds are, but new information's going to reveal itself to me later that might matter to me. Right? Where I may want to do something different. So that's another reason that we really want to preserve optionality because we're not omniscient. So even if I kind of know everything about the state of the world today, I don't have a time machine. So I ca- I can't see what the world will be tomorrow. And sometimes the things that I find out about how the world is tomorrow are really going to matter to me in terms of what I'd like to choose. So I think that razor's amazing. Um, and I'm going to steal it.

• 12:15Chris Williamson?

  Cool, George.

• 12:16Speaker 00

  With credit.

• 12:17Chris Williamson?

  We are, we are, uh, pushing your razor out into the world. Also, a mutual friend of ours, Morgan Housel, was on, uh, last week and-

• 12:24Speaker 00

  Love him. Is, is his book not like ridiculous?

• 12:28Chris Williamson?

  Look, I told everyone that was listening, they had to go and get it. I, I referred to it, what did I say? It was, um, Robert Kiyosaki with a better, a better looking author. Um, it was like the handsome, the handsome rich dad, poor dad.

• 12:40Speaker 00

  I'll tell you something, what I love is, first of all, the book is so incredible, but I've seen- Um. It's particularly good to get Morgan on video.

• 12:48Chris Williamson?

  He's hilarious, man.

• 12:49Speaker 00

  I mean, you get, you get on audio, it's, it, you also get this across, but on video in particular, you really get this. That when he starts, like, you can just see, like, he's so passionate. And he's so excited about what he's talking about. And if you get to see him talk, you know, when you get to see him. It's like you just see, it's like a kid in a candy store, like, he's so excited to be exploring these ideas and be expressing them. Um. And it's so much passion. And so I just think it's like really fun to watch him talk about his ideas. Um. Separate and apart from the fact that his ideas are amazing, like, he's incredible at communicating these things. Um. And really like getting to the heart of what matters without a lot of extra words around it.

• 13:29Chris Williamson?

  Mm hmm.

• 13:30Speaker 00

  Um. I, I would really love to aspire to express the concepts that he does so eloquently in, in as few words as he does. I tend to be wordier than he is. But anyway, everybody should get his book, The Psychology of the Money, of Money. Get it.

• 13:44Chris Williamson?

  I, uh, I said that that was one of the first comments I made about the book. I did. Look, it's linked in the show notes below. You should have already gone back and listened to Morgan's episode, which was on last week. Go and check it out. It'll be available there. Um. I said, I really enjoy the fact that, because he's coming from a blogging background where attention's very scarce. He's, uh, uh, had to hold people's attention so, so sharply. And that's ported across really well into a book that's only like one hundred and ninety five pages, I think, of like actual writing. Short, short, very short, punchy sentences. One of the chapters is like three quarters of a page long. And, uh, apparently sent it off to his publisher and his publisher was like, do you not wanna? Should we, should we make this a little bit longer? And he was like, no, no, no, it's fine. Uh, he also said that he would, uh, mud wrestle me on the internet for pay-per-view money.

• 14:28Chris Williamson?

  Um, if that's, and I'm still proposing that to people, you know, everyone has a price and it's been a tough year for investments this year. So me and Morgan are, if the particular threshold is met for interest on, on the internet, we might mud, we might end up mud wrestling, so.

• 14:43Speaker 00

  Oh my gosh. Okay. I don't know. I don't know if I want to see that or not.

• 14:48Chris Williamson?

  You do. Look, everyone wants to see that and it will be available for $9.97 for a one-time purchase.

• 14:55Speaker 00

  There you go. I'm so happy you had him on though. Everybody needs to hear what he has to say.

• 14:58Chris Williamson?

  The third time this year he's been on the show. Oh yeah, it's getting pretty serious between me and him now.

• 15:04Speaker 00

  Yeah, I guess so.

• 15:06Chris Williamson?

  I know, yeah.

• 15:07Speaker 00

  You're like, you were dating. Now it seems like maybe this is, yeah.

• 15:10Chris Williamson?

  Yeah.

• 15:11Speaker 00

  It'll just become the Chris Williamson, Morgan Housel show.

• 15:14Chris Williamson?

  That's what we, that's, that's, that's my dream. That's what I really want to do. So we've talked about sort of just the matrix of decisions. Why should I be concerned about having a process for decision making at all? Why can't I just use my gut? Most people probably do.

• 15:30Speaker 00

  Yeah, that's a real, well, I mean, the short answer is I assume you'd like to do well in life.

• 15:38Speaker 00

  With some reliability. I mean, look, here's the thing, like, it's not so much that you can't have a good outcome when you use your gut. Of course you can. I mean, in the same sense that I can flip a coin and call heads and sometimes it's going to land heads. I'm going to win a bunch of money, right? And that's like, I could be in a situation where I'm offering you two to one.

• 16:02Speaker 00

  Right? So I'm losing to the bat and I happen to win. I get your dollar. Okay, great, you know? So the, the, the issue has to do with, how are we constructing a process? That's actually going to accomplish the things we wanted to accomplish, which is number one, sort of get us to whatever the best possible judgment is, given the fact that it's subjective. So we can think about there's what's objectively true of the world. What would objectively be the best judgment? And then how close are we getting to that? Right? And given the limits of our knowledge, um, I would call the, you know, the, if you can get as close as possible to that thing, you're gonna do better.

• 16:47Speaker 00

  And generally, you're just not gonna do better with your gut. Why? Because your gut isn't actually any kind of decision tool. It's not repeatable. You can't look back at it and say, well, why did my gut make that choice? Um, your gut is where all the cognitive bias lives. Your gut is where all the noise lives. Like, all the things that frustrate your decisions are living in your gut. Basically, aside from all the bacteria and undigested food. So, um, so, in order to be a great decision maker, you have to do two things, and you have to do them pretty intentionally. You have to understand and see the luck

• 17:30Speaker 00

  that's go- going to be in the way that your decision turns out. You can't control the luck, because luck by definition is outside of your control. But you wanna see it as accurately as possible. Here, here's the spread of outcomes that can occur. Um, and here's how likely those things are to happen. Okay? But then, the other thing you need to do is start really examining, what are the beliefs that you have that inform, um, the decisions that you make? And you need to actually pick those apart into, uh, a bunch of things. Number one is, you have to think about what the component parts of any decision that you make are, which is kind of the antithesis of gut, right? Like, gut is like, I just think I'm supposed to do this.

• 18:12Speaker 00

  And there's no kind of why that follows up with it besides, well, it's just my gut. Right? Um, so, when you think about what are, what are the component parts? What are, what are the actual motivations? If I think about a statement that I make, what must be true of the world? And how can I think about whether those things are actually true of the world? That's kind of the antithesis of gut, because you're actually opening that up to a process that you're examining. So, as an example, if, if I'm investing, um, and I choose to buy a stock, um, by definition, uh, this was something I discussed with Michael Ma- uh, with Michael Mobison, actually, just a few days ago. But when, when I make a, an investment in a stock, I'm saying that I think that I know something about this company that, or this stock, that the market doesn't know.

• 18:57Speaker 00

  So it's not acceptable to just say like, well, it looks like a good stock and I have a nose for value. Right? Okay. I don't really know what that means. You have to say, you have to pick it apart and say, what is the thing that I think that I know? That the market doesn't know. So it could be something super simple, right? I think the market is estimating the number of widgets that they're gonna sell. Okay, great. So now, now you're pulling that apart. Notice how this is very opposite to gut. Right? Because I'm starting, now I actually have to forecast that thing. So you also have to express yourself precisely. As you're forecasting, those things that you're pulling apart that must be true. For you to actually believe the thing that you do, that you should buy the stock, for example.

• 19:40Speaker 00

  So, so that's kind of like the first thing about what you're doing in terms of your beliefs. So you're pulling these things apart into their component parts. You're thinking about what are the possibilities? You're, you're trying to see the luck accurately. And then because you're doing that in all this very explicit way. You now allow yourself a way to go check back. And to say, essentially what I've done is in this very explicit way. I've created what my forecast of the future is. Not in a vague sense of I think this stock is gonna go up. But I've forecasted what I think the states of the world are gonna be in the future. How likely those states of the world are going to exist.

• 20:21Speaker 00

  Such that I can now actually close that loop and go back and check to see how well calibrated I am. And I can now do that because this is all very explicit. That's the problem with your gut, is that it's by definition implicit. It's by definition sort of intuitive. So I don't have that way, first of all, to think about the decision process in, in any kind of rigorous way as I'm making it. But worse yet, I have no way to go back and close the feedback loop in to, in order to create a really good learning loop. Because I can't really go back and check. I can't say, well, here are the, here are my assumptions. Here are the things that I thought would be true of the world.

• 21:03Speaker 00

  Here was my forecast of the states of the world that were going to exist. Here's the data that I was using in order to inform that decision. And so I, I think that when you're deciding with your gut, you're, it's like the, the thing I say is it's kind of like playing pin the tail on the donkey. Right? It's like that old children's games. Like you just put a blindfold on and you're sort of like, and then sometimes you actually get it right there and you're like, look how smart I am. And it's like, yeah, but that's not really a process.

• 21:31Chris Williamson?

  Repeatability is one of the important things that we need for a decision tool. What else do we need to make a decision tool effective?

• 21:40Speaker 00

  Um, so, repeatability, meaning if I use this process in the same way over time, I should expect to get similar results. Um, I also need it to be something that can be examined. Right? So I can examine it like an object and understand whether I'm actually using it in the appropriate way. Um, I need to be able to, uh, give, uh, the process to somebody else, such that they could actually apply that same process. Now we can start to see the problem with the gut, right? Is I can't be like, ew, here's my gut.

• 22:16Speaker 00

  Right? Well, I guess in Star Wars, they, they open that animal, sleep inside of it. They're always sleeping inside of guts in Star Wars. Um, I don't know if that helps them with, and make decisions, but it keeps them alive. Um, ew, but anyway, so I can't, I can't just say to you, like, the, the problem is if you say to me, like, how'd you make that decision? And I say, oh, well, I had gut feel, like I just felt like that was the right thing. This is completely unhelpful to you. Um, because that doesn't mean that you can do it. Right? So I need to also be able to give it to somebody else and explain how you would use this process, such that they could also use it in the same way. That is not to say that they will get the same results from it, because, um, there are issues of talent, right? I mean, two people can learn how to hit a tennis ball and one person will be much better,

• 23:00Speaker 00

  because people just sort of are better at some things than others. But I should be able to get you part of the way there. I can get you to hit a ball, right? Um, and hopefully I can get you to hit a pretty good ball. If it's a really good process, I should get more fidelity out of what you're doing. So, re- repeatability, I have to be able to examine it, um, uh, so that I can understand what its use is, what its purpose is. I have to be able to teach it to somebody else. That's really important. Um, and then I have to be able to go back, I have to loop back, to see, I have to be, it has, there has to be accountability. I have to be able to loop back to see whether it produced, it, I actually applied it well. Um, did I use the right tool for the right job?

• 23:43Speaker 00

  That kind of thing. So uh, it's, it's very simple. It's like a really good decision process, which has a whole bunch of decision tools in it. It's really like uh, just using tools, right? So I, I wanna know, uh, we can think about like a screwdriver, right? Okay, I can look at it. I can figure out what its purpose is. Right? I can, for the purpose that I use it for, to screw screws in the wall, I can repeat that with fidelity over and over again. Um, I could hand you the screwdriver and I could teach you how to use it. Such that you could put screws in the wall as well. And then I can go back and I can check your work. And I can see, oh wait. It seems to me that you stripped the, the threads on this. Did you use a hammer?

• 24:28Chris Williamson?

  Well, I mean, a hammer is the ultimate tool. As everyone knows, there's no job that can't be fixed with a hammer. Which would be the ultimate decision tool, yeah.

• 24:35Speaker 00

  Yes, well, yeah, actually, that's what I was gonna say, like breaking asphalt, but you could do it. It would just be really tiring. Cause you could use the claw part of that. It would just take you a very long time, but you could actually do it.

• 24:45Chris Williamson?

  Hammer is an underrated tool, as far as I'm concerned.

• 24:48Speaker 00

  You could use a hammer, right? So, so you can think about gut as like,

• 24:54Speaker 00

  what if you came in and you saw a screw was in the wall? And you said to me, how'd you get it in the wall? And I was like, it just happened. Because I'm really good at putting screws in the wall.

• 25:06Speaker 00

  Like, I mean, think about how bad that is. First of all, you can't do it. Second of all, I can't actually figure out, like, was that the best way to do it? Was there a better way to do it? Because I just left it in a black box, right? So, and one of the things I wanna be really clear about is that, uh, there are lots of decisions you're gonna make in your life where you are actually using your gut. Just because we, you know, you're not gonna take a really long time on every single decision that you make. The issue is, first of all, you have to know what a good decision process looks like. So you can understand when it's okay to kinda go with your gut. Um, because there is just less risk to sort of not having a high fidelity process. Um, and then the second thing that, that's just really kind of important about that

• 25:51Speaker 00

  is that you have to have a way, even with your gut decision, to go back and say, well, there were things that were implicit in my gut decision. That-

• 26:01Speaker 00

  For me to make that decision, I had to believe were true. So let me go back and actually think about that now. Right, let me think about what were the things that I had to believe were true. Were those things actually true? Was I missing anything? What wasn't I incorporating into my gut feel about it? Um, so that the next time I make a gut decision, having gone through that examination, one would assume I would have honed it better. So, uh, look, uh, there's lots of things that you're just gonna kind of go fast on. And that's okay, as long as you understand what a really good process looks like. And you understand how incredibly important it is to make the implicit explicit in your decision process, so that when you go and take a look back on really what is the quality of the decision that my gut produced,

• 26:46Speaker 00

  that you have a way to actually answer that question, such that you could tweak and hone and realize, you know, and then sometimes you'll realize, um, for say the market that I was in, uh, my gut feel was pretty good, but there's something that's changed about the market that I didn't notice. And now I actually have to go back to being pretty deliberate. You know, I, I actually think about it like, like this, so that you can see these videos. So, um, when you're riding a bike, we know that when you turn the handles to the right, the bike goes to the right. When you turn the handles to the left, the bike goes to the left. So when we're starting to learn how to ride a bike, it's a very deliberative process. Right, we're trying to figure out like, how do we balance the bike? How do we keep it upright? If we turn this way, are we gonna turn?

• 27:31Speaker 00

  And you've seen people, they're sort of, you know, wobbling and whatever. And then eventually we become really good at riding bikes. And it's pretty much something that we're not thinking about very much. Right, like I, I just sort of get on a bike and I go. And, um, not really deliberating about it. So that would be like a, it's turned into a gut decision and it works really well. But, um, the world changes.

• 27:54Speaker 00

  So what happens when someone hands you a bike where now when you turn the handles, it goes to the right, it goes left? And what you can see is that people fall down a lot again because their gut is telling them when they want to go right to actually turn the bars right. And now they have to sort of re-figure out, well, hold on, this is like a different type of bike. And so I have to now learn this process and it takes a while. But then, once they learn it, they can go back to sort of go, you know, using their gut. Decisions and gut decision making are just like a lot like that, that your gut can get you pretty far, but you always have to be sort of checking back in to see if the world is handing you a bike where where the handles bars work differently. Right?

• 28:38Speaker 00

  Because that happens all the time. Environments change and and markets change. So that's like things external to you changing. But then the other thing is that sometimes it turns out that there's a better way to ride the bike. And the way that you were doing it worked pretty good. Right? Like you were winning to it. You were getting to where you wanted to go. It was all kind of fine. But someone shows you something that you could do that's even better than that would actually help you to be more efficient or or whatever it is. And you so that's why you actually, another reason why you want to be examining your gut is even if nothing has changed about the world, you may not have the best strategy.

• 29:22Speaker 00

  The big danger is you have a strategy that's winning.

• 29:27Speaker 00

  That's like the biggest danger that you've never been.

• 29:29Chris Williamson?

  Why?

• 29:30Speaker 00

  Because it means you're very unlikely to go explore if there's a way to win more.

• 29:36Speaker 00

  And, you know, you, by the way, you can see this in investing. There are people who are winning in investing and you they haven't answered the question is, okay, you're winning, that's fine. Would you be winning more if you were indexing?

• 29:50Chris Williamson?

  Yes, Morgan Morgan says yes.

• 29:53Speaker 00

  Yes.

• 29:54Chris Williamson?

  Dollar dollar price average into the market index funds, leave it for fifty years, millionaire. That's what that's what Morgan says.

• 30:00Speaker 00

  Right. So and but that's not true of everyone. But what happens is that we get focused on the fact that we're winning. So we're doing better than zero. So I'm making four percent with my active, you know, I'm sitting here on Robin Hood.

• 30:14Speaker 00

  The worst thing that happens is you're making 4% on Robin Hood.

• 30:18Chris Williamson?

  That is the worst situation to be in. It's so interesting to hear this same sort of reflection come about in multiple different areas of human nature, right? So let's think about the system one versus system two thinking, or the fast and slow from Daniel Kahneman, the explore versus exploit that James Clear talks about in Atomic Habits when you're doing habit setting. Taylor Pearson's consciously designed life versus execution designed life, which is sitting back, thinking very, very deliberately and having to pay the entry price that is being super, super conscious about every little area that you do. And then the hope is that over time, that becomes absorbed into your mode of being. Because you can, perhaps when you're

• 31:01Chris Williamson?

  a child, it's really, really important that you don't put your hand on the hot stove. As you get a bit older, that decision still has the same level of consequence if you get it wrong, but the chance that you get it wrong is significantly lower. So you don't have to be conscious about it, but you do when you're a kid. The same as the reason why they've got those little foam cups on the edge of all of the sharp tables that are at eye height for children in houses, because they might run around and not see a table and then lose an eye. When you're older as an adult, that's usually not the concern that you have, despite the fact the consequences are the same. So this is how we expand our domain of competence, by being deliberate, by learning, by probably making some mistakes, hopefully not losing an eye. And then we allow ourselves to move on to whatever comes next. What do you think

• 31:44Chris Williamson?

  most people presume wrongly about good decision making?

• 31:51Speaker 00

  That a good decision will get you a good result. And that a bad decision will get you a bad result.

• 31:57Chris Williamson?

  So this is the difference between being-

• 31:59Speaker 00

  The main thing that people get wrong.

• 32:01Chris Williamson?

  This is the difference between being right and being accurate, right?

• 32:04Speaker 00

  Yeah, exactly. So mostly, if you have enough cracks-

• 32:12Speaker 00

  At it, uh, good decisions will actually get you good results, but that's in the aggregate. So in the aggregate, that that is true. Um, that, uh, assuming that you have managed your risk of ruin well. So that you still have something to bet. Um, whether it's an eye or money. Um, making repeated, you know, repeatedly making winning decisions, you will actually end up winning. That's true in the aggregate, but what people think is that that's true in the short run as well. Um, and I think that's, that's really like the biggest mistake that people make.

• 32:55Speaker 00

  So what ends up happening is that, uh, essentially we over index on the outcome. This is just resulting, um, in order to sort of try to work backwards.

• 33:05Chris Williamson?

  What does that mean? Just take us through over index on the outcome that's resulting. What's that mean?

• 33:09Speaker 00

  Okay, so, um, let's, I'll, I'll, so let's say that I, uh, traveled through an intersection safely.

• 33:19Speaker 00

  Um, did not get in an accident, nothing remarkable happened. What people will say is that that must mean that I made good decisions as I went through the intersection. And if I travel through the intersection, I get in an accident, they'll say, I must have made poor decisions. This is true, like, um, uh, I make an investment and it loses. I made a mistake. I make an investment and wins. I must be a genius. Um, I ordered the chicken instead of the fish in a restaurant and it's gross. I made a mistake. I ordered the chicken instead of the fish and it's amazing. I knew it, right? Like, must, uh, obviously, great decision. Um, so that's resulting. It's basically saying if I know what the quality of the outcome is, that this tells me something about the underlying decision process.

• 34:05Speaker 00

  Um, the issue of course, is that I might've gone through a red light when I got through safely. And I hardly think, now that I've told you this details, they would consider me a good decision maker in that case. Um, I may have gone through a green light and a car came from the other direction and, you know, what could I do? Um, and that doesn't really make me a bad decision maker that it happened to turn out poorly. Um, obviously same things with that investments. I mean, the, the, the whole thing obviously is that there's volatility. So, uh, you can make a perfectly sound investment.

• 34:37Speaker 00

  When you're thinking about in terms of just expected values, sort of what's my expectation in terms of whether I'm going to win more often than I lose? And if you keep making choices where you're going to win more often than you're you lose, you're probably making pretty good decisions. But you can lose, like in the short run, I can make an investment, I can lose. And it just turns out really poorly. So I think that this is the biggest thing that kind of like really frustrates our ability to learn because the only thing that we can learn from is experience. That's it. That's what we have. Aside from, you know, sort of what the built-in software is that we come with. We've got to learn about the world as the world kind of tells us, like, you know, here's

• 35:22Speaker 00

  some data. And then we sort of process the data and we figure this out and we're trying to close these feedback loops and we're trying to learn. And what ends up happening, of course, is that we're pretty poor at learning because we aren't aggregators. We just naturally don't sort of wait around until we have enough data to understand something that's significant about the set of outcomes that we might be observing. And instead, we process those outcomes in sequence.

• 35:47Speaker 00

  And we don't really know what we're supposed to take from the outcome. And in general, we just kind of take really bad lessons from it. And it's a really strong cognitive illusion. So in Thinking in Bets, you know, I talk about the Pete Carroll example in the Super Bowl. He makes a pass at the end, mathematically a really good play. He loses the Super Bowl because he chooses this this particular play to call. And people still today talk about it as a mistake. Likewise, you know, I think people are pretty familiar with America's 2016 election and Hillary Clinton losing that election.

• 36:28Speaker 00

  So again, to show you like how widespread this is and how much it really frustrates our decision-making and our learning and the lessons that we learn, there's general consensus that Hillary Clinton made a really huge error, like really bungled her election strategy, her campaign strategy. And it was the error that everybody seems to agree upon is that she did not campaign in the Rust Belt, which is really kind of combined of, we're talking about three states, Michigan, Wisconsin, and Pennsylvania. Interestingly enough, Ohio is one of those states that doesn't get mentioned.

• 37:08Speaker 00

  As one of the states that she bungled. Uh, even though it's part of that Rust Belt. And I think the reason why is that she, she would lost that state by a lot. So there would have been no reason for her to go and actually campaign there. But, so she loses Michigan, Wisconsin, and Pennsylvania. Razor thin margin across the three states. It's a total of like 80,000 ish votes. Okay. So, um, so everybody starts writing pieces about her terrible Rust Belt strategy and how she really screwed it up and what a mistake she made and how she bungled that part of the US. And, uh, and this was horrible, what her strategy was. So here's how we can understand, um, where our ability to understand what a good decision

• 37:53Speaker 00

  and a bad decision is really goes wrong. Because what people are saying is she made very poor decisions.

• 37:59Speaker 00

  But here's the thing about presidential elections. It's true about any national election. It's true whether you're in Britain or Germany or France or America or wherever, is that everybody has something to say about it. While it's happening. That's whether it's a seasoned political strategist, a pundit. People come crawling out of Silicon Valley who are like data analysts to write a billion pieces on Medium. Um, so everybody's got something to say.

• 38:27Speaker 00

  So I think that we should assume that, given the data that was available at the time, that if this was a really huge mistake, there would have been lots of people writing about it.

• 38:39Speaker 00

  Because we have to remember, that's that idea of like, what do we know at the time that we decide, right? And our knowledge is always limited in some way. We don't have perfect knowledge. We certainly can't foresee the future. And there's just a whole bunch of stuff we don't know. Um, so we have to judge decisions based on the knowledge that's available at the time that somebody makes the decision. And I would agree, it would be a huge mistake if there were a lot of people who were yelling and screaming about Michigan, Wisconsin, and Pennsylvania and the fact that, uh, it seemed like even though the poll said that she was way ahead there and running even in places like Florida and New Hampshire and evenish in like Arizona, um, which is where she was spending her time, that maybe she should skedaddle out of Florida and get to these three states.

• 39:23Speaker 00

  There, if there were lots of people saying that, uh, and she did not know that herself, I would agree. She was not using the information that was available to her. But you can look it up on Google for yourself. There are crickets.

• 39:38Speaker 00

  Now, you do start to see tons and tons and tons of articles about this particular issue, but the first one of them appears on November 9th, which is the day after the election.

• 39:50Chris Williamson?

  How fortunate.

• 39:51Speaker 00

  Yeah, there were, by the way, two really big, long articles about, I think it was Pennsylvania in particular, and they were critiques of Trump.

• 39:59Chris Williamson?

  Post talking like that, to me, reminds me a lot of when I watch my housemate play COD. So I've, I've got into during the pandemic, I've really, really gotten into watching him play Warzone online terribly. He's awful. Toby, I love you, man, but you suck. God, you know, if this is, not only does he suck in terms of how I get to see the more subjective areas of his game, like when he crouches and when he sprints, but there's objective measures as well in that he sometimes goes through a whole game with zero damage being made, et cetera, et cetera. But I'll sit there, I know, it's, you could just pull the trigger for the whole game.

• 40:42Speaker 00

  I literally don't even know the game, and I know that has to be a bad strategy.

• 40:46Chris Williamson?

  It's not a good thing. It's not a good thing. But I'll happily sit there and abuse the guy with no control of myself, just with like a chocolate bar or like a glass of water in my hand, and I'll happily abuse him and tell him, no, no, no, you should have gone left. Oh, why didn't you go left? Like, I knew that left was a good thing, but I don't, I only ever say that like after he's dead and on the floor, or like doing the signal thing where he needs someone to come and revive him. So it's precisely the same as this, right? And everyone's, everyone's a critic. Everyone's uncle.

• 41:14Speaker 00

  Exactly. It's like, and this is why I say it's a, it's a, it's a really robust cognitive illusion. In the same way that when I look at a visual illusion, you can explain to me what the illusion is, but I cannot unsee it. Right? Like you can see, like on Twitter, you'll see like all these great illusions, like it'll be a stationary cube, and because the background is flashing, you'll see the cube spinning.

• 41:42Chris Williamson?

  Is that Steve Stewart's thingy that you follow?

• 41:46Speaker 00

  Well, so he, yeah, so he'll post them, Phil Tatlock will post them. There's a Japanese person who posts tons and tons of visuals.

• 41:53Chris Williamson?

  Yes, Steve reposts his stuff, doesn't he?

• 41:56Speaker 00

  Yeah, exactly. I'll repost some sometimes. Phil Tatlock has been reposting a lot of them lately. So the thing is, you can explain to me what's happening, right? You can say to me, the cubes are not moving. It's the fact that the background is flashing that's making you see the cube spin. And you can, and I'll go, oh, that's really interesting. And then I look at it and the cubes are spinning.

• 42:13Chris Williamson?

  Yeah.

• 42:15Speaker 00

  Um, and this issue of resulting is like that. So, um, I know what, I know that nobody was talking about this before November 9th. I know that. And it's very hard for me not to feel, deep down inside of me, that Hillary Clinton made a mistake. It's hard for me to get over that. Um, because it really does feel that way. Like, God, when that, that food comes to you in the restaurant and it's disgusting.

• 42:42Chris Williamson?

  You shouldn't have ordered that.

• 42:45Speaker 00

  I should, uh. So, so that's kind of the, the first thing that has to do with like this over indexing on outcomes. But the other thing that comes from that is also that we misremember what our state of knowledge was at the time. So that's what you're talking about with, I knew you should have gone the other way. Right? So there's a little bit of a difference between, um, uh, you know, somebody saying, it was the wrong choice for you to go right. As if that has now revealed itself to them afterwards. Versus saying, I knew beforehand that you should have gone the other way, regardless of whether you said it or not. Um, this actually happened to me with this Hillary Clinton thing, because I pitched it to somebody at a, a major newspaper. And they said, oh, I can't run this because

• 43:30Speaker 00

  I did actually know this. And I had a bunch of friends who knew it, and we're all talking about it and reading about it. And I'm thinking, you're at a major newspaper, you could have literally published that. But you did not. So I'm not exactly sure why you kept that incredibly contrarian opinion that, that may have gone viral to yourself.

• 43:49Chris Williamson?

  It's so, so.

• 43:50Speaker 00

  But you apparently did. But, but literally, they're misremembering what their state of knowledge is. So now think about how hard it is when we talk about, like, we were talking about what makes a good decision process. And part of it is this ability to go back and examine. If you can't even remember what you thought at the time, if you can't remember what your state of knowledge was when you made the decision, how on earth are you going to create any fidelity around that process? Like, how are you going to go back? And then what are the lessons we're learning? So I'm seeing it in the United States right now where people, everybody, like, you can, everybody's talking about like, oh yeah, you know, Biden's running ahead in these three states, but I'm sure it's wrong. That, you know, because obviously there was a polling error in those

• 44:33Speaker 00

  three states. The thing about a polling error is you can't find out until after the fact. So now everybody has decided that we all knew this and, you know, now the only strategy that you could possibly apply to a presidential election would be to spend all your time in Pennsylvania, Wisconsin, Wisconsin and Michigan, apparently. What a terrible lesson to take from that. The lesson should actually be, is there a way for you to detect which polls are more accurate and which aren't? And if they're not accurate, can you predict which way they might not be accurate? Because there's no reason in this election cycle that we should assume if there's a polling error, that it would go in Trump's favor versus Biden's favor, right? It can go, that, that can actually go both ways. And we know that this is actually quite hard because in 2016, the polls

• 45:16Speaker 00

  nationally were dead on. And in states like Florida and New Hampshire, they were also dead on. Right? So we had this very specific polling error. I would actually argue that in this election cycle, it's probably more likely, and I don't know, I'm just making this, I would have to talk to a pollster because there may be just structural problems to polling those states. But assuming there aren't structural problems, I would assume that they would be polling more accurately. Because I have to imagine the pollsters are like, super duper like, holy crap, I can't get these three states wrong. And so if there were a polling error, my guess is, if I were to bet, given my knowledge, I would want to go find out more before I place money on it. But if I had to bet, I would say the polling error would be more likely to occur outside of

• 45:59Speaker 00

  those three states right now. Just because I have to assume that people are really kind of looking at those. That's assuming there's not something structural about those states that would make polling them quite hard. But you can see that people are kind of taking the wrong lessons. So you can have a candidate now come along and just sort of based on this idea that she made a horrible mistake, say, my campaign strategy should be to really campaign hard in those three states. But that could be completely a terrible campaign strategy. You're just learning really bad lessons.

• 46:27Chris Williamson?

  I don't envy the job of a campaign manager at all. It's just.

• 46:32Speaker 00

  No, you're just gonna like, if you win, you're a genius. If you lose, you're an idiot. There you go.

• 46:36Chris Williamson?

  It's it's so, so hard. How can people spend their decision making time more wisely? Talk about this in the book, and I'm very, very, very interested in the opportunity cost of not making decisions of how long it takes to make one. Obviously, it's all well and good as saying here, be conscious with your decision making process and let's have this, that and the other. But you need to spend that time as wisely as possible, right?

• 47:00Speaker 00

  Yeah, so so we've hinted at it a little bit because we've been talking about like, what road are you on? And are you getting off the road and that kind of thing, which is a little bit of unlocking it? So the idea is, look, you want to, you want to obviously understand what a really robust decision process looks like. But then I said, sometimes you're going to go really fast. I would not want someone to think that you should never go fast. The question is, what are you going fast on and what are you going slow on? How are you valuing the time that you're taking in order to make the decision? Because every minute that you spend gathering information and considering options and try to kind of map these things out is a minute that you can't get back.

• 47:39Speaker 00

  So what we want to generally be thinking about is how can we be sort of maximizing the amount of time that we're spending on things that are going to

• 47:50Speaker 00

  bear fruit? And minimize the amount of time that we're spending on things that aren't going to be particularly fruitful. And that includes your decision making time. So you want to spend your decision making time on places where there's really going to be a lot of value to doing so. And not spend so much time on places where there isn't going to be a lot of value to doing so. And actually, sometimes

• 48:11Speaker 00

  you're costing yourself a lot by not going super fast. And there's two ways that you can cost yourself a lot by not going super fast. One is that options expire. So sometimes by going slow, the opportunity actually goes away.

• 48:27Speaker 00

  So actually, that was a really big problem with coronavirus response. Was that options kept expiring? So a simple option, for example, would be.

• 48:39Speaker 00

  Uh, make sure when you get the very first intelligence, that there's something bad brewing, um, that you ramp up production of PPE. Right? So, uh, the longer you wait to do that, uh, the option to actually have that be a reasonable solution to mitigating the risk of the virus kind of goes away. Cause obviously as soon as you have like widespread community spread, it's helpful, but then you're trying to sort of roll back a disaster as opposed to prevent a disaster. So that that's an example of like the opportunity going away. Obviously, if you're thinking about investing in a company, at some point they close the round.

• 49:21Speaker 00

  So, so there's that cost to it, but the other, the other cost can be that sometimes the thing that you really need to know or that you wanna know is, um, how the world responds to the decision that you made or how you respond to the decision that you made is you're trying to sort of gather information. Um, and the best way to do that is to like, to do a lot of stuff really quickly in order to, to get lots and lots of information. So that when you do have to make really significant decisions, you have much better models of the world. And this is true like of our own preferences. You can think about this as like, um, a priori, like ex ante, there are certain things that I won't know whether I like or not. Right? It, it could be a food that I've never encountered before and I, I kind of don't know ex ante.

• 50:05Speaker 00

  The only way that I can know whether I like that food is to actually taste it. So there's no reason for me to research it and go ask a lot of people their opinions about the food and so on and so forth. I might, uh, the best thing I can do is just try it. Um, and that's true of a lot of stuff that we can, that, that we wanna be doing. So the question then is, okay, so how do you sort of sort through that stuff to figure out? What are the things that you can go pretty fast on? And what are the things that you can go pretty slow on? Or you should go pretty slow on? And the answer basically all comes down to a core principle, which is in cases where when you experience the downside, it's not that bad. You can go really fast. Because what we're really trying to do when we make decisions and we take our time

• 50:50Speaker 00

  is to try to reduce the probability of a bad outcome occurring and increase the probability of a good outcome occurring. So we want to make sure that on balance, we're winning the decision, that the upside potential is greater than the downside potential. And the more that we can create a spread between those two things, the better off we are. Occasionally, it's sort of like you're choosing among bad options because that does come up. Sometimes you're kind of in a no-win situation. And even then, you're trying to increase the spread between those. You're trying to figure out which one has the least, is sort of the least bad. Sometimes you have all good options and you're trying to figure out which one is the most good. But all of these are just sort of weighing upside and downside potential. So presumably, what we're getting out of sort of mostly,

• 51:34Speaker 00

  what we can get out of spending more time is that our forecast of what the upside and downside potential is gets more accurate.

• 51:46Speaker 00

  So if we spend less time, we're sacrificing some accuracy. So we may end up with a little bit more downside than we would have if we had spent a little bit more time. That's why it all comes down to, in terms of the time that you spend, you spend, how bad is the bad? Because if the bad isn't that bad, it doesn't really matter. You should probably just like move fast and break things, as they say. You should just go around because what you're getting in return is you're not missing out on opportunities and you're learning a lot. I mean, this is a way to do this fast cycling learning. So let's sort of figure out like, okay, so how do we figure out when bad isn't bad? So we've already talked about a couple of those things, right? So one is when something's really quittable.

• 52:31Speaker 00

  So if I'm thinking about a decision that I'm making and I'm trying to decide, do I want to exit?

• 52:40Speaker 00

  When, whenever I choose a route, um, I might experience downside on that route. There could be a road closure. But also, I'm giving up the gains that are associated with other options. Okay, so this is one of the things that I'm risking. When I know that I can easily get off the road that I'm on and go get on another road, what that means is that I'm naturally mitigating the downside. Because I can just quit. When the world starts to reveal itself that a particular future is unfolding that I don't find particularly favorable. Or when I think that I'm gonna like durian and I taste it and it's yucky. I don't have to finish the whole fruit.

• 53:23Speaker 00

  I, I can just throw it in the trash and go find something to wash the taste out of my mouth. Right? So, um, so the, the, basically, we can sort of think about this as the more liquid the thing that we're doing is, the faster we can go. Because we can gather information and as we gather that information, if we don't like what we're seeing, we can go, we can get out and go to other options. So that's quittability. Then, we also already talked about doing things in parallel. So, we know that we're always exposed to a certain amount of risk. Um, but what we can do when we do things in parallel is we can just basically sort of have a bar. Right? We can say, this is good enough.

• 54:08Speaker 00

  I'm gonna do it because I'm gonna do enough of it that I don't really have to worry about whether I'm gonna realize the downside on this particular choice. Because across all the choices I make, I know that I'm gonna be positive expectancy and so therefore I should make a bunch of money. Um, so obviously, that's just portfolio theory, right? So I can own a whole bunch of stocks and then, um, that's great.

• 54:31Speaker 00

  Another way to think about doing things in parallel is that if I can get on both sides of a position at the same time, then I can obviously go faster because I'm sort of betting on both things at once. I'm betting that I'm right and I'm wrong at the exact same time. So that would be like, not necessarily in every single market, but in a lot of markets, stocks and bonds would work that way against each other.

• 54:55Chris Williamson?

  Would perhaps doing the night class of the career you're considering doing whilst staying in your current mainstream job to see if you like it, or moonlighting on a weekend as a veterinary nurse whilst you continue to do your art midweek or whatever it might be.

• 55:10Speaker 00

  Exactly. Yeah. You can't decide between the chicken and the fish on the menu. So you ask me if I'll order the chicken and you order the fish. Do that. So hedging would go into this category in general. So I want to have an outdoor wedding. It's my dream, but I'm concerned that it might rain. I could choose to have my wedding indoors, like in a banquet hall or something like that. But notice that I lose my optionality then because I can't actually have it outdoors then. But what I could do instead is I could choose a location where I could set up a tent. And I may never use the tent. That's

• 55:53Speaker 00

  okay. Because I'm thinking about that in advance. So I'm basically paying something for the optionality because the optionality is worth something to me. So I don't want my day ruined. And if the price of the tent is tolerable to me for insuring against having my day be ruined, I'd rather have an indoor and outdoor wedding essentially at the same time in a theoretical sense.

• 56:15Chris Williamson?

  Yeah.

• 56:15Speaker 00

  And so I can do them both theoretically at the same time, even though only one of those things is actually going to occur. But I've allowed myself a future where I could do either. Right? So that's exercising options in parallel. So that also really helps mitigate the risk. And you can go much faster on decisions that are liquid and hedgeable, basically. So you can think about those two things, or where you can just straight up do two things that you think are winning at the exact same time. So you're not, then that's not really a hedge. The other thing you can think about is just generally, there's a whole bunch of decisions that we take a lot of time with that actually aren't particularly high impact.

• 56:56Speaker 00

  And if we could actually sort of step back and say on its own, separate and apart from whether there's any optionality, um, is this something that's just generally low impacted? I'm just getting hung up on it for reasons other than, um, what the impact of the bad outcome might be, or at least the long-term impact of the bad outcome, um, maybe I could actually speed up a bunch of those decisions. So, uh, so an example of that is, so I, I know a lot of people who do this. I may have sometimes done this myself. Um, people really spend a lot of time, like, you know, back in the old days when we went to restaurants and dined in, um, where people will spend like forever trying to decide between two things on a menu. Do you know anybody like that?

• 57:41Chris Williamson?

  Yes, including, including some of my good friends.

• 57:43Speaker 00

  Yes. And they're like literally, they're asking everybody like, what are you ordering? What are you ordering? What are you ordering? What are you ordering? Yeah. Like, okay, well, they're different people. They have different tastes than you do. Um, and then, you know, they'll layer it down and then they're calling over the waitstaff and they're being like, which one do you think is better? Um, uh, and it's like literally they're like taking forever on this decision. But the pro, that the reason why that's such a bad use of your time is that, first of all, you could be using that time for things that might make you bring you more enjoyment by like actually talking to the people that you're with. That'd be fun. Um, uh, but it's really more that, that the impact of sort of like not get, being quite as accurate about which dish you're gonna like and ending up with a dish that maybe

• 58:28Speaker 00

  isn't so good, or is a little bit more likely to maybe not be so good, is so small that there's no reason to be spending really any time on this decision whatsoever. And the way that you can get there is you can, I, I talk about this thing called the happiness test, um, which is to do a little time traveling to try to figure out what the impact of the decision is. So, um, if I see you, uh, if you order something and you eat it and it's terrible, I guess we do takeout right now, and you get, it's like gross and it's like yucky and you end up throwing it out. Um, and I see you a year after this happens. And I just say to you, hey, do you remember a year ago when we ordered takeout and the dish you got was like gross? Um, and I, I just say, how much of an effect did that have on your happiness today?

• 59:16Chris Williamson?

  Almost, almost none, unless it was that dish I had that had salmonella in just after I came back from Africa last year, which that that did have a fairly long lasting impact.

• 59:25Speaker 00

  Yes, so apart from food poisoning, but food poisoning is all things being equal. Because if you're at a restaurant where you might get food poisoning, that's going to be true of every single dish that you get. So, so I agree, separate and apart from food poisoning. Um, so what if I catch you in a month? You didn't get food poisoning, I was just yucky.

• 59:46Chris Williamson?

  Don't, I can't even remember it.

• 59:48Speaker 00

  Right, what about a week? Are you still like a week later, are you like, I'm so sad, my week was ruined because I had a bad takeout dish?

• 59:55Chris Williamson?

  Probably not, probably not.

• 59:58Speaker 00

  So basically what you can get out with the happiness test, and I use happiness as like a proxy for, um, in general, like, are you reaching your goals? Right, um, is that what you're, what you can see is what is the long term impact of, of sort of observing the bad result, right? Like, if I actually observe the future in which the dish is gross, or I don't like what I watched on television, or the book was kind of bad, or the movie that I went to go see kind of sucked, or I had a bad date, or whatever it is. Um, the, the shorter the time period in which I can catch you and you tell me that didn't have any effect on your happiness, the lower impact that decision is going to be.

• 1:00:43Speaker 00

  So we can see that, like, if you're making a choice about getting married, where for most people, but not all, but for most people, that would be, um, a choice to be committed to one person and not have other people involved in the equation. Um, that is going to have a pretty long term effect on your happiness. If your marriage isn't going to go, isn't going well, it's going to matter to you. Right, but if you kind of have a bad date, it's not going to have a long term effect on your happiness. So here we can separate out the things that really are going to have a long term effect on your happiness and the things aren't. And the things that aren't going to have a long term on your happiness, honestly, you spent, shouldn't spend too much time with. And in general, as we think about where are we going to allocate our decision making time?

• 1:01:28Speaker 00

  What we want to be doing is thinking about it like a menu. The, like, a really good approach to a menu is basically to sort the menu into options that you like and options that you don't like.

• 1:01:42Speaker 00

  And once you get to options that you like, that are kind of in that bucket, you can generally pretty much sort of be flipping coins among them. Um, and one of the good ways to kind of clarify that for yourself, like what belongs in the bucket of things that I like, is to apply the only option test, uh, which is essentially to, to take anything out of that bucket. So we can think about like a venue, right? If this were the only dish I could order, would I be happy with it? And if the answer is yes, you can put it in the like bucket. If the answer is no, I'd be really sad if that was the only thing I could order. You, you sort it out. So your, your decision-making time should be spent on the sorting, but not on the picking. You can think about, once things are sort of in this bucket of stuff that I like,

• 1:02:26Speaker 00

  then anything in there is just picking. And you may be under the illusion that you can separate out whatever small differences there are in expectation for any of those individual options. But we're deciding under uncertainty, right? Like, we don't have a whole bunch of information, least of all how it's going to actually turn out, which is the thing that you would want to know. So the, you're not actually probably going to be able to sort out with any fidelity, with any accuracy, what the slight differences are between the items that are in your, okay, this is a thing that I think I'm going to win to bucket. Um, so stop trying. Like, just go ahead and flip a coin once you've got them in a bucket. Now, obviously, um, the more information that you have available to you, the more, the more accurately you're modeling

• 1:03:12Speaker 00

  that, that information and that data, your bucket is going to be smaller. But regardless, that bucket is always going to hold more than one option. Right? So the less information you have, the kind of, you know, if your model isn't very exact, your bucket may be much bigger. The more information, the more data, all that stuff that you have, that bucket mu- might be much narrower. But once things are in the bucket, stop trying to pretend that you can actually split them apart. Really bad use of your time. That's where you really lose opportunities to go find other things to get into the bucket. That's where I want to be spending my time. I want to find other things that I can put in the bucket. I don't want to spend my time sorting among things in the bucket.

• 1:03:56Speaker 00

  I want to go get other things into the bucket. That's where all of my time should be spent. And then you're going to have fewer opportunities expire. When you do that. And the only thing that I would say is that once you do have things in the bucket, to your point, um, one way to decide not, instead of trying to figure out like, what's the difference in expected value, just say they're both in the bucket. Is there one that's more equitable? Can I do these two things at the same time? Right? You can ask those types of questions about optionality and that that will help you do some sorting of the things in the bucket. That may help you actually get a higher fidelity choice. But all things being equal, once they're in the bucket, just flip a coin.

• 1:04:37Chris Williamson?

  I wrote a newsletter last week, which is actually about my golden rule for decision making, which is far, far less sophisticated than the stuff that you go through in your book. But um, I found this quote, I can't even remember where it was from, that said, the quality of your life depends upon the quality of your decisions. Therefore, making precise decisions is is quite important. And the rule that I came up with was, or the rule that I tend to use, which takes the third party perspective from Shane Parrish from Farnham Street, and a little bit of stuff from James Clear, was to ask myself the question, what would me tomorrow have wanted me today to do? Like, you've got these two things. I want a cookie. I want to not go to the gym. I want to do whatever. The third party perspective,

• 1:05:20Chris Williamson?

  being able to take that that third party perspective is super useful, because it helps to give you just a little bit more distance between the present self, which we all know is just a total bastard, and the remembering self, which is actually really, really useful. So if you were to write a couple of golden rules that people can take away and can help them to make better decisions, is there stuff that people can remember to sort of keep in their mind? A couple of little maxims, any Duke's maxims that you've got that they can take away?

• 1:05:50Speaker 00

  Yeah. So, well, first of all, let me just say, your point about the time travel is really, that's really key. That's what the happiness test is doing. Right? In the moment, you're like, oh, if I get the chicken and it's bad, I'm going to beat myself up thinking I made a mistake. Whereas what this does is says, when you're looking back in a year, are you even going to care? And that allows you to get into a more rational place about, about um sort of how much time you're spending and how much worry and how much anxiety you have around that decision. Do I have any maxims? Um, that will help people with their decisions? Hmm, yes. Okay, here's one. If you want to know what somebody thinks, don't tell them what you think first.

• 1:06:32Chris Williamson?

  Why?

• 1:06:34Speaker 00

  So, one of the things that you're talking about when you say it like the remembering self is that's actually, you know, in thinking fast and slow, obviously, it's a discussion of the inside and outside view, right? So getting to the remembering self is actually a way to get to the outside view. In other words, to get outside of the things that are driving the decision in the moment, where you're driven by your own perspective and your own biases and kind of what you want to be true of the world, um, to outside of yourself by imagining yourself as an outside observer. Because that's really what you're doing when you're thinking about the remembering self is you're trying to imagine how somebody else would view this decision from the outside. So, uh, one of the most valuable decision tools that we have

• 1:07:18Speaker 00

  is to get to the outside view, to think about how, how would somebody else view this? The, the problem that I'm considering. Um, and we know that two people with the exact same data can come to very different conclusions. People apply different mental models. They have different perspectives. They may have different information than you have. They could have a different fact that would be very helpful to you. And so, uh, we'd like to, in order to become better decision makers, to live less in the inside view and more in the outside view. We wanna, we wanna be living the outside view, which I suppose is another maxim. All right, so. So, how do you get to the outside view? Well, one of the ways is to go find out what other people think. Because, um, a lot of the outside view by definition kind of lives in other people's heads.

• 1:08:02Speaker 00

  Uh, they've got a lot of the information you don't have and they have a lot of perspectives that you don't have. That's, you know, separate and apart from base rates, you should go Google. So, one of the problems that we have is that when we communicate to each other,

• 1:08:16Speaker 00

  human beings really like to be members of a tribe. And part of what creates cohesion in a tribe is that you believe the same things. To be true, we see this in politics. And so, human beings are kind of like agreement machines. If you've ever been in a meeting, which I'm sure you have, you'll notice that a lot of the meeting is spent on areas where people agree. That makes us feel good, it makes us feel like we're like on the same page in a team player. But that's true just like interpersonally as well. Right, so, you know, we've all been at like cocktail parties where someone says something that you totally disagree with and you're like, "Don't speak up." Like, you don't want to be rude. So, what you always need to be thinking about is that

• 1:09:01Speaker 00

  your beliefs can really infect other people.

• 1:09:05Speaker 00

  Yeah, so, when I tell you what I believe to be true of the world, I am much less likely to find out what you think is true of the world.

• 1:09:16Speaker 00

  When I tell you what I believe the best course of action is, you are much less likely to tell me what you think the best course of action is. Not with any fidelity. And the worst thing I could do is if I'm asking you about something that happened in the past and I'm trying to figure out whether I should have turned left or right in my video game. If I tell you I died, then I've ruined the whole thing. Right, so, you need to hide that information from the other person. And actually, what your goal is by not telling them all the things that you think or, you know, that are, or like the outcome, is that I'm trying to put you into the same state of knowledge that I was in at the time that I made the decision. Because when I tell you my belief, there's sort of a couple of things that could happen.

• 1:10:00Speaker 00

  One thing is that you could believe a different thing, but you could think I'm wrong. And so, you don't speak up. Another thing is that um you could believe a different thing, but either you don't want to embarrass me or you want to move the meeting along, or you don't have enough skin in the game to like tell me or whatever. So you just suppress it for that reason. Not not because you think you're wrong. You're actually think you're right, but you don't actually speak up. And the third thing that can happen is if I'm speaking and I tell you what my opinion is, that your opinion can shift while you hear me. And so by the time that you actually speak up, you've come to a place that agrees with me more, even though you didn't start in that place. This is obviously particularly problematic for people who are perceived to be leaders or subject matter experts.

• 1:10:41Speaker 00

  So you can imagine if I were asking for your advice on a poker hand that I played, and I told you how I played it, I said, the the person in front of me raised, and I tell you all sorts of details about the person that you would need to know. In other words, the knowledge that I had at the time. And then I said, I looked down and I had an ace queen. And so I raised him back because I'd seen him play and I felt like uh he was probably pretty weak because he'd been playing a lot of hands. And so I felt like A, I probably had the best hand and B, I thought there was a good chance that he was gonna fold and so on and so forth, which is really how we communicate to people because somehow we think that my opinion is really valuable data for you to have. What do you think's gonna happen now?

• 1:11:25Speaker 00

  If you didn't think that if your your instinct was to call, you go, I must be wrong. And now I actually don't get to hear that opinion from you.

• 1:11:33Chris Williamson?

  Interpersonally, that's a really interesting challenge for people to come up against, I think. It's something that I learned during the process of this show and I brought up a couple of times with uh communication experts that allowing a short question to sit during a conversation is really uncomfortable. And I noticed when I was talking to people that there would be this sort of weird after effect echo thing. I'd say a question and then and then ask a question and have this, what's this trait? What's all that? What was all of this like extra words that I needed to say at the end of it? But I think sometimes, especially if you were speaking to someone who perhaps was an authority

• 1:12:16Chris Williamson?

  in a particular subject field, or even just someone whose opinion you respected generally, you would ask a question and then Clarify, I don't just do something that dampens the impact of sitting in silence, because we hate silence. And if it's silent, it's because I have nothing to say and you think I'm boring. And I'm a did it. Oh, my God, I better say something. And you get it out. And that totally ruins the end result. And the end result is to try and get this other person's point of view to replicate the situation that you were in. So you can iterate the learning experience so that it's replicable. So that you actually understood it and all that stuff.

• 1:12:50Speaker 00

  Right. I mean, you kind of think about it this way. It's like, if I could like, create a map of the things that you believe and the perspectives that you have. And I also was able to map my own perspectives and beliefs and knowledge and facts and so on and so forth. And I laid those on top of each other. There are places where those maps would be totally overlapping. We both agree that the earth is round. Yeah, yes. But then there's places where, where it would diverge, right? So you can think about like laying your hands on top of each other, right? And, you know, the places where the fingers aren't lined up perfectly would be places where we have a dispersion of opinion. And the palms would be places where we agree.

• 1:13:29Speaker 00

  We live our lives trying, basically forcing more agreement than there actually is, or like creating the illusion of lots of agreement. And that's because when we talk to people, we talk in a way, we do two things. One, we speak to them in a way where we're much less likely to hear the dispersion in a real way to find out that they have a different opinion than we do. Number one. And number two is, we'll also talk in ways that are loose enough that the target area is so broad that while we could have very different opinions, it wouldn't appear so.

• 1:14:06Chris Williamson?

  Colloquial sort of changeability, almost.

• 1:14:09Speaker 00

  Right. So, you know, that an example would be, well, I mean, the easiest way to think about it is like, when you use natural language terms to describe probabilities. So if we both say, I think that we should invest in this stock and there's a real possibility that we're going to win. And you say, yes, I agree, there's a real possibility we're going to win. We didn't actually find out what both of us mean a real possibility is. And Andrew Mobison and Michael Mobison actually did like this great survey where they actually just had a whole bunch of people online like answer when I say this word like there it's a real possibility it's going to rain tomorrow. What probability do I actually intend when I say that? And it turns out somewhere between 20 and 80% of the time and it's a pretty even

• 1:14:53Speaker 00

  spread across. Strangely.

• 1:14:57Chris Williamson?

  It's so useless.

• 1:14:58Speaker 00

  Yeah, it's like they're totally useless words, but think about how often you're using those kinds of useless words, right? And that's what I said about like a good decision process breaks things into the component parts. Right? Like you have to sort of think about precisely what does it mean? If I think that I'm supposed to invest in this stock, what are the things that I mean? That I can sort of pull out of there that I think the market doesn't know that I do?

• 1:15:26Chris Williamson?

  The specificity is important, right?

• 1:15:28Speaker 00

  Specificity is really important because I'm trying to find out maximally where we have a dispersion of opinion, which I can't do unless I'm really specific. So first of all, I have to say, what are the things that for me to make this investment? What are the things that I think need to be true of the world in order to make a winning decision? Right? But so, so I mean, you can literally regress on this. So I might define that and then just ask you to tell me in very precise words. You hopefully using some kind of precision like a scale of zero to five where we can see that I'm a four in year or two, actually forecasting a probability.

• 1:16:11Speaker 00

  If it's like, I want to know how many widgets you think that they're going to produce, you would give an estimate with a lower bound and an upper bound instead of saying a lot or something like that. Right? If you, if you're saying like, I really think like, I know, I know that the unit economics aren't great, but the market opportunity is huge. Okay, but what does that mean? How big does the market need to be in order to balance out the unit economics? Can you specify that for me? Right? So the more specific that we can get, so instead of saying, I think there's a real possibility that it's going to rain, what I want to know is what probability do you actually intend? Like what, what's the probability it's going to rain? And I'm going to let you give me a lower bound and an upper bound.

• 1:16:54Speaker 00

  I don't care because that's going to help me understand how uncertain you are, like how much uncertainty there is in your forecast. So, um, so here's how a great conversation would go. Um, what do you think the chances of rain are tomorrow?

• 1:17:10Chris Williamson?

  In the northeast of the UK, probably about ninety percent.

• 1:17:14Speaker 00

  Right. So, great. And hope, and I've hopefully written down my forecast myself. What I don't want to do is say to you, I think there's a seventy percent chance it's going to rain tomorrow. What do you think? I definitely don't want to do that. And worse yet, I don't want to say, I think there's a real possibility it's going to rain tomorrow. What do you think? Because you'll, you'll probably just endorse that. Because if it's real possibility, I could be thinking forty percent and use the term real possibility and you, you think it's seventy percent. And now, you know, or ninety percent or whatever, and now we thought we agreed when we actually didn't.

• 1:17:50Chris Williamson?

  I'll tell you what's, what's an interesting.

• 1:17:52Speaker 00

  I think that's really important.

• 1:17:53Chris Williamson?

  It really is. Um, there's an interesting insight I had with a, a, an author who was looking into how to have productive disagreements. And he talked about the danger of overusing analogy. And that's the linguistic equivalent of what we're talking about here. That if it's, well, you know, when the rubber really meets the road with this, you know, as soon as you start to abstract yourself away from the conversation and the real talking points, that is a very manipulative, very effective manipulative tool. And he said that a lot of the time, um, people that talk in hyperbole, so use Trump as an example, you know, spectacular, fantastic, wonderful, majestic, all of this sort of stuff.

• 1:18:35Chris Williamson?

  And when you start to create, um, use these analogies, you no longer can actually pin down what someone's talking about. You know that he's not talking about the frog trying to cross the road. Like, it's not, it's not about that. That's a proxy for what he's meaning that he's talking about. Um, and yes, it's so easy for us to tumble down this, right? Where.

• 1:18:56Speaker 00

  So I, yeah, I do this with my clients all the time where, where they'll say something like that, right? Where, well, where, where they'll say, uh, I think we should do this, even the, the economics aren't great because it's going to really increase the value of our investment pipeline. And I'll say, hold on a second. I don't know what that means, what you just said. So I need to, and literally, it's you have to write down how, what, how much of an increase in the pipeline, unexpected value, you have to talk about unexpected value. You have to forecast that. And then you have to tell me, how much are you losing to the things that you don't like about the deal? And you have to do that. And then all of your partners have to do that, right? And then now you can start to compare.

• 1:19:41Speaker 00

  And you can start to, to break it out and say, I don't care if you use this word spectacular, but I'm gonna pause. And I'm gonna say, what does spectacular mean? Right? Because you can think about like, for example, like, a spectacular job on handling coronavirus might be, um, that, uh, you know, one out of every a hundred million, a hundred thousand people died. But somebody else could describe spectacular as one out of every 10,000 people died. I don't, I don't know what your spectacular is. And the problem is, it's not just in terms of us trying to have real conversation where we can sort of dig into this stuff in a pro- productive way

• 1:20:25Speaker 00

  in order to inform each other of our opinions. It's that we always have to think about that look back. So if you say spectacular, and then I try to call you on it later, e- even if you didn't mean that at the time, you get to say, no, that, that's what I meant by spectacular. Okay, but that's because we didn't know, I don't, nobody really defined the term. So of course you can, as, as Phil Tetlock would call them weasel words, right? Like you can sort of weasel out of any kind of accountability to what the forecast is that you made. When you, when you have these very broad target areas, because you can just kind of like, move around like, oh no, I meant the, you know, over here. Which, you know, and I, I think that we need to not allow ourselves to do that.

• 1:21:11Speaker 00

  The second thing is that, in, in terms of having really productive disagreement, I think we need to stop thinking that disagreement is bad.

• 1:21:20Speaker 00

  This agreement is amazing because the whole thing about this problem of incomplete information is that I don't care what your decision process looks like. If there is a problem with the beliefs that are informing your decision, it doesn't matter what happens after that because your forecasts are going to be off. The things that you think your goals are going to be wrong, how much any particular option is going to advance you toward your goal or cause you to retreat away from it, that's going to be ill-informed. What the options are that you think are available to you are going to be ill-informed. You know, I mean, it just all falls apart if we don't have our beliefs in order.

• 1:22:02Speaker 00

  And the thing about our beliefs is that like what we know fits on the head of a pin and what we don't know is like the size of the freaking universe. So we better start walking around and taking these walks through that universe of stuff that we don't know in order to fix this problem, which is that we know so little. And if we can just increase the accuracy of what we believe and broaden that knowledge just a little bit, oh my gosh, are we going to be in amazing shape? And the issue is that we do not interact with that world in order to maximally, maximally come across corrective information. We actually interact with that world in order to minimally come across corrective information. And that's a freaking disaster.

• 1:22:46Speaker 00

  That's just a disaster for our decision-making. So I need to know what you believe. And when I find out that you believe something different than me, I am like, oh, thank you. And now we get to talk about it. And I think that one of the ways to get to a place where we can have really, really productive discussions around dispersion of opinion is to realize, first of all, that you should have gratitude that you exposed it because that allows you to become a better-informed decision-maker. And I'll circle back to that again in a second. But the second thing is that you need to realize that there is no reason that you need to agree at the end of the interchange. And I think a lot of the stress and a lot of the avoidance of disagreement that, that

• 1:23:27Speaker 00

  we end up with is because we believe that the goal of the conversation is to agree at the end. And that's poppycock because you have your models and I have mine. You have your beliefs and I have mine. And isn't that wonderful? Because that's what a freaking market is. A bunch of people have different beliefs and then they trade on their beliefs. And then you sort of, the market sort of finds out what's true. And hopefully through that process and through you observing all these people trading on their opinions. Right? Um, that you start to get your opinions to be a little bit more accurate. Recognizing that none of their opinions are probably exactly right either. Right? So you're just trying to sort of see everything from lots of different sides. And I don't need to end up coming around to your side. You don't need to end up coming around to my side.

• 1:24:11Speaker 00

  Because the goal for both of us is to just become better informed. I want to be informed of the rationales for why you believe what you do. And you want to be informed of the rationales for why I believe what I do. Because only good stuff comes from that. And there's like, basically three things that can come from that. Thing number one is, we both moderate our opinions. Yes. We don't necessarily come all the way to agreement, but you know, you, you shave a little here. I shave a little there. We, we're maybe get to a place that's a little more accurate. Um, so that's one thing that could happen. Another thing that could happen is, let's say, Chris, you're right and I'm wrong. Alright. I now have the opportunity with something that's where I need to do a 180. To do as much of a 180 as possible. Right?

• 1:24:56Speaker 00

  So I may be able to reverse a belief that I have that's actually wrong. And man, should I be thankful for that? Because that, that belief, as painful as it is to find out that I'm wrong in the moment, is going to inform every single decision that I ever make again in the future. So I will trade the pain of whatever it is to find out I'm wrong right now. And be incredibly grateful to you that we exposed that. Because now all of my future decisions are going to be better off for it. But here's the interesting thing. It may feel like when you're right and I'm wrong, that the only person who's benefiting is me. But that's not true. You're getting a tremendous amount of benefit out of the conversation. Because you have to convey to me, who believes a different thing than you. Why the thing you believe, you believe so strongly?

• 1:25:41Speaker 00

  And what happens when you have to explore it in that way? That my job, I'm, I have to convince you. My job is to convey why I believe what I do. And that the person that I'm talking to can say, but why? Okay, I get this part of it, but I'm not understanding this part of the rationale. Or what does the rubber meet the road actually mean? When you say spectacular, what does that actually mean? And I'm allowed to quiz you on that, not in a way where I'm having an argument with you, just out of curiosity. I want to really understand why you believe what you do. The act of you having to give me a good rationale for the thing that you believe allows you to believe, to know your beliefs better. Because what happens is that we give ourselves a pass.

• 1:26:23Speaker 00

  We think we know things really well, but when we're forced into a situation where we must actually convey, like where we have to teach, we very often find out that actually there's a whole bunch of crap. I'm not exactly sure why I believe that. This could be even if you're true. Like the example that I like to give is, I really believe the earth is round. But I can tell you that if I were having a conversation with a flat-earther, I would stink at explaining why. So do I really know it?

• 1:26:51Speaker 00

  Like, I know it because I believe that all the scientists say so, and I've seen the pictures and stuff. And it seems to me that that, you know, just from a logical standpoint, it would make no sense that we could keep this conspiracy going for so long that the rational thing to believe is that the earth is round. But yet, I would not be able to convey it to someone who really believes something different. I would have to go and Google it. And in the act of Googling it and sort of figuring out like, what are the scientific experiments to tell you this? And how can you tell if photos are doctored and things like that? I would come away from that, understanding my belief much, much better. And it's only through that process that I get to do that.

• 1:27:32Speaker 00

  So, look, I want to, I want to expose when somebody has a different opinion than me, because that's where all the fun stuff happens.

• 1:27:38Chris Williamson?

  It is indeed, it is indeed. Look, Annie, today's been really fun. How to Decide: Simple Tools for Making Better Choices will be linked in the show notes below. Anywhere else that people should go to check out your stuff?

• 1:27:49Speaker 00

  Yeah, well, you can go to annyduke.com.

• 1:27:53Speaker 00

  There's a contact form there. I love to hear from people who have read my stuff or heard me speak on podcasts or whatever. Actually, I would say that How to Decide, I wrote How to Decide because of people who wrote to me. And were asking me about Thinking in Bets. And through those conversations with them, I realized that people who read Thinking in Bets were kind of like they were craving something more practical. You know, like, how would I actually, how would actually create a good decision process? How would I do that? And that's, that's why I wrote the book was because of conversations with people who'd heard me or read my stuff or whatever. So I love to hear from people. So please go to AnnieDuke.com.

• 1:28:38Speaker 00

  Please, you know, you can follow me on Twitter at AnnieDuke. And then the other thing I would love for people to check out is the Alliance for Decision Education, which is a nonprofit that I co-founded, which is trying to take all the stuff that like, we've been talking about, you know, from James Clear to Morgan Housel to Phil Tetlock to Daniel Kahneman to, you know, so on and so forth. And say, there's all this work for adults on how do you become a better decision maker? How do you shape better habits? How are we thinking about what's true and not true? How do you think probabilistically that we know? All are the things that are going to create a better life for people. But where is that in K through 12 education? It's just absent.

• 1:29:23Speaker 00

  So what we're trying to do with the Alliance for Decision Education is to really bring decision education, this, this body of work that you're talking about and start to pull that down into curricula for K through 12. So that we can start teaching kids, you know, from the start, how to become better decision makers. Because, you know, our motto is better decisions lead to better lives and a better society. And we really, really believe that. And so I would love if people could go check out that nonprofit.

• 1:29:52Chris Williamson?

  Everything will be linked in the show notes below. Annie, thank you so much for today. It's been really fun.

• 1:30:09Speaker 02

  Mr. President, my company tore me of the explosion on our rig. On October 2nd. Causing massive water displacement and triggering a tsunami. They all keep the blame on me. Tom Cruise is. Digger got us into this mess, and Digger's gonna dig us out. From the director of Birdman and The Revenant.

• 1:30:26Speaker 00

  What would happen if I stopped going? You stall, you freeze to death, that's what.

• 1:30:30Speaker 02

  With great power comes great deniability.

• 1:30:33Speaker 00

  I'm not the first then.

• 1:30:34Speaker 02

  Digger. See the trailer online. Only in theaters October 2nd. Rated R. Under 17. Not a minute without parent.