Ep. 1397: Algos to Live By with Michael Covel on Trend Following Radio

Please enjoy my monologue Algos to Live By with Michael Covel on Trend Following Radio. This episode may also include great outside guests from my archive. --- I'm MICHAEL COVEL, the host of TREND FOLLOWING RADIO, and I'm proud to have delivered 10+ million podcast listens since 2012. Investments, economics, psychology, politics, decision-making, human behavior, entrepreneurship and trend following are all passionately explored and debated on my show. To start? I'd like to give you a great piece of advice you can use in your life and trading journey… cut your losses! You will find much more about that philosophy here: https://www.trendfollowing.com/trend/ You can watch a free video here: https://www.trendfollowing.com/video/ Can't get enough of this episode? You can choose from my thousand plus episodes here: https://www.trendfollowing.com/podcast My social media platforms: Twitter: @covel https://twitter.com/covel Facebook: @trendfollowing https://www.facebook.com/trendfollowing/ LinkedIn: @covel https://linkedin.com/in/covel/ Instagram: @mikecovel https://www.instagram.com/mikecovel/ Hope you enjoy my never-ending podcast conversation!

• 0:00Speaker 00

  My special July 4th offer, a brand-new product, is underway. It started July 4th, but it's still open. You can still jump in right now. Go to trendfollowing.com/join. Again, that's trendfollowing.com/join, J-O-I-N. And to put you in the headspace of this special July 4th offer, of course, I've got one of my favorite Trend Following archival guests.

• 0:40Speaker 00

  The kind of mind, the kind of mindset that puts you in the headspace. And once you're in that headspace, if you want that hand-holding and you want to go the extra mile, my brand-new product is available right now. Again, trendfollowing.com/join. I hope you enjoy this episode.

• 1:16Speaker 00

  So this great book came across my desk, Algorithms to Live By: The Computer Science of Human Decisions. Let me give a brief overview from the website. "A fascinating exploration of how computer algorithms can be applied to our everyday lives, helping to solve common decision-making problems and illuminate the workings of the human mind. All our lives are constrained by limited space and time, limits that give rise to a particular set of problems. What should we do or leave undone in a day or a lifetime? How much messiness should we accept? What balance of new activities and familiar favorites is the most fulfilling? These may seem like uniquely human quandaries, but they are not.

• 2:01Speaker 00

  Computers, too, face the same constraints, so computer scientists have been grappling with their version of such problems for decades, and the solutions they've found have much to teach us."

• 2:14Speaker 00

  Again, that is from the book Algorithms to Live By by Brian Christian and Tom Griffiths. And my guest today is one of those authors, Brian Christian. Oh, and by the way, if you invest, or if you are a trader, it's a must read. Now, on to Brian Christian.

• 2:40Speaker 00

  How in the world did you even start to go down this path with your co-author, really?

• 2:46Speaker 01

  I've really been going down this path since my, really since my teenage years. When I was in college, I was a double major in computer science and philosophy. You know, in some sense, each, each way of looking at the world started to permeate the other. And I began, you know, thinking of computer science in philosophical terms, but also thinking about computer science as a way to approach the problems that I was facing in everyday life. The way that we would turn to a discipline like philosophy for advice on, you know, the, the problems of the world, the things that we're grappling with. I actually found that computer science offered me in many ways, an, an as compelling

• 3:29Speaker 01

  or more compelling framework, and really a vocabulary for kind of understanding the problems around me. And so, you know, this is something that I've been thinking about for a very long time.

• 3:40Speaker 00

  My background, having written several books on trading, I was just stunned to go through your work because, and I don't know if anyone said this to you yet, but if somebody is a trader, an investor in 2016, and they're serious about it, this is almost foundational, your work. Now, maybe that was not the intent, but in my world, so many of the quant traders, and I'm not talking about the high-frequency stuff, but the, the kind of the longer-term trend traders that have 100% automated strategies, all backtested, finding a positive mathematical expectation, understanding overfitting, understanding robustness, having it all automated. And to read your work, it's like, oh, wow, okay.

• 4:22Speaker 00

  Whether intended or not, you guys just laid down the foundation for traders today.

• 4:27Speaker 01

  Yeah, well, thank you. I mean, that's not an area that I know especially deeply, but it's certainly, I mean, I'm certainly aware that we're kind of living in this time in which a lot of the, you know, a lot of the decision-making that's going on in the world, finance being a great example, is increasingly being, you know, executed by algorithms. You know, there's this increasing need to understand just some of the larger ideas and the real principles of the field of computer science as a way of kind of making sense of how these things operate. And I think there's also, there's a real opportunity

• 5:08Speaker 01

  to bring some of those principles back to our own decision-making processes in our own lives. And so that's, that's really where I see this book coming in, is it, it kind of completes this feedback loop where for the last 50 or 60 years, you know, algorithm designers have begun by asking themselves how they would solve a problem. And that's often, you know, any programmer's first draft of a piece of code that they're writing. But of course, as the field has matured, we've developed this formal rigor around being able to make assertions and make proofs around optimality and expectation and bounds

• 5:50Speaker 01

  and so forth. What we haven't seen is bringing that knowledge back around to improve our own intuitions. And so that, that's where I see this book coming in, as kind of completing that circuit.

• 6:04Speaker 00

  Let's talk the word "algorithm." For people that know, they'll hear your words, they'll read your work, and they'll be like, "Okay, it makes perfect sense." But for a lot of people, I know the word "algorithm" immediately brings about fear and confusion. And why don't you give a little of a historical perspective? Because an algorithm is just really a way to take things that we could do as a human being on a piece of paper with a pencil and taking it to the land of computer science, to the land of computers. So it's, why don't you give a little bit of a narrative there? Because it's not necessarily so complicated. It's, I think sometimes it's the vocabulary that causes people to pause.

• 6:42Speaker 01

  Yeah, I think that's true. You know, it's a word that today we associate specifically with the world of computer science, but the principle of what makes an algorithm an algorithm is it's really just a series of steps that you follow to solve a problem or make a decision. You know, algorithms in this sense are much older and much broader than the computer, of course. And, you know, we all in school learned, for example, algorithms for doing long division, or, you know, you subtract by carrying the one and so forth. But, you know, it goes back even farther than that. I mean, you can think of cooking

• 7:23Speaker 01

  following a recipe as being algorithmic in that sense. And so, you know, mathematical procedures have been found on clay tablets dating back thousands of years. And so, in many ways, they are not only older than the computer, but they are some of the first and oldest technology, if you will, of the human race, at least as old as the Stone Age.

• 7:48Speaker 00

  So let's bring people into an example. I was thinking of one in my life inspired by your work, and perhaps you can take my example and run with it and pass along the lessons. Years ago, I was working on a film, a documentary film. I just moved to San Diego. I needed to hire a film assistant. I put a Craigslist ad out. I think approximately 30 female candidates showed up. The very first one was the most competent, but she scared me. I just moved to California. She was in a pantsuit. I don't know. She was, she just scared me, and I didn't want to hire her on the spot. So I interviewed 29 other ladies, and they were just all not anywhere near her level. And then, in fact, the very last candidate, I can't even make this up. I laughed to myself about the very

• 8:33Speaker 00

  last candidate, as we were leaving, attempted to kiss me on the lips as an exit interview.

• 8:40Speaker 01

  Oh, my gosh.

• 8:41Speaker 00

  So I went back and I did hire, I hired the very first lady. And as it turns out, she ended up going to USC's Film School and has a successful career now in LA. So it was a good choice. But I wasn't really using any kind of an algorithm probably described in your work. I was probably doing everything wrong.

• 9:05Speaker 01

  Well, you certainly found yourself in one of the canonical problems of the 20th century, which is called the secretary problem. And it is almost exactly as you describe. You know, the canonical setup is you're hiring for a position and you get, you know, N candidates. So in your case, 29 or 30 show up in a random order. At each step of the way, you evaluate the next candidate that comes in. And you have to either hire them on the spot and send everyone else away and never know if there might have been a better candidate out there. Or you dismiss this candidate, in which case, in the classical version of the problem,

• 9:46Speaker 01

  you do not get the ability to call them back later and try to bring them back in. The crux of this problem is, how do you try to, of course, hire the very best person for the job without knowing in advance who that person is going to be? And at each step of the way, you're faced with making this irrevocable choice. Do you take the person in front of you and not know what else is out there? Or do you send them away, but maybe you've just blown your best opportunity? This field in general is known as optimal stopping. And in the book, we sort of unpack the structure of optimal stopping problems and show that

• 10:29Speaker 01

  there are really a whole host of situations in life that take this form. And so, one of the examples that we give is real estate. If you're looking to buy a house or rent an apartment, in most competitive markets, you are faced with exactly this kind of a dilemma where you, let's say, you go to an open house and people are submitting bids pretty much on the spot. And so, you have exactly this dilemma. How do you know when your best opportunity is without the ability to gather information and synthesize it before making a choice? And the classical version of the problem has this very famous solution,

• 11:10Speaker 01

  which is that you should refrain from making any offers for the first 37% of the applicant pool, and after that point, instantly extend an offer to the first person you see who's better than everyone you saw in the first 37%.

• 11:27Speaker 00

  And it's very similar in thinking to something like a mathematical expectation-type formula where you're just saying, okay, you're not trying to say you know the absolute best, you're saying, it's the best for right now, let's make a decision, let's move on.

• 11:41Speaker 01

  Yeah, I think there's a deeper point here about this problem, which is that even following the optimal solution, A, it does not guarantee that you're going to get the optimal result every time. What it can offer you is, in this case, the best probability of getting the best outcome. And I think at a slightly deeper level, what it can offer you is really peace of mind, that usually when we make a decision and we don't necessarily get the outcome that we want, the first thing that we do is kind of play back the entire thought process that led to the actions that we took, and sort of call into question everything that we did along the way. And there's something, I think, very consoling

• 12:22Speaker 01

  about having a mathematical or a computational understanding of the form of the problem that you're facing, which is that it enables you to be confident even when you don't get the result that you want. You can still know, hey, I followed the right procedure, I followed the right thought process in this case. Even if I didn't get the outcome I wanted, I don't have to call all that into doubt, because I know that I followed the right process.

• 12:52Speaker 00

  Process versus outcome, they're not, I think that's one of the biggest issues we have in humanity today. We can have that great process, but we all still fixate on the outcome, whereas we should be saying to ourselves, let's just follow a great process and know that we're going to have errors, we're going to have mistakes. But if we keep following the good process, we're going to be in a much better place in the long run.

• 13:18Speaker 01

  I think that's exactly right. An understanding of the the structure of the problem that we're facing, the basic characteristics of the optimal solutions, I think is a really powerful starting point in order to get a sense of what those processes should be.

• 13:33Speaker 00

  I'll give you an example in my world, too. When you look at some of the traders today, let's say they have a 100% automated trading system and they'll take, they'll get a signal that says buy. Well, you know, most of the, most of the time, 60% of the time, perhaps that buy signal turns around, goes the other direction and you have to take a small stop. As a human being, though, if you're just sitting there making the discretionary decision, you call into question, did I make the right move? Did I do the right thing? The computer just says, hey, I'm going to follow the rules and we know in the long run, we want to get out if it doesn't go our direction. It's very, very similar to what you're outlining in your work.

• 14:05Speaker 01

  Yeah, I would add also that, you know, we, there are some people who are skeptical, for example, that the the structure of a classic optimal stopping problem really does describe the way, the types of decisions that we face in everyday life. I mean, there is, there is this very strict assumption in, in the problem that you, that each, at each moment in time, you have this irrevocable choice. And in your case, you know, you, you encountered a, a real life example where the, the assumptions of the model were a little bit relaxed, you know, in, in the case of making that hire, you were able to go back and extend an offer to someone, even if you didn't hire them on the spot.

• 14:49Speaker 01

  In fact, there are optimal algorithms for this version of the problem, too, which is known as recall. Trading, I think, offers a wonderfully stark example where, you know, as you describe, at each moment in time, you have this extremely fleeting and ephemeral opportunity, you must make a decision on the spot. If you buy, then you can't

• 15:16Speaker 01

  You can't reverse that without some pain, right? But also, if you fail to act, then the opportunity may go away and you don't have the opportunity to rectify that also. In fact, you know, many aspects of the financial world are implicitly modeled using optimal stopping under the hood. I know that American stock options, for example, you know, if you want to assign a value to the option to buy something or the option to sell something, how do you assign a value to that? Well, implicitly, you have to make some assumptions about, you know, whether that

• 15:56Speaker 01

  person is going to be able to know the correct time, the best time to exercise that option. The mathematics of optimal stopping are at work under the hood of some of these things that are familiar to traders.

• 16:09Speaker 00

  As you talk, as you explain, I think back to my MBA program and I say to myself, I thought I really only learned two things of great value in my life, the definition of a sunk cost and an opportunity cost.

• 16:20Speaker 01

  Yes, absolutely.

• 16:22Speaker 00

  Yeah, it's right, it's right there in that, right in the same headspace. Let me let you tear apart another word. You mentioned it earlier and, you know, plenty of people are going to know, but probability is a sticky wicket for people. But why don't we, let's talk about probability because, again, it's the process versus outcome thing and it's, it's like whether we're talking poker or the MIT blackjack team, trading, you know, VC financing, probability is, is really the success foundation of so many businesses today. And I don't think that's necessarily properly understood by so many people that don't have the training or the understanding.

• 17:01Speaker 01

  Yeah, I think, you know, furthermore, one of the, one of the key things that compounds the problem of, of dealing in probabilities is we're also dealing in uncertainties, that you might have, you might have information about two things and you're trying to decide between them, but not only is the information probabilistic in nature, it's also uncertain. So to give you a concrete example, you know, let's say you, there are two airlines that you fly, one of them you fly more than the other. Let's say, one of them has been on time nine out of 15 flights, so 60%.

• 17:41Speaker 01

  The other one, you've only flown twice and it was on time once and it was late once. So only 50% performance. Simply put, you know, you're you're trying to decide what's the smarter airline to take for your next flight. There's one that's been on time 60% of the time, nine out of 15. The other's been on time 50% of the time, one out of two. But in this case, it's compounded by this other factor, which is that you just don't have as much information about the second airline as you have about the first. How should we try to make the best decision when we're comparing these two two different quantities? There's a famous problem in computer science called the multi-armed bandit problem, which

• 18:25Speaker 01

  is sort of an odd name, but it comes from the idea of the slot machine as the one-armed bandit. And it's exactly this. It's like you you walk into a casino, there are all these slot machines that pay out at different rates with different probabilities. But crucially, you don't know which machines pay off at which rates and which are better than others until you start playing them. And I think that's something that's very true in many aspects of life. And so there the the critical thing to a computer scientist is what's called the explore-exploit trade-off. So you you have to very consciously trade off how much energy you're going to spend gathering information and how much of your energy and your time and your money you're

• 19:10Speaker 01

  going to spend leveraging the information that you've already gained to try to get the best outcome that you can. And I think there are so many situations in life where we have to trade off in exactly this dimension. I mean, careers being one example where, you know, you have people weighing, you know, is it is it worth trying to start over in a different career track? Or is it worth going back to school to get this additional credential and things like this? Is it worth exploring this new business possibility? Or should I just sort of hone and refine the thing that I have? And in fact, you know, mathematicians and computer scientists that have done Some of the most important work on the multi-armed bandit problem and the explore,

• 19:55Speaker 01

  explore trade-off have done a lot of it for companies that were explicitly trying to decide how to allocate their their resources between these more exploratory things and the the things that that have a known payoff. So the one of the best examples was that in the 1970s, the Unilever Corporation pharmaceutical company hires this mathematician, John Gittens, and they say, you know, tell us what percentage of our budget we should be spending on R and D for experimental drugs and what percentage of our budget we should spend, you know, on just marketing for drugs that are already profitable. And so this gives us a formal framework for thinking about what are otherwise kind of apples and oranges types of comparisons.

• 20:40Speaker 00

  I love the one expression used in your work, and I could see it as a book title in the future. Win, stay, lose, shift.

• 20:47Speaker 01

  Yes. So this was this was one of the very first results that came out in this area. So the the multi-armed bandit problem has this really colorful history in in the history of mathematics where for most of the 20th century, it was considered unsolvable, not just a difficult problem, but literally unsolvable. And in fact, the the Allied mathematicians in Great Britain during World War II joked about dropping the multi-armed bandit problem over Germany as as the ultimate instrument of intellectual sabotage. But to the field's own surprise, they started making a series of gradual breakthroughs in the second half

• 21:27Speaker 01

  of the 20th century, kind of chipping away at the problem. And one of the first was this finding that this very simple algorithm called win, stay, lose, shift was better than just playing at random, which is a very modest result. But it was the it was really the first key stepping stone towards the full solution of the problem. And the basic intuition behind win, stay, lose, shift is just if something worked, do it again. If you flew the airline and it was on time, fly them again the next time. If you, you know, went to a conference and it was, you know, Profitable or otherwise kind of worthwhile, go again. If you went to a restaurant and you had a good meal there, go back.

• 22:09Speaker 01

  But the lose shift part of it means that as soon as you don't get a good outcome, try something else. So as soon as your airline is late, then try the competitor the next time.

• 22:22Speaker 00

  You can see why I love this because I'm thinking immediately, again, in investment terms, and I'm thinking, ride your winners, cut your losers.

• 22:29Speaker 01

  Yeah, exactly. And so it's a great example of what is just this wonderfully intuitive and succinct policy that we're actually able to say, with the full rigor of the field. There are some really positive bounds that we can put on this. We can sort of certify that it's a reasonable approach. It's not necessarily the optimal approach, but it's pretty good and it's extremely easy to implement.

• 23:01Speaker 00

  You know, the win, stay part of it is momentum, almost. It's kind of like, just follow the momentum. You don't necessarily know why. You might not be able to answer the why, but the win, stay, stay with it. And there's a lot of value to that. And that's very counter to human intuition, I believe.

• 23:17Speaker 01

  The basic logic, at least if you're thinking about the formal version of the problem, is, you know, if at time T, let's say, it made sense to go with a certain choice, a certain restaurant, a certain airline, you name it. If it already made sense to do that, and then you do it, and it pays off, well, it can only make more sense to do it again, because it can only raise your estimate of it paying off in the future.

• 23:46Speaker 00

  Let me say for the audience right now, too, since I could literally sit here and ask you questions for hours, I'm not going to get you for hours. But even if I talked for hours, I would not even come close to covering 5%, 2%, 1% of what's in this book. This is extremely dense. And so I really, I do highly recommend it to people to jump in. It's just a fantastic foundation. That said, I want to jump around a little on you, and I love one example, one story. And you can you can keep tying it into where we're going. But you talk about Jeff Bezos and you talk about Jeff. Jeff is an early in life at an investment firm, looking forward in life and then imagining looking backward and thinking about that from an algorithmic standpoint. Why don't you explain that story?

• 24:31Speaker 00

  I think it's a fantastic way to bring people into your thinking.

• 24:35Speaker 01

  Yeah, absolutely. So Jeff Bezos, of course, we all know him now as the CEO of Amazon. But he started off in investment banking. He was at a very prestigious and successful firm. I believe it was D E Shaw and he was working in New York City. And he has this inkling, as he explains to his boss at the time, that he wants to leave this investment firm and start an online bookstore in Seattle, which at the time just seemed like, you know, insanity of why? Why would you leave this extremely lucrative, successful position to do this thing that to pretty much anyone at the time seemed like just a total, total lark?

• 25:19Speaker 01

  You know, this, this thing that was just so unlikely to work. And the way that he found himself framing that decision in his own mind was what he calls a regret minimization framework, which I think is a really lovely approach to, to decision making, where, as you say, he projected himself forward in time and imagined himself at 80, looking back. What he realized was that he would not regret having tried, even if he failed, but, but he would regret not having tried. And so from that perspective, it just made the decision incredibly easy for him. And of course, we all know what happened after that.

• 26:02Speaker 00

  Well, you know, if you put aside even the, the current day, Amazon's success, I really do think this, this idea of the regret minimization framework, I've never thought of it that way. And I've really, it is, it is quite beautiful because if you're a young person or whatever age you are, to say with the limited time that I have left, and that's always uncertain too. But to try and look ahead and say it that way, it changes the dynamics so much because it really, if you're honest about it, it forces you out of the comfort zone to say, "Oh, wow, what am I afraid of, really?" Because if I'm going to get to 80 and I'm going to look back, I don't want to just say, "I just clocked in every day for 50 years and I got a gold watch." It really is a, it's a great way to think.

• 26:47Speaker 01

  Yeah, absolutely. And it's, you know, it's a little bit of a memento mori of sorts. It's kind of a lights of fire. But also, you know, as you say, I think it's a really good analytical framework as well. Part of what I think is so lovely about this idea of regret minimization is that it has this formal parallel as well within the field. There is explicitly a class of algorithms that are called regret minimization algorithms. For example, in the multi-armed bandit problem, you can formalize the notion of regret. You know, so in this problem, you walk into this casino, there are all these different slot machines, but you don't know which ones pay out more than others until you start trying

• 27:30Speaker 01

  them. And so you can, you can formalize the notion of regret as the money that you could have made if only you knew at the beginning of your time in the casino what you knew by the end. And so you can assign regret this very specific number. From there, you can ask all these sort of interesting mathematical questions. For example, how does your regret grow over time as a function of following, you know, the best possible strategy? As it turns out, following the optimal algorithms in this problem, your regret will grow, specifically it will grow logarithmically. The bad news is that even following the best possible

• 28:12Speaker 01

  strategy, you are still going to make mistakes. You're still going to continue making mistakes all the way through your life. But the frequency and the intensity of those mistakes will go down and it will keep going down as you go through your life. And so that is something that I think is oddly consoling or very sort of heartening. Specifically, there are a set of algorithms called regret minimization algorithms that can offer this guarantee or this bound of logarithmic regret in that problem.

• 28:48Speaker 01

  And specifically, my favorite of these is what's called upper confidence bound. And this is another one of these things where I really feel like there's a profound life lesson in this very technical subject. So, the idea of the upper confidence bound algorithm is that it says, you know, for each of these opportunities, there is a kind of a confidence interval. There are these error bars around how good I think it could reasonably be. You know, there's a middle of the bell curve where my sort of expectation lies, but there's also these tails that surround that, or these error bars that go both above and below.

• 29:31Speaker 01

  The upper confidence bound algorithm says, just very simply, ignore your actual expectation of what's probably going to happen, ignore the worst-case scenario, focus exclusively on doing the thing that has the best best-case scenario. And this is an algorithm that's been shown to minimize regret. And to me, that's just this very sort of human takeaway from this problem, that we can literally minimize our regrets in life by focusing on the things that have the most positive potential. To me, that's very profound.

• 30:06Speaker 00

  From there, so we think about Bezos and this great framework. Now, of course, once he jumped, he then had to have his stopping point, Amazon, to some degree. Perhaps that wouldn't have worked out, and he would have had to try another venture. I don't know. But it did work out. But there, on the flip side of this, there are many people that have that spirit that Jeff had, but perhaps, and I'm kind of reading from what you say here, but there's many people that they tend to over-explore. So, they favor the new disproportionately over the best. You have these people that have the energy and excitement to go jump, like Bezos did, but they keep jumping and they can never sit still.

• 30:46Speaker 01

  Right, that there is the possibility to to go too far in that direction. Yet, one of the things that I think is quite interesting is having looked at the problem and looked at what characterizes the optimal strategies, the optimal algorithms in the scientific literature, there's this irresistible question, which is, is that what real people actually do in real life when they're confronted with these types of problems? In each chapter, we kind of delve into the psychology literature and say, what does it look like when you actually throw people into a lab setting and have them go through these sorts of tasks? Do they do what the optimal algorithm would have them do?

• 31:30Speaker 01

  In almost every case, the answer is no, they don't. And so there's a real opportunity there to ask, well, why is that? Is that because people are just, you know, stupid or misinformed or kind of poorly designed by evolution for, you know, for these sorts of tasks? Is it perhaps that it's the models that are wrong, that the people are in some sense doing something smarter than what the theory has gotten us to? In the case of explore-exploit problems, there is this very consistent result, which is that people tend to over-explore.

• 32:09Speaker 01

  They gather more information than they need to, they don't commit as much, they second-guess themselves more than they should and that sort of thing. The famous experiment in which this comes up was one by Amos Tversky and some of his colleagues back in the 19, either 60s or 70s. They they had this setup where they had a box with two lights on it. One of the two lights would come on with varying frequency. And they said, okay, you have, we're going to press this button that will turn one of these two lights on. We're going to do it 1,000 times, and each time you have the opportunity to either observe the light or place a bet on what you think the light, which light you think is going to come on.

• 32:54Speaker 01

  But you do not get to observe which light came on, and you do not find out until the end of the experiment whether you won or lost money on that bet. It turns out that the optimal strategy in this case is, I believe, you make thirty-eight observations in a row, and then you blindly place a series of nine-hundred-sixty-two bets on whichever light came on more at the beginning. This is not what people do at all. They are much more exploratory. They gather much more information. They will observe for a long period of time, then they will make a series of bets, then they will go back to observing, then they will make a series of more bets, then they will go back to observing.

• 33:35Speaker 01

  Well, someone had a closer look at the raw data from this study and came to a realization that I think is quite significant, which is if the probabilities that these different lights were coming on was on a random walk, if it was varying over time, well, then what the human subjects were doing is, in fact, extremely close to the optimal solution. This is a version of the problem known as the restless bandit problem. Then you can ask the question, well, okay, why are subjects behaving as though these payoffs are volatile when we've told them ahead of time that they're fixed?

• 34:15Speaker 01

  I think an appropriate response to that is, well, there's subjects in a psychology experiment, so I think it's perfectly reasonable for them to doubt the description of the scenario that the experimenters gave them, because there is a long storied history of psychology researchers lying to subjects. So I think it's probably appropriate that the subjects didn't take it on face value. But the deeper point here is that when you find yourself in one of these restless bandit problems, when the fundamental truths can change, the world itself can change, the

• 34:55Speaker 01

  environment that you're in can evolve, well, then you really should spend more of your time gathering information and commit for smaller stretches of time before gathering more information. And so it turns out that humans' instinct to quote-unquote "over-explore" is, in fact, very well-adapted for the sort of dynamic and changing world that I think Better describes a lot of the scenarios that we're actually in.

• 35:25Speaker 00

  We also run the risk that, let's say, an investor, for example, wants to make an investment in Tesla. How much information do you gather and when do you stop? These become tricky questions because in many ways, there I can think of many investors that would say, "I want the most simplistic data, a simple price momentum, and I'll bet off the price momentum alone." Versus some other investors might say, "I need to have, you know, untold amounts of fundamental information." But the untold amounts of fundamental information never end.

• 35:59Speaker 01

  Yes, right. While you are gathering all that information, you might be missing out on your best opportunity, your best window, right, to buy.

• 36:06Speaker 00

  Right.

• 36:07Speaker 01

  And, you know, one of the, one of the key factors in any sort of explore or exploit problem is this question of how much time you feel you have. What is, what is the interval of time over which you are trying to optimize? You know, by analogy, if you are living abroad, let's say, for a year, you know, the, the first week that you're there, even if you discover an amazing restaurant, you probably don't want to go back twice in the same week. Because, you know, the, the second restaurant that you try in that country has a 50% chance of being better. And then the third restaurant you try has a 1 in 3 chance of being better, which is, which is pretty high. And moreover,

• 36:52Speaker 01

  if you do find a restaurant that's better than the best one you already know about, you have 51 weeks to enjoy it and keep going back. Conversely, if you imagine yourself in the last week of your trip, well, the, the odds that a restaurant that you try are, the odds that a new restaurant you try is better than the best ones you already know about are quite small. You know, you're able to set a pretty high bar as a result of your experience. But furthermore, even if you did find a restaurant that was better than the ones you already know about, you've, you've run out of time to continue to enjoy that.

• 37:33Speaker 01

  One of the things that you find when you look at some of the solutions to the explore-exploit trade-off is that... Our decision making can and should change as a function of where we perceive ourselves to be within that interval of time. And this is, of course, something that comes up in investing context. If you have a passive retirement account, for example, you sort of plug in your expected retirement year. A lot of these funds will typically pursue a very high-risk strategy 40 years out from retirement and then gradually, for example, move the money from stocks into bonds as you get closer to retirement.

• 38:15Speaker 01

  And I think the same type of dynamic exists in this question of, do I continue to gather information or do I just leverage the information that I have? It's partly a function of the amount of time over which you're going to be committed to your action. So if I'm committing a bunch of money into Tesla, to use your example, and it's going to be locked in for, let's say, 50 years, then it probably makes sense to do a lot more of that fundamental research. But if I'm just going to be parking my money there for a month or six months or something,

• 38:57Speaker 01

  then it's more important to just sort of try different things out and commit on the basis of what we already know.

• 39:07Speaker 00

  Let me jump to another part of the book, something that I really liked, because I can stay in one chapter and just start picking your brain for the next 10 hours. Computers have this really great technique, this capacity to multitask, and I'm going to let you explain it. It's called threading. But we, as human beings, we really don't have this ability to thread, do we, effectively?

• 39:35Speaker 01

  Yeah, I mean, not really. Despite the fact that one of the things that really distinguishes the brain from a computational architectural standpoint, one of the things that really distinguishes the brain is this massive parallelism.

• 39:54Speaker 01

  It's ironic, given that, that we can't really process more than one train of thought at a time, right? Or have more than one conscious deliberate experience at a time. We have to kind of yank our attention back and forth. And in many ways, this is, in fact, quite like what computers do. They just do it at a much faster timescale, and so they can create the illusion of doing more than one thing at the same time through this process that's called threading. The basic idea behind threading is that you have a series of processes that are running, but the CPU can only attend to one of them at a time.

• 40:37Speaker 01

  It will spend a little bit of time flashing your cursor, and then yank itself to your inbox, and spend a little bit of time pinging to see if there's new mail in your inbox, and spend a little bit of time buffering your audio so that your Spotify in the background keeps playing smoothly. And by hopping around from thing to thing nimbly enough and quickly enough, it can create this wonderful illusion of simultaneity. But, as anyone who has used computers in the last 50 years can attest, sometimes this breaks down, and we end up staring at the hourglass or the spinning beach ball of doom.

• 41:21Speaker 01

  And so, I think it can be very instructive to understand what's actually going on under the surface in those cases, because I think there's really a connection between how a machine's time management can fall to pieces and some of the human traps that we all recognize from our own lives. The biggest pitfall for a system that's managing a multitude of threads is what's called thrashing. The basic idea behind thrashing is, as the computer is managing these different things, there's a certain amount of overhead associated with changing from one task to another.

• 42:05Speaker 01

  And this is known as context switching. So in a computer, that means you're repopulating your caches with different information. You are kind of bookmarking and saving your progress on, you know, the previous task so that you can come back to it later. You're finding your place in the next task and sort of queuing it up and getting started. And so there's a certain amount of fixed overhead associated with every context switch. And so if you imagine, let's just say for the sake of argument, you know, it takes you a, it takes the system a tenth of a second to context switch. It's really more like, you know, a millionth of a second, but let's just say for the sake of argument, you know, it's a tenth of a second.

• 42:50Speaker 01

  If you're doing 10 different things and you have a policy that you don't let something go longer than one second before you do at least a little bit of work on it, then the system ends up in a state where it only has a tenth of a second allocated to each task, but it requires the entirety of that tenth of a second just to page the, the sort of now where was I information back into its proverbial head. And so you have, you can have a situation in which the computer is spending literally 100% of its time doing these sort of bookkeeping meta tasks.

• 43:32Speaker 01

  And what's really brutal about this situation is that because it is not actually advancing the state of any of its threads, it's extremely difficult to get out of because it's not doing any real work. And so it's not likely to terminate any of those processes and bring the system back to a more stable state.

• 43:51Speaker 00

  Nice description of human nature, huh?

• 43:53Speaker 01

  Yeah, I mean, I honestly, I, I see more of myself in that description than, than I'm, you know, comfortable with. I think we've all had moments where we've felt, you know, if, if you've ever spent, you know, more time Mentally keeping track of all of the things that you have to do than actually doing them, then you're basically in a position of thrashing where the meta work and the bookkeeping has overwhelmed the actual work that you're doing.

• 44:27Speaker 00

  Brian, let me bring up one more topic with you before I let you leave. This one really is near and dear to me. I'm so sad to see its demise, but I started as a Eudora Mail user probably in the mid 1990s. And what I loved about Eudora Mail is that, and I had to stop, I think in 2013, they stopped supporting it on the Mac. But what I loved about it is that I could sort things, right? But, but if I needed to search on Eudora Mail, and it is, they've basically, all the folks in Silicon Valley have eliminated this opportunity for me. It does not exist. Eudora Mail still cannot be beat for search.

• 45:07Speaker 00

  If I wanted to search a decade of email and find anything from a thought to an IP address, you name it, Eudora Search was just absolutely awesome. The searching versus sorting conundrum is something, it's something you cover in your work a lot. Why don't you, why don't you speak to it some from your perspective and maybe perhaps a lesson or a story that might give people a window into what you guys have discovered and put out for us?

• 45:34Speaker 01

  Yeah, absolutely. One of the, you know, one of the canonical domains of computer science that, you know, would be familiar to any, any computer science student is sorting algorithms. You know, in many ways, this is one of the canonical, you know, computer tasks. How do you sort a set of things as efficiently as possible? In the book, we get into the history of sorting algorithms and, and what advice they might hold for us about the best way to, you know, organize our household items like our bookshelves and so forth. But I think that the deeper principle here is that for a computer scientist or for, you know, a computer system, there is really one and only one reason that you would sort

• 46:19Speaker 01

  things in the first place, which is to save yourself time searching for those things later.

• 46:27Speaker 01

  That I think is a very, a very deep point that, that leads to a way of thinking about the, the question of sorting, which is really, should you be sorting at all? Will the act of sorting something save you more time in subsequent searches than the act of sorting it would itself take?

• 46:48Speaker 00

  It's a really deep thought.

• 46:50Speaker 01

  And so you end up with, in many cases, you know, a computational argument for mess. That there are a number of things in life that, you know, we as humans attach kind of a, a moral dimension to, you know, or, or guilt with something being messy. When in fact, there is this argument that can be made i- in a number of situations where mess is not only the, the easy choice, but it can also be the optimal choice. Computer science of all things, I mean, it's, it's ironic because we think of computers as kind of the, the paragons of organization. But computer science gives us the language to actually formulate the rigorous argument for mess.

• 47:35Speaker 01

  And that for me is really sort of counterintuitive and cool.

• 47:38Speaker 00

  It's very cool. Really, it's, uh, there's, there's so many kind of aha moments in this work. I really, I really appreciate you putting it together. I do want to read one last sentence, which I think the, for the audience to keep you thinking as to why this is important. And this is from, from your work. "The underlying directive of any good algorithm is to minimize the labor of thought." We all want to do that.

• 48:01Speaker 01

  Absolutely. Yeah. I mean, I think that's one of the, that's one of the kind of unintuitive things about computer science is that the, the goal of any algorithm is to minimize computing. To, to a computer scientist, computing in some sense is the enemy or, or is bad. And so I think there's something we can all sort of take heart in that there's, in any decision-making process, there is this critical, critical component of just knowing when to stop thinking, or knowing how hard to think about something. And computer science gives us a way of not only making the best decision when we have

• 48:46Speaker 01

  perfect information and all the time in the world, but it also gives us a way of thinking about how to make smart trade-offs when we don't have those luxuries. And I think that's much more often the case that we find ourselves in.

• 48:59Speaker 00

  Brian, the work, Algorithms to Live By: The Computer Science of Human Decisions, with your co-author Tom Griffiths, available everywhere. I think it's doing really well, isn't it?

• 49:10Speaker 01

  Yeah, in fact, we're just in our third printing now, so that's really, it's been really encouraging to see the reception it's been getting.

• 49:17Speaker 00

  You've got all Silicon Valley reading this, right?

• 49:19Speaker 01

  Yeah, yeah, yeah. In fact, I just ran into a friend of mine on the subway. We were talking about the book, and the guy standing next to me, you know, whips around and says, "Oh, my God, you wrote Algorithms to Live By?" I have read that book. So, yeah, it's really, it's really satisfying when you can feel like, you know, the work that you're doing is reaching people who are sensitive to it and who are able to take something out of it. That's really meaningful.

• 49:46Speaker 00

  Beyond Amazon to buy, is there a website people should check out?

• 49:51Speaker 01

  AlgorithmsToLiveBy.com.

• 49:53Speaker 00

  Brian, I thank you for taking the time. Hopefully, I can have you on again when the next work comes, because I know a next work will come.

• 50:00Speaker 01

  Absolutely. It's been a total pleasure, so I would love that. Thank you very much for having me.

• 50:09Speaker 00

  I see a time when those awake will understand how to make money in up, down, and surprise markets. Whether new trader or experienced, college student or financial advisor, protecting against a crash, or just trying to make a lot of money, trend following offers everyone an answer in uncertain times. To get started immediately, send me an email, [email protected]. I will send you the right trend following steps to take, along with my free video.

• 50:45Speaker 00

  But if you want to buy and hold, trust the government and trust Wall Street, this is absolutely not for you.