Leigh Drogen - Sink or Swim--How to Combine Quant and Traditional Asset Management Techniques - Invest Like the Best, EP.48]

Several weeks ago my conversation with Leigh Drogen on quant investing proved timely and popular--because everyone in asset management is facing the rise of big data, and the use of data science in investing strategies. Because of the rise of quants, many are asking themselves how to survive and thrive in a changing industry. In short, how can traditional managers compete with quants? This second conversation with Leigh was set up to answer many of the questions posed in the first one. If quants are taking over, what should other investors do about it? Leigh proposes a method by which old school asset managers can restructure their thinking and their process to compete with and even beat purely quantitative competitors. The method involves pulling the best from both worlds and combining them into a hybrid structure. But it will be impossible without a wholesale change in mindset, which is where we begin. Please enjoy round two with Leigh Drogen. Links Referenced Revenge of the Humans Part II: A New Blueprint For Discretionary Management Show Notes 2:14 – (First question) – What role will ego and mindset play for traditional hedge funds looking to transition into quantitative investing strategies 4:21 – Describes the traditional process that hedge funds use to make investment decisions and how the internal politics can hamper it 6:08 – What value has portfolio managers played at hedge funds traditionally as the quarterback of a fund 9:57 – A look at what Leigh has seen as he sits with teams 12:20 – A look at places that have tried to simply add quant to their firm’s strategies without “tearing it down to the studs” and properly integrating them into the process 15:00 – Leigh is asked to define the basics of a good investment firm’s strategies 16:57 – Strategies for writing down core beliefs, whether it’s for yourself or your firm 17:49 – Exploring the second step, finding a differentiating view and how to succeed with it. 21:43 – The importance of force ranking and structuring the unstructured 26:14 – Building factor models 29:42 – How the portfolio manager position should have less room for subjectivity than at the analyst level 33:44 – Is anyone integrating this kind of high level data at the portfolio manager level into the decision making the way Leigh describes 35:07 – What blind spots are created by systematizing their processes 36:18 – Why much of this applies more to shorter and structured periods 38:23 – Shifting to portfolio constructions and what Leigh would do to create the right mix 43:39 – Shifting to management structures in these firms starting with the role of the CIO 45:24 – Looking at the different quant roles that exist in a firm and what they should be responsible for; data engineers, data analysts, pure quants, and quantitative engineer 48:20 – If you are an undergrad or grad student right now interested in asset management, what are the roles you should be thinking about targeting 49:25 – Why communication skills are still so important, no matter what role you are in 50:25 – With all of the tools and skills that Leigh has at his disposal at Estimize, why not institute an active strategy 52:01 – What has Leigh observed in the dispersion of skill in the Estimized data set 53:47 – What is the relationship between specialization and accuracy among funds 55:29 – The pros and cons of the generalist 56:56 – A look at Leigh’s background into War Theory and what lessons that he still draws on today 1:00:19 – How the field of study around war and battle relates to the investing world Learn More For more episodes go to InvestorFieldGuide.com/podcast. For complete shownotes, go to InvestorFieldGuide.com/leigh. Sign up for the book club, where you’ll get a full investor curriculum and then 3-4 suggestions every month at InvestorFieldGuide.com/bookclub Follow Patrick on twitter at @patrick_oshag

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Patrick O'Shaughnessy?

  I know firsthand how complex the tech stack is for asset management firms. And seemingly every new tool and data source makes the problem even worse, adding more complexity, more headcount and more risk. RidgeLine offers a better way forward. One unified platform that automates away the complexity across portfolio accounting, reconciliation, reporting, trading, compliance and more, all at scale. RidgeLine is revolutionizing investment management, helping ambitious firms scale faster, operate smarter and stay ahead of the curve. See what RidgeLine can unlock for your firm. Schedule a demo at ridgeline.ai.

• 0:32Patrick O'Shaughnessy?

  This podcast is sponsored by CFA Institute, the global association of investment professionals whose mission is to lead the investment profession by promoting the highest standards of ethics, education and professional excellence for the ultimate benefit of society. CFA Institute serves a global community of investment professionals, working to build an investment industry where investors' interests come first, financial markets function at their best and economies grow. The Chartered Financial Analyst credential is the most respected and recognized investment management designation in the world. The views expressed in this podcast do not necessarily represent the views of CFA Institute. Hello and welcome everyone. I'm Patrick O'Neal and this is Invest Like the Best.

• 1:15Patrick O'Shaughnessy?

  This show is an open-ended exploration of markets, ideas, methods, stories and of strategies that will help you better invest both your time and your money. You can learn more and stay up to date at investorfieldguide.com.

• 1:31Speaker 02

  Patrick O'Neal is a Principal and Portfolio Manager at O'Neal Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of O'Neal Asset Management. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of O'Neal Asset Management may maintain positions in the securities discussed in this podcast.

• 1:54Patrick O'Shaughnessy?

  Several weeks ago, my conversation with Lee Drogan on quant investing proved timely and popular because everyone in asset management is facing the rise of big data and the use of data science in investing strategies. Because of the rise of quants, many are asking themselves how to survive and thrive in this changing industry. In short, how can traditional managers compete with quants? This second conversation with Lee was set up to answer many of the questions posed in the first one. If quants are taking over, what should other investors do about it? Lee proposes a method by which old-school asset managers can restructure their thinking and their process to compete with and even beat purely quantitative competitors. The method involves pulling the best from both worlds and combining them into a hybrid structure. But it will be impossible without a wholesale change

• 2:39Patrick O'Shaughnessy?

  in mindset, which is where we begin. Please enjoy this great second round with Lee Drogan. So Lee, round one was us describing sort of the state of quantitative investing and the process broadly speaking. We're going to get even more specific today, which is to describe how a more traditional discretionary asset management firm, specifically hedge fund, can successfully make this transition from discretionary to at least part quant. And we'll begin with something that is pervasive across this process that we'll describe step by step, which is this notion of mindset and ego. So I'll let you start there by describing

• 3:22Patrick O'Shaughnessy?

  what role ego will or will not play in this transition for those that are successful, and why this needs to be first and foremost, a shift in mindset versus, say, tactics and strategies.

• 3:32Speaker 03

  Yes. So I started, it's funny that you bring up tactics and strategies. I studied war theory in school. And often, we like to say that war is politics by other means. And if you don't have a political goal, it doesn't matter what tactics and strategies you use to fight the war, or to fight the battle, you're going to lose in the end. And as we've seen, you know, Americans in Afghanistan, like, we've used all the right tactics and strategy, but we have no political, you know, outcome that we actually are attempting to affect. And in the kind of long only and long short hedge fund equities world, it's very much similar today. So you're seeing all of these funds wake up and attempt

• 4:15Speaker 03

  to use tactics and strategies, but really have no kind of, the political aspect has not really been thought through yet. And I think they're doing it all backwards. And the political aspect is mostly the ego associated with PMs having control over the investment process and final decision making and things flowing up to the PM instead of the PM basically just being like the offensive coordinator on a football team. That, that is the shift that needs to take place. And look, these, these guys make a lot of money and even more like the power associated with it, I think is an important thing for them. And in order to make this shift, they're going to have to give some of that up.

• 4:53Patrick O'Shaughnessy?

  Let's level set by talking about where it is now. So describe the, to the extent there is a process, describe the traditional process between researchers, analysts, sort of the hierarchy that exists now, how decisions get made, and then we'll go from there into what it maybe should look like.

• 5:13Speaker 03

  Every fund is going to be a little bit different and you're going to get differences between big long only funds and, you know, smaller long short funds. But generally, what you've got is these analysts who will be given a sector or industry and they'll be told, go cover 40, 50 stocks. And you'll have a PM, maybe at like, let's say our, you know, our hypothetical firm, there's 15 analysts and there's, you know, three or four PMs. The analysts will then come to the PM with an idea and say, I think this thing's worth X a year or two from now or whatever. And here's the big 10-page research report Y. Please go read it. And then they'll have some kind of discussion and maybe the PM

• 5:58Speaker 03

  will put it into action and maybe they won't. And there's this whole kind of politics process of getting your ideas implemented. And the issue is that a good analyst isn't necessarily a good politician within a firm, but these two things have been unfortunately combined. And so you get really good analysts with really good differentiated ideas that may be very accurate, but they just can't sell it. And then the PMs don't use it and then it's very hard to judge the accuracy of the analysts and how good they are relative to how the PMs use it and the whole system does not align itself towards understanding how good are we, when are we good, when are we bad, when should we listen to that person, when should we not listen to that person?

• 6:39Patrick O'Shaughnessy?

  A good analogy that you've used is already is this notion of PM as offensive coordinator versus PM as quarterback, so kind of calling versus making the plays. And that's going to be an important part of the discussion, which is how have PMs traditionally added value and how should they in the future, assuming they migrate to this more quantitative or man plus machine type model? So in the, talk a little bit about that in the past and the more, we'll call it the traditional model, where there's information flowing up to a PM. Again, we're speaking in generics and there's obviously an exception to everything we're going to say, but talk about that process in terms of how they've added value, quote unquote, historically, whether it's diving deeper into the names or just having a nose

• 7:23Patrick O'Shaughnessy?

  for, you know, more top-down type stuff. Talk about kind of the flavor of the quarterback version of portfolio managers.

• 7:30Speaker 03

  Yeah, I mean, the quarterback version is they're basically deciding on everything from position sizing to market timing to risk management and they're basically relying on the analysts to just give them kind of, you know, the fundamental view and thesis. That puts a lot of different variables in the PM's hands and they're juggling a zillion different things and how could you possibly be good at all of those things? I think, you know, while we're going to talk about, you know, maybe some of the beginning processes of idea generation and stock selection, the one that I think is just even more egregious is the portfolio optimization part of it. Computers do this much better than humans and God knows, a human should not be deciding

• 8:12Speaker 03

  how to weight their portfolio and just like the very stark example is you could be generating alpha but have negative returns because you're exposed to betas and the PM be like, well, I, you know, like I've done everything right, like why, why is my portfolio not performing? Well, well, it's, it's, you know, because you didn't, you should have optimized. To not be exposed to certain betas, or if you wanted to be exposed to them, like the machine should be doing that for you. And so they're, they're literally just juggling too much. They're having a macro view of the whole thing from the top down. And then they're literally overriding the best ideas on average from the individual analysts because inherently,

• 8:53Speaker 03

  an analyst's idea that is very differentiated should be controversial. And we, we know that in a study of venture investing, there's some really good funds, like first round capital that have looked at the variables associated with making a good investment. And in many ways, when you look at like deep value investing in this same process, it's, it's exactly the same. Union Square Ventures has also done a, a study internally of this kind of stuff where they find that when they go into the partner meeting with an idea, what they want to see is a barbell of, I think that's an awesome idea and you're fricking crazy. Like, why would we ever do that? They find that their worst investments are the ones where everybody is like, oh yeah, that makes total sense.

• 9:37Speaker 03

  You do not want to be making those investments. And the problem in, in our world, long, short and long line investing, is that ideas that are like, oh yeah, that makes total sense are usually the ones that get put into the portfolio. The ones that are really controversial, which are where you really should be investing, eh, you know, they get thrown out by the PM.

• 9:54Patrick O'Shaughnessy?

  Yeah, this is, uh, this is something that I have to remind myself all the time, which is that consensus and alpha do not mix.

• 10:00Speaker 03

  Exactly. Yeah. If you are in consensus, you are not generating alpha and like by definition.

• 10:06Patrick O'Shaughnessy?

  My favorite part of the conversation with Jerry Newman, who's a venture capital guy here in New York, was that he would log on to some, I think it might've even been Union Square Ventures website and like look at the portfolio and just think, this is crazy. Like, what are they doing? And you look at the fund's IRR and it's like 65% or something outrageous like that.

• 10:23Speaker 03

  Fred Wilson will say, he knows he's made a great investment when other people call him up and say, why the hell did you do that?

• 10:29Patrick O'Shaughnessy?

  So talk a little bit about your recent experience where you sit in a very neat seat, which is you're not necessarily a competitor to, you're certainly not a competitor to your clients and the kind of hedge fund landscape, but you get an intimate kind of look into the structure of the firms with an eye towards helping them. So tell a couple of stories about recent experiences where you're sitting with teams and sort of what you've seen and what they've been asking for.

• 10:54Speaker 03

  The last couple of years, as we've scaled out distributing the estimate, earnings estimate data to these long short and long only funds, inherently, we sit at a really interesting part of their investment process of them judging their own estimates against ours, of them looking at ours as, you know, certain signals to how stocks might perform. And I'll go into these funds and sit down and maybe it'll be like the CIO and a PM and a couple of analysts and a technology guy and maybe another person or two. And I don't go in there to like rip their whole thing apart. It's not my job. I go in there to show them how they can use our stuff to make better decisions. And then inevitably, they end up asking, you

• 11:39Speaker 03

  know, other questions around that. But recently, I've been in some of these funds and one of them, I sat down with a group similar to that. It was a, you know, $20 billion fund. And we got 15 minutes into the conversation and one of them just turns to me and is like, "Could you please actually rip our whole process down to the studs?" And I'm like, "This is not what I'm here for." And it becomes obvious when you ask all these kind of questions as good, you know, I act as the salesperson in these conversations. Good salesperson should ask a lot of questions to understand how would you use it, you know, what's your process? But when you start asking all these questions about what's your process, so that I can tell you where our stuff fits into it, a lot of them kind of just like wince and go, "Well,

• 12:24Speaker 03

  you know, like the analysts bring the, you know, the ideas and like then we discuss it." And I'm like, "Well, do you have a meeting that this is discussed in?" They're like, "No, not really. It happens ad hoc." And you can just see their faces. I don't have to say it. They realize we have to get, we have to get, we have to get this together. And then they inevitably say, "Well, how would you do it?" And I'm like, "That's not why I'm here, guys. I'm glad to try and find you somebody to like go consult for you on this." But that's also why I wrote that, you know, the follow-up piece to the first one. So that I didn't have to go into every one of these funds and kind of try and teach them how I would do it, or how I see other firms who are successful at certain parts of it doing it.

• 13:02Patrick O'Shaughnessy?

  Yeah, so I hear all the time, especially when presenting a purely quantitative strategy, often that one of the first questions is, is there some sort of fundamental overlay? It's always that term overlay. So the mindset is often that this is a two-tiered process, whatever direction that might be in. So you run a quant screen and narrow your universe, and then you pick from within that, or you do the work first, and then a quant screen sits on top. And I think really the key insight here is how difficult it is to integrate those two mindsets together. So I'd love for you to talk again about that mindset. You mentioned that term "tear down to the studs." And I think that's really important, right? If you're going to emerge with a process that successfully integrates pure quantitative

• 13:43Patrick O'Shaughnessy?

  methods, data science, et cetera, that it can't be something that you bolt on. So talk about people that have bolted on, and maybe why that's gone wrong, and then we'll get into, we'll get actually into the step-by-step process.

• 13:55Speaker 03

  Yeah, so I think a lot of these funds that have woken up first, the easiest thing to do was to say, "Let's go hire five data scientists and an engineer, and then let's go buy a whole bunch of unique data, and then let's go throw it at the PM at whatever part in the process we feel, whatever." And this simply doesn't work, because, yeah, these are two different worlds, two different ways of thinking. The PM won't pay attention to it. There are some firms that have tried to do this on a larger scale, and get the PMs to actually pay for it, like through P&L, as if the internal team was almost like a sell-side shop. And that hasn't worked, obviously. But the reason they're doing this is because you don't have to rock the boat politically, then.

• 14:38Speaker 03

  These guys sit over here, those other guys sit over here, and neither the two shall meet until some work product gets handed over. And this is obviously not working. And the scary part of it is, we're starting to see these firms, a couple of them, retrench, because a couple of years down the road, after trying this, everybody looks around and is like, "Is anybody using it?" And then everybody looks at this cost center over here and says, "Why are we doing this?" And it might actually, like, they've taken a step forward, but they might actually take two steps back at this point, because, you know, they did it wrong. And I think that's going to retard the industry a little bit, the fact that they tried to take a shortcut here. Because the buzzwords were "big data" and "data science" and all these things, instead of like, "Let's actually look at the..."

• 15:22Speaker 03

  Let's look at the process by which we make investments in our culture. You know, like, what do we value in terms of how we should act?

• 15:32Patrick O'Shaughnessy?

  Part of the nice thing about that idea, the visual of bringing it down to the studs is the studs themselves. So let's go there, which is what the studs, I think, represent being kind of core beliefs about the universe and the universe itself. Like, where are we going to hunt? We can't hunt everywhere, probably. Maybe some firms do that are massive. But talk about defining the studs themselves and why that is sort of the first step in either this building or rebuilding process.

• 15:58Speaker 03

  Yeah, one of these firms that I was in talking to, they literally couldn't elucidate the universe of names and the type of investing strategy that they employ. They were like, "Oh, yeah, we kind of do everything." It's like when an analyst comes up with a good idea. So it's a couple of things. What is the market cap that you're going after? Like, are you a small and mid-cap investor or are you a large-cap investor? What industries and sectors do you believe that you have really good analysts in where you can analyze things and have an edge? Do you believe in growth and momentum? Or do you believe in value or some other characteristic? That you have a serious belief that you want to align your portfolio towards? I personally was taught how to be a momentum investor, right? You want to buy high and sell higher. You want to use relative strength in your favor.

• 16:43Speaker 03

  And then you want to get the hell out of there before margins collapse at the top and then the multiple collapses. You want to ride the multiple up instead of trying to catch a multiple at its bottom. That was my philosophy. God knows there are a lot of different philosophies that work perfectly well. I think the problem is a lot of these firms don't even know what their core philosophical beliefs are. And you can't build the rest of the process without that. Because the rest of the process is really, it's aligned to those things. If you believe in momentum and growth and relative strength and stuff like that, you're going to look at all sorts of different factors and variables and different universe of names than if you believe in value and all those other things.

• 17:22Patrick O'Shaughnessy?

  I've actually been going through this exercise myself, not just in investing, but just more broadly speaking, trying to come up with a set of core beliefs or guiding principles. And it's incredibly hard to do, like to actually put pen on paper, which I think is the right way to do this, and codify core beliefs is really freaking hard. Are there any suggestions that you have for going through that process itself? Like, maybe you've done it?

• 17:48Speaker 03

  It's like writing a constitution. The firm needs to sit down and write a constitution. And this is, we talk a lot historically about this idea of like style drift, and how managers across the board perform poorly when they drift from their stated style. Every firm should have a stated style. And we know that when you have a specific strategy and you try and take trades outside of that strategy, those trades are going to be your biggest losers, mostly because you don't know how to manage risk under those scenarios. And when you don't know how to manage risk, those are going to be the long tail of your bad returns.

• 18:21Patrick O'Shaughnessy?

  So assuming that people can sit down, identify a universe where they have strengths, identify some core idea of how markets work and, or at least how they can operate.

• 18:31Speaker 03

  Yeah, how they can operate.

• 18:32Patrick O'Shaughnessy?

  In a way that has some sort of edge. So then step two is, okay, ride that edge. So develop sort of a differentiated view versus the market, back to that consensus idea. So talk about that second step and what's important there.

• 18:45Speaker 03

  So this is where I think firms need to get much more systematized in how they run the process of having analysts come up with ideas. Historically, there's some software like Code Red and Tamale, and some of these have been bought by like a FactSet or Advent. And it's the system where the analysts will basically go in and put notes about things. And some funds like Citadel, even Point 72 have kind of these internal estimate capture systems. Nobody really uses it because they're terrible pieces of software. And they know it too. Like, this is not a secret at all. And so they can't really get the whole firm to actually give them structured views on expectations.

• 19:28Speaker 03

  And so my belief is that they want to and they know it's what they should be doing in there. They want to try to do this. And the way that I'd like to see firms set this up is basically, Analysts have to give a centralized database, a forward-looking view on EPS, revenue, EBITDA, and then like whatever key performance indicators the company is really kind of bound to. So Netflix would be subscriber growth numbers and Starbucks would be same-store sales, whatever it is. Give a forward year of estimates quarterly, or two forward years, or whatever your timeframe aligns to. And timeframe is another one of those, like Core Belief's many-firm sale, "Oh yeah, we're long-term investors." And then, of course,

• 20:10Speaker 03

  like, you know, they're in and out of the stock in a quarter, and they don't give their idea the time. And I think the reason why this happens is because if you don't have all the structured variables that add up to why you believe in a long-term thesis, you will cut and run much earlier before letting it actually play out, because it's not actually put down on paper there. So the analysts on a regular interval, probably three times a quarter, have to update their forward-looking estimates. And then what they should be doing is, and this is my core kind of view on what fundamental investing is, you have some fundamental factor, let's say EPS, you have a forward one-year EPS estimate, let's say for Apple, it's, I don't know, what, it's four bucks or five bucks,

• 20:54Speaker 03

  whatever. Apple trades at 15x earnings now, you believe that if you're right, it'll trade at 18x earnings a year from now, you multiply that multiple by your forward-looking estimate, that's your price target. And so, three times a quarter, the PMs are going to get an updated view on all of your stocks, let's say you cover 30 or 40 names as an analyst, the PMs are going to know what is your forward one-year price target, what is the market's forward one-year price target, which is basically the consensus E times some terminal multiple that it's trading at now. And they're going to see where are the biggest deltas between your ideas and the market's consensus. And then, once we have all of that data from all of our analysts, we can do a lot of different things with that. But the point

• 21:38Speaker 03

  is that instead of the analysts coming to the PM whenever they feel it's necessary, it's incumbent upon the PM to then run the process of taking all of the analyst's structured ideas and dealing with them. And that will remove, hopefully, a lot of the politics involved in when the analyst actually shares an idea. Because sometimes they get pissed off at the PM and they're like, well, I don't want to share my best ideas with you. I'm just going to like, save them in this like Excel sheet over here on Google Docs. So that when I go to another firm, I can like, say, well, these were all my real ideas, but my PM screwed it all up. You don't want that. And so in this way, you would have a centralized kind of database of these views.

• 22:16Patrick O'Shaughnessy?

  One of the key ideas here, which is just hard, it's a mindset thing again, is this notion of structuring the unstructured, that a lot of this is gray area stuff and long reports, and it's not objective data points. You've talked a lot in the past about the power of just force shrinking stuff. Just, it doesn't need to be, the magnitude may not matter, but at least get something on paper that you can track to be able to look back and evaluate performance, evaluate effectiveness, maybe change your own methods. I think that's really key. So let's assume that there's some way, and maybe this is a big assumption, which is that there's some efficient way of structuring the unstructured that's not just like an Excel spreadsheet. And maybe that, you know, someone out there listening can build a very lucrative software company.

• 22:57Speaker 03

  Yeah, absolutely.

• 22:59Patrick O'Shaughnessy?

  If they do this for hedge funds.

• 23:00Speaker 03

  It's something like, you know, you take something like Tamale or Code Red, and it would have been nice to have them literally put in fields where you could say, okay, my field is corporate governance. And each time I go in there to like, update my whatever, I have to rate corporate governance for this company on a one to 10 scale. Well, if we collect that for several years, we can go back and say, was corporate governance or my expectation of what corporate governance was, was that in any way correlated to volatility or earnings volatility, or misses in beats, or stock performance? And if we collect that on a set of maybe 10 different variables for each stock that the analysts, you know, these kind of unstructured concepts, what's the probability that the company gets bought

• 23:44Speaker 03

  out this quarter, you know, stuff like that. You just have so much more data to deal with. And that's the stuff you hand over to the quant. That's where the quant starts to really like, get involved. But you have to have this stuff structured. You can't just have, and I've seen funds try and take all of the unstructured text that their analysts were putting into Tamale and Code Red and stuff like this, and have their quants go Through the semantic analysis of it and it's like you are trying so hard to do the wrong thing. You're putting so much effort into into the wrong thing and it's it's really, yeah, it's really funny.

• 24:20Patrick O'Shaughnessy?

  Just just a point here because it's something I've thought a lot about and we're talking about systematizing things and that, you know, systems in general are good in this sense, but a key point I think about systems is that simple systems are better than complex ones. It's the beauty and the elegance of a of a force ranking one through X versus like price targets, let's say. There's something much better about keeping things simple because if you try to drop, and there's a million examples of this, if you try to drop a complex system in like a design, top-down design, complex system, it will fail. And we've seen this as as pure quants a lot. It's why often equal weighting trumps everything else. It's why force ranking works so well.

• 25:02Patrick O'Shaughnessy?

  And it also makes it more accessible as you're trying to make this transition from kind of shoot from the hip, stock picking to a more systematic process. It's it's more of a baby step, right? You're not being asked to input a thousand things or have false precision is a big problem. When you take a traditional mindset and try to make it quantified, I think false precision is incredibly dangerous. So I would highly recommend people think in terms of system, yes, but simple systems at the beginning trump complex ones. And then, once you figure out what works within a simple system, then you can push a little harder, maybe make it more complex, but start with really, really simple.

• 25:39Speaker 03

  Financial people love complex things because they believe if it's more complex, it must be better. And it's just simply not true. It's also a marketing thing. Oh, we're doing all these like really complex things. Give us money because we're the only ones who can do it or, you know, something like that. And there's only a handful of funds in the world where that is actually true. Your rentex of the world, the rest of us should not attempt to go through that. It's, yeah.

• 26:04Patrick O'Shaughnessy?

  So to take stocks, so we've got core beliefs, guiding principles. We've got this notion of structuring the unstructured by taking whatever it is and at least entering it into a very simple database. So now we've got data. It's subjectively derived data, but that's okay.

• 26:21Speaker 03

  It's not even that it's okay. It is, that is the core of generating alpha is the subjective nature of your forward-looking expectations being differentiated from the rest of the world.

• 26:31Patrick O'Shaughnessy?

  So ideally, the process through which those subjective inputs are derived also itself is somewhat systematic or process oriented.

• 26:39Speaker 03

  Should be, hopefully, yeah. You're looking at the same data each quarter, you're doing channel checks, you're talking to customers, whatever, yeah.

• 26:46Patrick O'Shaughnessy?

  So now we've got data. So now we really start to get into our wheelhouse, which is actually building factor models. So talk about this crucial step, and this is really where, you know, instead of hiring a quant team to come sit, come sit in their own office and occasionally serve up, you know, a research report, this is where those skillsets become really interesting. So talk about building factor models.

• 27:07Speaker 03

  Yeah, so this is where the unique data comes in. This is where the quants get hired. Quants don't enter the process really at all up until this point. There are data engineers that enter the process before this point and data analysts, and they help the fundamental analysts get ahold of any data that they may feel is necessary to make those forward-looking expectations. But on the other side, there should be a set of quants that basically work with the PMs to say, okay, what is it about this industry or this sector that we believe all of your names share? And so we know that in technology, which is a growth industry, that history surprise

• 27:48Speaker 03

  of, you know, analyst estimates is a factor that is correlated to future momentum and outperformance in stocks. And so what the quants will do is they'll talk to the PM and they'll talk to the data analyst, and they will take a data set and they will create a factor model, which is basically a z-score. You know, we rank all the stocks in our sector, so let's say there's 300 of them.

• 28:12Patrick O'Shaughnessy?

  Just define what a z-score is.

• 28:13Speaker 03

  Z-score is basically The relative ranking of, you know, let's say one stock to the next based on some factor. So if we're talking about history surprise, if there's 10 stocks and two of them always miss and two of them always beat, the two that beat are going to get a 90 or 100 Z score. And the two that always miss are going to get a negative 90 and negative 100 Z score. And then it's a normal distribution. So, or it's not a normal distribution, but it's a normalized.

• 28:40Patrick O'Shaughnessy?

  It's a number of standard deviations away from the mean that any company is. So if a company is, I don't know, has a EPS growth rate of 50% and the average is 10 and the standard deviation is 20, it's 50 minus 10 divided by 20.

• 28:55Speaker 03

  The point, the point being, we're trying to normalize a range of, you know, of scores. So the, the quant is going to build a factor model that they can put into a system. And they're going to build not just one factor model, but maybe 10 of them. And the idea is that when the PM goes to look at all of their analyst's expectations, kind of sorted by the largest kind of outlier expectations. They're also going to be able to see what do the factor model scores say? And the idea here is that while the analyst's, you know, long-term year or two-year-out expectation is going to be whatever it is, the factor models might give better insight into

• 29:38Speaker 03

  timing, position sizing, risk management. Should we actually invest in this name right now? Or does the post-earnings drift model from SMI say we should hold off for five days? And that's where the PM really plays offensive coordinator and looks at all the different information, looks at the satellite photos, you know, looks at, talks to his quarterback on the ground. And they're going to be coordinating and this is where they add alpha. PM shouldn't be trying to add alpha in the what stock should we pick realm. They should be adding alpha in the how do I run a better process to incorporate all this different data? And make everything work together well.

• 30:14Patrick O'Shaughnessy?

  It seems like there's actually, you tell me, at the PM level, less room for subjectivity than at the analyst level.

• 30:21Speaker 03

  Should be, yeah.

• 30:22Patrick O'Shaughnessy?

  So. Talk about how that might marginalize that very position, right? It sounds, more and more of that sounds like it could be replicated by a simple, you know, piece of software.

• 30:32Speaker 03

  See, I think that, I think that the PM still needs to exist and they need to exist specifically for the purpose of, software is not good at seeing outlier things. And so, let's say you're just dealing with 10 names that the analyst, you know, ranks for their forward-looking expectations. And, you know, you've got your two that are really big positive delta outliers. Those are the analyst's two best long ideas. And the PM is gonna look at that and they're gonna say, hey, like, I have experience in this name. I really wonder why the analyst thinks that the earnings are gonna be so much different than the market does. And they're gonna go talk to the analyst and they're gonna say, have you thought about these three things? And maybe the analyst hasn't. And they're almost playing coach at that point.

• 31:17Speaker 03

  And it doesn't mean that they should overrule the analyst. It doesn't, it doesn't mean they should throw it out, but maybe the analyst goes back and, you know, kind of has another conversation with himself. Or looks at more data. It's a process the PM has to drive. The other part of it is that this is inherently an interpersonal kind of process. So I think that the core skillset of the PM has to go from being a stock picker to basically being a coach, like, they have to have a much better set of interpersonal skills. And frankly, I have to say that women probably should rule this part of the process. They're simply better at this, they have less ego, and they have less irrational behavior

• 32:00Speaker 03

  when it comes to, especially, you know, in, in professional environments, working with people in a very egotistically driven, you know, environment. It's about data and they have to follow it. Now, the, the interesting thing about having the PM move into that role is that I actually think that you can now measure the alpha that both sets of people are generating. Let's say you have your, you have your ranking of the stocks by delta, both upside and downside that the analyst is giving you, and they're giving it to you on a regular basis. You could theoretically- I'm going to create a virtual portfolio equally weighted of that analyst's top five long picks and top five short picks.

• 32:42Speaker 03

  And I can judge the alpha that the PM is creating by how much they deviated from those picks. And did they do better than the virtual portfolio or worse when the actual returns come out? If they did worse, fire the PM, right? Like, just go with the analyst model. And if the analysts are bad, fire the analyst, right? They're giving you bad inputs for the PM to deal with. Right now, you have no clue who's adding value in this process.

• 33:06Patrick O'Shaughnessy?

  The other cool thing there is that there's two layers of that, right? So assuming there's multiple analysts covering different lists of stocks, you can do it at the individual analyst level. So you can add value there as a PM. But then you can also be a risk allocator to which analyst. So you might be just equating or taking their exact picks. But if you give one twice as much money as the other, you know, that could be a key role.

• 33:27Speaker 03

  So this is the other kind of quantitative aspect to the PM's role, which is if we have all of the analyst picks and forward-looking expectations, the, you know, the fundamental view, the multiple view, all those other kind of unstructured variables that we've worked with, we can actually say, this analyst has been really good at these types of stocks and these types of scenarios. And we can put really easy confidence intervals next to their picks on that dashboard. So that when the PM looks at the fact that like, this is the biggest upside outlier pick, but like the confidence interval is really low for them to be accurate, they can go back to the analyst and be like, look, I know that this is one of your names. Like, I need a lot more from you here to get confident in this because statistically,

• 34:10Speaker 03

  you're not very good on this and like, I might not put it into the portfolio.

• 34:13Patrick O'Shaughnessy?

  To this point, what we've described so far, is anyone actually doing this? Like, is anyone, this is like the platonic version of a quant and fundamental integration. Is there anyone on the planet that you're aware of that's actually doing this sort of to a T?

• 34:28Speaker 03

  Not the whole thing. There are some firms doing pieces of it. They're collecting different pieces of information. But I think, and this is really where I think somebody needs to build a piece of software, which we'll talk about in a second, to actually run the central nature of the process. So people have given software to the analysts to fill out expectations à la Estimize in a structured way, Code Red and others in the unstructured way. People have built stuff like Omega Point, which will be the totally end of the process where it structures the portfolio. Once you've picked the stocks, like what should be weighted and underweighted and overweighted. But nobody's really built a piece of software, either internally or externally from these funds,

• 35:11Speaker 03

  that actually says, "You're going to review this on a Monday meeting every week. Here's the names sorted high to low. Here's all the factor models, red light, green light, or the factor scores. Here's the confidence intervals of the analysts. How often have they been accurate when they have these views? I think you need a centralized piece of software to run that, or else it'll be too ad hoc. And you let everybody go off in different directions. Something needs to be the center of that.

• 35:40Patrick O'Shaughnessy?

  I'll play Dell's advocate a little bit here, and everyone will know that this obviously is not my view. But I'll ask the question anyway, which is, let's assume a firm does this. What blind spots does it create? So what, nothing's perfect. So what are the negatives of, let's say, this perfect piece of software that totally systematizes the whole process? What are firms going to lose out on by doing this?

• 36:03Speaker 03

  One-time ideas, kind of, you know, your

• 36:06Patrick O'Shaughnessy?

  Special sits.

• 36:06Speaker 03

  Your special sits. This is not a special sit system. My own, you know, kind of, personal view on special sits is you're basically gambling, in a sense. Unless you are, unless you don't have to take that many bets. Soros is, in a sense, a special sit person fund whatever, right?

• 36:26Patrick O'Shaughnessy?

  His back hurts.

• 36:28Speaker 03

  Yeah, like he, yeah, like when he feels, you know, that the temperature drops and his back hurts, right? He comes back every two or three years, has one great idea and like makes a ton of money. You're not Soros. Like, I'm not Soros and, you know, people out there aren't going to be Soros. And we're also not Buffett. Like, we don't have enough capital to use the capital as the advantage.

• 36:51Patrick O'Shaughnessy?

  I want to pause for a minute here, especially with the reference. To Buffett because it's important to point out a key caveat to all this, which is this, we're kind of talking about one competitive. I don't know if corner, corner sounds too isolating, but one competitive part of the market. And timely, you mentioned duration or investment horizon or whatever earlier is a really key idea here. Because a lot of this applies much more to, I would say, shorter and more structured periods.

• 37:20Speaker 03

  We're talking weeks to two years.

• 37:23Patrick O'Shaughnessy?

  That's right.

• 37:24Speaker 03

  Three years.

• 37:24Patrick O'Shaughnessy?

  So if you are, if you are an investor or an allocator or anyone interested where the differentiated view, back to that idea, is much more long-term. And I would be very curious to hear what factors you think are relevant there. I've always wondered this. I've heard people say, you know, corporate culture could be something.

• 37:42Speaker 03

  And that's where hopefully you would be able to develop some structured variables and factor models that would give you an idea of, yeah, what are those things? And they're going to be different from the weeks to months to a year or two ones. Yeah.

• 37:54Patrick O'Shaughnessy?

  The idea still applied, but maybe just less so. So something like culture is going to change a lot less frequently than an EPS estimate.

• 38:01Speaker 03

  Or a view on an industry as a whole, right?

• 38:03Patrick O'Shaughnessy?

  Right, right. Exactly.

• 38:05Speaker 03

  Like I like, I like iRobot here, not because I really love the company, but just because it's a story stock in an industry that is going nuts. And that, that wouldn't be a like three-month pick for me. It would be like a five-year pick.

• 38:16Patrick O'Shaughnessy?

  One of the things, the exercises that I went through literally just for fun, you know, so, so we're purely quantitative, but I love reading 10Ks, right? Especially from companies that are very consistently coming through models. And it's fun to go through and literally just pick a stupid, simple system like one through five. Does this seem like it's a good buy right now relative to the price? And that exercise can be really, really interesting. And even if you're a pure quant, sort of inform future research or ideas that create, we talked to the first time about the importance of creativity here in designing the hypotheses first and foremost. So structuring, like any part of the process can lead to interesting outcomes. Now I would like to talk about portfolio construction specifically.

• 38:59Patrick O'Shaughnessy?

  And you've mentioned it a few times, which is optimization, exposure to risk factors. These are things that are better handled by software, but may still also require, you know, PM's involvement. So talk about sort of what your platonic portfolio construction slash optimization slash risk management system would look like.

• 39:18Speaker 03

  Yeah, so we're just at the beginning of this. Axieoma has a decent product. There are a couple of other big shops that have okay products. There's a smaller startup called Omega Point that has, I think, a good one. The idea here is that you can generate alpha, but I use the example of Tesla and Microsoft. So if you invested in Microsoft over the last like six months, you created a positive return, but almost all of that positive return is beta. There really was no alpha in Microsoft. If you invested in Tesla over the last six months, it was almost all alpha because Tesla is not correlated really in any way to the broader market or momentum or any of those other kind of like main factors.

• 40:03Patrick O'Shaughnessy?

  Just to be sure everyone understands, can you just describe like the basically the math behind what you just said? So how do you decide, why is Microsoft just beta? Why is Tesla more alpha?

• 40:12Speaker 03

  Yeah, it would be like, what is the return of the stock outside of the return of like a portfolio of momentum names? Or yeah, in the case of like Tesla, was it all momentum? No, it wasn't. Or was it all value? Obviously it was value. But Microsoft was basically like, there was some value in there. There was also some like general market, you know, beta like to the S and P exposure. It was stuff like that where if you wanted Microsoft, you could have just gone invested in like a smart beta ETF and gotten the same thing.

• 40:43Patrick O'Shaughnessy?

  Yeah, this is a really interesting development in this whole world where for a long time, alpha was knowing about

• 40:48Speaker 03

  Knowing about beta.

• 40:49Patrick O'Shaughnessy?

  About value momentum. Now maybe it's choosing or being patient in them. So there's the patience component too. But what's interesting is that the appetite now is a lot for certainly in the head, really specifically in the hedge fund space, pure idiosyncratic risk, which sounds bad, but really means something truly uncorrelated to what have become very cheap sources of market exposure.

• 41:13Speaker 03

  I don't have a complete disbelief in leveraging beta. I think there are times that the PM Should have a bit of a macro view on where the market is. I remember, you know, my mentor, David Geller, he certainly had an overarching, like, he would look at new highs, new lows. He would look at breadth as indicators of, you know, directionality of the indices. And we would take exposure on the long and short side, up and down, you know, relative to that on a, on a larger macro scale for the book. I think this is where PMs should focus more on the unstructured nature of decision-making.

• 41:55Speaker 03

  Is, is that like, how exposed do we want to be at, at different times? Again, this is the alpha that they're adding. It's not the pure ideas, it's the like, construction. Most of the construction should be done by the machine at this point, these systems. And what the PM can do is look at the system and say, okay, the system has completely beta-neutralized all of my positions. So my book is like, beta-neutral. I'm just gonna, you know, generate alpha theoretically, based on all the, like, you know, historical betas. But I really think, you know, momentum's gotten hammered here over the last couple of months. And I think that we're gonna have a really good earnings season and, you know, some of these other factors. And I think momentum's gonna make a comeback. And they can go into the system and say,

• 42:35Speaker 03

  I want to ratchet up the exposure to momentum here. And I think that's a good thing that they can do this. And look, they're gonna get measured. And every year, we're gonna look at the amount of alpha they created by toggling things up and down. And they're either gonna get fired or they're not, based on how good they are. But the machine should do most of the portfolio management. The other part that the PM can do is sit there and say, okay, my analysts have this really big outlier long pick. And all the factor model scores like line up and they're bright green. I want to size up on this thing right now. And they can take an outlier, large position and swing for the fences. And this is where I really think the machine and man kind of convergence allows

• 43:20Speaker 03

  the discretionary firms to potentially put up better returns than the quant.

• 43:25Patrick O'Shaughnessy?

  How do you though account for what, so we talked about how to evaluate a PM, right? Which is to say, okay, versus an equal weighted allocation to the analyst's long and short picks, what value did they add or subtract? How do you count for the fact that these kind of big fat swings represent typically a very small sample size? Like, how do you draw, and maybe it's, maybe there's just no way, right?

• 43:46Speaker 03

  Yeah, there's, I mean, it's, look, that's, that's supposedly the value that the PM is supposed to add, or else, why don't we just have a quant? Like, why don't we just be quants and have the analyst be the human input to the quant model, right? Like, it's, it's one or the other, and the PM can't get away with it. Like, at this point, you either hold the PM's feet to the fire and measure like that, or, well, I just don't have a PM, just have a quant run it.

• 44:11Patrick O'Shaughnessy?

  One of the things we haven't talked about is the role of a CIO. In a lot of the stories you've told me offline, a lot of these meetings you've had that have led to this piece that you wrote and this very conversation, the CIO's in the room. So, this will be a good switching point into us discussing organizational structure and sort of strengths and weaknesses, the roles that there should be, what, let's say, hiring managers should look for in those roles, what allocators should look for when doing due diligence, and so on. We'll start with the CIO. So, what do you view as the role of a really good CIO as it relates to a team of PMs?

• 44:44Speaker 03

  Yeah, I mean, I think the CIO at this point has to be the one with the strongest backbone, because they're gonna have to stand up to the PMs. And in many ways, you know, the, the PMs are your firm right now. I really think the analyst should be your firm, and not the PMs. The CIO is going to have to expend political capital and piss some people off. Now, my view is that PMs are mostly mercenaries anyway, and so, I think that in the future, PMs are gonna wanna work at the firms where the CIOs have a good grasp on this process, because you might have 15, 20 analysts at a firm, or if you're at one of these really

• 45:24Speaker 03

  big long only funds, like, you know, Fidelity or Alliance Bernstein, or, you know, whatever it is. You're going to have a lot more analysts than that and a lot more PMs. And if I was a PM, I would want to go to a firm that has a CIO that is willing to have an iron fist grip on this process and actually willing to put, you know, these processes in place so that I can get measured better and that I believe that it's fair what I'm getting in terms of comp relative to the other PMs. And I think that has to, that has to happen at this point.

• 45:56Patrick O'Shaughnessy?

  So let's go down the stack then. So obviously, we've already talked about CIO, portfolio manager, analyst. That's pretty clear. Let's talk about the quant roles that exist, both on the data and the research side. I'll just let you outline kind of what, who they are, what they do, who do you hire first? Because this is the nitty gritty of, we are into tactics a little bit now, but that's okay. So talk about the various quant roles, quant and data roles.

• 46:18Speaker 03

  Yeah, so I, I think you need to start with the data engineers because they're actually the first ones that are going to be flowing data into the analyst's kind of forward-looking decision-making process. And so these are classic kind of engineers that understand how to deal with data. They're not all that expensive. Then you need this data analyst. And this is actually, I think, the hardest position to find because it's such an interdisciplinary set of skills. You have basically an understanding of being an analyst, first off, what drives stocks and the models that the analysts are building and all that. But you also have to have some technical skills, be able to talk quant with the quants, and have some technical skills, you know, regarding being able to manipulate data in helping the analysts out.

• 47:02Speaker 03

  These people are few and far between, but they're, you know, there are more and more of them these days. You think about Matt Ober who went to Third Point, who is technically the, the head of quantitative something, something over there, or the head of data science. But, but really, even though he runs the whole group and there's somebody who needs to run this group, he really acts as that data analyst for all the different, you know, PMs and analysts over there. It's, and there's a reason why Third Point is up so much this year. And some of those things have to do, and I know this directly, like have to do with Matt and what he's done over there. So there's the data analyst. Then you have your, your pure quant. And these are the guys that are literally taking the data from the data analysts and

• 47:45Speaker 03

  doing research, like pure research into factor models and helping the analysts with like, yes, like this data set that I got from this credit card company is correlated to EPS and revenue when the company reports, you should use it. And then you have like the most expensive one, which is your quantitative engineers. And these are the guys that need to have a quant background, but also a deep engineering background, and they're going to be actually creating the factor models. They may also work with external companies, you know, like ours to integrate other factor models from third parties, but they're really expensive right now because they could go work basically anywhere.

• 48:21Patrick O'Shaughnessy?

  So there just aren't enough of these people. And let this serve maybe as some, some career or job advice. One of, one of the interesting things is when you find someone that can span or kind of straddle two different worlds and translate between the two, that's an incredibly valuable skillset.

• 48:37Speaker 03

  My, my interns this summer from Wharton, like that I'm telling, and they have these quantitative skills. And then a couple of them also have these like really good interpersonal skills. And I'm like, yes, like go study both and become the center of that.

• 48:51Patrick O'Shaughnessy?

  Let's go that direction a little bit more. So if you were an undergrad listening or a grad student listening and you're interested in, in asset management in general, and obviously this would depend on a person's inherent skillsets, but are there, is there one or more avenue that you think has the biggest like supply, demand imbalance in the kind of long only in hedge fund world right now?

• 49:12Speaker 03

  Right now, there's just too many econ and poli sci people. And then they're trying to hire all these pure data, you know, science people that know nothing about finance. And the problem is like, you really need somebody who knows both. So we have a, we have an intern this summer from Chicago, University of Chicago, and he's studying, you know, econ and comp sci. And I'm desperately trying to get him to also add some poli sci courses. Because if you, if you can't write and you can't communicate an idea, you won't, you won't be able to be the center of that kind of thing. And he doesn't want to be a pure quant and he also doesn't want to be a pure analyst. Like he would be perfect for this role, but you, you also have to have that skillset.

• 49:57Patrick O'Shaughnessy?

  It's amazing how, I was thinking about this the other day, that in any given hierarchy, the higher up you go, the more communication matters. You know, you mentioned backbone and things like that matter too, but it really is amazing. Like it's rare to see someone, even at the top of a really wonky, say quantitative stack, that isn't a pretty good communicator.

• 50:16Speaker 03

  Look at, look at Ross Guerin at Cubist. I mean, he's one of the best people in the industry and there's a reason his fund does so well. It's, you know, he's a great quant first, but he's also just generally a great manager and communicator and person. And you're not going to find, especially on the discretionary side, firms work. It's not going to be a pure quant at the top, I guarantee it. Like, it's going to be somebody with an interdisciplinary skillset.

• 50:39Patrick O'Shaughnessy?

  It's also encouraging that that's a skillset that you really can't, like, almost anyone can improve a lot in that skillset. You know, you can go read books on writing, on communication that really do actually work. It makes me think of, back to ESTIMIZE again, and this will just be kind of a fun tangent, which is, so you've got the ingredients for all this, which is the subjective inputs, the differentiated inputs, the ability to track which subjective inputs are good at which names, by what degree, and a software mindset. So it just begs the question, why not turn this into an active strategy?

• 51:18Speaker 03

  Yeah. I would love to someday go back to running money. I'm jonesing for it. But I view our next step in the evolution of what we do at ESTIMIZE as adding the multiple, which we're about to do, being able to aggregate up to full price target. Of course, we already have the big statistical engine that measures all these things and spits out confidence scores and intervals and stuff like that. We can already build factor models, which we do. But I want to give these firms the centralized dashboard now. And that's kind of our next big, probably two to three years of this company, is trying to be the center of that process for these firms. And it's obviously a big addressable market, which we're excited about. But more, I would really, just intellectually, I want to see if I

• 52:01Speaker 03

  can help solve this problem. Because if you can help solve this problem, you're going to get two things. Like, some of these firms, and I really honestly think it's only 20% to 30% of them are going to survive the shift, but you're going to get a bunch of these people from the old firms creating new ones and starting fresh with a process. And I think those are the people who are really going to win. And I'm excited to be friends and colleagues, in some sense, with those people who attempt to do that and be a big part of their success at the end of the day.

• 52:33Patrick O'Shaughnessy?

  Can you share at all the, some ideas around what you've observed in the dispersion of skill within the estimated data set? So I'm always curious to hear like, okay, so of course it's intuitive that, you know, one person's going to be better than the next and better than the next. But like, how tightly, how narrow is the most skilled group? Like, is the best two or three guys orders of magnitude better than, you know, the, the, the next five guys and the next 10 guys? Talk a little bit about like the distribution of skill that you've observed as your data sets gotten bigger.

• 53:08Speaker 03

  We were really interested in this from the beginning, mostly just because my co-founder and I are data nerds to begin with. And this was, in a sense, an academic experiment. We were not surprised to find that it is a normal distribution of, of skill. Although what happens, and this is one of the reasons why the data set is so good, is the people on the left side of the distribution who suck tend to show much more attrition at a rapid rate than the people, obviously, on the right side, which you really want to keep around. And theoretically, this is what you should want at your firm as well, right? You should measure and then get rid of the bottom 20 or 30%, like every single whatever interval. In terms of how much better, I don't really have like the, the pure statistics around that.

• 53:53Speaker 03

  But I can say, we were a little bit surprised at the, the persistence of who was better and who was worse. Analysts who come in are persistently better than the ones who are kind of more random right in the middle.

• 54:09Patrick O'Shaughnessy?

  Yeah, that was going to be my next question. Like, to what degree can you parse out luck versus skill?

• 54:14Speaker 03

  Really well, more than we thought we could. We thought that this was a more random environment than it is.

• 54:19Patrick O'Shaughnessy?

  And what about specialization? So do you find more accuracy among people who only enter, say, estimates for one sector versus more generalists? We, we skipped over this concept of generalists, which I know you have a strong opinion of. So I'm curious first, before we get to your opinion, I'm, I'm more curious about the relationship between specialization and accuracy.

• 54:40Speaker 03

  Yeah, you want, so we know specifically that our confidence- Scores and thus the select consensus model that weights each estimate in the consensus. Those models are dependent on your accuracy within that sector and have nothing to do with accuracy outside of that sector. Because we found zero correlation between accuracy in one sector and another. I'll bet that if we looked very, very closely at the data that there would be correlation between like tech and consumer. Or like energy materials, but it doesn't, you don't need to go that deep in order to literally just say. That analyst has this score and this Z score in this sector and you're going to overweight them, but in that sector, you know, don't pay attention to it. So yes, specialization is incredibly important. The other one was number of estimates was highly correlated.

• 55:24Speaker 03

  So we find that analysts who made more than 10 estimates, but less than 50 per quarter. Yeah, companies, yeah, companies were more accurate than people who made less than 10 or more than 50.

• 55:37Patrick O'Shaughnessy?

  Very interesting.

• 55:37Speaker 03

  Yeah, it seems to be relatively intuitive in the sense that like, if you're doing more than 10, you're probably like. You're being thoughtful, you're seriously involved. If you're estimating like one or two, you know, we don't have that much confidence in you because maybe you're just coming to do apple or whatever. If you're doing more than 50, you're probably a generalist and like spreading yourself a little bit too thin and. You don't really have as much conviction in your ideas and you're not paying as much attention.

• 56:00Patrick O'Shaughnessy?

  Talk about the pros and cons of the generalist idea. It's a term that, that is always asked, right? In every meeting and every due diligence questionnaire is, are you looking at the world broadly? Or are you very narrowly focused on material stocks or whatever, whatever it is? So obviously there are pros and cons. What, where do you, where do you kind of fall out in terms of generalist versus specialized analyst?

• 56:23Speaker 03

  Yeah, I think the analysts need to be super specialized. There are so many different variables associated with each individual sector and industry. The only reason to be a generalist is if you're a PM, in which case you should be. But then even then, hopefully your fund has a focus on a specific sector or industry and you're not just like doing everything across the board. Or if you're at one of these, you know, big, long only shops, you're not dealing with every sector. Like you're in charge of a book that looks at these things and it's the CIO's job to like allocate amongst different sector books or whatever. So I think the PM should understand markets in general and all these things. The analyst should come out of industry. And I've seen, the best analysts that I know in tech, some of these guys, if they're in enterprise tech, they worked in Silicon Valley. Like, they know all the players.

• 57:08Speaker 03

  They have friends at these companies. You need differentiated information or a differentiated understanding of this stuff in order to do this. If you trade energy names, you better have come out of working at one of these companies or at least been a consultant to them in the past. I don't think there's any other way to do it at this point.

• 57:28Patrick O'Shaughnessy?

  One thing that you've brought up both times that we haven't dug in on, I remember in college, my favorite, literally my favorite class, they technically called it a philosophy class, was just war theory. I would love to hear a little bit about your background there. What drew you to it? And I would love to hear some of the key things that you remember in terms of frameworks and lessons, maybe that you still draw on.

• 57:50Speaker 03

  I mean, we talked a little bit about one of them, which is all war is politics by other means. And what that basically means is that when you get involved in anything, know what the final outcome you want is. Don't just say, "We're gonna go fight a war against terrorism." That's a war on a tactic. That's not a thing. You're not alluding to the core of what you want. Always know what you want out of something. I mean, obviously, some of this stuff, there's a big part of what we learn, which is Clausewitz, and there's a big part of what we learn, which is Sun Tzu. Those are two sides of the same coin.

• 58:34Speaker 03

  But the Sun Tzu stuff is more like, know you've won before you've gotten into something. When you go to make a bet, put all the odds in your favor and know that you're making a sound bet there. Don't go to war and guess at whether you're gonna win. The thing that really drew me to the whole space was, I was also studying behavioral economics at the time on the other side, so poli sci and econ, and two different specializations. And in behavioral economics, you learn that markets are incredibly irrational because humans are irrational. And we learnt a very specific side of that irrationality and then how you can take advantage of it. On the war theory side, you learn how people behave

• 59:16Speaker 03

  in that realm. And I love stories about people and generals and decisions that they make. I named my two cats Hannibal and Alexander after Hannibal of Carthage and Alexander the Great. And it's not like, I'm really not a blood and gore kind of person at all. It's more the strategy associated with, they were innovators. Like, these guys were innovators in understanding the tools that they had and the outcome that they wanted and how to get from one to the other.

• 59:49Patrick O'Shaughnessy?

  Can you describe maybe a story or something specific for what their innovation was?

• 59:54Speaker 03

  Oh, man. I mean, Hannibal of Carthage, you know, there's a famous, and I'm gonna forget the name of the battle, but there's a famous battle where he basically outflanks the other army and comes up with basically one of the core tactical moves of kind of ground-based, hand-to-hand, you know, man-on-man combat. And it gets copied by generals for another, you know, thousand years.

• 1:00:22Patrick O'Shaughnessy?

  This is Battle of Cannae?

• 1:00:23Speaker 03

  Yeah, that rings a bell. I'm forgetting my exact history here. But, and he wins an improbable battle with this, with this strategy. And there's, you know, it's everything from just the how he comes up with it and how he implements it. Those kind of things are really, you know, interesting to me because in many ways, they dovetail with how we come up with investment strategies, what we believe in, what everybody else believes in at the time, you know, those kind of things.

• 1:00:50Patrick O'Shaughnessy?

  Yeah, I was just gonna say that the parallels are eerie, right? In terms of hierarchy and structure, you know, generals, CIO, analysts and grunts, you know, there's all sorts of parallels. And it's, my study of military history is much more limited than yours, but it does seem like the most interesting episodes Are always about the development of new weapons or strategic technologies. I'll call it strategic technologies. And you know, blitzkrieg's always been my favorite, which is this kind of discombobulating way of approaching battle. That's just incredibly philosophically interesting when you don't expect it. Then everyone gets used to it and it's an arms race like anything else. It seems like a great field of study. It's good inspiration for me to go back.

• 1:01:35Speaker 03

  There's another one for today specifically, I think is really important. Parallel, which is in the military, unfortunately, and we've suffered from this in the past as well in the US. We tend to elevate generals and colonels, you know, through the ranks because they've won a great battle or they had a big outlier event.

• 1:01:57Patrick O'Shaughnessy?

  Performance chasing.

• 1:01:58Speaker 03

  It is directly performance chasing, right? And so what will happen is the general will make a big outlier decision and he shouldn't have, based on all of the odds and the facts. But instead of the like three quarters of the time when he would have lost that bet, he wins the bet and we end up elevating him in the ranks because of that, because well, you can't not like promote the dude who just won this great, you know, this great victory. And there are some people, and we know like just by statistics, there will be some people that flip heads, you know, five times in a row where it was complete luck and they did the wrong thing, but got the good outcome and they end up becoming like a three star general and now they're making a bad decision the next time.

• 1:02:41Speaker 03

  This happens all the time and this happens all the time in finance as well. And this is why I think measurement versus outcome is so important because you might flip heads five times in a row, but like, you know, the next couple of times it'll be completely random and you'll be like, why is this person not performing?

• 1:02:59Patrick O'Shaughnessy?

  Yeah, I kind of want to wrap up with that idea, which is that. If you can get your mindset wrapped around this notion of simple systems of measurement and simple systems of measuring outcomes and then just constantly wonder and be curious about what matters and what's predictive. First of all, you know, there's a couple of key things here. First of all, you're going to be surprised often by what matters and what doesn't. But second, you kind of need to work hard at the beginning to get yourself in the mindset of this. Like I use this app every day that's called Way of Life. It's just like a bunch of checklist stuff. And the last thing on the checklist is like basically well-being, yes or no. Like, do I feel a high sense of well-being, yes or no?

• 1:03:41Patrick O'Shaughnessy?

  And then you can go, so I've got almost two years of data now on this thing and you can go.

• 1:03:45Speaker 03

  What was correlated to my well-being?

• 1:03:46Patrick O'Shaughnessy?

  And see which of these variables is most correlated with my binary yes or no well-being. And it turned out that it was sleep. Like something incredibly simple and I tracked all this other stuff, you know, did I do this, did I do that? All this complicated stuff and stupid dumb like, did I get a good night's sleep was the most correlated to well-being.

• 1:04:05Speaker 03

  I don't, I don't, uh, yeah, I'm not surprised.

• 1:04:08Patrick O'Shaughnessy?

  But that, had I, had I been designing, you know, some life change ahead of time, sleep would not have been the thing that I focused really hard on getting better sleep. But now that's what I'm thinking about because I did one of these simple, simple systems. And it just seems like, you know, your message and, and we'll make sure, I think this might be released simultaneously with this, with this new piece of yours. The piece itself covers some of what we've talked about, but, but really gets into some deep detail about what people, what asset managers, portfolio managers, et cetera, should be doing. Um, so we'll, we'll post that and I encourage everyone to, uh, to check it out. Thanks, man.

• 1:04:43Speaker 03

  Absolutely.

• 1:04:44Patrick O'Shaughnessy?

  Hey, everyone. Patrick here again. To find more episodes of Invest Like the Best, go to investorfieldguide.com/podcast. If you're a book lover, you can also sign up for my book club at investorfieldguide.com/bookclub. After you sign up, you'll receive a full investor curriculum right away and then three to four suggestions of new books every month. You can also follow me on Twitter, at Patrick underscore Oshag, O-S-H-A-G.

• 1:05:15Speaker 04

  If you enjoy the show, please leave a quick review for us on iTunes, which will help more people discover Invest Like the Best. Thanks so much for listening. [Music]