Protege CEO Bobby Samuels on the data nobody has unlocked yet

Bobby Samuels is the co-founder and CEO of Protege, a platform that licenses real-world data from industries like healthcare and media to companies building AI. Protege has raised $65 million from Andreessen Horowitz, CRV, and Flex Capital. Before Protege, Bobby led partnerships at LiveRamp and ran Privacy Hub at Datavant. In this episode of Summation, Bobby and Auren discuss: The auction for Spirit Airlines' emails, and what a model learns from a failed airlineWhy synthetic data has been "18...

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  One of the things that we think about is not just like the letter of the law, but the spirit of the law of like, yes, there is data that we can probably use. As a company, our general take is the creators of IP should be compensated accordingly. You know, it's easy to build certain things and whatnot, but I think that companies should continue to focus on what they're uniquely good at and not try to spin up all these tools. If there's like highly sensitive data that you want to be able to look at before you license or can only be shared in very sensitive situations, you basically can create a synthetic data set that just allows somebody to sort of understand the general nuances and relationships of the data.

• 0:43Auren Hoffman?

  Hello, fellow data nerds. My guest today is Bobby Samuels. Bobby is the co-founder and CEO of Protege, a platform that licenses real-world data from industries like healthcare and media to companies building AI. Protege has raised $65 million from integration to Horowitz, CRV, Flex Capital, where I am. Before this, Bobby led partnerships at LiveRamp and also ran Privacy Hub at DataVant. Bobby, welcome to Summation.

• 1:12Speaker 00

  Thank you. It's great to be here and yeah, always good to see you, Orin.

• 1:16Auren Hoffman?

  Okay, now going back, I just want to start with this. Okay, I, Protege, I actually went back on my notes. I was lucky enough to interview you for a job there at LiveRamp when you were a senior in college. This is 2013, 2014 timeframe. I think that was your senior year. I literally just checked my interview notes a couple of days ago and it said, and I don't exactly remember what it said, but it said, here's the, there's an interesting conversation on the economics of horse racing. Do you have a recollection of this conversation that we had about horse racing?

• 1:49Speaker 00

  So I was thinking about when we first met before this and for whatever reason, and kudos to you, I think I remember more interview questions from our interviews than like any other ever, pretty much. Okay, so there were, there were, there were three questions that, that came up. One was that, um, which, so basically,

• 2:12Speaker 00

  I almost like, I was very like into horse racing. I actually wrote my college essay on like betting on horse racing. I, cause it's like, you know, it's, it's very data-driven and I was like, data, you know, I grew up, I like loved baseball stats and like that permeated all this.

• 2:29Auren Hoffman?

  Were you betting on horse, horses and stuff?

• 2:31Speaker 00

  Yes, yes. I mean, I had like my, my dad, cause I was not technically legal. So I guess like, you know,

• 2:37Auren Hoffman?

  Your dad was placing the bets on your behalf?

• 2:40Speaker 00

  Yes. But then like, as I thought, I started writing a thesis and then decided ultimately, not to like, there was a lot around, uh, basically economic theory as a related to betting on horses. And it's actually like, there's a lot of interesting parallels now where, uh, getting data for horse racing is actually like incredibly difficult and expensive because there's like one company that essentially has a monopoly on it.

• 3:07Auren Hoffman?

  Oh, really?

• 3:08Speaker 00

  Yeah, called the Daily Racing Forum. It's like very, so one of the things I had to, actually like the, probably the hardest thing for the, for this was going to be getting the data, but there were interesting things like.

• 3:17Auren Hoffman?

  Susquehanna like bets on horse racing too somehow, right? Like even like companies like that.

• 3:22Speaker 00

  They just like pay a ton for data. Yeah. So, I mean, there are a lot of like, actually equivalents to things I think about now. So, uh, one was how difficult it was to get the data. The second was like, there are biases. So there was this thing that I think has been like more or less arbitrage away called like the favorite long shot bias, which was favorites were chronically undervalued, long shots were chronically overvalued.

• 3:45Auren Hoffman?

  That's true, like on Polly Market still, like they always have, they, they show these biases on Polly Market all the time.

• 3:50Speaker 00

  Yeah. Cause it's like, oh, it's a hundred to one or a thousand to one, like might as well. And then it ends up being like, actually it should have been a thousand to one, but because everyone did that's nine hundred to one or something. Um, and then the, uh, uh, the third is much like a FICO score or Nielsen rating. There's actually like this statistic that has developed called the Bayer Speed Rating, which is essentially this guy named Andrew Bayer. He also, he went to Harvard, he dropped out because he wanted to do horse racing stuff, created this like composite stat that is like used by some group is like the number that distills all these different variables is like the agreed upon speed rating for a specific race. So there's actually like all of these pretty interesting parallels.

• 4:35Speaker 00

  A lot of the data businesses.

• 4:36Auren Hoffman?

  So that's like the Black Shoals equivalent of horse racing or something?

• 4:40Speaker 00

  Exactly. Yeah. Yeah. So that was one. That was one question we talked about. The second was, uh, what do you think about Bitcoin? Um, so this is 2013.

• 4:52Auren Hoffman?

  Oh, that's a good question in 2013.

• 4:54Speaker 00

  And I said, I don't really get it. I don't know. An answer that I stand by today. I still don't really get it.

• 5:00Auren Hoffman?

  I don't get it either, I guess.

• 5:03Speaker 00

  Yeah. So that was the second. And then the third was, I think it was like, what is a good heuristic for what, how much of a risk taker somebody is?

• 5:12Speaker 00

  The answer I had was, what is their first job after, out of college, which just felt particularly top of mind because we were talking about my first job out of college. But, um, anyway, so three questions that, that stuck with me, which is three more than the vast, vast, vast majority of interviews I've had.

• 5:25Auren Hoffman?

  You, you don't strike myself as, as someone who's, who's a risk-taking personality, right? You're not jumping out of planes and stuff like that.

• 5:35Speaker 00

  No.

• 5:36Auren Hoffman?

  And so how, how do you think about, because there's this trope that founders are risk-seeking, but so many founders I know are actually risk averse.

• 5:45Speaker 00

  Yeah. I think it's like, the way I think about it is what is, like, what's the right sort of unit of risk you're looking at? So from my perspective,

• 6:01Speaker 00

  you know, it's, yes, on paper it is like risky to start a company, but I, all of the essentials in life I have, you know, like I have family, we have like the resources we need to live life. And so like from that perspective, it's not actually like at a fundamental level, it's not that risky.

• 6:19Auren Hoffman?

  Get a job or whatever, right?

• 6:21Speaker 00

  Totally, totally. And so like, I don't.

• 6:23Auren Hoffman?

  The downside of failing a company, a failing company is, you know, not that bad, you know, it's not that bad.

• 6:29Speaker 00

  Right.

• 6:30Auren Hoffman?

  Exactly.

• 6:30Speaker 00

  You just go do something like, and so, um, so to, to me, there was like almost a bigger risk of spending a bunch of time not wanting to go after something that I wanted to go after.

• 6:42Auren Hoffman?

  Yeah.

• 6:43Speaker 00

  Whereas like, yeah, I'm not going to skydive, um, both because it's like a risk that doesn't make sense to me. Also my mother would kill me. So, uh,

• 6:52Auren Hoffman?

  You risk to that relationship. You have a risk to your relationship with your mom.

• 6:56Speaker 00

  Yeah. Which, which we won't be able to, uh, fully unpack in the, in the course of a podcast episode, but, but yes, that is, that is correct.

• 7:04Auren Hoffman?

  Okay, that's fair. Okay. Now you are, uh, Protege, you're in the data business. Uh, you basically go get data that was never on the internet. Ideally, right? And then you give it to the models. Like, why can't the, why can't these AI companies go get that data themselves?

• 7:24Speaker 00

  The short is like, it's just, it's hard. So we have hundreds of different partners and a lot of that comes from pre-existing relationships that either I had or folks on the team. We have a handful of ex-Hollywood licensing execs who have gotten Hollywood, like data from Hollywood and sports leagues. And there's, so there's just a huge effort in going and procuring data that the labs would set like, having one to.

• 7:54Auren Hoffman?

  Is it like, you need all these data sets together to combine or could you actually license one and actually would be valuable or something?

• 8:01Speaker 00

  So I think in healthcare, which is our, their biggest area, I would contend that it is faster, cheaper and higher quality to go through us than to try to do it yourself. So we know all the data sources or a lot of them. We understand the linkages between them. We understand what's the process by which it needs to be curated. We understand the process by which, how you think about the identification. There's certainly some risk buffering too, where we are taking on certain risks around the identification that they don't need to.

• 8:33Speaker 00

  We have the research chops in-house to really be able to prude and curate the data. And so you add all of that together. Um, and by us becoming this sort of one-stop shop for all of that, they're able to get data so much faster. And because we know the data, higher quality. One of the things that I suspect we'll talk about sort of differences between data worlds that I've seen before and the AI world is just the speed of the AI world is so intense that the difference between doing something in a month versus doing something in three months in the world of pharma isn't a huge difference. In the world of AI is the difference between, does this make this training run or not?

• 9:17Speaker 00

  And does my model have a totally different look? And so the idea that we can save time and have higher quality, which requires less research time, huge, huge, huge deal. Um, and so, uh, so I think there's, there's that piece. When we do a good job with, with researchers, it very quickly pivots from, we need this thing, go get it for us to, okay, we trust you. What are the next areas of inquiry we should be thinking about? And so then it moves into more of this like true thought partner advisory capacity where we really can be sort of an extension of their team.

• 9:50Auren Hoffman?

  I mentioned the healthcare data, like part of, to me, value in data comes with, as these data sets are joined, they become more valuable because you can ask like interesting questions across data sets. Is that, am I on the right path here? Is that like part of the values? Like you're starting to join these data sets together and, and you're kind of like selling a bigger data set as you join it.

• 10:13Speaker 00

  Totally. Yeah, absolutely. And that, you know, we work with DataVant, it's part of the LiveRamp DataVant DNA. But it's, it's very much multiplicative in terms of its value and longitudinal multimodal data is, continues to be hugely, hugely impactful, uh, uh, in how models need to be trained.

• 10:34Auren Hoffman?

  And there's a lot of government data, which is amazing. Super longitudinal data. You look at the CMS data set, the IRS data set, et cetera. Obviously has a lot of privacy things, but could the government do a better job? I mean, obviously just purely for society of releasing that data in a de-identified way or allowing that data to be queryable in some way to help research.

• 11:02Speaker 00

  Yeah, and this comes down to, you know, probably someone's, you know, individual folks take on sort of utility and privacy trade offs. I mean, my take is yes, if done in a really thoughtful way, if, you know,

• 11:15Auren Hoffman?

  You can almost certainly have your cake and eat it too. You could almost certainly protect privacy of every extreme case. And, you know, it takes, it just, it takes some thoughtfulness of doing it, but you could probably release it in a way that does protect privacy completely.

• 11:32Speaker 00

  Yeah, there is this like privacy utility trade off where, you know, you could have total privacy and strip everything. You could have total utility and strip nothing. And so it's how do you, how do you navigate that? But I think there's also this like softer question of who are the parties you trust? Where there are probably a handful of folks who you would trust with more data access. Where if they're either smaller parties, uh, or, you know, folks who don't have a great track record, you may not, uh, it's, you know, if, if even with things that are de-identified, if someone makes a business out of trying to attack it, that still is pretty scary. But I think there's more that the government could do in releasing data.

• 12:12Speaker 00

  Uh, and ultimately the more data, especially on the healthcare side that can be released, the faster we can get to better treatment, better cares.

• 12:19Auren Hoffman?

  Yeah, and Raj Chetty has access to the IRS data and he's written hundreds of really valuable papers based on that, right? It's a super longitudinal study of hundreds of millions of Americans. It's incredible. Uh, and I think a few other people have access to it and that's maybe five total people or something.

• 12:38Speaker 00

  Exactly.

• 12:39Auren Hoffman?

  Have access to that. And that's great. I mean, all those people are amazing and they're super smart, but like, I don't know. It seems like we should give more than five people access to that data.

• 12:47Speaker 00

  There's probably more than five people who we could trust with access to that data. Yeah.

• 12:50Auren Hoffman?

  Yeah. And again, I think we could completely de-identify it. So, uh, maybe there's some, maybe not giving access to everybody, but there's like some level of, of data that gets out there. Now, okay, you have these models, okay, these models are getting smarter. Um, and then there's obviously this data that was on the internet, which they kind of initially got trained on. How quickly are they moving to what the new data set is? Like, okay, like we need this data and now we need this other data set. Like how, like I remember back in the day, this is, you know, even before you and I met, where you would be training on, uh, pictures of like a cat and not a cat or something like that. Like that, that was like the state of data, uh, 15 years ago or something.

• 13:34Auren Hoffman?

  Now we're like much further along, but obviously that data is not valuable at all anymore. There's no value in that kind of data. So how quickly does like the, the, the value of data move?

• 13:45Speaker 00

  It can move fairly quickly. And a lot of this is, to some extent, some of these things operate like the speed of different training runs. Because now you have, these are such massive investments that what we see is slow, slow, slow, and then a training run looms and then there's like just this mad dash.

• 14:05Speaker 00

  And so there is like, I'd say there's the consumption of data spikes around that time, at least at a pre-training scale, post-training tends to be sort of more fine grained. But we do see data that, uh, over the course of a couple of years, depending on the type of data, does, uh, depreciate or become more saturated.

• 14:30Auren Hoffman?

  Your company's only been around for roughly two years, right?

• 14:32Speaker 00

  Yeah.

• 14:33Auren Hoffman?

  Um, so is there data that was very valuable two years ago that has, you know, 10% of value today? Or how quickly is it decaying?

• 14:43Speaker 00

  Data like, we, we see fewer, but like bulk text data. Um, that's probably one where we see less and is a little bit less, uh, has a little bit less value. And so there's some depreciation there. Um, I'd say in general, health data has not shown the same level of depreciation. Uh, which makes sense in part because you always need the latest and the way people are treated changes. I think that-

• 15:12Auren Hoffman?

  Well, it's temporal essentially, right? It's not a static data set.

• 15:15Speaker 00

  It's temporal, but I'd say the, the more unimodal things are, the more, uh, depreciation there's been. Like I think the cat example is, is where that's very unimodal.

• 15:28Auren Hoffman?

  But as you move into like- You solve it. Now you know that's a cat.

• 15:31Speaker 00

  Yeah, exactly. But the more you can think about, okay, this is, uh, there is a video with audio associated and then you can add scripts and then there's all that, I think has more durability. Than anything that's just purely unimodal.

• 15:47Auren Hoffman?

  And there was a point a couple of years ago where Google licensed Reddit data uh, for quite a lot of money. Um, and I think that, that, that is going to come up again for a relicense soon. I assume it's just way less valuable today than it used to be, or do you think it, it, there is some value? Because there's some exclusivity that they're getting or some other type of thing.

• 16:10Speaker 00

  So for some of these things, I would differentiate between training and like rag applications. And so for training, I'd say probably, you know, there's more text that's coming, so probably a little bit less valuable. But for them, for, for Reddit, my understanding is a lot of the value comes from the sort of rag type applications. Or if I ask a question, it's not a training thing. It just needs to, you know, I need like a specific answer. And a specific answer may come from a Reddit thread that was two days ago. So I think that, and that's closer to like a almost perplexity type component of it. Um, so I imagine Reddit data will continue to be valuable,

• 16:50Speaker 00

  but the apportionment of that value probably changes from pre-training, um, more to like those sort of rag type applications.

• 17:00Auren Hoffman?

  And Google also recently licensed, um, the Spirit Airlines data where they got all like the emails and I assume Slack messages and basically employee communications between one another. How, how, what, what do you think about that? And that, that seems like at least like du jour, look cool as data right now. I don't know if that will be hot anymore six months from now.

• 17:23Speaker 00

  Okay. One, one quick question. Do you think there's, and I've been thinking about this, do you think there's any way Spirit used Slack? They're definitely, that's definitely a team shop, right?

• 17:31Auren Hoffman?

  Okay. Yeah, probably teams.

• 17:32Speaker 00

  No, no, no, we're just, we're just curious.

• 17:34Auren Hoffman?

  Yeah.

• 17:35Speaker 00

  Yeah.

• 17:35Auren Hoffman?

  Yeah, almost certainly.

• 17:36Speaker 00

  There's probably some tech team somewhere.

• 17:38Auren Hoffman?

  If they use Slack, the airline would still be around probably, right?

• 17:41Speaker 00

  Exactly, exactly. Yeah. It's like a good, it's a good harbinger of like-

• 17:44Auren Hoffman?

  Yeah, and they probably use Outlook and stuff like that. You know, they're using Outlook and Teams like your company is. It's probably, your days are probably numbered, right?

• 17:53Speaker 00

  Webex, you know, all these things. So, so yeah, yeah. So it's super interesting. And then I don't know if you, like, Micro One has come over the top. And so Merckor said, I think it was like a seven and a half million dollar bid. Google had a $10 million bid and then Micro One has come in with a $12 and a half billion bid. It doesn't-

• 18:12Auren Hoffman?

  Oh, I didn't remember that. Okay.

• 18:13Speaker 00

  Either I haven't seen the latest or there hasn't been clarity on it.

• 18:16Auren Hoffman?

  So it hasn't actually closed as of this taping of this podcast?

• 18:20Speaker 00

  Not, I don't think so. Yeah.

• 18:21Auren Hoffman?

  Okay, got it. Okay.

• 18:22Speaker 00

  Publicly lobbying, but exactly. Like ultimately, part of our thesis from the beginning has been, uh, real world data is what's most impactful for AI. So the way we think about it, you have three types of data. You have, uh, you have manufactured data, which is data made by people for AI. You have synthetic data, which is data made by AI to train AI. And then you have real world data, which is people living their lives, companies building their products. Ultimately, that type of data, our thinking is what's most impactful for training AI. In addition, enterprise workflows are clearly what's driving the most consumption to date. And Anthropic's meteoric rise is, is evidence of that.

• 19:05Speaker 00

  And so getting actual corporate, like data is from actual companies has taken off as an area of interest. And so I think this really caught, this Spirit Airlines, uh, deals really caught everyone's attention. And I think has been the most public demonstration of this. Uh, but there's a ton that is already going on, that had been going on behind the scenes. But I think this sort of brought it into the spotlight.

• 19:35Auren Hoffman?

  Yeah, it's interesting because you could see some companies where it, it's possible that the remnant data that they have is worth more than the actual, even if it's like a for-profit company that's making money, not a bankrupt, where the remnant data actually could be more, but it's just like, it's just there. And so you could buy a company for X dollars and immediately sell the data for X and then you have the whole company for free or something, you know?

• 20:03Speaker 00

  There's a, um, some of the companies that are like Sunset and things like that, that help companies go out of business like this.

• 20:10Auren Hoffman?

  Yep.

• 20:10Speaker 00

  They're public.

• 20:11Auren Hoffman?

  That's how they make money. Yeah.

• 20:12Speaker 00

  Yeah. And like they're selling the data. I mean, I actually, it's an interesting thing for like nonprofits to think about where this is like material, could be like a material revenue stream for them. Um, if they, if they're licensing some of this data.

• 20:26Auren Hoffman?

  Oh, interesting. And yeah, that, that could, and also probably like, there's a whole other set of ways you're interacting in a nonprofit that might be different than a for-profit or something like that.

• 20:36Speaker 00

  Yeah. I mean, that is one of, one of the jokes we have internally is the issue with buying Spirit Airlines, Spirit Airlines data is like, you're going to teach the models to be really good at operating a failed airline.

• 20:46Auren Hoffman?

  Right, right, right, right.

• 20:48Speaker 00

  And so you might, uh, run into the same thing if you're-

• 20:50Auren Hoffman?

  Figure out how to charge the, uh, consumers for literally every single thing that they do, including a bag of peanuts or something. Yeah. Perfect, perfect.

• 20:58Speaker 00

  Just what I want from our AI.

• 21:00Auren Hoffman?

  Uh, like about 20 years ago, one of my friends was doing early version of AI. And the way he was training the data was the Enron emails. Um, I guess all the Enron and already those were those, that was already five years old, 20 years ago. And he was, but those, those were all released. And so you could see all the interactions or maybe not every email in Enron, but a whole bunch of them were, were released. And, and so it was public and he was able to get access to it and it was, it was not easy to parse it. He was able to parse it all and then train it. Even back then, it was like considered something like really valuable to, to use.

• 21:39Speaker 00

  Totally. And that I, I, I think I've heard of other folks, I mean, I think it sounds like he was on the earlier side of doing that. But I've heard of other folks doing that again, like it is this meta question of what are you ultimately? Uh, what are you teaching the models to ultimately do if that is a key part of the, the corpus? Um, and so one of, I think as we think about alignment, which has come up a lot in the last few weeks. One of the things for folks to keep in mind.

• 22:10Auren Hoffman?

  No, I mean, in some ways you are a store, right? You're going sourcing interesting goods and then you're selling those goods multiple times. Uh, and you're, you know, part of what you do is, um, just being able to do procurement and sourcing it. And then what, how does like a data store end up being very valuable? Um, and it's, and it's, it's somewhat, you know, some of these things go out of fashion. So it's like a fast fashion store. It's a Zara or something like that.

• 22:41Speaker 00

  Yeah. One of the, either a handful of things. And the way I conceptualize us is sort of this store and also this like research lab smushed together. So we have a large group of researchers called a data lab led by our chief scientific officer, who does research on data and, uh, and papers and interacts with researchers at, uh, at labs, but treats data with the same seriousness and rigor that model builders treat models. And so I think that's been very important for us in building out these researcher to researcher relationships at, at different labs. But I think it's, it's a combination of continuing to do what we do well, which is sourcing new data,

• 23:23Speaker 00

  continuing to deepen these relationships, continue to think in, into new verticals. But we're also moving into different parts of the, the stack. And so we're making a big push into benchmarks and evaluations.

• 23:37Auren Hoffman?

  Yep.

• 23:38Speaker 00

  And, you know, I think, I think right now we see, uh, data is sort of the biggest bottleneck. I think that will likely be true in conjunction to how well are these models performing and how well are they performing on my data? And then from there it reinforces the, okay, if this is where it's missing and needs more data and you sort of create this loop.

• 23:57Auren Hoffman?

  And if you have a benchmark, you maybe have a currency and then a currency has value or something like that.

• 24:02Speaker 00

  Exactly. Yeah. That's, that's, so I, I think that's exactly right. Um, and so we're, we're thinking about it from, from that perspective where, and then, you know, the currency is both, you know, if done correctly, accepted, but also always needs to be updated. So I think if we can get that piece going, um, very positive release signals, I think it can be an incredibly valuable and, and durable business.

• 24:28Auren Hoffman?

  The, the best moats are marketplaces, right? Um, where you've got these buyers and sellers and in some ways that is also what you're building. Uh, and I assume though that there are a small number, I, I don't exactly know. Uh, but I assume there's a small number of buyers that are where most of the money is going to. And then there's maybe this long tail, you know, maybe five buyers have 80% of the money. And then there's a long tail of the other 20%. And that's not maybe the most enduring marketplace. Like ideally you have a broader set that make up the, the top 80%. Maybe it's 20 to 40 buyers or something that make up the top. Um, that would be a better long-term marketplace. Is that, do you think we're going to go there?

• 25:13Auren Hoffman?

  Or do you think there's always going to be a very small number of companies that are just like taking up most of the, most of the air in the, in the buying of the data?

• 25:21Speaker 00

  I think if you, if it's purely in the licensing, large scale pre-training, that's probably true. As you think of more about, uh, evals, benchmarking, enterprise.

• 25:35Auren Hoffman?

  Yeah.

• 25:35Speaker 00

  Exactly. Like.

• 25:36Auren Hoffman?

  You would think it goes, you would think a lot of people would want it, right?

• 25:39Speaker 00

  Yeah. I think that ends up going more distributed. Um, and look, I, I think, uh, you know, as we talk to investors, this is always the, the question that comes up. But I mean, others are very, you know, handshake and Mercor and others are, uh, very public about the move to, to enterprise. I think we need to be very sort of thoughtful in what's our game. Um, you know, cause I think, I think there's enough investment flowing around and enough dollars to chase that, uh, short-term like sort of revenue is, uh, in some ways easier to come by than sort of durable, long-term revenue. Um, and so that's sort of how we're approaching it and trying to build something, uh, longer lasting versus just sort of quarter over quarter success.

• 26:24Auren Hoffman?

  We found this with selling data like to hedge funds that it turned out there are actually only a very small number of hedge funds that were buying most of the data. Uh, and even very, very, very big hedge funds like Bridgewater bought very, very little data. Um, surprising how little data company like places like Bridgewater bought, but and the, there's like five hedge funds basically probably, probably about 70% of the data or something like that in terms of, of dollars. And so they had a lot of power. And also it turns out a lot of this data was like somewhat fungible. So yeah, each data set itself like could give you some alpha, but it, there was, there was usually another data set that was very similar to that one. And so it started to drive down the price.

• 27:09Auren Hoffman?

  It's like, oh, you're buying the credit card data or you're buying foot traffic data. They both maybe give you a sense of like how well the Starbucks is doing or something like that.

• 27:18Speaker 00

  And so I, I do think to that point, we're think, we're also trying to think through, and, and I think it's done well so far. Like what are the less, the, the rare, less fungible types of data? And, and how do you add value to that, whether through bringing more modalities to, to bear or building derived products on, on top of it? But yeah, I mean, the hedge fund one, it, it, it's something you see from being enough of the data businesses that whenever anyone has a slide of like, here's what we're going to sell to, and then it's hedge funds. It's like, just cross that out. Like it's not going to work. It's going to sound great.

• 27:48Auren Hoffman?

  Make much money. Right. Right. Um, and then now hedge funds also historically like gave very little feedback back to the, the, the data seller. And so it was hard for the data seller to make their data better. Cause they wouldn't actually understand like how it's working. And so the, it is, in some ways it was bad for the hedge fund because they weren't getting the data, the data better. But it was maybe good for the hedge fund because they didn't want to tell competitors what they were doing. I imagine with AI labs, you're getting more feedback from the customer. Um, and then of course that helps you make the data better. And then it helps all the, all buyers, uh, get better data.

• 28:25Speaker 00

  That, yeah, I mean, we, we definitely embed pretty closely, uh, and get, and get rich feedback, uh, which just makes us smarter on how to do everything going forward. Yeah. No question.

• 28:37Auren Hoffman?

  If you think about your business, you have these like incredibly smart competitors that are super driven. And then you're selling, your customers are probably the smartest people in the entire world. Um, that is a weird business, right? Usually your competitors are, you know, whatever, they're okay. And then your customers are fine. I mean, you are dealing with these like incredibly smart driven people, both as customers and competitors. Um, how do you think about that?

• 29:06Speaker 00

  Yeah. I mean, the, so the customer side, it's a, so it is a different prior profile than I've, you know, I've seen. Because it's not, even if they're not the ones signing the checks, it's ultimately all researcher driven. Uh, and so they're the ones who dictate the data strategy. They're the ones who dictate what is, you know, how do you define and what is quality? Uh, and so there are folks where.

• 29:35Speaker 00

  You know, they sniff out BS very quickly and they get deluged with different folks hawking their wares. And so, you know, the only ways to, to do it are to really know your stuff, to embed deeply, to show expertise and value early and repeatedly. And also, I think like, and this is what should like show you, you care about like what the research is. And so it comes down to these like deeply trusted relationships that you, you build and then you sort of continue to deliver again and again and again.

• 30:14Speaker 00

  And where you can, it's not just, hey, I need this data from you, but okay, where should we be, be going? On the, on the competitor side, you know, we historically haven't run into a ton of competitors. I think sometimes we'll button to partners and things like that. I think that's likely starting to, to change. We're seeing more folks sort of moving into the real world data space.

• 30:41Auren Hoffman?

  And you're also expanding all the things you're doing, so you're gonna hit them, right?

• 30:45Speaker 00

  There's a collision course here. Yeah, yeah, yeah. And so I think it's just being thoughtful on where we feel like, like, where do you feel like we have a right to win? And going after that, and then where do we not feel like we have a right to win? And despite the easy money that may exist, not going after that. So at least to date, coding has probably been the single most lucrative part of the data world.

• 31:08Auren Hoffman?

  Yeah.

• 31:09Speaker 00

  For training and, but like, that's not our world. Like, we don't have that data. We haven't sought the data. We don't have, has expertise, so we haven't touched it. And I think just being really thoughtful on where do we have a right to win in a durable way? Has to, is the only way versus trying to, you know, do every part of every stack and every domain.

• 31:31Auren Hoffman?

  I have found in my career that it is, and it is, I, I, maybe people in the comments are gonna beat me up for saying this, but the higher the IQ of the person, maybe there's some limit, maybe there's an asymptote, the easier it is to sell to them, the easier it is to explain your product and sell. And when we, when we were at LiveRamp together, pretty much all of our early customers were very high IQ people. And the lower IQ people only bought from us like when we hit a hundred million in revenue. It was much harder to, to sell into them. Is that a crazy thesis?

• 32:11Speaker 00

  No, no, no, it's for sure. I totally resonate with that. I mean, and it's like, probably like IQ times drive, which I think in, in, in the, in the spots we're in, they actually like, it's almost like table stakes that these folks were driven. But yeah, I, I think

• 32:29Speaker 00

  the, the sales cycles can be incredibly, I mean, some of them are slow, obviously, but some can be incredibly quick. And I think the

• 32:37Auren Hoffman?

  Cause they often know what they want, right? They have a sense or they can quickly ascertain what they want.

• 32:44Speaker 00

  Yeah. And like they, they get the concepts, like they get, if you are airtight,

• 32:50Speaker 00

  they get what you're talking about and they can move quickly and you can, so I've always, I've always, like, I totally agree with that. I've always found that. And it's just, we're fun too.

• 33:00Auren Hoffman?

  One, one of the other things about selling to a fast-moving company is that the, that the error, you know, there's kind of two types of error. They could, they could buy something that they end up like not really benefiting from. And they could not buy something that could like massively benefit them. And they are more worried about the latter error. Whereas almost every other company is more worried about the former error, especially because you could look stupid and then you're gonna get blamed for it. So, you know, if that's the case, you're gonna wanna, you're usually in a, in the, in the, in the former one, you're gonna get a committee with you to make a decision.

• 33:42Auren Hoffman?

  So now we made a decision together, so we're all to blame if this goes bad, right? Rather than like even if I myself making the decision to go do this, because I don't wanna be on the chopping block if it goes poorly. So, and that committee, of course, makes everything just take so much longer and that, that sales process to be so much harder.

• 34:03Speaker 00

  Yeah. I mean, the, the sales, the sales cycles can be, I mean, there's just so much like sort of FOMO in the space. And then the, I think, I think folks see that the prize here for winning is pretty massive. That, I mean, also the money that's getting spent on compute, for everything I've seen, still dwarfs the money spent on data. And so, you know, e- even though both are like incredibly central to everything that's going on. And so I think, you know, in certain cases, there's probably a little bit more latitude. I mean, at, at the same time, you know, it's the, the labs do have discipline, like they're not, they're not, you know,

• 34:44Speaker 00

  while they're spending aggressively, I haven't seen cases of like recklessness. They, they tend to be pretty thoughtful.

• 34:50Auren Hoffman?

  But they, at least they can like, there's someone there who can make a decision and then

• 34:54Speaker 00

  Oh, yeah.

• 34:54Auren Hoffman?

  They either make a decision or not, either, either they say it's not good and then it's done. Or they say it is good and then you can make it happen, but usually they're empowered to make a decision, one way or the other. And of course, they might make the wrong decision either way, but at least they can make the decision and they're willing to make the decision.

• 35:12Speaker 00

  They're, they can make the decision, they're willing to make the decision. And then because of the, I mean, again, more on the pre-training side.

• 35:19Speaker 00

  You know, sometimes in sales, you have to make up forcing functions. There are these beautiful forcing functions that exist of these huge training runs. And so it's like, you don't have to, you don't have to do that. So, so that's another, I, I think what you said is totally right. And I think that's like a third dynamic that, that really helps from a sales side.

• 35:38Auren Hoffman?

  There's a bunch of new, essentially AI labs that are either open source or open weights. I imagine they are going to be extremely data hungry as well, whether it's Nvidia doing their stuff. Or how do you think about these companies on the scene? Thinking machines, et cetera.

• 35:58Speaker 00

  Yeah.

• 35:59Auren Hoffman?

  I think they're all

• 36:01Speaker 00

  To your point, will be super data hungry.

• 36:05Speaker 00

  You know, we've seen some of that. I, I think there's a question on, presumably there will be some rationalization where you're not gonna have 60 Neo Labs that exist.

• 36:18Speaker 00

  And, but, you know, the expected value of these things is, you know, if you basically say it's probability of zero times one minus probability of infinity, like that's still worth the, the bet. So, so they, you know, folks still make those, make those wagers.

• 36:35Speaker 00

  But yeah, I, I think the same dynamics exist where there's, you still need massive pre-training. You still need to run the same post-training type work.

• 36:43Speaker 00

  Just because it's open source or just because it comes after Anthropic and OpenAI doesn't really change sort of the fundamental physics of it all. And so, yeah, I think, I think there's real demand there too.

• 36:52Auren Hoffman?

  What do you think is the most valuable data set that is just locked up right now? Because no one's either figured out how to unlock it or no one's figured out how to get it out safely that you're, that could come on in the next five years.

• 37:06Speaker 00

  So it depends on like how you sort of unlock, I mean, I, I think like by a lot, like genetics writ large is, and there's, and some of that is locked up in that there are organizations that have that data that can't or won't share that. So is, is locked up insofar as it is in people's bodies and has not been sequenced at any sort of like

• 37:26Auren Hoffman?

  Yeah, very small, very, only a very small number of people have, have ever been sequenced.

• 37:30Speaker 00

  Totally. And so that, from a, from the perspective of, I think, financial gain, but also sort of net positive to a society. Yeah, I think it's pretty hard to, to beat that one. And, and you're hearing more and more around this kind of topic and folks thinking through these questions. But that's one where like, as a consumer of healthcare, I'm, I'm very, very excited to see the inevitable unlocking of that. And I, I think it can, more than anything, like dramatically change how healthcare is administered in the US.

• 38:04Auren Hoffman?

  One of my friends, you know, there's this whole idea of this rare disease, a disease that only like 10,000 ish people have or kind of thing. He believes that most diseases are actually rare diseases. You get diagnosed with X, but really you're diagnosed with this very, very weird. And this is why it's so hard to treat these diseases is because we think X is the same. If you have X and I have X, if you have, you know, whatever, pink or cancer, I have pink or cancer, it's the same. But it's actually very, very different. It's manifesting in a very different way. And if we could actually start to get things like the genome, we could start to get all the other kind of data around this person. We can start to really understand the right personal treatment.

• 38:46Auren Hoffman?

  And it could all the way get down to like an N of one treatment potentially for that given person.

• 38:52Speaker 00

  And I think that's like the dream of all of this is can you get down to N of one type treatment? That couples genetics, family history, clinical data, environmental data, wearable data, like there's all, the more you can create this picture. But yeah, I mean, right now we've had to treat a lot of human illness with a pretty broad brush. And I am sure that one of the lessons we'll learn over the next bunch of years is like exactly what you said.

• 39:18Auren Hoffman?

  And if you think of a drug development, if there is an N of one drug that only works for Bobby Samuels, if that really was the case, then you don't need FDA approval. You cannot get FDA approval for, for a true N of one drug. So now we have a whole new, now we actually can get to market really, assuming like some, there's some committee who actually still, you know, there's probably still lots of regulation around it. But there's no FDA needed if there's just the Bobby Samuels drug that only works for Bobby.

• 39:48Speaker 00

  And this is where like, if you can, like thinking through where does evals and benchmarks come from all of this? Cause you could also imagine that, okay, we're worried about this reaction. And so, and you're putting, putting the, the price and economics of all this aside, which is like, great, we're gonna, before Bobby's administer this drug, we're going to take a set of cells, we're going to have the medicine in some small dose. We're gonna see how his cells react with this medicine. And only if it clears some threshold will we give him that. So even that is like probably a couple of days or weeks kind of thing. But I think the, the area for what is eval or efficacy at point of treatment,

• 40:32Speaker 00

  I think will end up being a, a bigger and bigger part of this. But right, you're not gonna need a 10-year cycle to approve a drug. It'll be way faster. And again, it's like, I'd say of everything in AI, it's probably the thing I'm like most excited about.

• 40:45Auren Hoffman?

  Now, there's, New York Times is suing OpenAI. And all these people are like fighting over who owns the data, like where does this whole thing end up landing?

• 40:55Speaker 00

  As a company, our general take is the creators of IP should be compensated accordingly.

• 41:04Speaker 00

  Where and that's sort of true wherever we operate. Where and how that lands from a sort of government regulatory perspective?

• 41:15Auren Hoffman?

  I think it's a question. It's like, like, okay, if you buy the book and you scan the book, it's that good enough as long as you don't like, output the copyright of the book or something like that. Like there's all these things like it's all sorted now, right?

• 41:28Speaker 00

  Yeah, totally. And then, you know, with what Anthropic did,

• 41:33Speaker 00

  they, I think they got in trouble, not because they scanned the book, but because they didn't buy the book.

• 41:38Auren Hoffman?

  So they didn't buy the book. Yeah, exactly. Yeah.

• 41:40Speaker 00

  And so I think there are a lot of gradations that will also be sorted out.

• 41:43Auren Hoffman?

  Like if I go to the library and get a book, like, like just check it out and read it and then check it back in and get smarter. Well, like, well, what, well, how that happened? Like, I don't even know. Like, exactly. I didn't buy it. Like, I don't even know how these things work in the human brain. Yeah.

• 41:57Speaker 00

  So, right. So I think that, and, and, and one of the reasons we focus on some of this, like, we focus on primarily non-public data is that you sort of, you don't need to think about these questions quite as much because they're not public. So.

• 42:14Auren Hoffman?

  There's a price right up front. Like there's a clear transaction that they're, both sides are agreeing to right up front.

• 42:20Speaker 00

  Exactly, exactly. But no, I mean, I think that, uh, I, I, I think it is anyone's guess how this all ultimately turns out. And there's both sort of legal implications. Obviously there's, you know, the sort of like geographic and political implications. I certainly, I think some of the leaders of the different models have basically said, if we want to stay competitive with Chinese models, this is what has to happen. There's also the anthropic strategy of, uh, we're going to get hit with a massive fine. That's only going to matter if we're a huge company and we're willing to take the risk. So, which I'm sure.

• 42:55Auren Hoffman?

  Fast and bring things or whatever.

• 42:57Speaker 00

  Fast break things and then we'll get sued. It'll be, it'll be radic here. So.

• 43:00Auren Hoffman?

  Yep.

• 43:01Speaker 00

  But in terms of where it ultimately lands, I think it's anybody's guess.

• 43:06Auren Hoffman?

  Now there are these personal agents out there that are attaching to one's emails, calendars, other kind of like personal data. Maybe your, your, your, your Google Drive or your Dropbox or your, you know, whatever iCloud files, et cetera. Your, your photos. And of course we've got like Instinct town, but even the AI labs have these personal agents that are training on data. How should like, should there be some sort of better notice for consumers? Maybe consumers don't always realize what they're training on or how they're training or what they're giving up or what they're not, or how should, how should we protect consumers in some sort of way? Or should we even worry about that?

• 43:46Speaker 00

  Look, they should give the right notices, but I think if it's, if it's a personal agent, to me, it's reasonably clear that this is being trained now. Ideally, it is.

• 43:58Auren Hoffman?

  Ideally, like my stuff is being trained for me.

• 44:01Speaker 00

  For me, for me. Yes.

• 44:02Auren Hoffman?

  Yeah, exactly. Now all of a sudden my emails are also being trained for Bobby. And Bobby knows how to write these like Orin Hoffman style emails. Well, like, I don't know if I want that. That's my idea, right? You know?

• 44:13Speaker 00

  Right. So sending, you know, I start sending people birthday emails the day before, like all this stuff.

• 44:17Auren Hoffman?

  Exactly, exactly. Yeah. Yeah. It's like, hey, Orin does this. This is a good strategy. Now all of a sudden everyone does this. I don't get any benefit from my birthday emails. Yeah.

• 44:26Speaker 00

  Yeah. It's like, uh, like consulting interviews we've gotten earlier. So now all these people start sending emails two days before the birthday and.

• 44:34Auren Hoffman?

  Yeah, totally. Totally. Yeah. 364 days before the birthday.

• 44:41Speaker 00

  So, uh, I think folks have done this correctly, like the, the opening eyes of the world, which is like, we won't train on your data. I think that, I, I do think if, if you are not doing something besides sort of like personalizing one's agent, there should be a real notice. And like, one of the things that we think about for us is not just like the letter of the law, but the spirit of the law of like, yes, there is data that we can probably use that we feel is like rights cleared, but you know, does it pass the, you know, New York Times test and, and things like that? And I think that even, you know, if something's buried in some terms and conditions, I, I think that still probably makes

• 45:24Speaker 00

  folks uncomfortable. And now, look, if it's like, hey, you can have this product for X price if you don't let us train and free if you do, like, great. Let consumers make that, make that trade off. Um, but I think you do have to be pretty transparent if you're going to do, you know, broad training off of folks' data.

• 45:41Auren Hoffman?

  Now, for at least the last 20 years, uh, there's been a company started every year that tries to buy data from consumers and pay them for their data. Um, and okay, I'm going to pay you for your, whatever data you have and sell it and try to have like a, a transaction. No, to the best of my knowledge, no company like that has ever worked. Do you think something like that could ever work?

• 46:05Speaker 00

  Not that I'm aware of. Like, a lot of the data is valuable at real scale and for.

• 46:12Auren Hoffman?

  Like maybe they're going to pay a consumer 20 bucks a year or maybe even a hundred bucks a year or something. Yeah. It'd be kind of the top end, right?

• 46:19Speaker 00

  Yes. And I think that, um, and then, you know, you obviously get like a pre-skewed distribution of folks who are willing to give up a bunch of data. Now, one of the things that is a little bit related to what you're, what you're talking about. There was an Anthropic hackathon that happened, I think a couple of weeks ago, where there was somebody who was sick. Like, I think some potentially terminal illness, who basically said, here's all of my data, like, go to town and do whatever sort of modeling you want on top of it. And so it is the first time or one of the first times where there are folks where there is like potentially a real social or like.

• 46:58Auren Hoffman?

  That's awesome.

• 46:58Speaker 00

  Reason to like donate their data.

• 47:00Auren Hoffman?

  Okay. Here's everything I've ever eaten that I've tracked. Here's all my genetic data. Here's all my medical data. Like, I need to be better. Like in some ways, like Brian Johnson is kind of doing this too in a different way, right?

• 47:12Speaker 00

  I mean, you can imagine, you know, when people at some point where folks have on their like driver's license, it's like donate my organs and like donate my data. Um, cause I think it is like, if, especially if someone has some sort of disease, like, if they can donate their, like, that is actually like a material good for the world if they can help folks come up with treatments for that disease, uh, down the line.

• 47:38Auren Hoffman?

  Okay. Yeah. I can see. Okay. You know, you have these like bankrupt companies that are selling data out. Like I could imagine maybe a, a, a country that is like defaulting to the World Bank. Maybe we can like, maybe they can pay in data as opposed to paying their interest or something.

• 47:52Speaker 00

  Totally, totally. And like, I would for sure, like, look, I would not give up all my data about everything about me today. But if, you know, something happened to me, I would, yeah, sure. Like fire away.

• 48:03Auren Hoffman?

  Yeah, yeah. Absolutely. Especially if you had a sense it would be good for humanity. It would help humanity.

• 48:09Speaker 00

  Yeah. Yeah. Totally.

• 48:12Auren Hoffman?

  What, what is a belief about AI and data that so much, like tons of people believe or some people repeat that you just think is flat out wrong?

• 48:20Speaker 00

  I think there's a lot of conversation about how like vertical applications are going to get eaten by the models and all that. And I just like, I'm super skeptical of that. Like, I think that, I think there will continue to be like a robust vertical ecosystem. Even if you could create or recreate some of that with the labs, like, there's a reason that people bought vertical software, even though like the software itself was not necessarily like super differentiated. Like, there's a lot of companies that, and folks who don't want to go rebuild whatever. And so I am, I'd say there's like a lot of bear, and it just comes and goes, but there's bearishness on some of the application layer.

• 49:02Auren Hoffman?

  And even if it takes you a week to create that software, it still might take you another, at least another week a year to maintain it. Um, and, and like if you're, and it's like, okay, that rant, and, and it might just be like one person in your organization who actually is maintaining and if they're on vacation or they leave your company, you're kind of stuck. So, so even just that, okay, there's some value right there of like using like some, some software.

• 49:26Speaker 00

  And then you, you aggregate that across a whole enterprise and it, so I think that, like, I'm skeptical that any of the underlying laws that, you know, make software work actually like, there's, there are things, you know, it's easy to build certain things and whatnot, but I think that companies should continue to focus on what they're uniquely good at and not try to spin up all these tools. You know, like I continue to be super bullish on Salesforce and this is like, it's not like, you know.

• 49:57Auren Hoffman?

  Even though like the product Salesforce.com is absolutely horrible. Like it's just so hard to use. Yeah.

• 50:03Speaker 00

  It's terrible. Yeah. But it's like, it's.

• 50:06Auren Hoffman?

  It's a database people use and lots of other people have connected to it and.

• 50:10Speaker 00

  And it's just, yeah, and it's terrible. Like, but it's, um, the thing that makes it unique is not the software. Um, and you could recreate the interface and all of that, but it's not. So I'd, I'd say I'm like.

• 50:20Auren Hoffman?

  It's 25 years of the corner cases that they somehow figured out or something.

• 50:25Speaker 00

  Exactly, exactly. Um, yeah. I mean, I think back to your, your tweet of the, uh,

• 50:32Speaker 00

  you know, I, a software is incredibly mispriced, but I don't know if it's way overpriced or way underpriced, um, which I've thought a lot about. And I, I, at that moment in time, I would have been on the, it's underpriced part of the world.

• 50:46Auren Hoffman?

  Well, I mean, I think, I think it's, it's almost on a per company basis.

• 50:50Speaker 00

  It is almost.

• 50:50Auren Hoffman?

  Like, yeah. And it's hard to know. So like the basket, like is the basket. And people are either overpricing or underpricing the basket on any given name. It's also so hard to figure out.

• 51:02Speaker 00

  Yeah. I think things like the, you know, Chegg and Nerdy, like the, like online tutoring, like they're hosed.

• 51:10Auren Hoffman?

  Well, you know, again, like they can go founder mode and figure out something new and they've got a good brand. So they, they can come out with new things that, that. Historically, in general, software gets hosed every, most software companies, like most software, most of the dominant software companies of the nineties, like did not exist, uh, just 10 years later. Uh, so they, very few ever make the transition in software. It's not like, um, you know, the hotel chains or car companies, like, you know, that's, that are much more enduring or something.

• 51:41Speaker 00

  Or pick your Berkshire company.

• 51:43Auren Hoffman?

  Yeah. Picture Burger Company. Yeah. And one of the interesting things about software is if, if, um, if you're JP Morgan and you want to stay at a hotel or you want to go fly somewhere, you use Marriott or United Airlines. You don't go like create your own hotel. Um, but if you're JP Morgan and you want to get software, probably my guess is their software budget today is like. 20% of the budget is actually software that they buy and 80% of the software they make. So again, you don't make your own hotel. You don't make your own airline usually, right? You buy, you buy a hundred percent of that, you know, a restaurant or whatever.

• 52:23Auren Hoffman?

  But like for software, for most organizations, most of their budget goes to making it, not buying it.

• 52:30Speaker 00

  Yep. So I, I think the, you know, there's probably, I think that where that needle is probably moves. But I think there's still going to be, I think there's a reason, you know, Harvey is taking off in the way it is. Now, and then, now, now you get some companies, you know, I think Givemorgan is an interesting one where like Kirkland and Ellis is publicly investing $500 million in terms of building AI. Whether they're going to be successful, that's like such a DNA clash, but there are probably going to be some folks who say, no, we want to build this internally in some capacity. But, um, but in, in general, I'd say I'm like long a bunch of, a bunch of applications subject to the same rules that make any company good. And so if you're just doing another me too kind of company, you're gonna, it's gonna be hard.

• 53:13Speaker 00

  But, but I think your point around like the, you're always gonna, things don't stay static.

• 53:18Auren Hoffman?

  Yeah, things don't stay static. Now, there was a point, maybe, maybe a couple of years ago where everyone was talking about synthetic data. And for whatever reason, people aren't talking about that as much anymore. Why was it so hot a few years ago? And why is it like less hot now?

• 53:37Speaker 00

  So in the healthcare world, when I started at DataVant, which would have been like 20, like 2020, we were 18 months away from synthetic data having a huge moment. When I left four years later, we were 18 months away from having a huge moment. And it always feels like we're like right there. But look, I think that there was a, there's something super compelling about the idea of in the context of a scaling, of scaling laws, where data is a huge constraint. If you can scale up with synthetic data, that is like a very, very compelling concept.

• 54:17Speaker 00

  And I think the research community has sort of gone back and forth over time, to your point on whether it is valid. I think that as data has gotten more, uh, more focused on, uh, not even niche use cases, but things that are a little bit more nuanced. It is clear that real world data is where things need to go. And I mean, you can think about it like if a company wants to build an agent to create a, a deck, PowerPoint deck. You want to have the deck itself, but you want to have all the conversations that lead into it and all the Slack messages and the presentations and all that.

• 54:56Speaker 00

  And like, if you just think about that being created, either by people sort of thinking it off their head or just like a, AI creating that, it's going to feel very stilted and weird. And it's not going to, you're going to have all sorts of biases embedded in the models. And so folks have, as real, have looked more at this real world data world. And that I think is, is certainly something we're seeing in healthcare too. Like you're going to have a fake pathology slide. Like so, so I think that these things come in and out of fashion. There's, uh, you know, I'm sure.

• 55:34Auren Hoffman?

  I guess you kind of like, um, if you're the Pentagon or something, okay. It like, you might need to create synthetic data around like a war with an adversary. Cause like the, okay, there is no data about, well, here, there is no data about that. So maybe somehow we need to create it. I don't even know how you do it in a real, in a good way. But like, maybe that's the only thing you could do is, is create synthetic data in certain types of scenarios.

• 55:58Speaker 00

  I think that there is a place for synthetic data, the ones that I, the use case I see is like most compelling is you basically like want to, if there's like highly sensitive data that you want to be able to look at before you license or can only be shared in very sensitive situations. You basically can create a synthetic data set that just allows somebody to sort of understand the general nuances and relationships of the data. And so, uh, that is a use case I understand. The, the other thing about synthetic data and, and manufactured data. One of the things just about models generally is they explore sort of hidden relationships between different modalities and different concepts.

• 56:42Speaker 00

  And so it's actually like impossible to create synthetic data because we don't know those relationships. And so it's impossible to preserve all those sort of distributions in the right way. So again, I, I tend to be pretty bearish on, on the concept.

• 56:53Auren Hoffman?

  Yeah, I tend to be bearish too. Yeah. But I'm, I'm open to be even proven wrong.

• 56:58Speaker 00

  Totally. Me too. You know? And, and then, you know, I say, okay, for us, even in the world where synthetic data does have more of a day, like you still need a core of real world data. So I, I think for someone like us, it's, you know, we'll be fine either way. But I'm just like, I'm, I'm sort of bearish on the concept generally.

• 57:18Auren Hoffman?

  No. Okay. How do you think about hiring that's different than the norm? Okay. And has it changed in the age of AI?

• 57:26Speaker 00

  The biggest shift of what I think about, uh, there's always been the talk of sort of mercenary versus missionary. I think that is, I think about that 10 times as much now than I did.

• 57:42Auren Hoffman?

  Interesting. You think it's even more important now?

• 57:45Speaker 00

  Yes, absolutely. The amount of investment that's going into the space, into sort of AI, AI data is so large that, you know, companies fortunes are going up and down and crazy valuations. And if somebody is motivated by what's the next dollar, like they will be impossible to retain. They're going to think about the.

• 58:06Auren Hoffman?

  I'm sorry. Are you just thinking about that now? Because obviously you, you're, you're running one of the hottest companies and probably all those, all, all the mercenaries might, might want to join you. Whereas maybe before you weren't in companies that mercenaries were aware of yet.

• 58:19Speaker 00

  I, I mean, I think that the concept as before, I was like fortunate to be at, at great companies. But like, I actually think the payoff to being a mercenary is higher now than it has probably ever been. Where you can jump to companies and, you know, on paper, make a huge amount of money very quickly. And there are a lot of cases where, you know, folks who are fresh out of school are making eye popping numbers. And so I think it's, you know, I, one is we've had, you know, the good fortune to have a fair amount of success. But I also think that there is more of a tendency for folks to sort of be looking out, okay, what's the next?

• 59:01Auren Hoffman?

  And why is it so bad to have, like, I mean, I presume generally people who joined, you know, uh, investment banks and private equity firms, like over the last 30 years were mostly mercenaries.

• 59:14Speaker 00

  Mostly mercenaries.

• 59:15Auren Hoffman?

  Right? I mean, I don't know why else you would join, um, probably, right? So like, and like, obviously those are good businesses and they have dedicated people and they're super smart people. So like, why does it even matter if like mercenaries join you?

• 59:28Speaker 00

  I, I think for us, a couple of things. One is the investment banking business was like pretty stable, uh, for a while, like for a long period of time. Like, yes, it's not, but like, you sort of know what next year is going to look like. I think for us, you know, the industry is changing so quickly that if you have folks who are just going to jump when things get a little bit rocky, it's very hard to build like a cohesive sort of culture and team. I think the other thing is we're selling to missionaries. Like the research community is unbelievably mission driven.

• 1:00:07Auren Hoffman?

  Yeah, your clients are missionaries.

• 1:00:09Speaker 00

  Yes, to, to a degree that, um, I am always like very, very pleasantly surprised about. Where folks like deeply, deeply care about these things and they can sniff out in two seconds whether you care to.

• 1:00:21Auren Hoffman?

  Yes.

• 1:00:22Speaker 00

  And like, you know, I.

• 1:00:24Auren Hoffman?

  Whereas like, yeah, if you're, uh, if you're a hedge fund person, like no one cares.

• 1:00:29Speaker 00

  No one cares. No one cares.

• 1:00:30Auren Hoffman?

  Money or not. Yeah, yeah.

• 1:00:31Speaker 00

  Yeah. Yeah. Yeah. And so I think that's like a big, a big, it's, it's both. We want to think about retention in an environment that is incredibly fast changing. And that, you know, the second is, you know, the, uh, who we're selling to. But then I think in the AI world, the just like ramifications of what one can do are pretty substantial. In terms of both for good and for, you know, not the, the anthropic safety lineman folks saying there's a greater than 10% chance that the human species is going to be wiped out in a period of time. And so like, I actually think for the world, like it's important that you have folks in these companies, uh, who were like genuinely care about those types of things.

• 1:01:10Speaker 00

  And like, you know, I think one of the anthropic interview questions was, is there a world where you, you know, say we should shut down the company? And I, I think just like from a personal perspective, the stakes are high enough that that's, those are the kind of folks you want on the team.

• 1:01:24Auren Hoffman?

  Okay. Travis Mayer, I wanted to ask you about Travis.

• 1:01:27Speaker 00

  Yeah.

• 1:01:28Auren Hoffman?

  You, you, this is now your third company where you work super closely with Travis.

• 1:01:34Speaker 00

  Yes.

• 1:01:34Auren Hoffman?

  As you know, he's one of my favorite people and I've also worked with him over multiple things. What have you learned from Travis?

• 1:01:42Speaker 00

  Yeah. I mean, I've, outside of my family, like there's very, very few relationships that have had the same impact on me that sort of my relationship with Travis has. Um, I think there's, there's a bunch of different ways to go with it. I think the most non-obvious is, because you know, there are pieces on strategy. And I think obviously there have been a lot of big wins for him. And he is like clearly as a strategic thinker, exceptional. But the, I think the, the thing that would be surprising is the focus, like huge focus on culture, huge focus on sort of, you know, the decision can be what the decision is going to be, but how do we get there in the right way?

• 1:02:26Speaker 00

  How do we build the right operational processes? And so I have learned a ton from him and like, what does it look like to build a good company in the right way? You know, how do you think about talent? How do you think about promoting talent? How do you think about insisting on a high bar? How do you think about hiring? Fire, like I, I feel across these three classes, these three companies that have sort of had like a masterclass in a bunch of these things. You know, and then there's also obviously like the strategy of, uh, you know, how do you think about data businesses? And what are the things that seem appealing, but aren't the hedge funds being a good example? So I think it's both the strategy piece, but also just like in some ways, the sort

• 1:03:08Speaker 00

  of day to day blocking and tackling, what is the way in which to run a company in a thoughtful, but also like an ethical way. And I think that the, uh, you know, he is someone who I like implicitly trust. We started working on protege, the two of us and a couple of other co-founders. We didn't have any like paper between us for months. And I was like, eh, we'll figure it out. We trust each other. And I think that just like speaks to him too. And the idea that you can't like, being a good person and like winning huge in a business context is like, uh, not just like mutually exclusive, but actually in certain businesses are like extremely additive and complimentary.

• 1:03:51Auren Hoffman?

  Yeah. Also like, are you a long-term maximizer or a short-term maximizer? Right?

• 1:03:57Speaker 00

  Totally.

• 1:03:57Auren Hoffman?

  And he's clearly a long-term maximizer.

• 1:04:00Speaker 00

  He's clearly a long-term maximizer. And I think the, you know, the thing that, uh, I think is attributed to you and to Travis is like they, the businesses that you guys have been in also like play to strengths in terms of like building networks of, uh, like how do you generate all sorts of win wins? I actually think it's a, it's an industry that, um, is conducive to low ego founders because you need to like, the only way you win is if everyone wins. So I think it's also.

• 1:04:31Auren Hoffman?

  Yeah. Basically a lot of, you know, if you're doing a deal with someone, they get 60% of the benefit type of thing, but you just do lots of deals with like that and you do well.

• 1:04:41Speaker 00

  Exactly. Exactly. Um, and so I think there's also just like, you know, a really good sort of industry founder fit that he's, uh, you know, hit on again and again.

• 1:04:52Auren Hoffman?

  All right. I already asked you the last question we ask all of our guests, um, the conventional wisdom one, which you gave me, which was great about the, the SaaS applications. But the, the last question I want to ask is how you, how are you personally using AI day to day?

• 1:05:06Speaker 00

  So I am not the like, interestingly, like I don't, I wish I used it in a more sophisticated way. The, I have it all the time. Like I have it think through tone, actually have a lot of like, that's a, you know, obviously there's like the, like I, I ask it a question. So like, if I'm like, hey, here's a note I'm thinking about writing, like how is this landing?

• 1:05:29Auren Hoffman?

  How is it going to be perceived?

• 1:05:30Speaker 00

  Yeah.

• 1:05:31Auren Hoffman?

  So it's like, it's like your EQ, uh, instead of your IQ.

• 1:05:35Speaker 00

  Yeah. And like there's, you know, I, I sometimes will use it for like different questions I throw off, but I think like I, um, you know, there's that. I also think, and this is like, you know, hardwired from a young age, I'll sometimes read a document and be like, give me a grade on this. And it always gives the same grade, which is like B plus, uh, because it's like, okay, great. Uh, you know, I gotta wait. But, but I, I think, um, there's also just like, cause and look, a lot of times I'm like, your advice is, is silly. Um, but I think it's like, you know, things land on different people, different ways. And it's helpful just to get like, okay, here's like a thing I had maybe hadn't thought about, or here's like a, a sentence that maybe I can like lessen a little bit.

• 1:06:19Speaker 00

  Um, so that's like one of the ways that I, I like do tend to lean on it in addition to like sort of the, the standard ones of.

• 1:06:28Auren Hoffman?

  Yeah. Yeah, exactly. Exactly. I like the tone idea. I'm going to try that, uh, because I, I have low EQ or I think I could use it to help me, uh, coach me on those things.

• 1:06:39Speaker 00

  It's good to get another perspective, you know? And especially during the day when, when we're remote during the day, when I can't turn to somebody to be like, can you like sanity check this? It's like a, you know, it is a form of sanity checking.

• 1:06:51Auren Hoffman?

  Amazing. I thank you, Bobby Samuel, CEO of Protege for joining us on summation. I knew this would be good. Obviously we have a long relationship with one another. I knew this would be amazing, but this exceeded my expectations. It was awesome. So thank you very much.

• 1:07:05Speaker 00

  Of course. Back at you. Always good to see you and thanks for having me.