Fivetran CEO George Fraser - data for the AI revolution

George Fraser is the co-founder and CEO of Fivetran, the category-leading data integration platform that does over $300M in revenues. In this episode of World of DaaS, George and Auren discuss: The Databricks/Snowflake “war”Building a data stack for AI workloadsEvidence based decision making for tech companiesParallels between neuroscience and artificial intelligence Looking for more tech, data and venture capital intel? Head to worldofdaas.com for our podcast, newsletter and events, a...

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Auren Hoffman?

  Hello, data nerds. Welcome to World of DAS. I'm your host, Orin Hoffman, CEO of Safegraph and GP of Flex Capital. Discover more episodes, get weekly data as a service news at worldofdaas.com. That's spelled world of D A A S dot com. Hello, fellow data nerds. My guest today is George Fraser. George is the co-founder and CEO of FiveTran, a category-leading data integration platform that does over 300 million revenues. George, welcome to World of DAS.

• 0:31Speaker 01

  Nice to be with you, Orin. It's been a while.

• 0:34Auren Hoffman?

  Super, super excited. Now, how does the rise of AI workloads change what customers need from data pipelines?

• 0:41Speaker 01

  We're trying to figure that out ourselves. I mean, we're monitoring the situation closely. So far, the answer seems to be not much. FiveTran has been delivering text data to data warehouses for 10 years. And that has historically been sort of opaque. There's not a lot you can do with it. And now there's something you can do with it, these language models. From one perspective, what they fundamentally do is they give you the ability to machine process unstructured text data.

• 1:07Auren Hoffman?

  Do you think that's going to change how people are actually processing the data in some sort of way?

• 1:13Speaker 01

  Yeah, I think the most obvious use case is RAG. We do that at FiveTran. It's been very successful here. We have an internal RAG chat integrated into Slack. It is used by salespeople. Sales engineers support people to answer the arcane questions about the zillions of data sources we support that we get every day. If you're talking to someone at FiveTran, there's a decent chance that in the background, they're talking to this thing. And we found that once we got all the data together, building the actual interface on top of it was the easy part. The hard part, as always, is getting all your data in one place.

• 1:46Auren Hoffman?

  It's just getting all that knowledge and stuff like that. And then how does one help tune it? If it's saying one thing, because maybe it's accessing an old knowledge base or it's having some sort of like, how do you actually tune it over time? How do you say, oh, in the future, you should respond more like this or something?

• 2:02Speaker 01

  We don't. We just do retrieval and we update the underlying data that's getting put into the context window. And that ends up being how it gets solved.

• 2:09Auren Hoffman?

  So just putting newer and newer data over time should make the answers better and better?

• 2:13Speaker 01

  Well, it's always searching against the latest version of our docs, our support tickets. And that's working pretty well for us.

• 2:20Auren Hoffman?

  Is that going to just alleviate the need for things like sales engineers and things like that?

• 2:25Speaker 01

  The value that people provide is often much more complex than you realize and less obvious and they do a lot of different things.

• 2:34Auren Hoffman?

  Especially since a lot of it's just understanding that what the customer has is almost like being consultant to the customer, which is hard to train.

• 2:40Speaker 01

  Yeah, so I don't think so. I think that we will discover that mastery of the details of every single data source is not the only thing you need to succeed in this role. Similarly to, I think AI is already playing a big part in medicine. It's a very analogous situation. Doctors have to have a mastery of this huge range of subjects, but that's not going to make doctors disappear. Being a guidelines bot is not the only thing that doctors do.

• 3:10Auren Hoffman?

  If you're bringing three people to the sales call, maybe you bring two in the future. There could be some scenarios like that, right? Or no?

• 3:17Speaker 01

  Totally. I think sales engineers will be less overworked in the future.

• 3:21Auren Hoffman?

  Yeah, they'll concentrate on more of the higher value stuff. The data warehouse itself is evolving. How do you see that playing out?

• 3:28Speaker 01

  Kind of a jump ball right now for this next layer of applications. The thing that sits on top of the data, everyone is trying to be the center of that universe.

• 3:38Auren Hoffman?

  Do you think they'll be the clear winner over time? Or do you think it'll just be very, very, very fragmented?

• 3:44Speaker 01

  I think it's going to be very fragmented, if I have to guess. I think our experience has been that the sort of application layer is pretty well served by sticking together a bunch of open source things. And it's a scenario where it actually works pretty well to put together your own little set of building blocks because a lot of it is about, like I said, curating the data correctly and then curating that retrieval step correctly, at least as far as RAG goes. The application layer is thin and benefits from customization. That tends not to lead to a single giant winner. But I'm not a VC. I'm not in the pragmastication business, nor are they really. Who is?

• 4:22Auren Hoffman?

  The tech vendors that basically like a company has all these different tech vendors as part of it. Sometimes they'll have like thousands of tech vendors and it almost constitutes a very unique DNA and where no two companies are the same. If you look at somebody's like tech stack, what does that tell you about the company itself? If I looked at your DNA, I might know that you're a man and you're maybe even of a certain height and some other types of things. What could I learn from a company just by seeing the tech stack?

• 4:52Speaker 01

  You could see how old it is based on the types of vendors it uses. You could see if it's done a lot of M and A. So companies that have done a lot of M and A will have many

• 5:02Auren Hoffman?

  Many of the same things.

• 5:03Speaker 01

  Many of the same thing. One of our largest customers is OpenAI, but they have exactly one of each thing, right? Because they are so young. Whereas if you look at LVMH, another big five train customer, they have many copies of each thing. And then I think on the data side, you can tell how much power the practitioners have in the organization. When the practitioners are disempowered and the decisions are being made by people five levels up, they tend to buy things that speak to exciting subject matter to IT leaders like master data management and tend not to work very well. Whereas when the practitioners have a lot of power in the

• 5:43Speaker 01

  decisions, you tend to see tools that solve real problems and work really well.

• 5:49Auren Hoffman?

  But also when the practitioners are doing it, you may have more vendors because you may have little bespoke things that solve a specific problem.

• 5:56Speaker 01

  Probably.

• 5:57Auren Hoffman?

  There's this kind of open source revolution in the area, whether it's like DuckDB, Postgress, all this other stuff that's going on. What's your take on this open source versus closed source stuff in the data infrastructure layer?

• 6:09Speaker 01

  Postgress is, we're watching the end of the movie. That's been going on for a long time, the takeover.

• 6:15Auren Hoffman?

  You think we're actually, it's in decline now?

• 6:18Speaker 01

  No, it's not that it's in decline. It's in a zenith. All those wheels were sent in motion 10 plus years ago. The funny thing about Postgres, it's so popular. Listen, Fivetran runs our application on top of a single vertically scaled Postgres database. It's continuous with the one that I set up over 10 years ago. There is in theory a continuous write-ahead log through all that time. However, there are much better databases out there that have come out of the research groups. It's just so hard for a application database to get traction, any new thing. Postgres is not actually the greatest technology in the world. I'm not saying it's bad.

• 6:54Auren Hoffman?

  You guys are still on it for a reason. It's just because there's inertia and everyone knows Postgres is great. Everyone knows how to use it and there's all these other tools that are built on top of it. And AWS has, or Azure, whatever, they have other internal tools to help scale it.

• 7:08Speaker 01

  It's impossible to change. The way to change your production database is to start a new company. That's why it's so hard.

• 7:16Auren Hoffman?

  I still know people who start new companies who started on Postgres, but you're saying that is likely today.

• 7:21Speaker 01

  At that moment, when you start a new company, you're under a very different set of considerations. The very best thing is not necessarily, you want something cheap, you want something that's available in the AWS console or whatever else you're using. You want something that's very safe. This is not something where you want to take risks. A counter example would be Snowflake, which was built on Foundation DB, which they chose because they believed that they had a unique set of challenges that they faced that were solved by this sort of high-risk choice of Foundation DB. But that's very unusual.

• 7:55Auren Hoffman?

  How do you see companies like Snowflake, Databricks, how do you see them evolving over time?

• 7:59Speaker 01

  Data lakes are really going to change things for those companies. The rise of truly vendor-neutral storage formats is a really significant development and they're leaning into it, to their credit, but it's really going to change their role. I think traditionally their role has been as the keeper of the database. And when I say database, I mean it in the academic sense, the data itself, the logical model, not the query engine, not the particular instantiation of it. And now, especially at the largest companies that generate the lion's share of their revenue, you're seeing people move towards vendor-neutral data lakes. And then it becomes a question of the value that they can offer higher up in the stack.

• 8:37Auren Hoffman?

  You're still seeing that like a Databricks is last I checked was about 2.6 billion ARR and it's still growing crazy fast. Somehow they're still gaining all this share and stuff. How do you see these things happening over time?

• 8:50Speaker 01

  Even though everyone likes to talk about Databricks and Snowflake as being in some kind of battle, I like to joke that the thing about the war between Databricks and Snowflake is the customers don't know about it.

• 8:59Auren Hoffman?

  Yeah, and the customers have both often.

• 9:02Speaker 01

  Yeah, they mostly are doing different things, even though they try to do each other's thing. And I think Databricks has benefited enormously from all of the hype around AI. They have been flogging that tagline data plus AI for 10 years. It's like skating to where the puck is going 10 years in advance. And it is there now. They are really benefiting from that. And I think that's why they've grown really fast recently.

• 9:28Auren Hoffman?

  Now, you wrote this article on evidence-based businesses. Where do you see most companies go wrong where it's like using data for business decisions?

• 9:37Speaker 01

  They use crappy data where they would be better off using no data at all. The best thing is evidence-based decisions that are based on high-quality evidence. The second best thing is just introspection or stories. It's not your gut. Just talk to people one at a time, but be systematic, really listen. If at the end, you think that all 10 people told you the same thing, you weren't listening. The world is not like that. There should be contradictory information, but just be like an anthropologist, just go out there and collect stories and be systematic. That's second best. The worst is what most people do, which is they use low-quality data that's highly confounded, that they analyze it to death. If you put a complicated enough model on a data, it will tell you anything.

• 10:21Speaker 01

  And it will end up telling you what you want to hear and you will do it without realizing. This is the thing people don't understand about P-hacking is it's subconscious. You do it not realizing you're doing it. And then at the end, you're like, look, this data said this conclusion that I was predisposed to believe. You made it tell you that.

• 10:38Auren Hoffman?

  What can companies do better? What's a simple thing companies should start thinking about and doing better?

• 10:43Speaker 01

  When you do something, when you make a big decision or you make a change to your business, decide in advance how you will say whether it worked or not and write it down. Say before you change the sales territories or the marketing strategy or whatever, here is how I will evaluate this after the fact. And then try to stick to that. Sometimes you won't be able to stick to it. You'll realize your plan was impossible to evaluate it the way you do. But really try to do that because that will prevent you. That is the only way to prevent yourself from torturing the data until it tells you what you want to hear. This is to decide in advance how you will evaluate.

• 11:20Auren Hoffman?

  Interesting. A lot of investors, when they invest, let's say before they put money into a stock or something like that, they'll write down, okay, here are the reasons I'm putting money into the stock. Here are the things that I think are going to happen. Let's say there's a macro thing that's going to happen. And therefore, because of that macro thing, I think the stock is going to go up. And let's say I believe that there's a war going on in the Middle East. Therefore, there's going to be higher oil prices. Therefore, the stock of Exxon is going to go up or something like that, that hierarchy. Because sometimes the stock went up, but not because of your belief. The oil price went down, but the stock still went up. Sometimes your belief was right. So then you can at least look down the chain and understand over time, what did you do that was right? What did you do that you think needs updating in the future?

• 12:05Speaker 01

  And writing it down is a key part of that story you just told. It's not only that it needs to work for the reason you thought it was going to work. You need to write it down beforehand to prevent yourself from making up a story after the fact about why you are right. And we all do this.

• 12:20Auren Hoffman?

  We will all do unless you write it down. Even when you write it down, sometimes if it's not clear, there's a story to be woven.

• 12:27Speaker 01

  Always, you can always make something fit. We just had a presidential election and there is this ridiculous figure, Alan Lichtman, who claimed that he could predict every presidential election based on these 13 keys. And they were a perfect example of this because they had been mostly determined retrospectively. You could weave together an explanation of anything.

• 12:48Auren Hoffman?

  Yeah, they have that sometimes. Who wins the NFC or the AFC and that determines the presidential election or something. Maybe that was true in the past, but it seems unlikely to be true always.

• 12:59Speaker 01

  The primary problem in making evaluations of data is actually fooling yourself. That's the main thing that goes wrong.

• 13:05Auren Hoffman?

  It was interesting. In March 2020, I thought stupidly that it was overblown and that it wasn't a big deal. So when the S and P 500 dipped, I'm like, oh, it's completely overblown. I'm going to invest in the S and P 500. And so it turned out to be a good investment, but for completely the wrong reason. Turned out the S and P 500 did go up massively after that, but it was a bad investment because it was the wrong reason to invest, if that makes sense.

• 13:29Speaker 01

  It's funny. I did the exact same thing and made a lot of money. I had a bunch of money in a money market fund because I had my first opportunity to sell some Five Trans stock on secondary, and it was sitting in a money market fund. And I looked at it and there was a point where the S and P 500 had dropped. The correction that had been priced in was greater than the 2008 financial crisis. And I looked at it and I said, this is too big. Covid is bad, but it's not that bad. I was following very closely at the time, the infection fatality rate. There was very little data, but there was some data from Iceland. There was some data from this cruise ship. There was some data from Ohio prisons and a couple

• 14:13Speaker 01

  other cases. And I said, it's worse than a flu, but it's three to 10 times worse or something. It's in that band. That is not as bad as the 08 financial crisis. And I don't know. I guess it depends what your definition of overblown is.

• 14:28Auren Hoffman?

  I thought the shutdowns were going to be a matter of weeks, whereas it was really for a year. She had all these things that were impacted. Obviously, you had fast food. You had, but fast food wasn't as impacted. I didn't realize that just because people use DoorDash. But I thought fast food, hotels, all these things that started going down, air travel, all these things that really started going down massively, then just the economy in general. People just presumably be out of work. They would have less money. So that was all getting priced in early March or mid-March 2020. And so I just thought, oh, that's overblown because we're going to come back to work very soon. Even when we shut down our office, I was like, oh, yeah, it's going to be super temporary. We're going to be back in a few weeks. Yeah, it just took way longer to get everything moving. It took until the end of the year, until they started rolling out the vaccines and

• 15:13Auren Hoffman?

  all these other things.

• 15:14Speaker 01

  I guess that was over optimistic. And I also shared that view until, I think, early March or something at that time. And then I saw that's not how it was going to play out. But I would say that you were right in that the correction was too big. The market came back quite quickly, by the way.

• 15:30Auren Hoffman?

  By April or May, Drew's back.

• 15:33Speaker 01

  My plan was I was dollar cost averaging. I was buying in each week. It crossed the threshold and I was like, okay, now this is a correction that I can believe in. My threshold, this is like about 2008 at worst. And so, once it hit that threshold, I stopped buying and then it kept going up and I didn't predict that.

• 15:49Auren Hoffman?

  So it hasn't stopped. Yeah, it's crazy. I'm sure you've reversed many decisions, maybe in that case, that's a good example of your investment case. But at Five Tran, you're probably reversing evidence-based decisions that you've made. How does one do that? Is it the same kind of thing where you're writing it down? And then when things aren't moving that way, you can reverse it somehow? Or how do you walk through that? Sometimes I feel like the reversing decision is harder than making it.

• 16:13Speaker 01

  Many times in small things and big things, I think most decisions, first of all, are not really evidence-based. Can't be. It's not possible to make most decisions in an evidence-based way. And I think this is more of a personal psychology thing. You have to try to practice being wrong and admitting that you're wrong, at least to yourself, but ideally to others as well. And if you're never doing that, then it's not that you're always right. It's just that you're refusing to admit it. So I think that's a tactic I try to use to maintain epistemic hygiene for myself. Another tactic I use a lot is I imagine myself as the new CEO of Five Tran. And I say, what do I come in here and immediately overturn?

• 16:57Auren Hoffman?

  You can't do that every day or you just go crazy. So is it like a once a year thing, a once a quarter thing that you think about that?

• 17:04Speaker 01

  No, I do that every week.

• 17:05Auren Hoffman?

  Okay, got it. So every week?

• 17:07Speaker 01

  I don't know. Maybe I am driving myself crazy.

• 17:10Auren Hoffman?

  Every week you're like, okay, what would somebody who just came in here be doing differently?

• 17:14Speaker 01

  Yeah, mostly I come up with a lot of the same answers and I'm trying to put those things into action. So it's not like a new revolution every week.

• 17:22Auren Hoffman?

  But presumably, somebody who replaced you, who have that same idea of what to do differently, would do those things very quickly, or no?

• 17:30Speaker 01

  It's just sometimes big changes are hard to make. For example, the big one right now is there are some aspects of our pricing model that are really a little too clever. I definitely overthought it a little bit, and we're trying to make some simplifications.

• 17:45Auren Hoffman?

  But then you have to wind through all your contracts.

• 17:47Speaker 01

  It's a huge project, right? So that's the prime example right now.

• 17:52Auren Hoffman?

  And too clever, meaning just too hard to understand for everybody, both for the company and for the customer?

• 17:57Speaker 01

  And high train is very hard to price, but it's a little too hard to understand right now. And so we're going to make some adjustments that are not going to be material in terms of company's revenue, but hopefully we'll make it a little easier to explain to each new person who comes through the door.

• 18:10Auren Hoffman?

  Yeah, it is hard when someone's like, okay, I have this bill and I've seen it in certain different tech stacks we use. You have this bill in October and you're just trying to understand. I just don't understand there's this number of workers doing this number of tokens, doing this number, like it becomes very hard to go through and predict.

• 18:29Speaker 01

  Yep.

• 18:30Auren Hoffman?

  What companies do you think do pricing really well?

• 18:32Speaker 01

  I think Snowflake has done pricing really well. They have basically changed nothing for their entire existence. Bob just hit the bullseye on the first shot. And he really helped me figure out five trans pricing models. So we joke about this, man, I didn't hit the bullseye as well as you did, Bob. He's like, nope. Good pricing models, they're familiar. People call them simple. Sometimes they're not actually simple, though. It's just that they're extremely familiar. They gear off something people already know and they align the willingness to pay. They align the material cogs. If you have material cogs, which not everyone does, those are the basic characteristics of a good pricing model. And they reduce friction. They make it easy to become a customer. That's key.

• 19:15Speaker 01

  And that is a criteria that is bailed by get in touch pricing, where there is no public pricing model and you just say you have to talk to a salesperson. That is a huge source of friction and a lot of salespeople like those models because they see the customers at the end of the pipeline and it gives them a high degree of control. And the ones who go past that gate, who see that and proceed, there's a heavy selection that takes place there. So then it makes it, it looks really good at the other end. The problem is that you just eradicated 80% of your pipeline by having that.

• 19:47Auren Hoffman?

  I don't even like to engage with companies, which by the way, it was most of them. I went about and I see it, I'm like, I can't just sign up and use it. First of all, I hate when I can't sign up and use it. So that just annoys me. Why can't I just sign up and start paying for it right now in some sort of small way? And start using it. And then I want to have a sense of, okay, if I do scale, my organization scales greatly. What is the pricing? Cause so many times you get bitten the butt where you're like doing something and then all of a sudden it's like 10x more than you thought it was going to be. And then you have to do this whole unwinding of moving to a new vendor. Didn't realize it. Probably everybody like me has experienced that multiple times.

• 20:23Speaker 01

  Yeah. And all of those negatives you just described, they prevent people from becoming customers in the first place. And so if you look at your existing pipeline or your, or your existing customers, you'll never see these issues. This is an example of where the only way to find this stuff out is introspection. You're not going to discover this by looking at data. If you look at data on pricing, it'll tell you, you can look at surveys. Surveys is actually how you address this.

• 20:47Auren Hoffman?

  On the flip side, a lot of salespeople will say, it gives us less power. Or look, we have these two customers and they're doing the exact same thing. And one's paying twice as much. So many enterprise software companies have that. So are we going to have to go back to the one paying more and give them a price break? Or how does one start to, once you have enough customers, how do you start thinking about it?

• 21:06Speaker 01

  Yeah, it's a trade off. If you're trying to extract the maximum willingness to pay out of each individual customer and your salespeople are really good negotiators, which not all are by the way, then custom pricing is the way to do that. The problem is it sacrifices every other characteristic, velocity, conversion rate through the funnel. That's my feeling about that.

• 21:30Auren Hoffman?

  You've been working on building an environment where internally where changing one's mind is celebrated. How do you institute that from a culture standpoint?

• 21:39Speaker 01

  I'm not sure I've really succeeded at that. I'm still working on it. I try to cite examples of myself. I try to live by example. I try to encourage people to do experiments when possible. If an experiment is not practical or not worthwhile, I encourage people, when you make change, make it all at once. So you can just see the discontinuity. If whatever you did work, try to write down what is your success criteria for this thing? And every quarter, I think we do a little better at it. It's tough, though. It's not the prevailing culture of the world. The prevailing culture of the world is to have arguments and to win the argument, which is not getting to the truth.

• 22:16Auren Hoffman?

  You did this acquisition of HVR is one of the biggest in the data integration space. How do you approach integrating these two larger companies?

• 22:26Speaker 01

  There's sort of two sides of it. There's the technology integration and there's the people integration. And within people integration, there was engineering and there was sales. The people integration, we did really fast. It was a considerably smaller company by headcount, by revenue. It was like 30% of the combined company, but it was a private equity backed company. It hadn't grown as fast. It was very lean and that helped. And there was a very similar culture in engineering. And so that was really easy.

• 22:53Auren Hoffman?

  Were you like very quickly replacing people's emails to an at fivetran.com email and stuff?

• 23:00Speaker 01

  I mean, that was within a month.

• 23:02Auren Hoffman?

  They kind of fell right away on this other team rather than like, because sometimes you might want to preserve the brand or preserve that internal culture or something.

• 23:10Speaker 01

  Our goal was to fuse everything together. The companies had really highly complementary strengths. So Fivetran was stronger in small and midsize companies. We were stronger in SaaS data sources and open source databases, Postgres and MySQL. We were the fast growing new kid on the block and we were bigger by headcount and by revenue. But we were weak on the older databases, the closed source databases like Oracle and SQL Server and SAP DB2. There's a bunch of them. There's a list. And we were weaker on performance.

• 23:52Speaker 01

  If you have a single really large database, that was something we really struggled with. And we were weaker with big companies. This sort of makes sense, like from a product perspective and from a sales perspective, those were our weaknesses. And then HBR was just like the perfect mirror image, strong with these older databases, strong with big companies, not the easiest thing to set up, not the easiest thing to manage, had been around since 1998. At the time of the acquisition, they were still a Windows application, so amazing in many ways, but also from another time in some ways. So the idea was we're going to combine the strengths of both companies. So we wanted to do a total merge of key people and the technology and everything. And so we very quickly merged all the systems. That was easy. Just moved everyone over, everyone's email moved, they were on Bitbucket,

• 24:35Speaker 01

  just moved it over to a GitHub repository. That happened in weeks. And then the engineering teams really melded pretty fast. They had a very similar culture. It turned out.

• 24:43Auren Hoffman?

  And by the way, I'm sure there and the HBR engineers are like so excited to finally be on Git.

• 24:49Speaker 01

  No, absolutely not. Engineers are so cantankerous about this stuff.

• 24:53Auren Hoffman?

  I thought they would have been like, woo-hoo, finally.

• 24:56Speaker 01

  Ask two engineers about their opinion about IDEs or version control or whatever, and you'll get 10 opinions. Okay, no one will be happy, that's for sure. But it was fine. It was fine.

• 25:08Speaker 01

  That part was pretty easy. The sales team was a little tougher because there was different sales cultures. As makes sense because they were selling more to older, larger companies. We were selling to smaller, younger companies.

• 25:19Auren Hoffman?

  Probably they're also using very different sales tools, even.

• 25:22Speaker 01

  Both companies were using Salesforce. I think their sales force is probably in better condition. In retrospect, we should have done the opposite. We should have adopted their instance and moved all of our stuff. That actually, we'd probably be in a better spot today. That took more time. There were a couple of iterations of that. And then from a technology perspective, there was a mixture of strategies. In some cases, we just rewrote things. We took the same people and the same tactics. About things like how you queue data internally and reimplemented them in Five Trans Codebase. In some places, we tried to actually have one codebase call the other. The first tactic has been a lot more successful and we're moving everything over time to that strategy

• 26:05Speaker 01

  of just rewriting rather than trying to actually integrate the code. It's a multi-year process, though.

• 26:10Auren Hoffman?

  Oh, really? Okay. In some ways, that sounds harder or it sounds like it would take a lot longer.

• 26:15Speaker 01

  It's harder upfront, but then it works better. The problem is troubleshooting. When things go wrong, it's so hard to troubleshoot two codebases communicating through RPC that were written with very different philosophies is really a nightmare. And meanwhile, a lot of the big ideas from HVR were totally possible to re-implement in another codebase and we had the same people. And so it was just asking them to speedrun the same idea. For example, they had a really good serialization format to queue the data internally. And so we were like, same guys, come on in here and just do it again, but this time in Java rather than in C. They basically implemented the same format. They changed a couple of small things that they regretted about the first implementation.

• 26:58Speaker 01

  It didn't take that long and it was highly successful, like Five Trans a lot faster now. And this is one of the reasons.

• 27:04Auren Hoffman?

  If you did something similar, would you run a lot of the same playbook today? Or what are some of the big things you do differently?

• 27:09Speaker 01

  I think acquisitions are super unique and highly circumstance dependent. And no matter what, you'll regret some aspects of your strategy. And most acquisitions fail, so acquisitions are tough.

• 27:22Auren Hoffman?

  Yeah, most acquisitions have a negative ROI. Like venture investing, every once in a while, some of them have a 10x. Some personal questions, before Five Trans, you were like a scientist, right? You had a PhD in neurobiology. What are some of the big, open questions in neuroscience right now?

• 27:39Speaker 01

  All of them. Why are we here? What is the physical basis of human consciousness? It's the question of questions and we have no idea. We don't even know how to state it precisely. A lot of neuroscientists get annoyed when you bring it up and they're like, that's not a scientific question. And I'm like, that is the most important question in the universe. And obviously it's a neuroscience question and you're just giving up.

• 28:01Auren Hoffman?

  Now that we like interacting with AI, it's clearly passed the Turing test, clearly passed Turing test quite a while ago. How has that changed your mind about things or opened your mind about things?

• 28:12Speaker 01

  Well, but it turns out the Turing test is a terrible test.

• 28:15Auren Hoffman?

  But if you had asked me five or six years ago, as a layman, I would have said, oh, that's the test. But then it turns out that was really easy.

• 28:22Speaker 01

  I wouldn't say it was easy, but.

• 28:24Auren Hoffman?

  Now every system has passed that test.

• 28:27Speaker 01

  Passed it, and yet still there's so much that they can't do. So clearly that is not a great test of general AI.

• 28:35Auren Hoffman?

  It's like we keep moving the goalpost. When the eighties, it was chess or something like we just keep moving the goalpost on what it means to be conscious or what it means to be AI or.

• 28:44Speaker 01

  I think the thing that is really surprising about AI and has always been surprising about it. And I go back not only to chess, but work queue planning software was a kind of AI. We make these huge leaps and these AIs astonish us with their capability along some dimension. But then they're bizarrely weak on others and things that seem easy to us are still elude these AIs. The most surprising thing I think about language models has been that precisely the things that Commander Data struggled with, human interaction, emotion, they're great at, they can blather, they can write poetry. They're so good.

• 29:26Auren Hoffman?

  Their EQ is off the charts.

• 29:28Speaker 01

  And the things they struggle with until the latest iteration of the models. And I think they put this into the training set because they've been called out as a weakness of AI. So I think they cheated to solve this problem is you ask an AI or a language model to do arithmetic in base nine. They couldn't do it and it's not that hard. If you know how that kind of math works, it's not that hard to do long form arithmetic and base math and base nine, but they couldn't do it. And there continue to be these bizarre gaps in there. So ironically, the things that Commander Data was great at, it turns out it's super hard for these language models to do. And the things that he struggled with are easy.

• 30:03Auren Hoffman?

  To some ways, it's like, it's so interesting. Whenever I meet a human who's super talented in one vector, they're like the best guitarist or the best mathematician or whatever, they're often so terrible in another area. You would think it translates really well from one to something else, even if it's like something somewhat analogous. A great scientist is sometimes terrible at policy involving science, something that's like right tangential to what they're doing.

• 30:29Speaker 01

  Interesting. It's hard to say.

• 30:31Auren Hoffman?

  You disagree?

• 30:32Speaker 01

  Maybe the great scientist is actually really good at it and you're just wrong, right?

• 30:35Auren Hoffman?

  That might be right, yeah.

• 30:37Speaker 01

  I think in general, in people, all intellectual skills are highly correlated. That's what IQ is. It's just the observation that people who are smart in one way are almost always smart in every other way. And AIs are not like that at all. Going back to chess again, it's like you can make an AI that's so good at playing what we think of as a very intellectually difficult task. But we struggled for years to do things that humans would think of as easy, like driving. Driving, we're just getting now. I think people find driving to be much easier than playing chess, to be great at it, right? But it's a much harder AI problem.

• 31:14Auren Hoffman?

  Yeah, it's a good point. It's a really good point. Now, you've also said your brain is not a computer. Why do you think that? Because I know there's a big debate around that.

• 31:21Speaker 01

  Yeah, I think it's just a bad analogy.

• 31:23Auren Hoffman?

  You don't think there's a scenario where we could upload our brain and have basically the same function on a machine or something?

• 31:30Speaker 01

  I didn't say that. I think that it's very mysterious how consciousness works. And I think you can simulate physical things in a computer. But the problem with the analogy of brains and computers is that computers are primarily software driven, right? You have a general purpose CPU that can basically do anything and then you write software that runs on that CPU. So they're very programmable. Brains are very much the opposite. We come with very much pre-baked. Great example of this, these language models to get to competence, to get to human-like ability to talk. They are trained on, the numbers are not public, but people think perhaps like a trillion words to train like a state-of-the-art language model. A human being, by the time of becoming a toddler where they're pretty fluent in language, they've heard tens of millions of words.

• 32:15Speaker 01

  So that is a 100 million times difference in the size of data. We come pre-programmed with something really important.

• 32:23Auren Hoffman?

  Probably even in our lifetime, we don't hit the same kind of level.

• 32:26Speaker 01

  Not even close. So that's a very different system you're describing there.

• 32:30Auren Hoffman?

  For those of you not watching on video, there's a bed behind you. I know you've got an interesting theory about beds and videos. So explain to the audience your theory about beds and videos.

• 32:40Speaker 01

  I have to credit my boyfriend with this theory, I think. I think there's a U-shaped relationship between your seniority in your job and the probability that there's a bed in the background on video.

• 32:51Auren Hoffman?

  So like super low seniority, there's a bed, super high seniority, there's a bed?

• 32:56Speaker 01

  Exactly. Super low seniority, you're not worried about it. You're like, I'm a junior associate or whatever. Everyone expects this. Then, once you get to the other end of the spectrum, you're like, I don't care.

• 33:05Auren Hoffman?

  I've heard that about webcams as well. The crappy webcam at the low end and the super crappy webcam when you're Jeff Bezos or something.

• 33:13Speaker 01

  That's funny. I'm not at that level yet. I'm very particular about webcams, but maybe someday.

• 33:18Auren Hoffman?

  When you're at the trillion dollar market gap, I expect you to have a very crappy webcam.

• 33:22Speaker 01

  Exactly.

• 33:23Auren Hoffman?

  All right. What is a conspiracy theory you believe?

• 33:25Speaker 01

  Santa. Santa is the real conspiracy theory. And this, I think, is part of why people believe in conspiracy theories as adults. Because when you're a child, you are the victim of this conspiracy theory. And then you discover that it's real. The conspiracy, I mean, not Santa. Santa is a global conspiracy of adults to deceive children. Think about that for a second.

• 33:48Auren Hoffman?

  One of the things I like about the Santa conspiracy is like, at some point, I don't know when that is, but let's say some point, you're like nine, 10 years old, you get in on the conspiracy, and then you actively participate against all the other younger. It's like, there's a point where you get read in very quickly.

• 34:05Speaker 01

  And what is the lesson that you are learning by having this experience?

• 34:10Auren Hoffman?

  You're learning to lie and deceive.

• 34:12Speaker 01

  What is your favorite answer you've gotten to this question, by the way?

• 34:15Auren Hoffman?

  I believe that there's conspiracy theories everywhere, so I'm into it. Usually, when you read something that is, let's say, negative about a company in the New York Times or something, it's pretty rare that the reporter came up with it on his or her own. Usually, it got fed it by a competitor or somebody who had a reason for that story to get out there. Whenever I read anything, I often try to figure out, okay, who's benefiting from this? And there's a pretty good chance that somebody in that subset of people who are benefiting is involved in that story.

• 34:50Speaker 01

  That's a fair one. So that's your favorite answer that you've gotten to this?

• 34:53Auren Hoffman?

  That's my own answer, maybe not my favorite answer. Santa's pretty good. I would say it's definitely up there. It's one of my favorites. Because I never thought about it as like a global conspiracy, but it's clearly one of the biggest conspiracies that there is. One of the things about the tooth fairy, you have kids?

• 35:08Speaker 01

  No.

• 35:08Auren Hoffman?

  Okay, so one thing about the tooth fairy is there's been no inflation in tooth fairy things.

• 35:15Speaker 01

  It's still like a quarter.

• 35:16Auren Hoffman?

  Maybe there has been inflation, but my kids have gotten a dollar and I think people have been getting a dollar for like 25 years. I don't think it's gone up at all. I think you just get the dollar and even if you're super wealthy, I don't think your kid is getting like $50 or $20 or something when their tooth comes out. Maybe they are.

• 35:33Speaker 01

  That is interesting. I'm trying to remember, what did I get? I feel like it was coins, but I'd have to check with my mother.

• 35:39Auren Hoffman?

  Yeah, sometimes we do the silver dollar, or Susan B. and whatever, that type of dollar as well, the coin dollar, but it's still $1. So I feel like kids are really getting the low end of the stick. We're just not keeping up with the purchasing power parity of a tooth loss has gone down pretty dramatically.

• 35:55Speaker 01

  Things have to go down. Otherwise, there's no productivity gains. You got to find the savings somewhere.

• 36:00Auren Hoffman?

  Yeah, that's a very good point. Prices have to go down somewhere. Our last question, we ask all of our guests, what conventional wisdom or advice do you think is generally bad advice?

• 36:08Speaker 01

  The precautionary principle is terrible advice. When in doubt, err on the side of caution. First of all, it's not a useful, it has no limiting principle. You can just go to infinity of cautiousness. And second of all, I think in general, people take too little risk, not too much. So I think the precautionary principle is extremely popular and absolutely terrible advice.

• 36:28Auren Hoffman?

  How do you deal with these Cassandra's that are out there? Because they're occasionally right. And there's a guy, if you remember the levees in New Orleans, when Katrina came 20 years ago, there's a guy who was like writing about the levees all the time. And then of course, in the levees broke, we should have done the rejiggered the levees, put some money behind it so they wouldn't have broke. And clearly that would have been a good thing. But then you dive into all the other things the same guy said, none of them have still come to pass. And it would have cost potentially hundreds of billions of dollars to do everything that he wanted to do. So how do you actually dive into these Cassandra's?

• 37:05Speaker 01

  Utter contempt is the right answer. I mean, first you have to understand that it's hard to persuade other people of this. That's a different question. But first you have to understand that if you evaluate people's predictions in retrospect. There will always be someone who is right on everything. Gets towards what we were talking about earlier about writing down your theory of success for whatever it is you're doing in advance to avoid fooling yourself. But if you just go out and look in the world, there's always going to be someone who looks like a great prognosticator in advance. And worse, there are a lot of people out there who sort of hack the system. So they make a lot of predictions.

• 37:42Speaker 01

  They're basically like mentalists that are just specific enough that they can go back in retrospect and pull them out and say, look, I was right about this thing.

• 37:51Auren Hoffman?

  Here's the tweet. Yeah.

• 37:53Speaker 01

  And it's so toxic.

• 37:55Auren Hoffman?

  It's like a game. People do it all the time.

• 37:57Speaker 01

  I know, and I think some people do it without realizing it. You just have to understand the sort of nature of large sets and retrospective analysis and how dangerous that is. And yeah, have really high, don't listen to them. I'll tell you a funny story along these lines. My grandfather was a OBGYN in a small town in Ohio. A fun fact, the football coach, Jim Harbaugh, was delivered by him and is actually named after him.

• 38:23Auren Hoffman?

  Oh, my gosh. Amazing.

• 38:25Speaker 01

  Yes, my grandfather, Jim Fraser, he was an OBGYN. He delivered an entire generation. And this was back before you could tell what sex a baby was going to be. And he was a little bit of a prankster. And so sometimes he would tell the mother that he could tell. He would look at her, he would do an examination. It's going to be a boy. And then he would write down in the chart, girl. And if he was right, they'd be like, Doc, you predicted it was a boy. And if he was wrong, he'd be like, no, I said it was going to be a girl. Look, you can see right here.

• 38:52Speaker 01

  And I think he eventually got caught by some of the Catholic families who had a lot of children. And that's kind of a harmless example, but this is what people do. I always think of that story. When you look at prognosticators, you just plant a bunch of seeds. And then after the fact, you go up, you go back and say, oh, look at this one. This one grew into a tree.

• 39:09Auren Hoffman?

  In your own company, there are going to be these, there's so many things that go wrong. There's so many, you can't patch everything. There's some things that you have to patch. There's some security thing that you have to go fix, or there's some thing that you have to do. You can't say, we're not going to fix everything, but you also can't go fix everything. So how do you even? Know how to deal with that. There's defense and offense. You can't always play offense. You have to play defense sometimes.

• 39:34Speaker 01

  Yeah, this is more of a general question about risk. And I think that, first of all, the tendency is to take no risk towards that precautionary principle. So that's the main thing you end up fighting against is you're actually pushing people to take more risks. And so you make the point, zero risk is not the right answer. You look at, how did you come to the decision that this was the right thing to do? And then you look at someone's framework and you say, would this framework ever lead to taking any risk? And if the answer is no, it's not a good framework. You point out to people that if nothing ever goes wrong, you didn't take enough risk. Things should go wrong. Bad things should happen. If nothing bad happens, then you played it too safe. There's a great Elon Musk quote about engineering that if you never put any parts back, you didn't take enough out.

• 40:17Speaker 01

  And I think there's a sort of endless cousins of that principle. It's a great principle. If you never reverse something, then you didn't go far enough in some direction.

• 40:26Auren Hoffman?

  Yeah, it's interesting. At Flex Capital, our seed fund, we do a lot of deals. And so occasionally, one of them turns out to be a scam or whatever it might be. There's always a question internally, what should our scam rate be? It shouldn't be zero because then we're not moving fast enough. We're doing too much due diligence on everything. The shrinkage rate at Walmart can't be zero. They have to frisk everybody on the way out, which would be a bad experience. So you always have to figure out there's some sort of optimal rate that you have to understand.

• 40:54Speaker 01

  I absolutely agree with that. The hardest thing is there are a few things where the rate you're going for is actually zero. There are certain kinds of hacks. At Five Tran, the prototypical example is we must never put a customer's data in the wrong customer's destination. That needs to happen zero times.

• 41:10Auren Hoffman?

  You have to know what that is.

• 41:12Speaker 01

  So it's like, how do you gauge risk on that dimension? Look at close calls. You say, okay, first of all, this is the thing that really needs to happen zero times. This is one of the few cases where actually zero is the right answer. And then you look at close calls and other things that are distant from it and say, okay, those should not be zero. If those are zero, then we're being too aggressive. But the core thing does actually need to be zero. And that's how you calibrate. Those are the hardest ones. And not everything is like that. Getting sued is not like that. Lots and lots of things that I think a lot of people would think of, you want to happen zero times. And once a company gets big enough, actually zero is the wrong answer about that.

• 41:47Auren Hoffman?

  There are certain parts of the organization, whether it be like the general counsel's office or the HR office or something like that, that their job is to be a little bit more risk averse. How do you actually work with those parts of the organization?

• 42:00Speaker 01

  You tell them that's not their job. There's a whole list. It's legal, security, SRE. Even procurement is can actually be like this. Tell them, no, no, no.

• 42:09Auren Hoffman?

  Price or something.

• 42:10Speaker 01

  That is not your job to advocate for one side of the equation. Your job is to balance it. Tell me about how you've made mistakes in both directions. And if you haven't, then you're not actually doing your job.

• 42:22Auren Hoffman?

  All right, this has been amazing. Thank you, George Fraser, for joining us on World of DAS. I follow you at Fraser George W. on X. By the way, I think you need a new name on X, but I follow you at Fraser George W. on X. And I definitely encourage our listeners to engage with you there. This has been a ton of fun.

• 42:38Speaker 01

  Nice to talk to you.

• 42:41Auren Hoffman?

  If you're a super data nerd, go to worldofdaas.com. That's D-A-A-S, worldofdaas.com, and sign up for our weekly Data as a Service Roundup Newsletter. Thanks for listening. If you enjoyed the show, consider reading this podcast and leaving a review. For more World of DAS, and DAS is D-A-A-S, you can subscribe on Spotify or Apple Podcasts or anywhere you get your podcasts. And also check out YouTube for videos. You can find me at Twitter at at Aurin. That's A-U-R-E-N, Aurin. And we'd love to hear from you.

• 43:18Auren Hoffman?

  World of DAS is brought to you by Safegraph. Safegraph is geospatial data for physical places. Check it out at safegraph.com. And by Flex Capital. Flex Capital invests in data companies like those we talk about at World of DAS. Check it out at flexcapital.com.