EP 124: Databricks CEO Ali Ghodsi Breaks Down the AI Hype-Cycle

Ali Ghodsi, CEO of Databricks (now a $61B company), has become one of the top minds in AI, evolving from researcher to leading one of the biggest AI-centered private companies. In our conversation, he broke down the AI hype cycle in detail and made bold predictions for the future. He also shared his biggest operating lessons from scaling Databricks to a market leader and pivotal moments in his journey. (00:00) Introduction to AGI and AI Challenges (01:00) Ali Ghodsi's Journey in AI (01:38) The Evolution and Impact of AI (03:59) AI in Enterprises and Consumer Adoption (05:25) Specialized AI Systems and Data Advantage (07:32) The Hype Cycle and Future of AI (08:22) Scaling Laws and Compound AI Systems (19:52) AI Use Cases and Industry Applications (28:29) Early Days of Databricks and Spark (38:43) Taking Over as Interim CEO (39:00) Focusing on Enterprise Sales (40:42) Innovating in Open Source (41:08) Challenges and Pivots in 2015 (44:31) Building and Scaling New Products (47:57) Leadership Evolution and Advice (54:45) Hiring and Promoting Leaders (01:10:44) Future of AI and Databricks Executive Producer: Rashad Assir Producer: Leah Clapper Mixing and editing: Justin Hrabovsky Check out Unsupervised Learning, Redpoint's AI Podcast: https://www.youtube.com/@UCUl-s_Vp-Kkk_XVyDylNwLA 🎥 Subscribe on YouTube: https://www.youtube.com/channel/UCugS0jD5IAdoqzjaNYzns7w?sub_confirmation=1 Follow on Socials 📸 Instagram - https://www.instagram.com/theloganbartlettshow 📱 X - https://twitter.com/loganbartshow 🎬 Clips on TikTok - https://www.tiktok.com/@theloganbartlettshow About the Show Logan Bartlett is a Software Investor at Redpoint Ventures - a Silicon Valley-based VC with $6B AUM and investments in Snowflake, DraftKings, Twilio, and Netflix. In each episode of The Logan Bartlett Show, we sit down with the people behind today’s most important startups and extract the tactics, lessons, and frameworks they’ve learned the hard way. Conversations span hiring to GTM, product, growth, fundraising and everything in between - collectively forming the ultimate playbook to make you a better CEO, investor or board member. Tap follow and enable notifications to stay ahead of the game.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:04Logan Bartlett?

  Welcome to the Logan Bartlett Show. On this episode, what you're going to hear is a conversation I have with Ali Ghodsi. Ali is the co-founder and CEO of Databricks, a business valued at over $50 billion. Databricks sits in the center of the artificial intelligence revolution that we're seeing today. Ali and I talk about the state of AI, as well as what he's seeing going on at the ground level, his view on the federation of models that he sees coming, as well as what he sees for the future of AI over the course of the next couple of years. We also go into the founding of Databricks, his journey to becoming ultimately the CEO, as well as the different operating principles that he runs the company with today. A really fun conversation that you're here with Ali now. Ali, thanks for doing this.

• 0:49Speaker 01

  Of course.

• 0:50Logan Bartlett?

  So you've been doing artificial intelligence since 2013 with Databricks?

• 0:56Speaker 01

  With Databricks, yeah. Before that, we were at Berkeley, we were doing also AI.

• 0:59Logan Bartlett?

  When did you actually start?

• 1:00Speaker 01

  I think 2009, we started participating early, 2009, 10. Spark, when it was originally created, was to participate in the Netflix competition, which was, hey, they gave you all these movies, and you had to do recommendations, and they would, you know, they give a million dollars, a half a million dollars to the winner. So we got the highest score on that, together with another team.

• 1:19Logan Bartlett?

  So you're 15 years in now?

• 1:20Speaker 01

  Yeah.

• 1:21Logan Bartlett?

  It obviously, it was considered somewhat niche, and maybe academic initially, and then obviously, we're at a very different point today. I'm curious, as you, as ChatGPT happened, and that moment where it seemingly went from enterprise and business use to sort of mainstream consumer adoption, did that change your perspective on artificial intelligence in any way?

• 1:43Speaker 01

  Not really. I think that it's kind of changed, I think from 2013, 14, 15, I just didn't understand why people don't really get into AI. You know, in 2014, I remember having this debate with many of the people at the company, that I wanted to call it AI. They said, no, that's like not authentic, we should call it machine learning, it's ML. And I said, look, once this stuff goes mainstream... You know, mainstream, it's not going to be called ML, right? They're like, no, but that's not accurate. AI is robotics. And I just never could understand why it's not much bigger of a deal. It was kind of a little bit esoteric, machine learning, AI. And in, you know, twenty twenty two, November, it just made all the sense.

• 2:23Speaker 01

  But then the magnitude of how much it sort of just shook everybody up, that was kind of a shock.

• 2:29Logan Bartlett?

  It was surprising to you. Were there points that you look back on actually in those fifteen years that were actually maybe groundbreaking to you that you're, but but not to the mainstream?

• 2:40Speaker 01

  Yeah, I mean, like when we started, people weren't really using machine learning for other things than ad tech. It was really ad placement, you know, click through, you know, click click streams for logs to place ads and so on. So I would say two thousand fourteen, fifteen, when we said really large enterprises started using this stuff for more interesting things like Expedia, how they listed the hotels, that how you ranked the listing of the hotels gave them a huge revenue uplift. It was kind of a game changer. Regeneron found a sort of, so they found the marker in the genetic DNA for a disease and they were developing drugs. So you could start seeing like, wow, that has really big impact on a lot of these companies. That was a big deal.

• 3:21Speaker 01

  But then I would say two thousand eighteen, nineteen, we started seeing the natural language processing breakthroughs coming. You know, with GPT two and so on, it was already pretty, pretty amazing. So it was pretty clear that the Turing test would be broken soon in a couple of years or so. So. So, you know, it's been many of them, but yeah, who would have anticipated that the whole world would kind of wake up in this way? And this would be like front news every day, you know?

• 3:44Logan Bartlett?

  Yeah, hard to believe. I've heard you compare the chat G P moment to the iPhone moment. Can you can you compare that analogy and what the implications are for apps? And maybe how enterprises or users should think about artificial intelligence?

• 3:56Speaker 01

  Yeah, I mean, when the iPhone came out, everybody realized kind of what the smartphone revolution would be. And what happened immediately was that every company on the planet said, we have to build lots of apps. We have to build our own iPhone apps.

• 4:08Speaker 01

  And so there was this flurry of activity, all the companies building apps. But, you know, did the apps really make sense? A lot of the apps that they built, you know, I I kind of joke about it and I say it's kind of like building flashlight application for your iPhone. But then a couple of years later, the iPhone just has the flashlight built in. And I think twenty twenty three was one of those years that everybody wanted to say, hey, I own AI at our company. I am the AI leader. I am the Gen AI. So there were a lot of chatbots that were built. And you know, there'll be just like, there's an app for that in two thousand and seven or eight, there's going to be an AI company for that that just does that. So, you know, for instance, people building chatbot for HR documentation, you can ask, you know, what's your policy on paternity leave or maternity leave? And companies building that, but it doesn't make sense that a company builds that for themselves.

• 4:52Speaker 01

  It makes more sense that there is a SaaS app for that and everybody just buys that. And a little bit, we're in this moment where there's a lot of like, hey, we're going to build our own flashlight app or whatever chat app for everything. And now we're seeing slowly the companies being built and the apps being built that are going to specialize and we're all going to just leverage them. I think we're like in the first innings of that.

• 5:14Logan Bartlett?

  So, so to take that analogy a step further, do you think that there ultimately will be disintermediation between the end customer and maybe the HRS system where a native AI business sort of takes in that natural language interface and no longer would work day or ADP or whoever it is on the backend sort of serve as the interface to the user?

• 5:34Speaker 01

  Could be. I think that this dynamic is playing out that in this AI era, data is extremely important. So, why shouldn't everybody build their own chat app? Well, you know, you should really ask yourself, what's your competitive advantage as a company? What is it you can do uniquely that no one else in your industry can do? Everybody can call OpenAI as LLM. So you're just calling, you know, GPT four or five or six, that's not going to help. So what does you have? Typically it comes back to the data or some customer relationship, but typically data, you must have some data advantage that nobody else has. So that's what it's going to come down to. The big companies, the incumbents, they have that data advantage, right? They have customers and they're collecting a lot of data. So the question is, can they leverage that data advantage and incorporate the AI and

• 6:19Speaker 01

  do the innovation that's needed? On the other hand, we've seen in these waves before that you'll have these smart founding teams that are just completely disrupted, thinking about things differently, and they'll rise and they'll do things in a way that the big companies are just, they're asleep at the wheel and they're not going to do. I think we're going to see both. I think we're going to see some, we're going to see a lot of company startups that fail, just like in ninety nine, when the internet was about to come. But we're also going to see a few that become the next Google, Facebook and so on. But I think we're also going to see some incumbents that actually figure out how to leverage the data that they have and do the investments needed to crack the code on this. We're going to see both of those. How exactly that? You know, flesh is out. I would probably bet more on the startups, to be honest, than the big ones.

• 7:01Logan Bartlett?

  Interesting.

• 7:01Speaker 01

  They move too slow. The big ones move too slow. And you know, if you have like a killer app, you can get the data flywheel probably collecting data on it and grow the usage. And then you'll have enough data advantage to be able to do that. And if the big guys are not paying attention, it'll move too fast. And before you know it, it's too late to react.

• 7:21Logan Bartlett?

  Where do you think we are in the hype cycle of AI right now?

• 7:25Speaker 01

  I think, I think that we're gonna see, there's, I don't think like, hey, we're in trough of disillusionment. I think it's, you know, frankly, I think it's like somewhere at the top. And we're gonna see that, you know, soon, we're gonna see more and more companies not doing so well. And I think there's gonna be a narrative that, hey, this AI thing was complete, you know, it was overpromised. There's nothing there. The quality of the models aren't there. They just don't work. This stuff doesn't work. It wasn't worth it, all these companies and so on. So I think we're kind of going in towards that very, very soon. But then I think right after that, you know, when we actually are gonna see those very companies start coming out with these applications that are completely game changing for humanity.

• 8:07Logan Bartlett?

  Yeah, the good news is crypto's back. So people can bounce back and forth between one and the other. I'm curious, as we record this, I guess, early November of 24, there's been a bunch of reports about scaling laws and maybe the limitations that we're reaching some of the outer ends and diminishing returns on that. Do you have any opinion on where we are with scaling laws right now?

• 8:26Speaker 01

  Yeah, I mean, we actually put out a research paper, you know, my co-founder, Matez Aria. It's about compound AI systems, we call it. And it was actually put out probably six months ago, or maybe even more. And it kind of predicts this, which, what it basically says is that to really eke out that last quality bit now and really like move up the other, we really need to build these agent systems where we have many parts, many components that work together. And in particular, it's gonna be really, really important that we can evaluate the quality of that whole AI system on the particular task you want to do. Like just to take that silly example that I gave, HR docs, Q and A. How well do you answer, you know, questions from employees on HR documentation?

• 9:12Speaker 01

  Not how well do you pass the bar exam or medical exam, general intelligence, not that, but this specific. Only once we have this evaluation, we can start building these agentic systems where we can start doing hill climbs, so keep improving the quality for that specific task. So we call these compound AI systems. And these will also leverage a lot of tools. And they will also leverage classic machine learning, because classic machine learning has much better numeracy, it's much better dealing with numbers and so on, calculators, things like nature forecasting and so on. So that's what we call compound AI systems. And the beauty of those is that you can, when you run them and they get something wrong, you can debug which agent got it wrong. Where was the reasoning step that was broken?

• 9:56Speaker 01

  And you can then improve it, you can get better and better, and you can do this hill climbing and keep increasing the quality of the model or the AI system. Scaling laws, you pre-bake these gigantic models, you wait six months and the model comes out and maybe it's bad at predicting something, maybe it's numeracy is not good. And you can't even know why it's not good. There's no way for us to know, hey, why did it get this thing wrong? I don't know. You do a bunch of matrix multiply, a bunch of attention layers, something plops out, it's inaccurate, I don't know, let's go train a bigger one with more data. So it just seems much easier to do this now, more agentic workflows at inference time. So it makes a lot of sense. I think there's going to be a lot of innovation. It is nothing, you know, I think everybody's now focused on this.

• 10:41Logan Bartlett?

  Do you think there's an implication for like, do you think that results in a federation of model types and different capabilities that people are respectively good for?

• 10:49Speaker 01

  Yeah, I think you will build AI systems that are very, very good at the particular task. We see it all the time. Our customers, they typically want to do the same thing again and again and again. Like they have massive data, like they have database with SEC filings, right? And or they're an insurance company and they have all these insurance documents. And they want to repetitively, again and again, on massive amounts of data, repetitively do the same task again and again and again. So then for them, it comes down to, what's the highest quality I can do on this specific task? I don't need to be good at AGI, artificial general intelligence. Can I do this again and again really, really well? And can you give me the lowest cost for that? Because I'm going to do this at scale. It's going to cost me a lot of money. So then it makes sense to specialize a system that's really, really good at that.

• 11:32Speaker 01

  You know, having a generic gigantic model is just way too costly. And actually, it won't do as well, even though it's really smart. It won't do as well at that particular task, repetitive tasks that you want to do. So we see this all the time. Like, one of the surprising things on Databricks is, you know, we have models of all kinds of sizes. You can use Lama 400, 5 billion parameters. But in aggregate, we get more revenue from tiny models. And that was surprising to me because, you know, per model, you make much more on a bigger model. Uses much more GPUs and so on. But customers want to take the big models, distill the smallest possible that can get good enough quality and just use that because that saves them money. And actually, in aggregate, we make more money on that. So, yes, I think we're going to see that. And you see that in life also, like, you know, are all the tasks that humans are doing, all the jobs that are happening all day?

• 12:16Speaker 01

  Do you need Terrence Tao for every task? You know, are you getting the smartest person possible on the planet doing them? Or many of the jobs are repetitive and some things, you know, it's good enough and you want to pay this particular price for it. So it makes sense that you have a smarter model that's specialized.

• 12:29Logan Bartlett?

  What do you think about the concept of like general reasoning? Or the path to, maybe not AGI, but, but a more holistic understanding and reasoning that sort of underpins these distinct models? Do you think we're, we're approaching a step function in that? Or do you think that that we've reached some certain plateau in it? And now it's about the data and the different functional areas?

• 12:48Speaker 01

  I mean, look, the scaling laws, it's like on a logarithmic scale. So moving to the right, we are going to move to the right. And yes, that means that we're going to get improved intelligence. But, you know, the next thousand X is not going to be as, you know, it's not going to yield as much as the previous thousand X. Plus, you know, one of our researchers put this out, you know, over the last 11 years, you know, we've had 100 million times improvement in compute that we've thrown at the problem. We're not being, going to be able to get the same over the next few years. We're going to improve it, but not as much. You know, there's just not going to be enough energy compute to throw at it. And we're not going to have enough data. Like, you know, Lama was trained on 15 trillion tokens. What's a 10x the size of the web?

• 13:31Speaker 01

  Are we, you know, are we going to 100 x 1000 x the amount of data that can do that? Getting that high quality data, it seems just, it gets exceedingly expensive. It's much easier to build these compound AI systems, these agentic systems where you can actually debug and get really good at the task. So yeah, I think there's going to be a slowdown. I also think that a lot of those things where people are like, look, we're going to really crack the code and come up with that like super AGI. That kind of would require some kind of, you know, recursive self-improvement loop. I don't see that happening because the next model is more expensive and requires even more people. You have to be even more careful about it. I would believe that if you would say, hey, GPT-5 will cost, you know, $1 and it's going to take one second to create.

• 14:16Speaker 01

  If we were headed in that direction, then you could see like, okay, well, then it can spawn off and we can just apply genetic algorithm, do lots of them and pick the best and so on. But we're not headed in that direction. It's the opposite. It's like, we're going to do a very specialized data center. We're going to build a model over one year period of time. We're going to be very careful, lots of humans involved. So I don't have a crystal ball, but my guess is that for many more years, we need humans in the loop. And we need this stuff to be verifiable. Like, we can't just, just like we saw with cars or with, you know, you know, autopilots in airplanes. We want to have a human in the loop that can verify that it's not veering off in the wrong direction. And it's just accelerating the work that they do today.

• 15:01Speaker 01

  And I think for that, we're going to use more of these agentic systems.

• 15:03Logan Bartlett?

  So it sounds like a lot of lead bullets. There's no silver bullet that you sort of see on the horizon at this point. When you hear people talking about like $50 billion models or, you know, throwing all this, very good for your business. I'm sure as people think about this, but do you think that, that all of the stuff we were just talking about, it means that, that that's a little pie in the sky and overly ambitious for diminishing returns? As people sort of talk about these models at that level of scale?

• 15:30Speaker 01

  Well, again, I don't have a crystal ball and maybe they're right. And if they are, it's amazing.

• 15:33Logan Bartlett?

  You've been pretty good historically in predicting some of these things.

• 15:35Speaker 01

  Yeah, well, I think that, you know, the reasoning that they have makes sense, which is like, hey, we're freaking creating AGI. It's worth any cost. It doesn't matter what it costs and it doesn't matter what dilution and your economic model is done. I don't care about them because look at what we're creating. Makes sense to me if you actually achieve it. But, um, you know, I would probably lean more on the skeptical side. Um, you know, spending that much money, like my, I guess I'm too, I guess I'm too much of an operator running a company that needs to think about, you know, uh, you know, growth rate, LTV, CAC, you know, you know, so on and so forth, operating margin that I think, you know, if you got to spend that much money to produce a model. And then you have to do inference on that model to, you know, for it to sort of pay back the investments that you need to train that model.

• 16:19Speaker 01

  And it seems right now the case is that before you've even gotten the money back for how much it costs to train the model. It's already commoditized and there's a better version by someone somewhere. Um, so then economically, it's just right now not feasible to do that. Um, it seems that for those particular tasks, the cost to create these, uh, you know, compound agentic systems. That's worthwhile and ROI is there. Why not do that? And it can drive a lot of value and continue doing that. And then maybe we'll be surprised and one of these, you know, labs come up with that super model. But I'm skeptical. I think that there's going to be a barrier. We got to go through some, we got to overcome some big barrier to get to that complete autonomous. That can, you know, just get the accuracy right and it's reliable.

• 17:03Logan Bartlett?

  If we froze, uh, model development right now, do you think that the existing capabilities that we have in AI is enough to, uh, catch up at some point with all the tooling and the applications and all that? With the excitement and the promise that we're seeing, or do you think these incremental gains that we're sort of talking about? And the, the more niche sort of specialized models, we need to keep making those improvements to, to catch up with, you know, the potential of what we could actually accomplish?

• 17:34Speaker 01

  Absolutely. I think that, uh, You know, humans are the longest pull in the tent here. If no human uses any of this stuff, it's absolutely useless. You can't even come up with that super, you know, AGI thing. If nobody's using it, it doesn't matter. Humans take a very long time to change all of their behaviors, habits, processes in these companies and so on. It's just very, very hard. So I think with what just we have now, there's huge untapped potential. And it's not yet been applied in every workplace, in every function, in every process, in everything that we're doing. And I think we can even improve what we have today with those kind of techniques that we discussed. So I think like even if we freeze it and say, hey, that's it. Actually, we didn't know how to create a single model that's even better. This is the best we'll see in the next 20 years.

• 18:17Speaker 01

  I think it's going to be massive implications on the economy and, you know, and what we can do and so on. So I would still be extremely bullish on this. That's really true to what I believe in.

• 18:28Logan Bartlett?

  Is there something that you see from your seat and purview that you have that maybe wouldn't be intuitive to the average listener? That's, you know, paying attention to all this stuff, but you're able to see so clearly the same way that you maybe did 10 years ago. You're like, hey, why isn't anyone, why does anyone care about this AI thing? Is there another insight, maybe not at that quantum, but something else that you sort of see that you feel like might be mainstream or well understood in a couple of years that is underappreciated today?

• 18:55Speaker 01

  Yeah, I mean, I think it's the fact that you're focusing the task. I kind of said it, you focus the tasks on the repetitive stuff that today is costing you a lot of money. And if you can just get the reliability up on those specific tasks, it's massive improvement for these organizations. And it's pretty game changing. We haven't actually reaped that yet. Those use cases are sort of right now being worked on. Folks are building them. They're not in production. It hasn't had impact on all the companies that are using this stuff, but it's just around the corner. It's like guaranteed it's going to appear. It will probably take two, three years before we see it and then we'll see the impact of it. And actually, you know, I fear or, you know, suspect that there might be before that we hit this trough of dissolution and then actually these use cases land and we'll see the companies

• 19:37Speaker 01

  that are doing that. So if you can see it baking already, it's just not in production yet.

• 19:42Logan Bartlett?

  Are there use cases that you've seen people use Databricks or other systems for that have gotten you particularly excited? Back to the Regeneron analogy that you were saying 10 years ago, maybe, maybe something today that you guys see on the field that is exciting or makes you, you know, particularly looking forward to the future of what could happen?

• 19:59Speaker 01

  In every vertical, in every industry, we see lots and lots of use cases, you know, like financial services, you know, there's so many financial advisors, FAs out there, you know, they basically now we generate automatically for one of our customers, you know, large bank automatically in advance, you know, what advice I should give to you, they it's customized, they understand everything about you, your wealth situation and so on, and they can give you customized advice, you know, it's affects, you know, thousands of thousands of these folks that are doing that, right? In insurance, sifting through massive, massive amounts of documents to assess risk, right? It's like they can do it. And by the way, one thing that people underestimate is that the models are very fast at reading data. They're slow at producing text, right? They're, you know, much slower.

• 20:44Speaker 01

  The other insight I would say is like, you know, that's surprising is, you know, it's probably at least a 10 to one ratio between how much people are using the models to read stuff than to produce. So they have, when they use Databricks, they use us 10 times or 100 times, sometimes more or even more to read stuff and then produce very little text or very little data out. Whereas people are focused on that output that comes out, but it's usually much, much smaller. So, you know, underwriting is another one that we're seeing. There's so many going through all the SEC filings and figuring out what's happening, you know, extracting signal. So figuring out alpha, we see it in all the financial sector. So there's so many use cases in healthcare, electronic medical records, there are just so many of them. They're all handwritten, being able to actually finding

• 21:29Speaker 01

  structure in that. It's, you know, that's a treasure trove by itself. So use cases around that. So there's, you know, hundreds of hundreds of these you see all the time in every industry. They just haven't been productionized yet. They're not fully being used by the workforce.

• 21:46Logan Bartlett?

  Do you think that read to write ratio, do you think that stays around that? Are there any trends that you sort of see as read going up in comparison to write? Or you think at some point the writing and the creation catches up with the reading and summarization?

• 22:01Speaker 01

  I mean, hard to see that because, you know, how much are we going to read as humans? You know, how much we, I mean, there's only so much and we read slowly, right? Whereas having the AI crunch a lot of information, you know, so that it has that context makes a lot of sense. So if anything, that probably increases and if we can do that faster. So that's, that's where the trend is headed. We're seeing even more and more of that when we look at our data than when we started.

• 22:25Logan Bartlett?

  As you counsel customers, be it big enterprises or smaller ones, about figuring out where AI and automation can, can exist within their organization? And how to continue to dive deeper into that journey? Are there any takeaways or key points that you keep hitting that, that you feel like, hey, if you could tell all customers? This is how to set up a team or this is how to think about it. Any frameworks or primitives or whatever that you think about?

• 22:54Speaker 01

  First of all, you gotta get your data strategy right. Like a lot of companies now want to sort of skip that step and jump straight to the AI. And, you know, the data teams have been at it, trying to get the data of their company organized, secured, you know, get it in one place. So they can actually get access to it, not in different silos. And, you know, that's has been a journey and it's not finished work. It hasn't, that work has not been on the CEO's radar of the Fortune 500. That's, you know, it's been sort of a esoteric topic. Maybe IT was sort of worrying about it. Now the CEOs are waking up after ChatGPT and they're saying, I need AI. It's game changing. I see it now. I've used it myself. Give me AI. And they're sort of skipping that step.

• 23:37Speaker 01

  So that's, it's extremely important that you go back to the basics. The data is extremely important, so you have to get that in a good place. And if you don't have that in a good place, you're not going to actually going to be, be able to succeed with AI. The second thing is focus the use cases of AI that you want to do on where you have a data advantage. So for your company, what is your data advantage? And frankly speaking, oftentimes I find myself talking to customers, telling them, maybe don't do this project. And so why not? We're very excited about it. We're gonna do this and that. We are ready to go. We'll pay you a lot of money for it. And it's like, yeah, but what's your advantage in this? You know, there'll be an app for that by someone else. So let's focus on places where you have a competitive advantage against your competition. And it typically has to come back to something you have around the data and you do have it.

• 24:21Speaker 01

  So let's focus on those use cases because we want our customers to be successful. Only then, long term, will we succeed as well. So I would say that's the second thing I say. The third is

• 24:33Speaker 01

  there's a lot of people problems in large organizations. You know, it's, there's now many people that are fighting over owning AI in the organization. Like I'm the person who owns AI at our company and so on. So figuring out a way so that you can get the strategy aligned in large organizations is, I know it's boring, but this is the hardest part of a large organization with so many different people running in different directions, having so many ideas of which vendor to use, what kind of models to use, what kind of people to hire, which project to do. So if you can concentrate sort of the decision making to one person in the organization and not have lots of departments have political fights over it, it streamlines and moves things faster.

• 25:17Speaker 01

  Typically, you actually have the head of that department being a change agent person. It's someone that doesn't do things that everyone will love. They'll come in and make the difficult decisions, streamline the data processing, the security around the privacy around it, because everybody's worried about that. And then focus on those use cases that actually matter for that organization. This is as important as a technology. I care a lot about technology and geek out on that, but this is just as important. Projects will get not funded, or they get stuck in security, or they go nowhere because different departments are fighting each other. So we need to overcome these things as well. This is why I'm saying it's gonna take a few years before we see the impact of these things.

• 25:54Logan Bartlett?

  As you think about the landscape and the term that you used earlier to sort of talk about was a compound?

• 26:00Speaker 01

  Yeah, compound.

• 26:01Logan Bartlett?

  Compound data and all that.

• 26:03Speaker 01

  Systems.

• 26:03Logan Bartlett?

  Yeah, compound agent systems. How do you think about what areas to potentially pursue? There's a bunch of venture companies in spaces like inference or evals or whatever it is. Like, as you think about different areas to potentially go into for Databricks as a business, how do you evaluate, no pun intended, I guess, those markets to potentially pursue?

• 26:26Speaker 01

  Yeah, it makes sense. Look, it's just for us, it makes a lot of sense since you already have all your data with Databricks. You already trust us that it's secure. You know that we can handle the security, the privacy. So the question is, what's really adjacent to that? And for me, the CEO, where is it we can actually bring extra value to the customer that they otherwise couldn't get elsewhere? Typically comes back to the data that they have with us already. So typically comes with governance around that data, with guardrails around that data. You know, you don't want to build, this actually happened with the customer. They had a chatbot, and if you asked it, what's the best car I could buy? And if this was a car manufacturer, it recommended a competitor's car, you know? So how do you put guardrails around that? You know, how do you deal with PII data? How do you deal with AI regulation that's coming? A lot of this, we can standardize in Databricks.

• 27:11Speaker 01

  So it makes sense. We can like automatically produce documentation. So you're compliant with the regulation that's coming. We can put guardrails so that it doesn't do those kind of things. And since we have the data, we can create this data loop. So that you get this flywheel of when they use the model, we create the evals and automatically feed it back to making this model smarter and smarter and doing that. It seems we can do that because they already have the data with us. So it's just advantageous for us to do that. I think in this space, like for startups, they should focus on the applications. I think applications are going to be where a lot of value in the AI space is going to accrue in the future. We'd love to be their infrastructure. So they use us for their data processing, for their governance, and for their AI.

• 27:53Speaker 01

  The applications, I mean, there's a lot of creativity. There's a lot of things that you're seeing happening. I can't predict what's going to be the apps that are actually going to win in the future. It's kind of like in 2000 trying to predict Facebook or Twitter. No one could have done that. No one ever would have said, oh, you know, there's going to be a company that does. You know, 140 character tweets, it's going to be the same thing now. So we just want to be that infrastructure for them so that they can build their AI systems on top of us. And we help them with that data flywheel.

• 28:19Logan Bartlett?

  If we back up a little bit to the original early days of Spark. So we talked about the Netflix competition and that. I had heard that maybe you tried to actually sell the technology in the early days and there weren't really any takers. Can you just go back to that early story? Like, what did you see? And what was your path to getting it off the ground into some commercial entity?

• 28:40Speaker 01

  Yeah, I mean, what was really happening is that we were seeing that the Silicon Valley tech companies were leveraging AI. This is 2009 or 10. We were seeing what Facebook was doing. We were saying what Google was doing. And it was very clear that doing AI on the data is going to be the most important thing. And at the time, everybody was stuck on doing basic analytics backwards looking. Call it BI, business intelligence, which is backwards looking. Like, how was my revenue last week? And they were excited that they could do that really, really fast. But we were asking, hey, what about, what's your revenue going to be next week? AI. And there was just not much appetite for that at the time. And we went around and we tried to push the technology out because we just wanted to do research. But it was very hard to actually get anyone to actually see this.

• 29:25Speaker 01

  You know, you might remember the phrase, it was the Hadoop wars, Hadoop days. It was impossible to get anyone's attention for that stuff. So it was kind of a little bit out of despair that we started the company and said, look, if we don't do it ourselves, no one is going to do it. I remember actually one of our first customers, we said, hey, this is open source. Just take it because we don't want to make money on this. And I said, no, you gotta charge us for it because we want an enterprise that's behind this. And that's kind of when we realized that, okay, there needs to be like an enterprise business. No one else is going to do that. We probably have to do it ourselves.

• 29:57Logan Bartlett?

  Was it an issue of just pure focus for these vendors? Or was there something from a data structure standpoint that needed to actually happen to have the market evolve to the point that they actually were able to do a lot of the things that you wanted them to do?

• 30:12Speaker 01

  I think there's these hype cycles that come and everybody's just focused on something. So that era was that Hadoop hype cycle and they were obsessed with it. And we couldn't for the life of us tell us AI is more interesting. It's just that, you know, you open the newspaper and everything is just about that. And it's happening now too, right? It's like, I think the market works like that. And to create a successful company, you have to kind of bet on something that's non-consensus at the time, and you have to be right about it. So for us, it was AI, it was the cloud, it was open source. Those were the things that we were excited about, but there wasn't much excitement around that. And actually, we were told that this is not going to work. All the data is going to be on premises, you know, open source, you cannot monetize it, and so on and so forth. So I think it's similar now, right?

• 30:56Logan Bartlett?

  Were you intellectually, I heard a story that you, that a customer was going to offer you, I don't know if it was seven figures or eight figures or something to build an on-prem version of the product, and you rejected it at a point in time that maybe you could have used this now, now less so. But at that point in time, you definitely could have used the seven or eight figures or whatever it was. Did you just see this as an inevitable path you were building towards? And therefore, anything but that was a distraction along the way? Or what was that philosophical decision?

• 31:23Speaker 01

  Yeah, 100%, like, our view was, we have to have an asymmetric bet on something that no one else believes in, and we better be right about it. That's the only way. So if we're going to go on-prem and do what everyone else is doing, then there are way bigger sharks that can do that than us. And then this is going to fizzle out, and it's not going to work. So yeah, it was a month or so in to my CEO job, 2016, that one of the largest banks told me, you need to bring this stuff on-premise. And I said, no, we're not going to do that. And they said, well, how much? I said, how much what? They said, how much would we pay you for this? I said, we're not going to do it. And they said, five million. I said, we're not going to do it. And they said, 10. I said, we're not going to do it. They said, 20. I'll give you 20 right now.

• 32:02Speaker 01

  And at the time, 2016, you know, I guess we ended the year with 10 million revenue that year. So, you know, 20 million revenue was a lot. But the idea was like, if you go on-prem, they're a much bigger company. So we have to have an asymmetric bet. That we believe in and we continue to double down on. And then, once that bet becomes true, we'll be big enough that it's too hard for others to catch up with us. Similar thing, as I said, is going to happen in this AI and LLM era right now. I think you have to pick something that you think is different than the rest of the market and then bet on it. And then hopefully you're right.

• 32:40Logan Bartlett?

  As you reflect on the journey, one of the stories that I had heard was giving a lot of credit to Ben Horowitz in actually turning this into a business. Can you tell that story? Typically, I don't allow other VC plugs, but I thought it was an interesting one that he actually was pushing you guys to think big and turn this into a commercial product.

• 33:00Speaker 01

  Yeah, I mean, look, we came out of the research lab. We really wanted to raise only two, three, $400,000 seed and just code away for a year or so. And he's the one that pushed hard, no less to the Series A, and I want to be an exclusive investor. And we were actually not very excited about it, but then he gave us a price we couldn't refuse. And he said, no, go big. And we gave us, you know, 14 million at the time, which is a lot of money back then. I know it's not much these days. And he said, go, go, go, go, go. So he really pushed us, go aggressive, disrupt the market, think big. You know, he always was a couple of orders of magnitude bigger sort of vision for Databricks than we ever had.

• 33:39Speaker 01

  Like whatever was our biggest grandiose vision of Databricks, he always was a couple of orders of magnitude ahead and was pushing aggressively, saying, no, no, no, this, you're thinking too small, like way too small. And I think he really truly believed it as well. So that kind of set the tone from the early days for us.

• 33:55Logan Bartlett?

  Because the early founders, I don't know if you remember these numbers, but you guys bet. You remember the actual numbers? So what were the bets for the early?

• 34:02Speaker 01

  We were around the table when we started the company and said, hey, what's like a great outcome for the company? And I think when we were around the table, the numbers were something along the lines of like 150 million exit for us or 200, and someone said 250 and so on. So that's how we were thinking. And Ben was immediately like, no, no, no, billions for sure we're going to do. And then when we got to that, you know, it was like, this is easily going to be a hundred billion dollar company. And you know, now where we are now, he's like, definitely a trillion is what you need to think about. And, you know, I would also give credit for Marc Andreessen in one of the meetings, I think, we, early days, we said, he said, what's your biggest bottleneck? I think this was 2017. So hey, we can't hire engineers. We can't get the engineers from Facebook and Google. They're just too expensive.

• 34:46Speaker 01

  And he said, huh, what's so what's the so Fang is the problem? And I said, yeah, Fang is the problem. And he said, OK, so you just need to add yourself to that list. It needs to be Fang DB. Ha, and that's it. And then he moved on. But after that meeting, we're like, yeah, OK, he kind of is right. We should actually just offer up and, you know, hire those people and become like one of those tech magnets. And it changed our mindset. Like, we went back. We changed our compensation. We changed how we hire, just based on that one comment.

• 35:11Logan Bartlett?

  I'm curious, like, at a tactical level, I mean, that's very aspirational. It sounds great. But but like in changing the compensation or thinking bigger, like, what did you then go do?

• 35:22Speaker 01

  We literally went back and moved the compensation for engineering to 90th percentile of the market. You know, we did the math and we saw how much is it going to cost us? Are we going to be able to do this? Is it going to be? Is it something that we can afford as a company? What's the dilution going to be? This is a long time ago. And actually, when we ran the calcs, we were surprised that actually we can do this and it's sustainable. And it was surprising to us. We're data companies. We crunch the data and all the numbers. What we found is that, you know, it's really your market cap divided by number of employees. That's how equity rich you are when you're paying someone equity. And these companies have a lot of employees. So then it turned out, actually, we are actually richer than we think if you take our market cap and divided by the number of employees. Because that's, you know, that's not how you're doing financially.

• 36:05Speaker 01

  That's not about your revenue growth, but that's about what you, you know, what could you offer someone in equity? And it turned out that actually it's absolutely within our reach to do it. So let's explore it a little bit. And it started working. And then we were able to hire the people that we couldn't hire before. And they came in and they completely revamped what we were doing with the product. We just could increase the feature velocity and the innovation cycle at Deterix massively. And that actually helped us kind of slowly start separating from the pack. Because in 2017, if you had talked to us, you know, we would have been one of the startups among hundreds of other ones out there. So these small things kind of that they did to push us think really, really big, go way bigger. And we did that, but we always kind of did it judiciously. We didn't want to kind of. Go crazy and you know, burn down the bank account. We wanted to be careful about it.

• 36:49Logan Bartlett?

  Databricks was founded in 2013.

• 36:52Speaker 01

  Yes.

• 36:52Logan Bartlett?

  That's right. And for the first couple of years, you weren't the CEO. What were you doing through that the first couple of years in the journey? And then what was the impetus for you to step in and become the CEO?

• 37:02Speaker 01

  Yeah, I was running product and engineering, which I loved. And actually, in that phase, it made a lot of sense because you know, what do you do the first two, three years of a company? When you start a startup, you want to get product market fit, build something that just works. So that was the focus for us. Just how do we get one customer that wants to use this? And then how do we get a customer that wants to repeatedly use it? And then how do we get a customer that repeatedly uses and pays us for it? And so that was the first two, three years of the focus. The technology that we had innovated with, with Spark exploded in 2015. And you know, at that point, we felt like we had succeeded, but we just didn't have any monetization. So we went through a difficult time in 15. We had to figure out, okay, if we can't monetize, maybe there is nothing here.

• 37:46Speaker 01

  And the company kind of went through a little bit of sort of introspection. And then we said, okay, let's kind of do a little bit of reset of some of the things and rethink some of the stuff and start anew in 2016. So I became CEO in 2016. And we made a bunch of changes that actually started helping us with monetization.

• 38:03Logan Bartlett?

  Initially, were you the interim CEO or were you the CEO and they were still talking to other potential outside candidates?

• 38:11Speaker 01

  Well, I thought I was. Well, first of all, I heard through the grapevine that they're interviewing people. So I found out, okay, they're actually interviewing other external folks. So it's

• 38:18Logan Bartlett?

  Were you the CEO at the time?

• 38:19Speaker 01

  No, but they were like, hey, do you want to be the CEO? I was like, sure. And then I found out, okay, they're actually interviewing other people. All right. They didn't tell me about that. But then eventually I was like, okay, I think I am CEO. They've announced it to the company. I am CEO. I'm running it. But then I kind of, you know, my compensation didn't change. And I learned kind of soon that, hey, I'm actually interim CEO. I didn't know that.

• 38:38Logan Bartlett?

  You didn't have the title until?

• 38:39Speaker 01

  Yeah. So then it's turned out, no, no, no. Actually, it was just a test period. And then eventually, I would say after one year, I kind of, I think they really kind of like, okay, no, we're gonna go with him.

• 38:50Logan Bartlett?

  So the first, when you came in in 2016, there were three things that you kind of focused on. And it sounds like monetization was one of them as well. But can you talk through what those three were? I think it was up-leveling the management team, sort of changing the focus of the customer type and then monetization?

• 39:06Speaker 01

  Yeah. Enterprise sales. Total pivot into enterprise B2B. We were PLG. I know it's still hot. Everybody wants to build PLG companies, but we learned the hard way after two, three years that PLG doesn't really work. Might work for very few companies. It's a great thing. We still do PLG. But if you, guess what? If you want to sell a data platform, we want to manage the most sensitive data of a large organization. You need to get the buy-in of that organization. Nobody's going to just swipe a credit card and start giving their data to you, and you're going to succeed. So PLG just won't work in that model. In B2B in general, it's hard. If you want to have big contracts with large enterprises, PLG probably don't work, does not work. So, so it's number one. Going all in on enterprise sales.

• 39:51Speaker 01

  We had tried earlier to just be this PLG, product-led growth. The second thing is up-leveling the management. So we up-levelled all the exec team. Like, every person on the exec team was new. So I basically got a new exec team that had much more experience. They knew what they were doing. And a little bit went more with experience over just say, we're just going to be smart PhDs that can figure this out on our own. We went instead with, you know, someone who's done sales for 20, 30 years, or someone who's done marketing at scale at Oracle, knows how marketing is done and so on. So that really helped also with the execution, because I would say we were really weak on go-to-market. We were great at product innovation, research. So those really helped. And then a change to how we actually innovate in open source.

• 40:35Speaker 01

  Just open sourcing everything and giving it away is not going to work. So we need proprietary secret sauce that we're not just giving away to the hyperscalers. So that was critical also in changing. And that's difficult to do. If you're a pure open-source company, you come from academia, it's, it's, it's difficult to get all your employees to be all in on that. But it was the only way to succeed. We wouldn't have been around if we hadn't done that pivot.

• 40:58Logan Bartlett?

  Did you have those views as you were running product and you sort of came in prepared to do that? Or was there some point of introspection that ultimately led to those conclusions?

• 41:07Speaker 01

  Yeah, I think 15, we were not doing super well. And in 15, it was sort of like, okay, we are trying PLG. It's clearly not working, right? Yeah, we don't want to do enterprise B2B because we're not excellent at it. We don't have experience in it. But, you know, we ended 2015 with 1.6 million revenue. 2015. So if we only have 1.5, 5.6 million revenue, from what we could glean from blogs and whatnot, an enterprise salesperson can sell a million dollars, a million and a half of software. That's just one AE. Why don't we hire three of them and try it out? What do we have to lose? So that kind of emerged in 2015. The other thing was it was very clear that no matter what we did in the innovation cycle side, the hyperscalers immediately had the same thing.

• 41:52Speaker 01

  They would just pick that. And in fact, they would add a little bit of stuff on top of it. And then they had the whole platform, the whole cloud. So they were always the good enough vendor. So the customers were like, well, you're great. You have great innovation, but I can also get it from the hyperscaler. I'm going to try that first because I already have a contract with these folks. I got security approvals. I've already paid them a bunch of money. Try that first. And if it's not good enough, I'll come back to you. So it became pretty clear that we have to be able to articulate then, yeah, you can go over there, but you won't get XYZ if you go over there. Like, we have XYZ that they don't have. And so if you couldn't articulate that, then there was nothing you could give your sellers to sell. And then, you know, it was the experience that we were lacking. So I think throughout the hardships of 2015 kind of made it pretty clear that we have to try something new.

• 42:37Speaker 01

  And these three directions seem to be pretty viable.

• 42:41Logan Bartlett?

  I mean, the one that stands out the most as like potentially a one-way door is the licensing or monetization shift. And it's very different than the culture, I would say, of the people that are very open source purists in some way. Was that, was that a debated decision in a very meaningful way? Or was it just kind of declarative? Hey, guys, we have to do this to be able to build a business.

• 43:04Speaker 01

  I think when I took over, I was just a little bit more forceful in what needs to be done. But it was big debate internally. And I made it pretty clear. I think I used the analogy and I said, hey, we have a gun and it has one bullet left in it. We're not going to do thousand things. We have like one bullet left. We better aim it really clearly and we better hit the target. If we don't, we're done here. And so I made it pretty clear that this is existential, the things that we're gonna do. And and I think when I kind of got people to wake up, because you know, in the startup days, you're excited, you're doing stuff and but if nobody makes it clear to you that hey, we're actually gonna go out of business in a couple of years. It's not all great, just because we have a big valuation. I think people got in line and and started thinking about, okay, what are the ways we can actually do this?

• 43:47Logan Bartlett?

  Did they quickly start showing up in the the business or did it take a few quarters?

• 43:50Speaker 01

  No, I I think pretty soon the engineer started thinking and I made it very clear to them is that this is great that you built this. It's literally available now also on the hyperscalers cloud. So what's the differentiation literally? And I made it very viscerally clear to them and I shared it. We had an extreme transparency culture to make it really clear. So get them to buy and let them understand themselves that we can't win. We're guaranteed gonna lose if we continue this way. And you know, they kind of soon got it that we want to be successful here. So they started coming up with innovations that we're gonna this, maybe we don't need to open source.

• 44:21Logan Bartlett?

  Now, as you think about expanding into new products, I've heard you actually will set up different teams for each product as it gets going. What have you learned about product management and organization as as you can need to scale and iterate on these different product areas?

• 44:34Speaker 01

  Yeah, I think that when you build a product zero to one, you kind of have to do the same thing that you did in the original kind of first two, three years of the company. So there's a lot of processes in any company once you're over 150 employees, let alone thousands. So you gotta cut that and say, look, for a new product, you don't need to satisfy all the security stuff. You don't need to have all the processes, have a small core team that iterates really, really quickly. Figure out, first of all, who's the sort of innovator type that you have in those early startups? Put them on the project. These are people that are not necessarily good at when the company scales and they're leading mature products. You have to have one such anchor person that you put everything around and you put them in charge, they start building that product. And then you let them iterate really quickly.

• 45:19Speaker 01

  So get them off the iteration cycle that the big company goes on, but get them on this much quicker, smaller one. And you know, encourage them just iterate inner dev loop, outer dev loop, that's super, super fast and just focus on getting users and getting product market fit. Once that happens, you can kind of move it back into the mothership. So ideally, even do it outside of the whole product if you can. So we have a product right now called Genie, which lets you just, you ask in English questions and it can answer immediately. It produces, it understands all the semantics of your data. We call it data intelligence. So it understands your data and it then programs and asks the questions and give you just straight up the answer. But to know that it's not hallucinating, you can actually see how it coded it up and behind the scenes and so on.

• 46:03Speaker 01

  That product, we actually just built it completely outside of Databricks codebase. And it didn't satisfy any of this stuff. We didn't build on anything. We just iterated quickly, just do that outside of the norm, like a startup almost. And then only when we saw that the results are really good and people like it, we went through the kind of painful process of, okay, let's rewrite it, let's move it into the mothership. So it's almost like an acquisition at that point happens. And you have to integrate it into the whole system and make it secure and so on.

• 46:30Logan Bartlett?

  So you'll give people autonomy to make the decisions on, not the shared primitives necessarily, or if you guys are using some type of, I don't know, single sign-on or whatever cloud structure underneath, you'll give people full autonomy to do it and then you'll come back and reconcile it once it actually finds product market fit.

• 46:45Speaker 01

  Literally do it outside in a separate repository, do it in a separate GitHub repository outside of the big system. And do it, iterate very quickly is the most important thing, because that's what you're doing as a startup. As a startup, you operate three to four times. I've seen it, we've done acquisitions. And if you measure it, the startups move three to four times faster than a company like Databricks. And I think we move very fast and we're very innovative, but still it takes us three, four times longer. Why is it? It's the security processes, it's the legal sign-off, it's making sure that it works on all the clouds in all of the environments. And the biggest thing that slows you down is that you have thousands of thousands of customers that you can't regress. So you gotta unshackle the new products. Otherwise, you're just gonna move too slowly.

• 47:27Speaker 01

  And that's what we're seeing. Honestly, the really, really big companies, they operate that way. It takes them too long. That's why you have startups in the ecosystem. So you have to rekind of go back and reproduce the kind of environment that you had when you had a small startup.

• 47:40Logan Bartlett?

  I'm curious, so now you're coming up on, I guess, nine years-ish at the CEO job, is that right? As you reflect on like your leadership style, today you sort of seem like the canonical leader of Databricks and it's like the job that you've always done. I assume leadership style has kind of evolved in some ways over the course of this period of time. Are there anything that you wish you could, anything you could wish you could go back and tell your earlier self, taking the CEO job for the first time, or maybe your earliest founder self when you're sort of stepping into the seat?

• 48:09Speaker 01

  Yeah, I mean, I think that you would want to grow leaders earlier on, like build trust with few leaders that you groom and continue doing that. Because what you find nine, 10 years later, and I have a lot of awesome co-founders that are still active, and they run the company where it's almost a partnership where we run it together. But it's not that many people, like, you know, even though we're almost 8,000 employees, how many people are there that are really, can really help you build the business together? It's hard to find those leaders that build you trust with them and, you know, grow the company with. So I would just invest more in that, much more actively, and always be lookout, on the lookout for those kind of people in the company. You know, we were just busy executing and we didn't pay attention too much to that, maybe. So that's

• 48:50Speaker 01

  one thing I would do differently. The second thing I would do differently is when these trade-off decisions come up, and it's really hard to decide, do we do, do we go on-prem or not, or whatever it is, bet, make a bet that, you know, is much more for the future. Don't worry so much about, you know, the implications that it has on our finances or on the revenue and don't, like, don't let short-termism creep in. Easy to say now, but in the early days when you're, it's far from clear that you're going to be successful as a company. And, you know, the VCs are asking you, how's it going? You have your board meetings, they're looking for your revenue targets, they are super obsessed, think, how's that? Is it hockey-sticking? Is it going up?

• 49:32Speaker 01

  What's it looking at? How do you compare with others in the industry? Continue making the bets on the future and don't get carried away on those numbers. That's another advice I would give, because we took many shortcuts that backfired. They were not worth doing. We kind of knew it, that this was a shortcut, but it was like, it was worth it, because, you know, this is going to bring so much revenue if we just take this shortcut. I would have avoided many of those shortcuts now. You know, err on the side of the future than the present. Now, of course, maybe it's path dependent and I'm saying it just because we're where we are. If we had done the things that I'm saying we should do, we wouldn't have succeeded. But I still kind of, on balance, feel like we could have been more futuristic in our bets than we were.

• 50:13Logan Bartlett?

  I get the feeling you focus on empowering individual leaders within your organization and having them deputized to make the best decisions for the organization rather than you serving as a central hub for each department. Is that a fair character?

• 50:30Speaker 01

  I think it's both. I think I also micromanage them.

• 50:32Logan Bartlett?

  Okay, maybe they would disagree.

• 50:34Speaker 01

  Yeah, they would probably not say.

• 50:35Logan Bartlett?

  How do you think about the tension between, you know, a central figure that's able to make definitive decisions that sort of sit across an organization versus more decentralization and empowerment at an individual level?

• 50:48Speaker 01

  Yeah, I mean, you get more clarity if it's one person and it's more consistent. And, you know, that's great. The disadvantage is that it slows things down and it bottlenecked things. And it's maybe less empowering. So you have to strike a balance and you have to do it in places. Like, you know, you build trust over time and you figure out which are the parts which you can completely delegate and they're going to do an excellent job. And, you know, they're probably better than you at it. So don't pretend that you're better than them. And what are the areas where you really need to deep dive? And then there are some areas that there's things that only you can do as a CEO. Like, you know, it's literally tie breaking. It might sound silly, but over time overlapping product areas appear. And you get leaders that are kind of building similar-ish things and the Venn diagram intersects.

• 51:33Speaker 01

  And if you leave it to them, no matter how many times you tell them, you know, figure out how to unify this, do one, we're not going to have two similar offerings. They're not going to be able to do it on their own. So you've got to step in and push so that you've got to use the CEO card judiciously. I think that matters a lot and you can make, build a much simpler product. So I think if you want really simple unified product that makes sense, one person makes much, much more sense. Just have to watch out for bottlenecking things and slowing things down.

• 52:00Logan Bartlett?

  Why is a willingness to say the same thing over and over again an important part of the CEO's job in your mind?

• 52:05Speaker 01

  Because I think people are not paying attention to what you're saying. Like, you know, we all have busy lives. We're paying attention to our own life. You know, I didn't even notice this thing that you said. I wasn't even there. Or maybe you said it, I didn't really rock it. That was on my phone and I had actually fired that day. And I was having other issues that I was dealing with. So I didn't really, I didn't really internalize it. I didn't really get it. So many, many times you tell someone something for a third time. And then only then you notice that they are like, you know, and then they repeat it back to you as if it's like they came up with it. That's great. I think it's just you gotta repeat it. But what I tried to do is figure out a way to repeat it in different ways. Don't repeat it the same way all the time. It gets boring and old. Challenge yourself and come up with a new angle. Find a customer that says the same thing that you want to say. Or use case that's a little bit different. Use different slides, make new slides.

• 52:48Speaker 01

  And you'll be surprised because, you know, people didn't really get it the first time. They'll say, oh, this was completely awesome what you just said now. Even though you're like, well, that's literally what I said like a month ago, but I just refurbished it a little bit. So I think it's super important.

• 52:59Logan Bartlett?

  Being truth seeking, I think is one of your values. Is that right? How do you implement that in practice? Or how does that be manifest itself as a cultural, cultural practice for you guys?

• 53:09Speaker 01

  Make sure in the hiring process that you test folks on truth seekingness. You know, it's not that hard, actually, to ask someone, you know, tell me, what's your biggest screw up? You know, tell me where you really made a mistake. Who did you really not hire well? Why did you make that mistake? How was it and so on? Push them, push them, push them till you see, you know, until they get uncomfortable. Do they start to come up with lots of excuses? Well, I really actually didn't want to hire this person, but I was, my boss made me hire him. And that was a bad hire. I knew it, but yeah, that was a mistake I made, but technically my boss's mistake. So like, you know, keep pushing them and you can, in the interview process, actually suss out if someone is truth seeking and honest. Or whether they're trying to a little bit, you know, make it look much nicer. You know, so that's, that's number one. Number two, don't promote the people in the company that are not truth seeking.

• 53:53Speaker 01

  Like at least if you made the mistake of hiring them in, then at least make sure you don't promote them. So in the promotions of the company, make sure that you're really evaluating the cultural values. Like really do it. It's easy to have the cultural values, but are you evaluating the promotion? I have a rock star, rock star is doing great. They moved the needle on the business. Should they get the promotion or not? You don't want to be the person saying, well, wait a minute. Can we like, look at if they were truth seeking? Oh, they're not. Let's not promote them. It's like, are you out of your mind? This person had a huge impact on the company. Yeah, but they're not truth seeking. But if you mean it, if you want that to be the culture, you have to actually do that in the promotion. And then, of course, if it's an extreme case, we might have to part ways. You have to do that as well. You can't have people around that don't fit the culture. So that's what I would say.

• 54:35Logan Bartlett?

  You spend thirty, forty percent of your time on hiring, is that right?

• 54:38Speaker 01

  Yep.

• 54:38Logan Bartlett?

  One of the things you said is that you, when you're hiring executives, you'll just find reasons to keep in touch and find a bunch of different touchpoints. Can you maybe talk through that as a philosophy or how you go about doing it?

• 54:50Speaker 01

  Yeah, I think actually the bigger philosophy is that I don't believe in interviews. Because I think some people interview well, and they might not be good at all. And some people interview really poorly, but they might be excellent. So I actually just think it's a very noisy channel. You can't, there's not that much value in it. So then, what can you do? One of the things, this is the one that you mentioned, but there are other ones that we use as well, is I try to actually have them do the job. And I think, we think, oh, you know, in programming, we can do that. Ask them to program this thing, and then if they do it well. But we can't do it for any other job. And I disagree with that. Just, if you're hiring someone for head of marketing, work with them to fix something that's broken in marketing right now, whatever it is. If it's a personnel problem, or if it's the webpage, or if it's the product marketing positioning, work with them. Call them, call

• 55:35Speaker 01

  them, you know, in the evening, call them, you know, in the morning, ask them, go through it, like, work with them. Basically, make them do the job, as if they were already an employee in the company. And you will have a very good sense of culturally how they are, you know, are you on the same wavelength? Can they actually do a good job at this? The kind of stuff, especially, pick the things that are not working right now in your company, because that's, those are the things you would give them to fix when they come in. And if they could fix those things, they're clearly a good hire. So I'd rather do that than coming in an interview and just asking them, you know, the, you know, the gotcha questions that they might have practiced, you know. Tell me your own weakness. What's your biggest weakness? And so on. So that's, that's one of the big things that we do. The other is backdoors. And I believe you should do the backdoors upfront.

• 56:19Speaker 01

  So don't go through a lot of firms, especially recruiting firms, they will do, run you through a cycle, they have you meet all these people and then at the end, they want to just check the box. By the way, they'll even do the references for you, which is a big mistake. I think you should use the references upfront. I already know who you're hiring. So that the first time I meet a person, I already know a lot about them. I've already asked. And I've also found this thing that if you do five to 10 backdoors on people, their bosses, previous bosses, you actually kind of know the person better in some regards, not in their full entirety, better than they know themselves. Cause you know, all the things they went through and you know, the perspective with three, four people that saw that happen to them in the previous job, they might not actually fully comprehend what everybody thought about that situation.

• 57:01Speaker 01

  Of course, they have a unique perspective. So I think the backdoors are extremely important. You know, and then others, just look at their trajectory of work, you know, where were they? Have they seen great? Have they done something stellar? Are they hopping around a lot? You know, those are the three things that we kind of emphasize in hiring.

• 57:16Logan Bartlett?

  What are your techniques on the backdoor references? Are there, are there specific questions you like to ask or ways that no one likes to talk poorly about someone they know? So how do you sort of drill in to make sure you're getting the actual, high quality information?

• 57:29Speaker 01

  Yeah, it's really dependent on who you're talking to and they're all different. And you can tell immediately, some people are not just going to tell you the truth. This is going to be, you know, they're going to just say good things or they don't want to say anything. So, you know, that's that. So you just move on and end the call. But, and I guess I'm revealing my, you know, recipe here. So it will never work again. But I think that if you can find the form of person that, you know, opens up, I think one thing that I like to grill them on is really, really ranking of how good the person is. And don't ask, you know, score from one to 10 and they say 10. Really ask them like, does it rank number one, number two? How many have you? And then what to go really deep, who did you, who's number one, who's number two in your list? By the way, you go after those as well, of course. And then really ask them, you know, go through scenarios, like it's just the follow-up questions. Like what, what did they do well? Tell me something that went wrong.

• 58:15Speaker 01

  Give me a scenario. What was that scenario? Really push the boundary. You got to kind of almost to the point that they become uncomfortable. So many times I've done it so far that, you know, they call back and say like, you know, this is, you know, what's going on here?

• 58:27Speaker 01

  But yeah, you want to, but you'll find when you're doing the backdoors that, okay, that guy and that gal, they're like super forthcoming. That's a straight shooter. They're going to tell you the whole thing. And others are not. Front door references, the references that the candidate gives you are similar, but kind of inverted. They're, you know, the vast majority are actually people that the person is going to hire next. If it's a manager, they're just literally giving you their, and that's useful too, because you can interview those people, you know, who they're going to hire next. So you can gauge the quality of the people that they're going to hire. But occasionally you find the front door that also tells you the real truth and tells you like really what's going on. And you learn a lot there too. And when I do these references, I'm not looking for, hey, what's your one fault that I'm going to cancel, you know, the, I'm not going to hire you for. It's just to get a complete picture.

• 59:10Speaker 01

  You're going to have faults, you're going to have strengths. We want to understand that full picture before we go ahead and hire you. So that's kind of what we're going for. But yeah, for some execs, I've done 15 of these backdoors before I hired them.

• 59:21Logan Bartlett?

  From a, as a leader, you've had to ramp up in different functional areas, one of which was enterprise sales. I don't think you had experience to it. How did you go about learning in that area? And I think you copied maybe best practices or sort of internalized, stitched together to build your own model from a bunch of different organizations. Maybe you can talk through learning a new area or discipline and how you went about it?

• 59:42Speaker 01

  Yeah, but first of all, I would just say, you know, we hired Ron Gobisco, who, you know, took us from essentially zero to now over $2 billion ARR, right? Basically $3 billion ARR. And so he's been phenomenal.

• 59:54Logan Bartlett?

  And what did you see in him?

• 59:56Speaker 01

  Yeah. I thought that he was extremely smart. So I thought he's actually different from many of the salespeople. He is the classical enterprise B2B sales guy, you know, so he's not like the engineer type that, you know, he is the classic archetype of a sales leader, you know, with all the rah rah and motivational speech and get everybody riled up. So it's like, he has all that. So that's great. That's the stuff we didn't understand or have. He had the big enterprise experience because there's all sellers that sell to be, to mid-market SMB, but he had actually sold to really big gigantic enterprises. So that's also something we didn't understand anything about. Like he was thinking very big. Like when he came in, you know, it's like, oh, we're going to do a $10 million contract or $100 million. We're like, this, no one's going to ever pay

• 1:00:39Speaker 01

  $10 million for data mix. It was our mindset. So that was also useful. But, but he also was,

• 1:00:47Speaker 01

  you know, he was cerebral and he was extremely smart and went by first principles. So there was a compatibility. He had an engineering degree. So he was sort of the unicorn of kind of someone who like spoke our language and couldn't understand us and was extremely smart, but also had this whole other thing that we couldn't understand. So that hire was important. We took us a year to hire. We went through a lot of candidates and I think we kind of nailed that one. It was luck. So that's important. The other thing I would say is in the interview process, when you're searching, like we took us a year to hire a salesperson. Take the, I've seen people make this mistake. Take the exec hiring process super seriously. It's a learning process. You're basically

• 1:01:29Speaker 01

  going to school to learn what a head of sales looked like. So in that, talk to all the top head of sales and grill them and pay attention and use them also in the interview, since I don't believe in interviews, as I said, use the interviews a little bit of as a tutorial for yourself to learn how does excellent sales look like in your organization, for your company, for your company, for your company. And you get the, after, you know, talking to 20 people, you get a pretty good picture of, okay, this is kind of it. This is what they're doing. They all have that. All the stellar ones seem to have this in common. This is what it's about and you can go deeper. And then in the backdoor process, you figure out who's who, you figure out who are the top people in this industry that you can't hire. Try to get some time with them as well to just learn from them and build a relationship

• 1:02:11Speaker 01

  with them or get them to be advisors. So really put yourself in the, hey, you're going to university to learn. That's, that's what that process is. You know, you're not an expert and trust the people that are the best in the industry at that. Try to find the number one person and go with them.

• 1:02:26Logan Bartlett?

  In hiring for Databricks, are there certain things that you feel you uniquely prioritize? Or the types of people that succeed within the organization that are, is maybe unique to Databricks?

• 1:02:39Speaker 01

  Yeah, well, so our culture principles have been reverse engineered to exactly answer your question.

• 1:02:43Logan Bartlett?

  So you didn't start with the principles and went the other way? You started with the people that succeeded and came back?

• 1:02:47Speaker 01

  Succeeded and failed.

• 1:02:48Logan Bartlett?

  Okay.

• 1:02:48Speaker 01

  We also looked at people that had perfect CVs that we hired and they bombed out. Why was that? And we went back and said, okay, we couldn't agree on the hiring bar. They were like, I've worked with these 10 people. They're stellar. And we're like, no, that doesn't look like that our bar. That was a big issue. So raise the bar. Don't settle. We'd rather have more, you know, we'd rather have more false negatives than have false positives. So we'd rather reject someone that's really great by mistake than hire someone by mistake that's not great. And that's painful. We learned that that way. Truth seeking, a lot of execs from the big companies had this attitude of, you know, everything's going great. I got this. My team's got that. We're working on this. Here's a nice slide deck. Manage up.

• 1:03:33Speaker 01

  Nothing to look for here. But that didn't work for us. It's like, that's what where the truth seeking principle came in. We really need to know, like, what's broken? Who did you, who in your team is not working out well? What's not, what are we not doing well? Where are we not satisfying the customer? So that was the second principle. You know, third principle, I've done this before. I know how it's done. This is how a much bigger, better company is doing it. We should do it this way. So that was that first principle. We needed to know exactly how and why and go from first principles on every decision. So, so being first principle became another culture principle. So our culture principles are the recipe for succeeding at Databricks. Like execs ask me all the time at the end, you know, before just we shake hands and I hire them. They say, hey, so you think I'm going to be successful here? What do you think is going to be needed? And it's like, it's super simple.

• 1:04:17Speaker 01

  I swear to God, read the culture principles. If you really do those, you will be, I mean, of course, the results have to, the output has to be great, but you will work out here. If any of this stuff doesn't look good to you, let's argue about it now because that's, that's going to be an issue. Those are really the principles of the company. So our culture principles aren't like the aspirational, cool stuff that you'd like to have on the wall. It's literally what worked and didn't work in the past empirically.

• 1:04:40Logan Bartlett?

  When, when looking for people within or to come in Databricks, I think people that have a chip on their shoulder a little bit. Do you, is that something you can? Do you try to tease out in the conversations? What are the types of things that you'll ask to try to figure that out?

• 1:04:55Speaker 01

  Yeah, I mean, the people that, you know, really, really, really went against all odds, really pushed, they have grit. One of the things that Ron said in the sales, you know, interview that kind of I remember now from 2015, I think, November was, he said, hey, if there's a deal lost, you know, I want to know like, why, because there's no way anyone else on the planet could have won that deal. Like, I just know it's humanly impossible because I will have done everything humanly possible to win that deal. So I just want to, and it was a side comment he was doing with something. And it was like clear to him that this, this guy is going to go over and beyond. And then, of course, the negotiation with him to hire him was a real pain.

• 1:05:37Speaker 01

  So that was also reaffirming that, okay, he's probably going to be great at this job. You don't want to have sales that's, you know, terrible at negotiating their own comp for themselves. So, you know, you want to suss out, you know, is there something, something that the people who have something to prove, are they trying to prove to you? If they're super content and happy with things, you know, then it's probably not going to work. It's probably for people that are a little bit broken, like all of us are, like we're all, we have something to prove and we're going to work our ass off, we're going to do everything in our power. And you can suss that out in interviews. It's not hard.

• 1:06:12Logan Bartlett?

  You mentioned work your ass off there. That's something that I think has went out of vogue, working hard for a while. It's sort of come back in vogue. I'm curious, you had said something along the lines I had here. You've never met someone successful who didn't work hard. Can you elaborate on that point?

• 1:06:26Speaker 01

  In research, in academia, like genius professors, you know, who had won amazing awards, you know, and that's like, you know, you can be really, really smart in research and maybe you don't need to work hard, but all of them that I met, they were, you know, working harder than us. And we're like shocked, they're there 24/7. And then in industry, the same thing. So I just never, ever met someone who's just phoning it in and they're super successful. You find talented people, but they're not successful. It has to go with extremely hard work. So we had that from day one at Databricks and we never deviated, whether it was in vogue or not. We were a super hardworking culture from day one. We continue to be. And we said, look, we have lofty goals. To achieve them, we're going to work extremely hard. Totally understandable if you don't want to do it because maybe you have family, maybe other things you want to do in life.

• 1:07:09Speaker 01

  That's totally understandable, but then probably Databricks is not for you. So it's been always, always true here at there. In fact, we had, you know, our work-life balance scores weren't great in surveys. And, you know, I try to be upfront. We don't want to actually have terrible work-life balance. We actually focus on teams that had terrible scores and try to, you know, change things. And we got actually improvements in those teams. But I also was upfront, I don't want to be best in class and have the highest work-life balance score in the industry. That's not, we're not going to win that way. So it's as simple as that. Now, I try to myself also match that. So I don't want anyone to work harder than me. Like, so I gotta kind of match that myself. But, and it has to be sustainable. What I mean by that is I've been doing this for a decade more, and I want it to be the

• 1:07:54Speaker 01

  same way for the employees. So if they're like really burning themselves out and they can't do it for any more years, that's not, I don't want them to do that. I want them to be able to do it and it's sustainable. And then hopefully we can achieve great things together. And if they don't, that's okay, too. No hard feelings. You know, or maybe in a period of their life, they want to do it, but they don't want to do it another, that's okay, too.

• 1:08:13Logan Bartlett?

  One of the hardest things our companies deal with in scaling is when the, the people that got you there, they're starting to question if they're the ones that can get you there to the next step along the journey. It sounds like when you came in, you, you made that decision as CEO to up-level a lot of leadership. And I'm sure along the way, you've, you've kept Ron and he's continued to scale in his role, but I'm sure there's other functional areas that you've decided to upgrade. How have you gotten about making the decision that, hey, it might be time to part with someone who, who's done a great job up to that point, but you might need someone new to take you to the next level?

• 1:08:45Speaker 01

  I think you have to just be truth-seeking with yourself. So number one is like, when do you know yourself that, you know? I think we kind of know there's the signals, but we just don't want to see the truth. We want to ignore it because it's, it's so hard because this person helped build the company with you. It's, so you don't want to kind of admit it to yourself. But deep inside, you know, deep, deep, deep, deep inside. So just immediately when that is there, just recognize it and face the truth. Be truth-seeking with yourself. Don't lie to yourself at least as an exec. Okay, once you know it yourself, now you have to take action. I found it easiest if you can explain to the person and get them on board that it's not really working. Don't make it about like, just make it like, is it working or not? Is, are you winning or not here? Are you winning or not? Just focus on that. And if they realize that, hey, okay, I am not winning, well, let's discuss why. Then I think everything else becomes much simpler.

• 1:09:29Speaker 01

  It's like, okay, but then what do we do about, what about you then? So after, you know, like, okay, I'm not winning. It's not quite working. There are many reasons. Okay, let's figure out a way to make this work. Then you can change, steer the conversation towards, okay, what's the best thing for you, your career? I want to help you. I want you to be successful. I have found that there are new roles for people where they can continue to crush it. I have had people that, you know, they, in that role, it wasn't working anymore. And we found a new person that leads it or, you know, you do the layering or whatever, but you find a new role for this person and they crush it. And they're awesome at it and they'll be happier, actually. You know, after a while goes on, they'll be like, A, Ali is no longer bugging me, you know, about it not working. Second, I'm actually happier when I'm succeeding.

• 1:10:11Speaker 01

  When I'm winning, I'm happier than when I'm constantly, it's not working and everything is going, you know, I'm having a bad day every day. So I've actually found that many of those folks, you can keep them in the company. But you should also part ways if they're like just phoning it in and they're not interested anymore. And they're just there for some memory of what happened in the early days when everything was great. It's, you're better off also parting ways too. Let's not waste their time and our time.

• 1:10:34Logan Bartlett?

  Maybe zooming back out as we, as, as we wrap, as if someone's listening to this in a couple of years' time and we're sort of looking forward with your crystal ball that you claim not to have, but I think you're pretty good on it. Are there, are there elements of the world around Databricks or what you guys are operating that you, you think are on the cusp of being possible in the next five years? Just things that get you really excited to get out of bed every day and do this job? And just the potential, maybe at an industry level, not even at a company level, that you think we're on the cusp of seeing?

• 1:11:09Speaker 01

  Yeah, I think what's going to happen is that we're going to see many of the things that we see today are going to be disrupted. And there's going to be a sort of AI app for those things in the future. We don't even know what they're going to look like, you know? But what do they all need at their core? Like, what did apps need in the last 10, 20, 30, 40 years? They needed an Oracle database under the hood. You know, the joke is like you put all the complex, you stuff all the complexity into a database and then you build workflows and UIs on top or the two-tiered or three-tiered model. I think in the future, you need the AI database under the hood. And if you have an AI database, you have all your transactional data. You have all your analytics and AI in there with the model. And then you can do wonders on top.

• 1:11:50Speaker 01

  So I think we're going to see an ecosystem of these applications in the AI space being built, and they all need an AI database under the hood. And that's what basically Databricks is. Databricks is basically AI database. We don't call it that. We don't market it that way, but essentially it's an AI database under the hood. And you will need that for every application. Every company in the planet will need that. And, you know, I hope that we can be the company that powers all of these applications in the future. There will be others also that do that as well. But that's the goal. I think, you know, we're seeing already the seeds of this happening.

• 1:12:22Logan Bartlett?

  The seeds of it, I guess I'm curious just to elaborate on that. Like, the seeds of this happening, are you seeing just particularly exciting use cases, or just customers?

• 1:12:30Speaker 01

  Look, there's a lot. Look, in the SIM space, they need to do massive log processing securely and detect zero day attacks. Okay, that's like kind of what Databricks bread and butter is. Real time processing of massive logs with AI built in. In the observability space, same thing. You need to observe, you know, your log processing, get, you know, understand the metrics, detect anomalies, and so on. In the customer 360 space, you know, CDPs and, you know, understanding what your customers are doing, same thing again there. It's like, you know, massive data that you have to collect from many, many different places and then get telemetry on it. It's huge data processing. You need to do it in real time. You need to do AI to get the insights and doing those. So each one of these spaces,

• 1:13:11Speaker 01

  I think, are going to be built on basically AI database. I hope we can be the AI database that powers all of them. This disruption, I think, is going to happen in the next two, three, four years.

• 1:13:20Logan Bartlett?

  It's exciting. Well, thank you for doing this.

• 1:13:22Speaker 01

  Likewise.

• 1:13:26Logan Bartlett?

  Thank you for joining this episode of the Logan Bartlett Show with co-founder and CEO of Databricks, Ali Godsey. If you enjoyed this conversation, we'd appreciate it if you shared to anyone else that you think might find it interesting, as well as subscribe to us on whatever podcast platform you're listening to us on. We look forward to seeing you back here next week with another great guest on the Logan Bartlett Show. Have a good weekend, everyone.