Pod Insights a16z Podcast Data Alone Is Not Enough: The Evolution of Data Architectures About this episode Just having data is not enough: it takes an entire system of tools and technology to extract value from data. a hallway style conversation between Ali Ghodsi, CEO and Founder of Databricks, and a16z general partner Martin Casado explore the evolution of data architectures.
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Speaker 00 Hi and welcome to the A16Z Podcast. I'm Doss. Data, data, data. It's long been a buzzword in the industry, whether big data, streaming data, data analytics, data science, even AI and machine learning. But data alone is not enough. It takes an entire system of tools and technology to extract value from data. A multi-billion dollar industry has emerged around these data tools and technologies, and with so much excitement and innovation in the space, it raises the question, how exactly do all these tools fit together? This podcast, featuring Ali Gotze, the CEO and founder of Databricks, explores the evolution of data architectures, including some quick history, where they're going, and a surprising use case for streaming data,
0:45 Speaker 00 as well as Ali's take on how he'd architect the picks and shovels that handle data end to end today. Joining Ali in this hallway-style jam is A16Z general partner Martine Casado, who, with other A16Z enterprise partners, just published a series of blueprints for the modern data stack. You can find that, as well as other pieces on building AI businesses, the empty promise of data moats, and more at a16z.com/meconomics. In this conversation, we start with Ali answering the question, how did we arrive at the set of data tools we have today?
1:21 Ali Ghodsi? It kind of started in the 80s. Business leaders were kind of flying blind, not knowing how the business were doing, waiting for finance to close the books. And this data warehousing paradigm came about, where they said, "Look, we have all this data in these operational data systems. Why don't we just get all that data and we'll take it out of all these systems, transform it into a central place, let's call it the data warehouse, and then we can get business intelligence on that data." And it was just a major transformation, because now you could have dashboards, you could know how your product is selling by region, by SKU, by geography. And that itself has created at least a $20 billion market that has been around for quite a few decades now. But about 10 years ago, this technology started seeing some challenges.
2:06 Ali Ghodsi? One, more and more data types like video and audio started coming about, and there's no way you can store any of that in data warehouses. Second, there were on-prem big boxes that you had to buy, and they coupled storage and compute. It became really expensive to scale them up and down. And the third thing was people wanted to do more and more machine learning and AI on these data sets. They saw that we can ask future-looking questions. Which of my customers is going to churn? Which of my products are going to sell? Which campaigns should I be offering to who? So then, the data lake came about 10 years ago. And the idea was, here's really cheap storage. Dump all your data here, and you can get all those insights. And it turns out, just dumping all your data in a central location,
2:51 Ali Ghodsi? it's hard to make sense out of that data that's sitting there. And as a result, what people are doing now is they're taking subsets of that data, moving it into classic data warehouses in the cloud. So we've ended up with an architectural mess that's inferior to what we had in the 80s, where we have data in two places, in the data lake and in the data warehouse, where the staleness and the recency is not great. In the last 2-3 years, there's some really interesting technological breakthroughs that actually now are enabling a new kind of design pattern. We refer to it as the lake house. And the idea is, what if you could actually be able to do BI directly on your data lake? And what if you could do your reporting directly on your data lake, and you could do your data science and your machine learning straight up on the data lake?
3:35 Martin Casado? I would love to tease a part of few things that have led us here. You know, this is very clearly a large existing data warehouse market around BI and analytics. And, you know, it's typified by people using SQL on structured data. It seems like the ML AI use case is a little bit different than the analytics use case, right? The analytics use case, it's normally human beings that are looking at dashboards and making decisions, where the ML AI use case, you're creating these models, and those models are actually put into production, and they're part of the product. They're doing pricing, they're doing fraud detection, they're doing underwriting, et cetera. The analytics market is an existing buying behavior in existing customer, and ML AI is an emerging market. And so, the core question is, is are we actually seeing the emergence of multiple markets?
4:20 Martin Casado? Or is this one market?
4:21 Ali Ghodsi? Well, there are big similarities and there are big differences. And let's start with the similarities. Roughly the same data. is needed for both. There's no doubt that when it comes to AI and machine learning, a lot of the secret sauce, you get those really great results or predictions, comes with augmenting your data with additional metadata that you have. In some sense, you have the same data and you're asking analytical questions. The only difference is one is backwards looking, one is future looking. But other than that, a lot of it is the same. And you want to do the same kind of things with the data. You want to sort of prepare it, want to have it so that you can make sense of it. If you have structural problems with your data, that actually causes also problems for machine learning, actually. The difference is today is that it's lighter business that's typically is doing AI
5:04 Ali Ghodsi? and data science or hardcore R and D. Whereas data warehousing and BI oftentimes sits in IT. Users of the data warehouse and the BI tools are data analysts, business analysts. In the case of machine learning, we have data scientists, we have machine learning engineers, we have machine learning scientists. The personas are different and it sits in a different place in the organization. And those people have different backgrounds and they have different requirements on the products they're using today.
5:28 Martin Casado? If you talk to some folks that come from the traditional analyst side, they'll say, you know, AI and ML is cool. But if you really look at what they're doing, they're just doing simple regressions. Why don't we just use the traditional model of data warehouses with SQL? And then we'll just extend SQL just to do basic regressions? And we have 99% of the use cases.
5:50 Ali Ghodsi? Yeah, that's interesting that you ask because we actually tried that at UC Berkeley. There was a research project and that research project tried to basically look at, is there a way we can take an existing relational model and augment it with machine learning? And after five years, what they realized is that it's actually really hard to bolt on on top of these systems, machine learning and data science. And the reason is a little bit technical. It just has to do with the fact that these are iterative recursive algorithms that continue improving the statistical measure until it reaches a certain threshold and they stop. That's hard to implement on top of data warehousing. So if you look at the papers that were published out of that project, the conclusion was we have to really hack it hard and it's not going to be pretty.
6:33 Ali Ghodsi? If you're thinking of the relational COD model as it was with SQL on top of it, it's not sufficient for doing things like deep learning and so on.
6:41 Martin Casado? Is the same statement true about going from something architected for AI and ML and then having it support more of a traditional analyst relational model?
6:49 Ali Ghodsi? So interestingly, I think the answer is no. Because there is now widespread data science API. What has emerged as lingua franca for the data scientist, data frames. A data frame is essentially is a way where you can take your data and you can turn it into tables and you can start doing queries on it. And that sounds a lot like SQL, but it's not. Because it's actually built with programming language support. So you can do that in programming languages like Python or R, which enables you to do data science. So now your data is in sort of tables. And that's great. It turns out you can now also build SQL on top of data frames. So you can get sort of a marriage between the world of data science and machine learning
7:32 Ali Ghodsi? and the world of BI and data analytics using data frames.
7:37 Martin Casado? I get what you're saying about the data warehouse. But there's a lot more than just the data sitting in the data warehouse. And I'm just trying to grok, like you still have this entire world of data and SQL and ETL. Is there a dissonance there or do they stay two worlds or like what happens?
7:53 Ali Ghodsi? Well, so every enterprise we talk to, they have the majority of their data in the data lake today. And then subsets of it goes into the data warehouse. So there's like two-step ETL that they do. The first ETL step is getting into the data lake. And then there's a second ETL step that they use. So organizations are definitely paying a hefty price for this architectural redundancy. But the question is, do you really need two copies of it? And do you really have to maintain those two copies and keep them in sync? Are you going to have a world in which you have all of your data in the data lake? And then you do your machine learning and data science on it? And then subsets of it moves into a data warehouse again? And you clean it up and you put it in that structured form? So you can do SQL and BI? Or can we do it all in one place?
8:33 Martin Casado? Okay, so let's actually ask that specific question because. Even though the AI ML is a large market with a lot of value, there's a ton of existing workflow around BI. So you've got all the dashboarding and all the tools that are based on SQL for data warehouse. But then you also have folks that want to interact with the data very quickly, which is they'll use something like a ClickHouse or a Druid in order to do that in OLAP. OLAP stands for online analytical processing. It's effectively a fast database that supports fast queries. And then you've got more traditional batch processing, which normally folks have thought about Spark. What you're saying is, is that you think that you can combine all of these things
9:15 Martin Casado? in the same data lake, including OLAP query loads?
9:19 Ali Ghodsi? Yes, I actually think you can get all the way there. The data lakes are blob stores, big, large, you know, cheap storage, but kind of data swamps. Turns out there's some recent technological breakthroughs that show you how you can basically turn them into a structured relational storage system. And the way you do that is you build basically transactionality into these data lakes. Once you have that, you can now start adding things like schemas on top of them. Once you add schemas on top of them, you can add quality metrics. And once you have that, you can start reasoning about your data as structured data in tables instead of data that's just files.
9:58 Martin Casado? I get that, like putting structure on top of a blob store or whatever. You still need a query layer, right? Building a query engine that's super fast, that can like respond to analytical queries. I mean, there's entire companies that do that.
10:09 Ali Ghodsi? Yeah, so it turns out there's two APIs you need. One is the data frame API. That'll enable all the data science and machine learning. And then you can build a SQL layer on top of it. And there's nothing that really gets in the way of making this as performant as the state of the art fastest MPP engines out there. So the same tricks that you apply to get speed ups, you can apply the same tricks now because you're actually dealing with the structured data.
10:34 Martin Casado? It feels like especially in data, there's always kind of the trend de jour that everybody's exciting about, but like you're not ever really sure if the market's real or not. And people have been saying this a lot for like real time and streaming use cases. And so it's very clear that like people want to process data at different times, speeds. Batch, we know, is a very large market, which is like, listen, you've got a bunch of data. You want to do a whole bunch of processing and then, you know, it's stored somewhere else. You do some queries. And then there's more and more people are talking about streaming analytics. That means that actually, as a stream comes in, you do the queries before like it hits disk. I sit and pitches basically as a full-time job. A lot of the things motivating the streaming use case is seeing a little contrived.
11:09 Ali Ghodsi? There's the latency and the speed and how fast you can get this stuff. That's one side of the equation, and that's what everybody focuses on. And oftentimes, when we ask the business leader, hey, so what kind of latency would be okay with you? Like, what are you okay with? They'll say, oh, I don't know. I mean, we want it to be super fast, like every five minutes, every 10 minutes. And you can accomplish that with batch systems. Or sometimes they say every half an hour, half an hour is fine. But, you know, we want it to be as recent as half an hour. And then when you dig into like, wouldn't you want it to be even faster? It turns out that streaming systems, the weakest link will dictate the latency. So there'll be some upstream process that has nothing to do with the system that you're putting in place. And if that upstream link, if that one place where you loading the data in or something, if that's coming in every half an hour,
11:53 Ali Ghodsi? and it doesn't matter how fast the rest is. I think the actual latency, this obsession with we need it in less than five milliseconds. For most use cases, you don't have that. There's another side of the equation, which people don't focus on because it's harder to understand or explain. But it might be the biggest benefits out of the streaming systems, which is it takes care of all the data operations for you. So if you don't have a real-time streaming system, you have to deal with things like, okay, so data arrives every day. I'm going to take it in here, I'm going to add it over there. Well, how do I reconcile? What if some of that data is late? I need to join two tables, but that table is not here. So maybe I'll wait a little bit and I rerun it again. And then maybe once a week, I rerun the whole thing from scratch, just to make sure everything is consistent.
12:36 Ali Ghodsi? In some sense, all the ATL that people are doing today, and all the data processing that they're doing today, could be simplified if you actually turn it into a streaming case. Because the streaming engines take care of the operationalization for you. You don't have to worry anymore, you know, did this data arrive late? Are we still waiting on it? Is it inconsistent? They'll take care of all of that.
12:56 Martin Casado? So you think ultimately, a large part of this becomes stream processing?
13:01 Ali Ghodsi? What I'm kind of saying provocatively is that in some sense, all of the batch data that's out there is potential use case for streaming. I think the stream processing systems have been too complicated to use. But actually, under the hood, they take care of a lot of data ops that people are doing manually today.
13:17 Martin Casado? I would love to talk through what you actually think a modern data stack looks like. We talked to a whole bunch of folks and it seems like there's definitely a best practices stack forming, but very, very few people know what it looks like. Let's say you get hired, Ali, you decide to have a new job as a VP of data for like whatever, and you were to build a data infrastructure that does both analytics and it does AI ML. Like what product category, again, not specific products, but product categories would you use? You know, end to end.
13:50 Ali Ghodsi? If you get hired into a big company, I'll spend the next five years fighting political battles on who owns which part of the stack and which technology I'm going to get rid of. And if this technology doesn't fit this one, then how does it? So there's a lot of org chart and human and process problems. But let's say I get in there and they say you get to have it his way.
14:08 Martin Casado? He's getting all the juice. That's right.
14:09 Ali Ghodsi? Exactly. So obviously trying to do something on prem makes absolutely no sense at this point. And when you're building that cloud native architecture, don't try to replicate what you had in the past on prem. Don't think of it as big clusters that are going to be shared by users. One big change that happens in the cloud that on prem vendors don't think of often is that the networks in the cloud are invisible. So any two machines can communicate at full speed. And it can also communicate to the storage system, to the data lake at full speed. This was not the case on prem. So things like Hadoop and so on, they had to optimize where you put the data and the computation has to be close to the data and all that. So you move it into the cloud. Typically, you have data flowing in from some of your systems. That depends on what kind of business you're in.
14:53 Ali Ghodsi? But you have IT devices or you have something from your web apps. Sometimes it goes to, you know, streaming sort of queuing systems, whether it's a Kafka or whatever they're called, it goes into those. And from there, it lands into the data lake.
15:08 Martin Casado? Into a data lake. So you're saying the data goes directly into your data lake?
15:10 Ali Ghodsi? That's the first landing place. If you don't do that, you're actually going to go back a decade or two in the evolution. Because if you don't put in a data lake, then you have to immediately decide what schema you're going to have. And that's hard to get right from the beginning. So the good news with the data lake says you don't have to decide the schema. Just dump it there. Step number two, you need to build a structural transactional layer on top of it so that you can actually make sense of it. And there are technologies for that. There's three, four of these that appeared roughly at the same time. And they all enable you to actually take your data lake and turn it into a lake house. Step number three, you need some kind of interactive data science environment where you can actually start interactively working on your data and get insights of it.
15:55 Ali Ghodsi? Typically, people have notebooks based solutions where they can iterate with notebooks. Typically, there's things like Spark under the hood and they're interactively processing their data and getting insights from it. And that's really important because a lot of data science in organizations ends up not being advanced machine learning. It just ends up being, okay, so we have this data coming in from our products or from our devices or whatever it is. We have to massage it, get in a good form, and we need to get some basic insights out of it. If you want to get into the predictive game, you need a machine learning platform. And there are now these machine learning platforms that are emerging. Many of them are proprietary and you find them inside the companies. You can read about them, but you can't get your hands on one. And there are lots of startups building them, right?
16:35 Martin Casado? And this is for operational ML?
16:37 Ali Ghodsi? Well, I think this actually goes from like training the ML model. So actually featureizing it, creating a model that can do the predictions, tracking the results. Make sure that you can actually make them reproducible and you can start actually sort of reasoning about them to moving it into production, which is the hardest part. And moving it into production where you can actually serve it inside products. So that's the job of the machine.
16:59 Martin Casado? And machine learning platform in your world is the folks that work on it are the data scientists and data engineers both, right?
17:05 Ali Ghodsi? Yeah, so it's different organizations today. Unfortunately, the serving part, the production part, sometimes is owned by IT. And the creating of the models happens by data scientists that sit in line of business. And there is a friction in those organizations because IT operates differently at a different wavelength than from the data scientists. But the machine learning platform needs to span both. If it doesn't, you're basically not going to get the full value out of the machine learning sort of work that you're doing.
17:32 Martin Casado? Can you talk a little bit to like where like the like data pipeline and DAG tools fit in on all this?
17:36 Ali Ghodsi? Yeah, I mean, that would be the first step of this, right? I talked about training immediately. But you know, the really hardest part really is to take that data that's now sitting in the data lake. And first step of it is sort of building the pipelines that create, featureizes it and gets it in the right shape and form. So that you can start doing machine learning on it. So that's step number one. And then, you know, after that, you start training the models. To orchestrate that automatically and make that workflow just happen, you need software that does that. So that's definitely like the first mile in the ML platform.
18:07 Martin Casado? And if I want to like take my traditional BI dashboard and attach it to this, where does that attach?
18:12 Ali Ghodsi? Yeah, good question. So that's the last mile. BI itself typically uses something like J D B C or D B C. And you know, to make that really fast and snappy and work on top of the data, you need some capability that makes that possible. In the past, your only option there has been put it in a data warehouse and then attach a BI tool to it. Yeah, I'm claiming that with the lake house pattern that we're seeing and with some of those technological breakthroughs with those technologies I mentioned, you can connect our BI tool directly now on that data lake.
18:39 Martin Casado? To where? To the transactional layer that's built on top of it to?
18:42 Ali Ghodsi? Yeah, yeah. So if you have something like Delta Lake or if you have something like Iceberg or Hive Asset, you can connect it to those directly.
18:49 Martin Casado? If you didn't have any legacy problem, like it seems like doing a data lake makes a lot of sense. Is there a simple migration path to this?
18:57 Ali Ghodsi? I think it's hard in West. In Asia, it's easier because there's not lots of legacy. It's harder in the West because the enterprises have, well, I have 40 years of technology that I've bought and I've installed. I have data in them, I've configured them. I need to make that work with what you're talking about and how do you fit the two together? Whereas if you're building it from clean slate, you can actually get it right more easily.
19:17 Martin Casado? So you're actually seeing more usage of data lakes for companies that aren't encumbered by legacy?
19:23 Ali Ghodsi? Well, I mean, the companies that are really succeeding with this stuff, I mean, take an Uber. They're doing predictions and the predictions are the competitive advantage. You press within a second, it tells you what the price of the ride is. And it's basically simulated the ride. It basically knows what that meter is going to tell you after an hour ride with traffic conditions and everything. And it gives you exactly the right price, can't overprice, can't underprice. It matches supply and demand of drivers with surge pricing. It can even put people in the same car to lower the cost. All of these are machine learning use cases. And those stacks, these are all companies that are 10 years old. They didn't exist. They don't have lots of legacy data warehouses and legacy systems. They built it custom for this use case and it's a huge competitive advantage.
20:03 Martin Casado? Is this the durable stack that lasts for the next decade? Or is this converging on something that looks a little bit different that you can articulate from here?
20:11 Ali Ghodsi? I can't predict the future, but I'll tell you a few ingredients of it just makes sense long term. If I'm an enterprise and I'm sitting there as a CIO or someone that's picking the data strategy, I would make sure that whatever I'm building is multi-cloud. There's a lot of innovation happening between the different cloud vendors. They have deep pockets and there is sort of an arms race there. So make sure that you have something that's multi-cloud. The second thing I would do is as much as possible, try to base it on open standards and open source technology if possible. That gives you the biggest flexibility that if the space again changes, that you can sort of move things. Otherwise, you find yourself being locked into a technology stack the way you were locked into technologies from the 80s and 90s and 2000s.
20:52 Ali Ghodsi? Storing all your data, dumping it first in raw format into a data lake is something that's going to remain that way. Because there's so much data that's being collected. You don't have time to figure out exact perfect schemas for it and what we're going to do with it. So either we dump it somewhere or we throw it away. And I think no one wants to be that employee that threw away the data, especially when it's so cheap to store it. And the third thing I would do is I would make sure that this stack that you're building, the way you're laying it out, that it has machine learning and data science as first-class citizen. Because that's a way actually to turn the data that we all had and we get business insights out of it. Machine learning platforms didn't exist 15 years ago. That probably will change quite a bit.
21:36 Ali Ghodsi? I think the exact shape of the machine learning platform, I don't think will look exactly the way it is today. But many of the ingredients are right.
21:43 Martin Casado? Perfect. Thank you so much. I don't know if we're on a race to see who speaks faster, but I think you win.
21:49 Ali Ghodsi? Thank you for having me.
21:50 Martin Casado? Yeah, I really appreciate you taking the time.
21:52 Ali Ghodsi? Always a pleasure.
Previous Designing for, Marketing to, and Partnering With Gen Z Next Textiles as Tech, Science, Math, Culture... or Civilization Page updated September 30, 2026 · transcript and summary are machine-generated
