Turing Award Winner: Postgres, Disagreeing with Google, Future Problems | Mike Stonebraker

Mike Stonebraker is a Turing Award winner famous for his contributions to fundamental database technologies. We discussed the story behind building Postgres, where he disagrees with Google/Amazon on databases, and what he's working on now. 🔸 My keyboard Kickstarter: https://www.kickstarter.com/projects/ryanlpeterman/compose-simple-ergonomics-beautifully-done 𝗣𝗼𝗱𝗰𝗮𝘀𝘁 𝗹𝗶𝗻𝗸𝘀: • YouTube: https://youtu.be/YPObBOwIrHk • Apple: https://podcasts.apple.com/us/podcast/the-peterman-pod/id1777363835 • Transcript: https://www.developing.dev/p/turing-award-winner-postgres-disagreeing 𝗘𝗽𝗶𝘀𝗼𝗱𝗲 𝗹𝗶𝗻𝗸𝘀: • Red book of database readings: http://www.redbook.io/ • BEAVER: An Enterprise Benchmark for Text-to-SQL: https://arxiv.org/abs/2409.02038 𝗧𝗶𝗺𝗲𝘀𝘁𝗮𝗺𝗽𝘀: 0:00 - Intro 1:03 - How he got into databases 6:43 - Competing with Oracle 9:07 - What made Postgres special 15:55 - One size fits none 21:37 - Why he disagreed with Google 29:14 - Why he chose academia over big tech 30:58 - Replacing state in an OS with a DB 42:02 - Future problems in databases 51:36 - Technical book recommendations to learn databases 52:20 - Advice for younger self 55:52 - Outro 𝗪𝗵𝗲𝗿𝗲 𝘁𝗼 𝗳𝗶𝗻𝗱 𝗠𝗶𝗸𝗲: • His current company DBOS: https://dbos.dev/ 𝗪𝗵𝗲𝗿𝗲 𝘁𝗼 𝗳𝗶𝗻𝗱 𝗥𝘆𝗮𝗻: • Newsletter: https://www.developing.dev/ • X/Twitter: https://x.com/ryanlpeterman • LinkedIn: https://www.linkedin.com/in/ryanlpeterman/ • Threads: https://www.threads.com/@ryanlpeterman • Instagram: https://www.instagram.com/ryanlpeterman • TikTok: https://www.tiktok.com/@ryanlpeterman

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Computer science may well not be a growth industry going forward.

• 0:04Ryan Peterman?

  This is Mike Stonebraker. He's a Turing Award winner, famous for his fundamental contributions to database systems like creating Postgres and more. What was the hardest part of that meditation?

• 0:15Speaker 00

  Uh, query optimizer. It's just algorithmically difficult.

• 0:19Ryan Peterman?

  How do you identify the people who aren't smart?

• 0:22Speaker 00

  Well, I mean, it's, it's very easy.

• 0:24Ryan Peterman?

  He shared interesting technical takes from his experience.

• 0:27Speaker 00

  On our benchmarks, large language models get 0%.

• 0:32Ryan Peterman?

  Why did you disagree so much with MapReduce?

• 0:35Speaker 00

  That wasn't the only thing Google was stupid about.

• 0:39Ryan Peterman?

  I'm curious, your thoughts on unsolved problems in databases and what you think the future might look like? Here's the full episode.

• 0:52Ryan Peterman?

  The first thing I want to go over is the story of how Postgres got started, but for that, I kind of want to start the beginning. How did you get into building database systems?

• 1:03Speaker 00

  When I graduated, I had the good fortune of being hired at Berkeley. And I, it was clear I had to, you know, continuing what I did for my PhD was not going to go anywhere. Then, as well as today, you're way ahead if you get adopted by a mentor who knows the ropes. So Gene Wong, who is still alive and still, still kicking, took me under his wing and said, well, let's do something together.

• 1:40Speaker 00

  And this was 1971, which was the year after Ted Codd wrote his pioneering paper in CACM. Gene Wong said, well, let's, let's take a look at database stuff and. At the time, the competitors were a thing called the COTUSIL proposal, which you're probably too young to have ever heard of. And so it was a low-level spaghetti network proposal, where you executed queries by following pointers. And then the alternative was the IBM proposal, which was a higher thing called IMS, which is still available.

• 2:25Speaker 00

  And it's hierarchical data. It's a tree. Organized your data as trees. And even at the time, IBM realized that trees were not general enough to solve many people's problems. So they hacked on a way to make it a limited network structure. So it was clear that was a horrible hack. The COTUSIL proposal had all kinds of bad properties, besides being low-level and really hard to debug. It also had the property that if anything changed in your, what's now called your schema,

• 3:10Speaker 00

  you basically had to throw away everything and do it all again because it was absolutely rooted at the physical level. Whereas TED COD stuff made perfect sense. And so Gene said, well, let's build one of these puppies. That's clearly the next thing to try. So we started building Ingress in 1972. I was an assistant professor at Berkeley. As you know, if you're an assistant professor, you have to, you have about, you get five years to prove that you're a big shit. And they fire you or they give you tenure. So Ingress was my ticket to getting tenure, which happened in 1976.

• 3:55Speaker 00

  That was where it started. And then again, you know, happenstance. At the time, you know, a lot of people would build prototypes, which were sort of studenty-like code, which means you could get it to run. But if you gave it to anybody else, they couldn't. So we put in the first 90% to get something we could run.

• 4:21Speaker 00

  And then for whatever reason, we put in the next 90% to get it to where it really worked. So the University of California version of Ingress was really worked. And so over the next couple of years, about 100 universities started running it. Cause Unix became the big thing. And so this was a database, a free database system that ran on Unix. And so it was quite popular in, in the academic world. And so we got, started getting, you know, lots of visitors at Berkeley who would say, gee, this is really nifty looking stuff.

• 5:06Speaker 00

  What's the biggest, biggest Ingress application you have?

• 5:12Speaker 00

  And we'd be forced to say, not very big. And so this was brought home in spades when Arizona State University considered running Ingress on their student records data, all 40,000 students worth. And they could get over that they had to get an unsupported operating system from Bell Labs.

• 5:40Speaker 00

  They could also get over, they had to run an unsupported, unsupported database system from these guys at Berkeley. But the project went down in flames when they realized there was no COBOL available for Unix. And they were a COBOL shop. So unsupported operating system, unsupported database system, no COBOL doomed us to, you know, irrelevance. And it was clear the only way out of that was to start a company. And so in 1980, we got venture capital as it existed then.

• 6:20Speaker 00

  And started Ingress Corporation to move Ingress to, to Dex VMS, you know, a real.

• 6:34Speaker 00

  A real operating system and we had a real company that would support Ingress. And that was the start of the commercial journey.

• 6:43Ryan Peterman?

  I saw that Ingress was competing with Larry Ellison's offering at Oracle.

• 6:49Speaker 00

  Yes.

• 6:50Ryan Peterman?

  I saw that Ingress was was certainly better than what they were offering. But they were still competing somehow. How did they compete?

• 7:02Speaker 00

  Larry Ellison is a fabulous salesman. And he, at the time, he made present tense and future tense indistinguishable. And so he basically lied to customers. He would ship stuff that didn't work. And have his initial customers help him debug it. So I think he engaged in what I consider very shady business practices. But lying to customers, I think, is unconscionable. So for instance,

• 7:42Speaker 00

  there was a thing called referential integrity. Which is if you if you fire an employee and he's the last person in a given department, do you want to delete the department or do you want to have it be a department, a ghost department? It's all that kind of stuff. And so Ingress Corporation implemented referential integrity. Oracle Corporation wrote two manual pages that said, here's the definition of referential integrity, which everybody agreed to. And then down at the bottom, it said, not yet implemented.

• 8:25Ryan Peterman?

  Interesting. Yeah, I had interviewed someone who worked at Sun Microsystems, and they had a similar opinion that they... Larry Ellison was a little bit shady. So it seems to be a commonality. I also saw somewhere else in something that you had said is that when Oracle acquired MySQL, that everyone kind of got afraid of that and moved to Postgres.

• 8:55Speaker 00

  That was the genesis of Postgres replacing MySQL as the preferred open source relational database system.

• 9:06Ryan Peterman?

  So you created Ingress, and there was a lot of technical innovations in it so that it was better than the incumbents. But ultimately, it went away and you developed Postgres. What was the thing that Ingress didn't do that Postgres would do?

• 9:24Speaker 00

  The big thing that guided us at the very beginning was the original reasoning for the academic version of Ingress was we were going to support a geographic information system that the neighboring professor Praveen Varia wanted. And so to support a GIS system, you need points, lines, polygons, line groups, that sort of stuff. And it was clear that Ingress couldn't do it because the data types we put into Ingress were the standard ones:

• 10:06Speaker 00

  integers, floats, text, strings. And you couldn't support, you couldn't efficiently support GIS types on top of that. So as a GIS, the academic version of Ingress was a complete failure.

• 10:27Speaker 00

  And that was in the back of our mind. The other thing that happened, this is a little out of chronological sequence, but it helps make the point, is that the commercial version of Ingress, I think around 1985, you know, there was, ANSI had just proposed a date and time standard for relational databases. And so, commercial Ingress implemented date and time, you know, using the standard Gregorian calendar. And so, I was associated with the commercial version of Ingress,

• 11:10Speaker 00

  as well as, I was still at the University of California as a professor. So I got a call from, from an Ingress customer, who said, you know, "You implemented date and time wrong." And I said, "Huh? We implemented the Gregorian calendar and you can subtract, and you know, if it has, you know, days have 30 or 31 months, except for February, except for leap years. So subtraction on dates works exactly the way you would expect it to." But he said, "That's not what I want." In his particular world, he said he was, he was dealing with,

• 11:55Speaker 00

  with bond financial instruments. And for some reason, I mean, you got the same amount of interest on his financial bonds during each month, no matter how long the month was. So he had the date you bought the bond, the date you sold the bond. He wanted to do a subtraction, multiply it by the coupon rate, and say, "That's what, that's the interest we paid you."

• 12:27Speaker 00

  But of course, his version of subtraction was March 15th minus February 15th is 30 days. Because that's the definition of his calendar. And so he had to retrieve two dates out to user code, do the subtraction in user code, put the answer back, and it cost him a factor of two or three in efficiency. And he said, why can't I just overload your definition of subtraction with what I want? And of course, with Ingress, it was hardcoded. And the problem was, this is a case where you wanted bond time,

• 13:08Speaker 00

  just like you wanted points, lines, and polygons. And so Postgres was engineered to have an extendable type system. So you could have whatever data types you wanted. And they were very efficient. And that was the main gist of Postgres, was that it had that flexibility. And as, you know, in business data processing, a lot, most people were happy with the standard data types. But relational databases started to spread to all kinds of other places, what are called abstract data types, or, you know,

• 13:52Speaker 00

  stored procedures, you know, a bunch of names they're called, you know, had great applicability. And so Postgres, that was, that was the big thing in Postgres. We also, Postgres also supported what the AI guys at the time wanted in the way of inheritance.

• 14:18Speaker 00

  We also supported time travel, but the implementation absolutely sucked. And it got taken out after a while. So there were a huge number of really nifty things in Postgres.

• 14:33Ryan Peterman?

  You mentioned you want to hire extraordinary software engineers. And I think you've you've said before that you have no trouble finding those people. How do you identify those people in your hiring that they're the extraordinary ones?

• 14:48Speaker 00

  It's usually pretty obvious. I mean, I have a good feel for how difficult stuff is. If they get 3X the amount done in school that I think is reasonable, then they're incredible.

• 15:05Ryan Peterman?

  On the flip side, you had this interesting quote. I wrote it down. He said, I can't stand people who aren't really smart. It's challenging to talk to them. How do you identify the people who aren't smart?

• 15:20Speaker 00

  Well, I mean, it's very easy. You talk to them and it rapidly, you can rapidly surface whether they're smart or not. What was your master's thesis? What did you do? Well, how did it exactly work? Well, how did you deal with error conditions? How many processes did you have? Why didn't you use threads? I mean, you ask them deep technical questions.

• 15:54Ryan Peterman?

  You gave a talk, and I think there's also a paper behind it, of this idea that one size fits all database systems is not optimal. One size actually fits none. And that what you really want is database solutions that target specific needs. What database offerings do you see today that are one size fits all?

• 16:16Speaker 00

  In 2004, when I wrote the paper, we had an academic project which was building what became Streambase. And so a stream processing engine looks nothing like a relational database.

• 16:30Speaker 00

  And we had the gist of an idea for column stores for data warehouses, which was popularized by Vertica. Looks nothing like a row store. So here were three wildly different implementations that had no resemblance to each other. And in each case, they were an order of magnitude faster than the other guys. So it's pretty clear that with those three instances, you give up an order of magnitude when you're running a database system that isn't architected for your kind of stuff. I think that's still true.

• 17:15Speaker 00

  I mean, I think ClickHouse is a column store. Pinecone is faster than user-defined types on text-based vector processing. And so I think it's still very much the case. And I think there's no difficulty putting a common parser on top of multiple implementations. Postgres has so far chosen not to do that. They don't implement a column store.

• 17:54Speaker 00

  And so I think they are not competitive on sizable data warehouses. They also don't have multi-node support. Again, for people with big data warehouses, that's table stakes. So I think it's just as true today as it ever was. I think that what is true is that if you want to get going, you have a database problem. You know, the answer is choose Postgres. And there's a huge programming community, all kinds of, all kinds of, you know, data type implementations, it's free.

• 18:37Speaker 00

  And you can find Postgres talent easily and get going. And so I think it's, it's, it's a great choice for lowest common denominator. And until you're trying to do a million transactions a second, it works just fine. Until you're trying to support a petabyte data warehouse, it, I say at the low end, it's absolutely the right. One size fits all at the low end, it's Postgres. At the high end, that's just not true.

• 19:12Ryan Peterman?

  GPUs, do they make available some new opportunities to optimize databases?

• 19:19Speaker 00

  Probably, but I think the, the big challenge is that GPUs are,

• 19:27Speaker 00

  you know, SOMD, you know, single instruction, multi-data. And that's, that's the anathema of indexing.

• 19:36Speaker 00

  And so whenever indexing is the right answer, they're probably not a good idea.

• 19:43Speaker 00

  And I think

• 19:46Speaker 00

  also you've got to architect them so that the, so that the bandwidth, so that the bandwidth from storage is, is not, not the bottleneck. And so if they're an add-on to the CPU, as often as not, the bus connecting it to the GPU to the CPU is a bottleneck.

• 20:10Ryan Peterman?

  Can you explain why indexing would be not as effective when there's SIMD?

• 20:17Speaker 00

  So let's, let's say I'm, I'm looking for Ryan's,

• 20:24Speaker 00

  I'm looking for Ryan's salary and I have a B-tree. So you go to the root of the B-tree.

• 20:32Speaker 00

  You find, you find the divider that has both sides of Ryan. You follow the pointer. That's a memory access for sure. Then you do it all again. And you do this like three or four times. So that doesn't parallelize well. So the answer is indexing doesn't parallelize well.

• 20:55Ryan Peterman?

  You mentioned B-trees. When you first implemented that first version of Ingress, did you write all of that by hand? Because I imagine there's probably not some existing B-tree library or something.

• 21:08Speaker 00

  Yeah, we wrote, the original version of Ingress was all written from scratch.

• 21:12Ryan Peterman?

  What was the hardest part of that implementation?

• 21:16Speaker 00

  Query optimizer.

• 21:18Ryan Peterman?

  And why was that hard?

• 21:20Speaker 00

  It's tough. It's, it's just algorithmically difficult. It's still, if you ask most any senior database programmer, what's the hardest, hardest part? They'll still say the optimizer.

• 21:36Ryan Peterman?

  MapReduce came out at some point in the early 2000s, and it kind of took the data world by storm. People were really impressed by it. They thought Google really knows what they're doing. This is the best thing since sliced bread. But it seems like when I look at the literature and what you thought at the time, you kind of disagreed heavily. Why did you disagree so much with MapReduce?

• 22:02Speaker 00

  Well, I think there were a lot of not very enlightened people who said, Google, Google is really smart. They must know what they're doing. And so we'll do whatever they say. And so they would, they would, they would engage in Hadoop or engage with Hadoop. But Hadoop is ridiculously inefficient. And so at the time,

• 22:32Speaker 00

  You know, others, you know, Dave DeWitt and others who were involved in our 2011 paper. We understood distributed databases and understood that you could beat the heck out of Hadoop with a distributed database system, which is basically what that 2011 paper says. And of course, it was, it's true. And, but that wasn't the only, that wasn't the only thing Google was stupid about. So, Google also had the opinion that eventual consistency was the right way to do concurrency control.

• 23:16Speaker 00

  And so that was postulated from on high by Google all during that same period of time. And it wasn't, and all the database people said, you know, you're out of your friggin' mind because it doesn't, it solves one particular kind of problem, but only, and that very rarely occurs in practice. Why did they pursue eventual consistency? Okay, well, the idea is that you have an East Coast database and a West Coast database, and they're replicas. So you want them to be the same. If you say, "I'm going to do a transaction.

• 23:57Speaker 00

  I'm going to decrement by one the number of widgets in the West Coast warehouse. Then I'm gonna, before I commit that transaction, I'm going to update the East Coast warehouse, pay, pay a message over and back to update it. And then to make sure everything goes well, it takes, it takes another round trip of message to make sure that both of them actually do the commit correctly. So it's expensive to do a distributed commit. And it still is. And so, the idea was, well, you, you do the, you do the West Coast update. You decrease the widgets by one.

• 24:39Speaker 00

  You just send a message asynchronously and not in a transaction. So that eventually, the East Coast warehouse gets decremented by one.

• 24:51Speaker 00

  So meanwhile, if you're on the East Coast, you decrement, you know, food stuffs by one. You send an asynchronous message. Eventually, the West Coast gets it. And eventually, everything settles out. So if you're allowed to go below zero, then what will happen is if the East Coast guy and the West Coast guy simultaneously sell the last widget, then eventually, the state of the warehouse will be minus one. And somebody won't get their widget.

• 25:36Speaker 00

  And so if you're allowed, like Amazon, to say usually ships in 24 hours, then maybe you're allowed to oversell. But most enterprises can't do that. And so eventual consistency just doesn't work. So we talked a million hours ago about referential integrity. So referential integrity in a sales system is, integrity constraint is stock is greater than minus one. And that fails with eventual consistency.

• 26:16Speaker 00

  And so Jeff Dean of Google finally figured that out. And when they did Spanner, Spanner had a conventional transactional system. And so Google completely abandoned eventual consistency and completely abandoned MapReduce.

• 26:37Ryan Peterman?

  So the trade offs basically correctness for performance.

• 26:42Speaker 00

  So it's performance versus data integrity. And if you don't care about your data, then you're willing to deal with with bad things happening.

• 26:54Ryan Peterman?

  So did you ever talk to the Google team while they were doing those things that you thought were so wrong?

• 27:00Speaker 00

  We talked to them before the 2011 paper.

• 27:08Speaker 00

  And said, why don't we, why don't we partner up and do some stuff? And they weren't, they weren't interested. So they declined.

• 27:19Ryan Peterman?

  Have you seen other examples in other big tech companies where their databases or database solutions where you actively disagree with them, like maybe Amazon or, or Facebook?

• 27:31Speaker 00

  Well, I gave a talk at Amazon, maybe three years ago. And I told them all the things I thought they were doing wrong. And I think Amazon's problem is that they are supporting, you know, 15 different database systems. And that's about 12 too many. So, so I think they have their own culture and I told, I, I said, you're supporting too many database systems. And at this point, they haven't chosen to retire any of them.

• 28:09Ryan Peterman?

  Why do you say that the 15 should be three?

• 28:12Speaker 00

  Well, they're supporting a graph based database system. And it's well understood that a graph based database system is almost never the performant option. And so if you want a graph, if you want, if you like the idea of having a user interface that deals with nodes and edges, that's fine. Put, put a layer on top of a relational database system that gives you that user model. And so most of their database systems, there's some other of their database systems that

• 28:53Speaker 00

  better at what it does than, than it is. And so, so the answer is you should retire.

• 29:04Speaker 00

  You should retire any database system that isn't performant in a big enough market to justify the maintenance.

• 29:14Ryan Peterman?

  You've influenced industry significantly from academia. And my one thought that I had is, why not work directly in industry? Or why do you prefer the position of being in academia and having influence in the way that you have versus just taking a job at AWS or something like that, being a very distinguished engineer there?

• 29:41Speaker 00

  Because that gives you a boss. And that gives you company rules, limits your ability to publish, limits your ability to go talk at conferences, limits your ability to go poke at what various competitors are doing that they won't tell their competitors. But mostly, I really like being in startups. After the commercial version of Postgres got acquired by Informix, I was working part-time for Informix, which was a 2,000-person company.

• 30:26Speaker 00

  And I didn't feel like I could make a difference because it was bureaucratic and whatever the president wanted, he got. So I think I'm just not cut out for, I'm not cut out for politicking. I don't do that very well. And I have a hard time interacting with people I think are dumb. And that, again, so I guess I have some problems with big companies.

• 30:56Ryan Peterman?

  I wanted to talk a little bit about DBOS. I just thought it was a really interesting technical model. Can you explain what DBOS is?

• 31:07Speaker 00

  We started the academic project in

• 31:12Speaker 00

  2019, 2020, something like that. And the gist of it was, at that point, Mateo Zaharia, who was on the faculty at Stanford, was also one of the founders of Databricks, was the original creator of Spark. And so, he said, at the time, Databricks, you know, basically was running people's Spark jobs on the cloud. And so, he said, at any given time, we might be orchestrating a million Spark jobs. And so, we have to write a scheduler

• 31:57Speaker 00

  that's going to decide who to run next at scale a million. And he said, there was no, we tried all the schedulers written by the OS folks, and they couldn't, they didn't scale. So, we put all the scheduling data in a Postgres database, and basically, a Postgres application was doing scheduling. And then, it sort of clicked that, by and large, most everything you do in an operating system is managing data at scale. And you should do that using database technology. So, why don't we just replace

• 32:40Speaker 00

  at least the upper half of Linux with a database system? So, that was the gist of the academic project. And we worked on it at Berkeley and Stanford in the early 20s. And it was, it was very successful. It clearly, it clearly worked.

• 33:04Speaker 00

  And in the process, the Stanford folks wrote an extension to JavaScript so that you could program, you need some programming world that can, can talk to your implementation. So if you're doing what amounts to a programming language, and you're running on top of what amounts to an operating system that is a database, then the obvious thing to do is put all your state in the database. And that's exactly what they did. And so we had an innovative programming language model, an innovative operating system model.

• 33:49Speaker 00

  And, and of course, then the idea was, well, can we start a company? And so we talked to the VCs, who, to a person said, "You're, you're dreaming if you think you're gonna displace Linux. However, this programming language stuff is really nifty." We had what amounted to extensions to JavaScript that would allow any, any program to have all of the nice features of a database system. You know, stuff was durable, you could have transactions.

• 34:31Speaker 00

  If it failed, you'd fail over, you know, it's all that nifty stuff. So we got funded to start a company in 2023. And that was DBoss Incorporated. And we decided that, that was the name of the project. It's always been the name of the project. But we were, we were basically in the programming language business. And so at the current time, DBoss has a version of TypeScript, a version of Java, a version of Go.

• 35:10Speaker 00

  And a version of Python, which which are basically seamless. It runs what looks like vanilla programs. In the world of the cloud, there's every incentive for you to structure your your your application as a workflow. And so we decided that we would support a workflow system, period. And so the workflow that that D-Boss supports in those four languages is the steps in

• 35:49Speaker 00

  in a workflow, the individual micro apps, whatever you want to call them, are transactional. Workflows are durable, so that once you do a step, it's not forgotten. And it's clear that we can make workflows atomic if there was a market for it, which means the whole workflow would either finish or look like it never happened. So it has very, very nice properties and is a great deal faster and a great deal easier to use than the competition.

• 36:32Speaker 00

  So the company is selling and innovating in this area. And so so the idea is that you want to make state of your application persistent when you put it in the database. And then it and then you figure out how to do it fast. And I think their their business model, as we were talking earlier, is very much get get leaf level programmers interested. So it's been very much, you know, tell us, leaf level programmer, tell us what you need

• 37:14Speaker 00

  that we don't have, get it quickly and convince people to try it. And

• 37:24Speaker 00

  We've been very, very successful with other startups who want to choose the best thing. And we're starting to be successful with the big boys. So it's an interesting, interesting market. And I think the key thing so far is probably two thirds of the customers are doing agentic AI, which means that they have a large language model surrounded by a bunch of stuff that adds more signal. And so far, most of agentic AI is read only.

• 38:09Speaker 00

  Meaning you want to produce a prediction for whether Ryan is going to be a good customer or not. And so that just runs some stuff and then produces a new thing that's given to somebody. So basically read only, which means that you're not, you're not actually updating Ryan's credit rating. Or, and so I think fairly quickly, this, the whole world is going to move to using, you know, agents to do read-write applications.

• 38:52Speaker 00

  And that's going to make, that's going to make them very, very databasy. And D-Boss does that stuff really, really well. And so, you know, for instance, if you want to write an agent or two agents that move a hundred dollars from my account to your account. And so you debit my account, you increment your account. And these two agents have to agree to commit.

• 39:25Speaker 00

  Or you have to back everything out. Which is to say, the workflow needs to be what I called atomic, which is it all happens, or it looks like it never happened. And so I think the demands on, in this market, will escalate with, with things, with people wanting stuff to be read, write. And so I think that, that will bode well for the market and bode well for DBOS.

• 39:56Ryan Peterman?

  And this, this, what's being offered in the market today to application developers differs from the original research project, where that was actually swapping out the guts of an operating system with a database. I see. That, I mean, that's really cool. I never imagined replacing all the state of a, of an operating system with a database. What's the, there, there's gotta be some trade off there.

• 40:21Speaker 00

  Well, a file system written on top of a DBMS is faster than, than the Linux file system. The scheduling engine is competitive with other scheduling engines. You can make everything fail over, so you get high availability without having to do anything else. The answer is, there, there's really no downside.

• 40:47Ryan Peterman?

  Then why wouldn't Linux incorporate that and upgrade itself with this?

• 40:55Speaker 00

  You hope they would. In other words, you should, you should keep all the device driver junk down at the bottom, because that's, there's a lot of it, and no one wants to do that and replace everything else with a database implementation.

• 41:11Ryan Peterman?

  Is that something that you've mentioned to Linux people? And what's their typical reaction?

• 41:17Speaker 00

  Back in the academic project, when I'd mention that to operating system folks, they would get very, very threatened. Which is, this is the database guys trying to take over their turf. And I think the programming language guys ditto, you know, which is, you know, the,

• 41:40Speaker 00

  The way to implement the runtime for a programming environment is with a database.

• 41:46Ryan Peterman?

  That's, uh, that's interesting. I mean, if it's objectively true, then maybe it will take over.

• 41:53Speaker 00

  Well, I mean, it took Java 10 years to become widely accepted. I just think the time constant is substantial.

• 42:02Ryan Peterman?

  I think we talked a lot about the past of databases and I'm curious your thoughts on unsolved problems in databases and what you think the future might look like.

• 42:13Speaker 00

  Okay, so I think two different things that I'd like to talk about. The first one is, like everyone else,

• 42:23Speaker 00

  three years ago, we started to look at what were large language models good for. So we've been trying to get what's now called text to SQL.

• 42:37Speaker 00

  To, we've been, we've been trying to make it work

• 42:42Speaker 00

  on real world databases,

• 42:46Speaker 00

  especially real world data warehouses. So we've been trying the technology on four different production databases warehouses, where we've gotten the workload, the actual workload that's run.

• 43:05Speaker 00

  And, you know, from the actual users using the system. And we've gotten them to reverse engineer the text that corresponds to that SQL. So we have text and SQL for, we have four benchmarks.

• 43:26Ryan Peterman?

  When you say text to SQL, you mean like a human prompting model or something like, I would just, in English, that text would be, you know, everyone over four years old.

• 43:38Speaker 00

  Tell me all the professors at MIT who won the Turing Award. And so an LLM is supposedly good at that.

• 43:48Speaker 00

  And so the text to SQL benchmarks, there's one called Spider, another one called Bird. And the best LLM systems are pretty good at those benchmarks, you know, like 80% accuracy or better.

• 44:04Ryan Peterman?

  So not superhuman?

• 44:05Speaker 00

  Not superhuman, but they're pretty good. Like, you would consider using them. And, you know, like, current leaderboard is something like 85% accuracy, which, I mean, it's getting there. You say, maybe it's not quite ready for prime time, but it's simply, it certainly looks, it looks pretty good. Well, on our benchmarks, large language models get 0%. And if you enhance them with RAG and, and all the tricks, goes to 10%. And if you give as a prompt, the FROM clause, in other words, all the actual tables that need to be accessed,

• 44:50Speaker 00

  and all the actual join clauses that need to be joined, then accuracy goes to about 35%. So, the definition of this stuff doesn't, is not ready for prime time, and not going to be for a while, if ever. So what, what's the difference? Number one, data warehouse, you know, LLMs are trained on the pile. Data warehouse data is not in the pile. And there's an adage that if you haven't seen the data a couple times before, you have no chance

• 45:29Speaker 00

  of regurgitating it. That's number one. Number two, query complexity on Spider and Bird is maybe 10 to 20 lines of SQL. Real-world data warehouses, it's 100 lines of SQL.

• 45:49Speaker 00

  Complexity is bigger. Number three, the schema in spider and bird is clean. You know, the table names are mnemonic, the column names are mnemonic, and there's no duplication. In data warehouses, people have materialized views all the time. It means there's redundancy. And, and column names are often underscore Z, uppercase, blah. And so they're not mnemonic. That makes it a lot harder. And then they also have idiosyncratic data.

• 46:31Speaker 00

  So J term is popular thing at MIT. It's a one month term in January. Not unique to, to MIT, but not very popular. So not in the pile. Idiosyncratic data, simple queries, schema, schema is a mess. Make, make it not work. And those are true of every data warehouse I know of. And so I think the, the technology simply doesn't work and isn't going to work anytime soon. So we've been, so what do you do?

• 47:15Speaker 00

  So, well, first of all, we published our benchmark. It's a thing called Beaver, which is an anonymized and abstracted version of these four data warehouses. And so if you think you're really good at doing text to SQL, try a real benchmark, not a fake one. So number two, borrowing from what I just said. If you don't have all the join terms and you don't have the from clause, you're toast.

• 47:50Speaker 00

  What's more, if you don't break down the query into simpler pieces, you're toast. So that says to me that you want to give your retrieval system simpler pieces, which include the FROM clause and include join terms. That's number one. Number two, the minute you want to talk to two different structured databases, you know, like your data warehouse and your CRM system, then it's pretty clear to me that doing a structured data join

• 48:34Speaker 00

  using an LLM is a bad idea. It's just, you're much better off leaving them as tables and doing a join in SQL. So our point of view is we are trying out turning everything into tables. You know, we're, we're, we're working with the Department of Transportation in the city of Munich, Germany. And they have six people full time who are answering citizens complaints, queries, which are of the form, how come I, I don't have enough time to cross this intersection next to my house before the light turns? All kinds of stuff.

• 49:20Speaker 00

  How come the trolley doesn't stop for enough time for me to get on the trolley? You know, it's how come the trolley doesn't come

• 49:30Speaker 00

  more than once an hour? I mean, it's all this stuff. Their database is the trolley schedule, that's SQL. The light sequencing, that's SQL.

• 49:42Speaker 00

  The intersections, that's CAD. The federal, you know, country of Germany regulations of this stuff. That's text. City of Munich regulations for this stuff, which is text. So you've got to join SQL, SQL, CAD, text and text. So our point of view is turn it all into SQL, all into tables. And do a join with what amounts to a query optimizer. So that's what we're working on. I think other people will have other ideas, but I think it's extremely fertile area. Cause people really want to do it.

• 50:27Speaker 00

  So that's number one. Uh, number two, we talked earlier about agentic AI. The minute this becomes read, write, it's a distributed database problem. And you want atomicity, consistency, all that stuff. I think a very interesting area. So that's pretty much what I, what I am, what I'm working on now.

• 50:52Ryan Peterman?

  On that benchmark where it's 0% right now, what percent is human? Like if you took someone who really knows SQL, what would they score like the average human?

• 51:03Speaker 00

  So once you disambiguate the, the text.

• 51:07Speaker 00

  A, a knowledgeable SQL user programmer with the schema will do, will, will get very high accuracy.

• 51:15Ryan Peterman?

  Okay, like 90% or something.

• 51:17Speaker 00

  At least.

• 51:18Ryan Peterman?

  Okay, okay. Wow, it's, I'm surprised that the LM so scores so lowly on, on this kind of benchmark. Maybe when this goes out, someone who works at Anthropic will reach out to you or something and say.

• 51:31Speaker 00

  I'd love to, I'd love to find out, cause I mean, it's a terrific success story.

• 51:38Ryan Peterman?

  For people who want to deeply understand databases and they're looking for some material to study. Is there a book that you'd recommend that's a top technical book to learn databases?

• 51:50Speaker 00

  Papers in the literature. I think Joey Hellerstein and I published a red book. What's called the red book, which is called readings and database systems. It's now.

• 52:05Speaker 00

  Eight years old. I mean, I think that would be a great set of readings for eight years ago. And beyond that, papers, popular papers from the literature.

• 52:19Ryan Peterman?

  If you could go back to yourself when you just graduated, give yourself some advice, knowing what you know today, what would you say?

• 52:27Speaker 00

  Back when I first took the job at Berkeley, without thinking about it much, we said, let's write a database system. And we knew nothing about databases, nothing about implementations. We were not skilled programmers like Bill Joy. So, starting off doing something that was that crazy was really pretty crazy. And, and, you know, you, you effort and you make stuff work and you learn along the way. And so I think the answer is, think outside the box, think crazy thoughts, and try and do them.

• 53:10Speaker 00

  And I think, to me, it's not at all obvious. The better question is, if you were starting out today, what would you major in? Because I think, you know, computer science may well not be a growth industry going forward. And I'm not sure I would recommend 18-year-olds to major in computer science. I mean, I think health, healthcare and the building trades are, are safe bets and everything else looks much riskier. If, if you're about to get your PhD and are trying to decide what to do, then I think life is pretty easy.

• 53:55Speaker 00

  You know, take, take the most prestigious job you can get.

• 54:00Speaker 00

  And find a mentor who's willing to help you. And then pick some area that isn't, you know, like our, our stuff, you know, which is called Rubicon is definitely not going with the flow. So choose something that's not, that isn't going with the flow. And try and make it work. Both my wife and I said, follow, follow your passion. Somehow the money will work out. And I don't believe that for a minute, but I think that's what you have to tell your kids. And your grandkids. But if you don't believe that, then why do you have to tell them that? My wife is, is a good example.

• 54:45Speaker 00

  So she has a, she has a master's degree in computer science, undergraduate degree in computer science. And she wanted to be a teacher, you know, you know, K, K 12 teacher. And her parents said, you can't do that. It doesn't pay enough money. And so I think, and I think she, ever since that time has regretted that decision. She wasn't passionate about doing computer science. It was simply a trade. And so I think, find something you're passionate about and, and you will, you know, either you, you won't starve. You may not make a lot of money.

• 55:29Speaker 00

  But I think chances are you'll be happier than if you do something you're not passionate about. Cause I think a lot of people I know view their job as simply a job. And life is what happens between 5:00 PM and 8:00 AM. I don't feel that way at all. I really like what I do. Wouldn't matter whether I made a lot of money or didn't. Awesome. All right. Well, thank you so much for your time. Really appreciate it.

• 55:57Ryan Peterman?

  Thank you for listening to the podcast. It's a passion project of mine that I've really enjoyed building. Another passion project that I've been working on, kind of in secret, is building an ergonomic keyboard that I wish existed. And I finally have a prototype, so I'd love to show you what we've built. It's ultra-low profile and ergonomic, and I couldn't find anything like it on the market, so that's why we built it. I'll put a link to the keyboard in the description. You can take a look and learn more about the project there. We could definitely use your support. Also, if you have any feedback for me about the show, I'd love to hear it. Comments on YouTube have led to guests coming on like Ilya Grigorik and David Fowler. I wasn't aware of them until someone dropped a comment.

• 56:38Ryan Peterman?

  Also, feedback in the comments helped me learn to reduce the number of cliffhangers in the intros. So your comments definitely make a difference. Please keep letting me know what you'd like to see more of in the show, and I'll see you in the next episode.