The Future of Software Development - Vibe Coding, Prompt Engineering & AI Assistants

Is AI the Fourth Pillar of Infrastructure? Infrastructure doesn’t go away — it layers. And today, AI is emerging as a new foundational layer alongside compute, storage, and networking. Erik Torenberg interviews a16z’s Martin Casado, Jennifer Li, and Matt Bornstein breaking down how infrastructure is evolving in the age of AI — from models and agents to developer tools and shifting user behavior. We dive into what infra actually means today, how it differs from enterprise, and why software itself is being disrupted. Plus, we explore the rise of technical users as buyers, what makes infra companies defensible, and how past waves — from the cloud to COVID to AI — are reshaping how we build and invest. Timestamps: 00:00 Introduction to Infrastructure 00:48 Defining Infrastructure and Its Components 02:27 The Fourth Layer: AI Models 06:34 The Evolution of Infrastructure 17:46 Developer Tools and the AI Wave 21:27 Data Engine Systems 22:11 Defensibility in AI Infrastructure 25:28 Expansion and Contraction Phases 27:09 Challenges in AI and Infrastructure 28:32 AI Models and Generalization 30:59 Thoughts on Andrej Karpathy's Talk on AI 34:09 AI and Human Expectations 36:18 The Role of Developers in the AI Era 40:31 Synthetic Data 43:17 AI Agents 45:16 Vertical Integration vs. Horizontal Specialization Resources: Find Martin on X: https://x.com/martin_casado Find Jennifer on X: https://x.com/JenniferHli Find Matt on X: https://x.com/BornsteinMatt Stay Updated: Let us know what you think: https://ratethispodcast.com/a16z Find a16z on Twitter: https://twitter.com/a16z Find a16z on LinkedIn: https://www.linkedin.com/company/a16z Subscribe on your favorite podcast app: https://a16z.simplecast.com/ Follow our host: https://x.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Matt Bornstein?

  It's like faster than light speed travel has just been invented.

• 0:03Jennifer Li?

  For many of the developers and programmers I talked to today, they are like going to Disneyland just because how many great tools there are to help them move faster.

• 0:11Martin Casado?

  We do spend a lot of time just trying to be very honest with ourselves. What has the new behavior been created? Where is this stuff getting used?

• 0:18Jennifer Li?

  Infrastructure never goes away, it just gets layered. In this case, it's definitely all the infrastructure that I've been using and leveraging in the past are still very relevant, but it's definitely getting layered by having this fourth pillar.

• 0:28Martin Casado?

  Software was always the disruptor. One of the most exciting things about the AI wave is like software is being disrupted. Like we're being disrupted.

• 0:35Matt Bornstein?

  This is a pretty big deal. It's by far the biggest thing that I've seen happen sort of in my life.

• 0:40Speaker 03

  Is AI the fourth pillar of infrastructure? Today on the podcast, we're joined by Martine Casado, Jennifer Lee, and Matt Bornstein from A16Z's infra team to unpack how infrastructure is evolving in the age of AI, from compute, storage, and networking to models, developer tools, and agents. We get into what infra actually means today, how it differs from enterprise, and why software itself is being disrupted. We also talk through the rise of technical users as buyers, what makes an infra company defensible, and how past infra waves from cloud to COVID to the current AI boom have shaped how we invest and build. Let's get into it.

• 1:19Speaker 03

  As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com/disclosures.

• 1:49Speaker 03

  We're here today to discuss the state of infra. First, can we get a definition of infra? Where does infra differ from enterprise? How do we think about it internally?

• 1:57Matt Bornstein?

  I would say infra is basically what makes software work. We'll probably get pretty deep into a set of technical definitions. You mentioned sort of networking, storage, compute. Where does AI fit into that? But I think at the simplest possible level, if you want software, infra is what engineers are using behind the scenes to make all this possible.

• 2:13Martin Casado?

  And our formal definition internally is technical buyer. So it's the stuff you use to build the stuff, the stuff you use to build apps. And if it's used by a technical user, we consider it infrastructure. Whereas something like, let's say, vertical SaaS could be used by a flooring company or by a marketer or by sales, that would not be considered infrastructure.

• 2:33Matt Bornstein?

  And technical user, for the record, is developer, data scientist, analyst, cybersecurity professor. Yeah, right. There's a DevOps, right? There's a wide range of sort of like people. These are our people, right? Like the kind of nerds behind this.

• 2:46Jennifer Li?

  And in a system term, so you can think about it as compute, networking and storage, but also all the tooling that goes around the developer's day of what you're using to build software and what are the tools and products that are operating these ever growing, more complex software as well, all the way to semi-technical users that may want to either prototype or tinker with building applications. We're very interested in anything in the technical domain and used by technical people.

• 3:12Erik Torenberg?

  So you mentioned compute, networking, storage. How should we think about models? This is the fourth layer of infra, how do they interface? How should we think about that?

• 3:20Jennifer Li?

  I certainly think of it as a fourth layer of infrastructure. It's certainly leveraged and built on top of all the three pillars we're talking about. It has a lot of demand of compute and of course it's trained and also producing a large amount of data. And to leverage and use these models for our purposes, latency and networking capabilities is also very important. But it's going to be as prevalent as any piece of infrastructure software. I don't know like the analogy these days anymore. Is it a database? Is it sort of like a new form of compute? So really, to me, it's like a fourth pillar that incorporates everything, but also provides intelligence for the software we're using and building today.

• 3:59Martin Casado?

  Yeah. So I think this is exactly right. I think it's probably worth asking like why a piece of infrastructure is a piece of infrastructure. And generally, a new piece of infrastructure changes the way that you program computers and then changes the stack that's around it. Like it's got different memory requirements. It's got different latency requirements. And so it just requires rethinking how we build software and how we build infrastructure. You know, I would say in addition to compute, networking, storage, I'd say distributed systems would also be included just because things like state consistency require you to think about like proximity and guarantees. I would say databases probably did too, because it changed our programming model. You have different guarantees. And these models very much fit in that for a couple of reasons. I think Jennifer's exactly right. Like you just build different data centers and different chips if you want to build these models.

• 4:43Martin Casado?

  So it has that impact, but programming them is like non-obvious. Like we're just still trying to grapple like how you program with them. Like they don't really listen to you. Sometimes they do the coding themselves. And if I were to try and distill, like, what is the one biggest difference that these models provide to infrastructure? It's the following. I don't remember ever in the history of computer science where we've, from an application standpoint, we've abdicated logic. Like in the past, we've abdicated resources. Like you're like, give me compute, give me swords, like these abstracted resources. But the logic, the yes or no, like the what it's doing always came from the programmer. But in these ones, we're like, come up with the answer for me. And so it's requiring us to rethink, what does it mean to be a programmer? What does it mean to be software, et cetera?

• 5:26Martin Casado?

  So it's clearly very fundamental to computer science. And I would say, again, to Jennifer's point, is very, very much a new piece of infrastructure.

• 5:34Matt Bornstein?

  And I think a lot of people are trying to reason by analogy. You sort of alluded to this, Jennifer. It's like, oh, is it like a database because it can answer queries? Or is it like a network because it's sort of non-deterministic and we need to handle retries and weird edge cases? But like, I think people are really just trying to figure out how to program these things, which you sort of said, Martin. But like, we've got to start from a blank sheet of paper, which is what makes our jobs like really exciting right now. Cause there's a lot of people trying to figure it out and coming up with ideas.

• 6:01Martin Casado?

  Many of us have been in computer science for a long time, right? We've been in our schools and in our operating lives and in our investing lives. And like, software was always the disruptor, right? Like we disrupt the taxis or we disrupt like sales, we disrupt the back office or we disrupt everything. Like, one of the most exciting thing about the AI wave is like, software is being disrupted. Like we're being disrupted, right? Yes, disrupting software and eating software. I know, I know. Like, ah, so we have to think about.

• 6:28Erik Torenberg?

  Software is a self-eating ice cream.

• 6:30Martin Casado?

  Yeah, that's right. Honestly, I think this is the first time I could honestly say that like, the profession that I've dedicated my entire life to is being disruptive. And it's very exciting because it's eating itself in a way.

• 6:39Erik Torenberg?

  Yeah.

• 6:40Matt Bornstein?

  Yeah. And it's tempting to be a curmudgeon, right? Because we've all been doing this our whole lives. So it's like really being open to and embracing the new stuff is like the key thing.

• 6:48Jennifer Li?

  What still applies is infrastructure never goes away, it just gets layered. In this case, it's definitely all the infrastructure that I've been using and leveraging in the past are still very relevant. But it's definitely getting layered by having this fourth pillar, which is AI and models.

• 7:01Erik Torenberg?

  What's different from past supercycles versus what can we learn as we enter this new one?

• 7:06Martin Casado?

  So there's two things that happen. So one of them is often when you bring the marginal cost of something down, like with compute, we did it for computation and for the internet, we did it with distribution, it increases the TAM a whole bunch. So for one, you almost always see this massive TAM expansion. And part of that tends to be because the TAM is bigger, you've got new users. And because you have new users, like there's normally like a new behavior that happens, right? This is very much the case with the internet, which is like, people weren't used to going to a computer and talking to everybody around the world on top of the internet. And existing companies don't know really how to think about new behaviors. Like, they've built these sales motions and operating things

• 7:48Martin Casado?

  around like the old behavior. And so you see TAM expansion, you see new behaviors, those new behaviors provide white space for challengers, new like startup companies to come and to go ahead and fill those models. And I think we're seeing exactly that happen with this one as well. Like, clearly, this market is massive. If you look at how successful these model companies are. But also, you're seeing use cases that like computers just never really have done before, and you're seeing that too. And so in that way, I think it rhymes very much with, say, the internet, and it rhymes very much with probably even the microchip.

• 8:17Jennifer Li?

  Maybe I'll answer that question just from my personal experience. I'm always like a sort of a tools person and also wanting to like having tools fulfilling certain creativity because I came to coding and computer science as a late bloomer after my twenties and really enjoyed all the tools available for me at that point of just building software and like learning computer science as well. Now we just have massive and massive leverage in trying to create anything as long as you have a good idea. Like, Martine laughs at me about this, like I was a bigger like local no code champion for, let's say, the last five, 10 years. Because, again, these are tools for people who have good ideas but may not be like educated in the computer science term. But the retools of the world, the Wix and Squarespaces, like you can build applications, can build software easily with

• 9:01Jennifer Li?

  these tools. But now you're given like the next level of thought partners, tools to... Really, as any role in the company, prototype software interfaces for your end customer, for end user, as long as you know what they need, what they want to see, like you can really realize these ideas really quickly in like at your fingertips.

• 9:19Matt Bornstein?

  So low code's finally happening. It just takes a lot of coding.

• 9:22Jennifer Li?

  It just turns out the code is natural language.

• 9:23Martin Casado?

  I know. It's so funny because when Jennifer joined the team, she was very excited about low code. But from my view, low code is like Python. It's like your scripting language when you write.

• 9:32Matt Bornstein?

  Interpretative languages.

• 9:33Martin Casado?

  NPM is low coding. So we kind of had to bridge that gap. And it was in a way like a bit irreconcilable until AI came out. And it's very, very clearly like what the promise of low code was. And so you're right, it really is disrupting software.

• 9:47Matt Bornstein?

  So when I was a kid, right, the internet was sort of a new thing. I just remember really vividly, there was this movie with Sandra Bullock called The Net, and she orders a pizza from her computer.

• 9:58Martin Casado?

  Yeah.

• 9:58Matt Bornstein?

  And this was like completely mind blowing. And like now, actually, this is a common user behavior. But like, what we're dealing with now is just so much bigger than that, right? I just think it's really hard to like, I think some of these points about how will the infrastructure evolve, like how will the companies adapt and things like that are probably transferable. But this is a pretty big deal. It's by far the biggest thing that I've seen happen sort of in my life.

• 10:17Erik Torenberg?

  Yeah. Let's zoom out and take this long view. And Martina, you're actually the perfect full circle because weren't you the first infra investment ever as a portfolio founder?

• 10:25Martin Casado?

  I think it was either me or Okta. But I will say, me and Todd were like the infra portfolio. Fund one LP days when they would prod us out in front of the LPs. It was like me and Todd from Okta. So for sure, I was one of the first two.

• 10:38Erik Torenberg?

  At what point did we develop a clear infra practice at Aizen Z?

• 10:42Martin Casado?

  So we always had strong infra people, right? So like Ben Horowitz is an infra guy. And honestly, I would say Mark is like he masquerades as a consumer guy, but he's actually revolutionized the way we use computers in this deep infrastructure way. And by the way, many things came from that JavaScript, etc. We had Peter Levine, ZenSource, we had Scott Weiss. And so like, there's always been deep infra. But when I joined the firm, we didn't think of it as infra. We thought of it as enterprise.

• 11:09Erik Torenberg?

  Right.

• 11:10Martin Casado?

  Right? And so you're either in the consumer team, you're in the fintech team, or you're in the enterprise team. And the thing about just classifying this stuff as enterprise, The go to market motions for something that touches technology and being able to reason about that is so different than reasoning through the go to market motion that's purely through like sales, right? And we've just learned over time that like, we do deep market diligence as a firm and we do deep diligence on companies before we invest in them. And that the type of diligence we do of it was deep infrastructure, just required a different type of junior partner and type of analysis. Over time, we realized that companies where you can evaluate more by like the business

• 11:51Martin Casado?

  model, the market buyer, the unit economics, you know, those kind of class of companies are just sufficiently distinct. So we decided to just pull them apart and then that's why we have the apps fund. And then of course, the infra fund.

• 12:04Jennifer Li?

  If you think about enterprise, there is of course the horizontal piece of it, but also a lot of vertical enterprise, whether applications, buyers, sectors where infrastructure is almost always horizontal. And that's another thing we realized along the way of we still want to like appeal to these like technical audience and technical buyers, but we want to think about sort of the space in a horizontal fashion where these technologies can be distributed to all these different sectors and verticals as well. And it impacts how we think about the stack, it impacts how we think about what is going to drive the form of like how the stacks integrate with each other, just like different from enterprise generally.

• 12:38Martin Casado?

  Yeah, so it's actually a very, very important distinction. I'm glad you brought it up. So like technical buyers tend to be centralized buyers in a way, right? Like IT will buy compute network and storage. And yes, you've got the compute person, the networking person, but it kind of rolls up to IT. And developers are kind of a centralized buyer. So like you can understand IT and you can understand the developers. But when I was looking at vertical SaaS apps early in my investing career, to understand how to sell into a flooring company, you have to understand the flooring market. And that's entirely different than like the pet food market and that's entirely different than the construction market. And so I felt like There was no real central software buyer for these kind of verticals. So as you get further away from core infrastructure, you get away from this notion of a centralized educated buyer. And just the level of analysis becomes very different.

• 13:23Martin Casado?

  I think that's exactly right.

• 13:24Matt Bornstein?

  And there's even sort of like the horseshoe theory of software buyers happening now where you have like consumers over here. And then you kind of work up to like apps and like you would think that infra is way over here, but it's actually bending back around. Like developers are making a lot of the decisions and a lot of the like marketing and sales to developers looks more like consumer these days than it used to.

• 13:43Jennifer Li?

  That's an incredibly important point. I was researching the stats the other day. When I first joined venture or maybe even just started thinking about infrastructure, developers deal like in the low tens of millions. And they're becoming like the next generation of consumers. Now they're definitely the next generation of consumers. It's going above 50 million on the way to even bigger than that. Now we're talking about natural language being a programming language for everybody. Like they're making decisions just as a consumer. So we're of course like training ourselves to also understand how developers as individuals adopt tools, but also like IT buying centers as customers and enterprise organizations like evaluating tools. So there's just a lot to learn and figure out, given these are the technical audience we really care about.

• 14:24Erik Torenberg?

  Yeah, I want to get to how we think about our investable universe and what sort of subcategories are mature versus newer or sort of ripe. But before that, I want to better trace the evolution a little bit. So maybe Martin, we can start with you. If you had to categorize, like since you've been investing, like the different waves of info or like maybe the different inflection points at which it changed how we even thought about.

• 14:45Martin Casado?

  Totally.

• 14:46Erik Torenberg?

  How would you characterize?

• 14:46Martin Casado?

  Let's just say for like the lifecycle of the firm. So sort of what 2009 and that was actually pre-cloud. And so a lot of the early investments were right in the cusp of cloud. And like, for example, my company, like the cloud was out there, but it wasn't sufficiently deployed. You could use it as a core thesis and a lot of the early investments were like this. And we talk about it in terms of tech, but it rips through the entire business model like. Early software was on-prem with a perpetual license. That's just different economics and different analysis, right? And so like the pre-cloud installable software area was one. And then we saw the cloud transition, right? So we went to recurring revenue, totally different deploying model, totally different operating model. And during that time, we'd see things like net dollar retention being more important,

• 15:31Martin Casado?

  expansion being more important, gross churn being more important.

• 15:34Jennifer Li?

  Gross margin.

• 15:35Martin Casado?

  Gross margin. Everything changed to that. By the way, meanwhile, of course, we saw consumer being disrupted by mobile, right? With the rise of the Uber's and the Lyft's, the R B and Bs. And then again, we're seeing it happen again during this AI, which I would say the AI transformation of the last three years has been the most dramatic I've seen in the last 30 years of being in this industry. And there's one interesting blip along that path, which was COVID.

• 16:01Martin Casado?

  We come from the realm of like enterprise sales and like being on the ground and deploying things. And like that totally evaporated. So like, actually, that was a dramatic shift as well, but it was driven by this kind of force majeure as opposed to a secular technical wave like the other two did.

• 16:16Jennifer Li?

  I would say one of the benefits COVID brought is there was already the trend going of like developers buying and adopting tools, like this bottom-up motion for infrastructure and for dev tools. It got accelerated during the COVID time where lots of people are just like tinkering, trying out products and tools, and also more people building dev tools during that time. I feel like there was a flourishing ecosystem just of these like PLG or product-led dev tool companies that are giving birth to now where we're seeing a lot of like AI dev tools are coming up.

• 16:45Matt Bornstein?

  There's nothing developers like better than being forced to stay home. Not interact with other people and write code.

• 16:56Jennifer Li?

  That's news, however.

• 16:58Erik Torenberg?

  That's amazing. Okay, that's a good overview. Let's go deeper into the present a little bit. Can you guys share how we think about the different subcategories or landscape that kind of makes up Infra? We could also plug some examples of portfolio companies, spaces where we made some bets.

• 17:12Matt Bornstein?

  Here's a few important categories. Developer tools, meaning anything developers use to make their lives better, easier, faster, more efficient. Cursor is probably our top developer tool company right now.

• 17:22Martin Casado?

  And before that GitHub, right? I mean, like

• 17:23Matt Bornstein?

  Yeah, GitHub before that.

• 17:24Martin Casado?

  So we've seen very much.

• 17:26Matt Bornstein?

  And Jennifer, you've actually backed a bunch of interesting dev tools companies in the last few years, too.

• 17:30Jennifer Li?

  Yeah, from Monify, Stenless.

• 17:32Erik Torenberg?

  And there was a long time where the dev tools was written off by VCs.

• 17:35Martin Casado?

  Oh, yeah. The TAM was too small.

• 17:37Erik Torenberg?

  Yeah.

• 17:38Martin Casado?

  I mean, at the time of GitHub, would you ever have imagined a repository would be a huge company? It was like almost a joke. Yeah.

• 17:44Erik Torenberg?

  Right. And people were out there about the business model and all these things.

• 17:47Matt Bornstein?

  Small TAM is the classic, like, red flag for infra investing. If you're at home listening to this and someone tells you, "Small TAM, run."

• 17:53Martin Casado?

  They're not an infra investor. Infra creates TAM. If there's one takeaway from this thing is TAM creative. So, yeah, so like, I think dev tools, you've got core infra, which is computing, network and storage, right? This is like to IT. And then you tend to actually be quite a bit above the core infra stack. So maybe you talk through kind of the areas you focus on.

• 18:11Jennifer Li?

  I think about both, like, how developers are, like, using tools to improve their efficiency, but also how customers are getting value out of that as well. So a lot of packaging, maybe dev tools into SaaS forms, like, I mean, I've invested in this company called Pylon. It is a SaaS company that does customer support, but fundamentally is like being a data pipeline. So that's like core infrastructure that's very good at connecting with systems and providing context to a lot of, like, agents, AI models. So to me, that's infrastructure. And we're spending a lot of time on, of course, like, the cutting-edge AI research. So a lot of foundation model investing. And we can probably enumerate for the next 10 minutes of.

• 18:45Martin Casado?

  I will say, the reason that we're a little bit skittish on this question is, early in supercycles, it's very hard to distinguish between an infra company and the application companies. And the reason is because the TAM is so small and so new, the new technology becomes the app, right?

• 18:58Jennifer Li?

  Right.

• 18:59Martin Casado?

  So let me refer to, like, the original supercycle that started the firm, which was the internet. Like Netscape, I remember it when it came out. Like, it was a consumer thing, or at least a student in school thing, right? And so this is one company, which everybody was downloading from an FTP server, Netscape, and using it as individuals. The enterprise didn't know what to think about it and banned it or whatever. And the same company that built JavaScript, that's building this core technology, is also doing the browser. And over time, it matures. And then, of course, you have all of these internet companies, then all the applications show up. We're seeing the same thing in the AI wave. Like, is Midjourney? Is that an infra company? They build the model or is that an app company? Well, it's both in this sense. And so I do think that at this stage, it's very hard to distinguish.

• 19:43Jennifer Li?

  It's exactly right. Like, it's very hard to answer if OpenAI is a app company or is it an infra company? Literally, it's building like infrastructure that's like a cloud running these models for different sector and different use cases, but at the same time, building a consumer app that's chat GPT. We think of foundation model companies are similar, like 11 Labs. They're a voice AI provider and they have the creator application that can use a studio to create voices. But at the same time, they're also supplying the voices to these large scale enterprise use case that are like fine tuning, cloning your own voice and distributing that through API. So it's sort of both. Yeah.

• 20:17Matt Bornstein?

  Another area we've done a lot of work is data systems. And this goes all the way back to Databricks. And there've sort of been these two branches. One is this like kind of backend data edge driven, big data systems, spark, doop sort of thing. And the other is this sort of data analyst, more tabular, snowflake kind of thing. And I think as a firm, we've been super, super active and super aggressive, you know, invested in companies like Databricks, I mentioned, Five Tran D B T, which you guys invested in together, Hex, which is doing really, really well, Tabular, which was acquired by Databricks. So we're still, I think, really, really bullish on this. Unfortunately, AI had sucked the air out of the room for a lot of data companies from a bunch of different angles. But I think we'll continue to do more of this as well.

• 20:57Erik Torenberg?

  How do we think about defensibility for AI companies? Whether it's the app layer or the model layer? Do they all have their respective sort of areas of defensibility? Or how has sort of our notion of defensibility evolved?

• 21:08Matt Bornstein?

  So we once wrote a blog post that there was no defensibility anywhere in the stack.

• 21:12Martin Casado?

  For anything.

• 21:13Matt Bornstein?

  For anything.

• 21:14Martin Casado?

  And yet people make lots of money.

• 21:19Matt Bornstein?

  You know, the argument at the time was like, okay, like Nvidia has sort of a moat because chip designs are hard to copy. But if you go sort of up or down, it's kind of like, okay, they're all sort of manufactured at the same place, right? At TSMC, if you go down, if you go up, the cloud providers provide effectively the same product. Like the models are training on the same data and had of like similar capabilities. The apps are all kind of like all using the same model. So that was sort of the naive theory, I think, when we were just trying to understand this in the beginning. And I think maybe it was true at the time. What Martine was sort of alluding to a second ago is meanwhile, every company at every layer of the stack is doing like fantastically well right now. And During this kind of initial phase of industry development, which we sometimes call the Brownian motion phase, I actually think it's really hard to make sort of pronouncements like this

• 22:04Matt Bornstein?

  about what's going to work, what's not going to work, where is value going to accrue, et cetera, et cetera. App companies are doing really, really well. And we're pretty clear past the wrapper phase. I don't think there are any wrappers anymore. Building good products with AI is really hard. And the founders doing it now have really good intuition for how to do it. The models are clearly pushing the whole industry forward. And they've built huge companies from that. So it's kind of all working right now. And I think you could actually make a case for how defensibility will work. It's quite different from the way defensibility has worked before. And maybe you guys want to add on to that.

• 22:34Jennifer Li?

  Yeah, on how defensibility worked for infra companies before, largely, it's really hard to, let's say, if you're building a new database, like a new framework, it just takes a lot of expertise in the domain of understanding what has happened in the past, where did the field come through? And what are the innovations that need to happen to polish, let's say, a software into this new abstraction to provide to developers? Maybe one example I'm thinking of is DuckDB. It's such a high-performance, small, but really nimble database. It took four years for the team to write it. To replicate that is really hard. And that's generally what happened in the past. Infrastructure space, it takes these experts a lot of time to build a new piece of security software,

• 23:19Jennifer Li?

  if it's UBI keys, or if it's, again, like data breaks on Spark. But now, as the AI infrastructure comes through, I think a lot of those defensibilities still stays and are true because these are the secrets of what are the downfalls and guarantees other software run into, where these founders know the past and also looking into the future. But I do feel like the adoption phase is just really massive. Who is going to earn the distribution? And earn the developer attention is going to be a different game.

• 23:48Martin Casado?

  Can I just do a quick mental model? What has suited me very well in thinking about this? I think the sloppiest thinking in our industry is around defensibility. It's just, we're just so sloppy about it. And so maybe this is worth the price of listening to this podcast, like this mental model. The industry tends to go through these expansion and contraction phases. Think of like the big bang or something like it expands and then it contracts. So what happens when it expands? When it expands, zero sum thinking is deadly because you're just getting more market. We're clearly in an expanse phase. Everybody is, oh, Nvidia can't like sell more chips, but they keep selling more chips. Oh, like the hosting platforms can't continue to get margins yet. They keep continuing to get margins. I mean, Matt said it perfectly. I totally agree. So if you're in the expansion phase, then there's just more to sell. You should be aggressive investing.

• 24:32Martin Casado?

  So what happens in the collapse phase? So look at any layer of the stack. So in the collapse phase, which things start to consolidate again, you have consolidation. But what is the end state of consolidation? The end state of consolidation will always be an oligopoly or a monopoly. It's not like layers ever go away. If you have an oligopoly, say the clouds, then you have, which is effectively price fixing, but it's tacit, right? Which is everybody is like, we're going to price at this and we're going to maintain our 30% margins. So you still have value there. You have margins. Or in the case of like a monopoly, you'll end up with, say, like an Intel at the time. Then you can also maintain margins. So in none of this, do you lose margins, right? And I just think this is why people think so sloppy about this.

• 25:14Martin Casado?

  People use the words like commoditization and no defensibility. That tends to be a battle between layers of the stack. But the only way you can do that is actually move down the stack and enter somebody else's layer, which is incredibly hard to do. And you do see it like, so of course, Google is going to build their own chips. So they start moving down the stack, but that's a very, very different layer than somehow Google playing the different layers off against each other. And so I would encourage anybody that does invest at least in infrastructure to not think zero sum and to realize that historically, every layer of the stack has maintained some level of value and margin. And if not, it was because a layer above them managed to verticalize themselves. But then it's that one player against the rest of the world.

• 25:58Matt Bornstein?

  Yeah, I mean, it's like faster than light speed travel has just been invented, right? And we're sending all the spaceships out in all directions and there's plenty of planets like in stars to claim for everybody. Like, you know, we're not even close enough to each other to like fight.

• 26:09Martin Casado?

  For the AI wave, yeah, for sure. Like this is.

• 26:11Matt Bornstein?

  Yeah.

• 26:11Martin Casado?

  But it will, it will, it will slow down and then the consolidation will happen. But I guarantee you'll just end up with these great companies that maintain margin. Like AWO still has great margins. Google still has great margins.

• 26:22Jennifer Li?

  And still growing at insane speed.

• 26:24Matt Bornstein?

  Databricks too is growing at incredible speed.

• 26:26Martin Casado?

  Yeah, that's good.

• 26:27Matt Bornstein?

  Scale. You know, I think people underestimate how hard these problems are in many cases, right? You're sort of applying consumer thinking because this is how we live most of our lives. It's like, oh, wouldn't it be relatively easy to move to a different part of the stack? Or take out your competitor or someone, a customer could just switch back and forth. And it's just different laws of physics, I think, in.

• 26:43Jennifer Li?

  It just turns out in general, the switching cost of infrastructure piece is so much higher. Even with like API business, people tend to think you can just switch over to another API. There's so much logic embedded in calling the API in the software itself. There's a lot more switching cost compared to your regular SaaS or a consumer.

• 27:01Matt Bornstein?

  Yeah, totally. Because you're actually integrating systems, right? It's not necessarily a person who can just have a preference for one thing or another. You're sort of integrating.

• 27:06Jennifer Li?

  For sure.

• 27:07Erik Torenberg?

  Sam Altman once had the advice to startups last year. He was like, if you're worried about us improving our models, you're in a tough spot. But if you get more excited about your business by us sort of improving our models, then you're in a good spot. Do you think that's a helpful framework?

• 27:21Martin Casado?

  I think it's helpful for OpenAI.

• 27:24Matt Bornstein?

  No comments.

• 27:26Martin Casado?

  I want to say that too. If, you know, A16Z, if you think us investing in this company isn't good for you, then. If you want to buy from our customers, our companies, that's great. There's a very open question. That's actually a technical question. It isn't a business question, which is how much does general training generalize, right? So we know in the pre-training world. It generalized really well. So you'd create one model and that model is just as good at code as it was at like writing a poem or right. So we know that it was very general. And in that world, sure, as the models get more powerful, then they can do all of the things. So they compete with all of the things, right? But it seems clear to me, and again, this is an observation and it may not be correct, that as we get

• 28:10Martin Casado?

  more into the RL world, that you make some trade offs. And then let's say I RL something for code, it's not going to be as good as something else. And like you're making these trade offs. And in that world, then it's not the case that the model is going to generally be good. So I think it's great to compete at the model layer. And so again, I think this is maybe a reasonable rubric, certainly for OpenAI to have people believe, maybe a reasonable rubric if you believe that these models are going to be generally great. But I just don't think it holds up to how things are going to play out.

• 28:39Jennifer Li?

  This was a debate we had two years ago. I think whether the general model and the most capable model will rule, or a lot of small, medium sized models that are very good at specific tasks, that's going to be the future. It turns out both are true.

• 28:49Martin Casado?

  Both, yeah.

• 28:50Jennifer Li?

  When we're talking about complex systems, you cannot just use one model that drives everything, at least not today. But you can compose very capable and powerful models to like take certain tasks and also chain together processes from like processing document to like feeding in a model to have some reasoning and give you back really clean and structured data to make decisions and put into your application to serve end users. Like that's a complex system that invokes many model calls instead of just like one big model's task.

• 29:18Erik Torenberg?

  Did you guys have any reactions or is it worth talking at all about Karpati's talk? Did that framing resonate with you? Did you have any sort of differences of how you would frame certain things? One thing he mentioned is that he thinks it's not the year of agents, but the decade of agents, perhaps it's not as immediately upcoming as we might have thought.

• 29:35Jennifer Li?

  Probably should have talked about it more.

• 29:38Martin Casado?

  There's something he said recently that I thought, and he's actually responding to somebody else that I thought is actually very insightful. I actually thought his talk was great, but I thought his talk was kind of an overview of what, you know, a lot of people know. I thought it was pretty good, general overview. I saw the tweet today about who knows how old it was. So there's this idea of prompt engineering that people have talked about. And somebody, it wasn't Karpati, but Karpati piled on top is this, it's really not about prompt engineering, it's context engineering. And so what is context engineering? So if you're going to call a model, you have to know what to put in the context, in that prompt. And what tools do you have to do that? Well, you could use other models, but at some point, you're probably going to use traditional computer science. You're going to use things like indexes. You're going to have to do prioritization, et cetera.

• 30:22Martin Casado?

  And to really drive the best performance out of those models, you do want the context to be correct. And I do think it's probably the right framing of this problem. And the next step is in as much as we're going to provide formalism to how to use these models, to how to use existing tools, to how do you improve the performance? You should be thinking about what's the right way to get the right context into those models. And I bring this up because, like we said before, new infrastructure pieces create new patterns and new methods of software and building systems. And this is a great example of that kind of emerging before our eyes and people reasoning about it. And I truly believe in five years, we'll look back, we'll come up with a whole new set of formal ways to build software, and they will have strong guarantees, and we'll understand them, and there'll be all the tools for it, et cetera.

• 31:07Jennifer Li?

  The way I think that relates to our world is if you think about, what is the new form factor of infrastructure that needs to become part of this context engineering? It goes back to a lot of, well, we're obsessed about it with data pipeline. How do you feed the right data and context into the models or into the context? And how do you have agencies' tools or the infrastructure that will provide a discovery and guarantees the visibility of these tools as well? Like, it's the classic infrastructure problem that's still unsolved. So it's a very exciting time.

• 31:35Matt Bornstein?

  There's a few different types of infrafounders, right? There's the sort of infrafounder who loves solving really messy, long-tail, like nasty problems, right? There's the type that kind of just gets fed up with a problem and they're like, "I'm going to solve this finally. I'm sick of this." And there's the type that just sees the world in a new way, right? And it's kind of like, "This is actually how we should marshal these resources." And like, React is a great example of this. We had all these like, kind of progression of front-end development frameworks. And finally, React was like, the way that's stuck. And for years now, it's been sort of the default front end. So like, I think what Karpathy's talking about is trying to figure that out, right? And I think his Software 2.0 thing was really interesting. We were investing in a bunch of like, traditional ML companies at the time. And I think Software 3.0, I think he's sort of right about that too, in sort of directionally.

• 32:20Matt Bornstein?

  And my hope is it'll inspire a lot of new infrafounders to do this work, see the world in a new way, and figure out how these primitives should really be arranged.

• 32:28Martin Casado?

  One of the difficulties of having any conversation around AI is it just exploits this weakness in the human imagination to dump all of our fears and hopes and dreams into this anthropomorphic fallacy, right? And this goes all the way back to the Promethean legend. And so, let's talk about even this context, right? We're building systems to build other systems. Those systems have constraints, right? And so, you can fail on either side of this when it comes to this anthropomorphic fallacy. On one side, you can be like, this stuff doesn't work, you shouldn't use it, you should only use traditional things, which, okay, that's clearly not the case, it seems very useful. But on the other side, you can like, believe they'll solve all of our problems, and you don't need formalism, and you just kind of like, go to the beach, and you come back,

• 33:11Martin Casado?

  when AGI is done, it'll do it for you, type thing. And so, a part of our job, and what we spend a lot of time talking about, is trying to find that pragmatic, non-blankered, non-pessimistic middle, despite all of the rhetoric. And then you hear all the rhetoric, right? The stuff is gonna like, no, we're not gonna have to work, and we're all gonna be on... The beach. Right? Or, you know, like, it'll kill us, like, the whole thing. And I think where we've landed is, this is a real disruption, it is changing all of software, it'll look something, place different, but it is still gonna require professionals. And I do think that the statement, "it'll require professionals," is a very meaningful one. It means that you actually still need people that understand the specifications of the systems.

• 33:56Martin Casado?

  And not everybody agrees that. Some people are out there like, "Listen, you will never need a programmer again, because people are beginning to just say some high-level thing, and it'll show up." And the only one statement I'll say to that is, formal systems came out of natural languages for a reason. And like, either you care about specifying what you're designing, or you don't, and if you do, you need to be a professional. And that's why every professional discipline, even though they started with a natural language, has ended up with a formal system.

• 34:23Erik Torenberg?

  What is your mental model on coding specifically? Will there be like fewer engineers who are just higher powered?

• 34:29Matt Bornstein?

  I think the best way to think about this is simply that we're going to have more developers. I think it's very unlikely that we're going to shrink development teams because we have amazing new tools. That's just not how these markets have worked in the past. I think exactly the opposite is going to happen. It's like, we're going to be creating so much great software. It's going to be so accessible to so many people who may work at a big company or may just be sort of hacking on the weekend. I think that's by far the simplest way to think about it. And it gets back to Martin's point of you can't anthropomorphize these models. A model is a file on a hard drive in a computer somewhere. And when you run a Python script, you can transform one piece of data into another piece of data. Like, that's what this is. And programming is a fundamentally creative job, right? You are literally creating things in the most strict sense of the word,

• 35:14Matt Bornstein?

  which is you're creating software that doesn't exist before. And that's something only a person can actually do at some level of abstraction. So I personally think this is a huge boon for programmers. And you have to change the way that you're working. And it's like a huge productivity boost. And I think that creates more not less.

• 35:30Jennifer Li?

  I cannot agree more. I feel like for many of the developers and programmers I talked to today, they're like going to Disneyland just because how many great tools there are to help them move faster, like build things they have always wanted to do on both side projects and also their main job. I do think it also changes the dynamic of like how people are picking up new languages, picking up new frameworks. And it is, again, a next level of iteration speed, given what we're seeing with AI agents and also AI coding tools.

• 35:59Martin Casado?

  Here's another, I think, useful mental model on this stuff. Like, I think it's worth asking the question, why do people buy software? Like, why does someone buy some random SaaS tool? Is it because it's so hard to build it? No, like, most SaaS tools are like CRUD. Like, they're just these basic kind of read write databases. They're all the same. So why do people buy them? And Aaron Levy, who's the CEO of Box, I think, said this so beautifully. The reason people buy software is because somebody else made the decisions of what the workflow should be and what the operational logic should be and what data is important, how you use that data is important. Like creating a product is a lot of understanding what is being used and guiding the user along that direction. So if not, I just give you a compiler. Like you do whatever you want, or I'll just

• 36:44Martin Casado?

  give you a database and you do whatever you want. There's a reason that we have a proliferation of vertical SaaS and it is this kind of articulation or this transfer of domain understanding. And that just doesn't go away independent of how you create the software. And so we will still need to design products based on whatever problem is being solved, guide people so that they're the most effective with them and they can understand the best. And we did this with assembly before and then we did it with high level languages and then we did it with high level frameworks and then we're going to do it with AI. But like the fundamental process of that articulation will not go away.

• 37:19Matt Bornstein?

  And it's totally orthogonal to creating the software itself, right? It turns out to be a much harder problem to go out and collect requirements from an unknown set of users with an unknown set of needs. And like figuring out what to build, that turns out to be much harder than actually building it.

• 37:31Martin Casado?

  So what do you think is the average number of lines changed in the PR in the industry?

• 37:36Matt Bornstein?

  Two.

• 37:36Martin Casado?

  Yeah. But it shows you to the point. Like it literally is understanding the need from the business and the need from the user and making some minor tweak. That is the long tail that goes into software. And by the way, it turns out it's two. Well, I think it's the median. It's the median is two.

• 37:53Matt Bornstein?

  I've written a lot of like GTRs in my life. Just for the record, by the way, Martine said crud. Crud is a technical term, right? Create, read, update, delete. We actually think applications are great. We don't think they're crud.

• 38:04Martin Casado?

  Yeah, crud is a.

• 38:05Matt Bornstein?

  Just hit that footnote.

• 38:07Erik Torenberg?

  Good to know. Jennifer, you were mentioning earlier how two years ago, we were having this debate on generalization and what are the debates we're having now? Internally or with your peers? Or what are the main questions that we're asking? That we can't wait to see how they're going to reveal themselves in the next few months or next year that are going to impact our business?

• 38:24Jennifer Li?

  Oh gosh, every week is different. What are some recent ones? Definitely like how realistic are agents today? Like really producing production level software? That's more on the coding agent side, but also in general, like the agent. Evolution, where are we in going from demo wear to producing real value, tangible value? What are some other ones?

• 38:43Matt Bornstein?

  Synthetic data is one we talk about a lot. We've been talking about for 10 years. This is what's so great about infra. You can have the same debates for 10 years and they never quite go away. It's like change the background. It's like everything else is the same. Consistency versus availability. Literally every system has those trade offs. So the synthetic data thing, right? It's almost an information theory question. It's like, can you make models meaningfully better without introducing new information to the system? And I think it's now pretty clear you can do a little bit. But the question is, does this lead to sort of like a self improving utopia of models or not? And I think we have some pretty strong opinions on the not side of that. Generalization, Martine mentioned, is a pretty interesting one. If you train a model to be really good at math, does that mean it's going to be really good at other things? Or is it just like really good at math? Which I get excited about. Not everybody gets excited about this.

• 39:27Martin Casado?

  Another one that we talk a lot about is like what these things are actually good for. Just because the path we came from used a lot of AI, pre-gen AI, used a lot of AI. So there's a lot of AI shaped holes in the enterprise, like chatbots and this and that. And that's very much on the brain. And it seems to sometimes confuse the discussions from the new use cases we're seeing. Like if you look at the most common use cases of something like ChatGPT, I think the top one is like companionship and therapy. And then it's like managing my schedule. It's like the top of the pyramid of need stuff. And then number five is professional development, not like low code development or whatever. And so I think what is happening is we have this idea of what we thought AI was going to do.

• 40:07Martin Casado?

  And like just kind of the stilted attempts previously, and then Jennifer actually ran product for a chat company prior. And then what it really is good for, and clearly there's some overlap and convergence, but it's not nearly as big as people say. And so we do spend a lot of time just trying to be very honest with ourselves. Or what is the new behavior being created? Where is this stuff getting used? What is it just being trying to cram into places it's not actually quite good at?

• 40:30Erik Torenberg?

  To that point, how do we think about agents right now? What are they good for, going to be good for soon enough? What is the state of them? How do we think about the broader conversation?

• 40:38Martin Casado?

  Coding agents are awesome. They're amazing.

• 40:41Matt Bornstein?

  It's really amazing. So I have a very simple way to think about this. And I'm like the anti-agent guy, by the way. I think it's kind of a marketing thing. This is the other thing about infra people. We are allergic to marketing, which is not always a good thing. Which is a good thing you're here, Eric.

• 40:54Jennifer Li?

  What do you mean?

• 40:54Matt Bornstein?

  Yeah, yeah, exactly. Can you explain? If you take the simplest definition, that is basically an agent is an LLM running in a loop. A very simple way to think about this is errors propagate throughout the loop, right? So if you have a small error, it gets worse and worse. And this is why a lot of agents doing, say, general web browsing don't perform very well yet. On the flip side, if you have a way to correct those errors in the loop, which is one thing that you have in code, right? You can lint, you can interpret, you can try to compile and things like that. You actually do see good performance over time. So that's a very simplistic and maybe not quite the right way to look at it. But if you can do this kind of error correction, I think you're seeing a lot of improvement from this iterative approach.

• 41:32Martin Casado?

  I mean, it's incredible. I'm actually on the GitHub mailing list of a lot of the companies that I work with, just mostly for interest. And I have, even in the last week, seen a bunch of cursor agent commits. And even in the last 24 hours, the Slack integration, seeing it come from Slack. And so I do think for kind of bite-sized tasks you can articulate very well, we're starting to see them really work. So in the coding space, I would say I'm a convert. But to Matt's point, like the, you know, go wander out in the woods and bring back a bear. I think we're...

• 42:09Matt Bornstein?

  There are a lot of errors you could run into.

• 42:13Erik Torenberg?

  Will we see more vertical integration or more horizontal specialization?

• 42:16Martin Casado?

  You know, historically, we've seen both. And what's interesting is we're already seeing both now, right? Apple, of course, has just been historically vertically integrated. Microsoft and Intel historically horizontally. And often, companies will start horizontal and then go vertical. So Google is horizontal, built on top of normal servers. But then they built their servers and then they built their own chips. They built their own networking gear. And so I think you always get a mix of the two. What's interesting about now is we're actually really seeing both. I would say that OpenAI is very much a vertically integrated company now with ChatGPT driving a lot of it. I would say Anthropic, a lot of the usage really is more horizontal. And they're doing a great job of that. I think we're seeing this on the model layer too. Very interesting discussion we haven't had, but it's a very interesting one is like,

• 43:00Martin Casado?

  open source quote unquote really seems to work with these models just because you can't as a user recreate it. So if you look at BFL, they've done a great job building like a horizontal layer for these models. But then you've got companies like Ideogram, which have built a great kind of vertical experience as well. And so I would say for AI, we've got already this early on, great examples of both. And I don't see any reason that that will change.

• 43:22Speaker 03

  Yeah.

• 43:23Jennifer Li?

  I think from the business front, it just poses new interesting questions and challenges too of how do you capture the value? Of course horizontally can capture the value through being able to address every single use case by providing that to developers or enterprises. But if you're vertically integrated, you kind of have to pick the lane of do I want to focus on image models? Do I want to focus on graphic designers? Do I want to focus on, you know, people who are generating photographies? You have to understand the market and the user personas use cases pretty well to capture the maximum value where you probably can take a easier path to just provide API so everybody can use it.

• 43:54Speaker 03

  I think this is a great place to wrap. Guys, thanks so much for coming on the podcast.

• 43:57Martin Casado?

  Thank you for having us.

• 43:58Jennifer Li?

  Thank you. This is fun.

• 44:02Speaker 03

  Thanks for listening to the A16Z podcast. If you enjoyed the episode, let us know by leaving a review at ratethispodcast.com/a16z. We've got more great conversations coming your way. See you next time.