The State of AI with Marc & Ben

In this latest episode on the State of AI, Ben and Marc discuss how small AI startups can compete with Big Tech’s massive compute and data scale advantages, reveal why data is overrated as a sellable asset, and unpack all the ways the AI boom compares to the internet boom.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Marc Andreessen?

  Most of the content created on the internet is created by average people. And so kind of the content on average, you know, as a whole, on average, is average.

• 0:09Ben Horowitz?

  The test for whether your idea is good is how much can you charge for it? Can you charge the value? Or are you just charging the amount of work it's going to take the customer to put their own wrapper on top of OpenAI?

• 0:22Marc Andreessen?

  The paradox here would be the cost of developing any given piece of software falls, but the reaction to that is a massive surge of demand for software capabilities.

• 0:31Ben Horowitz?

  And I think this is one of the things that's always been underestimated about humans is our ability to come up with new things we need.

• 0:41Marc Andreessen?

  There's no large marketplace for data. In fact, what there are is there are very small markets for data.

• 0:47Steph Smith?

  In this wave of AI, big tech has a big compute and data advantage. But is that advantage big enough to drown out all the other startups trying to rise up? Well, in this episode, A16Z co-founders Marc Andreessen and Ben Horowitz, who both, by the way, had a front row seat to several prior tech waves, tackle the state of AI. So what are the characteristics that'll define successful AI companies? And is proprietary data the new oil? Or how much is it really worth? How good are these models realistically going to get? And what would it take to get 100 times better? Mark and Ben discuss all this and more, including whether the venture capital model needs a refresh to match the rate of change happening all around it.

• 1:31Steph Smith?

  And of course, if you want to hear more from Ben and Mark, make sure to subscribe to the Ben and Mark podcast. All right, let's get started.

• 1:41Ben Horowitz?

  It is kind of the darkest side of capitalism when a company is so greedy that they're willing to destroy the country and maybe the world to like, just get a little extra profit. And they do it like the really kind of nasty thing is they claim, oh, it's for safety. You know, we've created an alien that we can't control, but we're not going to stop working on it. We're going to keep building it as fast as we can, and we're going to buy every friggin' GPU on the planet. But we need the government to come in and stop it from being open. This is literally the current position of Google and Microsoft right now. It's crazy.

• 2:15Marc Andreessen?

  The content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investor or potential investors in any A16Z fund. Please note that A16Z and its affiliates may maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com/disclosures. Hey folks, welcome back. We have an exciting show today. We are going to be discussing the very hot topic of AI. We are going to focus on the state of AI as it exists right now in April of 2024, and we are focusing specifically on the intersection of AI and company building. So hopefully this will be relevant to anybody working on a startup

• 3:00Marc Andreessen?

  or anybody at a larger company. We have, as usual, solicited questions on X, formerly known as Twitter, and the questions have been fantastic. So we have a full lineup of listener questions, and we will dive right in. So, first question, so three questions on the same topic. So, Michael asks, in anticipation of upcoming AI capabilities, what should founders be focusing on building right now? Gwen asks, how can small AI startups compete with established players with massive compute and data scale advantages? Then Alistair McLei asks, for startups building on top of OpenAI, et cetera, what are the key characteristics of those companies that will benefit from future exponential improvements in the base models versus those that will get killed by them? So, let me start with one point, Ben, and then we'll jump right to you. So, Sam Altman recently gave an interview, I think maybe

• 3:43Marc Andreessen?

  Alex Friedman or one of the podcasts, and he actually said something I thought was actually quite helpful. Let's see, Ben, if you agree with it. He said something along the lines of, you want to assume that the big foundation models coming out of the big AI companies are going to get a lot better. So you want to assume they're going to get like 100 times better. And as a startup founder, you want to then think, okay, if this current foundation models get 100 times better, is my reaction, oh, that's great for me and for my startup, because I'm much better off as a result? Or is your reaction the opposite? Is it, oh shit, I'm in real trouble? So, let me just stop right there, Ben, and see what you think of that as general advice.

• 4:17Ben Horowitz?

  Well, I think generally that's right, but there's some nuances to it. Right. So I think that from Sam's perspective, he was probably discouraging people from building foundation models, which I don't know that I would entirely agree with that and that a lot of the startups building foundation models are doing very well. And there's many reasons for that. One is there are architectural differences which lead to how smart is the model? There's how fast is the model? There's how good is the model and the domain? And that goes for not just text models, but you know, image models as well. There are different domains, different kinds of images that respond to prompts differently.

• 4:58Ben Horowitz?

  If you ask Mid Journey and Ideogram the same question, they react very differently, you know, depending on the use cases that they're tuned for. And then there's this whole field of distillation where, you know, Sam can go build the biggest, smartest model in the world, and then you can walk up as a startup and kind of do a distilled version of it and get a model very, very smart at a lot less cost. So there are things that, yes, the big company models are going to get way better, kind of way better at what they are. So you need to deal with that. So if you're trying to go head to head, full frontal assault, you probably have a real problem just because they have so much money.

• 5:41Ben Horowitz?

  But if you're doing something that's different enough or like, you know, different domain and so forth, for example, you know, at Databricks, they've got a foundation model, but they're using it in a very specific way in conjunction with their kind of leading data platform. So, okay, now if you're an enterprise and you need a model that knows all the like nuances of how your enterprise data model works and what things mean and needs access control and what needs to use your specific data and domain knowledge and so forth, then it doesn't really hurt them if Sam's model gets way better.

• 6:21Ben Horowitz?

  Similarly, 11 Labs with their voice model has kind of embedded into everybody. You know, everybody uses it as part of kind of the AI stack. And so it's got kind of a developer hook into it. And then, you know, they're going very, very fast at what they do and really being very focused in their area. So there are things that. I would say like extremely promising that are kind of ostensibly, but not really competing with OpenAI or Google or Microsoft. So I think it sounds a little more coarse green than I would interpret it if I was building a startup.

• 6:58Marc Andreessen?

  Right. Let's dig into this a little bit more. So let's start with the question of, do we think the big models, the god models are going to get 100 times better?

• 7:04Ben Horowitz?

  I kind of think so. And then I'm not sure. So if you think about the language models, let's do those because those are probably the ones that people are most familiar with. I think if you look at the very top models, you know, Claude and OpenAI and Mistral and Llama, the only people who I feel like really can tell the difference as users amongst those models are the people who study them. You know, like they're getting pretty close. So, you know, you expect if we're talking 100x better, that one of them might be separating from each other a lot more. But the improvement, so 100% better in what way? Like for the normal person using it in a normal way, like asking it questions and finding out stuff?

• 7:48Marc Andreessen?

  Well, let's say some combination of just like breadth of knowledge and capability.

• 7:52Ben Horowitz?

  Yeah, like I think in some of them they are, yeah.

• 7:55Marc Andreessen?

  Right. But then also just combined with like sophistication of the answers, you know, sophistication of the output, the quality of the output, sophistication of the output, you know, lack of hallucination, factual grounding.

• 8:05Ben Horowitz?

  Well, that I think is for sure going to get 100 times better like that. Yeah, I mean, they're on a path for that. The things that are, so against that, right? The alignment problem where, okay, yeah, they're getting smarter, but they're not allowed to say what they know. And then that alignment also kind of makes them dumber in other ways. And so you do have that thing. The other kind of question that's come up lately, which is kind of, do we need a breakthrough to go from what we have now, which I would categorize as artificial human intelligence as opposed to artificial general intelligence, meaning it's kind of the artificial version of us. We've structured

• 8:47Ben Horowitz?

  the world in a certain way, using our language and our ideas and our stuff. And it's learned that very well. Amazing. And it can do kind of a lot of the stuff that we can do. But are we then the asymptote or?

• 9:04Ben Horowitz?

  You need a breakthrough to get to some kind of higher intelligence, more general intelligence. And I think if we're the asymptote, then in some ways it won't get 100 times better. It's because it's already like pretty good relative to us. But yeah, like it'll know more things, it'll hallucinate less. On all those dimensions, it'll be 100 times better, I think.

• 9:24Marc Andreessen?

  You know, there's this graph floating around, I forget exactly what the axes are, but it's basically shows the improvement across the different models. To your point, it shows an asymptote against the current tests that people are using. That's sort of like at or slightly above human levels.

• 9:36Ben Horowitz?

  Yeah.

• 9:36Marc Andreessen?

  Which is what you would think if you're being trained on entirely human data. Now, the counter argument on that is, are the tests just too simple, right? It's a little bit like the question people have around the SAT, which is if you have a lot of people getting 800s, you know, on both math and verbal on the SAT, is the scale too constrained? Do you need a test that can actually test for Einstein?

• 9:52Ben Horowitz?

  Right, right, right. It summarizes the tests that we have, and it's great.

• 9:56Marc Andreessen?

  Right. You can imagine SAT that like really can detect gradations of people who have like ultra-high IQs, who are ultra-good at math or something. You could imagine tests for AI, you know, you could imagine tests that test for reasoning above human levels when assumed.

• 10:08Ben Horowitz?

  Yeah, well, maybe the AI needs to write the test.

• 10:11Marc Andreessen?

  Yeah, it needs to write the test. Yeah, and then there's a related question that comes up a lot. It's an argument we've been having internally, which is also, I'll start to take some sort of more provocative and probably more bullish, or as you would put it, sort of science fictiony predictions on some of this stuff. So there's this question that comes up, which is, okay, you take an L M, you train it on the internet. What is the internet data? What is the internet data corpus? It's an average of everything, right? It's a representation of sort of human activity. Representation of human activity is going to kind of, you know, because of the sort of distribution of intelligence in the population, you know, most of it's somewhere in the middle. And so the data set on average sort of represents the average human.

• 10:40Ben Horowitz?

  You're teaching it to be very average, yeah.

• 10:42Marc Andreessen?

  Yeah, you're teaching it to be very average. It's just because most of the content created on the internet is created by average people. And so kind of the content on average, you know, as a whole, on average, is average. And so therefore, the answers are average, right? You're going to get back an answer that sort of represents the kind of thing that an average 100 IQ, you know, kind of by definition, the average human is 100 IQ. It's IQ is indexed to 100. That's the center of the bell curve. And so by definition, you're kind of getting back the average. I actually argue like that may be the case for the default prompt today. Like you just ask the thing, does the earth revolve around the sun or something? You get like the average answer to that and maybe that's fine. This gets to the point is, well, okay, the average data might be of an average person, but the data set also contains all of the things written and thought by all the really smart people. All that stuff is in there, right? And all the current people who are like that, their stuff is in there. And so then it's sort of like a prompting question, which is like, how do you prompt it?

• 11:27Marc Andreessen?

  In order to get basically, in order to basically navigate to a different part of what they call the latent space, to navigate to a different part of the data set that basically is like the super genius part. And you know, the way these things work is if you craft the prompt in a different way, it actually leads it down a different path inside the data set, gives you a different kind of answer. And here's another example of this. If you ask it, write code to do X, write code to sort a list or you know, whatever, render an image, it will give you average code to do that. If you say, write me secure code to do that, it will actually write better code with fewer security holes, which is very interesting, right? Cause it's accessing a different corpus of training data, which is secure code. And then if you ask, you know, write this image generation thing, the way John Carmick would write it, you get a much better result because it's tapping into the part of the latent space represented by John Carmick's code, who's the best graphics programmer in the world. And so you can imagine prompting crafts in many different domains,

• 12:10Marc Andreessen?

  such that you're kind of unlocking the latent super genius.

• 12:13Ben Horowitz?

  Yeah.

• 12:13Marc Andreessen?

  Even if that's not the default answer.

• 12:15Ben Horowitz?

  Yeah, no, so I think that's correct. I think there's still a potential limit to its smartness in that. So we had this conversation in the firm the other day where you have, there's the world, which is very complex and intelligence kind of is, you know, how well can you understand, describe, represent the world? But our current iteration of artificial intelligence consists of humans structuring the world and then feeding that structure that we've come up with into the AI. And so the AI kind of is good at predicting how humans have structured the world

• 12:56Ben Horowitz?

  as opposed to how the world actually is, which is, you know, something more probably complicated. Maybe the irreducible or what have you. So do we just get to a limit where like it can be really smart, but its limit is going to be the smartest humans as opposed to smarter than the smartest humans? And then Kind of related, is it going to be able to figure out brand new things? You know, new laws of physics and so forth. Now, of course, there are like one in three billion humans that can do that or whatever. That's a very rare kind of intelligence. So it still makes the AIs extremely useful, but they play a different role if they're

• 13:37Ben Horowitz?

  kind of artificial humans than if they're like artificial, you know, super duper mega humans.

• 13:45Marc Andreessen?

  Yeah. So let me make the sort of extreme bull case for the hundred, because, okay. So the cynic would say that Sam Altman would be saying they're going to get a hundred times better precisely if they're not going to.

• 13:56Ben Horowitz?

  Yeah, yeah, yeah, yeah, yeah.

• 13:57Marc Andreessen?

  Right? Because he'd be saying that basically in order to scare people into not competing.

• 14:00Ben Horowitz?

  Well, I think that whether or not they are going to get a hundred times better, Sam would be very likely to say that. Like Sam, for those who don't know him, is he's a very smart guy, but for sure he's a competitive genius. There's no question about that. So you have to take that into account.

• 14:18Marc Andreessen?

  Right. So if they weren't going to get a lot better, he would say that. But of course, if they were going to get a lot better, to your point, he would also say that. He would also, you know, why not? Right? And so let me make the bull case that they are going to get a hundred times better or maybe even, you know, on an upward curve for a long time. And there's like enormous controversy, I think, on every one of the things I'm about to say, but you can find very smart people in the space who believe basically everything I'm about to say. So one is there is generalized learning happening inside the neural networks. And we know that because we now have introspection techniques where you can actually go inside and look inside the neural networks to look at the neural circuitry that is being evolved as part of the training process. And, you know, these things are evolving, you know, general computation functions. There was a case recently where somebody trained one of these on a chess database and, you know, just by training on lots of chess games, it actually imputed a world model of a chessboard, you know, inside the neural network and, you know, that was able to do

• 15:03Marc Andreessen?

  original moves. And so the neural network training process does seem to work. And then specifically, not only that, but, you know, Meta and others recently have been talking about how so-called overtraining actually works. Which is basically continuing to train the same model against the same data for longer, you know, putting more and more compute cycles against it. You know, I've talked to some very smart people in the field, including there, who basically think that actually that works quite well. The diminishing returns people were worried about, about more training.

• 15:26Ben Horowitz?

  And they proved it in the new LAMM release, right? That's the primary technique they use.

• 15:31Marc Andreessen?

  Yeah, exactly. Like what one guy in the space basically told me, basically is like, yeah, we don't necessarily need more data at this point to make these things better. We maybe just need more compute cycles. We just trained it 100 times more and it may just get actually a lot better. So.

• 15:42Ben Horowitz?

  One day the labeling, it turns out that supervised learning ends up being a huge boost to these things.

• 15:48Marc Andreessen?

  Yeah, so we've got that. We've got all of the kind of, you know, let's say rumors and reports of various kinds of self improvement loops. You know, they're kind of underway and most of the sort of super advanced practitioners in the field think that there's now some form of self improvement loop that works, which basically is. You basically get an AI to do what's called chain of thoughts. You get it to basically go step by step to solve a problem. You get it to the point where it knows how to do that. And then you basically retrain AI on the answers. And so you're kind of basically doing a sort of a forklift upgrade across cycles of the reasoning capability. And so a lot of the experts think that sort of thing's starting to work now. And then there's still a raging debate about synthetic data, but there's quite a few people who are actually quite bullish on that. Yeah, and then there's even this trade off. There's this kind of dynamic where like LLMs might be okay at writing code, but they might be really good at validating code. You know, they might actually be better at validating code than they are at writing it.

• 16:33Ben Horowitz?

  That would be a big help.

• 16:34Marc Andreessen?

  Yeah, well, but that also means like.

• 16:36Ben Horowitz?

  They can validate their own code. Yeah, yeah.

• 16:39Marc Andreessen?

  They can validate their own code and it's, and we have this anthropomorphic bias that's very deceptive with these things because you think of the model as an it. And so it's like, how could you have an it that's better at validating code than writing code? But it's not an it. What it is, is it's this giant latent space. It's this giant neural network. And the theory would be there are totally different parts of the neural network for writing code and validating code. And there's no consistency requirement whatsoever that the network be equally good at both of those things. And so if it's better at one of those things, right? So the thing that it's good at might be able to make the thing that it's bad at better and better.

• 17:07Ben Horowitz?

  Right, right, right, right, right. Sure, sure.

• 17:08Marc Andreessen?

  Right, sort of a self improvement thing. And so then on top of that, there's all the other things coming, right? Which is it's everything, there's all these practical things, which is there's an enormous chip constraint right now. So every AI that anybody uses today is its capabilities are basically being gated by the availability of chips. But like that will resolve over time. You know, there's also your point on like data labeling. There is a lot of data in these things now, but there is a lot more data out in the world. And there's, you know, at least in theory, some of the leading AI companies are actually paying to generate new data. And by the way, even like the open source data sets are getting much better. And so there's a lot of like data improvements that are coming. And then, you know, there's just the amount of money pouring into the space to be able to underwrite all this. And then, by the way, there's also just the systems engineering work that's happening, right? Which is a lot of the current systems, you know, we're basically, we're built by scientists. And now the really world-class engineers are showing up and tuning them up and getting

• 17:51Marc Andreessen?

  them to work better. And, you know, maybe that's not, maybe that's not a

• 17:55Ben Horowitz?

  Which makes training, by the way, way more efficient as well. Not just inference, but also training.

• 18:01Marc Andreessen?

  Yeah, exactly. And then even, you know, another improvement area is basically Microsoft released their phi small language model yesterday. And apparently it's competitive. It's a very small model competitive with much larger models. And the big thing they say that they did was they basically optimized the training set. So they basically deduplicated the training set. They took out all the copies and they really optimized on a small amount of training data, on a small amount of high-quality training data, as opposed to the larger amounts of low-quality data that most people train on. You add all these up and you've got eight or 10 different combination of sort of practical and theoretical improvement vectors that are all in play. And it's hard for me to imagine that some combination of those doesn't lead to like really dramatic improvement from here.

• 18:35Ben Horowitz?

  I definitely agree. I think that's for sure going to happen, right? Like if you were, so back to Sam's proposition, I think if you were a startup and you were like, okay, in two years I can get as good as GPT-4, you shouldn't do that.

• 18:49Marc Andreessen?

  Right.

• 18:49Ben Horowitz?

  That would be a bad mistake.

• 18:51Marc Andreessen?

  Right, right. Well, this also goes to, you know, a lot of entrepreneurs are afraid of, I'll give you an example. So a lot of entrepreneurs, this thing they're trying to figure out, which is, okay, I really think I know how to build a SaaS app that harnesses an LLM to do really good marketing collateral. Let's just make it very similar, a very, very similar thing. And so I build a whole system for that. Will it just turn out to be that the big models in six months will be even better at making marketing collateral just from a simple prompt, such that my apparently sophisticated system is just irrelevant because the big model just does it?

• 19:18Ben Horowitz?

  Yeah.

• 19:19Marc Andreessen?

  Yeah, how we, let's talk about that, like apps, you know, there are, another way you can think about it is that the criticism of a lot of current AI app companies is their quote unquote, you know, GPT wrappers. There's sort of thin layers of wrapper around the core model, which means the core model could commoditize them or displace them. But the counterargument, of course, is it's a little bit like calling all, you know, old software apps, you know, database wrappers, you know, wrappers around a database. It turns out like actually wrappers around a database is like most modern software. And a lot of that actually turned out to be really valuable. And it turns out there's a lot of things to build around the core engine. So, yeah. So Ben, how do we think about that when we run into companies thinking about building apps?

• 19:51Ben Horowitz?

  Yeah, you know, it's a very tricky question because there's also this correctness gap, right? So, you know, why do we have co-pilots? Where are the pilots, right? Where are the AI? There's no AI pilots. There are only AI co-pilots. There's a human in the loop on absolutely everything. And that really kind of comes down to this, you know, you can't trust the AI to be correct in drawing a picture or writing a program or, you know, even like writing a court brief without making up citations.

• 20:27Ben Horowitz?

  You know, all these things kind of require a human and it kind of turns out to be like fairly dangerous to not. And then I think that so what's happening a lot with the application layer is people saying, well, to make it really useful, I need to turn this co-pilot into a pilot. And can I do that? And so that's an interesting and hard problem. And then there's a question of is that better done at the model level or at some layer on top that, you know, kind of teases the correct answer out of the model, you know, by doing things like using code validation or what have you? Or is that just something that the models will be able to do? I think that's one open question. And then, you know, as you get into kind of domains and, you know, potentially wrappers

• 21:11Ben Horowitz?

  on things, I think there's a different dimension than what the models are good at, which is what is the process flow, which is kind of in database for all to. So on the database kind of analogy, there is like the part of the task in a law firm that's writing the brief, but there's 50 other tasks and things that have to be integrated into the way a company works, like the process flow, the orchestration of it. And maybe there are, you know, a lot of these things, like if you're doing video production, there's many tools or music even, right? Like, okay, who's going to write the lyrics? Which AI will write the lyrics and which AI will figure out the music?

• 21:55Ben Horowitz?

  And then like, how does that all come together and how do we integrate it and so forth? And those things tend to, you know, just require a real understanding of the end customer and so forth in a way. And that's typically been how like, applications have been different than platforms in the past is like, there's real knowledge about how the customer using it wants to function. That doesn't have anything to do with the kind of intel or is just different than what the platform is designed to do. And to get that out of the platform for a kind of company or a person turns out to be really, really hard. And so those things, I think are likely to work, you know, especially

• 22:40Ben Horowitz?

  if the process is very complex. And it's something, it's funny, as a firm, you know, we're a little more hardcore technology oriented and we've always struggled with those, you know, in terms of, oh, this is like a some process application for like, plumbers to figure out this. And we're like, well, where's the technology? But, you know, a lot of it is, how do you encode, you know, some level of domain expertise and, and kind of how things work in the actual world back into the software?

• 23:13Marc Andreessen?

  I often think, I've been to founders that you can think about this in terms of price, you can kind of work backwards from pricing a little bit, which is to say sort of business value and what you can charge for, which is, you know, the natural thing for any technologist to do is to kind of say, I have this new technological capability and I'm going to sell it to people. And like, what am I going to charge for it? And it's going to be somewhere between, you know, my cost of providing it and then, you know, whatever markup I think I can justify, you know, and if I have a monopoly providing it, maybe the markup's infinite. But, you know, it's kind of this, it's, it's sort of technology forward, you know, kind of supplier supply forward, you know, pricing model. There's a completely different pricing model for kind of business value backwards and or sort of, you know, so-called value pricing, value-based pricing. And, and that's, you know, to your point, that's basically a pricing model that says, okay, what's the business value to the customer of, of the thing? And if the business value is, you

• 23:58Marc Andreessen?

  know, a million dollars, then can I charge 10% of that and get a hundred thousand dollars, right? Or whatever. And then, you know, why is it cost a hundred thousand dollars as compared to $5,000 is because, well, because to the customer it's worth a million dollars and so they'll pay 10% for it.

• 24:12Ben Horowitz?

  Yeah, actually, so great example of that. Like, we've got a company in our portfolio, Crest AI, that does things like

• 24:22Ben Horowitz?

  Debt collection.

• 24:24Marc Andreessen?

  Yeah.

• 24:24Ben Horowitz?

  Okay, so if I can collect way more debt with way fewer people with my, you know, it's a copilot type solution, then what's that worth? Well, it's worth a heck of a lot more than just buying an OpenAI license because that OpenAI license is not going to easily collect debts or kind of enable your debt collectors to be massively more efficient or that kind of thing. So, so it's bridging that gap between the value. And I think you had a really important point. The test for whether your idea is good is how much can you charge for it? Can you charge the value?

• 25:04Ben Horowitz?

  Or are you just charging the amount of work it's going to take the customer to put their own wrapper on top of OpenAI? Like, that's the, that's the real test to me of like how deep and how important is what you've done.

• 25:20Marc Andreessen?

  Yeah. And so to your point of like the kinds of, you know, the kinds of businesses that technology investors have had a hard time with, you know, kind of, kind of thinking about, you know, maybe accurately is sort of, it's the company that is, it's a, it's a vendor that has built something where it is a specific solution to a business problem where it turns out the business problem is very valuable to the customer. And so therefore they will pay a percentage of the, of the, of the, of the value provided

• 25:41Ben Horowitz?

  Yeah.

• 25:41Marc Andreessen?

  back to, back, back in the terms, for price for the software. And, and that actually turns, that actually turns out, you can have businesses that are not very technologically differentiated that are actually extremely lucrative.

• 25:51Ben Horowitz?

  Yeah.

• 25:51Marc Andreessen?

  And, and then, and then because, because that business is so lucrative, they can actually afford to go think very deeply about how technology integrates into the business, what else they can do. You know, this is like the story of a Salesforce.com, for example, right?

• 26:02Ben Horowitz?

  And, and by the way, there's a kind of a, a chance, a theory that the models are all getting really good. There are open source models, there are like, that are awesome. You know, Lama, Mistral, like, these are great models. And so the actual layer where the value is going to accrue is going to be like tools, orchestration, that kind of thing. Because you can just plug in whatever the best model is at the time, whereas the models are going to be competing, you know, in a death battle with each other and, you know, be commoditized down to the, you know, the cheapest one wins and that kind of thing. So, you know, you could argue that the, the best thing to do is, is to kind of connect

• 26:47Ben Horowitz?

  the power to the people.

• 26:50Marc Andreessen?

  Right, right. So that actually takes us to the next question, and this is a two in one question. So Michael asks, and these are diametrically opposed, which is why I paired them. So Michael asks, why are VCs making huge investments in generative AI startups when it's clear these startups won't be profitable anytime soon, which is a loaded question, but we'll take it. And then Kaiser asks, if AI deflates the cost of building a startup, how will the structure of tech investment change? And of course, Ben, this goes to exactly what you just said. So it's basically the questions are diametrically opposed because if you squint out of your left eye, right, what you see is basically the amount of money being invested in the foundation model companies kind of going up to the right at a furious pace. You know, these companies are raising hundreds of millions, billions, tens of billions of dollars. And it's just like, oh, my God, look at the, they see these sort of capital, you know, sort of, I don't know,

• 27:33Marc Andreessen?

  infernos, you know, that hopefully will result in value at the end of the process. But my God, look at how much money is being invested in these things. If you squint through your right eye, you know, you think, wow, that now all of a sudden it's like much easier to build software. It's much easier to have a software company. It's much easier to like have a small number of programmers writing complex software because they've got all these AI co-pilots and all these automated software development capabilities that are coming online. And so on the other side, the cost of building an AI like application startup might, you know, crash. And it might just be that like, you know, the Salesforce, the AI Salesforce.com might cost, you know, a tenth or a hundredth or a thousandth amount of money that it took to build the, you know, the old database-driven Salesforce.com. And so, yeah, so what do we think of that dichotomy, which is you can actually look, you can actually

• 28:14Marc Andreessen?

  look out of either eye and see either cost to the moon as like for startup funding or cost actually going to zero.

• 28:21Ben Horowitz?

  Yeah, well, like, so it is interesting. I mean, we actually have companies in both camps, right? Like, like I think probably the companies that have gotten to profitability the fastest, maybe in the history of the firm, have been AI companies or been, you know, AI companies in the portfolio where the revenue grows so fast that it actually kind of runs out ahead of the cost. And then there are like, you know, people who are in the foundation model race who are raising hundreds of millions, even billions of dollars to kind of keep pace and so forth. They also are kind of generating revenue at a fast rate. The headcount in all of them is small. So I would say, you know, where AI money goes.

• 29:07Ben Horowitz?

  And even, you know, like if you look at OpenAI, which is the big spender in startup world, which, you know, we are also investors in is, yeah, headcount wise, they're pretty small against their revenue. Like, it is not a big company headcount. Like, if you look at the revenue level and how fast they've gotten there, it's pretty small. Now, the total expenses are ginormous, but they're going into the model creation. So it's an interesting thing. I mean, I'm not entirely sure how to think about it, but I think like, if you're not building a foundation model, it will make you more efficient and probably gets profitability quicker.

• 29:47Marc Andreessen?

  Right. So the counter, and this is a very bullish counterargument, but the counterargument to that would be basically that falling costs for like building new software companies are a mirage. And the reason for that is this thing in economics called the Jevons paradox, which I'm going to read from Wikipedia. So the Jevons paradox occurs when technological progress increases the efficiency with which a resource is used, right? Reducing the amount of that resource necessary for any one use, but the falling cost induces increases in demand, right? Elasticity enough that the resource use overall is increased rather than reduced.

• 30:20Ben Horowitz?

  Yeah, that's certainly possible.

• 30:23Marc Andreessen?

  Right. And so this is, you see, you see versions of this. For example, you build in your freeway and it actually makes traffic jams worse, right? Because basically what happens is, oh, it's great. Now there's more roads. Now we can have more people live here. We can have more people that, you know, we can make these companies bigger and now there's more traffic than ever. And now the traffic's even worse. Or you saw the classic example is during the industrial revolution, coal consumption, as the price of coal drops, you, people use so much more coal that actually the overall, overall consumption actually increased. But people are getting a lot more power, but the result was, was the use of a lot more coal in the paradox. And so the paradox here would be, yes, the cost of developing any given piece of software falls, but the reaction to that is a massive surge of demand for software capabilities. And so the result of that actually is, although it even, it

• 31:07Marc Andreessen?

  looks like starting software companies, the price is going to fall. Actually, it's going to happen, it's going to rise for the high quality reason that you're going to be able to do so much more.

• 31:14Ben Horowitz?

  Yeah.

• 31:14Marc Andreessen?

  Right? With software, the products are going to be so much better and the roadmap is going to be so amazing of the things you can do and the customers are going to be so happy with it that they're going to want more and more and more. So the result of it. And by the way, another example of Jevons paradox playing out in another related industry is in Hollywood. You know, CGI in theory should have reduced the price of making movies. In reality, has increased it because audience expectations went up.

• 31:35Ben Horowitz?

  Yeah.

• 31:36Marc Andreessen?

  And now you go to Hollywood movie and it's CG, it's wall to wall CGI. And so, you know, movies are more expensive to make than ever. And so the, the result of it, you know, so, but the result to Hollywood is at least much more, let's say visually elaborate, you know, movies, whether they're better or not is another question, but like much more visually elaborate, compelling, kind of visually stunning movies through CGI. The version here would be much better software.

• 31:54Ben Horowitz?

  Yeah.

• 31:55Marc Andreessen?

  Like radically better software to the end user, which causes end users to want a lot more software, which causes actually the price of development to rise.

• 32:01Ben Horowitz?

  You know, if you just think about like a simple case like travel, like, okay, booking a trip through Expedia is like complicated. You're likely to get it wrong. You're clicking on menus and this and that and the other and like, you know, da, da, da, da. An AI version of that would be like, you know, send me to Paris, put me in a hotel I love at the best price, you know, send me on the, the, the best possible kind of airline, airline ticket. And then, you know, like make it like really special for me. And like, maybe you need a human to go, okay, like we're going to, you know, or, or maybe the AI gets more complicated and says, okay, well, we know the person loves chocolate and we're gonna like, you know, FedEx in the

• 32:44Ben Horowitz?

  best chocolate in the world from Switzerland into this hotel in Paris and this and that and the other. And so like the quality, you, you, you can, the quality could get to levels that we can't even imagine today, just because, you know, the software tools aren't, aren't what they're gonna be. So.

• 33:01Marc Andreessen?

  Yeah, that's right.

• 33:02Ben Horowitz?

  Yeah, I kind of buy that actually. I think I brought the argument. You're both nice.

• 33:08Marc Andreessen?

  How about, yeah, or how about I'm gonna land in whatever, Boston at six o'clock. I want to have dinner at seven with a table full of like super interesting people.

• 33:15Ben Horowitz?

  Yeah, right, right, right, right, right.

• 33:17Marc Andreessen?

  You know?

• 33:17Ben Horowitz?

  Yeah.

• 33:18Marc Andreessen?

  Right? Like.

• 33:19Ben Horowitz?

  Yeah.

• 33:21Marc Andreessen?

  You know, no, no, no travel agent would do that for you today, nor would you want them to.

• 33:24Ben Horowitz?

  No.

• 33:26Marc Andreessen?

  No.

• 33:27Ben Horowitz?

  Right. Well, then you think about it, it's got to be integrated into my personal AI. And like, and this, you know, there's just like unlimited kind of ideas that you can do. And I think this is one of the kind of things that's always been underestimated about humans is like our ability to come up with new things we need. Like that has been unlimited and there, there's a very kind of famous case where John Maynard Keynes, who the, the kind of prominent economist in the kind of first half of last century, had this thing that he predicted, which is like, nobody, because of automation, nobody would ever work a 40 hour work week, you know, like, because once their needs were met,

• 34:11Ben Horowitz?

  needs being like shelter and food. And, you know, I don't even know if transportation was in there. Like, that was it. It was over and like, you would never work past the need for shelter and food. Like, why would you? Like, there's no reason to. But of course, needs expanded. So then everybody needed a refrigerator. Everybody needed not just one car, but a car for everybody in the family. Everybody needed a television set. Everybody needed like glorious vacations. Everybody, you know, so what are we going to need next? I'm quite sure that I can't imagine it, but like somebody's going to imagine it. And it's quickly going to become a need.

• 34:48Marc Andreessen?

  Yeah, that's right. By the way, as Keynes famously said, it was his essay, I think, was Economic Prospects for Our Grandchildren, which was basically that.

• 34:56Ben Horowitz?

  Yeah.

• 34:56Marc Andreessen?

  You know, what you just articulated. So Karl Marx had another version of that. I just pulled up the quote. So that the society, when, you know, when, when the Marxist utopia socialism is achieved, society regulates the general production. Thus makes it possible for me to do blah, blah, to hunt in, this is a quote, to hunt in the morning, fish in the afternoon, rear cattle in the evening, criticize after dinner.

• 35:18Ben Horowitz?

  What a glorious life.

• 35:20Marc Andreessen?

  What a glorious life. Like, if I could just list four things that I do not want to do, it's hunt, fish, rear cattle and criticize.

• 35:29Marc Andreessen?

  Right? And by the way, it says a lot about Marx that those were his four things.

• 35:31Ben Horowitz?

  Well, criticizing being his favorite thing, I think.

• 35:35Marc Andreessen?

  Yes.

• 35:36Ben Horowitz?

  Basically communism in a nutshell.

• 35:39Marc Andreessen?

  Yeah, exactly.

• 35:40Ben Horowitz?

  I don't want to get too political, but yes.

• 35:42Marc Andreessen?

  Yes, 100%. And so, yeah, so it's this, this, yeah, do you think what they have, what came with Keynes and Marx had in common is just this incredibly constricted, it's incredibly constricted view of what people want to do. And then, and then correspondingly, you know, the other thing is just like, you know, people, people who want, people who want to have a mission. I mean, probably some people just want to fish and hunt.

• 35:57Ben Horowitz?

  Yeah.

• 35:57Marc Andreessen?

  But, you know, a lot of, a lot of people want to have a mission. They want to have a cause. They want to have a purpose. They want to be useful. They want to be productive.

• 36:01Ben Horowitz?

  It's actually a good thing in life, it turns out.

• 36:04Marc Andreessen?

  It turns, it turn, it turns out.

• 36:07Ben Horowitz?

  Yeah.

• 36:07Marc Andreessen?

  In a startling turn of events. Okay. So, yeah. So, yeah, I think that I've long felt, you know, a little bit of the software eats the world thing a decade ago. I've always thought, I've always thought that basically demand for software is sort of perfectly elastic, possibly to infinity. And the, the theory there basically is if you just continuously bring down the, the cost of software, you know, which has been happening over time, then basically demand, you know, basically is, is like, basically perfectly correlates upward. And the, and the reason is because, you know, kind of as we've been discussing, but it's kind of, there's, there's always something else to do in software. There's always something else to automate. There's always something else to optimize. There's always some, something else to improve. There's always something to make better. And, you know, in, in the, in the moment with the constraints that you have today, you may not, you know, think of what that is, but the minute you don't have those constraints, you'll imagine what it is. I'll just give you an example. I mean, so I'll give you an example, they're playing out with AI right now, right? So there have been, we have, you know, we have companies that do this.

• 36:53Marc Andreessen?

  You know, there have been, you know, there have been companies that have made AI, you know, or they're, that have made software systems for doing security cameras forever, right? And it's like, and for a long time, it was like a big deal to have software that would do like, you know, have different security camera feeds and store them on a DVR and be able to replay them and have an interface that lets you do that. Well, it's like, you know, AI security cameras all of a sudden can have like actual like semantic knowledge of what's happening in the environment. And so they can say, you know, hey, that's Ben. And then they can say, oh, hey, you know, that's Ben, but he's carrying a gun.

• 37:19Ben Horowitz?

  Yeah.

• 37:20Marc Andreessen?

  Right? Right? And by the way, that's Ben and he's carrying a gun, but that's because like he hunts on, you know, Thursdays and Fridays as compared to that's Mary and she never carries a gun and like, you know, like something is wrong. And she's really mad, right? She's got a, yeah, really steamed expression on her face and we should probably be worried about it, right? And so there, there's like an entirely new set of capabilities you can do, just as one example, for security systems that were never possible pre-AI. And a security system that actually has a semantic understanding of the world is obviously much more sophisticated than the one that doesn't and might actually be more expensive to make. Right?

• 37:50Ben Horowitz?

  Right. Well, and just imagine healthcare, right? Like, like you could. Wake up every morning and have a complete diagnostic, you know, like, how am I doing today? Like, what are all my levels of everything? You know, how should I interpret them, you know, better than, you know, this is one thing where AI is really good is, you know, medical diagnosis, because it's a super high dimensional problem. But if you can get access to, you know, your continuous glucose reading, you know, maybe sequence your blood now and again, this and that and the other. Yeah, you've got an incredible kind of view of things. And who doesn't want to be healthier? You know, like now we have a scale. That's basically what we do.

• 38:35Ben Horowitz?

  You know, maybe, maybe check your heart rate or something, but like, pretty primitive stuff compared to where we could go.

• 38:41Marc Andreessen?

  Yeah, that's right. Okay, good. All right, so let's go to the next topic. So on the topic of data, so Major Tom asks, as these AI models allow for us to copy existing app functionality at minimal cost, proprietary data seems to be the most important mote. How do you think that will affect proprietary data value? What other moats do you think companies can focus on building in this new environment? And then Jeff Weishaupt asks, how should companies protect sensitive data, trade secrets, proprietary data, individual privacy in the brave new world of AI? So let me, let me start with a provocative, let me start with a provocative statement. Let's see if you agree with it, which is, you know, you sort of hear a lot, this sort of statement or cliche is like data is the new oil. And so it's like, okay, you know, data is the key input to training AI, making all this stuff work. And so, you know, therefore, you know, data is basically the new, the new resource.

• 39:26Marc Andreessen?

  It's the limiting resource. It's the super valuable thing. And so, you know, whoever has the best data is going to win. And you see that directly in how you train AIs. And then, you know, you also have like a lot of companies, of course, that are now trying to figure out what to do with AI. And a very common thing you'll hear from companies is, well, we have proprietary data. Right? So I'm a, you know, I'm a hospital chain or I'm a, you know, whatever, any kind of business, insurance company or whatever. And I've got all this proprietary data that I can apply, you know, that I'll be able to, you know, build things with my proprietary data with AI that, that, that won't just, you know, be something that anybody will be able to have. Let me argue that basically, let's see, I may argue in like almost every case like that, it's not true. It's basically what the internet kids would call cope. It's simply not true. And the reason it's just not true is because the amount of data available on

• 40:09Marc Andreessen?

  the internet and just generally in the environment is just a million times greater. And so while it may not, you know, while it may not be true that I have your specific medical information, I have so much medical information off the internet for so many people in so many different scenarios that it just swamps the value of quote, "your" data. You know, just, it's just, it's just like overwhelming. And so your proprietary data as you know, company X will be a little bit useful on the margin, but it's not actually going to move the needle. And it's not really going to be a barrier to entry in most cases. And then let me cite as proof for the, for my, my belief that this is mostly cope is there has never been, nor is there now, any sort of, basically any level of sort of rich or sophisticated

• 40:50Marc Andreessen?

  marketplace for data, market for data. There's no, there's no, there's no large marketplace for data. And in fact, in fact, what there are is there are very small markets for data. So there are these businesses called data brokers that will sell you, you know, large numbers of like, you know, information about users of the internet or something. And they're just small businesses. Like they're just not large. It just turns out like information on lots of people is just not very valuable. And so if the data actually had value, you know, it would have a market price and you would see it transacting and you actually very specifically don't see that, which is sort of a, you know, yeah, sort of quantitative proof that the data actually is not nearly as valuable as people think it is.

• 41:25Ben Horowitz?

  Where I agree, so I agree that the data like just says, here's a bunch of data and I can sell it without doing anything to the data is like massively overrated. Like I definitely agree with that. And like, maybe I can imagine some exceptions, like some...

• 41:47Ben Horowitz?

  You know, special population genomic databases are something that are that were very hard to acquire that are useful in some way that's, you know, that's not just like living on the internet or something like that, I could imagine, where that's super highly structured, very general purpose and not widely available. But for most data in companies, it's not like that in that it tends to not, it's either widely available or not general purpose. It's kind of specific. Having said that, right, like companies have made great use of data. For example, a company that you're familiar with Meta uses its data to kind of great ends itself, feeding it into its own AI systems, optimizing its products in incredible ways.

• 42:30Ben Horowitz?

  And I think that, you know, us, Andreessen Horowitz, actually, you know, so we just raised $7.2 billion and it's not a huge deal, but we took our data and we put it into an AI system and our LPs were able, there's a million questions investors have about everything we've done, our track record, every company we've invested and so forth. And for any of those questions, they could just ask the AI, they could be, wake up at three o'clock in the morning and go, do I really want to trust these guys? And go in and ask the AI a question and boom, they'd get an answer back instantly. They didn't have to wait for us and so forth. So we really kind of improved our investor relations product tremendously through use of our data. And I think that

• 43:15Ben Horowitz?

  almost every company can improve its competitiveness through use of its own data. But the idea that it's collected some data that it can go like sell or that is oil or, or what have you.

• 43:32Ben Horowitz?

  That's, yeah, that's probably not true, I would say. And, you know, it's kind of interesting because a lot of the data that you would think would be the most valuable would be like your own code base, right? Your software that you've written, so much of that lives in GitHub. Nobody is actually, I don't know of any company, we do work with, you know, whatever, a thousand software companies. And do we know any that's like building their own programming model on their own code? Like, or, and would that be a good idea? Probably not, just because there's so much code out there that the systems have been trained on. So like that's.

• 44:12Ben Horowitz?

  Not so much of advantage. So I think it's a very specific kind of data that would have value.

• 44:18Marc Andreessen?

  Well, let's ask, let's make it actionable then. If I'm a, if I'm running a big company, like if I'm running an insurance company or a bank or a hospital chain or something like that, like how, or you know, I a consumer packaged goods company, FC or something like what, how should I validate, like, how should I validate that I actually have a valuable proprietary data asset that I should really be focusing on using versus maybe versus in the alternate, by the way, maybe there's other things that maybe I should be taking all the effort I would spend on trying to optimize use of that data. And maybe I should use it entirely trying to build things using internet data instead.

• 44:48Ben Horowitz?

  Yeah. So, so I think, I mean, look, if you're right, if you're in the insurance business, then like all your actuarial data is both interesting and that I, I don't know that anybody publishes their actual actuarial data. And so like, I'm not sure how you would train the model on stuff off of the internet. You know, similarly.

• 45:09Marc Andreessen?

  That's a good, let me, let me, can I challenge that one? So that, that would be a good, a good thing. That'd be a good test case. So I'm an insurance company. I've got records on 10 million people and you know, the actuarial tables and when they, when they get sick and when they die. Okay, that's great. But like, there's lots and lots of actuarial, general actuarial data on the internet for large scale populations, you know, because governments collect the data and they process it and they publish reports. And there's lots, there's lots of academic studies. And so like, is your, is, is your large data set giving you any additional actuarial information that the much larger data set on the internet isn't already providing you? Like, are your, are your insurance clients actually actuarially any different than just everybody?

• 45:47Ben Horowitz?

  I think so. Cause on intake, on the, you know, when you get insurance, they give you like a blood test. They've got all these things. They know if you're a smoker and so forth. And in the, I think in the general data set, like, yeah, you know, who dies, but you don't know what the fuck they did coming in. And so what you really are looking for is like, okay, for this profile of person with this kind, with these kinds of lab results, how long do they live? And that's where the value is. And I think that, you know, interesting, like, you know, I was thinking about like a company like Coinbase where, right, they have incredibly valuable assets in the terms of money. They have to stop people from breaking in. They've done

• 46:30Ben Horowitz?

  a massive amount of work on that. They've seen all kinds of break in types. I'm sure they have tons of data on that. It's probably like weirdly specific to people trying to break into crypto exchanges. And so, you know, like, I think it could be very useful for them. I don't think they could sell it to anybody. But, you know, I think every company's got data that if, you know, fed into an intelligent system would help their business. And I think almost nobody has data that they could just go sell. And then there's this kind of in-between question, which is, what data would you want to let

• 47:08Ben Horowitz?

  Microsoft or Google or OpenAI or anybody get their grubby little fingers on? And that, I'm not sure. That's a, but that I think is the question that enterprises are wrestling with more than, it's not so much should we go like sell our data, but it's should we train our own model just so we can maximize the value? Or should we feed it into the big model? And if we feed it into the big model, do all of our competitors now have the thing that we just did? And, you know, or could we trust the big company to not do that to us? Which I kind of think the answer on trusting the big company not to F with your data is probably I won't do that.

• 47:52Marc Andreessen?

  Well, I mean, yes.

• 47:54Ben Horowitz?

  If your competitiveness depends on that, you probably shouldn't do that.

• 47:57Marc Andreessen?

  Yeah. Well, there are at least reports that certain big companies are using all kinds of data that they shouldn't be using to train their models already. So.

• 48:05Ben Horowitz?

  Yep, I think like I think those reports are very likely true.

• 48:10Marc Andreessen?

  Right.

• 48:11Ben Horowitz?

  Or or they'd have open data, right? Like this is, you know, we've talked about this before, but you have the same companies that are saying they're not stealing all the data from people or taking it in an unauthorized way, refuse to say, open their data. Like, why not tell us where your data came from? And in fact, they're trying to shut down all openness, no open source, no open weights, no open data, none of open nothing, and go to the government and try and get to do that. You know, if you're not a thief, then why are you doing that?

• 48:40Marc Andreessen?

  Right, right, right. What are you hiding? By the way, there's there's other twists and turns here. So for example, the insurance example, I kind of deliberately loaded it because you you may know it's actually illegal to use genetic genetic data for insurance purposes, right? So there's this thing called the Genoa Law Genetic Information Non-Discrimination Act of 2008. And basically, it basically it basically bans health insurers in the US from actually using genetic data for the purpose of doing, you know, health assessment, actuarial assessment of of it, which, by the way, now, because now the genomics are getting really good, like that data probably actually is, you know, among the most accurate data you could have if you were actually trying to predict like when people are going to get sick and die. And they're literally, they're literally not allowed to use it.

• 49:18Ben Horowitz?

  Yeah. It is. I I think that this this is a interesting, like weird misapplication of good intentions in a policy way that's probably going to kill more people than ever get saved by every kind of health FDA, et cetera, policy that we have, which is,

• 49:43Ben Horowitz?

  you know, in a world of AI, having access to data on all humans, why they get sick, what their genetics were, et cetera, et cetera, et cetera, is the most that is, you know, you talk about data being the new oil, like that is the new oil, that's the healthcare oil is, you know, if you could match those up, then we'd never not know why we're sick. You know, you could make everybody much healthier, all these kinds of things. But, you know, to kind of stop the insurance company from kind of overcharging people who are more likely to die, we've kind of locked up all this data.

• 50:21Ben Horowitz?

  A kind of better idea would be to just go, okay, for the people who are likely to, like, we subsidize healthcare massively for individuals, anyway, just like, differentially subsidized. And then you solve the problem and you don't lock up all the data. But, you know, it's typical of politics and policy. I mean, most of them are like that, I think.

• 50:47Marc Andreessen?

  Well, there's this interesting question, it's like in insurance, like, basically, one of the questions people have asked about insurance is like, if you had perfectly predictive information on like individual outcomes, does the whole concept of insurance actually still work, right? Because the whole theory of insurance is risk pooling, right? It's precisely the fact that you don't know what's going to happen in the specific case that means you build these statistical models and then you risk pool and then you have variable payouts depending on exactly what happens. But if you literally knew what was going to happen in every case, because, for example, you had all this predictive genomic data, then all of a sudden it wouldn't make sense to risk pool because you just say, well, no, this person's going to cost X, that person's going to cost Y, there's no...

• 51:24Ben Horowitz?

  Health insurance already doesn't make sense in that way, right? Like, in insurance, the idea of insurance is kind of like, it started with crop insurance, where like, okay, you know, my crop fails, and so we all put money in a pool in case, like, my crop fails so that, you know, we can cover it. It's kind of designed for it to risk pool for a catastrophic unlikely incident. Like, everybody's got to go to the doctor, all the fucking time. And some people get sicker than others and that kind of thing. But like, the way our health insurance works is like, all medical gets paid for through this insurance system, which is this layer of loss in bureaucracy and giant companies

• 52:08Ben Horowitz?

  and all this stuff when like...

• 52:12Ben Horowitz?

  If we're going to pay for people's healthcare, just pay for people's healthcare. Like, what are we doing? Right? Like, and if you want to disincent people from like going for nonsense reasons and just up the co-pay like it's like, what are we doing?

• 52:27Marc Andreessen?

  Just Yeah. Well, and then from a yeah, and then from a justice standpoint, from a fairness standpoint, like what it makes sense for me, you know, would it make sense for me to pay more for your healthcare? If I knew that you were going to be more expensive than like, you know, I'm direct, you know, if you, if, if everybody knows what future healthcare costs is per person.

• 52:42Ben Horowitz?

  Yeah.

• 52:42Marc Andreessen?

  There has a very good predictive model for it, you know, societal willingness to all pool in the way that we do today might really diminish.

• 52:48Ben Horowitz?

  Yeah, yeah. Well, and then like, like you, you could also, if you knew, like, there's things that you do genetically, and maybe we give everybody a pass on that. It's like, you can't control your genetics. But then like, there's things you do behaviorally that like dramatically increases your chance of getting sick. And so maybe, you know, we incentivize people to stay healthy instead of just like paying for them not to die. The, there, there's a lot of systemic fixes we could use with the healthcare system. It, it couldn't be designed in a more ridiculous way, I think. Well, it couldn't be designed in a more ridiculous way. It's actually more ridiculous than some other countries, but it's pretty crazy here.

• 53:27Marc Andreessen?

  Nathan, Nathan Odi asks, what are the strongest common themes between the current state of AI and Web 1.0? And so let me start there. Let me give you a theory, Ben, and see what you think. So I get this question, you know, because of my role and, you know, Ben, you, you with me at Netscape, you know, we, we get this question a lot because of our role early on with the, with the internet. So there's an, you know, the internet boom was like a major, major event in technology, and it's still within a lot of, you know, people's memories. And so, you know, the sort of, you know, people like to reason from analogy. So it's like, okay, the AI boom must be like the internet boom. Starting an AI company must be like starting an, an internet company. And so, you know, what, what is this like? And we actually got a bunch of questions like that, you know, that are kind of analogy questions like that. I actually think, and you know, and then Ben, you know, you and I were there for the internet boom. So we, you know, we lived through that and the bust and the boom and the bust.

• 54:10Marc Andreessen?

  So I actually think that the analogy doesn't really work for the most part. It works in certain ways, but it doesn't really work for the most part. And the reason is because the, the internet, the internet was a network, whereas AI is a computer.

• 54:24Ben Horowitz?

  Yep.

• 54:26Marc Andreessen?

  Okay. Yeah. So, so some people understand what we're saying.

• 54:29Ben Horowitz?

  It'll be more like the PC boom.

• 54:32Marc Andreessen?

  Or even the PC, or even I would say the microprocessor, like my best analogy is to the microprocessor.

• 54:37Ben Horowitz?

  Yeah.

• 54:37Marc Andreessen?

  Or even like the original computers, like back to the mainframe era.

• 54:40Ben Horowitz?

  Uh huh.

• 54:41Marc Andreessen?

  And the reason is because, yeah, look, what the internet did was the internet, you know, obviously was a network, but the network connected together many existing computers. And then, of course, people built many other new kinds of computers to connect to the internet. But fundamentally, the internet was a network. And then, and that's important because most of, most of the sort of industry dynamics, competitive dynamics, startup dynamics around the internet had to do with basically building, either building networks or building applications that run on top of networks. And this, you know, the internet generation of startups was very consumed by network effects. And, you know, all these positive feedback loops that you get when you connect a lot of people together, and, you know, things like so-called Metcalfe's law, which is sort of the value of a network, you know, expands, you know, kind of the way it expands is you have more people to it. And then, you know, there were all these fights, you know, these fights, you know, all the social networks or whatever, fighting to try to get network effects and try to steal

• 55:26Marc Andreessen?

  each other's users because of the network effects. And so it was kind of, you know, it's dominated by network effects, which is what you'd expect from a network business. AI, like, there are some networks effects in AI that we can talk about, but it's more like a microprocessor. It's more like a chip. It's more like a computer in that it's a system that basically, right, if data comes in, data gets processed, data comes out, things happen. That's a computer. It's an information processing system. It's a computer. It's a new, it's a new kind of computer. It's a, you know, we like to say the sort of computers up until now have been what are called von Neumann machines, which is to say they're deterministic computers, which is they're like, you know, hyper literal and they do exactly the same thing every time. And if they make a mistake, it's the programmer's fault. But they're very limited in their ability to interact with people and understand the world.

• 56:11Marc Andreessen?

  You know, we think of AI and large language models as a new kind of computer, a probabilistic computer, a neural network based computer that, you know, by the way, is not very accurate and is, you know, doesn't give you the same result every time. And in fact, might actually argue with you and tell you that it doesn't want to answer your question.

• 56:25Ben Horowitz?

  Yeah, which makes it very different in nature than the old computers. And it makes you get kind of composability, you know, the ability to build things, big things out of little things more complex.

• 56:41Marc Andreessen?

  Right, but but the capabilities are new and different and valuable and important because it can understand language and images and you know that all these do all these things that you you see when you use.

• 56:49Ben Horowitz?

  All of that means we could never solve with deterministic computers. We can now go after, right?

• 56:54Marc Andreessen?

  Right, right. Yeah, exactly. And so I think I think Ben, I think the analogy and I think the lessons learned are much more likely to be drawn from the early days of the computer industry or from the early days of the microprocessor than the early days of the internet. Does that? Does that sound right?

• 57:06Ben Horowitz?

  I think so. Yeah, I definitely think so. And that doesn't mean there's no like boom and bust and all that because that's just the nature of technology. You know, people get too excited and then they get too depressed. So there will be some of that, I'm sure. There will be over build outs, you know, potentially eventually of chips and power and that kind of thing. You know, we start with a shortage, but but I agree. Like, I think networks are fundamentally different in the nature of how they evolved in computers and and the kind of just the adoption curve and all those kinds of things will be different.

• 57:39Marc Andreessen?

  Yeah. So then this kind of goes to where how I think the industry is going to unfold. And so this is kind of my best theory for kind of what happens from here. This kind of this, you know, this this giant question of like, you know, is the industry going to be a few god models or a, you know, a very large number of of models of different sizes and so forth. So the computer, like famously, the, you know, the the original computers, like the original IBM mainframes, you know, the big computers, you know, they were very, very large and expensive and there were only a few of them. And the prevailing view actually for a long time was that's all there would ever be. And there was this famous statement by Thomas Watson, senior, who was the creator of IBM, you know, which was the dominant company for the first, like, you know, 50 years of the of the computer industry. And he said, he said, he said, I believe this actually true quote, he said, I don't, I don't know, I don't know that the world will ever need more than five computers. And I think the

• 58:24Marc Andreessen?

  reason for that, it was literally, it was like, the government's going to have two, and then there's like three big insurance companies, and then that's it.

• 58:32Ben Horowitz?

  Who else would need to do all that math?

• 58:34Marc Andreessen?

  Exactly. Who, yeah, who else would need to, who else needs to keep track of huge amounts of numbers? Who, who else needs that level of, you know, calculation capability? It's just not a relevant, you know, it's just not, not, not a relevant concept. And by the way, they were like big and expensive. And so who else can afford them? Right. And who else can afford all the headcount required to manage them and maintain them? I mean, and this is in the days, I mean, these things were big. These things were so big that you'd have an entire building that got built around a computer. Right. And they'd have like, they'd famously have all these guys in white lab coats, literally like taking care of the computer. Cause everything had to be kept super clean or the computer was not working. And so, you know, it was this thing where, you know, there, today we have the idea of an AI God model, which is like a big foundation model. That, you know, then we have the idea of like a God mainframe, like there, there would just be a few, a few of these things. And by the way, if you watch old science fiction, it almost always has this sort of conceit.

• 59:18Marc Andreessen?

  It's like, okay, there's a big supercomputer and it either is like doing the right thing or doing the wrong thing. And if it's doing the wrong thing, you know, that's, that's often the plot of the, of the science fiction movies is you have to go in and try to figure out, figure out how to fix it or defeat it. And so it's sort of this, this idea of like a, a single top down thing. Of course, and that, and that held for a long time, like that held for, you know, the first few decades. And then, you know, even when computers, computers started to get smaller. So then you had so-called mini computers was the next phase. And so that was a computer that, you know, didn't cost $50 million. Instead it costs, you know, $500,000, but even still $500,000 is a lot of money. People aren't putting mini computers in their homes. And so it's like mid-sized companies can, can buy mini computers, but certainly people can't. And then of course with the PC, they shrunk down to like $2,500. And then with the smartphone, they shrunk down to $500.

• 1:00:01Marc Andreessen?

  And then, you know, sitting here today, obviously you have computers of every shape, size, description, all the way down to, you know, computers that cost a penny. You know, you've got a computer in your thermostat that, you know, basically controls the temperature in the room. And it, you know, probably cost a penny and it's probably some embedded ARM chip with firmware on it. And there's, you know, many billions of those all around the world. You buy a new car today, it has something, new cars today have something on the order of 200 computers in them. May, maybe, maybe more at this point. And so you, you just basically assume with the chip today, sitting here today, you just kind of assume that everything has a chip in it. You assume that everything, by the way, draws electricity or has a battery because it needs to power the chip. And then increasingly you assume that everything's on the internet because basically all computers are assumed to be on the internet or that, or they will be. And so, so, and so as a consequence, what you have is the computer industry today is this massive pyramid.

• 1:00:46Marc Andreessen?

  And you still have a small number of like these supercomputer clusters or these giant mainframes that are like the God model, you know, the God, the God, the God mainframes. And then you've got, you know, a larger number of mini computers. You've got a larger number of PCs. You've got a much larger number of smartphones. And then you've got a giant number of embedded systems. And it turns out like the computer industry is all of those things. And, you know, what, what is a, what, you know, what, what size of computer do you want is based on, well, what exactly are you trying to do and who are you and what do you need? And so if that analogy holds, it basically means actually we are going to have AI models of every conceivable shape, size, description, capability, right? Based on trained on lots of different kinds of data at running at very different kinds of scale, very different privacy, different policies, different, you know, security policies.

• 1:01:28Marc Andreessen?

  You know, you're just, you're just going to have like enormous variability and variety. And it's going to be an entire ecosystem and not just a couple of companies. Yeah, let me see what you think of that.

• 1:01:37Ben Horowitz?

  Well, I think that's right. And I also think that the, the other thing that's interesting about this era of computing, if you look at prayers of computing from the mainframe to the smartphone, a huge source of lock-in was basically the difficulty of using them. So, you know, nobody ever got fired for buying IBM because like, you know, you had people trained on them. You know, people knew how to use the operating system. Like it was, you know, it was just kind of like a safe choice due to the massive complexity of like dealing with a computer. And then even with the smartphone, like the, you know, why is the Apple computer smartphone so dominant?

• 1:02:21Ben Horowitz?

  You know, what makes it so powerful as well? Cause like switching off of it is so expensive and complicated and so forth. It's an interesting question with AI because AI is the easiest computer to use by far. It speaks English. It's like talking to a person. And so like, what is the lock-in there? And so are you completely free to use the size, price, choice, speed that you need for your particular task? Or are you locked into the God model? And, you know, I, I think it's still a bit of an open question, but it's, it's pretty interesting in that, that, that thing could be very different than prior generations.

• 1:03:01Marc Andreessen?

  Yeah, yeah, that makes sense. And then just to complete the question, what would we say? So Ben, what, what would you say are lessons learned from the internet era that we live through? That would, it would apply that people should think about?

• 1:03:12Ben Horowitz?

  I think a big one is, is probably just the, the boom bust nature of it that like, you know, the demand, the interest in the internet, the recognition of what it could be was so high that money just kind of poured in in buckets. And, you know, and then the underlying thing, which in internet age was the telecom infrastructure and fiber and so forth, got just unlimited funding and unlimited fiber was built out. And then eventually we had a fiber glut and all the telecom companies went bankrupt and, and that was great fun. But, you know, like we entered in a good place. And I think that that's something like that's probably pretty likely to happen in AI where like, you know, every company is

• 1:03:57Ben Horowitz?

  going to get funded. We don't need that many AI companies. So a lot of them are going to bust. There's going to be a huge, you know, huge investor losses. There will be an over build out of chips for sure at some point. And then, you know, we're going to have too many chips and yeah, some chip companies will go bankrupt for sure. And then, you know, and I think probably the same thing with data centers and so forth, like we'll be behind, behind, behind, and then we'll overbuild at some point. So that, that'll be very interesting. I think that, and that's kind of the, that's every new technology. So Carlotta Perez has a great kind of, has done, you know, amazing work on this where like that is just

• 1:04:40Ben Horowitz?

  the nature of a new technology is that you overbuild, you underbuild, then you overbuild and, you know, and there's a hype cycle that funds the build out and a lot of money is lost, but we get the infrastructure and that's awesome because that's when it really gets adopted and changes the world. I want to say, you know, with the internet, the other, the other kind of big kind of thing is the internet went through a couple of phases, right? Like it went through a very open phase, which was unbelievably great. It was probably one of the greatest booms to the economy. It, you know, it certainly created tremendous growth and power in America, both, you know, kind of economic power and soft cultural power and these kinds of things. And then, you know, it became closed with the next

• 1:05:24Ben Horowitz?

  generation architecture with, you know, kind of discovery on the internet being owned entirely by Google and, you know, kind of other things, you know, being owned by other companies. And you know, AI I think could go either way. So it could be very open or like, you know, with kind of misguided regulation, you know, we could actually force our way from something that, you know, is open source, open weights, anybody can build it. We'll have a plethora of this technology will be like, use all of American innovation to compete, or we'll, you know, we'll cut it all off, we'll force it into the hands of the companies that kind

• 1:06:04Ben Horowitz?

  of own the internet today. And, you know, and we'll put ourselves at a huge disadvantage, I think competitively against China in particular, but, but everybody in the world. So, so I think that's, that's something that definitely, you know, that we're involved with trying to make sure it doesn't happen, but it's a real possibility right now.

• 1:06:24Marc Andreessen?

  Yeah. The sort of irony is that networks used to be all proprietary and then they opened up.

• 1:06:29Ben Horowitz?

  Yeah, yeah, yeah. Landman, Apple Talk, NetBuoy, NetBios.

• 1:06:34Marc Andreessen?

  Yeah, exactly. And so these are all the early proprietary networks from all individual specific vendors. And then the internet appeared and kind of TCP IP and everything opened up. The AI is trying to go the other, I mean, the big companies trying to take AI the other way. It started out as like open, just like basically just like the research.

• 1:06:48Ben Horowitz?

  Everything was open source in AI, yeah.

• 1:06:50Marc Andreessen?

  Right, right, right. And now they're trying to, they're trying to lock it down. So it's, it's a, it's a, it's a fairly nefarious turn of events.

• 1:06:56Ben Horowitz?

  Yeah, yeah, very nefarious. And, you know, like, it's remarkable to me. I mean, it is kind of the darkest side of capitalism when a company is so greedy, they're willing to destroy the country and maybe the world to like, just get a little extra profit. But, you know, and they do it like the, the really kind of nasty thing is they claim, oh, it's for safety. You know, we've created an alien that we can't control, but we're not going to stop working on it. We're going to keep building it as fast as we can. And we're going to buy every fricking GPU on the planet. But we need the government to come in and stop it from being open. This is literally the current position of Google and Microsoft right now. It's crazy.

• 1:07:39Marc Andreessen?

  And we're not going to secure it. So we're going to make sure that like Chinese spies can just like steal our chip plans, take them out of the country and we won't even realize for six months.

• 1:07:46Ben Horowitz?

  Yeah, it has nothing to do with security. It only has to do with monopoly.

• 1:07:50Marc Andreessen?

  Yes. The other, you know, just Ben, going back on your point of speculation. So there's this critique that we hear a lot, right? Which is like, okay, you idiots. Basically, it's like, you idiots, you idiots entrepreneurs, investors, you idiots. It's like, there's a speculative bubble with every new technology. Like, basically, like, when are you people going to learn to not do that?

• 1:08:06Ben Horowitz?

  Yeah.

• 1:08:07Marc Andreessen?

  And there's an old joke, there's an old joke that relates to this, which is the foremost dangerous words in investing are this time is different. The 12 most dangerous words in investing are, the foremost dangerous words in investing are this time is different. Right? Like, so like, does history repeat? Does it not repeat? The, my sense of it, you referenced Carolina Perez's book, which I agree is good, although I don't think it works as well anymore. We can talk about it sometime, but, but, you know, is a good, at least background piece on this. You know, it is just like, it's just incontrovertibly true. Basically, every significant technology advance in history was greeted by some kind of financial bubble, basically, since financial markets have existed.

• 1:08:41Marc Andreessen?

  And this, by the way, this includes like everything from, you know, radio and television, the railroads, you know, lots and lots of prior. By the way, there was a, there was actually a so-called, there was an electronics boom bust in the 60s called, it was called Tronics. Every, every company had the name Tronics. And so, you know, there, there was that. So, you know, there was like a laser boom bust cycle. There are all these like boom bust cycles. And so basically, it's like any new, any new technology, that's what economists call a general purpose technology, which is to say something that can be used in lots of different ways. Like it inspires sort of a speculative mania. And, you know, and like the critique is like, okay, why do you need to have this speculative mania? Why do you need to have the cycle? Because like, you know, people, some people invest in the things, they lose a lot of money. And then there's this bust cycle that, you know, causes everybody to get depressed, maybe delays the rollout.

• 1:09:24Marc Andreessen?

  And it's like two things. Number one is like, well, you just don't know. Like if it's a general purpose technology like AI is, and it's potentially useful in many ways, like nobody actually knows upfront, like what the successful use cases are going to be, or what's successful companies are going to be. Like you actually have to, you have to learn by doing.

• 1:09:38Ben Horowitz?

  You're gonna have to miss this. That's venture capital.

• 1:09:40Marc Andreessen?

  Yeah.

• 1:09:40Ben Horowitz?

  We, we.

• 1:09:41Marc Andreessen?

  Yeah, exactly.

• 1:09:43Marc Andreessen?

  Yeah, exactly. So yeah, the true venture capital model kind of wires this in, right? We, we basically, core venture capital, the kind that we do, we sort of assume that half the companies fail, half the projects fail. And, and you know, if, if any of us, if we are any of our.

• 1:09:55Ben Horowitz?

  Fail completely, like lose money.

• 1:09:57Marc Andreessen?

  Yeah, like lose money. Exactly. Yeah. And so like, and of, and of course, if we are any of our competitors, you know, could figure out how to do the 50% that work without doing the 50% that don't work, we would do that. But, you know, here we sit 60 years into the field and like, nobody's figured that out. So there, there is, there is that, that unpredictability to it. And then the other, the other kind of interesting way to think about this is like, okay, what would it mean to have a society in which a new technology did not inspire speculation? And it would mean having a society that basically is just like inherently like super pessimistic about both the prospects of the new technology, but also the prospects of entrepreneurship and, you know, people inventing new things and doing new things. And of course, there are many societies like that on planet earth, you know, that just like fundamentally like, don't have the spirit of invention and adventure that a, that, you know, that, that a place like Silicon Valley does. And, you know,

• 1:10:42Marc Andreessen?

  are they better off or worse off? And, you know, generally speaking, they're worse off. They're, they're just, you know, less future oriented, less, less, less, less focused on, on building things, less focused on, on figuring out how to get growth. And so I, I think there's a, at least my sense, there's a comes with the territory thing. Like we, we, we would all prefer to avoid the downside of a speculative boom bust cycle, but like, it seems to come with the territory every single time. And I, at least I have not, nobody, I'm aware, no society I'm aware of has ever figured out how to capture the good without also having the bad.

• 1:11:09Ben Horowitz?

  Yeah. And like, why would you? I mean, it's kind of like, you know, the, the, the whole Western United States was built off the gold rush. And like every kind of treatment in like popular culture of the gold rush kind of focuses on the people who wasn't, who didn't make any money. But there were people who made a lot of money, you know, and found gold. And you know, in the internet bubble, which, you know, was completely ridiculed by, you know, kind of every, every movie, if you go back and watch any movie between like 2001 and 2004, they're all like, how only morons did a dot com and this and that and the other. And there were all these funny documentaries and so forth. But like,

• 1:11:54Ben Horowitz?

  that's when Amazon got started, you know, that's when eBay got started, that's when Google got started, you know, these, these companies, you know, with that were started in the bubble, in the kind of time of this great speculation, there was gold in those companies. And if you hit any one of those, like you funded, you know, probably the next set of companies, you know, which included things like, you know, Facebook and X and, you know, Snap and all these things. And so, yeah, I mean, like, that's just the nature of it. I mean, like, that's what makes it exciting. And, and, you know, it's just, it's an amazing kind of thing that, you know, look, the transfer of money from people who have excess money to people who are trying

• 1:12:38Ben Horowitz?

  to do new things and make the world a better place is the greatest thing in the world. Like, and if we, some of the people with excess money lose some of that excess money and trying to make the world a better place, like, why are you mad about that? Like, that, that's the thing that I could never have seen. Like, why would you be mad at, you know, young, ambitious people trying to improve the world, getting funded, and some of that being misguided? Like, why is that bad?

• 1:13:08Marc Andreessen?

  Right, right. As compared to, yeah, as compared to, especially as compared to everything else in the world and all the people who are not trying to do that.

• 1:13:14Ben Horowitz?

  So you'd rather like, we just buy like, you know, lots of mansions and boats and jets and really like, what are you talking about?

• 1:13:22Marc Andreessen?

  Right, exactly. Donate money to ruinous causes.

• 1:13:26Ben Horowitz?

  Right.

• 1:13:28Marc Andreessen?

  Such as ones that are on the news right now. Okay. So, all right, we're at a minute 20. We made it all the way through four questions. We're doing good. We're doing great. So let's call it here. Thank you, everybody for joining us. And I believe we should do a part two of this, if not parts three through six, because we have a lot more questions to go. But thanks, everybody for joining us today.

• 1:13:45Ben Horowitz?

  All right. Thank you.