Pod Insights a16z Podcast The Race for AI—Search, National Infrastructure, & On-Device AI About this episode The AI race is on, and 2025 could be its most transformative year yet.
In this episode, a16z General Partners Anjney Midha and Jennifer Li, and Partner Alex Immerman dive into the trends reshaping AI and its impact on search, infrastructure, and devices.
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Anjney Midha? In 2024 alone, I think there were more than 700 pieces of state-level legislation that were AI-specific.
0:06 Speaker 01 The average query on Perplexity is 10 to 11 words. The average search on Google is 2 to 3 keywords.
0:13 Anjney Midha? We've had an enormous amount of Nvidia's purchasing orders come from the balance sheet of governments.
0:19 Jennifer Li? You can actually reimagine the AR experience.
0:22 Anjney Midha? If you're a founder like that who has the guts, then your impact on humanity ends up being quite generational.
0:28 Steph Smith? Here we are again, inching even closer to the end of 2024. And as we near 2025, here are a few dates to give you some perspective. We've had 24 incredible years of Wikipedia, 18 years of the iPhone, and 16 years since the Bitcoin white paper release. So as we look to 2025, and the speed of innovation is only increasing, we continue our coverage of A16Z's big ideas, together with the dozens of partners who are meeting daily with the people building our future. Last year, we predicted:
1:00 Speaker 04 A new age of maritime exploration. Programming medicine's final frontier.
1:05 Anjney Midha? AI for schemes that never end.
1:08 Steph Smith? Democratizing miracle drugs. On deck this year: Closing the hardware-software chasm.
1:14 Speaker 04 Game tech powers tomorrow's businesses. Super staffing for healthcare.
1:18 Steph Smith? And throughout our four-part series, you'll hear from all over A16Z, including American dynamism, healthcare, fintech, games, and more. However, if you'd like to see the full list of 50 big ideas, head on over to a16z.com/bigideas. And of course, if you missed it, check out part one, all about the intersection of hardware and software. As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast.
2:01 Steph Smith? For more details, including a link to our investments, please see a16z.com/disclosures.
2:12 Steph Smith? Today in part two, we'll be talking about the topic of the day, the month, and quite frankly, the year: artificial intelligence.
2:19 Speaker 01 There's certainly AI, AI, AI.
2:22 Steph Smith? And the race is on, whether it's across companies like Google and the disruptors chasing away 10 blue links, or sovereign countries trying to capitalize on the next frontier, or even device companies figuring out their role as AI meets the edge. That is all on deck today.
2:39 Speaker 01 There's a bit of an innovator's dilemma here, and I'm excited to watch it play out.
2:43 Steph Smith? That was...
2:43 Speaker 01 Alex Zimmerman, and I'm a partner here on the Growth Fund.
2:48 Steph Smith? Here's his big idea.
2:49 Speaker 01 The search monopoly ends in 2025. Google controls 90% of US search, but its grip is slipping. Its recent US antitrust ruling encourages Apple and other phone manufacturers to empower alternative search providers. More than just legal pressure, Gen AI is coming for search. ChatGPT has 250 million weekly active users. Answer Engine Perplexity is gaining share, growing 25% month-on-month and changing the search engagement form. Their queries average 10 words, three times longer than traditional search,
3:33 Speaker 01 and nearly half lead to follow-up questions. Claude, Grok, Meta AI, Poe, and other chatbots are also carving off portions of search. 60% of US consumers used a chatbot to research or decide on a purchase in the last 30 days. For deep work, professionals are leveraging domain-specific providers like Causally, Consensus, Harvey, and Hebea. Ads and links historically aligned with Google's mission: organize the world's information and make it universally accessible and useful. But Google has become so cluttered and gamed that users need to dig through the results.
4:18 Speaker 01 Users want answers and depth. Google itself can offer its own AI results, but at the cost of short-term profits. Google as a verb is under siege. The race is on for its replacement.
4:33 Steph Smith? Maybe start by setting the stage. So how big is the search market today?
4:37 Speaker 01 The search market is enormous. Anyone listening to this uses search. Virtually anyone with the internet uses search. But to put some numbers around it, Google that I just mentioned, they're the biggest game in town. They're approaching 200 billion of revenue annually. They're still growing double digits, highly profitable. Microsoft Bing, which has been the number two player for a long time, mid-single-digit market share, so pretty small. They have 12 billion of revenue. This is a massive, massive market, one of the largest out there.
5:10 Steph Smith? And it does feel like there are forces reshaping this industry. So tell me about those.
5:14 Speaker 01 So there's certainly AI, AI, AI. But before we jump into that, we can level set with the legal pressure that's mounting on Google. So earlier this year, Google was declared a monopoly. The court ruled that they're spending billions of dollars to phone manufacturers. In the case of Apple, tens of billions of dollars is monopolistic. It's anti-competitive and it's preventing their competitors from gaining share in the marketplace. They are the default search engine on all these phone manufacturers. And it basically makes it impossible for any of the others to gain share. So the exciting technological change, of course, is around AI. The gen AI search providers are fantastic.
5:59 Speaker 01 And the market has been dominated by Google for close to 25 years at this point. And as a monopoly, they're not really innovating. They have no incentive to, until now. If you think about the Google experience as it is today, it's just a long list of links. And the first few are sponsored. They're ads. And I, as a user, when I make a query on Google, I then need to make the decision. I have to sift through the information on that site and find the answer that I'm looking for. That's actually a pretty long process. Instead, with gen AI, I can just get the answer. So on ChatGPT, Perplexity, Claude, or when I'm chatting with my AI friends on Character AI or on Poe,
6:44 Speaker 01 I get an answer immediately. And that's just a much, much better experience.
6:50 Steph Smith? And I hear tons of people saying they're trying these new tools, but maybe you can give us a sense, is this shift really stark? Are people really moving over?
6:58 Speaker 01 People are definitely moving over. And I'd say there's four main reasons. One is the one we just talked about, the shift from links to answers. A second one would be how they are personalized, how the answers feel interactive, how they are conversational. The average query on one of these new services has a follow-up question. So it's not just about that initial engagement. It's about the ongoing conversation. The third difference is about how these new services engage with complex queries. So the average query on perplexity is 10 to 11 words. The average search on Google is two to three keywords.
7:40 Speaker 01 As you can imagine, their ability to leverage that information and get you what you want is much higher. On Google, you're going to get a list of links. One of those links might have one side of the debate. Another link may have the other side. You may never make it to that second side. On perplexity, on ChatGPT, they're going to synthesize both sides. They're going to give me both perspectives. And then the fourth, they're just not cluttered with ads. That may change over time. But putting it all together, it should be no surprise that these AI-native services are gaining share. For 10% of consumers, they're now referring to ChatGPT as their search engine of choice. Perplexity queries are growing 25 to 30% month over month.
8:25 Speaker 01 And I saw a survey last week that 60% of consumers for purchase decisions in the last 30 days used a chatbot. All of these services are growing massively.
8:37 Steph Smith? So if we look at some of the tools that exist today that are coming for that market share, you take a perplexity or you take a ChatGPT, those are also broad-based search engines or chatbots. But there are these other players. You mentioned Consensys or Habia, for example, that are verticalized. Does it surprise you that the last wave of search engines weren't verticalized?
8:57 Speaker 01 It does not surprise me that the last wave or the next wave will result in a winner-take-most dynamic. It's a pretty monopolistic market. Search is very much a distribution game. Google has an incredible brand. They have incredible direct traffic. They are the default search engine on most browsers. They are the Kleenex, the Band-Aid of online search. It's important to note that search engines definitely benefit from network effects. More users coming to Google provides more data around preferences for Google. That means the next search has more relevance and brings more users. It also means more users means more advertisers, more profit dollars
9:42 Speaker 01 that you can invest back in the service or into ensuring that Apple puts them as the default provider.
9:49 Steph Smith? Absolutely. And so as we think about this next AI wave, it sounds like you think maybe a similar dynamic might be at play. Or how do you think about maybe these smaller players actually finding a wedge or differentiating?
10:02 Speaker 01 One framework for thinking about how search could fragment or verticalize is just looking at why does vertical software beat horizontal software in some cases? And it's typically the vertical requires a specific user interface, proprietary data, features, workflows, compliance, et cetera. And that's why Viva in life sciences and pharma can beat Salesforce in horizontal CRM. For the average query on ChatGPT, Perplexity, or Google, they're pretty good. You don't need a vertical-specific application. But for deep domain research, I can imagine a world in which standalone apps thrive.
10:47 Speaker 01 In the case of our portfolio company, Hebia, they have a unique interface. It looks like a spreadsheet, which is native to financial services and their customers. It brings in public filings, earnings transcripts, but also private data. It can bring in research. It can bring in survey results. You can query, and then the output can also be specific to that industry. Not just look like a spreadsheet, but populate a meeting agenda. So you can go in immediately prepared. And so when I think about vertical search or vertical apps, it's not just about the search, it's about everything around the search.
11:27 Steph Smith? That's a good distinction. And as we think about how this maybe continues, the progression of this industry, if we go back to the last wave of search, we did see a bunch of search engines get traction to start, maybe AltaVista or Ask Jeeves.
11:40 Speaker 01 We all pick on Ask Jeeves.
11:41 Steph Smith? All the Gen Z'ers will not know what the hell any of that is, but they had some traction and then Google exploded and took over. Do you expect that same kind of consolidation or is this time different?
11:53 Speaker 01 I think consolidation should be expected in general purpose search because of distribution, because of network effects. Google won the last time around in part because of their page rank algorithm. It produced superior results. It had a minimalist UI. It was really simple. Ask Jeeves, AltaVista, they had tons of ads. It was cluttered, links on the homepage. No one wanted to use it. The irony is that today, what we're all complaining about with Google is that their pages are cluttered with ads on the results and that creates the opportunity for these new search engines. I would expect consolidation again.
12:34 Steph Smith? And I think something else that's interesting is what you're pointing at with some of these verticalized solutions is that it's not just for the everyday consumer, but also for the lawyer or for the academic researcher. Do you think that is also part of the fragmentation? Do you expect that to continue?
12:51 Speaker 01 Search historically has been thought of as a consumer product and for good reason. I make a lot of searches that are related to my personal lives, but I use, as do all the professionals you named, search at work. I probably make more Google searches at work than I do around personal matters. But there is a category of enterprise search in a market that has existed. And you can think of that as querying Box, Dropbox, Salesforce, all in one. But I think those two worlds are going to blend together. Consumer search should not just be limited to what's on the web and enterprise search should not be limited to just proprietary data. It should have both and a lot of these AI native services are working on that.
13:35 Steph Smith? Yeah, that's a great point. And as we think about those two models maybe blending together, we think of consumer search for sure. It's always been an ad-based model, or at least today. Google is this massive economic engine, but no one's paying a subscription for that. The new entrants do seem to have a more subscription-based model. Is this a temporary thing? Or how do you see those two dynamics playing?
13:56 Speaker 01 So the subscriptions could stick around, but I think this is going to continue to be a digital advertising-focused market. And I think digital advertising is going to grow because of these AI services. So these AI services today, they don't have ads. I imagine that's going to change. It's been in the news that Perplexity is already talking to advertisers. They have a high-income, highly-educated user base that should be attractive to these advertisers. But as we discussed earlier, the queries on these services, they're longer, they're more complex, they're more detailed, they're more personalized. And because of that, there's greater intent, which should be more helpful to advertisers in producing the best results on their end
14:39 Speaker 01 and creating an even larger market. But in the meantime, these subscription business models, they make a lot of sense. They bootstrap the business, they cover costs. And from a personal perspective, I'm very happy to pay $20 a month for ChatGPT, Perplexity, and Poe.
14:55 Steph Smith? I mean, they provide so much value.
14:57 Speaker 01 A lot more than $20 a month.
14:58 Steph Smith? Yes, yes. So obviously, this was your 2025 big idea. And I know this is going to be a many-year, maybe even decade-long progression. But what are you paying attention to in the next year? What opportunities are still on deck?
15:11 Speaker 01 As I looked at 2025, Google it may be on decline, but I don't expect them to go down without a fight. Google and Meta, I think they can be big players. Google AI Overviews already has a billion monthly active users. Meta's not far behind. They have 500 million monthly active users. Again, this shows the power of distribution for search. And maybe to that end, Apple, if you could call them a dark horse, is a dark horse. They are not investing the capex like Meta and Google and Microsoft and Amazon. But they control a central node to the consumer. And if they wanted to build a search application, the next day, they could have a billion users.
15:55 Steph Smith? We just heard how quintessential winning AI is to multi-trillion-dollar companies like Google and Meta. But what about nation-states?
16:03 Anjney Midha? In the race for AI dominance, compute has become critical national infrastructure, but not every country is equipped to compete in that race.
16:12 Steph Smith? That was...
16:13 Anjney Midha? Manjane Mida, I'm a general partner here at A16Z, where I focus on AI infrastructure.
16:17 Steph Smith? And here's his big idea.
16:19 Anjney Midha? My big idea is infrastructure independence. And it's the idea that a lot of countries and regions are starting to realize that modern AI, deep learning-based AI, generative models, are a form of what have been called general-purpose technologies. In the history of humanity, we've only had maybe 20 or 22 or so general-purpose technologies. And these are usually types of technologies like electricity, the printing press, that have very broad-based applications in society. They end up being largely horizontal economic multipliers and progress multipliers across a whole set of pillars and domains in society. There are usually two moments in the adoption of a general-purpose
17:03 Anjney Midha? technology where first, countries, nation-states start asking, are we going to welcome this technology? Or are we going to be hostile to its development?
17:11 Steph Smith? Okay.
17:12 Anjney Midha? And that's the first step that becomes pretty important in a country's progression, or in a nation-state's progression, or a region's progression, is do we even want to adopt it?
17:19 Steph Smith? Do we allow this in, regardless of whether we own it?
17:21 Anjney Midha? Do we embrace it or not? And then the second is, do we build or buy?
17:25 Steph Smith? Right.
17:26 Anjney Midha? Which is, can we trust somebody else to provide it for us? We're well past the stage of, do we embrace it or not? We're already well into billions of people around the world now, having already embraced it. So the governments don't really have a choice, so to speak. So in a sense, AI has already percolated throughout society at one of the fastest diffusion rates of any general-purpose technology, because it's piggybacked off of years and years of digital infrastructure. And so now the question everybody's asking is, do we build or buy?
17:53 Steph Smith? Yeah.
17:54 Anjney Midha? It's the single largest, probably, purchasing decision that's going to happen in the next 24 months is, do nation-states... Start buying.
18:03 Steph Smith? Do they build or buy?
18:04 Anjney Midha? Do they build or buy?
18:05 Steph Smith? Yeah, I love the parallel of companies because there are many companies that do choose to build, but also companies that choose to rent or buy.
18:12 Anjney Midha? Right.
18:12 Steph Smith? And so as you think about that, there's large nations around the world like the United States, which are clearly building.
18:19 Anjney Midha? Right.
18:20 Steph Smith? But talk about the argument for smaller nations or the 190 plus
18:25 Anjney Midha? Right.
18:26 Steph Smith? that should be thinking about buying.
18:27 Anjney Midha? The good news here is that we've got hundreds of years of human history to look at for clues about what happens next. If you're a small country and you were in the early 1900s, you were watching the modern electrification of the developed world, the United States or Europe. If you chart what happened with many of those countries, many of them decided to actually enter into what were called joint venture agreements. It starts with a joint venture with a country that's at the frontier.
18:55 Steph Smith? Right.
18:55 Anjney Midha? These countries at the frontier of AI is what I call hypercenters. These are countries that have the ability to develop, train, build, and host their own frontier models. I call them hypercenters mostly as an homage to the word hyperscaler.
19:08 Steph Smith? Right.
19:08 Anjney Midha? Which is that there have been a handful of companies that have the compute and the talent to actually build frontier AI. And now I think what we're seeing is a shift from just those companies driving a bunch of frontier AI to countries and regions driving it. And so if you're a small country and you're going, well, we certainly believe that it's important to have our own AI infrastructure. We want to be independent, but we don't have all the compute required to train these models, or we don't have all the talent locally. Then what you enter into is a joint venture with a country or an overseas partner that matches your values. And this is the really important thing about AI and AI models and how they're different from infrastructure like electricity is there's a fundamental encoding of human values in AI models
19:52 Anjney Midha? because they're trained on data.
19:54 Steph Smith? Yeah.
19:54 Anjney Midha? And the data has these local norms and cultural values embedded. And so if you happen to train a model on a bunch of internet data collected in the US, the models are just generally.
20:06 Steph Smith? American.
20:07 Anjney Midha? American. They're encoded with that.
20:09 Steph Smith? Yeah.
20:09 Anjney Midha? And if you're trained the models on. Data in France, they actually subtly have a bunch of different values encoded in the models that reflect those cultural norms. Uh huh. And so I think step number one, if you're a small country, is actually being a little bit crystal clear about which value systems you align with most out of the hypercenters. Uh huh. Now, it's not lost on people that the way the internet worked out was there essentially ended up being two internets, right? The Chinese internet and the rest of the world. Yeah. AI may not end up looking that different. And if you're a small country, what you really have to figure out is whose values align more with yours. A good historical precedent to look at here is the technology of money, right? Money is a pretty general purpose technology.
20:51 Anjney Midha? And what happened in the early 1900s with the modernization of finance is a number of countries started to ask the same question is, do we build or buy our own currency? Do we rely on the dollar? Right. Or do we have our own currency? And that led to the modern day currency regime where the dollar is a single global reserve currency. And that happened through a bunch of allied cooperation where a number of countries realized they did not have the local resources required to hold the peg of gold, right, to the dollar. And so I think what we're going to end up seeing is the emergence of, very similar to what happened with currency flows, where you have a couple of large countries that control their own sovereign currencies. Yeah. Right?
21:35 Anjney Midha? You have the US, you have China, you have India, and then you have a number of smaller countries that decided they wanted to be flow points. So you have Singapore and Ireland, and you have Luxembourg and Zurich that become massive global leaders in modern finance because they decide they want to ally with one of those power centers. Yeah. Right? So if you think about in the AI world, let's call regions at the frontier hypercenters. And then we have compute deserts, and these are places that have literally no install base of compute capacity to even be relevant. All the smaller folks have to figure out which of the hypercenters they want to align with. Right. And how do you become a modern day Singapore, Ireland, Luxembourg, et cetera, for the world of AI infrastructure?
22:17 Anjney Midha? And so it starts with deciding whether you want to be a compute desert or not. And if you're not, and you're going to actually embrace AI infrastructure as a government, then I think you've got to figure out which hypercenter you want to align with most.
22:27 Steph Smith? Right.
22:28 Anjney Midha? And then it becomes actually quite easy to reason about how to be a valuable ally.
22:33 Steph Smith? That's such a good parallel, because a lot of people think about resources in terms of the farmland that you have, the people who are working in that economy. But what you're pointing out is that countries for a long time have offered value or offered a resource in other ways. And as we think about AI, there's a few things that you've pointed out that countries can invest in, whether it's the compute capacity that they have, the energy resources to power AI and forward-thinking policy. So maybe we can break down each of those. How do you think about each of those blocks and how countries should be maybe maneuvering or investing in those things?
23:04 Anjney Midha? The good news is that there's only three or four ingredients here that really matter. The first is compute, which we've talked about. The second is abundant and low-cost energy, which powers the data centers. The third is data, just the availability of really high-quality tokens for these models to learn on. And the fourth is regulation.
23:20 Steph Smith? Yeah.
23:21 Anjney Midha? So that's the good news. Now, the bad news is that world is pretty unevenly split up, right? Some countries just have dramatically more compute than others.
23:28 Steph Smith? Yeah.
23:28 Anjney Midha? Others have dramatically more energy than others because of their natural reserves.
23:33 Steph Smith? Yeah.
23:34 Anjney Midha? So if you are in the Middle East, you may not have massive data centers yet, but what you do have is vast reserves of oil. And how you translate that into becoming a hypercenter is quite simple. It's the law of comparative advantage, right? You've got energy. You should use that to attract the world's best teams and companies and foundation model labs and so on by trading what you have with what they have. And so I'm quite bullish on allied ties between countries that recognize what their strengths are and then partner with other countries to fill that gap. And by countries, I mean private companies too, from other countries.
24:16 Anjney Midha? One of the things we may end up seeing in the coming years is jointly trained models between countries. Basically, I think for most countries, it's impossible to have total infrastructure independence at all parts of the stack.
24:27 Steph Smith? Sure.
24:27 Anjney Midha? What is much more feasible is to be great at one part of the stack and then collaborate with another sovereign or another country or region to achieve joint independence from a value system that you don't subscribe to, like the CCP. And so I actually think What's more important is for countries, regions, and frankly, in some of the world's largest companies that operate at nation scale, to assess which parts of the stack are critical to them that they must have independence from. And the answer there is a function of what asset they already have, right? It's their strengths. And then if they've got a critical gap to fill, it's to go and buy that. Now, in the long term, you might be able to build things out. But with infrastructure, especially of this kind, it can often take years, if not a decade-long scale.
25:11 Anjney Midha? So as an example, lower down in the stack from the model layer, you have the chip layer. And even below that, you have the lithography layer, right? There's a company in Tallinn called ASML that builds literally the world's most important machines.
25:24 Steph Smith? How many machines do they make per year? It's some very small number.
25:27 Anjney Midha? Each machine costs about $200 million. And I think they did $23 billion in revenue this year, 40% of which came, by the way, from China, because China was stockpiling ASML machines before a bunch of export restrictions kicked in.
25:41 Steph Smith? Yeah, and they're the only company that can actually make these EU lithography.
25:44 Anjney Midha? EU lithography of this precision. Now, is it feasible for the US to say we're going to build our own ASML like tomorrow? No, I mean, it's just going to take 10-plus years, right? EUV lithography just takes a really long time. On the other hand, is it feasible for a smaller country to say, "We're going to train our own local models at the frontier"? That's a little bit easier to do over the quarters of time scale. If you've got a leading research team.
26:08 Steph Smith? If.
26:09 Anjney Midha? If, and that's a big if, right? There's only a handful, really, of research teams globally that are capable of this. And so to answer your question, yes, I don't think sovereign AI or infrastructure independence means you have 100% ownership over every part of the stack. That's infeasible over the short term. It means that you don't rely on somebody for a critical part that you don't trust.
26:28 Steph Smith? Right. Can we talk about private companies for a second? Because you've brought them up a few times. How do you think about that dynamic where, as a nation-state, you're saying, "Okay, you need this sovereignty"?
26:40 Anjney Midha? Right.
26:40 Steph Smith? But at the same time, can you rely on that sovereignty through the companies that exist within your nation? Just using America as an example.
26:47 Anjney Midha? Right.
26:48 Steph Smith? Does the government really need to be involved? Or can they just let Anthropic or OpenAI kind of command that part of the stack? Or how do you think about the difference between government versus private enterprise?
26:59 Anjney Midha? The line is pretty stark in a few countries and is more blurry in others. So in China, the line is very clear. There's a law called the PRC 2017 National Intelligence Law. That says Chinese individuals and entities are required to support PRC national intelligence work by law. Which means if there's any technology that a PRC company has access to, they are automatically obliged to make that available to the government. And that's not the case in the United States.
27:31 Steph Smith? Right.
27:32 Anjney Midha? There are some cupboard types of technology, like dual-use technology, like classified defense technology, where if you are developing it, particularly if you're funded under a defense program, then you're required to make that available to government because the government's paying for the development of that technology.
27:46 Steph Smith? Right.
27:46 Anjney Midha? But by and large, the private sector in the United States and most other allied countries is by default protected from having to make its technology available to the government. It's not the case in the CCP. So I think that the question becomes for most countries is where on that spectrum do you want to exist? There's a general framework through which most infrastructure is categorized. Every country approaches it slightly differently, but the G5, the Five Eyes, the US, Canada, UK, Australia, New Zealand, we generally have a joint approach or framework to categorizing this infrastructure. And by and large, AI models have not been categorized as being dual-use or protected under national security.
28:27 Steph Smith? Yep.
28:27 Anjney Midha? The short answer is the history of technology has largely shown that if you'd like to win, then unlocking the best talents of a country with as few bureaucratic slowdowns usually ends up winning.
28:42 Steph Smith? Well, if we think about wanting to keep America at the frontier and we think about the different layers or ingredients that we talked about earlier, are there any high-risk areas that we think or that you think we're falling behind?
28:55 Anjney Midha? I think we go back to the four ingredients we talked about earlier of the frontier of AI, which is compute, data, energy, and laws.
29:03 Steph Smith? Yep.
29:03 Anjney Midha? Now, on the compute front, I think the private market in the United States is doing a pretty good job. It's pretty responsive to market demand. And I think there's no coincidence that the largest infrastructure businesses in the United States are chip companies and computing companies. Because I think we've generally done a pretty good job of letting the market feed that demand. I think on the data side, things are extraordinarily tough. Because, one, the Biden executive order last year was a starting gun that said, "Oh, AI is important, please do something about it," and left it to the states to figure it out. And the states have all taken a complete patchwork of approaches to data regulation. In 2024 alone, I think there were more than 700 pieces of state-level legislation that were AI-specific.
29:48 Steph Smith? Wow.
29:48 Anjney Midha? And a bunch of those laws, if you look at it, are really well-intentioned but atrociously implemented ideas for data regulation.
29:57 Steph Smith? Right, and impossible to adhere to.
29:59 Anjney Midha? Basically impossible to adhere to. And so I think one area where we're just handicapping ourselves is that there's no unified framework in the United States at the federal level yet for data, especially around training. And I think we needed that yesterday. Overseas, in a number of countries where rule of law, especially on copyright and IP and so on, is just less stringent, those labs are happy to just race ahead, whereas our companies here are trying to figure out what they should even comply with. And that greater hurts you more than actually a laissez-faire approach.
30:31 Steph Smith? Right.
30:31 Anjney Midha? I think our companies would be totally fine, the best founders at the frontier of AI would be fine in the United States being compliant. They just want to be told what to comply with.
30:38 Steph Smith? Not across 50 different states with different regulations that are changing and unclear, and in some cases, impossible.
30:44 Anjney Midha? Right. And there's also the fundamental scientific problem that they're just very real data walls that these models run into. And I do think one of the things that hurts frontier research in the United States and allied countries is a lack of government support in collaborating across borders to make more data available to allied regions. So that's number two. On energy, I think we've obviously hamstrung ourselves in the United States with nuclear. France, for example,'s embrace of nuclear 20 years ago has positioned them to have extraordinarily efficient data centers today. Whereas in the United States, I think we've basically shot ourselves in the foot around that. And then lastly, I think around inference regulation, what we're not doing enough of is making it clear who the liability rests on.
31:26 Anjney Midha? I've seen a number of proposals ahead of legislative sessions next year that want to hold model developers liable for the outputs of the inference, even of the misuses being done by somebody else.
31:38 Steph Smith? And what does that do that drives those very important developers elsewhere?
31:42 Anjney Midha? Essentially forces most startups to lose much-needed ground to big tech companies, and that entrenches incumbents more.
31:52 Steph Smith? So as we think about 2025, whether it's in the US or elsewhere, because I mean, this idea really is truly global. What are you looking out for? Or what should maybe, let's say, a legislator or let's say, the head of a nation, what should they be thinking about? And what are you looking out for in some of those decisions? Are you looking for countries that are buying GPUs or building out new energy centers? What are you paying attention to?
32:14 Anjney Midha? The leading indicators definitely compute. If you think about the AI supply chain, the first mile starts at the data center. That's the new atomic unit of sovereignty, I would say, which is a new thing. We've never actually had nation-states think about atomic units of an AI data center as a thing that countries should be purchasing. And I think about 24 months ago, we started seeing nations reason about that first mile as being important. So you've had an enormous amount of Nvidia's purchasing orders come from the balance sheet of governments. Just unprecedented demand they've been seeing from nation-states realizing that they want to be hypercenters.
32:48 Steph Smith? Yeah.
32:49 Anjney Midha? And that starts with them placing orders 12 to 36 months in advance to take delivery of GPUs, because if you don't get in front of that line, it's over. You're getting it after everybody else. So that was step one. The second thing I look for is founders who are deeply both technical, who often come from deep research backgrounds and scientists who've led frontier model development already, often inside of large hyperscaler labs. So an example is Arthur Mensch, who started Mistral. He worked at DeepMind. Or Guillaume Lemp, who led the initial Lama family at Meta, who are deeply mission-led and believe that they can help solve a bunch of these infrastructure problems for the world's largest governments. So there's a new class of founder who's both primarily technical and has their training in academia,
33:34 Anjney Midha? but is motivated to solve all the really hard problems that come with having to deliver, solve a bunch of these infrastructure problems for really large nation-states and regions. But I think if you're a founder like that who has the guts, then your impact on humanity ends up being quite generational.
33:51 Steph Smith? But now, let us convince you that the future of AI may not be so straightforward. Instead of models running in the cloud, perhaps we're bound for a future where many more applications will run on device.
34:02 Jennifer Li? I expect smaller on-device AI models to dominate in terms of volume and usage. This trend will be driven by use cases as well as economic, practical, and privacy considerations.
34:13 Steph Smith? That was
34:14 Jennifer Li? Jennifer Lee. I'm a general partner on infrastructure team.
34:17 Steph Smith? Here's her big idea.
34:19 Jennifer Li? My big idea is on-device and smaller generative AI models will become more popular in the next year. If you're a frequent user of Uber, Instacart, Lyft, Airbnb applications, I'm sure there are many, many machine learning models already running on our device. Very easily, when you load up a Uber screen, it's 100 models that's coordinating routes and giving you real-time price. What I'm more referring to is the generative models that are creating image, voice, video will become more prevalent in the same way to run on-device and within your applications, similar to these other traditional machine learning models.
34:55 Steph Smith? The models that we've seen in the last few years do take a lot of compute. And so can you square that with how much compute we can get from something like a smartphone? And also, these models, whether they're getting smaller, or how this kind of comes together?
35:07 Jennifer Li? Yeah. First, never underestimate the compute power on our smartphone is probably as powerful as a computer 10 or 20 years ago, that's thanks to Moore's Law. At the same time, the models are for especially smaller sizes of 2 billion, 8 billion parameter models. That's enough compute for them to run on-device, and it can generate and create very robust experience already, be it text or image or audio. And some of these models, if they are diffusion models, they're intrinsically smaller than large text models to be very capable. And there's another new set of tooling and also technology developed around distillation is if you have a very powerful large model can be distilled to a smaller parameter size model and still maintain a lot of capabilities that large model contains.
35:51 Jennifer Li? So both on the infrastructure side and also on the device compute power side, it's a perfect setup for the smaller models to be more popular.
36:00 Steph Smith? Totally. So I'm hearing a few things. I'm hearing that the smartphones are becoming more powerful. Some of these models are becoming more efficient, but that kind of brings us to the question of why. So why would we want to run these models on device? What are the advantages of that and also the disadvantages?
36:15 Jennifer Li? As consumers and day-to-day users, we are already spoiled by real-time and very performant applications. If you're talking to a chatbot, if you're talking to a conversational AI, if you're adding filters to your video and images on Instagram or TikTok, you don't want to wait for multiple seconds to load a new filter. You don't want to wait for multiple seconds for the chatbot to respond to you. Those are many real use cases that can really delight and improve user experience. Also optimization for compute. There are a lot of harder, more complex questions or video processing that requires going into the cloud. But largely, if it's, again, changing user experiences and improving the visual and sound effect of things,
36:58 Jennifer Li? it doesn't have to route through multiple servers, going through a network. So both from a user experience and efficiency perspective, it's a much better design to run some of the models on device. And then the last part is just privacy. Users do care about if my meeting notes is taken locally, I probably will use this meeting note ticker much more often than knowing some of the data is being sent to a server and they're processing a lot of my private conversations. So depends on the use case, again, for the application, I think that also improves the adoption.
37:31 Steph Smith? Absolutely. And that has my wheels spinning for sure in terms of maybe this unlocks new applications. So on that note, you mentioned a few already, but where might we see applications pop up? Or where perhaps are we already seeing applications with these on-device models?
37:45 Jennifer Li? First come to mind is real-time voice agents. It's a very popular topic and it's something I'm very excited about. We invested in and work very closely with this company called Eleven Labs, and that's one of the areas they're spending also a lot of efforts on is not just having the human-like synthetic voice, but being able to handle conversations fluently with end users and to get the latency down and also to think about what type of realtime exchanges you want to have with your AI companion, your support agents, or any sort of life coach. I think we do need to think about the modality and the latency in a much more, I guess, improved fashion. So I won't be surprised if some of those inference
38:29 Jennifer Li? workloads are running locally coming into the next top 18 month.
38:33 Steph Smith? Absolutely. And as we think about how maybe these different models also interact with other parts of a smartphone, let's say the camera, do you expect this to also maybe change user behavior and what we can do?
38:44 Jennifer Li? A hundred percent. You can actually reimagine the AR experience of if I point a camera to this room and I want to see a new surface and wallpaper and furnitures, there the technology is already there. Like we can actually leverage both generative AI and the camera and also prompting interaction to create new experiences already of how we interact with real physical life. And that's where I also think a lot of on-device models will play a big role of how to interact with a 3D world, how to interact with physical world, and not just using the camera for capture, but also using it as a projector.
39:25 Steph Smith? Definitely. And let me ask you about economics then, because a lot of the models that exist today do rely on inference and sending that inference up to the cloud and that costs money. Do the economics change if you all of a sudden have these models running on-device on the smartphone compute that already exists? Do the economics actually shift, or can we come up with new ways of monetizing in this new world?
39:47 Jennifer Li? Yeah, it's a great question, and I honestly don't really have the answer because even for larger models, the inference price has been dropping really significantly for the optimizations to be done. If it's a very workload-intensive compute, let's say using your computer or phone, I think it'll still have economic benefits, but I don't think it's a very direct answer of it's going to. Substantially reduce infrastructure costs for some of these applications. But architecting and structuring, sort of the whole tool chain, it does change sort of economics on the developer efficiency and sort of iteration speed. There is pros and cons when shipping in the cloud where you can launch more continuously.
40:28 Jennifer Li? On-device has its own challenges because you'll have to go with the updates with the application and with hardware. So there is that side of economics that I think will have impact from how teams are being structured in launching models in a hybrid mode. So I would encourage teams who are thinking of leveraging this technology, consider it more holistically.
40:47 Steph Smith? Super interesting. And as we think about that world, are there any players that you think really succeed here? Like, in one sense, I could see maybe the phone manufacturers. I could also see maybe, you know, the manufacturers of wearables being able to introduce all kinds of new applications. Think of wearing an Apple Watch, Fitbit, Whoop, things like that. Is it Nvidia that benefits in some way? Who do you think actually benefits from this idea of the models becoming more efficient? And these on-device models becoming a thing?
41:15 Jennifer Li? Right now, I've seen more interest and enthusiasm from the hardware development side, whether it's chips, it's the phone makers. I do think there is also a lot of interest from the model developers as well of just like proliferating the model adoption across different setups and devices. But I think over the long run, it's probably going to impact the whole supply chain.
41:38 Steph Smith? We talked about some of these macro trends throughout. How do you specifically see those trends shaping up in 2025? And is there anything in particular that you're putting your eye toward?
41:47 Jennifer Li? This will sound more like a consumer investor. I've been like a hardcore infra investor, but I am very excited about the mixed reality where generative models, 3D models, video models that really, again, makes the reality of what we are seeing today and through the camera lens, through the microphones. Much more creative world, even when sitting at home or when going on a ride. That's the type of experience I'm very much looking forward to. I think the foundation model technology is pretty mature. The infrastructure is getting ready. So I'm personally very excited about that sort of a new, new consumer experience.
42:24 Steph Smith? All right. I hope these big ideas got you geared up and ready for 2025. Stay tuned for parts three and four. And again, if you'd like to see the full list of 50 big ideas, head on over to a16z.com/bigideas. It's time to build.
Previous Jobs of the Future, Harnessing Earth Observation, & Gaming Tech Advances Next Super Staffing Healthcare, Codifying Compliance, & Scaling Services Page updated September 30, 2026 · transcript and summary are machine-generated
