Listener Q&A: 2024 Tech Market Predictions, Long Term Implications of Today’s GPU Crunch, and Will AI Agents Bring Us Happiness?

This week on the podcast, Sarah Guo and Elad Gil answer listener questions on the state of technology and artificial intelligence. Sarah and Elad also talk about the 2024 tech market, what type of companies may reach their highest valuation ever and the (former) unicorns that may go bust. Plus, how do Sarah and Elad define happiness? Hint: it’s a use case for a specialized AI agent. Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil Show Links: Cerebras Systems signs $100 million AI supercomputer deal with UAE's G42 | Reuters Our World in Data Show Notes: [0:00:37] - Impact of GPU Bottleneck in the near and long term [0:10:30] - Timeline for existing incumbent enterprises to use AI in products [0:11:50] - Vertical versus broad applications for AI Agents [0:19:33] - 2024 tech market predictions & how founders should think about valuations

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:05Sarah Guo?

  Hey, everyone. Welcome to No Priors. I'm Sarah Guo.

• 0:08Speaker 01

  I'm Alad Gil.

• 0:09Sarah Guo?

  This week on No Priors, we're back with another episode where we answer your questions about tech, AI, and everything in between.

• 0:15Speaker 01

  I think we have a lot of different questions that people have brought up this week that they were hoping we could cover and some topics that we thought would be kind of interesting.

• 0:21Sarah Guo?

  I want to go to one of our listener questions and I think a topic that's really popular with many of the companies that you and I work with in terms of access to computing for much smaller scale experiments.

• 0:35Speaker 01

  What's going on with the GPU crunch?

• 0:37Sarah Guo?

  Yeah, the companies that you and I work with, many of them are companies that, they need to use very specific infrastructure to train and serve large models, right? These work on GPUs and the structure of the industry is like, it's just not very robust, right? So you have a very small number of producers, Nvidia and AMD generally. And then Nvidia is very far ahead on the high-end processors that are most efficient for large-scale training and inference. Then you have the pandemic supply disruption, which we haven't fully recovered for. If you actually look at the supply chain,

• 1:20Sarah Guo?

  you go from the actual designers to the reliance on a few major foundries like TSMC. Expansion of this capacity is not easy, right? New fabs are billions of dollars. Yield is a very complicated thing. You can think of it as a massive precision manufacturing problem where temperature, pressure, chemical concentration, tool imperfections, new processes, materials issues, like anything can make production have lower yield or lower quality, right? And so like if you think about the speed with which the industry, driven by both large and small players, has decided that they want to do AI,

• 2:05Sarah Guo?

  like the physical processes cannot keep up with that demand. It's as if, you know, half the companies in the world over a Year-long period decided like, yeah, we need supercomputers, not superconductors, but gigantic networked GPUs.

• 2:23Speaker 01

  So like, what is the actual gap? So to your point, it sounds like much of the AI world is dependent on GPUs in order to train and then do inference on these big AI models. And the big suppliers are basically Nvidia, AMD, and then there's like a long tail of smaller folks. What is the delta between the amount of capacity that exists today and that's needed? Are we off by 2X, 10X, some other number?

• 2:47Sarah Guo?

  It's hard to say because right now, there's no way to explore like the price elasticity of these things, right? So, you know, just very specifically, like, the industry is kind of looking at deliveries in small quantity in September, larger quantities in December, January. Most of the large cloud providers are sold out for any scale for at least through April of next year. And so you have like really interesting dynamics, like large cloud players who, you know, are the biggest consumers of these GPUs already, like a Microsoft, going and buying from other providers for near-term, near-term supply, right?

• 3:32Sarah Guo?

  So I think one question that I ask you is like, hey, do you think this is a long-term thing? Do you think it's a very short-term thing? But I think it just goes back to like, the fundamental dynamics are, do you expect the demand for these chips to continue increasing at a pace that out creases the ability to scale a very physical, like, real-world process, right? Just to even be more specific, one of the challenges, like, I was talking to Jensen about this, and a bonder, like, not part of the GPU itself, but like a critical tool in the manufacturing and assembly of GPUs is very specialized. And so the ability to build any of these tools as well to enable these processes is a blocker.

• 4:16Sarah Guo?

  If you look at the demand from large labs today to continue increasing model scale and training time by magnitudes, I think it's hard to see that dynamic going away. What do you think?

• 4:30Speaker 01

  I feel like there's a couple different sort of second-order implications of the fact that we're seeing this giant GPU bottleneck. I think the first one is that we're seeing new sort of models that are dependent on GPU access or ownership as ways to create all sorts of really interesting monetization and potentially eventually cloud services. So that's things like CoreWeave or Foundry ML or other companies that are basically providing now GPUs in different ways. In some cases through aggregation or federating different sources of GPUs. In some cases, it's just having these large GPU clouds and being able to use them in really interesting ways. And one of the interesting, I think, side notes is that GPUs used to be very heavily used for crypto mining. And while

• 5:11Speaker 01

  crypto is down, it may actually be more economic to just use them for to rent out for AI training purposes or inference purposes. So I think that's one really interesting, almost like sectoral shift in terms of existing GPU capacity. The second is that a lot of the different players that are startups who've built their own semiconductors specifically for AI training, I think, are starting to see a lot of really strong pull. So for example, Cerebras, and I think we're going to have Andrew from Cerebras on our podcast in a couple of weeks, they just signed a $100 million deal with UAE for building nine supercomputers using their chips, which are optimized for AI.

• 5:48Sarah Guo?

  Amazing.

• 5:49Speaker 01

  And so I think they and Grok and other sort of semiconductor providers are going to find really strong pull during this period where people are desperate for any solution and they're willing to do, take the extra steps to really be able to utilize other forms of silicon. And so I think it creates a bit of an opening for other players in the market. And so it does seem like it's going to have these really interesting sort of cascading effects on members of the startup ecosystem and, you know, new players that are working against all this.

• 6:16Sarah Guo?

  Two sort of second order things are like, what do you do when scaling is blocked on capacity? Like, you try to be more efficient. It's not been an area of massive focus to date because people have been chasing the state of the art following Chinchilla scaling as the simplest path forward. But there are really interesting lines of research that are undervalued today unless the hardware supply crunch continues, including in dynamically figuring out or routing to Efficient models, so think of like the frugal GPT work or generally like distillation or even just a more intelligent choice of data for your pre-training or your fine-tuning

• 7:01Sarah Guo?

  training mix so you can use less compute, right, for the same or for improved quality. And I think like, everybody's been on this one path and an interesting second order effect is like, does it spread people out into lots of different directions in terms of chasing performance? I personally don't think the supply crunch goes away immediately and like, a part of the dynamic is just, you know, how much more people want to scale? And another part is like, you know, if this stuff is actually useful, then inference, like inference already dominates OpenAI compute usage, right? And so that demand will continue to go up.

• 7:40Speaker 01

  Yeah, I do think demand will only rocket from here, at least in the short run. And so the real question is the degree to which the semiconductor industry adjusts to that. And the reality is that people really view Nvidia's chips as the most advanced on the market right now. And so that means that a lot of it is just a bottleneck and how much can Nvidia scale up manufacturing? And there's other players like AMD, there's the startups we mentioned, Cerebras, Grok and others. But a lot of the capacity is just going to be how much can Nvidia and maybe AMD scale up in the short run at least? And so that may just cause some ongoing bottlenecks, assuming again that we continue to see this very rapid growth and AI and AI applications.

• 8:22Speaker 01

  I'm working on a blog post right now actually about this because it feels to me that we're still in the very, very early innings of this wave of AI adoption, right? It's not a continuum where we had CNNs and RNNs and now suddenly we have transformers. Transformers created a whole new capability set. And we're only, you know, eight months since ChatGPT and a few months since, five months, I think, since GPT-4. And so the only people who've really adopted this technology yet are the AI-native companies like OpenAI and Midjourney and a few other folks. And then we had the first wave of startups come, the perplexities and Harveys and characters of the world, as well as the first wave of incumbents adopting it, Notion and Zapier and sort of the very, very early founder-driven adopters.

• 9:06Speaker 01

  And so we've had zero real enterprise adoption in terms of real products at scale or close to zero. And you know, most enterprises, big businesses, take six months, nine months to see to their planning cycles. And then they'll spend a year prototyping. And then finally, they'll launch these AI apps. And so we're probably a year or two years before we really start to see large scale AI applications by existing incumbent enterprises in real products live everywhere. So from a ramp perspective, one could imagine that a lot of the future ramp in AI is coming in about two years or, you know, one to two years, something like that. So there's still a lot of room, I think, for the hype cycle, for increasing ongoing

• 9:47Speaker 01

  excitement, sometimes irrationally so. And then also for a sort of adoption of semiconductors and other underlying infrastructure. So there's still a lot to come, it feels like.

• 9:56Sarah Guo?

  I agree with you. And I still think we're really early in, let's say, like, the collective exploration of applications and constraints, right? Like, you had the people who were bleeding edge of just personal interest. Like, I think ChatGPT was, is looked at correctly as the starting gun for people to begin developing these AI applications, generally. But if you think about how long it takes to ship actual interesting products to market, and then the buildup of some collective understanding of like, how to make these models more useful in different applications, and then, you know, turn them into workflows,

• 10:38Sarah Guo?

  and then advance the state of the art, given a particular workflow, if you have a hypothesis on value, like, that all, that all takes time. So I think we're in anyone.

• 10:47Speaker 01

  Yeah, it's all been demos so far. Yeah. So I guess related to that, a lot of the interest and excitement right now is around agents. You know, I spoke recently, there's a group called the AGI House, which, you know, hosts these different hackathons in the Bay Area and stuff like that. And they had me come and help kick off like a agent hackathon they had and things like that. What do you think happens with the agent world? Like, what form does that take? And is it a handful of very broad agents? Is it highly specialized ones? Like, what do you think is coming there?

• 11:15Sarah Guo?

  Yeah, it's such a like powerful, broad idea that I think both will happen. Right? And so like the the overall idea is you you don't just talk to a chatbot or or query an interface. You have some sort of planning mechanism that is model driven that allows you to take asks autonomously, take actions autonomously, and like complete a more sophisticated task, often using other tools, and then return that result or report back on your work to an end user. Right? And so, you know, I think that is going to range from the pure

• 11:57Sarah Guo?

  consumer applications. So things like Inflection, which is going to, you know, have personalized that do more for you. Minion, which is working on like web agents. And and then, you know, I think like, there's been very recently more attention, or just more understanding of how powerful it is to have agents that in some way write executable code. Right? Because you can programmatically use many more tools, you can call APIs. And I think if that is do a task that is not a single query, but requires multiple steps in in analytics or in enterprise automation, or even, you know, within like, you know, companies that

• 12:42Sarah Guo?

  we work with, like Harvey, like a single legal task is actually a composition of thoughts, planning, attempts of research, like writing that a associate might do. And so I think it's going to be a pretty dominant paradigm.

• 12:57Speaker 01

  Yeah, it's kind of interesting, because if you look at past technology waves, and you ask about specialization versus sort of broadness, you know, are you building a broad-based platform that you can use for anything, or a vertical application that really helps you with one or two things well? Most of the things that really work are these vertical applications that help you really well. Now, some of them broaden and grow into the broad-based platform for everything. Right? Even in consumer, that's true. Like, Facebook started off as a college network. And in fact, it started with like five colleges and then they added all colleges. And then later they added the ability to add your work email as a way to register. And then they opened it up to everybody and then they started building the platforms on top of it and gaming and other things. Right. But it kind of happened sequentially.

• 13:40Speaker 01

  And there's kind of examples to that, you know, Google would be a very broad based thing from day one. It helped you discover information on the web, right? You needed a tool for that. But it feels like in the agent world, a lot of the people that I hear talking about ideas have these very broad sort of abstract ideas. And so an idea would be, um, I'm going to build an agent that is going to be your assistant. And you're like, okay, well, what is it going to help me with? And they say, everything. It's going to make you happy. And you say, well, I don't, you know, I'd love to be happy, but at the same time, you know, starting with a very targeted focused initial use case tends to be the best way to build product. A, because you know who you're building it for, B, you can really nail the use case. And there's the old sort of YC ism, which I think, which is really good, which is it's better to delight

• 14:23Speaker 01

  a small number of people than to have a very large number of people indifferent to your product. And so I think my, my bias for the agent world is if you're building an agent, start with something really targeted. If it's an assistant to help you, what exactly does the assistant do? Does it do background information searches on all the meetings you have that day? Does it specifically help with certain forms of scheduling? Does it help with other aspects of your day planning or synthesis of what you've done or follow up action items or whatever it may be? But choose one or two things and do them very well versus do everything. And then eventually you may build the thing that you start off that does one thing very well, but then broadens into everything. But usually starting with everything means you're not really doing anything deeply or well.

• 15:04Speaker 01

  And so I think that's, that to me is one of the main patterns, at least in terms of prior ways of technology development.

• 15:12Sarah Guo?

  I very much feel like this is like a very classic tension between, um, what I consider to be like, uh, I don't know, the like infrastructure platform engineering, like even research agenda driven approach that is like,

• 15:27Sarah Guo?

  Oh, you don't understand, like, the technology is general. We don't want to be taken off the research path that pollutes our data mix in a way that it is not a general purpose technology anymore, right? Or, you know, it can do anything, why limit it? Or even getting feedback from users, because you release this stuff, it is broadly capable, that they're doing everything with it. Some things much more successfully than others. And I think more of a, like, a product engineering, like, traditional, like, startup mindset that is, like, actually complete the task, right? And I definitely think the overall exploration has been skewed to one side, not as productively today. And one of the, like, even if you think

• 16:11Sarah Guo?

  from the research agenda, one of the reasons it is interesting to think about the, like, you know, have more focus, everybody's thinking about, but have more focus on accomplishing the specific task is, like, you want to be happy a lot. All I want to do is, like, never write boilerplate code again, right? And so if you think about-

• 16:32Speaker 01

  That's how I define happiness.

• 16:34Sarah Guo?

  Okay, great, then we're still the same. But, like, if you think about, like, okay, let's, like, complete one task. If I want to ask, you know, an agent to just, like, fix all the bugs in my software, then my ability to, like, successfully complete that task includes a lot of, like, bug fixing, specific techniques, right? Like, you could do test time search and then see if all of the different things that you generated actually execute, as one very simplistic example, right? And so, like, I think there are a lot of ways to advance in the research in very specific tasks that are much more tractable. But maybe I'm not thinking big enough.

• 17:19Speaker 01

  No, that makes sense. I think I would add one third piece to that framework you have, which is the research-driven versus product-driven. I think there's a third approach, which is infrastructure tooling-driven. And that's where you're like, I'm not going to build the agents, but I'm going to build the infrastructure that allows anybody else to build them rapidly. Now, sometimes those types of businesses or approaches work really well, and sometimes those things are solely an outgrowth of a vertical product that works really well, that you then open up the infrastructure for everybody else to use. And it's very case by case dependent. It's the difference between Stripe, where it's just like, we need to build payments for everybody. Everybody keeps building it over and over again. And the Facebook Auth platform, which only existed because you got to hundreds of millions of users, you could open up Auth as like a third-party service.

• 18:01Speaker 01

  And so I think as people think through that third angle of building infrastructure for others, they need to understand whether that infrastructure will be an outgrowth of an existing product area and benefit from the characteristics of the market liquidity of that product, or whether it's just a piece of infrastructure everybody keeps building over and over, and therefore it's a really good thing to just provide to the world. So I think it's kind of an interesting future topic.

• 18:23Sarah Guo?

  We are on a, you know, couple-month bull run at this point. 2024 tech markets, what's coming? Like, will people be able to fundraise? Will funds be able to fundraise our customers purchasing?

• 18:37Speaker 01

  You know, I think there's going to be basically four markets next year in some sense. One market is just AI, and I think AI will continue to run in different ways. And it'll look very expensive at the time, and a handful of companies will look really cheap in hindsight, just like with every other technology wave. And I think that's separable from the rest of tech that existed prior to the AI wave. For companies that fundraised in 2021, prior to being like AI companies, a subset of them, I think if I were to sort of divvy up that pie of those companies, sort of mid to late-stage private tech companies, not in AI, and what's going to happen to them next year and in 2025, I think a third of them are just going to go under, or a third of, I should say unicorns, will eventually just go under, be fire sales, whatever.

• 19:21Speaker 01

  They won't be able to ever raise money again. A third will be at the highest valuation they'll ever be at, ever, in the lifetime of the company. They'll reach their terminal value. And there's examples from 2014 of companies that went through that same wave. They, you know, raised in 2014, they went public a few years later, and then they never surpassed their market cap again. And then I think lastly, there'll be a third of companies that grow past it. And so I do think there's going to be a lot of carnage next year, and a lot of companies going under. And as those companies go under, three things will happen. Number one, it'll be much easier to hire people. I mean, people are already seeing that at startups. It's easier to hire again. Second, it should have follow-on effects and ramifications for commercial real estate, and we'll see a second shoe drop there.

• 20:02Speaker 01

  And then third, the venture capital community will be impacted because a lot of the things that they've been using to fundraise new funds or do other things with will suddenly go to zero. Their big unicorn success will go from a multi-billion dollar or billion dollar company to basically a company that isn't worth anything. And so I think that's going to have knock on effects to the venture ecosystem. But I think that'll take like two, three years to play out because all these things are a bit time delayed. But yeah, I think that other shoe still hasn't dropped in private tech markets. And a lot of it is just companies raised so much money in 2021, they still have lots of money. So everything still feels like it's continuing to go. But at some point that money's going to run out.

• 20:43Speaker 01

  So I think it's going to be a pretty bumpy 2024 and 2025 potentially.

• 20:48Sarah Guo?

  Yeah, my advice to companies that, you know, raised a very healthy valuations during that period of time and then are actually building businesses is to try to completely disassociate from that valuation. Because people will put themselves into all sorts of contortions to do a flat or up round to a valuation that makes no sense. Right? And if you don't have the historical context of that making no sense, it's an extremely painful sort of realization to have. But if you look at, there's this one analysis of actually the very best technology companies

• 21:30Sarah Guo?

  and the ones that endured from the internet bubble and how long it took those companies to reach the valuations they were at before the bubble burst. And it's a decade, right? And it's like startups don't have a decade to try to, you know, get to at par valuations.

• 21:47Speaker 01

  Yeah, I'm actually less worried about valuation. I think valuation is ephemeral, right? Effectively, every, roughly every tech company in public markets did a down round over the last year and a half, right? They all lost, or many, many companies lost 30 to 90% of their value, right? Effectively, they just did a down round in public markets because every day you're repricing a public stock. I'm more worried about the people who burn tons of cash. Yeah, they don't have a lot of revenue to show for it. And then when they're going to go out to raise more money, people say, well, you burn $50 million. You burn $100 million to generate $5 or $10 million of revenue. And so the issue isn't that your valuation's off. We can always reset valuation. It's the fact that you burned all this money and you don't have anything much to show for it. Right. And that's where I think the real issues will happen.

• 22:30Speaker 01

  Because you can always reprice things and people will be forced to and, you know, it'll just happen. But I think it's the underlying business case and business model that's going to be the real issue.

• 22:39Sarah Guo?

  Yeah, I guess like the unforced error there for companies who actually have the time to make the decision is the thing you want to avoid is like not adjusting your cost profile or, you know, holding on to that valuation until it's too late.

• 22:54Speaker 01

  Yeah, or just deciding it's the wrong business and it's not working. And, you know, the most important precious thing for you as a founder is your time. And I think people forget that you have this golden period in your life where you don't have, hopefully, a lot of other complications in terms of sick family members or school-related issues or whatever it is. And you can take risk and you have a low cost basis and you can do all these things. And that's the moment when you can best take risks to start a company, for many people, not for all. And, you know, you're really giving up the best years of your life, working on things that potentially may not work.

• 23:31Speaker 01

  Thanks for the discussion. It's a lot of fun.

• 23:33Sarah Guo?

  Yeah, super fun. Thanks to everyone who sent us your questions. Find us on Twitter at NoPriorsPod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.