Big tech earnings and the current AI debates, with Sarah Guo and Elad Gil

Host-only episode discussing NVIDIA, Meta and Google earnings, Gemini and Mistral model launches, the open-vs-closed source debate, domain specific foundation models, if we’ll see real competition in chips, and the state of AI ROI and adoption. Don’t miss our episodes with: Mistral NVIDIA AMD Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil Show Notes: (0:00) Introduction (0:27) Model news and product launches (5:01) Google enters the competitive space with Gemini 1.5 (8:23) Biology and robotics using LLMs (10:22) Agent-centric companies (14:22) NVIDIA earnings (17:29) ROI in AI (20:43) Impact from AI (25:45) Building effective AI tools in house (29:09) What would it take to compete with NVIDIA (33:23) The architectural approach to compute (35:42) the roadblocks to chip production in the US (38:30) The virtuous tech cycles in AI

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:05Speaker 00

  Today on No Priors, we're having a special episode of Sarah and me just talking. Hello, Sarah. How are you?

• 0:14Sarah Guo?

  Hey, Elon. What's going on? I see you a lot.

• 0:16Speaker 00

  Not much. Good to see you.

• 0:17Sarah Guo?

  Let's talk about models. What's going on in the model world?

• 0:20Speaker 00

  Yeah, I guess there's a lot of hand models that are emerging, so I was thinking of maybe trying to do that eventually.

• 0:27Sarah Guo?

  It's almost as good of a business as investing.

• 0:30Speaker 00

  I know, right? Yeah, so there's been a lot that's happened in the model world recently. Obviously, Google launched Gemini, which I think had a few interesting characteristics, both in terms of performance, but also the huge context window, right? It was million token context window. Companies like Magic, I think, in the past have actually put out like a 5 million token context window model and things like that, but it's really exciting to see that. And I think for certain application areas, like biology, longer context windows actually seem to be quite important. And so, for example, if you're doing a protein folding model and you have a short context window, you're often actually not encapsulating much of the protein, right? The average protein is, I think, something like 300 amino acids long, at least in the human genome. But there are things that are dramatically larger than that. And so you just can't capture it in some of the context windows being used for biological models.

• 1:15Speaker 00

  And so I do think this is going to be one of those areas that will end up being more important than people think, at least in the short run. But Gemini 1.5 seems to have some really interesting performance characteristics. There's obviously Sora from OpenAI, which was the video model that, you know, is just beautiful to watch. You know, there's other model companies like Pika and others that I think are doing exciting things as well. And then Mistral or Les Mis launched Le Chat, which is really the name of the product.

• 1:43Sarah Guo?

  Le Big Model.

• 1:44Speaker 00

  Le Big Model, Le Big Mac.

• 1:46Sarah Guo?

  I believe they call it Mistral Large. Yes.

• 1:48Speaker 00

  Yes, Le Large, Mistral Large. They launched that. And the thing that's really, really impressed me about Mistral is just the velocity of shipping. It's incredibly impressive. They went from basically starting the company to almost GPT-4 level in less than a year, right?

• 2:06Sarah Guo?

  Nine months, yep.

• 2:07Speaker 00

  It's amazing. And they have, you know, small performance models. They have Le Big Mac or, you know, the large model. They have chat. They have multiple languages. It's just, it's very impressive execution. So, and then I think the other thing that they just launched or announced was that deal with Microsoft, where, you know, they're now being licensed onto Azure.

• 2:29Speaker 00

  Yeah, and so I think the main models in Azure now are OpenAI, Llama, Mistral, and then some of the Microsoft models. So again, that's striking as well. So just very impressive progress by that company so far.

• 2:39Sarah Guo?

  I think the design space for what you actually want from models is certainly going to include state-of-the-art capability, and Mistral is very much going after that, and they've said so. But I think like, from the beginning, the company has talked about efficiency and latency and the ability to serve different use cases with that, and also, you know, being long-term proponents of retrieval, right? Like, one of the big debates in the research world right now, I don't know how much of it is a debate, but people are talking about it, I'm on one side of this, is that like, rag and retrieval is dead with sufficient context. And curious what you think here, but I'm more of the belief that it just opens

• 3:23Sarah Guo?

  up the set of trade-offs you can make between retrieval, more sophisticated retrieval, and model reasoning by having a larger context window versus saying like, we don't need any ability to work with a specific data set versus just retrain or stuff something into context.

• 3:41Speaker 00

  Yeah, we're gonna have both, in my opinion. The other thing I think that's very under-discussed, and this could lead into agent stuff, but I'd like to also spend a little bit of time on Gemini before I move to agents, is if you look at a lot of the optimizations that are done for areas where you had human-related sort of reasoning or other components, pre-LLM-based reasoning, a lot of it was happening at inference time, right? So when you were doing, when you were trying to build a better poker AI, a lot of what you did was, you know, certain types of tree searches or other things when you hit inference time, right? You built the model, but at inference, it did a lot of extra work. And I think that's also a little bit under-discussed in terms of probably a lot of what's going to happen in the future, particularly we get into agents and reasoning,

• 4:26Speaker 00

  is stuff that's happening at that point of inference, and then it's used to sort of feedback over and sort of continuously train or retrain a model over time, because I think that's the other piece of it is, you know, from a model perspective, you Spin up a giant data center and you spend $100 million over 12 months overall between all the different works that you do and everything to launch your next model. And then you have a file and then you use that file for the next year as you train the next model versus saying you're going to use some sort of continuous upgrading or training. And so all these things are going to shift over time. I think it's early in the technology cycle. And so all these things are going to happen. You know, one of the companies that has a lot of capabilities to do interesting things over time, of course, is Google. So I'm a little bit curious if Gemini has changed

• 5:10Speaker 00

  your opinion of sort of the AI model race and what role Google plays in the future. Has it not changed your mind much?

• 5:17Sarah Guo?

  I think the question on like whether or not Google has the ability to do the research work to have a competitive product has been answered, right? Gemini is a very impressive model. I think the capabilities that they have internally that they haven't released yet around additional like function calling and multimodality are also really, really impressive. And so the questions around Google are less about do they have, like they have all of these extraordinary advantages and you're the ex-Googleer. Like, I want to hear your opinion, but they have the distribution, they have the consumer behavior, they have all the data on like what the search behavior is, they have the

• 6:02Sarah Guo?

  data on what queries are valuable and which they would peel away and turn into like an answer. They know how to build like advertising auction systems and they have a great research team and enough GPUs and the model capabilities.

• 6:22Speaker 00

  Do you think it's progressive enough, though?

• 6:23Sarah Guo?

  Do I think the models are progressive enough?

• 6:26Speaker 00

  Yeah.

• 6:26Sarah Guo?

  One might actually ask if they're perhaps a little too far in that direction.

• 6:30Speaker 00

  Hmm.

• 6:30Sarah Guo?

  Right. And so I think like the question is actually can they steer Google to like focus on being competitive versus the many other demands from their employee base and like different missions that are not Brokering the world's information and like market cap.

• 6:52Speaker 00

  Yeah, it's interesting because the launch of 1.5 has made me more bullish on Google. And I was always actually quite positive on it, right? Like, I think I read a blog post a year or a year and a half ago, basically, about the model world. And one of the things that I mentioned at the time was I felt like Google was kind of a sleeping giant. And once it awoke, you know, it could really make enormous progress quickly. And just as Mistral is executed from scratch as a startup, which is extremely hard to do, right? You're literally building everything from the ground up. Although obviously there's open source to support you and all these other things. But fundamentally, you're just building an entire company. It's pretty amazing, right? Google has really accelerated its efforts. And it's had a series of launches over the last two, three months that have been quite impressive

• 7:35Speaker 00

  in terms of the velocity from cold start to having things that are externally accessible. And they have all the resources that one would need in order to do extremely well in AI, right? They have the compute, they have unique proprietary data, as well as all the data from the web, all the data from YouTube. They have specialized data that you could potentially opt into, like, you know, all your emails and your Google Docs. And, you know, they have this immense corpus of really valuable information. And then they have amazing talent. And so really, I think the thing that was lacking until recently was the will. And it seems like now, because of the competitive dynamic, the will has been reborn, right?

• 8:17Speaker 00

  And so it really feels to me like they are going to make really big strides going forward. And, you know, it's always possible that the velocity only increases from here for them.

• 8:29Sarah Guo?

  If I think about the domains in which these general LLMs are still not as capable, I mean, it's every domain, but in particular, not as capable as we want.

• 8:44Sarah Guo?

  Like, two of the areas, one you already mentioned.

• 8:48Sarah Guo?

  That I I'm I'm excited about includes like biology and then robotics. So maybe let's talk about that for a second. As a as a task, for example, if you ask ChatGPT to design a DNA sequence that can express Prisper Cast nine, it can't do that yet. Right. And if we think about cell design, protein design, protein optimization, a lot of these are areas where you have researchers showing like really exciting progress in use of transformers and diffusion models to get to much better predictions for, for example, drug discovery and target identification.

• 9:29Sarah Guo?

  And so I think, you know, I've seen a number of companies in this area of better understanding of biology that really feels like a different type of reasoning, a different type of data set. And as you said, even like specific context window constraints. And so I think that's an interesting one. And then on the, I don't know if you wanted to mention the robotics side, or if that's something you've been looking at too.

• 9:50Speaker 00

  The robotic stuff seems super interesting. It's a little bit earlier on than some of the other models and part due to data constraints, but it seems like there's pretty reasonable ways to generate some of that data now. So it seems like the, you know, in general, I wouldn't be surprised if 2024 and 2025 is the year of proliferation of models. Where we're going to start to see an expansion in terms of the different types that are covered, you know, chemistry and material sciences, et cetera, et cetera. Robotics will be part of that. Biology will be part of that. Maybe physics and math. I think maybe the last thing that is happening from a model perspective is I think the last few weeks have seen a lot of different sort of agent centric companies get up and running.

• 10:32Speaker 00

  And I think that's been a really interesting wave. And some of them again are taking very different approaches from the traditional, let's just build a giant LLM. And they're looking at things like AlphaGo or some of the game centric work that have been done in the past, you know, how do you build a better poker paper? Or how do you build diplomacy? How do you build Go? And there you have a very strong notion of acting sequentially based on changing information. You have some forms of what's known as self-play. You know, you you have the machine play itself a billion times ago and it learns new patterns based on that. You have really interesting approaches in heuristics and algorithms that time of inference versus training.

• 11:16Speaker 00

  And so I think that that harvest of knowledge is about to hit the world in the context of new products. And it'll take time for those products to emerge, you know, six months, 12 months, a year. But it does feel like that's another wave is coming where you're taking a fundamentally different approach to read that involves reinforcement learning, but is just different in terms of how you think about what you're actually doing in architecting and what you're inferencing and all the rest. So that's the one other area on the model side that I think is very exciting.

• 11:46Sarah Guo?

  One thing that I've seen here is that people are getting much smarter about agents as part of systems versus expecting to simply like instruct an agent and have it work with compounding failure across a bunch of tasks, right? In a general environment across any type of software, right? And so if it is operating in an environment that supports reinforcement well, like a game environment, or even a web application environment, but one that is constrained to particular tasks, or working agents working in domains that better support a sampling and validation, like code generation, like I'm really excited

• 12:31Sarah Guo?

  about that. And I feel like I've begun to see the glimpse of some of those things work. Whereas a very real question you could have asked in Q3, Q4 of last year would be like, is any of this stuff useful, right? Is it anything? And I think now it's like, it is.

• 12:48Speaker 00

  Yeah, people went too broad too early versus just saying, I'm just going to focus on a handful of targeted use cases or domains. And I'm going to figure out how do you create feedback loops in those domains so I can actually train effectively. And so, you know, the very early versions of this, even predating this LLM wave was, you know, hey, we're going to have a browser plugin and it'll watch everything you do, and then it'll do everything you do. Which is a very different problem from saying, hey, we're gonna make our PA better. We're gonna make good better. We're gonna make customer support better. We're gonna make, you know, X, Y, Z thing better. So I think the targeted approach makes a lot of sense.

• 13:21Sarah Guo?

  Yeah, and I think some of the teams working on this have also, they've actually experimented with post-training in environments where you can pay for, um, for, uh, human feedback data. Right? And if you do that, then you actually understand like the, um, the distribution of data you need, the scale of data you might pay for. And that's very exciting because it turns it like the agent problem from one that is, um, like, open-ended, untenable to just like, how much is it gonna cost to make a particular task work? And I'm massively oversimplifying here, but that is a very different proposition when scoped than, uh, like, as you described,

• 14:04Sarah Guo?

  the initial set of forays into agents, which is like, you know, we'll try to do anything.

• 14:10Speaker 00

  Yeah, that makes sense.

• 14:10Sarah Guo?

  And I think we'll still get there, but there is, um, like, rapid success on this front.

• 14:19Sarah Guo?

  Nvidia, everybody's talking about earnings. What do you make of it?

• 14:23Speaker 00

  I think earning money is an excellent idea. How about you?

• 14:27Sarah Guo?

  I think Jensen understands this better than everybody else. I think one thing that people have been talking about is whether or not this was a, like, short-term phenomenon, right? Like, if, um, there was only so much demand and once the supply chain caught up a little bit, um, there would be less insane growth. And I, I think now people are pretty confident, especially hearing Jensen's comment that they expect to continue to be supply-constrained through the rest of the year. Demand is just like much, much larger than I think most people expect on the capex side. And I think it's like worth understanding the upgrade cycle that drives that. Right? Because there's this huge efficiency incentive to upgrade from

• 15:12Sarah Guo?

  A100s to H100s to H200s to B100s. I was talking to one of my portfolio companies that's buying in the tens of thousands of GPU size and is skipping to B100s because they described it as like... Free money in terms of training efficiency. It's funny when somebody describes spending hundreds of millions of dollars as free money, but free money in terms of training efficiency, if you can actually get access to a cluster of a certain size. And so if if others feel that way, it is wild how much this like expands the server market.

• 15:49Speaker 00

  Yeah, it's probably a good time to run a hedge fund. I think in general, one thing that's a little bit under discussed is a lot of the emphasis on startups and startup rounds. I know, look, the startup raised a hundred million dollars or whatever. And the reality is a lot of the spend is the big hyperscalers and then other clouds that are building out right now. And then I think the other thing is that if you were to look at at least enterprise adoption of AI, it's still really, really, really early days. And despite that, if you look at Microsoft Azure revenue in the last quarter, they mentioned that revenue grew by 5% from AI related products, which if I'm doing the math right, if it's a $25 billion a quarter Azure sort of revenue, then that means they're adding something

• 16:32Speaker 00

  like one, one and a half billion a quarter in new spend due to AI, right? So that's five or six billion annualized. And so, you know, one thing that is a little bit perhaps not talked about is there's a lot more stuff coming. And over the next two years, three years, et cetera, as enterprises really adopt this at scale, we should anticipate as well that, you know, the need for compute will continue to grow. So it's really interesting to see both this replacement cycle you're talking about, the massive spend by big tech on LLMs, because they're driving most of the spend on LLMs, because they're they're the big rounds, right? The big rounds aren't venture capitalists. Investing billions of dollars is the big tech companies.

• 17:16Speaker 00

  It's Amazon and Google and Microsoft and Salesforce and Nvidia, actually, right? And then there's the enterprise adoption, which is still TBD. So yeah, there's a lot going on.

• 17:28Sarah Guo?

  On this point, if you look back a month, you know, AI years are like dog years. So a year to the meta earnings beat at the end of January. Did you did you see this article that David Kahn wrote at Sequoia, the 200, like AI's $200 billion question?

• 17:46Speaker 00

  Was this where you basically said, based on the spend, if you think of the ROI you need, then you need to generate hundreds of billions of dollars in return?

• 17:52Sarah Guo?

  Yeah.

• 17:53Speaker 00

  Or justify all the, yeah, all the spend that you had, yeah?

• 17:55Sarah Guo?

  Yeah, very succinct summary. And I was like, okay, yeah, that is the question. And I feel like the Meta earnings beat was the like, one day answer to that question. Right? So to your point, they're one of the large spenders. They said they're going to spend 30 to 37 billion dollars on CapEx in 2024, driven by like AI, driven by servers, right? Mark has this great like quote where he's talking about 600K H100 equivalent units of compute and saying like, there's no room for other people. But the response to all of the investment that has in in CapEx for

• 18:40Sarah Guo?

  training and inference at Meta over the prior years has been like a huge earnings beat from better targeting, leading to better conversion, better recommendations, leading to better engagement, better advertising tools, leading to better ROI, as well as like the cost controls that the rest of the industry is doing. And so they had this one day, I thought it was really nice that the number was exactly this too. They had this one day ad of 197 billion of market cap. Biggest single session ad before Nvidia, I forget where Nvidia ended up landing after their beat. But like, that's the answer, right? Like, you know, 197 billion of increase in enterprise value on 25, 30 billion of CapEx. Like, you should keep doing it.

• 19:25Speaker 00

  Yeah. Yeah, it's kind of amazing. It's kind of a related question because I remember Yuri Milner showed me this chart, which basically he looked at the aggregate increase in startup market cap and the aggregate increase of what at the time was like FAANG market cap. And obviously now there's like the magnificent seven or eight or whatever it is.

• 19:43Speaker 00

  And so if you looked at the top tech companies at the time, they added like, I don't remember, it was five or 10 times the market cap of all the startup ecosystem combined during the same period of time. And to some extent, you could argue we're going into the same thing, at least in the short run for AI, and we still haven't seen the monster AI companies emerge from scratch. And undoubtedly, those will exist. But at least for the next few years, it seems like where we're going to see that really huge market cap incremental add, maybe companies like OpenAI and some of the model companies, but also it seems like increasingly it's just going to be existing companies adding huge amounts of revenue and earnings and compute and everything else along the way. So it's back to like, maybe the right thing to do right now is just start a hedge fund.

• 20:28Sarah Guo?

  I think that also begs a question of how to think about like all of the other companies, like tech and not in terms of amount of impact from AI. I actually think it would be like a really fun lens to run a hedge fund with, because you can take a very long-term view of something that feels very secular, just classify companies this way and long, short, like take that strategy as the only lens. Because like, I do think that there are a number of services companies that are squarely in the sights of things that you will be able to significantly automate. And the only question is, which of these

• 21:13Sarah Guo?

  management teams is going to have the investment capability, technical talent, guts, conviction to invest the way Mark did through, you know, people were really mad about the capex spend for a few years at Meta, right? And I think the answer is mostly, especially some of these services firms, like, maybe they partnered to get there, but they mostly will not make the transition, I think.

• 21:38Speaker 00

  The other thing that isn't really discussed is the impact it's already having on some businesses. So obviously, ServiceNow had like a blowout quarter in part due to AIs. We're starting to see a lot of that enterprise adoption. One of the folks from Klarna posted today that they built an AI assistant that's powered by OpenAI that in its first four weeks handled 2.3 million customer service chats for them.

• 22:00Speaker 00

  And so I ended up handling two thirds of all their customer service inquiries. It was on par with humans in terms of customer satisfaction. It was higher accuracy, so it led to a 25% reduction in repeat queries. Customers resolved their errands in two minutes versus 11 minutes. It's live 24/7 in over 23 markets, communicating in over 35 languages. And it performed the equivalent job of 700 full-time agents. And so basically Klarna, in a few months or a year or however long it took them to build this, built this customer service chat product and replaced 700 people's worth. And they say that at this point, they have something like 3,000 full-time agents.

• 22:42Speaker 00

  And so it cut the agents needed by about 25%, right? And so it's this really interesting post from Klarna where they announced this. And then one of the things they announced as part of that is, you know, longer term, society needs to think about what this means for society. Because this technology seems to be so good for certain human level tasks. And this is back to that point of AI adoption and the enterprise is just starting. But how many years is it before every enterprise realizes that they can cut customer support dramatically, at least for certain types of products? Just through adding, you know, simple apps, you know? And so I think that's the other thing that is kind of happening in the background that isn't talked about that much, but you know, it's already starting to really show its face

• 23:26Speaker 00

  in pretty interesting ways.

• 23:28Sarah Guo?

  Yeah, well, I do think you're going to get this accelerated adoption that goes use case by use case, right? Where like in any market, you have early adopters that build it in-house or go get these solutions and are willing to take the risk when you don't actually know like what the impact will be, how well it will work. But as soon as one payments company does that and it's a better experience for the customer or has real like impact on operating cost. I think like you switch very quickly over to the entire sector being like, we have to adopt it in order to be competitive on both fronts.

• 24:05Speaker 00

  Yeah, yeah, this stuff tends to happen slowly and then suddenly all at once. And I think we're in the slowly phase right now. And I actually had my team go and take global services and look at that, right? And so if you look at spend on software in the US right now, it's about half trillion dollars. And software spend a year, if you look at

• 24:27Speaker 00

  human centric services, just payroll for things where GNI can probably impact things. It's three and a half to five trillion. So if you convert just 10% of that spend into AI revenue, you've effectively recreated the entire US market software industry and market cap. Right? And so these are huge trends that are coming and you can kind of imagine vertical by vertical. What are those things going to be? And then you can ask, is it going to be built as internal tools for companies? Is it going to be a new company that emerges that serves these things? Or is it going to be an incumbent who figures it out and adds it? And so this sort of customer support chatbot thing, you know, you would have thought that there's a company doing this for everyone.

• 25:09Speaker 00

  And it looks like in this case they're, they just did it internally or in-house.

• 25:16Speaker 00

  But you could also imagine an existing company like a Zendesk or somebody adopting to this. And the real question is, which of those three scenarios is going to happen? At least from a startup perspective, but from a technology wave perspective, this is massive. Right? And you can build in the feedback loops really easily for this type of product. Right? Because you can have the customer rated or thumbs up, thumbs down at the end of the session, et cetera. So you have a really good sort of RLHF or some sort of training set for it as well. So it's a, it's a product that should get better and better and better over time as you use it more.

• 25:45Sarah Guo?

  Yeah, I think one of the things that is an indicator of like where that services spend might be that gets externalized is actually like the big tech companies actually have, you know, they're tech companies, but they have broader businesses than I think sometimes they're given credit for. Right? Like Facebook, Meta interacts with SMBs as advertisers. If you look at anybody who has this like large commerce type customer base. So as you just mentioned, Klarna or Square or Meta or Shopify, like they've all done this now and it's working. Right?

• 26:23Sarah Guo?

  And, and so I think the fact that these are the companies that have the technical teams that are capable of doing it in-house is a nice indicator for like, well, if it's that effective, everybody else should too. And the question is, I think not every segment of customer, like retailers with. Enough of a technical team to build an e-commerce presence may not build this themselves. Then it's a more likely scenario that either an incumbent or a new company, be it Sierra or something else, ends up owning that customer service segment.

• 26:53Speaker 00

  Yeah, 100%. Yeah, we have a long list internally of like the companies that I think should exist in this space, right? Because there's so many obvious ones and very few companies exist for most of them, if any companies. And so I think it's back to this idea that there are these human capital waves happening in AI. And the very first wave we saw was researchers and they built early model companies and they built some of the early applications like perplexity and Harvey and all these things were actually started by people who were working on models initially. And they were just closest to the technology. So they knew what to do. And then the second wave of human capital was like infra people because there was the second closest to LLMs. And then the third wave, of course, is going to end up being application builders, but many of them were not aware that any of this stuff

• 27:38Speaker 00

  was important until ChatGPT came out 15 months ago. And they're just starting to show up, right? It takes them nine months to quit their job and a few months to figure out what to do and find a co-founder and a few months to build a prototype. And so we haven't seen anything yet really on the app wave. You know, all the apps or many of the apps so far were started by people who were very close to the research community. And then it's kind of permeated into other areas with some things growing really fast, right? There's like half a dozen medical scrapping apps that all seem to be growing at a pretty good pace. Or there's a few other application areas where it seems like there's a number of people working, but then there's lots and lots of spaces where it seems like nobody's doing anything, which is which is kind of weird, honestly.

• 28:15Sarah Guo?

  Yeah, there's a joke that the foundation model companies are here to replace all the jobs, but they don't understand what any of the jobs are. And I think there's like a little, a little bit of truth in the sort of exposure to what happens in, you know, a broad range of companies in terms of functions and outsourced services. And so I think that is the opportunity, right? Like now it's a race for people who are just great engineers, smart about a domain to go experiment on the fringe of that. And I still think there's opportunities around, like you and I have talked about. Um, the domain areas where you might want specific models or verticalized companies still,

• 28:56Sarah Guo?

  and we should, we should talk about that. But I, uh, I, my team and I just gave a presentation at this AI and production conference about how if 2023 was the year of infrastructure, like 24 is the year we begin to see applications. So I think we're pretty aligned there. I do want to ask you like one thing before we move away from all of the earnings stuff, which is, um, the most obvious place somebody is already making money is either like cloud providers, inference providers, or just Nvidia as a chipmaker. What would it take to compete, to have like a second source with Nvidia?

• 29:31Speaker 00

  I think there's a few different approaches, right? I mean, fundamentally, if you look at what people claim is a defensibility and part of Nvidia, it's a mix of chip performance, CUDA and interconnect. Um, you know, Nvidia bought, uh, Milonex back in 2019. It was an Israeli company, uh, to basically provide the interconnect side. I think that was like a $5 billion acquisition. So it's quite large relative to Nvidia's market cap at the time. Um, and then obviously CUDA has been developed over many years. Uh, uh, and then obviously they, they've iterated really well on these sort of different generations of chips. Um, so minimally, you at least need the, you need some form of silicon as procurement, and then you need to make sure that it's actually able, you're, you're able to, um, use it effectively, and then you're able to scale it, which is sort

• 30:15Speaker 00

  of the interconnect side. Um, and there's the incumbent side of it, right? AMD is obviously working on this. Intel's trying to, et cetera. And then there's the, um, the startup side of it where we've seen things like Rock, uh, emerge where they have very fast inference for open source models as well as language models, which is pretty striking. You have Cerebras, which has taken a fundamentally different approach to the chip side as well. So, you know, there's a few startups that I think have some interesting early hardware. And then there's some new companies like ads that have talked publicly about how they're really focused on transformer-based models and architectures for chips that they're building. So, um, there will be this potential wave of second sourcing over time, but. You know, in general, if you look at many of the most advanced chip markets historically,

• 30:57Speaker 00

  at least, there's tended to be a winner, or I should say a leader, and then there's been, there's tended to be a second place party. And that was, you know, during the microprocessor world, that was Intel, and then AMD was number two. And, you know, in mobile, it kind of morphed a little bit, right? You had Qualcomm and ARM doing different things, but both quite successfully. But I think Qualcomm was always, at least for a period of time, the bigger company, although ARM is much larger now. I should actually check that in terms of market cap.

• 31:27Speaker 00

  Yeah, Qualcomm is 176 billion, and then ARM is 140 billion. So they're pretty close, actually, now. There used to be a pretty big disparity between the two. In part, that's because ARM is being used now in sort of broader ways. So, you know, you kind of tend to see these market structures in semiconductors where there's a leader and then a second place. And I think part of that is traditional Moore's Law chip generation related stuff. I don't know how that will hold up or how that'll morph in AI. I don't know if you have an opinion on that.

• 32:01Sarah Guo?

  Yeah, the, you know, the way Jensen has described advancements in chip performance tend to be more memory management and new techniques versus just like transistors fitting on a particular die size. And I think somebody else at Nvidia called it Jensen's Law of like ability to get performance from full system. But the only thing I'd add to your description of competitiveness here is also like manufacturing, even for these fabulous chip design companies, is a big deal, right? Like, so you got to do what you said, design something better, including interconnect, design an entire, like, build an entire software ecosystem, CUDA's been around since 2006. But after that, you have to go get capacity at TSMC,

• 32:46Sarah Guo?

  right? And then you need to get yield up, and then you need it all to be competitive in terms of pricing. I think the desire, like, the economic pressure given to trillion of market cap and more demand than Nvidia can support is higher than ever, but I think the moat is actually really, really deep. And so when I think about like, what could be, what could be enough to go disrupt that? I've seen, I'm sure you've seen many of these companies, but I've seen a few different Approaches, it could be a chip and system designed for like specifically very much around latency. But the other thing that you said, right, like something, for example, optimized to

• 33:28Sarah Guo?

  transformers as an architecture, you're taking a bet around how much stability there is around a particular architectural approach. And I think that's felt like a quite good bet for a while now, but for the first time in a long time, there is some interest in in things like state space models with companies like Cartesian and some some alternatives, right? If you're a really big company with your own use case, right? If you're Meta or you're Google and you you know, either have like the entire ad system recommendation, surveying spam, et cetera, or all that like search and your own cloud, then you don't need to make everything work on the software ecosystem side.

• 34:13Sarah Guo?

  You just need to make one application work. And, you know, these companies also acquired Teams in, but that's how you end up with like TPUs and trainiums and all that. But I would love to meet companies in this area and still haven't haven't seen something that's gotten me over the edge, even in a place that is so obviously economically fertile.

• 34:34Speaker 00

  Yeah, I think one thing you pointed out, which was interesting to expand a little bit on is TSMC and the whole fable semiconductor world where you're basically, you know, outsourcing the development or the manufacturing of the chip to a handful of players, TSMC being the bigger, but there's the biggest, but there's there's there's one or two others that are big enough to at least handle some volume. And, you know, there's been this push to try and repatriate semiconductor manufacturing to the US and it's run into all sorts of obstacles that are pretty avoidable, environmental reviews that go on endlessly or other things that have prevented. People are actually starting to build these things that take many years to build. And it's been interesting to watch that in Japan, they're starting to actually have

• 35:15Speaker 00

  really interesting development of fabs specifically for this purpose. And so I'm increasingly wondering whether Japan emerges as sort of a second source location in part to geopolitically hedge Taiwan. But I think that's something also to kind of watch in terms of where are you actually seeing fabs go out? And how do you think about that geographic distribution? But also, why is the US in some sense getting in its own way for something that has pretty broad-based strategic importance on multiple levels, including national security?

• 35:44Sarah Guo?

  If you listen to the TSMC CEO about this, he talks as much about the human capital and the cultural elements of human capital required to make a place like TSMC work as the CapEx spend, right? And the access to equipment and the need to actually build the fab. I think that's pretty interesting because we can invest a great deal, but it's very hard to change culture. And so I do think that there's one version of like, maybe you have fabs in Japan or

• 36:28Sarah Guo?

  Mexico or Southeast Asia or just a broader global supply chain for chip production? Or maybe you have robots making chips?

• 36:38Speaker 00

  Yeah, I mean, that's all true, but the flip side of it is Intel has manufactured chips in the US for a long time. TI did historically, right? But Intel still does. So I don't think there's a complete lack of human capital. Obviously, it's concentrated in part in Taiwan and to secondary extent in Korea right now. But I do think there's the capability to do it. And I think, again, there are other things that are getting in the way, I think, even before that. Can you even break ground on the plant? Maybe step one, right? We should start with the basics and then we can deal with culture when we actually have a fab.

• 37:10Sarah Guo?

  Yeah, well, and I'm, I'm, I'm, I guess, very willing to believe that these companies and industries didn't exist in the place they, places they do without like great leaders for TMCS, MC or otherwise. And so like, maybe it's not a solvable problem. Like, I'd be curious if you believe in the Intel fab business that they're trying to push and push to other customers now. But to me, it's not binary. It's like, of course, we can like make chips in America. The question is, can we make them without the churn and with the yield and cost to make them competitive? But maybe it's so important, like you don't need them to be competitive for some period.

• 37:51Speaker 00

  Yeah, and also my point is, we're already doing that for Intel, right? Intel's fab business is in the US, not the, not the, the fabulous TMCC style business. Just making their own chips. They've been doing it for decades in the US. It's been fine. It's been high yield, you know?

• 38:06Sarah Guo?

  Yeah, it's been, it's been fine, but it's also been behind in terms of process technologies, right? But maybe, maybe that's not a human capital issue. Maybe it's other, other issues at Intel.

• 38:16Speaker 00

  Yeah, it seems like it's a other issue. Yeah, I think my general take on the whole market is the more I learn, the less I know in AI. And it's the opposite of every other field I've ever been in. Usually the more you learn about something, yeah, usually the more you learn about something, the more you can create sort of these straight line hypotheses or you know, what you know kind of compounds and it's static. And I feel in the AI world, like every week, there's like so many new things that your entire world model shifts.

• 38:45Sarah Guo?

  In like a fun way?

• 38:49Speaker 00

  Yes, it's, it's fun to be exhausted. But I think, um, you know, there, there's just so much going on and the pace of it really feels like, you know, that, that early slope into the, you know, the exponent that is a singularity or however you want to phrase it. But it really feels like this self reinforcing loop of new stuff. And honestly, a lot of it was kind of held back. And the larger tech companies and now it's kind of flourishing externally and that's creating competitive pressure on the larger companies and the larger companies are reacting and that's spawning more startups and it's just this really interesting virtuous cycle. And to some extent, the big tech companies are helping fuel it all by then funding the companies that are working at very late stages with huge rounds and they're funding

• 39:32Speaker 00

  a lot of the compute in the industry in a way that's, you know, at least in order of magnitude, maybe two orders of magnitude more than what the venture community is doing. And so it's this really interesting virtuous cycle of startups come out that accelerates big tech doing stuff that causes some people to leave big tech to do some interesting things externally. They then get funded by big tech and that accelerates both themselves and big tech and you have this kind of interesting cycle happening right now. So it's very exciting days.

• 39:58Sarah Guo?

  Yeah, I drew, I drew a slide that has like, as you might hope, like a bunch of reinforcing cycles. It's very fancy. And the one I would add to that is like what we started talking about, which is when something begins to work, if it is actually valuable, like the Klarna thing that you described, like at some point, if it's valuable and it moves the needle in the business, you have to do it like as a part of the competitor set. And so I think like we started with this like narrative driven thing where, you know, CEOs would say that they're going to do AI because like the markets believed that was the future and it was very generic. And you see that show up in the spending numbers or at least the expectations around spend, right? I was looking at this survey from

• 40:43Sarah Guo?

  one of the investment banks that says like Fortune 1000 IT budgets go to five to eight percent this year instead of three to five percent generally. And it's all because of AI. Like that's pretty big, right? That's like two X. And like if that's true, then that's also part of the reinforcement cycle here. Cause if the companies start to work, then they get to continue building these products, VCs, you know, or investors like us will, will keep, keep trying. So I think it's pretty exciting.

• 41:09Speaker 00

  Yeah, it's RLPA. Yeah, RLPA.

• 41:13Sarah Guo?

  Rolls right off the tongue.

• 41:15Speaker 00

  Reinforcement learning through product adoption feedback. You're welcome.

• 41:21Sarah Guo?

  Well, I'm just going to plug that in to chat GPT and have it write the paper, but I will be sponsoring author if you'll be first author.

• 41:31Speaker 00

  Yeah, I'll see if I include you.

• 41:36Sarah Guo?

  Academic violence.

• 41:39Sarah Guo?

  Find us on Twitter at NoPriorsPod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.