Dylan Patel - The Infinite Demand for Tokens, Claude Mythos, and Supply Constraints - [Invest Like the Best, EP.469]

This is my second conversation with Dylan Patel. Dylan is the founder and CEO of SemiAnalysis, where he tracks the semiconductor supply chain and AI infrastructure buildout. This conversation is about the supply and demand of tokens. On demand, Dylan describes something completely explosive. He explains why the frontier model is the only model anyone wants, and willingness to pay for it is nearly unbounded. His own firm has gone from tens of thousands of dollars in AI spend last year to seven million this year. On supply, we walk through the bottlenecks across memory, logic, and fab equipment that will determine how fast any of this can scale. We also cover Claude Mythos and what the leading labs need to do to fix their growing public perception problem. For the full show notes, transcript, and links to mentioned content, check out the episode page ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠here⁠⁠⁠⁠⁠. ----- Become a Colossus member to get our quarterly print magazine and private audio experience, including exclusive profiles and early access to select episodes. Subscribe at ⁠colossus.com/subscribe⁠. ----- ⁠Ramp’s⁠ mission is to help companies manage their spend in a way that reduces expenses and frees up time for teams to work on more valuable projects. Go to⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠ramp.com/invest⁠⁠ to sign up for free and get a $250 welcome bonus. ----- Trusted by thousands of businesses, ⁠Vanta⁠ continuously monitors your security posture and streamlines audits so you can win enterprise deals and build customer trust without the traditional overhead. Visit ⁠vanta.com/invest⁠. ----- WorkOS is the infrastructure B2B and AI-native companies use to sell to enterprise. It covers everything enterprise security requires: SSO, SCIM, RBAC, Audit Logs, AI governance, and more. Trusted by 2,000+ fast-growing companies, including OpenAI, Anthropic, Cursor, and Vercel. ----- Rogo is the AI platform for finance. They're building agents for Wall Street that are trained to understand how bankers and investors actually do work: from diligence and modeling, to turning analysis into deliverables. To learn more, visit rogo.ai/invest. ----- ⁠Ridgeline⁠ has built a complete, real-time, modern operating system for investment managers. It handles trading, portfolio management, compliance, customer reporting, and much more through an all-in-one real-time cloud platform. Visit⁠ ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ridgelineapps.com⁠. ----- Editing and post-production work for this episode was provided by The Podcast Consultant (⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://thepodcastconsultant.com⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠). Timestamps: (00:00:00) Welcome to Invest Like The Best (00:02:29) Intro: Dylan Patel (00:03:09) Semi Analysis AI Spend: Zero to $7M (00:05:16) Real-World Examples of Claude Code (00:11:41) Token Demand: “Completely Explosive” (00:14:48) Why Everyone Wants the Frontier Model (00:15:36) Mythos: Biggest Model Capability Jump in Two Years (00:20:54) Fear of Rapid Model Progress (00:23:45) Robotics as the Next Demand Wave (00:26:03) Scaling Laws & Compute Efficiency (00:27:24) OpenAI vs. Anthropic (00:31:33) Supply Side: Bottlenecks Across the Stack (00:33:26) TSMC CapEx Could Cause a Shortage (00:36:45) CPUs, ASICs, and FPGAs (00:40:12) Tokenomics (00:42:20) Protests & AI Backlash

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Patrick O'Shaughnessy?

  Ramp is the only platform built to make your finance team leaner, faster, and better, saving businesses 5% annually on average, so you can stay focused on growth. Ramp just opened for business in the United Kingdom, so if you're running a business in the UK, you can now use Ramp's AI-powered finance platform to manage cards, expenses, bills, approvals, and accounting all in one place. I run my business on Ramp, and so should you. Learn more at ramp.com/invest. Rogo is the AI platform purpose-built for financial institutions, serving tens of thousands of bankers and investors at hundreds of leading firms worldwide. Rogo's AI agents autonomously execute large chunk of your firm's workflows. They can screen deals,

• 0:42Patrick O'Shaughnessy?

  draft SIMs, run buyer outreach, and do diligence on data rooms, all with full security and compliance handled. Every action feeds Rogo Intelligence, the firm's context layer, turning years of deal experience into a secure, governed institutional memory that makes each new transaction stronger than the last. To learn more, visit rogo.com/invest. OpenAI, Cursor, Anthy, and Vercel all have something in common. They all use WorkOS. To achieve enterprise adoption at scale, you have to deliver on core capabilities like SSO, SCIM, RBAC, and audit logs. Instead of spending months building these mission-critical capabilities yourself, you can just use WorkOS APIs to gain all of them on day zero. Visit workos.com to get started.

• 1:30Patrick O'Shaughnessy?

  Hello and welcome, everyone. I'm Patrick O'Neal, and this is Invest Like the Best. This show is an open-ended exploration of markets, ideas, stories, and strategies that will help you better invest both your time and your money. If you enjoy these conversations and want to go deeper, check out Colossus, our quarterly publication with in-depth profiles of the people shaping business and investing. You can find Colossus, along with all of our podcasts, at colossus.com.

• 1:54Speaker 01

  Patrick O'Neal is the CEO of Positive Sum. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of Positive Sum. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of Positive Sum may maintain positions in the securities discussed in this podcast. To learn more, visit psum.vc.

• 2:22Patrick O'Shaughnessy?

  This is my second conversation with Dylan Patel. Dylan is the founder and CEO of Semi Analysis, where he tracks the semiconductor supply chain and AI infrastructure buildout. This conversation is about the supply and demand of tokens. On demand, Dylan describes something completely explosive. He explains why the frontier model is the only model anyone wants, and willingness to pay for it is nearly unbounded. His own firm has gone from tens of thousands of dollars in AI spend last year to 7 million run rate this year. On supply, we walk through the bottlenecks across memory, logic, and fab equipment that will determine how fast any of this can scale. We also cover mythos and what the leading labs need to do to fix their growing procession problem. Please enjoy my conversation with Dylan Patel.

• 3:02Patrick O'Shaughnessy?

  You told me this incredible story about how your own team's use of tokens has changed dramatically this year.

• 3:09Speaker 02

  Yeah.

• 3:09Patrick O'Shaughnessy?

  Can you retell that story? And what it is teaching you about what's going on in the world?

• 3:14Speaker 02

  Last year, we thought we were heavy users of AI. Everyone's using ChatGPT. Everyone's using Claude, providing whatever subscriptions anyone wants, on the order of spend of tens of thousands of dollars for our firm. This year, the spend has just skyrocketed. And it really started in late December with Opus. That included Doug Loflin, who's president. And he's very much like leading the charge in the sense of non-technical people using AI for coding. And so he's basically pilled the whole firm slowly over time. I think he's been the leader in doing that. Obviously, the engineers were using it anyways. But the spend in January just started to inflect and rocket and rocket and rocket and rocket. We signed an enterprise contract with Anthropic, and it's gone to the point where now,

• 3:58Speaker 02

  I think when I last talked to you, it was 5 million spend rate. It's actually 7 million spend rate now. So we're spending 7 million.

• 4:03Patrick O'Shaughnessy?

  That was last week, by the way.

• 4:05Speaker 02

  And a lot of that is just the usage. People who have never coded before are using Claude code and spending thousands of dollars sometimes a day. And it also like some people spend thousands of dollars one day or spend a couple hundred dollars a couple of days, and then they go back to a thousand dollars. It's very variable across each individual user. But across a firm, we're spending $7 million a year now on Claude code at the current rate versus our salary expense. Being in the neighborhood of $25 million. So we're north of 25% of spend on Claude code as a percentage of salary. And if this trajectory continues, then you know, we'll spend more than 100% by the end of the year, which is a bit terrifying. Thankfully, I don't have to decide between people and AI because

• 4:47Speaker 02

  our company is growing so fast. It's more so like, okay, well, I don't have to hire nearly as fast and I could spend a lot more on AI and it works and we just grow faster. But I think other folks will start to reckon with the fact that huh, if this person can do the work of five to 10 to 15 people using Claude code, then all of a sudden I should probably cut people. And the use cases are so broad.

• 5:09Patrick O'Shaughnessy?

  Give a couple of examples.

• 5:10Speaker 02

  Okay. So for example, one thing is we have a reverse engineering lab in Oregon that we've been building for a year and a half. We have a bunch of fancy microscopes, scanning electron microscopes. The whole purpose of this is you reverse engineer chips. You get the architecture out of it. You get the materials that they're using to manufacture. And this is some of the data we sell. This is a very slow process of analyzing that data. Instead, one person on the team, they've been able to spend with a couple of thousand dollars of Claude tokens. They've been able to create this application that is GPU accelerated, runs on a server that we have at CoreWeave. And anytime we send it an image, it will take the picture of the chip and overlay where every single material is, oh, this part is copper. Oh, this part of the gate is tantalum. This part of the gate is germanium. This part of the gate is cobalt. And so you can do a finite element analysis of the entire stack up of the chip

• 5:55Speaker 02

  very, very quickly, visual with a dashboard, GUI, it's everything, few thousand dollars of Claude. The person previously worked at Intel and he said that was an entire team's job to build that and maintain that. Rack that up across the entire firm. It's insane. Another example that I think is super fun is Malcolm. He's an economist at a major bank before. Their economist department was like 100 or 200 people. What he built was the most incredible thing ever. He piped all of this different data, Fred data and all these other data, right? Employment reports and all these other things from various APIs. We signed a couple of contracts with folks to get API access to data. Pulled it all in, started running regression, started looking at the impact of various economic revolutions on the economy from a deflationary inflationary perspective.

• 6:41Speaker 02

  The Bureau of Labor Statistics has this entire set of 2,000 tasks. And so he did that with AI, which ones can be done by AI, which ones cannot, and grading them across a rubric. About 3% are doable now with AI. And so he's created this metric so that you can measure things that can be done by AI, what the cost of being able to do those with AI, and therefore the deflationary aspect of it. Phantom GDP is what he's called it. Output can go up, but because cost falls so much, actually GDP theoretically shrinks. So he created this whole analysis and a brand new benchmark of language models, a set of evals across 2,000 different evals.

• 7:13Patrick O'Shaughnessy?

  He does all by himself?

• 7:14Speaker 02

  It's all by himself, yeah. And he's like, dude, this would have taken the team of 200 economists a year. He's like completely cracked out on Claude. He's like, everything has changed.

• 7:22Patrick O'Shaughnessy?

  How do you think about as a business owner, going from close to zero to 25%, accelerating towards whatever percent of total spend? At what point are you like, whoa, I need to put the brakes on this? And be careful how much we're spending. Maybe we don't need to spend on the most cutting it on Opus 4.7, which came out today. Maybe I can throw it back to something that's a little bit cheaper.

• 7:42Speaker 02

  I think I'm in the information business. We sell analysis. We do consulting. We create data sets. I don't see why this wouldn't be completely commoditized on a pretty rapid basis if I'm not constantly improving. My first product that I was selling as a data set, there's more people trying to do it. Now we've made it constantly better and better and better and more detailed. And so therefore it sells a market. But the way we were doing it in 2023 is not terribly different. It's basically what everyone else is doing now. If I don't move up the bar, then I will be commoditized. If I don't move fast enough, I will also lose my edge. So the question is, yes, AI commoditizes things, just like it commoditizes software. Those who can move fast and keep control of their customers and keep providing them an awesome service

• 8:26Speaker 02

  and keep improving the service, won't shrink, they'll grow faster. Those who are incumbent and not doing anything, they're going to lose. And so it's a bit of an existential. If I don't adopt AI, someone else will and they will beat me. Another easy example is the energy space. So we've had a few energy analysts for like a year now. We've been trying to build out this energy model. It's very complex. Energy's data services market is something like $900 million. It's obviously a huge market for me to try and break into. Been like slowly grinding at it and it's been helpful for our data services business. We really hadn't broken into the energy data services business despite a year of having multiple people on the team. Then Claude Code psychosis hits. One of the people who leads the data center and energy and industrial business at semi analysis. Jeremy hits him.

• 9:07Speaker 02

  And now all of a sudden, in three weeks, he spent a lot. He was spending like $6,000 a day. It was an insane amount, but he scraped every single power plant in the US. Every single transmission line above a certain voltage and created this entire mapping of the entire US. Grid as well as a lot of demand sources, all from various public sources of data. And it's got like this dashboard where you can view and check, you can see all the micro regions of the US where there's power deficits and surpluses. All of these details built in a handful of weeks. We started showing some of our customers who buy our data center data set, but are energy traders. We showed some of them and they're like, wow, how long did this take you? This is really good. This is better than XYZ company. And then we like dig deeper. XYZ company has 100 people and have been working on this for a decade.

• 9:51Speaker 02

  Obviously, our thing is not fully as robust, but in some ways it is better. I'm going to commoditize these energy services companies, data services company. Who's going to come commoditize me if I don't move faster? And so the question from a business owner's perspective is, yeah, I'm spending a lot, but what does that spend getting me? Is it getting more revenue?

• 10:08Patrick O'Shaughnessy?

  Are you worried that in the limit, the people that control capital and are investing capital, who are often hiring you for what you do? Will just say, well, we have analysts too who are really smart about this. Like we'll just build this ourselves. If it's getting that easy, at what point does it just all pool into the investment firms? That stand to gain the most because they have the most leverage on top of the data or the insights that they glean.

• 10:30Speaker 02

  First of all, any information services business, obviously. I don't generate as much value as my customer does from said information. Because if I sell you information for a dollar, you're only buying it for a dollar because you know that information helps you make a decision that lets you make. More than $1. And so therefore you have made more money off of me than I did from the information myself. These investment funds all have their own information services, especially like the super, the Jane Streets of the world and the citadels. They're really detailed on their data. And yet these sort of folks also purchase data from us and continue to do so and continue to grow with us because I think there's just some it factor, right? We move faster, we're more nimble.

• 11:07Patrick O'Shaughnessy?

  You're at the edge, you're at the front.

• 11:07Speaker 02

  We're a smaller team that's focused on just one specific thing. AI infrastructure and the huge revolution that causes in AI and tokenomics and all these things. And we see where it's headed and so we're moving faster and building faster. I think investment professionals, yes, they'll try and build some of the stuff we do and more likely they'll just buy the data from us. And it's cheaper for them to buy the data from us and then build on top of it than it is to build it themselves.

• 11:34Patrick O'Shaughnessy?

  I feel like every conversation I have with you, what I'm always getting at is just supply and demand of tokens. That's the thing that's interesting to me in the world right now. What has this experience taught you about the demand? Has it changed your view on the demand side of that equation, just feeling it viscerally yourself?

• 11:49Speaker 02

  If we take a step back and look at the macro lens, right? Anthropic has gone from $9 billion revenue to what they're at 35, 40 billion now. Probably by the time this airs, 40, 45 billion, who does ARR? Their compute has not grown to the same degree. And if you do the calculations and you assume they didn't decrease their research and development compute, they clearly didn't. They're really, they have mythos, they have Opus 4.7. So they clearly didn't decrease their research compute spend. So ultimately what they've done, even if you assume all incremental compute they've gotten has gone towards inference, their margins are at a floor of 72%. In reality, some of that incremental compute they've got probably went to research and development and maybe higher than 72% gross margins. To be clear, at the start of the year, there was a leak from their funding round docs, someone leaked it. 30 something percent gross margins.

• 12:34Speaker 02

  Where on earth does a business like this grow margins like that? It's in principle, right? Their demand is so high, they're able to cut back on usage limits, rate limits, all these things. What really matters is having an Anthropic rep and having an enterprise contract with them and getting the rate limit increases that you need. Because otherwise, tokens are ultimately super, super in demand. Whoever can pay for them, Anthropic has the same problem. I mean, not problem, it's just the reality of how capitalism works. Yes, people are sending them $40 billion ARR in tokens, but those tokens are generating way more than $40 billion in value. Various businesses will have different value generation per token, but as we get more and more intelligent, what really matters is access to these most intelligent tokens and leveraging them at things.

• 13:19Speaker 02

  You as a person deciding what is the best way to leverage these tokens to grow business and generate value. Because a lot of folks will want tokens and generate tokens, but the shitty SaaS startup in SF who is using Claude to generate their software product is not necessarily actually creating a ton of value. And therefore, they're going to get priced out of tokens soon enough.

• 13:41Patrick O'Shaughnessy?

  Vanta automates compliance so your team can spend less time on security reviews and more time getting customers. Vanta cuts audit prep by 82% and gives you instant up-to-date proof of trust without all the manual work. So you can close deals faster and with less friction. Customers report a 526% return on investment and more than 16,000 companies use Vanta, including Ramp, Harvey and Snowflake. Get started with $1,000 off at vanta.com/invest. RidgeLine is the first end-to-end system of record with embedded AI for investment management firms running portfolio accounting, reconciliation, reporting, trading and compliance on one unified platform. Firms are moving off legacy technology and onto RidgeLine

• 14:25Patrick O'Shaughnessy?

  because of how far ahead RidgeLine's AI features are compared to anything else in investment management software. I've been hearing from a lot of investment managers about AI and they fall roughly into two camps, with some unsure where to even start and others convinced that they can build their own order management system over a weekend. The reality is that running an investment firm will always require governance, controls and a single source of truth for your data. And no amount of AI enthusiasm changes that requirement. RidgeLine is built on exactly that foundation, which is why I believe that the firms that come out ahead in the AI era will be the ones running on RidgeLine's unified platform. If you're serious about your firm's AI strategy, RidgeLine should be part of that conversation and you can request a demo at ridgeline.ai.

• 15:09Patrick O'Shaughnessy?

  I had this experience just today where on the flight here, I got re-limited out on something. I saw 4.7 came out and what I immediately wanted was to be on 4.7 that second. I couldn't think about using 4.6 anymore, not this 4.7 is out. I was perfectly happy with 4.6 for the last many weeks. It's amazing. Are you surprised that people are so insistent on going to the most expensive, leading edge thing to the degree they are?

• 15:34Speaker 02

  Without a doubt. I think one of my funniest memories in the past month and a half is myself and a buddy of mine, Leopold, being on our knees in front of an Anthropic co-founder begging him for access to Mythos and then pretending it doesn't exist. Because we knew it existed and we're like, please give us access. And he's like, I don't know what you're talking about.

• 15:57Patrick O'Shaughnessy?

  What was your reaction to that rate card or that eval card coming out?

• 16:01Speaker 02

  It was rumored in the Bay Area. We knew it was supposed to be really good, but if you just look at the benchmarks, obviously benchmarks change over time. Mythos is potentially the biggest step up in model capabilities in two years. I think that's really, really an important detail that it's so good that they're like, don't want to release it. Even though they already announced the price to their people that they did a selective release for Cyber for, and it's five or 10x the token cost. They just don't want to release it because they're worried about the impact on the world. And they're releasing a worse version, Opus 4.7, to us. And they explicitly said in the model card, hey, we actually preferentially made it worse at Cyber. I don't know if you read that. Whoever you are, if you have enough capital, you should get a freaking Enterprise Anthropic subscription where you pay per token, not with these subscriptions.

• 16:46Speaker 02

  Because then you won't get rate limited much. And then you need to figure out how to leverage those tokens to the highest value task and make money off of it. Because ultimately what you're doing, maybe like a year from now or two years from now, the business is actually just arbitraging tokens. Right? The tokens are amazing, but let's figure out what direction to point them in. And then three or four years from now, the model will know what to do with the tokens and how to make the most value. You need to look at this retroactively, pick any benchmark. The cost to hit a certain capability tier used to cost X and now it costs 1 one hundredth or 1 one thousandth of that. DeepSeek, for example. On GPT-4 was one six hundredth the cost. And since then, the cost have fallen further for GPT-4 class models. Of course, no one gives a crap about GPT-4 class models.

• 17:28Speaker 02

  They want the Frontier because the Frontier lets them create the economically valuable things. But GPT-4 class models can still be used in stuff. And so people are using them in some like tiny use cases. It's just the cost have fallen so fast. But it's not really what's driving the demand. What's driving the demand is all these new use cases. Yep, current 4.6 OPUS or 4.7 OPUS tier models. A year from now. My spend for the same exact quality of the model would probably be like 70K. I bet you it'll be 100 times cheaper. Irrelevant because I'm going to be using a way, way, way better model, which can do way, way better things. Anthropic Mythos is more expensive as a model, but it spends a lot less tokens to do the thing. And therefore it is actually cheaper in most tasks than 4.6 OPUS.

• 18:12Speaker 02

  Because it's just way more efficient, even though each individual token is smarter. So yeah, there's crazy geniuses. Creating huge cost efficiency improvements every day. They work at the labs and they're making the models way more efficient. You see it every generation. A GPT, what was it, 5 Nano or whatever was better than GPT-4. Or 5 Mini was better than GPT-4 and it was like one one hundredth the cost. This just happens and we accept it at its face value. But ultimately you keep making things cheaper and then you keep scaling them up and you keep getting humongous improvements.

• 18:43Patrick O'Shaughnessy?

  When I last saw you, Mythos had just come out, maybe the day before or something. Or the card had just come out. And you said something like it actually made you feel like a little scared. It was so good. What did you mean by that?

• 18:54Speaker 02

  Anthropic's whole goal in 2025 and even a lot of 2024. They're like, hey, by the end of 2025, we need an L4 software engineer in our model. And they by and large achieved that with 4.6 OPUS. What they didn't say is that, and if you look at Mythos and if you compare benchmarks, it's like an L6 engineer. So L4 is like pretty new. L6 is like quite well experienced. I think Anthropic said that the model internally was available in February. So in two months, they've gone from L4 engineer to L6 engineer. What's next? When you think about the model progress, it's only accelerated. Anthropic's release cadence has compressed. OpenAI's release cadence has compressed. Why? Because generally, to make a better model, you need a few things, right? You need amazing compute. Compute is very expensive,

• 19:39Speaker 02

  and it has a time scale that we track. And it's like, it's growing, but it's set in stone. For the next short term, it's like set in stone what you've already signed. And there will be delays and shifts, and somehow you can find a little more. But it's generally pretty set in stone. There's amazing researchers that people are paying tens of millions of dollars for. And then lastly, there's implementation. And implementation historically has been very difficult. If I have an idea, now I have to implement it. Implementing is hard. Now, ideas are there. Implementation is very easy. It's expensive, but it's very easy. So how does one decide what ideas to implement? And it turns out, if your implementation is just so much easier, now you can just implement more ideas and move on the treadmill faster and faster and faster. Whether that is AI model research, and so now your model release cadence has shrunk down to two months

• 20:24Speaker 02

  from where it was six months before. Or I want to take every power plant in the US and every transmission line and model it and run regressions and see the micro supply and demand. I can also do that. The idea is cheap. Which idea makes sense? Which idea is worth the capital that you have to spend on the tokens? Because the implementation is there. That's the key learning. And if implementation costs continue to tank, which they are. We don't even have Meetos yet. It's only been a handful of hours since Opus 4.7 launched, but my team is pretty excited about it internally. What now comes to the world? It's a complete reordering of how economies work. What used to matter a lot was execution was very, very fucking difficult, and ideas were cheap. Now, ideas are cheap and plentiful,

• 21:08Speaker 02

  but execution's very easy. So really, only the good ideas are the ones that can justify the spend on super-cheap implementation.

• 21:15Patrick O'Shaughnessy?

  So are you actually scared? Or does it just introduce an uncertainty that's hard to grapple with?

• 21:21Speaker 02

  Uncertainty is there. But I do think that causes some fear in terms of how does society reform itself? How does one exist in a world where actually your ability to implement something is not actually that important? Your ability to just choose the correct idea for AI to implement, and then your ability to sell that idea or sell what the AI has implemented is what matters. Your ability to garner capital towards that is what matters. And going back to the point of it's very important to have the newest model always. Who's going to have access to the newest model? Anthropic's project, I know it's not called Earwig, but I troll Anthropic people by calling it Earwig. Glasswig. Anthropic Earwig, where they only release Mythos to certain companies for cyber.

• 22:06Speaker 02

  That's just going to be something that continues. Models will have less broad and less broad deployment. I know OpenAI and Anthropic and all these people are like, we want to have great AI for everyone. AI is very fucking expensive. Who's going to pay for the trillion dollars of infrastructure? People who have money and can build useful things with AI. And then you don't want people to distill your model, so you don't release them broadly. You release them to fewer and fewer set of customers. Those customers are also now wrestling over the tokens. Unless Anthropic jacks them. They could double their pricing on Opus and I would continue to pay and I bet most users would continue to pay. I bet that wouldn't solve their humongous capacity problem that they have. So then the question becomes, where does this cycle end?

• 22:47Speaker 02

  Where token usage and therefore the benefits of those tokens, the additional value generated on top of those tokens, aggravates among fewer and fewer and fewer companies. I don't have Mythos. You know who has Mythos? Top freaking banks. Now, they're only using it for cybersecurity, but at some point I can envision a world where, hey, maybe I, because I have an enterprise Anthropic contract and because Anthropic people kind of like me, they're willing to give us slightly earlier access or slightly higher rate limits or something for a model. I hope that's what happens. And then my competitor, whoever that is, doesn't have that and I'm able to fucking crush them. There are people like Ken Griffin of Citadel, super well-connected and super rich. He goes and signs a deal with OpenAI and Anthropic that's like, yeah, I'm going to get access to your models

• 23:29Speaker 02

  and I'll buy the first $10 billion worth of tokens each year. So whenever you release the model, I'll spend the first $10 billion tokens and then everyone else can get the model after that. Yeah. And it's like, okay, well now what does that do? Now he's going to crush everyone in the markets. That's just an example. It could be any number of things. It could be cyber, like Anthropic is worried about, oh, now I can hack people. Could be information services business like myself where I crush someone else. I think it's such a broad base. We don't know what these models can do. Anthropic doesn't know what these models can do. No one knows what these models can do. It's up to the end user to figure out where they can leverage the tokens to see what they can build and imagine, which is tremendously productive and uplifting for humanity. But then what happens to the concentration of resources and usage of it?

• 24:06Patrick O'Shaughnessy?

  Presumably right now robotics or robots consume relatively zero tokens versus everything else. What's your view of that? If that's like a second demand curve that could start to ratchet. There's a new startup every single day within a mile of here trying to build something interesting in robotics.

• 24:23Speaker 02

  And so there's this concept of software-only singularity, which is that the world has AI singularity, but only in software. And now what about the rest of the world? The vast majority of the world is physical. You can see the world orient around hardware, not software. That's actually why I think software-only singularity is like just a blip and not like we do get everything else. Because once software is super easy, what makes robots really hard? It's programming, microcontrollers and actuators and controlling all this stuff is very difficult. And right now, the interesting thing about models, AI models, is they're actually really inefficient in learning. It's just we're able to give them so much data that they're able to learn and pass us in certain ways.

• 25:04Speaker 02

  Currently, the robot models, VLA's, vision, language, action models, which is very popular right now, is probably not going to be the thing that ultimately scales beyond. They're inefficient in data and we can't scale the data for them fast enough. There is going to be some way to large-scale pre-trained robot models where, just like humans, see all this data throughout their lives. And what's interesting is humans, the reason why we're so good is we're sample efficient. One example, two example, we're good. So applying that to robotics, so once you have this software-only singularity, implementation is super cheap. Anyone can start to build these models that now robots are actually useful. And so I think in the next six to 18 months, we'll start seeing real breakthroughs in robotics

• 25:45Speaker 02

  that enable few-shot learning, i.e. there's a pre-trained robot model. And now there's a robot that you have hired or bought or whatever. You show it a few examples and it's able to do it. Right now, you know, there's a lot of companies doing robots for advertisement or robots for simple stuff like that. But it'll be like, oh, folding clothes, sure, sure, sure. No, but it's going to get really niche. Robots just for cleaning chalkboards and it's a rental service. Or it'll be a model package that you download onto your standard robot that then does that. And anyways, there'll be a huge explosion in physical good acceleration and deflationary effects there. But that's ultimately going to keep token demand going crazy. I don't think token demand slows down personally.

• 26:24Patrick O'Shaughnessy?

  Did you learn anything else about the world based on Mythos's results and how it was built? It's my way of asking if you break down the components of the scaling laws.

• 26:33Speaker 02

  Mythos is a materially larger model than prior models. And so, yes, it is a much larger model. What chip it's trained on is not really relevant. It's the scale. And obviously, 100,000 black wells is equivalent to hundreds of thousands of prior generation chips. GPUs and Tranium have their different release cadence, so it's not exactly mirrored one to one. But ultimately, yes, Mythos is a significantly larger model. It's proof that the scaling laws still work. Everything about it shows the trend line continues of models. More compute into model makes model better. And along the whole way, it's not just more compute into model makes model better. Along the whole way, we're also getting these compute efficiency wins, which are, as all this research compute that the labs are spending is actually turning into, if I want X capability tier model, every six months that cost, or every two months,

• 27:17Speaker 02

  that cost is dramatically decreasing. But then if I scale it up massively, I get a humongous capability drop as well. And so, yes, it's proof that this is still happening. Google and Anthropic are not heavy, heavy users of GPUs on the training side. But OpenAI, they'll start having their new class of models. I think they're taking a more sensible principled approach to scaling in small steps. Anthropic really went for a huge jump. We'll see better and better models throughout the year, and the release cadence is only going to get faster.

• 27:45Patrick O'Shaughnessy?

  We've gone a long way in the conversation with saying almost nothing about OpenAI, which would have been so strange a couple of months ago.

• 27:50Speaker 02

  So this is the interesting thing. Everyone's like, okay, so Anthropic's just won, right? They had Mythos in February. They never even released it because they didn't feel the need to. They're already sold out. Their revenue's already adding $10 billion a month. And then you've got Opus 4.7 today, all before OpenAI's alleged spud release, which media such as The Information and others have posted about. So clearly Anthropic is in the lead, and OpenAI is cooked. What's interesting is because Anthropic has such bounds on compute, and they can only grow it so fast, and to the point of, Dario used to gloat about how OpenAI was being too aggressive on compute, and Anthropic was more sensible in their scaling. And now Anthropic is like, fuck, I wish we had a lot more compute. OpenAI is able to pay the bills perfectly fine. In fact, they raised

• 28:34Speaker 02

  A ton of money to get incremental compute in addition to the irresponsible levels of compute that they were buying from Oracle and Core even SoftBank and all these people in Microsoft such as Tranium now they're getting Tranium as well from Amazon they've done this insane thing on compute they also know they need more but what's interesting is if you were to say Opus 4.6 let's ignore models getting better over time let's just take diffusion of this technology you and I may jump on the model immediately day one but other businesses take time and it takes time for people to learn and the spark of oh shit Claude psychosis moment doesn't hit everyone at the same time and so by the end of the year let's say a 4.6 Opus tier model the economy would spend $100 billion on I don't think that's unreasonable it's spending 40 billion right now

• 29:17Patrick O'Shaughnessy?

  that's like a linear extrapolation

• 29:19Speaker 02

  it's a linear extrapolation not an exponential to get the exponential you need the better models anthropic won't have enough compute to do that and presumably OpenAI and Google will hit that tier soon enough whoever hits that tier next sure anthropic may get to charge 70 plus percent gross margins but if OpenAI hits it next they charge 50% in gross margins they still get all of this incremental demand and probably they also won't have enough compute to serve all the users sure maybe mythos is a model where if the world had enough compute it'd be $500 billion of revenue or something crazy there is such demand for these tokens and such limitations on compute we see this with H100 prices skyrocketing and all these other things the useful life of these GPs continue to extend it extend it's pretty clear even the tier two lab

• 30:04Speaker 02

  is going to be sold out of tokens let alone the tier one lab the tier one lab will have better margins but the tier two lab will be sold out and probably the tier three lab will also be close to sold out economic value that the best model can deliver is growing faster than our ability to actually serve those tokens to people via the infrastructure and so this gap will continue to grow and the model labs will continue to have expanding margins until people in the hardware supply chain infrastructure supply chain are like wait no why don't I just jack up my margins

• 30:30Patrick O'Shaughnessy?

  so suffice to say I think the assessment today or your assessment of the demand side is completely explosive in your own particular example here at semi analysis but just more broadly that you call it AI psychosis as people fall into this experience of what they can do the implementation difficulty going completely away I've certainly felt that my own token spend is just through the absolute roof just in the matter of weeks so that feels like a pretty good assessment anything we're missing on the demand side

• 30:54Speaker 02

  if you don't use more tokens you'll never escape the permanent underclass Either you use more tokens and you generate economic value, outsized economic value for the use of those tokens. A lot of people are doing it the boring lazy way. Oh, I guess I'll just work one hour a day instead of eight hours a day and I'll have AI do most of my job. That's the boring way. The cool way is I'll still work eight hours a day and I'll do eight X the work and maybe I'll make five X the money. You can't do this with a job, obviously. There's people who have multiple jobs. There's people who start companies and start selling stuff. There's people who are hustling, which is what I view like you and I is doing is we're mostly hustling. Get that economic value on this AI before everyone is using it, it's table stakes, because it's still not table stakes. So if you don't use more tokens and generate the value from them and capture that value, there's three different problems here.

• 31:39Speaker 02

  Using more tokens, generating value from those tokens and capturing value from the value that you created from the tokens. If you don't do these three things, you'll never escape the permanent underclass, i.e. as models continue to skyrocket in capability and the concentration of resources potentially happens.

• 31:54Patrick O'Shaughnessy?

  All right, let's talk about supply. What is changing at the frontier of supplying the entire stack that's required to serve all these tokens as the demand curve explodes?

• 32:03Speaker 02

  As demand skyrockets, prices are going up for everything on the supply side, whether it be the end GPUs, their prices are going up. In addition, their useful life is extending.

• 32:13Patrick O'Shaughnessy?

  H100 prices look like this?

• 32:14Speaker 02

  Yeah, exactly. There's people who have argued GPU useful lives are less than five years. Complete nonsense. There are clusters now resigning, three or four-year-old Hopper clusters resigning for three or four more years. There's A100 clusters that are resigning for another couple of years. So the useful life is clearly not five years. It's maybe even seven or eight years, arguably. We don't know yet. We'll see when Hopper gets there, but it's clearly not five years. So useful life is extending and the prices are going up on that renewal. In effect, the gross margin was not 35% on a cluster, it's beyond that. So margins are expanding in the cloud layer. Margins are extremely healthy on the hardware layer with Nvidia still charging 75 or whatever percent gross margin. As we move down the stack memory, obviously, margins have skyrocketed there.

• 32:58Speaker 02

  Places like Optics and Logic, there are large prepayments and margins are growing slowly, more so the companies that are making chips like Nvidia are paying. Huge prepayments, so in effect the cast of capital or timing of cash flow, the return on invested capital is going up even if the gross margin isn't. And you see this across the whole supply chain. You see ASML is completely sold out and they need Carl Zeiss to expand faster. Everyone's either sold out and margins are going up, or they're getting prepayments, which increases the return on invested capital because the invested capital is lower. And so this is a consistent trend across any part. It's even like to make a PCB requires copper foil. And that copper foil is sold out and people are making prepayments for it. Anything and everything that has a pulse and is sold out,

• 33:41Speaker 02

  people are jumping to get more incremental supply and fighting over the supply for the years after.

• 33:47Patrick O'Shaughnessy?

  What do you think are the most important bottlenecks? Typically in economic history, when there's this kind of demand, supply reorients and rises very, very quickly to meet the demand. It seems like it's almost impossible for supply right now in this moment to keep up. Famous last words, every shortage is followed by a glut historically. But what are the most interesting bottlenecks to you across the supply side?

• 34:10Speaker 02

  Supply chains are usually very fast to react. One unique thing is that our supply chains now are more complex than ever and the things we're building are more complex than ever. And therefore the lead times are longer. And it's not like we haven't seen 18 month long lead times in other industries. It's just building incremental supply didn't take years. And this is the case with memory. Memory can only grow capacity low double digit percentages a year, right? 20s, 30% a year, even less for NAND, a little bit higher for DRAM, but whatever. Even though the demand signal was very strong at the end of 2025, the memory companies immediately started reacting. None of that incremental capacity really gets here until the second that they've decided to do in addition to the typical 20 to 30%. And they can stretch a little bit, but really

• 34:52Speaker 02

  the true incremental supply doesn't come till 28, which is a very unique thing. Even if they wanted to build as fast as possible, it doesn't come till 28. Late 27 at best. So the result is memory prices have gone through the roof. And guess what? They're going to double and triple again, at least on DRAM especially. People are like, oh, the memory story is overplayed. Everyone gets it. It's like, no, no, no, you don't get it. DRAM will double or triple from here still because that's how much capacity is required. And they have to steal capacity from somewhere else. And the only way to steal capacity from somewhere else in a capitalist economy is demand destruction via higher pricing. We're not rationing stuff here. And so ultimately, that's what's going to happen. And so margins continue to go up. I think logic also has humongous capacity problems. TSMC just had their earnings.

• 35:37Speaker 02

  They keep upping CapEx. Ultimately, it takes them quite some time to build fabs. They're trying to do everything they can to squeeze every little output out of every fab that they have. But ultimately, they're not raising prices fast because they're good people, it seems like. Single-digit price increases instead of triple-digit price increases like the memory guys have had. So you ultimately have this market where, yeah, TSMC is a great company, but are they actually going to extract all the value? I mentioned things like copper foil, glass fibers for PCBs, lasers. These are things that are well-understood in niche supply chains, but they're very, very tight. And ultimately, upstream, the semiconductor wafer fabrication equipment supply chain is one that's gone up a lot, but it's still very underappreciated. TSMC CapEx this year, they say 56.

• 36:20Speaker 02

  We've had 57.4 billion since January, and we may up it slightly more just because we see some ways that they can get incremental CapEx. But what people aren't focusing on is what does that mean next year? And what does that mean the year after? And it turns out, three years from now, TSMC is going to spend $100 billion on CapEx. Maybe two years from now, it might be 28. Sincerely, they may spend $100 billion on CapEx in 2028. And people just can't fathom that. But what does that mean for their downstream supply chains? Companies like Lam Research, or Applied Materials, or ASML, or their further downstream supply chains like MKSI and all these other companies. The tail whip, it just gets whipped harder and harder and harder. And that's a shortage if TSMC wants to spend $100 billion in 2028, which is a real possibility.

• 37:02Speaker 02

  I think people would think that's insane, but that's a real, real possibility.

• 37:06Patrick O'Shaughnessy?

  What about other parts of the chip ecosystem where GPUs have been completely dominant? What about like CPUs or ASICs or things that start to pop out as both opportunities and bottlenecks beyond just Nvidia's GPU dominance?

• 37:18Speaker 02

  Yeah, I mean ASICs are obviously taking off, but I'll pivot away from AI chips to talk about these other things. There's a project we did on FPGAs, and it turns out there's 120 FPGAs per next-generation rack, AI rack, and then what about all the FPGA names? CPU-wise, all these reinforcement learning environments, plus all the slop code you and I are generating that is now running on some Versal instance, or whatever it is, or some AWS instance, or some bucket that we've spun up. All of that requires CPU. And so CPUs are completely sold out and demand is skyrocketing there.

• 37:49Patrick O'Shaughnessy?

  How people understand the role that CPU plays and everything?

• 37:52Speaker 02

  There's two main reasons why you need tons of CPUs. One is when you're doing reinforcement learning, the CPU is very critical to that. So before, you would throw all the internet's data into the model, train it, and it spits some stuff out. Now, you train all the world's internets, you put all the internet data into the model, then you put it in this environment. This environment is like, hey, model, try this out. And it tries stuff out, tries a bunch of different things. And in the end, there is an environment which scores whether or not what it tried out is successful, and it grades it. And these environments can be anything. It can be, hey, check if the text was outputted in the right way, structured output. It can be very simple stuff, or it can be very complex stuff. And people are starting to get into very complex things. Like, hey, I want you to open this file,

• 38:34Speaker 02

  change it, edit it, update it, submit it to this website. I want you to open up this physics simulation from Siemens and edit this CAD model. So the environments can get more and more complex, and those environments run on CPUs. They don't run on GPUs, they don't run on ASICs. The ASICs run the model that takes the input data from the environment, runs it through the model. The model creates outputs of various different trajectories, ways that it think it could solve it in different instances. Those trajectories are graded/scored, and the ones that are successful, you train on and you update, and you iterate, iterate, iterate. And so CPUs are very useful for that one. And then, once you have these great models and you're deploying them, those models are generating code. They're generating useful output. That useful output,

• 39:18Speaker 02

  it doesn't go from a GPU straight to the human brain. It goes from a GPU or an ASIC through to a deployed app that you're deploying somewhere that actually just runs on CPUs. So that's another area where there's a lot of demand, and things are sold out in a large, large way.

• 39:32Patrick O'Shaughnessy?

  As you continue to assess and try to be the world's best-informed person on both the trajectory of supply and demand, what are things that you wish you knew to make that understanding that you don't know?

• 39:43Speaker 02

  I think the hardest area for us and for everyone is understanding tokenomics, economics of tokens. I think we have a really tremendously good insight into how much it costs to run infrastructure, what the cost of tokens are, what the cost of models are, what the margins of these labs are. But the usage and adoption is what's really difficult to model continuously, right? January, we had crazy estimates for February. Anthropic smashed them. How do we calibrate this model? What are the data sources for this? February, we had crazy assumptions for March. I know people were like, "You're crazy, Dylan." And then they smashed them. Everyone sees the number of 10 billion and they're like, "What the f-" How do they add 10 billion to revenue? Who is using all these tokens? Why are they using them? What are they building with them? And then more importantly, with what they're building with these tokens, how is that actually

• 40:28Speaker 02

  diffusing into the economy and what value is that generating? Because it's not really something that you can capture in any GDP statistic. All of the value of the tokens that I use get transformed into better information, which I then sell at a discount to what people used to sell information for relatively. And therefore, that information is now making its way throughout the economy and people are making better investment decisions or better competitive decisions if they're a semiconductor company or a data center company or hyperscaler. What is the value of this? What has that done to the economy? It's clearly, by every subjective metric, amazing. But where is the phantom GDP? What is the phantom GDP? How do we track the real economic value? Because the GDP metrics are not accurate if you were to say, "What is the GDP that

• 41:13Speaker 02

  Dylan Patel is making?" It's tiny compared to the value that I think is being created. And I think you would say the same for Patrick. What is the value being created by these tokens? Not on a basis of simple, "What is the knock-on effect of all the things that these things are doing?" I think that's the real question and challenge. It's hard to measure. I think we've got a tremendous reading on the supply side of things. I think we've got a tremendous reading on even a lot of the demand-side signals, but it's, "What is the value these tokens are generating?" That's hard to quantify and measure.

• 41:40Patrick O'Shaughnessy?

  I hope we get a chance to do this every three months because this changes so quickly. What do you think's going to happen next? When I come back three months from now and we're in San Francisco together again? What do you expect?

• 41:51Speaker 02

  Large-scale protests.

• 41:53Patrick O'Shaughnessy?

  Really?

• 41:53Speaker 02

  Yeah, I think there will be a large-scale protest against Anthropic and AI. People hate AI. AI is less popular than ICE, less popular than politicians. With Anthropic adding so much revenue, that's going to start causing business changes downstream. People are going to get more and more scared of AI. They'll start blaming more and more of their own problems and things that are global. Have been deep-seated problems for a long time. Those will bubble up and be blamed on AI. Probably some politician or some influencer will be able to start taking and weaponizing AI against people. You look at the comments of news articles where Sam Altman had a Molotov cocktail thrown at his house twice in two weeks. People are cheering it on. And this is just the beginning. So I think we'll see large-scale protests against AI in three months.

• 42:38Patrick O'Shaughnessy?

  What is the counterweight to that? How should the AI industry head that off?

• 42:43Speaker 02

  First of all, Sam Altman and Dario have to stop getting in interviews. They're so uncharismatic. I don't know what they're doing. Every interview they do is like, wow, normal people are going to hate you even more. Sam being on Tucker Carlson probably made all Republicans hate OpenAI. I'm just guessing. Same with Dario. I think that's first. Two, they need to start showing uplifting things that can be done with AI. Three, they need to stop talking about how the capabilities are going to change the whole world constantly. Because then people are going to get fear of that capability because they have no connection.

• 43:13Patrick O'Shaughnessy?

  They don't know how to use it, yeah.

• 43:14Speaker 02

  There's no connection to it either. The average person doesn't know an Anthropic employee. The average person doesn't know an OpenAI employee. The average person doesn't know who these people are, what their goals are. And they just view them as a sneaky cabal of 5,000 people at this company. They're going to change the world and automate all the jobs and destroy society. That's what they view it as. And as people who are funding the building of all these data centers and power plants that are going to pollute the world, they don't quite understand what's happening. So they have to stop talking about the future thing that's going to happen and only talk about present, how uplifting AI is. I think it's a huge reorg and rebranding that needs to be done.

• 43:48Patrick O'Shaughnessy?

  This is so much fun. I love doing this with you. Thanks for your time.

• 43:50Speaker 02

  Haas and Bakes.

• 43:53Patrick O'Shaughnessy?

  If you enjoyed this episode, visit colossus.com. You'll find every episode of this podcast complete with hand-edited transcripts. You can also subscribe to Colossus, our quarterly print, digital and private audio publication, featuring in-depth profiles of the founders, investors and companies that we admire most. Learn more at colossus.com/subscribe.

• 44:32Patrick O'Shaughnessy?

  You know how small advantages compound over time? That's true in investing and just as true in how you run your company. Your spending system is your capital allocation strategy. Ramp makes it smarter by default, better data, better decisions, better economics over time. See how at ramp.com/invest. As your business grows, Vanta scales with you, automating compliance and giving you a single source of truth for security and risk. Learn more at vanta.com/invest. The best AI and software companies, from OpenAI to Cursor to Perplexity, use WorkOS to become enterprise-ready overnight, not in months. Visit workos.com to skip the unglamorous infrastructure work and focus on your product. RidgeLine is redefining asset management technology as a true partner, not just a software vendor.

• 45:13Patrick O'Shaughnessy?

  They've helped firms 5X and scale, enabling faster growth, smarter operations and a competitive edge. Visit ridgelineapps.com to see what they can unlock for your firm. Every investment firm is unique and generic AI doesn't understand your process. Rogo does. It's an AI platform built specifically for Wall Street, connected to your data, understanding your process and producing real outputs. Check them out at rogo.ai/invest.