Ep 95: Modal CEO on the Future of Inference, the Chaos of the GPU Market & Origins of the Sandbox Bet

Jacob Effron and Pat Chase sit down with Erik Bernhardsson, co-founder and CEO of Modal, for his second appearance on Unsupervised Learning, now with the company past a $1B run rate. They dig into how the AI compute market actually works behind the scenes, why Modal’s early bets on sandboxes and low-level infrastructure paid off, and Erik’s takes on NeoClouds, NeoLabs, and where GPU prices are headed next.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  How much of like in organized marketplaces compute these days versus like the Wild West?

• 0:04Speaker 01

  I went to GTC last year. It was like an open air drug market, but like GPUs, walking around, like talking to people, trying to get GPUs and meeting people in hotel lobbies, exchanging numbers. It's very far from an efficient market. I mean, some people are working on this. Compute comes in many different shapes or forms. I also feel like structurally, it will be a little bit like sort of unwieldy for a long time. Like it's just not something you can like sort of buy and sell in an exchange because it's not like a fungible contract.

• 0:34Speaker 00

  Welcome to Unsupervised Learning. As you can probably tell from my voice, I am not Jacob. I am Patrick Chase, GP at Redpoint. So awesome to have Eric on Unsupervised Learning for the second time. He is the founder and CEO at Modal. I have learned so much from working with him over the last three years. He is one of my favorite people to jam on AI ideas with. He has an incredible ability to move seamlessly from the lowest altitude of developer shipping code to the highest altitude of where the market is going and building what is missing. We talk about the modal journey over the last three years, the crazy technology they have built to make it all work, inference, sandboxes, and what's coming next. Hope you enjoy the episode.

• 1:15Speaker 00

  For you guys, there is obviously a ton of AI compute demand and lots of people going out and trying to reserve, you know, GPUs and CPUs. What's it like for you guys finding compute in the market today?

• 1:29Speaker 01

  It's pretty rough. I mean, like, you know, not rough in the sense that like we can actually find compute. I think there's a little bit of a misconception that like we're sort of, we are certainly scarce, you know, in terms of compute in certain ways, but like there is capacity you can get. You just have to look a little bit further, accept higher prices, accept longer terms, et cetera. And we do that a lot. I mean, we probably do like two compute deals every week at this point, if not three. And so we keep, you know, stacking up, you know, more and more compute. And I really think that's the beauty of Modal is we can aggregate all this demand and manage the capacity in aggregate much more efficiently. And the value prop of Modal has always been like, don't think about capacity. Like we had a lot of that like annoying stuff to sort of the day to day knife fighting of

• 2:14Speaker 01

  getting capacity and working. We work with 25 different clouds at this point and adding more all the time.

• 2:18Speaker 00

  How much of like in organized marketplaces compute these days versus like the Wild West, like random people that somehow have capacity and, you know, late night text messages?

• 2:27Speaker 01

  It's a lot of late night WhatsApp groups and like it's pretty unorganized. Like, I mean, I went to GTC last year. Or sorry, earlier this year in San Jose, the Nvidia

• 2:39Speaker 01

  big conference they're doing. And it was like an open air drug market, but like GPUs, like you're just like walking around, like talking to people, trying to get GPUs and you know, meeting people in hotel lobbies and, you know, exchanging numbers and, you know, I mean, at some point we, I think we got connected with someone who had GPUs because I think our VP of Sales cousin was golfing with someone and we got connected. There's many such cases, like it's very far from an efficient market. I mean, some people are working on this and I think over time it got a little bit more efficient, but it's very heterogeneous, like compute comes in many sort of different shapes or forms. So I also feel like structurally, like it will not, like it will be a little bit like sort of unwieldy for a long time.

• 3:24Speaker 01

  Like it's just not something you can like sort of buy and sell in an exchange because it's not like a fungible contract.

• 3:29Speaker 00

  Yeah, but the key differences being like, I guess, the kind of actual powered shell or size of the cluster and like the chips themselves or to your point, there's a lot of people that have been trying to build like, hey, you know, compute is this fungible single asset that you could build a marketplace around. What are some of the kind of the key differences you observe?

• 3:46Speaker 01

  I think a lot of it is actually boils down to reliability. I mean, first of all, it's like, you know, differences in like disk and networking and of course CPU and memory and things like that. Like all these things vary and region or like location, like they're all over the world. Like we have GPUs in the US, of course, like all over the US, but we also have GPUs in like Indonesia, like all over Europe, et cetera. So so location is one of those big factors and then, but also reliability. And that's another thing we do exceptionally well is like we aggregate all this capacity and then when Neo Clouds die or like when they have failures, which they often have because some of these are like former crypto miners who haven't really thought about, you know,

• 4:29Speaker 01

  running a 24/7 operation, we can just take it out of rotation or like when GPUs fall off the bus or whatever, like we, in many cases, we just file a ticket automatically and someone goes and looks at it.

• 4:38Speaker 00

  Have you found the data providers, the companies that provide pricing for GPU and that sort of things? Have you found them to be helpful?

• 4:45Speaker 01

  I think moderately helpful. I think the pricing less so than just their ability to connect with a lot of providers. I don't fully trust the pricing today. But I think these, some of these providers are actually very good at like, you know, knowing all the Neo Clouds and we work with them sometimes to get connections with cloud providers. The pricing, I think over time will get more accurate. Like our internal pricing does not match what we see publicly when we buy capacity. But I think over time, hopefully, like we will have more efficient like external instances. I think that would actually be very useful.

• 5:16Speaker 00

  You've been on this like rocket ship journey these past years. And so lots of things we'd love to dig into for our listeners on the, on the business side. Maybe starting, you know, I think one of the interesting things about the history of Modal is you know, for a while it felt like you, you know, publicly said, I think on your first Unsupervised Learning episode, that you were less focused, you know, on these LLM inference calls, specifically like a simple API, you know, call for a model. And, you know, you kind of recently released a product there. Maybe talk a little bit about the journey and kind of that initial reticence and what made you decide that like now is the right time to really, you know, double down on this?

• 5:53Speaker 01

  Yeah. So we started building this in late 2021, basically, or early somewhere around that time. And and the initial product was this serverless function. And we thought it would be really useful for a wide range of stuff around like data, AI, machine learning, broadly speaking.

• 6:09Speaker 01

  Kind of like looked at a lot of different use cases and but then like ChatGPT came late 2022 and Stable Diffusion also came around the same time.

• 6:18Speaker 00

  Right, right.

• 6:18Speaker 01

  It turned out like we were like super well positioned to host a number of stable diffusion applications. So that provided, that was like the first year of growth was like all stable diffusion. And then we started like expanding into like adjacent use cases like audio and video and stuff like that. And it was like early teams that, you know, they were deploying GNI inference. They didn't want to deal with downloading infrastructure. They wanted to move fast. Like we, we started aggregating a lot of different GPUs across a lot of different clouds even back then. We always knew we wanted to be multi-product, but we started with the serverless, like the sort of compute primitives. We felt like it was so general purpose. And we focused a lot on like kind of what I think was like high code use cases. Like focus on like machine learning engineers and researchers who want to write code and focus a lot on like being kind of low in the, lower in the stack, right?

• 7:03Speaker 01

  And pretty early, we built also a storage layer, a file system to store data in particular container images. Then we launched a sandbox product in 2023 and the reason why we launched it back then, we started seeing people like messing around with LLMs to generate code. We figured actually like a lot of stuff we built on the infrastructure side is like really well suited to run that code. And they're going to need a place to run that in an isolated way. Didn't immediately, it wasn't like an overnight success, but like basically 2025, it kind of blew up because I think models got really good at writing code back then. So about a year ago. So that was sort of our venture into like kind of becoming like a multi-product vendor, like starting about 12 months ago, like sandboxes has been on a tier and grown

• 7:47Speaker 01

  like, I don't know, thousand X almost or something like that. It's about a third of our revenue right now. Standalone is a 300 million run rate business, sandboxes. So that's been incredible in terms of, you know, becoming a multi-product vendor. We just launched also a training product.

• 8:02Speaker 01

  So sort of, you know, building, trying to lay out like all the different building blocks and, you know, what does that suite of tool looks like and what are the toolboxes? In this like new world of like AI applications, like they're all going to need a bunch of new primitives and like we want to offer all those primitives and what does that, what does that look like? So LM inference was something, we were a little bit on the sidelines for a while. Until actually, I would say the DeepSeek moment. So that was about a year and a half ago, like early 2025. When DeepSeek came out, which was like the first Chinese open source model that was like actually really good. And then I was like, fuck it. Like we got to get in here. And like, we started building up a team

• 8:40Speaker 01

  to do a lot of inference optimization. Modal has always been really good at LM inference, like, but we didn't necessarily have a product for it. In fact, we didn't even have an inference product until very recently. We were just like really good at hosting inference, including LM inference, but it was like kind of custom. And a lot of the teams that want to run LM inference a year or two ago, like they had to deploy it themselves or like we would work with them. But we started seeing a lot of demand with the DeepSeek moment over the last 18 months, started seeing a lot of growth. LM inference is now about a third of our revenue too. So we've seen incredible growth with that. And a few months ago, we started, we shipped a number of LLM specific features because we think, you know, we kind of laid this like foundational

• 9:21Speaker 01

  layers, like the storage, the compute, the serverless functions, the file system, et cetera. I think there's a lot of opportunity to move up in the stacks. We started moving up in the stack now and launched dedicated inference products. So you can get like a single tenant models, like all the state of the art Chinese models or a bunch of US ones too, like one single click or CLI command, you can just deploy on Modal as well as much as shared endpoints. Like you can just call models off the shelf without having to deploy them, fully token based pricing. And so I think those are, you know, the first out of like many more products to come, like slightly higher up in the layer. We think there's a lot of opportunity because we're sitting on this like very solid foundation in the compute and the storage.

• 10:02Speaker 00

  Maybe like you guys will, we'll start at the bottom of the stack and work our way up, but going back in time, you touched on it, but you guys invested in some really hardcore technology in the early days. You built your own file system, container runtime, those types of things. How does that show up in the product today when the experienced developers can have?

• 10:23Speaker 01

  Yeah, I think, I think a massive source of alpha in this market is just like not being afraid to do hard things. So we went all in when we realized like in order to develop, to, in order to To deliver on the developer experience we wanted to have in the very early days, we realized that we kind of have to build our own file system to do container images really fast. So we started building that in the very early days. And that become like over the next few years, you know, it still is like a very massive source of alpha. It's been, it's been a longer journey for sure. Like we've had to go a much longer route, like building our own file system, building our own container runtime, our own schedule. Like we don't use Kubernetes and Docker. Like we use like very little off the shelf software.

• 11:06Speaker 01

  We use Gvisor, which is a fantastic Google product for isolation. Recently started using also KVM for certain tasks. But like decided pretty, pretty early on that like a lot of the stuff we're going to have to build ourselves. And I'll just do some really cool stuff. I think, you know, one of the best examples of where that shows up is we can do cold starts extremely fast. Turned out like the initial developer experience that we wanted, like very fast feedback loop when you're launching stuff in the cloud, that is also very helpful when you do building auto scaling applications that need to scale up and down very fast. So first of all, the loading the container is very fast. But over time, we also launched a couple of really cool features around CPU snapshotting and GPU snapshotting.

• 11:50Speaker 01

  So we can like very fast load data from the disk or from network straight into the GPU and fire up like even these like large models in a few seconds, which is really cool. And over the last six months, we built this incredibly complex system that basically uses RDMA over peer to peer between the nodes to copy data very fast. So I think a lot of this stuff is like kind of, you know, accumulating and really starting to pay off. Like we built some crazy stuff and now that's like, you know, we're really kind of, you know, reaping the dividend dividends of that, those sort of investments that we made for many, many years. It was a longer route, but you know, now I think we have some incredibly unique technology that's very hard to replicate.

• 12:32Jacob Effron?

  And then kind of moving up to inference, what do you view as you guys have made, you know, tons of investments there as well and optimizations? What do you view as the unsolved problems in the inference world or in the inference stack?

• 12:45Speaker 01

  I think we're still pretty early with inference. I think, you know, a lot of stuff is like, it's like one of those like very empirical sciences. Like I think people are trying different things and seeing what works. I mean, I mean, over time, like people have kind of figured out a lot of stuff, you know, take something like this idea you get a pre felt, right? Like it's a very non-obvious idea, like you sort of, you know, do the pre fill on a set of nodes and then you copy the data over RDMA and do the decode and kind of swap. There's some like very non-obvious things, but like we've invested a lot in those types of things, the KV caching, there's a lot of complexity around like KV caching, of course, like running more efficient kernels, that's something people do a lot. There's a lot of these things that, you know, we have a team of, I think, 20, 30 people

• 13:26Speaker 01

  right now, who are extremely focused on just like LM performance optimization. And, you know, it takes a lot of work. You have to stay up with, with research, you have to read papers, you have to look at all the software coming out all the time and try, try and see what works, what doesn't work. It's still kind of an empirical science. Over time, I think, you know, patterns will emerge. I think also the models themselves will change very quickly. Like now we have like mixture of experts and more complex things over time, like who knows, like maybe we'll swap out the transformers for something else and, you know, and that would change how like KV caching works. Like there's all these things like that could, you know, change overnight if someone figures out like a very different paradigm for all these things. And maybe they will, maybe they won't, I don't know.

• 14:07Speaker 01

  There's also obviously a lot of stuff around quantization and, and other things. So, so yeah, it's, it's, it's a wild world out there, but, but yeah, we're, we're, we're playing around with all these things.

• 14:20Jacob Effron?

  But it feels that there's so many folks that are working on, you know, different parts of, of, of improving inference. And obviously what's I think been really interesting in the last two, three years is a lot of those optimizations and improvements have been in open source, right? And like, you know, for the broader community, maybe shared via research. As like a CEO, how do you think about like the level of, of what the kinds of problems you want your team to go solve versus, you know, it's like, hey, we're going to benefit from just the entire community like pushing this space forward?

• 14:45Speaker 03

  Hi, I'm Anne Maris and I'm the producer of unsupervised learning. Thank you so much for listening to the show and if you've enjoyed the conversation, please subscribe. Subscribing helps us grow, which helps us continue to lock down some amazing guests. Thank you so much for your listenership and see you next episode.

• 15:01Jacob Effron?

  As like a CEO, how do you think about like the level of, of what the kinds of problems you want your team to go solve versus, you know, it's like, hey, we're going to benefit from just the entire community like pushing this space forward?

• 15:13Speaker 01

  We actually love open source because I think a lot of our alpha is, is in the proprietary side, in the infrastructure layer, like all this like cool stuff around like auto scaling, for instance. So I think unlike other inference providers, we've actually leaned very heavily into open source. We work very closely with both the VLM team and the SGLA team. We, we contribute very actively to both those repos and, and do a lot of optimizations with them. And so I think this may be a little bit of a different approach than maybe other inference providers. Of course, there's going to be, you know, certain things that, you know, we, we keep internal, but, but almost everything we, we push to, to open source. And I think it's, you know, it's because like we have so much alpha in the infrastructure anyway, like we might as well just, you know, work with the amazing open source engines

• 15:58Speaker 01

  that are out there that maybe were, you know, they weren't state of the art like, you know, a few years ago, but now they are. And, and that's an incredible thing that like we'd love to leverage frankly.

• 16:07Jacob Effron?

  You know, you were talking about heterogeneous compute and the, the different places where you're getting compute from and also how the, you know, kind of inference stack is changing. How important are, you know, the long tail of chips to you guys, the non Nvidia chips and, and things like that, as you think about your supply or kind of what the, what your serving stack looks like over time?

• 16:29Speaker 01

  I mean, I think Nvidia is still like the king here, like they have the best chips. There's way more demand for Nvidia versus anything else. Like, you know, we're, you know, curious about some of the other chips, like, you know, we made some experiments, like taking a look at it. But Nvidia definitely has the best chips, but you know, like looking at AMD, Tranium, TPUs, et cetera, like starting to be like pretty good. So we're always curious about that. And then, and of course, there's, there's a lot of like older chips too. And, and I think the, the challenge has been like for a lot of the LLMs, like you're kind of constantly pushing like, you know, how many parameters can you fit? So, so typically the latest and greatest LLMs, like they will always require, you know, a full node, maybe even in the future, even multiple nodes, who knows?

• 17:11Speaker 01

  So like eight B300s, like they'll require the latest chip and like eight of them, because that's how many you can fit on a node. But there's plenty of models outside of that, like some older LLMs, smaller ones that have, you know, they're optimized for performance, might be fewer parameters. And then outside of language, there's a lot of stuff around like voice and, and, you know, audio transcription or speech synthesis or like, you know, pharma, like bio stuff, like a lot of that stuff, actually they don't require like the latest and greatest GPUs. Like they actually get more cost efficient serving on, on Hoppers or even Empires or like, you know, different types of like L40X or RTX Pro 6000 and stuff like that. So we actually, I mean, we are ramping up like very, very aggressively on, on black

• 17:55Speaker 01

  walls, primarily GBs now. But we see actually like plenty of demand also in the older chips, which I think is quite exciting. There's just so much stuff going on. People forget about the Hoppers, but like they, they are our most common type right now, still at modal, H100s. We run some very like large production use cases based on Hoppers.

• 18:18Jacob Effron?

  How efficient like is model to chip matching today, like, you know, in the industry at large? Do you feel like most people are using the like optimal, you know, chip or, or, you know, from a price perspective for what they're, what they're working or is there still a lot of room for improvement there?

• 18:30Speaker 01

  There's a little bit of room to improve. I mean, one of the things that we want to do over time is like kind of automate that for people. I, I, I think in practice is like less of a problem than it might seem like, because often like you're kind of memory constrained. So like knowing how much memory you need for a model, like you basically pick the cheapest GPU that fits the model. That's usually like the heuristic that works in almost all cases. So if you have, you know, if your model is 80 gigs, like you can either fit it on a hopper or an older A100, 80 gig. I guess in that case it depends a little bit on the flops. But, but in many cases, it's like whatever, whatever GPU like is the smallest that fits your model. I think over time, like there's a bunch of stuff we've been thinking about that where like basically you just deploy something, a model, and then we manage like different

• 19:14Speaker 01

  chips and maybe automatically give you discounts because like we can, you know, the more options we can have to like run it on a lot of different chips like that lets us aggregate capacity in a better way and, and, and run things at higher utilization. Because then we can switch between different GPU types as, you know, there's like under utilization on those. So, so that's something actually we're, we're quite interested in doing over the next six months. It's like giving people an option to, to deploy things in a way where like we automatically switch types underneath in a way that's still consistent with your dollar per request or dollar per token or whatever. I think there's incredible cool stuff to do there.

• 19:49Jacob Effron?

  That's super interesting. And then one thing I imagine is so interesting in, in your role too, is like you're obviously predicting for like what your compute fleet should be. And obviously you see what the workloads are today and you probably have some sense of, of, you know, what models from what on what chips. And you're also trying to predict like, you know, hey, a year or two from now, like what, you know, what kind of workloads will we have? I think there's lots of, you know, there's, there's lots of different debates about whether we're going to, you know, most of the, you know, most tokens running will be really large models or actually, you know, we've saturated most tasks and these models will get smaller and smaller. Like, you know, as you think about kind of the evolution of your fleet in the next year or two, like what, you know, what's your perspective on, on like where you'll see relatively more different kinds of workloads and less?

• 20:26Speaker 01

  I, I think there's been kind of like basically over a trend of like, like larger and larger models over the last three, four years.

• 20:33Jacob Effron?

  You won't bet against that.

• 20:34Speaker 01

  I kind of, you know, I, I definitely feel like that trend is going to continue, like we're going to need, like if B300s had more memory, I'm sure, you know, a KIMI K3 would be bigger. People are kind of like, you know, training the models in a way that they fit on these, these chips. So I, I think on the, at least on the LM side, like on the, you know, LLMs are in a way like kind of accrued like type of world model, like you're trying to like encode as much information as you can in these LLMs. Like you need like very big, like almost like lookup tables of facts, you know, in a way that's kind of a conceptual, I think about it. And, and, and that sort of pushes like the memory requirements a lot. I mean, at some point like it might taper off, like who knows, but, but like for now, like people will make use of every possible memory, like the more memory, like they'll

• 21:19Speaker 01

  just train more parameters for sure.

• 21:21Jacob Effron?

  You obviously sit in like the, the, in some ways, I'm like the eye of the hurricane of like a very exciting, large and competitive market. And so I'm curious, as you think about like, you know, what ultimately will kind of differentiate players in this space and like, you know, and, and win out. I think there, there's so many true believers in the size of the overall inference market. And I think lots of folks are, are asking what actually, you know, wins out in the end of the day. And I know you have a good thesis on this, so I would love it for you to, to share with our listeners.

• 21:49Speaker 01

  I mean, I think a lot of the inference market will be quite rough, right? Like I think the sort of straight down the middle, just like, you know, serving tokens, you already see sort of open router basically commodizing the underlying providers. I, I tend to think, you know, like. The real alpha or like the real like point of differentiation. Like there will be a lot of differentiation just in like more efficient serving. And but but it is a little bit of a race to the bottom. And I think and I think there's going to be a lot of opportunities to sort of stay ahead of competition. And you're gonna have to like put a lot of effort into that. But but I think a more sort of durable differentiation will definitely be in like, you know, providing the entire platform of a lot of different things, which is like, I think something we are quite well positioned to do.

• 22:29Speaker 01

  We have the sandboxes, we have reinforcement learning, a lot of customers doing also normal fine tuning. A lot of customers, obviously, you know, deploying other types of models. We have batch processing. I think I I think my my my view in general is like customers, they don't want to buy like single trick ponies. Like they want to buy like a multi-product uh set of tools. So that to me is like ultimately like probably the biggest source of differentiation is like, you know, how do you provide that that entire set of tools? How do you provide this like very general purpose thing that lets engineers like iterate very fast, deliver value? And then like this sort of day to day, like we're still gonna have to like go and do the knife fighting of like, you know, staying ahead of your competitors in terms of performance and use the latest kernels and the much more efficient KV cache. And there's gonna be a lot of that stuff for a long time because there's just so much money in it.

• 23:15Speaker 01

  But I I I'm kind of pointing out, I I think that's sort of that's like less durable as an advantage. I think that, you know, because over time, like people, th- these tricks will sort of uh spread and people figure out how to how to copy them. Uh so the more durable thing, I think, is like being that sort of portfolio of a lot of stuff around inference, not just the inference in itself. You know, having all the models and and and and a lot of, you know, stuff like around reinforcement learning, maybe, or or other things.

• 23:41Jacob Effron?

  You know, the biggest modal customers are are on multiple products. Maybe can you touch on a few of uh the more interesting ones?

• 23:49Speaker 01

  So yeah, we we have some incredibly cool customers like cognition is using us a large scale for uh both for inference and and sandboxes. They do a lot of reinforcement learning on model, these sandboxes for all the rollouts. We have customers like Ramp using us for background agents, for the internal coding agent that they actually launched as a as a standalone product today that they're selling. We have customers like Suno and and Runway that use this for generative video and audio. Dakagon is another customer that uses us for a lot of customer support stuff, voice and and text. DoorDash uses us for a bunch of internal applications, mostly LMS, but also all kinds of other stuff. We actually have a lot of customers in in in computational biotech using us for for various types of drug discovery.

• 24:28Speaker 01

  Using, you know, molecular simulations and sequence alignment and all kinds of stuff. A big bunch of customers in robotics, um one X and and a few other ones using us to train robot models, like doing a lot of training, uh but also batch processing of video and things like that. A lot of customers in the data generation space. So it's the customers who are building environments and and running benchmarks on model off using um uh sandboxes.

• 24:56Jacob Effron?

  Maybe one thing you kind of mentioned this portfolio of products, and I think that's like, you know, one of the most interesting parts of the of the business you're building. And and I imagine like honestly, one of the hardest parts must be just like prioritization of all the different things you could you could build. There's probably so many, you know, new things that come up of of of really exciting tools you can build for developers. And and, you know, I guess I'm curious, obviously, you kind of, you know, with sandboxes, you have, you know, incredible products around, you know, coding agents around, you know, reinforcement learning. You know, how do you think about kind of building down that vector? There's also kind of like the fine tuning or all as a service stuff that some of your competitors seem like they they prioritize. Like, how do you think about, I guess, you know, where to prioritize within that portfolio?

• 25:34Speaker 01

  I I think it's an incredibly hard problem at, I mean, most startups, but especially multi-product companies, right? Like the sort of balance of like, you know, first of all, like, when do you go multi-product? And and then where do you put your bets? Uh like, we're a hundred and fifty engineers or something like that to today. And and I struggle with it every time, like every day, like, you know, I wish we could, you know, do that and then, you know, and whatever. And and like, but I kind of know that even if we had a thousand engineers or ten thousand engineers, like it's all, you know, there's lots of dollars, like, you know, I wish we could build a rocket and go to the moon and whatever. Like, it just never goes away. So so like, you kind of have to be just like ruthless with prioritization and really think about, you know, what's the highest ROI? And like, where do you draw the line?

• 26:16Speaker 01

  And you have to be sometimes like pretty aggressive also saying like, actually, like, I know we started on this project. I know you're really excited about it, but like this other thing kind of came up. So like, we're gonna have to pause this for a second, we'll get back to it later. We've had to do that many times too. It sucks when it happens, but um I I really think, you know, that sort of prioritization and like, how do you how do you allocate resources? I mean, resource allocation is like basically like, you know, the number one thing that I think a CEO has has to has to really, you know, be ruthless about at a company and like so much value is being created, like allocating the the work to the right resources.

• 26:50Jacob Effron?

  The modal roadmap in each board meeting is extensive and, uh, you know, very daunting. And I'm always impressed by the amount of uh of software that the the team can ship and hardcore software while while keeping the system running. How much has uh, you know, you guys were building the company pre chat GPT and then, you know, post chat GPT moment. How much has that changed, you know, maybe on the product side?

• 27:17Jacob Effron?

  You know, what you guys were doing, but then also when you were thinking about building uh and the velocity that you can ship software at?

• 27:24Speaker 01

  Yeah, I I mean, we always benefited from, you know, being quite well positioned at a time where like AI kind of blew up. And that's been incredible to see. And we're powering so many different AI applications today that like I had like no idea would exist like, you know, four years ago, five years ago when we started, but also so that's been kind of incredible. And and like you said, like software in itself is also like how we build software has changed a lot. People write like, I I mean, like, I I think probably like at this point, like ninety five percent of the lines of code that gets merged at modal is probably written by agents to some extent. Infrastructure still, I feel like relies a lot on like agents are like kind of good at it. I've actually been on up here like the last few days, like building or last few months, actually, like writing like an insane amount of code using agents.

• 28:05Speaker 01

  And like, it's kind of incredible like what they can build, but like you kind of still need humans to like supervise and quite a lot. And like, you know, infrastructure has a lot of tail risks around the reliability and taking and making sure that things scale and like stuff like that. So so so so at least for now, I I think software engineers are pretty safe. And and I think we we still benefit from having incredibly good engineers that can supervise these agents. And I think probably for the next few years. Another thing I I think has been kind of interesting to see is like, actually, like, I think the adoption pattern is actually the opposite of what I would have expected. Like, I thought it would be the junior people that would adopt agents the most. I've actually seen almost the other way around. It's like the very senior people, because for them, it's like giving them like ten interns. Like, they've been kind of constrained by like typing speed for a long time, but they have all these ideas in their heads.

• 28:50Speaker 01

  And I think using agents, now they can get it like, they can get it out like much faster. They're really good at building a test suite, like, you know, give the agents like, here's an extensive text test suite. It implements all of, you know, the Post six standard for file systems. Now go write a new file system. And it just goes on like this writes a file system and takes like, you know, three days to do it. So so we've seen, you know, that some of the very senior engineers like go like complete agent maxing. And I don't know, I I mean, it's kind of incredible like what you can do with agents. You can move very fast with them. Uh at the end of the day, it seems like humans are still safer software engineers for the next few years. I don't know, we'll see.

• 29:23Jacob Effron?

  How have you found the velocity of shipping software has changed what you guys need to do with code review or C I C D? Or uh deployment, have have those pieces of your development lifecycle changed at all?

• 29:36Speaker 01

  I think a little bit. I think we're coming up against some of those constraints, like we've had to spend a lot more time on C I C D. Like, I always like, you know, even long before agents, I always like, you know, I was a CTO for many years. Like, I was always like obsessed with C I C D as like, you know, very, very important to to figure out. So like, in that sense, like, I don't know if it's like changed like my view as like, you know, you like, you really need to have like automated deployments, automated tasks, like all these things. Now, by the way, agents are pretty good at writing tests too, they weren't like a year ago. So I I I don't know if it like we've necessarily like yet like run up against the foundational limits. But I mean, Anthropic, like, you know, a couple of days ago, they announced they're moving off Git, I believe, or at least GitHub. Uh I we might, we might run up against those limits at some point when we have, you know, uh ten X the amount of commits every day.

• 30:20Jacob Effron?

  But it's interesting to think about kind of the the, you know, how different the world is with these really powerful coding agents. And obviously, I think, you know, part of of of what I believe your your co-founders talked about is is kind of the shift even in who the consumer of modal is over time, right? From a from a developer to, you know, uh to agents themselves, right? And I'm curious like how you think about what that means for modal as a business and some of the decisions you might make differently as, you know, more and more of of of folks interacting with modal are either doing it via an agent or purely end to end an agent.

• 30:48Speaker 01

  It's actually changed like a lot less than than you than I think I would have expected. It turns out agents, I mean, like in in the early days, like we we built Model in order to give engineers a better developer experience and the ability to move faster. It turns out like agents kind of want the same thing. Like they want fast feedback loop, they want like good error messages, good documentation, a lot of the stuff like that that in engineers want to. So so so most of the stuff is actually quite similar. I I do think it's like certain things a little bit harder, like one of the things like we found, for instance, like deprecations is much more annoying. Cause uh agents are trained often on like a lot of old the code that exists online. So removing features is much harder for us, like we can't just rely on like, you know, remove it from the documentation and wait three months.

• 31:33Speaker 01

  Because like it's in the training data, people will use, you know, L M's to generate model code and that will be like, you know, using deprecated features. Uh and and that's like quite hard to change. So we've had to change our deprecation we know quite for quite a lot. Uh another thing we we've started you doing quite a lot is also uh setting up a lot of automated pipelines to to basically have like L M's continuously write model code and making sure it it compiles and or not compiles, but it builds and and runs, trying to just like catch like things that agents struggle with. Uh and and then maybe another uh thing we we spent a lot of time on is more like kind of like agent optimization stuff, which is actually more like similar like search engine optimization, like if you

• 32:15Speaker 01

  you you can go and ask, you know, Claude Code, like something like, hey, like, I want to do something with inference, I want to deploy this model and and not say anything about Model. And just like see like which tool does it recommend? And and and that obviously is something we we wanted to recommend Model as much as possible. Uh so so that's something we track internally, make sure that, you know, over time, like. And I actually think that's like something that that um will change software engineering a little bit. Maybe almost like to do worse, like I like I I feel like in a way like because of the training data and and the training loops, like there's more like, you know, like incumbency power in a way, like there's more staying power in software today. Like if you get inside the training data and you get the models to recommend, you know, a cer- a certain framework.

• 33:00Speaker 01

  Uh I I've heard people say like, oh, I'm just gonna use React for everything because agents are really good at it. I'm not gonna try these like new fancy front end frameworks, right? Like I I think there's like something similar with like a lot of the frameworks today is like that staying power is actually very, very valuable now. So so we obviously want to make sure that agents recommend Model as much as possible because that creates like a little bit of flywheel.

• 33:19Jacob Effron?

  And one thing I'm I'm struck by is you obviously had this like very pression bet in twenty twenty three on on sandboxes, right? It sounds like you guys had kind of this instinct of like, hey, this is gonna be something that's required for for coding agents. And then, you know, it took a few years for it to really take off and then, you know, it became this really important primitive. I'm curious, is there anything else that you guys are working on today that you feel like you're a little early on? But like, you know, you're you're you're equally as excited and and how much of your product portfolio is like those kind of bets versus like, okay, we know this is a hair on fire, like here and now problem.

• 33:48Speaker 00

  Jacob, are you are you fishing for the next thing we should be investing in?

• 33:51Jacob Effron?

  A hundred percent, a hundred percent. I'm nothing if not shameless.

• 33:54Speaker 01

  I I I feel like sandboxes, they did create a little bit of like a high bar for us, like everyone's like, what's the next thing? You know, what's the secret sauce you're working on and like what's cooking in the lab, you know, and and I I I wish I had like an easy answer, like, yeah, like a hundred percent confidence, you know, this thing is.

• 34:09Jacob Effron?

  Would you have answered sandboxes in twenty three or was it just like part of a bunch of things you were doing?

• 34:13Speaker 01

  No, because what sandboxes represented back back then was like a little bit of like an asymmetric bet, like sandboxes was not like a very hard thing for us to build back then, like to getting out like a prototype of sandboxes took like Akshat like a few days. And then I spent a little bit of time, you know, like improving the S D K and some other people like started building. And then we started seeing some customers, we rewrote it. And then in twenty twenty five, when it started exploding, then we had to rewrite the system like five times just to handle like more and more scale and make it faster and and and more reliable and and things like that. So but but but the initial investment was actually pretty pretty basics. So in in that sense, like it was a little bit like a sort of asymmetric bat, like a kind of a low downside, high upside bat. I don't know if there's like anything similar today that that we're working on.

• 34:55Speaker 01

  I am, I mean, like, just just kind of called out like a couple of like, you know, things that I'm quite bullish on and and these probably represent like way larger investments. Uh like, you know, larger risks and and, you know, in terms of like the that size. Low latency, like I I think, you know, there's gonna be a lot of voice to voice, a lot of real time video type stuff. Uh that requires us to rethink a lot of how we manage to compute like globally, how do we get down the number of hops, like how do we scale up and down in different regions, like almost like building a C D N for inference? Uh so I'm pretty excited about that. Um I think training is actually kind of an underrated thing. I think there's a lot of opportunity in in in training, like midsize models and reinforcement learning.

• 35:35Speaker 01

  That's been a big bet for the next for for the last couple of years, like getting all the R M A, InfiniBand type stuff working. Um but I'll say, I think a lot of V M like stuff is is quite interesting, like I think, you know, supporting full blown operating systems like Mac and Windows and Android and iPhone could be kind of cool. Um I think there's a lot of upside there. So I I I have a lot of those ideas in my head and we we we we're like building a lot of those things. Um I don't know if there's anything that represents the same like sort of explosive potentials like sandboxes. I mean, maybe like if we have this podcast again in like three years, like we'll we'll kind of look back and.

• 36:11Jacob Effron?

  We're we're gonna make it a recurring at this point, so we'll uh we'll we'll we'll look we'll look forward to uh to revisiting that. You know, I I guess like obviously, you know, sandboxes have become this fundamental primitive for, you know, coding agents. And it feels like that then brings you into this world where there's like, you know, I imagine you look at the current state of coding agents, you're like, God, there are so many more tools that would be helpful for these agents to make them uh more effective for people that use them. How how do you think about like, you know, whether you further build into that world or like the set of the things that are interesting for you there?

• 36:39Speaker 01

  We we do think about it, but we I think we think about it more from an infrastructure perspective. I I think it's very easy to kind of get lost in like, you know, whatever is like the latest today. When I look at like the architecture of like agent harnesses and like how we're building agents and stuff like that, I still kind of feel like we're so early in these architectures like evolving so fast and like what what what our job to some extent is like trying to figure out is like what are like the sort of core building blocks that we can put out today that like will last, you know, like people, we need them for the next five years. And and that to me is like less obvious, like we we are interested in like sort of agent orchestration uh as a category. I think there's a lot of interesting stuff there.

• 37:16Speaker 01

  We're interested in like, you know, evals and model gateways and all kinds of other stuff around like, you know, like the the whole like, you know, life cycle of of of learning from agents and and innovating on our agents, but but I think it's like less clear exactly like how the architecture. My my view is like, it will be more cloud centric and it will be probably more, you know, like like durable objects, like kind of workflow engine type driven, and we have some ideas of like, you know, stuff to build in that space. Uh but but I I I find it like harder to like, you know, lay out exactly what agentic development will look like in five years. Part of it because like I also think like to some extent like all software will like basically be agentic, like I I kind of hate and hate the word agentic in certain ways because I I feel like it's like,

• 38:01Speaker 01

  okay, so it's using A I to make decisions, like, well, shouldn't all software do that? So like I I almost like feel like if you look at it on a five year horizon, like I I think like software, like all software will become agentic. And then what does software look like then? Like, I I don't know, but I, you know, I I I I hope we can build a lot of the building blocks to support all kinds of aspect of like, you know, in general, just like how software uses A I.

• 38:22Jacob Effron?

  Yeah, I mean, it feels like one of the hardest parts of building an for a company, right of, you know, across the board for many people, you know, building A I for companies of, you know, what what actually persists, right, versus is a pattern, you know, is is like rag from two years ago or like, you know, whatever it is.

• 38:35Speaker 01

  Rag is a good example, exactly, right? Like if we had made a bet on like rag like two years ago, like it it would have been kind of a bad thing to do. So so yeah, but but I I would also, so so it's hard, but I would also say like one of the benefits of building an infrastructure is like you can take a longer view and you don't have to like listen to all the noise. I mean, you you you have to listen to the noise, but like you kind of have to like ignore a lot of it and trying to figure out, okay, what are the patterns here?

• 38:58Jacob Effron?

  You kind of alluded to some of these other, you know, other workloads that like low latency workloads like voice to voice and and and real time video. I think there's a lot of of chatter in the air about like multimodal inference, robotics inference, these other categories. I'd love to hear you just riff a little bit on like, to what extent are those problems, you know, similar to the set of problems the inference markets had before? To what extent are they like different and new?

• 39:18Speaker 01

  I think they're all quite similar, but all, I mean, they're like snowflakes, unique in their own way. Uh robotics is gonna be a lot of it on on device, of course. So that's a big difference and and, you know, I I don't sort of envision like we're gonna support, you know, most of what robotics looks like, because a lot of it is gonna be on the devices themselves. But I think we're gonna be able to support a lot of the testing and development and training of the of robotics and a lot of the simulation environments. So that's something I'm very excited about. Uh on on the multimodal stuff, I mean, there's audio and music and image and stuff like that, like people don't talk about it as much, but that keeps growing like crazy too. I I feel like L L M's have kind of dominated the the airspace in the last few years, but I I'm just incredibly

• 40:01Speaker 01

  you know, excited about all the different applications, like why are like voice assistants so bad in twenty twenty six? Like, I don't get it, like, they're just horrible. Like I have a Google Home and and like every question, it just bungles, it just doesn't understand what I'm like, you know, so I just go to Chat G P T. Like, why can't you just connect all of it like end to end and train like a a system so that that that can, you know, do like speech to speech?

• 40:25Speaker 01

  I I I I think there's a lot of, we we're gonna see a lot of sort of incredibly cool applications in that space. I I always think about, you know, you look back like, you know, four years ago, three years ago, whatever. You know, could you have guessed that like things like Instinct or Suno or like ignition or like, none of those things like are very easy to guess, like, you know, like three years ago? So I I kind of feel like also like if you look down the road like three years, like most people don't have an idea of like what the world is gonna look like in three years, it's so hard to to know. But like what I do know is like kind of judging by the past three years, it's gonna be like, there's gonna be some really cool applications. I just don't know what it's gonna be, but they're gonna be very cool. And hopefully they'll use modal.

• 41:04Jacob Effron?

  I was looking back at an email you sent me with the modal series A pitch deck. And this is what you said. This was after we had signed a term sheet. By the way, we never actually reviewed a modal deck. But here's something we made earlier that I just updated with the latest numbers. Could be good review at some point. When you sent this, you were roughly 2 million in run rate. You now, you know, you've crossed a billion run rate, which is insane in three years. What's been the biggest kind of change to supporting that kind of scale in just, you know, a couple of years?

• 41:38Speaker 01

  So first of all, I actually realized like the other day, I said on another podcast that I've never made a pitch deck. And then I was actually thinking about it, I was like, maybe I did make a pitch deck after this series A. Cause we had a couple of angels that came in and they were asking about it. So I was like, oh, fine. I guess I'll put something together. So I realized I did lie, like I did make a pitch deck, but it was after.

• 41:59Jacob Effron?

  You never showed it to us, so I don't know that it counts.

• 42:02Speaker 01

  So I don't know if it counts. Like I never like sent a pitch deck to an investor. I sent it to a few angels, actually. But that, and that's the only pitch deck I've ever made, actually. But in terms of the scale, yeah, I mean, we've grown, you know, 500x, I guess, in terms of revenue, something like that since then, over the last three years.

• 42:19Speaker 01

  So a lot of growth. I think, I think, you know, I like, I don't know, maybe I'm like, just, you know, my, my experience has mostly been at hyperscale startups. Like I was at Spotify for seven years and some other places. So, so for me, like this, like fast growth is kind of all I've seen, like, you know, there's like, you know, every, every six months, you have to rewrite all the systems. You have to kind of rethink everything you're doing. You have to kind of change all the processes. Like you always feel like you're behind. Like, I don't know, I kind of like the mess.

• 42:50Jacob Effron?

  I guess we would love to, you know, since, since we have your, your mind on the podcast, would love to kind of zoom out. Maybe talk a little bit about the broader info market and some things that have been in the news of late. You know, maybe most top of mind these last few weeks, I feel like there's been tons of discourse about neo clouds, right? They like, you know, how the debt's trading, they traded down, they're kind of back up a little bit now. And I'm curious, like, what do you think happens to this landscape over time?

• 43:12Speaker 01

  I think the neo clouds are like fine business models. Like I, I think people have been like confusing it like a little bit. They're like, wow, there's so much demand, like revenue is going to grow like so fast. And then, you know, there's a little bit of sort of a, almost like a VC mindset of like, oh, like thinking in like revenue multiples. But I think all of these, like, these neo clouds, like growth is actually quite expensive for them. Cause they have to go and raise a lot of debt. And so I don't know if necessarily like, you know, just because like they grow a lot, like, you know, like it's very like capital inefficient. Like these companies are at their core, like, you know, financing vehicles, like they're very, they're doing leasing, like of hard assets. And that's, you know, and that's like inherently, you know, harder to scale the capital now that that market is kind of saying, whoa, we deployed a lot of capital into neo clouds.

• 43:57Speaker 01

  Like maybe, maybe hold on for a second, like, you know, we don't have that much more capital to lend. And so the interest is starting to go up.

• 44:05Speaker 01

  But also, by the way, like these companies have incredibly high leverage. So of course they're going to have more volatility, like the, the residual equity value is going to go up and down much faster on the stock market with GPU prices and stuff like that. But they're like fine business models, like, you know, they kind of locked in customer contracts and raised debt against it. And like, you know, now they're paying off that debt and, and in many cases, it turns out the residual value has been higher than people expected. Lifetime of GPUs has been longer than people expected. So they can resell once they paid it off, like at a higher price than people thought. So like, they're going to make a bunch of money from that. Like it takes a few years to get there. But I think in that sense, like neo clouds will do well. They're not particularly differentiated. So I, you know, long-term, I think they're going to.

• 44:47Jacob Effron?

  You work with a lot of them, like, is, is there any differentiation you see?

• 44:51Speaker 01

  It's like a little bit, but like at its core, it's like, you know, like relatively undifferentiated space. So I, I mean, my view is like, you know, I mean, Elon was like tweeting the other day, like, oh, the neo clouds are going to die. Like, I, I think that's kind of bullshit. I do think Elon had a point when like basically like when he implicitly was saying like SpaceX has a much lower cost of capital. I don't think there's a coincidence. And if you look at the, like the, the three successful hyperscalers, Amazon, Azure, or AWS, Azure and GCP, that they're all attached to companies that had very solid cash flow. And they're all public companies that had very low cost of capital. Cause that at its core is, is makes the life a lot easier if you want to build something that's very capital efficient, capital intensive.

• 45:36Speaker 01

  So I, I think in general, like that is something that neo class will struggle with. Like they have, you know, they're going to have a high cost of capital and, and it's going to be hard for them to grow fast because of that. So, so we'll, we'll see about the market, but they're like decent business models. I think there might be an opportunity to buy the dip. I don't know.

• 45:54Jacob Effron?

  You had an interesting tweet that was a lot of the, the neo labs could become neo clouds. You know, if their, their model doesn't work out, but they've locked in these contracts.

• 46:04Jacob Effron?

  I definitely could see the, the world going that way. If, you know, if you have valuable compute that continues to appreciate because there's so much demand, but.

• 46:12Speaker 01

  Yeah. I mean, Mr. All kind of did that, right? And, you know, like it seems to be fine. I think a lot of people are making these like very large buying decisions and they kind of rationalize it like, they're like, well, if you don't need the GPUs, we can just resell them, right? Like that's almost like a form of insurance. And you have to, you know, assume with like all these neo lab, you know, there's so many neo labs out there that have each raised 1 billion. They always raise exactly 1 billion dollars for some reason. I don't know why. I guess they like that number and then buying GPUs.

• 46:39Jacob Effron?

  Very precise financial modeling that points them to the optimal amount.

• 46:42Speaker 01

  Yeah. So, so some of them I think will probably work out really well and, you know, but I imagine in the grand, you know, like some of them will not work out. And if they don't work out, well, now they're sitting on a ton of GPUs, like, you know, let's resell those GPUs, right? Like, especially if there's ever like a scenario where like VC funding pulls back. Cause I, I think what's, what's kind of a scary thing about some of these commitments is like, you know, effectively like what, what happens today when you make a GPU commitment is you only have to pay about 30% upfront, sometimes a little bit more, sometimes a little bit less. But what that means is like, you can raise a billion dollars, you can go out and get GPUs for $3 billion and only pay a billion dollars upfront. I mean, not exact math.

• 47:25Speaker 01

  But the problem is like, you know, in a year, now you're on the hook for another few hundred million and like every month, now you have to pay. So, so I, I think that creates like a little bit of like implicit leverage in a lot of these companies. And, and, and a dependence on subsequent fundraising. And, you know, that works as long as they can go and raise VC dollars to fund the GPU bills. But at some point, it might not work. And in that case, I think you're going to see it a little bit more of a secondary market where some of these neo labs might, you know, pivot to doing a little bit more of the sort of neo cloud style. Like we have a bunch of GPUs or just resell it back to the cloud or. I don't know. I'm very bullish on AI as a whole, but you know, I always like look for these like sort of structural like risks. So overall, I think there's going to be like incredible application, incredible demand.

• 48:09Speaker 01

  But like, you know, is a hundred percent of the neo labs going to succeed and, you know, be, you know, trillion dollar company, like probably not.

• 48:15Jacob Effron?

  You know, I feel like on the AI emphasis side, one big category that has gotten a lot of buzz is like this. Harl is a service market and kind of a bunch of folks, you know, almost forward deploying and like helping companies build their own models. And, you know, you have had folks like Satya saying, hey, every company is going to have their own model. Like, do you subscribe to that? Like, what do you kind of think of this, of that overall market?

• 48:35Speaker 01

  Now, I think some companies will have their own models. I, I think that like weighted by dollar revenue, like I, I think most models will probably be post-trained, right? Like, but, but like weighted by company, like in terms of like numbers of companies, like most companies will not have the, the need to like post, do their own post-training, right? But like the large ones, right? Like cognition is training their own code models and, and cursor date and, and all these companies are. And like those are the ones also serve a lot of tokens. So I, I think when you look at like reinforcement learning and post-training, like certain types of, you know, parts of it is very attractive, right? Like you can work with companies and, and serve, you know, better models and do it at lower cost and higher throughput and all of this stuff.

• 49:19Speaker 01

  Some of it looks like it's kind of like just a service business. So I, I think a lot of this sort of tension over the next few years, when you look at post-training is going to be, can you turn this into product? Like, can you sell that product to these companies? This is something we think a lot about. It's like, how do we create this like amazing product you can just kind of sell? And not turn it into just like a service business.

• 49:39Jacob Effron?

  And then, you know, I guess last one before we move to the quickfire round would just be like, you know, I think there's this, this question about, you know, how many tools live under like a single info provider in AI world versus, you know, our separate categories? I've always struck when we talk to you about like just how much you want to go build and, and all these kind of building blocks. I'm wondering if there's categories of AI infra, you know, that you think will be unique and distinct from the stuff you're building that you're optimistic on and that will kind of like live as, as still large standalone opportunities.

• 50:06Speaker 01

  Yeah, I mean, we're not going to go into model training, I think. Like, so I think type safe with Jav is going to be, you know, there's probably going to be a lot of these like custom models. I'm sure in other verticals too, you know, other modalities like audio and video and music and like, obviously on the application side, it's going to be a lot of incredible companies. And then there's probably going to be a lot of companies like focusing more on like software development and and stuff like that. And, you know, I, so I, I don't know, like, my, my general view on these things is like, there's always like, more boxes on the market map than you expect. But I also do think that overall, every company that is successful in the end, to some extent, has to become a multi-product company too. So I don't know

• 50:48Speaker 01

  if that's a tension or not. Like, I, I, I think, you know, we obviously want to find, like I said, a cluster of like very adjacent things and do all of it in a way that like, you know, maximizes the revenue and the shareholder value. But I, you know, by no means, you know, are we going to try to expand into everything. I think people kind of make this like, false assumption also, like, you know, you look at like Anthropic or OpenAI, and they're like, why don't they move up to stack and launch their own law firm and launch their own, whatever, like, they're very good at like, serving these models and doing that at scale. That's how they achieve scale, right? Like, and they're very good at doing that. Like, I, I, I don't think it makes sense for every company to do everything. And so similar for us, like, we're not going to do everything, but we're going to do a very large cluster of adjacent things and do exceptionally

• 51:31Speaker 01

  well. And I think there's going to be a lot of room in the market to create, you know, a bunch of, you know, very large, you know, 100 billion or trillion dollar opportunities in the market.

• 51:40Jacob Effron?

  We'll end on our quickfire round where we stuff, probably a similar set of questions, what we asked you three years ago, maybe a little bit of an evolution. To start, maybe I'd love just one thing you've changed your mind on in AI in the last year.

• 51:52Speaker 01

  Well, the opportunity size of sandboxes has obviously been kind of crazy, right? Like, I, I didn't think it would be anywhere close to, you know, I mean,

• 51:59Jacob Effron?

  I thought you knew that in 23, and that's why you made the, made the big bet.

• 52:03Speaker 01

  Yeah. So that's been kind of incredible.

• 52:06Jacob Effron?

  Coding agents have changed a lot of what it means to be an engineer. How, if at all, has it changed how you think about hiring?

• 52:15Speaker 01

  Not much. I, I think we're still going to need humans for a long time. I think the sort of demand for humans is just going up the more productive they get. I, I think, you know, we've seen this for the last 30, 40, 50 years of software engineering, you know, whether it's compilers or high level programming languages or CICD, like, every time you make engineers more productive, like, the demand just goes up. And so I think we're going to need a lot of different, a lot more engineers. I do think maybe on the short term, it's going to be like, a little bit harder for like new grads, like, you know, except for that, the very high side of the spectrum. But in the long run, I'm very bullish on still, you know, computer science. Actually, ironically, I didn't study computer science, but, but I'm very bullish on studying computer

• 53:00Speaker 01

  science in general. Like, if my kids wanted to do it, like, I would definitely encourage them.

• 53:03Jacob Effron?

  What's the biggest mistake you've made in building models so far?

• 53:06Speaker 01

  Ooh, I don't know. I think one mistake I made in the past is that, I think sometimes, like, you kind of want to sequentially do things. You're like, well, first we need to do this. This is the blocker. We'll get to this other thing later, because this other thing, you know, is blocked on this. But like, in a world where like, everything is moving fast, like, one of the things I've learned is like, sometimes you kind of need to do both at the same time. Like, you need to assume that, like, oh, we're doing this thing to unblock it. You might as well do this other thing at the same time, just in the interest of moving fast. That's something I, I'm naturally, like, not mega aggressive, which is kind of ironic, because like, we've grown 15x in the last 12 months. I've sort of had to retrain my brain to like, be a little bit more like aggressive and saying like, this is a massive market opportunity. We need to be even more,

• 53:49Speaker 01

  you know, deploy, like, hiring, deploying capital, like, whatever it is, like, massively just like paralyzing everything we can.

• 53:56Jacob Effron?

  Sweden has a ton of interesting startups that have emerged. What are your thoughts on, on why that is? And has the Swedish house mafia been replaced by the Swedish tech mafia?

• 54:07Speaker 01

  Did you know that, like, one of the Swedish house mafia members was my classmate? It's a fun trivia.

• 54:12Jacob Effron?

  Really? I did not know that.

• 54:15Speaker 01

  Yeah. Back in the early 90s, growing up in Sweden, when I was eight, he told me my bike was ugly. I was very upset. Yeah, I mean, we've had a lot of tech in Sweden, Spotify, Klarna, you know, a bunch of gaming studios. More recently, obviously, Lovable and Agora, stuff like that. I think, my theory about like, a lot of this stuff is like, success kind of just creates success. Like, you know, a lot of like, what makes people go into tech is like, they know about some cousin's friend who made a lot of money in tech or something like that. So, a lot of it is, I think, is just like, a little bit of a flywheel effect. I do think that Sweden, as a European country, has always been pretty entrepreneurial and commercial. So I do think

• 54:58Speaker 01

  there's a little bit of that. Like, we've always been a little sort of, maybe going back to the Vikings, like, kind of, let's go conquer the world. In the past, maybe it's been through music and furniture and stuff like that, but now increasingly with tech. So, you know, there's that. Like, I think there's sort of a mentality in Sweden, like, you can actually, Stockholm is like funny, because like, everyone in Stockholm thinks they're like, this like mega global city that's like part of the world, in a way where I actually think they aren't. But like, I think the fact that they think they're so global is like, kind of good. Like, they have this like vision that like, everyone cares about Stockholm. But I actually kind of like that, because it means like, they're out there like, building, you know, things, and they're saying from day one, like, we're going to conquer the world. Like, we're going to do this startup, it's going to be awesome. And I like that, like, the sort of mega mania.

• 55:39Jacob Effron?

  And now this vision of like, your elementary school reunion, and just what a wild, a wild group that would be. That's quite, that's quite the cast in what I imagine was not a particularly large class.

• 55:50Speaker 01

  Yeah, yeah, it's kind of a random fact.

• 55:52Jacob Effron?

  I guess a shameless question. Do you think GPU rental prices 12 months from now will be higher or lower?

• 55:57Speaker 01

  I think higher. You know, it's funny, like, I actually changed my mind on this, like, in the last 48 hours. And I'll tell you why. I, I read the Anthropic S1, and I actually had a, you know, agent summarize it for me. But it was kind of incredible how profitable that company is. And to the degree that I hadn't anticipated. I thought that, like, in my mind, like, 48 hours ago, my argument was like, oh, Anthropic, like, they have really good gross margins, but they're spending so much money on training that, like, as soon as they go public, they're going to have to like, slow down in how they grow training and post-training, and they're going to, you know, maybe grow inference as much, but not the other stuff as much. But then, like, I realized, like, they're basically a profitable company. So like, they have like, zero incentive to slow

• 56:40Speaker 01

  down on like, capacity, capacity acquisition. And I imagine there's a similarly dynamic going on in OpenAI. And so, and OpenAI and Anthropic has obviously been like, the biggest two buyers of capacity out there. And so the fact that, like, I don't think they're going to slow down buying, you know, because of, you know, clearly, like, financially, they're doing very well. I think it will mean that, like, there's going to be a lot more demand. And I think for that reason, prices will probably go up in the next 12 months.

• 57:09Jacob Effron?

  I've heard you say that you've always thought Modal, you've thought of Modal as your 20-year project, you're five years in. What do the, what do years 10 through 20 look like?

• 57:21Speaker 01

  Wow. I mean, hopefully, I'll hire a bunch of people that can do a lot of the sort of, you know, I'm too in the weeds of like, everything right now. So I, but, I don't know, like, I, I just like, love this stuff. Like, I, I love to keep working on this. Like, I think, you know, year 10 to 15, we're going to look more like a conglomerate. We're going to have a lot of different products. We're going to have probably some like crazy bets, going to moon and Mars, or whatever. But I still hope to be quite involved in this sort of the high level, like, resource allocation, and what are the things we're betting on. And I'll probably be less involved in sort of day to day, like, you know, let's get a contract from this GPU vendor, like, whatever. But like, I think that, like, sort of like, high level, like, resource allocation

• 58:02Speaker 01

  and stuff like that, to me is, that's the most fun. And I hope to stay involved in that for another 20 years, hopefully, maybe longer.

• 58:09Jacob Effron?

  Yeah, I mean, if we keep current pacing, at least five more unsupervised learning episodes in that timeframe, right?

• 58:15Speaker 01

  Yeah, exactly.

• 58:16Jacob Effron?

  Well, Eric, thank you so much for joining us. We really appreciate it. We'll give you kind of the, the last call to action for the audience. If there's, where can they go to learn more about Modal, or anything that you would direct them to?

• 58:29Speaker 01

  Well, you should obviously go to modal.com. You should also run pip install modal in terminal and play around with it. You know, there's a lot of people maybe listening to the show who are building startups, or thinking about building startups, and, you know, would love to see all the cool stuff they're going to build. And, you know, hopefully at Modal, we can help you with those startups. So let me know, and feel free to reach out, by the way. Like, I always love talking to customers. But feel free to play around with Modal, and looking forward to see what everyone's going to build.

• 58:55Jacob Effron?

  I'm Jacob Effron, and this has been Unsupervised Learning, a podcast where I get to talk to the smartest people in AI, and ask them tons of questions about what's happening with models, and what it means for businesses in the world. As I hope is clear, I have a ton of fun doing this. It's a nights and weekends project, in addition to my day job as an investor at Redpoint. But our ability to get these incredible guests on really comes from folks like you subscribing to the podcast, sharing it with friends. It's really what ultimately makes this whole thing work. And so please consider doing that. And thank you so much for your support and listening. We'll see you next episode.