Ep 21: Modal CEO Erik Bernhardsson on Bringing Development to the Cloud, the GPU Market, and GenAI Music

Jacob and Pat sit down with Erik Bernhardsson, the founder of Modal Labs, a data infrastructure company providing GPU compute to data teams. On this episode we discussed Erik’s thoughts on the AI chip market, the most popular GenAI use cases on Modal, and even Oracle Cloud’s resurgence in the AI start-up market.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Welcome to Unsupervised Learning, Red Points AI Podcast. Today, we had an awesome episode with Eric Bernhardsson, the CEO and co-founder at Modal. Modal is one of our most recent investments in the AI space and what I think is one of the most exciting companies in data infrastructure and AI infrastructure. They're recently on the information top 50 companies. In this episode, we talk with Eric about what Modal does, why it's an amazing developer experience, and then also just the broader AI ecosystem, how GPU prices affect the business, what's happening with the cloud vendors, open source versus closed source models, and much more. Thank you so much for joining us.

• 0:38Speaker 00

  Eric, thank you so much for joining us on Unsupervised Learning.

• 0:41Speaker 01

  Yeah, thanks for inviting me. It's great to be here.

• 0:44Speaker 00

  You started Modal as a general compute platform. Basically, anything a data team wanted to do, they could run on Modal. This is obviously still true, but over the last couple of years, Modal's really emerged as a great way of building generative AI applications. And so maybe just to start, what was your motivation for founding Modal? And did you expect that it would be such an amazing way of building with generative AI when you first founded the company?

• 1:10Speaker 01

  Well, generative AI barely existed back then. I don't think people talked about it. I don't know, maybe it did exist, but at least term is to me pretty new. But yeah, so I started working on Modal end of 2020, roughly. And my background is I've been working on data for most of my career. I was at Spotify for many years and built a music recommendation system in particular. So I spent a lot of time on large scale, like numerical methods and big data. In general, I always felt like working with data was hard. And, you know, most of my personal experience, like I always wanted to build better tools to work with data. It's fairly agnostic as to like where to start in the stack. I built a workflow engine called Luigi a million years ago at Spotify. So that was actually one of the things I thought a lot about is like, maybe I should

• 1:55Speaker 01

  build a better workflow engine. And then at some point, like researching that idea, I was like, well, actually, like, you kind of have to run the code somewhere. Like, what, like, that seems like a really hard problem. Like, maybe I should do that instead. Uh, so, so decided to, to, to go a layer deeper in the stack and focus more on like, okay, it's hard to run code in the cloud. Like, data teams seem to really struggle, you know, in particular, data teams, you know, whatever they're using, like Kubernetes or whatever. Like, just the point of like, you know, I have some ad hoc thing I want to like scale out, or I want to productionize a machine learning model, whatever it is. It's kind of hard. And so started working on that in, towards the end of 2020. Pretty quickly realized like, that's a hard thing. Like, it's a hard infrastructure problem. Like, I have to go down like very deep and like, you know.

• 2:38Speaker 01

  Like, like, just a few months in, I realized like, I kind of want to build my own file system, actually, because I want to load containers very quickly. And like, people that I was talking to back then, they're like, you're nuts. Like, just use like Kubernetes and Docker, like everyone else. But like, but my, my, my feeling was always like, no, like, in order to have the user experience that I wanted, like, I'm gonna have to rebuild those things. So, so it took a lot of time to, to do that, uh, to, to rebuild a lot of that foundational infrastructure. And then about a year and a half in, so like, around the summer last year. Uh, we built something we felt really excited about. Like, we had this, you know, product where like, you can write code and you can launch it in the cloud and kind of this works. Uh, and we had support for all kinds of different features. We didn't really have any customers.

• 3:21Speaker 01

  So, so that's, you know, so then we, you know, started looking around, okay, like, what are like the actual use cases? Like, just like, in, within a few months, like, we started seeing a lot of people coming to Model. You know, uh, looking for a way to, to deploy a lot of GNI applications or a lot of GNI models. Uh, and, and we realized there was something there. Uh, and, and I think what they saw in Model was in particular, like, we, we had GPU support. Uh, we, we could do things serverless, we were able to start containers very quickly. Uh, and in particular, also, I think there's like a lot of people coming to machine learning or coming to AI that really have sort of a traditional, like, bias towards existing tools. They just want to start from like a clean slate and like use something new.

• 4:03Speaker 01

  Uh, so, so we found a, a pretty good fit in that, and that's effectively now like what's been driving most of our revenue and growth. Do have some other customers, like we do have a bunch of people using this for like computational biotech or 3D rendering, web scripting, whatever. We're super excited like long term to also like, you know, focus on those use cases too. I think right now, like, you know, effectively, like, you know, Jana AI has been, you know, it is like by far a biggest use case. But, but yeah, like we are building a general purpose platform still. So this is hopefully many other types of use cases in the future too.

• 4:34Jacob Effron?

  I, you know, I think it would be awesome maybe to just contextualize a little bit the, the different options that, you know, developers have today that want to run, you know, Lama or stable diffusion. And, and how do you think about kind of the, the set of options in front of them today?

• 4:47Speaker 01

  I mean, obviously you can like roll it yourself, right? Like you can like, you know, run your own Kubernetes cluster and like, you know, spend like a few days, like, you know, dealing with CUDA driver versions or whatever. And, and, you know, set up Kubernetes and like, you know, on EC2 or 4K or whatever and like run it yourself. And I think, I mean, some people do that. Like, first of all, it's like super annoying to do from developer experience. Like, another problem is it's also from a utilization point of view, it's actually like kind of expensive, right? Like GPUs are expensive. And so like, what, like one of the benefits of Model is it's all serverless. You actually only pay for usage, which I think is like very beneficial when you're dealing with these like very expensive GPUs. Another option is there's a lot of providers out there that basically do like AI behind an API. And, and so replicate, I think, like there's a few others too.

• 5:33Speaker 01

  There's like zero customization. Like as soon as you have like your own proprietary model or your own proprietary like custom workflow. Like now you still have to write code and I have to deploy it somewhere yourself. And so you're kind of back to square one. And so like, I think what Model does is really well is like for custom code, like when you really need to do something custom, like you have your own custom models or you have, you know, it's a complicated workflow. Like we solve that and we also solve that in serverless ways. So you actually end up like lowering your costs pretty substantially. And we also do a much better developer experience. Like so, so you don't have to like, you know, as much as I love AWS, like I also like banging my head against AWS for like, you know, probably like multiple person years at this point, like, and like AWS is fantastic, but it's like incredibly frustrating to work with.

• 6:16Speaker 01

  So I, I think, you know, we represent like a new way of building, you know, cloud native AI applications in a certain sense that makes it a lot easier to work.

• 6:23Speaker 00

  I think it's really cool, just all the foundational technology that you guys have built. And you talked about building, you know, container runtime and file system. Like, what is the, you talked a little bit about what that means, but I guess why do that? Like, what is the advantage that you can give developers because of the way that you've kind of architected Modal?

• 6:45Speaker 01

  I think the main one like that that we started with was just like, I wanted to make engineering teams more, data teams more productive. And like, I thought a lot of it, like, what does that even mean? Like, you know, like, what was productivity? And like, a lot of that, like, to me, means like, fast feedback loops. Like, that's like, when I feel happy is like, when I write code and I have fast feedback loops. And also when I feel productive, like, like, I think the best way to like, understand developer productivity is actually like, from the point of view of like, you're iterating on code. Like, how quickly do you get feedback? And like, working with the cloud is like, super annoying. It's like, traditionally, like, if you had to like, deploy, you know, run something and like, iterate on it in the cloud. Like, you have to like, build a container and push it to the cloud and like, trigger something and then download logs, whatever. It's just super annoying.

• 7:22Speaker 01

  So, to me, like, you know, that basically like, precludes you from like, including the cloud in, you know, your day-to-day like, developer feedback loop. And then, which is like, for data, like, kind of bad, because like, I think there's a lot of reasons why, like, data teams actually kind of need to, they need to use the cloud in a lot of cases. Like, dealing with GPUs, for instance, or, or, you know, doing like, ad hoc, like, you know, scaling things out or whatever. Like, you kind of want to use the cloud, right? Like, you don't want to like, have to go and buy a GPU machine and like, like, so, so, like, I started thinking about like, you know, what, what can you do to actually like, take code? And like, launch it in the cloud in a way where like, it basically feels like you're, you're running things locally, but you actually run things in Modal's environment, in the cloud. Uh, in order to do that, you have to boot containers really quickly, for instance.

• 8:06Speaker 01

  And it, and it turns out like, you doing that with Kubernetes and Docker, traditional tools, there's no way you can, you know, get down to like, a second or two. Like, taking code on someone's computer, building a container and executing, launching that in the cloud, especially if you want to scale out or do all kinds of other stuff. Kubernetes is not optimized for like, fast feedback loops. It's, you know, it's optimized for like, run things at scale and handing out our requests, but that, that, it doesn't work for this particular application, so. So the biggest reason was just like, we wanted a fast feedback loop. And then later it turned out, actually, like for serverless stuff, actually it's kind of the same criteria. Like you want to be able to boot containers very quickly. Turns out it's extremely useful for serverless, like Lambda style containers too. So they ended up having kind of two benefits in the end.

• 8:50Speaker 00

  I remember the, in like the early days of Hadoop and Spark of like how brutal it would be to run a job in the cloud. It was like, okay, you would kick something off, then go get coffee and see the result. And like, we've come so far from that, but it still is like, Modo is really the first thing that I've seen that, that feels like it's local, right? Or you don't really care. It's just like it runs instantly and you get the result. And I think that kind of opens up a lot of opportunities for data people to, to build with it.

• 9:19Jacob Effron?

  Have you seen any concentration in like some of these initial use cases?

• 9:22Speaker 01

  Like definitely, I mean, I mentioned we do, you know, a lot of our use cases are in G N A I. And I would say specifically, we see most of the applications in audio, video and images, like stable diffusion, like Dreamboots, like that kind of stuff. Like you're seeing like music generation, like audio pipelines, like less language models. I think that the reason is like, just the best language model today is behind the proprietary API, right? Like OpenAI. And so for that reason, like people don't quite deploy language models like as much themselves. It's just hard to like, you know, get the same performance. But we're starting to see like more and more people doing L M's to on Model.

• 10:00Speaker 01

  But I would say like the, in terms of revenue, like the main, like, you know, the vast majority of it comes from like audio, video, image. And I would say also like inference, like we focus a lot on like inference, which I think to me is like long-term, like where the money is going to be. And like, you know, like where like most large scale deployments, you know, are going to be focused more on inference. We are starting to do more of fine tuning and a little bit of like doing training and like other types of stuff too. But definitely like inference right now is like where we see vast majority of applications.

• 10:31Jacob Effron?

  Yeah, can you talk through just like some of the specific companies like a Substack? Like you mentioned, they're kind of in video or audio or those types of things. But like how one of those or scale is using Modal and maybe why it was a great fit for them?

• 10:47Speaker 01

  Yeah, Substack is like they're doing very subset podcast descriptions with Modal. So they like one of the things they like with Models, like they can do this fan out and use like Whisper to transcribe a lot of audio very quickly. So you can take like an audio files, like chop it up into small pieces, just got in parallel. So that's pretty exciting. I think another example of like a company that I've been pretty excited about is this company called Suno that does music generation. And so they basically have a provider model running a model that like, you know, you type in like, I want to have, I don't know, a German hip hop song with bongo drums. And then it like generates that music using their own custom model, which is like, so there's a lot of like, there's really cool use cases that I see that, you know, to me is like very exciting.

• 11:32Jacob Effron?

  Yeah, it must be fun to see, you know, going back to your Spotify days. I feel like, you know, cool, cool, full circle there.

• 11:38Speaker 01

  It's true. Yeah. Yeah. It's actually funny because like, there's another thing like I always thought of like Spotify, like the main thing with Spotify was like, you know, you played a music, you double clicked on that chunk and you're like, wow, is this actually in the cloud? It's so fast, it feels local. And I always like feel like there's Models kind of.

• 11:54Jacob Effron?

  I guess that's the unifying theme behind the work you do. I'm sure obviously Modal solves a bunch of problems for developers already. And I'm sure you look out as at how this space is going to evolve over the decade and see a ton more problems that need to be solved on the developer side. And so I'm wondering what you're seeing firsthand of like, some of the pain points developers run into when they're, when they're working with some of these models and, and how you think about, you know, how, how the kind of developer stack evolves.

• 12:19Speaker 01

  Yeah, I mean, I think there's like so many different like challenges like everywhere, right? Like dealing with just like CUDA drivers and like productionization and stuff like that. Plus the software stack is like incredibly immature, like, like, you know. You know, a lot of, you know, hugging phase or like, you know, integrating that with towards like, there's just so many like janky ways where you end up having to call one Python script from another Python script in a sub-process because there's no like, you know, unified interface, like, everything around fine tuning is like based around like kind of janky frameworks and like, this is, you know, obviously like that will mature, but it's like, it's a pain today to like build, you know, end to end pipelines using, you know, um, some of these like Gen AI models, uh, which is also why I think like models exciting, uh, like we can help, you know, people do that things. Um, I think,

• 13:04Speaker 01

  you know, the cost is going to go down, like it's going to get easier and easier to do it. But yeah, it is pretty significant right now. It's pretty annoying.

• 13:10Speaker 00

  I'm curious how you kind of see the model, the model layer of the stack evolving over time, um, or the models, like there's kind of one camp, which is like one model to rule them all in the cloud. And then there's like open source for everything and, you know, a bunch of different smaller models. Um, or maybe there's somewhere in between, like, how do you see the, you know, the kind of model layer evolving? Will, will models be commoditized or, um, like, what do you think will happen there?

• 13:38Speaker 01

  Yeah, I mean, like, I think like historically, if I look at like, you know, the models and like open source and, and machine learning research in general, like, I feel like the cutting edge stuff has always been kind of open. You know, there's so many like, you know, tens of thousands of like machine learning researchers, like AI people, like building things and sharing it. Like, to me, like, it feels like what we've seen in the last couple of years, like, has been a little bit like, uh, you know, retraction from that. So like, I do think that's going to revert back. It like, I also think that's because like models, like, you know, so, so like some people like they, they have like privacy concerns, like price or performance concerns, like they're clearly like, they're going to want to run like smaller models. In those cases, like, you know, like, there is not going to be the sort of proprietary models like necessarily, like it could be like smaller versions that's still, that's still like fine tuned or, or if they have

• 14:23Speaker 01

  like more data, they might want to like start with like a open source, you know, model and like fine tune it to like a specific domain. And then you look at like, you know, the types of stuff that's left. It's like, you know, the like OpenAI or whatever, like OpenAI, I feel like. Or, you know, they clearly have the best model today. And I kind of like I mentioned, like, you know, like, open source has always been where I feel like, like, long term, like, the research happens. I, I, I do wonder what's gonna happen with that. Like, I, I, like, cause like, so far, like, you know, they've been able to build up their lead by like, throwing like billions of dollars at like H one hundreds and like, hiring the best engineers in the world.

• 14:58Speaker 01

  But, but I always like, wonder if it's like diminishing returns from that. And like, you know, like, Yann LeCun and other people have been like, making good compelling arguments, but like, maybe like, open source will like, will also take over. Like, I don't know, like, I, I think there's like a point where like, I, I could also see like smarter model architectures, like, you know, being like, you know, it might be like more, much larger gains coming from that than from just like throwing like H one hundreds at the problem. So, I, I don't know, like, I'm, I'm somewhat like, I don't know, maybe like, somewhat naive and maybe like aspirationally, but like, I would love for open source to be like the source of like the best models. I don't know, like fundamentally, like I, like, I do think that, like, if you have an unlimited budget, like, obviously you will be able to train much larger models than anyone else. But I do think like OpenAI, like the delta is gonna definitely shrink over time.

• 15:43Speaker 00

  Totally. Yeah, I think it's fascinating because I do think the delta will shrink over time. And I think a lot of it comes down to how much better the models get with scale. Because if they don't get much better, it's like for the same quality model, the cost is going down a ton for training and inference. But like, if they do get a lot better with scale, then you know, maybe you would have to call a model with more compute in the cloud or one of OpenAI's models. And I think it'll be interesting to see if kind of those returns continue. You mentioned something that I thought was really interesting around the, like, GPUs and people having, you know, thousands of, of GPUs that they can train on or run for inference.

• 16:29Speaker 00

  Like, what have you seen around GPU access and the dynamics in the market today? And how does that affect the startup like you guys that, you know, are running workloads on, on GPUs?

• 16:42Speaker 01

  I mean, I think like some of the headline news is true, right? Like, it's hard to get A100s. And in particular, we've seen like sort of weird dynamics where like, like, you know, normally like in function economics, like most economics, like the more GPUs you want, like the more you want those X, like the lower price you get. It's kind of the opposite in the cloud. Like if you want like a thousand GPUs, like the price actually goes up. Or, or, you know, most people say you have to go like then, like, you know, go to different places and get them. So there's definitely like weirdness. Like, at that being said, like, I kind of feel like it's like a little like overhyped. Like, I feel like people are like obsessed with A100s. For instance, like one of the things like, you know, for instance, we like, I'm pretty happy about is like, that's actually freed up a lot of A100s.

• 17:25Speaker 01

  And A100s are like better price performance, in my opinion, for most use cases. We're also very like inference focused. So that means like we, we don't actually like the biggest, the main GPU we use A10s, which is kind of lower performance, but like per dollar, like very good performance and like more suited towards inference. We see less of a capacity shortage for those, like relatively to get them in a couple of different cloud vendors. We've had to do like a bunch of tricks, though. Like, we've had to like build model in a multi-region, like to support multiple regions, for instance. Like, and part of that is like, we see, like, you know, throughout the day and, you know, like depending on like who's running things in different like regions, like we see, like

• 18:08Speaker 01

  sometimes we have to go to Canada to get a bunch of like GPUs, sometimes we have to go to Europe, sometimes we can get them to use East One. And so there are some tricks like that that we have to do in order to get the GPU capacity we need. But I do think like, you know, I don't know, like my hope is like GPU prices will normalize. I think there will be more capacity. It's a little funky right now and maybe not quite as bad as like you might think reading, you know, like some of the news. But it is a little funky.

• 18:36Jacob Effron?

  Totally. And I mean, I think it makes sense, obviously, given the focus on inference, you guys have a bit more flexibility in the type of hardware you use. You know, I'm curious, obviously, the costs of inference have been more in the news of late and I think are constantly changing. And I'm wondering, like, where you think we are in kind of the optimization of inference costs for like this current set of models? Like, where do you think we end up as more and more folks are working on optimizing this, you know, as a percent of like where what they cost today to run like, you know, Llama or, you know, Stable Diffusion or some of these other models?

• 19:10Speaker 01

  I don't know. I mean, I think like GPUs, like, well, you know, they'll get like faster and faster every year. And I think people like optimize and like squeeze out more and more performance. But I also feel like there's like so much latent demand for a lot of this stuff, like that, like, you know, in the end, like, you know, like, when you make something more efficient, like, does usage go up or go down? Like, you know, or like total spend? Like, I don't know, like, sometimes like lowering the price like unleashes more. So is that like, I feel like, you know, it might have the opposite effect. Like, you might actually drive up the usage.

• 19:38Speaker 00

  I think we're a long way from the efficient frontier. We'll see.

• 19:41Jacob Effron?

  It certainly seems that way.

• 19:43Speaker 00

  This is a bit of an aside, but given your background, I thought it'd be interesting to ask you. You obviously built annoy at Spotify, nearest neighbor vector search. With a lot of the generative AI workloads, like, there's been a lot of attention around the kind of vector DBs and people running nearest neighbor to go get the, you know, for rag pipelines that they're going to go send to OpenAI. Do you have any thoughts on like the latest vector DB markets or the latest vector DB companies? And, you know, like, how you see that layer of the stack evolving?

• 20:19Speaker 01

  I'm incredibly bullish on vectors, and I've been there for a decade. Like, I started dabbling with vectors in like 2008 or something like that, 2009 at Spotify. And like, when I encountered it, I was like, this is obviously the way you should build music recommendations, because it's such a convenient way to think about items or like, you know, things like abstractly, like, you just have this like array of Of floating point numbers and that turns out to be like super convenient, except for searching for nearest neighbors. And then I ended up building my own vector database back then in like C++ and used a bunch of tricks. And like, you know, I talked to some VCs back then, like starting a company, you know, like I just felt like it was like way too early and I'm kind of glad I didn't do.

• 20:58Speaker 01

  And then like suddenly like, you know, 2000, I guess like 2021 or 22, like we went from like, you know, like zero open source, almost like vector databases, like only annoy essentially to like, you know, not suddenly it's like 25 startups doing this.

• 21:14Speaker 01

  I don't know, like I like, I'm still extremely bullish on vectors as a whole. I do think like we're so early. And so like my, like my general thought on this space is just like, I feel like, you know, when there's like a new technology like coming, like, like people's, people's, you know, people's like gut reactions trying to shoe worn it into some old technology. And so like, people look at vectors and they're like, yeah, it's a database. Like we have to shoe worn it into like existing SQL databases. But I don't know, like actually like, like when I look at vectors, like it represents the only way to like, think about like information. And so I kind of suspect that the final incarnation of like vector databases will look nothing like a traditional database. Like it looks something very different. Like, I don't know what the interface looks like. Do you put in like vectors into it? Like, I don't know, like maybe you put in like text and image.

• 21:59Speaker 01

  But then on the other hand, you need to think about like embeddings yourself, like, you know, you need to pick an embedding model. I don't know, but like maybe like the embedding model should like, you know, like do the vector like representation itself. Like, like, I don't know. I think there's so many like different things that we still haven't figured out. So I think that's ahead of us. And so like my feeling is like vector databases today look a lot like databases 10 years ago. I think vector databases in 10 years, like they will look nothing like that.

• 22:26Jacob Effron?

  You know, one thing I'm broadly curious about is, you know, obviously I feel like you've, you've built and been at the forefront of a lot of really interesting infrastructure and kind of improvements to developer experience. And now you have this front row seat to all the stuff that's getting done with Gen AI. Like, how do you think generally these tools are going to impact like developer experience going forward?

• 22:44Speaker 01

  I don't know. Maybe I'm not cold, but like I'm actually not like, you know, convinced like Gen AI is like, you know, like I do think like developer experience will change a lot, but I think it's going to change a lot more from cloud. Like I think we're still like relatively early in like cloud adoption. Especially like developers or like, you know, like when you're writing code today, like you, like most people still do it locally. Uh, which is like kind of funny because like every single other profession is like effectively moved their entire workflow to the cloud, but developers haven't. Uh, so I think that's kind of funny, but I don't know about like Gen AI, like obviously like, I think like, you know, Gen AI can like help us like, you know, do like, you know, tab completion better, like and stuff like that. And I think that's like very valuable, but I like, I think it's like valuable in the sense that like it's not going to like double developer productivity.

• 23:28Speaker 01

  I think when you're like, like developer productivity, like historically over the last, like, you know, a few decades, like we've gotten more like, we've gotten more productive, like, like through like a series of like 10% improvements, right? Like, you know, but, but, but they add up on a logarithmic scale, right? Like if you, if you do like, you know, if, if you have like 50 improvements that are each like 10%. Like something you have, like, I don't know, like, you know, multiple orders of magnitude in total, right? So, so like I, I like that, that's the kind of feeling I see, I get when I look at like Gen AI for developers. It's like, yeah, maybe they'll like make engineers like 10% more productive, maybe 20%, maybe 30%. It's nowhere near like 10x. But like, it's so good, like all these things add up.

• 24:09Speaker 00

  I think it's really interesting and you were talking about kind of the development in the cloud and, um, and what that means for developers going forward. And I think one thing that was awesome was, um, you wrote this, this post, um, years ago, kind of the, the storm in the stratosphere. Um, the reach, how the cloud will be reshuffled and, um, I won't do this justice, but, uh, but kind of the oversimplification that I think of in my mind is like, developers are going to be interacting with services built on top of the cloud, not the cloud themselves. And I think that's really played out over the last kind of couple of years, um, as you know, things like Snowflake and, um, Versal and others have taken off.

• 24:49Speaker 00

  And I think of Modal as one of those for, um, for data teams. Um, so like, I guess, why would you say that we are so early in our kind of cloud adoption and, um, and what is kind of the future of cloud, um, seem like to you?

• 25:04Speaker 01

  Yeah, I, I, I think. Part of it is like kind of what I mentioned, like engineers like still write code as if the cloud doesn't exist. Like we, we deploy code into the cloud, but like a lot of like how we build software like pretend like it's like as if the cloud doesn't exist at all, right? Like, like a lot of the developer workflow doesn't exist in the cloud. Like what particular example is that, you know, like, like when you work with the cloud, like there's often like proprietary things like, let's say you're like, like writing something to run a Lambda, right? Like Lambda doesn't have a, a mocking framework. So, so in order to like run code locally, like you basically have to like mock out Lambda itself. And there's a bunch of frameworks that do that, but fundamentally now like you have like this like big rift between like your like local development workflow and their like cloud development workflow.

• 25:49Speaker 01

  And, and so, but you know, so like I used to think that like, well, what if we take like everything in the cloud and like mock it out locally? Then you can sort of do that. And then like, it's like people do that instead of using RDS, which is like a hosted Postgres and AWS, like people just like run Postgres in a Docker can, you know, whatever. But then I realized at some point like, what if you just like have this ability to like run like ephemeral, like, you know, you know, short-lived objects? Like what if you could just like create a database in the cloud and have it only exist for like a few seconds? And like run all your tests against it and then like tear it down, whatever. Like what if you actually like can bring the cloud into your local development workflow, like sort of, you know, iteration cycle, right? Like, and, and, and, you know, and, and in that case, like, there is no like difference between local, like it's same environment, right?

• 26:32Speaker 01

  And you solve all this like agony of like switching to different environments, deploying things, et cetera. But like, you know, like we're talking about more like obviously like this is like what Model is for data teams, right? Like, so I think like that, that to me is like represents like some way of like how I think about the future of like cloud and how we use it as developers is that like to me, all development will move into the cloud eventually. Model is sort of like how I mentioned that like happened for data teams specifically. And I think you see that across like other domains too, like you look at like how people work with Snowflake or other domains, or, or how other tools are basically like abstracting out, uh, the, the underlying like compute layers and like, you know, offering like a higher level service.

• 27:13Speaker 01

  Like I love AWS, you know, and like it, it, it's tremendous, like the amount of things I can build with AWS, like it's like, it's crazy like how much value it brings to me. But it's also like from a developer experience point of view, like so bad, like it's like just. You know, and it's like they had 15 years to figure this out. Like, I just don't think they're actually fundamentally interested. And like, I actually don't think they like care. Like, it's like, they make so much money just being a compute provider. And that's the sort of like where I like long term see. Like, people used to say like, oh, well, just doing compute is like pure commodity. Like, they have to go up the software like layer and like do more like stuff higher up. But I actually don't think that's true. Like, AWS is like incredibly good margins and just pure compute.

• 27:55Speaker 01

  So like, what I see is like a stable oligopoly of like a few hyperscalers doing incredibly well. You know, offering like compute at scale, which is an incredibly hard problem. And then on top of that, like higher level services that offer like true like developer experience around like, you know, writing code, deploying code, and then maybe do that focusing on more specific use cases and workflows. Because I think that makes it a lot easier to build those cases. Like, AWS like has to build for everyone at the same time, and that's much harder. If you're Vercel, you can focus on a particular workflow. If you're Model 2. So I think that, you know, that's sort of like, I guess, like part of my blog post in a nutshell.

• 28:30Speaker 00

  Yeah, I totally agree. And I think the, you know, the two points that really stick out to me are, one, the cloud business is quite good if you're, you know, just running storage and compute. And then, two, like the AWSs of the world, they have to build for everyone. They're building for data teams in front end and back end and all of those experiences in one, you know, definitely leaves something to be desired. And it is kind of crazy that with all of the advancements in like cloud computing and all this stuff, most development still doesn't happen in the cloud, right? It's like, people don't iterate in the cloud. They build locally and then figure out a way to get it into the cloud, maybe via Docker or via something else.

• 29:10Speaker 01

  Yeah. Which is funny, because if you go to like whatever, like a dentist, like all their workflows have moved to the cloud.

• 29:16Speaker 00

  Yeah.

• 29:18Speaker 01

  Or like, pick any profession today. Like, everything happens in the cloud, except for software engineering today.

• 29:23Speaker 00

  Never thought I'd be behind the, looking back at the dental industry ahead, but so it goes sometimes.

• 29:29Speaker 01

  They're ahead in cloud adoption.

• 29:32Speaker 00

  I guess like, you know, it's interesting. You kind of mentioned the stable oligopoly on the, on the cloud side. And obviously, one thing that can, can, you know, potentially cause a shift in those dynamics is, you know, these AI workloads, right? And I think it's been interesting to see the, you know, the early days of kind of the cloud companies and their strategies, you know, toward, toward kind of getting a lot of that volume. But I'm curious like how you think their strategies evolve over time and like, you know, what role you ultimately see them playing.

• 29:57Speaker 01

  I'm very bullish on the like the big three, maybe four, however you want to count like hyperscalers.

• 30:06Speaker 01

  I just think there's like such enormous like economies of scale and like, you know, benefits of like building hard technology and like running like, you know, capital investment that it takes to build, you know, data centers.

• 30:18Speaker 01

  I mean, that being said, like, there is sort of a weird thing happening right now where like, you know, Nvidia versus, you know, AWS versus GCP, where like, you know, GCP AWS have their own chips and like, and you know, there's some like weird kind of beef going on where like, like, you know, maybe they don't get enough, maybe they don't get enough like Nvidia GPUs and then suddenly like CoreWeave comes from nowhere and like, you know, which seems like kind of incredible to me, like, but like what they pulled off, like suddenly that's like, looks like a pretty sizable, you know, like cloud vendor coming from nowhere. So I don't know, there's definitely some like interesting stuff happening on the GPU stuff. I am very, like overall in the long run, like I am like very bullish on like just like AWS, GCP and Azure.

• 31:02Speaker 01

  And maybe Oracle frankly, actually like Oracle also kind of came back from the dead. I think we actually used Oracle at Model and pretty happy customers. Like it kind of surprised me to say that, but it's actually a pretty good service, Oracle Cloud. So, so, but all in all, like, you know, this, this is a, this isn't a business that favors scale and favors, you know, very high capital investments. It's a tough business. Like it's not going to be a lot of players in this space long term.

• 31:29Speaker 00

  I'd actually be curious to get your perspective on Oracle Cloud. It seems like that is, they have partnered with a lot of kind of AI type companies or companies running GPU type workflows. Like what's been your experience working with them?

• 31:41Speaker 01

  I think they're, I mean, they're great, right? Like, and you know, I like, I don't know under the hood, like maybe they're like, maybe they're bleeding money to like a market share. Like I don't know, but like, you know, like what we see is like great prices. I think they definitely benefit from taking a more neutral stance versus, you know, with Nvidia versus AWS and GCP, which, which are trying to do their own chips, which like obviously like from Nvidia's point of view, like that's not good.

• 32:08Speaker 01

  I think Oracle, you know, benefits from that, like having a more like, you know, being a better terms with Nvidia. Yeah, because like, you know, if you want like H100s or 8100s, like Oracle seems to be, you know, relatively good place to get those.

• 32:23Jacob Effron?

  Yeah, I'm curious, as obviously as the hyperscalers kind of fight to differentiate and win some of this business. I mean, obviously, there's potentially building their own chips. There's certainly close ties with certain, you know, closed source foundation model companies. But how else do you think they end up, you know, trying to differentiate or kind of what are you seeing today around the difference in the offerings?

• 32:41Speaker 01

  I mean, like, there's definitely like nuances, like, like, I, I don't know if there's like, in the grand scheme of things, like, like, at the end of the day, like, and like, I would relatively like, just, just like the problem of like, doing compute at a large scale, my feeling is like, that's like a hard enough problem that like, you almost like need no differentiation, like, on top of that. Like, I am sort of skeptical of the idea that, like, in order to be a successful hyperscaler, you need to have a data warehouse, you need to have like blah, blah, blah, blah, blah. Like, I actually don't know if you need that, like, CoreWee seems to be doing like, okay, like, entering the market without those things.

• 33:17Speaker 01

  But, but like, I think you need a lot of scale, and I think you need a lot of, you know, compute and, you know, like, you definitely need like, you know, some software, you need a storage system like S3, you need a, you know, compute system like EC2, and those are very hard problems. I like, in terms of like how they differentiate, like, long term, my feeling is like, what I want to see is obviously like, I want to see more diversity in terms of GPU vendors. Like, I have a dream that like, I can wake, you know, like, in like a few years, like waking up in a world where like, there's like multiple GPU vendors, you know, like, maybe AWS, like, you know, finally figure out how to, you know, run Infrasound, Trainium with like, good Torch support, TPUs like too.

• 33:57Speaker 01

  I would love it if like AMD has like a few, you know, maybe there's like some like wildcard vendor also like figuring it out. Like, I think, I think the world would benefit from that, because like, I think the world benefits from like cheap compute at large scale. I don't know, like, I've been saying this for like five, 10 years, though, that like Nvidia is going to have much more competition next year, and it doesn't seem to happen. So like, clearly Nvidia is doing a phenomenal job staying ahead of the competitor, competitors.

• 34:25Speaker 01

  But I do think, you know, at some point, I definitely would love to see that, and I think eventually it will happen.

• 34:30Speaker 00

  How opinionated do you find kind of modal users are when it comes to the GPUs or the compute that their workflows are running on? Is it like, okay, if you run it at scale, it's totally fine? Or like, I want to run it on H100 because I have a specific thing about something that's happening?

• 34:47Speaker 01

  I think they're actually fairly unopinionated. And I think that's good. Like, I actually kind of like think we're actually putting a little bit too much of an onus on them to like pick a GPU. Like, I would love to have some sort of automated, like GPU selection mechanism where we just like automatically run people's code on like different GPUs and then kind of, you know, continue to say, like, either you can optimize for like maybe lowest latency or just like lowest cost. And we just like automatically do that. It is sort of annoying, you know, a vector you have to think about when you're deploying code. Should you use A10s? Should you use T4? Or L10? Like, whatever, L4s, P4s, like,

• 35:26Speaker 01

  A40s, A100, H100. You know, there's so many different GPUs. It would be nice if there's sort of more a simple way to select the GPU.

• 35:35Speaker 00

  Totally. And that's like the dream as a developer is you can kind of, you write your code and then you specify kind of the scale that you need and the cost that you're willing to pay. And then you don't have to think about anything else, right?

• 35:47Speaker 01

  Exactly. And I think like, when I think about like CPUs, like, you don't really think about like whether you're like, you know, cloud CPU is like 3.2 gigahertz or like 3.6 gigahertz. And, you know, like, maybe you think a little bit like about like how many cores it has. You might think about how much RAM it has. But like, I feel like, you know, GPU, like CPUs have come to a point where like, you don't really care that much. And GPUs is going to take a long time to get there. But at least like maybe Model could enable, can make it a little bit easier.

• 36:18Speaker 00

  One thing I think is fascinating about your history is obviously, you know, you started to solve, you know, kind of these specific infrastructure problems, then have kind of moved in, you know, just by user demand into this Gen AI space. I'm curious, like, was it immediately, as these models gained popularity, was it like immediately obvious to you, you're like, yeah, like, Model is clearly going to be an amazing place to run this? Or was it like some users that came in initially and started using Model for these use cases? Like, what was that like, early days of realizing like, hey, this actually might be a pretty big chunk of the platform?

• 36:47Speaker 01

  I was kind of skeptical for a long time about like, model deployment. Like, I always felt like model deployment is like the easiest part. Uh, cause, cause like looking back, like, you know, the, the sort of traditional model deployment, like often you have like very complex feature pipelines or like complex like training, you know, requirements and like, and often like when you look at like, you know, traditional like simple models that, that cause, you know, like the, the, the most models that like say in e-commerce company, whatever, like you have like some very complex like feature pipeline and then, but the model itself is usually like fairly small. It's like X boost. Like you just like dump it on S3 and you have some like simple Kubernetes thing that just like loads that model and like it's both.

• 37:29Speaker 01

  And, and so when, when people started asking about like, whoa, can I do like model deployment on modal? Like I was like skeptical. I'm like, oh, I don't know. Like it feels like a relatively undifferentiated industry. Uh, but then we started realizing like, there's actually like a couple of things that have completely changed. Like first of all, like for a lot of these models, like you need GPUs. Uh, second of all is like you need serverless because it's like very expensive to run these models at scale. And then like, and then like, there's actually a lot less of the sort of production pipeline stuff around like feature pipelines, training, like all of that stuff. We want to do those eventually too, like, like definitely like want to like own the whole ecosystem. But a lot of the like, what happened with like GNI models is like, it's, it's just a model. It's just like code like comes in a box.

• 38:13Speaker 01

  It already has all the weights. Like all you need to, but, but however, these models are very large and they need to run on a GPU. And that's a hard problem. And, and so, but it's like a compute only problem. It's like, you know, it's like, whereas like previously machine learning tend to be like very complicated around like state management. But like stable diffusion is like, it's just like a pure compute problem. Uh, and so, so we realized it's actually quite well suited for, for that reason. But, but that took me a little bit of time to realize. I, I, I think, I think, you know, initially like I was thinking a lot about like modal as like sort of a Kubernetes replacement, which I still think, you know, long-term like that's what I want to build. But it turns out like the first killer use case was really serverless GPU inference.

• 38:57Jacob Effron?

  Well, one thing I definitely want to make sure to get your thoughts on too is I feel like you obviously, you know, earlier in your career at Spotify, worked a lot on like music recommendations and some really cool like early, you know, use cases in the space. I met, you know, your former employer has done a lot of cool things in generative AI these days. And I'm curious, like, if you were still there, what are some of the things you'd be thinking about or potentially building there?

• 39:21Speaker 01

  I mean, I think the craziest thing would be to just like generate music.

• 39:25Speaker 01

  You know, and I was actually like, I've been thinking about that. Like, I actually feel like music is like, kind of more of a pure art form in the sense that there's like no metadata almost. Like when you look at like paintings, like it's often like, oh, like you need to know the story of the artist, whatever, to truly appreciate it. But like, music is like, it's just like waveforms that goes into your brain and then it's either good or bad. So like, to me, that's like the perfect setting for like gen AI. Like, I actually feel like, you know, if I had to pick an art form, where are we going to see like the greatest prevalence of like, you know, AI-generated content? Like, I would actually pick music, like, and I think we're going to see it in the next couple of years more and more. I don't, I don't think we're going to see like purely AI-generated initially. I think it's going to be more like, you know,

• 40:07Speaker 01

  like good producers and good artists are going to figure out how to use AI technologies for, to the benefit for making music. But, you know, so, but like, you know, but also if I was Spotify, like I would maybe start to think about like, what we're paying a lot of money is in royalty, like in royalties, like can we, you know, can we actually substitute that with like, automatically generated music? I don't know. Like, it's kind of a crazy idea because I also feel like music is kind of like a social thing. Like, you want to know that your friends are listening to the same track so you can talk about, you know, there's a little bit of that aspect that I think is like would be lost, that I think is hard to replace. But I don't know, like, I'm kind of excited about the prospect of like AI-generated music, actually.

• 40:50Jacob Effron?

  I also think if Spotify were to get disrupted, that generative AI music is probably one of the biggest vectors that someone could go down. Like, I could imagine there being some sort of, I don't know, gen AI music social network where, you know, you, me and Jacob are creating a song together and then we can share that with our friends and each kind of riff on it and add things to it.

• 41:13Speaker 00

  I really think that will exist and in that world, it's like, I'm sure there'll still be a market for the, you know, Taylor Swifts of the world. But I do think a lot of kind of creation moving to people, individuals and consumers could be an interesting trend.

• 41:29Speaker 01

  Totally. And it's like funny, like, like, you know, like music tends to be also like, kind of like the first carrier of like new technology in many cases. Like, I feel like Steve Jobs has talked about that. You go back to like craft work that they talked about that, like, you know, music is like carrier of new ideas. Like, it's often like the first place, I guess, music and porn, like, but music tends to be like one of the first places where like new technologies really come to life. With the cloud, like, I think it happened pretty early with Spotify.

• 41:57Speaker 00

  Yeah. And all these people creating, creating their own generative music now have easy infrastructure to do it with, right? So all the easier to do. Awesome. Well, we'd love to move to a quick fire round, if that is, if that is cool. What is one thing that is overhyped in AI and one thing that is underhyped in AI?

• 42:14Speaker 01

  Overhyped? I don't know. Like, I actually kind of, I don't know if it's overhyped. Like, I feel like transformers. Like, I kind of want them to go away. Cause like, I don't, it feels like weird to me. Like, I feel like we're like, you know, confusing, like, we want to learn like a model of the world and like encode that as like facts with like the language model itself, which seems to be like more like a decode, like it's want to like read things. I don't know. Like, to me, like, maybe I'm also like annoyed because like I stopped following like machine learning research, like, you know, around the same time the transformer paper came out. So if like people moved away from transformers, I would have a chance to actually like start over and like learn everything. But, but like, I would love to like, you know, like, I feel like there's going to be like a paradigm shift at some point, maybe, hopefully.

• 42:58Speaker 00

  Yeah. Aren't transformers kind of good for business though, with their relatively high inference costs?

• 43:03Speaker 01

  I guess, yeah. I don't know. I don't know. I mean, I think whatever we're going to find probably will also have high inference costs, tends to be that way.

• 43:10Speaker 00

  It is very surprising to me that there would be one architecture that is so good at so much. Like, I feel like it will, people will discover new types of architectures that will be kind of good or the next iteration over time.

• 43:23Speaker 01

  Yeah, I think so too.

• 43:25Speaker 00

  All right, second one. What is one thing you wish you knew when starting Modal that you know now?

• 43:32Speaker 01

  Okay, that's a good question. I think like probably like my biggest mistake the first year was like, I thought it would be easier to go to big companies and say, hey, you're using Kubernetes and you have this like internal data platform. Actually, you should rip that out and replace it with Modal. Like it turns out it's much harder to tell people to to switch technologies than it is to people to adopt technologies from scratch. And I think that was like a realization we made a year or two in that made it so much easier to think about, okay, like where should we start to sell?

• 44:02Speaker 00

  Anything you've learned about like, you know, selling to AI people, you know, within these companies that you feel like you've gotten better at, you know, since since since starting?

• 44:11Speaker 01

  I think one thing I see a lot of like enterprise companies is that there's often like kind of a weird split. Not weird, actually, it's going to make sense, but like there's like a complex split between like platform teams and like people actually doing the AI. And they often like say very different things. Like you go to the AI team, they're like, yeah, like our platform is like crap. It's so hard to iterate. Like it's like, like I can't get anything into production. And then you go to the platform team, they're like, yeah, like we have like a great system we built internally. Like it works. Like, you know, everyone can ship things in production. It's really easy. You just need to like define this YAML stuff and then, you know, run these five commands and then go to get, you know. So there's like a big rift and like you kind of need to sell to both of those people at the same time, I think, a little bit in order to do.

• 44:55Speaker 01

  I'm like slowly learning about enterprise sales. I'm not an expert by any means, but like, you know, those are some of the challenges we've seen with like enterprise, like selling to, you know, selling, selling to to tech teams.

• 45:06Speaker 00

  Totally makes sense. And then to to close out, where can people go to, you know, follow your progress with Modal or learn more about what you're building?

• 45:16Speaker 01

  Modal.com is probably the best place. So M O D A L dot com. I'm also like somewhat active on Twitter. I tend to post a lot. So it's just like Bernhard Sun. Like my last name is my handle. You can always like follow me like posting stuff. But yeah, Modal, M O D A L dot com is probably the best place to to get started with Modal and follow our progress.

• 45:38Speaker 00

  Awesome. Well, Eric, thank you so much for joining us on unsupervised learning. This was really fun and always fun to have someone in the Redpoint family on on the podcast. And I really appreciate you taking the time.

• 45:49Speaker 01

  Awesome. It's great being here. Thank you.

• 45:52Speaker 00

  Well, I thought it was super fun to have Eric on, on the podcast. He's such an interesting dude in that he has, he can be down in the weeds and writing code, but then also zoom out to what's happening at the, you know, in the markets and cloud and, um, and AI and all that stuff. And so it's cool to get his perspective on, you know, everything from the developer experience up to like, who is going to win in generative AI?

• 46:17Jacob Effron?

  Yeah, he is, he is so smart. Um, what a, I'm glad, I'm glad he's on our side, uh, in, in the portfolio.

• 46:24Speaker 00

  Oh my God, I was so much that I get to learn from him and that everyone can. So, um, yeah, I, I agree.

• 46:31Jacob Effron?

  Totally. And I love how, I, I love that this is like, um, well, two things. I love one that this is like a, you know, a byproduct of, of a solution that he was building for non-generative AI reasons at all. And I love at the end when he was like, I still want to take out Kubernetes. Like, you know, it's like even, he's going to build a massive business doing this, but like, I still also like down the line, I'm going to, going to do the original goal as well. And like, this is kind of a, a step on the way, which is really funny.

• 46:53Speaker 00

  Yeah, I love the example that he gave around, um, Spotify and how when you're, you know, playing a song on Spotify, it's from the cloud, but it feels like it's local. And, uh, in some ways there's, there's some parallels to modal where it is in the cloud, but it's so fast that it feels like it's local and.

• 47:10Jacob Effron?

  Yeah, there's not many times I hear that like software, uh, development is behind dentistry, uh,

• 47:15Speaker 00

  I know.

• 47:16Jacob Effron?

  any sort of technology, but I think it was, uh, yeah, I think there's clear reasons why. And I feel like, um, it just seems like such a seamless experience using modal. Um, it's, it's a really cool, uh, cool tool. I don't know if I tell you, know, it's funny. He was talking about the, uh, the, the music stuff. Um, and I feel like people would take offense with the idea that like the context behind the music does not matter at all. And it is purely like the, uh, the output. I feel like, you know, everyone knowing Taylor Swift's latest breakup just makes that Taylor Swift song like hit so much harder, but.

• 47:45Speaker 00

  That is true. You know, you know that it's about Jake Gyllenhaal and then you're like, oh, we are never ever ever getting back together. Rashad, please cut this.

• 47:56Jacob Effron?

  No, that has to stay in a world of AI generated music. We never get that because you'd like just use someone else's voice or something. And we never have the pleasure of your, uh, your karaoke there.

• 48:05Speaker 00

  The the story, it does it does matter a lot, though. I don't know. I think about like some of the the great songs that that I really like. It seems like the story matters a little bit, but maybe not. I I think there is. And it's also one thing that was was cool was just to hear about all the different types of applications that people are building on modal where it's like, you know, Substack using Whisper for their podcast stuff and running that on modal, but also people doing protein folding and just the wide range of applications.

• 48:35Jacob Effron?

  Don't forget Ramp, you know, Red Point, Red Point Porco. Always have to shout out them.

• 48:39Speaker 00

  There you go. There you go. Well, you're also a healthcare guy. I thought you might like the protein folding.

• 48:44Jacob Effron?

  I do. I I love Ramp. I love protein folding. It's all. It's all up the alley. I mean, I I do. This is like, you know, clearly A plus company like, how did this come about? Like, how did you, you know, how do we we get the chance to work with these folks?

• 48:57Speaker 00

  I mean, it was. It was like a year long process where we had met Eric and then we tried modal and then we were trying to convince him to raise in the fall, then he kept pushing us off and then a year later, the timing was right, but it was a it was a long, a long, slow burn.

• 49:17Jacob Effron?

  I assume you sung him that Taylor Swift song and that was what that was what pushed him over the edge.

• 49:21Speaker 00

  It was interesting. You you were asking about the the process and how smart Eric is. There were two references that stuck out to me when we were, we did a bunch of references on him and one was like, dude, if you have the opportunity to write the check, just do it. It was someone that had worked with him at Spotify. And then there was another one that was like, Eric is actually a hyper genius, like one of the smartest people that I've ever met. So I

• 49:46Jacob Effron?

  It's clear. You could tell even from just the stuff. Like, I love that he casually built one of the first, you know, vector data, like, like

• 49:51Speaker 00

  I know. It was like before, before they were cool, too.

• 49:55Jacob Effron?

  I feel like, you know, there's probably 30 companies that could have been built, you know, at various times with things that he's done. And it's awesome that this is kind of where he decided to focus.

• 50:02Speaker 00

  I was surprised at how much he was raving about Oracle's cloud experience. You know, we we knew obviously that they were on Oracle and that a bunch of companies were using them for GPUs and to run AI workflows, but I thought it was kind of interesting to hear about. That he had so much praise for Oracle's cloud product.

• 50:22Jacob Effron?

  Yeah, and I just love that. Like, you know, I do think like the like transformer architectures are objectively good for his business, given the high inference costs. But I love that he's like, I wish we just move on to we find some new architecture. I think he's just so clearly loves technology and is. Yeah, he's fantastic. And I think it was a really interesting time to have him on, right? Obviously, there's been, you know, a bunch of raises and kind of this like model inference space. I mean, I think most notably together, which, you know, raised around from Kleiner and then, you know, it was rumored to have raised more money from Emergence. And, you know, we've, we've seen a whole host of other folks like replicate and any scale and fireworks.

• 51:01Jacob Effron?

  And I think it's clearly like the space is clearly incredibly interesting for investors right now. And, you know, I think there's, there's obviously, you know, it will the types of models that get run, the optimizations that get run on inference. But then also, I think what makes modal so unique is, is the ability to kind of be able to, you know, actually do far more than just bring your model and run inference. And I think it's just such a complete developer tool. It's part of what makes it so interesting.

• 51:27Speaker 00

  Totally. Yeah, I think that's, that was a big reason why we got so excited about modal was that they can do inference. It's great for that. But also, there's an incredible developer experience to do all of the things that a data team would want to do. And Eric touched on that in the episode of like, you know, if you just want to run to deploy a model, there's other ways that you can do that. But if you want to, you know, do some data transformation or do some customization and then do something else and then run that model at infinite scale without managing any infrastructure yourself, then model is a pretty good platform to do that kind of end to end. And I think that's from an investor perspective, something that that we got really excited

• 52:07Speaker 00

  about because you can own, you know, there's model hosting, but I think there's an opportunity if you own the whole pipeline to layer on a bunch of higher margin businesses and product lines over time. And I think that's one of the cool things about, about modal was that they are end to end.

• 52:25Jacob Effron?

  But it's such a great wedge, right? Like, everybody needs some sort of access to compute. Nobody wants to run this themselves. I mean, I think, you know, what we see with these AI applications that launch, it's like, you have no idea if, like, anyone's gonna use it, or like, the entire world's gonna use it. And the pace at which these applications can scale, you need someone that's flexible on the backend with compute.

• 52:44Speaker 00

  Yeah, I mean, that's what's sweet about Modal is like, you can go build your AI app and not worry about any infrastructure. If it takes off, then you're totally fine, right? If you're the next Lenza or whatever. Or if you're running at large scale already, like a Bramper, or Substack or whatever. I think that's, I don't know, like, the engineer in me gets really excited about that. And we actually, we ran a bunch of stuff on Modal before we invested. Like, almost a year before we invested, we built some flows to scrape Packer News and do a bunch of things. And it's like, it's so nice to just write Python and not have it, not have to worry about any infrastructure. It just runs like on a keyboard.

• 53:24Jacob Effron?

  That was awesome.

• 53:25Speaker 00

  Yeah, it was, it was pretty cool.

• 53:28Jacob Effron?

  I feel like your signature line is, you know, usually, I feel like every pitch we're on, you're like, "I don't write code anymore, I write emails." But then, you know, you were, you were back to it then.

• 53:35Speaker 00

  I know. I had to dust off the old keyboard and write some Python.