#229 Mitesh Agrawal: Why Lambda Labs' AI Cloud Is a Game-Changer for Developers

This episode is sponsored by Netsuite by Oracle, the number one cloud financial system, streamlining accounting, financial management, inventory, HR, and more. NetSuite is offering a one-of-a-kind flexible financing program. Head to https://netsuite.com/EYEONAI to know more. In this episode of the Eye on AI podcast, we dive into the transformative world of AI compute infrastructure with Mitesh Agrawal, Head of Cloud/COO at Lambda Mitesh takes us on a journey from Lambda Labs' early days as a style transfer app to its rise as a leader in providing scalable, deep learning infrastructure. Learn how Lambda Labs is reshaping AI compute by delivering cutting-edge GPU solutions and accessible cloud platforms tailored for developers, researchers, and enterprises alike. Throughout the episode, Mitesh unpacks Lambda Labs' unique approach to optimizing AI infrastructure—from reducing costs with transparent pricing to tackling the global GPU shortage through innovative supply chain strategies. He explains how the company supports deep learning workloads, including training and inference, and why their AI cloud is a game-changer for scaling next-gen applications. We also explore the broader landscape of AI, touching on the future of AI compute, the role of reasoning and video models, and the potential for localized data centers to meet the growing demand for low-latency solutions. Mitesh shares his vision for a world where AI applications, powered by Lambda Labs, drive innovation across industries. Tune in to discover how Lambda Labs is democratizing access to deep learning compute and paving the way for the future of AI infrastructure. Don't forget to like, subscribe, and hit the notification bell to stay updated on the latest in AI, deep learning, and transformative tech! Stay Updated: Craig Smith Twitter: https://twitter.com/craigss Eye on A. I. Twitter: https://twitter.com/EyeOn_AI (00:00) Introduction and Lambda Labs' Mission (01:37) Origins: From DreamScope to AI Compute Infrastructure (04:10) Pivoting to Deep Learning Infrastructure (06:23) Building Lambda Cloud: An AI-Focused Cloud Platform (09:16) Transparent Pricing vs. Hyperscalers (12:52) Managing GPU Supply and Demand (16:34) Evolution of AI Workloads: Training vs. Inference (20:02) Why Lambda Labs Sticks with NVIDIA GPUs (24:21) The Future of AI Compute: Localized Data Centers (28:30) Global Accessibility and Regulatory Challenges (32:13) China's AI Development and GPU Restrictions (39:50) Scaling Lambda Labs: Data Centers and Growth (45:22) Advancing AI Models and Video Generation (50:24) Optimism for AI's Future (53:48) How to Access Lambda Cloud

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  I think it's important that people stay very optimistic about this technology. Of course, we have to have, you know, controls around, a little bit of controls around how people, it doesn't go in the wrong side of things. I, I want to encourage people to, to be in the know about, about this. Underlying it all, like of course, we want to build a big, successful, long-term business. We really care about that. It's, it's kind of our baby to do that. But underlying of it all, we really want to be one of the ones that provide a lot of compute and ease of compute to people that are building the next step of applications for this world. And we really focus on, on the deep learning aspect of it. We spend a lot of time building for it and, and for a lot of listeners, again, I would encourage to try Lander Cloud if you're building any deep learning application.

• 0:42Craig S. Smith?

  What does the future hold for business? Ask nine experts and get 10 answers. Bull market, bear market, rates rising or falling, inflation going up or down? Can somebody please invent a crystal ball? Until then, over 40,000 enterprises have future-proofed their business with NetSuite by Oracle, the number one cloud ERP, bringing accounting, financial management, inventory, HR into one fluid platform. With one unified business management suite,

• 1:22Craig S. Smith?

  there's one source of truth, giving you the visibility and control you need to make quick decisions. With real-time insights and forecasting, you're peering into the future with actionable data. If I were a larger organization, this is the product I'd use. Whether your company is earning millions or even hundreds of millions, NetSuite helps you respond to immediate challenges and seize your biggest opportunities. Speaking of opportunities, download the CFO's Guide to AI and Machine Learning at netsuite.com/ionai. That's NetSuite.

• 2:07Craig S. Smith?

  netsuite.com/eyeonai E Y E O N A I all run together to get the CFO's guide to AI and machine learning. The guide is free to you at netsuite.com/eyeonai. netsuite.com/eyeonai. Why don't you start by introducing yourself and how you got to Lambda into this business?

• 2:42Speaker 00

  Very happy to and thanks for having me here today, Craig. Look, I think the best place to start is always with the beginning of a company and for Lambda. You know, I think the best thing was we all were engineers and in machine learning fields. And you know, when Lambda was initially started off as a face recognition API company. So basically, we had an iPhone app that used to take your photos and convert it into a Van Gogh style painting or a Picasso something, it was called a transfer style learning back then. And we used to have our processing done on the cloud.

• 3:25Speaker 00

  And it used to cost a lot of money. We were never profitable doing that business. What we realized at that point of time was that actually we could purchase the hardware that we were running it on, the GPUs, and we could deploy it and we could install the, what we now call Lambda stack, but at that point of time was just what is called Debian repository, just a collection of all the libraries like CUDA, CUDA and PyTorch, TensorFlow languages on it and make it super easy to run. And it immediately like took out all of our kind of spend on the cloud. So we saw that there was a gap in the market where, you know, you could buy a Dell, HP, whatever hardware, but really like to make it work for machine learning, you have to install this software stack and really support it.

• 4:10Speaker 00

  And that's kind of how Lambda got started. And for me, I think, you know, kind of an interesting journey, you know, in being in Bay Area and Stephen, who is the founder of the company and machine learning researcher himself. You know, we were just introduced and we're talking and he was like, hey, there is this opportunity. We are pivoting into infrastructure.

• 4:33Speaker 00

  I I I was from a background of chemical engineering and and have been working in quantitative finance. But was trying like starting to see early inklings of kind of at that point of time, you know, it was just like advanced statistics, basically, like just prediction prediction based and and things like those was trying to understand these models. And I was like, hey, this is a good way for me to foray into this world. I really understand the computational layer. And, uh, you know, I joined in as in a very, very early employee, uh, number of fifth at at Lambda and pretty much as as as as the saying goes, did everything that the company needed to be was was an engineer, was a support person, uh, you know, was an

• 5:15Speaker 00

  a HR person, was an office manager, was a salesperson, you know, everyone did everything. Uh, and and that's how Lambda kind of grew, uh, really from both my perspective and this external perspective is like we we really branded ourselves into this infrastructure world, but only very much focused on what we call deep learning applications, right? Really like making sure that at that point of time, the the state of the art models were convolutional neural nets, the ResNets, all of those models really making it work for that. And then over time, you know, the company's mission was really focused on bringing compute as quickly as possible to the machine learning community. And we realized we had an opportunity, uh, in, you know, twenty twenty, twenty twenty one with, uh, cloud starting to grow.

• 6:01Speaker 00

  But what we noticed was the hyperskillers were still and everyone was in early innings, actually, like and no one really foresaw ChatGPT maybe other than OpenAI. Uh, and even they would probably say that they didn't foresee the the enormous success of it. Uh, but we we realized that the compute will be very critical category and compute design for AI will be very uh, will be a very critical category and and we. Kind of stepped into that where it's like instead of just selling infrastructure to be deployed in an end person's environment, whether that's through a desktop or laptop at your office or through a cluster in a data center. We said, why not build, we, we, we are experts at building clusters. We're expert at the software stack about it.

• 6:42Speaker 00

  Let's build it in our data centers and give people remote access, AKA like an AWS or GCP like service, but for deep learning. And really optimize on kind of the networking stack, the software stack, the speed at which you could do the model. And that's kind of how Lambda got the first like really like foot down into the space. Uh, and then, you know, we saw the growth curve, uh, we got attracted to the growth curve and, and really then double down on building this kind of AI cloud and, and cloud for developers and researchers and enterprises. Again, keeping to a theme of only focusing on, on, on deep learning applications, but really building around that. So, so that's been the last journey of eight to nine years for, for Lambda.

• 7:22Craig S. Smith?

  I'm just curious, this, uh, style transfer app that you guys have, what was its name?

• 7:29Speaker 00

  Yeah, it was, it was called Dreamscope. Uh, so you know, it was like, look, I mean, of course, uh, we look at it with the, uh, with the very like loving eyes and it was like the mid journey before the mid journey, right? You know, I think mid journey is a brilliant, brilliant application that has been developed right now. And, and, and, and we, we, we did this back in 2016, 2017. Uh, and, and really it got a niche following in the arts community because you can imagine, right? It just, it, it wasn't like a generative app, right? You couldn't type something and it would generate it. But what it would do is take an existing piece of, whether it's an image or a series of images and then convert it into any, any piece of art that you'd want. And we really focused on

• 8:11Speaker 00

  kind of the artistic, like the noir lifestyle, Picasso, like all, all the artistic styles that people do. And, and, and it got over a million downloads within, within, you know, a few months of launch. Uh, we, we had a package called Paid package call at nine ninety nine and we had a lot of paid users at one point touching, you know, well over hundred thousand ARR, which at that point in time, now it's it's very small compared to what the AI AI companies do, but at that point of time was was a new and radical kind of way of approaching the consumer marketplace. And, you know, it is is maybe, but like, you know, we had to, first of all, a lot better models came out, people who developed their own models. And then second, because we focus so much on the AI learning infrastructure,

• 8:56Speaker 00

  we realize that focus is key to grow in this market. And so we, as a company and as an expertise area, we really pivoted into building out infrastructure for deep learning engineers, because we ourselves were users of those.

• 9:08Craig S. Smith?

  Yeah, the reason I ask is I was fascinated by style transfer back in two thousand sixteen, two thousand seventeen. And I did use a few apps. I use Picasso with the K P K A S. And there was another one, I can't remember.

• 9:26Speaker 00

  Yeah, even I'm blanking on the name, but it was a very popular Russian app. Yeah, I'm blanking on it. But yeah, like I know too, I've used, yeah, yeah, exactly. Style Transfer were the first really consumer applications of.

• 9:41Craig S. Smith?

  Yeah, I'm actually surprised that it didn't take off or become more popular. And then

• 9:52Craig S. Smith?

  there's another company that's doing something kind of similar to Lambda, Paper Space. I got to know those guys years ago, and I can't remember what they were doing, but it's the same kind of story. They were doing something in that you guys were working on style transfer, and they they saw this opportunity for for

• 10:18Craig S. Smith?

  G P U. And they moved into that space. Why are you cheaper than a hyperscaler?

• 10:28Speaker 00

  Yeah, I think that's a fair question that you can imagine we get asked across the board, whether it's investors or the media or just users. Look, I think there is a few things that go into it. First and foremost, let me be honest, there's not like a huge magic wand here that is somehow making like, you know, Lambda has like a way of producing the cost structure or because we get GPUs that cheap or nothing. Nothing like that. So there is a few things that go into it. One is just Lambda's philosophy. I'll be very frank with you and with everyone listening and we have been with our investors too. We really wanna grow the access to compute for all of both, not just like incoming developers and students and things like those,

• 11:12Speaker 00

  but across even enterprises before the advent of this kind of AI cloud companies, including Lambda. The GPU marketplace was a little hazy in terms of pricing. Like, you know, you had your website, even today, like if you go to AWS and you look at AWS prices on the website, it's like for Hopper, it's like five dollars or six dollars or eight dollars, right? Like, no one, almost no one is paying for that and that's been the reality for a long time. But you had this different pricing strategies that hyperscalers did. Lambda went in with a simple task of transparent pricing, straight pricing to the market that we believe allowed us to both grow and capture enough margins to continue to run the business and be successful.

• 11:57Speaker 00

  So that was just, that was one, like it's just philosophy around pricing. Two is this, and this is the technical reason, one of the technical reasons, again, not a magic wand, but if you think about hyperscalers, they have to support many, many hundreds or thousands of use cases, right? As they go. So they have microservices and supporting infrastructure for those services that they have to build. So when they deploy a GPU cluster, they don't just think about the GPUs, the CPUs around it, the memory, the storage, the networking. They have to think about a hundred more myriad different things, how to get interconnected to all the data center regions, how to have all the services. So what that results into is slightly an additional both a cost structure that they wanna recoup,

• 12:42Speaker 00

  but also a little bit. Time where like, you know, if Lambda were to get the GPUs today, and let's say, you know, hyperscale were to get the GPUs today, we would get that deployed way faster. Not again, because, not because we have more people or anything like that. It's just because we don't have to think and optimize for that. We have to optimize for deep learning. So the focus there helps. And as such, if you have less time to market, so you'll spend less on the operational costs, you have to don't think about the microservices and things like those. So it really allows you to price way more competitively. And the last thing I will say is this. Look, most, all, almost all, all hyperscalers are, are public companies. They have certain expectations to meet on what their growth margins are, what their operating margin structures are, and things like those.

• 13:25Speaker 00

  For Lambda, I think, being a private company here and, and really optimizing for growth allows us to take lesser margins than, than hyperscalers. Again, as long as the business is sustainable, you know, we, we go through that, right? So, so there is this multiple things that contribute to it. You know, including, as I said, the first thing that really contributed to it was like, us ourselves as a philosophy, we wanted to provide as much compute as possible. And, and we were advised, you know, like by, by fellow deep learning researchers and engineers who help us at the company and, and who are advisors to us, saying that, look, the best way you could grow is by making sure that everyone understands and learns about deep learning and that there is an accessible way to use that compute. So, kind of makes sense from, from that perspective.

• 14:09Craig S. Smith?

  Yeah. And then for growth, as you grow, do you, as demand grows, you add GPUs? Or do you add GPUs and then sell the demand and sell the capacity? How, how does that work? And do you have any trouble getting GPUs since there's famously a shortage of GPUs globally?

• 14:33Speaker 00

  Yeah. Two, two, two very kind of circular situations or catch 22 situations, as you mentioned, right? I think, I'll be frank, I think the last two years have been, we've been, we've been blessed in the sense that on the demand curve, the demand curve has grown so fast that no matter how much supply you're providing to the market, it was above, above the supply curve, right? The demand curve was above supply curve. So, so to your point, whether, you know, How many ever GPUs we kind of order and deployed is a function of less of of the what the demand curve of the market is. So to your point, whether we saw demand and then deployed, now it was more around, it was more a function of how much capital do we have to purchase the

• 15:16Speaker 00

  GPUs, how much data center do we have, space and power do we have to to deploy these GPUs, uh, and and, you know, how much can we, to your point, how much can we get allocation uh, of the GPU. So it was more a function of those, uh, you can think of like commercial factors rather than really the demand curve. The demand curve was all, has always been there. So from Lambda's perspective, we pretty much always, if if someone is doing like a reserve, what is called a reserve contract, so like in committing for a year, two year, three year, uh, for compute, we kind of always, always sold ahead of the curve. Like, so people have come to us and say, hey, I want a thousand GPU, I want a five thousand GPUs. And you say, okay, great, you know, give us three months and we'll get it deployed for you, or things like those. So that's on the reserve side. Now, where Lambda has kind

• 16:00Speaker 00

  of differentiated ourselves is a a little bit more on the on-demand compute, where, to your point, we deploy compute, um, without having an associated reserve or committed contract to it. We, because we want to, we want to create a marketplace where their people can come to Lambda, see a GPU available and spin it up and and and use it the moment they want it, for how much ever time they want it, and then spin it down. Uh, we want to create that, so we encourage that marketplace. So we basically assign a certain percentage of the GPUs that we get for that market. Now, we have, and again, as I said, both lucky and and and as a business, really, uh, you know, I I guess having that kind of visibility

• 16:41Speaker 00

  into the pipeline that GPUs as a demand curve has always stayed a supply curve. So it has allowed us to, even when we have booked it for on-demand, those are pretty much immediately sold out. Like, so we reach high levels of utilization within a few days to weeks of launching, uh, those GPUs, right? So, so, so, so far, it's been always ahead of the curve. In terms of securing GPUs, I think for Lambda, you know, I think, uh, you know, we have been, you know, I think Nvidia has been kind of like a very partner-friendly, has created a partner-friendly ecosystem. So like, you know, if if you think about it, how much GPUs Microsoft would like or want. Probably Nvidia could park all its GPUs at Microsoft, you know, maybe, I think.

• 17:22Speaker 00

  But no, Nvidia has been very, I think, much more, I guess, in a way democratic. And I think in the sense of like how they have provided GPUs. And and and part of it is just that they follow the usual traditional supply chain principle of first in, first out. So in the sense that you mentioned, how do we get allocations, you know? Again, not a huge, no matter kind of what you see in the media, it's not like a, like a, oh, like, you know, somehow we go to Nvidia and say like, hey, we are your favorite people and can you please give us GPUs? It's more around the fact that like, look, we have this demand curve. We are placing a purchase order six months, nine months ahead of time. And and we are committing to this purchase.

• 18:01Speaker 00

  And and and as such, you get into the queue of where the GPUs are going and you you get those allocations. And so so you have to do the supply chain planning very much so to to get the allocation, you know? So I think over time, you know, as Hopper, when they first came out, it was hard to get for everyone in the amount that you wanted to get, because again, demand was about supply. But, you know, everyone got whatever they placed their queue into the, how much they placed their order and then queue in. And accordingly, everyone got the ratios as Nvidia also ramped up its production system. Now it's like very, very much accessible to get a Hopper GPU in the H, H generation, H one hundred or H two hundred. But now the same story is unfolding for the Blackwell generation.

• 18:46Speaker 00

  You know, you you see it in the media report, Nvidia constantly saying they're sold out and they truly are because and all of us have placed, you can imagine hyperscalers and us, you know, all of us have placed orders for Blackwell GPUs now almost, you know, probably like, you know, twelve months to six months previous to today's date for Blackwells that are going to get delivered later this year, early next year, right? So so you you are in a queue to get those. But like, let's say if you were to start a cloud company today or or go out and place an order today. I mean, you're not, you're potentially waiting for like eight, nine months to get those of Blackwell GPUs, such as the demand for for for those Nvidia Blackwell GPUs, right? So so, yeah, that's kind of how the allocation curve goes. So you have to take certain risks to to get the allocation.

• 19:30Speaker 00

  And you have to constantly assess what your sales pipeline looks like, what the market demand curve looks like. But so far, it's always you, you kind of plan your GPU deployments well ahead of time. By both having visibility on the on the demand curve and also assuming the demand curve continues to stay above supplier, which we really believe in.

• 19:51Craig S. Smith?

  Yeah, a couple of questions. Your the workloads that you're running, are they primarily training workloads?

• 20:02Speaker 00

  So that's been an evolution. I think if you had asked me that question, like, let's say, twelve months ago, I'd have very strongly said, yes, like, I think a lot of the new spend dollars in infrastructure, especially in the deep learning space, is going towards primarily training. And I I include fine tuning as part of the the training bucket in this, especially since open source has really kind of allowed people to fine tune a lot more now with the models being of very high quality, right? So but but even like, you know, until very recently, I think training was the new major spend because for training, you need a huge chunk of compute all at once, all interconnected in a way, and so on. And inference is an

• 20:43Speaker 00

  application based usage, right? You know, if you are a consumer like company that is providing some application, if it takes out great, you need a lot of GPUs. If you're just starting now, you need you can start up with a few GPUs. And so so until very recently, training was the was the majority spend and and and that reflected in Lambda's own deployments and and in sort of Lambda's own usage as well. I I do want to very specifically answer generically. The way Lambda Cloud is built, it can be used for both training and inference. That's a that's a result of both what Nvidia has done with its high end GPUs where they have made it extremely good both for training and inference. So if you look at Hoppers, if you look at Blackwells, they are extremely effective training and inference GPUs. You

• 21:28Speaker 00

  you really can't say that, oh, it's really great for inference, but it sucks for training. It's really not true. They're absolutely top of the line for both those applications. So first of all, that that. Architecture and infrastructure itself helps both training and inference. And then the way Lambda designed its data center too, it's like, of course, we designed more for training in the sense that we try to get as many GPUs all interconnected together. But the same setup can be used for inference from that perspective. And so far, inference is not bottlenecked by some of the factors that you think that we bottlenecked by, such as latency, right? If you're a user in New York and your GPU is in San Francisco and you're using ChatGPT or an image generation,

• 22:10Speaker 00

  you're okay because the time lag from the networking time lag, the latency lag is in milliseconds, is hundreds of milliseconds.

• 22:17Craig S. Smith?

  Yeah.

• 22:17Speaker 00

  But the actual computation time is in, you know, ten, sorry, thousands of milliseconds or even seconds, right? So that networking lag is not effectively your bottleneck. It's still the model processing time. So right now inference actually works very well. It doesn't matter whether you run it in a training type cluster. And as models are becoming bigger and bigger, they actually need that kind of interconnected GPUs. So in terms of the market use case, yeah, training was the primary driver of revenue or GPU use cases for Lambda. I think if, you know, we constantly do surveys, it's very hard to always know, like have posts on the market, exactly how much usage. But when we right now speak to our users, I think inference probably, in terms of the new dollar and new GPU spin up,

• 23:02Speaker 00

  I think inference has overtaken training in terms of how many GPUs are spin up. Like we still have training customers, we still have fine tuning customers. But in terms of when we're deploying new GPUs and how they're getting allocated, it's more towards inference. When Blackwells do go live though, I expect that ratio to again flip a little because the first foray into Blackwell will all be training customers, right? Because that's what they're gonna try and get advantage of the Blackwell. So it kind of switches like that. But overall market, I think inference is starting to take over in terms of the dollar spent.

• 23:33Craig S. Smith?

  Yeah. Yeah. And the reason I said, yeah, I think the same thing too, because.

• 23:42Craig S. Smith?

  You know, foundational models are the foundation, but

• 23:47Speaker 00

  Yeah.

• 23:48Craig S. Smith?

  The applications on top of them are all about inference. So and I've been talking to

• 23:57Craig S. Smith?

  Andrew Feldman at Cerebras

• 24:00Speaker 00

  Yeah.

• 24:00Craig S. Smith?

  And Rodrigo Leong at at San Benova.

• 24:08Craig S. Smith?

  Are you starting to buy those chips for the inference business? Or if not, why are you sticking with GPUs when these are so much better for inference? Or Grok for that matter?

• 24:21Speaker 00

  Yeah. So the first answer I can very explicitly clearly say like we have, we currently only deploy Nvidia GPUs, both for training and inference applications. And we have not yet forayed into deploying ASICs or alternative accelerators. Inference demand is definitely starting to take off. I think one of the key points there, just quickly on that note is if you think about foundational models, you know, when the whole kind of AI wave, especially around transformer started, there were way more than three, four dozen companies all trying to do foundational model training. I think now it's getting concentrated. The models are becoming massive. Like, I mean, you hear about 100K GPU clusters and so on. But it's getting concentrated how many people do it too, right?

• 25:05Speaker 00

  So more people are going towards fine tuning and then figuring out applications for it. So which is also driving inference use case. Now to your point where like, you know, like, you know, companies like Cerebras, Grok, San Benova, even few others, they're focusing on inference as a service and inference as an application, right? Because and all of them have their own kind of ways of missions, like, you know, for example, you know, Grok really focuses on the speed, you know, tokens per second throughput. Cerebras can fit a big model, but also then also is also competing on tokens per second throughput kind of. So the speed, they compete on more speed than a GPU, Nvidia GPU could. For us, there is two things. One,

• 25:48Speaker 00

  These are still kind of up and coming accelerators and use cases that yet don't have that massive market economic drive to for us to to to to be the first adopters. So, look Lambda, like I'll be frank, like we are a growing company, we are, you know, I would say like, you know, we have limited amount of money in our balance sheet and we want to basically put that money towards guaranteed revenue generating ability.

• 26:18Craig S. Smith?

  Yeah.

• 26:19Speaker 00

  And Nvidia GPU today is is that, right? And and and it's a universal card, every single application runs on it, people optimize for Nvidia GPUs, people build for Nvidia GPUs, you know, you know for a fact that every single application will work on that. I think it will be very interesting and and it's really, I think even Nvidia will like say that it's great to have competition because it keeps everyone on their toes and and from that perspective. But from our perspective, really, right now, it's more around both capital allocation and what we see as all purpose applications, like, you know, where we want to be, like, we we can't yet determine the niche of where do we want to really support that. So so that's kind of what has led us to really focus on the Nvidia GPUs in general.

• 27:03Speaker 00

  I think as the software stack on on on this company has become more robust, such that we can deploy infrastructure with them, you know, I think they will, there will be considerations, of course, for this. But to be frank, from our viewpoint, you know, I don't yet know, like, look, things can change, but I think we'll be looking for guidance at other hyperscalers, at other players to first introduce to this to market.

• 27:26Craig S. Smith?

  Sure.

• 27:27Speaker 00

  See the market capture and then maybe consider it. But for us, very clearly, right now, it's it's a, you know, Nvidia GPUs are, I mean, if you look at the market capture of Nvidia GPUs, it's well over 90% of the total compute, right? So so so we will stay within that lane for for now.

• 27:43Craig S. Smith?

  Yeah. The the other question is, you were talking about inference using your cloud and the latency is all in the computation, not in the transmission or how far away are your are are your big biggest customers or your rather how?

• 28:12Speaker 00

  The distance between data centers.

• 28:14Craig S. Smith?

  Yeah, I mean, are you are you are people globally using you guys to train? Or is it really a US based?

• 28:28Speaker 00

  Yeah.

• 28:30Speaker 00

  Yeah, I mean, like, look, I think doesn't matter which cloud you're speaking to, the majority of revenues coming from US based clientele. We do have, you know, as as as it's always the case, we do have global customers and and the part of it is for training truly. We have had customers from Europe and Asia because it doesn't matter and and vice versa. We have deployments in certain other countries that US customers have used for training. For inference, though, I think majority of the use user base for us is US based first and foremost. And a lot of our our inference deployments are in the United States. You know, and it's the usual regions, you know, whether it's, you know, kind of West Coast, East Coast in, you know, near Virginia,

• 29:15Speaker 00

  Chicago area, Texas areas, kind of, and then the new up and coming regions like in a Salt Lake City area and some other kind of regions where there is more power like Atlanta and things like those. But, yeah, generally, it's it's it's United States based. And and by the way, for the latency ones, for the smaller models, it is starting to become more important, right? And again, this is, you know, not kind of trying to harp on it, but like, going back to your point around Grok and Samanowas of the world, they kind of for the the throughput that they they have, the speeds that they have, latency would matter, right? Especially for smaller models like an eight billion parameter or now a three billion parameter model. I think that that's where latency does start to matter matter.

• 29:58Speaker 00

  And and and I think if you probably, you know, probably I think Andrew will agree with this or things like those as. As AI starts to feed input into other AI applications, latency will also matter. Like right now, you know, yours, like an I, like even if you are the top reader in the world, uh, the throughput speeds are high enough that, you know, uh, throughput can be faster than yours or at least my reading speed. Let me just talk about myself, right? So it kind of doesn't matter to me when that output is like, whether it's faster by 10x or 2x. As long as it's faster than my reading speed, great. But as those outputs get inputted into other AI models and there's a chain of AI models

• 30:39Speaker 00

  or agent and behaviors or actionable behaviors from AI, I think it will really matter, the, the speeds and then latency. And that's, I think, that's why these guys are also correct in saying that like latency, like throughput speed is important. So it will matter in the near future as more agentic AI applications come in. You know, people will try where, you know, we, we do do load balancing where like a user in West Coast, he is hopefully hitting our West Coast GPU and a user in East Coast is, he is just not as important today for us. Like right now, it's, we basically hit them wherever there is capacity. Like, you know, if you're sold out in, you know, data center in West Coast, like doesn't matter if a San Francisco user is hitting it, we'll just divert it to Atlanta.

• 31:20Craig S. Smith?

  Yeah.

• 31:20Speaker 00

  Uh, but, but as, as you go forward on inference applications, you will see this optimization. I guarantee you hyperscalers already do it by regions. Uh, you, you will see that across the board, but it's still not as important. But things move fast and especially smaller models as they're getting better. You will absolutely see, um, latency matter on, on, on being closer to the metropolitan areas. But, uh, right now, um, most of the, first of all, most of the demand in the world is US based. Uh, cause almost all startups, and again, I, I really don't mean to say that they're not amazing European and Asian companies. But if you look at the percentage spread out of the demand curve for compute, a lot of it's tremendous.

• 31:58Craig S. Smith?

  Yeah. Uh, in, uh, you know, I, I, as I said, I don't know if I said it to you, but I spent a lot of my life in China and. And I still travel quite a bit. I was in Armenia recently. And the AI community there is struggling because they don't have GPUs. Or adequate GPUs. China, of course, is under these export controls. They can't buy the GPUs. But all of these places theoretically could access GPUs in the cloud. So. Is is that, do you see much of that?

• 32:42Craig S. Smith?

  I mean, and and is it practical for a Chinese company? I mean, I don't know what the regulations are in China, but to access GPU cluster in the cloud to train a model. Does does that work?

• 33:00Speaker 00

  China is a is a China for China for China. No, because it's it's on the restriction list for for United States. So, for example, we have to have a block on on all IPs that are that originate from.

• 33:15Craig S. Smith?

  Oh, is that right? I wasn't sure.

• 33:17Speaker 00

  Yeah, yeah. And that's true, because, you know, even though even though the block states that, okay. Nvidia can't sell their GPUs, but but even then, like, it's even about the cloud access. Now, like, look, they it is a tough one to because, like. Before Chinese companies did used to have access, you know, you have the you have the biggest clouds in the world, like Alibaba and and other clouds as well that have these GPUs. And China has, you know, if you read reports, China has figured out a way to to get access. In other countries and then things like those. But for for Lambda, very clearly, we we we have to follow the US regulatory guidelines and as such.

• 33:55Speaker 00

  With Chinese stuff, but like if you're speaking about like in European country or an Asian or an African country, right, that are not on the sanction list. You know, they they absolutely can, you know, spin up a GPU on Lambda Cloud, doesn't matter where they're accessing it from and and do it. We do see a decent amount of demand coming from, you can imagine, Europe. I think, you know, that's that's another hub in terms of of the machine learning community and then and and starting to see, you know, from other nations as well. But like, this is part of where, you know, when you see Nvidia's report, they talk about sovereign clouds or more like more than sovereign clouds. I mean, they use that word, but like, I think it's like more like internal nation cloud, like so for

• 34:38Speaker 00

  Indian users, like GPUs in India, for for for, you know, French users, GPU in France, GPUs in France. I think there is like a lot of these things happening, both hyperskaters going into different countries and and deploying this, but also like local local companies spinning up. But yeah, China, Russia, like these, these are the sanctioned nations, which they they definitely, there's a lot more regulatory kind of thoughts that you have to and and regulatory laws that you have to understand to make sure that you're not getting used there. But for other nations, yeah, so it so it depends on on where you're from. China, definitely, I mean, like, both, they they want to have access. I mean, I didn't touch upon China. I mean, other than

• 35:21Speaker 00

  the United States, the biggest user, or if you think about demand computationally, will be from China. It's just we don't see it or we don't get to talk about it because we don't see it because we we can't. If it was a free trade, then we would probably be talking about China as probably competitive to US demand usage in terms of of the total usage in the AI. But we don't see it. So it's all it's all internal there from their perspective.

• 35:45Craig S. Smith?

  Yeah, and this is completely off topic. But do you think those sanctions and restrictions and shortage of getting their hands on on GPUs is going to cause China to fall behind in, let's say, AI development vis vis the US?

• 36:05Speaker 00

  I can only give you my personal opinion on this. I I I'm not an expert. I I definitely am nowhere close to be like even a novice in geopolitical realms of of of of things like those. But look, what I can tell you is from technical side. This is that that's my foray or or viewpoint. Just today, there was a model launched by Tencent, which was it's an open source model. I don't know if you saw it. It's a mixture of experts. Based on the performance evaluations, it beats Lama four or five B, which is the current state of the art open source model, at least in the United States, right? And and so what that tells me is the answer is no. We have had the sanctions

• 36:45Speaker 00

  for multiple years now. If you look at and and okay, so that's one one thing. Like if you look at one of the best video generation models there and and just now the first open source, like really good quality first open source model like by Genmo, you know, Mochi when got launched and and it's an amazing open source model. That's a US based company. But we have had video generation close source models, both in the United States by companies like Runway and Pika, but then by Chinese companies like Cling and then others, right? Who there those models are, if you look at the results, they're really, really good, right? So I think overall, it's, you know, you know, people say that, oh, like, you know, they are maybe behind in L L M's and things like those. But really, the the evidence is starting to go flip side. Now,

• 37:30Speaker 00

  they don't have an O one reasoning model yet, at least not that we know of in terms of both close source and open source. But but even other than OpenAI in the United States, no one else has that either, right? And I think it's just about development time for even other companies to catch up to that because it's a new research stream. So I I do not think from a technical perspective that that that they are behind. Maybe it has slowed them. I don't know what the hypothetical looks like if they had free access to the GPUs, what that would look like. But from a technical perspective, at least what I can say is the answer is no to me.

• 38:07Craig S. Smith?

  Yeah. And with this Tencent model, I mean, they haven't said what chips they trained on, did they?

• 38:16Speaker 00

  No, no, no.

• 38:17Craig S. Smith?

  And I mean, there certainly they were stockpiling GPUs, but at some point that that supply has to run out and.

• 38:26Craig S. Smith?

  I don't know. It'll be interesting to see what happens.

• 38:28Speaker 00

  I mean, thi- this is the part of that. And this is like, I know that this is a little bit outside of our thing, but like outside of discussion point, but like, I think that's the interesting part. I mean, people do speak about how companies within China get access to these GPUs and and none of us can comment on it in a sense, definitely, whether they do or not. I think that's that's that's

• 38:48Craig S. Smith?

  Sure.

• 38:48Speaker 00

  that's that's that's not, you know, kind of domain area. But what I can say is that if they don't have access to it and they're getting models like what the tensor model is, versus all that's that in itself is an amazing feat of technological progress from there. If they do have access, well, then in a way, then that's as the sanctions are not working. All right. So so I think that's kind of how you you look at it. But again, and then the interesting thing is the the Chinese companies have open sourced a lot of their models, like, which is uh, quite an interesting tactic that they've taken, right? And I think people do use Qwen and now I'm sure they will use the tensor model and then so on and so forth.

• 39:26Craig S. Smith?

  Yeah, yeah. Uh, okay. So where where are you guys going? I mean, first of all, scale, you're you're not a hyper scaler, but what how many locations, data centers do you have around the world? How many GPUs do you have deployed? And what's the growth tra- trajectory?

• 39:50Speaker 00

  Yeah, I think the the the interesting thing is that hyperscalers are also growing so fast that I think what our current deployment would have been considered like a hyperscale deployment, just like literally three years ago. Uh, but now, like, obviously, like, I think the the amount of GPUs that are getting both manufactured and deployed is is is is really un- unseen of, which makes sense because the demand curve for Lambda, right? Look, we we have uh, data centers in now double digits, right? Now, um, I want to also be transparent, like, you know, it it's not like the hyperscalers where we have like, oh, each data center is fifty megawatts, so people can multiply fifty times twelve and say you have six hundred megawatts. No, because I think the way when we started, we had smaller data centers, you know, three, five, ten megawatts. Now, as we're growing, now we have data centers.

• 40:35Speaker 00

  Thirty, fifty, maybe hundred in the future, right? Uh, so, so, so it's a much more of like a uneven mix of, of the sizes of data centers. But we have now double digit data centers uh that we are managing both in the United States. And what we publicly discuss in, in terms of international, we have a place in South Korea that we are uh deploying GPUs in um as well. Um, in terms of number of GPUs, I think, you know, the, the scale keeps on growing. I think, uh, you know, for, for Lambda, uh, we, we are, I mean, there have been public reports and, and I think they're pretty uh close to the mark in terms of we are in multiple tens of thousands of GPUs.

• 41:14Speaker 00

  And, and interestingly enough, uh, it's not as interesting a number because, you know, when, when you hear singular companies like even Meta and, and, and X who are not, uh, XAI who are not cloud companies having GPUs in hundreds of thousands. So, so that's kind of where we want to go towards scale, uh, if you think about the growth rate, correct? So, uh, it's about how do we get more and more computation under our management? And, and, uh, and then, you know, build the software stack on top of it. So, yeah, Lambda is in, in, I would say, uh, somewhere in, in middle tens of thousands today. And then, you know, ideally really like gunning for that kind of six figure mark in terms of

• 41:55Speaker 00

  number of GPUs deployed, uh, hopefully in the next year or two as, as our data centers, the bigger data centers uh come up and as we get more and more uh capacity uh under uh us. So that, that's kind of where the scale is. Uh, in terms of growth rate, look, I think for us, starting from a lower denominator, the growth rate looks amazing, right? You know, I think, you know, you could get kind of, uh, uh, last year, you know, you, you look at almost a magnitude growth. Now we're talking about, uh, you know, over a hundred percent, you know, in three figures, uh, percentage growth rates. We want to continue that. We want to be near that like an over hundred percent, hundred fifty percent growth rate mark.

• 42:39Speaker 00

  It'll be interesting to see what the, you know, this is the point where we, we constantly think about like, how do we keep on innovating? Both on software side as well as really offering what people want because to continue that. You think about like what hyperskitters are deploying in terms of GPUs as well. Um, you know, it's, it's gonna have competition, um, but. Underlying thesis still remains that I think, you know, everyone still thinks that, hey, there's gonna be some plateauing of the compute curve. Personally, I, look, I have roasted their glasses on and, and, and, and so, so, you know, of course people should take. You know, what I say with a grain of salt, but like, I do not believe so that there is a plateau. I think that with the, with the rise of inference demand, with the rise of new types of model structures, whether that's reasoning curve.

• 43:23Speaker 00

  Whether that's multimodal, whether that's video generation models, I think, you know, um, where we will end up is, is just a continued rise in demand. And as long as the underlying market force has strong, uh, you know, growth rates will continue. I mean, think about the video models, right? Like you are in, in, in podcast and video generation industry. You know, someone like me, I, I don't take out time to generate videos of my own because. There, there is, I, I know that there's millions of creators in the world, but that's still a barrier to entry among the eight billion population worldwide. But imagine if you could just type something and say, hey, make my test sound very smart about, I don't know, investing.

• 44:01Speaker 00

  And, and, you know, I upload my photo and it generates a nice video about, uh, me being as smart as Warren Buffett in, in, in investing value stocks. And imagine how many people will generate that or, or I wanna show myself as Luke, see myself as Luke Skywalker in a Star Wars movie and, you know. Keeping aside the legal content generation, all those things aside, you think about what that unlocks. Whether it's content generation, whether it's reasoning model, you think about how AI today is used in a lot of, you think about kind of activities that are not doers, right? You think about chatbots and stuff. Yes, you have interactions with it. You get answers. You use it in legal and support services. But you know, when we go into doing like what people like to use the buzzword of either agent application.

• 44:46Speaker 00

  So like writing code and and and taking your appointments and things like those, but all the way from that to little physical activities like, you know, whether in robotics or whether in any kind of machining or whether in, you know, civil engineering jobs, chemical engineering jobs, jobs that require complex problem solving that comes from reasoning. I think that the amount of computation required, like the curve will in fact keep on getting steeper as we figure it out. Now, whether it takes some time to figure out, that's sure that that might be the that might be the air pocket that will cause it to people to think that it might plateau. But personally, I mean, as I said, I truly fall on the extreme side of that belief of where we are going.

• 45:28Craig S. Smith?

  Yeah. And and on the agentic side of of that, as you were saying, that latency will become much more important when you have networks of agents working together. And I wonder then if if that means that,

• 45:51Craig S. Smith?

  you know, eventually every community will have its own

• 46:00Craig S. Smith?

  AI data center, you know, GPU data center serving its whole community. Where are your your data centers? Are they? You don't own the data centers, right? You're you're placing machines in data centers. Is there a logic to building your own and operating your own data centers? Or or is that just another business?

• 46:25Speaker 00

  I think. The way we look at it, Lambda for us, we are much more focused on the layers about data centers. So putting an infrastructure, making it run very well, putting software stack on top of that infrastructure layer for people to use it as a cloud service, both for training and inference and then keeping on going up the stack. But I think, you know, if you look at the marketplace, I mean, hyperscalers have certainly gone towards building their own data systems as they've gotten bigger in size. So, you know, never say never, I think this, at a certain scale, you know, operationally, it starts to make sense to get into either a building yourself or getting into a partnership to build one. But for today, Lambda, yeah, absolutely. We, we work with the big data center infrastructure companies.

• 47:09Speaker 00

  That's their, they're, they're excellent at it. They're efficient at it. They're fast at it. Right? You know, think about all, all the digital infrastructure players that are out there. And, and I think, so we, we, we do like a long-term leasing partnerships with them to, to deploy this. In terms of the local data center that you talked about, I think, look, I think the extreme version of it is, you know, obviously, people will move towards like, you know, local kind of inference at the edge. There, there's going to be absolute part of that. I always think of it as like, uh, you know, not, not anything, but like, kind of like, uh, you know, like different sizes of brain, like small brain problems, medium brain problem, big brain problems, right? Small brain problems solved on, on your edge device with this laptop, phones, glasses, whichever, whichever, whichever edge devices

• 47:53Speaker 00

  that, that humanity creates in the future, right? I think then you think about kind of like, you talked about either local data centers in, in areas or even kind of like generators, right? You know, you might have computing devices per house to run your local household robots and, and decision-making by the, or maybe they'll be all on. Right now, all of the humanoid robots, it's all, all on either local, um, the robots itself, like the computation is there or they, they interact with the cloud and, and things like those. So it will depend on kind of how that latency requirements

• 48:28Speaker 00

  progresses, but I, I won't be shocked where people have like, kind of we have a local water heater and local generator, like you can kind of have that on a house level as well, but I don't, I think that's too far away still, I think. But like, you were absolutely spot on that. I think what we'll have is. I do believe, I think as as more application, like you know, you think about whether it's uh you know, robotics application, agentic applications, I think, and AI kind of daisy chaining into other other AIs, I think you will see more and more um local and local creations of data centers that that that go there. And then there will be absolutely the massive ones. I mean, right now, all the all the talk in the industry and and is rightly so on this, you know, four hundred megawatt data center, half a

• 49:10Speaker 00

  gigawatt, a gigawatt data center, nuclear power to power a gigawatt, solar renewable power to to be a backup, uh figuring out ways to get it to multi gigawatt. I mean, uh you know, if you think about kind of the the level of the data center powers where we are going, you think about like three, four, five gigawatts, that's like a New York City style power consumption by a data center that is built in a hundred acre plot of land. I mean, the density of power uh that is required there. And that's where the a lot of focus today is because the big architectures are getting built here, you know, hundred thousand, four hundred thousand GPUs kind of structures that are going to come in the market uh in the next year or two years, right? So so that's where the focus is, I think, once that like,

• 49:52Speaker 00

  and because that's because the underlying thesis is scaling is critical for for making better AI models. I think once that moves on to how do we assign more computational spread to multiple areas, more reasoning, uh smaller models, but like doing a lot more of it a lot faster, then the conversations will grow into like, it can probably conversion of office real estate into data centers, like, you know, uh all that thing. So there will be waves of this kind of uh talk points around how we build data centers and power areas.

• 50:24Craig S. Smith?

  Yeah. Wow, fascinating stuff. So is there anything I haven't talked about that you want to talk about?

• 50:31Speaker 00

  I think uh from from my perspective, what is interesting, the the like

• 50:37Speaker 00

  Like, and again, because for for me, the most interesting thing are are kind of the technical progresses that are happening, both from the model architecture and the applications perspective, right? Like, we today see what we see, like, whether it's chatbots, whether it's uh, support agents, whether it's uh, code writing assistant tools, uh, you know, people question sometimes the the quality of the outputs, uh, and then so on. But but just for for me, the essential thing is that that's just like any technology, right? It takes time. Like, if you look at an iPhone in two thousand, whatever, nine or two thousand seven, you know, you'll you'll think it's a shitty iPhone. It's like, wow, this is so slow, you know, you can barely do call, you can do calls plus maybe a little bit of mail and things like those, right? I think

• 51:20Speaker 00

  and I think every cycle, the technology pace has become faster, you know, what it took for us to to get from, uh, you know, like getting really internet, uh, being invented to people using internet at a wide scale, it took like ten, twenty years of life cycle. Phones or smartphones was ten years, web services and cloud was like eight or seven years. I think AI will be faster in terms of how it gets really percolated in everyone's life. And I think the the most interesting thing so for me is to really keep track of some of this kind of interesting points, you know, chatbot was one. I think now for me is the video models, like, how do they get integrated for us? I think that that will be the next kind of, you know, like, a milestone point in terms of like

• 52:04Speaker 00

  how easy it is to do multimodal and video models in our day to day life, how easy it is to use it. Then it'll be this, like, what O one did was a reasoning models around, you know, kind of complex problems, and then you'll be like the low latency on the edge devices, kind of. So these are kind of the step function points that I I'm looking forward to. And it's very interesting how quickly we are moving through through this, right? And I think uh, even when we go from like, you know, what was G P T four was state of the art to now cloud three point five, now back to O one, these are happening in months. So so from my perspective, I think, you know, it's important that people stay very optimistic about this technology. Of course, we have to have uh, you know, controls around, a little bit of controls around how people

• 52:48Speaker 00

  it doesn't go in the the wrong side of things. But I just wanna make sure, you know, there are a lot of listeners to to to your podcast. I I wanna encourage people to to be in the know about about this. I think it's a very fascinating field. And then lastly, of course, on Lambda side, right? Look, I think we are here to really Underlying it all, of course, we want to build a big, successful, long-term business. And we really care about that. It's kind of our baby to do that. But underlying of it all, we really want to be the ones that provide a lot of, or one of the ones that provide a lot of compute and ease of compute to people that are building the next step of applications for this world.

• 53:30Speaker 00

  And so, you know, to that point, I think, you know, we really focus on the deep learning aspect of it. We spend a lot of time building for it. And for a lot of listeners, again, I would encourage you to try Lambda Cloud if you're building any deep learning application out there.

• 53:48Craig S. Smith?

  Okay, and how do they reach Lambda Cloud? What's the URL?

• 53:53Speaker 00

  Well, our classic URL is lambalabs.com. But, you know, our fun one is gpus.com. We own gpus.com. So we've been for a long time.

• 54:07Craig S. Smith?

  That was a coup, yeah.

• 54:08Speaker 00

  Exactly, a cool grab. That was done six years ago. So at least on that front, we were ahead of the curve for grabbing that domain.

• 54:16Craig S. Smith?

  What does the future hold for business? Ask nine experts and get 10 answers. Bull market, bear market, rates rising or falling, inflation going up or down? Can somebody please invent a crystal ball? Until then, over 40,000 enterprises have future-proofed their business with NetSuite by Oracle, the number one cloud ERP, bringing accounting, financial management, inventory, HR into one fluid platform. With one unified business management suite, there's one source of truth, giving you the visibility and control you need to make quick decisions.

• 55:04Craig S. Smith?

  With real-time insights and forecasting, you're peering into the future with actionable data. If I were a larger organization, this is the product I'd use. Whether your company is earning millions or even hundreds of millions, NetSuite helps you respond to immediate challenges and seize your biggest opportunities. Speaking of opportunities, download the CFO's Guide to AI and Machine Learning at netsuite.com/eyeonai.

• 55:39Craig S. Smith?

  That's netsuite, N-E-T-S-U-I-T-E, dot com slash eye on AI, E-Y-E-O-N-A-I, all run together. To get the CFO's Guide to AI and Machine Learning, the guide is free to you at netsuite.com/eyeonai. netsuite.com/eyeonai.