The Future of AI Infrastructure with CoreWeave

As AI applications become more complex, the infrastructure powering them needs to evolve. Corey Sanders, SVP of Product at CoreWeave, joins Chris to discuss why AI requires a fundamentally different approach than traditional cloud computing. They explore AI-native infrastructure, training and inference workloads, the rise of agentic development, optimizing GPU performance, AI research workflows, and why the future of software will be built around AI-first experiences rather than websites and apps. Featuring: Corey Sanders – LinkedIn https://www.linkedin.com/in/corey-sanders-842b72/ Chris Benson – Website https://chrisbenson.com/, LinkedIn https://www.linkedin.com/in/chrisbenson, Bluesky https://bsky.app/profile/chrisbenson.bsky.social, GitHub https://github.com/chrisbenson, X https://x.com/chrisbensonLinks: CoreWeave https://www.coreweave.com/Sponsors: Framer: The enterprise-grade website builder that lets your team ship faster. Get 30% off at framer.com/practicalai http://framer.com/practicalaiUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars! Midwest AI Summit 2026 https://midwestaisummit.com/

• 0:02Speaker 00

  Welcome to the Practical AI Podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights. You can learn more at practicalai.fm. Now, onto the show.

• 0:41Speaker 01

  Welcome to another episode of Practical AI. We are the podcast that tries to make AI practical, productive, and accessible for everyone. We get to talk to all sorts of cool people in industry. And today, I want to introduce you to Corey Sanders, who is the Senior Vice President of Product at CoreWeave, which we've been seeing in the news quite a lot lately and looking forward to learning more about CoreWeave and the various problems it solves and the position in the AI world that it's fulfilling. Welcome to the show, Corey.

• 1:14Speaker 02

  Thanks, Chris, for having me. I was gonna say, you got to talk to all kinds of cool people, but today, you're stuck with me. So here we are.

• 1:19Speaker 01

  Oh, no, no.

• 1:23Speaker 01

  No, I gotta say, you have a really interesting background. I know with things changing as fast as they are, I know you did a long period at Microsoft before coming to CoreWeave. And you have seen so much of the cloud landscape change. Like, I can't think of anyone better to address that in terms of going, kind of like, all the things you did in Azure, and then coming to a little bit of a different problem set as I'm looking forward to hearing from you about. And so, anyway, yeah, if you could kind of start us off, and kind of, what problem are you, what problems are you interested in?

• 2:07Speaker 01

  And kind of paint a picture of how you see the world from where you're sitting?

• 2:12Speaker 02

  Yeah, no, it's a great question. And yeah, I mean, absolutely right. Like, I've been, you know, did 20 years at Microsoft before I joined CoreWeave and worked in the early days of Azure. And so, you know, it's been it's been a trip now to kind of take a step into CoreWeave and think about many of the same approaches we had with Azure in the early days now applying with an AI-specific lens. And that's really sort of the right way to think about what, you know, I think CoreWeave is doing and sort of my view on sort of the world and the industry right now is, you know, we are seeing the opportunities for AI-based applications to evolve and grow into a similar way that cloud-based opportunities back in the early days grew and evolved,

• 2:56Speaker 02

  where, you know, it starts off with one or two things, and suddenly, people are expanding and growing and building. And then, seemingly overnight, obviously, it took many years, and I think will take many years this time, every single application, everything, single service, every single thing that we sort of touch and experience in tech has AI as a significant component of it. And that, you know, I think that pivot is a critical part of the places that I'm working, right? Which is, with that being a pivot of how applications and how services are built, the need for AI-specific capabilities, AI-specific, AI-centric application models are paramount, right? Sort of asking people to apply

• 3:40Speaker 02

  that AI concept to their legacy way of building applications will end up slowing them down, end up creating challenges. And so, the opportunity and what I get to really focus on every single day is asking the questions, you know, great, I understand this is how it was done, but how could it be done different with an AI-centric perspective, right, AI-focused? And how are we building these services to make it so that people who want to go build AI have a place they can do it, where that's the entire focus of the platform?

• 4:08Speaker 01

  When you, when you talk about that kind of AI-centric approach that you're taking with a lot of people, and we certainly are guilty of it on the show, like, people here, for a long time, we were talking about the latest models coming out, and we were in that. And the last year, it's been agentic everything and stuff like that. And so, can you talk, like, there's a perspective shift here, I think, where, you know, where people, when they're thinking AI, and there's first things that pop into their head, can you talk a little bit about, like, what was missing? As, like, if you were coming from kind of the old way of doing cloud, what was missing that needed to be there to, instead of just as an add-on? Because that's what happened early on,

• 4:52Speaker 01

  is like, oh, hey, you can get a GPU in our cloud and everything. But now, with this kind of different, you know, approach that you've just talked about, which goes all the way from infrastructure to application, what is, what has been missing that needs fulfilling? How are you thinking about that problem? And like, what are some of the details around what infrastructure should be going forward?

• 5:14Speaker 02

  Yeah, absolutely. Well, you know, maybe as a starting point, it would help to explain maybe the different, sort of different streams of AI, right? And you sort of mentioned agentic. And so, maybe I'll start there and talk, because I think it helps a little bit of a frame. And they, streams have separated, and then there's a convergence now that we're heading towards. That's also worth talking about. So, look, I mean, at a very basic level, there are kind of two big aspects of sort of what, of how AI is used or leveraged. One is on the training side, right? One is the actual creation of these models, right, creation of these, of these weights, right, the making, sort of doing the learning in a very similar way to the way the human brain works, but doing the learning to actually

• 5:57Speaker 02

  create the output that we then engage with, whether it be AI, whether it be Anthropic, whether it be, you know, Meta, sort of, all of those have that sort of background in training. And that requires very specific infrastructure to be able to deliver. And it requires, and the infrastructure is very expensive, right? As we all know, and it's very, you know, deployed in sort of large tranches, and very interconnected, right? One of the biggest requirements for many of these training workloads is there's sort of these large tranches of deployments that are all deeply interconnected, so they're all working together. And, you know, as part of that training area, some of the things that, you know, CoreWeave identified, and, you know, certainly even the broader market identified, is the potential places where

• 6:42Speaker 02

  that work slows down. Because the GPUs are so expensive, right? Any sort of slowdown in being able to get the job done is impactful. And so, that includes things like failures in the GPUs, that includes things like storage being loaded into the GPUs, right? And so, all the way down at the infrastructure level, what's going wrong with the GPUs? Are there failures? Into, all the way up to the orchestration over which job is running on which infrastructure to be able to optimize the output, right? And so, this is where I think AI-focused direction, that's a very different set of requirements than just your standard public cloud with a whole bunch of computer racks, right? Right, it requires a bunch of both design and how the hardware's being laid out, and specifically, how it's interconnected,

• 7:26Speaker 02

  that's very unique to AI workloads, all the way up to then how the jobs are being orchestrated with knowledge over what's being run down at the core level of the platform. And this is a bunch of places that I think, you know, Corey has a bunch of differentiation on things like our durability platform, our storage platform, right? There's a bunch of places where we offer unique services targeting that type of AI massive-scale training workload, that then is just very different than your, again, your classic cloud workload. So, yeah, go ahead, pause there, because the second half of the story I'll get to. No, but go ahead, Chris, I'm ready. You had a follow-up. And I'll talk for, I'll talk 45 minutes uninterrupted, if you let me. So, let me, let me pause and let you kind of jump in. Go ahead.

• 8:05Speaker 01

  No, no, it's fine. And don't lose the second half of that story, because you had me, which is why I had the follow-up on there. I think, I am curious, as you're talking about that transition, like, if you'd bring, like, for you personally, when you were, before you came to CoreWeave, when did you have that realization? Like, is this, as whatever this epiphany is occurring, you know, was it all at once? Did it happen over time? Like, when did you realize, I want to do something different? And don't lose that second half of the story that I was cutting off as I gotta go.

• 8:37Speaker 02

  Yeah, no, no, no, right. And the second half of the story is very similar to the first half, just a different, a different sort of outcome, actually. But yeah, I mean, for me, look, I got the, I got the honor of my final years at Microsoft getting to work on sort of a lot of the AI infrastructure aspects in the platform. And it was, it was kind of a funny thing, because it was sort of not my day job. Like, my day job, I was working on industry solutions. So, like, financial services, you know, retail services, et cetera, trying to think through how these large verticals could leverage AI in their workloads, which actually is the second part of the answer. But the first part of the answer, I ended up getting pulled into a lot of the discussion over how, at Microsoft, were we optimizing

• 9:21Speaker 02

  the deployment of that infrastructure, right? And that, that realization really struck me there, which was like, you know, the, prior to that, a lot of the, you know, a lot of the big cloud strategy was, you end up having a bunch of space and power, and as the needs come up, you deploy what you need to, right? So, you need more compute in a given location, you deploy it. And within, you know, days, you have a huge amount of additional compute. You need a bit more storage in a location, you can go deploy a bit more storage, and you can kind of add. And that sort of commoditization, that sort of fungibility is a big value add for the big clouds at the scale that they're running. But it turns out, you can't really do that. When you're dealing with sort of AI workloads, again, all interconnected, all wired together with,

• 10:05Speaker 02

  let's say, Finna Band or Rocky, very specialized network behind it, very specialized storage underneath it, suddenly it's like you've got to pre-plan things, right? You've got to, you've got to be thinking about it ahead of time. And so it definitely was, it definitely for me was an experience of learning, right? It started there. And then as I moved over to CoreWeave, I think the, the realization of how many places from, again, caching, specific caching for AI workloads to specific approaches to Kubernetes, which is of course a standard orchestration solution, but can deliver focused AI enablement for bare metal-based deployment to really squeeze out all the juice of that GPU, right?

• 10:49Speaker 02

  All the way up to, again, the orchestrator being sort of aware, that realization of like, how many aspects of the platform needed to be, could be, and then needed to be customized for AI-specific workloads? That, that was really, I feel like I didn't fully get it until I came to CoreWeave. But I, I saw it in the beginnings of it when I, when at Microsoft and I don't know whether that's a Microsoft or Corey thing or more just a point in time thing for me personally. Like it's, it's more probably that learning over time that you start seeing more and more of this customization, realizing, wow, like this is because of how much value and cost there is behind this, it is worth it to do this customization because people will pay for it to get the most out of those very expensive GPUs.

• 11:31Speaker 01

  So, so you raise a great question. If you are, if you are, it's you or you, you're coming with the background that you have and the experiences that you had professionally and, and you're making that adjustment and, and you're seeing that you can come to CoreWeave and see the capability. How, how do you like, as a, as a problem, like you've identified a particular set of problems that are part of this evolution? And, and you're doing it from a great vantage point, given, you know, professionally where you've been. So, like, all the other AI practitioners that are out there don't have the benefit of your background. So how do you teach the, like, how do you teach them that there is something that they're

• 12:13Speaker 01

  not seeing that is a set of concerns that they need to address that they don't even know that they have? How do you approach that, that education piece?

• 12:22Speaker 02

  Yeah. I mean, look, it's, it's, it's very hard. By the way, it's very, it was, it's very hard for me. I mean, I think even so, like I, as much as I appreciate your sort of lead in that, that I'm obviously uniquely capable at doing this. It's actually, you know, that experience, that sort of big cloud experience, while it's super, super valuable, it also ends up actually being a limiter because you come in with a bunch of assumptions, right? To your point, you come in with a bunch of like, well, this is how we built it. Like, we learned that over 10 years and this is why we built it. And you don't necessarily question all the things you should question over like, well, but maybe this is a different world, right? And so I do think it's, you know, in some ways it's an innovator's dilemma sort of at

• 13:07Speaker 02

  the core, right? It is basically, you know, what, what, and I say this a lot, like what got us successfully here today does not necessarily get us successfully to the next wave, right? And, and that to me is such an important, just life lesson, but it's certainly a tech lesson. And so for me, like, it really is like asking those questions, right? It's, it's asking sort of the point of question of like, okay, so, so that's true, but, but that's true based on 10 years of commoditized, you know, general purpose, public cloud assumptions. What if we assume this is different here in AI? Or what if we assume that the data requirements are different or the container's image sizes are different? Or the overall flow of, of AI innovation is different from typical software development?

• 13:52Speaker 02

  Now what problems do we need to solve? So in some ways, it's all about asking the right questions and it's, it's also one of the things I always like to push on as being, I think, one of the most protected aspects of humans when it comes to the innovation of AI is that creativity, asking questions that maybe are different from the way things are. As we all know, AI is all based on the way things are, right? It is deep learning and regurgitation with a ton of amazing magic in what exists, right? But sort of the ability to now say, well, but maybe what exists is wrong, that's the difference. And I think that's where, when we approach AI and frankly, when we approach a lot of tech innovation, that's the questions that need to be asked.

• 14:37Speaker 02

  And then I would argue, as much as I think I'm decent at it, hiring really good people who are amazing at it is actually my secret sauce. So I look for those people in the market and, and basically find, try and find the people who are going to challenge me and say, well, but Corey, maybe your baseline is wrong. Like this is what you did in Azure and it worked in Azure, but maybe it doesn't work here anymore. And it's like, oh, wow, that's, that's good. Harsh, but, but fair. And so, you know, that type of approach, I think is.

• 15:01Speaker 01

  Okay. So one of the things that you mentioned a few minutes ago kind of caught my attention and I wanted to go back to it and that's having to do with training. And, you know, when you have, you know, training jobs fail and you talked a bunch about like where, you know, where in the system that could happen, you kind of identified some of those points. But could you talk a little bit about like, what actually goes wrong in the systems before an AI practitioner or researcher or something has identified that, that things have gone wrong and that they're not making progress along, you know, the line that they're, that they're trying to work toward?

• 15:42Speaker 01

  You know, I think there's, there's a little bit of, I don't know, you know, just black cloud mystique about like where that can happen. Can you talk a little bit about what you've seen and where, in terms of where that problem arises and, and how and, and what it is, what the implications are on that?

• 15:59Speaker 02

  Yeah, absolutely. I mean, it's, yeah, there's, it's one of those special areas where there's both science and art to it, right? I think and, and that's you, you find some of the best people in the industry who do this. It is that sort of instinct and sort of knowing how things work. But like, look, I think that, that, the concept of a failure in a job can mean a whole lot of things, right? There's certainly sort of failure from an infrastructure perspective. You know, the, the infrastructure fails, the sort of a node slows down, things are running slower than you'd expect. And that can be sort of a plethora of things, right? That can be a storage slowdowns, that can be GPU slowdowns, right?

• 16:37Speaker 02

  One of the things that we've recently launched, which is a surprisingly, surprisingly enthusiastic capability is our GPU straggler detection, right? This concept of like GPUs slow down for a variety of reasons inside their infrastructure. And then when your job is hundreds or even thousands or sometimes tens of thousands of GPUs, finding that one slowed down and you're, the output of that is maybe your job is slower, but like which GPU slowed down, right? Like which GPU isn't performing? And so that type of detection and observability is actually quite challenging. And so, you know, a bunch of that observability, this is a big part of, I think, where that

• 17:19Speaker 02

  sort of AI-centric work, that sort of, you know, we believe our secret sauce, but certainly everybody's working on that sort of like AI-centric focus on sort of being able to detect those types of hardware failures, the hard failures and the, the soft failures, the sort of slowdowns and, and so on, that can really dramatically impact the job. So that's kind of more on the like pure failure hardware impact. There's also the other side, which is, you know, a big part of this, it's, it is research, right? I mean, and we call it research for a reason, which means you run a ton of experiments, right? You've drawn a ton of experiments with a bunch of different parameters and different controls, right? And then you, and then you need to compare them all and sort of study them and understand which ways went what and, and sort of now based on your output, what should the next experiment be and so on.

• 18:03Speaker 02

  And this is where, you know, a product like Weights and Biases, which is the top of our platform, really focuses on that. But one of the biggest is sort of that experiment tracking and comparisons and so on. One of the biggest things we've heard feedback on and recently sort of announced, hopefully a response to this is very new, is it's really hard. Your point about sort of the art versus the science, like it's really hard that, that detecting and finding and then learning from that experiment what the right next step is. And so we recently launched what we call ARIA, CoreWeave ARIA, which is short, it's clever name, short for AI Research and Iteration Agent. And its entire job is to try and do this sort of continuous analysis on experiments that are being run and what the right next step should be.

• 18:47Speaker 02

  And so, look, I think the simple answer is right now it is a lot of experience and expertise in this area that's driving that type of work. But as is so common in technology, a big step of platforms like ours and, and others in the market is to enable that democratization. How do we make it so that many more people can do that type of iterative model improvement work? And, and, and that's, and then, and using AI itself to do that, I think is sort of a clever, a clever approach to this.

• 19:19Speaker 01

  With, with you talking about ARIA, could you talk a little bit about like that, I presume is introducing a kind of new workflow? It, new capability, it gives new insight, but there's new workflow there. Could you, could you talk a little bit about like how that changes the practitioner's day to day activity and like what's introduced or taken away and, and kind of, you know, talk about the benefit of that?

• 19:44Speaker 02

  Yeah. Yeah. I mean, look, I think if you take my, my full, full sort of view, full extension of time, I think this concept of, of portal experiences and console experiences and, and sort of standard views, I think they're all going to fade away. You know, I think the experience is going to be an interaction with an agent, right? Now, how that interaction actually sits, is it in a website? Is it on a console screen? Is it in a, you know, my VS Code console? Like, I don't know, right? Like, I think there's a lot of different places or all of the above, frankly.

• 20:24Speaker 02

  But like, I think the idea will be, hey, here is the type of improvement I want in my model based on what I'm seeing from my traces. You know, what's the best way for me to run some experiments? Great, run them, right? And then, you know, on my mobile app overnight, I see, okay, the experiments ran and the agent's popping in and saying, hey, you know, the experiments ran. It turns out this worked, this didn't, right? I think we should do another run here. What do you think? Yes, let's go do it. And so then that run kicks off, right? And then it comes back and says, oh, you know, this improved by quite a bit. But based on this data, perhaps it actually would be helpful to go and iterate and iterate and iterate and loop and loop and loop. And so this sort of like approach to AI loop, I think will be that type of that development experience.

• 21:08Speaker 02

  It will still be the brains of the researcher, like it will still be the, well, I don't know if that's gonna work. Let's try this instead. But it is going to be heavily guided by this sort of agent analysis of the data versus you go into your console, you look at the line charts of your six runs and you compare your success rate to each other and your performance rate to each other. And you visually look at it, which is, by the way, state of the art today, but not the way that the world's going to move. And, you know, I'm excited about that direction. I think that that agent led is the right direction. And candidly, I would argue, I think that's going to be the case for all of our interactions

• 21:50Speaker 02

  moving forward, no matter what we're working on, whether it be banking or buying clothes, right? It's going to be that type of interaction model versus click on a button that says buy, which I just, I think will be antiquated in five years and, and sometimes even laughable in 10, but we'll see, we'll see sort of how far it goes out quickly.

• 22:08Speaker 01

  I know. Well, I'm on the older side of things and can remember back. And yeah, when I think about some of the things from, from a few years ago, you're like, wow, we got through it. One of the things I wanted to ask about is, is you've mentioned agents a few times. And I think, you know, typically people are thinking about agents in terms of, okay, I have my model and I'm going to, you know, figure out in my application, you know, how many agents are assigned to the model? Is it one to one or one to many? And I'm going to give them the tasks and give them the interactions. But I'm, I'm wondering, is it a little bit different on the infrastructure side? Like, how do you see? You've kind of already made reference to it.

• 22:48Speaker 01

  So like, how do, how does agent architecture work in a, in a, in an infrastructure environment like that? What do, what do they mean to you and what does it bring? And, and can you talk a little bit about like, like this is what we get on our backend as a provider versus what this is what you as a customer would get on the front end? Can you talk a little bit about that infrastructure agentic?

• 23:13Speaker 02

  Absolutely. Yeah. Yeah. I mean, the, so I'll go back even to, I think the question I answered, I feel like many moons ago, where I said there were two paths, right? And we sort of went down the first path. The second path is this inference side of the house, right? Which is basically the AI application, right? You know, I, I like to call it AI application because I do, to your point, I think you said like everything, everyone's talking about agentic. And I think that's, that's fine, but like in some ways it's all about an application that's AI centric, right? And agentic happens to be one type of that. But all of it will sit on some aspect capabilities of an inference call, basically leveraging

• 23:56Speaker 02

  all those models that we just built and asking it questions or asking it to do an analysis or asking it to simply spell check something, right? Like all these things could be on the backend there. And look, I think to your point, as we get more and more complex and, and there's in some ways, two different approaches to AI applications. There's certainly the productivity internal facing ones, the, the co-pilots of the world. And then there's the, what I like to call the sort of mission critical, the business critical applications that are either, you know, external facing for a customer. So like a customer's customers or sort of critical to their business workload, like a gene folding application for a pharmaceutical company. Now that, that second category, and now kind of returning back to your, your, your,

• 24:41Speaker 02

  your, your most recent question, that second category, I think will be a very complex

• 24:49Speaker 02

  application model, right? Like over time, right? You know, today, I think there's a lot of this concept of like, I have a problem and I'm calling a frontier model and I'm getting an answer, right? And that's my app. But over time, it is going to be, I think, a consist of many agents or many models, many inference calls of many different types that come together to build an application, right? So in some cases, the very deep analytical, sort of the biggest, hardest problems may go to those frontier models and call into them and ask them to kind of do that sort of analytical work. But like translating, someone asks a question in, in Japanese and you want to translate it to, you know, English, like that doesn't need that level of depth.

• 25:31Speaker 02

  That, that may not, that, that may, that may enable sort of a different, smaller, cheaper model. And, you know, I think as you look at some of the very specialized workloads, the gene folding example, it may need a very specialized approach to a model versus, again, this general purpose feels like a big, a big giant hammer for maybe a screw, right? And so, you know, you can certainly push it in, but it doesn't quite work. And so the, the opportunity, I think for some of those workloads is to think much deeper about how their application is going to work. And this is to your point, what, how is it going to leverage the infrastructure, right? You know, what, what is the, with the infrastructure being sort of a significant component of the cost, how is it leveraging it? Is it fully optimized for that infrastructure?

• 26:15Speaker 02

  Is it fully infrastructure aware? All the things that I mentioned before about being infrastructure aware and storage caching and so on, all of that makes the inference workloads work better, work faster, work cheaper.

• 26:27Speaker 02

  And then I think the other question is, is it leveraging the right model, the right parameters, the right prompt?

• 26:35Speaker 02

  You know, and that to me is, is opening up a new approach to AI application development, which we like to call this AI loop, where you're never done making it better, right? You're never done making the prompt better, or you may want to take a model and make it a little bit better and do a little bit more training on it and it feeds back into that training component. And so this concept of doing inferencing, running an AI application, having 30, 50 different models, all interacting with each other, and then finding the part that needs improvement or could be cost, could be a cost benefit, or could have a little bit better result or could run a little bit faster. And then running that through the system and using products like weights and biases to take that output, run it through the system, use something like ARIA to learn and make it better and better.

• 27:20Speaker 02

  And so I think that's our future for those types of applications. And that's all going to be built on the best infrastructure, all the way up to that workflow that I think is going to be crucial to, to building the best apps.

• 27:32Speaker 03

  Seems like the amount of webpages, landing pages, et cetera, that we have to create every week just keeps growing. And often those turn into a pile of tickets and handoffs amongst team members. And that's why I love what Framer is doing, one of our partners who has a pro website builder platform where teams can collaborate in real time and iterate on the same page and publish instantly. And you can even tie in agents into this work where agents and humans work in tandem. Agents bring speed and scale. People bring taste and judgment and control.

• 28:12Speaker 03

  So I would encourage you, if you are starting a new website from scratch or needing landing pages or relaunching a website, check Framer out. Learn more how you can get more out of your site from a Framer specialist or get started building for free today at framer.com/practicalai for 30% off a Framer Pro annual plan. That's framer.com/practicalai for 30% off. Framer.com/practicalai. Rules and restrictions may apply.

• 28:46Speaker 01

  Okay, Corey, so as you were, were talking a moment ago in answer to that question, I've got a whole ton, a whole ton of questions.

• 28:55Speaker 02

  I'm sorry.

• 28:56Speaker 01

  No, it's good. This is, this is what we're here for. But no, it's, it's like, so as, you know, when people are out there and they're thinking about their own workflows and they're looking at, you know, the, the, the, the current incarnation of agentic engineering is kind of a loop engineering and getting all your loops going and, and everything like that. And you made reference to that in your answer. And so as people are trying to constantly level up in that way, and there's a learning curve for them to, to do that, what with, with CoreWeave, if they go, I'm moving, they've, they've listened to you today, they're like, I definitely want to move to this platform. What changes in their agentic or loop engineering that, that they should be thinking?

• 29:41Speaker 01

  If they're already trying on whatever platform, maybe Amazon, Azure, Google, whatever they're on, they're already kind of trying to do that. As they come and, and, and get onto CoreWeave and you've talked through our conversation about all these advantages, all these different capabilities that you're bringing in, how does that change what they're setting up in their prompts and how they're setting up agents to address? How do they take advantage of that in their current workflow? What, what adjustments are made?

• 30:11Speaker 02

  Yeah. I mean, I think. That one of the key things is being able to quickly execute and learn from that loop without a lot of, you know, additional noise or disruption, right? And what I mean by that is, and this is one of the things that I'm pretty excited about, what I get to kind of work on here at CoreWeave, is how are these, how are these, how is that loop all basically playing out in interconnected services, right? And how do you basically execute against that loop? And so by running your inference service and having it then leverage our Weave platform for traces and running evaluations, basically saying, okay, in production, I'm now seeing

• 30:53Speaker 02

  that with these types of questions or these types of operations, I'm either going down this wrong path, so I need to figure out how to fix that, or let's say it's taking too long, right? And so I need to think about, am I trying a different model, right? And so like, the tracing from production allows you to sort of understand where you need to make these improvements, right? And with the agent, sort of the ARIA agent, helping you sort of find those, right? And say, oh, well, gosh, this looks like it's slower when it has this type of question. To then being able to directly execute, okay, great. If I change my prompt, does that fix it? If I, if I change my model, hey, this new KIMI model is out. Wow, like this, I'm hearing it's really, really fast. Like, let me change my model and see if I get impact.

• 31:36Speaker 02

  And then I can quickly set up an evaluation that then gets executed, right? You can use our sandbox product to be able to just run that evaluation, say, okay, now I see these two together. And I understand now, okay, the performance did get improved on this one with KIMI. Let me make sure that the pricing isn't sort of a big difference. Okay, it's okay. So now let me kind of swap that in and get it back into production. And now let's take that loop again, right? So the loops can be fairly small, right? Like, every single modification, in fact, most modifications, I'd say, are not, let me put this into my training workflow and actually do some reinforcement learning or do some, you know, fine tuning. That is a possibility. Like, in some of these cases, like, okay, gosh, this is great.

• 32:19Speaker 02

  It's working well, but like, my accuracy is just a little bit off in these types of questions, and I don't think a new model's gonna help. Let me actually do some fine tuning, right? Or let me do some reinforcement learning, since I maybe don't have the data to do my fine tuning. And you can run those services again, all integrated into Core with, again, the Weave traces, to see, did it work? Run your evals, it did, plug it back in, right? And so, like, I think what we're going to find is that you're, you know, developers, at least what I believe developers are gonna look for is, one, being able to do much, if not all of it, all in one place, because just the simplicity of being able to track those traces, having a registry, which, you know, CoreWeave offers for lineage, like, every update in model, every update in parameter, you know, et cetera, being able to try, oh,

• 33:04Speaker 02

  that version was three yesterday, okay, that's the one I want. With then an agent that sits on top with the knowledge of like, oh, that's what you need to go try, or you should go work on that. To me, that's, I think, the real power here. And I'm excited that, like, you know, these components, like CoreWeave delivers a bunch of this, but a lot of this is also, we're continuing to, you know, to make it super easy for that flow to happen at a, at a, at a much more accelerated pace than it even happens today on CoreWeave, candidly.

• 33:32Speaker 01

  Gotcha. With, you know, as you're talking about, like, that, that flexibility to get different models, but in the same conversation, we've been talking about kind of just all the benefits and the virtues of, of the performance wins you get from that kind of vertical integration of your stack. Is there, is there any kind of tension between having, about kind of supporting an open approach on one side and having it deeply optimized in terms of portability, you know, for, for a customer and things like that? How do you, you know, how, what's your position on the tension there in terms of portability versus performance optimization?

• 34:08Speaker 02

  It's an awesome question. And, you know, I'll give an answer that I've actually believed, for, for quite some time, I had the, the luxury of deploying the first Linux infrastructure on, on, on Microsoft, on, on, it was called Windows Azure at the time, so it was Windows Azure hosting Linux. And so, you know, my, my belief system from then to today, and I think the CoreWeave belief system is, we need to expect and build for multi-cloud. Like, this, this belief that, you know, CoreWeave is going to be the only cloud people use for their AI applications, I think is false. Like, I think, you know, and, and so I talk about this loop, and I'm enthusiastic about the loop, you know, having all the components on CoreWeave to deliver on this loop, but

• 34:51Speaker 02

  with the strong realization that people are not, that many people are not gonna use every service that CoreWeave offers for the loop, they're gonna use some other services. They're either already have, you know, an evaluation service that they really love, right? Or, or, or they already do their training somewhere else and they wanna run sort of the inference here. Or they already do their inference somewhere else and they wanna run their training here. I also think in a, so, so I think being very open and, and, and delivering upon consistent services is, is a key part of our strategy, right? And that's sort of like, when you think about things we offer, SUNK, which is Slurm on Kubernetes. Slurm and Kubernetes are both, you know, open source projects, right? And so, you know, we take a lot of pride in believing we deliver the best.

• 35:33Speaker 02

  And that really, to me, is where I think the differentiation and value comes in, is can we take some of these open source, multi-cloud, cross-cloud capabilities and make people want to use us because we offer the best services? And so, lock in with love is sort of the joke I use, I typically make. And so, you know, I think that really is how I think about it. And so, it's also is why we take some of our SUNK Anywhere, we recently announced, so you can actually take our SUNK offering and deploy it on other infrastructure. Like, we think it works best on our infrastructure. Yeah, go ahead.

• 36:02Speaker 01

  Go ahead and go there. No, I was just gonna say, go there. You've mentioned SUNK a couple of times, along with Slurm and Kubernetes, and like, go there and tell me a little bit about it. And add in also, you know, for people who are using, you know, Kubernetes in their, in their old world, the old cloud and stuff, like, what's different? What's it doing well? What does SUNK bring? Can you, can you kind of package that up so people can, can kind of follow there?

• 36:25Speaker 02

  Yeah, absolutely. Yeah, so, so, you know, put simply, like, I think Kubernetes is such a market leader and, and, and, and there's deep awareness and understanding on how to operate and orchestrate infrastructure, right? And work goes on infrastructure, right? The concepts around Kubernetes, sort of the, the, the platform that's been built on top of it, certainly has, has, has effectively dominated that market for, for years, which is, which is awesome. And then Slurm, which is kind of a, a job scheduler orchestrator placement engine that has, has, you know, I would argue, been a dominating force in research-based workflows, right? In sort of AI-based research. It's not the only one, right? There's quite a few, quite a few out there, and we could have probably a full hour show

• 37:10Speaker 02

  debating the merits of each, but, but, you know, many, many AI researchers know Slurm and are comfortable with Slurm. The challenge is, is that the Slurm on top of infrastructure is, is actually tricky to manage, right? Slurm on top of infrastructure, particularly for sort of AI researchers, is challenging, sort of managing ups and downs of infrastructure, failures in infrastructure, and these are the things that Kubernetes is awesome at, right? And so, what, what CoreWeave built was SUNK, which is short for Slurm on Kubernetes, although it's with a U, but, and, and it is about bringing the best of both worlds, right? Enabling the sort of, the, the power, the capabilities of Slurm job scheduling with

• 37:54Speaker 02

  the orchestration power and ease that Kubernetes brings, and sort of jamming them together and creating sort of a platform. And this is, this is sort of a key, you know, key product that we offer directly on our infrastructure, but like I said, we have now recently announced that we offer that to be taken anywhere. It's called SUNK Anywhere, it's a clever name. And so you can deploy this on other clouds too, and take that, take the power of that Slurm plus Kubernetes support onto any cloud.

• 38:18Speaker 01

  So, really, really interesting. And, and like, one of the questions I really wanted to get in for, I'm taking a slight curve on this, is, is as you are, you know, servicing and providing, you know, more and more capability to people, people are, you know, always talking about the cost of, of things. And I don't mean the cost of any given provider, but just in general, to be able to do meaningful experimentation and research at, at the frontier is so, so capital intensive in terms of the compute and stuff. And like, how is that something that could be addressed? Is, is, how do you guys approach it?

• 39:00Speaker 01

  And how do you, and in a broader context than just CoreWeave, how do you think the industry at large needs to approach the problem? Because it, it kind of, there's a, there's a, there's a special few in, you know, in a sense, that, that have a lot of resources, and a lot of others are saying, how do I compete against that?

• 39:17Speaker 02

  How do I do that?

• 39:18Speaker 01

  Yeah. A lot of those folks are listening right now and watching the show, and like, what, how, what do you, what do you, what would you say to them? How would they approach that? And what can CoreWeave do?

• 39:26Speaker 02

  Yeah, I mean, I think the part of the goal with this AI Loop concept, part of the goal with this sort of integrated story, is to enable people to do more with less, right? Or, or, or do, do the same with, with less, I guess, depending on how you look at it. So, and, and so what I mean by that is, you know, optimization can mean a lot of things. So it can mean being able to choose a cheaper model that gets the work done, done well enough, right? And so it requires less infrastructure, requires sort of a less deployment model. It may be, you know, doing model improvements and fine tuning to reduce the size of a model,

• 40:10Speaker 02

  or enable it to be more effective with a smaller sort of scale. Part of the goal with ARIA and things like ARIA are to enable less trial and error, right? Enabling faster movement from experimentation to production, which is actually a big component of that cost construct, which is like, great, like, I wanna go build this in production, but how expensive is it going to be for me to get there? And with something like ARIA, with something like the AI Loop, you, you, you have sort of this flexibility to get things out, get them to start be revenue generating, sort of, get them to start building on your business, and then experiment and learn as you go. And with the help of like, hey, I wouldn't spend the money to fine tune this, I think you could just shift your prompt and improve it, says agent.

• 40:55Speaker 02

  And so like, I think the opportunity to sort of get that guidance to, to help direct, that is built on the back of what, you know, the learnings we've gotten with all of those massive labs still working. By the way, those massive labs are still on the very cutting-edge frontier, where like, an agent that talks about historical knowledge may not be as, as helpful. But for everyone who's kind of one step behind the frontier labs, which, by the way, I think every enterprise and every sort of large company would wanna be there, because they're like, here now, right? That is where I think some of these capabilities, coupled with sort of the metal-based optimizations that I've already talked about, where I do think this type of platform could really, could really be meaningful.

• 41:36Speaker 01

  That's really cool. With, with. Another thing that that is so big this year and exploding outward being kind of embodied agentic engineering at the edge, robotics is just taking off with AI integrated. What is CoreWeave's story going forward in that space as you are getting where you're not strictly based on cloud, but there's some hybrid between cloud and edge platforms that are out there with different levels of embodied intelligence and agentic capabilities?

• 42:10Speaker 02

  Yeah, I mean, there's a few places that we really focus when it comes to robotics and, you know, let's say some of the scenarios around robotics, industrials and manufacturing and so on. You know, one is, I talked a little bit about, you know, our models, our experiment tracker technology, right? And I kind of talked a little bit about looking at the different charts and so on and so forth. You know, one of the things that we recently added to our weights and bias platform is a specific robotics focus, right? And something even kind of as trivial, but is a huge, huge value is shifting from those sort of line charts showing success to visuals showing success, right? Showing the actual robot and how it's

• 42:54Speaker 02

  moving with all of your experiments and being able to detect what went wrong and what didn't. Obviously, this is pre-agent where I do think then add on top of that aria and you sort of have all that benefit of that robotics sort of training and iterative work adding up to being able to sort of drive a bunch of the robotics workloads. And then, you know, the other area that I think is quite interesting is work with Monolith, which was a company we acquired a few months back. And, you know, one of the things that we've realized in many of these cases is the need to go engage directly with customers on the types of scenarios they're building, right? And where we can go help. And so that's where, you know, I think a lot of this sort of focused engineering resources going in and sitting

• 43:37Speaker 02

  with customers and saying, let's talk through what you're building, let's talk through what problems you're trying to solve, and let's figure out how we can help. And, you know, we've had this for a while as part of our overall mission control offering. We called it direct to expert, which is really where we kind of lean in with resources and expertise. That sort of additional model has brought some of the industrials, the manufacturing expertise to bear here. And I think it's a big factor in sort of helping people build out their solutions and we'll continue to look at other verticals as well. And then the last point, you know, as we mentioned, I think more and more of the capabilities that we're building, we are enabling to run on other infrastructure, right? And so I mentioned Sunk already, our caching service for our storage platform can run on other

• 44:19Speaker 02

  infrastructure. And so most of these are based on just Kubernetes operators, which means an edge environment, an on-prem environment, like these also have the opportunity to leverage these same services and take advantage of them in those types of environments. And so, you know, even our services, our software solutions, our ways of advice, the solution, all can be brought to bear in those environments as we see some of that evolving and maturing.

• 44:43Speaker 01

  That is super cool. As we start to wind up, longtime viewers and listeners will know that we ask you to start waxing poetic a little bit in the last question. And that is, we always like to get your, what we describe is like, it's the day is over, you've chilled out, however you chill out at the end of the day, you know, you're kind of getting in bed, your brain's spinning on different problems and what might be. And if you can share a little bit of your vision, your dream about where you think things are going, it might be for CoreWeave, it might be for the broader industry or both, but just like what's in your head for like, for, you know, several years out where it's not on your product

• 45:28Speaker 01

  map, you know, I'm not asking for what's your next release. I'm asking for like, where are things going in the large in your view? And recognizing that just as I will do this and get it wrong regularly, it's fine to get it wrong. But I love to hear people thinking at any given moment about where things might go, whether you get it right or wrong. So what's yours? The I'm chilling out in bed before I go to sleep?

• 45:50Speaker 02

  Yeah, I mean, I'd probably say, and I talked a little bit about it already early on the call, but like maybe double down on it. Like, I think the progression that we saw with public cloud to the point now where every enterprise, every company in the world has a public cloud engineering team. Like, it's like, in fact, they've far exceeded their, you know, on-prem engineering team in most cases, right? There's obviously exceptions to that. But for the most part, like everybody, and you're sort of, you know, every single application or experience out there has some backing in the cloud. We, I think we will go through the same progression with AI and probably faster to the point where we will

• 46:35Speaker 02

  have AI engineering teams at every single enterprise company and they will, probably sooner rather than later, exceed the amount of people who are on the sort of cloud engineering team, right? Cloud engineering teams will be legacy and the AI engineering team, and they will be the supporting team for the AI engineering team who's going to build the AI applications. And AI applications, I think, will far exceed our current interaction model with AI applications, right? Like, I think, and I'll use sort of, again, the cloud as my example, like, like the initial wave of cloud was click a button, click a button, click a button, right? But now, we record podcasts on cloud-based experiences, right? We, we watch, you know, World Cup games on, on cloud-based infrastructure. Like,

• 47:19Speaker 02

  I prefer that over going to my TV and being like, gosh, what channel is this on, right? It's like, and so that transition, like, will be much more than just I chat to an agent, right? It will be, it will be visual response to the face you're making to your screen when something comes up. It will be, you know, human interaction models that will far exceed what we experience today and will replace what we know of as modern today. And I think it's going to happen much faster. Cloud took, what, maybe 20 years, 15 years to kind of really get there? I think we'll probably half it, right? I think within, you know, five to seven years, that's going to be our world. And websites with, you know, buttons that you click are going to be, are going to be legacy. Like, I can't

• 48:03Speaker 02

  believe they haven't, they haven't AI'd this yet, right? Or whatever the verb may be. And so that, to me, is the direction and the requirement there for a company like CoreWeave, but frankly, for a company, all the companies in this space, is how do we enable that progression by democratizing what's required to get there? Because right now, those enterprises don't have those people. They don't have those frontier lab researchers, right? They don't have the talent that they need to be able to be innovating in this way. And so they're limited by it. And so, just like they didn't have the cloud people in the original world, right? And so, they will ramp up, they will train, they will skill, they will grow, they will hire, right? That's going to happen. But in parallel, just again, like the cloud

• 48:45Speaker 02

  wave, we have an obligation to democratize the services so that more people can deliver upon that value for less. And that, to me, like, I think of why I'm here, and I don't even mean here, Corey, I mean, in the industry right now, that is why.

• 49:00Speaker 01

  That is very, I really appreciate that insight. That was, not only do I definitely think that you're on to something there, but it's interesting to listen to and fun. Thank you so much for coming on the show. That was a great conversation. I learned a lot. So Corey Sanders of CoreWeave, hope to have you back again.

• 49:20Speaker 02

  Appreciate it. Thanks for the time, Chris, and I'll be come back whenever you want.

• 49:28Speaker 00

  All right. That's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Blue Sky. You'll see us posting insights related to the latest AI developments, and we would love for you to join the conversation. Thanks to our partner, Prediction Guard, for providing operational support for the show. Check them out at predictionguard.com. Also, thanks to Breakmaster Cylinder for the beats and to you for listening. That's all for now, but you'll hear from us again next week.