[AIEWF Preview] CloudChef: Your Robot Chef - Michellin-Star food at $12/hr (w/ Kitchen tour!)

One of the new tracks at next week’s AI Engineer conference in SF is a new focus on LLMs + Robotics, ft. household names like Waymo and Physical Intelligence. However there are many other companies applying LLMs and VLMs in the real world! CloudChef, the first industrial-scale kitchen robotics company with one-shot demonstration learning and an incredibly simple business model, will be serving tasty treats all day with Zippy (https://www.cloudchef.co/zippy ) their AI Chef platform. This is a lightning pod with CEO Nikhil Abraham to preview what Zippy is capable of! https://www.cloudchef.co/platform See a real chef comparison: See it in the AI Engineer Expo at SF next week: https://ai.engineer Full Video Episode Timestamps 00:00 https://www.youtube.com/watch?v=t_MQ1Ms_Zp8 Welcome and Introductions00:58 https://www.youtube.com/watch?v=t_MQ1Ms_Zp8&t=58s What is Cloud Chef?01:36 https://www.youtube.com/watch?v=t_MQ1Ms_Zp8&t=96s How the Robots Work: Culinary Intelligence05:57 https://www.youtube.com/watch?v=t_MQ1Ms_Zp8&t=357s Commercial Applications and Early Success07:02 https://www.youtube.com/watch?v=t_MQ1Ms_Zp8&t=422s The Software-First Approach10:09 https://www.youtube.com/watch?v=t_MQ1Ms_Zp8&t=609s Business Model and Pricing13:10 https://www.youtube.com/watch?v=t_MQ1Ms_Zp8&t=790s Demonstration Learning: Training the Robots16:03 https://www.youtube.com/watch?v=t_MQ1Ms_Zp8&t=963s Call to Action and Engineering Opportunities18:45 https://www.youtube.com/watch?v=t_MQ1Ms_Zp8&t=1125s Final Thoughts and Technical Details This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe https://www.latent.space/subscribe?utm_medium=podcast&utm_campaign=CTA_2

• 0:03Speaker 00

  Okay, we are in the remote studio with this very special podcast. We actually recorded a while ago a tour of Cloud Chef's kitchen, but we wanted to record a little bit of an intro in our remote studio so that we at least get a nice audio podcast intro to the company. And we're here with my friend and co-host, Vibhu Sabra, and as well as Nikhil, who is the founder of Cloud Chef. Welcome Nikhil.

• 0:28Speaker 01

  Thanks for having me, Srik.

• 0:31Speaker 00

  Okay, so yeah, welcome back Vibhu. But I think by the time this launches, like, people will have heard the Anthropic podcast that we did. But yeah, so I think the headline that people will see when people see Cloud Chef is that it is an AI chef. You had this like pretty viral video on Twitter recently, you know, when you launched and told everybody. But also, people don't know that this is a real restaurant. Like, you actually run a real restaurant. You can order food on, I think Uber Eats.

• 0:57Speaker 01

  Yeah.

• 0:57Speaker 00

  And it's really good. Yeah. So like, what is Cloud Chef? Like, what is the scope of it? How do you pitch the company?

• 1:02Speaker 01

  At a very high level, what we're trying to do is we want to make high-quality nutritious food available to everyone. And the reason why it's possible to even think of a future like that is because you can automate practically all non-managerial work inside a commercial kitchen with culinary intelligent robots. And culinary intelligent robots is basically just robots that

• 1:28Speaker 01

  act like human beings, learn like human beings, and work like human beings, or work like human chefs. So we actually have our first, the video that Sean was talking about is the launch video of our first robot. And basically, it's a robot that has a mobile base, has two hands, goes around and does work inside a kitchen. So it's just like how you would hire a human employee or a human chef. You would hire our robot and the robot will come to your facility, cook, and come to your facility, learn recipes from the chefs inside the facility with one single demonstration and cook that dish over and over again, or participate in that workflow over and over again, like a human employee would.

• 2:11Speaker 01

  And then you pay the robot an hourly wage, like how you would pay a human. And so far, our robots are used by like Michelin star chefs. They're used by...

• 2:22Speaker 01

  Fresh fast food restaurants, airline caterers, a whole bunch of like commercial facilities use our robots as early wage labor as compared to

• 2:34Speaker 01

  as early wage labor as compared to buying a robot. The thing that makes our robot special is the fact that it can execute at chef level or the fact that it has culinary understanding better than even the best chefs in any single cuisine. Like what that means is, okay, if you if a robot is going on, if a robot is cooking, it needs to know how brown the onions are, how far along you are in the cooking process, if you're cooking it in a slightly different appliance, like what state is the recipe in, how much heat do you give it? All this thermodynamics modeling of cooking, understanding visually what's going on.

• 3:15Speaker 01

  This is what we call culinary intelligence and this is something that was not possible until like recently when like multimodal models got good enough and we basically built out some thermodynamics modeling to aid that and now the end result of that is a robot that can reason and make decisions in the real world, in cooking processes like a chef would.

• 3:44Speaker 01

  With more and more robot foundation models coming up, then getting better, these robots are finally also able to do real actions or real motions inside a kitchen. Right now, they're good enough to only do stuff like gross manipulation where you, where you don't like if a human requires more than two fingers or three fingers to do a task, the robot's probably not able to do it. But the good part is most tasks inside a kitchen can actually be done with just two fingers. So if you go around any commercial kitchen and if your two fingers had enough strength, you could probably do most tasks inside that kitchen. So we start with line cooking, which is like the biggest labor cost for restaurants and restaurants and other food producing facilities.

• 4:26Speaker 01

  And our robot is able to do line cooking for about forty to fifty percent of the world's commercially valuable cuisine to a point that if we put our robot against an expert chef in that cuisine, our robot is able to consistently make the Food better than even the chef whose source recipe it is like it's something that computers just do inherently much better than like the human brain. So that's a quick overview on this. Basically, we've trained our in-house models to we've trained our in-house models to do like the more dynamic perception and we've we leverage current V L M's and voice models to

• 5:09Speaker 01

  do perception and also to enable the robot to do tasks or for the robot to actually talk to human beings, interact with other coworkers in the facility to course correct its goals and whatnot. So that's a quick overview on what we are. The high level of goal, like I said, is to replace all non managerial work inside commercial kitchens with culinary intelligent robots. And when that plays out, we think we'll all live in a future where we all have access to really high quality food at fast food price points. At McDonald's price points, you should be able to eat the tastiest food that you've ever had in your life. That's the thing that we want to create. And we think that now that the robots have started to work in the real world, we see

• 5:54Speaker 01

  a future in which we see a soon enough future in which we'll make that possible. And to Sean's earlier point, we actually started experimenting with these robots in our own facilities. So we basically just built an in-house delivery kitchen at our office in Palo Alto. We weren't expecting it to do this well. I mean, it just like picked up really well on DoorDash. We just and we had moved from like my me and my co-founder, we had moved from India to Palo Alto and we were actually just missing really high quality Indian food here. So we just went to our favorite restaurants in Bombay and Delhi and asked, can you record your recipes? We'll serve them in California and we'll give you a royalty. But that's not the core business that we're focusing

• 6:36Speaker 01

  on. It's just something that we use to validate our technology and the fact that it's doing so well and the ratings are so good. This is like testament to how the tech is and how good the robot is functioning right now.

• 6:48Speaker 02

  We tried the food, we'll see later in the videos, really, really good food. I like the term you used there, artificial culinary intelligence, A C I has been achieved internally.

• 6:59Speaker 01

  Yeah.

• 6:59Speaker 02

  It's interesting though, because when you, when you frame it like that, you guys are doing something pretty different than like robotics, right? Like you mentioned that the robots are more so off the shelf parts, it's not like specialized robotics, it's actually the software underneath, right? So, yeah, we can talk a bit more about that.

• 7:14Speaker 01

  Correct. So when we started the company, we had one core ideology, which is that we will only solve problems that can be modeled as software problems. And culinary intelligence was the first like big, like culinary intelligence and decision making was like the first big open problem that we could model in software and solve. But when we had started, robots, you, robots were still like very much an electromechanical problem, like they weren't really a software problem. And now with like robot learning models and robot foundation, robot foundation models. It has gotten to a point where you can now start modeling the physical actions that somebody does also in the software domain and solve it in software and have software iteration cycles.

• 7:56Speaker 01

  And we didn't want to build any hardware, like we didn't want to be a hardware company because that was not our, like, we, we, we weren't good at it. So we just, we just didn't want to do hardware and we didn't think that the hardware hydration cycles would be beneficial for a company like this. Very recently, it has gotten to a point where you can just take off the shelf parts, put a bunch of like, uh, robot intelligence, quote unquote, I'll expand on that later, and get the robots to work. And that is a software iteration cycle, you don't have to build your own hardware, you don't have to get into what motors, like how do you, how, uh, how do we manufacture our motors? How do we manufacture these robots? How do we design? Like all, all those open questions, we rely on the ecosystem for us to solve.

• 8:39Speaker 01

  And then we basically source general purpose robot parts and, or general purpose robots. And these general purpose and write software on top of these general purpose robots. So we take general purpose robots, we leverage all the general purpose intelligence, like L L M's, L L M's, V L M's, robot foundation models. And then build this proprietary culinary layer on top, which has everything to do with like the thermodynamics modeling of cooking, custom evals for like manipulation, understanding, like, uh, understanding through perception, what stage of the cooking process you are in, all of those things we've. Basically built and we are hoping to serve the tide on both the

• 9:21Speaker 01

  uh advances in multimodal models and robot foundation models and use general purpose robots as like the vehicle to make that happen. So that's that that's in a nutshell how the uh our approach kind of works and we are very focused on like, like I said, modeling every part of the workflow as a software process and and now that we are able to do it, we are able to basically do full stack work inside a kitchen and not just be a assistant robot that can be prompted by somebody on site or just a guidance system that tells humans what to do like it's now able to do full stack work because the entire workflow can be modeled as a software process.

• 10:03Speaker 02

  Uh I

• 10:06Speaker 02

  I was gonna say, I know that like, you know, we we see how the robots work and what they're doing under the hood later. But uh the one, I guess, like overall question that I'm sure a lot of people have is like, what's the business model? How do people kind of hear about this? How do you see like, you know, rent a robot for twelve dollars an hour versus hire a chef? How do you how do you come up to this hourly rental and all this stuff? It's very cool to see and you know, good to see it works. Yeah, I'm just curious, how how does that side of the business?

• 10:31Speaker 01

  So for a business perspective, the main thing to keep in mind here is that like food prep is the most labor intensive industry of all labor intensive industries here and they can quantitatively, the way you measure it is how many full time employees do you need per million dollars of revenue generated? Food requires about thirteen people per million dollars of revenue generated and the second most labor intensive industry is hospitals, which require four people per million dollars of revenue generated. So food is like three times or more than three times as labor intensive as the second most labor intensive industry. Labor costs are just like going through the roof and labor costs have been increasing year year over year and depending on like what Trump does with illegal immigration could could even

• 11:16Speaker 01

  higher and like the the start of turnovers are really high. Uh the average restaurant is operating at like one thirty percent staff turnover. By the the end of ten months, you're like practically your entire staff is new. So high turnover, very high cost and the most labor intensive industry. And because the reason why we landed up At this price point or like this sort of pricing models because food service is not a very profitable industry. So they don't have free cash just lying around to do experiments. And there are no fixed, there aren't like fixed budgets set out for buying new robots, testing things out. If it doesn't work, it doesn't work. They don't take that sort of an attitude.

• 12:00Speaker 01

  Whereas there is a very readily available labor budget that we can tap into. Just like when you hire somebody, you don't pay for their college tuition, you just pay them a salary. We thought like, why should that be any different for robot robotics? And these robots are now cheap enough to a point where you are able to put that business model out there. Not lose money every robot you sell. So the robot costs, basically the point is the robot costs have gotten to a point where an hourly labor price, pricing model works. And the robots are also good enough to now do the entire chain of work so that it's possible. And at twelve dollars an hour, it's like forty percent of what loaded human cost, uh, would cost.

• 12:42Speaker 01

  So our customers get their ROI on day one, the robot starts working from day one and over time, these robots just get better. The hope is that at some point, they also even start making better, like the food at any given facility that they're cooking substantially better. Not just by cooking the same thing, but then enabling the facility to make uh, recipes that they weren't able to do earlier.

• 13:02Speaker 00

  Yeah, awesome. I think the, the, the last part, you know, we'll, we'll go, we'll cut right into the, the walk, the kitchen walkthrough video later. But the, the last part I think was, I think there's this general goal of demonstration learning, right? Learning from experts, learning from the Michelin star chefs. How realistic, you know, is this like a marketing promise or do you really just learn from one example? Cause like, I mean, obviously food is messy, right? Food needs a lot of different demonstrations. How realistic is this?

• 13:27Speaker 01

  So I, I want to clarify that the, I want to, I want to clarify two things. One, it is not a marketing thing. It's actually true. Two, the reason why it might feel counterintuitive is because our entire pipeline is not one end to end model. Like if you had one end to end model and you had to like, uh, train it to do, do a new thing, like being a one-shot learner is a, is a very big deal, like. But in our case, we have many AI subsystems that work with each other. Some are like, some of those are end-to-end neural networks, some of them are like hard-coded software pathways. So we basically use the best of both worlds to.

• 14:07Speaker 01

  Function and sort of architecture choice means that we don't have to like, we're not going in from like pixels of what a chef is doing and text or whatever to directly a generalizable recipe that can be cooked across any robot, any time scale. There are like software workflows and pathways before that that take this chef demonstration, convert that into chef demonstration, convert that into like an intermediate format that is easily digestible by different parts of our system. And yeah, I mean, one example for you would be, so say if you're like making an omelet and if you want to teach her, teach her how to make an omelet. What we are doing is we're not learning a new omelet making skill while we're, while you're showing the robot the sick.

• 14:51Speaker 01

  If we had that capability, we would be a robot foundation model and we would uh already be single shot learning. It is that from a single demonstration, assuming that we have all the base skills for the robot to do it. It is extracting what kind of decisions the chef is making, is it visual, is it thermal? What kind of skill the chef would be in, is invoking in themselves, is it like stir, sorta? And what are the parameters to those skills? So for us, learning is basically configuring this AI system and not going into a end to end model that's going directly from pixels to robot actions. We wouldn't be able to do a single shot recipe learning. We would have to have the chef cook the recipe in various different backgrounds, various different sizes, various different appliances.

• 15:36Speaker 01

  Because we have these like engineered midpoint, like engineered engineered midpoint go from one expert demonstration to uh uh from uh to a recipe form that can then be recreated across different kitchens, different appliances. And uh in the future, also different robot morphologies.

• 15:56Speaker 00

  Oh, other, other robot morphologies is fun. Yeah, now you only do the two arms, right? Okay, so we'll get, we'll get people to call to action and then we'll, we'll cut to the video. You are going to be at the AI Engineer World's Fair next week. Um, well, if people want to see the robot live, they can, they can see it there, probably taste some food, although I don't know how much food we can serve.

• 16:14Speaker 01

  Yeah, we'll see.

• 16:16Speaker 00

  We'll see. And obviously, I think part of the reason you're doing this is you're trying to hire, right?

• 16:20Speaker 01

  Yes.

• 16:23Speaker 00

  This is a like immediately applicable use case like you know what's the what's the what's the pitch for engineers?

• 16:30Speaker 01

  Pitch for engineers is that like there are only a handful of applied robotics companies that have a path to deploy more than a hundred robots in the next one year. And now that our robot is working and we have early signs of it being super helpful to customers, you will actually be working on a robot that is in production. There are people in robotics who are working on more complex hardware, more complex hardware problems, more complex software problems. But I think we are at the efficient frontier of value being delivered to the customer using cutting edge techniques and having a rapid scale up pipeline. I think that's not a lot of companies can say that they have all these three things.

• 17:13Speaker 01

  And cooking, like I said, it's I think we have a very powerful mission in the sense that today the food that you're eating is fast food, most cheap food is fast food. Whereas fast forward 10 years, you can eat very high quality food. And in fact, if you work with us, you can already eat very high quality food in our office as Sean and Vivu can confirm. But I think coming back to it, I think the mission is very powerful. I think if you are excited by creating value in the real world while also doing it in a way that serves all the current capabilities of the state of the art general purpose models, you're probably one of the maybe two or three companies that are at the intersection of that. And if that excites you, you should come up to me.

• 17:56Speaker 00

  Yeah, I think that's actually a pretty very, very strong pitch. I would say that you can actually even just try the food on the on Uber Eats. You can just kind of order here. It's one of those like virtual cloud kitchens and it looks so good. We've tried it. We'll cut to the video later, but yeah, thanks for jumping on and sharing your journey with us. I think this is very exciting. I think you've like somehow found the way towards like the most immediate applicable industrial use case of robots and like and and there's obviously a lot of scope for vision language models and solving a lot of hard engineering problems. Like and one thing you did say is like, oh, this is within very tight engineering parameters, which I think is is is pretty hard. Like you have to run it, run it a lot of frames per second and also like do a lot of that on device.

• 18:41Speaker 00

  So yeah, cool. Well, I'm looking forward to see you next week at the the conference and we'll cut to the video now.

• 18:48Speaker 03

  All the culinary decision-making is like 100% autonomous and the actions are 90% autonomous.

• 19:10Speaker 03

  Water has gone out or like the probability of all water going out is more than 90% and we have safety filters like that. That's what makes it deployable, otherwise it's not very deployable. So basically how appliances are controlled is we go into like any, so any appliance in any kitchen is controlled either using a knob or a touchscreen. So we go in, remove the knobs that control the appliance and then put these knobs in that can turn themselves. Oh. So every, that gives us a activation surface across all appliances. So we don't need to teach a robot to, for a salary. Ah. Yeah, so the, yes. It's an hour.

• 19:55Speaker 03

  Yes, so 12 bucks an hour is what you pay this, no capex. What? Yeah. Oh, is there anything else that's going on, like any other equipment? I mean, for example, the basic, the other thing is all the ingredients that are required basically get measured in these weighing scales. Okay. And regardless of which kitchen we go to, all kitchens store their ingredients in boxes. We just slap, yeah, we just slap a bunch, bunch of two.