Training General Robots for Any Task: Physical Intelligence’s Karol Hausman and Tobi Springenberg

Physical Intelligence’s Karol Hausman and Tobi Springenberg believe that robotics has been held back not by hardware limitations, but by an intelligence bottleneck that foundation models can solve. Their end-to-end learning approach combines vision, language, and action into models like π0 and π*0.6, enabling robots to learn generalizable behaviors rather than task-specific programs. The team prioritizes real-world deployment and uses RL from experience to push beyond what imitation learning alone can achieve. Their philosophy—that a single general-purpose model can handle diverse physical tasks across different robot embodiments—represents a fundamental shift in how we think about building intelligent machines for the physical world. Hosted by Alfred Lin and Sonya Huang, Sequoia Capital

• 0:00Speaker 00

  Just like the fact that this whole thing works. It's kind of mind blowing. Right? Like you, you build this like loosely brain inspired thing that has very general purpose learning algorithm, you feed it data and it somehow gets it and gets it way better than anything we've ever had before. And this applies to robots and it applies to vision and language and sound and all kinds of other things. And like, I think if you stop for a second and just think about it, how it works and and that it works, it's just like absolutely mind blowing.

• 0:49Speaker 01

  In this episode, we sit down with Carol and Toby of Physical Intelligence, a company building foundation models for robotics. Carol and Toby explain why the classical approach of breaking robotics down into perception, planning and control was fundamentally wrong, and how end to end learning with reinforcement learning is finally making deployment possible. You'll hear how they achieved robust, real world performance, getting robots to make coffee for 13 hours straight, and how these models generalize across radically different tasks from surgical robots to drone flying in ways that we don't fully understand. We also talk about the technical insights behind Pi star 0.6, which is Physical Intelligence's newest model that learns from experience using reinforcement learning. Enjoy the show.

• 1:31Speaker 01

  Carl, Toby, thank you so much for joining us here today.

• 1:33Speaker 00

  Thank you for having us.

• 1:34Speaker 01

  Excited to talk everything, physical intelligence, general robotics, et cetera. Maybe before we get into it, just for our audience, can you share a little bit about what physical intelligence is and the mission that you're after?

• 1:44Speaker 00

  Yeah, so at Physical Intelligence, we are building robotic foundation models. These are models that in principle should have, should be able to have any robot do any task. And over the past one and a half years or so, we started building the, we created the right building blocks that show how these models could scale. So we've shown that they're able to control many different robotic form factors, many different types of robots. We've also shown that they're able to generalize. So you can bring it to completely new environments and what it takes for them to generalize. And this, this last release that we just had called Pi Star O Six, that we also wanted to tell you more about,

• 2:26Speaker 00

  shows how we can get them to a good performance so that they're starting to become deployable. And this is really important to us because we want to see this technology actually deployed in the real world. But also because we don't have the benefit of having the, the free data on the internet. There is no data of robot actions. So we need to create the data sets ourselves. So we are after the, the problem of physical intelligence, after the problem of creating foundation models for robots. And we've made quite a lot of progress.

• 2:58Speaker 01

  Wonderful. And can I ask why the decision to build foundation models as opposed to, you know, there are companies that are building fully vertically integrated robotic products right now. You know, the, the Sunday lunch last month is in the back of my head. You can buy a cute little robot helper for your household. There's companies working on cooking robots. There's obviously humanoid companies. Why build a foundation model versus build, build a robot yourselves?

• 3:21Speaker 00

  Yeah. So I think if you look at the history of robotics, it's very, very clear to me, and I think to many roboticists, that we've been always bottlenecked on intelligence. We've had robots that are capable of doing incredible things, whether it's in the home or in industrial settings. We've seen robots more than a decade ago that if teleoperated, they can clean the entire house. And the really important caveat is if teleoperated. So if there is a human mind behind it, it's clear that the hardware is capable of doing lots of different things. And for a very long time, robotics companies have been structured the way you described,

• 4:01Speaker 00

  where you kind of think of creating a specific robot that's designed to do just a single task or a single application. And instead, what we thought would be, would really help the field is to focus on the bottleneck, on the intelligence. So we created a company to focus on that bottleneck because we think that this is, if we, if we address that bottleneck, we can actually make robots happen. And if you do it any other way, you're basically not making as much progress on the bottleneck as you could be. So we thought we would just target this problem head on, focus on the intelligence. And if we can do that, that would lead to many different vertical products. It will lead to, you know, robots being deployed in a home, in, in industrial settings, basically anywhere.

• 4:44Speaker 01

  Can I just pressure that, test that a little bit? So on the hardware side, like I've seen the latest videos, for example, of the, the Optimus hand. It, it's like, it's exquisite, it's a, it's a piece of art. And I hadn't seen the videos of people, you know, teleoperated robots cleaning houses 10 years ago. But I'm wondering if there's a set of tasks that's, you know, maybe now just on the cusp of becoming possible. For example, cooking or like being able to, you know, peel and dice an onion that like you couldn't have done with, with hardware prior to where we currently are. So how much of a why now do you think hardware is or isn't?

• 5:18Speaker 00

  So there's a lot of progress in hardware, especially in humanoid hardware. Like dexterous hands, for instance, as you mentioned, they're, that, that, I think they're much better now than they were even a few years ago.

• 5:31Speaker 00

  But it still doesn't address the bottleneck. We could have had robots operating, you know, chopping vegetables or doing cooking even with simple grippers before. The problem is that we don't have the intelligence to operate these robots. And the more complex the hardware is.

• 5:49Speaker 00

  It it doesn't really resolve that bottleneck, right? Like, it it allows you to do more potentially, but you're still bottlenecked by the fundamental challenge of robots not being intelligent enough.

• 5:58Speaker 01

  I see. So hardware may raise the ceiling on what you're able to do, but like the the capability floor, we're not even there yet.

• 6:04Speaker 00

  That's right. So even with simple robots, we are not yet at the level of a human operator.

• 6:09Speaker 02

  So the limit being the intelligence layer, what's the limit to developing the intelligence? Is it collecting data? Is it doing it cheaply? Because, you know, you've broken down the problem. We're going to keep asking you why, why, why? And just drill down further. So what's the next layer of the, okay, what's the bottleneck for solving intelligence? Generalization?

• 6:29Speaker 00

  It's a it's a good question. So we thought about it in terms of three factors. We refer to them as capability, generalization and performance.

• 6:40Speaker 00

  With capability, our our idea was that we want to get to the point where, as long as you can collect data for something, for a task or for a robot, you should have a model that should be able to to replicate that, to automate that task. This is something that we've gotten to fairly quickly. This was our Py2 release around a year ago or so, showing that it's basically possible, that if you can collect data for any task, for any robot, you should be able to automate it and the model should be able to learn it.

• 7:09Speaker 00

  The next challenge is around generalization, and this is still an open challenge. So we wanted to get to the point where the robots can just work zero-shot and you can just bring them to a new home, for instance, and they should know how to operate in that home. And this is a really, really difficult problem, right? Like, if you if you put a robot in a new home, it needs to understand, you know, where different items are, that the counters look different, the lighting is different than what you've seen in the past and so on. And I wouldn't say that this problem is solved, but I think we start to get a handle on how to how to solve it and how it scales. And the only answer to generalization that we know in machine learning is through diversity of data. So if you see a lot of different diverse data sets, you should be able to generalize to to a setting that is similar to the one you've seen.

• 7:54Speaker 00

  And this is something that we've seen with our Py2.5 release in April of this year, that we we got to the point where we can bring a robot to a new home that it's never been to before and is able to to to operate in that home. It's not perfect yet, but at least it has some kind of common sense on, you know, how to go about simple tasks like cleaning up the kitchen and things like that. And then the last challenge that is also not fully solved yet is performance. So how can we get these models to the point where the performance is good enough so you can actually deploy them?

• 8:22Speaker 02

  Right.

• 8:23Speaker 00

  And deployments here are really, really important because, as I mentioned before, we also need to gather data. And I think that is going to be the most scalable way of collecting data because you'll have robots out there in the world doing economically valuable tasks. And that way, the cost of that data collection is basically negative. And the more broadly you can deploy this technology, the more data you'll be getting. And I think in the limit, that will be the biggest source of the data you can imagine, much bigger than internet data, for instance.

• 8:52Speaker 02

  And how far away do you think we are from generalization or from a performance level that maybe it's a controlled environment, maybe it's a general environment in homes or offices, but not the whole whole world? If you could limit that, where do you think generalization performance will will need to be before we can deploy these kind of robots?

• 9:14Speaker 00

  I think we are actually fairly close to deploying these robots. We started deploying them ourselves already. We thought this was something that was going to take something like five years to get to the point where the technology is actually ready to deploy a robot in a in a commercial setting and have it do something valuable. But we've done it, I think, two months ago or something like that.

• 9:35Speaker 00

  So I think we're now getting to that threshold that that the models are useful enough, they're performant enough, and they can do enough of variety of tasks to be actually useful. So that's a really, really exciting moment. I think we just crossed that threshold. I think it's still to be determined how wide is the aperture of where we can deploy. There are some tasks where the failure can be really catastrophic. Maybe these are not the best tests to deploy just yet. There are some tasks that require a ton of generalization, like deploying in the homes or that are, you know, have privacy concerns or safety concerns and so on. Maybe these are not the best places to deploy just yet. But I think there is that the aperture is growing. As we collect more data, as these models get better, we can deploy them in more and more settings.

• 10:20Speaker 00

  So I think we're starting to get there.

• 10:23Speaker 02

  Where is the current aperture that you're deploying right now?

• 10:27Speaker 00

  So we're actually, this is a really difficult question to answer because with these foundation models, sometimes you don't fully know.

• 10:36Speaker 00

  So kind of similarly to how with large language models, you know, you train this model, you kind of cook it in the house, you try to make the best job possible. And then at the very end, you get this artifact and you can't really predict how good the artifact is going to be. You kind of have to test it. And that's where we are with these models as well. So for instance, we open source them so that we are not the only ones testing it. And we're not the bottlenecking knowing what their capabilities are. And by open sourcing them, we see them being applied to actually many more applications that we could have imagined. Things like driving or surgical robots or agriculture and places like that.

• 11:20Speaker 00

  So I don't have a very good estimate of what the aperture is. I think it's wider than what I had expected. And I think it's also will be, it will be growing over time. The more data these models get, the more mature they get, I think the aperture will continue to grow.

• 11:34Speaker 03

  I would add maybe like on the performance level, like as you said, the aperture is probably wider. The starting point is wider than we thought. But at the same time, of course, if you actually want each of those starting points that you start out for each of those applications to be at a level where people would want to use this as a day to day driving, you know, their businesses. Well, that's probably still quite a bit of hill climbing to do in terms of performance, right? So with this release that we're going to talk about a little bit in a bit, I guess, the PyTorch, we've made progress on like learning from experience data, getting that back and making the models better when they are deployed. It's still for a lot of things that I can naively imagine that there'd be lots of scenarios where

• 12:18Speaker 03

  there's a really, really long tail of things that can go wrong or that you can encounter that we don't yet have a great grasp on like how to completely solve, I would say as well.

• 12:28Speaker 01

  And you guys have been really great about publishing your results with a lot of transparency, releasing it open source. So whatever you're comfortable sharing, can you, can you talk about what your overall technical architecture, so to speak, is? And do you think that the architecture to kind of get to this promised land is, you know, pretty much baked? And it'll be variations on the theme of where we are and we just need to collect a ton of data? Or do you think that, you know, the architecture is still, still being figured out?

• 12:55Speaker 03

  I would say, so we can maybe start with like a little bit discussing where we are at now. Yeah. And then we can like go into the details of like how that might change. So at the moment, you know, the the architecture is very analogous to how, um, you know, V L M's and are are built today is that probably, you know, most of you interact with on a day to day basis, right? Type something in and put a image in and ask it to read what's on the image and and so on. And and we've kind of like started from um the same standpoint of, you know, there's a model that's trained on internet scale data and it's ingested um image data and text. And and we're adding all this robotics data and our training actually predominantly now is on robotics data, on data that we have collected ourselves.

• 13:37Speaker 03

  We have a little bit of that internet data in the mix, but the majority of it is robotics data. The architecture is kind of this uh vision language uh model and we add something on the side, which is what we call uh the action model, the action expert. The the part of the model that actually then has to drive the robot, right? That basically uh looks at the image and the instruction is getting and and has to perform the task, has to send um commands to the robot. And so broadly, it's like a transformer model that is a a fairly large model up to like some billion parameters at this point uh that we use, that we pre-train on our robotics data and on on internet data. And uh it is trained largely in initially from human demonstration data.

• 14:21Speaker 03

  Carol mentioned this earlier a little bit and we we have this demonstration data, teleoperated data of humans trying to get the robot to do stuff. So that's the the architecture that it looks like now and like roughly the scaling um that we're getting is from scaling our data. And we use models similar to what comes from the V L M world. How that might change, I think is an is an open question. I I think there's lots of opportunities in in adding more capabilities to these models that we're also exploring, right? You can imagine that um, you know, you you might want more context in these models. You might want more um more more cameras added to to the robots that the model needs to be able to uh to use.

• 15:05Speaker 03

  You might want uh to have a better understanding of the physical world in the sense of. You know, understanding exactly what's in the room, what can break, what is easily movable and so on. So there's lots to be done, I think, in both capabilities and also changing the architecture around. And I I wouldn't be surprised if in like five, six years, we look back and we say, oh, you know, maybe the the backbone of the model that we used at the time, which currently is comes from this vehicle and has changed. Maybe we've moved on and we we use something slightly different. I think that will evolve over time. But I think the the foundation of like the data and how we bring it into the model will probably stay.

• 15:48Speaker 01

  Got it. And should I think about it as it's pixels or signals in and then actions out? Is that

• 15:54Speaker 03

  Yeah.

• 15:55Speaker 01

  And like like a single, single big neural net?

• 15:57Speaker 03

  It's one big model. Yeah, it's really just basically images in text in uh text out and and actions out at this point. Yeah.

• 16:04Speaker 01

  And are you, I guess, do you have a separate kind of locomotion versus manipulation stack? And maybe this this might be a good time to talk about kind of just the historical evolution in in robotics and the various different waves of learning uh and how it pertains to your stack.

• 16:21Speaker 00

  Yeah, so for a long time, even before learning arrived here, people thought that robotics is one of these problems where you can, if you put enough people on it, enough engineers, they can think really hard about it. And eventually write the code that will, you know, have the robot do anything in the world.

• 16:39Speaker 00

  And, you know, people have tried really, really hard to do it this way. And then it turned out that the world is just way too complex.

• 16:45Speaker 01

  Yep.

• 16:45Speaker 00

  Right? Like you can't just write every single, every single case you'll encountering the real world. So that doesn't work. And and also, as we were, you know, trying to work on on that version of the problem, what ended up happening is people did what they usually do. They tried to break down this problem into smaller sub problems. So like rather than working on the full robotics problem, you would say there was a perception aspect of the problem, there's a control aspect of the problem, there's the planning part of the problem. And this is almost growing to a different communities. There's a planning community, there's controls community, there's they have their own conferences, their own problems and all of that.

• 17:18Speaker 01

  Yeah.

• 17:19Speaker 00

  So then, as we realized that, you know, it's not really possible to to handwrite all of these rules, people thought that we should learn them. We should learn them from data, which is seems like a really good idea, right? This is how we learn too.

• 17:32Speaker 00

  But what ended up happening is that they started learning each one of those components, these these broke down components.

• 17:38Speaker 01

  Separately.

• 17:38Speaker 00

  Through learning separately.

• 17:40Speaker 01

  Yeah.

• 17:40Speaker 00

  So you would have a perception layer that is fully learned. Maybe you'll have a control layer that is learned. Maybe you'll have a planner that is learned. And that showed some progress. It was better than what we had before.

• 17:51Speaker 01

  Yeah.

• 17:52Speaker 00

  But then it turned out that breaking down these prob- this problem into these subcomponents, it actually is the piece that doesn't work. Because, you know, when I try to, you know, pick up this glass, I don't think about it in terms of perception and then planning and then control. I just, I just go for it. I just pick up the glass and it's just all very natural. So it turned out that the, that this pipeline approach where you have these predefined interfaces that like perception gives you the position of the object and then the planner gives you the trajectory and the control executes it, those interfaces are the pieces that that broke down. So everything that we thought we knew how we work was always wrong. So then we then arrived to the next stage of this where we said, well, maybe just breaking down this problem was a bad idea to to begin with.

• 18:36Speaker 01

  Yeah.

• 18:37Speaker 00

  So let's just train the whole thing end to end, right? So we'll take whatever, uh, uh, the, the, the, kind of the sensory inputs as as input to the network and we'll have actions as the output. That's what we refer to as the end to end approach, where you try to go straight from pixels to actions. And we'll, we'll have the network figure out or the learning algorithm figure out how to split it into these different components, if it's even possible.

• 18:59Speaker 01

  Yeah.

• 19:01Speaker 00

  And then while we were doing that, we figured that it actually requires a ton of data to do this. And often it breaks when it requires some kind of common sense and to gather that common sense through first, first person action data sets is really, really hard because you would need to experience every single thing in the world to do this.

• 19:21Speaker 00

  And that's where we stumbled upon vision, language, action models, where we can use models that were pre-trained on internet data that already have pretty good understanding of how the world works. And we can utilize that knowledge so that we don't need to experience everything firsthand. You can just add some action components on top of it and have a common world understanding and connect it to how to actually perform things in the world. And that's more or less where we're at today.

• 19:48Speaker 00

  Now, at Physical Intelligence, we figured a few other things. So how do you scale, how do you start to scale these models? How do you get them to generalize? How do you get them to perform much better? How do you have them move much faster? How do you get them to the point where you can start deploying them? But I think largely, we're still in this, in this era of how do you bring some of the common sense knowledge from the internet pre-training? How do you make these models very general so that they can work on any robot and perform motions?

• 20:14Speaker 01

  And can I ask for something like reasoning, right? There's, there's so much stuff happening in the, you know, reasoning side of the large language model space. Do you get the benefits of that? And as part of your VLA backbone, do you have reasoning kind of emerge as a consequence of what you're doing as you train these end to end? Or, Kesh, I think about, you know, some of the benefits of what's happening in the LLM world. Do they, do they benefit you or not?

• 20:35Speaker 03

  I mean, I think definitely the models that we have today, they are already planning actions, not just at a, what is the immediate action? But kind of what is the, what are the next 50 things I, I need to do, right? So like the next 50 time steps in, in some sense, it's, it's a very short horizon. 50 steps means like a, like a second or, or two, right? And it also additionally kind of decomposes tasks into subtasks in language space already. So when you, when we ask it, oh, clean the kitchen, the first subtask it might pick out to do is like, oh, I have to drive to the counter and then I have to like, blah, blah, blah, blah, pick up the, the glass, move the glass into the sink. So it already has those aspects in, in some sense, right?

• 21:19Speaker 03

  So like it decomposes task into subtask because it gives itself its own top task. And it predicts like a little bit of a horizon of how actions go. So, so some of it is already there, I think.

• 21:31Speaker 03

  In the future, there will probably be more of it. I do totally expect that. You know, all the advances on like R L training for reasoning, all these things will will also make their way into robotics.

• 21:41Speaker 00

  Yeah.

• 21:42Speaker 03

  Um. And I think it's kind of interesting to think about because it's it's maybe a little different than than the R L for math problems that people do, for example, right? Because I think those are very easy to for easy for us humans to think of as like textual problems, right? You think through them in your head in like text. Okay, if I change this formula this way, I will get this outcome and so on. I think for the physical intelligence part of it, it will probably be a bit more than that, right? It's it's gonna be a little bit different when you try to learn a new sport, for example, when I I recently started to try to learn how to play tennis. And, you know, I don't think through in my head of like, I need to now grab the racket, I need to move it here, and I need to do this swing.

• 22:27Speaker 03

  But it's more like you think through the motion itself, right? I you you think about like, how does your body move? How maybe um maybe your plan in some sense trajectories of objects around you in your head? And so those things, I think, we'll see come into the models more over time.

• 22:43Speaker 00

  Yeah, I I I suspect that over time. Right now, we're in a place where we benefit quite a bit from vision language models. I think it's it's very, very likely that that that's gonna reverse. Hmm. That a lot of the the shortcomings that we see in L L M's today are kind of baked in are because we are focused on on the text problem, on problems like math and coding. Yeah. And I think robotics would uh will offer this this new avenue. Yep. Where you need to kind of rethink how how to think about reasoning. Hmm. Reasoning should probably happen in some kind of abstract space where, you know, you can reason a little bit in text, you can reason a little bit in images, maybe you can reason in trajectories or in, you know, all kinds of different spaces to arrive at the answer.

• 23:29Speaker 00

  And robotics provides this really nice test bed where um you're grounded in the physical world. There is not that much data yet, so you kind of need to deal with some of the the difficulties that come with that. But I think it will provide for for new findings that will then be reapplied to to the L L M world.

• 23:52Speaker 02

  Speaking about data, give us a sense of, I don't know how you measure the sort of magnitude of data you've already collected and how much you would like to collect to the next year. Not, I'm sure more is better, but like, what is the magnitude we're talking about?

• 24:07Speaker 00

  Yeah, data is, it's one of those things that is actually fairly nuanced. It's not just a matter of quantity.

• 24:13Speaker 02

  Yeah.

• 24:14Speaker 00

  Quality obviously matters, but also things like diversity. And even when you think about the quality or diversity of robot data, these are not very strictly defined terms, right? Like if you, if you go for the same tasks in like 10 different ways, is this diverse data or not? Or how do you compare it to the diversity of the data if you go for like 10 different classes?

• 24:35Speaker 02

  Right.

• 24:37Speaker 00

  So this is something that I don't think we as a community fully understand, like how to characterize the data, how to describe diversity, how to describe the quality of the data, how to make it very, very rigorous.

• 24:50Speaker 00

  And we're also finding out that those, there are some aspects of the data that really, really matter. Like for instance, if you want to get to a certain performance on a task, you're not going to get there by just increasing the quantity of the data you already have. We've been working on these three different tasks for the PyStar 0.6 release, and we've noticed fairly early on that if we just keep on collecting more and more data, the same way that we've been collecting so far, the performance plateaus, you're not going to just keep on getting better. So you need to find either new ways of collecting it, or you need to start thinking about what kind of data will result in better performance. And this is where things like reinforcement learning and things like this can really, can really, really help.

• 25:36Speaker 01

  Let's talk reinforcement learning and let's talk

• 25:38Speaker 00

  Let's do it.

• 25:38Speaker 01

  PyStar 0.6, is the star a nod to Q-Star or?

• 25:44Speaker 03

  Yeah.

• 25:44Speaker 01

  Okay.

• 25:45Speaker 03

  Effectively, yeah, trying to get to like policy star actually optimal.

• 25:48Speaker 01

  Policy star, okay, wonderful. Okay, maybe just say a word on what you guys are doing with PyStar 0.6 and then we can dive into what RL means for your world.

• 25:57Speaker 03

  Yeah, for sure. So, I mean, I think the main, if we want to contrast it to what we talked about earlier, the main difference is that up to that point, basically. All of the robotics foundational learning that that we've done was basically demonstration data, teleoperated, going into the model, the model is trained, kind of like, just imitate that data, right? And now with this new model by Star or Six, what we're using is basically RL from experience that the robot collects itself by actually running a policy. So we start with the initial policy is this demonstration trained policy and then you deploy it. You try to actually have the robot solve the task. And then it additionally gets kind of reward signals given by humans.

• 26:40Speaker 03

  And it can also get corrections, so where the human intervenes and says, oh, actually, you know what? This is not right. Let's let's do this a little differently. And that data, that process, basically that data is collected, gets, comes back in and the model kind of uses that data to try and figure out which of the data can I, kind of, kind of like, should I reinforce? Should I do more of and which of it should I do less of? And and basically improve itself over time, basically. That's kind of the big distinction. And having that stream of real data coming in is kind of the missing piece that Carol was talking about that allows us to now escape this plateau that we would otherwise, we were finding, we were kind of like getting to.

• 27:19Speaker 01

  Yeah, and I guess in my brain, I I think VRL is, you know, you're you're hill climbing on your reward signal. And so how do you make sure you're, how do you make sure you're generalizing as you as you hill climb on these specific tasks?

• 27:31Speaker 03

  The way we're thinking about this for for this specific kind of problem is like, you have this sort of general model and it achieves some performance that isn't isn't great. And now your first goal actually isn't to further generalize. You want to kind of solve this specific task first, right? Like, so we deploy it and we have, we've picked like three, four tasks. So it has to generalize across tasks, nonetheless, the method has to generalize. But when you're actually kind of deploying it and trying to start this RL process, you really care about, let's make sure I nail down this task and I kind of nail it down in a way where I can, where I can can solve it from many different positions and I can deal with all the long tail of failures that I that I will encounter, right? So in some sense,

• 28:15Speaker 03

  The the generalization and the performance here may seem at odds when you look at it from like, oh, wait, but now you're like just doing this one task. But um but really, at the end of the day, what what what we want to do is we have the same method, the same process that deploys to each of these tasks and then kind of gets the performance high. And then we can have all of that data across all of these tasks and we can bring that data back, basically, right? So in in in that sense, it's not actually at odds, if that makes sense.

• 28:43Speaker 01

  Yeah, that makes sense. How much of the RL are you doing? It sounds like this is a this is in real life RL. Can you talk a little bit about the approach to how much RL you're doing in SIM versus in real life?

• 28:55Speaker 03

  Um, so we have taken a quite uh like real world first approach as opposed to uh using SIM. We are exploring SIM, of course, uh as well as uh as a research tool. But all the RL we've done for the PyStar six paper is actually on real systems in the real world. And the reason for that is that it's actually really, really hard to model. Again, we can we can get back to the long tail of failures that you see when you when you do deployment. I can give you a lot of examples from the tasks that we've actually looked at for this release, where there were failure modes that we we saw that if you had just done done a simulation of it, you might not have seen it. So to give you an example, um we have this one task, which is uh you have to build a box, right?

• 29:38Speaker 03

  So this is an actual deployment task where the goal is um we build these little uh cardboard boxes to put chocolate into such that uh they can then be packaged up and and and and sent out, basically. So that's building a chocolate box, basically. Um, and building this box uh initially, you know, worked great. And then there is these new shipments of boxes coming in and they come in as like a flattened sheet of cardboard. And then these cardboards that came in in this new shipment were kind of not perfectly perforated, so they were sticking together, right? And then the robot starts like grabbing them, puts them on a table to to to try to build this box and like has two boxes suddenly on the table, right? And this is something that wouldn't happen in SIM if you had written like a nice simulator

• 30:22Speaker 03

  that where you would just get individual cardboards and like fold them. Um, and so now you have to deal with this problem, right? And if you just learn everything in SIM and then try to deploy it, you wouldn't encounter it. So we encounter it and then our um kind of. Method can kind of figure out that, oh, actually, what I need to do is I need to separate this. And I can, I need to move that, that second piece back and build the box.

• 30:45Speaker 00

  And we see a lot of successes for RL being applied in sim and transferred to the real world, especially in locomotion. And we, we haven't really seen that kind of success in manipulation for, for these kind of methods. And I think maybe one reason for that is that with, with locomotion, with trying to move around. It seems that the, the biggest part of the problem is modeling your own body. So if you can figure out how to model you yourself as a robot, you're basically like almost there. So you can do this modeling simulation exercise once, because you only can do it, you only have to do it for you yourself, for this one robot. And then you're basically done. If you do it really, really well, it should transfer.

• 31:28Speaker 00

  With manipulation, however, the problem is not how you move your own body, it's how the world reacts to it. You're actually changing the world around you. It's not difficult to figure out how to move your hand from A to B. It's difficult to figure out how this affects the objects you're interacting with. And now the problem is no longer just modeling your own robot. You have to model the entire world, right? Like every single object that you might be interacting with, every single task you can think of. And that's where we see scaling problems. And that's, I think, why we haven't seen those kind of methods be as effective in, in manipulation.

• 32:03Speaker 01

  What was the headline of the results from PyStar 0.6? And, you know, where, where did you see the model get after RL on the, on the task that you cared about? And what do you think that means about your overall training recipe going forward?

• 32:16Speaker 03

  Yeah, so I think for me, the most impressive thing, honestly, for me personally to see was just how these models run for hours at a time, recover from lots of different failures. And, and basically just keep going. And at the same time, do that at a, at a, at a rate that is actually much better than the initial model that we started with, right? So the headline figures were, we increased kind of throughput of the policies by over two X on, on these three tasks. So there's one task was this box splitting task I already talked about. Was the making coffee with an actual kind of industrial scale espresso machine. And the other one was kind of like folding laundry.

• 32:54Speaker 01

  Yep.

• 32:54Speaker 03

  And so for each of them, we managed to like make the base policy that was trained just from demonstrations much, much faster. And also make it be able to recover from failures much, much better. And so seeing that actually in action, when you you you sit there, right? We have, if you go to our website, you can look at the videos. We have the robot serve coffee for thirteen hours in a row or fold laundry for four hours, things like that. Actually, seeing that life changes the way you think about these models. You know, changes the way, at least I think about it actually being realistic that we can deploy them, that we can do it in a way

• 33:35Speaker 03

  where it's not just a toy demo, which is shown once, but is actually kind of doing the real thing fully.

• 33:41Speaker 00

  And that's been really a challenge in robotics that I don't think many people are aware of.

• 33:45Speaker 03

  Yeah.

• 33:46Speaker 00

  Like, you know, you see so many videos of robots doing cool things. And, you know, we post these videos too. There's basically like anything you want robot to do, there's probably already a video of a robot doing that.

• 33:56Speaker 01

  Yeah.

• 33:57Speaker 00

  But, you know, you can take as many takes as you will, as you want. You can, you can keep on recording until you get the perfect shot. And the problem that I think everybody encounters is the reliability of these models. How performant they are, how fast they can, they can go about the task, how, for how long you can actually deploy them without failure. And I think this is the biggest bottleneck in terms of deploying these models in the real world. Because, you know, if you, if they break every, every other trial, they're not really deployable.

• 34:27Speaker 03

  Right.

• 34:28Speaker 00

  And this is, this is, I think, the most important breakthrough for us with this Pi Star O six release that we can actually start getting to a place where they are deployable.

• 34:38Speaker 01

  Yeah.

• 34:38Speaker 00

  Where we use these robots in our office to serve us coffee, or we can give them to, to people at Pi to fold laundry in their home. Or we can deploy them and have them fold boxes for real. And that is really, really exciting.

• 34:51Speaker 01

  Should we think about what you guys are doing with reinforcement learning as primarily a, you know, a customer deployment reliability?

• 35:00Speaker 01

  Points then, like you can now make sure that you can, you know, go reliably deploy the coffee making model on a customer site and it's going to be fast enough. It's not going to fail over long time horizons. So it's more of a customer deployment innovations versus like a fundamental kind of capability innovation, or is it both?

• 35:20Speaker 03

  I think it's both. I think, I mean, Kyle, you said this a little bit earlier. I think to some extent, the robots that we really, really want, right? The robot that you want as home, which can do your laundry, do your dishes, cook for you, drive around, and also the robot that people want in these smaller businesses, maybe solving a real problem that they have that they don't want to automate with a classical way because it's too expensive, like building a chocolate box. Those are things where the robot has to be reliable, it has to be good, and it has to have the capability to do a new task that it hasn't seen in initial training stages. I think it's unrealistic for us to assume that, you know, we can just go with like more and more human data collection, go bigger, bigger, bigger.

• 36:02Speaker 03

  We will do that, but there is always going to be a limit to how good and how much data you can get and how good the initial policy is going to be. So I think it is that, what you said in terms of we want deployment, we need this. But also, I think increasingly over the next years, we will, I expect we will see that we will do these deployments and that data will actually become really valuable as a source for pre-training, for making our models better themselves. And we'll rely more and more on autonomous data collection, is my prediction at least, over the next coming years, to kind of build that host of data, that convex hull of all the tasks that we want robots eventually to do, such that the models like ingests this

• 36:46Speaker 03

  and becomes good at doing them and interpolating between them.

• 36:49Speaker 00

  And I think of it as a new capability. We haven't so far figured out how to learn from your own experience, or there's been many attempts, but I don't think we've seen it done at scale. Uh, to like a very, to, to, to the extent that I actually shows a convincing result that allows you to deploy something.

• 37:05Speaker 01

  Yeah.

• 37:06Speaker 00

  And this is why this result was, was really, really important to us. We wanted to get to the point where they can learn from their own experience.

• 37:13Speaker 01

  Yeah, yeah.

• 37:13Speaker 00

  Uh, because you know, similarly to, to, to how we learn, you know, you can learn a little bit from watching videos and practice and then, you know, maybe learning from others. But at some point, you need to learn on the job. You need to try the thing yourself. You need to see how your actions impact what you actually want to achieve.

• 37:29Speaker 01

  Yeah.

• 37:30Speaker 00

  And make your own conclusions and try to learn that way.

• 37:33Speaker 01

  Yeah.

• 37:33Speaker 00

  And I think this is the first step towards that.

• 37:35Speaker 01

  You're reminding me of the, uh, do you guys read the Rich Sutton Age of Experience paper this year? I, I love, I thought it was very profound. Um, do you think that this unlocks kind of continual learning in robotics for you all? Will this be part of that?

• 37:50Speaker 00

  Uh, it kind of depends what people mean by continual learning. I think it's, um, it's definitely more continual than what we've done in the past. Where you know, you have like a big pre-training mixture and maybe like a post-training mixture and you like, uh, you know, you, you, you, you sit down, you work really, really hard. And then you come up with an artifact and like, that's it.

• 38:09Speaker 01

  Yeah.

• 38:10Speaker 00

  Right? Like the artifact is done and there's not much you can do to change it. Now this is a much more of a living thing, right? Like we, we start with a process similar to this, but then you deploy it and then it keeps on learning. Right? So it's much more continual in that sense that, that it tries new things. It, it tries to learn from its own experience and it keeps on getting better.

• 38:31Speaker 01

  Yeah.

• 38:32Speaker 00

  Now, I, I think there is still room to, for it to be much more continual, where it can acquire new skills that way. Or it can be even much faster in, in doing this.

• 38:43Speaker 01

  Yeah.

• 38:43Speaker 00

  Um, it can probably reason throughout this process. So I, I think there was a spectrum of like how much you can learn on the job. And this is really promising because it shows that you can do it, but I think we can make it much, much better.

• 38:57Speaker 03

  Yeah, I would agree. I would say we're at the very beginning of of this, right? And it's not, it's not, it's definitely not continual learning in the classical sense that people would have thought about it of like data streams and then the the whole thing turns and it just ultimately leads leads all the way to, I don't know, AGI or something like this yet. But, you know, it's it's a first step, I I would say, and we're moving in the right direction there and there's lots more to be done. And I think I will say from even from this release, like I was personally impressed and to some extent, you know, shocked how good these models actually are at picking up little things that you put back into the data. I was surprised that even with just like human corrections for, there was one example

• 39:39Speaker 03

  for for tamping when when we do, so tamping is a specific part of making an espresso, right? You like put the the beans in and you have to tamp down the

• 39:48Speaker 01

  The best part.

• 39:49Speaker 03

  Yeah, the best part, you have to tamp tamp down.

• 39:51Speaker 02

  I don't get it right myself, so.

• 39:54Speaker 03

  There you go. See, I'm not a not a coffee expert.

• 39:56Speaker 02

  It's a skill issue. I'm gonna get it just right.

• 40:00Speaker 03

  That's right. And so our robot in the beginning like tamped way too hard because it's just happened to be the case that, you know, the initial human demonstrations were just making sure that, you know, let's make sure the coffee grounds are flat so we can put it in. And then the robot was like tamping really hard and like almost lifting itself off the table. When we looked at it, we're like, oh, that's that's a bit much. And so with just, I don't know, it was, I think, thirty four to fifty episodes, there's a really small range of corrections that humans did and we feed that data back and the model actually starts like being much more gentle and doing the correct thing. And I was really surprised by that because you you think, you know, this model has been pre-trained on these millions and millions of episodes and now you're just doing a little correction and that actually works. So seeing that happen was was a thing that I think is pointing towards this continual learning part, which I find impressive.

• 40:46Speaker 01

  Can I ask though, and I'm, the thing I'm still hung up on is generalization. So as I learn how to tamp better, does that make me better at folding boxes or not?

• 40:56Speaker 03

  Uh. In this specific case, no, but the mechanism is the same that you can also employ to fix the, oh, I have two boxes in front of me that are sort of stuck together and I need to pull them apart, right? Because you can get thirty corrections for the stamping part, you get thirty corrections for the pulling boxes apart.

• 41:17Speaker 01

  Yeah.

• 41:17Speaker 03

  But you get thirty corrections for, oh, you know, this box wasn't like neatly folded together. And all of this accumulates together to then give you this more generalized improvement, I would say.

• 41:28Speaker 01

  Okay, so it's a repeatable recipe, but they don't necessarily cross, cross-pollinate.

• 41:32Speaker 03

  Yeah, I mean, we, I would expect that as we scale this up, we might see also things actually kind of transfer from, from A to B if there is motions that are kind of similar across tasks. But at this point, yeah, I would say it's more like a repeatable recipe.

• 41:46Speaker 01

  Yeah.

• 41:47Speaker 00

  And, and yeah, we, we see a lot of generalization from pre-training where you train on more and more tasks, more and more data. You see that it's much easier to onboard a new task, or you see tasks that appear zero-shot that you didn't expect before. And this keeps on improving. We, uh, we kick off a pre-training run at certain cadence. And every single time we start seeing that the model keeps on getting better because there's more data being fed in. There is more improvements that we're making to the pre-training process and so on. And I also suspect that as we have more and more of these models deployed, doing all kinds of different tasks, they also bring data back in. And I think one way where we, where I'm, I'm quite certain where we'll see more generalization is from that process.

• 42:30Speaker 00

  That as you deploy these models, the, the, the data comes back, the models get better, you can deploy them more. Then the, the models get better, you can deploy them more and so on.

• 42:39Speaker 01

  Yeah.

• 42:40Speaker 03

  And I think maybe it's worthwhile for this point that you brought up, we haven't really talked about one, like crucial detail aspect of this five star six recipe, which is that the model has kind of two parts. One is the policy that is trying to like improve, right? Via corrections and, and RL feedback. And the other part is how do you actually get this RL feedback, right? So we've talked a little bit, I've mentioned like, you know, humans might correct and that's the human correction part. And the RL feed. Back part is is a little different and it's kind of in already has some of these aspects of of generalization that I think you're you're like trying to search for. Which is that the way we do this is we we first basically get humans to to to tell us basically whether a specific

• 43:23Speaker 03

  attempt of making the coffee or doing the box was successful or not. So there will be like human labels provided with these episodes. And then we train something which is called a value function to try and predict basically from my given point of where I am in my in the task. Will I likely be succeeding or failing basically? And this value function is then used as kind of a baseline to to decide whether for this data point, should I like bump that up or should I bump that down depending on whether I expect that I will be moving towards success or or I'm more likely to move towards failure.

• 44:00Speaker 01

  Yeah.

• 44:01Speaker 03

  And one thing that we saw when we trained these value functions, so those are trained basically from the same kind of backbone and the same kind of model, but they're pre-trained before the actual policy is trained that that actually runs the task. When we train these value functions, we see that adding more data from different tasks actually helps there. And the model starts being actually really quite good at at least for for certain task and knowing when it will fail beforehand and before it is obvious for me, for example, when I look at a video of it trying to insert the

• 44:37Speaker 03

  the porter filter, thank you. See, I'm I'm not good at making coffee. And it's trying to insert a porter filter into the the coffee machine. It kind of knows that it doesn't quite have the right angle before that happens. So like thirty forty steps before that actually happens. The value function kind of, if you look at the prediction, drops and and saying, oh, this this is not good in this specific episode. So I I shouldn't include this data.

• 44:59Speaker 01

  Interesting.

• 45:00Speaker 03

  And this gets better with more data and more tasks.

• 45:02Speaker 01

  And so this is an interesting counterpoint to the the Carpathian like slurping bits from a straw thing, right? Because you're not you're not waiting for that final bits at the end. You're You're actually getting a signal on the way.

• 45:13Speaker 00

  I I think AREL is just like a such a vast field and there's so many different approaches to it. And people often associate AREL with something like a policy gradient method. Or, um, you know, very specific on policy learning approaches. And to me, AREL is more of a problem definition. And there is many, many approaches that get around the problem that you're referring to, which is that, you know, you only get the reward at the very, very end and it's not really scalable for very long horizon tasks. There are things like value functions, there are things like temporal difference learning that try to get around this problem. Where you constantly make predictions and you do it in a sequential way.

• 45:53Speaker 00

  And this is maybe another one of these, one of these things where I think robotics can really help the the broader AI community. Because we don't have the advantage of having a perfect language simulator where you can run as many simulations as you would like. Instead, you need to do it in the real world, so you need to make more efficient methods. And therefore you need to learn value functions and things like this. And I think this will, these will be really valuable everywhere.

• 46:19Speaker 01

  Yeah. Can I push a little bit on, I'd love to understand, you know, internet video seems like it's part of the recipe, but not a huge focus right now. As I see it like, do you think that there's gold left to be mined in internet video? And then if you look at what's happening in video models right now, uh, world models.

• 46:39Speaker 01

  To what extent do you think that's going to be a, you know, discontinuous jump in model capabilities? And, you know, an important part of your, your model pipeline?

• 46:48Speaker 00

  Yeah, um, I think maybe there are two questions there. One is about the data. Like, how do you bootstrap yourself to the point where you can start deploying? And the other question is, you know, what about, what about video models and kind of the world model aspects of it? So on the data point. I I think we are now in this bootstrap phase where basically anything goes. Like whatever you you can figure out how to like add to the model and to to to its benefit, I think it's good. Yeah. Whether you can add sim, whether you can add human videos, some kind of handheld devices, human teleoperations. I think it kind of doesn't matter. You just need to figure out some way to bootstrap yourself to the point where you can deploy these models.

• 47:32Speaker 00

  Because I think in the long term, there's going to be this bootstrap phase, but then there's going to be the deployment phase. And I think deployment phase will be, will will provide much, much more data than anything you could do in the bootstrap phase. So we're in this kind of like weird spot right now where we tried many different things, try to see what what sticks to just get us to the deployment threshold. I see. And once you can deploy, I think that will vastly be be much greater than than anything you can do before that. So so that's also what we are sprinting towards. That's why we want to start deploying these models. That's why we want to do this, you know, with many different tasks in in in many different environments so that we can just have this very powerful data engine.

• 48:15Speaker 00

  Now, on the on the world modeling side of things, I think the world models and Orel approaches are kind of targeting the same problem, the problem of counterfactuals of how do you or a credit assignment problem, right? Like, how do you figure out which actions were the ones that actually matter for your success? And how would the world have evolved had you had you taken a different action? And one way you can do this is by predicting what would have happened, right? Like rolling out a full video of, you know, if I if I put this porter filter a little bit differently, you know, where would I end up? And would this be a failure or a success? Or you can do this through reinforcement learning.

• 48:54Speaker 00

  And it does it through a slightly different mechanism, a little bit more implicitly, but it fundamentally targets a very similar problem. We are exploring all of those approaches and try to see, you know, how to really solve the counterfactual problem. I don't think there is an answer yet, but we see a lot of progress with reinforcement learning that we've just shown with PyStar, with PyStar 6. But I think there is probably room for many, many other approaches, too.

• 49:24Speaker 01

  Awesome. Can we talk about, once you guys get past that bootstrap phase, let's talk about customer deployments a little bit. What do you bring to a customer? What do you sell them? And then, how do you imagine that's going to evolve over time? Like, are you selling them a fully vertically-integrated robotic solution? Are you selling them a model that they have to figure out how to integrate into their operations? Like, how does this all work?

• 49:45Speaker 00

  The real answer is, we don't know yet.

• 49:47Speaker 01

  Yeah.

• 49:49Speaker 00

  We are still figuring that out.

• 49:50Speaker 01

  Yep.

• 49:51Speaker 00

  We are still quite early in the technology. As you can, as you can tell, we are just starting to even get to the threshold where we can start deploying these things. So, we believe we should focus on the, on the technology first to figure out how to get it to the point where it's actually easy to deploy and expand this aperture that we were talking about initially. And robotics, the history of robotics startups is, very often gets to this point where you develop a technology for some period of time. You started with this grand vision of what it should be able to enable, how general purpose it will be. And as soon as you pick an application that you want to apply it to, you're kind of stuck.

• 50:35Speaker 00

  You start cutting corners, you start figuring out very special purpose solutions just for this application, and very quickly you become, you know, an application company that just focuses on, let's say, warehouse pick and place robots, and that's it. And we really want to avoid that future. We think we have a chance to really solve physical intelligence, and the benefits of doing this will far outweigh any single applications that we can focus on now. So we want to make sure that the technology is as general as possible, as easily deployable as possible, this aperture is as wide as possible. And and then we'll start figuring out how to commercialize it. And as you said, there could be many different ways of doing this.

• 51:18Speaker 00

  There's probably ways that we can't think of just yet because they will depend on how the technology goes. Whether it's in, whether you can be a model provider, a fully vertical solution, or you sell robots or whatever else. But I think it's a little too premature to answer this question. It will give you a lot of comfort, you know, just to like pick one of those.

• 51:36Speaker 01

  It will give Alfred a lot of comfort.

• 51:38Speaker 00

  Alfred will be happy with us, but I think it's just too early.

• 51:42Speaker 02

  No, you guys have a grand, grand vision. So thank you for working on physical intelligence. It's a wonderful, wonderful improvement just for Pi star zero six. It's just a huge sort of breakthrough. And so congratulations on all the success you've had.

• 51:57Speaker 00

  Thank you.

• 51:58Speaker 02

  Can I follow up with a spicy question?

• 52:01Speaker 00

  Sure.

• 52:01Speaker 02

  So as you said, this vision is so grand, so broad, you're doing all these different things. If I'm sure you've you've studied all previous robotics like efforts and they've largely, as you said, applied in application to an application. And they get narrower and narrower. And one of the most successful cases of a large application is self-driving. And Waymo or Tesla have done enormously well. But if I had to go back in history, you know, I learned about self-driving when Sebastian Thiem was on the stage of TED. And I think two thousand and two thousand and nine, two thousand and ten.

• 52:42Speaker 02

  And he talked about the thing where they won the DARPA challenge that was two thousand and seven. And we're in twenty twenty five and the thing barely goes from San Francisco down here. They kind of can do it now, but they take local roads. They can't even get on the freeway. If you do such a generalized job, how long is the runway or the timeline that you're thinking about to build? For generalization and performance.

• 53:08Speaker 00

  Yeah, so there are some aspects of the problem that make it easier than self-driving and some that make it harder.

• 53:15Speaker 02

  Yeah.

• 53:16Speaker 00

  One thing that makes it easier is that we don't need to deploy it only when it's hundred percent reliable, right? There's many, many tasks out there that even if you're at ninety-five percent reliability, you're totally fine. If you have a robot in your home folding your laundry and every hundred item, you know, it doesn't fold it perfectly, you'll be totally fine.

• 53:36Speaker 02

  You just call your your child to go fold the hundred.

• 53:38Speaker 00

  That's right.

• 53:39Speaker 02

  Somebody has still, we still, we still need chores.

• 53:42Speaker 03

  We still have that, yeah.

• 53:43Speaker 00

  Exactly. And with self-driving, that's not the case, right? Like, if you fail every hundredth time, catastrophically, that's that's a big problem.

• 53:51Speaker 02

  Yeah.

• 53:52Speaker 00

  So I think in terms of deploying this technology, it might be easier. Now, we also benefit from the fact that this is a different era of technology.

• 54:01Speaker 02

  Sure.

• 54:01Speaker 00

  We we are at the era of vision language models, of foundation models that that have some some common sense. And we learned a lot of lessons between, what was it, two thousand nine and twenty twenty-five? And we can benefit from all of those.

• 54:16Speaker 00

  So I think that also really, really helps. And these are much more general purpose solutions than what we had in the past.

• 54:23Speaker 00

  At the same time, there are some things that will be very challenging, right? Like, there isn't just a single application. This is a very general purpose solution that can be applied to driving, but also to manipulation and locomotion and flying and all kinds of other things. And I think it's to be seen how much harder this is. So far, based on what we've experienced, it doesn't seem to be that much harder, to be honest. It seems that if you if you tackle this with with a very general purpose kind of mindset from the get-go, it turns out that it can generalize fairly well. And there is something about physical intelligence that we don't fully understand that allows these models to generalize between

• 55:04Speaker 00

  driving and making coffee and flying a drone and operating a surgical robot. Even though they seem so far apart from each other, and it seems that these should be all different models and different applications, these models somehow can make sense out of all of that data. And that gives me a lot of hope that maybe the problem is not that much harder and it might be actually easier.

• 55:25Speaker 00

  So I think it's a fair question, but I also don't want to draw the wrong conclusions from what we've seen from self-driving.

• 55:32Speaker 02

  That's beautiful. Congratulations. What result is impressed you the most outside of results?

• 55:38Speaker 00

  It's a great question.

• 55:39Speaker 03

  Yeah, it's a good question, actually.

• 55:41Speaker 00

  I can start. I've been really impressed by the video models, what you mentioned earlier. I saw them a few years ago. I worked on aspects of them a few years ago. And I didn't expect this trajectory to be, the improvement to be so steep. Like, they're basically indistinguishable right now from reality, and they can do incredible things. So that's been really, really impressive and really surprising to me.

• 56:06Speaker 03

  Yeah. I would say I'm still in awe, to some extent, that we've gotten to this place where we do seem to get models that do seem generally intelligent to a level that I really didn't foresee coming out of just next token prediction. I'm still amazed with this. And like every little advance that I see, you know, winning IMO mass challenges or, you know, applying it to finding new stuff in science. To me, yeah, there are so many things this year where I thought like, wow, there's still, there's still a lot of progress to be made, even though it felt like at the beginning of the year, maybe this whole pre-training business of LLMs is kind of maybe petering out a bit. Yeah, realizing that there's like

• 56:50Speaker 03

  this whole almost second breath of, yeah, fresh air basically coming in.

• 56:54Speaker 00

  Yeah, I was maybe add to this, just like the fact that this whole thing works. It's kind of mind-blowing. I don't think we like fully realize how ridiculous this is, right? Like you build this like loosely brain-inspired thing that has very general-purpose learning algorithm. You feed it data and it somehow gets it and gets it way better than anything we've ever had before. And this applies to robots and it applies to vision and language and sound and all kinds of other things. And like, I think if you stop for a second and just think about it, how it works and that it works, it's just like absolutely mind-blowing. Like the fact that we can have robots, you can put it in a home and it kind of knows what

• 57:36Speaker 00

  to do in a home that it's never been to before, or it can make coffee for 13 hours straight or, you know, things like that. And this is from this very general purpose. Thing that that trains fully end to end that we don't fully understand, but it seems to start to get it. That to me is just mind blowing.

• 57:57Speaker 01

  We're in a simulation.

• 58:00Speaker 02

  It's what that's what Sonia believes that we're living in a simulation. But it is interesting, right? Like in science, they teach you to take a big problem and break it up into smaller and smaller problems. And then basically somebody realized that that's maybe not the best way to train machines or robots of any kind.

• 58:17Speaker 03

  And to be honest, the whole machine learning, like AI field made that same mistake, actually, to some extent, right? We were working for a long time, people were working on solving individual problems very deeply, basically, right? And then over time, there is this like notion of, oh, if we can put it all together, like do multitask learning, if we could do that really, really well, we'd do much better. And then, but then the fact that that all happened just because we switched to this, you know, general pre-training objective, and then it just all falls out. That's the part that is disappointing bit, right?

• 58:47Speaker 02

  Do you think it's like an accordion where we go from one framework to the other framework? We take big problems, break them up into smaller and smaller and smaller ones. That worked for a period of time. Then it stopped working and then we're like, all right, let's go back to the big problem and try to solve it more generally. You go back and forth.

• 59:02Speaker 03

  I don't see us going back.

• 59:04Speaker 00

  Yeah, I don't see us going back. I think there is a lot of approaches or a lot of people saying that, you know, you need the best of both worlds and you need some kind of way of incorporating the rules that you already know about. Like, you know, Newtonian physics, you don't need to learn that. We already know how it works. So can you just like put it somehow into the weights? But from what we've seen so far, it doesn't work. If you try to do this, you kind of limit the ability to to learn new things. And I don't think there is the best of both worlds. I think we just go all the way learning. And it's kind of interesting to, you know, how similarly to how we learn, you would think that if there was a way to pre-bake all of the intelligence, the evolution would have figured this out. You would have just been born, you know, knowing everything there is to know.

• 59:48Speaker 00

  And we see this with some other species, right? Like, I think deer, when they, when they get born, they're basically like as smart as they'll ever be. Like, they don't really learn much throughout their lifetime. But for intelligent species like like humans, but also I think Krauss, for instance, they have these childhood periods, adolescence period, where they're not very smart to begin with, but they have to learn from their own experience. And it doesn't come pre-baked. You kind of have to earn it on your own. And I think there is something, something to that. You need to just experience the world and learn from that. And I think that's the lesson we're learning in in machine learning as well, in AI, that, you know, we, we think

• 1:00:31Speaker 00

  we know how we think, but we actually don't. And we just need to let the algorithm learn it from data.

• 1:00:37Speaker 02

  Same thing with raising a child. I think I know how my son is thinking, but I don't.

• 1:00:42Speaker 00

  Yeah, I have a, I have a small daughter. And yeah, it's just so surprising.

• 1:00:48Speaker 02

  They learn so fast.

• 1:00:49Speaker 00

  They learn so fast, and you don't know where they get it from.

• 1:00:52Speaker 02

  Hopefully from the parents.

• 1:00:54Speaker 00

  Hopefully. She definitely knows some things that I didn't teach her.

• 1:00:59Speaker 02

  Thank you guys so much.

• 1:01:00Speaker 01

  It's a really beautiful mission you're building after. Thank you for coming to share.

• 1:01:04Speaker 00

  Thank you. Thanks for having us.

• 1:01:05Speaker 01

  Thank you.