#374 – Robert Playter: Boston Dynamics CEO on Humanoid and Legged Robotics

Robert Playter is CEO of Boston Dynamics, a legendary robotics company that over 30 years has created some of the most elegant, dextrous, and simply amazing robots ever built, including the humanoid robot Atlas and the robot dog Spot. Please support this podcast by checking out our sponsors: – NetSuite: http://netsuite.com/lex to get free product tour – Linode: https://linode.com/lex to get $100 free credit – LMNT: https://drinkLMNT.com/lex to get free sample pack EPISODE LINKS: Boston Dynamics YouTube: https://youtube.com/@bostondynamics Boston Dynamics Twitter: https://twitter.com/BostonDynamics Boston Dynamics Instagram: https://www.instagram.com/bostondynamicsofficial Boston Dynamics Website: https://bostondynamics.com PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ YouTube Full Episodes: https://youtube.com/lexfridman YouTube Clips: https://youtube.com/lexclips SUPPORT & CONNECT: – Check out the sponsors above, it’s the best way to support this podcast – Support on Patreon: https://www.patreon.com/lexfridman – Twitter: https://twitter.com/lexfridman – Instagram: https://www.instagram.com/lexfridman – LinkedIn: https://www.linkedin.com/in/lexfridman – Facebook: https://www.facebook.com/lexfridman – Medium: https://medium.com/@lexfridman OUTLINE: Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time. (00:00) – Introduction (07:18) – Early days of Boston Dynamics (15:39) – Simplifying robots (19:37) – Art and science of robotics (24:20) – Atlas humanoid robot (41:14) – DARPA Robotics Challenge (55:34) – BigDog robot (1:09:23) – Spot robot (1:30:48) – Stretch robot (1:33:36) – Handle robot (1:39:10) – Robots in our homes (1:47:57) – Tesla Optimus robot (1:56:39) – ChatGPT (1:59:43) – Boston Dynamics AI Institute (2:01:14) – Fear of robots (2:11:36) – Running a company (2:17:13) – Consciousness (2:24:46) – Advice for young people (2:26:42) – Future of robots

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Lex Fridman?

  The following is a conversation with Robert Playter, CEO of Boston Dynamics, a legendary robotics company that over 30 years has created some of the most elegant, dexterous, and simply amazing robots ever built, including the humanoid robot Atlas and the robot dog Spot, one or both of whom you've probably seen on the internet, either dancing, doing backflips, opening doors, or throwing around heavy objects. Robert has led both the development of Boston Dynamics humanoid robots and their physics-based simulation software. He has been with the company from the very beginning, including its roots at MIT,

• 0:45Lex Fridman?

  where he received his PhD in aeronautical engineering. This was in 1994 at the legendary MIT Leg Lab. He wrote his PhD thesis on robot gymnastics, as part of which he programmed a bipedal robot to do the world's first 3D robotic somersault. Robert is a great engineer, roboticist, and leader, and Boston Dynamics, to me, as a roboticist, is a truly inspiring company. This conversation was a big honor and pleasure, and I hope to do a lot of great work with these robots in the years to come. And now, a quick few-second mention of each sponsor. Check them out in the description. It's the best way to support this podcast.

• 1:28Lex Fridman?

  We've got NetSuite for business management software, Linode for Linux systems, and Element for zero-sugar electrolytes. Choose wisely, my friends. Also, if you want to work with our team, we're always hiring. Go to luxfreeman.com/hiring. And now, onto the full ad reads. As always, no ads in the middle. I try to make this interesting, but if you must commit the horrible, terrible crime of skipping them, please do check out our sponsors. I do enjoy their stuff. I really do. And maybe you will as well. This show is brought to you by NetSuite, an all-in-one cloud business management system.

• 2:07Lex Fridman?

  Running a business, as this podcast reveals from Robert Plater and Boston Dynamics, is really hard. It's not just about the design of the system. It's not just about the engineering, the software, the hardware, all the complicated research that goes into it, all the different prototypes, all the failure upon failure upon failure in the early stages, in the middle stages of getting these incredible robots to work. It's all the glue that ties a company together. And for that, you have to use the best tools for the job. I hope to run a business, a large business, that actually builds stuff one day. And boys, and much more than just

• 2:48Lex Fridman?

  the innovation and the engineering, I understand that deeply. And you should be hiring the best team for that job and use the best tools for that job. And that's where NetSuite can help out, hopefully can help you out. You can start now with no payment or interest for six months. Go to netsuite.com/lex to access their one-of-a-kind financing program. That's netsuite.com/lex. This episode is also brought to you by Linode, now called Akamai, and their incredible Linux virtual machines. I sing praises to the greatest operating systems of all time, which is Linux. There's so many different beautiful flavors of Linux. My favorite is

• 3:32Lex Fridman?

  probably the different sub-flavors of Ubuntu, Ubuntu Mate. That's what I use for my personal development projects in general, when I want to feel comfortable and fully customized. But I've used so many other Linuxes, distributional Linuxes. But that's not what Linode is about. Or it is in part, but it actually takes those Linux boxes and scales them arbitrarily to where you can do compute, not just on one machine, but on many machines, customize them, make sure everything works reliably. And when it doesn't, there's amazing human customer service with real humans. That's something that should be emphasized in this day of ChatGPT. Real

• 4:14Lex Fridman?

  human beings that are good at what they do and figure out how to solve problems if they ever come up. Linode, now called Akamai, is just amazing. If compute is something you care about for your business, for your personal life, for your happiness, for anything, then now you should check them out. Visit linode.com/lex for a free credit. This episode is brought to you by a thing that I'm currently drinking as I'm saying these words. It's the Element Electrolyte Drink Mix, spelled L-M-N-T. My favorite is the watermelon, that's what I always drink. You know, we have all explored. In college, things got wild, things got a little crazy, things got a little out of hand.

• 4:59Lex Fridman?

  All of us have done things we regret, have eaten ice cream we should not have eaten. I've eaten ice cream at Dairy Queen so many times in my life, especially through my high school years, and to contradict what I just said, I regret nothing. I think Snickers and, if memory serves me correctly, there's something called the Dairy Queen Blizzard, where you could basically shove in whatever you want into the ice cream and blend it and it tastes delicious. Like, I think my favorite would be like the Snickers bar, any kind of bar, Mars Bar, and anything with kind of chocolate caramel, maybe a little bit coconut, that, that kind of stuff. You know, I don't regret it, but we've experimented, all of us have experimented with different

• 5:43Lex Fridman?

  flavors, with different things in life, and I regret nothing. You should not regret any of it either, because that path is what created the beautiful person that you are today. And that path is also the reason I mostly drink the watermelon flavor of, I guess it's called watermelon salt, I don't know what it's called, but watermelon is in the word, of Element. I highly recommend it, you could try other flavors, chocolate's pretty good too, like chocolate mint, I think it's called. Totally different thing, all the flavors are very different, and that's why I love it, so you should explore. Anyway, it's a good way to get all the electrolytes in your system, the salt, the magnesium, the potassium, not salt, sodium is what I meant to say.

• 6:24Lex Fridman?

  It doesn't matter what I meant to say, what matters is it's delicious and I'm consuming it and I'm singing it praises, and I will toast you when we see each other in person one day, friend. And we should drink Element Drink.

• 6:39Lex Fridman?

  To our deepest fulfillments together as brothers and sisters in arms. Get a simple pack for free with any purchase. Try it at drinkelement.com/lex.

• 6:53Lex Fridman?

  This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description. And now, dear friends, here's Robert Plater.

• 7:18Lex Fridman?

  When did you first fall in love with robotics?

• 7:22Lex Fridman?

  Let's start with love and robots.

• 7:25Speaker 01

  Well, love is relevant because I think the fascination, the deep fascination is really about movement. And I was visiting MIT, looking for a place to get a PhD. And I wanted to do some laboratory work. And one of my professors in the Aero Department said, "Go see this guy, Mark Raibert, down in the basement of the AI lab." And so I walked down there and saw him. He showed me his robots. And he showed me this robot doing a somersault. And I just immediately went, "Whoa." You know? Robots can do that. And because of my own interest in gymnastics, there was like this immediate connection.

• 8:07Speaker 01

  And I was interested in, I was in an Aero-Asteroid degree because flight and movement was all so fascinating to me. And then it turned out that robotics had this big challenge. How do you balance? How do you build a legged robot that can really get around? And that was a fascination. And it still exists today. You're still working on perfecting motion in robots.

• 8:33Lex Fridman?

  What about the elegance and the beauty of the movement itself? Is there something maybe grounded in your appreciation of movement from your gymnastics days? Did you?

• 8:47Lex Fridman?

  Was there something you just fundamentally appreciate about the elegance and beauty of movement?

• 8:52Speaker 01

  You know, we had this concept in gymnastics of letting your body do what it wanted to do. When you get really good at gymnastics, part of what you're doing is putting your body into a position where the physics and the body's inertia and momentum will kind of push you in the right direction in a very natural and organic way. And the thing that Mark was doing, you know, in the basement of that laboratory was trying to figure out how to build machines to take advantage of those ideas. How do you build something so that the physics of the machine just kind of inherently wants to do what it wants to do? And he was building these springy pogo stick type.

• 9:34Speaker 01

  You know, his first cut at Legged locomotion was a pogo stick where it's bouncing and there's a spring mass system that's oscillating, has its own sort of natural frequency there. And sort of figuring out how to augment those natural physics with also intent, how do you then control that but not overpower it? It's that coordination that I think creates real potential. We could call it beauty, you know, you could call it, I don't know, synergy. People have different words for it, but I think that that was inherent from the beginning. That was clear to me that that's part of what Mark was trying to do. He asked me to do that in my research work. So, you know, that's where it got going.

• 10:14Lex Fridman?

  So part of the thing that I think I'm calling elegance and beauty in this case, which was there even with the pogo stick is maybe the efficiency. So letting the body do what it wants to do, trying to discover the efficient movement.

• 10:27Speaker 01

  It's definitely more efficient. It also becomes easier to control in its own way because the physics are solving some of the problem itself. It's not like you have to do all this calculation and overpower the physics. The physics naturally inherently want to do the right thing. There can even be, you know, feedback mechanisms, stabilizing mechanisms that occur simply by virtue of the physics of the body. And it's, you know, not all. Not all in the computer, or not even all in your mind as a person. And I, there's something interesting in that melding.

• 11:04Lex Fridman?

  You were with Mark for many, many, many years. But you were there in this kind of legendary space of Leg Lab and MIT in the basement. All great things happen in the basement. Is there some memories, is there some memories from that time? That you have, because it's so, it's such cutting edge work in robotics and artificial intelligence.

• 11:29Speaker 01

  The memories, the distinctive lessons, I would say, I learned in that, in that time period. And, and that I think Mark was a great teacher of was it's okay to pursue your interests, your curiosity, do something because you love it. You'll do it a lot better if you love it. That, that is a lasting lesson that I think we apply at the company still. And really is a core value.

• 12:02Lex Fridman?

  So the interesting thing is I got to, with people like Ross Cedric and, and others, like the students that work at those robotics labs are like some of the happiest people I've ever met. I don't know what that is. I meet a lot of PhD students, a lot of them are kind of broken by the wear and tear of the process. But roboticists are, while they work extremely hard and work long hours, there's a, there's a happiness there. The only other group of people I met like that are people that skydive a lot. Like for, for some reason, there's a deep fulfilling happiness, maybe from like a long period of struggle to get a thing to work and it works and there's a magic to it.

• 12:45Lex Fridman?

  I don't know exactly, because it's so fundamentally hands on and you're bringing a thing to life. I don't know what it is, but they're happy.

• 12:53Speaker 01

  We see, you know, our, our attrition at the company is really low. People come and they love the pursuit. And I think part of that is that there's perhaps an natural connection to it. It's a little bit easier to connect when you have a robot that's moving around in the world. And part of your goal is to make it move around in the world. You can identify with that. And, and this is on a, this is one of the unique things about the kinds of robots we're building is this physical interaction lets you perhaps identify with it. So I think that is a source of happiness. I don't think it's unique to robotics. I think anybody also who is just pursuing something they love, it's easier to work hard at it and be good at it.

• 13:35Speaker 01

  And not everybody gets to find that. I, I do feel lucky in that way. And I think we're lucky as an organization that we've been able to build a business around this and that keeps people engaged.

• 13:50Lex Fridman?

  So if it's all right, let's linger on Mark for a little bit longer. Mark Raebert, so he's a legend. Uh, he's a legendary engineer and roboticist. What, what have you learned about life, about robotics from Mark? Through all the many years you worked with him?

• 14:06Speaker 01

  I think the most important lesson, which was, you know, have the courage of your convictions and do what you think is interesting. Be willing to try to find big, big problems to go after. And at the time, you know, legged locomotion, especially in a dynamic machine, nobody had solved it. And that felt like a multi-decade problem to go after. And so, you know, have the courage to go after that because you're interested. Don't worry if it's going to make money. You know, that, that's been a theme. So that, that's really probably the most important lesson, I think, that I got from Mark.

• 14:49Lex Fridman?

  How crazy is the effort of doing legged robotics at that time especially?

• 14:56Speaker 01

  You know, Mark got some stuff to work, uh, starting from the simple ideas. So, so maybe the other, another important idea that has really become a value of the company is try to simplify a thing to the core essence. And, and while, you know, Mark was showing videos of animals running across the savanna or, uh, uh, climbing mountains, what he started with was a pogo stick because he was trying to reduce the problem to something that was manageable and, and getting the pogo stick to balance had in it the fundamental problems that if we solve those, you could eventually extrapolate to something that galloped like a horse. And so look for those simplifying principles.

• 15:40Lex Fridman?

  How, how tough is the job of simplifying a robot?

• 15:43Speaker 01

  So I, I'd say in the early days, the, the thing that made Boston, the researchers at Boston Dynamics special is that we, we worked on under figuring out what that, that central principle was and then building software or machines around that principle. And that was not easy in the early days and, and it, it took, um, real expertise in understanding the dynamics of motion and feedback control principles. How to build and, you know, with the computers at the time, how to build a feedback control algorithm that was simple enough that it could run in real time at a thousand hertz and actually get that machine to work.

• 16:29Speaker 01

  And that was not something everybody was doing, you know, at that time. Now the world's changing now and I, I think the approaches to controlling robots are going to change. But it, and they're going to become more broadly, um, available. But at the time, there weren't many groups who could really sort of work at that principled level, uh, with both the software and, and make the hardware work. And I'll, and I'll say one other thing about you were sort of talking about what are the special things. The other thing was it's, it's, it's good to break stuff, you know? You know, use the robots, uh, break them, repair them.

• 17:13Speaker 01

  You know, fix and repeat, test, fix and repeat. And that, and that's also a core principle that has become part of the company. And it lets you be fearless in your work. Too often, if you are working with a very expensive robot, maybe one that you bought from somebody else or that you don't know how to fix, then you treat it with kit gloves and you can't actually make progress. You have to be able to break something. And so I think that's been a principle as well.

• 17:43Lex Fridman?

  So just to linger on that psychologically, how do you deal with that? Cause I remember I had, I built a RC car with that some, it had some custom stuff like compute on it, all that kind of stuff, cameras. And because I didn't sleep much, the code I wrote had an issue where it didn't stop the car and the car got confused and at full speed at like 20, 25 miles an hour slammed into a wall. And I just remember sitting there alone in a deep sadness, sort of full of regret, I think, almost anger, but also like sadness because you think about, well, these robots,

• 18:28Lex Fridman?

  especially for autonomous vehicles, like, like you should be taking safety very seriously, even in these kinds of things, but just no good feelings. It made me more afraid probably to do this kind of experiments in the future. Perhaps the right way to have seen that is positively, like it's too.

• 18:46Speaker 01

  It depends if you could have built that car or just gotten another one, right? That would have been the approach. I remember when I got to grad school, you know, I got some training about operating a lathe and a mill up in the machine shop and I could start to make my own parts. And I remember breaking some piece of equipment in the lab and then realizing, cause I, maybe this was a unique part and I couldn't go buy it. And I realized, oh, I can just go make it. That was an enabling feeling.

• 19:23Lex Fridman?

  Yeah.

• 19:23Speaker 01

  Then you're not afraid. It might take time. It might take more work than you thought it was going to be required to get this thing done. But you can just go make it. And that's freeing in a way that nothing else is.

• 19:37Lex Fridman?

  You mentioned the feedback control, the dynamics. Sorry for the romantic question, but is in the early days and even now, is the dynamics probably more appropriate for the early days? Is it more art or science?

• 19:54Speaker 01

  There's a lot of science around it. And, and trying to develop, you know, scientific principles that let you extrapolate from like one-legged machine to another, you know, develop a core set of principles like, like a spring mass bouncing system. And then figure out how to apply that from a one-legged machine to a two or a four-legged machine. Those principles are really important and, and, and we're definitely a core, a core part of our work.

• 20:25Speaker 01

  There's also, you know, when we started to pursue humanoid robots, there was so much complexity in that machine that, you know, one of the benefits of, of the humanoid form is you have some intuition about how it should look while it's moving. And that's a little bit of an art, I think. And now it's take, or maybe it's just tapping into a knowledge that you have deep in your body and then trying to express that in the machine. But that's an intuition that's a little bit more on the art side. Maybe it, it predates your knowledge, you know, before you have the knowledge of how to control it, you try to work through the art channel. And humanoids sort of make that available to you. If it had been a different shape, maybe you wouldn't have had the same intuition about

• 21:10Speaker 01

  it.

• 21:11Lex Fridman?

  Yeah. So your knowledge about moving through the world is not made explicit to you? She just, that's why it's art.

• 21:20Speaker 01

  And it might, yeah, it might be hard to actually articulate exactly, you know. There's something about, and being a competitive athlete, there's something about seeing movement. You know, a coach, one of the greatest strengths a coach has is being able to see, you know, some little change in what the athlete is doing and then being able to articulate that to the athlete, you know? And then maybe even trying to say, and you should try to feel this. So there's something just in seeing, and again, you know, sometimes it's hard to articulate what it is you're seeing. But there's a, just perceiving the motion at a rate that is, again, sometimes hard to put into words.

• 22:02Lex Fridman?

  Yeah, I wonder how it is possible to achieve sort of truly elegant movement. You have a movie like Ex Machina, I'm not sure if you've seen it, but the main actress in that, who plays the AI robot, I think is a ballerina. I mean, just the natural elegance and the, I don't know, eloquence of movement. It's, it's, it looks efficient and easy and just, it looks right.

• 22:34Speaker 01

  It looks right is sort of the key, yeah.

• 22:36Lex Fridman?

  And then you, you look at especially early robots, I mean, they're, they're so cautious in, in the way they move. That it's not, it's not the caution that looks wrong. It's, it's something about the movement that looks wrong that feels like it's very inefficient, unnecessarily so. And it's hard to put that into words, exactly.

• 22:58Speaker 01

  We think, and part of the reason why people are attracted to the machines we build is because the inherent dynamics of movement are, are closer to right. Because we, we try to use, you know, walking gates or we build a machine around this gate where you're trying to work with the dynamics of the machine instead of to stop them. You know, some of the early walking machines, you know, you're essentially, you're really trying hard to not let them fall over. And so you're always stopping the tipping motion, you know? And sort of the insight. Of dynamic stability in a humanoid machine is to go with it, you know?

• 23:40Speaker 01

  Let the tipping happen, you know, let yourself fall, but then catch your, catch yourself with that next foot. And there's something about getting those physics to be expressed in the machine that people interpret as lifelike, or elegant, or just natural looking. And so I think if you get the physics right, it also ends up being more efficient, likely. There's a benefit that it probably ends up being more stable in the long run, you know, it could, it could walk stably over a wider range of conditions. And it's, and it's more beautiful and attractive at the same time.

• 24:19Lex Fridman?

  So how hard is it to get the humanoid robot Atlas to do some of the things it's recently been doing? Let's forget the flips and all of that. Let's just look at the running. Maybe you can correct me, but there's something about running, I mean, that's not careful at all. That's you're falling forward. You're jumping forward and are falling. So how hard is it to get that right?

• 24:41Speaker 01

  Our first humanoid, we needed to deliver natural looking walking, you know, we took a contract from the army. They wanted a robot that could walk naturally. They wanted to put a suit on the robot and be able to test it in a gas environment. And so they wanted the motion to be natural. And so our goal was a natural looking gait. It was, it was surprisingly hard to get that to work. And we, but we did build an early machine. We called it Pet Man Prototype. It was the prototype before the Pet Man robot. And it had a really nice looking gait where, you know, it would stick the leg out, it would do heel strike

• 25:25Speaker 01

  first before it rolled onto the toe. So you didn't land with a flat foot, you extended your leg a little bit. But even then, it was hard to get the robot to walk where, when you're walking, that it fully extended its leg and essentially landed on an extended leg. And if you watch closely how you walk, you probably land on an extended leg, but then you immediately flex your knee as you start to make that contact. And getting that all to work well, took such a long time. In fact, I probably didn't really see the nice natural walking that I expected out of our humanoids until maybe last year. And the team was developing on our newer generation

• 26:06Speaker 01

  of Atlas, you know, some new techniques for developing a walking control algorithm. And they got that natural looking motion as sort of a byproduct of a, of a, just a different process they were applying to developing the control. So that probably took 15 years, 10 to 15 years to sort of get that from, from, you know, the Pet Man prototype was probably in 2008 and what was it, 2022, last year that I think I saw good walking on Atlas.

• 26:36Lex Fridman?

  If you could just like linger on it, what are some challenges of getting good walking? So is it, is this, is this partially like a hardware, like actuator problem? Is it the control? Is it the artistic element of just observing the whole system operating in different conditions together? I mean, is there some kind of interesting quirks or challenges you can speak to, like the heel strike or all that?

• 27:02Speaker 01

  Yeah. So one of the things that makes the, like this straight leg a challenge is you're sort of up against a singularity, a mathematical singularity where, you know, when your leg is fully extended, it can't go further the other direction, right? There's only, you can only move in one direction. And that makes all of the calculations around how to produce torques at that joint or positions makes it more complicated. And so having all of the mathematics so it can deal with these singular configurations is one of many challenges that we face. And I'd say in the, you know, in those earlier days,

• 27:44Speaker 01

  again, we were working with these really simplified models. So we're trying to boil all the physics of the complex human body into a simpler subsystem that we can more easily describe in mathematics. And sometimes those simpler subsystems don't have all of that complexity of the straight leg built into them. And so what's happened more recently is we're able to apply techniques that let us take the full physics of the robot into account and deal with some of those strange situations like the straight leg.

• 28:19Lex Fridman?

  So is there a fundamental challenge here that it's, maybe you can correct me, but is it underactuated? Are you falling?

• 28:26Speaker 01

  Underactuated is the right word, right? You can't, you can't push the robot in any direction you want to, right? And so that is one of the hard problems of legged locomotion.

• 28:37Lex Fridman?

  And you have to do that for natural movement?

• 28:40Speaker 01

  It's not necessarily required for natural movement. It's just required, you know, we don't have, you know, a gravity force that you can hook yourself onto to apply an external force in the direction you want at all times, right? The only, the only external forces are being mediated through your feet and how they get mediated depend on how you place your feet. And, you know, you can't just, you know, God's hand can't reach down and give and push in any direction you want, you know, so.

• 29:09Lex Fridman?

  Is there, is there some extra challenge to the fact that Alice is such a big robot?

• 29:14Speaker 01

  There is. The humanoid form is attractive in many ways, but it's also a challenge in many ways. You have this big upper body that has a lot of mass and inertia and throwing that inertia around increases the complexity of maintaining balance. And as soon as you pick up something heavy in your arms, you've made that problem even harder. And so in the early work in the leg lab and in the early days at the company, you know, we were pursuing these quadruped robots, which had a kind of built in simplification. You had this big rigid body and then really light legs.

• 29:55Speaker 01

  So when you swing the legs, the leg motion didn't impact the body motion very much. All the mass and inertia was in the body. But when you have the humanoid, that doesn't work. You have big, heavy legs, you swing the legs, it affects everything else. And so dealing with all of that interaction does make the humanoid a much more complicated platform.

• 30:18Lex Fridman?

  And I also saw that at least recently you've been doing more explicit modeling of the stuff you pick up.

• 30:25Speaker 01

  Yeah.

• 30:26Lex Fridman?

  Which is very, really interesting. So you have to, what, model the shape? The weight distribution.

• 30:37Lex Fridman?

  I don't know what like you have to like include that as part of the modeling as part of the planning. Cause okay, so for people who don't know. So Atlas, at least in like a recent video, like throws a heavy bag, throws a bunch of stuff. So what what's involved in picking up a thing, a heavy thing?

• 30:58Lex Fridman?

  And when that thing is a bunch of different non-standard things, I guess they also picked up like a barbell. And to be able to throw in some cases, what are some interesting challenges there?

• 31:09Speaker 01

  So we were definitely trying to show that the robot and the techniques we're applying to the robot to Atlas. Let us deal with heavy things in the world because if the robot's going to be useful, it's actually got to move stuff around. And that and that needs to be significant stuff. That's an appreciable portion of the body weight of the robot. And we also think this differentiates us from the other humanoid robot activities that you're seeing out there. Mostly they're not picking stuff up yet and not heavy stuff anyway.

• 31:40Speaker 01

  But just like you or me, you know, you need to anticipate that moment, you know, you're reaching out to pick something up. And as soon as you pick it up, your center of mass is going to shift. And if you're going to, you know, turn in a circle, you have to take that inertia into account. And if you're going to throw a thing, you know, you've got all of that has to be sort of included in in the model of what you're trying to do. So the robot needs to have some idea or expectation of what that weight is and then and sort of predict. You know, think a couple of seconds ahead. How do I manage my now my my body plus this big, heavy thing together to get and and still maintain balance? Right? And so I that's a big change for us.

• 32:24Speaker 01

  And I think the tools we've built are really allowing that to happen quickly now. Some of those motions that you saw in that most recent video, we were able to create in a matter of days. It used to be it took six months to do anything new, you know, on the robot. And now we're starting to develop the tools that let us do that in a matter of days. And so we think that's really exciting. That means that the ability to create new behaviors for the robot is going to be. A quicker process.

• 32:52Lex Fridman?

  So being able to explicitly model new things that it might need to pick up, new types of things.

• 32:58Speaker 01

  And you know, to some degree, you don't, you don't want to have to pay too much attention to each specific thing, right? There's sort of a generalization here.

• 33:09Speaker 01

  Obviously, when you grab a thing, you have to conform your, your hand, your end effector to the surface of that shape. But once it's in your hands, it's probably just the mass and inertia that matter. And the, the shape may, may not be as important.

• 33:23Lex Fridman?

  Yeah.

• 33:24Speaker 01

  And so, you know, for some, in some ways, you want to pay attention to that detailed shape. And in others, you want to generalize it and say, well, all I really care about is the center of mass of this thing, especially if I'm going to throw it up on that scaffolding.

• 33:38Lex Fridman?

  And it's easier if the body is rigid. What if it's, there's some, doesn't it throw like a sandbag type thing?

• 33:43Speaker 01

  That tool bag, you know, had loose, had loose stuff in it. So it, it managed that. There are harder things that we haven't done yet. You know, we could have had a big jointed thing, or I don't know, a bunch of loose wire or rope.

• 33:55Lex Fridman?

  What about carrying another robot? How about that?

• 34:00Speaker 01

  Yeah, we haven't, we haven't done that yet.

• 34:01Lex Fridman?

  Carry spot.

• 34:02Speaker 01

  I guess we did a little bit of a, we did a, a little skit around Christmas where we had two spots holding up another spot that was trying to put, you know, a bow on a tree. So I guess we're doing that in a small way.

• 34:15Lex Fridman?

  Okay, that's pretty good. Let me ask the all important question. Do you know how much Atlas can curl?

• 34:25Lex Fridman?

  I mean, you know, this, for us humans, that's really one of the most fundamental questions you could ask another human being.

• 34:34Lex Fridman?

  Bench.

• 34:35Speaker 01

  It probably can't curl as much as we can yet. But a metric that I think is interesting is, you know, another way of, of looking at that strength is, you know, the box jump. So if, how high of a box can you jump onto?

• 34:50Lex Fridman?

  Question.

• 34:51Speaker 01

  And Atlas, I don't know the exact height. It was probably a meter high or something like that. It was a pretty, pretty tall jump that Atlas was able to manage when we last tried to do this. And, and I have video of my chief technical officer doing the same jump. And he really struggled, you know.

• 35:07Lex Fridman?

  Oh, the human.

• 35:08Speaker 01

  But the human, getting all the way on top of this box. But then, you know, Atlas was able to do it. We're now thinking about the next generation of Atlas and we're probably going to be in the realm of a person can't do it, you know, with this, with the next generation of the robots, the actuators are going to get stronger where it really is the case that at least some of these joints, some of these motions will be stronger.

• 35:31Lex Fridman?

  And to understand how high I can jump, you probably had to do quite a bit of testing.

• 35:35Speaker 01

  Oh yeah. And there's lots of videos of it trying and failing. And that's, you know, that's all, you know, we don't always release those, those videos, but they're a lot of fun to look at.

• 35:44Lex Fridman?

  So we'll, we'll talk a little bit about that. But if, can you talk to the jumping? Cause you talked about the walking and it took a long time, many, many years to get the walking to be natural. But there's also really natural looking, robust, resilient jumping. How hard is it to do the jumping?

• 36:04Speaker 01

  Well, again, this stuff has really evolved rapidly in the last few years. You know, the first time we did a somersault, um, you know, there was a lot of kind of manual iteration. What is the trajectory? You know, how hard do you throw you? In fact, in these early days, I actually would, when I'd see early experiments that the team was doing, I might make suggestions about how to change the technique. Again, kind of borrowing from my own intuition about how backflips work. Um, but frankly, they don't need that anymore. So in the, in the early days, you had to iterate kind of in almost a manual way, trying to change these trajectories of the arms or the legs to try to get, you know, a successful backflip to happen. But more recently, we're running

• 36:50Speaker 01

  these model predictive, uh, uh, control techniques where we're able to, the robot essentially can think in advance for the next second or two about how its motion is going to transpire. And you can, you know, solve for optimal trajectories to get from A to B. So this is happening in a much more natural way. And we're really seeing an acceleration happen in the development of these behaviors. Again, partly due to these optimization techniques, sometimes learning techniques. Um, so it's, there's, it's hard in that there's a lot of mathematics and, uh, behind it. Uh, but we're figuring that out.

• 37:33Lex Fridman?

  So you can do model predictive control for? I mean, I don't even understand what that looks like when the entire robot is in the air, flying and doing a back.

• 37:43Speaker 01

  Yeah.

• 37:44Lex Fridman?

  I mean.

• 37:45Speaker 01

  But, but, but that's the cool part, right? So, you know, you, you know, the physics, we, we can calculate physics pretty well using, you know, Newton's laws about how it's going to evolve over time and the road. Here this, this, the sick trick, which was a front somersault with a half twist is a good example, right?

• 38:03Speaker 01

  You saw the robot on various versions of that trick. I've seen it land in different configurations and it still manages to stabilize itself. And so, you know, what this model predictive control means is, again, the, in real time, the robot is projecting ahead, you know, a second into the future and sort of exploring options. And if I, if I move my arm a little bit more this way, how is that going to affect the outcome? And so it can do these calculations, many of them, you know, and, and basically solve where, you know, given where I am now, maybe I took off a little bit screwy from how I had planned, I can adjust.

• 38:42Lex Fridman?

  So you're adjusting in the air.

• 38:43Speaker 01

  Adjust on the fly. So the, the model predictive control lets you adjust on the fly. And of course, I think this is what, you know, people adapt as well. We, we, when, when we do it, even a gymnastics trick, we try to set it up so it's, it's close to the same every time, but we figured out how to do some adjustment on the fly. And now we're starting to figure out that the robots can do this adjustment on the fly as well, using these techniques.

• 39:07Lex Fridman?

  In the air. It's so, I mean, it just feels from a robotics perspective, just surreal.

• 39:13Speaker 01

  Well, that's sort of the, you talked about under actuated, right? So when you're, when, when you're in the air, there's something, there's some things you can't change, right? You can't change the momentum while it's in the air because you can't apply an external force or torque. And so the momentum isn't going to change. So how do you work within the constraint of that fixed momentum to still get from A to B, where, where you want to be?

• 39:35Lex Fridman?

  That's really under actuated.

• 39:38Lex Fridman?

  You're in the air. I mean, you, you become a drone for a brief moment in time. No, you're not even a drone because you can't.

• 39:44Speaker 01

  Can't hover.

• 39:45Lex Fridman?

  You can't hover.

• 39:46Speaker 01

  You're gonna, you're gonna impact soon. Be ready.

• 39:49Lex Fridman?

  Yeah. Are you considered like a hover type thing or no? No, it's too much weight. I mean, it's just, it's just incredible.

• 39:58Lex Fridman?

  And just even to have the guts to try backflip with such a large body. That's wild. But like,

• 40:05Speaker 01

  We definitely broke a few robots trying. But that but that's where the build it, break it, fix it, you know, strategy comes in. You gotta be willing to break. And what ends up happening is you end up, by breaking the robot repeatedly, you find the weak points. And then you end up redesigning it so it doesn't break so easily next time, you know?

• 40:23Lex Fridman?

  Through the breaking process, you learn a lot, like a lot of lessons and you keep improving not just how to make the backflip work, but everything.

• 40:31Speaker 01

  And how to build the machine better.

• 40:32Lex Fridman?

  Yeah.

• 40:32Speaker 01

  Yeah.

• 40:34Lex Fridman?

  I mean, is there something about just the guts to come up with an idea of saying, you know what, let's try to make it do a backflip?

• 40:42Speaker 01

  Well, I think the courage to do a backflip in the first place. And and to not worry too much about the ridicule of somebody saying, why the heck are you doing backflips with robots? Because a lot of people have asked that, you know, why? Why? Why are you doing this?

• 40:56Lex Fridman?

  Why go to the moon in this decade and do the other things, JFK?

• 41:02Lex Fridman?

  Not because it's easy, because it's hard.

• 41:04Speaker 01

  Yeah, exactly.

• 41:07Lex Fridman?

  Don't ask questions. Okay, so the the jumping, I mean, it's just, there's a lot of incredible stuff. If we can just rewind a little bit to the DARPA Robotics Challenge in 2015, I think, which was, for people who are familiar with the DARPA challenges, it was first with autonomous vehicles, and there's a lot of interesting challenges around that. And the DARPA Robotics Challenge was when humanoid robots were tasked to do all kinds of,

• 41:39Lex Fridman?

  you know, manipulation, walking, driving a car, all these kinds of challenges with, if I remember correctly, sort of some slight capability to communicate with humans. But the communication was very poor. So it basically has to be almost entirely autonomous.

• 41:58Speaker 01

  It could have periods where the communication was entirely interrupted and the robot had to be able to proceed.

• 42:03Lex Fridman?

  Yeah.

• 42:03Speaker 01

  But you could provide some high level guidance to the robot, basically low bandwidth communications.

• 42:09Lex Fridman?

  Yeah.

• 42:09Speaker 01

  To steer it.

• 42:10Lex Fridman?

  I watched that challenge with kind of tears in my eyes, eating popcorn.

• 42:15Speaker 01

  I was too.

• 42:18Lex Fridman?

  But I wasn't personally losing, you know, hundreds of thousands, millions of dollars and many years of incredible hard work by some of the most brilliant roboticists in the world. So that was why the tragic, that's why the tears came. So anyway, what have you, just looking back to that time, what have you learned from that experience? I mean, maybe if you could describe what it was, sort of to set up for people who haven't seen it.

• 42:44Speaker 01

  Well, so there was a contest where a bunch of different robots were asked to do a series of tasks, some of those that you mentioned, drive a vehicle, get out, open a door, go identify a valve, shut a valve, use a tool to maybe cut a hole in a surface and then crawl over some stairs and maybe some rough terrain. So it was, the idea was have a general purpose robot that could do lots of different things. Had to be mobility and manipulation onboard perception. And there was a contest, which DARPA likes, at

• 43:26Speaker 01

  the time was running, sort of follow on to the grand challenge, which was let's, let's try to push vehicle autonomy along, right? They, they, they encourage people to build autonomous cars. So they're trying to basically push an industry forward. And we were asked, our role in this was to build a humanoid. At the time, it was our sort of first generation Atlas robot. And we built maybe 10 of them. I don't remember the exact number. And DARPA distributed those to various teams that sort of won a contest, showed that they could,

• 44:09Speaker 01

  you know, program these robots and then use them to compete against each other. And then other robots were introduced as well. Some teams built their own robots. Carnegie Mellon, for example, built their own robot. And, and all these robots competed to see who could sort of get through this, this maze the fastest. And again, I think the purpose was to kind of push the whole industry forward.

• 44:34Speaker 01

  We provided the robot and some baseline software, but we didn't, we didn't actually compete as a participant where we were trying to, you know, drive the robot through this maze. We were just trying to support the other teams. It was humbling because it was, it was really a hard task and honestly, the robots, the tears were because mostly the robots didn't do it, you know, they fell down, you know, repeatedly. It was hard to get through this contest. Some did and, and, you know, they were rewarded and won, but it was humbling because of just how hard, these tasks weren't all that hard. A person could have done it very easily, but it was really hard to get the robots to do it, you know.

• 45:18Lex Fridman?

  Well, the general nature of it, the, the variety of it.

• 45:21Speaker 01

  The variety.

• 45:22Lex Fridman?

  And also that I don't know if the tasks were sort of, the task in themselves help us understand what is difficult and what is not. I don't know if that was obvious before the contest was designed. So you kind of tried to figure that out. And I think Atlas is really a general robot platform and it's perhaps not best suited for the specific tasks of that contest. Like for, just for example, probably the hardest task is not the driving of the car, but getting in and out of the car. And Atlas probably, you know, if you were to design a robot that can get into the car easily and get out easily, you probably would not make Atlas that particular car.

• 46:06Speaker 01

  Yeah, the robot was a little bit big to get in and out of that car.

• 46:09Lex Fridman?

  Right. It doesn't fit.

• 46:11Speaker 01

  This is the curse of a general purpose robot, that they're not perfect at any one thing, but they might be able to do a wide variety of things. And, and that is, that is the goal at the end of the day. You know, I think we all want to build general purpose robots that can be used for lots of different activities, but it's hard. And, and the wisdom in, in building successful robots up until this point have been go build a robot for a specific task and it'll do it very well. And as long as you control that environment, it'll operate perfectly. But, but robots need to be able to deal with uncertainty. If they're going to be useful to us in the future.

• 46:55Speaker 01

  They need to be able to deal with unexpected situations. And that's sort of the goal of a general purpose or multi-purpose robot. And that's just darn hard. And so some of the, you know, there's these curious little failures. Like I remember one of the, a robot, you know, the first, the first time you start to try to push on the world with a robot, you, you forget that the world pushes back and, and will push you over if you're not ready for it. And the robot, you know, reached to grab the door handle. I think it missed the grasp of the door handle, was expecting that its hand was on the door handle. And so when it tried to turn the knob, it just threw itself over. It didn't realize, oh, I had missed the door handle.

• 47:37Speaker 01

  I didn't have, I didn't, I was expecting a force back from the door. It wasn't there and then I lost my balance. So these little simple things that you and I would take totally for granted and deal with, the robots don't know how to deal with yet. And so you have to start to deal with all of those circumstances.

• 47:56Lex Fridman?

  Well, I think a lot of us experience this in, even when sober, but drunk too. Sort of you pick up a thing and expect it to be, what is it, heavy? And it turns out to be light.

• 48:09Speaker 01

  Yeah.

• 48:10Lex Fridman?

  And you, whoa. Oh, yeah. And then, so the same, and I'm sure if your depth perception, for whatever reason, is screwed up, if you're, if you're drunk or some other reason, and then you think you're putting your hand on the table and you miss it. I mean, it's the same kind of situation.

• 48:24Speaker 01

  Yeah.

• 48:25Lex Fridman?

  But there's a-

• 48:26Speaker 01

  Which is why you need to be able to predict forward just a little bit. And so that's where this model predictive control stuff comes in. Predict forward what you think's going to happen. And then if, and if that does happen, you're in good shape. If something else happens, you better start predicting again.

• 48:40Lex Fridman?

  So re, like re, re, regenerate a plan?

• 48:43Speaker 01

  Yeah.

• 48:44Lex Fridman?

  When you don't, I mean, that, that also requires a very fast feedback loop of updating what your prediction how matches to the actual real world.

• 48:57Speaker 01

  Yeah, those things have to run pretty quickly.

• 48:59Lex Fridman?

  What's the challenge of running things pretty quickly? A thousand hertz. Of acting and sensing quickly.

• 49:08Speaker 01

  You know, there's a few different layers of that. You want at the lowest level, you like to run things typically at around a thousand hertz, which means that, you know, at each joint of the robot, you're measuring position or force and then trying to control your actuator, whether it's a hydraulic or electric motor, trying to control the force coming out of that actuator. And you want to do that really fast, something like a thousand hertz. And that means you can't have too much calculation going on at that joint. But that's pretty manageable these days and it's fairly common. And then there's another layer that you're probably calculating, you know, maybe at 100 hertz, maybe 10 times slower, which is

• 49:49Speaker 01

  now starting to look at the overall body motion and thinking about the larger physics of the robot. And then there's yet another loop that's probably happening a little bit slower, which is where you start to bring, you know, your perception and your vision and things like that. And so you need to run all of these loops sort of simultaneously. You do have to manage your computer time so that you can squeeze in all the calculations you need in real time in a very consistent way. And the amount of calculation we can do is increasing as computers get better, which means we can start to do more sophisticated

• 50:30Speaker 01

  calculations. I can have a more complex model, doing my forward prediction. And that might allow me to do even better predictions as I get better and better. And it used to be, again, we had, you know, 10 years ago, we had to have pretty simple models that we were running, you know, at those fast rates because the computers weren't as capable about calculating forward with a sophisticated model. But as computation gets better, we can, we can do more of that.

• 51:03Lex Fridman?

  What about the actual pipeline of software engineering? How easy is it to keep updating Atlas, like do continuous development on it? So how many computers are on there? Is there a nice pipeline?

• 51:17Speaker 01

  It's an important part of building a team around it, which means, you know,

• 51:23Speaker 01

  You need to also have software tools, simulation tools, you know, so we have always made strong use of physics-based simulation tools to do some of this calculation, basically test it in simulation before you put it on the robot, but you also want the same code that you're running in simulation to be the same code you're running on the hardware. And so even getting to the point where it was the same code going from one to the other, we probably didn't really get that working until, you know, a few years, several years ago. But that was, you know, that was a bit of a milestone. And so you want to work, certainly work these pipelines so that you can make it as easy as possible and have

• 52:06Speaker 01

  a bunch of people working in parallel, especially when, you know, we only have, you know, four of the Atlas robots, the modern Atlas robots at the company. And, you know, we probably have, you know, 40 developers there all trying to gain access to it. And so you need to share resources and use some of these, some of the software pipeline.

• 52:25Lex Fridman?

  Well, that's a really exciting step to be able to run the exact same code in simulation as on the actual robot. How hard is it to do realistic simulation, physics-based simulation of Atlas such that, I mean, the dream is like, if it works in simulation, it works perfectly in reality. How hard is it to sort of keep working on closing that gap?

• 52:49Speaker 01

  The root of some of our physics-based simulation tools really started at MIT. And we built some, some, some good physics-based modeling tools there. The early days of the company, we were trying to develop those tools as a commercial product. So we continued to develop them. It wasn't a particularly successful commercial product, but we ended up with some nice physics-based simulation tools so that when we started doing legged robotics again, we had a really nice tool to work with. And the things we paid attention to were, were things that weren't necessarily handled very well in the commercial tools you could buy off the shelf, like, like interaction with the world, like foot-ground contact. So trying to model those contact.

• 53:31Speaker 01

  Events well in a way that captured the important parts of the interaction was a really important element to get right and to also do in a way that was computationally feasible and could run fast because if you if your simulation runs too slow, you know, then your developers are sitting around waiting for stuff to run and compile. So it's always about efficient, fast operation as well. So that's been a big part of it. You know, I think developing those tools in parallel to the development of the platform and trying to scale them has has really been essential, I'd say, to us being able to assemble a team of people that could do this.

• 54:15Lex Fridman?

  Yeah. How to simulate contact periods of foot ground contact, but sort of for manipulation? Because don't you want to model all kinds of surfaces?

• 54:27Speaker 01

  Yeah. So it will be even more complex with manipulation because there's a lot more going on, you know, and you need to capture, I don't know, things slipping and moving, you know, in your in your in your hand. It's a level of complexity that I think goes above foot ground contact when you really start doing textures manipulation. So there's challenges ahead still.

• 54:51Lex Fridman?

  So how far are we away from me being able to walk with Atlas in the sand along the beach and us both drinking a beer?

• 55:01Speaker 01

  Out of a can, out of a can. Maybe Atlas could spill his beer because he's got nowhere to put it. Atlas could walk on the sand.

• 55:11Lex Fridman?

  So can it?

• 55:12Speaker 01

  Yeah. Yeah. I mean, you know, have we really had him out on the beach? We take them outside often, you know, rocks, hills, that sort of thing, even just around our lab in Waltham. We probably haven't been on the sand, but I'm

• 55:25Lex Fridman?

  Soft surfaces.

• 55:26Speaker 01

  I don't doubt that we could deal with it. We we might have to spend a little bit of time to sort of make that work. But we did take a we had a had to take Big Dog to Thailand years ago and we did this great video of the robot.

• 55:44Speaker 01

  Walking in the sand, walking into the ocean up to, I don't know, its belly or something like that, and then turning around and walking out, all walk playing some cool beach music.

• 55:55Lex Fridman?

  Yeah.

• 55:55Speaker 01

  Great show, but then, you know, we didn't really clean the robot off and the salt water was really hard on it. So, you know, we put it in a box, shipped it back. By the time it came back, we had some problems with corrosion.

• 56:06Lex Fridman?

  So it's a salt, it's a salt water. It's not like

• 56:08Speaker 01

  Salt stuff.

• 56:09Lex Fridman?

  It's not like sand getting into the components or something like this. But I'm sure if this is a big priority, you can make it waterproof and stuff.

• 56:17Speaker 01

  Right, right. That just wasn't our, our goal at the time.

• 56:20Lex Fridman?

  Well, it's a personal goal of mine to walk along the beach. But it's a human problem too. You get sand everywhere. It's, it's just a jam mess. So soft surfaces are okay. So, I mean, can we just linger on the robotics challenge? There's a, there's a pile of like rubble they had to walk over. Is that's, how difficult is that task?

• 56:44Speaker 01

  In the early days of developing Big Dog, the loose rock was the epitome of the hard walking surface because you step down and then the rock, and you had these little point feet on the robot, and the rock can roll. And, and you have to deal with that last minute, you know, change in your foot placement.

• 57:04Lex Fridman?

  Yeah. So you, you step on the thing and that thing responds to you stepping on it?

• 57:08Speaker 01

  Yeah. And, and it moves where your point of support is. And so it's really, that, that became kind of the essence of the test. And so that was the beginning of us starting to build rock piles in our parking lots. And, and we would actually build boxes full of rocks and bring them into the lab. And then we would have the robots walking across these boxes of rocks because that became the essential test.

• 57:33Lex Fridman?

  So you mentioned Big Dog. Can you, can we maybe take a stroll through the history of Boston Dynamics? So what and who is Big Dog? By the way, is who?

• 57:45Lex Fridman?

  Do you try not to anthropomorphize the robots? Do you try not to? Do you try to remember that they're? This is like the division I have because for me it's impossible. For me, there's a magic to the being that is a robot. It is not human, but it is. The same magic that a living being has when it moves about the world is there in the robot. So I don't know what question I'm asking, but should I say what or who? I guess. Who is big dog? What is big dog?

• 58:18Speaker 01

  Well, I'll say to address the meta question. We don't try to draw hard lines around it being an it or a him or a her. It's okay, right? People, I think part of the magic of these kinds of machines is by nature of their organic movement of their dynamics. We tend to want to identify with them. We tend to look at them and sort of attribute maybe feeling to that because we've only seen things that move like this that were alive. And so this is an opportunity. It means that you could have feelings for a machine.

• 59:02Speaker 01

  And you know, people have feelings for their cars. You know, they get attracted to them, attached to them. So that's inherently could be a good thing as long as we manage what that interaction is. So we don't put strong boundaries around this and ultimately think it's a benefit, but it's also can be a bit of a curse because I think people look at these machines and they attribute a level of intelligence that the machines don't have. Why? Because again, they've seen things move like this that were living beings, which are intelligent. And so they want to attribute intelligence to the robots that isn't appropriate yet, even though they move like an intelligent being.

• 59:43Lex Fridman?

  But you try to acknowledge that the anthropomorphization is there and try to, first of all, acknowledge that it's there.

• 59:52Speaker 01

  And have a little fun with it.

• 59:53Lex Fridman?

  Have a little fun.

• 59:54Speaker 01

  Our most recent video, it's just kind of fun, you know, to, to, to look at the robot. We started off the, the video with Atlas, um, kind of looking around for where the bag of tools was. Cause the guy up on the scaffolding says, send me some tools. And Atlas has to kind of look around and see where they are. And there's a little personality there. That is fun. It's entertaining. It makes our jobs interesting. And I think in the long run can enhance interaction between humans and robots that in a way that isn't available to machines that don't move that way.

• 1:00:30Lex Fridman?

  This is something to me personally is very interesting. I've been, um, I happen to have a lot of legged robots.

• 1:00:39Lex Fridman?

  I hope to have a lot of spots in my possession. I'm interested in celebrating robotics and celebrating companies. And I also don't want to, companies that do incredible stuff like Boston Dynamics. And I, there's, um, you know, I'm a little crazy. And you say, you don't want to, you, you want to align, you want to help the company. Cause I ultimately want, uh, a company like Boston Dynamics to succeed. And part of that we'll talk about, you know, success kind of requires making money. And so the kind of stuff I'm particularly interested in may not be the thing that makes money in the short term.

• 1:01:15Lex Fridman?

  I can make an argument that will in the long term, but the kind of stuff I've been playing with is a robust way of, uh, having the quadruped. As the, the robot dogs communicate emotion with their body movement. The same kind of stuff you do with, with a dog, but not, not hardcoded, uh, but in a robust way. And be able to communicate excitement or fear, uh, boredom, all this kinds of stuff. And I think as a base layer. Of function of behavior to add on top of a robot. I think that's a really powerful way to make the robot more usable for humans for whatever.

• 1:01:54Speaker 01

  I think it's going to be really important and it's a thing we're beginning to pay attention to. We really want to start. A differentiator for the company has always been, we really want the robot to work. We want it to be useful. Making it work at first meant the legged locomotion really works. It can really get around and it doesn't fall down. And but beyond that, now it needs to be a useful tool. And our customers are, for example, factory owners, people who are running a process manufacturing facility. And the robot needs to be able to get through this complex facility in a reliable way, you know, taking, taking measurements.

• 1:02:38Speaker 01

  We need for people who are operating those robots to understand what the robots are doing. If the robot gets into, needs help or, or, you know, is in trouble or something, it needs to be able to communicate. And a physical indication of some sort to, so that a person looks at the robot and goes, oh, I know what that's, that robot's doing. The robot's going to go take measurements of my vacuum pump with its thermal camera. You know, you want to be able to indicate that. And we're even just the robot's about to turn, you know, in front of you and maybe indicate that it's going to turn. And so you sort of see and can anticipate its motion. So these, this kind of communication is going to become more and more important.

• 1:03:23Speaker 01

  It wasn't sort of our starting point, you know, but now the, the robots are really out in the world and, you know, we have about a thousand of them out with, with customers right now. This layer of physical indication, I think, is going to become more and more important.

• 1:03:41Lex Fridman?

  We'll talk about where it goes because there's a lot of interesting possibilities. But if you can return back to the origins of Boston Dynamics with so that the more research, the R and D side before we talk about how to build robots at scale. So Big Dog, who's Big Dog?

• 1:04:00Speaker 01

  So the company started in 1992 and in probably 2003, I believe, is when we took a contract from DARPA, so basically 10 years, 11 years. We weren't doing robotics. We did a little bit of robotics with Sony. They had IBO, their IBO robot. We were developing some software for that that kind of got us a little bit involved with robotics again. Then there's this opportunity to do a DARPA contract where they wanted to build a robot dog. And we won a contract to build that. And so that was the genesis of Big Dog.

• 1:04:45Speaker 01

  And it was a quadruped. And it was the first time we built a robot that had everything onboard that you could actually take the robot out into the wild and operate it. So it had onboard power plant, it had onboard computers, it had hydraulic actuators that needed to be cooled. So we had cooling systems built in. Everything integrated into the robot. And that was a pretty rough start, right? It was 10 years that we were not a robotics company. We were a simulation company and then we had to build a robot in about a year. So that was a little bit of a rough transition.

• 1:05:21Lex Fridman?

  Can you just comment on the roughness of that transition? Because Big Dog, I mean, this is this big quadruped, four legs robot.

• 1:05:32Speaker 01

  We built a few different versions of them, but the first one, the very earliest ones, you know, didn't work very well. We would take them out and it was hard to get, you know, a go-kart engine driving a hydraulic pump. And, and you know, having that all work while trying to get, you know, the robot to stabilize itself.

• 1:05:55Lex Fridman?

  So what was the power plant? What was the engine? It seemed like my vague recollection.

• 1:06:02Lex Fridman?

  I don't know. It felt very loud and aggressive and kind of thrown together.

• 1:06:09Speaker 01

  Oh, it absolutely was, right? We weren't trying to design the best robot hardware at the time. And we wanted to buy an off the shelf engine. And so many of the early versions of Big Dog had literally go kart engines or something like that.

• 1:06:25Lex Fridman?

  Those gas powered?

• 1:06:26Speaker 01

  Like a, yeah, gas powered two stroke engine. And the reason why it was two stroke is two stroke engines are lighter weight. But they're also, and we, we generally didn't put mufflers on them because we're trying to save the weight. And we didn't care about the noise. And so these things were horribly loud. But we're trying to manage weight because managing weight in a legged robot is always important because it has to carry everything.

• 1:06:49Lex Fridman?

  That said, that thing was big. Well, I've seen the videos.

• 1:06:52Speaker 01

  Yeah. I mean, the, the early versions, you know, stood about, I don't know, belly high, chest high. You know, they probably weighed maybe a couple of hundred pounds. But, you know, it, over the course of probably five years, we were able to get that robot to really manage a remarkable level of rough terrain. So, you know, we started out with just walking on the flat and then we started walking on rocks and then inclines and then mud and then slippery mud. And, you know, by the end of that program, we were convinced that legged locomotion in a robot could actually work. Cause you know, going into it, we didn't, we didn't know that. We had built quadrupeds at, at MIT.

• 1:07:38Speaker 01

  But they were, they used a giant hydraulic pump, you know, in the lab. They use a giant computer that was in the lab. They're always tethered to the lab. This was the first time something that was sort of self-contained, you know, walked around in the world and balanced. But and the purpose was to prove to ourselves that the legged locomotion could really work. And so Big Dog really cut that open for us. And it was the beginning of what became a whole series of robots. So, once, once we showed to DARPA that you could make a legged robot that could work, there was a period at DARPA where robotics got really hot and there was lots of different programs. And, you know, we were able to build other robots.

• 1:08:19Speaker 01

  We built other quadrupeds to hand like LS3, designed to carry heavy loads. We built Cheetah, which was designed to explore what are the limits to how fast you can run? We began to build sort of a portfolio of machines and software that let us build not just one robot, but a whole family of robots. To push the limits in all kinds of directions. Yeah, and to discover those principles. You know, you asked earlier about the art and science of legged locomotion. We were able to develop principles of legged locomotion so that we knew how to build a small-legged robot or a big one. So leg, leg length, you know, was now a parameter that we could play with. Payload was a parameter we could play with.

• 1:09:04Speaker 01

  So we built the LS3, which was an 800-pound robot designed to carry a 400-pound payload. And we learned the design rules, basically developed the design rules. How do you scale different robot systems to, you know, their terrain, to their walking speed, to their payload? So, when was Spot born? Around 2012 or so. So, again, almost 10 years into sort of a run with DARPA where we built a bunch of different quadrupeds. We had a sort of a different thread where we started building humanoids.

• 1:09:44Speaker 01

  We, we, we saw that probably an end was coming where the government was going to kind of back off from a lot of robotics investment. And in order to maintain progress, we just deduced that, well, we probably need to sell ourselves to somebody who wants to continue to invest in this, this area. And that was Google. And so at Google, we would meet regularly with Larry Page. And Larry just started asking us, you know, what's your product going to be? And, you know, the logical thing, the thing that we had the most history with, that we wanted to continue developing was a quadruped. But we knew it needed to be smaller. We knew it couldn't have a gas engine.

• 1:10:29Speaker 01

  We thought it probably couldn't be hydraulically actuated. So that began the process of exploring if we could migrate to a smaller electrically actuated robot. And that was really the genesis of Spot.

• 1:10:45Lex Fridman?

  So not a gas engine and the actuators are electric?

• 1:10:49Speaker 01

  Yes.

• 1:10:49Lex Fridman?

  So can you maybe comment on what it's like at Google, working with Larry Page, having those meetings and thinking of what will a robot look like? That could be built at scale, like starting to think about a product.

• 1:11:06Speaker 01

  Larry always liked the toothbrush test. He wanted products that you used every day.

• 1:11:15Speaker 01

  What they really wanted was, you know, a consumer level product, something that would work in your house.

• 1:11:23Speaker 01

  We didn't think that was the right next thing to do. Because to be a consumer level product, cost is going to be very important. Probably need to cost a few thousand dollars. And we were building these machines that cost hundreds of thousands of dollars, maybe a million dollars to build. Of course, we were only building a two, but we didn't see how to get all the way to this consumer level product.

• 1:11:47Lex Fridman?

  In a short amount of time.

• 1:11:48Speaker 01

  In a short amount of time. And he suggested that we make the robots really inexpensive. And part of our philosophy has always been. Build the best hardware you can. Make the machine operate well so that you're trying to solve, you know, discover the hard problem that you don't know about. Don't make it harder by building a crappy machine, basically. Build the best machine you can. There's plenty of hard problems to solve that are going to have to do with, you know, under actuated systems and balance. And so we wanted to build these high quality machines still. And we thought that was important for us to continue learning about really what was

• 1:12:32Speaker 01

  the important parts of that make robots work. And so there was a little bit of a philosophical difference there. And so ultimately, that's why we're building robots for the industrial sector now. Because the industry can afford a more expensive machine because, you know, their productivity depends on keeping their factory going. And so if spot costs, you know, $100,000 or more, that's not such a big expense to them. Whereas at the consumer level, no one's going to buy a robot like that. And I think we might eventually get to a consumer level product that will be that cheap. But I think the path to getting there needs to go through these really nice machines.

• 1:13:15Speaker 01

  So we can then learn how to simplify.

• 1:13:18Lex Fridman?

  So what can you say to the almost engineering challenge of bringing down cost of a robot? So that presumably when you try to build the robot at scale, that also comes into play when you're trying to make money on robot, even in the industrial setting. But how interesting, how challenging of a thing is that? In particular, probably new to an R and D company.

• 1:13:46Speaker 01

  Yeah, I'm glad you brought that last part up. The transition from an R and D company to a commercial company, that's the thing you worry about, you know, because you've got these engineers who love hard problems, who want to figure out how to make robots work. And you don't know if you have engineers that want to work on the quality and reliability and cost that is ultimately required.

• 1:14:08Speaker 01

  And indeed, you know, we have brought on a lot of new people who are inspired by those problems. But, but the big takeaway lesson for me is we have good people. We have engineers who want to solve problems. And, and the quality and cost and manufacturability is just another kind of problem. And because they're so invested in what we're doing, they're interested in and will go work on, on those problems as well. And so I think we're managing that transition very well. In fact, I'm really pleased that, I mean, it's a huge undertaking, by the way, right? So even having, to get reliability to where it needs to be, we have to have fleets of robots that we're just operating 24/7 in

• 1:14:53Speaker 01

  our offices to go find those rare failures and eliminate them. It's just a totally different kind of activity than the research activity where you get it to work, you know, the one robot you have to work in a repeatable way, you know, at the, at the high stakes demo. It's just very different. But I think we're making remarkable progress, I guess.

• 1:15:14Lex Fridman?

  So one of the cool things, I got a chance to visit Boston Dynamics and I mean, one of the things that's really cool is to see a large number of robots moving about. Because I think one of the things you notice in the research environment is MIT, for example, I don't think anyone ever has a working robot for a prolonged

• 1:15:37Speaker 01

  Exactly.

• 1:15:39Lex Fridman?

  So like, most robots are just sitting there in a sad state of despair, waiting to be born, brought to life for a brief moment of time. But just to have, I just, I just remember there's like a, there's a spot robot just had like a cowboy hat on and was just walking randomly for whatever reason, I don't even know. But there's a kind of A sense of sentience to it because it doesn't seem like anybody was supervising it. It was just doing his thing.

• 1:16:07Speaker 01

  I'm gonna stop way short of the sentience. It is the case that if you come to our office, you know, today and walk around the hallways, you're gonna see a dozen robots just kind of walking around.

• 1:16:18Lex Fridman?

  Yes.

• 1:16:18Speaker 01

  All the time. And that's really a reliability test for us. So we have these robots programmed to do autonomous missions, get up off their charging dock, walk around the building, collect data at a few different places and go sit back down. And we want that to be a very reliable process because that's what somebody who's running a brewery, a factory, that's what they need the robot to do. And so we have to, we have to dog food our own robot. We have to test it in that way. And so on a weekly basis, we have robots that are accruing something like 1,500 or maybe 2,000 kilometers of walking and, you know, over a thousand hours of operation every week.

• 1:17:04Speaker 01

  And that's something that almost, I don't think anybody else in the world can do because A, you have to have a fleet of robots to just accrue that much information. You have to be willing to dedicate it to that test. And so that's, but that's essential.

• 1:17:18Lex Fridman?

  That's how you get the reliability.

• 1:17:20Speaker 01

  That's how you get it.

• 1:17:21Lex Fridman?

  What about some of the cost cutting from the, from the manufacturer's side? What, what have you learned from the manufacturer's side of the transition from R and D?

• 1:17:29Speaker 01

  And we're still, we're still learning a lot there. We're learning how to cast parts instead of mill it all out of, you know, billet aluminum. We're learning how to get plastic molded parts. We're, and we're learning about how to control that process so that you can build the same robot twice in a row. There's a lot to learn there and we're only partway through that process. We've, we've set up a manufacturing facility in Wolfam. It's about a mile from our headquarters and we're doing final assembly and test of both spots and stretches, you know, at that factory. And

• 1:18:07Speaker 01

  And it's hard because, to be honest, we're still iterating on the design of the robot. As we find failures from these reliability tests, we need to go engineer changes. And those changes need to now be propagated to the manufacturing line. And that's a hard process, especially when you want to move as fast as we do. And that's been challenging. And it makes it, you know, the folks who are working supply chain, who are trying to get the cheapest parts for us, kind of requires that you buy a lot of them to make them cheap. And then we go change the design from underneath them. And they're like, what are you doing? And so, you know, getting everybody on the same page here, that, yep, we still need to move fast, but we also need to try to figure out how to reduce cost.

• 1:18:49Speaker 01

  That's one of the challenges of this migration we're going through.

• 1:18:53Lex Fridman?

  And over the past few years, challenges to the supply chain. I imagine you've been a part of a bunch of stressful meetings.

• 1:18:59Speaker 01

  Yeah, things got more expensive and harder to get. And yeah, so it's, it's all been a challenge.

• 1:19:05Lex Fridman?

  Is there still room for simplification?

• 1:19:07Speaker 01

  Oh, yeah, much more. And, you know, these are really just the first generation of these machines. We're already thinking about what the next generation of Spot's going to look like. Spot was built as a platform. So you could put almost any sensor on it. And we provided data communications, mechanical connections, power connections. And, but for example, in the applications that we're excited about, where you're, you're monitoring these factories for their health, there's probably a simpler machine that we could build that's really focused on that use case. And that's the difference between the general purpose machine or the platform versus the purpose-built machine.

• 1:19:50Speaker 01

  And so, even though, even in the factory, we'd still like the robot to do lots of different tasks. If it's, if we really knew on day one that we're going to be operating in a factory with these three sensors in it, we would have it all integrated in a package that would be easier, more, less expensive and more reliable. So we're contemplating building, you know, a next generation of that machine.

• 1:20:11Lex Fridman?

  So you mentioned that, so Spot, for people who somehow are not familiar, so is a yellow robotic dog. And has been featured in many dance videos. It also has gained an arm.

• 1:20:29Lex Fridman?

  So what can you say about the arm that Spot has, about the challenges of this design and the manufacture of it?

• 1:20:36Speaker 01

  We think the future of mobile robots is mobile manipulation. That's where, you know, in the past 10 years, it was getting mobility to work, getting the legged locomotion to work. If you ask, what's the hard problem in the next 10 years? It's getting a mobile robot to do useful manipulation for you. And so we wanted Spot to have an arm to experiment with those problems. And the arm is almost as complex as the robot itself, you know? And it's a, it's an attachable payload. It has, you know, several motors and actuators and sensors. It has a camera in the end of its hand.

• 1:21:20Speaker 01

  So, you know, you can sort of see something and the robot will control the motion of its hand to go pick it up autonomously. So in the same way, the robot walks and balances, managing its own foot placement to stay balanced. We want manipulation to be mostly autonomous, where the robot, you indicate, okay, go grab that bottle. And then the robot will just go do it using the camera in its hand and then sort of closing in on that, the grasp. But it's, it's a whole nother complex robot on top of a complex legged robot. And so, and of course, we made it, the hand look a little like a head, you know? Because again, we want it to be sort of identifiable.

• 1:22:03Speaker 01

  In the last year, a lot of our sales have been people who already have a robot now buying an arm to add to that robot.

• 1:22:11Lex Fridman?

  Oh, interesting. And so the, the arm is for sale?

• 1:22:15Speaker 01

  Oh, yeah. It's an option.

• 1:22:17Lex Fridman?

  What's the, what's the interface like to work with the arm? Like, is it pretty? So are they designed primarily, I guess, just ask that question in general about robots from Boston Dynamics. Is it designed to be easily and efficiently operated remotely by a human being? Or is there also the capability to push towards autonomy?

• 1:22:41Speaker 01

  We want both. In the next version of the software that we release, which will be version 3.3, we're going to offer the ability of, if you have an autonomous mission for the robot, we're going to include the option that it can go through a door, which means it's going to have to have an arm and it's going to have to use that arm to open the door. And so that'll be an autonomous manipulation task that just, you can program easily with the robot, strictly through, you know, we have a tablet interface. And so on the tablet, you know, you sort of see the view that Spot sees. You say, there's the door handle. You know, the hinges are on the left and it opens in. The rest is up to you. Take care of it.

• 1:23:24Lex Fridman?

  Oh, so it just takes care of everything?

• 1:23:25Speaker 01

  Yeah. So we want, and for a task like opening doors, you can automate most of that. And we've automated a few other tasks. We had a customer who had a high-powered breaker switch, essentially. It's an electric utility, Ontario Power Generation. And they have to, when they're going to disconnect, you know, their power supply, right? That could be a gas generator, it could be a nuclear power plant, you know, from the grid, you have to disconnect this breaker switch. Well, as you can imagine, there's, you know, hundreds or thousands of amps and volts involved in this breaker switch. And it's a dangerous event because occasionally you'll get what's called an arc flash as you just do this disconnect.

• 1:24:08Speaker 01

  The power, the sparks jump across and people die doing this. And so Ontario Power Generation used our Spot and the arm through the interface to operate this disconnect in an interactive way. And they showed it to us. And we were so excited about it and said, you know, I bet we can automate that task. And so we got some examples of that breaker switch. And I believe in the next generation of the software, now we're going to deliver back to Ontario Power Generation, they're going to be able to just point the robot at that breaker, they'll be, they'll indicate that's the switch. There's sort of two actions you have to do.

• 1:24:51Speaker 01

  You have to flip up this little cover, press a button, then get a ratchet, stick it into a socket and literally unscrew this giant breaker switch. So there's a bunch of different tasks. And we basically automated them so that the human says, okay, there's the switch, go do that part. That right there is the socket where you're gonna put your tool and you're gonna open it up. And so you can remotely sort of indicate this on the tablet and then the robot just does everything in between.

• 1:25:21Lex Fridman?

  And it does everything, all the coordinated movement of all the different actuators that includes the body.

• 1:25:25Speaker 01

  Yeah, maintains its balance, it walks itself into position so it's within reach and the arm is in a position where it can do the whole task. So it manages the whole body.

• 1:25:38Lex Fridman?

  So how does one become a big enough customer to request features? Cause I personally want a robot that gets me a beer. I mean, that has to be like one of the most requests, I suppose, in the industrial setting. That's a non-alcoholic beverage of picking up objects and bringing the objects to you.

• 1:25:58Speaker 01

  We love working with customers who have challenging problems like this and this one in particular because we felt like what they were doing, A, it was a safety feature, B, we saw that the robot could do it because they teleoperated it the first time, probably took them an hour to do it the first time, right? But the robot was clearly capable and we thought, oh, this is a great problem for us to work on to figure out how to automate a manipulation task. And so we took it on, not because we were gonna make a bunch of money from it and selling the robot back to them, but because it motivated us to go solve what we saw as the next logical step. But many of our customers, in fact, we try to, our bigger customers, typically ones who

• 1:26:43Speaker 01

  are gonna run a utility or a factory or something like that, we take that kind of direction from them. And if they're, especially if they're gonna buy 10 or 20 or 30 robots and they say, I really need it to do this, well, that's exactly the right kind of problem that we wanna be working on. And so.

• 1:26:58Lex Fridman?

  Note to self, buy 10 spots and aggressively push for beer manipulation. I think it's fair to say it's notoriously difficult to make a lot of money as a robotics company. How can you make money as a robotics company? Can you speak to that? It seems that a lot of robotics companies fail. It's difficult to build robots. It's difficult to build robots at a low enough cost where customers, even the industrial side, want to purchase them. And it's difficult to build robots that are useful, sufficiently useful. So what can you speak to? And Boston Dynamics has been successful for many years of finding a way to make money.

• 1:27:41Speaker 01

  Well, in the early days, of course, you know, the money we made was from doing contract R and D work. And we made money, but you know, we weren't growing and we weren't selling a product. And then we went through several owners who, you know, had a vision of not only developing advanced technology, but eventually developing products. And so both, you know, Google and SoftBank and now Hyundai, you know, had that vision and were willing to, you know, provide that investment. Now, our discipline is that we need to go find applications that are broad enough that you could imagine selling thousands of robots.

• 1:28:24Speaker 01

  Because it doesn't work if you don't sell thousands or tens of thousands of robots. If you only sell hundreds, you will commercially fail. And that's where most of the small robot companies have died. And that's a challenge because, you know, A, you need to field the robots, they need to start to become reliable. And as we've said, that takes time and investment to get there. And so it really does take visionary investment to get there. But we believe that we are going to make money in this industrial monitoring space. Because, you know, if a chip fab, if the line goes down because a vacuum pump failed

• 1:29:09Speaker 01

  someplace, that can be a very expensive process. It can be a million dollars a day in lost production. Maybe you have to throw away some of the product along the way. And so the robot, if you can prevent that by inspecting the factory every single day, maybe every hour if you have to. There's a real return on investment there. But there needs to be a critical mass of this task. And we're focusing on a few that we believe are ubiquitous in the industrial production environment. And that's using a thermal camera to keep things from overheating, using an acoustic imager to find compressed air leaks,

• 1:29:51Speaker 01

  using visual cameras to read gauges, measuring vibration. These are standard things that you do to prevent unintended shutdown of a factory. And this takes place in a beer factory. We're working with AB Inbev. It takes place in chip fabs. You know, we're working with Global Foundries. It takes place in electric utilities and nuclear power plants. And so the same robot can be applied in all of these industries. And, and as I said, we have about, actually it's 1,100 spots out now. To really get profitability, we need to be at 1,000 a year, maybe, maybe 1,500 a year, you know, for that sort of part of the business.

• 1:30:36Speaker 01

  So it still needs to grow, but, but we're on a good path. So I think that's totally achievable.

• 1:30:42Lex Fridman?

  So the application should require crossing that 1,000 robot barrier.

• 1:30:46Speaker 01

  It really should. Yeah. I want to mention our second robot, Stretch.

• 1:30:51Lex Fridman?

  Yeah. Tell me about Stretch. What's Stretch? Who's Stretch?

• 1:30:54Speaker 01

  Stretch started differently than Spot. You know, Spot, we built because we had decades of experience building quadrupeds. We just, we had it in our blood. We had to build a quadruped product. But we had to go figure out what the application was. And we actually discovered this, this factory patrol application, basically preventative maintenance, by seeing what our customers did with it. Stretch is very different. We started knowing that there was warehouses all over the world. There's shipping containers moving all around the world full of boxes that are mostly being moved by hand. By some estimates, we think there's a trillion boxes, cardboard boxes shipped around the world each year, and a lot of it's done manually.

• 1:31:39Speaker 01

  It became clear early on that there was an opportunity for a mobile robot in here to move boxes around. And the commercial experience has been very different between Stretch and with Spot. As soon as we started talking to people, potential customers, about what Stretch was going to be used for, they immediately started saying, oh, I'll buy, I'll buy that robot. You know, in fact, I'm going to put in an order for 20 right now. We just started shipping the robot in January, after several years of development.

• 1:32:09Lex Fridman?

  This year.

• 1:32:10Speaker 01

  This year. So our first deliveries of Stretch to customers were DHL and Maersk in January. We're delivering to Gap right now. And we have about seven or eight other customers, all who've already agreed in advance to buy between 10 and 20 robots. And so we've already got commitments for a couple hundred of these robots. This one's going to go, right? It's so obvious that there's a need. And we're not just going to unload trucks, we're going to do any box moving task in the warehouse. And so it too will be a multipurpose robot. And we'll eventually have it doing palletizing or de-palletizing, or loading trucks, or unloading trucks. There's definitely thousands of robots. There's probably tens of thousands of robots of this in the future.

• 1:32:54Speaker 01

  So it's going to be profitable.

• 1:32:56Lex Fridman?

  Can you describe what Stretch looks like?

• 1:32:58Speaker 01

  It looks like a big, strong robot arm on a mobile base. The base is about the size of a pallet. And we want it to be the size of a pallet because that's what lives in warehouses, right? Pallets of goods sitting everywhere. So it needed to be able to fit in that space.

• 1:33:13Lex Fridman?

  It's not a legged robot.

• 1:33:14Speaker 01

  It's not a legged robot. And so it was our first, it was actually a bit of a commitment from us, a challenge for us, to build a non-balancing robot.

• 1:33:28Lex Fridman?

  To do the much easier problem and to do.

• 1:33:31Speaker 01

  Well, because it wasn't, you know, it wasn't going to have this balance problem. And in fact, the very first version of the logistics robot we built was a balancing robot. And that's called Handle.

• 1:33:43Lex Fridman?

  And there's. That thing was epic.

• 1:33:44Speaker 01

  Oh, it's a beautiful machine.

• 1:33:46Lex Fridman?

  It's an incredible machine. So it was, I mean, it looks epic. It looks like out of, I mean, out of the sci fi movies of some sorts. I mean, just can you actually just linger on the like the design of that thing? Cause that's another leap into something you probably haven't done. It's a different kind of balancing.

• 1:34:05Speaker 01

  Yeah. So let me, I'd love, I love talking about the history of how a handle came about. Because it connects all of our robots, actually. So I'm going to start with Atlas. When we, when we had Atlas getting fairly far along, we wanted to understand, I was telling you earlier, the challenge of the human form is you have this mass up high. And balancing that inertia, that mass up high is its own unique challenge. And so we started trying to get Atlas to balance standing on one foot, like on a balance beam, using its arms like this. And you know, you can do this, I'm sure I can do this, right? Like if you're walking a tightrope, how do you do that balance?

• 1:34:47Speaker 01

  So that's sort of, you know, controlling the inertia, controlling the momentum of the robot. We were starting to figure that out on Atlas.

• 1:34:56Speaker 01

  And so our first concept of Handle, which was a robot that was going to be on two wheels. So it had to balance, but it was going to have a big, long arm. So it could reach a box at the top of a truck. And it was gonna, it needed yet another counterbalance, a big tail, to help it balance while it was using its arm. So the reason why this robot sort of looks epic, some people said it looked like an ostrich or maybe an ostrich moving around was the wheels, the leg, it has legs. So it can extend its legs. So it's wheels on legs. We always wanted to build wheels on legs. It had a tail and had

• 1:35:37Speaker 01

  this arm and they're all moving simultaneously and in coordination to maintain balance because we had figured out the mathematics of doing this momentum control, how to maintain that balance. And so part of the reason why we built this two-legged robot was we had figured this thing out. We wanted to see it in this kind of machine and we thought maybe this kind of machine would be good in a warehouse. And so we built it and it's a beautiful machine. It moves in a graceful way like nothing else we've built, but it wasn't the right machine for a logistics application. We decided it was too slow and couldn't pick boxes fast enough, basically.

• 1:36:14Lex Fridman?

  It would do it beautifully with elegance.

• 1:36:16Speaker 01

  Beautifully, but it just wasn't efficient enough. So we let it go.

• 1:36:20Lex Fridman?

  Yeah.

• 1:36:21Speaker 01

  But I think we'll come back to that machine eventually.

• 1:36:25Lex Fridman?

  The fact that it's possible, the fact that you showed that you could do so many things at the same time in coordination and so beautifully, there's something there.

• 1:36:33Speaker 01

  Yeah.

• 1:36:33Lex Fridman?

  That was a demonstration of what is possible.

• 1:36:36Speaker 01

  Basically, we made a hard decision and this was really kind of a hard-nosed business decision. It was, it was, it indicated us not doing it just for the beauty of the mathematics or the curiosity, but no, we actually need to build a business that can make money in the long run. And so we ended up building Stretch, which has a big, heavy base with a giant battery in the base of it that allows it to run for two, two shifts, 16 hours worth of operation. And that big battery is, sort of helps it stay balanced, right? So you can move a 50-pound box around with its arm and not tip over. It's omnidirectional. It can move in any direction. So it has a nice suspension built into it so it can deal with

• 1:37:20Speaker 01

  gaps or things on the floor and roll over it. But it's a, but it's not a balancing robot. It's a mobile robot arm that can work to carry or pick or place a box up to 50 pounds anywhere in the warehouse.

• 1:37:34Lex Fridman?

  From, take a box from point A to point B, anywhere.

• 1:37:37Speaker 01

  Yeah. Palatized, de-palatized. We're starting with unloading trucks because there's so many trucks and containers that where goods are shipped and it's a brutal job. You know, in the summer, it can be 120 degrees inside that container. People don't want to do that job. And it's backbreaking labor, right? Again, these can be up to 50 pound boxes. And so we feel like this is a productivity enhancer. And for the people who used to do that job unloading trucks, they're actually operating the robot now. And so by building robots that are easy to control, and it doesn't take an advanced degree to manage, you can become a robot operator. And so as we've introduced these robots to both DHL and Mariscan Gap,

• 1:38:23Speaker 01

  the warehouse workers who were doing that manual labor are now the robot operators. And so we see this as ultimately a benefit to them as well.

• 1:38:32Lex Fridman?

  Can you say how much stretch costs?

• 1:38:37Speaker 01

  Not yet. But I will say that when we engage with our customers, they'll be able to see a return on investment in typically two years.

• 1:38:47Lex Fridman?

  Okay, so that's something that you're constantly thinking about, how?

• 1:38:50Speaker 01

  Yeah.

• 1:38:51Lex Fridman?

  And I suppose you have to do the same kind of thinking with Spot. So it seems like with Stretch, the application is like directly obvious.

• 1:38:59Speaker 01

  Yeah, it's a slam dunk.

• 1:39:00Lex Fridman?

  Yeah, and so you can, you have a little more flexibility.

• 1:39:03Speaker 01

  Well, I think we know the target. We know what we're going after.

• 1:39:06Lex Fridman?

  Yeah.

• 1:39:07Speaker 01

  And with Spot, it took us a while to figure out what we were going after.

• 1:39:10Lex Fridman?

  Well, let me return to that question about

• 1:39:14Lex Fridman?

  maybe the conversation you were having a while ago with Larry Page, maybe looking to the longer future of social robotics, of using Spot to connect with human beings, perhaps in the home. Do you see a future there? If we were to sort of hypothesize or dream about a future where Spot like robots are in the home as pets of social robots?

• 1:39:36Speaker 01

  We definitely think about it, and we would like to get there. We think the pathway to getting there is likely through these industrial applications and then mass manufacturing. Let's figure out how to build the robots, how to make the software so that they can really do a broad set of skills. That's going to take real investment to get there. Performance first, right? The principle of the company has always been really make the robots do useful stuff. And so the social robot companies that tried to start someplace else by just making acute interaction, mostly they haven't survived.

• 1:40:17Speaker 01

  And so we think the utility really needs to come first. And that means you have to solve some of these hard problems. And so to get there, we're going to go through the design and software development in industrial. And then that's eventually going to let you reach a scale that could then be addressed to a commercial, a consumer level market. And so, yeah, maybe we'll be able to build a smaller spot with an arm that could really go get your beer for you. But there's things we need to figure out still. How to safely, really safely. And if you're going to be interacting with children, you better be safe.

• 1:40:57Speaker 01

  And right now, we count on a little bit of standoff distance between the robot and people so that you don't pinch a finger in the robot. So you've got a lot of things you need to go solve before you jump to that consumer level product.

• 1:41:11Lex Fridman?

  Well, there's a kind of trade off in safety because it feels like in the home, you can fall. Like you're, you don't have to be as good at, like you're allowed to fail in different ways, in more ways, as long as it's safe for the humans. So it just feels like an easier problem to solve because it feels like in the factory, you're not allowed to fail.

• 1:41:36Speaker 01

  That may be true. But I also think the variety of things a consumer level robot would be expected to do will also be quite broad.

• 1:41:46Lex Fridman?

  Yeah.

• 1:41:46Speaker 01

  And they're going to want to get the beer and know the difference between the beer and a Coca Cola or my snack. And, or, you know, they're all going to want you to clean up the dishes, you know, from the table without breaking them. Those are pretty complex tasks. And so there's, there's still work to be done there.

• 1:42:06Lex Fridman?

  So to push back on that, here's where I application, I think they'll be very interesting. I think the application of being a pet, a friend. So like no tasks.

• 1:42:17Lex Fridman?

  Just be cute, because I not cute, not cute. Like the dog is more, a dog is more than just cute. A dog is a friend, is a companion. There's something about just having interacted with them. And maybe because I'm hanging out alone with the robot dogs a little too much. But like, there's a, there's a connection there. And it feels like that connection is not, should not be disregarded.

• 1:42:41Speaker 01

  No, it should not be disregarded. Robots that can somehow communicate through their physical gestures are, you're going to be more attached to in the long run. Do you remember Ibo, the Sony Ibo?

• 1:42:54Lex Fridman?

  Yeah.

• 1:42:54Speaker 01

  They sold over a hundred thousand of those, maybe a hundred and fifty thousand. You know, what probably wasn't considered a successful product for them. They suspended that eventually, and then they brought it back. Sony brought it back. And people definitely, you know, treated this as a, as a pet, as a, as a companion. And I think that will come around again. Will you get away without having any other utility? Maybe in a world where we can really talk to our simple little pet, because, you know, chat GPT or some other generative AI has made it possible for you to really talk in what seems like a meaningful

• 1:43:34Speaker 01

  way. Maybe that'll open the social robot up again. That's probably not a path we're going to go down, because, again, we're so focused on performance and utility. We can add those other things also, but we really want to start from that foundation of utility, I think.

• 1:43:55Lex Fridman?

  Yeah. But I also want to predict that you're wrong on that. So which is that the very path you're taking, which is creating a great robot platform, will very easily take a leap to adding a chat GPT-like capability, maybe GPT-5. And there's just so many open source alternatives, you could just plop that on top of Spot. And because you have this robust platform, and you're figuring out how to mass manufacture it, and how to drive the cost down, and how to make it, you know, reliable, all those kinds of things, it'll be a natural transition to where just adding chat GPT.

• 1:44:32Speaker 01

  Oh, I do think that. Being able to verbally converse or even converse through gestures. You know, part of part of these learning models is that, you know, you can now look at video and imagery and associate, you know, intent with that. Those will all help in the communication between robots and people for sure. And that's going to happen, obviously, more quickly than any of us were expecting.

• 1:45:00Lex Fridman?

  I mean, what else do you want from life? Friends, get your beer and then just talk shit about the state of the world. I mean, there's a deep loneliness within all of us. And I think a beer and a good chat solves so much of it. Or it takes us a long way to solving.

• 1:45:21Speaker 01

  It'll be interesting to see, you know, when when a generative AI can give you that warm feeling that you connected, you know, and that, oh, yeah, you remember me, you're my friend, you know, we have a history. You know, that history matters, right?

• 1:45:41Lex Fridman?

  Memory of joint like.

• 1:45:42Speaker 01

  Memory of, yeah.

• 1:45:44Lex Fridman?

  Having witnessed, that's what friendship, that's what connection, that's what love is. In many cases, some of the deepest friendships you have is having gone through a difficult time together and having a shared memory of an amazing time or a difficult time. And kind of that memory creating this like foundation based on which you can then experience the world together. The silly, the mundane stuff of day to day is somehow built on a foundation of having gone through some shit in the past. And the the current systems are not personalized in that way. But I think that's a technical problem, not some kind of fundamental limitation. So combine that with an embodied robot like Spot, which already has magic in its movement.

• 1:46:29Lex Fridman?

  I think it's a very interesting possibility of where that takes us. But of course, you have to build that on top of a company that's making money with real application with real customers and with robots that are safe and work and reliable and manufactured scale.

• 1:46:48Speaker 01

  And I think we're in a unique position in that because of our investors, primarily Hyundai, but also SoftBank still owns 20% of us. They don't, they're not totally fixated on. Driving us to profitability as soon as possible. That's not the goal. The goal really is a longer term vision of creating, you know, what does mobility mean in the future? What, how is this mobile robot technology going to influence us? Can we, and can we shape that? And they want both. And so we are, as a company, are trying to strike that balance between, let's build a business that makes money.

• 1:47:30Speaker 01

  I've been describing that to my own team as self destination. If I want to, if I want to drive my own ship, we need to have a business that's profitable in the end. Otherwise, somebody else is going to drive the ship for us. So that's really important. But we're going to retain the aspiration that we're going to build the next generation of technology at the same time. And the real trick will be if we can do both.

• 1:47:55Lex Fridman?

  Speaking of ships, let me ask you about a competitor. And somebody's become a friend. So Elon Musk and Tesla have announced, have been in the early days of building a humanoid robot. How does that change the landscape of, of your work? So there's sort of from the outside perspective, it seems like, well, from a fan, as a fan of robotics, it just seems exciting.

• 1:48:24Speaker 01

  Right, very exciting, right? When, when Elon speaks, people listen. And so it suddenly brought a bright light onto the work that we'd been doing, you know, for over a decade. And, and I think that's only going to help. And in fact, what we've seen is that in addition to Tesla, we're seeing a proliferation of robotic companies arise now.

• 1:48:50Lex Fridman?

  Including humanoid?

• 1:48:51Speaker 01

  Yes.

• 1:48:52Lex Fridman?

  Oh, wow.

• 1:48:52Speaker 01

  Yeah, so, and interestingly, many of them, as they're, you know, raising money, for example, will claim whether or not they have a former Boston Dynamics employee on their staff as a criteria.

• 1:49:07Lex Fridman?

  Yeah, that's true. That's a I would do that as a company. Yeah, for sure.

• 1:49:12Speaker 01

  Yeah, so.

• 1:49:13Lex Fridman?

  Shows you're legit.

• 1:49:14Speaker 01

  Yeah. So you know what? It's bring it has brung a tremendous validation to what we're doing. And excitement, competitive juices are flowing, you know, the whole thing. So it's all good.

• 1:49:28Lex Fridman?

  Elon has also kind of stated that, you know, maybe he implied that the problem is solvable in the near term, which is a low cost humanoid robot that's able to do that's a relatively general use case robot. So I think Elon is known for sort of setting these kinds of incredibly ambitious goals.

• 1:50:01Lex Fridman?

  Maybe missing deadlines, but actually pushing not just the particular team of these, but the entire world to like accomplishing those. Do you see? Do you see Boston Dynamics in the near future being pushed in that kind of way? Like this excitement of competition, kind of pushing Atlas maybe to do more cool stuff, trying to drive the cost of Atlas down perhaps. Or, I mean, I guess I want to I want to ask if there's some kind of exciting energy in Boston Dynamics due to this a little bit of competition.

• 1:50:40Speaker 01

  Oh, yeah, definitely. When we released our most recent video of Atlas, you know, I think you'd seen it, the scaffolding and throwing the box of tools around and then doing the flip at the end. We were trying to show the world that not only can we do this parkour mobility thing, but we can pick up and move heavy things. Because if you're going to work in a manufacturing environment, that's what you got to be able to do. And for the reasons I explained to you earlier, it's not trivial to do so. You know, changing the center of mass, uh, you know, by picking up a 50 pound block, you know, for a robot that weighs 150 pounds, that's a lot to accommodate.

• 1:51:25Speaker 01

  So we're trying to show that we can do that. And, um, so it's totally been energizing. You know, we see the next phase of Atlas being more dexterous hands that can manipulate and grab more things that we're going to start by moving big things around that are heavy and that affect balance. And why is that? Well, really tiny dexterous things probably are going to be hard for a while yet. You know, maybe you could go build a special purpose robot arm, you know, for, for stuffing, you know, chips into electronics boards, but we don't really want to do really fine work like that.

• 1:52:05Speaker 01

  I think more coursework where you're using two hands to pick up and balance an unwieldy thing, maybe in a manufacturing environment, maybe in a construction environment. Those are the things that we think robots are going to be able to do with the level of dexterity that they're going to have in the next few years. And that's the, that's where we're headed. And, and I think, and you know, Elon has seen the same thing, right? He's talking about using the robots in a manufacturing environment. We think there's something very interesting there about having this, a two armed robot. Because when you have two arms, you can transfer a thing from one hand to the other. You can turn it around, you know, you can, you can reorient it in a way that you can't do it if you just have one hand on it. And so there's a lot that extra arm brings to the table.

• 1:52:50Lex Fridman?

  So I think in terms of mission, you, you, you mentioned Boston, the name X really wants to see what are the, what's the limits of what's possible. And so the cost comes second. Or it's a component, but first figure out what are the limitations? I think with Elon, he's really driving the cost down. Is there some inspiration, some lessons you see there? Of the challenge of driving the cost down, especially with Atlas, with a humanoid robot?

• 1:53:18Speaker 01

  Well, I think the thing that he's certainly been learning by building car factories is what that looks like and scaling. By scaling, you can get efficiencies that drive costs down very well. And, and the smart thing that they have in their favor is that they know how to manufacture, they know how to build electric motors, they know how to build our computers and vision systems. So there's a lot of overlap between modern automotive companies and robots. But hey, we have a modern robotic, I mean, automotive company behind us as well.

• 1:54:01Lex Fridman?

  So bring it on.

• 1:54:02Speaker 01

  Who's doing pretty well, right? The electric vehicles from Hyundai are doing pretty well.

• 1:54:07Lex Fridman?

  I love it. So how much, so we've talked about some of the low level control, some of the incredible stuff that's going on and basic perception. But how much do you see in currently and in the future of Boston Dynamics sort of more higher level machine learning applications? Do you see customers adding on those capabilities or do you see Boston Dynamics doing that in house?

• 1:54:33Speaker 01

  Some kinds of things we really believe are probably going to be more broadly available, maybe even commoditized. You know, using a machine learning, like a vision algorithm. So a robot can recognize something in the environment. That ought to be something you can just download. Like I'm going to a new environment, I have a new kind of door handle or piece of equipment I want to inspect, you ought to be able to just download that. And I think people besides Boston Dynamics will provide that. And we've actually built an API that lets people add these vision algorithms to Spot. And we're currently working with some partners who are providing that. Levitas is an example of a small provider who's giving us software for reading gauges.

• 1:55:19Speaker 01

  And actually another partner in Europe, Reply, is doing the same thing. So we see that we see ultimately an ecosystem of providers doing stuff like that. And I, I think ultimately you might even be able to do the same thing with behaviors. So this technology will also be brought to bear on controlling the robot, the motions of the robot. And, you know, we're using learning, reinforcement learning to develop algorithms for both locomotion and manipulation. And ultimately, this is going to mean you can, you can add new behaviors to a robot, you know, quickly. And that could potentially be done outside of Boston Dynamics right now.

• 1:56:04Speaker 01

  And that's all internal to us. I think, I think you need to understand at a deep level, you know, the robot control to do that, but eventually that could be outside. But it's certainly a place where these, these approaches are going to be brought to bear in robotics.

• 1:56:21Lex Fridman?

  So reinforcement learning is part of the process. So you do use reinforcement learning?

• 1:56:26Speaker 01

  Yes.

• 1:56:29Lex Fridman?

  So there's increasing levels of learning with these robots?

• 1:56:32Speaker 01

  Yes.

• 1:56:33Lex Fridman?

  And that's for both for locomotion, for manipulation and for perception?

• 1:56:38Speaker 01

  Yes.

• 1:56:40Lex Fridman?

  Well, what do you think in general about all the exciting advancements of transformer neural networks most beautifully illustrated through the large language models like GPT-4?

• 1:56:58Speaker 01

  Like everybody else, we're all, you know, I'm surprised at how much, how far they've come. I'm a little bit nervous about the, there's anxiety around them, obviously, for I think good reasons, right?

• 1:57:18Speaker 01

  Disinformation is a curse that's an unintended consequence of social media that could be exacerbated with these tools. So if you use them to deploy disinformation, it could be a real risk. But I also think that the risks associated with these kinds of models don't have a whole lot to do with the way we're going to use them in our robots. If I'm using a robot, I'm building a robot to do, you know, a manual task of some sort.

• 1:57:52Speaker 01

  I can judge very easily, is it doing the task I asked it to? Is it doing it correctly? There's sort of a built in mechanism for judging, is that, is it doing the right thing? Did it successfully do the task?

• 1:58:06Lex Fridman?

  Yeah, physical reality is a good verifier.

• 1:58:08Speaker 01

  It's a good verifier. That's exactly it. Whereas if you're asking for, yeah, I don't know, you're trying to ask a theoretical question in ChatGPT, it could be true or it may not be true. And it's hard to have that verifier. What, what is that truth that you're comparing against? Whereas in physical reality, you know, the truth. And this is an important difference. And so I'm not, I think there is reason to be a little bit concerned about, you know, how these tools of large language models could be used. But I'm not very worried about how they're going to be used. Well, how learning algorithms in general are going to be used on robotics. It's, it's really a different application

• 1:58:53Speaker 01

  that has different ways of verifying what's going on.

• 1:58:57Lex Fridman?

  Well, the nice thing about language models is that I ultimately see, I'm really excited about the possibility of having conversations with Spot.

• 1:59:05Speaker 01

  Yeah.

• 1:59:06Lex Fridman?

  There's no, I would say negative consequences to that, but just increasing the bandwidth and the variety of ways you can communicate with this particular robot.

• 1:59:15Speaker 01

  Yeah.

• 1:59:16Lex Fridman?

  So you could communicate visually, you can communicate through some interface and to be able to communicate verbally, again, with the beer and so on. I think that's really exciting to make that much, much easier.

• 1:59:28Speaker 01

  We have this partner Levitas that's adding the vision algorithms for day reading for us. They just, just this week, I saw a demo where they hooked up, you know, a language tool to Spot and they're talking to Spot to give commands.

• 1:59:42Lex Fridman?

  Yeah. Can you tell me about the Boston Dynamics AI Institute? What is it and what is its mission?

• 1:59:49Speaker 01

  So it's a separate organization, the Boston Dynamics Artificial Intelligence Institute. It's led by Mark Rayburn, the founder of Boston Dynamics and the former CEO and my old advisor at MIT. Mark has always loved the research, the pure research, without the confinement or demands of commercialization. And he wanted to continue to, you know, pursue that unadulterated research. And so suggested to Hyundai that he set up this institute and they agree that it's worth additional investment to kind of continue push, pushing this forefront.

• 2:00:35Speaker 01

  And we expect to be working together where, you know, Boston Dynamics is again both commercialize and do research. But the sort of time horizon of the research we're going to do is, you know, in the next, let's say, five years. You know, what can we do in the next five years? Let's work on those problems. And I think the goal of the AI Institute is to work even further out. Certainly, you know, the analogy of legged locomotion again, when we started that, that was a multi-decade problem. And so I think Mark wants to have the freedom to pursue really hard, over the horizon problems. And that's, that'll be the goal of the institute.

• 2:01:14Lex Fridman?

  So we mentioned some of the dangers of, some of the concerns about large language models. That said, you know, there's been a long running fear of these embodied robots. Why do you think people are afraid of legged robots?

• 2:01:32Speaker 01

  Yeah, I wanted to show you this. This, so this, this is in the Wall Street Journal, and this is all about ChatGPT, right? But look at the picture.

• 2:01:41Lex Fridman?

  Yeah.

• 2:01:41Speaker 01

  It's a humanoid robot.

• 2:01:43Lex Fridman?

  That's saying I will.

• 2:01:44Speaker 01

  That's saying it looks scary and it says, I'm going to replace you.

• 2:01:47Lex Fridman?

  Yeah.

• 2:01:48Speaker 01

  And so the humanoid robot is sort of, is the embodiment of this ChatGPT tool. That there's reason to be a little bit nervous about how it gets deployed.

• 2:02:00Lex Fridman?

  Yeah.

• 2:02:01Speaker 01

  So I'm nervous about that connection.

• 2:02:05Speaker 01

  It's unfortunate that they chose to use a robot as that embodiment. For, as you and I just said, there's big differences in this. But people are afraid because we've been taught to be afraid for over 100 years. So, you know, the word robot was developed by a playwright named Carol Chapek in 1921, Czech playwright for Rossum's Universal Robots. And in that first depiction of a robot, the robots took over at the end of the story. And, you know, people love to be afraid. And so we've been entertained by these stories for 100 years. But I, and I think that's as much why people are afraid as anything else,

• 2:02:49Speaker 01

  is we've been sort of taught that this is the logical progression through fiction. I think it's fiction.

• 2:02:59Lex Fridman?

  I think what people more and more will realize, just like you said, that the threat, like say you have a super intelligent AI embodied in a robot, that's much less threatening because it's visible, it's verifiable, it's right there in physical reality, and we humans know how to deal with physical reality. I think it's much scarier when you have arbitrary scaling of intelligent AI systems in the digital space, that they could pretend to be human. So a robot, Spot is not going to be pretend, it could pretend it's human all it wants. It could tell you, you could put your GPT on top of it, but you're going to know it's not human because you have a contact with physical reality.

• 2:03:44Speaker 01

  And you're going to know whether or not it's doing what you asked it to do.

• 2:03:46Lex Fridman?

  Yeah, like it's not gonna, like if it like, I mean, I'm sure you can start, just like a dog lies to you, like I didn't, I wasn't part of tearing up that couch. So Spot can, Spot can try to lie that like, you know, it wasn't me that spilled that thing, but you're going to kind of figure it out eventually. It's, if it happens multiple times, you know. But I think that.

• 2:04:10Speaker 01

  Humanity has figured out how to make machines safe.

• 2:04:12Lex Fridman?

  Yeah.

• 2:04:13Speaker 01

  And there's, you know, the regulatory environments and certification protocols that we've developed in order to figure out how to make machines safe. We don't know and don't have that experience with software that can be propagated worldwide in an instant. And so I think we needed to develop those protocols and those tools. And so that's work to be done, but I don't think the fear of that and that work should necessarily impede our ability to now get robots out. Because again, I think, I think we can judge when a robot's being safe.

• 2:04:50Lex Fridman?

  So, and again, just like in that image, there's a fear that robots will take our jobs. I just, I took a ride, I was in San Francisco, I took a ride and there were way more vehicles than autonomous vehicle. And I've done it several times, they're, they're doing incredible work over there. But people flicked it off.

• 2:05:11Speaker 01

  Oh, right.

• 2:05:12Lex Fridman?

  Off the car. So, I mean, that's a long story of what the psychology of that is. It could be maybe big tech or what, I don't know exactly what they're flicking off.

• 2:05:23Speaker 01

  Yeah.

• 2:05:23Lex Fridman?

  But there is an element of like, these robots are taking our jobs or, or irreversibly transforming society such that it will have economic impact and the little guy will be, would lose a lot, would lose their well being. Is there something to be said about the fear that robots will take our jobs?

• 2:05:44Speaker 01

  You know, at every significant technological transformation, there's been fear of, you know, an automation anxiety.

• 2:05:54Lex Fridman?

  Yes.

• 2:05:55Speaker 01

  That it's, it's going to have a broader impact than, than we expected. And there, there will be, you know, jobs will change.

• 2:06:07Speaker 01

  Sometime in the future, we're going to look back at people who manually unloaded these boxes from trailers and we're going to say, why did we ever do that manually? But there's a lot of people who are doing that job today that it could be impacted.

• 2:06:22Speaker 01

  But I think the reality is, as I said before, we're going to build the technologies so those very same people can operate it. And so I think there's a pathway to upskilling and operating, just like, look, we used to farm with hand tools and now we farm with machines and nobody has really regretted that transformation. And I think the same can be said for a lot of manual labor that we're doing today. And on top of that, you know, look, we're, we're entering a new world where where demographics are going to have strong impact on economic growth. And the, you know, the advanced, the first world is losing population quickly.

• 2:07:04Speaker 01

  In Europe, they're worried about hiring enough people just to keep the logistics supply chain going. And, you know, part of this is the response to COVID and everybody's sort of thinking back what they really want to do with their life. But these jobs are getting harder and harder to fill. And I just, I'm hearing that over and over again. So I think frankly, this is the right technology at the right time. Where we're going to need some of this work to be done and we're going to want tools to enhance that productivity.

• 2:07:39Lex Fridman?

  And the scary impact, I think, again, GPT comes to the rescue in terms of being much more terrifying.

• 2:07:48Lex Fridman?

  The scary, scary impact of basically, so I'm a, I guess, a software person, so I program a lot. And the fact that people like me can be easily replaced by GPT, that's going to have a.

• 2:08:04Speaker 01

  Well, and lot, you know, anyone who deals with texts and writing a draft proposal might be easily done with chat GPT now.

• 2:08:13Lex Fridman?

  Consultants.

• 2:08:14Speaker 01

  Wasn't before.

• 2:08:15Lex Fridman?

  Journalists.

• 2:08:17Speaker 01

  Yeah.

• 2:08:18Lex Fridman?

  Everybody is sweating.

• 2:08:19Speaker 01

  But on the other hand, you also want it to be right. And they don't know how to make it right yet. But it might make a good starting point for you to iterate.

• 2:08:28Lex Fridman?

  Boy, do I have to talk to you about modern journalism. That's another conversation altogether. But yes, more right than the average, the, the mean journalist, yes.

• 2:08:46Lex Fridman?

  You spearheaded the anti-weaponization letter Boston Dynamics has. Can you describe what that letter states and the general topic of the use of robots in war?

• 2:09:02Speaker 01

  We authored a letter and then got several leading robotics companies around the world, including Unitree in China and Agility here in the United States and Animall in Europe and some others, to co-sign a letter that said, "We won't put weapons on our robots." And part of the motivation there is, as these robots start to become commercially available, you can see videos online of people who've gotten a robot and strapped a gun on it and shown that they can operate the gun remotely while driving the robot around.

• 2:09:47Speaker 01

  And so, having a robot that has this level of mobility and that can easily be configured in a way that could harm somebody from a remote operator is justifiably a scary thing. And so, we felt like it was important to draw a bright line there and say, "We're not going to allow this." For reasons that we think ultimately it's better for the whole industry if it grows in a way where robots are ultimately going to help us all and make our lives more fulfilled and productive. But by goodness, you're going to have to trust the technology to let it in. And if you think the robot's going to harm you,

• 2:10:32Speaker 01

  that's going to impede the growth of that industry. So, we thought it was important to draw a bright line. And and then publicize that. And then our plan is to, you know, begin to engage with lawmakers and regulators. Let's figure out what the rules are going to be around the use of this technology. And use our position as leaders in this industry and technology to help force that issue. And so we are, in fact, I have a, I have a policy, you know, director at my company

• 2:11:13Speaker 01

  whose job it is to engage with the public, to engage with interested parties and including regulators to sort of begin these discussions.

• 2:11:24Lex Fridman?

  Yes, it's a really important topic, and it's an important topic for people that worry about the impact of robots on our society with autonomous weapon systems. So I'm glad you're sort of leading the way in this. You are the CEO of Boston Dynamics. What's it take to be a CEO of a robotics company? So you started as a humble engineer, PhD. Just looking at your journey, what does it take to go from being, from building the thing to leading a company? What are some of the big challenges for you?

• 2:12:05Speaker 01

  Courage, I would, I would put front and center for multiple reasons. I talked earlier about the courage to tackle hard problems. So I think there's courage required, not just of me, but of, of all of the people who work at Boston Dynamics. I also think we have a lot of really smart people. We have people who are way smarter than I am. And I, it takes a kind of courage to be willing to lead them and to trust that. You have something to offer to somebody who probably is maybe a better engineer than I am.

• 2:12:44Speaker 01

  Adaptability, you know, part of the, it's been a great career for me. I never would have guessed I'd stayed in one place for 30 years.

• 2:12:53Speaker 01

  And the job has always changed. I didn't, I didn't really aspire to be CEO from the very beginning, but it was the natural progression of things. There was always a, there always needed to be some level of management that was needed. And so, you know, when I saw something that needed to be done that wasn't being done, I just stepped in to go do it. And oftentimes, because we were full of such strong engineers, oftentimes that was in the management direction, or it was in the business development direction, or organizational hiring. Geez, I was the, I was the main person hiring at Boston Dynamics for probably 20 years. So I was the head of HR, basically.

• 2:13:38Speaker 01

  So I, you know, just willingness to sort of tackle any piece of the business that, that needs it and then, and be willing to shift.

• 2:13:47Lex Fridman?

  Is there something you could say to what it takes to hire a great team? What, uh, what's a good interview process? How do you know the guy or gal are going to make a great member of, uh, of a engineering team that's doing some of the hardest work in the world?

• 2:14:06Speaker 01

  You know, we developed an interview process that I was quite fond of. It's a little bit of a hard interview process because the best interviews, you ask somebody about what they're interested in and what they're good at.

• 2:14:21Speaker 01

  And if, if they can describe to you something that they worked on and you saw they really did the work, they solved the problems, and you saw their passion for it.

• 2:14:33Speaker 01

  And you could ask, but what makes that hard is you have to ask a probing question about it. You have to be smart enough about what they're telling you they're expert at to ask a good question. And so it takes a pretty talented team to do that. But if you can do that, that's how you tap into, ah, this person cares about their work. They really did the work. They're excited about it. That's the kind of person I want at my company. You know, at Google, they taught us about their interview process. And it was a little bit different. You know, we evolved the process at Boston Dynamics where it didn't matter

• 2:15:13Speaker 01

  if you were an engineer, or you were an administrative assistant, or a financial person, or a technician. You gave us a presentation. You came in and you gave us a presentation. You had to stand up and talk in front of us. And I just thought that was great to tap into those things I just described to you. At Google, they taught us, and I think, I understand why, you're right, they're hiring tens of thousands of people. They need a more standardized process. So they would sort of err on the other side, where they would ask you a standard question. I'm going to ask you a programming question, and I'm just going to ask you to, you know, write code in front of me. That's a terrifying, you know, application process.

• 2:15:53Lex Fridman?

  Yeah.

• 2:15:54Speaker 01

  It does let you compare candidates really well, but it doesn't necessarily let you tap in to who they are.

• 2:16:01Lex Fridman?

  Yeah.

• 2:16:02Speaker 01

  Right? Because you're asking them to answer your question, instead of you asking them about what they're interested in. But frankly, that process is hard to scale. And even at Boston Dynamics, we're not doing that with everybody anymore. But we are still doing that with, you know, the technical people. But we've, because we too now need to sort of increase our rate of hiring, not everybody's giving a presentation anymore.

• 2:16:27Lex Fridman?

  But you're still ultimately trying to find that basic seed of passion.

• 2:16:31Speaker 01

  Yeah.

• 2:16:32Lex Fridman?

  For the world.

• 2:16:33Speaker 01

  You know, did they really do it? Did they, did they find something interesting or curious, you know? And do they care about it?

• 2:16:41Lex Fridman?

  I think somebody admires Jim Keller, and he

• 2:16:48Lex Fridman?

  He likes details. So one of the ways you could, if you get a person to talk about what they're interested in, how many details, like, how much of the whiteboard can you fill out?

• 2:17:00Speaker 01

  Yeah. Well, I think you figure out, did they really do the work if they know some of the details?

• 2:17:04Lex Fridman?

  Yes.

• 2:17:04Speaker 01

  And if they have to wash over the details, well, then they didn't do it.

• 2:17:08Lex Fridman?

  Especially with engineering, the work is in the details.

• 2:17:11Speaker 01

  Yeah.

• 2:17:13Lex Fridman?

  I have to go there briefly, just to get your kind of thoughts in the long-term future of robotics.

• 2:17:23Lex Fridman?

  There's been discussions on the GPT side, on the large language model side, of whether there's consciousness inside these language models.

• 2:17:33Lex Fridman?

  And I think there's fear, but I think there's also excitement, or at least the wide world of opportunity and possibility in embodied robots having something like, let's start with emotion, love towards other human beings, and perhaps the display, real or fake, of consciousness. Is this something you think about in terms of long, long-term future? Because as you, as we've talked about, people do anthropomorphize these robots. It's difficult not to project some level of, I use the word sentience, some level of

• 2:18:18Lex Fridman?

  sovereignty, identity, all the things we think as human. That's what anthropomorphization is, is we project humanness onto mobile, especially legged robots. Is that something almost from a science fiction perspective you think about? Or do you try to avoid ever, try to avoid the topic of consciousness altogether?

• 2:18:42Speaker 01

  I'm certainly not an expert in it, and I don't spend a lot of time thinking about this, right? And I do think it's fairly remote for the machines that that we're dealing with.

• 2:18:54Speaker 01

  Our robots, you're right, the people anthropomorphize, they they read into the robot's intelligence and emotion that isn't there because they see physical gestures that are similar to things they might even see in people or animals.

• 2:19:11Speaker 01

  I don't know much about how these large language models really work. I believe it's a kind of statistical averaging of the most common responses, you know, to a series of words, right? It's it's sort of a very elaborate word completion.

• 2:19:30Speaker 01

  And I'm dubious that that has anything to do with consciousness.

• 2:19:38Speaker 01

  And I even wonder if that model of sort of simulating consciousness by by stringing words together that are statistically associated with one another, whether or not that kind of knowledge, if you want to call that knowledge,

• 2:19:54Speaker 01

  would be the kind of knowledge that allowed a sentient being to grow or evolve. It feels to me like there's there's something about truth or emotions that's just a very different kind of knowledge that is that is absolute. Like, the interesting thing about truth is it's absolute, and it doesn't matter how frequently it's represented in the world wide web. It's if you know it to be true, it can only be it may only be there once, but by God, it's true. And I think emotions are a little bit like that, too. You know something, you know, and and I just think that's a different kind of knowledge than the way

• 2:20:35Speaker 01

  these large language models derive sort of simulated

• 2:20:40Lex Fridman?

  It does seem that the things that are true

• 2:20:45Lex Fridman?

  Very well might be statistically well represented on the on the internet, because the internet is made up of humans. So I tend to suspect that large language models are going to be able to simulate consciousness very effectively. And I actually believe that current GPT-4, when fine tuned correctly, would be able to do just that. And that's going to be a lot of very complicated ethical questions that have to be dealt with. They have nothing to do with robotics, they have and everything to do with.

• 2:21:16Speaker 01

  There needs to be some process of labeling, I think, what is true. Because because there is also disinformation available on the web, and these models are going to consider that kind of information as well. And again, you can't average something that's true and something that's untrue and get something that's moderately true. It's either right or it's wrong. And so how how is that process? And and this is obviously something that the purveyors of these, Bard and ChatGPT, I'm sure this is what they're working on.

• 2:21:52Lex Fridman?

  Well, if you interact on some controversial topics with these models, they're actually refreshingly nuanced. They present because there's there's you realize there's no one truth. You know, what caused the war in Ukraine? Right? Any geopolitical conflict, you can ask any kind of question, especially the ones that are politically tense, divisive and so on. GPT is very good at presenting, here's the like, here's the it presents the different hypotheses. It presents calmly the the amount of evidence for each one. It's very, it's really refreshing.

• 2:22:35Lex Fridman?

  It makes you realize that truth is nuanced and it does that well. And I think with consciousness. It would very accurately say, well, it sure as hell feels like I'm one of you humans, but where's my body? I don't understand, like you're going to be confused. The cool thing about GPT is it seems to be easily confused in the way we are. Like you wake up in a new room and you ask, where am I? It seems to be able to do that extremely well. It'll tell you one thing, like a fact about when a war started. And when you correct it, say, well, this isn't, that's not consistent. It'll be confused. It'd be, yeah, you're right.

• 2:23:19Lex Fridman?

  It'll have that same element, childlike element with humility of trying to figure out its way in the world. And I think that's a really tricky area to sort of figure out with us humans of what we want to allow AI systems to say to us. Because then if there's elements of sentience that are being on display, you can then start to manipulate human emotion, all that kind of stuff. But I think that's, that's something, that's a really serious and aggressive discussion that needs to be had on the software side. I think, again, embodiment, robotics are actually saving us from the arbitrary scaling

• 2:24:04Lex Fridman?

  of software systems versus creating more problems. But that side, I really believe in the, that connection between human and robot. There's magic there. And I think there's also, I think, a lot of money to be made there. And Boston Dynamics is leading the world in the most elegant movement done by robots. So I can't wait. Thank you. To what maybe other people that built on top of Boston Dynamics robots or Boston Dynamics by itself. So you had a one wild career, one place and one.

• 2:24:46Lex Fridman?

  Set of problems, but incredibly successful. Can you give advice to young folks today in high school, maybe in college, looking out to this future where so much robotics and AI seems to be defining the trajectory of human civilization? Can you give them advice on how to have a career they can be proud of or how to have a life they can be proud of?

• 2:25:14Speaker 01

  Well, I would say, you know, follow your heart and your interest. You know what? Again, this was an organizing principle, I think, behind the leg lab at MIT that turned into a value at Boston Dynamics, which was follow your curiosity, love what you're doing. You'll have a lot more fun and you'll be a lot better at it as a result. I think it's hard to plan, you know? Don't get too hung up on planning too far ahead. Find things that you like doing and then see where it takes you. You can always change direction. You will find things that, you know, that wasn't a good move.

• 2:25:59Speaker 01

  I'm going to back up and go do something else. So when people are trying to plan a career, I always feel like, yeah, there's a few happy mistakes that that happen along the way and just live with that, you know? But just, but make choices then. So avail yourselves to these interesting opportunities, like when I happened to run into Mark down in the lab, the basement of the AI lab. But be willing to make a decision and then pivot if you see something exciting to go at, you know? Because if you're out and about enough, you'll, you'll find things like that that get you excited.

• 2:26:32Lex Fridman?

  So there was a feeling when you first met Mark and saw the robots that there's something interesting.

• 2:26:37Speaker 01

  Oh, boy, I gotta go do this. There is no doubt.

• 2:26:42Lex Fridman?

  What do you think in 100 years? What do you think Boston Dynamics is doing? What do you think is the role even bigger? What do you think is the role of robots in society? Do you think we'll be seeing billions of robots everywhere? Do you think about that long-term vision?

• 2:27:03Speaker 01

  Well, I do think that, um, I think that robots will be ubiquitous and they will be out amongst us.

• 2:27:16Speaker 01

  And they'll be certainly doing, you know, some of the hard labor that we do today.

• 2:27:24Speaker 01

  I don't think people don't want to work. People want to work. People need to work to, I think, feel productive. We, we don't want to offload all of the work to the robots because I'm not sure if people would know what to do with themselves. And I think just self-satisfaction and feeling productive is such an ingrained part of being human that we need to keep doing this work. So we're, we're definitely going to have to work in a complimentary fashion. And I hope that the robots and the computers don't end up being able to do all the creative work, right? Because, because that's the part that's, you know, that's the rewarding, the creative part of solving a problem is the thing that gives you that serotonin rush

• 2:28:06Speaker 01

  that you never forget, you know, or that adrenaline rush that you never forget. And so, you know, people need to be able to do that creative work and, and just feel productive. And sometimes that you can feel productive over fairly simple work that's just well done, you know, and that you can see the result of. So I, you know, I, you know, there is a, I don't know, there was a cartoon, was it Wally? Where they had this big ship and all the people were just overweight, lying on their beach chairs, kind of sliding around on the deck of the, of the movie because they didn't do anything anymore. Well, we

• 2:28:48Speaker 01

  definitely don't want to be there. You know, we need to work in some complimentary fashion where we keep all of our faculties and our physical health and we're doing some labor, right? But in a complimentary fashion somehow.

• 2:29:00Lex Fridman?

  And I think a lot of that has to do with the interaction, the collaboration with robots and with AI systems. I'm hoping there's a lot of interesting possibilities.

• 2:29:08Speaker 01

  I think that could be really cool, right? If you can, if you can work in a, in an interaction and really be, be helpful. Robots, you, you know, you can ask a robot to do a job you wouldn't ask a person to do, and that would be a real asset. You wouldn't feel guilty about it, you know? You'd say, just do it. It's a machine. I, and I don't have to have qualms about that, you know?

• 2:29:30Lex Fridman?

  The ones that are machines, I also hope to see a future. And it is hope. I do have optimism about the future where some of the robots are pets, have an emotional connection to us humans. And because one of the problems that humans have to solve is this kind of a general loneliness. The more love you have in your life, the more friends you have in your life. I think that makes a more enriching life, helps you grow. And I don't fundamentally see why some of those friends can't be robots.

• 2:30:00Speaker 01

  There's an interesting long running study, maybe it's in Harvard. They just nice report article written about it recently. They've been studying this group of a few thousand people now for 70 or 80 years. And the conclusion is that companionship and friendship are the things that make for a better and happier life. And so I agree with you. And I think that could happen with a machine that is probably, you know, simulating intelligence. I'm not convinced there will ever be true intelligence in these machines, sentience, but they could simulate it and they could collect your history and they

• 2:30:45Speaker 01

  could, you know, I guess it remains to be seen whether they can establish that real deep. You know, when you sit with a friend and they remember something about you and bring that up and you feel that connection, it remains to be seen if a machine is going to be able to do that for you.

• 2:30:59Lex Fridman?

  Well, I have to say, it's inklings of that already started happening for me. Some of my best friends are robots. And I have you to thank for leading the way in in the accessibility and the ease of use of such robots and the elegance of their movement. Robert, you're an incredible person. Boston Dynamics is an incredible company. I've just been a fan for many, many years, for everything you stand for, for everything you do in the world. If you're interested in great engineering robotics, go join them, build cool stuff. I'll forever celebrate the work you're doing. And it's just a big honor that you would sit with me today and talk. It means a lot. So, thank you so much. Keep doing great work.

• 2:31:37Speaker 01

  Thank you, Lex. I'm honored to be here and I appreciate it. It was fun.

• 2:31:40Lex Fridman?

  Thanks for listening to this conversation with Robert Plater. To support this podcast, please check out our sponsors in the description. And now, let me leave you with some words from Alan Turing in 1950, defining what is now termed the Turing Test.

• 2:31:58Lex Fridman?

  A computer would deserve to be called intelligent if it could deceive a human into believing that it was human. Thank you for listening and hope to see you next time.