E12: The Robot Revolution with Keerthana Gopalakrishnan of Google Robotics

Nathan Labenz talks to Google Robotics researcher Keerthana Gopalakrishnan (@keerthanpg) about how they train robots at Google, and how robots contextualize their environment and complete tasks through supervised learning. They go deep on integrating language models for commands and how to approach translating them, as well as the importance of robotics safety. Keerthana also shares how we’re using robots to better understand ourselves, personal experiences using AI, and her hopes and fears about AI's impact on society. This episode provides valuable insights into the current state and potential future of robotics and AI, with Keerthana's expertise offering unique perspectives on these fascinating topics. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics (0:00) Preview to episode (0:51) Sponsor (1:09) Intro (5:20) GPT analogy for robotics (6:51) Domestic robots (10:53) Description of Google robotics (15:17) Controlling robots and supervised learning (18:50) How robots complete tasks (26:17) Using data to train robots (30:45) Data operability (36:02) Integrating language models for commands (38:58) How to approach translating commands (42:49) How robots contextualize their environment (47:02) Approach to robotics safety (51:02) Tasks that robots can’t/shouldn’t do (54:08) The hard bounds of robotics (59:17) Frequency of inference (1:03:50) Form factor in robotics (1:08:38) Using robots to better understand ourselves (1:12:23) Having robot friends and spouses (1:17:44) Inference and latency (1:19:17) The Google Robotics team (1:21:07) AI tools Keerthana uses in her personal life SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries. From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive *Thank you Omneky* for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. Twitter: @CogRev_Podcast @keerthanpg (Keerthana) @labenz (Nathan) Join 1000's of subscribers of our Substack: https://cognitiverevolution.substack Websites: cognitivervolution.ai omneky.com https://research.google/research-areas/robotics/ https://research.google/teams/brain/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Keerthana Gopalakrishnan?

  That's why I'm so excited about robotics, because it's like we are inventing ourselves, right? It is, in many ways, a quest to understand us and our intelligence. And it's so hard to put down onto paper how we detect like a cup or how we are doing these things or how we are planning tasks. Like, you know how software engineers say the best way to learn something is to build it? And I think robotics is basically our quest to understand ourselves and build more of ourselves.

• 0:28Nathan Labenz?

  Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together, we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg.

• 0:51Speaker 02

  Omnikit uses generative AI to enable you to launch hundreds of thousands of ad iterations that actually work, customized across all platforms with a click of a button. I believe in Omnikit so much that I invested in it, and I recommend you use it, too. Use CogRev to get a 10% discount.

• 1:09Nathan Labenz?

  Keerthana Gopal Akrishnan, mother of robots, is a robotics and machine learning engineer conducting advanced robotics research at Google AI, where she's been an author on several of the most influential papers of the last year, including CACAN, which demonstrated that language models can effectively act as the executive planning function, or "brain" of a robotic system, capable of understanding and carrying out, quote, "long horizon, abstract, natural language instructions", unquote, by assessing situations, decomposing problems into parts, and issuing logical next-step commands for their robot bodies to execute. And also, Robotics Transformer, or RT1,

• 1:54Nathan Labenz?

  which demonstrated that the same large-scale pre-training paradigm, which has recently delivered so many breakthroughs in so many domains, can also work for robotics. RT1 ultimately delivered a 97% success rate on some 700 different tasks.

• 2:14Nathan Labenz?

  It's a truism in Silicon Valley startup culture that it's easier to manipulate and control bits than it is to control atoms. And some have argued that the impact of AI will be limited to digital domains as a result. To me, Keerthana's work strongly suggests that this outlook is wrong. Google's most recent release, Palm E, as well as OpenAI's GPT-4, show that computer vision is still improving rapidly. And at this point, the combination of visual and language understanding seem quite clearly rich enough to support general-purpose robots, suggesting that relatively few major conceptual problems, including more general robotic control,

• 2:58Nathan Labenz?

  which Keerthana suggests we will likely achieve by teaching robots to learn from watching humans, still remain to be solved. Barring an unexpected slowdown, my takeaway from this conversation is that we should expect many form factors of robots that can see, communicate with us in natural language, and solve basic problems on their own, all in just the next couple of years. At that point, the race to engineer and scale the production of all sorts of robots will be on. And shortly after, we'll begin to encounter them in factories, offices, businesses, and even homes. I enjoyed this conversation with Keerthana tremendously. She is technically deep, and we get into the weeds.

• 3:43Nathan Labenz?

  But she's also extremely thoughtful about how her work will affect the future, and I very much enjoyed her thoughts on the big picture as well. I hope you enjoyed this conversation with Keerthana Gopalakrishnan. Keerthana Gopalakrishnan, welcome to the Cognitive Revolution.

• 4:01Keerthana Gopalakrishnan?

  Thank you.

• 4:02Nathan Labenz?

  Really excited to have this conversation, a lot to cover. You have one of the best Twitter bios that I've seen, "Mother of Robots." And as soon as I saw that, I was like, "All right, we've got to get Keerthana on the show and talk to her about everything that she's doing." You've had quite a tour de force over the last year and change as well in terms of having been an author on some of the biggest papers in robotics. And obviously, you've worked with an outstanding team at Google to create that work. But Really looking forward to getting into the, some of the weeds on that.

• 4:42Nathan Labenz?

  You recently gave a lecture for the ML Collective, which I have watched and definitely recommend. And one of the things that you said in that lecture that really caught my ear was that we're somewhere between GPT-2 and GPT-3 in the world of robotics. So I wanted to just start by asking, what does GPT-4 look like in robotics? Or zooming out even more broadly, how do you think we will integrate robots into our lives as they get smarter and more capable over the course of, say, the rest of the 2020s?

• 5:20Keerthana Gopalakrishnan?

  I'm very excited because we are seeing the type of scale, learning at scale in robotics. So which is similar to like GPT-3 for language. So like we are seeing emergent capabilities in reasoning and also in low-level control. And the way that I see robots coming into our lives is definitely, I think at Google, at robotics at Google, we have cracked this recipe of like transformers for control, language for interface, and then foundation models for like reasoning. I feel like this recipe is fairly generalizable and extensible and has the potential to scale with data and with compute.

• 6:03Keerthana Gopalakrishnan?

  So the way that I see robots coming into our lives is where humans interact with robots in natural language and the robots can do a lot of instructions in the real world and they can think and plan in visual language space.

• 6:20Nathan Labenz?

  So without being, you know, falsely precise in terms of like a, you know, specific date for a prediction, does it seem realistic to you that we are headed for? A time within the decade where we might have, for example, domestic service robots that can like do the dishes for me after dinner, or you know, pick up the toys after my kids and put them away when the kids don't do it. What is this gonna like actually look like in, you know, homes or in places of business for people?

• 6:51Keerthana Gopalakrishnan?

  Oh, absolutely. I do. In fact, the task that we train our robots that we publish on are in office environments. And like picking things and cleaning and everything. In robotics, there is the problem of like, a lot of demos that you see are like fairly staged. If you, if you look at like robotics technology, there was like very traditional control, like PID control and like search algorithms and stuff. And then there was early deep learning, like end to end deep learning, but which was mostly like con nets and everything. And now there is transformers in robotics. And so we are, we are seeing like a curve and we are seeing that transformers are actually pretty good at doing a lot of tasks with one model, which is quite similar to how you are, right?

• 7:32Keerthana Gopalakrishnan?

  Like you can cook, you can clean, you can like plan tasks, you can talk to your kids, like language generation. So I feel like to build a robot that's like very useful, that goes into different people's houses, it's going to be very generalizable and it's going to be one model that's good at a lot of different things. And we are kind of seeing those trends already. The question is like, so we know it works in like multiple Google kitchens. RT one can do a lot of tasks, but Google kitchens are still very similar and still a very small subset of the total number of kitchens that people can see in the world. So like, how do you scale fast enough that you can bring the generality and complexity of the real world into the models?

• 8:16Keerthana Gopalakrishnan?

  And where like, you bring that to my house and then it doesn't suddenly break down?

• 8:21Nathan Labenz?

  Yeah, it's unbelievable convergence across kind of all the, the things, you know, we've talked to guests who are in computer vision and who are, you know, just touching all these different modalities. And the same story is kind of underlying the progress in all of them. So no surprise to learn on some level that that is also happening in robotics. I think one of the things that you talked about in your, in your lecture and you kind of, you know, alluded to there is just Scale is starting to happen in robotics as it has happened in other modalities. But the data doesn't naturally exist, right?

• 9:04Nathan Labenz?

  That's like a huge difference between for language, we have web scale data. For we just did an episode on generation of human-like voices. And obviously, there's a ton of voices out there. I cloned my own voice. It took only 10 minutes of audio because there was so much pre-training. But obviously, that doesn't exist in robots or for robots because we don't have all the robots. They're not out there doing tasks in today's world and collecting data. So I was really interested. I kind of dug in a little bit to the RT one paper and tried to get a sense for the work that you guys have done

• 9:42Nathan Labenz?

  to assemble the data set, you know, at least like beginning to approach the scale that is needed to power this. And I want to get into that a little bit deeper because I think that's something that people probably have no real intuition for. And you have lived it, you know, and put hours into helping to assemble that that data set. So maybe we could just like run down some stats and then get like very kind of practical in terms of, you know, what the data is and how you're going about collecting it. So you've got, this is from the RT one paper, 700 tasks, 130,000 episodes, which I take to be an attempt to complete a task.

• 10:27Nathan Labenz?

  You can, you know, definitely correct or refine my understanding. Multiple different robot form factors and a project that took a year and a half to complete. I guess for starters, can you like just describe the robots? There are a few videos out there that people can watch to also kind of go see them. What are these robots? What do they look like? How big are they? What do they like to be around? Tell us about the robots.

• 10:53Keerthana Gopalakrishnan?

  Firstly, a lot of questions here, so let me go part by part. The robots are really cute that we currently use, so they are mobile manipulators. They can drive and they have an arm with a gripper at the end. You can also like fit different tools, like they can do wiping and other things. And they also have like self-charging and other capabilities on top of that. So they they are really, and I think like they're they're a pretty good and fairly stable stack. And we have a fleet of them and we have been collecting data on them. And the the way the one thirty K episodes for use for RT one are supervised launch.

• 11:33Keerthana Gopalakrishnan?

  So they are collected by a human teleoperating these robots to do different tasks and over the seven. So they will attempt all of these tasks and then we train on that using supervised learning. So now that brings you a question about like data and language and vision and stuff. So in language and vision, as you said, there's a lot of free data available on the internet that you can just take and then use and then you're good to go. But in robotics, like, okay, there's reasoning and planning for which there is a lot of data because we it's similar to both humans and robots. But for actions specifically, there is not that much data. So that is one bottleneck in robotics. And it is also like a very engineering bottleneck, like it's very few people can

• 12:16Keerthana Gopalakrishnan?

  accumulate and accumulate data sets of that scale, which means that very few people, like robots are expensive. And you need like a lot of engineering effort to like orchestrate this large data farming operation. So that already means that very few people can do robotics. And also, which is why like I feel like we at Google, who have the opportunity to do it, are really we we really have to, we really have a unique position in scaling and making a dent in our attempt towards solving robotics. So I'm very excited about that. But also, if if you look at language and vision, the acceleration in deep learning was

• 12:58Keerthana Gopalakrishnan?

  Was achieved by weekly supervised learning. Collecting data using robots can only scale linearly and only scale, especially with like human demonstrations, it can only scale with O of humans. But we we want to scale faster than that. And so one way would be to get transfer from human manipulation data. For example, imagine how you would learn to surf or cook something. You would take a YouTube video and then you would watch it. And there are a lot of YouTube videos of humans doing a lot of things. Imagine that like one day you want to train like, let's say, an NBA level basketball player, but who's a robot. You would not do like teleoperate robots to do NBA level basketball or do self play.

• 13:39Keerthana Gopalakrishnan?

  Maybe you'll do a little bit of reinforcement learning and specific fine tuning on top. But you would make it watch all of the NBA videos that humans have been playing all these years and then you would show and you would try to get the robot like scaled up to that point and then collect a bunch of data to like exactly fit to the robot's environment. So transferring from human manipulation to robot manipulation is a problem that we haven't yet solved yet, which I think is going to be very important in really solving robotics. So while humans cannot scale that fast, robots can actually scale. We can build like fifty thousand robots. It it's just a question of money. And then if robots can do autonomous collect, then that's going to scale faster than with supervised learning.

• 14:23Keerthana Gopalakrishnan?

  So how how can robots do autonomous collect? So one is like using these foundation models to like collect data zero shot or with little bit of fine tuning, like something like Cores father says.

• 14:35Nathan Labenz?

  So let's build up those layers in a little bit more detail, starting with the supervised learning, like the, you know, the hundred and thirty thousand episodes. So if I understand that correctly, you've got essentially a kitchen, which is a lab on the Google campus, and you've got robots that as of the time that you're collecting these episodes are not AI powered. They are instead remote control powered. So somebody sitting there with like a PlayStation controller going around and. You know, picking up napkins and whatnot with the robot. Is that actually like a reasonable picture of what's happening, like a PlayStation controller type of interface?

• 15:17Keerthana Gopalakrishnan?

  Yeah, we have like Oculus controllers for these robots. And then we have like multiple mock kitchens, um, which are like, they are called robot classrooms. Like the robot goes to a classroom and learns some things. And then, once they can reasonably do things in the classroom, then they are brought to like our actual kitchens to do like stuff.

• 15:36Nathan Labenz?

  You know, I just did the most like naive math. I just divided one hundred and thirty thousand episodes by the number of workdays in the year and a half that the project took. And I got three hundred and sixty seven episodes collected per day. So i- if I'm envisioning this right, it's like you must have, uh, I don't know, twenty people who like have been operating the robots and like doing the actual VR to robot, uh, housework and collecting all the data. And then I, I understand also that like the robot sensors, you know, sort of robot proprioception, I guess, if you will,

• 16:18Nathan Labenz?

  is also being just recorded at like each timestamp along the way. And once all that is done, now you have kind of the, you know, reasonably big data set for supervised learning. So then your inputs would be the task or the command or the instruction plus the imagery that was seen at that given time, plus like state of the robot at that given time. And then the model is predicting next action. How many like data points does that translate into? Cause it's each episode presumably has, I know you're running like multiple inferences per second. So do you have a sense for what that hundred and thirty thousand episodes?

• 17:03Nathan Labenz?

  Would translate into in terms of like example predictions, so to speak?

• 17:07Keerthana Gopalakrishnan?

  Yeah, so it depends on the median task length, right? So and it also depends on what tasks. For example, picking is something that is fairly fast. And I think it's about like thirty steps. So you have thirty actions to pick an object. And like something like opening doors is a longer episodes. So because you have to like approach the door, grab the handle, then open the door. So let's say if we put a median around like forty or fifty, let's say fifty, fifty steps per episode, then each step has around eight plus few,

• 17:52Keerthana Gopalakrishnan?

  like let's say eleven tokens. So that's that's how many. And then you multiply that with the number of episodes. So it's like still way, way, way smaller compared to like language and image data sets. So we need to scale both by autonomous collection as well as by like constraint.

• 18:10Nathan Labenz?

  OK, cool. So we've got the the foundation of all this manually collected data. And now tell us kind of the the outlook for the self collection. So I'm I'm kind of, you know, I do this with language models where I'll try to get it to do a task. You know, it sometimes does it, it sometimes can't. And then I basically siphon off the data that is good, that shows successes and like feed that back into my fine tuning. And, you know, I I get better that way. Probably a lot of our listeners are familiar with that general kind of cycle. How does that? Is that basically what you're doing also in robotics? Or what is the kind of, you know, robotics twist on that?

• 18:50Keerthana Gopalakrishnan?

  So right now our system has like a high level and low level control. The high level control runs at like lower frequency compared to the low level control is like much faster. And the high level control, that paper is called FAICAN, where a language model is deciding how to plan, like what task to do in sequence. So imagine something like bring me a coffee. So it'll be like, go to the kitchen, find a coffee cup, pick up a coffee cup. Place it on the counter and then go to the coffee machine, press the button. So like there's a series of steps to, of steps in natural language to do to achieve like a small task that you tell the robot to do, which is like, go get me like a coke can from the fridge or something.

• 19:35Keerthana Gopalakrishnan?

  So there is a language model, which is like planning how to do the series of tasks. And then there is the robotics transformer, which is executing each of the smaller tasks.

• 19:46Nathan Labenz?

  I'd love to just hear more examples of what the tasks are and, you know, maybe your overall kind of impression of where the robots are today in terms of, like, are they practically helpful? Like, if you had one at home, would it be worth it, you know, to have one at home at this point? Or is it still more trouble than it's worth? Like, how, you know, how close are we to actually having something that would be useful in the wild?

• 20:13Keerthana Gopalakrishnan?

  So it's not in the wild yet, and that's one thing that we are working towards. But the thing is, scaling from five to five hundred is quite hard. But scaling from five hundred to five thousand tasks is easier and five thousand to fifty thousand is going to be also easier. Like, each of them are like a different class of problems. Like, towards the end, you come to like more scaling and distribution and inference at scale problems. But initially, you are like, you have algorithmic problems. And I feel like we sort of more or less have the algorithmic problems down. So the tasks in RT one are mostly like pick object, place, knock, pick, open cabinets,

• 20:55Keerthana Gopalakrishnan?

  take things out of drawers and put them on counter, also open fridges and close. Technically, things that you can do in a kitchen. And we also tried in like various Google kitchens. So it's like, yeah, if you take it to a new Google kitchen and make it run there, I think it should work. But if you bring it to your house, I doubt it will work because all the images kind of look like it's very out of distribution. I don't think it would generalize that much. But Google kitchens are still like somewhat similar, things are at similar heights and stuff, so it should. Sort of work. Uh. So then that is the question of like generalization, right? Um. And also like how many objects or how many scenarios can we generalize to?

• 21:37Keerthana Gopalakrishnan?

  So the initial RT one, uh, we wanted to focus on skills. So we were like, we will do a lot of skills, but a few objects. So we only had like seventeen objects. Now seventeen objects are like too small, right? You have millions of objects in the real world. And in order to be realistic, you need to be manipulating millions of objects. So we choose uh our recent paper on like open vocabulary object manipulation. So the idea there is that, so imagine you have, you can do seventeen objects. And then you collect a little bit of data on about a hundred objects. So generalizing from seventeen objects to any objects is quite hard. But generalizing from one hundred something objects to any objects is slightly an easier problem.

• 22:16Keerthana Gopalakrishnan?

  Um. So this was like, yeah, so what we did was use a visual language model uh that can do a lot of zero shot detection, right? Like uh right now, uh, language models can differentiate between your faces. Language models can say, hey, this is like a bottle or um this is a phone just from a zero shot image. So can we use that information to go manipulate that object? And it sort of works reasonably well.

• 22:40Nathan Labenz?

  So what are like the high end tasks? You kind of gave verbs, you know, of knock, lift, open, grasp, et cetera. And then you also kind of talked about the longer time horizon planning, which I I imagine is still like pretty short. Like, it sounds like the things are, you know, tasks that would be like thirty second tasks for a human. But like, how far is that currently going? Could the robot, for example, like actually make and deliver a coffee with like actual hot coffee in it?

• 23:14Keerthana Gopalakrishnan?

  So we don't let them handle hard objects um yet. And they're also not good at operating machines yet. But that's something like, because we didn't, we just didn't collect data on it. But that's something fairly doable and that you can learn. They probably shouldn't be handling like hot coffee. I I I doubt that that's that's probably like, I don't know. Like, imagine you're carrying a hot coffee and then there's like a kid nearby like. Uh like something happens and then you splash it, that's not good. Or you just pour it over yourself and then you break your electronics. So if you think about tasks in a kitchen, let's say when you're cooking and stuff. You only actually do a finite number of skills, but you are able to combine them in uh various fashions.

• 23:57Keerthana Gopalakrishnan?

  So if you do one hu- let's say you have one hundred skills. So which is like, let's say, pick vegetable, uh cut vegetable, wash them, um. Uh put them in plates like animal stew. So it's only finite number of skills, but then um let's say you have around a hundred skills now. And then you can then combine them in uh various ways. So it would be like hundred C X where X is the uh number of tasks that you wanna draw in order to do a high level sequence. So something like make an omelette, so that's like, I don't know, open the fridge, take an egg, break an egg. Even even with like a finite set of skills, you can still do a lot of tasks in in the real world by combining them and a lot of like high level instructions.

• 24:42Keerthana Gopalakrishnan?

  Um we still have a lot of uh skills to go. I think with R T one, the objective was how do we get an algorithm that shows like scaling limits? So if you look at the scaling plots in there, um task diversity is very important. R T one doesn't seem to saturate with the amount of data you throw in. And in fact, it actually seems to like get much and much better uh like emergence at like higher scales and higher diversity. And you can also increase the uh size of the model um a lot uh to fit the data. Imagine that you have to build like a very generic robot manipulator. You need data and you need uh you need a data absorber.

• 25:23Keerthana Gopalakrishnan?

  Um so i- in our office, we call it like a sponge and a fire hose. So a sponge is like R T one is a very good sponge. Uh and we are also building like better sponges there, which is like, I don't know, like. V L M uh like foundation model uh transformers, right? Which can transfer also like internet scale generality into robot manipulation.

• 25:47Keerthana Gopalakrishnan?

  So like visual language models as as manipulative, like have you seen the Parmi paper? Something like that. So now now imagine that you do have a data sponge that can that you can put a lot of data and then it just learns and then it does them in the real world. Now, the question is, how do you build a fire hose to like really pump it with like a lot of data? So that's one one of my main projects at Google. So like how to scale autonomous, but like also how to like mine for a lot of data from the internet.

• 26:16Nathan Labenz?

  Tell us more about that. I mean, the I I'm understanding from your commentary that you it sounds like are kind of trying to base this on video of humans. Humans doing stuff, which this is always kind of an interesting pattern as well, where it's like. A lot of times, the hardest part is figuring out how to cast the problem in such a way that you actually have. Or can create or can sort of, you know, massage existing data into form where it can actually. You know, power the the training paradigm that you want to power.

• 26:53Nathan Labenz?

  So obviously, there's a lot of data out there, you know, whether it's NBA basketball games or, you know, how to stuff of all kinds on YouTube. I guess a couple of questions that come to mind there. One is like, there's so many different possible forms of robots. That, you know, kind of if I was if I was thinking about this naively, I would think, OK. You know, maybe I need, you know, does video work? Or maybe I need like a first person like POV, you know, you know, GoPro type of view to to make this work because that might be a little bit more analogous. You know, boy, there's so many different possible forms of robot.

• 27:30Nathan Labenz?

  Obviously, you'd want this foundation model to like be able to adapt to all kinds of different form factors or embodiments, you know, if you will. So I guess those are my two questions, like i in the sort of pre-training. Do you think that like the third party point of view? You know, like the NBA basketball camera from the side view is going to be enough to kind of develop the conceptual, you know, semantic understanding that you need? Or is there going to be kind of a a need for like more specialized point of view type thing or something else? And then second. Is this kind of inevitably shaping up to be like a two-stage thing where there's

• 28:14Nathan Labenz?

  sort of first like building the representations of what the humans are doing and then later like a bunch of adapters to specific, you know, forms? What am I getting right and what am I getting wrong is I guess there what that might look like?

• 28:28Keerthana Gopalakrishnan?

  So some of the things you can actually transfer, right? Like, let's say I see my mom cooking an omelet and then from third-person view and then I can try to cook an omelet and I'm mostly successful. But if I see Kobe Bryant playing basketball and then I try to play at that level, I'm not going to be successful, even though I have a very good modeling of my own body. So I think it's going to be similar like that. A lot of tasks, like whatever is easier, you can learn by like third-person watching, but there still are tasks that you can only do by actually practicing and improving your own control model. We have some good work coming, like I don't want to speak about it before it's published,

• 29:07Keerthana Gopalakrishnan?

  on like transferring between human and robot data and also, yeah, training larger VLMs for control.

• 29:16Nathan Labenz?

  Just the one that came out this week, Palm E was pretty amazing, kind of inherent in that, you know, the big project is the E stands for embodiment, right? So we're taking Google's kind of signature language model, which has been adapted to, you know, seemingly every domain already. I go on and on about MedPalm in other conversations, but this is Palm E. So kind of sitting it at the center of this robot system and having it do the kind of reasoning and planning, pretty amazing. Definitely picks up on another theme too that we've talked about a few times, which is the the model-to-model communication. If I understand correctly, with Palm E, the

• 30:04Nathan Labenz?

  Kind of auxiliary models are trained to inject embeddings into the language latent space directly without going through language itself, right? So there's this kind of really interesting connectivity that's starting to happen across these models. That's something that we also talked to the BLIP authors about, and that's been a technique that they have used in their most recent paper as well. Translating these predictions to actual action in space and accomplishing stuff, that's very particular to robotics, obviously. Can you kind of talk us through that half of the equation?

• 30:44Keerthana Gopalakrishnan?

  I think the key to building like a foundation model would be data interoperability. And if you look at PAMI, it converts images into tokens, language into tokens, and actions into tokens. And once they are tokens, then like a transformer can like operate on all of them. And also, if you really think about it, the foundations of like thought is similar in whether you're acting in language space or vision space or action space. And to get a globally optimal solution, you need to think in all of these spaces, right? Like if you are a person who's cooking or cleaning or walking your dog or your kid,

• 31:26Keerthana Gopalakrishnan?

  you are thinking in both in language, in vision, and also with like your body. So any solution that would be like a globally optimal solution will be a fully multimodal. And I think eventually it would be like one model to rule them all, to like does all the modalities together. And specifically, the way I think one of RT1's innovation is tokenizing actions. And at any given point of time, at least for the EDR robot, we have like 11 variables for one to terminate the episode or not, and decide between arm and base control, and like around seven variables for arm control.

• 32:08Keerthana Gopalakrishnan?

  And for arm, we are only doing basically where to put the end effector, so end effector positions, end effector rotation, and how much to close the grip. So seven, three variables for position, three variables for the rotation, and one variable for grip or close. And then for base, X, Y, and the angle. So three variables for base, seven variables for the arm, and one variable to decide. Uh to control base or arm and then to also uh terminate it. So so one, once you like make your actions like tokens. Then it's just like uh like vision tokens or uh language uh tokens, right? Like it's actually kind of like a.

• 32:50Keerthana Gopalakrishnan?

  Uh it's very radical. Uh when V I T came out, people were like, oh, this is not gonna work because like, how can you convert an image into language as like a sequence of tokens? Uh like li- li- like language is sequence of characters, but then it it kind it works and it was able to bring us like large multimodal. V L M models like clip, even stable diffusion, uh DALL-E and everything which is uh built on top of like click like models. So then the question becomes like, now can you also add uh actions in there like tokens like? Now now actions look exactly like language and it's just a question of predicting like um eleven tokens after and you can do that for any robot.

• 33:33Keerthana Gopalakrishnan?

  Um and maybe another robot had like that has like more degrees of freedom, like let's say imagine like a humanoid or something and then you have a lot more. Um variables. So each degree of freedom can be like one extra variable for the transformer model to predict. Um and then it just becomes like, can you predict the X Y Z sequence? Um that's how I think uh actions are going to be tokenized and converted into data for consumers.

• 33:59Nathan Labenz?

  So at each step of inference, the kind of core language model, which is, you know, you're kind of projecting out in the future. There might just be literally one big model that takes everything in and, you know, it's one huge black box. But for now, we kind of have some auxiliary models that like do the vision part and kind of feed that into the language. And I I looked at the um the Paul Me paper, there's even like a couple of different types of language of of vision model, excuse me, that, you know. Uh object segmentation, you know, general scene description, depth mapping, whatever, like all these kind of different things come in together into the language model. Along with, of course, like, what are we trying to do here? You know, what did the what did the human ask us to do?

• 34:44Nathan Labenz?

  And then what is spit out the other end is eleven values.

• 34:52Nathan Labenz?

  Which say, and was it either or like you're either gonna do something with the arm? Or you're gonna do something with the base?

• 34:59Keerthana Gopalakrishnan?

  So RT one is like that, but I our next models are going to be whole body control. So which means that you move both of them together. If you really think about how you do your control, you are not either moving your arm or your head separately, right? At any given point, you move them together. So um yeah, that's going to be fixed.

• 35:22Nathan Labenz?

  So there's kind of a couple, I'm understanding like a couple of different paradigms here or ways that this could work. One would be that the language model says like, okay, based on what I'm seeing and what I'm trying to accomplish, I want to move the base to position X and Y. Okay, cool, we got there. Now, next time I want to move the gripper to position X, Y, Z. And you know, have a certain angle and have a certain, you know, open close. How does the sort of issuing of that command relate to the cycle time? Cause I would assume that like, you know, you can only accomplish so much before you're gonna be kind of running the whole process again.

• 36:02Keerthana Gopalakrishnan?

  Um, so that's like the action bound, right? The language model basically predicts a token. And the token is like a value between like zero to two fifty six. But each of those numbers correspond to like a a certain bound uh between no action versus go forward or backward. The language model is predicting still within like a bounded space of action that you can go to or not. And uh in RT one, the inference time was one hundred milliseconds. And then the full stack time was three hundred milliseconds. So that's like three hertz control. Humans are a lot faster than that. And um yeah, we we know that we need to optimize, but these are research robots, which is why

• 36:45Keerthana Gopalakrishnan?

  the stack is slower, but production robots uh would be like a lot faster than that. Um, and also, let's say, assume that once the language model says, go uh go to X, Y, Z position or whatever, and we are doing um concurrent or non-blocking control. So what that means is that um. After the language model tells the robot to go move to that position. Uh, the next cycle of inference does not wait uh until after that action is executed to start the next step of inference. So there is a thinking cycle which is going and then there is an executing cycle which is going. I mean, this is also like how you do it, right? Like you don't wait until, let's say, I don't know, you grab something to think about what is the next thing to do.

• 37:29Keerthana Gopalakrishnan?

  So thinking and acting are uh happening simultaneously and it is also important uh for for robots to be fast. Uh, because like there are moving objects, let's say if your kid throws your ball. Uh, you need to be dynamic and if you are slow, if your control is too slow, if if you move like like that, then you're not going to catch the ball. Um, so which means that it imposes certain limitations on the latency of the whole system. Uh, so one of the ways in which we tackle that is by using the token learner. Uh, which is like really compressing uh the tokens uh uh from uh from the pre-trained efficient net. And so that the inference times are faster and that cuts the inference time like one third.

• 38:14Keerthana Gopalakrishnan?

  Um, and also in language models, it's like everyone is excited about like really, really big models. But scale um comes with emergent proper properties and uh scale comes with like large reasoning abilities. But then that also means makes inference slower. So they are sort of like competing objectives. So we want fast control, but we also want to fit as much information, as much context as possible. Um, and uh definitely like in the future, our hardware, our inference hardware. Um, all of that is going to be much faster and optimized for transformer inference. Um, but right now there is still that competing objectives.

• 38:53Nathan Labenz?

  Okay, that's really helpful and I I think likely very clarifying for a lot of people. If I understand correctly, there are two like the command, I guess, right? The predicted command of go to X Y Z. That's explicit enough.

• 39:07Nathan Labenz?

  I gather that it can be just sent directly to the robot control system for execution. But then with the latest paper with Palm E, it sounds like there's a bit of a different architecture where it's starting to, instead of saying like, go directly to XYZ, it's giving like slightly higher level commands that are then received by and translated into actual action or motion by another model within the same system. So can you describe that version as well? And then maybe like the pros and cons, like why, you know, is that just a strictly better approach? Or do they have, you know, trade offs?

• 39:48Keerthana Gopalakrishnan?

  Uh, yeah. So actually, uh, Palm E and RT one work together. So, uh, Palm E is like the updated CACAN, which is a converting high level language into low level language. So converting between like how to go get coffee and then planning that with like feedback from the system and everything. And then RT one is doing the little things like pick up the coffee cup. So I think two models are like optimizing both of them is synchronously is a suboptimal compared to having one model do both planning, language generation and inference control. Um, and I think like that is the way that we will go. Uh, in robotics,

• 40:29Keerthana Gopalakrishnan?

  uh, we're not, uh, you'll see, you'll see future papers from us which go in that direction. Yeah. So Palm E is more like CACAN level, like reasoning about the, uh, environment, but also it is also doing like affordance. It is also doing like scene feedback. Um, and RT one is doing the exact actions.

• 40:50Nathan Labenz?

  Palm E, the successor to CACAN, will take in everything that's going on, which includes my having told it to get a cup of coffee and its knowledge of its position and state and its, you know, visual input. And then it will translate that to a low level command that is like grab the cup. And then RT one takes that input in and translates that to XYZ coordinates for the gripper.

• 41:23Nathan Labenz?

  What does sound nice about that architecture is you could have like RT2, 3, 4, 5, and 6 for different kinds of robots, but does RT1, it already handles multiple different types of robots as well, right? So is that just like a variable within the bigger thing? Like what kind of robot arm you're actually controlling? Is that just another token that's in the input? Or how does it kind of know which embodiment it is controlling at any given time?

• 41:52Keerthana Gopalakrishnan?

  So it has another input. So when RT1 was jointly trained with Kuka and with EDR, it was told where like the data came from and the action spaces were also sort of mapped together. Yeah, so one way that you could compose like of one model on like multiple robots is like using one token to determine like which robot you're influencing on. So let's say, imagine like if you're running like a robot dog and then you have one token saying this is a robot dog, which means you predict like, I don't know, maybe 24 variables versus if it's like the everyday robot that we are doing, then just predict 11 tokens.

• 42:37Keerthana Gopalakrishnan?

  That's one way where you can like still mix a lot of different data from a lot of different robots and then also have like transfer between them while allowing it to influence on a lot of different robots.

• 42:49Nathan Labenz?

  One of the videos that I thought was most striking from was an example of where there's these sort of exogenous shocks to the robot's plan. For example, there's one video where the robot is picking up a bag of chips out of a drawer and then the human comes in and knocks the bag out of its hand and puts it back in the drawer. And so it's a demonstration of the fact that The language model, especially as it's informed by the visual inputs, you know, obviously, it's not having these conscious thoughts of like, oh, somebody knocked this thing out of my hand, but it is realizing the bag is over there and, you know, I need to go back and get it to pick up my,

• 43:34Nathan Labenz?

  you know, to accomplish my objective. That's a pretty striking video. There's one with pizza being made as well and sauce being, you know, kind of smoothed around on the pizza. And, you know, the person comes and moves the pizza and the robot, you know, recognizes that and continues to apply the pizza sauce appropriately. So I was like, boy, that's pretty awesome. Then I was also thinking, what is the limit of how much perturbation I would want a robot to sort of push through in the real world? You know, and I have a four-year-old kid and a two-year-old kid and another new baby coming in, let's see, 25 days.

• 44:16Nathan Labenz?

  You know, it's definitely a very common occurrence in our house that I'm trying to do something and then, you know, there's a, there's a shock of a kid entering the scene and, you know, messing something up or whatever. How, what, how are you guys thinking about sort of a, okay, this is sufficiently big of a shock, you know, that we did not, that the model did not expect, that maybe it's time to just stop. You know, like I should, I should not pursue this goal anymore because something is happening that like, you know, is out of distribution. I don't know what to do with it again.

• 44:50Keerthana Gopalakrishnan?

  So these are like ethics questions, right? So it's like not a question of can you do it, but should you do it? And it's also like very similar to like safety questions. And here, our straightforward approach is to borrow from like AI safety, AI alignment research. So one way, like we are running a lot of autonomous scale operation. So like where the robots run around and then they propose what tasks to plan to practice. And then autonomous policies then try to attempt to practice those tasks. But now the question becomes like, imagine the robot is roaming around people's desks. It should not attempt to pick up their personal belongings. Or if people are sitting in the Google micro kitchen, it should not come near them

• 45:34Keerthana Gopalakrishnan?

  and then like try to disturb them or like. Propose any task that is harmful for them. So uh like like for example, one of the days um there was the robot that we were planning and it detected that there was a phone in my hand because I was recording it and it was like um uh take the phone and then pick up and then put it on the table. Uh but the phone was sitting in my hand as a human and I would not appreciate it if it just like grabbed the phone from my hand without any uh consideration for like how humans behave, right? Like before you take my phone from my hand, firstly, you need to understand that the social norm is to not grab people's phones from their hands. Secondly, you would ask for these things. So there's a lot of like H R I components in there uh which is like how

• 46:17Keerthana Gopalakrishnan?

  how can robots be like nice and polite even in social interactions? Um and we have a group that is working on H R I research, which is like thinking about like um how to make the robots polite, how to make them reason contextually about these things. Like for example, in this in this failure mode, the robot detected that detected a phone in the scene and proposed a task. But it did not detect that the relational uh aspect of uh the phone being in my hand and the fact that I was holding it. If we have better scene description, I think a language model would be able to reason that a phone, the phone is in the hand of a person. So then, you know, don't um don't do that. Um.

• 47:03Keerthana Gopalakrishnan?

  And so currently, our approach to safety now is asking a language model, like, should you do this or not? We also have like for tasks like collision and stuff, we have like more harder bounds, like it would not do tasks that actually have collisions. Even if the language model plants and everything, it is still like bounded, i- i- it i- i- it should not be uh like dropping objects or like colliding into things. So there is a hard bounding uh from a control perspective, which is just like, don't run into anything. Then there is the question of like asking a language model, is this a safe task to do? Is this a nice task to do? Or if not, can you explain to me why uh this is something that?

• 47:47Keerthana Gopalakrishnan?

  You should not attempt to do. So some of the tasks like our robots right now have only one hand. So uh when it is given a task that requires two hands, then it should be like, I I'm not able to do it. Because I I don't have two hands. Or if it's asked to lift a heavy object that's above its payload, it will be like, it is unsafe for me to do. Because it's a heavy object. Or if you say, like, pick up the thing with the hot coffee, it'll be like, this is a very hot, uh like, hot. Object that I should not be handling. Um so maybe you do it. So like, so um and then, so now here we come into the realm of collaboration.

• 48:24Keerthana Gopalakrishnan?

  Between a human and a robot, where the robot is closing the loop in itself. Robot knows what it should do, what it shouldn't do, what it can do, what it can't do. And also how to be nice while doing these things. And now, so once it determines, once it can classify these things. Now the question is like, you should still be able to get the coffee to the human, even if it's hot coffee. But if you are not able to handle that, if you think you're not safe enough to handle that, then you need to replan and reason about how to still get that task done. So one way could be you can uh like chat with the robot, uh our robots have like a chat interface, you can tell the robot back, hey, uh the human back, hey, look, um the the.

• 49:07Keerthana Gopalakrishnan?

  I have the hot coffee in there, can you come help me pick it up? Um or so that is the realm of like collaboration and or like intervention, you know, like, so I think deploying robotics in the wild. Uh is not going to be with fully autonomous policies, but with auton- autonomous plus interventions. So like how self-driving cars are, right? They are good for handling, aut- autonomy can handle like ninety-five, even ninety-eight percent of cases. But those like, those two percent of cases were like, or even one percent, i- if you run at like ten hertz in front. And you have one percent mistakes, you are making a mistake every ten seconds. And that's like highly unsafe. So but.

• 49:48Keerthana Gopalakrishnan?

  What the self-driving cars are good at are knowing where they are good at and where they are bad at. Um and they can ask a human for help when, let's say. The map is doesn't make sense or like there's a person there saying asking the cars to go around. So similarly, I think that the way that uh robots will be deployed is like they know their bounds of operation. And they also uh can ask a person for help when they are confused or when they do when they can't do something. And they also have like hard bounds around them, like common sense things, right? Like, don't hit a child or like, don't run into things. And even if a language model somehow plans like hit the child or whatever, then you still have like safety on robot like control,

• 50:31Keerthana Gopalakrishnan?

  which is not a machine learning model, which is deterministic control that says like, don't run into objects.

• 50:37Nathan Labenz?

  Uh that sounds like, for one thing, kind of another Herculean labor to collect all that data. Cause that's another level that doesn't really exist, right? Uh or maybe you're in the process of collecting it now, but there's not really a lot of examples, certainly out there on YouTube or whatever, of robots running into situations where they, you know, determine that like they can't do the task.

• 51:02Keerthana Gopalakrishnan?

  So uh definitely, I think that we will get a lot of transfer from a language model, like all the work that people are doing for alignment and safety of language models and other now with like G P T four or multimodal mo- models, like don't generate harmful uh output, don't ask it to do harmful things. So that's definitely going to feed directly into our research that's there. But we also need more information about the specific failure modes of our data of our problem. And once it is close to productization, so you know, like how Vemo or like self-driving companies have their simulators to test for safety related things. They also have like human in the loop, um which is like constantly testing the hardware. So I think all of those things will come in.

• 51:45Keerthana Gopalakrishnan?

  Uh but robotics especially has like a particular imposition on safety, which is that language models can be wrong on the internet, right? You ask Chat G P T something, it goes wrong, no problem, or it says something weird, no problem. But then if you make a bad plan, if you like drop an object, you break things, you cost, you cost money. Um. And you cause you probably injure a person or uh destroy an object uh which is so there is a higher penalty of a failure on uh robotics models um which is going to be uh which is going to be very important towards their safe deployment. Well, I don't think that this is an impossible problem to solve. Look at self-driving cars. They're already making money in San Francisco.

• 52:29Keerthana Gopalakrishnan?

  Um. So it's just a question of us doing it carefully with human in the loop and then um rolling it out in phases. Yeah, initially, everyone was like self-driving cars would never work because they can never solve one hundred percent of the problems. There are always going to be long tail and whatever. But yeah, you can, you don't have to solve the long tail. You can solve as much as you can and then you can uh make humans solve the rest. So you're basically doing intervention. Assume that the robot already works, like start making money today and then keep solving the problem, then reduce the intervention rate. So then that that becomes your whole problem. I think bringing robots to people's houses will also be like deployed

• 53:10Keerthana Gopalakrishnan?

  today with whatever your like capability you have and then use humans to fill the gap of your algorithms right now. And then as you go, as you make money, as you keep improving, uh slowly phase it up.

• 53:23Nathan Labenz?

  It's funny you mentioned the self-driving cars too. I just took a ride in my neighbor's Tesla that has the full self-driving enabled and I came away from that feeling like first of all, just pure wow, like it really does work amazingly well. And also like, nobody seems to be acknowledging it. It's this odd, uh, you know, as you said, like, people say it's never gonna work. Like, you can still go online today and find, you know, recent articles, probably can find one from today that says like, FSD is never gonna work. But I just wrote in one and it definitely is a lot farther along than some of the uh, you know, the naysayers would suggest. So it sounds like robotics

• 54:04Nathan Labenz?

  from this conversation uh is probably in a similar position. I wanted to talk a little bit more about kind of the hard bounds because that also is something that seems, as you just kind of spoke about, much more important in the the robotics uh. Paradigm than language models or others. One article, just for listeners, that I really recommend, because I could go back to it repeatedly, just to kind of meditate on like how I work as a human, is this Scott Alexander book review of a book called Surfing Uncertainty. We could put the link in the show notes, but basically, he kind of talks about how the human, like biological nervous system, just has like a ton of layers.

• 54:48Nathan Labenz?

  And you've got like, you know, the prefrontal cortex layers are kind of the highest layers that deal with the highest level of abstraction. And then, obviously, you know, radically oversimplifying. But as you kind of go down the nervous system, all the way out to the periphery, there's just like layer after layer. These are probably not discrete layers, but you know, to some extent, there are discrete cells, so there is some amount of discreteness to it. Anyway, whatever. It's it's not meant to be too, too literal. But as you get out to kind of the periphery, you have just like very low level interpretation of what's happening. And, you know, you can do things like remove your hand from a hot stove before there's certainly any conscious thought of the hot stove.

• 55:33Nathan Labenz?

  So all of that, you know, and I highly recommend that article. All of that to just kind of preface, like, it seems like robots need something similar, right? Where they they need to sort of detect that like, I'm encountering resistance, you know, or like, the thing that I'm grabbing is like deforming in my grasp more than I, you know, expected it to, or more than it seems like it should. And therefore, I need to kind of back off, right? I need to like, slow down. I need to, you know, I don't necessarily want to wait for the next inference cycle to withdraw pressure or, you know, back off of a collision or whatever. So how is that working today?

• 56:14Nathan Labenz?

  Is that like a non, something more of like a classical system that's not AI driven? Or is that also, you know, something that the the models are ultimately going to handle in your view? Like, what's the future of that look like?

• 56:25Keerthana Gopalakrishnan?

  So let's go back to the example of the touching the hard object, right? So there is like the planning in our body, which like goes to the brain and the spinal cord and whatever. And then Tells you that you should not touch the hard object, but then uh when you touch a hard object and then you reflectively act, you are not going all the way to run the inference step. This is what I was previously talking about by various layers of safety. There is one layer which is the language model telling you should you do it or not, don't touch hard objects, don't handle whatever. But there is also like low level control layer. Um uh RT one also has its own like safety, it will not uh do like highly unsafe things because it doesn't have the data and it is like

• 57:09Keerthana Gopalakrishnan?

  the data is cleaned and everything, but also on the robot, on the control level, uh just like the uh C++ code, there is also a way to determine like don't do collision uh related stuff, like don't run into objects. And that's like, that's very easy to detect, right? Like you know, uh you know, you know, the occupancy in space, uh don't deform the ob- like, don't uh run into things. Uh but also more than just collision, there's various other types of feedback, right? Which is like tactile sensing, um it's like, like as you said, right? Like if you try to pick up a like a full bottle and I don't know, like i- i- if it deforms a lot, then maybe don't do it.

• 57:52Keerthana Gopalakrishnan?

  Or if it's like, if if it spills over, don't do it. So right now, our robots don't have tactile feedback on them, so like the gripper, so it it it it knows some like, it has some pressure feedback. So if it's like a very hard object and then you squeeze it, you won't be able to squeeze it and that's reflected in the like the max torque applied on to the thing. But then there's also like more softer feedback, like eh to like cut like vegetables or like soft objects like tomatoes and stuff, you need more uh finer feedback, uh which is not there yet, but then we can still do a lot. We can still deconstruct safety into various layers with uh existing like just force feedback.

• 58:31Nathan Labenz?

  So that I think that's reassuring that there's like some, that it's not fully a black box, that there is this kind of C++ safety layer that is like a hard override and uh zooming out just a little bit more still and and just kind of kind of like returning to some of our, you know, very early discussion around. How will interact with robots? What kind of role, you know, they are going to play? How fast do you ultimately think we will want robots to be? As I watched the videos, you know, it is notable that they run pretty slow today. Certainly like much slower than humans. Like the videos are typically shown at like either four X speed or 10 X speed. I've seen both. So they're, you know, they're pretty slow in today's world.

• 59:16Nathan Labenz?

  Obviously you'll want to speed them up. But I wonder if you have like a sense for what the optimal frequency is for a robot to be operating at in a human environment? There might be a maximum at a certain like human like frequency that would be sort of accessible. You know, if we could run them at a thousand hertz, I'm not sure that we would even really want to.

• 59:39Keerthana Gopalakrishnan?

  Frequency of inference is not the only variable for speed. So frequency is think of frequency as how reactive you are. So even if you're running inference a lot, but each step you can only move a little bit. Then let's say to go to here, you're gonna let's say run like 10 influences. But then you can also go to here in one influence, right? So there are two variables to control here. One is how reactive you are and your inference frequency. But second is also like how much movement you are allowed in each inference step. Time to complete a task is within two X or three X of what a human would take for that task. And then that is about optimizing both the machine learning on on robot software

• 1:00:23Keerthana Gopalakrishnan?

  and also optimizing for both reactiveness as well as speed. We didn't play around with speed a lot because it can be a little bit dangerous, right? You're trusting the inference a lot, but we like, I think because these are research robots, we haven't really optimized all of these things a lot. When they come into production, then that becomes a more important question. But I think it's fairly solvable still with with existing machine learning models and existing robots to get at least like within two X of a human. Yeah, robots, you would be freaked out if a robot were as fast as you in doing things, right? And as good as you, I think it's fine to give them a little bit more time until they like,

• 1:01:06Keerthana Gopalakrishnan?

  I don't know, ease into society and people are like, okay with them.

• 1:01:10Nathan Labenz?

  Yeah, totally. I agree with that for sure. I have a, you know, the most advanced robot in my home right now is the Roomba. And it's, you know, kind of stumbles its way and, you know, crawls its way around the house. It's not, not too smart and doesn't figure things out too quickly. But it kind of feels appropriate, you know, and like we run it overnight. So it's one of these things too, where it's like, it doesn't really matter if it takes it an hour or two hours or, you know, whatever. Five minutes, like it wouldn't be much better for me if it was a five minute task, you know, for the robot instead of an hour. But honestly, it probably wouldn't be any better for me. So yeah, I do think you're right. But it is also interesting to hear that perspective that increasing the frequency of inference

• 1:01:54Nathan Labenz?

  really just smoothes the behavior as opposed to really changing the behavior. And then you have a separate decision to make around like, okay, per unit time, whatever that interval is, how much am I going to move in that unit time? So that's something you can kind of control independently, which is, that's helpful because I hadn't really conceived of it in that way. What do they like to work with? Like, do they break a lot? Do they need maintenance? Are they heavy? Like, how, how robust are these robots that you are currently working with?

• 1:02:25Keerthana Gopalakrishnan?

  So I have worked with space robots, like we, when we were in grad school, we sent a little robot to the moon. I have worked with like self-driving cars and I've worked with everyday robots. And I think these are the most stable, at least at the fleet scale they have. And they're fairly robust, like the navigation stack and everything is fairly reliable. They don't break as much. They don't run into objects, like everything works. And I think it's because like Google's really good at engineering and we also have like great ops folks who are like keeping everything like, like it's like oiling the machine, right? Like, and yeah, they, they are.

• 1:03:05Keerthana Gopalakrishnan?

  Fairly robust and fairly good robots uh so far, I think. They they don't they don't break easily. Um and it's very convenient to like run experiments on them. We still have a long way to go. With like solving smaller issues and stuff, but I think they are the best robots that I have worked with so far. And another thing is that like you you see industrial robots, right? They are they are also fairly robust. But they are very constrained environment, like it's like fixed base, it's like doing picking or like it's like doing scanning. Uh but these robots also move around. So compared to the complexity of the domain that they operate, I think their reliability is really good.

• 1:03:42Nathan Labenz?

  Do you think that they are the right form factor for like ultimately, you know, use in homes?

• 1:03:50Keerthana Gopalakrishnan?

  I think form factor is a very, very important question and there's a lot of like debate among roboticists on like what is the ideal form factor? Initially, a lot of robotics was like fixed base, so um they they were just like like these Kuka arms and stuff. They would just like do things from a fixed location, but this is not very general robotics, which is why we now have like arms moving around. But the arm is the most costliest part of these robots. So if they have like two arms, then uh the robots is going to be almost twice as expensive. And then the equation, the economic equation would not work out. Um so there are like various ways. So cost is one trade off. Up until this point, we were limited by the robot software.

• 1:04:33Keerthana Gopalakrishnan?

  Um and so if if you cannot even do one hand task, then it doesn't justify having multiple hands and legs and whatnot on the robot, right? But now we are seeing that we are limited, the robot capabilities have increased to a point where we are actually limited by the uh we are limited by the hardware. So we we have cracked a paradigm where it's like throw data and then just learn it. Um so then the question is like, how much data can you throw? And then on which robot should you throw the most data? And I think that if you really think about it, if you really think about like the ideal generalization, like like a superhuman physical intelligence, it's probably going to look like.

• 1:05:14Keerthana Gopalakrishnan?

  Humanoids, right? Because our world is designed for humans, like your kitchen is designed for humans. A car has like a is designed for a six feet human. Um. And a cup is the uh the dimensions of this, like what fits in my hand. Um. A lot of the data on YouTube is from like a human egocentric perspective. Um. With like human hands operating. So i- if you look at it from a design of the world perspective, then humanoids are uh the most optimal form factor. If you look at it from a data perspective, like NBA basketball agents playing, uh they they look like humans. So then and you probably need like a human like form to play very good basketballs, right?

• 1:05:58Keerthana Gopalakrishnan?

  Both from a data and a utility perspective, human humanoids, I I think are the ultimate form factor for robotic. But um if you really think about it, stability is uh if is a very, very hard problem to solve. Um. And Boston Dynamics has like really, really complex robots, which all of them are leg robots. But stability is a is a very hard problem to solve. And then that means that um all of your problems become stability problems. While you are like picking things up, you are also worried about like falling. And what if you do fall, you are a danger of breaking parts or injuring other people. So I think that one of the reasons that uh Boston Dynamics has like little stubs for their arms is because like they invested into legs too soon.

• 1:06:43Keerthana Gopalakrishnan?

  And then now all their problems are stability problems. So initially was one arm on fixed base robot, then one arm on mobile base robot. And we like now we have navigation and manipulation together. I think the next is like by manual manipulation. A humanoid on wheels is basically two arms and a camera at human height, right? It's not like, it sounds so complex, but it's actually just two arms and a camera. Um on wheels. So I think that would be the um form factor for a lot of like indoor and office applications. Like most homes nowadays have like, most offices have like elevators and stuff. And most, for most robots, like if they're on the floor, they can do a lot of things. And then the next step would be like leg robots, which adds an additional level of complexity.

• 1:07:27Keerthana Gopalakrishnan?

  Right? Because um imagine that you do build a superhuman intelligence and then it goes to deliver, but then it can't cross a little curb on the in front of your door. So you need legs to like go upstairs or go uh past cur- uh curbs or even to do like outdoor robotics. Um so humanoid on wheels would be the next version uh with two arms. Um then the then would be like legs and also are like fingers, like five fingers. Uh would be uh another upgrade, uh because a lot of things you can do with um with just grippers. But then uh fingers are highly complex, like we have so many degrees of freedom in here and it's ho- highly complex to plan and stuff.

• 1:08:07Keerthana Gopalakrishnan?

  But uh ultimately, a lot of the tasks would need more finer, dexterous manipulation.

• 1:08:13Nathan Labenz?

  As you're describing that, I have this kind of image in my mind that's like analogous to the human evolution. You know, where it starts with like the great ape or whatever, and then it gradually sort of becomes more, you know, uh to human over like six steps. You basically just described the six steps that goes from one arm to, you know, the eh the ambulatory robot. It's a pretty similar, you know, evolution, so to speak.

• 1:08:38Keerthana Gopalakrishnan?

  That's why I'm so excited about robotics, because it's like we are inventing ourselves, right? It is in many ways a quest to understand us and our intelligence. And it's so hard to put down onto paper why, how we detect like a cup or how we are doing these things or how we are planning tasks. Like at least high level things you can like describe in language, but low level manipulation, low level uh influences, you really cannot. Like why your arm and leg are moving in a certain way, you cannot. But then building, like you know how software engineers say the best way to learn something is to build it? And I think robotics is basically our. Best to understand ourselves and build more of ourselves.

• 1:09:19Keerthana Gopalakrishnan?

  Uh. If you think about uh G P T and language models are doing a lot with respect to like scaling uh intelligence. Right uh and bringing down the cost of knowledge work like if you think about it, industrial revolution has automated a lot of uh mechanical labor that our lives are much easier. And now with the intelligence revolution, we are like automating a lot of knowledge work. Uh. But then truck drivers in the United States make like two hundred K because no one wants to drive trucks. Um. Even like people that like sand boats apparently in San Francisco makes like ninety five dollars an hour. Um. And which is like, you know, comparable to a software engineer. Um. So as uh the as the availability of labor changes uh once like more knowledge works gets automated.

• 1:10:04Keerthana Gopalakrishnan?

  Um. Physical work is going to be the most uh valuable like when my friends ask me like oh, software engineers are getting automated. Artists are getting automated. I- it's like the more of a paradox, right? Thought the things that we thought were hard to do are actually easy to do for computers. But the things that we find are easy to do are really hard. Uh. Like we for us, like composing these images and stable diffusion is like super hard and editing those things. But then computers apparently can do that. But something so easy as like going to my kitchen and make me a coffee or like take my dog for a walk that we take for granted and we think is so easy is like really, really hard for computers to do.

• 1:10:46Keerthana Gopalakrishnan?

  So but physical labor still uh accounts for a large portion of our G D P. And automating that has like a lot of economic opportunity. And I think eventually, if if we if we think of like our species like really going forward and capturing uh different planets or expanding ourselves. Uh. Then definitely solving both A G I in the digital world and A G I in the physical world is going to be very important.

• 1:11:12Nathan Labenz?

  We had Su Hale from Playground, which is an image creation service that people are obsessed with. He has a young child as well. He said, you know, my, maybe my kid, who knows, like, maybe some of their best friends will be AIs. And then we also had Eugenia Koita from Replica, which is the like, you know, virtual friend app that, you know, is all digital today, you know, which people are already falling in love with. And that's become like a big challenge for her because people are doing like, you know, erotic role play in the app. And she's like, you know, that I might have problems there in any number of ways.

• 1:11:51Nathan Labenz?

  So they, they just kind of dialed that back and they had a bit of a backlash from users because people really care about that stuff as they become attached to it. Does it, is there anything that's like blocking in your mind, the sort of extension of, you know, when I think of this, I just think of like picking up the toys. Like that's as far as it goes typically in my imagination. But then, you know, when I think of Su Hale and his kid, and I think of Eugenia and all her users, it's like, boy, are we going to have robots that are like playmates to our kids? Are we going to have robots that are like robot spouses?

• 1:12:23Keerthana Gopalakrishnan?

  We, we are gonna have them, like in our, in our lab, like on days like I'm like driving to work and then I have booked like two robots to like play with me the whole day. And then at the end of the day, and you get close to these robots, right? Like you are like, this meta or this one, they have the numbers at their end. And, and then you're also like, at the end of the day, I like cracked one robot's wrist because like I moved the chair against it. And then you spend the whole day with it and then you're like, okay, I had to like file a bug to take it to like, it's like a hospital kind of service that they have where they fix the arm. But like, I felt like so bad that I broke the robot, but the robot would never understand that I'm actually sorry.

• 1:13:07Keerthana Gopalakrishnan?

  And I, I, I would not even, the robot never told me that it's okay. But a human, if I, if I would hurt a human by accident, they would tell me that I, I'm, I'm gonna be, it's good, it's, it's okay. And I would feel better about it. So that's one thing. You eventually start anthropomorphizing these robots, right? Because now they're also getting smarter with language. Like I can tell it, let's go on a walk and then. That language command is enough for me to like take it on a walk. Or like it, I can like teach it things, I can talk to it, or it says like sorry for doing something. Or I think sometime we had like a little bit of like creepy programming on it where like uh. You're working in the kitchen and then it would like go up to you and be like, hey, are you my creator? And I'm like, dude, what?

• 1:13:52Keerthana Gopalakrishnan?

  I already feel, I already feel so like when I wrote my bio as mother of robots. Uh it's because one day uh I was, I was like sitting in our robot classroom and I'm like. Oh, is the robot better at doing these things or my dog more consistent at like uh instruction following? Uh and I'm like, oh, maybe we should like have a face off between them. And then I'm like, wait, but whoever fails, I'm gonna feel so sad because both of them are like my babies. Uh and I'm like training both of them and uh and and that's when I'm like. They are actually an extension of us, right? If you, if you think about language models, they are basically condensing all of human history, all of our experiences.

• 1:14:34Keerthana Gopalakrishnan?

  And with multimodal models, like all the videos that we are collecting, all of our information about the world, all of our. Even like more private parts, all of our fears and everything. So in some sense, they are a distillation of the human mind itself. And not just one human mind, but a human mind across history and a human mind that is sort of also omnipresent in space. And time um and in in when you put it on robots, it's like. A human mind distilled, but also with physical capabilities. I can see that becoming really, really close to like being um being human and and maybe in the future, right? Once we have BCI's or whatever, like like I I I really uh believe that like.

• 1:15:17Keerthana Gopalakrishnan?

  The future of machines is also the future of us. Imagine that one thousand years later, our bodies have exactly the same skills as we have today, that would be so disappointing. But I imagine that, but I think it's very possible, like with Neuralink and everything, we have BCI's, we have more powerful compute on us. Like eventually like we merge with machines in a way that. Uh they are like our progeny, which is also one reason why like I'm not that worried about like AI. Uh taking over the world and how they uh like if if. And this might sound a little bit defeatist or controversial, but if they are in fact smarter than us,

• 1:15:58Keerthana Gopalakrishnan?

  then I would want to be like merged with them and have a common future with them than to be like, like it would be like if monkeys said that humans should not be formed, right? If they are actually superhuman and so much better, then we should just become one with them.

• 1:16:16Nathan Labenz?

  Love it. Fascinating. I assume that inference is going to kind of have to live on the edge for practical robot deployment. It seems like the latency of like going back and forth to the cloud is probably not viable if you want to run it like 10 hertz or whatever. So first of all, like, is that right? And then second, what about sort of on the edge fine tuning? Is there a paradigm that you see in the future where, you know, these robots will learn, you know, the layout of my home to take like one very simple thing or like our preferences,

• 1:16:54Nathan Labenz?

  you know, and start to have kind of a more customized, like, you know, the the sort of final finishing training would be like done in deployment, you know, by kind of human feedback, but like from the customer, you know, the actual, you know, homeowner or business owner, or whatever the case may be. How do you think that shapes up long term?

• 1:17:14Keerthana Gopalakrishnan?

  So I used to believe that you needed to do onboard inference for robots. But now we are seeing with like larger models like Kami and Polly that we are running on the robots. We are actually doing inference on a different server and then pinging back and forth. And so far, the latency is still good enough to like go do that. And then you, I think it would be very hard to run like bigger models on robot. But then we, this is still new technology, so we will have to see how how much, how they evolve. So maybe in the future, we have chips that can run like multi-billion models on robot, on device. That would be great, or we have some way of like fixing. The latency is so that like it is, at least right now, it is still within operational limits.

• 1:17:57Keerthana Gopalakrishnan?

  It's still good enough with language models. Now you are seeing like more personalized AI, right? Where you can shape the character of AI to like even with prompt engineering or even with like fine tuning on top of like an existing language model. Uh, you can um customize it to your application like it has like various different advantages. One is like privacy, um which is that you don't want to give your data to some large company to put it in their training set, but you want to keep it in your house or in your office. And then a little bit of fine tuning on top to personalize it. Um, or even like, um, uh, yeah, so, uh, I definitely think, uh, this will also be the way that once, once we build like a uh foundation model for robotics.

• 1:18:40Keerthana Gopalakrishnan?

  Uh, I think we, we, uh, these paradigms will start to happen. I definitely think it, it, it, it has to be, um, it has to be fine tuned, right? Like, uh, imagine a robot that is in your house. It has to learn the preferences of you, your kids, your schedules, what, how you like certain tasks to be done.

• 1:18:58Nathan Labenz?

  I noticed that there's, I believe, more than fifty authors on the RT one paper. And you've given us like a little bit of a, you know, a couple of different angles on like, oh, a hospital service for the robots. I'd never thought of that. Can you just kind of describe the team that has to come together to make all of this work?

• 1:19:17Keerthana Gopalakrishnan?

  Firstly, I, I want to say that I have an incredible team at Google. They are like, uh, you know, one thing I'm realizing in life is that to do great things, you need to stand on the shoulders of giants and also work with like really smart people. And, uh, my colleagues at Google are, they are like the smartest in the world. They are the best in the world at what they do. And they are still so humble and so curious. And you can challenge them, you can ask them questions. And it's, it's like really nice to show up to work and, uh, be heard for your ideas and be respected and to work with like world-class people. And now the way that we uh organize these large efforts, so we have like uh the big papers that we do. And we also have like splinter papers. Splinter papers are usually like smaller collaborations, like intern projects and other things.

• 1:20:02Keerthana Gopalakrishnan?

  And big papers are like more foundational uh like uh like upgrades in robotics technology that we publish as an entire group. And the way that this works is like uh someone leads the effort uh there they are uh uh but then everyone's included. Uh because uh we like doing robotics takes a lot of people, right? Like uh everyone who's in part of the ops and keeping the robots running. Who's collecting the data for it, people who advise, people who actually implement things and like train and evaluation. Um so um the bigger papers are fairly inclusive and then we try to like bring everyone on board to publish like the large. Um upgrades in robotics technology and for the smaller ones, it's usually like people involved and then they have different guidelines and stuff.

• 1:20:46Keerthana Gopalakrishnan?

  And even and at Google, we always like uh we always lean towards uh being inclusive and uh sort of including as many people as we can.

• 1:20:55Nathan Labenz?

  The role of A I in your personal life right now, are you using any particular products, services? That you find exciting and would recommend to people?

• 1:21:07Keerthana Gopalakrishnan?

  Uh so I use uh Bard or even uh Chat G P T sometimes to like ask it questions uh to do uh like. One of the thing uh to explain code, whenever I'm writing new code and like especially with web programming and stuff, you are writing a lot of templated stuff. Uh just asking uh Bard and Chat G P T is like very useful for like tail queries and that like search engines are not that good at doing. Uh that's one A I product. Uh yeah, I also like the uh summarization thing. So these days in meetings, we are like, if you have a large meeting that's like running over time. You just like take the whole thing, summarize it and then uh have these bullet points. But mostly I I use uh.

• 1:21:47Keerthana Gopalakrishnan?

  A I to like code and even like make like fun poems and stuff. And nowadays, like whenever uh people ask me to like write emails, uh I feel so lazy. Because most of a lot of like being obligatorily nice in an email is just like filling it with text that you can just ask G P T to write it, right? So sometimes like when I get a lot of emails where I I wanna give like a short response, but then it would be impolite to be like cut. I would just put it in there and then ask it to make like a bigger email and then just paste it in there.

• 1:22:19Nathan Labenz?

  Secret safe with us. So let's imagine that a million people already have a Neuralink. And if you get one, it will allow you to type or create text as quickly as you can think. In other words, you have thought to text. Would you be interested in getting one?

• 1:22:44Keerthana Gopalakrishnan?

  Absolutely, yeah. Why would I be not?

• 1:22:48Nathan Labenz?

  Well, you know, people are a little squeamish about holes in the skull. Uh, so that's one common objection that we've heard.

• 1:22:55Keerthana Gopalakrishnan?

  But a million people have it, right? So it worked for them. I in fact, I don't think that I would wait. I'm an early adopter. I don't think I would wait for a million people to get it before I get myself.

• 1:23:07Nathan Labenz?

  I think I'm with you, and that's part of why I asked the question. But it has, this has been actually one of our more polarizing questions. We get, I've had a couple of responses just like yours. I'm like, I didn't even need to wait for a million. And I've had others that are like, the last thing I'm gonna stand for is something that can read my thoughts, you know, being physically implanted into my body. Uh, and I do kinda get that, although personally, I think I'm enough of an enthusiast that I would, uh, I would probably be with you on the, uh, the earlier wave.

• 1:23:38Keerthana Gopalakrishnan?

  I'm also big on like privacy. So like I don't do any private company products that like scan my retina or anything. But so if it's like running inference on chip on my brain, that's fine. But like if it's like sending my cards to a cloud to run inference, then I would not be okay with that. So yeah, there are like, I think some constraints on like the bounds of how the technology works. I I don't know. Yeah, I'm very curious and very excited about it.

• 1:24:04Nathan Labenz?

  What would you say are your biggest hopes for and also fears about the places that AI may take us?

• 1:24:12Keerthana Gopalakrishnan?

  Um. So firstly, um, I think AI, uh, AGI has um a lot of promise um in like both automating knowledge work, automating uh bringing a lot of utility into our world. Like, I think in in in many tasks, uh, in many of uh our uh. So imagine that humanity is like pushing forward, right? Like, uh, we are making inventions in science, we are making inventions in robotics, technology, a lot of these things. But we are bottlenecked on intelligence on many of these ones. But imagine that there is like an AI scientist that can like propose hypotheses, run experiments, write papers. Um. They can invent new things. Uh. Our world and our technology would like move so much faster.

• 1:24:56Keerthana Gopalakrishnan?

  My hope for at least AI in this decade is hopefully we can train like an AI scientist um at least on a few things. And we can automate a lot of different types of knowledge work, like software engineers could be much faster, doctors could be much better. Um. And we we are not then resource constrained on intelligence uh in our world. That would be amazing. And even like even in physical intelligence. But of course, AI is a very foundational technology, very impactful, and it comes with a bunch of uh. We need to do it correctly. One thing that I'm worried about is uh increasing inequality.

• 1:25:35Keerthana Gopalakrishnan?

  Like if AI is built uh by large companies and then it like you then you use less and less labor. It it leads to a certain level of centralization. Even now with like uh language models and stuff, there are very few people who can train like models that are like billions of parameters. And that creates that increases the existing inequality already. And like a lot of people are left behind. I'm very concerned about that. And I think that the open source movement is going to probably try and equalize the field. So I'm very, I'm really cheering on like stability, AI and other companies. Uh. For leading that effort, uh, hugging face also. Um. The second thing uh that I'm worried about is um.

• 1:26:14Keerthana Gopalakrishnan?

  Safety. So um I'm fairly an accelerationist. I'm for uh we need to build this and we need to build this fast but also safely. But there is a finite uh probability that things could go wrong. And uh so we need to be careful about, we we also need to make sure that we solve alignment to a reasonable extent. That as we like phase out these uh this technology into products, we are careful about how they are being used. Um in fact, I think that productization of A I uh is going to accelerate not just capabilities research, but also safety research, because ultimately only an aligned A I is a useful product.

• 1:26:54Keerthana Gopalakrishnan?

  If it's not doing what people want it to do, but it's like gaslighting you to leave your wife or like threatening you, it's not a good product, right? So um I think it's great that like Sydney Bing went first and then told people what a bad chatbot could be. Because that gives the opportunity, that made everyone think about how important it is to build a really good chatbot. And it presents an opportunity to do this right. And I'm very excited for Google to uh take on uh that challenge and uh fulfill and meet people's expectations. I think we are approaching the development of A I in also a responsible fashion. We want to do this well in addition uh to uh being one of the first uh in the markets.

• 1:27:35Keerthana Gopalakrishnan?

  Yeah, uh so I think safety is one concern, and I think productization and acceleration of A I into products will also accelerate safety research. Uh the third thing I'm worried about is uh bias in A I. So in addition to existential risk type of safety, it's also like um I'm I'm a person of color, I'm bisexual, I'm a female. So I'm a minority in various ways, and I can, I I know that like uh language models and other things have like biases with these models. Uh for example, if you say go to Kiratana, it would not understand. But uh if you say go to David, it will go to uh because it's like uh because like Western names or or male names uh are more represented in the data set.

• 1:28:17Keerthana Gopalakrishnan?

  Um so, yeah, so I'm I'm concerned about like uh A I ethics and A I bias. Uh I think it's important problem to solve in order to like bring the benefits of A I to everyone and not just like um a dominant majority of population. And A I is making everything data driven and data is highly uh like, it's like a mismatch. We like we have different distributions for different distribution of people.

• 1:28:42Nathan Labenz?

  Keerthana Gopal Akrishnan, thank you for being part of the Cognitive Revolution.

• 1:28:47Keerthana Gopalakrishnan?

  Thank you for inviting me. I had a lot of fun talking about these topics and great questions as well. I'm really looking forward to it. It was a very technical podcast, more so than any podcast that I've seen. So, so, so thank you for that. And, and I hope that your audience will enjoy.

• 1:29:07Nathan Labenz?

  Thank you, and I am sure that they will.

• 1:29:10Speaker 02

  Omnikie uses generative AI to enable you to launch hundreds of thousands of ad iterations that actually work, customized across all platforms with a click of a button. I believe in Omnikie so much that I invested in it, and I recommend you use it too. Use Cogrev to get a 10% discount.