Ep 56: Distinguished Engineer at Waymo Vincent Vanhoucke Unpacks the Breakthroughs and Bottlenecks of Self-Driving

Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. The company operates a 24/7 public ride-hail service and provides over 150,000 trips each week across San Francisco, Los Angeles, Phoenix, and Austin, making mobility more accessible, sustainable, and safer for everyone. In this week’s episode of Unsupervised Learning, we dive deep into the frontier where AI meets hardware — and there’s no better guide than Vincent Vanhoucke, Distinguished Engineer at Waymo and former Head of Robotics at DeepMind.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Jacob Effron?

  Vincent Van Nieuw is a distinguished engineer at Waymo, which he joined after founding and leading the robotics team at Google. I'm Jacob Effron, and today on Unsupervised Learning, we got Vincent's takes on both of these spaces. He talked about how LLMs have changed autonomous vehicles and robotics. He talked about how Waymo thinks about the amount of sensors they have on their car and what the key milestones are left in the autonomous vehicle space. And then we also hit on AI and robotics, what the state of the space is today and what we'll learn over the next few years that will determine the trajectory going forward. These are topics I've wanted to hit for a long time on Unsupervised Learning and can't think of a better guest than Vincent to talk through them. I think you'll really enjoy the conversation. Without further ado, here's Vincent.

• 0:38Jacob Effron?

  Well, thanks so much for coming on the podcast. Really appreciate it.

• 0:41Speaker 01

  Thanks for inviting me.

• 0:43Jacob Effron?

  I've been looking forward to this one for a while. I feel like a lot of different places we'll dive into today, but I figure we'll start with your current employer and Waymo. Waymo was working on self-driving well before this kind of like explosion in advances of LLMs, diffusion models, everything that's happened in the last like, you know, four or five years. And I'm wondering to what extent that's changed the way that Waymo's technology works.

• 1:02Speaker 01

  Well, the interesting thing is that to some degree, nothing has to be thrown out, right? It's something that comes on top, right? The current sort of foundation model revolution

• 1:14Speaker 01

  is about building teacher models, like very large scale models that can run in the cloud, that basically suck in all the data that we have available in addition to internet data and to build a very large model of the Waymo driver and the behavior of the car and the environment. You can take that teacher and use it to train and distill the data into the onboard models that you have on the car without necessarily having to do a lot of retrofitting. You can

• 1:55Speaker 01

  As a starting point, not change anything and just use a different mode of supervision for for those models. Then you can also evolve then on that basis and try to get models that are bigger, higher capacity, higher expressiveness and things like this. But that's on top, right? So you don't throw away anything. You just give every model a better teacher and a lot more information to work from, basically.

• 2:22Jacob Effron?

  How has that actually been manifested? So you had this kind of way of doing things before and in a previous set of models you worked with. How have, you know, L L M's, V L L M's kind of made their way into the into the Waymo stack?

• 2:34Speaker 01

  The the big thing that L L M's and V L L M's can bring to the table. The first thing is really what we call world knowledge, essentially, is the semantic understanding of the world around it. The kind of things that are, you know, very obvious to you and I, what, you know, what a police car looks like, what an emergency vehicle looks like, that we, the the data that we collect from driving may not have experienced, right? Imagine we go into a a new city, their police cars look a little different. Until we experience that, we wouldn't have that data.

• 3:15Speaker 01

  Those models know what a general police car looks like, what a general emergency vehicle looks like. And so that world knowledge, and there are many examples of world knowledge, I'm just picking vehicles as an example, but just in learning about the what an accident scenes look like, right? We don't see a lot of those in our data, but if you ask a large multimodal model like Gemini or G P T, you know, you give them a picture of a scene with an accident, they will be able to recognize that this is what's happening and that's the semantic context that's relevant to this. So bringing all that knowledge from essentially the web into the driver to be able to give

• 3:59Speaker 01

  it more capabilities is essentially what we're seeking out of this.

• 4:05Speaker 01

  In addition to just scaling up, right? So those models are very large. They're pre-trained on a lot of visual data. They're pre-trained on a lot of text data that enhances their reasoning capabilities. You can just leverage that scale, you know, just bigger is always better essentially, and use that as a vehicle to build models that are just better.

• 4:30Jacob Effron?

  Conversely, what parts of the overall problem of self-driving are these models not actually that helpful for?

• 4:36Speaker 01

  You can distill every aspect of the driving problem into essentially a machine learning model. There are things that you want to also build on top of it on the outside of an AI model. Typically, you know, anything that has to do with strict contracts on safety, on regulatory constraints and things like this. You want to be able to express those in a very explicit way, not in an indirect implicit way, so that you can convince yourself that once the car drives and the AI model basically proposes a plan of driving,

• 5:20Speaker 01

  you can verify that this plan meets your requirements in terms of safety, compliance, and just general or just general be well behaved. Doing that from the outside gives you a very strong sort of framework for guaranteeing that the car behaves in a way that is reasonable at all times, while still enabling you to use the power of AI to come up with a plan, come up with a good idea in terms of what is the right strategy for driving.

• 5:55Jacob Effron?

  Yeah, super interesting. It's kind of putting, yeah, basically that checking layer, the guardrails around the output of the reasoning models. Makes a ton of sense. I guess you had a pretty storied run over at Google in the robotics world, you know, involved in, I think, a lot of the seminal work in the space. And then you chose to make this decision over to Waymo. What drove that transition? And, you know, maybe talk a little bit about what got you so excited?

• 6:19Speaker 01

  Yeah, it was. It was really accidental, pun intended, in the sense I got into an accident last year. Nothing bad, but it put me out of commission for a few months. And during that time, I had to, you know, not work, but I had to go to lots of physical therapy appointments. And since I live in the city and I couldn't drive at the time, I basically took Waymos everywhere. And I really found the product to be amazing.

• 6:48Jacob Effron?

  It's a pretty magical experience.

• 6:49Speaker 01

  Yeah, it's completely magical. It was the first time that I had this rapport with essentially an AI system that I thought, you know, touched everyone. That was so easy to use, no fancy UI, you know, it felt like the kind of AI that really applied to everybody. And that felt really exciting and magical to me. And then I got back to work after a couple of months of being out. And I also have a fantastic team back at DeepMind, who had basically picked up the reins while I was away and was doing just fine on their own.

• 7:32Speaker 01

  But I took the opportunity to do something different, knowing that the team was in good hands.

• 7:39Jacob Effron?

  I'm curious if you could talk about like maybe the similarities and differences with the robotics models work you were working. I mean, at the highest level, it feels like, you know, a lot of these problems are perception, planning and actuation. I'm sure in the details, there are lots of things that are bespoke to the self-driving world that may be different than some of the robotics work you did.

• 7:53Speaker 01

  Yeah, at the core, the problem is the same, in the sense that a, you know, an autonomous car is a robot, right? It has the same kind of inputs and outputs. You just have sensors, cameras as the inputs, and you have actuation. So, you know, turning the wheel and pressing on the gas pedal as the output. It's very similar to a manipulation robot that needs to see from its camera sensors and act with its, with arms and hands and fingers. What really... The big difference is the operational domain.

• 8:31Jacob Effron?

  Yeah.

• 8:32Speaker 01

  Um, as long as I was working on robotics and AI, I was in a more of a research environment where we were still chasing essentially the nominal behavior. We're still chasing, how do we get a robot to even do the thing we want it to do? Like, pick up an object and or tie a shoelace or, you know, make go make coffee or just.

• 8:58Jacob Effron?

  It's always go make coffee. I feel like everyone loves that.

• 9:00Speaker 01

  Go make coffee is great. It's amazing. Uh, it's a really good, uh, sort of a demo of, uh, of what the where the capabilities can be and are not quite yet. And so the, the, just chasing that was really the, the, the name of the game. Um, in the autonomous driving space, we know how to drive. We have a nominal system that works reasonably well and that, uh, you know, that is at the level of safety and performance and quality that, uh, you know, gets us into commercial products already. All the challenges here now are about scaling.

• 9:41Jacob Effron?

  How would you kind of classify the kind of state of autonomous vehicles today in terms of, you know, what we can do, what we can't do? And what are those kind of like challenges on the horizon or things that still need to be solved?

• 9:51Speaker 01

  There aren't that many big blockers in the sense that, for example, we don't drive, uh, we don't drive in snow right now, right? Uh, mostly due to lack of attention. So far, it was not the most pressing thing to work on, uh, in the areas that we were in. But this is something that requires more, uh, developing new, more capabilities. Most of the problems like this, like, uh, you know, fog or even driving on highways. We started driving on highways and testing there. I've, um, have been knocked down over time. And now the big challenges really are about, um, essentially scaling.

• 10:39Speaker 01

  All the issues that have to do with what happens when you drive millions of miles. The the long tail of problems that you have to deal with at that scale kind of dominates the equation of what you have, what problems you have to solve, right? You can imagine if you as a driver, you experience this thing maybe once in your lifetime, we will experience that thing pretty much every week or maybe every month, right? And so all of the things that are exceptional and weird and difficult

• 11:25Speaker 01

  are essentially becoming common occurrences for us and are kind of putting pressure on our scaling. So solving for this long tail is really what we're focused on and what we're hoping that, you know, AI and sort of large model capabilities can help us accelerate.

• 11:44Jacob Effron?

  And how do you go about solving that? I mean, obviously, I imagine, you know, the hard part is that there's not a massive amount of data to go get. And I assume when you're doing, you know, freeways or snow or fog, you can gather actually a decent amount of data. But for some of these long tail problems, is it mostly, you know, using simulation?

• 11:58Speaker 01

  Yeah, we do a lot of simulation and a lot of synthesizing scenarios that correspond to problems that we knew could happen, that we may never have observed in the world, but we know it's it's an eventuality. And so we we synthesize a lot of scenarios like this and validate our models against those. And then we try also to do a lot of looking at, you know,

• 12:29Speaker 01

  a lot of the situations in which, you know, nothing really bad happened. But there was a risk that something different could have happened. And so we just modify scenarios to make them worse. Right? You can imagine having you've experienced a thing in the real world. And then you dial a dial that says, you know, now make all the driver drunk drivers. Or make the drivers, you know, actively adversarial against you. Right? And how do you make the things more difficult for the car? So that you can learn and be more reactive and understand better what could happen in the worst case scenario?

• 13:11Jacob Effron?

  Are there areas of like, you know, research or technical breakthroughs? That still you still feel like need to happen on the autonomous vehicle side? Or is a lot of what you're describing just mostly like, you know, there's going to be this a large amount of edge cases and it's just identifying them?

• 13:24Speaker 01

  There is one technical advance that I think could completely yet again change the landscape for autonomous driving. It would be having reliable physically realistic world models. In order to be able to simulate the real world as it looks to you and me with physical realism, with a very accurate rendering of a scene. In order to cross to that, I think there is a growing body of work around world models that could really unlock that.

• 14:06Speaker 01

  The simple world model is video prediction models, right? The things like Sora or Vio, that's kind of the proto world model. In some sense, you can take a scene or an image and then push play. And, you know, it unrolls into a possible future.

• 14:25Jacob Effron?

  In a way, that seems reasonable with the physics of the world.

• 14:26Speaker 01

  That seems to be reasonable. Once you take that and you make it controllable, you make it physically realistic. While being rich and being very, very plausible from both the visuals and the way things behave in the scene. I think having essentially this. Digital twin of the world for autonomous driving could really sort of change the game. The question is, so there is work in that direction in research.

• 15:04Speaker 01

  I think there is a potential for that to become useful to some degree to autonomous driving today. But there is still a gap in the sense that, you know, it would be most useful to deal with the long tail of the problems. Those models are not very good at long tail right now.

• 15:22Jacob Effron?

  What have you seen around world model building? Like, how do folks build world models? It's interestingly come up, you know, mostly in the video context to date.

• 15:29Speaker 01

  It's come up in a video context first because you can build a world model that is not particularly physically realistic, but still look really good.

• 15:38Jacob Effron?

  Yeah, and the stakes are a lot lower.

• 15:40Speaker 01

  Yeah, so.

• 15:40Jacob Effron?

  Something happens in the video world, it's just fine if your trajectory was slightly off.

• 15:44Speaker 01

  And that's what's happened in image generation models. In the first place, you ended up trying to chase, making things look very realistic first, while not trying to make them too controllable. And but now, because a lot more people are trying to start using them for authoring, for creating content, there is a lot more of a trend to trying to make them actually controllable, useful, and have very tight knobs that you can actually use, and to tune the realism, to tune the style, but also to tune the geometry, the content, and everything like this. So a lot of work is going in that direction very naturally.

• 16:29Speaker 01

  It was very natural that first, you know, the wow videos and the magical creations would come out first. Now, it's the big challenge is turning that into a usable, a usable tool that you can use for functional uses, not just pretty pictures.

• 16:49Jacob Effron?

  What is like the limiting factor today?

• 16:51Speaker 01

  There is a deep question of causality at the heart of those world models, right? Right now, you can, by just learning correlation between data, you can generate very plausible videos because they have kind of a plausible sequence structure. You know, the objects don't disappear, they don't appear.

• 17:16Speaker 01

  People walk in the streets and that seems really reasonable. As soon as you want to make things controllable, you need to make sure your model understands causality.

• 17:26Jacob Effron?

  Yeah.

• 17:26Speaker 01

  And that, you know, you input, this output derives from this change to the input or discount to factual and things like that. And that's very hard. That's something that in general in machine learning,

• 17:42Speaker 01

  injecting causality into models we've always struggled with. And now we have to really solve that if we want to have a very plausible world model.

• 17:53Jacob Effron?

  Yeah. How do you then on the research side at Waymo think about, obviously, you know, having better world models would be really important for the product. There's also tons of people around the world working on it. So I could see two schools of thought, one of which is like, hey, there's lots of awesome, promising research directions. Let's, you know, stay on top of those. And as they become better and better, like it's something we could put in the product. The other is, you know, we should have our own folks working on this and pushing the frontier.

• 18:13Speaker 01

  It's always a trade off between, you know, pushing the state of the art by ourselves or leveraging what's been done outside. In for many, many tasks or many aspects of the problem. We're lucky that academia and other institutions are interested as well in the problem. And so you get a lot of leverage out of working with them. The AV problem has a lot of bespoke problems that, you know, people are not necessarily chasing in academia. And so we try to. Steer the conversation there.

• 18:58Speaker 01

  Waymo has put out, for example, the Waymo Open Data Sets. That is like the standard for AV research right now. And it was really designed in such a way that we helped steer the conversation and the research focus on the problems that we thought were relevant. And we have a pretty significant research effort that is really at the state of the art of AV right now. So we're more in a position today where we're at the forefront of AV. So we have to build the next thing.

• 19:42Speaker 01

  Like, we can't really necessarily rely on the rest of the community to build it for us and us to sort of inherit it.

• 19:48Jacob Effron?

  One thing I've always been curious about is just like what's required when you guys enter a new city?

• 19:52Speaker 01

  The models are remarkably robust and very portable across different cities. There's always things that come up here and there, but in a general, in general, we found that they were remarkably robust.

• 20:12Jacob Effron?

  What's an example of something that's like different city to city?

• 20:15Speaker 01

  Well, like, as I mentioned, sometimes the emergency vehicles look different. And it's not that we don't necessarily, the model doesn't recognize them. It's more that we want to be sure that we don't miss them, right? That those differences are well modeled by our model. So a lot of the time is spent on evaluation. It's not necessarily adapting the model, training it to do something. It's more convincing ourselves, convincing regulators, convincing the community that we've done our homework. And that we validated that indeed the model is robust in the communities that we're entering.

• 20:56Speaker 01

  There is a lot of just logistics to entering a new city, right? Sort of setting up depot and that's why we have, for example, this partnership with Uber to try and help scale up our operations and make the deployment faster. But a lot of it is really making sure we have the hearts and minds of the local community and we do things in a way that is respectful and supported by the communities that we work with. Because really, it's at the end of the day, it's really about trust.

• 21:37Speaker 01

  And trust is kind of a lot more than just technology in that context.

• 21:43Jacob Effron?

  There have been a lot of questions around kind of the rich sensor suite that Waymo uses today and kind of the extent to which that will be necessary as models improve. I'm curious like how you think about that.

• 21:52Speaker 01

  We use three different kinds of sensors, red cameras, lighters and radars primarily. And they're remarkably complementary. That's one of the nice features of this kind of sensor suite. They have strengths and weaknesses that complement each other extremely well. And because they're also kind of orthogonal to each other, we can use that diversity to check that, you know, what the camera sees is correlates with what the lighter sees. And if it disagrees, you know, we want to look into that more deeply.

• 22:37Speaker 01

  It's been, there's been kind of two schools of thoughts on this, right? A lot of the AV companies or companies that did driving assistance systems have been starting from kind of an L2 level driving, right? And have been trying to climb their way up to L4. And the economic constraints of L2 driving are very different from L4 driving. When you have a fleet of cars that basically earns money for every mile driven, you can afford to have more sensorization on the car compared to if you have a car

• 23:21Speaker 01

  that is owned by an individual, where you have to really cut down the price. So as a result of that kind of different business strategy, many companies have gone the route of let's start simple and cheap and try to climb our way up in terms of low complexity of the system. Waymo, at some point in its history, has decided, no, no, no, no. We're going to go the other way around. We're going to possibly over-sensorize and then see, solve the hard problem. Right? There is often the, you know, sometimes solving the 10x problem is easier than solving the 1x problem because you are the right North Star.

• 24:03Speaker 01

  So solve that first and then see which areas, which avenues do you have to reduce the costs and simplify. And but now we have the data, right? We have the data to inform those kinds of decisions because we've solved the harder problem and we understand what matters and what doesn't. So it'll be interesting over time how those technology stacks evolve. I think we have a good sort of way, a good trajectory with our next generation of cars to make the overall package cheaper and simpler. And how far we can take it is going to be interesting to see.

• 24:47Jacob Effron?

  You know, we've talked about some of the factors, you know, how good like world models get, other things like what are some of the other, you know, research areas or things we'll know in the next few years that might determine, you know, the ultimate question of, you know, four years from now, how many sensors have to be on these things?

• 25:00Speaker 01

  The sensor story is not just a performance problem, right? It's also a niche, it's also about redundancy. And the need for redundancy, I don't think is going to go away. In some ways, and it's not clear that, you know, the sensor suite will not evolve. But I think this kind of feature of having very different sensors that give you very different information and provide this complementary signal is a pretty powerful one. The... A lot of the argument around just using cameras, for example, has been about, you

• 25:40Speaker 01

  know, I can drive my car, I have eyes, I don't have a fancy LiDAR, so I can, you know, there is actual proof points that people can drive with just their eyes and you don't need anything more. The problem with that is that I'm increasingly convinced that the bar for L4 driving is not human level. Yeah. It's above human level, right? And how much of that is, and we've seen from, you know, the, our safety reports, you know, we are today in a place where we are safer than the average driver and we have fewer

• 26:20Speaker 01

  collisions and, you know, fewer reported injuries by some significant margin. That's essentially the driver being superhuman in some way. And I think that's actually a business requirement for successful for driving. So can we actually, you know, so the, will that bar ever change? I don't think so. Yeah. I think we'll be in a place where we need, and so the question is, can we get to that level of better than human driving with a simpler sensor suite?

• 27:01Speaker 01

  And that will be sort of, that will be something we'll experience in the next few years.

• 27:06Jacob Effron?

  Yeah, it's incredible to see some of like the superhuman displays of Waymo driving. I feel like there was this one viral video of like someone on a scooter falling right in front of a Waymo car and you're like, if that wasn't a Waymo driver, you know, that would have ended really badly and... But to your point, once you've seen that, it's hard to then as a society be like, "Well, we don't actually need that. Like, we could do that, but we don't necessarily need that. We need, you know, we take the the human level."

• 27:27Speaker 01

  Yeah. It's a lot easier to have this conversation based on data, is what I'm saying, instead of sort of litigating based on expectations. I think we now have the data and we are going to be able to figure out exactly how performance correlates with the sensor capabilities.

• 27:44Jacob Effron?

  What are like the the milestones that matter from here, like in the in the autonomous vehicle world? Like, what are what are you kind of, what do you think the next major one is?

• 27:53Speaker 01

  So it's it's funny, the the milestone I have in mind right now is this year marks the 30th anniversary of the first transcontinental autonomous ride.

• 28:07Jacob Effron?

  Yeah, when everyone thought we were right there.

• 28:09Speaker 01

  Yeah. So in 1995 was the first drive across the country. And and they, I think they had like more than 99% autonomy. And they drove across the US at more than 60 miles per hour on average. Right? So you could imagine that based on that data, people were like, "Yeah, we're done." Like, we, you know, it's just a matter of, you know, tying a bow around it and we'll we'll have autonomous driving. And it took 30 years to get to the point that we now have a commercial deployment. So I think, you know, thinking in terms of milestones and timeline, it's very hard to predict where things are at.

• 28:52Speaker 01

  The the place that, you know, we have the technology validation, we know that things work well in Phoenix, in San Francisco, and we have kind of the the user validation. Like, people love it. And that was not a given to me before I experienced it myself, that this was a product that people would actually just love and be attracted to, right? So the only thing that stands in, there's not really anything that really stands in the way of of of this becoming a big product. And now it's really just the scaling is going

• 29:33Speaker 01

  to be to be the thing. So I think the next milestones you'll see are going to be all about Expansion.

• 29:40Jacob Effron?

  Yeah.

• 29:41Speaker 01

  Right. And and sort of proving out in various geographies where I'm excited, for example, about where we started driving in Tokyo, collecting data. Right. That's going to be our first international experiment and the first time we drive left side of the road. So seeing that is going to be an interesting, interesting program.

• 30:10Jacob Effron?

  Bringing up the 1995, you know, like, drive across the country, I actually think is a really interesting lens to transition to kind of the broader robotics space where you spent a large part of your career over at Google and DeepMind. You know, I think a lot of times there's kind of these two competing forces when I talk to people in the broader AI world about robotics, where people feel like we're on the cusp of these really exciting breakthroughs and, you know, a lot has really changed in the last, you know, three, four years. And then at the same time, everyone loves to bring up the example of the 1995 drive and be like, well, it took 30 years to actually get this stuff into a commercial product that people can use and experience the light from. And I wonder how you think about those like two competing forces and how you'd kind of characterize where we are in that space today.

• 30:50Speaker 01

  Yeah, it's a really good question. The, like I said before, in the robotics space, we're still chasing the nominal use case, right? We're still chasing, how do we get a generalist robot to do anything we want, right? That's kind of the the problem statements that everybody is going after. And we don't have the 1995 ride yet, right? We have some examples of that, but I don't think we have a convincing generalist system quite yet.

• 31:30Speaker 01

  The, I wouldn't be surprised if we get that in the next couple of years, that we get that valid proof point in the next couple of years. And because the progress has been really rapid, there are still some phenomenal technology questions that need to be answered. We know how to generalize based on different visual inputs. We're not so good at generalizing motion. Like, a lot of the demos you see of robots doing stuff, they do the one thing.

• 32:03Jacob Effron?

  Yeah.

• 32:03Speaker 01

  Right? It might be on You know, different colored cups of coffee. If you're making coffee, it might be with objects that are randomized in space, but nothing that's really generalizing from a skills perspective that is enabling robots to do things that are very different. You may not need that to have a commercial success. That's totally fair. You could have a robot that completely optimized for one use case and one use case only, but does it very well, cheaply and dexterously. And that might be enough to be able to have a business behind it.

• 32:44Speaker 01

  But if you're thinking of the general vision of the AI robots that can, you know, make your coffee and tidy up your room and, you know, pick up your clothes. Before we're there, there is still some breakthroughs that are going to be needed.

• 33:03Jacob Effron?

  It feels like there's been kind of a wave of applying these, you know, LMs to the robotics world, whether it's, you know, VLMs on the perception side or LMs in planning or even like CodeLM on the actuation side. I think you said originally, like, it was kind of a surprise to you, like, how well these things work. I'd love to just unpack that, like, what, you know, what did you initially think might happen and how did it surprise you?

• 33:24Speaker 01

  The big surprise to me was kind of the fact that we could so quickly go from having a chatbot describe what it means to you to make coffee, right? To turning that into a plan that you can use on the robot, right? So what was really hard to build within the robotic environment with this common sense knowledge of, you know, what it means to make a coffee, what it means to, you

• 34:06Speaker 01

  know, that you have, if you have a cup, it goes on the table, it doesn't go on the floor. That if you look for a microwave oven, it might be in the kitchen, right? These kind of things that you and I know, that we don't even think about, that are a part of our world, that are... Is just basic knowledge of every day. And we didn't have access to that in robotics or in AI at large for a very long time. And LLMs really sort of brought that together. And the fact that we could take that sort of high level knowledge and quickly turn it into something that was actionable by robots, even though, you know, language is fuzzy,

• 34:50Speaker 01

  language is not very precise, but it was just precise enough to describe things that we could basically build language condition policies that would actually do things effectively and very high performance. So that was the first kind of aha moment. Oh, wow, actually having the language as the backbone of robotics is not completely crazy. And then kind of it evolved into, wait a minute, robot actions are just a different language. It's not that different from English or Chinese. It's just a different language that is not expressed in words. It's expressed in body actions.

• 35:32Speaker 01

  And if you think of robotics under that lens, suddenly you can leverage all the multimodal, large models, multilingual, and just see the robot actions as just a different dialect of expressing yourself in the world. And all the machinery there just works. And so that that was another sort of aha moment there where suddenly everything kind of started gelling together.

• 36:00Jacob Effron?

  Can you talk a little bit about like this question about, you know, you talked about we could have task specific models, we could have a generalizable model. Obviously with some of your work with RTX and other things, you've looked, you've worked on kind of like cross embodiment and this generalized robotics problem. Do you think it's likely that we find ourselves moving toward a general robotics model over the next three, four years? Or is it feel like something where actually, you know, the most immediate value will be provided through much more like specialized focused ones?

• 36:25Speaker 01

  I think it's going to be both in the sense that, you know, you want a generalized teacher. You want to generalize sort of backbone model that is easily retargetable and that can be optimized for a single task. It's a little bit the same paradigm as in large language models, where you have instruction tuning that enables you to develop very generic capabilities that are kind of related to the task that you might want to use your LLM for at the end of the day, but they're not necessarily the same task exactly. But having this instruction tune model enables you to quickly adapt your LLM

• 37:08Speaker 01

  to whatever the task you have in mind, either through prompting or through fine tuning or different strategies. But so I think we're going to end up in the same position in robotics, where you want to build a generalized, you know, robot model and then have the right tools to target it very specifically to specific tasks, even possibly on the fly, right? If you could do that, you know, prompting style at test time as opposed to doing at training time, then you've solved essentially all the hard problems.

• 37:52Jacob Effron?

  There's a lot of different efforts right now going on about people trying to build these kind of powerful, generalized robotic models today. I'm curious like how you'd broadly segment the different approaches that folks are taking and, you know, any thoughts on the relative efficacy of some of these things?

• 38:07Speaker 01

  There is a lot of push right now. There are kind of two approaches. One way to segment the approaches is some people have started like very hardware centric. I want to build the best, most capable humanoid robot in the world. Right. And then, once I have these degrees of freedom and those capabilities, I will be able to accomplish all the tasks I need. Versus people who have started kind of software first. Let's build the intelligence and trust that, once you have a general enough intelligence model,

• 38:48Speaker 01

  you'll be able to re-target it to a new platform relatively easily. The work we did with RTX kind of gave me some confidence that this path of going software first and building a very generalized robot model that can be easily re-targeted is a way to make progress fast. Because a lot of these, a lot of the problems in robotics still are about data. It's still about how do you acquire as much quality data as possible, as quickly as possible? And if you think of it, putting in the critical path of that data acquisition

• 39:30Speaker 01

  of very expensive and wobbly robots that is very hard to operationalize, that's a really tall order. It may make sense if you have a lot of money to throw at the problem, but there are really limits to the scalability of that approach. So I'm, I'm, I think, mostly, I'm also saying this because we haven't solved the problem. Right. It's not about hill climbing on a problem that we have kind of solved and we're trying to get better at. Right. The fundamental problem of robotic manipulation in high degrees of freedom is not solved.

• 40:09Speaker 01

  And so optimizing for the data collection and speed of execution is probably the most important thing to do right now.

• 40:19Jacob Effron?

  It also seems like there's kind of a debate between, you know, using kind of purely or as much simulation versus, you know, teleoperated data. Obviously, if you could use just simulation data, it would be, certainly be easier. But I think it seems like it works quite well in the locomotion context, maybe not as well in manipulation.

• 40:35Speaker 01

  Yeah, so we've really struggled with this for a long time. And in the locomotion context and navigation context, using simulation has been wonderful. The symmetry gap was not large enough to be problematic. In the manipulation space, we always struggled with getting the kind of diversity of experience and quality of the contact and performance from a simulation. Because there is a cost to simulation, right? It's not necessarily a monetary cost of, you know, buying lots of robots and operationalizing them.

• 41:20Speaker 01

  It's more of a cost of setting up the simulation environment, making it diverse, making it representative, tuning the physics so that the physics are realistic. And that, the amount of work you have to do to get that right in the context of manipulation is really, really high. And so my experience thus far has been that it was easier or a faster path if you could scale up your physical operations to collect lots of data in the real world and not have to deal with this simulation-to-reality gap versus doing the simulation.

• 42:02Speaker 01

  That said, I also want to say that we also took that path because we could.

• 42:10Speaker 01

  And because, you know, as a research organization, you also want to take the path less trodden in some way, right? A lot of other research labs were a lot more invested than us in simulation, had a vested interest in investing simulations work, for example. And it's also a lot more accessible to people in academia to do a lot of work in simulation. So we kind of explored more the, yeah, let's scale the real-world robotics and see what that part of the space can bring to the table. But I still think it's actually, it's still the better path for manipulation.

• 42:52Speaker 01

  To date.

• 42:53Jacob Effron?

  In hearing you talk, it becomes so clear that some kind of flywheel to acquire data and get data at scale is going to be crucial in the same way that you're experiencing at Waymo now, and you have all these edge cases that you get at the millions of miles you're driving. I know you thought a lot about the way that humans, we'll tell me, interact with these robotic interfaces, especially early on. Any early thoughts on what are effective ways to do that and what might end up being effective ways to get that flywheel going on data acquisition in the broader robotics world?

• 43:23Speaker 01

  Yeah, it's a really good question. I wish our colleagues in the HRI world would spend more time thinking about this, the HRI for data acquisition, because I think it's a very rich space, and it's really the big bottleneck for a lot of robot learning today. I made that pitch to the HRI conference a few months ago. I think there is really something there that we could do very interesting research on. There are different strategies right now that people are using, other kinesthetic teaching, puppeteering, teleoperation with

• 44:05Speaker 01

  gloves, or trying to synthesize behaviors in simulation. It's really, I think, a very empirical question of how do you maximize the throughput? The one thing that I would love to see take off is third-party imitation, being able to learn from watching videos of people doing stuff. But that, right now, I don't think anybody has really cracked. And I think it goes back to the same question of the world model that we were talking about

• 44:43Speaker 01

  before. It is about inferring causality and how if I do this, then that happens from observation, being able to model that and turn that into a useful learning signal for robots to learn how to behave. So, one thing on the data front that has been a big accelerator is the fact that we now have the big multimodal models and that

• 45:14Speaker 01

  Transferring visual information from those multimodal models to robots actually does work.

• 45:21Jacob Effron?

  Yeah.

• 45:21Speaker 01

  Right? We don't, we had the example of showing a robot that is moving a Coke can to a picture of Taylor Swift. We've never taught the robot who Taylor Swift was. We've never had to sort of act on, you know, the entity Taylor Swift or show the robot any data about Taylor Swift. That knowledge was part of the big multimodal model. So that solves one of the big model necks to data acquisition. Now, you really have to think about how do you acquire the right kind of data that is the motion data, about the actuation and the actual sort of physical skills. And I don't think, I think the jury's still out about what's the right way of doing that.

• 46:06Jacob Effron?

  Yeah. I mean, do you think like to get causality into these models, like it requires a new architecture or is like, you know, how do you think we ultimately, I'm sure there's a bunch of research paths being tried toward it.

• 46:15Speaker 01

  It's, it's a great question. I think it may just be proper data engineering because thus far, at least in the language model space, we've seen that you can elicit some form of causal reasoning and, you know, chain of thought and things like that without having to engineer it into the model. But, but that the, having the right data was very important and the right inductive biases in, in some ways. So it's possible that we'll, we'll, we'll get there without any major infrastructure or theory changes.

• 47:00Speaker 01

  But yeah, I don't know. It's a, it's a, you know, my hope is that we don't, it's, it's really a matter of scaling and data curation.

• 47:09Jacob Effron?

  As you think about kind of these unanswered questions right now in robotics and the ones that we'll know the answers to in the next two, three years, like, what do you feel like, I mean, it sounds like these, this question of causality and getting that into models is certainly one of them. What are the set of other questions that you feel like will, will be the key determinants of where the space goes over the next few years?

• 47:25Speaker 01

  Can we generalize motion, right? Can we generalize in the space of actions in the same way that we generalize in the space of perception? I think it's a key question that, you know, we cannot afford not to have an answer to. Um, I think the, one of the big questions is going to be,

• 47:52Speaker 01

  what are the differences, if there are differences, between robotics and all of the other areas of AI? Right now, the hypothesis that, you know, robotics is just another way, another, another language of AI, um, seems to hold. Uh, at what level does it break? Like, what, what, what, what is the, the one thing that we have to say, "Okay, now that's very different, and we have to invent some new techniques for this"? We thought that we would need, for example, to invent new techniques for, uh, motion generation. And it turns out the diffusion models, the same that are being used in video generation,

• 48:36Speaker 01

  were great for this use case. And right now are kind of state of the art for this kind of thing. So it was another area where we thought we, we were special in sense, and we were not. And I wonder if there is other areas like this, where, um, you need to have this specialization.

• 48:55Jacob Effron?

  And I feel like also, you know, it'd be super interesting to see if we find early signs of, of any sorts of scaling laws towards some of these things.

• 49:01Speaker 01

  Yeah, so, uh, you know, back, uh, on the, at, at WEMO, we've been looking a lot at scaling laws for large models, for behavior and for, for perception. And, um, our, we're seeing that the, you know, the same laws apply, uh, to some degree with different constants, uh, right? So you don't, you know, a, a autonomous driving model doesn't behave with the same, uh, characteristics as a L L M in terms of scale and everything. But you see the same kind of linear, uh, or log linear, uh, growth in terms of data and, and size and, and things like this. So there, so far, all points in the direction of...

• 49:48Speaker 01

  It's similar. Conceptually, it's the same. But we'll see if it hits the limits.

• 49:54Jacob Effron?

  Before you were in the robotics world, you were in the computer vision world, right? And I think certainly like a lot of VLMs, you know, the advances in those are obviously powering a lot of cool products and a big part of robotics. But it feels like in general, there's kind of all these classic computer vision use cases from back in the day. And it's not like, it doesn't feel like there's been any kind of crazy inflection in the usage of some of those. We haven't had like a chat GPT moment almost in that space. Like, I guess I'm curious, like what, you know, any reflections on that?

• 50:22Speaker 01

  Yeah, it's interesting, right? The computer vision has kind of been largely driven by, you know, a, the problem or the

• 50:35Speaker 01

  the solution, not the problem, right? In the sense that there hasn't been nearly as many applications of computer vision

• 50:46Speaker 01

  as, you know, the level of interest in the technology may be warranted.

• 50:53Speaker 01

  But I think that's really because vision is only useful when you're trying to act on the world.

• 51:00Jacob Effron?

  Yeah.

• 51:00Speaker 01

  Right? And the fact that we didn't have physical actors meant that just using vision as, you know, a tool to observe sort of limited the the scope of applications.

• 51:19Speaker 01

  Also, the the technology has been working extremely well in closed set environments. If you're trying to use computer vision at large in an open set environment where you're trying to just parse everything in the world, the level of performance is good on academic benchmarks. In the real world, it's not nearly as good, right? So there is a either a performance bar for sort of open set vision problems. For closed set problems, it's really that the applications of that are very specific. And I think can be unlocked really with robotics. Like, I think that's really the the best use case.

• 51:57Jacob Effron?

  You've obviously been on the cutting edge of a lot of AI research for a while. I'm curious, what's like one thing you've changed your mind on in the last year?

• 52:03Speaker 01

  I mean, what has been? Really fascinating to me is to observe the reasoning capabilities and how they evolve. And how this, you know, funny idea of chain of thought thinking that was mostly kind of a that started as a funny realization in some ways. And haha, this kind of weird prompting does something, isn't that funny? Actually sort of changed the way people think about reasoning and the path to getting a sort of performance in reasoning. I'll give you an example.

• 52:43Speaker 01

  I, I, so I like to write little stories and fiction as a hobby. And I had this idea, maybe like 10 years ago, for a topic in a premise for a science fiction story. Okay. And I sat on it for 10 years because I couldn't figure out if the physics worked. Like, I couldn't figure out if the premise behind the idea, the, the, the, you know, the, the physics it was resting on were sound. And I didn't know who to ask. I didn't know where to find the answers to that question. Couple of weeks ago, I put it in Gemini

• 53:28Speaker 01

  and just used the Gemini deep research to, you know, ask simply the question. It gave me a three-page summary of all the equation that were relevant. And the whole answer to my question was there in like five minutes. And so to me, that, that's something where suddenly you have access to the best, you know, physics knowledge or the best legal knowledge in a way that is at your fingertip. And I have been asking myself, what are the millions of things like? Should be asking Gemini right now that I'm not even thinking I should be asking because I don't have this mental model yet that this is something that is accessible to me.

• 54:12Jacob Effron?

  Yeah.

• 54:14Speaker 01

  I completely underestimated, I think, the the the power of having that level of accessibility and what it may mean for everyday use. And it's not, you know, just a technology, it's more, you know, are there many use cases for those things that we are not even thinking about? Or we haven't internalized, we haven't made a habit of it.

• 54:37Jacob Effron?

  Totally.

• 54:37Speaker 01

  Right?

• 54:38Jacob Effron?

  Research is a super powerful product and I guess, did the physics work in your story?

• 54:43Speaker 01

  Yes. So I've made it now.

• 54:45Jacob Effron?

  You're gonna go forward with it? The good, good, good, good writing assistant tool now. It's super interesting, obviously, I feel like the, you know, this, this wave of, of test time compute has been incredibly interesting and powerful. Obviously, one of the biggest question I feel like right now in the space is just, you know, it obviously works really well and easily verifiable domains, you know, how far does it, you know, does it get us to superhuman performance on, on coding and, and math with like zero impact on, on other things? Or how do you kind of think about that right now? You know, the, the extent to which it will be broadly applicable?

• 55:17Speaker 01

  Yeah, it's a good question. I think the, the, the, the space of problems that are, you know, hard to generate for, but easy to verify is pretty broad, right? There is a lot of things that have that shape where coming up with hypotheses or coming up with a plausible solution is hard, but once you have that solution, you can verify it relatively easily or you can, you don't need to necessarily verify it exactly. You can convince yourself that it's right or that it's in the vicinity of right.

• 55:54Speaker 01

  And so that's, in general, we've seen that, that, you know, generative versus discriminative in reinforcement learning terms, you have the actor versus critic model pop up everywhere, right, in a lot of places.

• 56:14Speaker 01

  You turn. The hard problem of generating a plausible answer into the other hard problem, possibly of verifying that answer. But it's a lot easier to be in that other world of verifying an answer, because you have all the ingredients there. You don't have to sort of imagine them. So I think it's going to evolve in such a way that it's not going to be just math. It's not going to be just coding. You could imagine in the autonomous driving case, we can verify that a plan is meeting all our requirements

• 56:53Speaker 01

  a lot more easily than actually generating the plan in the first place, because we have hard constraints that we can apply to the problem. So I think there's going to be a ton of different applications that will be able to leverage this.

• 57:07Jacob Effron?

  What other kind of areas that maybe are under-talked about today, do you think these models will actually be quite effective in?

• 57:12Speaker 01

  With reasoning in general. I think anything that is, you know, multi-step and requires essentially credit attribution is at stake. I think, to me, it's RL done right. I always had this love-hate relationship with RL in the sense that in the early days of robotics, a lot of people were banking on RL being the ultimate solution to everything, mostly buoyed by, you know, the success of AlphaGo and everything. And we spent years and years just focusing on RL

• 57:53Speaker 01

  and did not make a ton of progress as a result, because we were essentially, I think, as a community, we were really focused on trying to learn everything from scratch using RL, when it, you know, in hindsight, you know, there's very good ways of just using supervised learning to bootstrap yourself. And maybe you have RL as the little, you know, fine-tuning on top. So I think that paradigm is here to stay in a sense like bootstrap yourself with a large model, do a lot of supervised learning, and then use RL as the way to...

• 58:34Speaker 01

  Make that model even more of an expert at some specific reasoning things. That's going to be, you know, that has staying power, it feels. It's the right way of thinking about it.

• 58:48Jacob Effron?

  Kind of similar to the question I asked you about in robotics, like over the next few years, I'm sure we'll flip over a lot of cards about how the broader L M space is going to play out. What like two to three questions are most top of mind for you right now that we'll learn more on in the next 12, 24 months?

• 59:03Speaker 01

  I mean, I want to see this world model kind of thrusts that I think a lot of people are starting to seriously look into where that lands. I think it's having controllable video generation, controllable world generation, essentially having a purely generative video games, for example, will teach us a lot about if we can make that work, I think we, the current architectures of the large multimodal models that we have are going to be here to stay.

• 59:45Speaker 01

  If we cannot turn those models into good world models, then maybe there is another sort of leap that needs to be done in terms of architectures and performance. So I'm excited about that direction. I think there's a lot of important work that can be done there. Once you have models that can essentially act as digital twins of anything, you can turn every computer into a generative model, right?

• 1:00:24Speaker 01

  And so it might be that it will be completely impractical from a compute perspective and require sort of massive investments. I think that's why you see a lot of massive investments going into compute as well right now, is that a lot of people start seeing that the things that we really want to do as the next step are going to be yet another notch up in terms of the demands in computing.

• 1:00:50Jacob Effron?

  Well, it's been a fascinating conversation. We always like to end our interviews with a quick fire round where we get your thoughts on some standard set of questions. And so maybe to kick it off, what's one thing that's overhyped and one thing that's underhyped in the AI world today?

• 1:01:03Speaker 01

  Overhyped, underhyped. I'm struggling here because I think there is a lot of there is a lot of superficial hype that hides really deep things. So everything I come up as a potential hype. Like example, humanoid robotics. You could, a lot of people see this as extremely overhyped in the sense that there is a lot of investments that are going in that direction that are not justified by the current capabilities. If we manage to make humanoid robotics work in the next few years,

• 1:01:47Speaker 01

  the investment will be entirely justified. The risk is if we don't succeed and people lose patience, we are headed for a humanoid winter. And it's going to have a negative impact, I think, on all of robotics. So essentially, they're both overhyped and underhyped in the sense that if you're working robotics today, I think you should be working also on humanoid robotics because we can't afford not to make them work. So there is a, yeah, it's a tension there is, you know, it's all a matter of timelines and where do you think the timelines are going to align?

• 1:02:22Jacob Effron?

  Yeah.

• 1:02:23Speaker 01

  In terms of technology development versus amount of spend and amounts, you know, and focus of the ecosystem.

• 1:02:31Jacob Effron?

  Do you think L M model progress will be more, less, or the same this year as it was last year?

• 1:02:37Speaker 01

  I think it's going to be more.

• 1:02:38Jacob Effron?

  How about robotics models?

• 1:02:40Speaker 01

  More as well.

• 1:02:42Jacob Effron?

  The set, next set of questions is just an unfair set of prediction questions that are probably overly precise. But what year do you think self-driving car rides will exceed like human drivers in the U.S.?

• 1:02:54Speaker 01

  You know, I would love for, you know, when I'm an old grandpa and to be able to talk to my grandkids and tell them, you know, you know, in my day, we used to drive cars by hand. Can you believe this? Like, isn't that crazy? Right? Like, I feel like there is a potential future in which

• 1:03:22Speaker 01

  Look back at today and think, man, we were crazy to leave cars in the hands of humans, given the level of accidents that this generates and the complexity of the problem. So that's a future I would love to see. Whether it'll happen in my lifetime, I don't know.

• 1:03:46Jacob Effron?

  I like to think in the future, you'll have to go to the rural countryside and then maybe there they'll let you get behind the wheel of the car when no one else is around.

• 1:03:52Speaker 01

  Yeah, no, no traffic whatsoever.

• 1:03:58Jacob Effron?

  What's the go to thing that you try whenever a new model comes out to experiment with it?

• 1:04:04Speaker 01

  When a new model comes out, I don't try to follow too closely because there is so many models that come up all the time. I often, my reflexive reaction is to go to the LMCS leaderboard and see where it goes, where it stands, and convince myself whether there's something.

• 1:04:23Jacob Effron?

  So you're a metrics, not a vibes guy?

• 1:04:25Speaker 01

  Should I be paying attention or not? Yeah, more of a metrics. It's very easy to fool yourself into thinking something or another. And I find that when you go to a model and you ask for a hypothetical question, you get a very different answer than when you're trying to actually use the model for real, for a real functional application. So I try to really focus on, is this something, is it helping me in my life? Is it helping me in my business?

• 1:04:55Jacob Effron?

  That makes sense. What year do you think most Americans will have a robot in their house?

• 1:05:00Speaker 01

  So we have robots, right? We have dishwashers, we have laundry machines. They just don't look very much like robots.

• 1:05:08Jacob Effron?

  Very fair.

• 1:05:09Speaker 01

  So if you're thinking about, you know, sort of like Rosie the Robot.

• 1:05:13Jacob Effron?

  I want that coffee cup demo, like, you know.

• 1:05:15Speaker 01

  The mobile manipulator. I think it's going to take a long time. And the reason is, right, anything in your house needs to justify its square footage, right? It needs to be worth being there in the first place. And also, if a robot is in my house today and makes a little nick on my wall, I am very certain that this robot will be in the trash, you know, within half a second, right? The level of.

• 1:05:49Speaker 01

  The robot has to be so good and so safe.

• 1:05:54Jacob Effron?

  Yeah.

• 1:05:54Speaker 01

  For me, even as a robotics enthusiast, to accept it in my house. That's why the only robots that I've really succeeded in the home environment to date are the Roombas. Because they're only sort of, you know, hitting the part of the wall.

• 1:06:08Jacob Effron?

  That can only do so much damage, yeah.

• 1:06:09Speaker 01

  That, you know, it's okay to hit, right? And so I think the bar is going to be extremely high for anything that's mobile and that has access to, that can manipulate your environment. It may be less of a, of a lift for a fixed robot that would just be like a workstation where, you know, you just bring it material and it does things for you, like you falls your laundry or

• 1:06:39Jacob Effron?

  Right.

• 1:06:40Speaker 01

  things in the dish, in the laundry machine and things like that.

• 1:06:43Jacob Effron?

  But with an arm that's not long enough to nick a wall?

• 1:06:46Speaker 01

  Yes, yes. So I think it's going to take time. And I think, you know, we're going to see a lot more applications in logistics and industrial. I like the near home space. I think there is a lot of potential applications to, you know, last meter delivery.

• 1:07:01Jacob Effron?

  Yeah.

• 1:07:01Speaker 01

  Things like that, that could really come to fruition much sooner. Office environments, hospital environments, whenever there is, you know, there are people, it's complex, but there is scale and then it's someone's job to repaint the wall when there is a nick.

• 1:07:19Jacob Effron?

  Yeah.

• 1:07:19Speaker 01

  That feels more, a lot more accessible to mobile robots.

• 1:07:24Jacob Effron?

  Do you have any predictions on like the implications of all this AI progress on the future that you feel like are under talked about right now? Like how it might change our world or the way we do our, go about our day to day lives?

• 1:07:34Speaker 01

  It's going to, it's going to change education. And I don't think we have an understanding yet of how it's going to change education. I think a lot of the narrative around education is, oh, you can use chat GPT to cheat. And so, you know, how are you going to be able to evaluate students and things like this? And that completely ignores the bigger conversation of how do we, this is a magical tool to learn things.

• 1:08:04Jacob Effron?

  Yeah.

• 1:08:04Speaker 01

  It's interactive.

• 1:08:08Speaker 01

  Put my kid in front of a conversation the other day, and I don't remember what the topic was, but we interactively learned about a topic by just having a live conversation with the agent. And it was engaging and it was fast, it was memorable. There's a ton that we can do there. And it's right now, I feel like I don't see that in the public sphere being discussed.

• 1:08:36Jacob Effron?

  Yeah. You have a pretty popular course on Udacity. Are you going to take a second swing at it with some of these new tools?

• 1:08:41Speaker 01

  That's a good question. My class is long obsoles now. It was in the early days of TensorFlow. It was a ton of fun to put together at the time. Do you need me now? Or those tools, I mean, those tools would do a much better job than I do at explaining things and at forming the right curriculum.

• 1:09:06Jacob Effron?

  I encourage our listeners to check out your writing. You are a very, I feel like quite a clear thinker on a lot of these things and still better than the average of the entire internet.

• 1:09:16Speaker 01

  I try to be the anti-slot.

• 1:09:18Jacob Effron?

  Any area of like AI startups or research that we haven't talked about yet that you think is like particularly exciting or interesting?

• 1:09:26Speaker 01

  I'm excited about cheese.

• 1:09:28Jacob Effron?

  Cheese? I did not see that coming at all, I gotta be honest. What about cheese? It's a very French answer, I guess.

• 1:09:36Speaker 01

  Yes, yes. I've been, one startup I've been talking to recently is designing sort of plant-based cheese as opposed to, you know, not using any milk. So building basically a case in based on plant products in a way that's more cheap and sustainable and things like that. I think it's really cool to be able to sort of use AI techniques to design new products that are more like the day-to-day kind of products that can have a huge amount of impact on the world. I like this.

• 1:10:14Jacob Effron?

  They're using AI to explore the kind of design space for non-animal-based cheese?

• 1:10:20Speaker 01

  Yes. Yes, and I think it's exactly the kind of thing that, you know, is a little bit out of left field, but can have a massive impact just based on the scale of, you know, animal farming and milk production and how that can impact the world at large. I want to find the next, you know, the next AI-based cheese startup. I think this is exactly.

• 1:10:48Jacob Effron?

  Have they made the cheese yet?

• 1:10:49Speaker 01

  Yeah.

• 1:10:50Jacob Effron?

  Is it good?

• 1:10:50Speaker 01

  Fantastic. Blue cheese that is, you know, indistinguishable from the cow-based thing. It's served in top restaurants in the city.

• 1:11:05Jacob Effron?

  Wow. I'd be honest, this is the best answer. Usually we get like, oh, I like perplexity, or I like, you know, deep research or something. AI cheese is definitely, that's going to set the new high mark for this question.

• 1:11:15Speaker 01

  Yeah, I think, you know, AI plus something that you don't think about when you think about technology is really where I think a lot of the exciting things will happen. And probably making those connections and enabling sort of people who are not necessarily, you know, in the technology world to have access to those tools and vice versa is really, I don't know, it's fun.

• 1:11:40Jacob Effron?

  Well, this has been a fascinating conversation. I'm sure folks will want to pull on all sorts of threads. And so I'd love to leave a last word to you. Where can folks go to learn more about you, about Waymo, anywhere you want to point folks, the mic is yours.

• 1:11:51Speaker 01

  I, well, when I have the inspiration, I post, I have a blog on Medium where I post random thoughts about machine learning. And that's where I, that's my, that's my little water cooler to, with the rest of the world.

• 1:12:15Jacob Effron?

  Well, thanks so much. This has been an amazing conversation.

• 1:12:18Speaker 01

  Thanks.

• 1:12:19Jacob Effron?

  Hey guys, this is Jacob. Just one more thing before you take off. If you enjoyed that conversation, please consider leaving a five-star rating on the show. Doing so helps the podcast reach more listeners and helps us bring on the best guests. This has been an episode of Unsupervised Learning, an AI podcast by Redpoint Ventures, where we probe the sharpest minds in AI about what's real today, what's going to be real in the future, and what it means for businesses in the world. With the fast-moving pace of AI, we aim to help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Thank you for listening, and see you next episode.