Waymo’s Journey to Full Autonomy: AI Breakthroughs, Safety, and Scaling

In this episode of No Priors, Dmitri Dolgov, Co-CEO of Waymo, joins Sarah and Elad to explore the evolution and advancements of Waymo's self-driving technology from its inception at Google to its current real-world deployment. Dmitri also shares insights into the technological breakthroughs and complexities of achieving full autonomy, the design innovations of Waymo’s sixth generation driverless cars, and the broader applications of Waymo’s advanced technology. They also discuss Waymo's strategic approach to scaling amidst regulation, deployment in cities like Phoenix and San Francisco, and the transformative potential of autonomous driving on car ownership and urban infrastructure. Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @Dmitri_Dolgov Shownotes: 00:00 Introduction 00:15 History of Self-Driving at Google 00:29 DARPA Challenges and Early Involvement 01:39 Formation of Waymo 01:53 Industry Lineage and Early Skepticism 03:05 Initial Goals and Milestones 4:33 Pivot to Full Autonomy 04:50 Scaling and Deployment 05:29 Generational Breakthroughs 06:59 Choosing Deployment Cities 09:26 Technological Advancements 11:01 Evaluating Safety 14:41 Regulatory Stance and Trust 16:52 Future of Autonomous Driving 23:19 Business Strategy and Partnerships 26:06 Changing Urban Mobility Trends 26:40 Challenges and Misconceptions in Self-Driving Timelines 28:43 The Role of Traditional OEMs in an Autonomous Future 30:54 Designing Cars for Autonomous Ride-Hailing 33:42 Scaling Responsibly 35:18 Generalizability and Future Applications of AI 37:10 The Complexity of Achieving Full Autonomy 42:58 The Importance of Data and Iteration in AI Development 46:13 Reflecting on the Journey and Future of Waymo

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:05Sarah Guo?

  Hi listeners, welcome to No Priors. Today, we're hanging out with Dmitry Golgov, co-CEO of Waymo. Waymo started as the chauffeur project within Google back in 2009 and eventually spun off as its own company. Now, it provides over 100,000 paid rides each week across San Francisco, LA, Austin, and Phoenix. I love taking Waymos and I'm regularly campaigning for better South Bay coverage. We're excited to dig into all things robo taxis, self-driving, what it takes to deploy this technology on a mass scale, and what's next for Waymo.

• 0:35Speaker 01

  Well, Dmitry, thank you so much for joining us today.

• 0:37Speaker 02

  Thank you for having me.

• 0:39Speaker 01

  Maybe we can start off with just a little bit of a history of self-driving at Google, how you got involved, and how things have evolved over time.

• 0:45Speaker 02

  I've been doing this for quite a few years, I think about 18 now. I got started in around 2006. This was the time of the DARPA Grand Challenges. This is when DARPA organized a few competitions that they called the Grand Challenge in robotics with the purpose of advancing research in autonomous vehicles. And so, the first competition they had was the first Grand Challenge. The challenge there was to create a car that could drive autonomously in a desert. I just completed a stack and wired it, but it would drive for about 100 miles. Nobody succeeded, but there's a lot of great progress that was made. And then they repeated the challenge and a few teams succeeded.

• 1:30Speaker 02

  On the heels of that, they created another challenge that they called the DARPA Urban Challenge, where the setup was kind of a mock city that was supposed to imitate what driving on public roads is like. And that's the one that I was involved in. I was on Stanford's team. This was kind of my moment where it clicked for me. I saw the future and the benefits, and I've never looked back. That's what I've been doing ever since. And then, we started this project at Google in 2009. It was a small group of us, and then that grew into what now is Waymo. We started the company in the very beginning of 2017.

• 2:09Speaker 01

  Yeah, it seems like a lot of the lineage or history of this field all traces back to a handful of labs. You know, it's like Sebastian Thrim's lab at Stanford and a few others. And it seems like the founders of a lot of the companies that ended up eventually existing in this ecosystem all came out of the same sort of cohort of people, which I always think is fascinating to think about in terms of lineages.

• 2:28Speaker 02

  Yeah, yeah, no, definitely, definitely a small world. The CMU team and the Stanford team, there's a few people who came and started this project at Google, and then 2009 came from those teams.

• 2:38Speaker 01

  I think when you started working on this, this was considered a little bit of a crazy thing to do, right? It was early on, a lot of parts of the waves of deep learning hadn't really quite happened yet in terms of applications across all sorts of areas. Like, AlexNet hadn't existed yet, like all these other sorts of things that...

• 2:53Speaker 02

  Nothing existed, right? And you're absolutely right. People, you know, we heard a lot about us being crazy and never going to work in full honesty with ourselves. We're not exactly sure if, you know, we're just a little bit crazy or completely insane when we're trying to go after this problem.

• 3:11Sarah Guo?

  Did you treat it as an open-ended research project? Or you treat it as like, hey, you had an idea in your mind of like, an endpoint date where this would be viable on public road?

• 3:19Speaker 02

  More of the former, but it was not, it was not research, right? So, you know, and then the university, the upper challenge date, it was a research project. Then when we started at Google, it was under the belief that we can make it work. And if we can, then the impact, the positive impact of this technology on the world and the mission is worth it. But it was early days, so we actually had very little data to go by in terms of thinking about how long it's going to take and how hard the problem is going to take.

• 3:56Speaker 01

  How long did you think it was going to take? Like, at the time that you started?

• 3:59Speaker 02

  Oh, I don't know if we had a specific date, but I think that was actually the first question that we posed. It's like, you know, let's not build a product, right? In the first couple of years or so, we didn't have like a product in mind or a target data mind. The first order of business was to explore the space, right? So, we, towards that end, we created some milestones for ourselves with the goal of prototyping and learning. And just understanding. So after those two years, we said, ah, okay, you know, there's something there. Let's start talking about, you know, what the product could be. And actually, our first product that we thought was going to be viable was, you know, what nowadays you would call kind of an advanced driver system. Right. And we had some expectations of, you know, a small number of years that it would take for us to get there. When, you know, we, after working on it for a while and making more progress on kind of the core of the technology.

• 4:43Speaker 02

  We decided that was not the, you know, right path for us, that we want to go after full autonomy. That was, you know, that made that pivot around 2013.

• 4:49Speaker 01

  You're now doing something like a hundred thousand rides a week. So five million rides a year, sort of annualized out, which is incredible. What was the inflection point or what suddenly caused that sort of volume to happen or all these things to come together? Cause it feels like a reasonably recent phenomenon in some sense.

• 5:04Speaker 02

  Yeah, you're right. But there was a discreet, you know, some discreet jumps. So kind of if, you know, we rewind the clock a little bit, and in my mind, there were a few kind of generational discontinuous steps on kind of that progression from, you know, that point in 2013 when we said, let's go for it. To where we are today, I should say, exactly right. You know, a hundred thousand trips per week, more than a million miles per week and growing exponentially. So some interesting ones were, you know, in 2015, that was kind of our zero to one moment. This is when, you know, for the first time, we put a car on the road. There was that, you know, our, what we called the, you know, third generation of our system. This was the third generation of our, you know, self-driving hardware suite, you know, sensors, computer. And we put it on a custom-designed vehicle that we called the Firefly. We took a few rides, but nobody behind the wheel, zero to one moment.

• 5:50Speaker 02

  Then the next evolution, that kind of generational escape was our fourth generation of our driver. Those are the Pacific minivans with the fourth generation of the Waymo hardware suite. And we deployed those in a full autonomous mode in Arizona, in Chandler. And we actually opened it up to the public in 2020. But at that point, we, the focus was on doing it repeatedly. And the focus was on, you know, maturing the, you know, the technology, the building of the driver, the evaluation of the driver, kind of doing the leases in a regular cadence, getting it out to real customers, hearing from the customers. Understanding the feedback and iterating.

• 6:32Speaker 02

  So that was the focus of that fourth generation was not to grow and scale and capture the market. And then at that point, we made the decision to jump to what we now call the fifth generation of the Waymo driver. It's on the JLR IP basis. This is what you see in the fleet today in those four cities, Phoenix, San Francisco, LA and Austin.

• 6:52Speaker 01

  That was very smart, actually, that you started in Arizona versus in California. And I think, for example, I think Cruise ran into some issues in San Francisco where there was activists like putting cones on the cars and trying to stop them and doing other things. And so it seems smart to start in Arizona. I was just sort of curious, what are the criteria that led you to start that as a sort of a testbed or a place to?

• 7:12Speaker 02

  So I guess it depends on the different time horizon. So on the fourth generation, we picked a deployment area that was kind of medium complexity. And the goal there was, as I mentioned, is to go end to end. So we picked an environment where we thought it was, we checked enough of the boxes to help us learn the most important things that we wanted to learn and de-risk. While, and that was a deployment. And then there's the development of the system. So for the development of the system, you want to go after the hardest problems possible, right? You want to go after the densest environments, you want to go after the harshest weather. So we've kind of in parallel been doing that. So we've made a decision to deploy in Chandler in 2020 to learn from the end to end system while pushing on development of the system. Then when we've learned enough and we made that discontinuous jump to the fifth generation of our driver.

• 7:58Speaker 02

  And I said, okay, like that's the platform we believe that we want to take to scale.

• 8:01Sarah Guo?

  What's the hardest environment for a self-driving car?

• 8:04Speaker 02

  So density matters, speed matters, weather matters.

• 8:08Sarah Guo?

  Okay.

• 8:09Speaker 02

  So where a few of those come together is where the most complexity is.

• 8:13Speaker 01

  So New York in the winter is really bad and then?

• 8:16Speaker 02

  Yeah, that's right. Yeah, that's right. But then this is why we picked San Francisco is that it's good for learning and advancing the driver. It is a very interesting commercial market. We also, at the same time, went after downtown Phoenix. It has a different, makes more of the higher speed roads. And that gives us kind of the way we think about it in terms of the development and evaluation of the driver is kind of in the space. Of the operating domain, not necessarily areas or zip codes. And then you kind of take cities and you map that to the operating domain and you deploy it, right? And then looking forward is the kind of in terms of how we think about future cities. It is, you know, a few lenses is market. Like, you know, is there a good market from the commercial perspective? What is the technical complexity? You know, what is the regulatory environment?

• 9:02Speaker 02

  And that's how kind of the lens that we apply.

• 9:05Speaker 01

  Yeah. Well, with the big technology breakthroughs that got you to this fifth generation driver, that is really the one that you think you can scale now.

• 9:11Speaker 02

  So the biggest one's in AI, probably not surprising. Yeah, there was generational breakthroughs and with every generation of the hardware of the driver, there's new hardware, so it's getting more capable, it's getting simpler, it's getting, you know, more inexpensive and there's a lot of simplification, but that's a boost. Right. Really, it's all about, you know, AI, as you mentioned.

• 9:29Speaker 01

  And what specifically in AI was the shift or change that was important? Was it moving to sort of end to end DL for everything? Was it the transformer backbone? I'm sort of curious, like, was there a specific thing that?

• 9:39Speaker 02

  So for that last jump, the models, transformers, as you mentioned, played a huge role. Before that, you know, we had the big breakthrough before that, you know, it was Connetz. You mentioned AlexNet, that was around 2013. So that gave us a big boost, but was still kind of plateaued at the wrong place. And then it was a few of those things coming together, right? It is transformers, it is bigger models, more compute, coupled with kind of the whole evaluation. We often talk about the architectures and what's really more important, kind of the machines surrounding the architecture. You need to, the architecture is a name blur, but really to make it work at the level that we care about, you need like everything around it, the data engine, the flywheel of training the system, evaluating it. And you're going to have to think about the problem of evaluating the driver in tandem of building it, right?

• 10:23Speaker 02

  And you need the simulator, you need the data. So all of those coming together, I think is what leads to the breakthrough and discontinuity that you're seeing with where we are today.

• 10:32Sarah Guo?

  Can you explain how you guys think about evaluation internally? And then also, you know, how that might differ from how regulators evaluate this from the safety case perspective?

• 10:42Speaker 02

  So that's a big question, but I think I'm glad you're asking that because it is a super important and insanely difficult problem, right? We often talk again about the building of the driver, but there's two problems, the building of the driver and the evaluation of it, and they go hand in hand. So internally, it starts with figuring out what metrics you care about. Then bringing the data to support the evaluation of those metrics, you know, which we, you know, have hundreds. Then you need all the infrastructure, including things like the simulation. Some things you can evaluate in open loop, you know, and some things you need closed loop simulation for. So you need to build a realistic, scalable simulator to support all of that. Right?

• 11:23Speaker 02

  There are all of these metrics that guide our development of the system that help us improve and kind of train the Waymo AI, the Waymo driver. And then it funnels into what we call validation and evaluation. The aggregate of the evaluation validation methodology is what we call the readiness and safety framework.

• 11:40Speaker 01

  You know, if you look at miles driven in an urban setting, or, you know, some comparison to human drivers in a given city that you operate in, what is a relative safety level of what Waymo is doing versus a human driver at this point?

• 11:52Speaker 02

  We are pretty proud of our record. I think we have now, now that we've driven tens of millions of miles in fully autonomous, what we call rider only mode, and driving today more than a million miles per week. And we can, you know, with pretty good confidence and empirical data, say that we are, you know, better than human benchmarks. So we published some of that, you know, very recently on what we call the Safety Hub. The latest data point that we shared was based on 22 million fully autonomous rider only miles. And we compared our performance versus human benchmarks by, you know, different severity levels of, you know, contacts, collisions, different severity levels. So we see depending on the severity level, we see that, you know, for the lowest severity,

• 12:33Speaker 02

  lower severity outcomes, we're about, you know, a factor of two better than the human benchmarks. And as you look at more severe outcomes, the gap increases. So for kind of the most- That's pretty good. This is exactly what we want to see, right? So for airbag, you know, deployment tech collisions, we're about a factor of six better than human drivers.

• 12:52Speaker 01

  Yeah.

• 12:52Speaker 02

  And that's without any notion or attribution of causality or fault. So if you bring that into the picture, you know, most of those are unavoidable, there's very little they can do. Yeah. We have done a study with Swiss RE, the global, kind of, I think, the biggest global ratio in the world. And we partnered with them, we shared the data, they've done the analysis, and they found that, you know, for damage- Claims, we had about a four X reduction versus the human baseline. And for bodily injury claims, we had a hundred percent reduction. It was a smaller data set. This was earlier, but that was based on just a little bit less than four million miles. But also, you know, uh, it was from their point of view statistically significant. Uh, and um, that, that, you know, again, we're, we feel pretty proud of that.

• 13:34Sarah Guo?

  Yeah, it's amazing. Um, given that, what do you think the regulatory stance should be?

• 13:38Speaker 02

  I think we want to make sure that, you know, we enable uh this uh technology and service of the mission of making roads safer. Uh, and, you know, we've been engaged with regulators for many years and have that dialogue. And you know, so far we've uh had good success getting all the necessary, you know, permits and all of uh to a lot of scale. The way we think about it uh internally, and that's how kind of we have the dialogues with the regulators, with communities, with writers, it needs to be based on transparency and it needs to be a responsible iterative gradual process. And because this thing is very new, the technology is very new, the product is very different.

• 14:14Speaker 01

  If you're doing a million miles a week now, what prevents faster rollout? In other words, it seems like you've proven out that this is a safe solution. It's working very well at scale. You have your generation five driver that seems to be quite performant. Why not go bigger faster?

• 14:30Speaker 02

  We are scaling exponentially. Uh, it took us about three months to get from 50,000 miles to 100,000 miles. Right? So, you know, we're moving at a, at a, at a good rate. Uh, but the main thing uh

• 14:43Speaker 01

  What is the total number of miles that are driven in the U S per year?

• 14:47Speaker 02

  Uh, all vehicles, I think it's uh, you know, in different modalities, you know, large vehicles, small vehicles, I think so, just over three trillion miles.

• 14:54Speaker 01

  Okay.

• 14:55Speaker 02

  So quite a bit.

• 14:55Speaker 01

  Yeah.

• 14:55Speaker 02

  We have, we have, we have, we have

• 14:56Speaker 01

  That is way, way.

• 14:59Speaker 02

  Uh, but the way, again, you know, the way we think about it is that it's important for this to be an iterative process where we earn trust. It is not a thing where, you know, you, you, you know, build it and then you uh, you know, just turn on the switch and as you're right, it needs to be, it's new, it's different, needs to be a dialogue. It needs to be like we, you know, talked about establishing uh the safety

• 15:19Speaker 01

  Sure.

• 15:19Speaker 02

  record, right? And we needed to build up to that, right? We collected in tons of millions of miles. Now we feel pretty good about that. So then we, you know, that gives us confidence that earns us, you know, trust. Uh, so you have to be transparent about where we are and then we, you know,

• 15:31Speaker 01

  What do you think is the biggest shift that's coming from a technology basis going forward? We, we had Andrei Karpate on our podcast a couple of weeks ago and he sort of contrasted what he viewed as the Tesla approach, which is more in his view, sort of software driven. To the Waymo approach, which was more sort of hardware centric. Do you think that's a correct characterization? And also, how do you view things sort of changing over the next, you know, a couple of years or year or two in terms of your roadmap?

• 15:57Speaker 02

  Yeah, I would think, yeah, I mean, it's all about AI, full stop. Like we talked about, you know, a few big breakthroughs that allowed us to get to where we are today, you know, Conness, Transformers, Bigness, now, you know, most recently, kind of combining the Waymo AI with the general well knowledge of, you know, VLMs. This is at the core of it. And, you know, hardware matters, right? We have to operate in the physical world. So the hardware that we have on our cars gives us, I tend to think of it as an advantage, right? Like you have to see well.

• 16:27Speaker 01

  Yeah.

• 16:27Speaker 02

  Right? So, you know, as a human driver, right? Like if you can't see, you know, if you close one eye, you don't, it loses depth. If you, like, if you're, you know, your vision is not 2020, you don't have your glasses, like you'll still drive, right? You'll drive a K, but you know, you're not going to be a Skoda driver. But the point, I guess, the way I see it is it's all about AI. It's all about building the system, building the AI, building the software and being able to evaluate it. And that's what we have today, right? So far, I mean, we have, you know, achieved good quality of the driver. We have built all the machinery to evaluate it and know what it takes. So now for us, it's an optimization, right? It's a, it's an optimization, simplification. And if you have that, you know, that the thing that works and you have a good mechanism

• 17:09Speaker 02

  to value it, then, you know, it really is drastically different in terms of, you know, how fast it can move. I guess I've been in this other mode for years where we have not solved the kind of, haven't cracked the nut, right? And you're kind of hypothesizing what it would take and like, you know, what is the yield of this technical breakthrough? And you're kind of climbing uphill, right? And it's this, you know, old analogy of, you know, going to a mountain, you see the peak, then you get there and you thought it was the summit, but no, now you see it like that, you know, the landscape. And that's what it felt like for us, you know, for many, many years. That's kind of the trend that the youth industry usually follows. And I just find that it's a qualitatively different place to be in when you've cracked the nut and you have the valuation and now you can optimize and scale.

• 17:45Sarah Guo?

  As part of your optimization, do you think of like reducing or simplifying the sensor suite as an important priority?

• 17:52Speaker 02

  Every generation of the hardware would increase capability, but also simplify drastically. And then the cost comes down every generation. So that was true for the previous generations. We made a big jump from the previous generation to today's and then from the fourth to the fifth. And then now going to the sixth generation, this was the primary focus simplification and bringing down the cost. As well as with a vehicle, making it about the user experience. It's absolutely. And then, of course, the economies of scale, right? There's nothing like fundamentally look at the components that we use. You know, in our hardware, there's just, all of them are fairly commoditized and with scale. They just, you know, you'd think you get to ride the typical curse of the hardware. You know, radars used to be expensive in the past, right?

• 18:35Speaker 02

  You put them all in the cars, you bring down the cost. The same with computers, right?

• 18:39Speaker 01

  I guess qualitatively, we know that people can drive well with like a very simple vision system, right? And obviously, in the self-driving world, we have dramatically more and different types of sensors. Is there any sort of analytical framework that you all use in terms of the amount of data slash types of data you need to collect in order to have a performance system relative to the curve on the AI side? In other words, can you follow some sort of scaling curve on AI that you can pre-predict the set of sensors that you can do without?

• 19:08Speaker 02

  What we have now is, now that, to jump to the core of the answer to your question, now that we have cracked the knot of the driver and the valuation, we can answer that question with data. Right? So we have, you know, talked about different, so you start with humans. We have just two kind of cameras, but like on a pivot, right? And we can drive kind of okay-ish, right? And then if your vision is not very good, if you're, you know, like you maybe not drive as well as computers can, right? So that's why we have vision, we have cameras, we have lighters, we have radars, right? And they will give us some benefits. And we can talk about, you know, what the pros that they all and kind of how they bring to the table, the data they bring to the table and how they're nicely complementary.

• 19:50Speaker 02

  And what you get from the different sensing modalities just by the kind of the physics of it. But again, in terms of, you know, how do we answer that question? For years, it was more theoretical and you hypothesized. You know how much of a certain thing you need now that we have a thing that works and we can evaluate it. We can, you know, bring data to the table. We can empirically, more empirically, you know, answer that question and say, what, what happens to the Waymo driver? If you take away something, right? Maybe you add noise to the system, maybe you degrade, maybe you take away a full sensor modality. Maybe you take away lighter and you take radar away completely and you just drive with camera. You know, let's drive. Yeah, you know, I can use a human drive with one eye or, you know, with blurry vision. Yeah. Are you, is that acceptable performance? Will you get a license? No. Same, same for us. We can, you know, take some way and we can answer that question like, is the performance, can it still drive? Of course.

• 20:35Speaker 02

  Is the performance good enough for full autonomy? And is it good enough for our bar of readiness and safety? And the answer is no. Okay. And but again, this is in the context of full autonomy, is in a context of scale. And it is in the context of the, you know, responsible deployment and the high bar for safety and readiness, the wisdom for ourselves. If you change, you know, some of those inputs, let's say you talk about, you know, small scale where you talk about, you know, not full autonomy, talk about a driver system. Then the answer changes, right? It might have a different, you know, if you still have a human in the loop and they're responsible for safety. You know, a different configuration, you know, whether it's, you know, just to get rid of sensor modalities altogether.

• 21:18Speaker 02

  Or in, you know, maybe use all three modalities, but you pick a different operating points because, you know, it's cameras, it's cameras. So their solution, dynamic range, cleaning, you know, same for letters, but you know, the answer of your operating point would change based on kind of where you set the bar and what the product actually is.

• 21:30Sarah Guo?

  If you look forward a year or two, like now, you know, crack the nut, Waymo is looking at scale and probably more about the business. Like, do you think of robot taxis at scale as the, the near term business plan or are there other modalities or like deployment? Like, if I think about this as just like a capex problem, like other avenues that are important for you guys to explore.

• 21:50Speaker 02

  That's the main one. Right hailing is the main one that we're focusing. So we're, you know, focused on technology, we're focused on the product. We are learning from our users every day, we're earning trust. And we're setting up the ecosystem of, you know, partnerships in that space. So that's our primary focus, right? We are very excited about the commercial opportunity there. It's not the only one. Always think of Waymo as a technology company. We're building a generalizable Waymo driver with the mission to build the world's most trusted driver. And we want to deploy that driver not just in ride-hailing. There's more than three trillion miles in the US. There's more than 10 trillion miles driven worldwide. So the vision and mission is to deploy the Waymo driver in different commercial products

• 22:35Speaker 02

  and different applications and maybe in different modalities across all of that spectrum. So that includes things like deliveries, things like long haul trucking. It includes things like personally owned vehicles. But right now, we're very focused on ride-hailing.

• 22:49Speaker 01

  Do you think you'd license that out? Or you think you'd actually build the vehicles for these different use cases?

• 22:53Speaker 02

  We're not. We've never been in the business of building vehicles. We partner. And I guess this is how like this is. The mission is so important. The opportunity is so massive that we don't want to go it alone. So we think about partnering and we even together the ecosystem to pursue all of those different commercial applications and all of those products. And you see us doing that in the ride-hailing business. We don't build our own vehicles. We partner with partner with the tier ones. We partner with OEMs. We partner with other companies who help us on the operational side. We partner with companies who help us on the network side, so forth and so on.

• 23:30Speaker 01

  How do you think a car ownership will change over the next decade or two? So it's very exciting to see this ramp in terms of driverless rides that are happening. And one could argue at some point, some proportion of the population, just like people flip to Uber to ferry them around in major cities versus driving or taking taxis or other things, there could be a flip here to autonomous systems versus owning cars. You should just be able to order something on demand, have it show up at the right time, and you just get in and it takes you wherever you need to go. Do you view that as a 10% use case, a 30%? I'm just sort of curious, like what proportion of miles do you think will convert over time?

• 24:05Speaker 02

  Uh, I think over time we'll see more and more as you know, with the technology matures, as you know, it gets deployed in more of these different products and modalities. I think you're starting to see some of that even today in ride-hailing, right? And it's not uniform, right? But even kind of the densest cities, if you look at, you know, the people who live in San Francisco or people who live in New York, even before, you know, autonomy, right? There was a shift of fewer people wanted to, in those areas, wanted to own cars, especially the younger generations, right? It's not, not what they're excited about, right? And I think with, you know, with autonomy in those areas and kind of the densest course, you will see more of that evolution and continued trend, right? And then, you know, over time it will expand and so forth and so on. But, you know, the thing that I'm most excited about is all of those modalities

• 24:48Speaker 02

  will be bringing the safety benefits of this technology to the ecosystem.

• 24:51Speaker 01

  One of the reasons I ask is I remember talking with people in the self-driving world, I don't know how long ago, eight years ago, nine years ago, whatever it was. And at the time, everybody thought this wave was coming. I think a lot of people were off in terms of the timeframe, in terms of what actually happened. And there was sort of a flurry of startups all getting up and running at the time. And a lot of the conversations were around how urban environments would change over time. Where do you actually put a parking lot? Or where do you park your car versus having like a lot outside of the city? That would then, the cars would come in and pick you up at the right moment versus everything needing to be centralized. And so one of the reasons I was asking that question was to try and get a sense of your view of the order of magnitude and also timeframe by which some of these transformations may occur.

• 25:29Speaker 02

  You know, I don't want to speculate on specific timeframes. I think the vision is absolutely the correct one, right? And so, you know, we talked about the safety benefits of it, that's the primary one. Then there's accessibility. Once you have, you know, you get those benefits, once you can deploy at scale, right? And once you are deployed at scale, you can do things like, you know, use land in a better way, right? Instead of, you know, taking up so much space for parking lots, having, you know, personally owned vehicles that just sit around 90% of the time, you can do better. I think as a society, we can do better, right? All of those, you know. But the thing with those benefits is that come from scale. And again, safety is the primary one that we're very focused on. You know, we've talked about as kind of the North Star, as the mission, as the vision for, you know, many years.

• 26:14Speaker 02

  But you know, it was always kind of with us, you know, came at once we get to scale. So I think today we're starting to actually earn the right to talk about, you know, starting to realize that mission. Right? You know, tens of millions of miles that behind and well, but you know, more than a million per week. We and the benchmarks, the safety benchmarks that are statistically significant, seem, you know, purely ambiguous. Talk about actually, you know, real tangible safety benefits and like inducing, reducing, you know, harm and injuries that are happening on the road today. So that's our primary focus and, you know, I think that's the primary benefit that we'll see. And then beyond that, I think there's going to be auxiliary ones like the ones we mentioned.

• 26:51Sarah Guo?

  What do you think is the role of traditional OEMs in this world? When, you know, I'd say functionally, like a car takes you from point to point. Right? Hailing takes you from point to point with like, you know, different tiers of comfort level. But, you know, very large industry has been built around people buying passenger vehicles for like, you know, industrial design or brand or all these other things that, you know, you really need a Ford Raptor to run around the Bay Area, right? How do you, how do you think that evolves? When, when, you know, perhaps one of the more primary drivers of value is now AI and the ability to do this autonomously.

• 27:32Speaker 02

  Right. But I guess, you know, we think of what we're doing as building the driver.

• 27:36Sarah Guo?

  And the driver could still drive different cars.

• 27:38Speaker 02

  And the driver, exactly. You put the driver, you still need the car. And different four factors, you know, whether it's a car that's good for ride hailing in certain, you know, urban environments. Whether it's a different vehicle that you need for good transport or truck or, you know, something that you want to take on longer trips with your family. You know, we will need different cars, we'll need different form factors. And I think it's very, very complimentary in what we're doing and what, you know, the car industry is building.

• 28:04Sarah Guo?

  Perhaps to a lot's point about how cities will change. The answer has clearly been given the efforts to change both drivers and cars and the environment to just change the driver. Right? Which is what you guys have done. Do you think there are arguments still to change the infrastructure? Right? Like you can make, for example, in the public transport space, right? There's other form factors that require the participation of the public sector in order to deploy.

• 28:31Speaker 02

  Absolutely. Sustainability is very important for us. Safety is the primary thing, but I think all of those modalities can coexist. In fact, just in the last couple of days, we announced something that we're doing where we are incentivizing people to take Waymos in the cities where we operate to public transit hubs, and then everybody benefits.

• 28:53Speaker 01

  Sure. How do you think about the form factor of the car itself? I know that there was companies like Zoop that Amazon bought where they kind of hollowed out the inside of the car because you no longer needed the steering column and everything else, and they put seats facing each other almost like a London cab. Do you have any thoughts on what that experience will look like in the future as more and more things move to autonomous self-driving ride-hailing systems?

• 29:14Speaker 02

  Yeah, so designing a car around the passengers makes total sense. I agree. In the past, we've designed cars around primarily the driver, right? If it's the Waymo driver, it's all about the rider experience. So we have done quite a bit of work on the sixth generation of the Waymo driver and the car, and the car is designed with the passenger in mind, right? So it is more spacious, it is all about the user experience. You have flat floors, you have lower floor for entry, you have doors that slide to the side, so it's all about getting in. So absolutely. You know, there's different aspects of it. We don't have cars facing each other.

• 29:56Speaker 02

  I think it's an open question. Some people get nauseous when you do that. There's benefits on facing forward, but all of that, I think, will be for us as an industry to figure out as we move forward. But I think the key point, it becomes much more like the design is around the rider, not around the driver.

• 30:14Speaker 01

  It's like you could do very interesting SKUs too, where I've always wanted a car with like a Peloton in the back or something, so as you're commuting, you just kind of...

• 30:22Speaker 02

  I like that. I like it. I want to be in a bike, but I want my bike to be inside of a car. I, I, there might be like, let's see if we can.

• 30:28Speaker 01

  Why do you want to be outside?

• 30:30Speaker 02

  Yeah.

• 30:31Speaker 01

  A contained environment.

• 30:32Speaker 02

  Yeah. Especially in California. God forbid you actually, yeah, take a bike.

• 30:39Sarah Guo?

  I want to, I want a much less exotic form factor. And I just want to be able to take a Zoom with stable internet and have it not look weird.

• 30:45Speaker 01

  On a bike. Right. You want to be on your bike.

• 30:47Sarah Guo?

  On a bike. Sure. On a bike.

• 30:48Speaker 01

  That's very exciting.

• 30:50Speaker 02

  We do, we do increasingly have, you know, team members calling into meetings from Waymos. It is, but it is, you know, it actually, you know, we joke about, but it, it gets to the point of privacy. It just becomes, you know, endless. If you don't have another human car, right? You can, you know, you can do a work meeting. You can do a, you know, call. You can like, listen to your favorite music on full volume and not worry about, you know, that, that interaction of like, having another human that you're sharing the spaces. So we are seeing, that was one of the, you know, hypotheses of the benefits of, of our product. And we are, you know, seeing very positive feedback from our hours today. You know, I want that specific dimension as well.

• 31:27Sarah Guo?

  You know, it's just somebody excited to see this technology expand coverage range. Is the blocking factor to, let's say, you know, a billion miles a week? Is it like putting more cars on the road from a capital perspective? Is it just operationally, this can only happen so fast? Is it your view of like what you want to see from a safety and trust perspective, like consumer trust perspective? What's the bottleneck?

• 31:53Speaker 02

  Primarily it's the latter. So we've always, you know, our playbook has been to go about it responsibly and gradually and earn trust every step of the way and have this transparent dialogue. Again, this is a very new thing, new technology, new product, very different from what people are used to. I think it has to be a process. And again, you know, trust is. This thing that's hard to, you know, earn, but very easy to lose. So that's the main thing, right? Uh, and we see that, you see that like in places where, you know, we operate, uh, and we've engaged with communities and there's writers who have, you know, used Waymo's. It is, uh, there's a lot of trust. There is, you know, people, you know, use the word magic a lot about the experience. Uh, and then you go to a different place where people have not experienced

• 32:38Speaker 02

  it and there's more anxiety, more loss of trust. So you can't just get there in one step. You have to do it kind of responsibly and iteratively. So that, that's, that's the main thing that.

• 32:47Speaker 01

  Yeah, my sense is back when, uh, they had elevator operators, getting rid of the operator was a big deal, right? Cause you used to have somebody in the elevator who would close the door for you and push the button and control that experience. That's interesting to see that evolution of different types of technology over time or people's interpretation of it. Um, how do you think about generalizabilities? You mentioned you're building a general purpose driver that could potentially port into other types of vehicles. Do you think there's other extensions into other forms of robotics with what you're building? Or do you think those are all more specialized models? Or how do you think about where this could go from that perspective?

• 33:18Speaker 02

  On the driving part, uh, we've kind of designed it to be generalizable and we're very happy. You know, with what we're seeing with the fifth generation driver and like they AI generalize it really well. And based on, again, we've been using, you know, data from a very broad ODD to build it. Even, you know, if we're deploying, you know, responsibly and gradually. Once we, you know, believe that we've achieved the level of performance that we require for a certain ODD, which maps to, you know, certain areas and subcodes. And to the other part of the question of, you know, going beyond autonomous vehicles, some of the stuff by the nature of the problem and the complexity, I think the, some of the research that we do, uh, is pretty foundational. Uh, when we talk about, you know, perception, right? You know, you can be in a car, you can be in a different, you know, modality and like operating in the physical world.

• 34:03Speaker 02

  Uh, a lot of the research that we published, a lot of what we've done, I think can, you know, benefit those communities as well. When you talk about, you know, AI, uh, being deployed in a kind of real time system, in a safety critical system, a lot of the work that we have done, I think, you know, can translate, uh, to others, you know, so forth. When we talk about the evaluation of the system, kind of what, you know, many robotics applications, you know, beyond autonomous vehicles. Need a good, realistic, scalable simulator. That that fundamental work translates. So in your support and so on, we are very focused on the trillions of miles where we can have the positive benefits. So for us, like I focus, I think focus is very, very important. So we are being very laser focused on driving.

• 34:43Sarah Guo?

  Can I ask, go back and ask like perhaps a more technical question here? Like a while back, you said you wanted to focus on the full autonomy problem. There were, there are many other teams who actually have some lineage in the like, you know, Waymo, chauffeur, Google programs that chose a use case that looks like it was going to be easier. Trucking, like long haul trucking, deliveries. It's not clear that's much easier. Do you think there's a lesson to be learned here? Or at least, you know, there are more miles being driven autonomously on the road in passenger vehicles by Waymo than in these other applications today?

• 35:25Speaker 02

  Yeah.

• 35:25Sarah Guo?

  What lesson?

• 35:26Speaker 02

  Yeah, I think that's a great question. Kind of the big differentiation that I would draw, and this is like orders of magnitude, in the difference between full autonomy versus at scale versus, you know, a driver system. And that's the big, like, that's the chasm. To your question of the different vehicle platforms, different operating domain. You can have slower speed applications where, you know, you do local deliveries, or you can have, you know, a trucking application on freeways. And they're a little bit different. But if we're talking about full autonomy, maybe there's second order differences, but the first order complexity is still there. You can, like, if you think about, you know, the core, the heart of the problem

• 36:09Speaker 02

  of building a generalizable and safe driver and being able to evaluate it, and the incredibly high bar of safety. The complexity of the noisy, messy physical environment and the long tail of, you know, people doing all kinds of weird things. And the necessity of making real time decisions where milliseconds matter and like how hard that AI problem is. It's, you know, the distribution, the contours change a little. But if you're talking about freeways or all speed, but the fundamentals are like you don't get like there's no silver bullets. You don't get to skip the core complexity. Now, for example, freeways, the nominal case, they're a bit more structured. Right, like you're here, but you still encounter with lower frequency. But at higher speeds, where the severity, you encounter all kinds of things.

• 36:53Speaker 02

  You encounter construction zones, you encounter grills and mattresses and all kinds of stuff falling off of the cars in front of you. You encounter cars having getting into accidents and kind of spinning out in front of you. You encounter people driving recklessly with their own cars, with their motorcycles. You encounter pedestrians jaywalking, you encounter all kinds of things, right? And it happens much less frequently. So this is where it might be unintuitive. If it happens at once per million miles, none of us have seen examples like that in our world. So it can lead to this early stage optimism about like, okay, there's the simplification. But if you want to do it full autonomously and you want to do it at a scale, that complexity is still there. It's just the way it changes.

• 37:34Sarah Guo?

  And why is it breaking from like, you know, let's say advanced driver assistance? That it seems to work in more and more scenarios versus, let's say, full autonomy?

• 37:45Speaker 02

  What's the delta?

• 37:46Sarah Guo?

  Yeah.

• 37:47Speaker 02

  It's the number of nines, right? And it's the nature of this problem, right? If you think about where we started in 2009, one of our first milestones, one goal that we set for ourselves was to drive, you know, 10 routes, each one was 100 miles long, all over the Bay Area. You know, freeways, downtown San Francisco, around Lake Tahoe, you know, everything. And you had to do 100 miles with no intervention. So the car had to drive autonomous from beginning to end. That's the goal that we created for ourselves. It was about a dozen of us, took us maybe 18 months, we achieved that. 2009, no ImageNet, no ConvNets, no Transformers, no big models, tiny computers, you know, all this stuff, right? Very easy to get started.

• 38:30Speaker 02

  It's always been the property and with every wave of technology, it's been always very easy to get started. But that the hard problem, and it's kind of like that the early part of the curve has been getting like, you know, even steeper and steeper. But that's not where the complexity is. The complexity is in the long tail of the many, many, many nines. And you don't see that if you go, you know, for a prototype, if you go for, you know, a driver assist system. And this is where, you know, we've been spending all of our, that's the only hard part of the problem. Right? And I guess nowadays, it's always been getting easier with every technical, you know, uh, kind of cycle. So nowadays, you can take with like all of the advances of an AI and especially in the, you know, generative AI world and the LLMs and BLMs. You can take kind of an almost off the shelf, uh, you know, transformers are amazing.

• 39:13Speaker 02

  If BLMs are amazing, you can take a kind of a BLM, uh, that, you know, can accept images or video and is, you know, has a decoder where you can give it, you know, text prompts and it will output text. And you can fine tune it, you know, with just a little bit of, you know, data to go from, let's say, camera data on a car to instead of words to trajectories or, you know, whatever decisions you might know, you just, you know, take the thing because a black box, you know, you take whatever has been trained for a LLM, you fine tune it a little bit and like that without, you know, that's, I think if you're asking a good Christian in computer science to build, you know, an AI today, this is what they would do.

• 39:46Sarah Guo?

  Yeah.

• 39:47Speaker 02

  And out of the box.

• 39:48Sarah Guo?

  That's amazing.

• 39:49Speaker 02

  You get something that it, it's amazing, right?

• 39:51Sarah Guo?

  Yeah.

• 39:51Speaker 02

  The power of transformers, the power of BLMs is mind blowing, right? So with just a little bit of effort, you get something on road and it works. You can, you know, drive, I don't know, tens, hundreds of miles and we just, it will blow your mind. But then is that enough? Is that enough to remove the driver and drive, you know, millions of miles and have a safety record, you know, that is, demonstrates really better than humans? No. Right? And I guess this is, you know, with every tech, you know, evolution, technology in a breakthrough in AI, they've seen like that appreciated.

• 40:15Sarah Guo?

  Is it the right way to think of the, um, the iteration cycle for Waymo now still like many other AI companies where in eval, some, some set of cases comes up that you don't handle as well as you want, and then you collect more data and you put it into the pipeline? You retrain and you deploy. Or are there still architectural changes that are happening even past this point of cracking the nut?

• 40:37Speaker 02

  Both. So the first thing you mentioned where it's the data collection and understanding where performance is not going to kind of building the whole data flywheel and evaluation flywheel. That's at the heart of it. But I think this is where it gets a bit nuanced. What do you do? What is the architecture? And what is the training methodology? In particular, the simplest thing you can do is an end to end model that is trained just on imitative, kind of imitating human drivers. So very easy. Sensors, pixels go in, driving behavior that you have examples of, and you just train it to imitate human drivers. And you can run this flywheel and kind of run this circle that you described while operating under that paradigm. And you'll make progress, right?

• 41:22Speaker 02

  It's a very well understood kind of approach, right? You balance your data, you find some examples where you're not performing as well as you would have liked, if you figured out how to evaluate it.

• 41:32Sarah Guo?

  You simulate more examples that look like that.

• 41:34Speaker 02

  Now you're getting simulation, right? But this is exactly how it starts to get interesting, right? For what you described, you don't even need a simulator. You can run that, you can turn the crank on this whole machinery even without having a simulator. Just do the open loop to imitate. So you find more examples where you'd like, of example, things you want to imitate. And you reduce the number, you find examples where humans did something you don't like. So you've got a few, this is your dataset balancing. And you will continue to improve. You might plateau in the wrong place. You might plateau in the right place for a driver assist system. You will plateau in the wrong place for a fully autonomous system. So then you need to kind of build that machinery. You'll still like, at a high level, that principle of new data and augmentation holds. But to really go the distance to full autonomy, you need to do other things, right? You need to do synthetic data. You need to do closed simulation.

• 42:19Speaker 02

  Maybe sensor simulation is not enough because when you do it at scale, it's just highly inefficient. It's not practical, right? Just kind of doing, imitating sensor, piping it through the whole thing. So then you get into like an immediate representations. Can you simulate in that space? And you still are doing kind of at a very high level, the flywheel. The loop holds, right? But what's in the loop, I think it depends on.

• 42:39Sarah Guo?

  Get more creative.

• 42:39Speaker 02

  Exactly, exactly.

• 42:40Speaker 01

  So I guess, where are we today and where do you think we're going?

• 42:42Speaker 02

  I think we are reflecting on this journey that's been quite a few years. I find myself.

• 42:52Speaker 02

  I'm more excited than ever about where we are, the momentum and the future. Right. I've been doing this for close to two decades. The vision was always there, right? But we had these big existential questions. Can we build the thing? Can we figure out what's good enough and how to evaluate it? Can, will people want to use it? Can we do it in a way that's commercially viable? You know, and so forth and so on. And can we go the distance? Like now, where we are today, operating at the scale we are and scaling. We've demonstrated that we can build a thing. We are proud of our safety record, figured out how to evaluate it.

• 43:36Speaker 02

  We see that people want to use it and they get very positive feedback and people are excited about it. We see that we can do it in a way that's cost efficient and viable. So I am super excited about what the future holds. And we're starting to talk about realizing the mission of actually making, realizing those safety benefits. So now it's all about optimization, scaling and bringing this technology to more people and more places.

• 44:01Speaker 01

  Amazing. Very exciting. Thank you again for joining us today.

• 44:04Speaker 02

  Thank you for having me.

• 44:04Sarah Guo?

  And congratulations on the breakthrough progress over the last small amount of time.

• 44:09Speaker 02

  Thank you. Thank you.

• 44:12Sarah Guo?

  Find us on Twitter at NoPriorsPod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.