How End-to-End Learning Created Autonomous Driving 2.0: Wayve CEO Alex Kendall

Alex Kendall founded Wayve in 2017 with a contrarian vision: replace the hand-engineered autonomous vehicle stack with end-to-end deep learning. While AV 1.0 companies relied on HD maps, LiDAR retrofits, and city-by-city deployments, Wayve built a generalization-first approach that can adapt to new vehicles and cities in weeks. Alex explains how world models enable reasoning in complex scenarios, why partnering with automotive OEMs creates a path to scale beyond robo-taxis, and how language integration opens up new product possibilities. From driving in 500 cities to deploying with manufacturers like Nissan, Wayve demonstrates how the same AI breakthroughs powering LLMs are transforming the physical economy. Hosted by: Pat Grady and Sonya Huang

• 0:00Speaker 00

  You know, if you're building a vertically integrated robotic solution, maybe you can go deep, but our ambition is to be the embodied AI foundation model for all of the best fleets and manufacturers around the world. And, and, you know, to do that, unless we want to overload the company by building a separate neural network for each application, we need to be able to generalize. We need to be able to amortize our cost over one large intelligence and to be able to very quickly adapt to each different application that our customers care about. That's what what we're trying to push.

• 0:45Speaker 01

  Today, we're talking with Alex Kendall, CEO of Wave, about the shift from software 1.0 to 2.0, or from classical machine learning to end-to-end neural networks in autonomous driving. Wave sells an autonomous driving stack to auto OEMs, similar to Tesla FSD, but for non-Tesla automobiles. Major car manufacturers globally like Nissan are choosing Wave to power their AV stacks. Alex started Wave back in 2017, when most self-driving software stacks were massive hand-coded C++ code bases, covering every possible edge case, like navigating around double-parked cars. Alex bet the firm from the beginning on an end-to-end neural net approach to self-driving, and on the use of synthetic data and world models as the ultimate path to generalization and scaling. Today, that architecture is reshaping AV and all of physical AI, including robotics. Enjoy the show.

• 1:32Speaker 02

  Alex, thanks for joining us on the show.

• 1:34Speaker 00

  Hey, Pat. Hey, Sonia.

• 1:36Speaker 02

  One of the things that is very special about your company is that it sort of typifies AV 2.0, meaning a new architectural approach that I think is kind of demonstrated to be superior to the AV 1.0 approach that people toiled with for so many years. Can we just start by defining, what was AV 1.0? What is AV 2.0?

• 1:58Speaker 00

  For sure. When we started the company in 2017, the opening pitch in our C deck was all about the classical robotics approach at the time was to take a perception, planning, mapping, control, essentially break down the autonomy problem into a bunch of different components and largely hand engineer them. And our pitch was that, okay, we think that the future of robotics is not going to be a system that's hand engineered to drive with a lot of infrastructure like high definition maps. But instead, we thought that the future of robots would be intelligent machines that have the onboard intelligence to make their own decisions. And of course, the best way we know how to build an AI system is with end-to-end deep learning. So for the last 10 years, we've been

• 2:39Speaker 00

  promoting an approach, a next generation approach, AV 2.0, that replaces that stack with one end-to-end neural network. Now, of course, that may seem more obvious today, but it was, it has been contrarian for many, many years. But I think today it's maybe unfair to make that, just make that basic distinction because of course, you know, anyone who's worth a grain of salt will use deep learning in various parts of the stack. But what you see in more incumbent solutions to autonomous driving is of course, deep learning for perception and maybe for each different component, but still a lot of hand interfaces, still a lot of infrastructure on high efficient maps and perhaps, you know, reliance on a lot of hardware. So our solution is, is, you know,

• 3:23Speaker 00

  still somewhat moved on. But today, rather than just being an end-to-end network, today, of course, we start to talk about foundation models. We talk, start to talk about more of a general purpose intelligence, one that can understand not just how to drive that car, but many cars with different sensor architectures, with different use cases. And so really, it all boils down to how do we build the most intelligent robot that can scale without needing onerous infrastructure?

• 3:49Speaker 02

  So Wave is sensor inputs, motion output.

• 3:54Speaker 02

  Gigantic neural net in the middle.

• 3:56Speaker 00

  That's right, at a very, very, very, um, uh, uh, simple level. But some of the interesting things you see that are maybe different from the story we've all heard with large language models is with autonomous driving, of course, there are some, some interesting new factors. One is, uh, of course, safety. The system we need to make sure is safe by design. And what that means is that, you know, we can't just pump more data in and, and hope that hallucinations go away. But we need to design a, a, a architecture that is still end to end data driven, but is, um, is both functionally safe and we can build a, a robust, uh, behavioral safety case. Um, so that introduces some interesting, um, uh, architectural challenges. And then, of course, we also need to run real time onboard a, a robot onboard a vehicle.

• 4:41Speaker 00

  Uh, and so dealing with the onboard compute and onboard sensor, uh, limitations, uh, make, make it an interesting challenge. But yes, it's the same narrative we're seeing playing out in robotics that we've seen play out in all these other AI fields, like, like language or game playing agents. It's that an end to end data learned solution is out competing anything we can hand code. And, uh, what we're excited to be pioneering is that the exact same narrative here in, in robotics and autonomous vehicles.

• 5:05Speaker 02

  And when you guys started this in 2017 and it was a very contrarian approach, when people from the industry said, well, that'll never work because, how did they finish that sentence?

• 5:18Speaker 00

  Uh, I could count hundreds of those meetings. Uh, yeah, uh, typical arguments were, look, uh, it's not safe. Um, it's, uh, it's, it's not interpretable, can't understand what it's, what's doing, or, uh, even simply, um, uh, it, it doesn't make sense. We, we haven't heard of this, this AI thing. Um, and look, I think five, 10 years ago, it was probably reasonable to say end to end deep learning wasn't interpretable. But I, I don't think that's true today. I think today we have a lot of really great tools for understanding and responding to, to insights about, um, the way these, um, uh, deep learning systems reason. Uh, but moreover, I think if you have the ambition to build any intelligent machine,

• 5:59Speaker 00

  um, I think it's naive to think you can build a complex intelligent machine and actually make it, um, you know, let's say strictly interpretable to the point where you can point to a single line of code or a single thing that, that causally made the outcome occur. The the beauty of intelligent machines is that they are so wonderfully complex. And there, I think the way that we're going to not just design them, but understand them is through a data-driven structure.

• 6:27Speaker 01

  Can you say more about the before and after of the AV one dot of stack and the millions, billions of lines of code that goes into those systems versus the two dot O systems today? And how how quickly is that changing? Because my sense is that deep learning, large neural nets hitting hitting the physical economy is a much more recent phenomenon than people might appreciate.

• 6:51Speaker 00

  Well, especially when you think about the path to distribution and deploying these systems. I mean, the automotive industry has just gone through a seismic shift in bringing out software-defined vehicles and the right hardware on these cars to be able to make them drive. Maybe one common point of debate is is a camera only or camera radar, LiDAR as a as a sensor approach to autonomy. And just to be clear on on our position, Wave, we we want to build an AI that can understand all kinds of different sensor architectures. There's going to be sometimes where a camera-only solution makes sense, sometimes where camera radar, LiDAR. And we train our our embodied AI model on all of those permutations from very diverse data sources. And the car we just drove in is a camera-only stack.

• 7:34Speaker 00

  And we've got other cars that we work on with partners that have radar and LiDAR. And of course, there's there's different trade-offs that you you you take there. But more generally, we're seeing mass-produced cars from the best manufacturers around the world have a GPU onboard, have surround camera, surround radar, and sometimes a front LiDAR. And what's beautiful about that is there's now the opportunity to see this AI come out and benefit people around the world. I think that kind of software-defined infrastructure is happening in automotive, has perhaps not yet happened to the same degree in other robotics verticals, but I'm sure the market's going to move that way as well. And in in general, having the right level of computing infrastructure in a scalable way and opening up these platforms to AI, I think, is is what's really making this possible.

• 8:18Speaker 00

  And that's gone through a tipping point in the last couple of years.

• 8:21Speaker 01

  And, you know, your your perspective of AV two dot O has flipped from contrarian to, I'd say, consensus. Maybe in the last two or three years, was, do you think it was FSD twelve that did it? Or what when did that mindset start to shift?

• 8:36Speaker 00

  I miss the contrarian day, but even today, I still, I was in a conversation this morning where I still see a lot of, a lot of folks still say, yes, we need end-to-end AI. They've brought the, you know, the big tech narrative around the future of AI. But they say things like, we need end-to-end AI with, with hard constraints or with safety guarantees. And, and still there's, there's, there still can be some, you know, belief that some hybrid approach is the way to go where, where you want to try and, try and take a rules-based stack and an end-to-end learn stack. But often these approaches can get the worst of both worlds or just add costs and complexity. So, you know, I still think there is a distribution in the market of those that are leaning in and moving

• 9:18Speaker 00

  fast and those that are, you know, are perhaps, you know, have some, some catching up to do. But of course, crediting the, the breakthrough that all of us that have been working in deep learning that, that really made this world-changing and mainstream, of course, we've got to credit the large language model breakthroughs. I think they've, they've inspired the world and opened up the market's mind to be curious about this technology.

• 9:42Speaker 00

  But also what we've been doing at Wave, you know, a year ago, we were just driving in central London. Central London, I think, is a great proving ground because it's this unstructured, incredibly complex and dynamic city that our AIs learn to navigate around very smoothly, safely and, and reliably. But in the last year, we've taken it to highways, to Europe, Japan, North America. Our cars were in New York City last week, driving around there. And so bringing it global, being able to take it to different manufacturers' vehicles and show a product-like experience. This, this growth is, I think, also really, really opened up a lot of inspiration around, around the world.

• 10:23Speaker 01

  Why is it that you're able to launch in hundreds of cities worldwide? And, you know, some of the AV1.0 companies need to actually go out and build, build an HD map. Just say a word on, on the difference in how technical differences are actually leading to differences, how the machine's able to learn and how you're able to roll out.

• 10:42Speaker 00

  Autonomous driving is all about generalization. Generalization means being able to reason about or understand something you've never seen before.

• 10:51Speaker 00

  Every time you go for a drive, you're going to see something new for the first time.

• 10:56Speaker 00

  What did we see today? We saw a road worker rolling out some carpet thing in front of the road, but you know, on a pedestrian crossing, but not wanting to step out. And we had to reason about, could we pass them without yielding, for example? You know, there's just an example from earlier today, but you could think about all the new things you see on the roads every time you drive. You're never going to see every experience in your training data.

• 11:18Speaker 01

  Yeah.

• 11:18Speaker 00

  So that means that you have to be able to reason and generalize to things you haven't seen before to be safe, to be useful around the world. And that's what has motivated our entire approach. So whether it's us, a manufacturer giving us one of their vehicles, and within a couple of months, us being able to drive it on the road. A couple of weeks ago, in September this year, we unveiled a vehicle to media with Nissan in Tokyo. Just four months earlier was the first time we'd even driven in Tokyo and got hands on this vehicle. Four months later, we were having media drive in the car, experience it. And that was a new country and a new vehicle for us. So what that showed is that our AI was able to generalize. It's trained on very diverse data from around the world. It's trained on

• 12:03Speaker 00

  diverse sensor sets, vehicles. And so it was able to understand that vehicle's new sensor distribution, and of course, the complexity of driving around in central Tokyo. So I think that's a really great demonstration of generalization. And if we think about, you know, if you're building a vertically integrated robotic solution, maybe you can go deep. But our ambition is to be the embodied AI foundation model for all of the best fleets and manufacturers around the world. And, you know, to do that, unless we want to overload the company by building a separate neural network for each application, we need to be able to generalize. We need to be able to amortize our cost over one large intelligence, and to be able to very quickly adapt to each different application that our customers care about. That's what we're trying to push.

• 12:46Speaker 01

  You mentioned reasoning in there in terms of how the model is reasoning through, you know, this construction worker, what do I do now? In the LLM world, obviously reasoning is its own. Separate track of lots of scaling and print time computes techniques. Are you deliberately training your models to reason as an emergent property emergent behavior of the models? Like, say more about what you mean about reasoning.

• 13:09Speaker 00

  We are, and I think reasoning in the physical world can be really well expressed as a world model. In 2018, we put our very first world model approach on the road. It was a very small 100,000 parameter neural network that could simulate a 30 by three pixel image of a road in front of us. But we were able to use it as this internal simulator to train a model based reinforcement learning algorithm. There's a fun blog post if you want to see the history on that. But fast forward to today, and we've developed a Gaia. It's a full generative world model that's able to simulate multiple camera and sensors in very rich and diverse environments.

• 13:52Speaker 00

  You can control it and prompt the different agents or scene in it. And and that's an example of reasoning where we can train in the ability to simulate how the world works and what's going to happen next. What happens when you bring this kind of representation on the road is you get some really nice emergent behavior. Like today, when we saw we were driving around unprotected turns that were occluded, you saw the car nudge forward until it could see for itself and then completed the turn. Or when it's foggy in London, you see the car slow down and and, you know, drive to what it can reason about. And by training it with that level of understanding, it gives that that level of emergent behavior that helps it really understand particularly complex multi-agent scenarios.

• 14:34Speaker 00

  I think that's key for for getting safe and smooth autonomous driving.

• 14:39Speaker 01

  So the world models are really key to teaching the model how to reason through their new scenarios.

• 14:44Speaker 00

  Hundred percent.

• 14:45Speaker 02

  You mentioned earlier the diversity of your data. Say a word about where all the data comes from.

• 14:50Speaker 00

  It's it's becoming like an enormous amount of data because, of course, unlike the language domain or image domain, when we're dealing with, you know, a typical self-driving car that has. A dozen multiple megapixel cameras, radar, maybe a LiDAR, you know, you're dealing with when you aggregate that up, it's very quickly tens or hundreds of petabytes of data. So it's it's an enormous amount of data you have to train on, but it's the diversity that's really key. And we've solved for diversity in two ways. The first one is by becoming a trusted partner across the industry and aggregating data across many different sources. From dash cams to fleets to manufacturers to to robot operators.

• 15:32Speaker 00

  And the second one is being able to filter and and really understand the data. Here, we've really worked hard to develop different unsupervised learning techniques to be able to cluster and find unusual or anomaly experiences. And of course, find the scenarios that our system is is performing poorly at and then drive the learning curriculum on those. But yeah, today we we learn from a diverse set of of vehicles, a diverse set of sensor architectures of countries. And that's that's really one of the key things that drives the level of generalization.

• 16:07Speaker 01

  Does the increased growth of world models and, you know, simulated data, does that mean that you just don't need as many actual on-road miles?

• 16:16Speaker 00

  I I think there's two sides to to to that question, right? On the one side, yes, efficiency, learning efficiency really matters. The second, you you can't only rely on learning efficiency. At the limit, if we take our current approach and just scale it up, I'm I'm sure it'll produce generic level five driving. You know, at the limit, if you have unlimited training data, this is really just a lookup data table with some some prior experience. But that's not economically or technically feasible. And so the question is, how can you train this to be the most efficient, data efficient system? Because I think efficiency will lead to not just improved costs, but faster time to market and and more intelligence. So efficiency comes from a number of different factors.

• 16:59Speaker 00

  There's most importantly, how the the data curriculum you put in place. But then the the learning algorithms, how do you magnify the learning you have? And I think world models are a really great opportunity for that. They generate synthetic data and synthetic understanding that doesn't replace real world data, but it recombines it and magnifies it in new ways. It lets you pull in interesting insights and. And I think these kind of approaches can can really, really improve data efficiency. But across the board, I think working under resource constraints has forced our team to to develop so many innovations. I'd also call out just the the workflow, because in traditional robotics, when you're tuning parameters

• 17:39Speaker 00

  or algorithms or designing geometric maps and things like this, you know, there's very well established cultures and workflows. But, you know, our team, when we have fifty model developers working on one main production model, or when we have an end to end net that we need to understand and introspect, or even the way that we deploy these systems to simulation or to the road and and and feedback. We've developed the entire culture from the ground up at Wave has been developed for embodied AI, for end to end deep learning, for driving, the data infrastructure, the simulation, the the safety licensing before we put systems on the road. This is, you know, this has not been a hedge or a side bet for us, but this is the entire essence of our culture.

• 18:19Speaker 00

  And I think doing this under resource constraints and doing this with full mission driven conviction has has led to a bunch of interesting innovations that, look, getting to where we are today, everything is about iteration speed.

• 18:31Speaker 01

  Yeah.

• 18:32Speaker 02

  Speaking of your culture, I'm picturing, you know, a bunch of AI research types, machine learning engineers, that sort of thing. How does the culture of your organization differ from similar applied lab type environments? Given the customer base that you serve, given that you're going after the automotive industry specifically, with all of its quirks around, you know, supply chain and all of its requirements around safety. And so how does that influence the culture of your business?

• 19:03Speaker 00

  Hugely. In fact, you know, for the first few years of Wave, you know, we were really a a group of passionate embodied AI researchers. But in the last couple of years, I'm really, really proud of how our team has built out deep expertise in understanding the automotive industry, but also the ability to reliably deliver to our partners there. And that's a that's a different culture. It's a culture I've really grown to respect, because when you're building millions of cars, you know, the level of reliability and MTTF you need there is is extraordinary.

• 19:38Speaker 02

  What have you all learned from them? I mean, I'm sure part of your job is to teach them about what's going on in the world of AI. What have you learned from them?

• 19:46Speaker 00

  I think some of the main things I call out have been efficiency and reliability. The difference between technology and a product would be some of the the main themes. I mean, the the level of reliability required, but also the level of of quality that is seen to really robustly prove these systems out before deployment and the pride that these these companies take in that has been exceptional. Another thing has been perhaps the the sense of brand differentiation and the desire for, you know, look, do you do you want your car to to to drive? How can your driving person personality really match the brand's preferences? How can you provide that that experience that really gives gives brand differentiation?

• 20:31Speaker 00

  And the great news is that I think we've been able to riff and brainstorm off these and come up with some some really neat technical ideas down down, you know, down that vein. But yeah, ultimately, safe, high quality and personalizable AI has been some great feedback we've we've got from the industry.

• 20:50Speaker 01

  Can you talk about your path to market actually in partnering with the auto OEMs? How did you decide to do that? And then how do you think the market landscape will play out for how autonomy rolls out?

• 21:00Speaker 00

  Yeah, of course. Great, great question, Sonia, because since the beginning of Wave, we've been focused on the pitch I gave around end-to-end deep learning being the approach to autonomy. But we've tried a number of different go-to-market approaches over the years. But in the last couple of years, I've been hugely energized about working and partnering with the the biggest and best automotive, consumer automotive manufacturers around the world. Why is that? Well, I mentioned how they've begun to introduce software-defined vehicles, so they have the infrastructure to work with autonomy. There's the market belief that this is a technology that can can really thrive. And also, it's the chance to get to scale far beyond what we're seeing with the the city-by-city robotaxis we're seeing right now.

• 21:45Speaker 00

  But moreover, these are OEMs that are investing in the right infrastructure to go from not just driver assistance, but to eyes-off autonomy, where you can actually, you know, take liability for the drive and give the user a safe and give them time back from their driving experience. So that's awesome. I think when you think about the market, you know, there are 90 million cars built each year. And so, you know, some manufacturers that are building the autonomy systems themselves, like Tesla, build a couple of million, but the vast majority of the market, I think there's an opportunity to partner, to work with some of these innovative platforms and to bring our AI to market to make these autonomous products possible. And it will only grow from there. These manufacturers don't want

• 22:26Speaker 00

  to stop at driver assistance. We're working together to build eyes-off and driverless robotaxi products. But the key thing is that by avoiding retrofitting our own hardware on these vehicles, by putting them in natively as a software integration, we can move fast at scale, we can build low-cost vehicles that can be homologated all around the world. And this is going to be the path to see tens and hundreds of thousands of robotaxis rolled out around the world at an affordable price. And of course, this is all possible because of the level of generalization that this AI enables.

• 22:59Speaker 01

  Tesla FSD is just such a game-changing product. And, you know, my friends don't have it. It's just they can't imagine driving it the other way. And so it's really cool that you're going to empower, you know, the 88 million other vehicles sold every year to be able to sell that experience as well.

• 23:14Speaker 00

  Hundred percent. It's one of those things that a lot of people would jump in our car and come for a drive with some being skeptical about autonomy, but without exception, they step out with a smile on their face. It's a magical experience. And yeah, I can't wait for people to be able to try it around the world and make autonomy, not just a robotaxi tourism experience, but bring this experience to people in, yeah, eventually every city.

• 23:39Speaker 01

  What do you make of the sensor fusion confusion debate? You know, the one that plays out on Twitter every year or so of Tesla gets confused if there's both camera and and lidar coming in. Sorry, radar.

• 23:52Speaker 00

  I think it's the wrong debate to be having. It's not the frontier question. The industry is is really, I guess, outside of Tesla, is really coalesced around a common architecture of a surround camera, surround radar and a front facing lidar stack. Now, this costs under $2,000, so it's automotive grade components, not the retrofit robotaxi components you see today. But having a frontier GPU compute, automotive grade GPU on the car and that kind of sensor architecture is a really great platform to to build L3, L4 autonomy, eyes off or driverless. It gives you the necessary redundancy. It lets you deal with edge cases that, you know, cameras alone, I agree, they can get you to human level, but we want to go beyond human level.

• 24:34Speaker 00

  And so I think this kind of architecture is is affordable, scalable. It's got the supply chain for mass manufacturer. And and it can eliminate, I think, you know, eliminate all accidents and and really drive superhuman levels of performance. So that's that's what we're seeing many manufacturers bring out on their vehicles and where we're integrating our AI. Of course, for a driver assistance system, camera only can work for a human level driverless system or, of course, I should I should clarify, 90 something, you can look at different stats, but 95% or or above accidents unfortunately are caused by human error. So not only can you be human level, but you can eliminate a lot of human attention and and accidents caused by that.

• 25:18Speaker 00

  But there are still accidents that to be able to solve would require perception capabilities that go beyond beyond vision. And if we want to tackle that long tail, there are many ways to solve it. One of the ways would be to bring in some other sensing modalities like radar and lidar. So, you know, we're excited to be working with those kind of platforms, but crucially, natively integrated into into the OEM's vehicles themselves.

• 25:41Speaker 01

  Is it the same neural net that can drive on one OEM's car and another's car? And how does how does that even work? Because I imagine each vehicle has, you know, slightly different positioned cameras, things like that.

• 25:51Speaker 00

  It comes from the same family. So we we train a very large scale. We regularly train very large scale models. Of course, we iterate them on them month on month. But you know, that's one model that's common to all of the fleets that we work with. But as you optimize to a specific sensor set or a specific embedded target, of course, you can start to specialize the model. But the beauty is that 99% plus of the cost and the time and the effort is training their base model. And then we can build very efficient personalization to the specific customer. And so this lets us the scale, but gives us the ability to squeeze it to very efficient real time platforms and make it adapted to a specific, a specific use case.

• 26:36Speaker 01

  Are you gonna let Pat personalize a super aggressive driver model? I'm gonna need to.

• 26:41Speaker 00

  What driving style would you like, Pat?

• 26:42Speaker 02

  Yeah, pretty aggressive. Safe, very safe, but you know.

• 26:47Speaker 00

  We can do that. We, yeah, we find, it's really funny when you, when you build distributions around driving behavior. Yeah, you can, you can really tell from the human training data we have, you can really tell when it goes from being helpfully assertive, let's say, to unhelpfully aggressive. And we can, we can draw a clean line there.

• 27:05Speaker 02

  There you go.

• 27:06Speaker 00

  What about you, Sonia? How was the drive we just had?

• 27:08Speaker 01

  Fantastic. It was, it was, it was comfortable. It was, it was safe. It was, and it felt very human, actually. Like the way it was kind of nudging up when it couldn't see on the turn. It was very human.

• 27:18Speaker 00

  Yeah, well, it's, as, you know, complex as we can get in Silicon Valley, but come to Tokyo or London, or I was in the weekend in downtown San Francisco. And yeah, you really need the ability to predict and reason about other folks around you to be able to drive in a human like way. And what we find is that if you're not able to smoothly go around double parked vehicles or deal with other dynamic obstacles, or even the prevailing row of traffic might not be aligned to the specific lane, but maybe there's a human like way of driving, then you know, what's, what's awesome about the intelligence that we've built is it's able to reason about these things and keep the traffic flowing, keep interacting with road users in a very human like way. I think this is going to

• 28:00Speaker 00

  be key for societies to accept and love robot taxis. I can't wait to make that a reality.

• 28:05Speaker 02

  Are there any specific corner cases that your cars have a hard time with today?

• 28:10Speaker 00

  There are loads and it's it's uh uh really hard to generically talk about one because that's that's so rare. Yeah. Uh um you know, if I was, it's very hard to say, oh, it's always these types because it's always, you know, a corner case is a couple of edge cases coming together in a corner and it's it's always confounding factors when you get something really obscure. But we're driving, we've driven in over five hundred cities this year. Uh and so when you're driving at that level of scale, of course, you see things that you've never seen before. Road signs are written in a new language. Um actually, maybe one way to to break it down is often we talk about driving broken down into safety, utility and flow. Yeah. Safety being, of course, um safety critical behavior, flow being the style of driving. Is it smooth? Is it um enjoyable?

• 28:53Speaker 00

  Uh and then utility being the navigation and road semantics. Yeah. Safety and flow were found generalized exceptionally well throughout the world. We get almost uniform metrics in every country we operate in in terms of safety and uh um flow or uh uh comfort of the drive. But utility has been the really interesting one as we've gone global. How do you navigate? How do you deal with road signs? How do you read different languages? Hmm. Um how do you deal with different driving um cultures? Uh and so that's the one that's been interesting. But from uh uh we we we published some some results about this. When we went from the UK to the US, we needed uh hundreds of hours of data to be able to drive, um you know, within in ten percent of our frontier performance.

• 29:33Speaker 00

  Um but then when we went to uh uh Europe and to Germany, of course, we'd already learned to drive on the right side of the road. Coming to the US, we'd learn to do right turns at red lights. Then coming to Germany, uh we had to learn to still drive on the right side of the road. But of course, you can't turn right at a red light there. But then on the autobahn, you'd like this, uh you need to drive. We drive today up to a hundred and forty. Uh so uh so pretty fast there. But um yeah, uh uh it gets more efficient each time with exponentially less data in each new market because you've seen some of those things before. Yeah.

• 30:05Speaker 01

  You you mentioned at the beginning that large language models were part of what flips your approach from. From contrarian to consensus, are you integrating large language models at all into into your models? And I know some of the robotics companies that are getting started now are starting from this VLA, VLM base. Is that part of your architecture?

• 30:22Speaker 00

  Hundred percent. In 2021, we started working on language for driving. I remember my team came to me at the time and said, hey, we should start a project on language. I said, no, no, no, guys, startups are all about focus. Keep focus. But they actually gave some pretty compelling arguments. So we started to play around with these things. And a year or so later, we released Lingo, which is the first vision language action model in autonomous driving. And what was special about this model was it could not only drive a car, see the world, drive a car, but also converse in language. And it'll let you talk to it, ask it questions, you know, what are you finding this risky? What's going to happen next? Or even it could commentate your drive. And what's interesting about this is that,

• 31:04Speaker 00

  so there's a few benefits. One is bringing language into pre-training, of course, just improves the representation's power, gives, you know, more interesting information to learn from than than just imagery alone. But then second, aligning the representation with language opens up a ton of interesting product features. It enables a, you know, you to create a chauffeur experience where you could actually talk to your driver. You know, no longer do you need a PhD in robotics to understand the system, but actually you can just talk to it and like ask it to drive. Pat, if you want to, if you want to race around the commute super fast, then you can, you can demand that. But then third, it gives you a really nice introspection tool where you can start to actually, you know, you could imagine regulators or our engineering

• 31:47Speaker 00

  team converse with the system and language to really diagnose why it's doing what it's doing or get it to explain its reasoning. So I think these are really clear benefits, which we're really excited to be pushing.

• 31:56Speaker 01

  That's super cool. And you're running it on the embedded compute?

• 32:00Speaker 00

  We are. So we've put out demos that run offboard. Onboard's challenging with what's in the automotive market today, but some of the next generation compute, for example, the Nvidia Thor that our next gen development vehicle is going to be with, will be large enough to run it onboard. It's going to be cool.

• 32:14Speaker 01

  Very cool.

• 32:15Speaker 02

  You've talked about how autonomous driving sort of provides a path to more generalized embodied AI. Can you paint that picture for us, how you go from autonomous driving to humanoid robots, or whatever other things you might want to embody AI?

• 32:33Speaker 00

  I think we're going to be in the future, looking at a ton of interesting use cases for robotics. What we're seeing is that mobility is becoming possible, I think, much before manipulation. Manipulation is challenging in terms of access to data, global supply chains for hardware, and actually even the hardware designs themselves. Like, I think tactile sensing is still a really hard challenge. But inevitably, it'll be a massive transformative thing. But maybe, you know, maybe is it the maturity of where self-driving was in 2015? But today, you know, our system is rapidly becoming a general purpose navigation agent, giving it an arbitrary sensor view and a goal condition, it's able to produce a safe trajectory.

• 33:17Speaker 00

  So I think we're going to see a rapid advancement from not just consumer automotive, robot taxis, you can think about trucking and other applications. But, you know, this AI will enable manufacturers and fleets who want to build robots in any kind of mobility application. And of course, we're really excited to be working with frontier developers and applications over time as you go out across that robotics stack. And I expect we'll see more maturity in the coming years from manufacturing and manipulation use cases as well. But in the end, I think the benefits of having a large foundation model that certainly in automotive, we can, I think we have access to the largest robot and data supply chain.

• 34:01Speaker 00

  And so we're really lucky in that regard to be able to push forward the intelligence there. But generalizing that intelligence to new applications, I think there'll be benefits from the model being able to experience multiple different verticals, and it'll only make it more general purpose.

• 34:15Speaker 00

  Any applications you're excited about?

• 34:16Speaker 02

  I mean, I'm psyched to have humanoid robots walking around.

• 34:19Speaker 00

  Yeah, me too. I think I think they're going to be neat. You know, whichever form factor, I think humanoids will play a big part. I think other forms of locomotion as well. And then manipulation, there's some really interesting challenges in those space. But I think the same story is going to play out. You know, a working on a narrow application. You know, like when self-driving went to Phoenix, Arizona and put in a ton of infrastructure and expensive hardware to make it work is going to, I think, have limited runway. But working on general purpose, lean, low-cost hardware stacks that really focus on making the system most intelligent and robust. You know, I think this is the recipe for scale. So yeah, let's let's watch that space.

• 35:02Speaker 02

  Yeah.

• 35:03Speaker 01

  Do you think there are major research breakthroughs needed to reach kind of physical AGI, so to speak? And if so, what do you think is the most promising direction?

• 35:13Speaker 00

  Absolutely, I do. I think there's so much more ground to scale up the current approaches and we'll do that. But I think we will get compounding returns from, I always usually talk about four factors that that drive performance. There's of course data and compute, but then also the the algorithmic capabilities and the embodiment, the what is the hardware and capability on the on the robots? And I think we need to push all four. And on the algorithmic side, there are so many opportunities for for growth. I think a key one is is measurement. How do you actually measure and quantify these systems? How do you respond quickly, find regressions, be able to really have a simulator

• 35:55Speaker 00

  that that closes the the real world gap at scale and can run efficiently? I mean, it's no secret that these generative world models are very compute intensive. But having a good measurement system will just drive efficiency and iteration speed. So that's a that's a key one. People often talk about being a chicken and egg. If you have a perfect simulator, you've solved self-driving and vice versa. And I really believe that. I think AlphaGo showed that when you have a perfect simulator, you can just solve problems through. Monte Carlo tree search. And so I, you know, I, I think that's going to be the case in robotics as well. So yeah, one is, one is measurement. Another pillar is building more generality into the model.

• 36:36Speaker 00

  How can you build out more modalities and align those different modalities in their reasoning? I think this is going to open up new use cases, particularly when it comes to human robot interaction and navigation. I was going back to the utility problem before. Some of these things I'm, I'm, yeah, I'm, I'm super excited about. And then the last one is just engineering efficiency. I mean, training these systems and the data requirements is extraordinary. And so I wouldn't understate, I think the, the most sexy part of this problem is the efficient infrastructure to, to, to train and serve these models.

• 37:11Speaker 02

  Yeah.

• 37:12Speaker 00

  And getting that right, it's, uh, I, yeah, I, I think it's a real competitive advantage or disadvantage.

• 37:19Speaker 02

  We started by talking about AV two dot O. Someday, I imagine we might be talking about AV three dot O. What could AV three dot O look like? If you go five, 10, 15 years in the future, are there any other big leaps in this industry that you think we'll see?

• 37:34Speaker 00

  You, you said that with such deadpan.

• 37:39Speaker 00

  AV, so the, the whole premise of AV two point O was all about putting intelligence on the car and not needing infrastructure and a ton of, you know, a ton of overcooked hardware, but really making the system intelligent. And so I think we're, we're seeing that emerge now with the system that can generalize to the world with all of the onboard scalable intelligence and compute. If I were to speculate where AV three point O, I, I, we haven't thought, we haven't sort of thought about in depth lately. But one idea could be taking the intelligence outside the car. So, I mean, when you start to have. Majority prevailing autonomous vehicles, you could imagine a ton of new things you could do when they start to communicate, when they start to interact with each other.

• 38:22Speaker 00

  You know, why do we need traffic lights in the future if they can coordinate? Why do we need, you know, all these sensors if you can actually just communicate with the AV in front of you to be able to see around corners? I mean, of course, I'm speculating here. It opens up tons of interesting cybersecurity questions, communication latency questions, things like that. But I don't know, I'm all up for embodied AI. And if we can build a safer and more accessible system by taking the intelligence, not only in the car, but beyond, maybe, maybe that's a path, let's see.

• 38:53Speaker 02

  I think that's really interesting. If AV three dot O is the point at which it's sort of a mesh network. And, you know, at that point, maybe humans aren't allowed to drive because they can't communicate with the mesh network the same way that the robots can. Or maybe there are special places that humans go to drive just for recreational purposes. But for transportation, you know, it's all autonomous. Yeah, interesting.

• 39:15Speaker 01

  How do you hire and how do you attract people with how hot the AI market is these days?

• 39:19Speaker 00

  I love that question because, you know, at the end of the day, our team is our product, our team are the most important thing to making this possible. And we talk a lot about at Wave about being a place where you can do the best work of your career. And what that means for me in embodied AI is having a set of colleagues around you that inspire and excite world class in what they do, having the right resources, the right culture unblock you. But I think uniquely at Wave, we are able to bring together really a frontier AI environment with a near term product opportunity in automotive. So if you want to work on intelligent machines and see your system brought out with the scale

• 40:00Speaker 00

  of impact of chat GPT in robotics, I think that's the, this is a place where we can do it. Um. The other thing is is that we've gone global. I mean, we have teams in uh um London, Stuttgart, Tel Aviv, Vancouver, Tokyo, Silicon Valley. Um. And, you know, wherever there's an ama- almost, you know, some of the major AI and automotive hubs. Um. And, you know, we're really looking to to build a global culture that can bring this product to the world, work with customers around the world. Um. Uh. And, you know, and most importantly, collaborate with the the very, very, very best people. Um. Yeah. So anyone who's interested in uh pioneering embodied AI, pushing the frontiers and actually turning it into a game-changing product, um come chat, we'd love to speak.

• 40:45Speaker 01

  Wonderful. Alex, you've believed in the future for end-to-end neural nets and in self-driving and in the physical, physical economy for for longer than just about anybody. And it must be incredibly fulfilling to see that vision start to come to life. Congratulations, and thank you for joining us.

• 41:02Speaker 00

  Thank you, Sonia. Thank you, Pat. It's such a privilege.