Pod Insights a16z Podcast Tesla's Road Ahead: The Bitter Lesson in Robotics About this episode What does Rich Sutton’s "Bitter Lesson" reveal about the decisions Tesla is making in its pursuit of autonomy?
In this episode, we dive into Tesla’s recent "We, Robot" event, where they unveiled bold plans for the unsupervised full-self-driving Cybercab, Robovan, and Optimus—their humanoid robot, which Elon Musk predicts could become “the biggest product ever.”
Joined by a16z partners Anjney Midha and Erin Price-Wright, we explore how these announcements reflect the evolving intersection of hardware and software. We’ll unpack the layers of the autonomy stack, the sources of data powering it, and the challenges involved in making these technologies a reality.
Anjney, with his experience in computer vision and multiplayer tech at Ubiquity6, and Erin, an AI expert focused on the physical world, share their unique perspectives on how these advancements could extend far beyond the consumer market.
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Erin Price-Wright? This is the big race in robotics.
0:03 Anjney Midha? The smarter your brain, so to speak, the less specialized your appendages have to be.
0:08 Erin Price-Wright? AI has pushed every single one of these kind of to its limit into a new state of the art.
0:13 Anjney Midha? The way they're solving precision is instead of throwing more sensors on the car, is to basically throw more data at the problem. Data is absolutely eating the world.
0:22 Erin Price-Wright? What is good enough? We used to have the Turing test, which obviously we're blown past now.
0:28 Anjney Midha? His shorthand for it was like the AWS of AI. He's got this idea of this distributed swarm of unutilized inference computers.
0:36 Erin Price-Wright? Whether that's an oil rig, whether that's a mine, whether that's a battlefield, there's so many different use cases for a lot of this underlying technology that are really starting to see the light of day.
0:49 Anjney Midha? It's basically an if, not a when.
0:51 Erin Price-Wright? An inevitability.
0:54 Steph Smith? Earlier this month, Elon Musk and the team at Tesla held their WeRobot event, where they unveiled their plans for the unsupervised full self-driving Cybercab and Robovan, plus Optimus, their answer to consumer-grade humanoid robots, and also what Musk himself predicted would be, quote, "the biggest product ever of any kind." Now, of course, none of these products are on the market yet, but several demos were on show at the event. Naturally, the response was mixed. Supporters said we got a glimpse of the future, while critics said the details were missing. But in today's episode, we're not here to debate that. What we do want to talk about is what this indicates about the intersection of where hardware and software meet. So what does Rich Sutton's 2019 blog post, "The Bitter Lesson",
1:38 Steph Smith? tell us about the decisions that Tesla's making in autonomy? And how realistic is the quoted $30,000 price range? Also, what are the different layers of the autonomy stack? And where do we get the data to power it? And what does any of this look like when you exit the consumer sphere? We cover all this and more with A16C Partners, Anjuni Meta, and Aaron Price Wright. Anjuni previously founded Ubiquity 6, a pioneering computer vision and multiplayer technology company that sat right at this intersection of hardware and software, and was eventually acquired by Discord. Aaron, on the other hand, invests on our American Dymism team with a focus on AI for the physical world. And if you'd like to dig even deeper here, Aaron has penned several articles on the topic that we've linked in our show notes. All right, let's get to it.
2:26 Steph Smith? As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com/disclosures.
2:56 Steph Smith? So last week Tesla had their WeRobot event, and Musk announced the Cybercab, the Robovan, or as he liked to call it, the Robovan, and the Optimus. You guys are so immersed in this hardware, software world. I'd love to just get your initial reaction.
3:13 Erin Price-Wright? From my perspective, it wasn't that there was anything in particular that was super surprising, but what was exciting as just sort of a culmination of one thing that Elon Musk does really well and Tesla has done really well, which is continue to pour love and energy and money and time into a dream and a vision that's been going on for a really long time, like well past when most financial investors and most people kind of lost the luster of self-driving cars after their initial craze in the mid to late 2010s. And they've just continued to plod along and to continue to make developments. And now we're finally seeing like this glimpse of the future for the first time in a really long time.
3:54 Anjney Midha? I think that's right. I think it was very impressive, but unsurprising.
3:57 Erin Price-Wright? Yeah.
3:58 Anjney Midha? So I think the two schools of thought when people watched the event was, one was absolutely this whole, oh, my God, this is such vaporware. He shared literally nothing on engineering details. What the hell? Come on, give us the meat on timelines and dates and prices. And then the opposing view was like, holy shit, they're still going. They haven't given up on any of this autonomy stuff that he's been talking about for years. And I'm absolutely more empathetic towards the view that it was a lean towards the latter, which is that it's sort of an homage to the bitter lesson. It's sort of amazing blog post that I'm going to do a terrible job of summarizing by this great computer scientist, Rick Sutton, which basically says that over the last 70 years or so of computer science history, what we've learned is that general purpose methods basically beat out any specific methods in artificial intelligence in particular. Basically, the idea that if you're working on solving a task
4:44 Anjney Midha? that requires intelligence, you're usually better off leveraging Moore's Law and more compute and more data. Than trying to hand engineer a technique or set of algorithms to solve a particular task. And broadly speaking, that's been the big grand debate in self-driving and autonomy, I would say, for the last two decades, right?
4:59 Erin Price-Wright? Yes.
5:00 Anjney Midha? It's the sort of general purpose bitter lesson school versus the let's model self-driving as a specific task.
5:06 Erin Price-Wright? As a set of discrete decision-making algorithms, unconnected to each other.
5:11 Anjney Midha? A system to solve, let's say, edge detection around stop signs, right? Where self-driving is a really hard problem. And you could totally say, well, there's so many edge cases in the world that we should map out each of those edge cases. And I think it was an homage to the bitter lesson. So that's what I was most excited about is he did share actually details that their pipeline is basically an end-to-end deep learning approach.
5:31 Erin Price-Wright? Which is incredible and probably true only for the last, my guess is 18 to 24 months.
5:37 Anjney Midha? Right.
5:37 Steph Smith? Yeah. And I mean, in The Bitter Lesson, he also talks about the fact that it's really appealing to do the opposite because in the short term, you will get the benefit, but the broader deep learning approach ends up winning out in the long term. And a lot of people talk about Musk, Musk says it about himself, that the timelines sometimes are off, but he's basically banking on that premise in the long term.
5:59 Anjney Midha? It's basically an if, not a when.
6:02 Erin Price-Wright? An inevitability, yes. And I think the event was the first time that it really did feel, in an emotional sense, for the average American consumer. I'm not talking about the super-duper tech literate people who wanted the details of the underlying models and their weights, but like for the average American consumer, the first time that this version of the future felt like an inevitability.
6:25 Steph Smith? And before we get into maybe the specifics around where else hardware and software are intersecting, I'd love to just talk about that, that average person who's watching, because you guys are meeting with companies and investors and this has been going on for quite some time. So I'm just curious if maybe you noticed anything under the hood or maybe the meta in that announcement or event that maybe the average person watching is, you know, what are they seeing? They're saying things like, oh, maybe it was human-controlled and not like fully AI and device. Or other people are commenting on the fact that these humanoids are shaped like humans. Like, why do we need that?
7:01 Erin Price-Wright? On the topic of humanoids, I think humanoids are a great choice of embodiment for a robot to really emotionally connect to and speak to a human being watching because I can relate to a human form factor. Obviously, we found out that it was teleoperated, which is, in my opinion, Still doesn't take away from like how cool and amazing it was. The human form factor is a way to connect what is happening with robotics to a regular person who is like, okay, yes, I like see myself in that. This looks like Star Wars or some other sci fi movie. In reality, maybe this is like a controversial opinion. I don't see the vast majority of economic impact over the next decade from robotics coming from the humanoid form factor.
7:44 Erin Price-Wright? But that doesn't take away from the power of the symbol of having a humanoid make a drink at this event because it just it like connects back to the sort of science fiction promise of our childhoods getting sort of finally delivered.
7:59 Anjney Midha? The opening sequence, he started with like a sci fi. I think it was a Blade Runner visual and he was like, we all love sci fi and I want to be wearing that jacket that he's wearing in the picture, but we don't want any of the other dystopian stuff. And so that definitely stuck out to me is that he did not start the way he usually does. It's often a technical first sort of story, but he started with a, here's a vision for where I think the world should go. So it was much more Disney-esque in that and it was quite poetic. I think they literally did it on the Warner Brothers studio lot. And so they like recreated a bunch of cities and I think they had onsite at the event, the robo vans taking people around from these simulated cities. There was a sort of theatricality to it all that stuck out to me, which I thought was quite different and I thought it was refreshing because the core problem with this branch
8:42 Anjney Midha? of AI, which is largely deep learning based and bitter lesson based is that it's an empirical field, unlike call it Moore's law, which was predictive, where you basically know if you double that number of transistors, you get this much more performance on the chip. And it's just about pure execution. AI is much more empirical. You don't really know when the model is going to get done training and when it does get training, whether it will converge or not.
9:03 Erin Price-Wright? Or even what does converge mean? Like what is good enough? We used to have the Turing test, which obviously we've blown past now. It's a feeling more than it is a set of discrete metrics that you can really point to.
9:14 Anjney Midha? Right. So it made a lot of sense to me that he's trying to decouple this idea of progress from a specific timeline.
9:23 Steph Smith? I see.
9:23 Anjney Midha? Because I just think we're setting ourselves up for every time you ask a deep learning researcher, so when's that G P T five model gonna show up? It's like the most frustrating question ever, right? Because they don't know, we don't know. And frankly, sometimes they show up earlier than scheduled and show up later. And by the way, you can look at the stock market's reaction. It's a prime example of how people have been so conditioned by, I would say, the Steve Jobsian, Apple-like cadence year on year of like, here's your new iPhone. It's this incremental but predictable, I think, forecasting that the tech industry keeps trying to reward. And I think what he's doing is pretty refreshing, which is saying, look, here's a vision for where we want to go, but it's decoupled. The second thing on the humanoid piece that I was quite impressed by is actually the quality of the teleoperation. So everybody's talking about how, oh, this is fake, this is all smoke and mirrors. It's just people.
10:08 Erin Price-Wright? Teleoperation is really, really hard problem.
10:11 Steph Smith? I was gonna say, why is no one talking about that?
10:12 Anjney Midha? Have you ever tried? I mean, I've tried, unfortunately.
10:14 Erin Price-Wright? It's so hard.
10:15 Anjney Midha? I like, we were at a company two weeks ago and they've got these teleop robots. And the founder was demoing a mechanical arm that he was teleoperating with a gamepad. And he was folding clothes with it. And I was like, oh, that looks simple. He's like, here, try it. It was one of the hardest manipulation things I've ever tried. And by the way, we tried that with, you know, V R headset with six toff motion controllers, almost harder to do. Teleoperating something, especially over the internet, in a smooth fashion with precision is incredibly hard. And I don't think people appreciate the degree to which they've really solved that by a point.
10:49 Erin Price-Wright? Yeah, I was actually really impressed by that. And, you know, I think that there's huge opportunity for teleop in sort of production applications that will have like massive economic benefit.
10:59 Anjney Midha? Right.
10:59 Erin Price-Wright? Even before we have true robots running around managing themselves. Because if you think about, there's all these really hard and really dangerous or hard to get to jobs, or there's labor differentials where it's a lot harder to hire people to do certain things in certain locations. And if we can imagine a future where the teleop that we saw last week at the event is something that's like widely available, like that's incredible. Imagine not having to go and service a power line, but you can actually teleop. A robot to do that for you, but still have the level of sort of human training and precision needed to make a really detailed and specific evaluation. The promise of that is really cool, even before we get to robots. So that was a really exciting.
11:40 Steph Smith? Yeah, it's like a stop along this journey. And so if we talk about that journey, the arc of hardware and software coming together in maybe a different way than we've seen in the past. Just as an example, so Mark famously said software is eating the world. That was in 2011. We're in 2024. And it does feel like the last decade has been a lot of traditional software, not so much integrating with the physical world around us. And so where would you place us in that trajectory? Because we're seeing it with autonomous vehicles, but I get the sense that's not the only place where this is happening.
12:10 Erin Price-Wright? Yeah, this is where I spend 95% of my time in all of these industries that are just starting to see the glimmers of what autonomy and sort of software-driven hardware can bring. What's really interesting is just actually a dearth of skills of people who know how to deal with hardware and software together. You have a lot of people that went and got computer science degrees over the last decade and relatively speaking, a lot fewer that went and got electrical engineering or mechanical engineering degrees. And we're starting to see the rise of, oh, shoot, we actually need people who understand not just maybe how the software works in the cloud with wifi, where you have unlimited access to compute and you can retry things as many times as you want and you can ship code releases all day, every day. But you actually have kind of a
12:55 Erin Price-Wright? hardware deployment where you have limited compute in an environment where you maybe can't rely on wifi all the time, where you have to tie your software timelines to your hardware production timelines. Like, these are a really difficult set of challenges to solve. And right now, like, there just isn't a lot of standardized tooling for developers and how to do that. So it's interesting. We're starting to see portfolio companies of ours across really different industries that are trying to use autonomy, whether it's oil and gas or water treatment or HVAC or defense. They're like sharing random libraries that they wrote to connect to like particular sensor types, because there's not this like rich ecosystem of tooling that exists for the software world. So we're really excited about what
13:40 Erin Price-Wright? we're starting to see emerge in the space.
13:42 Steph Smith? Even Elon said when he's talking about these two different products that he's unveiling, right? Optimus, and then you have the robo-vans or cybercabs.
13:51 Steph Smith? Those seem like two completely different things, but he even said in the announcement, he said, "Everything we've developed for our cars, the batteries, power electronics, advanced motors, gearboxes, AI inference computer, it all applies to both." Right? So you're seeing this overlap.
14:04 Erin Price-Wright? That's super exciting.
14:05 Anjney Midha? When I was watching it, I was just nerding out because my last company was a computer vision, 3D mapping and localization company. So I unfortunately bet spent too much of my life calibrating LiDAR sensors to our computer vision sensors. Because our whole thesis when I started back in 2017 was that you could do really precise positioning just off of computer vision and that you didn't need fancy hardware like LiDARs or depth sensors. And to be honest, not a lot of people thought that we could pull it off. And frankly, I think there were moments when I doubted that too. And so it was just really fantastic to see that his bet and the company's bet on computer visions and a bunch of these sensor fusion techniques that would not need specialized hardware would ultimately be able to solve a lot of the hard navigation problems, which basically means that the way they're
14:47 Anjney Midha? solving precision is instead of throwing more sensors on the car is to basically throw more data at the problem.
14:53 Steph Smith? Yeah.
14:53 Anjney Midha? And so in that sense, data is absolutely eating the world. And you asked, "Where on the trajectory are we of software eating the world?" And I think we're definitely on an exponential that has felt like a series of stacked sigmoids. Often, it feels like you're on a plateau, but a series of plateaus totally make up an exponential if you zoom out enough. And earlier in the conversation, we talked about the bitter lesson. A number of other teams in the autonomy space decided to tackle it as a hardware problem, not a software problem, right? Where they said, "Well..."
15:17 Erin Price-Wright? More LiDAR, more expensive LiDAR.
15:19 Anjney Midha? More GPU, more sensors, right? And Elon's like, "Hmm, you know, actually, I want cheap cars that just have computer vision sensors. And what I'm going to do is use a bunch of the custom, really expensive sensors that many other companies put on the car, which is at inference time, and he's just gonna use them at train time." So Tesla does have a bunch of like really custom hardware that's not scalable, that drives around the world in their parking lots and simulation environments and so on. And then they distill the models they train on that custom hardware to a test time package. And then they send that test time package to their retail cars, which just have computer vision sensors. And the reality is, that's a raw arbitrage, right, between sensor stack. And it allows the hardware out in the world to be super cheap.
16:02 Anjney Midha? The result there is software is eating the sensor stack out in the world. That makes the cost of these cars so much cheaper that you can have a $30,000 fully autonomous car versus 100 plus thousand dollars of cars that are fully loaded with these LiDAR sensors and so on.
16:17 Erin Price-Wright? But I think in order to have the intuition that you can even do that, you really actually have to understand hardware. If you just understand software and hardware is like a sort of a scary monster that lives over here and maybe you have a special hardware team that does it.
16:32 Anjney Midha? Right.
16:33 Erin Price-Wright? It's going to be hard for you to have the confidence to say, no, we can do it this way.
16:37 Anjney Midha? I think you're totally right, which is that the superpower that Tesla has is his ability to go full stack, right? Because a lot of other industries often segment out software versus hardware, like you're saying. And that means that people working on algorithms and the autonomy part just treat hardware as like an abstraction, right? You throw over a spec, it's an API, it's an interface that I program against and I have no idea what's going on.
17:00 Erin Price-Wright? You don't have to worry about the details.
17:01 Anjney Midha? Don't have to worry about it.
17:02 Erin Price-Wright? It doesn't matter.
17:02 Anjney Midha? Which by the way is super powerful. It's unlocked this whole general purpose wave of models like ChatGPT and so on, right? Because it allows people who specialize in software to not have to think about the hardware.
17:13 Erin Price-Wright? It's also what's driven sort of the software renaissance of the last 15 years.
17:17 Anjney Midha? Absolutely, decoupling, right? Composition and abstraction is sort of the fundamental basis of the entire computing revolution. But I think when you're like him and you're trying to bring a new device to market, kind of like what Jobs did with the iPhone, by going full stack, you end up unlocking massive efficiencies of cost. And I think that's what this event may have been lost in the sort of theatricality of it all is the fact that he's able to deliver an autonomous device to retail consumers at a cost profile through vertical integration that would just not be possible if it was just a software team buying hardware from somebody else and building on top.
17:50 Steph Smith? Can we talk about those economics, by the way? Just attacking that head on. Both Optimus and Cybercab were quoted as being under the 30K range. Is that really realistic? And then tied into what you were saying, we see other autonomous vehicles, which are betting more on the LiDAR and the sensors, which also have come down in price pretty substantially.
18:11 Erin Price-Wright? My guess is Elon is backing into the cost based on what people are willing to pay. And he will do whatever it takes to get those costs to line up. I mean, it's the same thing he did with SpaceX. He will operate within whatever cost constraints he needs to operate within, even if the rest of the market or the research community is telling him it's not possible. Obviously, like a 30K humanoid robot is way less than what most production industrial robotic arms cost today, which I think are more in the 100K range for the ones that are used in like the high-end factory. So if he can get it down to 30K, that's really exciting. I also don't necessarily think you need even a 30K humanoid robot to accomplish a wide swath of the automation tasks that would pretty radically transform
18:55 Erin Price-Wright? the way our economy functions today.
18:57 Anjney Midha? Yeah, I think Aaron's right in that there's probably a top-down director, which is do whatever it takes to get into the cost footprint of-
19:02 Erin Price-Wright? This car has to cost 30K.
19:04 Anjney Midha? Right. But I think if you do a bottoms-up analysis, I don't think you end up too far. Because actually, if you just break down the kind of bomb on a Tesla Model 3, you're not dramatically far off from the sensor stack you need to get to a $30,000 car, right? This is the beauty of solving your hardware problems with software is you don't need a $4,000 scanning lighter on the car. So I think on the Cybercab, I feel much more confident that the cost footprint is going to fall in that range because it's frankly, we kind of have at least an ancestor on the streets, right? The thing that gets prices up is custom sensors. Because it's really expensive to build custom sensors in short production runs. And so you either have scale of manufacturing like an Apple, and you make a new CMOS sensor or a new Face ID sensor,
19:49 Anjney Midha? and you get cost economies of scale because you're shipping more like 30 million devices in your first run. Or you just lean on commodity sensors from the past era, and you tackle most of your problems in software, which is what he's doing.
20:01 Steph Smith? And to that point, when he's betting on software, another interesting thing that he announced was really over-specing these cars to almost change the economics potentially, based on the fact that those cars could be used for distributed computing. To your point, Anj, if you put a bunch of really expensive sensors on the car, you can't really distribute the load of that in any other way than driving the car, right? But if you actually have this computing layer, that's again, in his case, he's saying he's planning to over-spec, that actually can fundamentally change like what this asset is. And you kind of saw the same thing even with Tesla's today, where he's talking about this distributed grid, right? Where all of a sudden, these large batteries are being used not just for the individual asset. So do you have any thoughts on that idea or if we've seen that elsewhere?
20:44 Anjney Midha? He was a bit skimpy on details on that. But I think he did say that the AI five chip is over spec. It's probably going to be four to five times more powerful than the HW four, which is their current chip. It's going to draw four times more power, which probably puts it in that like 800 watts this range, which for context, you know, average hair dryer is at about 1800 watts. I mean, it's hard to run power on the edge, but I think what he said was something to the effect of like, your car's not working 24 hours a day. So if you're driving, call it eight hours a day in LA traffic. God bless whoever's having to do that.
21:19 Erin Price-Wright? For real. Hopefully they're using self-driving.
21:21 Anjney Midha? One would hope. Actually, he opened that his pitch with a story about driving to El Segundo and saying you can fall asleep and wake up on the other side. But I think the T-shirt size he gave was about 100 gigawatts of unallocated inference compute just sitting out there in the wild. And I think his shorthand for it was like the AWS of AI, right? He has got this idea of this distributed swarm of unutilized inference computers. And it's a very sexy vision. I really want to believe in it.
21:51 Erin Price-Wright? Ground us on is this realistic?
21:53 Anjney Midha? Well, I know, I think it's realistic for workloads that we don't know yet in the following sense, right? That the magic of AWS is that it's centralized and it abstracts away a lot of the complexity of hardware footprints for developers. And by centralizing almost all their data centers in single locations with very cheap power and the electricity and cooling, these clouds are able to pass on very cheap inference costs to the developer. Now, what he's got to figure out is how do you compensate for that in a decentralized fashion? And I think we have kind of prototypes of this today, like there are these vast decentralized clouds, I think one is literally called vast, of people's unallocated gaming rigs. People have millions of Nvidia 4090 gaming cards sitting on their desks that aren't used. And historically, those have not yet turned into great businesses or high utilized networks
22:37 Anjney Midha? because developers are super sensitive to two things, cost and reliability. And by centralizing things, AWS is able to ensure very high uptime and reliability, whereas somebody's GPU sitting on their.
22:50 Erin Price-Wright? Maybe available, maybe they're driving to El Segundo.
22:53 Anjney Midha? They're right.
22:53 Erin Price-Wright? And there are just certain things, especially with AI models that are hard to do on highly distributed compute, where you actually need good interconnect and you need things to be reasonably close to each other. Maybe in his vision, there's a world where you have Optimus robots in every home and somehow your home Optimus robot can take advantage of additional compute or additional inference with your Tesla car that's like sitting outside in your driveway, who knows?
23:17 Steph Smith? Right. Okay, well, this event clearly was focused on different models that are consumer facing. So again, Cybercab, that's for someone using an autonomous vehicle. Optimus is a humanoid robot probably in your home. But Erin, you've actually been looking at the hardware-software intersection in a bunch of other spaces, right? And as you alluded to earlier, maybe different applications with better economics, at least today.
23:44 Erin Price-Wright? I think long term, there's no market bigger than the consumer market. So everyone having a robot in their home and a Tesla car in their driveway that's also a robot taxi has huge economic value, but that's also a really long-term vision. And there's just so much happening in autonomy that's taking advantage of the momentum and the developments that companies like Tesla have put forward into the world over the last decade that actually have the potential to have meaningful impact on our economy in the short term. I think the biggest broad categories for me are largely the sort of dirty and unsexy industries that have very high cost of human labor, often because of safety or location
24:27 Erin Price-Wright? access, whether that's an oil rig out in the middle of Oklahoma somewhere that's three hours' drive from there, New York City, whether that's a mine somewhere in rural Wyoming that freezes over for six months out of the year, so humans can't live there, and mine, whether that's a battlefield where we're starting to see autonomous vehicles go out and clear bombs and mines from battlefields to protect human life. There's so many different use cases for a lot of this underlying technology that are really starting to see the light of day. So very excited about that.
25:00 Steph Smith? And as we think about that opportunity, you've also talked about this software-driven autonomy stack. So as you think about the stack, what are the layers? Can you just break that down?
25:09 Erin Price-Wright? Yeah, sure. So across whether it's a self-driving car or sort of an autonomous control system, we're seeing the stack break down into pretty similar categories. So first is perception. You have to see the world around you, know what's going on, be able to see if there's a trash can, be able to understand if there's a horizon, if you're a boat. The second is something Ange knows really well, which is location mapping. So, okay, what do I see? How do I find out where I am within that world based on what I can see and what other sensors I can detect? Whether it's GPS, which often isn't available in battlefields or in warehouses, et cetera. The third is planning and coordination. So that's, okay, how do I take a large task
25:52 Erin Price-Wright? and turn it into a series of smaller tasks? So what is more of an instant reaction? I don't have to really think about how to take a drink of water, but I might have to think about how to make a glass of lemonade from scratch. So how do I think about compute across those different types of regimes when something is more of an instinct versus when something has to be sort of taken down and processed into discrete operations? And then the last one is control. So that's like, how does my brain talk to my hand? Like, how do I know? What are the nerve endings doing in order to pick up this water bottle and take a drink out of it? And that's a really interesting kind of field that's existed for decades and decades. But for the first time, probably since the 70s,
26:36 Erin Price-Wright? we're starting to see really interesting stuff happen in the space of controls around autonomy and robotics. And I would say like, all of these are pre-existing areas. None of this is wildly new, but I think in the last two years, especially with everything that's happening with deep learning, video language models, broadly speaking, AI has pushed every single one of these kind of to its limit and to a new state of the art. And there just aren't tools that exist to tie all that together. So every single robotics company, every single autonomous vehicle company is basically like rebuilding this entire stack from scratch, which we see as investors as a really interesting opportunity as the ecosystem evolves.
27:15 Steph Smith? And as you think about that ecosystem, people kind of say that as soon as you touch hardware, you're literally working on hard mode compared to just a software-based business. So what are the unique challenges, even with maybe that AI wave today that's pushing things ahead? How would you break down what becomes so much harder?
27:32 Erin Price-Wright? I think Anj touched on this a little bit before, but the more you can commoditize the hardware stack, the better. So the most successful hardware companies are the ones that aren't necessarily inventing a brand-new sensor, but are just taking stuff off the shelf and putting it together. But still, like tying everything together is really hard. Like, when you think about releasing a phone, for example, Apple has a pretty fast shipping cadence, and they're still releasing a new phone only every once a year. So you have to essentially tie a lot of your software timelines to hardware timelines in a way that doesn't exist when you can just sort of ship whenever you want in a cloud. If you need a new sensor type, or you need a different kind of compute construct, or you need something fundamentally different in the hardware, you're bound by those timelines.
28:16 Erin Price-Wright? You're bound by your manufacturer's availability. You're bound by how long it takes to quality engineer and test a product. You're bound by supply chains. You're bound by figuring out how these things have to integrate together. So the cycles are often just quite a lot slower. And then the other thing is, when you're interacting with the physical world, you get into use cases that touch safety in a really different way than we think about with pure software alone. And so you have to design things for a level of like hardiness and reliability that you don't always have to think about with software by itself. If your ChatGPT is a little slow, it's fine, you can just try again.
28:56 Erin Price-Wright? But if you have an autonomous vehicle that's like driving a tank on a battlefield autonomously and something doesn't work, like you're kind of screwed. So you have to have a much higher level of rigor and testing and safety built into your products, which slows down the software cycles.
29:13 Anjney Midha? The holy grail is sort of general-purpose intelligence for robotics, which we still don't have. When you train a general model, you basically get the ability to build hardware systems that don't have to be particularly customized. And that reduces hardware iteration cycles dramatically because you can basically say, look, roughly speaking, these are the four or five commodity sensors you need. The smarter your brain, so to speak, the less specialized your appendages have to be. And I think what a number of really talented teams are trying to solve today is, can you get models to generalize across embodiments? Right? Can you train a model that can work seamlessly on a humanoid form factor or a mechanical arm, a quadruped, whatever it might be? And I'm quite bullish that it will happen.
29:57 Anjney Midha? I think the primary challenge there that teams are struggling with today is the lack of really high quality data. Right? The big unknown is just how much data, both in quantity and quality, do you really need to get models to be able to reason about the physical world spatially in a way that abstracts across any hardware? I'm completely convinced that once we unlock that, the applications are absolutely enormous, right? Because it frees up hardware teams, like Aaron was saying, from having to couple their software cycles from hardware cycles. It decouples those two things. And I think that's the holy grail. I think what Tesla, the victory of the autonomy team over there, is having realized eight years ago,
30:36 Steph Smith? Yeah.
30:37 Anjney Midha? the efficacy of what we call early fusion foundation models, right? Which is the idea that you take a bunch of sensors at training time and different inputs of vision, depth. You take in video, audio. You take a bunch of different 6D sensors and you tokenize all of those and you fuse them all at the point of training. And you build an internal representation of the world for that model. In contrast, the LLM world does what's called late fusion. You often start with a language model. Let's train just on language data. And then you duct tape on these other modalities, right? Like image and video and so on. And I think the world has now started to realize that early fusion is the way forward. But of course, they have an eight-year head start.
31:15 Steph Smith? Yeah.
31:16 Anjney Midha? And so I get really excited when I see teams either tackling the data sort of challenge for general spatial reasoning or teams that are taking these early fusion foundation model approaches to reasoning that then allow The most talented hardware teams to focus really on what they know best.
31:32 Steph Smith? Where are these companies getting training data? Because you mentioned Tesla, for example, yes, we've had cars on the road, tons of them with these cameras and sensors.
31:40 Erin Price-Wright? I still think that one of the smartest things Elon did was turn on full self-driving for everybody for like a month-long trial period last summer. I have a Tesla and I turned it on for my free month and it was like a life-changing experience using. And I obviously couldn't get rid of it. And so now, not only do I now pay for full self-driving, but I also
32:01 Steph Smith? You're feeding the pipeline.
32:02 Erin Price-Wright? I'm giving him all my data.
32:04 Steph Smith? So to me, that's really clever. And so I'm curious if you talk about some of these other applications. Do they have the number of devices, or in this case, cars for Tesla, capturing this data? Or how else are we going to get this spatial data?
32:19 Erin Price-Wright? This is the big race in robotics right now. I think there are several different approaches. Some people are trying to use video data for training. Some people are investing a lot in simulation and creating digital 3D world. And then there's a mad rush for every kind of generated data that you could possibly have. So whether that's robotic teleoperated data, whether that's robotic arms and offices. Most of these robotics companies have pretty big outposts where they're collecting data internally. They're giving humanoids to their friends to have in their homes. It's a yes and scenario right now where everyone is just trying to get their hands on data, literally however they can.
32:59 Anjney Midha? I think it's the Wild West. But if you're Tesla, then the secret weapon you have is you've got your own factories, right? So the Optimus team has a bunch of humanoids walking around the factories, constantly learning about the factory environment. And that gives them this incredible self-fulfilling sort of compounding loop. And then, of course, he's got the Tesla fleet, like Aaron was saying earlier with FSD. I'm proud to have been a month one subscriber for it. And I'm happy that I'm contributing to that training cycle because it makes my Model X smarter next time around. So the challenge then is for companies that don't have their own full stack, sort of fully integrated environment, right? Where they don't have deployments out in the field. And to Aaron's point, you can either take the simulation route for that and say, we're going to create these sort of synthetic pipelines. Or we're seeing this, yeah, huge build out of teleop fleets.
33:43 Anjney Midha? Like with language models, you had people all around the world in countries showing up and labeling data. You have teleop fleets of people piloting mechanical arms hardware around the world. I think there's an interesting sort of third new category of efforts we're tracking, which is crowdsourced coalitions, right? So an example of this is the DeepMind team put out this, maybe a year and a half ago, robotics dataset called RTX, where they partnered with a bunch of academic labs and said, hey, you send us your data, we've got compute and researchers, we'll train the model on your data and then send it back to you. And what's happening is there's just different labs around the world who have different robots of different kinds. Some are arms, some are quadrupeds, some are bipeds. And so instead of needing all of those to be centralized in one place, there's a decoupling happening where some people are saying, well, we'll specialize in providing the compute
34:28 Anjney Midha? and the research talent, and then you guys bring the data. And then it's a give-to-get model, right? Which we saw in some cases with the internet early on. Nvidia is an example of this, where their research team isn't stacking a bunch of robots in-house, so they're instead partnering with people like pharma labs who have arms doing the bedding and wet lab experiments and saying, you send us the data, we've got a bunch of GPUs, we've got some talented deep learning folks, we'll train the model, send it back to you. And I think it's an interesting experiment, and there's reason to believe this sort of give-to-get model might end up actually having the highest diversity of data. But we're definitely in sort of full experimentation land right now.
34:59 Erin Price-Wright? Yeah, and my guess is we'll need all of it.
35:01 Steph Smith? So it sounds like data is a big gap, and it sounds like some builders are working on that. But where would you guys like to see more builders focused in this like hardware-software arena? Especially because I do think there are some consumer-facing areas where people are drawn to, they see an event like this and they're like, oh, I want to work on that.
35:18 Erin Price-Wright? Yeah, I'm pretty excited about the long tail of really unsexy industries that have outsized impact on our GDP and are often really critical industries, where people haven't really been building for a while. Things like energy, manufacturing, supply chain, defense. These industries that really carry the US economy, where we have under-invested from a technology perspective probably in the last several decades, are poised to be pretty transformed by this sort of hardware-software melding and autonomy. I'd love to see more people there.
35:50 Anjney Midha? I'm very excited for all the applications Aaron talked about, and I think to unlock those, we really need a way to solve this data bottleneck, right? So startups, builders who are figuring out really novel ways to collect that data in the world, get it to researchers, make sense of it, curate it. I think that's sort of a fundamental emitter on progress across all of these industries. We just need to sort of 10x the rate of experimentation in that space.
36:14 Steph Smith? All right, that is all for today. If you did make it this far, first of all, thank you. We put a lot of thought into each of these episodes, whether it's guests, the calendar Tetris, the cycles with our amazing editor Tommy, until the music is just right. So if you like what we put together, consider dropping us a line at ratethispodcast.com/a16z. And let us know what your favorite episode is. It'll make my day, and I'm sure Tommy's too.
36:40 Steph Smith? We'll catch you on the flip side.
Previous The Story of Apple Pay with Jennifer Bailey Next AI, Robotics & the New Global Arms Race Page updated October 26, 2026 · transcript and summary are machine-generated
