NVIDIA's Auto Play and the Future of Autonomous Driving — With Danny Shapiro

Danny Shapiro is the Vice President of Automotive at NVIDIA. He joins Big Technology to discuss the current state of autonomous driving technology, its future, and NVIDIA's role. Tune in to hear how NVIDIA is pushing the boundaries of AI and simulation to make self-driving cars safer and more reliable. We also cover the challenges of full autonomy, NVIDIA's broader play in the automotive ecosystem, and how generative AI is transforming the industry. Hit play for an insider's look at the cutting-edge technology shaping the future of transportation. Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  Nvidia's automotive VP is here to speak with us about the state of autonomous driving and how the latest AI innovations translate to the car. That's coming up right after this. This episode is brought to you by Genesis. What does it actually take to put a Gentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial to go inside a large scale agentic CX transformation, including the business decisions, trade offs, and lessons along the way.

• 0:44Alex Kantrowitz?

  You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay? Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation. Welcome to Big Technology Podcast, a show for cool headed, nuanced conversation of the tech world and beyond. Well, look, the car is a computer, and so of course we got to speak with Nvidia. And we have a perfect person to do it with us here today. Danny Shapiro is here.

• 1:28Alex Kantrowitz?

  He's the Vice President of Automotive at Nvidia, someone I have spoken with before privately, but we're so excited to bring him here and have him share everything that Nvidia is working on with you and then also give a real important picture into the state of autonomous driving, which I think is one of the most interesting tech innovations that we're in the midst of currently. So Danny, great to have you here. Welcome to the show.

• 1:51Speaker 01

  Yeah, Alex, thanks so much for inviting me. Really excited.

• 1:54Alex Kantrowitz?

  So let's talk at the beginning, just looking at the big question. How far are we from fully autonomous driving?

• 2:02Speaker 01

  You know, they're operating on our roads today. There are vehicles, robo taxis, even some trucks that are truly driverless. They've gotten to the point where the safety driver can be removed and they're operating some as part of a revenue program as well for those companies. It's not widespread in the Bay Area where I am. I see them on the roads all the time. I've ridden in them. It's an amazing experience. It really is transformative. But I think we're seeing a lot of the technology come down into consumer vehicles. So you have driver assistance systems, things that are originally based on the technology for self-driving, but we're bringing it into vehicles like Mercedes, like Jaguar Land

• 2:44Speaker 01

  Rover vehicles, like Volvos and a number of other brands all over the world are integrating Nvidia technology to enable much safer driving on the streets.

• 2:54Alex Kantrowitz?

  Yeah, and I had experience in San Francisco last summer where I spent a bunch of time riding in Waymos and then riding in Cruises. And we had Cruise, the Cruise CEO at the time, Kyle Vogt, come on the show and talk about how his plan was to 10x Cruise rides every year going forward. And of course, we know it didn't happen. They ran into some safety concerns and he's out. And it looks like their ambitions have scaled back pretty dramatically, although maybe they'll ramp back up again. But I'm curious to hear your perspective of like, what is the difference between a Waymo and a Cruise? I mean, not specifically like going into the technical details, but you mentioned that we have autonomy already.

• 3:38Alex Kantrowitz?

  So why is it taking us so long to, and I know so long is relative, but so long to have that spread from one company that does it really well to every car through the economy? Is it a safety thing? Is it a cost thing? Like, what is the roadblock now?

• 3:56Speaker 01

  Well, for us, and I believe a lot of our partners, safety is the primary concern. I think if you look back to 2016, when a lot of predictions were made, everyone was talking about 2020 was the year. And so from a compute standpoint, from a software development standpoint, that really looked realistic. I think everyone underestimated the true complexity of being able to make sure you get it right.

• 4:21Speaker 01

  Virtually all the time, and that's what's really challenging. So the basics are easy. When you can drive down the freeway, cars are all going in the same direction. There's no pedestrians, there's good lane markings. That's really a solved problem. But as soon as you really introduce the complexity and the anomalies that come about from human behavior, people either falling asleep on the roads or driving recklessly or being impatient or road rage or whatever it is, it's really hard to predict that and creates hazards for the self-driving car. So I think what we're doing is we're seeing a whole new wave of innovation now. Part of it's based on the same fundamental technology from ChatGPT, these large language models, more of an end-to-end system that's able to

• 5:06Speaker 01

  look holistically at the whole environment around the car and be able to anticipate and predict what other drivers will do and understand how to react. Just like ChatGPT, you can say anything now and it knows how to respond. It's kind of, it's quite amazing. So while ChatGPT is a large language model and using a form of generative AI, that's text in and text out. What we're doing is applying that same type of algorithmic approach and training of the neural networks, really. The technology innovation from that, we're doing with training video in and imagery from cameras or sensor data to be able to understand that environment and then determine what is the best course of action for that vehicle to safely navigate.

• 5:50Alex Kantrowitz?

  Yeah, I was gonna ask you what the path forward is, given that we know that, okay, this is something that's, these models are struggling with. So you think that this is the path forward, that the way that you just described?

• 6:00Speaker 01

  Absolutely. You look at, you look at the innovation that's taking place from, from players out there. Companies like Wave in the UK have put together a really amazing underlying technology that lets the front-facing camera be interpreted in a way that the system could communicate to the occupants of the vehicle what's going on outside the car and also use that then as a way to determine what is the car going to do? How is it going to steer, accelerate, or brake? So it can interpret that video feed and explain that there's somebody jaywalking or somebody ran a red light, or there's a child waiting to cross, or whatever it may be. So this generative AI approach is really, I think, going to accelerate the adoption.

• 6:41Speaker 01

  Very recently, at a conference called CVPR, so that's Computer Vision Pattern Recognition, it takes place annually, last month it was in Seattle, Washington. And there was a competition that they held, so it's a research conference. Over 400 entries into this autonomous driving challenge. And it was basically looking at sensor data and trying to predict the best trajectory for the vehicle moving into the future. Nvidia submitted and won the challenge. Our research team had developed a new large language model, basically end-to-end training of that sensor data system for then controlling the vehicle. And so over 400 entries, Nvidia came out on top with this new large language model type of approach.

• 7:26Speaker 01

  And I think really that's where we're seeing a lot of innovation now. And instead of having a lot of individual neural nets that are trained on lanes and on signs and on pedestrians and all these individual things, a more end-to-end approach looks at the entire environment and can then understand what to do in the case that maybe there are no lane markings. Right? So again, I think as you look at how dynamic the environments are, how there's no standard of streetlights all look the same or signs are all the same or lane markings are all the same. This end-to-end approach is really going to help get us to that point where we don't worry like we used to worry about those end cases that haven't been expressly trained.

• 8:08Alex Kantrowitz?

  I think this is important, what you're saying. So basically, if I have it right, the current set of autonomous driving vehicles, at least the base layer of technology they use, they have a bunch of different, basically, artificial intelligence systems picking out each different feature in the road. And they combine that to eventually make their predictions of where to go. But you're saying that the cutting edge today is not these individual systems, but one system that looks at everything and then is able to predict?

• 8:39Speaker 01

  I think what's really key about being safe is the combination of diversity and redundancy. So you want backup systems, but also you want a variety of different algorithms. So I think we're seeing a layering of different technologies and that these things that you're looking for lane markings, but if you don't find the lane markings, there's an end-to-end network also that's there to guide the vehicle and determine what to do. And we have neural networks now that are looking at signs and can interpret complex signs. So if you're trying to figure out, can I park here or not? The sign net will actually be able to read that sign and understand, is it a Saturday? Is there street sweeping or whatever it is? It has the context. And so the complexity of these networks is quite elaborate as well.

• 9:22Alex Kantrowitz?

  Trust me, I could have been saved a lot of tickets if I had something in that. So what you're saying is basically with, if people have cars with this technology embedded, they'll be able to pull up to the curb and it'll tell you, no, you can't park there?

• 9:36Speaker 01

  That's right.

• 9:37Alex Kantrowitz?

  So it's interesting to me that Nvidia is developing a lot of this technology. Like I went to your automotive section on the website and was like, wow, there's a tremendous amount of models coming out of Nvidia. I thought it was largely the carmakers, like the Waymos or the Teslas that are developing the autonomous technology. So how involved is Nvidia in developing these models itself? And then who's the customers?

• 10:03Speaker 01

  We work with hundreds of automakers, truck makers, robotaxi companies, software startups, the sensor companies, the mapping companies. It really is quite an ecosystem that we've built. We're not creating the vehicles, but we work with those manufacturers. And so we offer the compute hardware, that's our drive platform. So that's the brain that goes inside the car. Our drive OS is the safe operating system that's part of that package. We have a lot of different middleware and libraries that they can use to develop their applications, algorithms, the neural networks.

• 10:36Speaker 01

  That application layer, though, is generally built by our customers. So Mercedes-Benz or Jaguar Land Rover, Volvo, Neo in China. And so they can pick whatever parts of the software stack they want. And in many cases, our customers are taking the full stack and they're developing some of their own algorithms as well. So there might be a pedestrian detection algorithm from Mercedes, running along a pedestrian detection algorithm from Nvidia. And we collaborate on that. So starting through the end of this year, the introduction of the new CLA, it's already been announced from Mercedes.

• 11:14Alex Kantrowitz?

  What's CLA?

• 11:15Speaker 01

  That's the new Mercedes model, the C-Class. And so every Mercedes will be built on Nvidia Drive with the software that we've developed and rolled out by Nvidia. So it starts with that C-Class vehicle and then we'll go through their entire line over time.

• 11:30Alex Kantrowitz?

  Do you sense a time where eventually Nvidia will be able to build, like using some of the technology that you've discussed already, build the software suite that empowers autonomous driving and then just it's plug and play for these auto manufacturers? Is that the goal?

• 11:46Speaker 01

  Well, that's essentially what we're doing. We're making the software available. So we're developing the whole stack. And really, it's a three-computer problem, we call it. So you have the computer we just talked about inside the car. So that's the drive platform. It's a very high-performance, energy-efficient, automotive-grade supercomputer. You plug all the sensors into that and it's purpose-built for a vehicle. And it's going to operate in the heat of the desert sun. It's going to operate in very cold temperatures, you know, in Alaska, unlike your phone, which if you gets too hot or cold, it'll shut off. So we have to make sure that the temperature range will work, that the shock and vibration, the dust environment. So all of that goes into making this computer automotive-grade.

• 12:31Speaker 01

  But then in addition to that, we make the computer that's used to train the artificial intelligence. That's our DGX. That's a supercomputer. And so we have a huge business in terms of the automakers building out their own data centers or using, you know, cloud providers like Azure, AWS, and Oracle to train. And then we also have our OVX. That's our Omniverse computer for simulation. So again, that's another data center solution for first developing and then testing and validating in simulation before the software even goes into the car. So Nvidia is the only company that has these three computers. And it's really this whole life cycle of developing, testing, and deploying the software. And really, it's a continuous

• 13:16Speaker 01

  flywheel, just like your phone gets software updates. All these cars are designed to get software updates and get smarter and smarter over their life.

• 13:24Alex Kantrowitz?

  Yeah, that's wild. You know, Danny, a lot of folks think of Nvidia as just like the chip company for AI training. And I'm always like, it's a little bit more than that. And it's just wild that like, just in this one discipline, it's deeply involved in pushing the cutting edge forward with autonomous.

• 13:45Speaker 01

  Absolutely. I think you're right. People tend to focus on what happens in the car, not realizing there's so much work before you can get to that point and so much development. So what's great about the customers that work with us, it's a single architecture. It's the same chip technology that's in their data center that they're training on, that we do testing. It's called hardware in the loop testing. So we actually test the whole software and hardware that goes in the car. We can test that in the data center first in these virtual environments called a digital twin. So we create a model of a city and we simulate the camera, the radar, the LiDAR signals that are detecting everything happening around the car, motorcycles cutting off vehicles or pedestrians jaywalking.

• 14:30Speaker 01

  And so all of that to be tested before we actually even put it on the road. So it's very efficient. It's very safe to test it that way. And it really helps create a much better product in the end.

• 14:41Alex Kantrowitz?

  Yeah, I'm going to talk to you a little bit more about that in a moment. But first, I want to ask you another technology question because it sort of seems like there's two different schools of thought when it comes to building autonomous cars. One is, I'm just going to call it shorthand, the Waymo school of thought, which is that you need like a gazillion sensors and cameras and your car is going to look like a submarine and it's going to cost hundreds of thousands of dollars, but it's going to work pretty freaking well. And another school of thought, I'm just going to call it the Tesla school of thought, is that you just need a few cameras and then eventually you'll be able to train machine learning models to the point where you can run your Tesla autonomously without a LiDAR.

• 15:27Alex Kantrowitz?

  Who do you think is right?

• 15:30Speaker 01

  I think they both have merits and the reality, as I mentioned before, this diversity and redundancy is really how you get higher levels of safety. So cameras are great, but they don't work in all conditions. And so when you combine radar, when you combine LiDAR, you have the strengths of many different types of sensors and they complement each other. So I think we see in the case of Waymo is really a higher level of security and confidence and that redundancy that comes from the system. And so they're operating fully autonomously with no drivers, whereas I do have a Tesla and it's quite remarkable being camera based, but every once in a while I still need to jump in and grab the wheel.

• 16:13Alex Kantrowitz?

  Right.

• 16:13Speaker 01

  And so it's not there yet. Can it get there? I think it probably can eventually, but it's not there today.

• 16:20Alex Kantrowitz?

  How's the latest full self-driving? Update.

• 16:23Speaker 01

  Like I said, it's pretty good. I rely on it every day. You know, it is. It is still considered beta. And so I'm, I'm watching it. I'm in the industry. So I'm curious about each software rev and what it can do and what it can't do. But it's, it's quite remarkable.

• 16:37Alex Kantrowitz?

  And there was a report in the Wall Street Journal this week that talked a little bit about some of the deficiencies of the way that Tesla operates and points exactly at this issue. Right. That it's decided to not use LiDAR. For instance, some of them have radars. But Elon basically wants to do everything cost as cheap as possible. Right. Seeing it in SpaceX, you're seeing it in Tesla. And it's because these systems are just good enough that people trust them, that we've seen some of the tragedies happen. There was a video that they put out, the Journal put out, that showed effectively a car driving at night. And there was an overturned tractor trailer blocking the road. And because the car's computer vision models hadn't seen effectively like

• 17:21Alex Kantrowitz?

  black, you know, dark underbelly of the, of the truck, dark evening, couldn't pick it up because it hadn't been trained on enough examples of this. And the car went right into the, right into the tractor trailer. But you, but thinking about your answer, you think that this is just a temporary thing? That eventually they'll be able to figure out these sort of outlier scenarios over time?

• 17:47Speaker 01

  I think if, if the models can be able to detect that there's a physical obstacle, even if it doesn't know what it is, then it will be able to take the right action. But again, this is where having the diversity of sensor data becomes a really big differentiator.

• 18:03Alex Kantrowitz?

  Right. Now, talking again about what Nvidia is doing internally, it's pretty wild. So you're actually simulating collisions. Is this something that you do in that sort of world that you talked about? Where you go through these?

• 18:17Speaker 01

  Absolutely.

• 18:18Alex Kantrowitz?

  So talk about it.

• 18:18Speaker 01

  What we try to do is create the scenarios. It's, it's less about creating accidents, but creating the scenarios to ensure that the systems will do, have a safe outcome. There's no way to ensure that there's zero accidents in the world. Right. There's this. There's always can be crazy stuff that's going to happen on the road and no human driver could could avoid something falling right in front of the car or somebody getting pushed in front of a vehicle or something like that. But what we want to do is be able to anticipate all that and be able to avoid it or mitigate what would happen in one of these hazard scenarios.

• 18:56Speaker 01

  One of the things we're able to do actually is record drives that we're taking and then use that as input to create a huge range of different scenarios, permutations on that and test the software. And so we can actually capture cars in a scene and make any one of those cars in the scene the autonomous car and see how it would behave. So we're building a massive database of scenarios and ways to test and validate that the technology is good. The other thing that we can do is we can take accident reports and now using these large language models, we can input these accident reports and be able to create scenarios

• 19:36Speaker 01

  from a text input explaining what happened or if there's a map or something like that.

• 19:42Alex Kantrowitz?

  So then how important is simulation in training, anything having to do with autonomous driving?

• 19:49Speaker 01

  Simulation plays a really, really big role in ensuring the safety of the system. There's really no way that, first of all, driving around, collecting data, you rarely are going to see the dangerous scenarios, the hazards, the things that very rarely occur, you're not going to capture them on your data collection. So we need to use simulation and really what we call synthetic data generation to create those kinds of scenarios. So we can create fake potential hazards, things falling off of trucks, people running across the street at night, whatever it may be, somebody running a red light. And so we can create that data to augment the real data for training the AI.

• 20:31Speaker 01

  And then we can actually simulate all these dangerous scenarios to ensure that the system will do the right thing. And the benefit of using simulation too is repeatable. So we can adjust the software and test something that maybe didn't pass a month ago, but we can run it through the same scenario and see, oh yeah, we fixed that. There may be situations, it often happens that the sensors are blinded by the sun right as it's setting, right? The sun is like coming right into the eyes of the car, into the driver's eyes, into the camera's eyes. And so you only have a few minutes a day where you can actually capture that data as well as test. And so in simulation, it can be sunset 24 hours a day.

• 21:12Speaker 01

  So we really can control that environment and we can create rain, snow, fog. It's really remarkable and it gives us the ability to test those things that again, may never be seen in the real world. So if you're driving a real vehicle trying to test it in autonomous mode, you may never know. In simulation, we can be sure it's going to work before we release it.

• 21:34Alex Kantrowitz?

  And you also have a assistant that you're working on that sort of appears within the car that helps you like navigate tough things. Like for instance, like it will look around, it's called Lada, I believe. And if you're in New York and you want to make a right on right, it will be like, hey, listen, like right on right is illegal here. Or if you're in Mexico and like you're taking a hairpin turn, you know, there's like some traffic patterns that allow you to kind of go outside the lane to take a peek and then come back in. And it will help you figure out what's going on with that. I mean, I'm about to go to Ireland and I would definitely need something like that to tell me what side of the road to be driving on. So talk a little bit about the, the progress there.

• 22:12Speaker 01

  So I think you're referring to a, a video series we put out called Drive Labs.

• 22:16Alex Kantrowitz?

  Right.

• 22:17Speaker 01

  And so what we do is each episode, we sort of pick one little piece of technology that's part of this huge software suite that we're, we're building. And so some of these things come out of our research team. So it might not be in our software today, but it's sort of a preview of what's coming out soon. And so what we're able to do is, is train these systems and create these different. Modules essentially that become part of the software stack that our customers can use. So how they decide to bring it to a customer sort of is the decision of a Mercedes or a Volvo or others. But the technology is there so that we can train it on what are the laws of a particular region. I mean, the signs, the light signals, the lane markings in different regions of the

• 23:00Speaker 01

  country or around the world differ quite a bit. And so basically creating these large language models for each region that understands what you're allowed, what the car is allowed to do and not to. And so it will, it could give you an alert if you try to, if it's been implemented in the car, it could give you an alert if you try to turn right on red in New York, for example.

• 23:19Alex Kantrowitz?

  Yep.

• 23:20Speaker 01

  Perfectly fine to do in California.

• 23:21Alex Kantrowitz?

  Okay, we're going to go to break, but before we do, I got to ask you a stock question. The stock is down like 7% today and 16% on the month. Do you notice that within the company?

• 23:32Speaker 01

  You know, things move around a lot. The market just shifts a lot. If you look at the page often, everything's green one day and then everything's red another day. And I think there's, there's really no way to predict. We'll put out great news and we never know what's going to happen. I think what, what we do know is Nvidia is just creating amazing products. We have really exciting technology we're bringing to market. It's not just in automotive, it's in healthcare, it's in energy, it's in finance. Gaming is still a big part of the company. People don't necessarily talk about it as much, but it's a huge, huge part of our company. It's super exciting. This week going on a SIGGRAPH, which is the big graphics conference annually, and that's something that Nvidia has always been a part of, and that's kind of our core accelerated

• 24:16Speaker 01

  computing for graphics. And now that conference is turning into an AI conference and it's all about robotics. And so Nvidia is still at the center of all these different industries, providing technology, platforms, ecosystems for other companies to build. So I think, I think things are feeling good.

• 24:33Alex Kantrowitz?

  Yeah, I have been tuning into SIGGRAPH this week and it is funny, right? At a gaming conference and I think Jensen is just so happy to be back in the gaming environment because at the start of every session that he's had, he just shouts out, this is my home turf, which is, you know.

• 24:49Speaker 01

  Yeah, I mean, it's so true. Stiggraph is it's just it's where all the graphics researchers kind of unveil the the new technology and graphics is so core to Nvidia, but so many other things, whether it's filmmaking, whether it's manufacturing, whether it's now visualization for, you know, engineering with fluid dynamic simulations or or weather forecasting. We have a system we're working on called Earth Two, which is going to just do amazing things for that for meteorologists and be able to predict weather with incredible detail beyond what has ever been possible and look way in advance. So that has huge implications for just a lot of predicting, you know, natural disasters, for example.

• 25:32Alex Kantrowitz?

  Right. All right, let's take a break, come back. You mentioned robotics. We'll talk about robotics, including how this technology will help translate the state of the art advances to generative AI as well, right? We've learned what generative AI can do to autonomous cars. What can autonomous cars teach generative AI? All right, that's coming up right after this. This episode is brought to you by Avpoint. Everyone's racing to roll out AI right now. Co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data, and most teams have no single way to see it, secure it, and prove it's under control. That's exactly what Avpoint does. For 25 years,

• 26:14Alex Kantrowitz?

  they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate. Your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. Avpoint, the unifying trust layer for AI. Head to avpoint.com to see how enterprises deploy AI with confidence. Learn more at avpt.co/bigtechnologypodcast. That's avpt.co/bigtechnologypodcast.

• 26:53Alex Kantrowitz?

  Violent threats against executives are growing at an alarming rate. When a company experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door. The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a wide-glove enterprise security service, they protect your people with continuous personal data removal, proactive prevention tools, and human-led emergency support. So when someone goes looking for your executives, Ironwell ensures they hit a dead end. So here's what to do. Go to ironwell.com/bigtechnology, fill in the quick form, and request your free risk assessment. Their team will show you just how exposed your executives are

• 27:38Alex Kantrowitz?

  and how to lock it down before a threat reaches their front door. That's ironwell.com/bigtechnology. Learn how to stop online threats before they become real-world attacks. The link's in the show notes.

• 27:53Alex Kantrowitz?

  And we're back here on Big Technology Podcast with Danny Shapiro. He's the VP of Automotive at Nvidia. All right, let's talk about robotics. So to begin with, it seems like you were talking a little bit about how when you're driving around with a car that has all these sensors, you start to begin building a world model. Is that helpful for the development of robotics? And is Nvidia already sort of taking what it's learning from cars and putting it into robots and what it's learning from robots and putting it into cars?

• 28:24Speaker 01

  Absolutely, yeah. We have an entire robotics group that sits alongside our automotive group. And there's a lot of related aspects to the problems. If we think about driving a vehicle, well, the whole thing is we don't want to hit anything, right? We have to understand the environment. We have to sense it. We want to plan how we're going to maneuver. And then we act, we actuate the vehicle and drive it. But we don't want to touch anything. Robotics is almost the other flip side of that. It's like robots, they need to interact. They're going to grab something and they want to touch something, but you have to do it very delicately. But the ability to sense, plan, and act is really the same. So so much of what we're doing is related. You know, a car is really, an autonomous car is really a form of robot. It's got wheels and it drives around.

• 29:09Speaker 01

  Some robots that we're doing in terms of factory robots still might be autonomous machines that are roaming around like at a warehouse or in a factory. Others are fixed. There's an arm that's moving around. I think the key thing is, again, this three computer model of training the systems, of simulating them in digital twins, and then deploying the software is the same in both of these cases. We're doing a lot of work in factories, all types of factories, but also factories to build cars. And so companies like Mercedes-Benz, like BMW, are working with our teams to develop this factory as a digital twin first. So it's a full simulation of the entire factory.

• 29:53Speaker 01

  All the robots, the workers, the assembly lines, the trucks pulling up, the logistics of moving parts around, all of that is modeled and run digitally in simulation before they even build the factory. And so the benefit of that is you're not halfway into construction, you realize, oh, wait a minute, the arm that has to swing here to take the body of the vehicle and rotate it around isn't going to clear the ceiling. We need to raise the roof another two feet, right? You plan all that in advance before you ever build the factory, and you can really optimize your layout. So digital twins, AI are a huge part of planning how robots are actually going to interact. And then we train those robots in simulation so that you can then take that software,

• 30:36Speaker 01

  just like you do in the autonomous car, we load that software into the robot.

• 30:41Alex Kantrowitz?

  Yeah, it is so interesting thinking about it. Like car, your one mission is don't touch anything. And robot, your mission is interact. But I imagine both type of technologies are building models of the world, trying to figure out what's going on. Is there shared, is there like a shared universe that both the robotics and the automotive elements of Nvidia work on?

• 31:04Speaker 01

  Absolutely. There's a lot of shared technology. So this is the strength of Nvidia as a company. We're relatively small company for the impact that we have on industries around the world. So the engineering team that's developing a lot of the core hardware and software is leveraged across groups from automotive to robotics to healthcare. An example is we're developing pedestrian detection algorithm. That same core tech can be used to detect cancer in an X-ray.

• 31:32Alex Kantrowitz?

  How?

• 31:32Speaker 01

  Or a CT scan.

• 31:33Alex Kantrowitz?

  Just computer vision?

• 31:35Speaker 01

  Yeah, well, it's AI, it's deep learning. So the same techniques, it's just different patterns of data, just trained on different types of data, different modalities. But the concepts are essentially the same. And that's where we've been able to go to market in many different industries with the same basic architecture, hardware and software with purpose-built applications or devices. But again, the core technology is leveraged across so many different groups. If you're looking for where to drill for oil, right, you have seismic data that you can then apply deep learning to, to figure out where is a pocket of oil buried miles below the earth. So it's the same, conceptually, the applications are markets are totally different.

• 32:18Alex Kantrowitz?

  Right. Now, let me ask you this. How does the company incentivize these divisions to cooperate? Because I imagine you're like the automotive group and you have one set of goals you're working towards. And they're the robotics group and they have another set of goals they're working toward. Now, if you work together, you could probably both help each other to the point where you're going to both exceed those goals or do better than you would have in a silo. But a lot of companies work in silos, whether it's by design, thinking about Apple, don't, you know, if you're working on Face ID and you're working on automotive road detection, don't talk to each other, you're on two different projects. And maybe, you know, wink, wink, that's why the automotive project within Apple failed. And then other companies incentivize silos just by the incentives in terms of

• 33:03Alex Kantrowitz?

  like your performance review. If you were, you know, coming up short, even though you were collaborating, you came up short, you get this grade, you don't advance. So how does Nvidia address this?

• 33:13Speaker 01

  Yeah, that's, those things you describe are not the culture of our company. I think one of the principles that we're founded on is one team. It's all about Nvidia first. And so the individual group comes second. And in fact, The notion of the group is kind of dynamic and we really don't have much of an org chart in the company. Jensen says the mission is the boss and so we have these virtual teams. There's a lot of cross-functional work that goes on. People have different roles and responsibilities and might be working on a variety of different things. And so it's really all about how does, what's the best thing for the company? And working across groups is really rewarded and in part of the way that

• 33:54Speaker 01

  just the culture is we want to help each other and the whole company succeeds as opposed to, hey, this is my thing. I own this. I'm just going to focus on this. So it really is is part of like just the culture of the company and embraced throughout and Jensen is constantly looking at, okay, if there's, if he finds two different groups that are doing related things, he's like, you guys get together and figure it out together. We don't need to have two separate programs going on here, but let's, let's pick the best. And so there really is a huge collaboration that goes on throughout the company.

• 34:28Alex Kantrowitz?

  So I'm wondering whether another collaboration is going to be or already is yours with the groups that are working on AI models, generative AI models? Because the biggest limitation with generative AI has been that it doesn't really understand the real world. I mean, at least with text. Now, you might push back on me on this, but this is what we've heard on the show. Which is basically like, there's only a small amount of human knowledge that's been codified in text. And the rest is just being out there in the world, interacting, using your eyes, figuring out what gravity is, right? You can't really understand that from, from text. I mean, you can read about it, but you don't really experience it until you're out there in the world.

• 35:12Alex Kantrowitz?

  So I'm curious how these real world interactions, whether it's something like an autonomous car, something like a robot, might be used to advance the knowledge that we have with large language models today.

• 35:25Speaker 01

  I think it's, I think it's happening. I think part of what we do is we're able to model physics, right? So that's a key thing. So we can model gravity. We can model how things interact. We can model, you know, motions of different types of, of materials or fluids or things like that. So it comes down to mathematical modeling of, of the real world and it's a big part of what we do.

• 35:48Alex Kantrowitz?

  Yeah. Have you thought about just like the immense progress that China is making? I mean, what's your view there? I've heard that you can buy electric vehicle in China for $10,000. Whereas in the US, if you could do that, that would be a sea change here. So what's your view there?

• 36:05Speaker 01

  Yeah, we work with a lot of companies around the world. We have a number of customers in China. They're doing remarkable work in terms of the development of EVs, but also driver assistance systems and autonomy with robot taxis. So there it's a big market. It's the biggest in the world and it's a big market for Nvidia. So we work closely. We have teams over there and we work in Japan, in South Korea, throughout Germany, UK, and of course, North America as well.

• 36:33Alex Kantrowitz?

  Is there anything the US can learn from the China market to price our cars better? Or is that?

• 36:39Speaker 01

  I'm not an expert on battery technology, but I think there's certainly, you know, economies of scale and some things they're doing to bring the cost down. I think there's also a lot of government support that they're getting in China as well.

• 36:52Alex Kantrowitz?

  I guess like last question for you. How long do you think it's going to be until anywhere in, let's say, the US, you can open an app and hail a robot taxi?

• 37:03Speaker 01

  Can do it today.

• 37:05Alex Kantrowitz?

  Anywhere, though. I can't do it in New York.

• 37:06Speaker 01

  Oh, anywhere. So I mean, I don't know if it's always going to be anywhere. Right? I mean, it's sort of, there's got to be a market for it. But I think cities will make a lot of sense. The suburbs in some, some areas, yes. In the rural areas, it's going to be a challenge. It's like you can't, you can't get an Uber everywhere today. Right? So you can't get a taxi everywhere. But I think in major markets, it's very soon.

• 37:32Alex Kantrowitz?

  Okay. Can't wait for it. Vane Shapiro, thanks so much for joining. Great to see you.

• 37:36Speaker 01

  Great to see you too. Thanks, Alex.

• 37:38Alex Kantrowitz?

  All right, everybody. Thanks so much for listening. We'll be back on Friday with Ron John Roy breaking down the week's news. We'll see you next time on Big Technology Podcast.