LIVE: Google's Jeff Dean on the Coming Transformations in AI

At AI Ascent 2025, Jeff Dean makes bold predictions. Discover how the pioneer behind Google's TPUs and foundational AI research sees the technology evolving, from specialized hardware to more organic systems, and future engineering capabilities.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Hi and welcome to Training Data. We are mixing it up for this week's episodes and dropping a conversation that was filmed live at Sequoia's annual AI conference in San Francisco with Google's chief scientist and AI lead Jeff Dean. Jeff is interviewed by our partner and Google alum, Bill Korn. We hope you enjoy this special conversation with Jeff about the future of model development and compute, whether or not he likes vibes coding, hint, he does, and his expected timelines for a 24/7 software developer agent.

• 0:30Pat Grady?

  We have Jeff Dean. And if you read Jeff's bio, he's run everything at some point in Google, including overseeing the genesis of this industry and the BERT paper that kind of sparked things so many years ago. And we're very fortunate at Sequoia to have our partner, Bill Korn, who spent about a decade before Sequoia running most of engineering at Google with Jeff. And so, please welcome Jeff and Bill.

• 0:56Speaker 02

  Thanks.

• 0:58Speaker 03

  No, thank you. And Jeff, it's great to see you. We got to work together for a few years. And Jeff still is occasionally willing to talk to me, which I'm very proud of.

• 1:10Speaker 02

  We have an occasional dinner, which is great fun.

• 1:12Speaker 03

  Yeah, no, he's now the chief scientist, I think, at Alphabet. So I thought we'd start. Obviously, a lot of the people in the room are excited about AI and what's happened. Google clearly introduced a lot of the tech that the industry's based on, transformers and other things. Where do you see things going these days as you look out, both within Google, but also in the industry as a whole?

• 1:41Speaker 02

  Yeah, I mean, I think this sort of period has been a fairly long time in developing, even though it's sort of come into sort of popular visibility only in the last three or four years. But really, starting maybe in 2012 and 13, people were starting to be able to use these really, you know, at that time, what seemed like large neural networks to solve interesting problems. And the same sort of algorithmic approach would work for vision and for speech and for language. And that was, you know, pretty remarkable and kind of Brought attention to, you know, machine learning as a way to solve those problems rather than sort of more traditional handcrafted approaches.

• 2:24Speaker 02

  And one of the things we were interested in in 2012 even was how can you scale and train very, very large neural networks? So we trained a neural network that at the time was 60x larger than anything else. And we used 16,000 CPU cores, because that's what we had in our data centers, and got really good results. And that really, you know, cemented in our mind that scaling these approaches would really work well. And there's been, you know, a whole bunch of evidence of that and, you know, hardware improvements to help increase our ability to scale to larger and larger models, larger data sets. You know, we had an expression, bigger model, more data, better results, which has been

• 3:09Speaker 02

  sort of relatively true for the last 12 or 15 years. And where things are going, I think, you know, now the models that we have are capable of doing really interesting things. You know, they can't solve every problem. They can solve a growing set of problems year over year because the models get better. You know, we have better algorithmic improvements that show us how to, you know, train larger models with, you know, the same compute cost, more capable models. And then we have scaling of hardware. We have, you know, increasing compute per, per unit of hardware.

• 3:51Speaker 02

  And also we have, you know, reinforcement learning and post-training kinds of approaches that are making the models better and sort of guiding them into the ways that we want them to behave. And that's really exciting, I think. You know, multimodality is another big thing, like having, you know, the ability to put in audio or video or images or text or code and have it sort of output all those kinds of things as well is, is pretty, pretty useful.

• 4:18Speaker 03

  The, the industry is, I think mesmerized by agents right now. How, how real do you think agents are? I know Google introduced an agent framework. Some of this stuff, not Google's necessarily, but some of the agent stuff seems to be a little bit vaporware to me. So.

• 4:35Speaker 02

  Yeah.

• 4:36Speaker 03

  Sorry, folks, I'm a little direct, as some folks will tell you.

• 4:40Speaker 02

  It's all good. I mean, I think there's a lot of promise there because, you know, we I do see a path for agents with, you know, the right training process to eventually be able to do many, many things in the virtual sort of computer environment that humans can do today. You know, right now, they can sort of do some things, but not most things. But the path for increasing the capability there is, you know, reasonably clear. You get more reinforcement learning going. You have more agent experience that it can learn from. You have, you know, early nascent products that can do some things, but not most things, but

• 5:20Speaker 02

  are still incredibly useful for people. And I think similar things will happen in sort of physical robotic agents as well. Like, right now, we're probably close to making that transition from, you know, robots in messy environments like this room kind of don't quite work today, but you can see a path where in the next few year or two, they'll start to be able to do 20 useful things in this room. And that will introduce, you know, pretty expensive robotic products that can do those 20 things. And then, learning from experience, they will then get cost engineered to now have something that's 10 times cheaper and can do, you know, 1,000 things. And that's going to engender even

• 6:04Speaker 02

  more cost engineering and more improvement in capability. So, it's exciting.

• 6:09Speaker 03

  It is. And it does seem like it's coming, even though it's very rare today. But they, I guess, one of the other things that comes up, I think, with a lot of young companies is what's happening with the large models. I mean, clearly, you know, Google has Gemini 2.5 Pro and Deep Research and so forth. And then there's OpenAI and a number of other players. I think there's an open debate about how many large language models, open source, closed source, where are things going? How do you think about that? Obviously, Google has a strong position and wants to, I'm sure, dominate in that area, but. How do you see the landscape?

• 6:52Speaker 02

  Yeah, I mean, I think, uh, clearly there it it takes quite a lot of investment to build the absolute cutting-edge models. Um, and I think there won't be 50 of those. Uh, there may be like a handful. Um, and, uh, there are an awful lot of, you know, once you have those capable models, it's possible to make much lighter weight models that can be used for many more things because you can use techniques like distillation that, uh, I was a co-author on and got rejected from NERIP's 2014 is unlikely to have impact.

• 7:27Speaker 03

  I've heard that. I've heard that technique may have helped DeepSeek, so.

• 7:32Speaker 02

  So that that's a really nice, uh, technique. If you have a better model and then you can put it into a smaller scale thing that actually is pretty lightweight and fast and all the kinds of properties you might want. Um, so, uh, I mean, I think there will be quite a number of different players in this this space because, you know, different shape models or models that focus on different kinds of things. But I also think, you know, a handful of really capable general purpose ones will will do pretty well.

• 8:03Speaker 03

  Fair enough. I guess hardware is the other thing that's interesting. It looks to me like every large player is building their own hardware. Obviously, Google has been very public about the TPU program, but Amazon has their own, rumors are Meta has one, rumors are OpenAI is building one. You know, there's lots of, uh, hardware and and yet the industry seems to only hear about Nvidia.

• 8:30Speaker 02

  Oh.

• 8:31Speaker 03

  Fairly. I'm sure that's not true in your office, but I, how how do you, how do you think about that? Uh, how how important is specialized hardware for this stuff?

• 8:42Speaker 02

  Yeah, well, I mean, it's it's very clear that having hardware that is focused on sort of machine learning style computations. And, you know, I like to say accelerators for reduced precision linear algebra are what you want, and you want them to be better and better generation over generation, and you want them to be connected together at large scale with super high-speed networking so that you can, you know, uh, spread your model computation out over as many, you know, compute devices as possible. I think it's super important. You know, I helped bootstrap the TPU program in 2013 because it seemed obvious we would want a lot of compute for inference at that time. That was the first generation. And then the next generation of TPUs, TPUV2, was focused

• 9:25Speaker 02

  on both inference and training because we saw a big need there. And I think we're on now, we stopped numbering them for some annoying reason. So now it's, we're on Ironwood, which is coming out any day now, and Trillium before that.

• 9:41Speaker 03

  Be careful. That sounds like an Intel chip naming strategy, which hasn't worked that well.

• 9:49Speaker 02

  Small edit distance from my chenium, which is a little

• 9:50Speaker 03

  Yeah, I know. I guess, going a little bit off topic, and then maybe we'll open to questions from folks in the room. I have a lot of friends who are physicists. They were a little surprised when Jeff Hitten and his colleagues won the Nobel in physics. I guess, how do you see AI, you know, some of the physicists I know are sort of offended that a non-physicist is starting to win Nobel prizes. How far do you think AI is going to go in various fields at this point?

• 10:31Speaker 02

  Pretty far, I think. I mean, also this year, my colleague Demis and John Jumper won it for

• 10:38Speaker 03

  I almost forgot that.

• 10:39Speaker 02

  Yes, yes. Double Nobel Prize celebration, Monday and Tuesday, or whatever it was. So, I mean, I think that's a sign that really AI is influencing lots of different kinds of science. Because, you know, at its core, you know, can you learn from interesting data? And a lot of parts of science are about making connections between things and understanding them. And if you can have AI-assisted help in doing that, you know, one of the things I've seen in many different fields of science is, many disciplines often have incredibly expensive computational simulators of some process, like weather forecasting is a good example, or

• 11:23Speaker 02

  You know, fluid dynamics or quantum chemistry simulations. And often what you can do is use those simulators as training data for a neural net. And then build something that approximates the simulator, but now is 300,000 times faster. And that just changes how you do science, because all of a sudden, well, I'm going to go to lunch and screen 10 million molecules. That's now possible, instead of, you know, I would have to run that for a year on computer I don't have. And I think that that just kind of fundamentally changes your, your, your process of what you, how you do things and will make faster discoveries.

• 12:00Speaker 03

  I, I think it's probably the most interesting, if there are questions from the audience at this point. I have other questions for Jeff, but.

• 12:10Speaker 04

  Well, actually, just to quickly follow up on the, you know, Jeff Hinton famously left Google after it's like studying, I guess, the effects of, or the differences between digital and analog computing as a future platform for inference and learning. And I'm wondering, is the future of inference hardware analog?

• 12:33Speaker 02

  It's definitely a possibility. I mean, I think like analog has some nice properties in terms of it being very, very power efficient. You know, I think there's a lot of room for digital things to be much more specialized for inference as well. So, and it's a little bit easier to work with typically. But, you know, I think there is a general direction of how can we make inference hardware that is, you know, 10, 20, 50, 1,000 times more efficient than what we have today. And that seems eminently possible if we put our minds to it. It's actually something I'm spending a bit of time on, so.

• 13:17Speaker 05

  Hi. I was just gonna ask about developer experience versus hardware. I think the TPU hardware is extremely impressive, but there's a lot of, you know, in the zeitgeist about how CUDA or different like, you know, technologies are easier to use than the TPU layer. And so I'd be curious for your perspective on that. And is that something you've been thinking about or getting a lot of angry emails about?

• 13:39Speaker 02

  Yeah, I mean, I don't connect with cloud TPU customers all that much, but definitely the experience can be improved. One of the things we started working on in 2018 is a system called Pathways, which is really designed to enable us to take lots of different computing devices and then give sort of a really nice abstraction with those where you have a virtual physical device mapping that is managed by the underlying runtime system. And, you know, we have support for that for both PyTorch and JAX. We primarily use JAX in-house. But what we have is a single JAX Python process just looks like it has 10,000 devices on

• 14:23Speaker 02

  it. And you just write your code as you would as an ML researcher, and off you go. You know, you can prototype it with four, or eight, or 16, or 64 devices, and then you change a constant, and you run against a different pathways back in with 1,000, 10,000 chips, and off you go. Like, our largest Gemini models are trained with a single Python process driving the entire thing with, you know, tens of thousands of chips, and it works quite well. So, pretty good developer experience, I think. One thing I would say is, to date, we had not offered that to cloud customers, but we just announced at CloudNext that we're now gonna have pathways available for cloud customers. So then everyone else can have

• 15:03Speaker 02

  the delightful experience of a single Python process with thousands of devices attached.

• 15:13Speaker 02

  And I agree, that's a much better experience than managing like, 64 processors for your 256 chips. Like, why would you want to do that?

• 15:26Speaker 06

  I love using the Gemini API. And it would be even easier if it would got one API key rather than like the Google Cloud credential setup. Do you guys have a plan to unify the Google Cloud Gemini stack with the Gemini project like setup right now that's more for testing stuff?

• 15:47Speaker 02

  Yeah, I think there's a bunch of streamlining that is being looked at. It's a known problem, not something I spend a lot of time on personally, but I know like Logan and others on the developer side are aware of this friction. We'd like to make it frictionless to use our bottles.

• 16:05Speaker 07

  Is that working? Okay. So it's an interesting time in computing. You've got the confluence of Moore's Law and denHard scaling being completely dead with AI just scaling like crazy. You have a pretty unique position in the world of driving these supercomputers and infrastructure that is being built. And you know like how to map the workloads onto these things, which is a unique sort of skill. What do you think the future of computing is going to look like? What is the computing infrastructure heading towards like from an asymptotic thought experiment level?

• 16:44Speaker 02

  Yeah, I mean, it's really clear that we will have dramatically changed the kinds of computations we want to run on computers in the last say five years, 10 years. And that was like initially a small ripple, but it's pretty clear now that you want to run incredibly large neural networks at incredibly high performance and incredibly low power. And you also want to train them. Training and inference are pretty different kinds of workloads. So I think it's useful to think of those two as, you know, you probably want different solutions for the two or somewhat specialized solutions.

• 17:27Speaker 02

  And I think you're going to see all kinds of adaptation of compute platforms for this new reality that you really just want to run incredibly capable models. And so some of that will be in low-power environments like your phone. Like you'd like your phone to run incredibly good models with lots of parameters, super fast, so that when you talk to your phone, it just talks back to you and it can help you do all kinds of things. You're going to want to run these on robots and autonomous vehicles. You know, we already do somewhat, but even better hardware for that will make those systems much easier to build, much more capable, you know, physical agents in the world.

• 18:11Speaker 02

  And then you want to run them at incredibly large-scale and data centers. And then you also then want to use lots of inference time compute for some kinds of problems, but not others. So you have problem, you know, it's pretty clear you want to use 10,000 times as much compute for some problems as for others. And that's a nice new scaling knob we have that can make your model much more capable or give you, you know, much better answers or make the model capable of doing things with that much compute that it can't do with, with, you know, 1X as much compute. But you shouldn't spend 10,000 times as much compute on everything. So how do you make your systems work well for that? And I think that's a combination of hardware, system software, you know, model and algorithmic

• 18:55Speaker 02

  tricks, distillation, all these things can help you make amazing models come to life in small compute footprints.

• 19:03Speaker 03

  One thing I've noticed is the computer science, at least traditionally, you know, when people are studying algorithms and computational complexity, it was all op-count based. And I, I think as people are rediscovering hardware and details of hardware and system design, I think one of the things that's come back into focus is you need to think about network bandwidth and memory bandwidth and so forth. And so I think a lot of the Kind of traditional algorithmic analysis needs to be completely rethought just because of realities of what real computation looks like.

• 19:42Speaker 02

  Yeah, one of my office mates in grad school did his thesis on like cache aware algorithms because the the order of magnitude, big O kind of notation didn't account for the fact that some operations are 100x worse than others.

• 19:55Speaker 03

  Yeah, no, that's right.

• 19:56Speaker 02

  And I think in in modern ML computing, you care about data movement at the incredibly small level, like moving things from SRAM into accumulators costs you some tiny number, some tiny number of picodules, but it's way more than the actual operation costs you. So it's important to have picodules at the tip of your tongue these days.

• 20:20Speaker 03

  The I one other quick question. Do you vibe code?

• 20:30Speaker 02

  I've been trying it a little bit. It actually works surprisingly well. Yeah, I mean, we've had some nice, we have like a little demo chat room. Actually, we have a lot of chat rooms. We sort of run Gemini via chat room. So I'm in like 200 chat rooms. And when I wake up and brush my teeth, I get like nine notifications because my London colleagues are busily doing things. Like we had one where people can send out cool demos of things I've seen and one that was particularly cool was you feed in a YouTube educational oriented video. And the prompt is just something like, please make me an educational game that

• 21:12Speaker 02

  uses graphics and interactivity to help illustrate the concepts of this video. And, you know, it doesn't work every time, but 30% of the time you get something that's actually kind of cool and related to differential equations or traveling to Mars or, you know, doing some kind of cell aspect thing. And, you know, that's just an incredible sign for education. Like the tools we now have and will have in the next few years really have this amazing opportunity to change the world in so many positive ways. So I think we shall remember that is is kind of what we should be striving for.

• 21:50Speaker 03

  Would you mind passing there and then maybe there?

• 21:53Speaker 08

  Yeah, we'd love to hear your thoughts about the future of search and especially given Chrome has such big distribution, right? And then especially Chrome already know the credentials like payments and then web signing credentials. Have you thought about like getting Gemini just directly into Chrome? You know, making the Chrome app Gemini app instead of have a separate app? You know, I say this because I'm long-term Googlers, so just, you know, think about...

• 22:26Speaker 02

  Yeah, I mean, I think there are definitely lots of interesting downstream uses one could make of the core sort of Gemini models or other models. You know, one is can it help you do stuff in your browser or on your full computer desktop by observing what you're doing and, you know, doing OCR on tabs or maybe it has access to the raw tab contents? You know, that seems like it'll be incredibly helpful and, you know, I think, you know, we have some early work in this area that we've published public demos of in video form that seem pretty useful, things like Mariner and things like that, so TBD.

• 23:09Speaker 03

  You pass the...

• 23:10Speaker 09

  Jeff, question for you. So, thank you for your comments, very insightful. You know, earlier you mentioned like, you know, the number of sort of foundational model players, you know, will likely only be a handful and, you know, this is largely because of, you know, kind of the infrastructure costs and sort of the scale of investment to sort of remain at that cutting edge. And so, sort of as this like battle for the frontier unfolds, like, how do you see the... like, where do you see this end game going? Like, you know, where's like the... where does this lead us? Like, is it just whoever writes the biggest check to build the biggest cluster wins? Or is it, you know, better... you know, you just talked about like better utilization of like unified memory optimization

• 23:54Speaker 09

  and sort of, you know, different, you know, efficient uses of what you already have? Or, you know, is it the consumer experience? Or like, how... where does this all... where does this arms race lead us?

• 24:06Speaker 03

  Isn't it just whoever gets to Skynet first, the game's over?

• 24:11Speaker 02

  Yeah, I mean, I think it's going to require really good insightful algorithmic work as well as really good systems, hardware and infrastructure work. I don't think either one of those is more important than the other because of what we've seen in, say, our Gemini progression to from generation to generation is the algorithmic improvements are as important or maybe even more so than the hardware improvements or the more, you know, larger amount of hardware we're putting to the problem. But both are incredibly important. And then I think from a product standpoint, you know, what it's there's sort of early

• 24:55Speaker 02

  stage products in this space, but I don't think we've collectively hit on what is the thing that or it's probably going to be many things that become the the daily used products for billions of people, right? I think there's probably some in the educational space or in, you know, general information retrieval that is search like but but sort of taking advantage of the strengths of of, you know, large, large multimodal models. I think probably helping people get stuff done in, you know, whatever work environment they find themselves in is going to be an incredibly useful thing. And how will that get manifested in product

• 25:39Speaker 02

  settings? You know, how do I manage my team of 50 virtual agents that are off doing things? And they'll probably be mostly doing the right thing, but occasionally they'll need to consult, you know, with with me about some choice they need to make. I need to give them a bit of steering. How do I manage, you know, 50 virtual interns? It's going to be complicated.

• 26:03Speaker 10

  Hi, Jeff. Thanks for being here, right here.

• 26:06Speaker 02

  Sorry.

• 26:07Speaker 10

  I literally cannot think of anyone better in the world to ask this question. How far do you believe we are from having an AI operating 24/7 at the level of a junior engineer?

• 26:26Speaker 02

  Not that far. Yeah.

• 26:31Speaker 03

  Is that six weeks or six years?

• 26:36Speaker 02

  Every year in AI seems like a dog, dog seven or something. I will claim that's probably possible in the next yearish. Yeah.

• 26:47Speaker 11

  Hi, you talked about scaling pre-training and now scaling RL. How do you think about like the future trajectory of these models? Will it be, you know, one large model with all the compute? Or a constellation of smaller models that have been distilled from these larger models? Both working in parallel? How do you see, you know, the future landscape?

• 27:08Speaker 02

  Yeah, I mean, I've always been a big fan of models that are kind of sparse and have different parts of expertise in different parts of the model. Because, you know, from our, our weak biological analogies, that's partly how our real brains get so power efficient is, you know, we're 20 watts or whatever, and we can do a lot of things. But our Shakespeare poetry part is not active when we're like worried about the garbage truck backing up at us in the car. And I feel like there's, we do some of that with mixture of expert style models. You know, we did some of the early work in that space where we had like 2,048 experts and showed that it gave you dramatic improvements in efficiency,

• 27:52Speaker 02

  like 10 to 100x more efficient sort of model quality per training flop. And that's, that's super important. But it feels like we're not really fully exploring this space yet. Because right now, the kinds of sparsity people tend to do is incredibly regular. Like, it feels like you want paths through your model that are like 100 or 1,000 times more expensive than other paths. And you want experts or pieces of your model that are tiny amounts of compute and some that are very large amounts of compute. Maybe they should have different structures. And I think you want to be able to extend your model with like new, new parameters or new, new bits of, of space.

• 28:37Speaker 02

  And maybe you want to be able to compact parts of your model, running a distillation process on this piece of it to make it one quarter the size. And then you have some background garbage collection thing that is now like, oh, great, I have more memory to use, so I'm gonna. Put those parameters or put those bytes of memory somewhere else and make more effective use of them somewhere else. And so that, to me, seems like a much more organic, continuous learning system than what we have today. So the only problem with this is what we're doing today is incredibly effective. So it becomes a bit hard to completely change what you're doing to be more like that. But I really do think there are huge benefits to doing things in that style rather than

• 29:22Speaker 02

  the sort of more rigidly defined model that we have today.

• 29:28Speaker 03

  I think one more question and then we'll probably wrap up.

• 29:34Speaker 12

  Hey, I wanted to return to the junior engineer inside a year. I'm curious, what advancements do you think we need to get there? Obviously, just maybe code generation gets better, but like outside of code generation, what do you think gets us there? Tool use, agentic planning?

• 29:48Speaker 02

  Yeah, I mean, I think this hypothetical virtual engineer probably needs a better sense of many more things than just writing code in IDE. Like, it needs to know how to run tests and debug performance issues and all those kinds of things. And we know how human engineers do those things. They learn how to use various tools that we have and can make use of them to accomplish that. And they get that wisdom from more experienced engineers typically, or reading lots of documentation. And I feel like junior virtual engineer is going to be pretty good at reading documentation

• 30:30Speaker 02

  and sort of trying things out in virtual environments. And so that seems like a way to get better and better at some of these things. And I don't know how far it will take us, but it seems like it'll take us pretty far.

• 30:45Speaker 03

  Jeff, thank you for coming and sharing your wisdom.

• 30:50Speaker 02

  Thank you. Great to see you.