Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI

This week I welcome on the show two of the most important technologists ever, in any field. Jeff Dean is Google's Chief Scientist, and through 25 years at the company, has worked on basically the most transformative systems in modern computing: from MapReduce, BigTable, Tensorflow, AlphaChip, to Gemini. Noam Shazeer invented or co-invented all the main architectures and techniques that are used for modern LLMs: from the Transformer itself, to Mixture of Experts, to Mesh Tensorflow, to Gemini and many other things. We talk about their 25 years at Google, going from PageRank to MapReduce to the Transformer to MoEs to AlphaChip – and maybe soon to ASI. My favorite part was Jeff's vision for Pathways, Google’s grand plan for a mutually-reinforcing loop of hardware and algorithmic design and for going past autoregression. That culminates in us imagining *all* of Google-the-company, going through one huge MoE model. And Noam just bites every bullet: 100x world GDP soon; let’s get a million automated researchers running in the Google datacenter; living to see the year 3000. Watch on Youtube https://youtu.be/v0gjI__RyCY; listen on Apple Podcasts https://podcasts.apple.com/us/podcast/jeff-dean-noam-shazeer-25-years-at-google-from-pagerank/id1516093381?i=1000691556147 or Spotify https://open.spotify.com/episode/4atx1POpKIL8WGvdVfdnbb?si=XYxo6SIyRi2qmZ1ZGfl6vw. Sponsors Scale partners with major AI labs like Meta, Google Deepmind, and OpenAI. Through Scale’s Data Foundry, labs get access to high-quality data to fuel post-training, including advanced reasoning capabilities. If you’re an AI researcher or engineer, learn about how Scale’s Data Foundry and research lab, SEAL, can help you go beyond the current frontier at scale.com/dwarkesh https://scale.com/dwarkesh Curious how Jane Street teaches their new traders? They use Figgie, a rapid-fire card game that simulates the most exciting parts of markets and trading. It’s become so popular that Jane Street hosts an inter-office Figgie championship every year. Download from the app store or play on your desktop at figgie.com https://www.figgie.com/ Meter wants to radically improve the digital world we take for granted. They’re developing a foundation model that automates network management end-to-end. To do this, they just announced a long-term partnership with Microsoft for tens of thousands of GPUs, and they’re recruiting a world class AI research team. To learn more, go to meter.com/dwarkesh https://meter.com/dwarkesh To sponsor a future episode, visit dwarkeshpatel.com/p/advertise https://www.dwarkeshpatel.com/p/advertise Timestamps 00:00:00 - Intro 00:02:44 - Joining Google in 1999 00:05:36 - Future of Moore's Law 00:10:21 - Future TPUs 00:13:13 - Jeff’s undergrad thesis: parallel backprop 00:15:10 - LLMs in 2007 00:23:07 - “Holy s**t” moments 00:29:46 - AI fulfills Google’s original mission 00:34:19 - Doing Search in-context 00:38:32 - The internal coding model 00:39:49 - What will 2027 models do? 00:46:00 - A new architecture every day? 00:49:21 - Automated chip design and intelligence explosion 00:57:31 - Future of inference scaling 01:03:56 - Already doing multi-datacenter runs 01:22:33 - Debugging at scale 01:26:05 - Fast takeoff and superalignment 01:34:40 - A million evil Jeff Deans 01:38:16 - Fun times at Google 01:41:50 - World compute demand in 2030 01:48:21 - Getting back to modularity 01:59:13 - Keeping a giga-MoE in-memory 02:04:09 - All of Google in one model 02:12:43 - What’s missing from distillation 02:18:03 - Open research, pros and cons 02:24:54 - Going the distance This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com https://www.dwarkesh.com?utm_medium=podcast&utm_campaign=CTA_1

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Dwarkesh Patel?

  Today I have the honor of chatting with Jeff Dean and Noam Shazeer. Jeff is Google's chief scientist and through his 25 years at the company, he has worked on basically the most transformative systems in modern computing, from MapReduce, Big Table, TensorFlow, AlphaChip. Genuinely, the list doesn't end. Gemini now. And Noam is the single person most responsible for the current AI revolution. He has been the inventor or the co-inventor of all the main architectures and techniques that are used for modern LLMs, from the transformer itself, to mixture of experts, to mesh TensorFlow, to many other things.

• 0:42Dwarkesh Patel?

  And they are two of the three co-leads of Gemini at Google DeepMind. Awesome. Thanks so much for coming on.

• 0:48Speaker 01

  Thanks for having me.

• 0:49Speaker 02

  Super excited to be here.

• 0:50Dwarkesh Patel?

  Okay, first question. Both of you have been at Google for 25 or close to 25 years. At some point early on in the company, you probably understood how everything worked. When did that stop being the case? Do you feel like there was a clear moment that happened?

• 1:06Speaker 01

  I mean, I know I joined and like at that point, this was like end of 2000 and they had this thing, everybody gets a mentor and you know, so you know, I knew nothing. I would just ask my mentor everything and my mentor knew everything. It turned out my mentor was Jeff. And it was not the case that everyone at Google knew everything. It was just the case that Jeff knew everything. Because he had basically written everything.

• 1:31Speaker 02

  You're very kind. I mean, I think as companies grow, you you kind of go through these phases. Like when I joined, you know, we were 25 people, 26 people, something like that. And so you eventually learned everyone's name and even though we were growing, you kept track of all the people who were joining. At some point, then you kind of lose track of everyone's name in the company, but you still know everyone working on, you know, software engineering things. Then you sort of lose track of, you know, all the names of people in the software engineering group, but you know, you at least know all the different projects that everyone's working on. And then at some point, the company gets big enough that, you know, you get an email that Project Platypus is launching on Friday and you're like, what the heck is Project Platypus? So I think.

• 2:13Speaker 01

  Usually it's a very good surprise. Like you're like, wow, Project Platypus. Like, I had no idea we were doing that. And it turns out.

• 2:20Speaker 02

  Yeah, but I think it is good to keep track of like what's going on in the company, even at a very high level, even if you don't know every last detail. And it's good to know lots of people throughout the company so that you can go ask someone for more details or figure out who to talk to. I think like with one level of indirection, you can usually find the right person in the company if you have a good network of people that you've built up over time.

• 2:42Speaker 01

  How did Google recruit you, by the way?

• 2:45Speaker 02

  I kind of reached out to them, actually.

• 2:48Speaker 01

  And Noam, how did you get recruited? What was it that you did that? I actually saw Google at a job fair in like 1999. And I assumed that it was like already this huge company that no point in joining. Because everyone I knew used Google, I guess that was because I was a grad student at Berkeley at the time. I guess I've dropped out of grad programs a few times. But, but you know, it turns out that like, actually, it wasn't really that large. So it turns out I did not apply in 1999, but like just kind of sent them a resume on a whim in 2000 because I figured I should let it was like my favorite search engine and figured I should apply to multiple places for a job.

• 3:30Speaker 01

  But then, yeah, it turned out to be, uh, be, uh, really, uh, really fun. Looked like a bunch of smart people, uh, doing good stuff. And they had this really nice crayon chart on the wall of the daily, uh, number of search queries that, you know, somebody had just been maintaining. And, um, yeah, it looked very exponential. These guys are going to be very successful. And it looks like they have a lot of good problems to work on. So it's like, okay, maybe I'll, uh, yeah, go work there for a little while and then have enough money to just go work on AI for as long as I want after that. Yeah, yeah. In a way, you did that, right? Yeah, yeah. It totally worked out exactly according to my. So you were thinking about AI in 1999?

• 4:15Speaker 01

  Yeah, this was like 2000. Yeah, I remember in.

• 4:20Speaker 01

  Yeah, in grad school, a friend of mine at the time had told me that his New Year's resolution for 2000 was to live to see the year 3000 and that he was going to achieve this by inventing AI. So I was like, oh, that's that sounds like a good idea. But, you know, then, you know, I didn't get the idea at the time that, oh, like, you could go do it at a big company, but, you know, I figured, hey, you know, a bunch of people seem to be making a ton of money at startups. Maybe I'll just make some money and then I'll have, you know, enough to live on, just work on AI research for a long time.

• 4:58Speaker 02

  Yeah.

• 5:00Speaker 01

  But yeah, it actually turned out that Google was a terrific place to work in AI.

• 5:04Speaker 02

  I mean, one of the things I like about Google is our ambition has always been sort of something that would kind of require pretty advanced AI. You know, organizing the world's information and making it universally accessible and useful. Like, actually, there's a really broad mandate in there. So it's not like the company was going to do this one little thing and stay and stay doing that. And also, you could see that what we were doing initially was in that direction, but you could do so much more in that direction.

• 5:32Dwarkesh Patel?

  How has Moore's Law over the last two, three decades changed the kinds of considerations you have to take on board when you design new systems, when you figure out what projects are feasible? What has stayed, you know, like, what are still the limitations? What are things you can now do that you obviously couldn't do before?

• 5:47Speaker 02

  I mean, I think of it as actually changing quite a bit in the last couple decades. So like the two decades ago to one decade ago, it was awesome because you just like wait and like 18 months later, you get much faster hardware and you didn't have to do anything. And then more recently, you know, I feel like the general purpose CPU based machines scaling has not been as good. Like, the fabrication processes improvements are now taking three years instead of every two years. The architectural improvements in, you know, multi-core processors and so on are, you know, not giving you the same boost that you that we were getting, you know, the, you know, 20 to 10 years ago. But I think at the same time, you were seeing

• 6:32Speaker 02

  Much more specialized computational devices like machine learning accelerators, TPUs, very ML-focused GPUs more recently, are making it so that we can actually get really high performance and good efficiency out of the more modern kinds of computations we want to run that are that are different than a twisty pile of C++ code trying to run Microsoft Office or something.

• 6:58Speaker 01

  Yeah. I mean, it feels like the the the algorithms are following the hardware. Basically, like what's happened is that at this point arithmetic is very, very cheap and moving data around is comparatively like much more expensive.

• 7:16Speaker 02

  Right.

• 7:17Speaker 01

  So pretty much all of deep learning has taken off roughly because of that because it you can build it out of matrix multiplications that are, you know, N cubed operations and N squared bytes of data communication, basically.

• 7:34Speaker 02

  Well, I would say that the pivot to hardware oriented around that was an important transition because before that we had CPUs and GPUs that were not especially well suited for deep learning. And then, you know, we started to build, say, TPUs at Google that were really just reduced precision linear algebra machines. And then, once you have that, then you want to.

• 7:58Speaker 01

  Right. You have to see the insight that seems like it's all about all about kind of identifying opportunity cost. Like, OK, this is something like Larry Page, I think, used to always say, like, our second biggest cost is taxes and our biggest cost is opportunity cost. And if he didn't say that, then I've been misquoting him for years. But but basically, it's like, you know, what what what is the opportunity that you have that you're missing out on? And like in this case, I guess it was that, OK, you've got all of this chip area and you're putting a very small number of arithmetic units on it. Like, fill the thing up with arithmetic units. You could have order of magnitude more arithmetic getting done.

• 8:43Speaker 01

  Now, what else has to change? OK, the algorithms and the data flow and everything else.

• 8:48Speaker 02

  And oh, by the way, the arithmetic can be like really low precision, so then you can squeeze even more multiplier units in.

• 8:54Dwarkesh Patel?

  No, I want to follow up on what you said, that the algorithms have been following the hardware. If you imagine a counterfactual world where suppose that the cost of memory had declined more than arithmetic, or just invert the dynamic you saw over the last decade.

• 9:09Speaker 01

  Okay, data flow is extremely cheap and arithmetic is not cheap.

• 9:14Dwarkesh Patel?

  What would AI look like today?

• 9:16Speaker 01

  That's interesting.

• 9:17Speaker 02

  You'd have a lot more lookups into very large memories.

• 9:21Speaker 01

  Yeah, I mean, I think it might look more like AI looked like 20 years ago, but in the opposite direction. I'm not sure. I guess I joined Google Brain in 2012. I'd left Google for a few years, happened to go back for lunch to visit my wife, and we happened to sit down next to Jeff and the early Google Brain team. And I thought, wow, that's a smart group of people doing something.

• 9:51Speaker 02

  You should think about it, you're real nuts. We're making some pretty good progress here.

• 9:55Speaker 01

  That sounds fun. So, okay, so I jumped back in.

• 9:58Speaker 02

  I ruled him back. It was great.

• 10:00Speaker 01

  To join Jeff. That was like 2012. I seem to join Google every 12 years. I rejoined Google in 2012 and 2024.

• 10:10Dwarkesh Patel?

  What's going to happen in 2036?

• 10:12Speaker 01

  I don't know. I guess we shall see.

• 10:17Dwarkesh Patel?

  What are the trade offs that you're considering changing for future versions of TPU to integrate? How are you thinking about algorithms differently?

• 10:26Speaker 02

  I mean, I think one thing, one general trend is we're getting better at quantizing or having much more reduced precision models. You know, we started with TPUV1. We weren't even quite sure we could quantize a model for serving with 8-bit integers, but we sort of had some early evidence that seemed like it might be possible. So we're like, great, let's build the whole chip around that. And then over time, I think you've seen people able to use much lower precision for training as well, but also the inference precision has gone. People are now using int4 or fp4, which sounded like if you said to someone, like, we're gonna use fp4, just like a supercomputing floating point person in front of you ago, they'd be like, what?

• 11:10Speaker 02

  That's crazy. We like 64 bits in our floats. Or even below that, some people are quantizing models to two bits or one bit. And I think that's a trend to definitely pay attention to.

• 11:22Dwarkesh Patel?

  What bit? Just like a 01?

• 11:23Speaker 02

  Yeah, just a 01. And then you have like a sign bit for a group of bits.

• 11:30Speaker 01

  It really has to be a code design thing because if the algorithm designer doesn't realize that he can get greatly improved performance throughput with the lower precision, of course, the algorithm designer is going to say, of course, I don't want low precision. That introduces risk and then that's irritation. And then if you ask the chip designer, okay, what do you want to build? And then they'll ask the person who's writing the algorithms today, who's going to say, no, I don't like quantization. It's irritating. So you actually need to basically see the whole picture and

• 12:15Speaker 01

  figure out, oh, wait a minute, we can increase our throughput, the cost ratio by a lot, by quantizing.

• 12:24Speaker 02

  Then you're like, yes, quantization is irritating, but your model is going to be three times faster. So you're gonna have to deal.

• 12:31Dwarkesh Patel?

  Through your careers, at various times, you've had sort of an uncanny, you worked on things that have an uncanny resemblance to what is actually what we're actually using now for generative AI. In 1990, Jeff, your senior thesis was about back progression. And in 2007, so this is the thing I didn't realize until I was working for this episode. In 2007, you guys trained a 2 trillion token N-gram model for language modeling. Just walk me through when you were developing that model. Like was this kind of thing in your head? What did you think you guys were doing at the time?

• 13:09Speaker 02

  Yeah, so, I mean, let me start with the undergrad thesis. So I kind of got introduced to neural nets in one section of one class on parallel computing that I was taking in my senior year. And I needed to do a thesis to graduate, like an honors thesis. And so I approached the professor and I said, oh, it'd be really fun to like do something around neural nets. So he and I decided we would, I would sort of implement a couple of different ways of parallelizing back propagation training for neural nets in 1990. And I called him something funny in my thesis, like, like, pattern partitioning or something. But really, I implemented a, you know, model, model parallelism and data parallelism on a 32 cube, 32 processor, hypercube machine.

• 13:50Speaker 02

  You know, in one, you split all the examples into different batches and every model, every CPU has a copy of the model. And in the other one, you kind of pipeline a bunch of examples along to processors that have different, different parts of the model. And, you know, I compared and contrasted them. And it was interesting, you know, I was really excited about the abstraction because it felt like neural nets were the right abstraction that could solve tiny toy problems that no other approach could solve at the time. But, and I thought, oh, you know, naive me, oh, 32 processors will be able to train like really awesome neural nets. But it turned out, you know, we needed about a million times more compute before it really started to work for real problems.

• 14:33Speaker 02

  But then, starting, you know, in the, you know, late 2008, 2009, 2010 timeframe, we started to have enough compute, thanks to Moore's law, to actually make neural nets work for real things. And that was kind of when I sort of re-entered looking at neural nets. But prior to that, in 2007-

• 14:51Dwarkesh Patel?

  So actually, can I ask about this?

• 14:53Speaker 02

  Sure.

• 14:54Dwarkesh Patel?

  First of all, unlike other artifacts of academia, it's actually like a really, like, it's like four pages and you can just like read it and-

• 15:02Speaker 02

  Yeah, it's four pages and then like 30 pages of C code.

• 15:05Dwarkesh Patel?

  But it's like, it's just like a well-produced sort of artifact. And then, yeah, tell me about how the 2007 paper came together.

• 15:11Speaker 02

  Oh, yeah. So that, we had a, we had a machine translation research team at Google, led by Franz Akk, who had joined Google maybe a year before, and a bunch of other people. And every year, they competed in a, I guess it's a DARPA contest on translating a couple of different languages to English, I think, Chinese to English and Arabic to English, I think.

• 15:34Speaker 02

  And the Google team had submitted an entry. And the way this works is you get like, I don't know, 500 sentences on Monday and you have to submit the answer on Friday. And so I saw the results of this and we'd won the contest. And by a pretty substantial margin, measured in BlueScore, which is like a measure of translation quality. And so I reached out to Franz, the head of this winning team. I'm like, this is great. When are we going to launch it? And he's like, oh, well, we can't launch this. It's not really very practical because it takes 12 hours to translate a sentence.

• 16:14Speaker 02

  I'm like, well, that seems like a long time. How could we fix that? So it turned out, you know, they had not really designed it for high throughput, obviously. And so it was doing like 100,000 disk seeks in a, in a large language model that they'd, they'd sort of computed statistics over. I wouldn't say trained really. And, you know, for each word that it wanted to translate. So like, obviously doing 100,000 disk seeks is not super speedy. But I said, okay, well, let's, let's dive into this. And so I spent about two or three months with them designing an in-memory compressed representation of n-gram data.

• 16:57Speaker 02

  And, and we were using, and n-gram is basically statistics for how often every n-word sequence occurs in a large corpus. So you basically have, in this case, we had like two trillion words. And most n-gram models of the day were like using two grams or maybe three grams. But we decided we would use five grams. So how often every five-word sequence occurs in basically as much of the web as we could process that, that in that day. And then you have a data structure that says, okay, you know, I really like this restaurant occurs, you know, 17 times in the web or something. And so I built like a data structure that would let you store all those in memory on 200 machines.

• 17:42Speaker 02

  And then have sort of a batched API where you could say, here are the 100,000 things I need to look up in this round for this word. And it would give you them all back in, in parallel. And that enabled us to go from taking a night to translate a sentence to basically doing something in, you know, 100 milliseconds or something.

• 18:00Dwarkesh Patel?

  There's this, there's this list of Jeff Dean facts, like Chuck Norris facts.

• 18:09Dwarkesh Patel?

  Like, for example, that for Jeff Dean NP equals no problemo.

• 18:15Dwarkesh Patel?

  And one of them, it's funny because now that I hear you say it's like, I think it's kind of true. One of them is the speed of light was 35 miles an hour until Jeff Dean decided to optimize it over a weekend. Just going from 12 hours to 100 milliseconds or whatever. It's like, I gotta do the orders of magnitude there, but.

• 18:33Speaker 02

  All of these, all of these are very flattering. They're pretty funny. They're like an April Fool's joke gone awry by Michael H.

• 18:43Dwarkesh Patel?

  Okay, so obviously in retrospect, this idea that you can develop a latent representation of the entire internet through just considering the relationships between words is like, yeah, this is, this is large language models. This is Gemini. At the time, was it just a translation idea? Or did you see that as being the beginning of a different kind of paradigm?

• 19:07Speaker 02

  I think once we built that for translation, the, the serving of large language models started to be used for other things like completion of, you know, you start to type and it suggests like what completions make sense.

• 19:20Dwarkesh Patel?

  Right.

• 19:21Speaker 02

  So it was definitely the start of, of a lot of uses of language models in, in Google. And, you know, Noam has worked on a number of other things at Google, like spelling correction systems that use language models.

• 19:32Dwarkesh Patel?

  Right. Yeah, I think, yeah, but that was like 2000, 2001. And there, I think it was just all in memory on one, on one machine.

• 19:40Speaker 02

  Yeah, I think it was one machine.

• 19:41Dwarkesh Patel?

  Yeah.

• 19:41Speaker 02

  But his spelling correction system he built in 2001 was amazing. Like he sent, he sent out this demo link to the whole company. And like, I just tried every butchered spelling of every few word query I could get. I like scrambled Uggs Bundy.

• 19:56Dwarkesh Patel?

  Yeah, yeah.

• 19:57Speaker 02

  Instead of scrambled eggs Benedict. And like, it just nailed it every time.

• 20:01Dwarkesh Patel?

  Yeah. And I guess that was language modeling.

• 20:03Speaker 02

  Yes. Yeah.

• 20:03Dwarkesh Patel?

  But, but at the time, when you were developing this systems, did, uh, did you have this sense of? Look, you make these things more and more sophisticated. You don't consider five words, but if you consider 100 words, 1,000 words, then the latent representation is intelligence. Or was that, like, basically, when did that insight hit?

• 20:21Speaker 01

  Not really. I mean, like, not, like, I don't think I ever felt like, OK, N-gram models are going to, you know, are going to

• 20:29Speaker 02

  Sweep the world.

• 20:29Speaker 01

  Yeah, the, the artificial intelligence. I think, I think at the time, I was, a lot of people were excited about Bayesian networks. That was, that seemed exciting. Definitely seeing like those early neural language models. You know, there's, but both, both the magic in that, OK, this is doing something extremely cool and also, also it's just struck me as like the, the best problem in the world. Like, in that, like, for one, it is very, very simple to state. Like, give me a probability distribution over the next word. Also, there's roughly infinite training data out there. There's like the text of the web.

• 21:10Speaker 01

  You have like trillions of training examples, like, you know, of unsupervised data.

• 21:17Speaker 02

  Yeah, or self-supervised.

• 21:18Speaker 01

  Self-supervised.

• 21:19Speaker 02

  Yeah, it's nice because you then have the right answer and then you can train on like all but the current word and try to predict the current word. And it's this kind of amazing, you know, ability to just learn from observations of the world.

• 21:32Speaker 01

  Yep. And then it's AI complete. If you can do a great job of that, then you can pretty much, pretty much do anything.

• 21:38Dwarkesh Patel?

  I'm excited to introduce our new sponsor, Meter. They're a networking company that is behind a growing fraction of the world's internet infrastructure. Fun fact, about three to four years ago, in the very early days of the podcast, I ran this podcast from a donation from Meter CEO, Anil, and I continue to benefit enormously from his advice to this day. The modern world runs on networks. Progress and feels as diverse as self-driving cars to giant LLM training runs to even broadcasting a podcast like this around the world is bottlenecked on designing and debugging large, complex networks. Meter wants to give network engineers a 100x multiplier by training

• 22:19Dwarkesh Patel?

  A large end to end foundation model using time series packet data and support tickets and networking textbooks and all the other proprietary data they have as a result of themselves building every layer of the networking stack in-house. Meter just announced a long-term compute partnership with Microsoft for access to tens of thousands of GPUs. They're currently recruiting a world-class AI research team. Their goal is to build autonomous networks that radically improve the digital world that we take for granted. To learn more, go to meter.com/markes. All right, back to Jeff and Noam. There's this interesting discussion in the history of science about whether ideas are

• 23:02Dwarkesh Patel?

  just in the air and there's a sort of inevitability to big ideas or whether it's sort of plucked out of some tangential direction. In this case, this way in which you're laying it out very logically, does that imply, like, basically, how inevitable does this?

• 23:18Speaker 01

  It does feel like it's in the air. There were definitely some, there was like this neural Turing machine. So, yeah, a bunch of ideas around this attention slash there's like having these key value stores that that could be useful in neural networks to kind of focus on things. So, yeah, I think in some sense, in the air and in some sense, you know, you need some group to go do it.

• 23:50Speaker 02

  I mean, I like to think of a lot of ideas as they're kind of partially in the air where there's like a few different, maybe separate research ideas that one is kind of squinting at when you're trying to solve a new problem and you kind of draw on those for some inspiration and then there's like some aspect that is not solved and you sort of need to figure out how to solve that. And then the combination of like some morphing of the things that already exist and some new things lead to some new breakthrough or new research result that didn't exist before.

• 24:21Dwarkesh Patel?

  Were there, are there key moments that stand out to you where you, looking at a research area and you come up with this idea? And you have this feeling of like, holy shit, I can't believe that worked.

• 24:34Speaker 02

  One, one thing I remember was, you know, we'd been in the early days of the brain team. We were focused on, let's see if we can build some infrastructure that lets us train really, really big neural nets. And at that time, we didn't have GPUs in our data centers. We just had CPUs, but you know, we know how to make lots of CPUs work together. So we built a system that enabled us to train, you know, pretty large neural nets through both model and data parallelism. So we had a, a system for unsupervised learning on actually 10 million randomly selected YouTube frames. And it was kind of a, you know, a spatially local representation.

• 25:17Speaker 02

  So it would build up unsupervised representations based on trying to reconstruct the, the thing from the high level representations. And so we got that working and training on 2,000 computers using 16,000 cores.

• 25:31Speaker 02

  And, you know, after a little while, that model was actually able to build a representation at the highest level where one neuron would get excited by, you know, images of cats that, you know, it had never been told what a cat was, but it sort of had seen enough examples of them in the training data of head on facial views of cats that that neuron would turn on for that and not for much else. And similarly, you'd have other ones for human faces and, you know, backs of pedestrians and this kind of thing. And so that was kind of cool because it's sort of from unsupervised learning principles, building up these really high level representations.

• 26:12Speaker 02

  Then we were able to get, you know, very good results on the supervised ImageNet 20,000 category challenge that like advanced the state of the art by like 60% relative improvement, which was quite good at the time. So that to me, and that neural net was probably 50x bigger than one that had been trained previously. And it got good results. So that sort of said to me, hey, actually scaling up neural net seems like a, I thought it would be a good idea and it seems to be, so we should keep pushing on that.

• 26:41Dwarkesh Patel?

  So the, these examples illustrate how these AI systems.

• 26:46Dwarkesh Patel?

  Fit into what you were just mentioning that Google is sort of a company that organizes information fundamentally and then you can, basically what AI is doing in this context is finding relationships between information, between concepts to help get ideas to you faster, information you want to you faster. Now we're moving with current AI models, like obviously they're very, you know, you can use bird in Google search and you can ask these things, questions and they obviously are still good at information retrieval. But more fundamentally, you know, like they're like, they can like write your entire code base for you and do all, you know, like

• 27:22Speaker 02

  That seems more useful.

• 27:23Dwarkesh Patel?

  More like an actual worker. Which is going beyond the, just like information retrieval. So has, yeah, has your, how are you thinking about like, is Google still an information retrieval company if you're like building an AGI? Like AGI can do information retrieval, but it can do many other things as well, right?

• 27:40Speaker 02

  I think we're an organized, the world's information company and that's broader than information retrieval, right? That's maybe organizing and creating new information from, you know, some guidance you give it. Can you help me write a, a letter to my, to my veterinarian about my dog? It's got these symptoms and it'll draft that. Or can you feed in this video and, you know, can you produce a summary of like what's happening in the video every few minutes? And, you know, I think our sort of multimodal capabilities are showing that it's more than just text. It's about, you know, understanding the world in all the different kind of modalities that information exists in, both kind of human ones, but also kind of non-human oriented

• 28:24Speaker 02

  ones like weird LiDAR sensors on autonomous vehicles or, you know, genomic information or health information. And then helping, how do you extract and transform that into useful insights for people and make use of that in, in helping them do all kinds of things they want to do. And that's, you know, sometimes it's, I want to be entertained by chatting with a chatbot. Sometimes it's, I want answers to this really complicated question. There is no single source to retrieve from. It's you need to pull information from like a hundred webpages and like figure out. What's going on and make a organized synthesized version of that data and then dealing with multimodal things or coding related problems. I think it's super exciting what these models are capable of and

• 29:09Speaker 02

  they're improving fast. So I'm excited to see where we go. I don't know what's

• 29:13Speaker 01

  I am also excited to see where we go. And yeah, I think definitely the organizing organizing information is clearly like a trillion dollar opportunity, but a trillion dollars is not cool anymore. What's cool is a quadrillion dollars. I mean, and obviously, the the idea is not to just pile up some giant pile of money, but it's to just create value in the world and so much more value can be created when these when these systems can actually like go and do something for you, write your code or figure out problems that that you wouldn't have been able to figure out yourself

• 29:56Speaker 01

  and to do that at scale. So I mean, we're going to have to be very, very flexible and dynamic as we improve the capabilities of these models.

• 30:09Speaker 02

  Yeah, I guess I'm pretty excited about kind of a lot of fundamental research questions that sort of come about because you see something that we're doing could be substantially improved if we tried this approach or things in this rough direction. And maybe that'll work, maybe it won't. But I also think there's there's value in seeing what we could achieve for end users. And then how can we work backwards from that to actually build systems that are able to do that? So as one example, organizing information, that should mean any information in the world should be usable by anyone regardless of what language I speak. And that I think we've done some amount of, but it's not nearly the full vision of no matter what language you speak out of thousands of languages,

• 30:54Speaker 02

  we can make any piece of content available to you and you make it usable by you. And any video could be watched in any language. I think that would be pretty awesome. And we're not quite there yet, but that's definitely things I see on the horizon that should be possible.

• 31:12Dwarkesh Patel?

  Speaking of different architectures you might try, I know one thing you're working on right now is longer context. If you think of Google search as like it's got the entire index of the internet in its context, but it's like sort of very like shallow search. And then obviously, language models have like limited context right now, but they can like really think, it's like dark magic like in context learning, right? Just like can really think about what it's seeing.

• 31:39Dwarkesh Patel?

  How do you think about what it would be like to merge something like Google search and something like in context learning?

• 31:44Speaker 02

  Yeah, maybe I'll take a first step at it. I mean, because I've thought about this for a bit. I mean, I think one of the things you see with these models is they they're quite good, but they do hallucinate and you know, have factuality issues sometimes. And part of that is, you know, you've trained on say tens of trillions of tokens and you stirred all that together in your tens or hundreds of billions of parameters.

• 32:06Dwarkesh Patel?

  Yeah.

• 32:07Speaker 02

  But it's all a bit squishy because you've like churned all these tokens together. And so the model has like a reasonably clear view of that data, but it sometimes like gets confused and will give the wrong date for something.

• 32:23Dwarkesh Patel?

  Right.

• 32:23Speaker 02

  Whereas information in the context window in the input of the model is like really sharp and clear. Because we have this really nice attention mechanism in transformers that the model can pay attention to things and it knows kind of the exact text or the exact frames of the video or audio or whatever that it's processing. And so right now, we have a model that can deal with kind of millions of tokens of context, which is quite a lot. It's like, you know, hundreds of pages of PDF or, you know, 50 research papers or, you know, hours of video or tens of hours of audio or some combination of those things, which is pretty cool.

• 33:04Speaker 02

  But it would be really nice if the model could attend to trillions of tokens, right? Could it attend to the entire internet and find the right stuff for you? Could it attend to all your personal information for you, right? Like, I would love a model that has access to all my emails and all my documents and all my photos. And when I ask it to do something, it can sort of make use of that with my permission to to sort of help solve what it is I'm wanting it to do. But that's going to be a big computational challenge because the naive attention algorithm is quadratic. And you can kind of barely make it work on a fair bit of hardware for millions of tokens. But there's no hope of making that just naively go to trillions of tokens.

• 33:47Speaker 02

  So we need a whole bunch of interesting algorithmic approximations to what you would really want to make a way for the model to attend kind of conceptually to, you know, lots and lots of more tokens, to trillions of tokens, and attend to your tokens. You know, maybe we can put all of the Google code base in context for every Google developer, all the world's source code in context for any open source developer. That would be amazing.

• 34:13Speaker 01

  It would be incredible. Yeah, I mean, right. The, yeah, the beautiful thing about, you know, model parameters is they are quite memory efficient at, you know, sort of memorizing facts. Maybe, you know, you can probably memorize order of one, one fact or something per model parameter. Whereas, you know, if you have some token in context, there are like lots of keys and values at every layer. It could be, could be a kilobyte, a megabyte of, of, of memory per, per token.

• 34:49Speaker 02

  You take a word and you blow it up to 10 kilobytes.

• 34:51Speaker 01

  Yes, yes. Yeah. So, I mean, there, so there are some, there, there, there's actually a lot of innovation going on around. Okay, A, how do you minimize that? And B, okay, what, what words do you need to have? There are, there are better ways of accessing bits of that, bits of that information. And, you know, Jeff seems like the right person to figure this out. Like, okay, what, what does our memory hierarchy look like, you know, from the, you know? SRAM all the way up the data center worldwide level.

• 35:25Dwarkesh Patel?

  I want to talk more about the thing you mentioned about, look, you know, Google is a company with like lots of code and lots of examples, right? If you just think about that one use case and what that implies. So you've got like the Google monorepo.

• 35:40Dwarkesh Patel?

  And if you maybe you figure out the long context thing, you can put the whole thing in context or you fine tune on it.

• 35:47Dwarkesh Patel?

  Yeah, basically, like, why, why, why don't, why hasn't this been already done? And, you know, because you can imagine like the amount of code that Google has proprietary access to, just like me, even if you're just using it internally for it to make your developers more efficient and productive.

• 36:02Speaker 02

  Oh, to be clear, we, we have actually already done further training on a Gemini model on our internal code base for our internal developers.

• 36:11Dwarkesh Patel?

  Yeah.

• 36:12Speaker 02

  But that's different than attending to all of it.

• 36:14Dwarkesh Patel?

  Right.

• 36:15Speaker 02

  Because it sort of stores together the code base into a bunch of parameters. And I think having it in context is, is, makes things clearer. But even the, the sort of further trained model internally is incredibly useful. Like Sundar, I think, has said that 25% of the characters that we're checking into our code base these days are generated by our AI based coding models with, with kind of human kind of.

• 36:41Dwarkesh Patel?

  How do you imagine in a year or two, based on the capabilities you see around the horizon, your own personal work? What will it be like to be a researcher at Google? You have a new idea or something with the way in which you're interacting with these models in a year, what does that look like?

• 36:57Speaker 01

  Well, I mean, I assume the, we will be, we will have these models a lot better and

• 37:04Speaker 01

  hopefully be able to be much, much more productive.

• 37:07Speaker 02

  Yeah, I mean, I think one, one of the. In addition to kind of researchy context, like anytime you're seeing these models used, I think they're able to make software developers more productive because they can kind of take sort of a high level spec or in-sentence description of what you want done and give a pretty approximate, you know, pretty reasonable first cut at that. And so from a research perspective, maybe you can say, I'd really like you to explore, you know, this kind of idea, like similar to the one in this paper, but maybe like, let's try making it convolutional or something like that. If you could do that and have the system automatically sort of generate a bunch of experimental code and maybe you

• 37:50Speaker 02

  look at it and you're like, yeah, that looks good, run that. Like that seems like a nice dream direction to go in and seems plausible in the next year or two years that you might make a lot of progress on that.

• 38:01Dwarkesh Patel?

  It seems underhyped because you've got like, you could have like literally millions of extra employees and you can immediately check their output, but employees can check each other's output. They like immediately stream tokens.

• 38:14Speaker 02

  Yeah, sorry. I didn't mean none to hype it. I think it's super exciting. I just don't like to hype things that aren't done yet.

• 38:21Dwarkesh Patel?

  Yeah, so let's, I do want to play with this idea more because, you know, it seems like a good idea if you have something like, kind of like an autonomous software engineer, especially from the perspective of a researcher who's like, I want to spec, build the system. Again, okay, so you'll legislate with the idea, like as somebody who has worked on developing transformative systems through your careers, the idea that instead of having to code something like whatever the today's equivalent of MapReduce is or TensorFlow is, just like, here, here's how I want like distributed AI library to look like, write it up for me. Could, do you imagine you could be like 10x more productive, 100x more productive?

• 39:06Speaker 02

  I was pretty impressed. I think it was on Reddit that I saw, like, we have a new experimental coding, like model that's much better at coding and math and so on. And someone external tried it and they basically prompted it and said, I'd like you to implement a SQL processing database system with no external dependencies and please, please do that and see. And from what the person said, it actually did a quite good job. Like it generated a SQL parser and a tokenizer and, you know, a, a query planning system and some storage format for the data on, on disk and actually was able to handle simple queries.

• 39:47Speaker 02

  So, you know, from that prompt, which is like, you know, paragraph of text or something to get, you know, even a certain, an initial cut at that seems like a big boost in productivity for software developers. Yeah. And I think you might end up with other, other kinds of systems that maybe don't try to do that in a single, you know, in semi-interactive respond in 42nd kind of thing, but might go off for 10 minutes and like might interrupt you after five minutes saying, I've, I've done a lot of this, but now I need to, you know, get some input, you know, do you, do you care about handling video or just images or something? Um, and that, that seems like you'll need ways of managing the workflow if you have

• 40:31Speaker 02

  a lot of these kind of, uh, background activities happening.

• 40:36Dwarkesh Patel?

  Yeah. Actually, can you talk more about that? So what interface do you imagine we might need if we have, um, uh, if you could literally have like millions of employees you could spin up, hundreds of thousands of employees you could spin up on command who are able to type incredibly fast and who, um, so it's, it's almost like you go from like 1930s, like trading of like tickets or something to now modern and the like, you know, of chainsuit or something, you know, like you need a better, you need some interface to keep track of all the sets going on for the AIs to integrate into this big mono repo, um, and leverage their own like, uh, strengths, um, for humans to keep track of what's happening. Basically, what, what, what is it like to be, uh, Jeff or Noam in three years, working day to day?

• 41:18Speaker 01

  It might be kind of similar to what we have now because we already have a sort of parallelization as, as a major issue because, you know, we, we have like lots and lots of really, really brilliant machine learning researchers and we want them to work, all work together and, uh, and, and build AI. Um, you know, so actually the parallelization among people might be similar to parallelization among, uh, uh, among, um, machines. But, um, I think there definitely, it should be good for things that require like a lot of exploration, you know, like I come up with, come up with the next, uh, breakthrough because, you know,

• 42:00Speaker 01

  If you have a brilliant idea that's just certain to work, you know, in the ML domain, then, you know, it has a 2% chance of working if you're brilliant. And, you know, mostly these things fail. But if you try 100 things, or 1,000 things, or a million things, then you might hit on something amazing. And we have plenty of compute, like modern, you know, top labs these days have probably a million times as much compute as it took the train transformer. So.

• 42:33Dwarkesh Patel?

  Yeah, actually, so that's a really interesting idea. If you have, like, suppose in the world today, there's like, on the order of 10,000 AI researchers, and this community coming up with a breakthrough every.

• 42:44Speaker 02

  Probably more than that. There were 15,000 at NERF.

• 42:46Dwarkesh Patel?

  Oh, wow. 100,000, I don't know.

• 42:50Speaker 02

  Yeah, maybe. Sorry.

• 42:52Dwarkesh Patel?

  No, no, it's good to have the correct order of magnitude. And the odds of this community, every year, comes up with a breakthrough on the scale of a transformer is, let's say, 10%. Now suppose this community is 1,000 times bigger, and it is, in some sense, like the sort of parallel search of better architectures, better techniques. Do we just like get like

• 43:13Speaker 01

  A breakthrough a day?

• 43:14Dwarkesh Patel?

  breakthroughs every year or every day?

• 43:17Speaker 01

  Maybe. Sounds potentially good.

• 43:21Dwarkesh Patel?

  But does that feel like what ML research is like? Is just if you have, if you are able to try all these experiments?

• 43:29Speaker 01

  It's a good question, because we, you know, I don't know that folks have been, haven't been doing that as much. I mean, we definitely have lots of, lots of great ideas coming along. Everyone seems to want to run their experiment at maximum scale. But I think that's, you know, that's a human problem.

• 43:47Speaker 02

  Yeah. It's very helpful to have a 1,000th scale problem, and then vet like 100,000 ideas on that, and then scale up the ones that are this team promising.

• 43:58Dwarkesh Patel?

  Yeah. A quick word from our sponsor, Scale AI. Publicly available data is running out. So major labs like Meta and Google DeepMind and OpenAI all partner with Scale. To push the boundaries of what's possible. Through Scale's data foundry, major labs get access to high-quality data to fuel post-training, including advanced reasoning capabilities. As AI races forward, we must also strengthen human sovereignty. Scale's research team, SEAL, provides practical AI safety frameworks, evaluates frontier AI system safety via public leaderboards, and creates foundations for integrating advanced AI into society. Most recently, in collaboration with the Center for AI Safety, Scale published "Humanity's Last Exam",

• 44:41Dwarkesh Patel?

  a groundbreaking new AI benchmark for evaluating AI systems' expert-level knowledge and reasoning across a wide range of fields. If you're an AI researcher or engineer, and you want to learn more about how Scale's data foundry and research team can help you go beyond the current frontier of capabilities, go to scale.com/dwarkesh. All right, back to Jeff and Noam. So I think one thing the world might not be taking seriously. People are aware that it's exponentially harder to make, like to do this, like make a model that's 100x bigger, is like 100x more compute, right? So it's like, people are aware that's like an exponentially harder problem to go from Gemini 2 to 3 or so forth. But maybe people aren't aware of this other trend

• 45:24Dwarkesh Patel?

  where Gemini 3 is coming up with all these different architectural ideas and trying them out, and you see what works, and you're constantly coming up with these algorithmic progress that makes training the next one easier and easier. How far could you take that feedback, Lou?

• 45:39Speaker 02

  I mean, I think one thing people should be aware of is the improvements from generation to generation of these models often are partially driven by hardware and larger scale, but equally, and perhaps even more so, driven by major algorithmic improvements and major changes in the model architecture and the training data mix and so on, that really make the model better per flop that is applied to the model. So I think that's a good realization. And then I think if we have automated exploration of ideas, we'll be able to vet a lot more ideas and bring them into kind of the actual, you know, production training for next generations of these models. And that's going to be really helpful because that's sort of what we're currently doing

• 46:23Speaker 02

  with a lot of machine learning research, brilliant machine learning researchers, is looking at lots of ideas. You know, winnowing ones that seem to to work well at small scale, seeing if they work well at medium scale, bringing them into larger scale experiments and then like settling on like adding a whole bunch of new and interesting things to the to the final model recipe. And then I think if we can do that, you know, 100 times faster through those machine learning researchers, just gently steering a more automated search process rather than sort of hand babysitting lots of experiments themselves, that's going to be really, really good.

• 47:00Speaker 01

  Yeah, the one thing it doesn't speed up is like experiments at the largest scale because you still end up doing like these N equals one experiments and they're really just try to put a bunch of really brilliant people in the room and have them stare at stare at the thing, figure out why this is working, why this is not working.

• 47:18Speaker 02

  More hardware is a good solution and better hardware.

• 47:21Speaker 01

  Yes, yes, we're counting on you.

• 47:23Dwarkesh Patel?

  So, okay, naively, I would, so there's a software, there's this like algorithmic side improvement that future AIs can make. There's also the stuff you're working on on AlphaChip, I'll let you describe it, but if you get into a situation where just from a software level, you can be making better and better chips in a matter of weeks and months, and better AIs can presumably do that better. Basically, I'm wondering, how does this feedback loop not just end up in like Gemini 3 takes two years, then Gemini 4 is like a six, or the equivalent level jump is now six months,

• 48:06Dwarkesh Patel?

  then like the level five is like three months, then one month, and you get to like superhuman intelligence much more rapidly than you might naively think because of this software, both on the hardware side and from the algorithmic side improvements?

• 48:22Speaker 02

  Yeah, I mean, I've been pretty excited lately about how could we dramatically speed up the chip design process?

• 48:28Dwarkesh Patel?

  Yeah.

• 48:29Speaker 02

  Because as we were talking earlier, the, you know, the current way in which you design a chip takes you roughly, you know, 18 months to go from, we should build a chip to something that you then hand over to TSMC and and then TSMC takes, you know, four months to fab it and then you get it back and you put it in your data centers. So that's a pretty lengthy cycle and the fab time in there is a pretty.

• 48:54Speaker 02

  You know, a small portion of it today. But if you could make that the dominant portion so that instead of taking, you know, 18 months, 12 to 18 months to design the chip, you could shrink and with, you know, 100, 150 people, you could shrink that to, you know, a few people with a much more automated search process, exploring the whole design space of chips and getting feedback from all aspects of the chip design process for the kind of choices that the system is trying to explore at the high level.

• 49:24Dwarkesh Patel?

  Yeah.

• 49:24Speaker 02

  Then I think you could get, you know, perhaps much more exploration and more rapid design of something that you actually want to give to a fab. And, and that would be great because you can shrink that, that time, you can shrink the deployment time by kind of designing the hardware in the right way so that you just get the chips back and you just plug them in to some, some system. And that will then, I think, enable a lot more specialization. It will enable a shorter timeframe for the hardware design so that you don't have to look out quite as far into what kind of an algorithm to be interesting. Instead, it's like you're looking at, you know, six to nine months from now.

• 50:03Dwarkesh Patel?

  Yeah.

• 50:04Speaker 02

  What should it be, rather than, you know, two, two and a half years? And that would be pretty cool. I do think that, that fab fabrication time is, if that's in your inner loop of improvement, you're gonna like.

• 50:15Dwarkesh Patel?

  How long is it?

• 50:16Speaker 02

  The, the leading edge nodes unfortunately are taking longer and longer because they have more metal layers than previous, you know, older nodes. So that, that tends to make, make it take anywhere from three to five months.

• 50:28Dwarkesh Patel?

  Okay. But that's how long training runs take anyways, right? So you could potentially do both at the same time?

• 50:34Speaker 02

  Yeah, potentially.

• 50:35Dwarkesh Patel?

  Okay, so I guess like you can't get sooner than three to five months, but the idea that you could get like, but also, yeah, you're like rapidly developing new algorithmic ideas. Right. That can move fast.

• 50:44Speaker 02

  That can move fast. That can run on like existing chips and explore lots of cool ideas.

• 50:49Dwarkesh Patel?

  Yeah. So isn't that like a situation in which you're like, I think people sort of expect like, ah, there's going to be a sigmoid. Again, this is not a sure thing, but just like, is this a possibility? The idea that you have like sort of an explosion of capabilities very rapidly towards the tail end of human intelligence that, you know, gets like a smarter and smarter to more and more rapid rate.

• 51:13Speaker 01

  Quite possibly.

• 51:14Speaker 02

  Yeah, I mean, I think I like to think of it like this, right? Like, right now, we have models that can take a pretty complicated problem and can break it down, you know, internally in the model into a bunch of steps, can sort of puzzle together the solutions for those steps and can often give you a solution to the entire problem that you're asking. But it, you know, isn't super reliable and it's good at breaking things down into, you know, five to 10 steps, not 100 to 1,000 steps. So if you could go from, yeah, 80% of the time, it can give you a perfect, perfect answer to something that's 10 steps long, to something that, you know, 90% of the time can give

• 51:55Speaker 02

  you a perfect answer to something that's 100 to 1,000 steps of sub, sub problem long, that would be an amazing improvement in capability of these models. And, you know, we're not there yet, but I think that's what we're aspirationally trying to get to is.

• 52:09Speaker 01

  Yeah, we don't need new hardware for that. We, but I mean, we, we'll, we'll take it.

• 52:13Speaker 02

  Yeah. Yeah, exactly. Never looked new hardware in the mouth.

• 52:19Speaker 01

  One of the, you know, like one of the big areas of improvement, I think, you know, in the near future is, is this inference time compute, like applying more compute, you know, at inference time. And I guess the way I've, I've liked to describe it is that, um, you know, a, like even some giant language model, you know, even if you're doing, say, a trillion operations per token, which is. You know, more than, more than most people are, are doing these days. You know, operations cost something like 10 to the negative $18. And so you're getting like a million tokens to the dollar, right? So, I mean, compare that to like a relatively cheap pastime.

• 53:04Speaker 01

  Like you, you, you go out and you buy a paper book and read it. You're paying like 10,000 tokens to the dollar. So it's so like, talking to a language model could be like, you know, is, is, is like 100 times cheaper than reading a paperback. So there is a huge amount of headroom there to say, okay, if we can make this thing more expensive, uh, but, but smarter, cause we're like two, like 100x cheaper than reading a paperback. We're like 10,000 times cheaper than like talking to a customer support agent. We're like a million times or more cheaper than, than, you know, hiring a software engineer or talking to your doctor or lawyer.

• 53:43Speaker 01

  Like, can we add, you know, the, yeah, add computation and make it, make it smarter? So like, I think a lot of, a lot of the takeoff that we're, that we're going to see in, in the very near future is of this form. Like we've, we've been exploiting and improving, uh, pre-training a lot in the past and post-training. And those things will continue to improve, but like taking advantage of, you know, think harder at, at inference time is going to just be a, an explosion.

• 54:17Speaker 02

  Yeah. And an aspect of, of inference time is I think you want the system to be actively exploring a bunch of different potential solutions. You know, maybe it does some searches on its own and get some information back and like consumes that information and figures out, oh, now I would really like to know more about this thing. So now it kind of iteratively kind of explores how to best solve the high level problem you pose to the system. And I think having a dial where you can make the model give you better answers with more inference time compute seems like we have a bunch of techniques now that seem like they can kind of, kind of do that. And the more you crank up the dial, the more it costs you in terms of compute, but the better the answers get.

• 55:01Speaker 02

  That, that seems like a, a nice trade off to have, cause sometimes you want to think really hard, cause this is a super important problem. Sometimes you probably don't want to spend enormous amounts of compute to compute, you know, one plus, what's the answer to one plus one? Maybe the system should decide to use-

• 55:19Dwarkesh Patel?

  You take that to 100 and it comes up with like new axioms of set theory or something.

• 55:21Speaker 02

  Should decide to use a calculator tool or something instead of, you know, a very large language model.

• 55:27Dwarkesh Patel?

  Are there any impediments to taking inference time, like, uh, ha- having some way in which you can just linearly scale up inference time compute, or is this a basically a problem that's sort of solved and we know how to sort of like 100x compute, 1000x compute and get correspondingly better results?

• 55:45Speaker 01

  Well- We're, we're working out the algorithms as we speak, so I, I, I, I believe, you know, we'll, we'll see, we'll see better and better solutions to this as these many more than 10,000 researchers are, are hacking at it, uh, any of them at Google.

• 56:01Speaker 02

  I mean, I think we do see some examples in our own sort of experimental work of things where if you apply more inference time compute, the answers are better than if you just apply, you know, uh, you know, X, you know, if you apply 10X, you can get better answers than X amount of computed inference time, and that seems useful and important. But I think what we would like is when you apply 10X to get, you know, even a bigger improvement in the quality of the answers than we're getting today. And so that's about, you know, designing new algorithms, trying new approaches, you know, figuring out how best to spend that 10X instead of X to, to improve things.

• 56:40Dwarkesh Patel?

  Does it look more like search, or does it look more like just keep it going in that linear direction for a longer time?

• 56:45Speaker 02

  I mean, I think search is, I, I, I really like Rich Sutton's paper that he wrote about the bitter lesson and the bitter lesson effectively is this nice one-page paper, but the, the essence of it is, um, you can try lots of approaches, but the, the two techniques that are incredibly effective are learning and search. And you can apply and scale those algorithmic, or, you know, computationally, and you often will then get better results than any other kind of approach you can apply to, to a pretty broad variety of problems.

• 57:17Dwarkesh Patel?

  Yeah.

• 57:18Speaker 02

  Um, and so I think search has got to be part of the solution to spending more inference time, is you want to maybe explore a few different ways of solving this problem and like, oh, that one didn't work, but this one worked better. So now I'm going to explore that a bit more.

• 57:31Dwarkesh Patel?

  How does this change your plans for future data center?

• 57:37Dwarkesh Patel?

  Planning and so forth, where if, you know, can this kind of search be done asynchronously? Does it have to be online, offline? How does that change how big of a campus you need and those kinds of considerations?

• 57:51Speaker 02

  I mean, I think one general trend is it's clear that inference time compute, you know, you have a model that's pretty much already trained and you want to do inference on it, is going to be a growing and important class of computation that maybe you want to specialize hardware more around that. You know, actually, the first TPU was specialized for inference and wasn't really designed for training. And then subsequent TPUs were really designed more around training and also for inference. But it may be that, you know, when you have something where you really want to crank up the amount of compute you use at inference time, that even more specialized solutions won't make a lot of sense.

• 58:33Dwarkesh Patel?

  Does that mean you can accommodate more asynchronous training?

• 58:36Speaker 02

  Training or inference?

• 58:38Dwarkesh Patel?

  Or just you can have, the different data centers don't need to talk to each other. You can just like have them do a bunch of.

• 58:44Speaker 02

  Oh, yeah, I mean, I think, I like to think of it as, is the inference that you're trying to do latency sensitive? Like a user is actively waiting for it?

• 58:53Dwarkesh Patel?

  Right.

• 58:53Speaker 02

  Or is it kind of a background thing? And maybe that's, I have some inference tasks that I'm trying to run over a whole batch of data, but it's not for a particular user. It's just I want to, you know, run inference on it and extract some information. And then there's probably a bunch of things that we don't really have very much of right now, but you're seeing inklings of it in our deep research like tool that we just released. I forget exactly when, like a week ago, where you can give it a pretty complicated, high level task like, hey, can you go off and research the history of renewable energy and all the trends and costs for, for wind and solar and other kinds of techniques? And put it in a table and give me a full eight page report. And it will come back with an eight page report with like 50 entries in the bibliography.

• 59:38Speaker 02

  It's pretty, pretty remarkable, but you're not actively waiting for that for one second. It takes like, you know, a minute or two to go, go do that.

• 59:46Dwarkesh Patel?

  Yeah.

• 59:47Speaker 02

  And I think there's going to be a fair bit of that kind of compute. And that's the kind of thing where you have some UI questions around, okay, if you're going to have. A user with 20 of these kind of asynchronous tasks in the background happening and maybe each one of them needs to like get from our more information from the user like I found your flights to Berlin, but there's no non-stop ones. Are you okay with a, you know, a non-stop one? How does that flow work when you kind of need a bit more information and then you want to put it back in the background for it to continue doing, you know, finding the hotels in Berlin or whatever? I think it's going to be pretty interesting and inference will be useful.

• 1:00:28Speaker 01

  Inference will be useful. I mean, there's also a compute efficiency thing in inference that you don't have in training and that, you know, in general transformers can use the sequence length as a batch during training, but they can't really in inference because you're when you're generating one token at a time. So, so there, there, there may be different hardware and inference algorithms that that we design for the purposes of being efficient at inference.

• 1:00:56Speaker 02

  Yeah, like as a good example, out of an algorithmic improvement is like the use of drafter models. So you have like a really small language model that you do one secret token at a time when you're decoding and predict like four tokens. And then you give that to the big model and you say, okay, here's the four tokens the little model came up with. Check which ones you agree with. And if you agree with the first three, then you just advance. And then you've basically been able to do a four, four token with parallel computation instead of a one token with thing in the big model. And so those are the kinds of things that people are looking at to improve inference efficiency. So you're not, don't have this single token decode bottleneck.

• 1:01:40Speaker 01

  Right. Basically, the big model is being used as a verifier as opposed to a generator and verification you can do.

• 1:01:47Speaker 02

  Hello, how are you? That sounds great to me. I'm going to like advance past that.

• 1:01:51Dwarkesh Patel?

  So a big discussion has been about. You know, you're, we're already tapping out like nuclear power plants in terms of delivering power into one single campus. And so do we have to like have just like even more like two gigawatts in one place, five gigawatts in one place, or can it be more distributed and still be able to train a model? Does this new regime of inference scaling make different considerations there plausible? Or how are you thinking about multi data center training now?

• 1:02:24Speaker 02

  I mean, we're already doing it. So we're, we're, we're pro multi data center training. I think, I think in the Gemini 1.5 tech report, we said we use multiple metro areas and trained with some of the compute in each place and then a, you know, pretty long latency, but high bandwidth connection between those data centers. And that works fine. You know, it's great. Actually, training is kind of interesting because each step in a training process is, you know, usually for a large model is a few seconds or something at least. So the latency of it being, you know, 50 milliseconds away doesn't matter that much.

• 1:02:58Speaker 01

  Just the bandwidth, you know?

• 1:03:01Speaker 02

  Yeah, just bandwidth.

• 1:03:02Speaker 01

  As long as you can sync, you know, sync all of the parameters of the model across the different data centers and then accumulate all the gradients. So it's in the time it takes to do one step, you're, you're, you're pretty good.

• 1:03:14Speaker 02

  And then we have a bunch of work on, in, you know, in even early brain days when we were using CPU machines and they were really slow. So we needed to do asynchronous training to help scale, where each copy of the model would kind of do some local computation and then send gradient updates to a centralized system and then apply them asynchronously. And another copy of the model would be doing the same thing. You know, it makes your, your model parameters kind of wiggle around a bit and it makes people uncomfortable with the theoretical guarantees, but it actually seems to work in practice.

• 1:03:48Speaker 01

  In practice, it works. It was so pleasant to go from async to sync because, because your experiments are now replicable, like rather than like every, like your results depend on like whether there was like a web crawler running on the same machine. It's like one of your computers. So I am so much happier running on like TPU pods.

• 1:04:12Speaker 02

  I love asynchronous. It just lets you scale some more.

• 1:04:14Speaker 01

  Two iPhones and an Xbox or whatever.

• 1:04:16Speaker 02

  Yeah. What if we could give you asynchronous but replicatable results?

• 1:04:21Speaker 01

  Ooh.

• 1:04:23Speaker 02

  So one way to do that is you effectively record the sequence of operations. So like which gradient update happened and when? And on which batch of data, you don't necessarily record the actual gradient update in a log or something, but you could replay that log of operations. So that you get repeatability. Then I think you'd be happier.

• 1:04:46Speaker 01

  Possibly. At least you could debug what happened. Yeah, but you wouldn't be able to like compare to necessarily to training runs. Because, okay, I made one change in the hyper parameter. But also like I had like a like

• 1:05:01Speaker 02

  Web crawler.

• 1:05:03Speaker 01

  Messing up. And there were like a lot of people streaming the Super Bowl at the same time.

• 1:05:11Speaker 02

  I mean, the thing that let us, the thing that let us go from asynchronous training on CPUs to fully synchronous training is the fact that we have these super fast TPU hardware chips and then pods, which have incredible amounts of bandwidth between the chips and a pod. And then scaling beyond that, we have like really good data center networks and even cross metro area networks that enable us to scale to, you know, many, many pods in multiple metro areas for our largest training runs. And we can do that fully synchronously, as Noam said, as long as the gradient accumulation and communication of the parameters across metro areas happens, you know, fast enough relative to the step time, you're golden. You don't really care.

• 1:05:54Speaker 02

  But I think as you scale up, there may be a push to have a bit more asynchronous in our system than we have now. Because like we can make it work. I've been, you know, our ML researchers have been really happy how far we've been able to push synchronous training because it is easier mental model to understand. You know, you just have your algorithm sort of fighting you rather than the asynchronous and the algorithm kind of battling you.

• 1:06:21Speaker 01

  As you scale up, there are more things fighting you, you know, like there's, I mean, the, right, that's the problem with, you know, with scaling that you don't actually always know what it is that's fighting you. Is it, you know, the fact that you've pushed like quantization a little too far in some place or another? Or is it your data? Or is it?

• 1:06:41Speaker 02

  Maybe it's your adversarial machine, MUQQ17, that is like setting the seventh bit of your exponent and all your gradients or something.

• 1:06:50Speaker 01

  Right. And all of these things just Make the model slightly worse, so you don't even know that the thing is going on.

• 1:06:58Speaker 02

  Yeah, that's actually a bit of a problem with neural nets is they're so tolerant of noise. You can have things set up kind of wrong in a lot of ways and they just kind of figure out ways to work around that or learn.

• 1:07:10Speaker 01

  And you could have bugs in your code. Most of the time that does nothing, some of the time it makes your model worse, some of the time it makes your model better. And then you discover something new because you never tried this bug at scale before because you didn't have. It didn't have the budget for it, but.

• 1:07:28Dwarkesh Patel?

  What practically does it look like actually to debug or decode what the. Like you've got these things, some of which are making the model better, some of which are making it worse. You, when you go into work tomorrow, you're like, all right, what's going on here? How do you figure out what the most salient inputs are?

• 1:07:45Speaker 01

  Right, I mean, well, at small scale, you do lots of experiments. So, so I mean, there's, I think, one part of of the research that involves. Okay, I want to like invent these improvements or breakthroughs kind of in isolation. In which case you want a nice, simple code base that you can fork and hack and like some baselines. And, you know, my dream is I wake up in the morning, come up with, come up with an idea, hack it up in the day. Run some experiments, get, get some initial results in the day like, okay, this, this looks promising. These things work, these things worked and didn't work. And, and I think that, that is, that is very achievable because.

• 1:08:28Dwarkesh Patel?

  At small scale.

• 1:08:29Speaker 01

  At small scale, as long as you keep your, you know, keep a nice experimental code base and.

• 1:08:34Dwarkesh Patel?

  Maybe an experiment takes an hour to run or two hours or something, not, not, not two weeks.

• 1:08:39Speaker 01

  It's great, it's great. So, so there's that part of the research and then there's some amount of scaling up. And then you have the part which is like integrating where you want to stack all the improvements on top of each other and see if they work in at large scale and see if they work all in conjunction with each other.

• 1:08:58Speaker 02

  Right, you think maybe they're independent, but actually maybe there's some funny interaction between. You know, improving the, the way in which we handle video data input and the way in which we, you know. Update the, the model parameters or something and, and that interacts more for video data than some other thing. You know, there's all kinds of interactions that can happen that you maybe. Uh, don't anticipate. And so you want to run these experiments where you're then putting a bunch of things together and then periodically making sure that all the things you think are good are good together. Uh, and if not, understanding why they're not playing nicely.

• 1:09:34Dwarkesh Patel?

  Um, two questions. One, how often does it end up being the case that things don't stack up well together? Um, is it like a rare thing or does it happen all the time?

• 1:09:46Speaker 01

  It it happens, happens all the time.

• 1:09:49Speaker 02

  Yeah, I mean, I I think most things you don't even try to stack because they they, you know, the initial experiment didn't work that well, or it showed results that aren't that promising relative to the baseline. And then you sort of take those things and you try to scale them up individually. And then you're like, oh, yeah, these ones seem really promising. Uh, so I'm going to now include them in something that I'm going to now bundle together and try to advance, you know, what the and combined with other things that seem promising. And then you run the experiments and then you're like, oh, well, they didn't really work that well. Like, let's try to debug why.

• 1:10:22Speaker 01

  And then there are trade offs because you you want to keep your like integrated system, you know, as clean as you can because, you know, complexity, code base, yeah, code base and algorithmically complexity, you know, complexity hurts, complexity makes things slower and introduces more risk. And then, you know, at the same time, you want to you want it to be as as good as possible. And of course, uh, uh, every individual researcher uh, wants wants his inventions to go into it. Um, so there there are, uh, there are definitely, uh, uh, challenges there, but, uh, but we've been working together quite well.

• 1:10:59Dwarkesh Patel?

  My sponsors, Jane Street, invented a card game called Figgy in order to teach their new traders the basics of markets and trading. I'm a poker fan and I'd say that Figgy is like poker in the sense that there's hidden information, but it's much more intense and social. In poker, you're usually just sitting around waiting for your turn, whereas in Figgy, you spend the whole time just shouting bids and asks at the other players. The game is set up such that there's a winner in the end, of course, but during each turn, you are incentivized to find mutually beneficial trades with the other players. And in fact, that's the main skill that the game rewards. Figma simulates the most exciting parts of trading. Jane Streeters enjoy it so much that they hold an inner office Figma championship every single year.

• 1:11:42Dwarkesh Patel?

  You can play it yourself by downloading it on the App Store or you can find it on desktop at F I G G I E dot com. All right, back to Jeff and Noam. Okay, so then going back to the whole dynamic of you find better and better algorithmic improvements. And the models get better and better over time, even if you take the hardware part out of it. Should the world be thinking more about and should you guys be thinking more about this? There's one world where you're just like AI is a thing that takes like two decades to slowly get better over time. And you can sort of like refine things over, you know, if like you've kind of messed something up, you fix it. And it's like, not that big a deal, right? It's like, not that much better than the previous version you released.

• 1:12:26Dwarkesh Patel?

  There's another world where you have this big feedback loop, which means that the year, the, the, the, the two years between Gemini four and Gemini five are the most important years in human history. Because you go from a pretty good ML researcher to superhuman intelligence because of this feedback loop. To the extent that you think that second world is plausible, how does that change how you sort of approach these greater and greater levels of intelligence?

• 1:12:55Speaker 01

  I've stopped cleaning my garage because I'm waiting for the robots. So probably I'm, I'm more in the second camp of what we're going to see a lot of acceleration.

• 1:13:05Speaker 02

  Yeah, I mean, I think it's, it's super important to understand what's going on and what the trends are. And I think right now the trends are the models are getting substantially better generation over generation. And I don't see that slowing down in the next few generations, probably. So that means the models, say, two to three generations from now are going to be capable of, you know, let's go back to the example of breaking down a simple task into 10 sub pieces and doing it 80% of the time to something that can break down a task, a very high level task into 100 or 1,000 pieces. And get that right 90% of the time, right? That's a major, major step up in what the models are capable of. So I think it's important for people to understand,

• 1:13:48Speaker 02

  you know, what's, what is happening in the progress in the field. And then those models are going to be applied in a bunch of different domains. And I think it's really good to make sure that we, as society, get the maximal benefits from what these models can do to improve things in, you know, I'm super excited about areas like education and healthcare, you know, making information accessible to all people. But we also realize that they could be used for misinformation. They could be used for, you know, automated hacking of computer systems. And we want to sort of put as many safeguards and mitigations and understand the capabilities of the models in, in place as we can.

• 1:14:32Speaker 02

  And that's kind of, you know, I think Google as a whole has a really, you know, good view to how we should approach this. You know, our responsible AI principles actually are a pretty nice framework for how to think about trade offs of making, you know, better and better AI systems available in different contexts and settings, while also sort of making sure that we're doing the right thing in terms of, you know, making sure they're safe and, you know, not saying toxic things and things like that.

• 1:15:02Dwarkesh Patel?

  Or I guess the thing that stands out to me, if, if you were like zooming out and looking at like this period of human history, if, if we're in the world where like, look, maybe if you do post training on Gemini three badly, it can do some misinformation. But then you like fix the post training and like, it's gonna, gonna stop doing them. Is it, it's a bad mistake, but it's a fixable mistake, right? Right. Whereas if, if you have this feedback loop dynamic, which is a possibility. Right. Then the sort of like mistake of like, the thing that catapults this intelligence explosion is like, misaligned is, uh, is like not trying to write the code you think it's trying to write and optimizing for some other objective. And on the other end of this very rapid process

• 1:15:47Dwarkesh Patel?

  that lasts a couple of years, maybe less, you have things that are approaching Jeff Dean or beyond the level, or no, Shazier, beyond level. And then you have like millions of copies of Jeff Dean level programmers. And anyways, that seems like a harder to recover mistake. And that seems like a much more salient like you really got to make sure you're going in the explosion.

• 1:16:11Speaker 01

  As these, as these systems do get more powerful, you, you have, you know, you've got to be more and more, more and more careful.

• 1:16:18Speaker 02

  I mean, one thing I would say is there's like the extreme views on either end. There's like, oh my goodness, these systems are going to like be so much better than, than humans at all things and we're going to be kind of overwhelmed. And then there's the like, these systems are going to be amazing and we don't have to worry about them at all. I think I'm somewhere in the middle and I've been a, I'm a co-author on a paper called Shaping AI, which is, you know, those two extreme views often kind of view our role as kind of laissez faire, like we're just going to have the AI develop in the path that it, that it takes. And I think there's actually a really good argument to be made that what we're going to do is try to shape and steer the, the way in which AI is deployed in the world so

• 1:17:03Speaker 02

  that it is, you know, maximally beneficial in the areas that we want to capture and benefit from in, you know, education, you know, some of the areas I mentioned, healthcare. And steer it as much as we can away, maybe with policy related things, maybe with, you know, technical measures and safeguards away from, you know, the computer will, you know, take over and, and have unlimited control of what it can do. So I think that's an engineering problem is how do you engineer safe systems? I think it's kind of the modern equivalent of what we've done in a kind of older style software development.

• 1:17:44Speaker 02

  Like if you look at, you know, airplane software development, that has a pretty good record of how do you rigorously develop safe and secure systems for, for doing a pretty, pretty risk, risky task.

• 1:17:58Dwarkesh Patel?

  The difficulty there is that there's not some feedback loop where the 737, you like put it in a box with a bunch of compute for a couple of years and it comes out with like the, the version 1000.

• 1:18:08Speaker 01

  I think the good news. The good news is that analyzing text seems to be easier than generating text. So I believe that the sort of ability of language models to actually analyze language model output and figure out what is problematic or dangerous will actually be the solution to a lot of these control issues. We are definitely, definitely working on this stuff.

• 1:18:51Speaker 01

  We've got a bunch of brilliant folks at Google working on this now. And I think it's just going to be more and more important, both from a do something good for people standpoint, but also from a business standpoint that you are a lot of the time limited in what you can deploy based on keeping things safe. And so it becomes very, very important to be really, really good at that.

• 1:19:28Dwarkesh Patel?

  Yeah, obviously, I know you guys take the potential benefits and costs here seriously, and you guys get credit for it, but not enough, I think, really. It's like, there's so many different applications that you have put out for using these models to make the different areas you talked about better. But I do think that there, again, if you have a situation where plausibly there's some feedback loop process, on the other end, you have like a model that is as good as Noam Chizzi or as good as Jeff Dean, if like, if there's an evil version of you running around, and suppose there's like a million of them.

• 1:20:02Speaker 01

  Yes.

• 1:20:03Dwarkesh Patel?

  I think that's like, really, really bad.

• 1:20:04Speaker 01

  Yeah.

• 1:20:05Dwarkesh Patel?

  That could be like, much, much worse than any other risk, maybe sort of like nuclear war or something. Like, just think about it, like a million evil Jeff Deans or something.

• 1:20:13Speaker 01

  Where do we get the training?

• 1:20:14Dwarkesh Patel?

  Yeah. But to the extent that you think that's like a plausible output of some quick feedback loop process. What is your plan of like, okay, we've got Gemini three or Gemini four, and we think it's like helping us do better job of training future versions. It's writing a bunch of the training code for us from this point forward. We just kind of like look over it, verify it. Even the verifiers you talked about of looking at the output of these models will eventually be trained by, or you know, a lot of the code will be written by the AIs you make. You know, like, what do you want to know for sure before we like have the Gemini four help us with AI research?

• 1:20:54Dwarkesh Patel?

  We really want to make sure, we want to run this test on it before we like let it write our AI code for us.

• 1:21:00Speaker 02

  I mean, I think having the system explore algorithmic research ideas seems like something where there's still a human in charge of that. That it's exploring the space and then it's going to like get a bunch of results. And we're going to make a decision like, are we going to incorporate this particular, you know, learning algorithm or change to AI system in into kind of the core code base. And so I think you can put in safeguards like that that enable the system, to enable us to get the benefits of the system that can sort of improve or kind of self-improve with human oversight, without necessarily letting the system go full on self-improving without any, any notion of a person looking at what it's doing.

• 1:21:45Speaker 02

  Right? That's the kind of engineering safeguards I'm talking about where you want to be kind of looking at the characteristics of the systems you're deploying, not deploy ones that are harmful by some measures and some ways. And you have in understanding what its capabilities are and what it's likely to do in certain scenarios. So, you know, I think it's not an easy problem by any means, but I do think it is possible to make these, these systems safe.

• 1:22:15Dwarkesh Patel?

  Yeah, I mean, I think, I think we are also going to use these systems a lot to check themselves, check other systems. You know, it's.

• 1:22:25Speaker 01

  I mean, even as a human, it is easier to recognize something than to generate it. So.

• 1:22:33Speaker 02

  One thing I would say is if you expose the model's capabilities through an API or through a user interface that people interact with, you know, I think then you have a level of control to understand how is it being used and and sort of put some boundaries on what it can do. And that, I think, is one of the tools in the arsenal of like, how do you? How do you make sure that what it's going to do is is sort of acceptable by some set of standards you've you've set out in your mind?

• 1:23:02Speaker 01

  Yeah, I mean, I think the goal is to empower people, but, you know, so for the most part, you know, we should be mostly letting letting people to do things with these systems that that makes sense and, you know, closing off as few few parts of the space as we can. But, you know, yeah, if you if you let somebody take your thing and create a million evil software engineers, then that doesn't empower people because they're going to they're going to hurt others with a million evil software engineers. So I I'm against that.

• 1:23:33Speaker 02

  Me too. I'll go on.

• 1:23:35Dwarkesh Patel?

  All right, let's talk a little few more fun topics.

• 1:23:40Dwarkesh Patel?

  Over the last 25 years, what was the most fun time? What period of time do you have the most nostalgia over?

• 1:23:47Speaker 02

  I mean, I think the early sort of four or five years at Google, when I was sort of one of a handful of people working on search and crawling and search and indexing systems and our traffic was growing tremendously fast and we were trying to expand our index size and make it so we updated it, you know, every minute instead of every, you know, month or two months if something went wrong. And seeing kind of the the growth and usage of our systems was really just personally satisfying, you know, building something that is used by, you know, today, 2 billion people a day, I think, is is pretty, pretty incredible. But I would also say equally exciting is sort of working with people in the Gemini

• 1:24:29Speaker 02

  team today. And I think the progress we've been making in what these models can do over the last year and a half or whatever is really fun. People are really dedicated, really excited about what we're doing. I think the models are getting better and better at pretty complex tasks. Like if you showed someone using a computer 20 years ago what these models are capable of, they wouldn't believe it, right? And even five years ago, they might not believe it. And that's pretty satisfying. And I think we'll see a similar growth and usage of these models and impact in the world.

• 1:25:02Dwarkesh Patel?

  Yeah, I'm with you.

• 1:25:03Speaker 01

  I'm with you. Early days were super fun. I mean, part of that is just like knowing everybody and the social aspect and the fact that you're just building something that millions and millions of people are using. And same thing today, we got that whole nice micro kitchen area where you get lots of people hanging out with. I love being in person. Work with a bunch of great people and build something that's helping millions to billions of people. Yeah, what could be better?

• 1:25:36Dwarkesh Patel?

  What's this micro kitchen?

• 1:25:37Speaker 02

  Oh, we have a micro kitchen area in the building we both sit in. It's the new so named gradient canopy. It used to be named Charleston East and we decided we needed a more exciting name because it's a lot of like machine learning researchers and AI research happening in there. And there's a micro kitchen area that we've set up with, normally it's just like an espresso machine and a bunch of snacks. But this particular one has a bunch of space in it. So we've set up like maybe 50 desks in there. And so people are just hanging out in there. It's a little noisy because people are always grinding beans and espresso. But you also get a lot of like face to face ideas of connections like, oh,

• 1:26:21Speaker 02

  I've tried that. Like, did you think about trying this in your idea? Or, oh, we're going to launch this thing next week. Like, how's the load test looking? There's just like lots of feedback that happens. And then we have our Gemini chat rooms for people who are not in that micro kitchen. We have a team all over the world. And you know, there's probably 120 chat rooms I'm in related to Gemini things. And you know, this particular very focused topic, we have like seven people working on this and there's like exciting results being shared by the London colleagues. And when you wake up, you see like what's happening in there, or it's a big group of like people focused on data and there's all kinds of issues, you know, happening in there. It's just fun.

• 1:27:04Dwarkesh Patel?

  What I find remarkable about some of the calls you guys have made is you're anticipating a level of demand for compute, which at the time wasn't obvious or evident, TPUs being a famous example of this, or the first TPU being an example of this. That thinking you had in, I guess, 2013 or earlier, if you were, if you think about it that way today and you do an estimate of, look, we're going to have these models that are going to be a backbone of our services and we're going to be doing constantly inference for them, we're going to be training future versions. And you think about the amount of compute we'll need by 2030 to accommodate all these use cases. Where does the Fermi estimate get you?

• 1:27:44Speaker 02

  Yeah, I mean, I think you're going to want a lot of inference compute is the, the rough highest level view of these capable models because if one of the techniques for improving their quality is scaling up the amount of inference compute you use, then all of a sudden, what's currently like one request to generate some tokens now becomes 50 or 100 or 1,000 times as computationally intensive, even though it's producing the same amount of output. And you're also going to then see tremendous scaling up of the uses of these services as, you know, not everyone in the world has discovered these, you know, chat based conversational

• 1:28:26Speaker 02

  interfaces where you can get them to do all kinds of amazing things. You know, probably 10% of the computer users in the world have discovered that today or 20% as they, that pushes towards 100% and people make heavier use of it. You know, that's going to be another, you know, order of magnitude or two of scaling. And so you're now going to have, you know, two orders of magnitude from that, two orders of magnitude from that. Models are probably going to be bigger. You'll get another order of magnitude or two from that. And there's a lot of inference compute you want. So you want extremely efficient hardware for inference for models you care about.

• 1:29:03Dwarkesh Patel?

  In FLOPs, the global, total global inference.

• 1:29:08Dwarkesh Patel?

  In 2030.

• 1:29:09Speaker 01

  I I think just more is always going to be better. Like, like, I, I, like, if you just kind of think about, OK, like, what fraction of world GDP will be, you know, will people decide to spend on, on AI at that point? And then, like, OK, what, what do the AI systems look like? Well, maybe it's some sort of personal assistant like thing that is in your glasses and can see everything around you and has access to all your digital information and the world's digital information. And like, maybe it's like you're Joe Biden and you have the earpiece in the cabinet that

• 1:29:54Speaker 01

  can advise you about anything in real time and solve problems for you and give you helpful pointers, or you could talk to it. And, you know, it wants to analyze like anything that it sees around you for any potential useful impact that it has on you. So, I mean, I can imagine, OK. And then, then say it's like your, OK, your personal assistant or your personal cabinet or something, and that every time you spend 2X as much money on compute, the thing gets like 5, 10 IQ points smarter or something like that. And, OK, do you, would you rather spend like $10 a day and have an assistant or $20 a

• 1:30:37Speaker 01

  day and have a smarter assistant, you know? And not only is it an assistant in life, but an assistant in getting your job done better, because now it makes you from a 10X engineer to a 100X or 10 million X engineer.

• 1:30:55Speaker 01

  I mean, OK, OK, so, so, OK, so, let's see. From first principles, right? So, so people are going to want to, want to spend. Some some fraction of world GDP on this thing. The world GDP is almost certainly going to go way, way up to like orders of magnitude higher than it is today due to the fact that we have all of these artificial engineers like working on improving things. Probably we will have solved unlimited energy and and like carbon issues by that point. So we should be able to have lots of energy. We should be able to have millions to billions of robots like building us data centers.

• 1:31:38Speaker 01

  Like, let's see, what like the the sun is, what, 10 to the 26 watts or something like that? You know, I mean, I I'm guessing that the that the amount of compute at the, you know, being used for AI to help each person will be astronomical.

• 1:31:59Speaker 02

  I mean, I would add on to that. I'm not sure I agree completely, but it's a pretty interesting thought experiment to go in that direction. And even if you get partway there, it's definitely going to be a lot of compute. And this is why it's super important to have as cheap and a hardware platform for using these models and applying them to to problems that NAM described, so that you can then make it accessible to everyone in in some form and have, you know, as low a cost for access to these capabilities as you possibly can. And I think that's achievable by focusing on, you know, hardware and model co-design kinds of things.

• 1:32:41Speaker 02

  And we should be able to make these things much, much more efficient than they are today.

• 1:32:46Dwarkesh Patel?

  Is the is Google's data center build out plan over the next few years aggressive enough, given this increase in demand you're expecting?

• 1:32:56Speaker 02

  I'm not going to comment on our future capital spending because our CEO and CFO would prefer I don't probably, but I will say, you know, you can look at our past capital expenditures over the last few years and see that we're, we're definitely investing in this area because we think it's important and that we're, you know, we're continuing to build new and interesting innovative hardware that we think really helps us have an edge in deploying these systems to more and more people, both training them and also how do we make them usable by people for inference?

• 1:33:31Dwarkesh Patel?

  One thing I've heard you talk a lot about is continual learning, the idea that you could just have a model which improves over time rather than having to start from scratch. Is there any fundamental impediment to that? Because theoretically, you should just be able to keep fine tuning a model or yeah, what does that future look like to you?

• 1:33:50Speaker 02

  Yeah, I've, I've been thinking about this more and more and I've been a big fan of models that are sparse because I think you want different parts of the model to be good at different things. And we have, you know, our Gemini 1.5 Pro model and other models are mixture of expert style models where you now have parts of the model that are activated for, for some token and parts that are not activated at all because you decided this is a math oriented thing and this part's good at math and this part's good at like understanding cat images. So, so that gives you this ability to have a much more capable model that's still quite efficient at inference time because it has very large capacity, but you activate a small

• 1:34:35Speaker 02

  part of it. But I think the current problem, well, one, one limitation of what we're doing today is it's still a very regular structure where each of the experts is kind of the same size. You know, they, they, the paths kind of merge back together very fast. They don't sort of go off and sort of have lots of different branches for mathy things that don't merge back together with the. The kind of cat image thing. And I think we should probably have a more organic structure in these things. I also would like it if the pieces of those model of the model could be developed a little bit independently. Like right now, I think we have this issue where we're going to train a model.

• 1:35:15Speaker 02

  So we do a bunch of preparation work on deciding the most awesome algorithms we can come up with. And the most awesome data mix we can come up with, but there's always trade offs there. Like we'd love to include more multilingual data, but that might come at the expense of including less coding data. And so the model's less good at coding, but better multilingual or vice versa. And I think it would be really great if we could have like a small set of people who care about a particular subset of languages. Go off and create really good training data, train, you know, a modular piece of a model that we can then hook up to a larger model.

• 1:35:54Speaker 02

  That improves its capability in say Southeast Asian languages or in, you know, reasoning about

• 1:36:03Speaker 02

  Haskell code or something. And, and then you then also have a nice software engineering benefit where you've decomposed the problem of it. Compared to what we do today, which is we have this kind of a whole bunch of people working. But then we have this kind of monolithic process of starting to do pre-training on this model. And if we could do that, you know, you could have a hundred teams around Google. You could have people all around the world working to improve. You know, languages they care about or particular problems they care about and all collectively work on improving the, the model. And that's a kind of a form of continual learning.

• 1:36:37Speaker 01

  That would be so nice. You could just like glue models together or rip out pieces of models and shove them into other like Dr. Frankenstein-y kind of thing. Or like you just attach a fire hose and you suck all the information out of this model.

• 1:36:51Speaker 02

  Yeah.

• 1:36:51Speaker 01

  Shove it into another model. There is, I mean, the countervailing

• 1:37:00Speaker 01

  interest there is sort of science in terms of like, okay, we're still in the period of rapid progress.

• 1:37:07Speaker 02

  Yep.

• 1:37:07Speaker 01

  So if you want to do sort of controlled experiments and, okay, you know, I want to compare this thing to that thing because that is helping us figure out, okay, what do you want to build? So for a thing, you know, in that interest, it's often best to just start from scratch. So you can compare one complete training run to another, you know, to another complete training run sort of at a practical level. Because it kind of helps us figure out what to, you know, what to build in the future. And it's less exciting, but, uh, but does lead to rapid progress.

• 1:37:42Speaker 02

  Yeah, I think there may be ways to get a lot of the benefits of that with kind of a version system of modularity. Like I have a frozen version of my model and then I include a different variant of some particular module. And I want to compare its performance or train it a bit more.

• 1:37:58Speaker 01

  Oh, yeah.

• 1:37:58Speaker 02

  And then I compare it to the baseline of this thing with, you know, now version, you know, N prime of this particular module that does Haskell interpretation.

• 1:38:08Speaker 01

  Right, actually that can lead to faster research progress, right? You've got some system.

• 1:38:12Speaker 02

  Yeah.

• 1:38:12Speaker 01

  And you do something to improve it. And if that thing you're doing to improve it is relatively cheap compared to training the system from scratch, then it could actually make. Yeah, that could actually make research much, much cheaper and faster.

• 1:38:27Speaker 02

  Yeah.

• 1:38:27Speaker 01

  So, okay.

• 1:38:28Speaker 02

  And also more, more parallelizable, I think.

• 1:38:31Speaker 01

  Yeah.

• 1:38:32Speaker 02

  Okay, because you, across people.

• 1:38:34Speaker 01

  Okay, let's figure it out and do that next.

• 1:38:36Speaker 02

  Yeah.

• 1:38:39Dwarkesh Patel?

  So this is, um, this idea that is sort of casually laid out there is actually it would be a big regime shift.

• 1:38:45Speaker 02

  Yeah.

• 1:38:45Dwarkesh Patel?

  You think this is the way things are headed. This is like, this is a sort of like very interesting prediction about, uh, you just have this like blob where things are getting pipeline back and forth. Then if you want to make something better, you can do like a sort of surgical incision almost, right?

• 1:39:01Speaker 02

  Or grow the model, add another little bit of it here. Yeah, I've been sort of sketching out this vision for a while in the sort of pathways.

• 1:39:09Dwarkesh Patel?

  Yeah.

• 1:39:09Speaker 02

  Under Pathways name.

• 1:39:10Speaker 01

  Yeah, you've been building the.

• 1:39:11Speaker 02

  And we've been building the infrastructure for it. So a lot of what Pathways, the system can support is this kind of twisty weird model with like asynchronous updates to different pieces.

• 1:39:22Speaker 01

  Yeah, we should go back and figure out.

• 1:39:24Speaker 02

  And we're using pathways to train our Gemini models, but we're not making use of some of its capabilities yet. But yeah, maybe we should.

• 1:39:31Speaker 01

  Maybe. There have been times like, you know, like the way the TPU pods were set up. I don't know who did that, but they did a pretty brilliant job. You know, the low level software stack and the hardware stack that, okay, you've got your, you know, you've got your nice, regular, high-performance hardware. You've got these great torus-shaped interconnects. And then you've got the right low-level collectives, you know, the all-reduces, et cetera, which I guess came from supercomputing, but it turned out to be kind of just the right thing to build, to build distributed deep learning on top of.

• 1:40:13Dwarkesh Patel?

  The, okay, so a couple of questions. One, suppose you do figure, suppose Noam makes another breakthrough and now we've got a better architecture. Would you just take each compartment and distill it into this better architecture? And that's how it keeps improving over time?

• 1:40:30Speaker 02

  Yeah, I mean, I do think distillation is a really useful tool because it enables you to kind of transform a model in its current model architecture form into a different form. You know, often you use it to take a really capable but kind of large and unwieldy model and distill it into a smaller one that maybe you want to serve with really good, fast latency inference characteristics. But I think you can also view this as something that's happening at the modularity, at the module level. Like, maybe there'd be a continual process where you have each module and it has a few different representations of itself. It has a really big one. It's got a much smaller one that is continually distilling into the small version. And then the small version.

• 1:41:15Speaker 02

  Once that's finished, then you sort of delete the big one and you add a bunch more parameter capacity and now start to learn all the things that the distilled small one doesn't know by training it on more data. And then you kind of repeat that process. And if you have that kind of running a thousand different places in your modular model in the background, that seems like it would work reasonably well.

• 1:41:38Dwarkesh Patel?

  This would be a way to do doing inference scaling, like the router decides how much

• 1:41:42Speaker 02

  Yeah, you can have multiple versions and like, you know, oh, this is an easy math problem. So I'm going to route it to the really tiny math distilled thing. And oh, this one's really hard. So.

• 1:41:52Dwarkesh Patel?

  At least from public research, it seems like it's often hard to decode what each expert is doing in mixture of expert type models. If you have something like this, how would you enforce the kind of modularity that would be visible and understandable to us?

• 1:42:07Speaker 01

  Actually, in the past, I found experts to be relatively easy to understand. I mean, I don't know, the first mixture of experts paper, you could just like.

• 1:42:17Dwarkesh Patel?

  I don't know. I'm only the inventor mixture.

• 1:42:21Speaker 01

  Like, yeah, you could just see, okay, like this expert, like we did, you know, 1000, 2000 experts. Okay. And this expert, like all of the was getting words referring to cylindrical objects, you know, like.

• 1:42:33Speaker 02

  This one's super good at dates.

• 1:42:35Speaker 01

  Yeah.

• 1:42:36Speaker 02

  Talking about time.

• 1:42:37Speaker 01

  Was actually, yeah, pretty easy to do. But I mean, like, not that you would need that human understanding to like, figure out how to like work the thing at runtime, because you you just have like some sort of learned router that's looking at the example.

• 1:42:54Speaker 02

  And. I mean, one thing I would say is like, there is a bunch of work on interpretability of models and what are they doing inside? And sort of expert level interpretability is a sub problem of that that broader area. I really like some of the work that my former intern Chris Hola and others did at Anthropic where they could kind of they they trained a very sparse autoencoder and were able to deduce, you know, what characteristics does some particular neuron in a large language. So they found like a Golden Gate Bridge neuron that's activated. When you're talking about the Golden Gate Bridge. And I think, you know, you could do that at the expert level. You could do that at a variety of different levels and get pretty interpretable results.

• 1:43:38Speaker 02

  And it's a little unclear if you necessarily need that. If the model is just really good at stuff, you know, you know, we don't necessarily care what every neuron in the Gemini model is doing as long as the collective output and characteristics of the the overall system are good. You know, that's one of the beauties of deep learning is you don't need to understand or hand engineer every last feature.

• 1:44:01Dwarkesh Patel?

  Man, there's so many interesting implications of this that we could just keep, I could just keep asking you about this. One implication is currently if you have a model that has some tens or hundreds of billions of parameters, you can serve it on like a handful of GPUs. In this system where any one query might only make its way through a small fraction of the total parameters, but you need the whole thing sort of loaded into memory. The specific kind of infrastructure that Google has invested in with these TPUs that exist in pods of hundreds or thousands would be like immensely valuable, right?

• 1:44:41Speaker 01

  I mean, for any sort of even existing mixtures of experts, you want the whole thing in memory.

• 1:44:47Dwarkesh Patel?

  Yeah.

• 1:44:47Speaker 01

  I mean, basically, if you are, I guess there's kind of this misconception running around with like mixture of experts that, okay, the benefit is that, you know, the you don't even have to like go through those weights in the model, you know, if some expert is unused. It doesn't mean that you don't have to retrieve that memory because really in order to be efficient, you're serving at very large batch sizes.

• 1:45:15Speaker 02

  Of independent requests.

• 1:45:17Speaker 01

  Of independent, right, of independent requests. So it's not, it's not really the case that, okay, at this step, you're either looking at this expert or you're not looking at this expert. Because if that were the case, then when you did look at the expert, you would be running it at batch size one, which is like.

• 1:45:35Dwarkesh Patel?

  Massively inefficient, like you've got like modern hardware.

• 1:45:39Speaker 02

  Right.

• 1:45:40Dwarkesh Patel?

  The the the operational intensities are whatever hundreds or, you know, so so. So so that's not what what's happening. It's that you are looking at all the experts, but you only have to send a small fraction of the batch through each one.

• 1:45:56Speaker 02

  Right, but you still have a smaller batch at each expert that then goes through. And in order to get kind of reasonable balance, like one of the things that the current models typically do is they have all the experts be roughly the same compute cost. And then you run roughly the same sized batches through them in order to sort of propagate the the very large batch you're you're doing at inference time in and have good efficiency. But I think, you know, you often in the future might want experts that vary in computational cost by factors of a hundred or a thousand.

• 1:46:30Dwarkesh Patel?

  Yeah.

• 1:46:31Speaker 02

  And or maybe paths that go for many layers on one one case and, you know, a single layer or even a skip connection in the other case. And there, I think you're going to want very large batches still, but you're going to want to kind of push things through the model a little bit asynchronously for at inference time, which is a little easier than than training time. And, you know, that's part of kind of one of the things that PathWave was designed to support is, you know, you have these components and the components can be variable cost. And you kind of can say for this particular example, I want to go through this subset of the model. And for this example, I want to go through this subset of the model and have them kind of, the system kind of orchestrate orchestrate that.

• 1:47:16Dwarkesh Patel?

  Ah. It also would mean that it would take companies of a certain size and sophistication to be able to, like right now, you you know, anybody can train a sufficiently small enough model. But if we if it ends up being the case that this is the best way to train future models, then you would need a company that can basically have a data center sized, a data center serving a single quote unquote blob or, you know, a model. So it would be it would be interesting change in. Paradigms in that way as well.

• 1:47:49Speaker 01

  You definitely want to have at least enough enough HBM to put to put your whole model. So depending on the size of your model, most likely that's how much, you know, that's how much HBM you'd want to have at a minimum. I mean, yeah.

• 1:48:08Speaker 02

  But it also means, I think, you don't necessarily need to grow your entire model footprint to be the size of the data center. You might want it to be a bit below that and then have, you know, potentially many replicated copies of one particular expert that is being used a lot so that you get better load balancing.

• 1:48:27Speaker 01

  Right.

• 1:48:27Speaker 02

  So like this one's being used a lot because we get a lot of math questions and this one on, you know, maybe it's an expert on Tahitian dance and it is called on really rarely. That one, maybe even page out to DRAM rather than putting it in HBM. But you want the system to kind of figure all this stuff out based on load characteristics.

• 1:48:47Dwarkesh Patel?

  How, right now, language models, obviously, like you put in language, you get language out, obviously it's multimodal. But you could imagine, the Pathways blog post talks about like, sort of like so many different use cases that are not obviously of this kind of auto-aggressive nature.

• 1:49:07Speaker 02

  Right.

• 1:49:07Dwarkesh Patel?

  Going through the same model. So could you imagine like basically Google as a company, the product is like, Google Search goes through this, Google Images goes through this, Gmail goes through, it's just like the server, the entire server is just this huge mixture of experts specialized.

• 1:49:24Speaker 02

  I mean, you're, you're starting to see some of this by having a lot of uses of Gemini models across Google that are not necessarily, you know, fine tuned. They're just sort of, you know, given instructions for this particular use case in this feature in this product setting. So I definitely see a lot more sharing of what the underlying models are capable of across more and more services. You know, I do think that's, that's a pretty interesting direction to go for sure.

• 1:49:53Dwarkesh Patel?

  I feel like people listening might not sort of register how, yeah, like how interesting we prediction this is about where it is. It's like sort of like getting like no one on a podcast in 2018 and being like, yeah, so I think like, you know, language models will be a thing. It's like, if this is where things go, this is actually, yeah, that's incredibly interesting.

• 1:50:14Speaker 02

  Yeah, and I think you might see, that might be a big base model, and then you might want customized versions of that model with different modules that are added on to it for different settings that maybe have access restrictions, like maybe we have an internal one for Google use for Google employees that we've trained some modules on internal data and we don't allow anyone else to use those modules, but we can make use of it. And maybe other companies, you add on other modules that are useful for that company's setting and serve it in our cloud APIs.

• 1:50:46Dwarkesh Patel?

  What is the bottleneck to making this sort of system viable? Is it, is it like systems engineering? Is it ML? Is it?

• 1:50:54Speaker 02

  I mean, it's a pretty different way of operating than our current Gemini development. So I think, you know, we will, we will explore these kinds of areas and I think make some progress on them, but we need to sort of really see evidence that it's the right way, you know, that it has a lot of benefits. Some of those benefits may be improved quality, some may be sort of less concretely measurable, like this ability to have lots of parallel development of different modules. And I think that would, but that's still a pretty exciting improvement because I think that then that would enable us to make faster progress on improving the model's capabilities for lots of different distinct areas.

• 1:51:37Speaker 01

  I mean, even the data control modularity stuff seems like really cool because then you could have like the piece of the model that's just trained for me. It knows all my private data.

• 1:51:45Speaker 02

  Yeah, like a personal module for you would be useful. Another thing might be you can use certain data in some settings, but not in other settings. And, you know, maybe we have some, some YouTube data that's only usable in a YouTube product surface, but not in other settings. We could have a module that is trained on that data for that particular purpose.

• 1:52:05Speaker 01

  We are, we're going to need like a million automated researchers to invent all of this stuff.

• 1:52:13Speaker 02

  It's gonna be great.

• 1:52:14Dwarkesh Patel?

  Well, the thing itself, you know, it's like you build the blob and it like tells you how to make the blob better and.

• 1:52:21Speaker 02

  Blob 2.0.

• 1:52:24Speaker 02

  Or maybe they're not even versions, it's just like an incrementally growing blob.

• 1:52:28Dwarkesh Patel?

  Yeah. Okay, Jeff, motivate for me, big picture, why is this a good idea? Why is this the next direction?

• 1:52:36Speaker 02

  Yeah, I mean, I guess this kind of like notion of an organic, like, kind of not quite so carefully mathematically constructed machine learning model is one that's been with you for a little while. And I feel like in the development of neural nets, like the biological analog, the artificial neurons, you know, inspiration from biological neurons is a good one and has served us well in the deep learning field. And we've been able to make a lot of progress with that. But I feel like we're not necessarily looking at other things that real brains do as much as we perhaps could. And that's not to say we should exactly mimic that because silicon and, you know,

• 1:53:19Speaker 02

  wetware have very different characteristics and strengths. But I do think one thing we could draw inspiration, more inspiration from is this notion of having different specialized portions, part, sort of areas of a model, of a brain that are good at different things. So we have a little bit of that, a mixture of experts' models, but it's still very kind of structured. And I feel like this kind of more organic growth of expertise. And when you want more expertise of that, you kind of add some more capacity to the model there and let it learn a bit more on that kind of thing. And also this notion of like adapting the connectivity of the model to the connectivity of the hardware is a good one.

• 1:54:02Speaker 02

  So I think you want incredibly dense connections between artificial neurons in sort of the same chip and the same HBM, because that's, doesn't cost you that much. But then you want a smaller number of connections to nearby neurons. So like a chip away, you should have some amount of connections.

• 1:54:26Dwarkesh Patel?

  Right.

• 1:54:27Speaker 02

  And then like many, many chips away, you should have a smaller number of connections where you send over a very limited kind of bottlenecky thing, the most important things for that this part of the model is learning that for other parts of the model to make use of. And even across, you know, multiple TPU pods, you'd like to send even less information, but the most salient. Kind of representations and then across metro areas, you'd like to send even less.

• 1:54:52Dwarkesh Patel?

  Yeah. And then that emerges organically.

• 1:54:55Speaker 02

  Yeah, I'd like that to emerge organically. Like you could hand specify these these characteristics, but I think you don't know exactly what the right proportions of these kinds of connections. And so you should just let the hardware dictate things a little bit. Like if you're communicating over here and this data always shows up really early, you should add some more connections.

• 1:55:13Dwarkesh Patel?

  Yeah.

• 1:55:14Speaker 02

  Then it will make it take longer and show up at just the right time.

• 1:55:18Dwarkesh Patel?

  Oh, here's another interesting implication potentially. Right now, we think about the growth in AI use as a sort of horizontal. So suppose you're like, how many AI engineers will Google have working for it? You think about like, how many instances of Gemini three will be working at one time? If you have this, whatever you want to call this like blob, and it can sort of like organically decide how much of itself to activate, then it's more of like, you know, if you want like 10 engineers worth of output, it just activates a different pattern or a larger pattern. If you want 100 engineers of output, it's not it's not like calling more agents or more instances.

• 1:56:02Dwarkesh Patel?

  You're just calling different subsets.

• 1:56:04Speaker 02

  I think there's a notion of like, how much compute do you want to spend on this particular inference? And that should vary by like factors of 10,000 for really easy things and really hard things, maybe even a million. And it might be iterative, like you might make a pass through the model, get some stuff and then decide you now need to call on some other parts of the model as another, you know, aspect of it. The other thing I would say is like, this sounds super complicated to deploy because it's like this this weird, you know, constantly evolving thing with maybe not super optimized ways of communicating between pieces. But you could always distill from that, right? Like, so if you say, this is the kind of task I really care about,

• 1:56:47Speaker 02

  let me distill from this giant kind of like organicy thing into something that I know can be served really efficiently. And you could do that distillation process, you know, whenever you want, once a day, once an hour. And that that seems like it'd be kind of kind of good.

• 1:57:02Speaker 01

  Yeah, we need better distillation.

• 1:57:03Speaker 02

  Yeah.

• 1:57:04Speaker 01

  Anyone out there invents amazing distillation techniques that instantly distill from a giant blob onto your phone, that would be wonderful.

• 1:57:12Dwarkesh Patel?

  How would you characterize what's missing from current distillation techniques?

• 1:57:15Speaker 01

  Well, I just want it to work faster.

• 1:57:17Speaker 02

  Yeah, a related thing is I feel like we need interesting learning techniques during pre-training. Like, I'm not sure we're extracting the maximal value from every token we look at with the current training objective. Right? Like, maybe we should think a lot harder about some tokens. You know, when you get to the answer is, maybe the model should, at training time, do a lot more work than

• 1:57:43Speaker 02

  when it gets to the.

• 1:57:45Speaker 01

  Right, right. Yeah, I mean, right, there's gotta be some, some way to get more from the same data, make it learn it forwards and backwards and.

• 1:57:53Speaker 02

  Yeah, every which way, like hide it, hide some stuff this way, hide some stuff that way, make it infer from like partial information. You know, these kinds of things. I think people have been doing this in vision models for a while, like you, you distort the model or you hide parts of it and try to make it guess the bird from half, like that it's a bird from this upper corner of the image or the lower left corner of the image. And that makes the task harder. And I feel like there's an analog for kind of more textual or coding-related data where you wanna, you know, force the model to work harder and you'll get more, more interesting observations from it.

• 1:58:32Speaker 01

  Yeah, the image people didn't have enough label data, so they had to, you know, all this.

• 1:58:36Speaker 02

  Yeah, and like they invented, I mean, dropout was invented on images, but we're not really using it for text mostly. That's one way you could get a lot more learning in a more large-scale model without overfitting is just make like a hundred epochs over the world's text data and use, use dropout.

• 1:58:57Speaker 01

  Yeah.

• 1:58:57Speaker 02

  But that's pretty computationally expensive, but it does mean we won't run it, like even though people are saying, oh no, we're almost out of like textual data. I don't really believe that because I think we can get a lot more capable models out of the text data that does exist.

• 1:59:13Speaker 01

  I mean, like a person has seen like a billion tokens, so.

• 1:59:16Speaker 02

  Yeah, and they're, they're pretty good at a lot of stuff.

• 1:59:18Speaker 01

  Yeah.

• 1:59:19Dwarkesh Patel?

  Obviously, human data efficiency sets a lower bound on how, or I guess upper bound. Uh, one of them on.

• 1:59:27Speaker 02

  Maybe it's an interesting data point.

• 1:59:29Dwarkesh Patel?

  Yes. The, uh, so there's a sort of like modus ponens, modus tollens thing here of one way to look at it is, look, LLMs have so much further to go, therefore we project, you know, orders of magnitude improvement and sample efficiency, just if they could match humans. Another is maybe they're doing something clearly different, given the orders of magnitude difference. What's your intuition of what it would take to make these models as sample efficient as humans are?

• 1:59:57Speaker 02

  Yeah, I mean, I think we, we should consider changing the training objective a little bit. Like just predicting the next token from the previous ones you've seen seems like not how people learn.

• 2:00:08Dwarkesh Patel?

  Right.

• 2:00:09Speaker 02

  It's a little bit related to how people learn, I think, but not, not entirely. Like a person might, you know, read a whole chapter of a book and then try to answer questions at the back. And that's, that's a kind of different kind of thing. I also think we're not learning from visual data very much. You know, we're training a little bit on video data, but we're definitely not anywhere close to thinking about training on, on all the visual inputs you could get. You know, so you have visual data that, that we haven't really begun to train on. And then I think we could extract a lot more information from every, every bit of, bit of data we, we do see. You know, I think one of the ways people are so sample efficient is they explore the world and take actions in the world and observe what happens.

• 2:00:52Dwarkesh Patel?

  Yeah.

• 2:00:52Speaker 02

  Right? Like you see it with very small infants, like picking things up and dropping them. They learn about gravity from that. And then that's a much harder thing to learn when you're not initiating the action. And I think having a model that can take actions as part of its learning process would be just a lot better than just sort of passively observing a giant data set.

• 2:01:12Dwarkesh Patel?

  Is GAT or the future then?

• 2:01:14Speaker 02

  Something where the model can observe and take actions and observe the, the corresponding results seems pretty useful. I mean.

• 2:01:26Speaker 01

  People can learn a lot from thought experiments that don't even involve extra input and like Einstein learned a lot of stuff from thought experiments or like Newton, like went into quarantine and got an apple dropped on his head or something and invented gravity and like mathematicians like, you know, map didn't have any extra input chess, like, okay, like you have the thing play chess against itself and it gets good at chess, that was DeepMind, but also like all it needs is the rules of chess. So like, there's actually probably a lot of somehow, a lot of learning that you can do even without external data.

• 2:02:06Dwarkesh Patel?

  Yeah.

• 2:02:07Speaker 01

  And then you can make it in exactly the, the fields that you care about. Um, you know, of course, there is learning that will require external data, but probably maybe we can just have this thing talk to itself and make itself smarter.

• 2:02:23Dwarkesh Patel?

  So here's a question I have.

• 2:02:25Speaker 01

  Yeah.

• 2:02:26Dwarkesh Patel?

  What you've just laid out over the last hour is potentially just like the big next paradigm shift in AI. That's like a tremendously valuable insight potentially. Um, uh, how do you know in 2017, you released the transformer paper on which tens, if not hundreds of billions of dollars of market value is based in other companies, not to mention all this other research that Google has released over time, um, which, uh, you know, you've been like relatively generous with. Um, in retrospect, when you think about divulging this information that has been helpful to your competitors, in retrospect, is it like, yeah, we'd still do it? Or would you be like, ah, we didn't realize how big a deal a transfer was. We should have kept it indoors. Um, how do you think about that?

• 2:03:11Speaker 01

  That's a, it's a good question because I, I think probably, you know, we, we, uh, didn't need to see the size of the opportunity like often reflected in, uh, you know, in, what other companies are doing. And also it's not a fixed pie. Like this is like the, the, the current state of the world is.

• 2:03:35Speaker 01

  Pretty much as far from fixed pie as you can you can get. I think we're going to see like orders of magnitude of improvements in GDP, health, well, and anything else you can think of. So, you know, I think it's, you know, definitely been nice that Transformer has got around.

• 2:03:59Speaker 02

  It's transformative.

• 2:04:02Speaker 01

  Thank God Google's doing well as well. So, you know, these days we do we do publish a little less of what we're doing. But, you know.

• 2:04:13Speaker 02

  Yeah, I mean, I think there's always this trade off of, you know, should we publish exactly what we're doing right away? Should we put it in, you know, the next stages of research and then roll it out into like production Gemini models and not publish it at all? Or is there some intermediate point? And for example, in our computational photography work in pixel cameras, you know, we've often taken the decision to develop interesting new techniques like the ability to do, you know, super, super good night sight vision for low light situations or whatever, put that into the product and then publish, you know, a real research paper about the system that does that after the product is released.

• 2:04:59Speaker 02

  And I think, you know, different techniques and developments have different treatments, right? Like, so some things we think are super critical, we might not publish. Some things we think are really interesting but important for improving our products, we'll get them out into our products and then make a decision, you know, did we publish this or do we give kind of a lightweight, you know, discussion of it, but maybe not every last detail? And then other things, I think we publish openly and try to advance the field and the community because that's how we all kind of benefit from, you know, participating. I think it's great to go to conferences like NURIPS last week with like 15,000 people, you know, all sharing lots and lots of great ideas.

• 2:05:40Speaker 02

  And, you know, we published a lot of papers there as we have in the past and, you know, see the field advance is super exciting.

• 2:05:48Dwarkesh Patel?

  How would you account for? So obviously, Google had all these insights internally rather early on, including the top researchers. And now, as of 2024, Gemini 2 is out. We didn't get a chance much to talk about, but people will know. Like, it's a really great model. Yeah, it's really good.

• 2:06:13Speaker 02

  As we say around the micro kitchen, such a good model, such a good model.

• 2:06:16Dwarkesh Patel?

  So it's top in LMSys, Chatbot Arena. And so now, Google's on top. But how would you account for, basically, coming up with all the great insights for a couple of years? Other competitors had models that were more, that were better for a while despite that.

• 2:06:33Speaker 02

  You want me to take us out?

• 2:06:34Dwarkesh Patel?

  Sure.

• 2:06:34Speaker 02

  I mean, I think, yeah, we've been working on language models for a long time. You know, Gnome's early work on spelling correction in 2001, the work on translation, very large-scale language models in 2007, and Seek2Seek, and Word2Vec, and, you know, more recently, Transformers, and then BERT, and things like the internal Mina system for, that was actually a chatbot-based system designed to kind of engage people in, you know, interesting conversations. We actually had an internal chatbot system that Googlers could play with, even before ChatGPT came out.

• 2:07:17Speaker 02

  And actually, during the pandemic, a lot of Googlers would enjoy, you know, everyone's locked down at home, and so they'd enjoy spending time chatting with Mina during lunch, because it was like a nice, indeed, a, you know, lunch partner. And, you know, I think one of the things we were a little, you know, our view of things from a search perspective was like, these models hallucinate a lot, and they don't get things right correctly, you know, a lot of the time, or some of the time. And that means that they aren't as useful as they could be. And so we'd like to make that better. And, you know, from a search perspective, you want to get the right answer, you know, a percent of the time, ideally, you know, and be very high on factuality. And these models are not near that bar.

• 2:08:02Speaker 02

  But they, I think what we were a little unsure about is that they were incredibly useful. Oh, and they also had all kinds of safety issues, like they might say offensive things and you had to work on that aspect. And get that to a point where we feel comfortable releasing the model. But I think what we've kind of didn't quite appreciate was how useful they could be for things you wouldn't ask a search engine, right? Like, help me write a note to my veterinarian or like, oh, can you take this text and give a quick summary of it? Or whatever. And I think that's the kind of thing we've seen people really, you know, flock to in terms of using chatbots as amazing new capabilities rather than as a pure search engine.

• 2:08:46Speaker 02

  And so I think, you know, we took our time and got to the point where we actually released, you know, quite capable chatbots and have been improving them through Gemini models quite a bit. And I think that's, that's actually not a bad path to have taken. Would be like to have released a chatbot earlier maybe, but I think, you know, we have a pretty awesome chatbot with awesome Gemini models that are getting better all the time and that's, that's control.

• 2:09:11Dwarkesh Patel?

  Yeah. So we've discussed some of the things you guys have worked on over the last 25 years and there's so many different fields, right? You start off with search and indexing to distributed systems, to hardware, to AI algorithms. And genuinely, there's like a thousand more. Just go on either of their Google Scholar pages or something. What is a trick to having this level of not only career longevity where you're having, you have many decades of making breakthroughs, but also the breadth of different fields? Both of you, in either order, what's trick to career longevity and breadth?

• 2:09:54Speaker 02

  Yeah, I mean, I think one, one thing that I have, that I like to do is to find out about a new and interesting area. And one of the best ways through that is to pay attention to what's going on. Talk to colleagues, like pay attention to research papers that are being published. Look at the kind of research landscape as it's evolving. You know, be willing to say, oh, uh, you know, check design. I wonder if we could use reinforcement learning for for some aspect to that and be able to dive into to a new area, work with people who know a lot about a different domain. Uh, or health AI for healthcare is on the act on a bit of work. I, uh, you know, working with clinicians about what are the real problems, you know? How could AI help?

• 2:10:37Speaker 02

  You know, it wouldn't be that useful for this thing, but it would be super useful for this. Getting those insights and often working with like a set of five or six colleagues who have different expertise than you do. Uh, enables you to collectively do something that none of you could do individually. And then some of their expertise rubs off on you and some of your expertise rubs off on them. And now you have like this bigger set of tools in your tool belt as an engineer and researcher to go tackle the next thing. And I think that's, that's one of the beauties of, you know, continuing to learn on the job. It's, it's something I, I treasure and I really like to enjoy diving into new things and, and see what we can do.

• 2:11:15Speaker 01

  I, I'd say like probably a big thing is like humility, like I'd say I'm like the most vulnerable. Uh, ever.

• 2:11:25Speaker 01

  But, but, uh, but I, but, but seriously, you know, there, there is, um, you know, to say, hey, you know, what I just did is, is not compared to what I can do or, or what can be done and to be able to like drop an idea as soon as you see something, as soon as you see something better, like you hear somebody, you know, with some better idea. And you see how maybe, maybe, maybe what you're thinking about, what they're thinking about, or something totally different can, um, you know, uh, it could conceivably work better. Cause I, I, I think there's a. Um, there is a drive in some sense to say, hey, the thing I just invented is awesome. Like, give me more chips.

• 2:12:12Speaker 01

  Particularly, if there's a lot of top-down resource assignment. But um, I, I, I think we also need to, um, you know, um, you know, incentivize uh, people to say, hey, this thing I am doing is not working at all. Let me just drop it completely and uh, you know, try, and try something else, which I think Google Brain did, did quite well with uh, with the very kind of bottoms-up uh, UBI kind of uh, chip allocation. Yeah, it's like basically everyone had one credit and you could pool them.

• 2:12:51Speaker 01

  Yeah, it's a good idea. Yeah, and, and then Jem and I, I mean, it has been like mostly top-down, which has been very good in some sense, because it, it has led to a lot more collaboration and, you know, people working together. You less often have like five groups of people all building the same thing. You're building interchangeable things uh, and, and uh, but on, but on the other hand, it does lead to some incentive to um, uh, to say, hey, what I'm doing is working great. And then, then, then like as I'll lead these, you hear like hundreds of groups and everything is. So you should give, uh, give them more chips.

• 2:13:31Speaker 01

  And there's less of an incentive to, to say, hey, what, what I'm doing is not actually working that well. Let me try, let me try something different. So I, I think going forward, we're gonna have, you know, some amount of top-down, some amount of uh, bottom-up. So as to incentivize sort of both of these behaviors, collaboration and like flexibility. Cause I think, I think both those things lead to, you know, a lot of innovation. Yeah, I, I think it's also good to kind of articulate.

• 2:13:58Speaker 02

  Interesting directions you think we should go and you know I uh I have an infernal slide deck called go Jeff wacky ideas.

• 2:14:09Speaker 02

  That I think is like, those are a little bit more like product oriented things of like, hey, I think now that we have these capabilities, we could do these.

• 2:14:17Dwarkesh Patel?

  Right.

• 2:14:17Speaker 02

  You know, seventeen things. And you know, I think that's a, that's a good thing because some, sometimes people get excited about that and want to start working with you on, on one or, one or more of them.

• 2:14:28Dwarkesh Patel?

  Yeah.

• 2:14:28Speaker 02

  I think that's a good way to kind of bootstrap, you know, uh, where we should go, uh, without necessarily ordering people, we must go here.

• 2:14:37Dwarkesh Patel?

  Yeah. Hey, this is great.

• 2:14:40Speaker 02

  Appreciate you taking the time and it was great, great.