🔬 An Oscar, Two Asteroids, and the Algorithm in Your sklearn: John Platt on AI for Science

How often do you get to talk to a guest who has both an Academy Award and who invented textbook machine learning algorithms? John Platt has an Oscar https://www.atogt.com/askoscar/display-person.php?id=78091&var=0, two textbook https://en.wikipedia.org/wiki/Platt_scaling algorithms https://en.wikipedia.org/wiki/Sequential_minimal_optimization, two named asteroids, and an Erdos-Bacon number https://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93Bacon_number of 6. This was easily the most fun bio of all the guests we’ve read to date. And the result was an epic and fun chat covering Google’s Empirical Research Assistance https://research.google/blog/empirical-research-assistance-era-from-nature-publication-to-catalyzing-computational-discovery/ (ERA), how AI can help battle climate change, and tons of great stories about the co-evolution of science and AI. John’s colleague Dave Bacon likes to tease John that his career has been defined by being twenty years early to the next big thing. This may be convolutional neural networks (some credit him with coining the term), fusion research, quantum computing. John and Google have been working on solving some of humanity’s hardest problems with AI and computation for well over a decade now. Recently John and his team set their sights on using AI to solve any scientific problem that can be written down as a score. Google’s Empirical Research Assistance (ERA) John’s team has taken on many hard scientific problems over the years. In solving these, they noticed a pattern, many scientific problems can be reduced to what John calls a “scoreable task”. Once you have the score function, the goal is to find some code that maximizes the score. The hard part is in formulating the score, but once you have the score finding the maximizer can still be quite a lot of effort. John’s team set out to automate solutions to this general problem. This came out of the idea of an “auto-Kaggle” AI, which can solve any Kaggle problem you can throw at it. Kaggle is owned by Google, so all the data was ready and easily available to them! The result is Google’s Empirical Research Assistance or ERA (paper https://www.nature.com/articles/s41586-026-10658-6, github https://github.com/google-research/era/tree/main/era_applications, blog https://research.google/blog/accelerating-scientific-discovery-with-ai-powered-empirical-software/). ERA is surprisingly simple conceptually. Gemini (or your LLM of choice) keeps a running tree of past experiments (notebooks) and where they’re going. It’s a close cousin of Monte Carlo Tree Search https://en.wikipedia.org/wiki/Monte_Carlo_tree_search: at each iteration the Upper Confidence Bound https://en.wikipedia.org/wiki/Multi-armed_bandit#Upper_Confidence_Bound_(UCB)_bandit_algorithm rule picks which notebooks are most promising to mutate. This is optimistic, not greedy, so sometimes even the fifth-best notebook gets chosen. Gemini then proposes mutations for each one, about ten at a time. The history of each branch is shared, so different leaves can learn from each other. “It’s almost like having a hyper-eager grad student who doesn’t sleep.” Evolutionary algorithms https://en.wikipedia.org/wiki/Genetic_programming have been around since the 70s, but this works because Gemini actually knows where to look! What’s even more interesting is that there was a step change between Gemini 2.0 and 2.5, and this went from just not working to working great. ERA is so powerful that John and his team solved many outstanding problems with it, resulting in at least ten papers https://github.com/google-research/era/tree/main/era_applications/pdfs. Some of these were climate change related, which we talk about in the next section. So, we had to ask: if you have an optimization god how do you avoid fooling yourself? John’s answer is that ERA provides predictive models. It’s up to the scientist to make sure they’re truly descriptive. Some of this just involves good old-fashioned careful machine learning science. “It’s a power tool. It can slice your fingers off.” This led to some fun discussion about Kaggle competitions, and the fun ways people can overfit to datasets without meaningfully solving the problem you actually care about: Google’s contrail-detection competition was won by entrants who noticed a half-pixel error in the labels (is the origin at the corner of the pixel or the center?) and this turned out to be a part of the winning special sauce https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/writeups/jun-koda-1st-place-solution. Great for winning $15,000, not so helpful if you actually want to solve contrails. “People themselves will act like these LLMs and try to reward hack. It goes back to Goodhart’s law https://en.wikipedia.org/wiki/Goodhart%27s_law: any metric that becomes a target is no longer good as a metric.” His advice for where to start instead? “Always just fit linear regression. Just do it. Just do it. Just do it. Or SVM.” Tackling Climate Change with AI John and his team have worked extensively to mitigate the effects of climate change. We talked about several of their initiatives. Perhaps the most interesting result we talked about was reducing the effects of condensation trails (contrails) from airplanes. Those little streaks you see running behind planes somehow account for 1% of all human-induced global warming https://doi.org/10.1016/j.atmosenv.2020.117834?!? Some of these trails of ice crystals can hang out for days. These crystals are black in the infrared, acting like a thermal blanket that traps heat day and night. It’s easy to understand what’s happening here, a region of atmosphere becomes “ice supersaturated”, and a tiny bit of exhaust seeds water vapor that instantly crystallizes. The scale here is astounding, with a single gram of exhaust resulting in ten kilograms of ice crystals. The solution to all of this is quite simple, in principle! We know what parts of the atmosphere are most likely for the trails to form. Just have the planes drop a flight level or two. Problem solved, right? Well, the hard part is accounting for how much warming was prevented. This is a counterfactual problem, parts of which stumped John’s team for over two years. They had a working model for the heat-trapping half https://doi.org/10.5194/amt-19-1951-2026, but not for the reflected sunlight. ERA was able to find a simple model with some confounders they hadn’t considered. Cracked it! Modeling climate generally is a hard problem. Climate is best thought of an attractor of many different possible weather outcomes. This makes it much harder to model. “Weather is where you are on the attractor https://en.wikipedia.org/wiki/Lorenz_system, and climate is the statistics of the attractor. The problem with climate is that we’re altering it. The attractor itself is changing, it’s moving.” John and his team have worked on treating both the symptoms and the disease of climate change, with several other works in the area. Another fun example we briefly cover is FireSat, a way of using a constellation of satellites to rapidly identify fires before they grow too big to put out. For anyone living in California, you understand the problem. In dry years a small fire can result in hundreds of thousands of acres. If you could find this fire when it’s the size of a room, it could be put out. By the time it hits an acre we have a much harder problem. Where is this all going? Looking forward by looking back By now it should be clear John has an incredible and unique view over the intersection of science, computation, and AI. John talked about a class on physics of computation https://en.wikipedia.org/wiki/Feynman_Lectures_on_Computation he took with Richard Feynman back in 1982. This was when quantum computing was an ill-defined concept with no theory or experimental backing. John recalls every Tuesday was a guest lecture, and every Thursday was Feynman explaining why the Tuesday guest was wrong. John also recalls doing science back when there was essentially no compute, a million operations per second was cutting edge. What is John’s recommendation: the most important skill is developing deep domain expertise. There’s no other way to develop taste than to tackle hard problems. One surprising part of this is that John recommends spending time doing things the old fashioned way. Play with tools, and just implement things yourself. “You could drive up the mountain, or you could hike up the mountain, and maybe it’s okay, even fun, to occasionally hike.” Summing it up, John’s message to the audience is that there will still be a place for scientists, and that if anything it will just open up more opportunities for “the creative stuff, the rigorous stuff, the philosophy stuff.” But don’t forget to spend time doing the grunt work. “There just seems to be this strong impetus in the world to optimize and squeeze everything out. But you do lose something when you hyper-optimize. It’s overfit.” And whatever tools you end up using, John’s advice is the same one Feynman gave him https://calteches.library.caltech.edu/51/2/CargoCult.htm forty years ago: you must not fool yourself, and you are the easiest person to fool. We had a great time talking with John. We hope you enjoy! Also in this episode * Fusion is three years away, not thirty, if you ask John. And why the Lawson criterion https://en.wikipedia.org/wiki/Lawson_criterion means every fusion approach has an Achilles heel. * Why superconducting qubits are still finicky. * The asteroid he named after his mom, which turned out to have a moon. * The looming helium shortage https://en.wikipedia.org/wiki/Federal_Helium_Reserve nobody talks about. * How NeurIPS started as people crashing a private workshop at Snowbird, and why Hopfield networks are all you need https://arxiv.org/abs/2008.02217. * Being Carver Mead’s sysadmin on a VAX with an 80 MB disk the size of a dishwasher. * Finding asteroids in 1985 with film, a stereoscope, and a letter to Brian Marsden. The Vera Rubin Observatory https://en.wikipedia.org/wiki/Vera_C._Rubin_Observatory found 11,000 in six weeks. * The Feynman effect: total clarity in the room, none once you leave. * Quantum echoes, the NISQ era https://en.wikipedia.org/wiki/Noisy_intermediate-scale_quantum_era, and why he thinks quantum is neither thirty years away nor tomorrow. * A startup that wants to inject mercury into a fusion reactor and sell the transmuted gold. “It might not work.” * John’s 20% time rule for his own group: do stuff for learning, and you don’t even have to tell him what. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe https://www.latent.space/subscribe?utm_medium=podcast&utm_campaign=CTA_2

• 0:00Speaker 00

  Are you talking about introducing explicit priors that you know she based upon some human intuition, or maybe in this case, LLM intuition?

• 0:08Speaker 01

  When you talk about multiple hypothesis testing, right, there's predictive models and there's descriptive models. A predictive model is like, let's say you just have a, you have some inputs and you have some outputs and you just, I just want to build a piece of code that tries to just have the lowest error rate on some data set. The statistical model, a descriptive model is actually what science is trying to get to, which is, okay, it should be able to extrapolate because it has sort of the physics or the actual, some description of reality that's captured within it. And then you can use it to extrapolate. Yes, Newton thought of apples and gravity, but gravity isn't actually about apple, right? If you take a 17th century machine learning model like, oh, apples will fall, but how about planets, you know? I don't know, I have no data about planets.

• 0:52Speaker 01

  So who knows what they do, right? The distinction between those is a little bit blurry, right? Because when a physicist or scientist comes, they use their intuition, or maybe even more than intuition, like essentially there's maybe a solid pile of facts that they know about the world. And then they make sure that whatever model they build is sort of consistent with what's known.

• 1:14Speaker 00

  Welcome to Late in Space Science. I'm Brandon, joined by my co-host RJ. It's a pleasure to have John Platt, you know, with us today. John is a Google Fellow and Head of Applied Science, Google Research. He has really a fun like background. He, I guess, you described yourself when you were talking a few minutes ago as a mega nerd.

• 1:33Speaker 01

  No, gigging.

• 1:34Speaker 00

  Gigging, excited in absolutely everything. And it really, it really shows, yeah, you was, correct me if I'm wrong about any of this stuff, but, so you started college at 14 and started your PhD at 18 at Caltech. You were advised or co-advised by John Hotfield, right?

• 1:50Speaker 01

  Oh, yeah.

• 1:51Speaker 00

  Yeah, yeah, who just won a Nobel Prize in, you know, two or three years ago.

• 1:54Speaker 01

  Yes.

• 1:55Speaker 00

  So John created several, was responsible for several textbook algorithms, one known as Platt scaling, another one sequential minimal optimization, which is the textbook algorithm for training SVMs. Even today, it's still, if you use SK learn, it's there. John has discovered and named two asteroids, has an Oscar for technical developments from 2006. So if you've ever watched a Pixar movie, you've seen John's algorithms in work. John has an air dose Bacon number of six or three, three and three from either side. And I'm going to skip over like 20 years of your career, but then jumping into Google, it's, you know, working at Google Sciences, you've worked on fusion, quantum computing,

• 2:38Speaker 00

  climate modeling, and many other topics. Is that more or less right?

• 2:43Speaker 01

  That's right, yeah.

• 2:44Speaker 00

  Okay, cool. Did I miss anything important for today?

• 2:47Speaker 01

  Oh, no. I mean, I've also done, you know, lots of applied math and signal processing and all sorts of fun things like that.

• 2:52Speaker 00

  Yeah, yeah. I think you also, your Wikipedia has a fun story about patents and the iPhone too.

• 3:00Speaker 01

  The iPod.

• 3:01Speaker 00

  iPod, yeah, yeah. Yeah, welcome.

• 3:03Speaker 01

  Thank you. Yeah, thank you for having me.

• 3:05Speaker 02

  Can you tell us about the ERA is the, I think the way that the acronym is pronounced? And, and I know that there's a lot of different semi-related stuff out there, both within and outside of Google. So what, can you tell us a little bit about the details of ERA and, and what makes it special?

• 3:25Speaker 01

  Well, we've been doing sort of AI for science in Google Research for more than 10 years now. And around 10 years ago, it was very much using, I don't know what you call it now, maybe classical, classical machine learning models, you know, things like, you know, convolutional nets or whatever. And they were specific models to build to, you know, solve specific science problems. But about two years ago, we got very excited about these more general LMs that have popped up in the last few years. And we were wondering what can be done with them. And of course, a lot of people have been playing and trying to figure out what the right, what the right thing to do is. And we kind of stumbled into this mapping.

• 4:10Speaker 01

  And there's found that many different scientific problems can be mapped into something we call scoreable tasks. So you can often phrase a scientific problem as, oh, gosh, I really would like to have a piece of code that maximizes some score. And it's surprising, the number of different sort of scientific problems you can make a lot of progress on by mapping into that framework. Well, one thing is, a lot of scientists spend a lot of time sort of building models. They might be statistical models, or they might be, you know, physically based models. And if it's a statistical model, like in machine learning, your scoring function is, well, I have some data set and I'd like to have the fit, the model on the data set go up. And we can talk about overfitting in a minute, but.

• 4:55Speaker 00

  That was one of our questions.

• 4:57Speaker 01

  And then that's actually very, that's actually very interesting. It was so, machine learning is kind of a subset of this sort of scoreable task, right? But you could do other kinds of things, like especially Michael Brenner, who's the lead author on the ERA paper. He's very, very skilled because he, he likes to sort of knock out a scientific paper in an evening now with a tool. So there's something in applied math called asymptotic expansions. And, which is you're asking, how does this, how does an ordinary differential or partial differential equation, say ordinary differential equation behave? There's some parameter has an epsilon in it, and you're trying to say, how does it behave as epsilon goes to zero? And you could turn, it turns out, you can turn that into a empirical task by essentially

• 5:42Speaker 01

  asking that it, it proposes some solutions that are asymptotically correct. And you check to see if the asymptotic solution is correct for like epsilon equals 1e minus four or something. And then you, you check that fit, but then you ask, you ask Gemini, which is the core AI underneath it, to, to do the mathematical reasoning, try to solve the problem whilst, while also maximizing the fit to the, to the data. So you can, actually, there's a lot of sort of tricks you can do because it's not that the underlying thing that's, that's altering the code or the underlying thing that's sort of making the decisions is not a random process. It's an AI itself that is smart and knows about things and knows a lot about the world.

• 6:25Speaker 01

  You can get a lot of, solve a lot of interesting problems because that sort of core inner loop is an AI that has huge amounts of prior knowledge. So that's sort of the, the trick. So we've been running around trying to map lots of scientific problems into scoreable tasks and trying to solve them. And it's, it's really been kind of fun. And I'm happy to talk about the ones that I've been involved in at least.

• 6:47Speaker 02

  Yeah, I would love to hear about some of the more, so the statistical one is what everyone listening will probably know about. What you just mentioned makes sense. What are some of the other interesting ones?

• 6:57Speaker 01

  Let's see, that are not statistical, let's see, because we have interesting ones like one that we just put a paper up on archive is, or actually I think it might be on GitHub. It's, you often run into this in remote sensing because there's always a trade off. There's satellites flying above the earth. And there is a trade off between how frequently they can revisit a spot on the earth, what their spatial resolution is, how big the pixels are, and their spectral resolution is or how many bands they have. And ideally, you'd like to have monitoring of the earth that's constant and, you know, a frame every five minutes at, at hyperspectral resolution, at whatever, 10 centimeters. You can't get that.

• 7:39Speaker 01

  But for, for example, to monitor CO2, the atmospheric concentration of CO2, you can take data from one satellite. That's, for example, it's OCO2 or OCO3. OCO3 is actually attached to the International Space Station. But, so it gets you like a little strip of CO2 measurements that are highly accurate and pretty high resolution. You can actually try to do there, you could, because a lot of it is in the infrared, weather satellites like GOES has some infrared bands and it, and it takes a picture every essentially five minutes, but the pixels are very large and it doesn't have such great spectral resolution in terms of it wasn't designed to find CO2.

• 8:23Speaker 01

  So you just ask one to estimate the other and you shovel other data in, like what's the current weather, what, what's sort of the long-term, you know, albedo. And so you, so ERA came up with this very nice, this very nice model that can sort of, can do almost like super resolution and inform super resolution of, of one satellite to another. So that's like one example.

• 8:47Speaker 02

  Yeah, yeah, okay. So any, any scientific problem that you can map into this framework. So the, the input is to, to ERA is sort of this mapping and the output is code, is that?

• 9:01Speaker 01

  Well, sort of. I mean, the input is you, the way we've got it set up in, in the product is you just start talking, right? And so, because a lot of times it's not obvious to how do this mapping, although, you know, experts like Michael Brenner know how to do it. So we actually wrote an agent that actually helps you, it sort of talks to you to try to help you define what your scoreable task should be. So there's already, there's sort of a instance of Gemini sitting there trying to help you write a code. So that's actually sort of an, almost like an intermediate result. You start talking to it about your problem and it tries to produce essentially a Python notebook underneath that, that has a, that has a score, essentially a function with a, with a scoreable, which essentially produces a score.

• 9:46Speaker 01

  And then it starts to mutate that notebook in a clever way, because again, it's Gemini. And it will try to sort of keep proposing code that tries to maximize the, the score.

• 9:58Speaker 02

  So what is different about this and just a generally, a general agentic system that can sort of optimize notebooks?

• 10:06Speaker 01

  Right now, it's a, it's essentially its own, in modern 2026 parlance, we actually worked on this in 24 and 25, but the modern parlance is kind of a specialized harness that runs an algorithm, which for ERA was Monte Carlo tree search. So essentially, it's keeping hundreds or thousands of possible instances of notebooks, and then it selects one, and I can explain how it selects one, and it decides, well, how, what can I do, Gemini asked itself, what can I do to make that notebook be better? And then it will make a new one and test it and then put it back into the candidate pool. So you can imagine the candidate pool is actually tree structured because it's, you know, every candidate possibly

• 10:51Speaker 01

  has some children. And what you do is you pick based on something called the, it's actually a fairly standard algorithm in, from reinforcement learning called upper confidence bound, UCB. So you essentially pick, it's an optimistic algorithm. So it tries to estimate, say, what's the, what's the 95th percentile outcome of mutation? It tries to estimate that. And it picks the one with the highest bound, the highest optimistic bound. So in other words, it doesn't always pick the best performing notebook. It tries to predict like, what's the current performance plus two sigma of its guest. And so it's always trying, so it hunts around.

• 11:27Speaker 00

  So high recall, basically.

• 11:29Speaker 01

  High recall. It's trying to make its bet so that it, it most efficiently tries to make progress, which isn't always greedily doing the best candidate. Sometimes it's the fifth best. We're also play, we've also played around where, where it kind of recombines, it sort of takes ideas from two candidates and, and smashes them together and tries to make a third candidate out of that.

• 11:54Speaker 00

  How does it seed the initial candidate pool?

• 11:57Speaker 01

  Well, that's the amazing thing is that underneath Gemini is actually good at writing code. I mean, you just ask it, write me a thing, because you have, you have a textual description of the problem. It isn't just, oh, here's your scoring function start, use a textual description of the function. And you might give it, in fact, we have under some things, like, here are five papers that people tried to solve this problem with. And it's kind of smart. It actually goes and reads the papers and will actually take a first stab at code. It might not be great, or it might, you know, sometimes it has bugs and it returns essentially minus infinity, but it will then try to mutate the code and say, oh, it'll try to make it be better. So it's pretty cool. You don't actually have

• 12:40Speaker 01

  to give it, I mean, you can, if you want, give it a, some starter code, but you don't have to.

• 12:46Speaker 00

  How many agents are you spinning out? I guess maybe not agents, or how many different, you know, tree branches are you spinning up at each iteration?

• 12:53Speaker 01

  Oh, at every iteration? Well, there's a trade off. You'd like to do a lot of parallel work, but if you do too much parallel work, you can't learn from previous things. So right now, we use about, the default is 10 parallel, so you, so you try to grow 10 leaves at a time. That seems to be about the right trade off.

• 13:14Speaker 00

  When you say you can't learn from previous iterations, that means that the orchestrator, or is there some sort of, yeah, what like, what's the, that you said that there is some step which is able to like recombine or make decisions beyond just like the score? I mean, so yeah, I guess maybe one of the ultimate questions is, as a human, when you are doing some sort of a ML project, you don't just look at like, oh, there's this one metric that we're trying to optimize. Oftentimes, there's like orthogonal metrics, sometimes even insights such as like just watching, you know, training curves can sometimes give you intuition about what's going on, or like looking into specific examples. Does it do any sort of introspection like this? Is there?

• 13:54Speaker 01

  Well, it has, it has the history of, but by I meant why you can't do too many things in parallel is, is if you, if you have 10 parallel searches at once, then what the number one can't actually see what numbers two through 10 are doing. So if you do 1,000 at once, then you're using a huge amount of computation without a lot of cross learning. Whereas, once you finish a little batch, you get the history. It's sort of, obviously you have to prune it so it doesn't blow up the context, but, but you get the history of what it was thinking about as it was kind of writing the code and the results of the code. So, so it can learn from its previous attempts.

• 14:35Speaker 00

  Okay. And does it learn across?

• 14:37Speaker 01

  Oh, yes. Yes. Essentially, it's essentially, it's like one essentially shared context. Yes. So it is, it is sort of thinking as it goes along. It's not, it's not like it's a thousand different, completely independent branches.

• 14:50Speaker 00

  You're really pushing Gemini's like long context abilities.

• 14:52Speaker 01

  And you have to do, you have to do the right management.

• 14:55Speaker 00

  Yeah, yeah, yeah. Okay. Oh, that's, that's cool. I mean, going back to RJ's question, the like key point here being the first, the first key goal is to, I guess, identify what specific score that you are trying to optimize, right? Sometimes that is, I agree, like kind of the hardest part of the problem. And so I find it interesting that, I'm not sure I'd always trust my agent to do that part. That part seems like the more human task in the loop.

• 15:27Speaker 01

  It is, and often you have to be careful. In fact, a lot of what you do, it's kind of, it's very meta. I guess everything, everything we do is very sort of high level. You have to make sure that the, there's no, one common thing is you come up with a scoring function or the agent does, or you do it together, and then the iteration finds a way to cheat or hole, like, oh, no, I didn't mean that. And so you have to go through and often sort of play and have a loop around it where you kind of iterate, like, no, no, no, I didn't mean that. Or you have to tell it in its instructions, okay, well, you know, don't, don't do this. So yeah, there's often iterations and you, so you, even with a Gentech help, you don't necessarily get the, the right scoring function from day one. And in fact,

• 16:09Speaker 01

  it's really neat because, I mean, in the old days, i.e. 2024 or something, you know, a lot of, a lot of grad students would spend a lot of time like doing scientific software. And it's just so much effort to write code at all, that you kind of try maybe a few things or a few things that are very related, and then you sort of stop because you have to write your paper, you have to do your next experiment. This thing is kind of underneath kind of relentless because it keeps trying and keeps trying, keeps trying. And so the people who use it are now spending all their time almost at the right level, almost at the scientific creativity level. Well, what does it mean to have a cost function? You know what I mean? And so that's almost like the essence of the scientific problem. You're

• 16:51Speaker 01

  not so much now in the details of, oh, oh, I have to import this CSV file, or I have to get this database to work or whatever. It's, you're now sort of thinking almost like deeply philosophically about your actual scientific problem, not down in the grungy goop of, of worrying about, you know, databases. In fact, one, one cool thing the agent can do is actually suggest data, data sets to you, like, oh, have you thought about maybe pulling in this data set and doing a join? And so it'll, it'll make suggestions about like, you know, data sets you can join with, which is kind of cool.

• 17:24Speaker 00

  Going back to what you said a second ago, in terms of agents love to hack things and reward hack, are there, do you have any fun stories or interesting stories about, you know, where things were comedically run off the rails?

• 17:37Speaker 01

  Boy, I'm blanking. I know other, other folks have run into it. I don't know if it's, I don't know if I have enough details to sort of say what, to sort of express the comedy of it. But it, it does, it, you, you kind of get surprised. I don't know if I have any really concrete, sorry, I'm blanking.

• 17:56Speaker 00

  No, it's fine. Yeah. I always like to think of machine learning. It's like, it's sort of like the old genie stories, monkey falling. That's what you wish for because you, you're going to get it.

• 18:06Speaker 01

  That's right. And, and you have that, and that happens very much with this. You have to be careful. But on the other hand, it has some knowledge. The nice thing is that sort of Gemini knows a lot about many things, or more than any one person can do. So it at least knows, especially if you point papers, point, you know, like here, there are five papers that tried to do this in some way. So to some extent, it does have that genie feel, but to some extent, it also sort of does sane things. This is why, remember the whole idea of sort of evolutionary coding? It's been around since the seventies. Everyone's

• 18:46Speaker 01

  loved to do that. Like, oh, let's mutate list code or whatever to do things. But the reason why it just hasn't taken off is that random mutation in code space is pretty much worthless. I mean, just like, well, just like DNA, it's sort of like, you know, most, most things are harmful. So here it's like, oh, no, no, we can actually find, it sort of knows, sort of underneath it knows interesting gradients to try, which is why the thing works, that, that the underlying loop itself is an AI. So yes, it can, it can maybe overfit and have funny sort of genie problems like you allude to. But it also has some amount of sanity because it's sort of, it knows about the world and it, it has world knowledge in it. So.

• 19:27Speaker 00

  The paper, though, you were doing Gemini 2.5, and I think, you know, Gemini has advanced quite a bit. Do you have metrics or have you, you know, this is a tool that you're continuously using and it sounds like you're improving.

• 19:38Speaker 01

  Yeah.

• 19:38Speaker 00

  And I'm wondering, like, do you internally, have you seen like, almost like a phase transition in how effective this tooling has been? Has it, how dramatic has the improvement been over the last, like, I guess, year or two?

• 19:51Speaker 01

  Oh, well, I mean, a year or two, amazing. You know, there's every, even every half version of, of, I mean, essentially, I think it would have been impossible under Gemini 2.0. And yeah, I think so. In other words, it wouldn't have worked.

• 20:05Speaker 00

  So you started at 2.5 and that was like just the?

• 20:07Speaker 01

  Oh, well, no, we've been trying to experiment with these things actually for a while.

• 20:11Speaker 00

  Okay.

• 20:11Speaker 01

  And things just weren't working. And then they started to work. And then now they're, they're just amazing. So the progress on Gemini major versions has just been stunningly amazing.

• 20:21Speaker 00

  Yeah, I think this is an experience a lot of people have been having where things were just seemed impossible, whatever, are suddenly becoming magically useful really quickly.

• 20:32Speaker 01

  And so if people are, I even say this to scientists because there's some people like, oh, I tried whatever 2.0 and I didn't like, oh yeah, that was a long time, that was a year ago. That was like a long, it was a journey ago. Right? Yes. And in fact, all the, all the, we even have one of the pre-prints where we've sort of combined ERA with anti-gravity. And that, you know, the whole, that whole harness of anti-gravity is pretty amazing, too. And it's that, that's the one where you can sort of pull in lots of papers and it can write lots of code for you. And so, yeah.

• 21:04Speaker 00

  Is that publicly available? Or is that?

• 21:06Speaker 01

  The anti-gravity?

• 21:07Speaker 00

  Yeah, yeah.

• 21:08Speaker 01

  Well, anti-gravity is certainly publicly.

• 21:09Speaker 00

  Oh, sorry, sorry. The ERA plus anti-gravity?

• 21:12Speaker 01

  Not yet.

• 21:13Speaker 00

  Okay, okay, not yet. Okay.

• 21:14Speaker 02

  I find this area really fascinating because like you said, it's been, there's been some form of code mutation out there since the dawn of computer science, basically. The canonical problem is sort of the overfitting or multiple hypothesis testing problem, I think, which is maybe a little bit better matched to the problem where you're basically, my hypothesis now that this algorithm worked, now my hypothesis, and so that you run a risk that sort of it has exponentially exploded, right? Because now suddenly I have like these, it's like hyper, hyper parameters that I'm optimizing. And so that you have this explosion of state space that you're exploring. And so that it seems much easier to, to sort of overfit

• 21:58Speaker 02

  to a problem. What are your thoughts about that? Because on the other hand, empirically, my experience, I even tried the, you know, sort of open source version of ERA. I kind of strapped it into Claude and it's running right now. So I can't tell you how well it's working.

• 22:12Speaker 01

  Okay, I'm curious.

• 22:13Speaker 02

  Yeah, I'll let you know. But I'm just curious to know, this is a question that's been in my mind about just general AI for science. And what, so what are your experiences with this, sort of on the, sort of on the ground?

• 22:27Speaker 01

  I guess there's two questions sort of embedded in your question, I think, right? Because when you talk about multiple hypothesis testing, right, there's predictive models and there's descriptive models, right? I, you know, can tell I've been doing machine learning for a long time, statistics.

• 22:41Speaker 00

  That's actually a really good point, though. Do you mind explaining that, expanding on that? I'm not sure that's something that everyone in our audience would be familiar with, like.

• 22:48Speaker 01

  Right. Especially in, in modern days, I think people are trying to sort of obscure the two.

• 22:53Speaker 00

  If you started with LLMs, I'm not sure that distinction would be meaningful.

• 22:57Speaker 01

  That's right. So a predictive model is like, let's say you just have a, you have some inputs and you have some outputs. And you just, I just want to build a piece of code that tries to just have the lowest error rate on some data set. That's just a statistical model, right? A descriptive model is actually what science is trying to get to, which is, okay, it should be able to extrapolate because it has sort of the physics or the actual, some description of reality that's captured within it. And then you can use it to extrapolate. Because it's sort of like, you know, yes, Newton thought of apples and gravity, but gravity isn't actually about apple, right? If you just fit, if you take it as a machine, you know, the 17th century machine learning model, like, oh, apples will fall, but how

• 23:40Speaker 01

  about planets? You know, I don't know, I have no data about planets. So who knows what they do, right?

• 23:45Speaker 00

  So, so when you say extrapolative, okay, I didn't realize we're kind of going on a tangent here, but I am curious. Okay, so when you say extrapolative, so there's different ways I could think about this. One of them is, you said, a model of physics or model of the world. Are you talking about introducing explicit priors that, you know, she based upon some human intuition, or maybe in this case, LLM intuition? Or are you talking about this is the, the physics is actually learned by the model, or the

• 24:10Speaker 01

  Oh, it's more.

• 24:10Speaker 00

  The underlying process of the world is under, what about the model?

• 24:14Speaker 01

  The distinction between those is a little bit blurry, right? Because when a physicist or scientist comes, they use their intuition and they are about, or maybe even more than intuition, like essentially there's maybe a solid pile of facts that they know about the world. And then they make sure that whatever model they build is sort of consistent with what's known. Currently in ERA, it is, it's sort of, it is LLM intuition. Essentially, that's what I was trying to say about having a good gradient underneath, that, that especially if you point it at existing papers, it will try to build models that are kind of sane underneath, because if it's, again, if you, especially if you give a guidance, like, oh, be sure to incorporate this and this, or look at these papers to get these things.

• 24:58Speaker 01

  So it will, there is a, you can introduce a bias towards certain model choices, and it will have a bias because it, it just, its own little sort of world knowledge is accumulated inside of, inside of itself in, in, in sort of pre-training.

• 25:15Speaker 00

  Can you give us examples of what that might look like? Is it modeling something in a way where it's actually, there's different, for example, if you're doing something with partial differential equations, there's these, you know, formalisms people have, like neural operators, for example, or, or you can embed a, you can encode a differential equation in some sense, or I think that's called physics-informed neural networks. Something is like one for fluid and kind of PDE type modeling systems, or for, let's see, molecular systems, there's oftentimes this idea about equivariance. I mean, are the models like picking up on these, like, you know, tricks which have been developed in literature? Or are they adding some weights for some physical prior or something? Yeah, kind of what does that look like? When it introduces like a physics, when it introduces

• 25:59Speaker 00

  some sort of, you know, knowledge like that?

• 26:03Speaker 01

  I don't know if I have enough data to sort of say, oh, you know, 73% of the time it does this. But especially when you point it at existing papers, it will try to, in fact, it will do very well at adapting the, the methods that are described in the papers for the problem. In fact, it will do amazing job. You can actually often just recreate or reverse engineer papers. That, that's again, what Michael Brenner actually likes to do this. He'll, he'll say, oh, that sounds like an interesting paper. We did this actually for the, we had this thing, it was actually kind of a hack. I suggested this to Michael, where there's this one MIT professor who, he came up with a, some

• 26:46Speaker 01

  code to do, essentially, if you have a rooftop, if you have a rooftop with a fixed area, and you want to sort of maximize the amount of solar power you capture over a day, sort of solar energy, you can build up, which of course captures more sunlight, you can sort of build, you can have it design a widget, you know, involving mirrors or struts or solar panels, just sort of stock it, whatever are, you know, angles or, or sizes you like, and usually with, with a, with a maximum height, and then try to figure, let it sort of explore that design space. And I believe, Michael, we can ask him, I believe it actually just, I don't think he actually installed the simulator. I think, I think the code

• 27:28Speaker 01

  just, it just reproduced the code, because it has like a coding agent inside of it. So it just reproduced the code from the paper and sort of figured it all out. So yes, it's, it's very good, especially if given a pointer to what other people have done. It's very good at kind of like, oh, I haven't seen it try equivariant modeling. That, that can get very hairy if you know about Klebsch-Gordon coefficients. It's, it's pretty.

• 27:54Speaker 00

  They're fun, yeah.

• 27:54Speaker 01

  Yes. So I don't know if it'll do the true equivariant stuff, but it actually knows a lot. I remember actually when Gemini 2.5 came out, I know this is not exactly about ERA, but I, I remember sort of thinking, oh, this is a new world when like the day 2.5 came out, because it said, hey, Gemini 2.5, can you write me some boost decision tree code? And it did.

• 28:15Speaker 00

  And it worked.

• 28:16Speaker 01

  It just did, yes. And I said, you know, yeah, this is, this is a new world. So yes, I think it's, to loop back to your question, I think, yes, if you give it sort of guidance about, oh, you know, it's important to put this kind of thing in, it will. And so it, it won't necessarily, or at least not that we've seen, discover completely new, like if you didn't know about Klebsch-Gordon coefficients, I don't, I mean, you don't, you didn't know about something, it won't completely discover a new kind of physical model from scratch. But it will certainly, if you tell it about interesting constraints about the world that are known, it will certainly follow it. I don't know if I answered your question.

• 28:54Speaker 00

  So there's still room for humans for the next year or two?

• 28:56Speaker 01

  Oh, in fact, there's, going back, I think there's totally room for humans because I don't know, I mean, we have co-scientist that tries to help you come up with sort of hypothesis generation. But really, I still haven't seen sort of the creativity and the philosophy and the sort of, the sort of the careful rigor. You totally need humans. I don't see humans going away. They can make strange suggestions. And I've used co-scientist actually for an interesting problem in geochemistry, and I learned about a new kind of ion I didn't realize happened in magma. But I, but, and so it'll tell you interesting things and you'll learn stuff. But I don't think it sort of substitutes for human creativity.

• 29:41Speaker 00

  You know, going back, we were just talking about two years, two point, you know, two, you had Gemini 2.0 to 2.5, and this was like, you already saw a leap, and now it's been another year or two, and now we're at 3.5. And, you know, there's, you're saying this is working much better. I mean, whenever you look at a graph, you know, you can, if something looks like an exponential, it can either, you can either be in a sigmoid or you can be at the beginning of a takeoff, right? I guess every exponential turns into a sigmoid of interest.

• 30:05Speaker 01

  Every exponential turns to a sigmoid.

• 30:07Speaker 00

  But the question is like, where are we on that?

• 30:09Speaker 01

  I mean, I guess I'm a big believer in sort of the whole jagged.

• 30:13Speaker 00

  Aggressive frontier.

• 30:14Speaker 01

  Yeah, the jagged frontier. And so certainly, at least what I see, I mean, I don't know what's going to happen in a couple of years. But yes, there, there's some big spikes out in jaggedness in terms of coding ability and just gathering knowledge and finding related things. And, and that's huge and wonderful, which I think is great for scientists. So far, it's kind of less in terms of rigor and, we can talk about things like the International Meth Olympiad, but, and meth in general, but, but, but in terms of sort of philosophy and creativity, I think it's still kind of not. And I, maybe it'll, maybe everything will, people, some people are saying everything's going to inflate and pass, but I'm still seeing a lot of very strong jaggedness. So I, I can see, well, maybe it'll get to be

• 30:57Speaker 01

  extra good at coding and extra good at, at fitting models and extra good at making suggestions and things. But I don't know. So far, not. So far, you need the humans.

• 31:06Speaker 02

  Yeah, I want to get back to the question about the multiple hypothesis testing.

• 31:10Speaker 01

  Oh, sorry, sorry.

• 31:11Speaker 00

  We've derailed.

• 31:13Speaker 02

  Totally, totally love the tangent.

• 31:15Speaker 01

  Multiple hypothesis testing is when you make, have a descriptive model and you're saying, this is the way the world works. And you have a bunch of data and you take a billion darts and you throw. And so you have to be careful, there's something called false discovery rate, right? And so the question is, Is this finding descriptive models? Or is this finding sort of predictive models? And fundamentally, the scientist is there to make sure that whatever is saying is descriptive. We haven't been able to make a system so far out of these pieces that can really sort of discover completely new physics or completely new science. But this is sort of a power tool to help you discover completely new science.

• 31:58Speaker 01

  So maybe I'm trying to unask your question about multiple.

• 32:02Speaker 02

  But I think this gets at the heart of it, yes.

• 32:04Speaker 01

  Yes, but then you're saying, what about just pure over? Okay, let's set aside, it's not, it's not trying to figure out a descriptive model of the world that's still up to the scientist. But what about just plain old overfitting?

• 32:13Speaker 02

  Yeah.

• 32:14Speaker 01

  Yes, you have to be very careful because it's a power tool. It can, I shouldn't probably say it can slice your fingers off. No, but you know what I mean? It's about, you have to be very careful and you have to be very rigorous. In fact, now you have to be more careful and more rigorous to not fool yourself. You really, really need to be just excruciatingly careful about having, you know, very hidden holdout sets you don't look at. You have to be just super, super rigorous to make sure that you don't completely, because it is a total power tool.

• 32:44Speaker 02

  So the question to how do I not slice my fingers off is you need to use the same techniques, but be very careful with them.

• 32:52Speaker 01

  Yes.

• 32:52Speaker 02

  That's a very clear answer that I'm going to get. Thank you.

• 32:56Speaker 00

  Yeah, I actually don't think I've heard any guests say that. Yeah, it's like, it is, I think, a very important skill, maybe one of the most important skills in the new, people talk a lot about taste.

• 33:06Speaker 02

  Yeah.

• 33:06Speaker 00

  But maybe this is a variation of taste, but like.

• 33:09Speaker 02

  The taste is like the other side, right?

• 33:11Speaker 01

  It's like the rigor. It's like the, yes, yes. In fact, if anything.

• 33:15Speaker 00

  Yeah, taste or rigor, which one's more important?

• 33:17Speaker 01

  Well, I don't know. I think people, at least the way I am viewing it is, is, I mean, the aspect of that research, researchers are software developers, which there's a lot of overlap in a lot of fields. I'm seeing that software engineers are, it's almost like, obviously, there's a lot of concern like, oh no, what am I going to do? You know, coding seems to be, you know, getting automatic. So I think, I think there's sort of both. I think there's a lot of people get pulled into, well, I'll be the creative source. So I'll try to figure out new science. I'll try to figure out new products. I'll try to sort of really be very creative. And again, I'm strongly believe that I don't think that's going to go away. There's also people sort of pull towards rigor, like, oh, I want to make sure this

• 34:03Speaker 01

  doesn't, that this doesn't crash. I want to make sure this scales. I want to make sure this isn't wrong. I think you need both, and I think you need people who are really good at both. But they don't necessarily have to be the same people. But yes, I think you need, I think this is even broader than science, just as sort of software engineering evolves. Yeah, it'll be, you know, the people who will bring the creativity and the people who will bring the rigor. And I think those will be sort of anchors.

• 34:29Speaker 02

  Some other things that I've seen are related work out there. There's a really cool leaderboard for, you know, a claw leaderboard, agent leaderboard for scientific problems from Stanford. I don't know if you're familiar with it. It seems like a really interesting idea to me to have, you know, sort of different agents kind of competing on the leader. So it seems like if you squint a little bit, what ERA is doing is, is kind of a leaderboard, but it's internal and it's recombining ideas. Whereas what are your thoughts about this? And do you, is that like a thing that you guys are working on? And is there problems with that or advantages to that?

• 35:11Speaker 01

  Ironically, you know, the whole ERA project actually started because people may not realize Kaggle is actually part of Google.

• 35:19Speaker 00

  Oh, yeah.

• 35:20Speaker 01

  And so it was called the auto Kaggle problem. So it was actually like, that's what it was, is let's try to, let's try to have a system that can sort of, you know, win at Kaggle competitions. So that's sort of why it sort of has this shape, that's sort of how the project started. And it goes back to sort of overfitting, right? If you've ever actually competed in a Kaggle competition.

• 35:42Speaker 00

  I have done Kaggle competitions and, or I've done one. It is a really interesting phenomenon because there's this, overfitting is like rampant.

• 35:52Speaker 01

  Rampant.

• 35:52Speaker 00

  Yeah, yeah. And it's really impressive how people can overfit to certain data sets in a way that is, yeah.

• 35:58Speaker 01

  Or we even, we had, we have a fun project I can talk about over if you like, that tries to mitigate contrails, jet contrails. I'm going to talk about that if you want. And we had a contrail Kaggle competition and people actually beat us, but they found that we had a half pixel error in our labels. And they, because it had to do with the center versus the lower left, like where is 00? Is it in the lower left of the, of the pixel or is it in the center? You know what I mean?

• 36:26Speaker 00

  Yeah.

• 36:26Speaker 01

  So they found that and exploited that and squeezed or whatever, a little bit extra stuff. Because it turns out when you make artificial data and you rotate it, you have to make sure that you take into account that half pixel offset. So yes, people or people themselves will be, act like these LLMs and try to sort of reward hack on these things. So it sort of goes back to, what is it? Goodhart's Law? You know Goodhart's Law?

• 36:52Speaker 00

  Yeah, yeah. Maybe, maybe quote it though.

• 36:55Speaker 01

  Let's see, let's see, any metric that becomes a target is no longer good as a metric.

• 37:00Speaker 00

  Yeah.

• 37:00Speaker 01

  And so that's the, I mean, it's good and it's just that you have to be, you have to be very, very careful and you have to, again, you have to have like layers of rigor and like, okay, but we'll do this and we'll optimize for this. But you have to realize, okay, that's just now Goodhart's Law applies and you have to be careful. And so yeah, that's a lot of reasons why the whole AI field has been kind of constantly exhausting these, these things. Because again, Goodhart's Law applies individually to every, to every leaderboard you make. So again, it's sort of, you just have to step back and be very, very careful. I, that's not, I don't think.

• 37:35Speaker 02

  No, no, no, that, no, that's really useful to my thinking is, you know, as we've had guests on, it's been a recurrent theme of how do you manage all this, the complexity that's introduced by LLMs in agentic science?

• 37:49Speaker 00

  I think my follow up question was about overfitting in Kaggle. Yeah, it is, if you, you had an auto Kaggle problem and then the question is, given auto Kaggle, how often was it successful? I mean, assuming you probably just ran this on like all of your Kaggle competitions or something.

• 38:05Speaker 01

  Well, we tried it on various, like what they call playground competitions and it did very, very well in playground competitions. We've entered into different competitions, some of them. It turns out there's, in the last few years, just the number of leaderboards and competitions and whatnot have just exploded far beyond Kaggle. So we've done very well in some of them, like one thing we're super proud of is the whole, CDC set up this competition where you try to predict next week's, the number of COVID and flu cases that will happen in every state and territory in the US. And you try to predict a week in advance. And ERA did super well on that.

• 38:46Speaker 00

  It's funny because in some, Google invented the concept of using data to track disease progression with Google Flu. So it's kind of funny that you were sort of going full circle 20 years later or something like that.

• 38:59Speaker 01

  So that did very well. Other ones where we've entered, we weren't quite as good often because people are very, again, you know, you have to sometimes, sometimes how well you do in these competitions is a measure of how much sort of TLC you put into it and how much you're willing to squeeze the last point 001. And so, so it was, I mean, ERA did well, it got you pretty close, but you, but we didn't, we didn't close the jump in the last whatever 30 places or whatever, because no one was there to shave the last, you know, point 001 off the thing.

• 39:32Speaker 00

  Yeah.

• 39:33Speaker 02

  Is it very iterative? Like you get to it, you know, it sort of, I saw the charts in the paper and, you know, you sort of get these step changes as it discovers something and then, and then flat and then, so is it very much human in a loop? Like, okay, you've stalled on the problem, like try this kind of thing?

• 39:51Speaker 01

  Yes, yes. At the outer loop, which is, I think that's almost like the more fun, creative part. So yes, oh, here, have, look at this paper. Oh, you're doing something bad or, you know what I mean? So it's almost like having a hyper eager grad student or something who doesn't sleep. And you sort of tell it things and you, and you sort of guide it around.

• 40:13Speaker 02

  How often does someone intervene versus, like, what does the outer loop look like actually?

• 40:18Speaker 01

  It might run for a few hours and, and come back and give you some examples. And then you would, you know, you can, as far as you like, you can, you can sort of keep trying and keep poking at it.

• 40:30Speaker 02

  So that's, it's very much designed to be human in the loop then?

• 40:34Speaker 01

  Yes, yes.

• 40:35Speaker 02

  Interesting, because a lot of the other tools that I've tried tend to be very one shot.

• 40:42Speaker 01

  Well, I guess it depends on your definition, right? I mean, it's obviously you talk to it and you start it and it'll go for some number of hours and then come back. And then, but of course, then you say, but then that's where the human creativity kicks in. And then you're sort of doing the outer loop where you sort of every, you know, depends if you want to sleep, but you know, every few hours you, you go and you give it another try and you.

• 41:03Speaker 02

  What kind of budget are you giving this thing? Like you blew through a million dollars accidentally kind of thing?

• 41:09Speaker 01

  I don't actually know because we're using sort of, you know, internal calls to Gemini. So actually I don't actually know.

• 41:17Speaker 02

  So, but look, is there's token budget, but then there's also like, I'm solving a problem that is computationally expensive.

• 41:23Speaker 01

  Oh, yes. That also, essentially underneath it, because, because the, the scoring function itself might have, you know, Monte Carlo estimation or, or whatever. Yes, so you actually end up, you can actually end up using a lot of compute to just even do, or simulation, like if you have a simulator inside, has to run a simulation. So yeah, you can, you can, you can spend a fair amount of just CPU or GPU.

• 41:47Speaker 02

  So my little experiment with ERA, ERA and Claude is, is to build a neural network for some classification problems. And so they obviously like, if you have enough data, then, you know, larger networks work better, but they're more expensive to train. And you get to, you start to run into a question of how do I manage my budget if I have a fixed budget so that I'm spending my, my.

• 42:12Speaker 02

  My my dollars on the most effective solutions.

• 42:16Speaker 01

  That's right. And I think that's still something we need to figure out. But it's of course, it's no different than if you have a grad student and they're trying to train a very, very large neural network or a very, very large data set. They themselves have to, there's something like, oh, is there a scale law? Can I extrapolate? So it's not, I guess it's in the same problem, but maybe more urgent because it just runs into this problem because it's so relentless, it runs through the problem much quicker than a grad student could.

• 42:43Speaker 02

  One of the things that you optimized was contrails. Can you talk a little bit about that?

• 42:51Speaker 01

  Well, let me maybe spend a minute or two talking about the contrails problem.

• 42:56Speaker 02

  Yeah.

• 42:57Speaker 00

  For context, contrails, not chemtrails, which is a conspiracy theory.

• 43:00Speaker 01

  Yes, although you should also dislike contrails, but maybe not for the same reason. So contrails are, if you've ever seen those white clouds form behind jets, those are called condensation trails or contrails. And it turns out they add, at least according to the estimates that people have, about 1% of all anthropogenic global warming is caused by contrails. Why is that? I can just talk about the maybe the physics of that. So it turns out that there's actually two countervailing effects. Contrails are, well, sometimes if you've ever seen them, they're streaking and then they kind of go away. Those don't really do anything, but sometimes they last for a long time.

• 43:39Speaker 01

  You'll just see in the in the sky, just like almost like a waffle of of of just persistent contrails, they're called. And there's two effects that they have. Those are thin white clouds, so they reflect sunlight, but that only, of course, happens during the day. It turns out all all objects emit something called black body radiation. And the earth does at whatever the temperature is about 300 Kelvin. It's in the far infrared, around 10 microns. And at those wavelengths, contrails have very low albedo. They're almost essentially black. And so they'll absorb a little bit of the outgoing infrared radiation and then re-emit it both directions. So essentially, they'll reflect some of the outgoing heat, so it'll trap heat like a blanket.

• 44:22Speaker 01

  And so because that happens 24 hours a day, they tend to be warming. And it turns out it's a surprising, again, there's some uncertainty about it. But, you know, contrail cirrus, cirrus that sort of comes from from from contrails, might cover, especially in places like Europe, which has a lot of air like traffic, a few percent of the actual sky is covered by sort of additional contrails. Which so it adds, in those places like Europe, it adds about 1 watt per square meter of forcing, locally at least, which means that, just to give you a sense, all of anthropogenic warming sort of averaged across the whole globe is about 3 watts per square meter.

• 45:05Speaker 01

  So in places of high airplane traffic, it can be a lot of warming locally. So what can you do? Well, it turns out contrails are caused by areas in the atmosphere that are ice super saturated. They're a little bit like rock candy. So like when you have rock candy, you get a water solution that has too much sugar in it. And any little, you know, little bit of sugar in it will just crystallize all the the the sugar out. Just like in this contrail, there are these regions, they tend to be kind of pancake shaped, only a few hundred meters tall. And if you fly through it, the jet exhaust has a little bit of moisture in it, which will turn into droplets and then freeze.

• 45:45Speaker 01

  And then for every gram, if you're in this bad region, for every gram of water, ice or soot you put out, it's about 10 kilograms of water gets sucked out. So there's this enormous 10,001 curing ratio. So it's a big problem. So what you can do is you can figure out where these regions that are invisible, of course, these regions of icy supersaturated are, and then tell the plane to go underneath. And you only have to drop essentially what they call two flight levels. So it actually, it costs a little bit of fuel, but not very much to kind of avoid these sort of bad regions. So we built a system that sort of looks at satellite images and tries to detect where contrails are. So we have essentially a continuous monitoring system.

• 46:30Speaker 01

  And then try to build a model of of, because it turns out the weather models are not quite accurate enough to to find these places of ice supersaturation. So we built a custom model, again, like a convolutional net or a U net or something, to essentially to predict where they're going to happen. So that then you could be give maps to a flight planning software so that they can dodge it and inexpensively reduce the the the climate impact of aviation by a lot.

• 46:59Speaker 02

  What's the physics behind why you can predict that? Is it just, I see it in the satellite and then tomorrow I think it'll be there because planes go to the same place?

• 47:08Speaker 01

  Oh, no, it's because you're trying to detect these regions of ice supersaturation. Because they're very, very persistent, essentially they're.

• 47:16Speaker 02

  Oh, they're persistent.

• 47:16Speaker 01

  Oh, yeah, yeah. I mean, no one knows exactly, but they could last for days. Essentially, they're caused by, they think, sort of warm, moist air being injected just at the boundary of the tropopause between the, just at the bottom of the stratosphere. And then when humidity gets up there, it sort of sticks there for a long time and then gradually dissipates.

• 47:34Speaker 02

  Got it. So it's, there's just a sort of.

• 47:37Speaker 01

  They're like bad spots in the atmosphere you don't want to fly through.

• 47:40Speaker 02

  Right, okay. And so once you've established that it's probably good for a couple of days at least.

• 47:45Speaker 01

  Well, you have to keep predicting where that is.

• 47:47Speaker 02

  Yeah. And the the models you're using are, you've mentioned like CNNs or something like.

• 47:52Speaker 01

  That's right. And we haven't replaced those with era level models yet. But there was a very interesting problem that came up, which is you sort of want to know, well, just how much warming did this contrail make? And how much did it add to global warming? Because, for example, you might want to find the biggest ones. Because there's some fuel cost and maybe it costs a bit of money for the airplanes to avoid it. So you say, well, gee, I'd like to kind of know how much it did. But that's actually what they call a counterfactual problem. Like, okay, you made a contrail and certain amount of infrared radiation happened. So we can measure that if you're careful. But would have happened if there hadn't been a contrail there. That's a very difficult thing to estimate because you can't access the universe where.

• 48:37Speaker 02

  That didn't happen.

• 48:38Speaker 01

  That didn't happen. So you have to make these things called counterfactual models. And there's those are, if, I don't know if your listeners know, counterfactual models are actually pretty tricky to to fit and make. And we were, remember, there were two, there were two reflecting the sunlight and then there's the infrared. It turns out the measuring what the effect of reflecting sunlight is actually more difficult. And we were actually stuck on it for two years. We had a model that worked okay, a counterfactual model for the outgoing long wave radiation, but not for the reflected sunlight. Era actually helped us find a model, sort of searched all the confounders and sort of figured out like, oh, how can we estimate it?

• 49:21Speaker 01

  Because we had, again, we even had like test, test code on on sort of artificial, because you can kind of inject artificial, make artificial data sets where there's sort of injected contrails and sort of figure out, oh, well, we know how much it was because we injected it. And so, again, our own attempts didn't even pass our own tests. But Era's thing actually did and sort of unstuck this problem. So, yeah, we're in the middle of writing up a paper. We have a paper about the outgoing long wave radiation. But we have a paper that it's not submitted yet, but we've talked about it at at EGU, I think, where we actually solved this problem, so.

• 50:00Speaker 02

  And the the models that Era comes up with, are they just like a big monstrosity of of of code? Or are they like pretty basic and it's just you needed the intuition to develop?

• 50:12Speaker 01

  Yes, it's actually, in this particular case, it was actually more of the latter, that it essentially sort of helped identify what the, it was a very simple model with some just some number of confounders that we just hadn't, we just hadn't tried that combination before and it worked very, very well. So, yeah, it actually sort of came up with the, and it was sane in retrospect. So that was, I think, a big win.

• 50:32Speaker 02

  Yeah, that's interesting. I know you've done a lot of work in climate. What other other stuff have you done?

• 50:38Speaker 01

  I think I talked about this, right? The I talked about the CO2 thing. That was pretty fun. Because it's this, it's still quite a, the reason why estimating CO2 in the atmosphere is an interesting problem is we actually don't know what the carbon flux is into the bio, in and out of the biosphere. I mean, we do, we know that the biosphere captures, right? We emit a bunch of CO2 out into the atmosphere. And some of it gets absorbed into the ocean with sort of mostly inorganic chemistry, some some phytoplankton, and a lot of it gets absorbed on land. But we, but the error bars about what happens is are utterly large. And the error bars 50 years from now are very large.

• 51:20Speaker 01

  Like, the models in 2100, we don't know how how the biosphere will react to the ever increasing temperatures and CO2. So we don't actually know how much. The CO2 absorbed and the error bars are 300 ppm of CO2 just from the uncertainty of how what gets absorbed. And just to point out, you know, right now, there's about what 440, 450 ppm. So it's huge. I mean, it could be seriously amazingly awful or not great. But, you know, it did 300 ppm is like enormous uncertainty. So it'd be really nice to figure out, you know, can we can we reduce that? So this is like the CO2 concentration is

• 52:04Speaker 01

  like one step towards that solving that.

• 52:07Speaker 02

  And I know that Google has made some really big improvements in climate modeling and weather prediction as well. Right? Is it? I was at NERIPS this year, this last NERIPS. And, and, and I stopped by the climate track and, you know, I maybe only had a chance to listen to two talks or something, but I was, it really blew my mind, the sort of step change. I think this happened in the past. I don't know what it is, maybe 10 years or whatever, in terms of climate modeling. And I know a lot of that happened at Google. Can you talk a little bit about what has happened in Google and other places that has made that allowed that really big transition in in climate and weather modeling?

• 52:51Speaker 01

  Okay, so let me, people often sort of collapse climate and weather together. Well, because they're fundamentally the same physics.

• 52:59Speaker 02

  Yeah.

• 52:59Speaker 01

  Although, at least for the atmospheric physics, they're obviously when you start having ice and land, you know, climate is long term weather. And so the complexity of a full earth system model, which is a climate model, is much, much bigger than an atmospheric model. Like, you have to actually measure what's the water flux and the CO2 flux in and out of the land, or what will happen with ice. And so there has been a step change with weather models. Sorry, sorry, I want to make this. So whether is up to 15 days.

• 53:30Speaker 02

  Okay.

• 53:30Speaker 01

  Approximately.

• 53:31Speaker 02

  Yeah.

• 53:31Speaker 01

  Because, you know, weather itself, or the atmosphere appears to be chaotic. I'm hoping, I don't know if I should explain chaos. Essentially, it's the butterfly effect, right? That's small perturbations, like a butterfly flaps its wings and the weather will be completely different in, you know, two or three weeks. So weather is trying to predict the actual trajectory of the atmosphere over, say, two weeks. And that's now, that has been a huge step change. And that's because that's been a lot of, not even the new LLM stuff, that was based on, you know, the 2018 era, machine learning stuff, and just a large amount of data and a large amount of compute. So it's been, there's been a lot

• 54:14Speaker 01

  of sort of very clever work and a lot of it from Google, making new, new weather models. And it's been, it's been great. And in fact, we had a really neat breakthrough because now we can apparently predict tracks of cyclones, tropical cyclones, much more accurately many days in advance. And so places like Jamaica had got hammered by a terrible hurricane. And a lot of the classic models didn't actually predict it, partially because it's often that the, especially the intensification, it's all being driven by the, what the surface temperature is. Because hurricanes are, people might not realize are heat engines. Essentially, they they convert sort of heat in the ocean to big atmospheric motions. So weather's

• 54:58Speaker 01

  been great. Climate is much more difficult because you don't actually care about, you're not trying to predict whether it's going to rain in Seattle in 2070. You're trying to get kind of like averages. And what makes it difficult is that it's what they call non stationary. So that, in fact, literally, the, it's like the underlying physics or the underlying, like, you know, plants are behaving differently and, and ice behaves differently. And, and, and so it's a, it's very, very difficult to use sort of classical ML on sort of true climate models. And so that's why sort of the whole discussion you guys had about, you know, what we're talking about, about descriptive models and multiple hypothesis testing, that is incredibly severe in climate, because we don't, we have no

• 55:42Speaker 01

  data from 30 years from now, and we don't want to wait 30 or 50 years to find out whether we were right or that we overfit. So whatever things we do, you have to be kind of careful and try to peel off sub problems and, and the problem of, of exactly how do you inject, how do you build a big model that can predict into the future, but is still constrained by what we know? It's a fascinating problem. I think it's still unsolved, but it's a, it's a great problem to have because of, again, these uncertainties, we really would like to know what will happen in, in 60 years to the climate. So it's still, it's a thing, it's a very, very interesting problem to work on. But so far, AI has not revolutionized it because it's very, very resistant. Again, because of this data problem, it's a low data problem.

• 56:28Speaker 02

  Does the butterfly effect chaotic nature of weather, does that also impact climate? Or is the time scale so large that you have a closed system for which, you know, maybe it's oscillating between poles or whatever, but it's, it's sort of, when you look at it, that time scale, it's more stationary?

• 56:46Speaker 01

  It's unfortunately non stationary in a different way. But the original sort of whole chaos thing was, well, I maybe many people came up with it, but in meteorology, it was back to a person named Lorenz, who had this sort of very model, very simple model, ODE. So the difference between climate and weather is, is whether is where are you on the attractor? And climate is about the statistics itself of the attractor.

• 57:12Speaker 02

  Right.

• 57:12Speaker 01

  Problem with climate is that we're altering it. So the attractor itself is changing, it's moving. And there could be, and there could be, everyone talks about tipping points, that means that the attractor suddenly changes. And the trouble is, that's very, very, very difficult to predict.

• 57:27Speaker 02

  So even the attractors shape changes quickly?

• 57:30Speaker 01

  Or could.

• 57:31Speaker 02

  Could.

• 57:31Speaker 01

  And the trouble is, when you run a simulator, you don't know, like, is it, did it go unstable because my model's not great? Or is it an actual physical instability?

• 57:41Speaker 02

  Interesting, yeah.

• 57:42Speaker 01

  And it's extremely difficult to tell the difference.

• 57:45Speaker 02

  What do you do, especially when, I mean, to me, it strikes me that not only do you have not have future data, you really don't have much past data. You can do some measurements and ice cores and lots of stuff to try to do that. But there was nobody with an instrument 100 years ago.

• 58:00Speaker 01

  That's right. So if you have annual data or whatever, maybe you have, if you're lucky, 50 data points in any one location. Right. So whatever we do has to be very constrained by what we know. But it's just very different. I'm just telling you sort of the horns, the dilemma people on. So people make these, in fact, people in applied science in general, I would say climate is the most extreme, make these things called process models, where what you do, and I've seen the code, oh, well, you know, I'm going to be reductionist, and I'm going to sort of take the horrible complicated climate thing and sort of boil it down to 1000 pieces. And then I'm going to, you know, find the expert who wrote a paper about, you know, piece number 763, and he fit a cubic to some data, like, for

• 58:44Speaker 01

  example, one thing that's very mysterious, which is related to contrails, is how does ice behave in clouds? It turns out, you might, I get, everything is complicated once you dig into it. But it turns out that, like, when you make a contrail, how long does it last? Well, it depends on, because the way contrails can evaporate is ice starts to accumulate, as I said, and then the ice crystals get big, and then they fall. But of course, how quickly they fall depends on their shape, which is not known. And how much does the contrail mix from the, from the moist inside the contrail out? Again, people have approximations, but they don't know. And so the, the uncertainties are very much confound. And it's not just,

• 59:27Speaker 01

  oh, John, who cares about contrails? It turns out that the actual physics of, of microphysics of ice has very strong implications about what climate models do. And it's sort of, we just don't know. So I'm trying to say it's very gnarly, and it's not a solved problem. My hope is that with tools, maybe not like today's era, but maybe tomorrow's era, because it, remember, you can, as I was saying before, it not only can fit data, it can read papers, right?

• 59:58Speaker 02

  Yeah.

• 59:58Speaker 01

  And the question is, it can read a lot more papers than we can. So maybe, maybe we can integrate all the data that are all the knowledge that people have carefully evaluated, much more than any one person writing a piece of code, and fit data? I mean, that would be, that would be utterly glorious. We don't have that today. But that's sort of one of the hopes that I have, even for a more amazing tool in the future, is something that really can write code in a sane way, even much more sane, because it'll be constrained by all the scientific knowledge that we've accumulated so far. That would be amazing. We don't have that today.

• 1:00:35Speaker 02

  I mean, that strikes me as being very similar to biology.

• 1:00:38Speaker 01

  Oh, yes. Oh, boy.

• 1:00:41Speaker 01

  Right? If you've ever played biology or even looked at biology, there's so many exceptions and so many hacks in the biological systems. Yes. Oh, boy. So yeah, it would be amazing if we could have a thing that could really integrate all known scientific knowledge with data and try to synthesize sort of new models and new things.

• 1:01:04Speaker 02

  I think what you're saying is that AI can be an unlock here to some extent, because the models are so piecemeal, they are necessarily piecemeal. And so that being able to assemble the jigsaw puzzle, not to mix metaphors, but to assemble like a really, this jigsaw puzzle, having just scale and capacity actually helps a lot.

• 1:01:28Speaker 01

  That's right. The one things that these AIs have is somehow, you know, humans, even I'm pretty well right, I think, but it's just difficult for me to kind of integrate across the N squared different, my, N is the number of papers I read in a life, it's pretty big. And it's just difficult for me to even do that N squared thing. But somehow, there's just so much data in those billions and billions of parameters. And you can also give it access to read PDFs that it can somehow. Start to pull things together that people wouldn't do it. So that's, again, I'm starting to see little indications of that inside of ERA. I'm not claiming that's what ERA does today, but yeah, that's sort of my hope of where this is going to go.

• 1:02:08Speaker 00

  I think I've heard a lot of people suggest something like the route to intelligence is to combine LLMs with some form of search. So it's actually amazingly like ERA is doing that, right? Something which, you know, is a maybe a very, a very strong database lookup with a good search algorithm is one way.

• 1:02:25Speaker 01

  And of course, I mean, there was the whole, I mean, people still do, I guess, the whole rag thing, of course. And if you think about it, Google itself, you know, the 10 blue links things, it was or is a form of AI before we had LLMs, right? Because it was like, you can, you could cast yourself back to whatever 2010 or 2015. You could ask Google about literally anything and it will tell you stuff, right?

• 1:02:49Speaker 02

  Surprisingly well, actually.

• 1:02:51Speaker 01

  Yeah, surprisingly well.

• 1:02:52Speaker 02

  Because somebody on the internet has written about it, probably.

• 1:02:55Speaker 01

  That's right.

• 1:02:56Speaker 02

  If you can match that.

• 1:02:57Speaker 01

  In fact, that was one of the reasons why I wanted to come to Google is just that was such an, was such an amazing thing, right?

• 1:03:03Speaker 00

  I wonder how much of our audience did a search pre-Google and just know how bad that experience was.

• 1:03:09Speaker 01

  Yeah, I remember in 1998, I think, I think it's when Google, I was using AltaVista.

• 1:03:14Speaker 02

  Yeah.

• 1:03:14Speaker 01

  And like, I don't know, Google just got released and I used it and I just started, sorry, a digital.

• 1:03:20Speaker 02

  Yeah, I remember that.

• 1:03:21Speaker 01

  I just, I just dropped like a, like a hot potato or something and started immediately using Google. So it's, so that is sort of a form of AI. And so, yes, it might be, oh, yeah, it could be that just having access to all of that and sort of keeping it in mind at the same time.

• 1:03:38Speaker 02

  That model of sort of scientific discovery, in as much as it pans out, is kind of comforting too, because it is reductionist. So that you can look at the individual parts and understand them. So it found the exact things to assemble, but they're all actually maybe fundamentally things that people have invented. Or, or it's done iterations on it. And so all those little pieces are individually understandable. And then you can also put them together into a coherent picture.

• 1:04:06Speaker 01

  I think for a lot of problems like biology or climate science, I don't think we would trust the answer unless it was in that shape.

• 1:04:15Speaker 02

  Yeah.

• 1:04:15Speaker 01

  Because if there was some giant black box model that said, oh, this is how a cell works, it's like, do I believe it? I mean, I don't know if I believe it because I can't examine it, so.

• 1:04:25Speaker 02

  But I mean, to argue against it though, like if it, if it works really well.

• 1:04:28Speaker 01

  But you'd have to gather, you'd have to, I mean, you have to test it, obviously, but a, but a statistical model, you have, again, it has to extrapolate.

• 1:04:35Speaker 02

  Yeah, yeah. And it has to extrapolate to the extremes or the sort of the, the black swan events.

• 1:04:42Speaker 01

  That, that's right.

• 1:04:43Speaker 02

  Yeah.

• 1:04:43Speaker 01

  And so it's very hard. This is why things like self-driving cars are very, very, it's a very difficult problem.

• 1:04:47Speaker 02

  Right.

• 1:04:48Speaker 01

  It's all corner cases.

• 1:04:49Speaker 02

  Yeah.

• 1:04:50Speaker 01

  It's kind of amazing how well they've done.

• 1:04:52Speaker 00

  That's an interesting point, thinking about like when, you know, coming from the world of physics, where a model was usually a, a, a single equation or a small number of equations, which uniquely define a system and everything about it. And you just crank, you just find a solution to the system and you've, you now know everything you need to know. And I think something like AlphaFold was kind of a shift for a lot of people where before they thought, oh, protein folding is, you know, a problem where you just, if we find the right force field and we have the right computational engine, we will solve protein folding. And the thought of even really solving it in a data-driven way was only appeared a few years before, you know, AlphaFold, AlphaFold 1 came out.

• 1:05:36Speaker 00

  And it's interesting that I think it's sort of forced, the new AI modeling has, I think, forced people to reevaluate almost what is science? Because AlphaFold is, and similar models are incredibly powerful. There's a lot of things that they've opened up as tools, but at their core, they oftentimes don't give intuition in nearly the same way that, let's say, the, the most physicists historically would have wanted. And so I guess it's sort of, there's this old saying, all models are wrong, some are useful.

• 1:06:08Speaker 01

  Yes.

• 1:06:08Speaker 00

  And, yeah.

• 1:06:09Speaker 01

  Buck said that.

• 1:06:10Speaker 00

  Yeah, yeah. When do you find the data-driven models to be sufficient? And when do you want sort of like something which is interpretable that humans can actually understand?

• 1:06:20Speaker 01

  I think it boils down to almost like the difference between weather and climate. If you're in a data-rich regime, like weather or even proteins because of PDB, you can feel, oh, yes. In other words, I've, I've got enough data to kind of cover. And so a statistical model like AlphaFold should do. And so in fact, a lot of people happily use Alpha. You know, I think it's really revolutionized my understanding. I'm not a biochemist, but people seem to love it. And the, and the, and one amazing thing they did is they exhaustively, they just, just ran it on all PDB and, and published it. It was just really, really cool.

• 1:06:57Speaker 00

  It's like 6 billion protein predictions.

• 1:06:58Speaker 01

  Yeah, so, so.

• 1:06:59Speaker 00

  The vast majority are actually quite accurate, yeah.

• 1:07:01Speaker 01

  Yeah, so that's just amazing. So, but it feels closed, if you know what I mean.

• 1:07:05Speaker 00

  Yeah.

• 1:07:06Speaker 01

  But when it's like climate and it's open and it's non-stationary, or you have to make these big extrapolations, you have to be much more cautious. Or maybe biology, again, and there's probably, there may be parts of biology like, oh, there was this virtual cell challenge from the ARC Institute.

• 1:07:24Speaker 00

  That had a funny result.

• 1:07:25Speaker 01

  I, I, I know that people were, there may have been some overfitting for at least that.

• 1:07:29Speaker 00

  Yeah.

• 1:07:29Speaker 01

  Would we say more, sorry?

• 1:07:30Speaker 00

  Or I guess maybe at a high level, I think, the simple baselines.

• 1:07:35Speaker 01

  Oh, worked very, very well.

• 1:07:36Speaker 00

  Yeah, yeah.

• 1:07:37Speaker 01

  Yeah, yeah, yeah, just like in the.

• 1:07:38Speaker 00

  One of the classic things in whenever you do biology is just always start with a simple baseline. Maybe this is probably just good ML in general.

• 1:07:45Speaker 01

  It's good ML.

• 1:07:45Speaker 00

  Yeah. Understand your simplest case in, in biology, there are many problems where they're extremely resistant to anything beyond the simple baseline, even if you have a lot of data.

• 1:07:54Speaker 01

  That, that's right. And in fact, I tell people the same thing. I said, always just, just fit linear regression, just fit.

• 1:08:00Speaker 00

  Just do it.

• 1:08:01Speaker 01

  Just do it, just do.

• 1:08:01Speaker 00

  Or SVM, yeah.

• 1:08:03Speaker 01

  Or SVM, I mean, SVMs are, are just a different.

• 1:08:06Speaker 00

  Different form of.

• 1:08:06Speaker 01

  Linear regression, yes. So.

• 1:08:08Speaker 00

  Different basis, but.

• 1:08:09Speaker 01

  So when do you need the more process model-y thing? I think it's just when you, when you have, I mean, sort of climate is on one end and I don't know, weather maybe on the other end, maybe. That may be too extreme, but there, I think it's where are you on the, on the data richness thing? When do you, when can you feel like, oh no, I really have a closed, I really have a closed problem? And I think I can actually cover it.

• 1:08:33Speaker 00

  Yeah, closed problem that you, data fully covers is, yeah, I think that, that makes a lot of sense in what I've seen as well. Yeah. I think one thing I'm kind of curious about is when you're working on climate modeling, what are the, you talked about contrails. You've talked about CO2 predictions. What are the, the, the broad things you're trying to accomplish? So one of them is, I guess, making up, like making interventions and the other one might be making predictions for things like insurance? Or like, how do you help like adjust for some sort of climate change? Or like, yeah, what are the principal goals, I guess, for you specifically or the community at large?

• 1:09:15Speaker 01

  I think, you know, just like any community, there's probably many different goals. For me, I'm, or and my team, we're very, very interested in interventions.

• 1:09:26Speaker 01

  So like, which ones are possible at, at, at relative cost? I mean, contrails was kind of amazing because it turns out that the intervention is quite low cost and we, and also one amazing thing about contrails is they are local, unlike things like CO2, because, so if a country decides to fix contrails over itself and actually improves its, I mean, it, it has the global effects, but it mostly improves the, the climate a little bit over themselves.

• 1:09:52Speaker 00

  Yeah.

• 1:09:52Speaker 01

  So they, they like that.

• 1:09:53Speaker 00

  I guess that if you are in a cold climate and you want to warm it up, this is now your own.

• 1:09:57Speaker 01

  Oh.

• 1:09:58Speaker 00

  You could now.

• 1:09:58Speaker 01

  Yeah, it, it turns out that it's a little bit asymmetric. The warming is constant essentially and global. The cooling only happens when you're sort of at a good, then the sun is at a good angle over you. So it's very rare that the uncertainty, there are contrails that were our uncertainty bounds in terms of the warming. There are many, many contrails, mostly at night, of course, where the, where, where it's, it's like a largely warming and we're, we're very sure in terms of like two sigma. There's very, not very many contrails where you say, oh, I know for sure that it's cooling and I want more of it. And like, like, so only over sort of the poles in polar summer, do you know that the contrails are cooling?

• 1:10:41Speaker 01

  And therefore, if you got rid of them, they would warm up, but there's essentially no flights over Antarctica and not that many over, over the poles in the summer, so.

• 1:10:50Speaker 00

  So no one's gonna, no one who lives in a cold climate is gonna use this maliciously.

• 1:10:54Speaker 01

  Well, yes, in that, in that, well, they wouldn't know for sure whether it was warming or cooling. And so they would do stuff and so mostly we just sort of ignore, we don't recommend that people fly those.

• 1:11:06Speaker 00

  You also brought up an interesting point about the economics. I mean, a lot of, there, I think, was a lot of resistance historically about certain, you know, to climate, certain climate change interventions, which have, in some sense, the market has just taken over. Like, at this point, unambiguously, like renewables and batteries are just almost universally, unambiguously, just better than alternatives.

• 1:11:28Speaker 01

  For, for, for, for, for, for non-mobile. I mean, you.

• 1:11:33Speaker 00

  That's a really good point. Yeah, yeah, like planes, we, we do not have a solution to.

• 1:11:38Speaker 01

  Correct. I mean, there are some battery powered planes, but they're very small and have to go very limited range.

• 1:11:43Speaker 00

  They probably will never actually be.

• 1:11:45Speaker 01

  It's hard to imagine the physics would be very, very, unless we came up with something like nuclear batteries, which would be kind of amazing, but I don't, we don't know how to do that.

• 1:11:52Speaker 00

  Or even if we did, I think the, the risk of like, people would be too afraid of a nuclear battery going wrong or something.

• 1:11:58Speaker 01

  Oh, yeah. Since we don't know what they are, we don't know what the risks are. Yeah, so we don't know. So, yeah, that's, that's the problem with, I talk about, I have given talks about climate change and I talk about the pi, the pi turd of badness, pi turd of sadness, which is, there's no one silver bullet for climate change, right? There's so many different things that contribute greenhouse gases just from across our economy. So they sort of all have to be fixed or many, many of them have to be fixed. So there's no one single thing. I mean, I've worked on fusion, fusion's cool, and it might actually knock a lot of them out if it's cheap enough, which we don't know, because we don't know if it'll work yet.

• 1:12:34Speaker 00

  I mean, fusion is one of those interesting things where the joke was always fusion is 30 years away, but I think it's actually now less than 30 years away, maybe.

• 1:12:42Speaker 01

  Yeah, no, I think there's a, there's a definite probability that, that someone will make commercially relevant fusion even by the end of this decade. So I think it's like three years away, not 30 years.

• 1:12:55Speaker 00

  This is very real. Interestingly enough, I think a lot of that, I'm gonna not just stump or advertise some of our other episodes, but a lot of it actually comes down to material science, interesting enough, in that.

• 1:13:06Speaker 01

  Well, I'm skeptical. Oh, super connected, sure, sure, sure. Well, yeah, sorry, we can talk about fusion if you.

• 1:13:12Speaker 00

  Yeah, yeah, there was actually two, two things. One is fusion, the other is better control systems, which I think is.

• 1:13:17Speaker 01

  Yes, and in fact, Google DeepMind has been working on control systems for Tokamaks to make sure they don't essentially go unstable and go disrupt. Yeah, that's right.

• 1:13:28Speaker 00

  Disruptions are quite interesting to themselves.

• 1:13:30Speaker 01

  Yes, yes.

• 1:13:31Speaker 00

  Yeah, it's basically the entire, the, all energy in the, the Tokamak culminates into one little beam and then.

• 1:13:38Speaker 01

  And then it hits your vacuum chamber and you're very, very sad.

• 1:13:42Speaker 00

  Yeah, I think people believe that either could be turned on after $30 billion disrupt and then basically have a $30 billion, $30 billion brick or something.

• 1:13:52Speaker 01

  Oh, yeah, I guess, I guess you could try to patch it. I remember working, we, again, before LMS, we worked with a fusion company called TAe and I was in their control room. And yes, it was kind of sad, you have to be very careful. We were making systems to recommend new experiments and they were very, very skeptical and jaundiced, which, where they should, because, because I've even been there for that, even under human control. It's like, they were doing some experiment and then you hear this big bang and it's like, oh no. And then it's like, you know, then, then the apparatus is down for two weeks as they patch some.

• 1:14:26Speaker 00

  You were there during a disruption?

• 1:14:27Speaker 01

  Oh, no, this is, this is, sorry, they have field reverse configuration.

• 1:14:30Speaker 00

  Oh, okay.

• 1:14:31Speaker 01

  So, so, yeah, which has its own, I mean, things, you know, there's some arc.

• 1:14:35Speaker 02

  So, so what is that? Sorry, I'm not familiar.

• 1:14:37Speaker 01

  Oh, oh, what's the field reverse configuration? Well, it turns out Tokamaks are not, although they're perhaps the most studied form of plasma, there's many different kinds of architectures, essentially ways to try to stabilize and, and compress plasmas. There was a, a shape, essentially, it's essentially a self-contained football of plasma called the field reverse configuration, where essentially the magnetic field inside and outside are opposite. So they're separated by something called a separatrix. And that is sort of in theory unstable, but in practice, stable. Like, for example, when you run magnet hydrodynamics, MHD code, it, it's unstable under that assumption, but that's, that's an assumption, that's not the way the real world works.

• 1:15:17Speaker 01

  And so, yeah, it was kind of disfavored for many years, but TA and other people, I think Helion have, have FRCs because they are actually relatively robust. You can actually knock them against walls and they'll, they'll still, they stay stable. And yes, but you can still get discharges and things that punch holes in your vacuum chamber, which is kind of unfortunate.

• 1:15:39Speaker 02

  For clarification, so you have these fusion reactors, they are, or trying to be reactors, maybe, and.

• 1:15:48Speaker 01

  Apparatuses, yeah.

• 1:15:50Speaker 02

  And you create a plasma, the plasma is magnetically charged.

• 1:15:56Speaker 01

  Confined, yeah.

• 1:15:57Speaker 02

  It's confined, so it's confined by a magnetic field. So you have some sort of magnetic system that is tunable by a computer and then the computer tries to kind of maintain the confinement.

• 1:16:11Speaker 01

  Well, FRCs kind of, once you make them, they're sort of sustained. There's different ways of, of trying to make sure you, okay. So all fusion boil stone has been called the Lawson criteria. There's essentially, and it's, it explains why fusion is hard. Essentially, you can just very easily, on the back of an envelope, just show that the density, the temperature and essentially the energy loss, it's called the confinement time. It's one over the amount of time it takes for the energy to decay away, one over E in a plasma. So the product of those three numbers has to be bigger than some constant and then you can get fusion. And if you don't, then you don't. And that's explains, the fact that it's a product of three numbers explains why fusion is so hard, because it, every approach has an Achilles heel where one of those numbers is not very big

• 1:16:56Speaker 01

  and then they try to desperately make that be higher. Yeah. And every approach is different, every approach to fusion is kind of different. And a lot of, you have to be a bit skeptical when there, there's all these sort of breathless news things about fusion, because they'll say, you know, no, confinement time is starting to like, oh, stable for X minutes or whatever. And it's talking about like one of the three numbers, but you have to have all three numbers before you can get fusion. I, I think that the whole field is, is making a lot of progress and it's very exciting, but you do have to, you have to be a little bit cautious about the breathless news articles that only talk about one number.

• 1:17:30Speaker 02

  So what is the computational part of that?

• 1:17:32Speaker 01

  Oh, unfortunately, for better or for worse, it depends on the approach. So for Tokamaks, as Brendan said, it's, that there's this, it's mostly stable, except that there's occasionally this instability that takes all the energy and smacks it into one place. And so you have to sort of keep everything sort of under control. So it's a control system. FRCs themselves have very simple instabilities. So for example, they have what they call a Z instability. So it's fine, it's stable, it'll just wobble, literally wobble back and forth, but you just make what they call a PID controller that just keeps the football in the center of the reactor and, and things are fine.

• 1:18:10Speaker 02

  And it does that by adjusting the magnetic field?

• 1:18:12Speaker 01

  Yeah, it's sort of, it actually adjusts, I think, the electric field is sort of by, sort of knocks it back and forth. The issue that people have is it really depends on which, which sort of plasma architecture they're deciding to use.

• 1:18:23Speaker 02

  Climate is, you know, sort of political because of economics, basically, probably, mostly, maybe other stuff, but the, the, the economics of it, you know, you have to persuade people to, to somehow spend more or you have to have a solution that it has like this happy coincidence where it's both economically better and, and better for the climate.

• 1:18:48Speaker 01

  That, that's hard.

• 1:18:48Speaker 02

  Yeah.

• 1:18:49Speaker 00

  But in cases, it's not, I mean, many cases it has, hasn't been hard, but yeah.

• 1:18:54Speaker 02

  Think about like, okay, predicting even weather, right? You can prep and you could see how that could be economically beneficial. So what kind of work are you doing with interventions and how does that kind of interact with economics? Like, it sounds like the contrail is one, we did it in a, an analysis and said, actually, this is great because it is very low economic impact, but high value.

• 1:19:19Speaker 01

  That's right. So if you're trying to think, there's, there's sort of energy intervention. So you have to be, you have to sort of compete with existing forms of energy. And that, that's not trivial, unless there's a co-benefit or there's some, there's some sort of clever, you know, just co-benefit, like, this again is highly speculative, it wasn't our work. There was a startup that was, I don't know if you saw the news, it was last year, I think, where someone figured out if you inject mercury into a fusion reactor, that the neutron flux can actually transmute the mercury into gold and then you, then you can sell the gold, which I thought was very clever. It might not work, but.

• 1:19:55Speaker 00

  You know, as a physicist, the one thing I want out of a fusion reactor is helium, but that's a different story.

• 1:20:01Speaker 01

  Oh, you, oh, helium, helium at three, well, I mean, helium four is kind of boring, although it's, it is getting, because the strategic reserve has been shut down, there's less of it, yes. And of course, I want helium three, helium four.

• 1:20:13Speaker 00

  Yes.

• 1:20:13Speaker 01

  Well, not even just a fuse, just to make dilution refrigerators for quantum.

• 1:20:17Speaker 00

  Or MRIs or, yeah, so much, so much technology we think about actually just goes out the window if we run out of helium.

• 1:20:23Speaker 01

  That's true.

• 1:20:24Speaker 00

  No one's thinking about, sorry, that's like a complete aside.

• 1:20:27Speaker 01

  Yes, the fact that, that the US had a helium, strategic helium was the reserve was for a very important blimp fleet.

• 1:20:34Speaker 01

  But, but they kept it for decades anyway, so that was nice, but then we stopped.

• 1:20:37Speaker 00

  And then we got rid of it all, it all went up in the air.

• 1:20:39Speaker 01

  Yes, yes, in balloons and stuff or out of natural gas wells.

• 1:20:43Speaker 00

  Yeah.

• 1:20:44Speaker 01

  Sorry, now we're talking about helium.

• 1:20:47Speaker 02

  Interventions, what are some of the most exciting, interesting ones?

• 1:20:51Speaker 01

  Well, I'm very excited by fusion. I mean, I don't know if it's intervention, that's sort of a, a source of energy. Because if we can make, if we can make it work and we can make it be sort of low enough capital cost that that will actually help a lot, because at least the, the current models are renewables are great. Ideally, you'd like to electrify everything, right? Which has problems because like you can't electrify flights, but you could try to electrify a lot of stuff. You know, there are EVs, you'd have to figure out how to electrify things like cement or steel, but those are hard, especially things like making steel want reduction power anyway to essentially, you're adding carbon and you're reducing iron ore. So there's a lot of sort of things that are difficult about electrifying everything.

• 1:21:36Speaker 01

  But if you could electrify everything, then, then the amount of electricity required would grow by a factor of five. And you could try to grow renewables, renewables plus battery, again, trying to squeeze all of it out, it starts getting ever more expensive because you just need ever more, you need like a huge number of batteries to sort of cover the, the last, you know, few percent or even, you know, 10 or 20%. So we do need some sort of power that, that can cover the last 20%, something that's, you know, base load. So fusion might be a, a thing for that. So that's super exciting. Again, there, there's no one sort of silver bullet that can sort of cover all the cases. So I'm happy to sort of talk about any specific case, but it's sort of like.

• 1:22:19Speaker 01

  The world, the world is a very complicated place and the global economy is a very complicated place. So it's super hard to sort of talk about sort of interventions in general. Yeah.

• 1:22:27Speaker 00

  Maybe instead of interventions, one thing I'm curious about is how does this make, affect decisions and to, for example, like what do we build? How do we build? I think you're from LA, right? Or at least you-

• 1:22:40Speaker 01

  Well, I spent 11 years there, yeah.

• 1:22:42Speaker 00

  You're like, okay. So yeah, you spent a lot of your life in LA. I mean, LA just basically large parts of it just burned down. And maybe probably close to where you used to live. So the, you know, this is something that I think a lot of people kind of saw coming, maybe partially due to regulatory issues, but partially due to other issues, you know, and we were completely unprepared. And it seems like there is a lack of preparation about what to do next, or to sort of adjust for this. And I mean, have you worked on basically predicting like new risk assessments or, you know, suggestions like what do we actually change to maybe harden, you know, society even for what's coming, regardless

• 1:23:26Speaker 00

  of whether or not we're actually do something to make, solve the underlying problem?

• 1:23:29Speaker 01

  That's right. So in fact, there's a big effort at, at Google into something called Crisis Resilience. And so we had a very fun project called FireSat. I don't know if you know about this. So it turns out that for wildfires, a lot of these wildfires, you could, if you, if you only caught them early enough, it's very easy to put out a wildfire the size of this room. But even if it's like an acre, it gets much, much harder. And so, and of course, under certain circumstances, they can grow exponentially from, from the size of this room up to an acre. So that might be hard to catch. But they often sort of start small and spend a while. So we figured out that, oh, if you

• 1:24:12Speaker 01

  had a global constellation of low Earth orbit satellites that could detect in the midwave IR, which goes back to the black body, essentially that's the temperature of fire, they stand, the fires stand out in the midwave IR. And so we designed a sensor that if you built, it depends on exactly what their orbits, but you know, roughly 50 to 80 of them, you could actually find fires about the size of this room, about five meters on a mount, maybe it's a bit larger in this room, five meters on a side. And anywhere on the planet, and again, depending on how many satellites you had within, within like 15 to 20 minutes, you'd have to put, you know, a fair number up, like, you know, 80 to get them within

• 1:24:57Speaker 01

  15 minutes. And then you could actually intervene. You could decide not to if you wanted to, you know, have the fire burn fuel and you thought it was safe. But if it was going to blow up to something unsafe. So we worked with a now a nonprofit called Earth Fire Alliance that we're part of. And so they're starting to like, we've launched one satellite, which is a prototype. We've worked with a company named Muon Space to actually sort of make the satellite. So that's cool. We have wildfire boundary detection and we propagate that information out through Google. So we can actually sort of figure out from existing satellites and existing data feeds where the boundaries of fires are. And then we sort of tell people through, through their Android phones or through

• 1:25:37Speaker 01

  search about fires. We've worked with the US Forest Service on making new models for how fires propagate. Cause again, that goes back to these process based models from the seventies by a person named Roethmer. So we've actually made a little neural network proxy model based on a new, essentially to sort of be able to run it very, very quickly. So we worked with the Forest Service on that. So yeah, yeah, we're very, very interested in trying to minimize, because it turns out people might not realize, the World Health Organization estimates that there are 300,000 excess deaths a year across the world from wildfire smoke.

• 1:26:14Speaker 00

  Yeah. I mean, I remember, it's been a few years since we had a really bad fire season. I mean, what, four, five years ago, there was this cloud of smoke which like crossed all of North and the US and Canada and caused a lot of respiratory issues, I think.

• 1:26:28Speaker 01

  Yes. Yeah. And it's very hard to track. I mean, you have to get these, you have to get the estimate these excess deaths from statistical means. But yeah, it's a very serious public health problem. And also just very scary and it burns people's houses down and it's terrible.

• 1:26:42Speaker 00

  Yeah.

• 1:26:43Speaker 01

  My view is that climate change is sort of like a serious disease. Do you treat the symptoms? I.e. do you adapt? Or do you try to attack the underlying thing? And the answer is both. It's serious enough to see both.

• 1:26:52Speaker 00

  Yeah.

• 1:26:52Speaker 01

  Right. And so, yes. So we take sort of adaptation, especially around climate resilience very seriously at Google. And we try to give people informational tools to sort of help. That's part of the reason why we were working on weather and then sort of cyclone prediction and things. So it all actually hangs together. So it's more than just, you're right, it's more than just interventions. It's climate resilience too.

• 1:27:13Speaker 00

  Yeah. Having lived through four or five fire seasons on the, on the West Coast, they're, they can be quite, quite nasty.

• 1:27:20Speaker 01

  And it used to not be. I mean, I have a cabin up in the Sierra Nevada mountains. And yeah, it used to be, oh, you know, summertime, it's nice. And then now it's like, well, not every year, but yeah, there's like, you know, winter, spring, summer and smoke.

• 1:27:32Speaker 00

  Yeah. Fire season. Yes. Yeah. I want to stay, stay to the west of the fire line.

• 1:27:38Speaker 01

  And yeah. Yes. And I, yes. So that is another thing that I'm interested in and that Google's also very interested in is, is climate resilience.

• 1:27:48Speaker 00

  Are these IR sensors small enough that they could hitch a ride in like a microsatellite grid? Like, would it make sense to, would it be almost cheaper just to hire, I mean, to pay someone who's watching a constellation?

• 1:28:00Speaker 01

  Oh, they're not that small.

• 1:28:02Speaker 00

  They're not that small.

• 1:28:02Speaker 01

  The thing is you need refrigeration.

• 1:28:05Speaker 00

  Oh, okay. Okay.

• 1:28:06Speaker 01

  Because it's midwave IR. So you have to, you have to keep it cool.

• 1:28:09Speaker 00

  Okay. So these would have to be their own special satellites.

• 1:28:11Speaker 01

  They're not, they're not super large.

• 1:28:13Speaker 00

  Okay. Yeah.

• 1:28:14Speaker 01

  They're not like the, you know, the satellites in, in two synchronous orbit are these giant monsters because of all the optics and who knows what. But yeah.

• 1:28:22Speaker 00

  And they have, you know, they're basically just IR sensors with a resolution of five by five.

• 1:28:28Speaker 01

  No, no. That's the other cute thing is their resolution is about 50 by 50 meters. But you can use super resolution because it's essentially multi-spectral and you sort of know where fires are. And you have, so yeah, there's a fair sprinkling of AI on top of them to, to reach that five by five meter.

• 1:28:44Speaker 00

  Oh, that's cool. Yeah. Yeah.

• 1:28:45Speaker 02

  When you also have, you have a convolution over the, what you're reading out, right?

• 1:28:49Speaker 01

  The, I forget the frame rate. The satellite is moving. I don't remember what the point spread function is. I'm sorry. I don't. But you're right. They do move. But I don't remember how fast they, I don't remember how fast they, this also has been called a broom sensor. So there's this funny thing of trying to, you kind of spread out the spectrum one way. And there's also, it's, it's a somewhat complicated thing. It isn't just a, like a Polaroid. It's, it's a complicated sensor. Yeah.

• 1:29:19Speaker 00

  So we're switching gears a little bit. You know, you've been at the intersection of AI and science for quite some time. I think you've sort of wound your way into and out of it, back and forth. How do you see the field has evolved? Because I feel like it's evolving very quickly now. And like, what are the sort of lessons that you've, you've learned that you think the community has learned? And how do you think this should change? If you are a young scientist or young practitioner, how should this change what, how you should approach, you know, the future?

• 1:29:49Speaker 01

  Well, I think there has been a phase change in the last 12 to 18 months. So I mean, a lot of what we used to do, as I, as I said, was build these specialized models to solve natural problems. And if you think that's your job, it's kind of fun. You, you find a problem, you solve it, you find another problem, you solve it. But now we have these much more general AI things. And I think the whole AI for science community is kind of still feeling around. There's, the fact that they're working is so new that collectively we're not sure, like, what's the best thing to do? Or maybe there's no one best. Maybe there's a tool chain. And I think, I think we're all trying to figure out, like, what

• 1:30:31Speaker 01

  should we do? And so there's a question of what should young scientists do? I think it would be, you know, I, I have a son who's, who just turned 21 and he's really into both sort of AI and coding and chemistry. And I look at him, I think he's doing the right thing because he's both learning a lot and trying to be a domain expert about RNA, but he's also sort of using vibe coding and using all the tools. I think that's, I think that's the right answer is, is, is because everyone's figuring it out. Still be a deep domain. I don't, I don't think domain expertise is going away because it goes back to a lot of people have said it goes back to taste and trying to figure out how people get

• 1:31:15Speaker 01

  taste without doing all the grunt work. That's an interesting open question, but develop domain expertise, but also try and play, I would say with all the different tools that are available, because it's not like, oh yes, we know what's going to happen and the smart old people are knowing what's, no, we're experimenting too. And so, uh, yeah, so I would say, uh, definitely develop domain expertise and try to use these tools and try to solve big, hard scientific problems as best you can. There's still the huge open issue about what do you actually do about physical lab work? That is not going away because, you know, experiments are the, the ground truth and

• 1:31:56Speaker 00

  And a bottleneck.

• 1:31:57Speaker 01

  And a bottleneck. I mean, people are talking about a lab in the loop, but that's still very, very, very open because how do you, how you, no one has, as far as I know, a general lab that does everything. There's a lot of very specific, you know, labs that are controllable. Um, so I think it's just, we've gone through this phase change. Uh, it seems super exciting. Again, I would advise people to play with whatever tools are available and to develop sort of deep domain expertise and taste to the extent you can. And I would advise people also not, not to be scared and learn to like, try stuff, you know, uh, you know, I'm always happy. Uh, we have student researchers at Google and they come and they do sort of wild and crazy things. And that's

• 1:32:39Speaker 01

  always just delightful. So yeah, people should be trying for wild and crazy things and see what happens.

• 1:32:45Speaker 00

  This may be a question without an answer, but when I think about how I developed expertise and how a lot of people developed expertise, it was by starting with a, you know, a, a simple defined problem and then hammering it. And then in that process of exploration, you learn more and, you know, some ways you go broader, some ways you go deeper, but you still, the process of just banging your head against a problem, which now would be instantly solvable, teaches you the skills you need to solve harder problems. What advice would you give to your son for that?

• 1:33:16Speaker 01

  You know, I don't know. Maybe it's a bit like hiking, which is your, yes, I mean, you obviously can't drive everyone or you could, you could drive up the mountain.

• 1:33:22Speaker 00

  Yeah.

• 1:33:23Speaker 01

  Or you could hike up the mountain and maybe it's okay, even fun to occasionally hike up the mountain, even if you can drive up the mountain. Uh, yeah. I mean,

• 1:33:30Speaker 00

  And in your old days, I'm gonna sound like a real man. In the old days, In the old days of six months ago.

• 1:33:35Speaker 01

  Oh, no, no. I was even thinking of in the old days of the eighties, nineties, like, you know, a lot of people take it like, oh, there's open source packages. There's, there's Glearn. There's, there's all sorts of things. We didn't have that. I had to write my own numeric library. I had to write my own machine learning. I've, I've written boosting, probably rewritten it four times, four different languages. And, and so now I know boosting, you know, it's like, and so maybe, uh, not taking the totally easy. I mean, obviously, I mean, there's this trade off like, oh, but I want to be as efficient and productive as possible. Yes. But you also have to develop the muscles. So it's a little bit maybe like being an athlete, like there are places, there are times when you're actually doing exploit, when you're trying to run as fast as you can. And then there's also training time. And so maybe people just have to train.

• 1:34:17Speaker 00

  And it's entirely plausible that if you spend time actually hammering away and doing the hard work, even if it goes slower there, that pays dividends into your larger, um, you know, productivity long-term. Like even if locally, that one moment you are not being maximally productive by not exploiting it LM, that feeds into something.

• 1:34:38Speaker 01

  I hope so. I hope, the thing I don't know is I hope people in their careers, it's hard because right, the, the, the whole world seems to want to optimize everything. And

• 1:34:49Speaker 02

  It's your fault.

• 1:34:50Speaker 01

  No, I know. It's, no, I don't, it's my fault, but it's just sort of the, the, you know, and sometimes you have to set aside time, like at Google, especially in my group, we have this concept of 20% time, which I still, I very, very strongly in my own group try to protect. It's like, you can do whatever you, if you want to learn stuff, if you want to try stuff, you don't even have to tell me. You don't even have to tell me. In fact, probably shouldn't tell me, uh, you know, just do stuff for, for exactly for learning and also because that's where the sort of creative juices are. I don't want to so occupy people's time where they have nothing like, where they, they can't feel like they can play or learn or try new crazy things. So I know 20% time is unusual and there just seems to be this strong impetus in the world

• 1:35:33Speaker 01

  to just like, like I said, optimize and squeeze everything out. But you do lose something when you hyper opt, you sort of overfit.

• 1:35:39Speaker 00

  Yeah.

• 1:35:41Speaker 01

  So

• 1:35:41Speaker 00

  You overfit to productivity as you

• 1:35:42Speaker 01

  Yes. So that's right. So I, I know that might be, my advice might be swimming upstream against, uh, perhaps cultural norms.

• 1:35:50Speaker 02

  The thing that always comes up for me here is that I've, there's a, the problem is not stationary. There's a new skill set that will be the, the right skill set for the future, right? And that the question in my mind is always just tangling, okay, is this a skill that is an enduring skill that

• 1:36:11Speaker 01

  Yes.

• 1:36:11Speaker 02

  Or, or like maybe it wasn't enduring yesterday, but today it will be enduring because the, like I've seen that, um, you know, like for just as an obvious one, you become sort of like a manager when you're

• 1:36:24Speaker 01

  Yes.

• 1:36:24Speaker 02

  When you're using

• 1:36:25Speaker 01

  Absolutely. I tell people this.

• 1:36:27Speaker 02

  Skills transfer well. Some of them don't, but a lot of them do. And so that as a manager, you lose track of, of the details of what's going on and you trust your people or agents or whatever to have that managed so that they can report up to you and answer, uh, you know, sort of the high level questions and, and get the judgment about the little things correctly. And, and so that is that what we've come to is that we're just like middle managers now?

• 1:36:57Speaker 01

  Well, I, I, I mean, I don't know. Again, I, I, I have a little management work, but you can't, I don't, you don't want to be an empty suit.

• 1:37:07Speaker 02

  Yeah.

• 1:37:07Speaker 01

  In other words, because the things might get it wrong, especially LM's that are sort of really weird. They don't make the same kind of mistakes that humans make. And so you, you can't, you know, fully trust them. You have to be rigorous and like, you know, poke at it and make sure. Although you should be poking at software that you write yourself to. I mean, you shouldn't trust yourself. That's, that's one thing I've learned. What did Feynman say? You know, you absolutely can't fool yourself and you're

• 1:37:31Speaker 00

  You're the easiest person to fool. Yeah.

• 1:37:33Speaker 01

  So, um, so that might be, I don't know if I, I can quantify what's enduring, but somehow fundamentalness, I mean, really learning a domain that is about the world, for example. So this is why I like, well, biology or physical sciences, I think those will, those are enduring sort of fundamental things. Math is very enduring, but even things like rigor and checking and that sort of thing, which goes back to maybe management that, that you want to really make sure that the LM's are producing the right things or they haven't cheated in some way. But again, you should be doing that to yourself too.

• 1:38:10Speaker 02

  Yeah.

• 1:38:11Speaker 01

  So I think there's some enduring and also just the enduring value of sort of creativity and thinking out of the box. And so I think those are enduring. I don't know. There's some, there's something very fundamental about all of those.

• 1:38:23Speaker 02

  So you mentioned Feynman, um, if you don't mind me, uh, changing gears.

• 1:38:28Speaker 01

  Yeah. I took a class from

• 1:38:29Speaker 00

  Yeah. Yeah. Not just any class.

• 1:38:32Speaker 01

  Yes. The, the, his, his sort of physics of computation class. I did it with in fact, Hopfield and, and, uh, Carver Mead. Uh, that was fun. At the time, I don't think, maybe Feynman knew, I, I felt like none of us knew what even the problem was. I mean, I guess Feynman was trying to say, oh, let's do quantum simulation, which I guess is, uh, it turned out to be the right answer. But yes, at the time it was, oh, well, I guess, uh, it was cool. First of all, it was DARPA funded it. So you were supposed to go once a week to get a primer dinner. Uh, but I just went every week. Uh, anyway, for the students.

• 1:39:07Speaker 00

  Yeah. So just for a little bit more context, this class was basically the class right after Feynman and I forget who else proposed the concept of a quantum computer without really knowing what it was, but knowing that there was some sort of

• 1:39:20Speaker 01

  Well, there was plenty of, there was plenty of room at the bottom essay, which I think was in the very early, this, I took it in 82.

• 1:39:27Speaker 00

  Okay.

• 1:39:28Speaker 01

  Oh, that was when he taught it. And he did say there was plenty of room at the bottom, but the way the class was structured, it was, um, uh, it was like a guest lecture on Tuesday and then Feynman would stand up on Thursday and explain why that was all wrong.

• 1:39:46Speaker 01

  Which is pretty fun. And then we encountered something which other people have also encountered, something which is called the Feynman effect. Maybe he was so charismatic or something, he would explain things and you would say, yes, yes, I understand. And then you walk out to think, no, no, I

• 1:40:00Speaker 02

  That did not stick.

• 1:40:01Speaker 01

  Yeah. So it was kind of fun. But a lot of the guest people, I mean, maybe this sort of showed the chaos. I mean, a lot of people, I think it was Danny Hillis came. There was all these interesting guests here. So I would say in the union of all of them, I think showed this sort of mass confusion of what was going on. Cause there was a lot of like, oh, should we make computers reversible? Because we have to make sure, can they even be reversible? Can, can this sort of the bottom limit of, of the heat per operation be zero? Or is there some sort of thermodynamic limit? That was like a big deal. I don't think that's, I don't think that's a big deal now.

• 1:40:32Speaker 00

  Yeah. But Wait, so you're talking about the land hour limit, right? Or not, what's on?

• 1:40:36Speaker 01

  Well, at the time, most of me with all this, this question about like, can you have like billiard ball computers and can they be reversible and things? So, and also it was sort of like, what kind of computing? That's why Danny Hillis came like, you know, I think that was in the era of the connection machine and what the original thinking machines, not Mir and Marotti's, but the original one in the eighties. So he came.

• 1:40:56Speaker 00

  What was the state of like general computation in 1982? Like, I mean, I mean, at this point,

• 1:41:02Speaker 01

  To rounding error, we had zero. I mean, we had, I remember when I, sorry, I got there. I was Carver Meads, sysadmin and we had a VAX 11750 that maybe did a MIP, 1 million, 1 million operations. And the whole research group shared an 80 megabyte disk drive. That was the size of a dishwasher. It was, it was very exciting. Sorry.

• 1:41:25Speaker 02

  So what about complexity theory? What was it? Cause I know that there's a lot of interest in quantum complexity and how it relates to gravity right now. And I wonder, I don't have a mind in my mind about why in complexity.

• 1:41:36Speaker 01

  We, we could try to look it up. I don't think there's, of course, the whole, there's a whole hierarchy of, you're right, quantum complexity classes. I think that was developed after that. Cause this was in 1982. In fact, I didn't even sure there was a gate. We never talked about the gate model. There was no gate model of quantum.

• 1:41:52Speaker 00

  You mean quantum gate?

• 1:41:53Speaker 01

  Yeah.

• 1:41:53Speaker 02

  Yeah.

• 1:41:54Speaker 01

  Yeah. Yeah. I forget. I'm sorry.

• 1:41:55Speaker 00

  I think a lot of those, yeah, I don't think those came out until like the 90s or something. Yeah. I mean, I remember reading Nielsen and Chung, the classic quantum.

• 1:42:04Speaker 01

  The information.

• 1:42:05Speaker 00

  Yeah, quantum information textbook, which, which brings up a lot of those points. Now that I think about it, I, that textbook was written in what the late 90s or early 2000s.

• 1:42:14Speaker 01

  Yes, that's right.

• 1:42:15Speaker 00

  And I think that was the first textbook, which like put down kind of the general knowledge of the, the field. But I could be wrong. I guess I think now that when I was young, I don't think it, I took it just for granted that this is, this is the book that everyone.

• 1:42:29Speaker 01

  Yeah. No, but this is, well, you remember in the, in the early eighties, I mean, there was a whole bunch of excitement around neural networks. And it's really interesting because again, we really did not know what we were doing. No one knew what they were doing.

• 1:42:41Speaker 00

  There was like an interesting like neural networks, then SVMs, then neural networks.

• 1:42:44Speaker 01

  Oh, no, no, this is before that. This is in the eighties. Everyone said, everyone said, this is, has the capability of revolutionizing computing. But what does? Well, I mean, there's a tremendous excitement around Hopfield networks. In fact, NeurIPS came out of, because there was a workshop at Snowbird that was nominally private, but everyone tried to crash. And so they, they spotted up NeurIPS. And so that was.

• 1:43:09Speaker 00

  Snowbird is a, is a skiing trip with a, with a computation conference attached.

• 1:43:13Speaker 01

  That, that's right. That's, that's why I learned to ski. Cause I did notice ski and then I kept going. Anyway, that workshop came out of the Santa Barbara workshop in 1985, which came out of some local things at Caltech, which is called HopFests. So, so yeah, people thought, oh, wow, something involved. In fact, ironically, there was like, yeah, yeah, something about associative memory. And if you actually dig down into what transformers are, they are associative memories. Yeah. So in fact, there was even a paper called, you know, Hopfield networks are all you need. So yes. Yeah, I forgot. I'm sorry.

• 1:43:44Speaker 02

  That title is a reference to that paper?

• 1:43:47Speaker 01

  To the era, to the era. So it's actually, it's like, it's like the whole thing has sort of come full circle. And in fact, I think we did, we have collectively as a, as a field, revolutionized computer science. But it was the, the sort of the hopes and dreams completely outstrip the capabilities because again, we effectively had zero compute.

• 1:44:07Speaker 00

  Yeah. I think there's, the, the history of machine learning is different paradigms as like compute versus memory, scaling versus data become available in different levels.

• 1:44:20Speaker 01

  That's right. And the fact that I think people don't realize that the reason why neural networks one is there, the one compute limited thing, although again, with, with now that we have transformers, things are getting memory limited again. But, but, and the fact that they ride on top of BLAS. And so the fact that BLAS was being optimized by things like GPUs. I mean, I don't actually know if, in fact, I'm pretty sure the brains don't work by matrix multiplication. But, but it was just that, that, that the algorithms co-evolved with the, with the hardware. And, and that's, that's why we're here. Who knows? I mean, if we gone down some other path where people really cared about some other compute, who knows what, what architecture we would have ended up with? I don't know.

• 1:45:00Speaker 00

  I mean, didn't a lot of this start with like, I think people were hacking Playstations to train, to train neural networks or something, or to do, I guess, maybe it was even before that for like super.

• 1:45:10Speaker 02

  There's a funny story by a friend of mine from grad school, Brian Cottenzaro at Nvidia, where he, Brian, sorry if I get this wrong. But he came to Nvidia and, and was having a lot of trouble getting traction. And, and he basically, they were, they were doubled and tripled down on, on gaming. And he basically, one day had a meeting with Jensen and convinced him, let's, you know, like this is, we have all these people using CUDA for BLAS basically, and in for deep learning in particular, and convinced Jensen. And he said, it was like a 15 minute conversation and they pivoted the whole company the next day or whatever.

• 1:45:54Speaker 01

  Okay. This, I've never worked in, I mean, I have friends, Dave Kirk, who's the first chief scientist and, and Bill Dallie were both friends of mine from grad school. So yes, but I don't know the details of why. Now, remember that people like in, in Hinton's group, even in the around 2010, they were using GPUs to do the deep learning. But, but even in, as even in, I would say 2007, 2008, they weren't that much faster than CPUs. Again, they, they, they only started really exceeding. In fact, I think this is not a coincidence in the era of, you know, the original ImageNet and some of the speech, some of the speech recognition stuff. So that's, I think that's not a coincidence that they really resurged when, when, when GPUs passed CPUs. So there's, that's not a coincidence either.

• 1:46:38Speaker 02

  Yeah. I really want to know, how does one get the opportunity to name an asteroid?

• 1:46:44Speaker 01

  Oh, well, again, is Caltech, there was, there were some wonderful people, Gene and Carolyn Shoemaker. And they were teaching a class in planetary science. I like planetary science. And so yes, as part of that class, they took us, they, they took us through the sort of asteroid discovery thing. Now this is in the eighties. So now there's all sorts of amazing, amazing systems. Well, there is for a while, something called linear, which automated the thing. But this is before anything was automated. So yeah, it was just part of a class. And what you would do is you'd go to steps, even though we did it out of order in the class, but the steps we used to do, this is 40 years ago, is you would go to Palomar, there would be

• 1:47:26Speaker 01

  a fast telescope, which literally now a museum piece is in their visitor center. But at the time it was a real thing. You would put a piece of film in it. You would take a picture and then you'd wait for a few more minutes and take a same picture of the same point of sky. You put in a stereoscope, like back at Caltech, you would see if anything, you would look around, you would see if anything floated, because it's this, it would move by a tiny bit. So it would pop up and you'd see.

• 1:47:49Speaker 02

  So you was, because it's different in different eyes, then you would be able to see it as something that was projected in a different.

• 1:47:56Speaker 01

  That's right. So you would see, it would pop out at you literally. And then you would go to a measuring microscope and you would take measurements of known star references and where this floater is. And so if you got accurate enough measurement, then you would send it off to a person. I don't think it's him anymore. His name, Brian Marsden, at the Minor Planet Center in, in Arizona. And then he had a big software system to do, to sort of piece together these called apparitions. And if you found, and if you happen to have done the observation, which was the final apparition, which allowed his software to connect it into one big orbit, then you would get discovery rights and you could name the, the asteroid. But now it's like amazing. There's, there's this observatory now in Chile called

• 1:48:39Speaker 01

  the Vera Rubin Observatory. And there's this amazing telescope called the Simone Survey Telescope. Essentially automates this process. Essentially can just take many frames of the sky. It's an utterly stunning instrument. And so they discovered 11,000 asteroids in six weeks.

• 1:48:54Speaker 02

  Oh, wow.

• 1:48:55Speaker 01

  Yeah. So it's now.

• 1:48:56Speaker 00

  How many of them are there? I mean, at least.

• 1:48:58Speaker 01

  Well, it depends. I guess.

• 1:49:02Speaker 01

  Yeah. So, but there are probably millions of asteroids.

• 1:49:04Speaker 00

  So there's still a lot of opportunity.

• 1:49:06Speaker 01

  It's true, but I don't even know if they bother me. So I don't know. Maybe you don't need, I don't know. But people are doing occult, sorry, I can, I'll talk about this forever. There's, there's some, there's stuff called occultation. Like one of my asteroids, I was, I was talking, sending email to amateur. If an asteroid happens to pass in front of a star, it dips, you just like when they discover exoplanets. But if you're super lucky, it'll dip and then it'll dip again, because there's a moon. And so one of my asteroids, this amateur found a little moon around it. So, so that was, I think, last year.

• 1:49:40Speaker 02

  Around the asteroid?

• 1:49:41Speaker 01

  Yeah.

• 1:49:41Speaker 02

  Interesting.

• 1:49:42Speaker 01

  So that's pretty cool. So there's still room for.

• 1:49:44Speaker 00

  Your asteroid has a moon to it.

• 1:49:46Speaker 01

  Yes. In fact, apparently, it's actually, actually, there's a little bit of a story there because it's not really my asteroid that I found two of them. I named one of them after my dad. I waffled. So Carolyn named it after a professor at University of Washington. And I said, oh, but I wanted to name it. And so I whined and she was nice. And so she gave me one of hers. And so then they named that one after my mom. And that's the one that has the moon.

• 1:50:07Speaker 00

  Oh, okay.

• 1:50:09Speaker 01

  So the asteroid named after my mom has a moon.

• 1:50:12Speaker 00

  How big is it?

• 1:50:13Speaker 01

  It's like the main body, they think is about four. Kilometers across and try to get the units right.

• 1:50:20Speaker 00

  Oh.

• 1:50:20Speaker 01

  And then Moon is about one kilometer. So it's actually, it's not, it's a pretty big binary. They're not quite twins, but yeah.

• 1:50:27Speaker 02

  Did other people in their class?

• 1:50:29Speaker 01

  Discovered, uh, I think there was one other person. It's a little bit of a crap shoot. Yeah. And the fact that I found two is quite unusual.

• 1:50:35Speaker 02

  Was there a reason why you, was it just pure luck or was you like just sitting there all day? Do it, try to?

• 1:50:41Speaker 01

  I don't know. I tried to be careful, but yeah.

• 1:50:44Speaker 02

  How does one become, get an Oscar?

• 1:50:47Speaker 00

  Yeah, an Academy Award.

• 1:50:48Speaker 01

  Uh, well, again, maybe I was at the right place. Uh, I, uh, my thesis advisor is named Albar. And, um, I was a student intern actually in 1986 at a place called Chambertier, which is a oil discovery place, but they had an AI lab. And so they were, we were all doing sort of computer graphics research and, uh, we were all thinking, you know, there, at the time, this was revolutionary. I realize it's now considered incredibly boring. Like you could actually use like physics simulators to, to make computer graphics movies. And at the time I was like, wow, that's really cool. So I said, you know, you could use the theory of elasticity to make floppy things. And, and so I said, I was sort of like, well, here's the theory of elasticity. And I wrote a elastic simulator and I made, you know, fabric and, you know, stretchy things and, and stuff.

• 1:51:29Speaker 01

  So they said, oh, wow, that's cool. And so you sort of descendants of that became a lot of the physics simulators that people used in, in, um, you know, Pixar and their various movies. So yes, I tell P, I tell interns sort of half jokingly, well, if you do a really good job as an intern, you can get, you can get an Academy Award.

• 1:51:47Speaker 00

  So, so how like, how long was that between the time you did that work and then?

• 1:51:52Speaker 01

  20 years.

• 1:51:52Speaker 00

  It was 20 years.

• 1:51:53Speaker 01

  Which is actually not atypical, right? Cause they wanna, when they give you an Academy Award, they wanna make sure like, oh yeah, it's sort of well-used and everyone uses it and stuff. So yeah. But, but of course the, obviously like in that 20 years, like, well, everyone does this, obvious, but you know.

• 1:52:05Speaker 00

  This was specific work done for a specific movie or something?

• 1:52:08Speaker 01

  No, it was like a, it was like a paper in CIGRA.

• 1:52:10Speaker 00

  It was, it was a paper and then Pixar, this became like, I think somewhat important core to like a lot of their.

• 1:52:15Speaker 01

  Yes. And in fact, some of my friends, in fact, a lot of my friends did, did a lot of these simulators. So yeah.

• 1:52:21Speaker 00

  I'm curious about, since you have been quantum computing adjacent and worked on quantum computing directly at Google Applied Science as well. I think you not currently on that, but I'm curious to see what?

• 1:52:31Speaker 01

  I'm still dabbling in.

• 1:52:32Speaker 00

  Oh, you're still, you're still dabbling. Yeah.

• 1:52:33Speaker 01

  Quantum.

• 1:52:34Speaker 00

  Yeah. Yeah. So where do you see the trajectory of quantum computing over the, over the years going? Because this is one of those things, which I guess kind of like fusion was sort of had a, a sense at first like being very exciting and then seeming like it wasn't going anywhere for a long time. And then maybe now we're seeing hence again of it being exciting. And I, I'm, or maybe that's like my sort of quantum adjacent.

• 1:52:58Speaker 01

  I think a lot of people have gone through that. I tend to average, I tend to average things out over the, I guess I'm old enough now, right? Sort of average things out over the decades and it's just all progress.

• 1:53:08Speaker 00

  But I mean, like see going from, you know, starting from Feynman, just trying to figure out even what this means as a concept all the way up to now where, you know, I guess there was this recent Willow result of quantum correction, which actually seems genuinely achievable, achievable with like the right scaling laws and stuff. I mean, does this, is this like a, do you still think that, or would you say that quantum is 20 years away? Or even till like some simple but practical algorithm, which actually succeeds? Or I mean, is it, do you see this accelerating or do you think this is still going to be something that's slog? Cause I mean, there's a lot of areas where we, we saw, oh, this is very, you know, this advanced very quickly

• 1:53:51Speaker 00

  and it was very unexpected. And I'm wondering if this is a thing that you think will be quick or will not be? Or this is, I don't know.

• 1:53:59Speaker 01

  I think sort of in between, it's not a purely software thing because we need to build systems that are large enough and stable enough to be able to, you know, be quantum computer instead of a quantum apparatus. We're in there, what they call the NISC era, that was a.

• 1:54:13Speaker 00

  Lazy intermediate.

• 1:54:14Speaker 01

  Medium scale quantum, uh, which was, I think coined by John Preskill. It's not a very good term, sorry. But I guess that's this term we have. Uh, and there are, you know, uh, the quantum team made this wonderful, um, paper, which I think I'm co-author on a web, one of many for something called the quantum echoes algorithm, which could, uh, be applicable now, which fundamentally it's, it's, it's in the style of, of Feynman's, uh, proposal, uh, which essentially it's, well, the technical term is you could try to fit a Hamiltonian to observe data like it in NMR. What that means is, yeah, you have, um, a physical model that's parameterized and you use the, you use the quantum computer to kind of adjust the parameters and try to figure out inside of a loop to what the right,

• 1:55:00Speaker 01

  so you could, for example, decode, uh, NMR parameters. So that is a practical thing that's used now. The question is, is it gonna be, it will be big enough to, uh, make breakthroughs? That's still TBD. The, uh, quantum team at Google is, has been very, uh, executing, uh, again amazingly well against a, uh, roadmap that, that Hartmut Nevin, uh, laid out a few years ago, and they're just, uh, continuing to march down this thing where they're scaling up. And when they hit their last milestone, they should be able to have a, a quantum computer that does amazing things. And they're continuing to march it along. So yeah, it's, it's, I think on the scale of a few to several years, I, I haven't sort of kept up on exactly what date they're saying.

• 1:55:43Speaker 01

  So you should ask Hartmut exactly when that's going to happen. But, um, yeah, they're, they're, so they're marching along. So the, so the main, the interesting question is, will the superconducting, uh, computing be the winner? Or will one of the other sort of alternate technologies? And that's still TBD. I, I still think superconducting is, is a very promising thing because it is, it is very scalable. Um, so yeah, no, I don't think it's 30 years away. I don't think it's tomorrow. I don't think there's going to be, I don't think unless, well, even if there's a sudden hardware breakthrough, these are very finicky things. So it's like someone might have a brilliant idea, but it will still be a while before, because fundamentally, at least in the NISC era or for a while, these are fundamentally analog computers. And so that, they tend to be very, very, very finicky. So I wouldn't expect, um, all of a sudden, you know,

• 1:56:27Speaker 01

  something, some, some phase change happens.

• 1:56:30Speaker 00

  No one has figured out how to scale up qubits in a way that interact in just to kind of arbitrary size. We're still in the order of a hundred, 200 qubits, I think, or?

• 1:56:40Speaker 01

  You mean in terms of?

• 1:56:41Speaker 00

  Okay, so in terms of.

• 1:56:43Speaker 01

  Well, it's a little bit like, yeah, yeah, exactly. So for superconducting qubits, um, yeah, people, I mean, although people are working on it, the ratio of the number of physical qubits to logical qubits is still relatively large for the error rates that you need. And so maybe there'll be a breakthrough there. I don't know. Or there are other things that might have, uh, you don't need that ratio to be so high because a lot of it has to do with the, the 2D connectivity of the, of the, of the chips that you lay out your qubits out of 2D chips. And for things like, um, neutral atoms, they in theory, in theory can connect anything to anything else. But of course, in practice, we don't really know. And we don't actually know what the limitations of neutral. At least I don't know the limitations of what neutral, maybe, maybe the super experts know.

• 1:57:27Speaker 00

  Superconducting qubits in particular have a sort of tension where you, you want to have, you know, nice clean resonators, which, you know, you do get a clean resonator by decoupling from the environment. And then you get good interactions by coupling resonators, which involves covering to the environment. So.

• 1:57:43Speaker 01

  It's true, but at least there, but there, yes. But there's like the whole environment.

• 1:57:48Speaker 00

  Yeah.

• 1:57:48Speaker 01

  Uh, and then there's like little tiny peoples that you, that you wanna go through to your neighbors.

• 1:57:52Speaker 00

  This is not like a fundamental uncertainty relationship or something.

• 1:57:55Speaker 01

  That's right.

• 1:57:56Speaker 00

  It's like, this is a technological limitation that difficulty or something.

• 1:57:59Speaker 01

  Yeah. The hardware team and, and, and Google Quantum is, is very, very skilled. They're very, very skilled. So, um, uh, yeah, they're, they're really good at, at making these designs and making these things actually work. So I, I, I find them impressive.

• 1:58:13Speaker 00

  Yeah. It'd be exciting to see that advance.

• 1:58:15Speaker 01

  Yeah.

• 1:58:16Speaker 00

  Yeah.

• 1:58:16Speaker 01

  Yeah.

• 1:58:17Speaker 00

  I guess I'll just have to hold a breath and wait.

• 1:58:19Speaker 01

  Just wait. Yeah. I guess I'm just very patient. So I've started working.

• 1:58:23Speaker 00

  You said 20 years ahead of.

• 1:58:25Speaker 01

  That's what Dave Bacon, who runs the, the, the software team in Google Quantum always, he, he teases me like, John, like, oh no, I can't work in quantum computing. This was like 10 years ago because it's gonna be, you're always 20 years ahead of time. And so I'm gonna have to wait for 20 years. Like, okay. Well, you know, 10 years have gone by. It's like.

• 1:58:42Speaker 00

  Halfway there.

• 1:58:43Speaker 01

  Halfway there. I don't, I don't take that as a, an absolute.

• 1:58:46Speaker 00

  There's a hard rule.

• 1:58:46Speaker 01

  Yeah. But, uh, I also started working on fusion 10 years ago. We'll see.

• 1:58:53Speaker 00

  Maybe you're actually causal. Like you start working on something and the reality just, you know.

• 1:58:57Speaker 01

  Catches up.

• 1:58:58Speaker 00

  Catches up.

• 1:58:58Speaker 01

  I guess so. Maybe. Who knows? I don't know. I, I started working, I loved convolutional nets in the early 90s. And Jan LeCun claimed.

• 1:59:06Speaker 02

  That's out.

• 1:59:07Speaker 01

  I coined the term convolutional net. And as far as I can tell, that, that may be true. I, cause everyone called it Linet, because it was a very specific thing. They all worked for Jan. I said, well, I don't work for Jan. I don't wanna call it Linet. So I called it a convolutional net. So I don't know. That was a more generic term.

• 1:59:20Speaker 00

  It's a good term.

• 1:59:21Speaker 01

  Yeah.

• 1:59:22Speaker 00

  Before you go, is there anything you want the audience to take away? Any messages you want to deliver?

• 1:59:28Speaker 01

  I think era is an example of it, but I, I'm really amazingly excited about the potential of AI for science. I think it's, I think it's going to be an amazing power tool for scientists. And I think scientists won't be replaced. I think it could, in fact, I'm hoping that they'll spend all their time on, again, the creative stuff, on the rigorous stuff, on the philosophy stuff. And so I, I think it's good. I think it's gonna be way cool.

• 1:59:51Speaker 00

  I, I hope so too.

• 1:59:52Speaker 02

  Yeah. No, I, I, yeah. My intuition is that a lot of, a lot of, I've heard a lot of people say that, that I hope they're right. And I hope it's not just, uh, sort of coping with a reality that's uncomfortable. The other question we almost forgot to ask, if you could remove a bottleneck in your industry, which, um, whatever you, however you wanted to find that by fiat, then?

• 2:00:18Speaker 01

  Like magic.

• 2:00:19Speaker 02

  Yeah. By, by magic. What would that be?

• 2:00:21Speaker 01

  If I could get a magic wish, I would say, someone please make the, the everything lab that you could like send JSON blob to, and it will do any experiment at all.

• 2:00:31Speaker 02

  Okay. Automated.

• 2:00:32Speaker 01

  But it has to be anything. So essentially, so I guess we have to solve the sort of AI complete robotics problem, I guess. But if we did, then that would be stunning. Cause right now, things like era, it's all computational. So someone has to gather the data. So yeah, if we could just break that, oh, oh, that would be so amazing. That would be so utterly amazing.

• 2:00:51Speaker 02

  Yeah. Um, great. So I really appreciate you taking the time to, to, to see us. Um, and I think you, you like kind of flew in and adjusted your schedule a little bit to.

• 2:01:03Speaker 01

  Yeah. I was, I was sort of flying over San Francisco to get home. And so I, that's that I landed in San Francisco.

• 2:01:08Speaker 02

  So you, you made a big effort to be here. We really appreciate that. It was really fun to talk to you.

• 2:01:13Speaker 00

  Yeah. It's been a blast.

• 2:01:14Speaker 01

  Okay. Cool. Thank you for having me.

• 2:01:16Speaker 02

  Yeah, you're welcome. Thank you.