How AI is changing weather forecasting

Weather forecasting drives billions of economic decisions — from grid operations to evacuation planning. Better forecasting could improve supply chain planning, disaster warnings, and renewable integration. The industry has decades of satellite observations and ground measurements, making it ripe for AI-driven advancements. And it’s already happening. But how exactly does AI get used in weather forecasting, and how does it actually lead to improvements? In this episode, Shayle talks to Peter Battaglia, senior director of research at Google DeepMind’s sustainability program, which launched a new AI-powered weather forecasting model in November 2025. They cover topics like: Why precipitation is so much harder to predict than temperature How the weather industry works, with governments creating global models and private companies refining them for specific use cases What AI models can see that traditional supercomputer simulations can’t Novel sources of data like cell phones, door bells, and social media Resources: Latitude Media: Where are we on using AI to predict the weather? Latitude Media: Could AI-fueled weather forecasts boost renewable energy production? Catalyst: Specialized AI brains for physical industry Credits: Hosted by Shayle Kann. Produced and edited by Daniel Woldorff. Original music and engineering by Sean Marquand. Stephen Lacey is our executive editor. Catalyst is brought to you by Uplight. Uplight activates energy customers and their connected devices to generate, shift, and save energy—improving grid resilience and energy affordability while accelerating decarbonization. Learn how Uplight is helping utilities unlock flexible load at scale at uplight.com. Catalyst is brought to you by Antenna Group, the public relations and strategic marketing agency of choice for climate, energy, and infrastructure leaders. If you're a startup, investor, or global corporation that's looking to tell your climate story, demonstrate your impact, or accelerate your growth, Antenna Group's team of industry insiders is ready to help. Learn more at antennagroup.com.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Speaker 00

  Latitude Media, covering the new frontiers of the energy transition.

• 0:07Shayle Kann?

  I'm Shayle Khan, and this is Catalyst.

• 0:11Speaker 02

  We don't really understand how the AI models forecast it, but they are capable of treating the hurricane as almost like a large macroscopic scale object that is moving.

• 0:22Shayle Kann?

  They have like spatial awareness in a way that the old models didn't.

• 0:25Speaker 02

  Yeah, that's it's a really interesting area, I would say, of like sort of the science of how AI works to understand exactly how they see the world in that sense.

• 0:35Shayle Kann?

  Coming up, where the winds are blowing using AI for weather forecasting.

• 0:49Speaker 00

  Demand flexibility is now an essential part of the data center's speed to power playbook. But how do data centers actually curtail load? Grid Beyond is the end-to-end solution for flexible energy management. Grid Beyond's advanced DERS and VPP platform is tailored to what data centers care about most: maximizing reliability, intelligently meeting curtailment obligations, reducing costs, and managing all your distributed resources in one place. Ready to transform your distributed energy management? Get started at gridbeyond.com. Catalyst is brought to you by Antenna Group, the communications and marketing partner behind the most influential companies in energy, climate, and infrastructure. For three decades, Antenna has helped ambitious companies stand out,

• 1:32Speaker 00

  build market leadership, and become category-defining brands. From breakthrough startups to global industry leaders, their team brings the expertise, relationships, and track record to accelerate your momentum. Learn more at antenna group.com.

• 1:47Speaker 03

  The power plant for the modern grid isn't a building. It's a network. Energy Hub's DERS platform connects batteries, EVs, thermostats, and commercial and industrial sites into virtual power plants that over 230 utilities dispatch as reliable grid capacity day in, day out. VPPs deploy in 60 to 100 days at 40 to 60% less cost than traditional infrastructure, and they perform like conventional generation from day one. Learn about the flexibility platform for the modern grid at energyhub.com.

• 2:22Shayle Kann?

  I'm Shayle Khan. I lead the early-stage investing practice at Energy Impact Partners. Welcome. All right, so here's a statement that I suspect would be pretty non-controversial. AI will improve weather forecasting. It's obvious, right? And it seems like it must be true. I certainly would have agreed with that statement had you asked me before this conversation you're about to listen to. But to me, the interesting question is, why exactly? Like, through what mechanism can AI improve weather forecasting? For that matter, how do we actually do weather forecasting today? And if it does get better, what are some of the likely outcomes that it will enable? It's an interesting set of questions for me for two reasons. First, weather forecasting itself is important to a whole host of other

• 3:05Shayle Kann?

  categories I care about. Obviously, resilience, but also energy and a variety of others, agriculture, et cetera. But also, it's interesting because I think it's exemplary of a whole host of next-wave applications for AI. LMS are, of course, finding their way through everything that requires language. Now, there are world models starting to show up to try to revolutionize robotics and things in the physical world. But what about things like weather, where we have used some machine learning historically, but can we do better with transformers and the new architecture of AI that we're seeing in other categories? Let's find out. My guest today is Peter Pataglia. He's a senior director at Google DeepMind,

• 3:46Shayle Kann?

  where he is leveraging the big brain inside the DeepMind to improve weather forecasting. Here's Peter. Peter, welcome.

• 3:53Speaker 02

  Thanks. Happy to be here.

• 3:54Shayle Kann?

  All right, let's talk weather forecasting. I want to start maybe by having you school me a bit on something that I realize I don't know, which is, how do we do weather forecasting? Like, currently, and maybe a little bit of history, like, have there been major shifts technologically in how we forecast weather historically? So maybe walk me through the history, such as it is, of how we forecast weather, and then, like, what do we actually do today?

• 4:20Speaker 02

  Yeah, so I have to admit, I'm actually a relative newcomer to the area of weather forecasting myself. So, we had gotten involved in this several years ago, and it was sort of built out of a research program that was trying to model complex simulations, including fluids. And the Earth's atmosphere is a fluid, and one of the big challenges that we were sort of interested in exploring was... Modeling the atmospheric fluid, which is weather forecasting. So I should say that I sort of have gone through this journey of learning about weather forecasting. So the stuff that I'll say, hopefully it's accurate, but forgive me if I make mistakes.

• 5:03Speaker 02

  So I think my understanding about the field is that really a lot of the, historically, weather forecasting was very important for agriculture and sort of other use cases that were very important for kind of day-to-day life. But I think it was about maybe 100, 150 years ago that you had agencies or bureaus that were starting to do like marine forecasting or like kind of more systematically, collecting observations systematically and treating it as a science. But then probably about 50 years ago or so, you started to see the emergence of like large government public weather agencies. So I think NOAA in the US was formed in the 70s. I think ECMWF, the European Center for Medium Range Forecasting, was also formed in the 70s.

• 5:46Speaker 02

  And these are two of the big prominent weather bureaus. But most governments have a weather bureau and they, it's sort of, weather has traditionally been viewed as a public good. So this is something that, you know, they collect tax money and then fund their weather service. And the idea is that a lot of the, you know, it's not only is forecasting the weather useful, just again, like what are you gonna, you know, you're gonna wear an, have an umbrella or wear a coat, but for things that are more like, you know, there's a dangerous storm coming, flood, extreme heat, extreme cold, those types of things. And also a lot of sort of decision-making, like agriculture, energy, transportation. And I think in general, weather forecasting has traditionally been understood to be something that's a very good investment on a dollar.

• 6:31Speaker 02

  So the public tax money that's invested in the, in the national weather bureaus has significant economic returns on those investments. So that's kind of, I think, my understanding of the history of the kind of standardized or official weather forecasting business in a sense.

• 6:51Speaker 02

  Maybe one more thing I can say about the way to think about the industry. I think my understanding is you can kind of think about the weather forecasting industry is divided into, it's almost like a pipeline, really. There's this sort of government official weather bureaus that are issuing these like global forecasts that are predicting all sorts of weather variables, but typically more at like a coarser spatial resolution. And then you have this full, this like big post-processing chain where they take the like sort of base forecast and then they specialize them for different use cases. So for example, like when you look at the app on your phone and you see, you know, the chance of precipitation, that's not coming directly from NOAA, that's coming from other

• 7:35Speaker 02

  intermediaries that are like taking, you know, local weather station data and other historical information, trying to kind of like tweak it and improve it and make it especially useful for your use case. And you see that sort of an energy and all sorts of other applications of weather forecasting.

• 7:51Shayle Kann?

  Is there a corollary to how that has worked historically to like what's, what's happening with LLMs today? Not to jump into the AI stuff too early, but just in the sense of like, is the global forecast, so let's say NOAA's forecast, is that a big mega model that spits out this one big forecast?

• 8:09Speaker 02

  Yeah.

• 8:09Shayle Kann?

  And then what people are doing in the post-processing world is saying, okay, I'm going to take that model, but then I'm going to like, fork it is the wrong word, but I'm going to, I'm going to fork it and add a bunch of additional data into it to try to make it better at a smaller spatial resolution. Like, I'm just trying to picture what it actually is.

• 8:25Speaker 02

  Yeah. So I mean, I could, I didn't, I didn't say much about that, where, where your actual weather forecast comes from in terms of like technically. So maybe if I say that, then maybe it will sort of open the answer to that question. So again, fluids, like the atmospheres of fluid, and in physics, we have fluid equations called the Navier-Stokes equations. And that's, they govern fluids like at all scales, like from the largest scale structure of the universe, which actually turns, turns out to also be a fluid, down to like what's happening in, you know, in your blood, basically. It's, there's turbulence that determines sort of how your blood flows and has kind of important implications. Now... All things in between that, right? You have like weather and you know, stream flow and

• 9:07Speaker 02

  other types of things like that, all fluids. And what happens, like engineers have figured out that, well, so this, the fluids are very complicated to simulate. So in order to simulate them accurately, they need to approximate the solutions and so they can run them on very large computers. Because they're, they're so complicated, they would never run on a computer natively. You have to sort of break up the computation and approximate certain things in order to actually, you know, model everything that's happening, for example, in the atmosphere. So that's called numerical weather prediction. The numerical is just saying that they're making a numerical approximation to these Navier-Stokes equations.

• 9:48Speaker 02

  And traditionally, it's been run on supercomputers. So like, a lot of the big supercomputer centers have either, you know, do a lot of weather forecasting, or even built to do weather forecasting. And in many ways, it's been sort of a triumph of science and engineering that we've been able to like, decade on decade, predict not, you know, one day, two days, but like 10, 12, 15 days into the future. It's hard to even sort of imagine, like the scale of that type of predictive accuracy was sort of unimaginable 100 years ago. People just didn't think like, in two weeks, we can kind of know what the weather's gonna be? That's crazy. That relies on knowing like, what's happening on the other side of the earth as the sort of prevailing winds carry the, you know, moisture and the temperature and all that kind of stuff. So.

• 10:33Shayle Kann?

  And I imagine there's like an exponential increase in complexity the further out into the future you get, just because like, the, there are a variety of possibilities of what actually happens today. And each one of those needs to be taken into account when I'm trying to predict what's going to happen tomorrow, and so on and so forth as you move into the future.

• 10:51Speaker 02

  That's, that's the butterfly effect, right? It's that a butterfly might or might not flap its wings, and then that will determine like, a week later, whether there's a hurricane or not. Right? So the idea is that little tiny

• 11:04Speaker 02

  Changes or little tiny effects or lax, you know, missing effects will cause, could cause huge changes in weather over time. And that's exactly right. So the fluid, the atmosphere of fluid is thought to be chaotic, which means that that's, that's sort of the definition of chaoticity. It's that little tiny changes can have huge, large impacts later. That's what makes it so hard. And there's coupling across scales. Again, the butterfly flapped its wings, but then you have like up in the top of the atmosphere, stuff is happening. Now, that's, that is exactly why it's very, very difficult to sort of, you know, find solutions to the exact equations that govern the atmospheric fluid.

• 11:46Speaker 02

  It's, so we have to make approximations and we use supercomputers and we have all kinds of tricks. I should also say like another important thing to recognize is that the, when you generate, when a weather forecast has been generated, the actual prediction of the future is only half of the process. That's the, it's the second half. The first half of the process is figuring out what the weather currently is. So if you, we have satellites and we have weather stations and balloons and ships and all sorts of information that are taking measurements of like what the weather is all over the earth. But again, using the butterfly as an example, you would have to know like where every butterfly

• 12:28Speaker 02

  is in principle to act to perfectly forecast the weather. So if you sort of think that through, you realize that it's not really, we're, we're, weather forecasting, weather forecasting is always going to be fundamentally uncertain to some level. We're never going to be able to make perfect observations of the weather everywhere on earth with the precision required to perfectly predict the weather a week out. And so when weather forecasting, that's why you have a chance of rain versus like it's definitely going to rain, right? And you have like a range of temperatures, especially as you go out in time.

• 13:05Speaker 02

  And that's again, sort of what makes weather forecasting so hard. So the first step in weather forecasting isn't actually predicting, it's taking all the satellite data and all the stations and all the different observations and estimating the current state of the weather across the earth. And once we have that estimate, then we can make the prediction with the supercomputer. And that's that that second part is what our team and a lot of the teams in the field who are working on AI-based weather forecasting have been especially focused on. But my guess is that over time, we're going to see other parts of the weather forecasting process being, you know, having having more and more AI methods that are coming in and trying to advance them.

• 13:46Shayle Kann?

  Before we get into the AI methods, I it seems like we have generally, even pre-AI, we've been getting, I mean, you tell me if the curve has been linear or exponential or flat, but like, it seems like there's been, I don't know, fairly linear improvement in our weather forecasting ability for for decades. Like, we're getting more precise. We are also getting better predicting further out into the future, as you said, like, you know, a week, two weeks, et cetera. To the extent that that's true, I'm sure it's all these things, but like, how much of the improvement that we have seen historically has come from, I don't know, A, as you said, just like having better ground truth data on the current

• 14:28Shayle Kann?

  state of the weather, B, more compute, as you said, has been running in supercomputers. We get power more and more powerful computers, we can just run more and more complicated Navier-Stokes equations, or C, additional tricks, basically, that allow you to like, do better predictions without adding more compute.

• 14:48Speaker 02

  Yeah, that's a that's a great question. So I'll just admit, I don't know the answer to that. I think all three contribute. So I can say, on the first one, data, yeah, we like, have, there's, you know, better satellites that are flown and there's more better systems for, you know, collecting balloon observations or these different sort of things. So we definitely are getting better data and we know that that improves the quality of the forecast. We are also getting better models. That's definitely true as well. We're the, we're getting, you know, we're building bigger supercomputers. They can operate at finer resolution.

• 15:22Speaker 02

  Just, I think, in the last less than 10 years, the ECMWF, which has the best weather forecast, they increased the resolution, meaning that they had finer detail and space in their forecasts, and that allowed the forecast to be more accurate. So you see both like adding just raw compute power, but also improving the quality of the models and the approximations can also, you know, has also made, I think, a pretty dramatic impact. And I think that sort of blurs into your third category of like other tricks. I think in general, you have, you know, like, without getting into the details of how the numerical models work, you can kind of think about them as a backbone that's making a

• 16:05Speaker 02

  sort of general prediction at a coarse scale, and then you have a lot of other parameterizations and trickery sort of under the hood in the making finer and finer grain predictions and also updating the backbone to be consistent with the fine grain. And those are all being advanced sort of in parallel, and like the teams, these, you know, engineering teams and sciences, scientific teams are sort of working together to make these better. The last thing I would also say too is, again, going back to that like post-processing part of the weather forecasting pipeline, it's not just, again, that like these large, you know, NOAA and ECMWF and these other large agencies, it's not just that they are improving the forecast, it's that other parts downstream in post-processing are improving what they're doing.

• 16:46Speaker 02

  So actually, the first advent of like AI and machine learning in weather forecasting, or at least some of the earliest, was not like trying to overhaul the whole weather forecast process itself, but making, you know, using more and more like statistical methods and linear regression and nonlinear regression, neural networks and other types of earlier machine learning techniques to improve not the base forecast, but like the specific application, so maybe we can calibrate your, you know, chance of rain better if we have a slightly better downstream model.

• 17:21Speaker 00

  For data centers, connecting to the grid quickly means two things: building distributed energy and having a plan for demand flexibility. GridBeyond helps with both. GridBeyond's DERMs and VPP platform unlocks the potential of distributed assets and dispatches flexibility into the right market at the right time. And this is more than software. GridBeyond supports developers every step of the way. From modeling your highest ROI energy mix to orchestrating the full set of assets around a data center. The track record speaks for itself. GridBeyond manages over 1,400 sites and more than 3.7 gigawatts of load across Europe, the UK, and the US. Learn more at gridbeyond.com or click the link in the show notes. If you work in energy or climate,

• 18:05Speaker 00

  you already know the company's Antenna Group has helped build. For three decades, they've been the communications and marketing partner behind the industry's breakout companies, category creators, and market leaders. Today, their team is working across the most consequential areas of the energy economy, from data centers and grid modernization to nuclear, geothermal, storage, critical materials, and next-generation technologies. Antenna helps ambitious companies become the names everyone in the market knows, trusts, and wants to work with. They build reputation, elevate executives, drive demand, and turn momentum into market leadership. If your company is ready to lead its market, Antenna is ready to help you get there. Visit antennagroup.com.

• 18:47Speaker 03

  When you harness the power of millions of DERs in a VPP and put artificial intelligence behind the dispatch, the outcomes exceed conventional generation. Energy Hub's DERMs platform has been trained on years of real dispatch data from over 2.6 million DERs, so utilities can plan with confidence. That means consistent load reduction across a four-hour event window, even during the widest, most uncertain peak periods. Utilities target, shape, and verify capacity down to the meter. Unlock flexible capacity that costs 40% to 60% less to deploy than traditional infrastructure in a fraction of the time it takes you to break ground on a peaker plant. Learn more at energyhub.com.

• 19:31Shayle Kann?

  So, okay, so we've been improving, we've been, you know, in more recent years, applying sort of the earlier versions of ML to continually improve. I guess I'm curious to your perspective, what the biggest gaps are. I mean, like, obviously, we don't, we don't have the ability today to generate a perfect forecast three months into the future. Like, it could always get better. But apart from just that element of it, are there any areas where you feel like, actually, there's like a real, it's really hard to do X? Is it like precipitation is a bugaboo? Or, you know, something else, right?

• 20:04Speaker 02

  Yeah. I mean, I would, I think there's sort of two ways to answer that. It's, you're always going to be limited by the quality of your data. So if you don't have good data about something, it's, it's gonna, you know, you're just, it's bad, you know, garbage in, garbage out, sort of thing, right? So these models take an estimate of the state of the current weather and then predict what's gonna happen. If your estimate isn't very good, because your raw observations weren't very good, you're not gonna get a very good forecast. So improving, just collecting more data and using the data you have collected to form a better estimate of the current weather, that's definitely gonna always improve things. So that's, it's sort of a known gap, right? Now, we don't know exactly what the ceiling is. We don't know, like, if we've, you know, if we do this satellite or that station observation, how's it gonna improve things?

• 20:49Speaker 02

  We might have an idea, but we don't always know, and sometimes we have to just test it out. But the, yeah, the, the other thing I would say is that you, you have different features of weather, which are harder or easier to predict. So an obvious one is temperature. So temperature is sort of very smoothly. Like, if you look at a map of the temperature across the earth, it's sort of, you know, it's not, you know, up a mountain, it's gonna be colder, and like in a valley, it'll be, you know, different. But it's sort of very smoothly. What doesn't vary smoothly is precipitation. So like a rap, you know, a violent thunderstorm that sort of emerges out of nowhere, and there's like high wind and low pressure and all this kind of stuff.

• 21:31Speaker 02

  That, the, like, where exactly that front will be, like, where exactly the precipitation will happen, what exactly the wind and these kinds of things are much, much harder, because the, the detail of, like, it's, everything's happening at a finer scale. Like, where, you know, even if you look at a radar map, you can see this. It's not like... Precipitation sort of varies smoothly over the earth. You see like there's a little, you know, thunderstorm right there or like there's a, you know, rain and it's and then, you know, a few miles over, nothing. So that type of very high resolution, complex, you know, patterns of precipitation, for example, wind as well, those are much harder to predict because you're effectively predicting a lot more information.

• 22:13Speaker 02

  You can't just sort of summarize it by saying, oh, every, you know, 25 kilometers or whatever, the temperature is this and then everything else is just kind of interpolated in between. You have a lot of stuff happening at a much finer scale, finer scale than a lot of our models even capture. And then we have to do secondary steps to try to resolve those finer details.

• 22:34Shayle Kann?

  All right, so let's talk about AI then. I mean, you mentioned, this is one thing, right? Whenever we talk about AI, quote, quote, fingers applied here, right? Like there's the, there is ML as a subset of AI, a related entity. We've been doing ML already. So I guess the first question that I have is, as you think about leveraging AI now and into the future for weather forecasting, what version of AI are we talking about? Like what version or versions are you actually, what is the, what are the actual capabilities and or model structures that are interesting here?

• 23:12Speaker 02

  Yeah, that's a good question. So yeah, these days, I mean, AI is a pretty catchall term. I find myself just using the word AI just to mean a lot of different things because I think it's kind of easier.

• 23:24Shayle Kann?

  Right.

• 23:25Speaker 02

  Usually people kind of know. The difference, the way, but the way I would say it is the difference between AI and, so machine learning is sort of the like statistical inner core of AI. It's trying to capture, it's taking data and trying to capture the patterns through a training process and then, you know, kind of use some inductive assumption like, oh, what we've seen in the past is going to be similar to what we see in the future.

• 23:53Speaker 02

  Modern AI, I think, is a broader family of things. It sort of involves like agents and your interactions with them and a lot of like language models are often sort of associated with AI. What we use in our weather forecasting models and a lot of folks out in the community are using is, you know, this field is advancing and this AI-based weather forecasting is developing. We're still mostly using fairly traditional machine learning, supervised learning. So supervised learning just means you take a data set that has a pair of examples, an input example and a target example, and you train a model to try to take input examples and accurately predict the target examples.

• 24:36Speaker 02

  And so if you think about weather as, you know, again, like I said before, you're estimating the current state of the weather and then the next step is to predict what's going to happen next, right? That's, it can be, a supervised learning method can be trained to do that and that's what we're doing in our models and like most folks at IC are doing as well. And then the only extra step is that just makes one prediction, but then we feed the output of the model back into itself and then we have it make another prediction. So the output becomes the input and then it generates another output. And if you just sort of chain those steps together, you get your first input and then you get a sequence of outputs that represent future steps in time. And so supervised learning, we use a lot of, these days with, in terms of AI architectures,

• 25:21Speaker 02

  transformers and graphical networks are what we use. People use convolutional neural networks, but I don't feel that neural network architectures these days tend to be the sort of exciting part. It's usually more of like the training and the sort of data, how you handle the data and that kind of thing.

• 25:37Shayle Kann?

  So, but you, you mentioned transformers, because I guess like if we had been having this conversation.

• 25:43Shayle Kann?

  Five years ago, right? I imagine that you still would have told me about supervised learning, for example, right? Like, that wasn't, that's not new. Transformers had been invented by that point, but like, had not been, you know, broadly applied the way that they are today. So what is it that, like, this new wave of AI unlocked by things like, by, by transformers and convolutional neural networks and so on? Like, what does that enable above and beyond what you would have been able to do five years ago?

• 26:13Speaker 02

  Yeah, that's a good question. So the way I look at it is, so transformers are very similar to graph neural networks. They, both of them are, so actually, let's, let's, let's take this back. So we used to use often convolutional neural networks. And the idea here is it learns a little function that's sort of local in an image. And then it sort of applies that same function everywhere. And then you stack up sequences of these layers. And that eventually lets you, like, one, the information on one side of an image communicate with the information on the other side of the image because the, it's a hierarchy. A transformer architecture allows you to make a direct connection between the information

• 26:55Speaker 02

  on one side of the image and the other side of the image, the same way that graph neural network does. And I like to think about it like graph neural networks because what it's like saying is, well, in a graph, you have nodes and you have edges or connections between the nodes. And a longer connection between nodes is for nodes that are farther away and shorter connections are for nodes that are closer. So if you use the graph neural network analogy to describe the older convolutional networks, it's like the graphs are all small. They're all kind of, everything's kind of close. It's like nearby in an image. Graph neural networks allow you to choose how far away you want information to interact. And in transformers, it's, it can be understood as a graph that has connectivity across any spatial scale.

• 27:39Speaker 02

  So in language models, the way a transformer works is, it says, when I want to make predictions about the next word, I want to be informed by the most recent word, but also every word that has happened in the text before. And that's important because in language, the next word is not predicted by just the previous word. It actually is predicted by stuff that happened earlier in the sentence, or in the paragraph, or in the, you know, book. And so the ability to make, to have information across large spatial scales that interact with one another, that ability allows you to, it opens sort of new patterns of computation and allows you to represent functions that

• 28:20Speaker 02

  have traditionally been harder to represent, but it allows you to make, you know, better predictions of the next word or the next, you know, in our case, you can kind of think about as the next spatial point or a faraway spatial point. And that allows the model to be more flexible and capture richer functions.

• 28:36Shayle Kann?

  There's a good comparison there. What do you think of as being, I guess, you just described something that is similar about what you can do in weather forecasting, thanks to a transformer architecture, to what you can do with large language models, which is what most people are going to be most familiar with in the, in the new wave of AI. What's different?

• 28:55Speaker 02

  So, that's a great question. I think what's, what's interesting is that the way that, so in, in language, the text is understood to be, is treated as a sequence. It's like, you know, token, token, token.

• 29:10Speaker 02

  We are also modeling sequences in weather, but we're not allowing our models to look too far back in time. So, it, because weather, weather is actually different from text in a fundamental way. In fact, most physical processes are. They are what's called Markov in that the most recent state of the system determines the subsequent state. So like I said, in text, that's not the case. You know, right now, I'll just pause. You didn't know what word I was going to say next, right? It kind of depends on the context, a bunch of words behind it or earlier. With weather forecasting in principle, if you know exactly what's happening right now, you can fully predict what's going to happen next. You don't need to look further back in the past.

• 29:55Speaker 02

  So, we actually use transformers not to model the spatial, the interactions in weather over time, like the sequence of text, but in space. So in text, you actually don't have a sense of spatial structure, right? You just have one sequence of text. It's this word, word, word. When you read it, you just see word, word, word. In weather, you have spatial structure. You have weather all over the earth at the same time. And it's all, you know, especially the close weather, it sort of determines and can be used to predict what's happening next at, you know, our current location. And so we use transformers and graphical networks to capture the short and long range

• 30:39Speaker 02

  spatial dependencies. And those interactions between, you know, what's nearby and what's about to happen next are what determine weather and that's how we sort of make these predictions. But one thing I should also add is that, similar to how I was saying earlier, that like it's kind of impossible to measure, like everything that's happening on the earth in the fine detail in weather, you have to make approximations. These models do too. And this actually brings us to a very fundamental difference between how AI models are making their predictions and how traditional models are. So AI models can take the statistical structure of weather patterns. So for example, if I'm looking at a, you know, hurricane that's traveling over the

• 31:23Speaker 02

  earth, right? In a traditional model, the way it simulates that is it, in very fine detail, it kind of figures out like, what's the pressure and the temperature and the wind and the moisture and what are those things? What's going to happen next is determined strictly locally. AI models, because they can look at a much larger spatial range, they can use what traditional methods use, or they can use other approaches. Because you know, when you look at a hurricane, it almost looks like an object sliding over a globe, right? That's not how a traditional model models it. And we don't really understand how the AI models forecast it, but they are capable of treating the hurricane as almost like a large macroscopic scale object that is moving.

• 32:08Speaker 02

  Because they can see all the structure of the hurricane and they can see, you know, sort of what's happening in the recent past.

• 32:15Shayle Kann?

  They have like spatial awareness in a way that the old models didn't.

• 32:18Speaker 02

  Yeah. And we don't know. That's, it's a really interesting area, I would say, of like sort of the science of how AI works, to understand exactly how they see the world in that sense.

• 32:29Shayle Kann?

  It seems like to me, on one hand, a much harder problem than an LLM, because you've got the entire physical world and the data is sparse, as you said, and there's lots of complex interactions. On the other hand, it's determinative in a way that LLMs are not, right? Like, there is no correct next word necessarily, right? There's like a, it's a, it's a best guess as to what the best next word should be. But in the case of weather forecasting, there is a correct prediction to make. And there is a universe of historical data that you can draw upon to do it. And so I'm, I'm back and forth on whether this is a harder problem or an easier problem than like making a really, really good LLM.

• 33:11Speaker 02

  I think, yeah, I think you'd have, I think, depending on, you know, what, uh, what, where your allegiances lie, that'll probably be, I think it's probably more of like a, an opinion question, but yeah, I think you're, you're absolutely right. So the, um, the way I think, so it's the one thing I just would like, uh, I might say it a different way is that it's still, like I said before, it still is fundamentally uncertain from the standpoint of the information that's available to the model. Now, yes, like the physics is truly deterministic underneath, but because the model doesn't again, see the butterflies or see the little fine scale stuff from its perspective, it actually is a random process, right?

• 33:51Speaker 02

  Because it doesn't, if it doesn't know whether the butterfly flapped its wings or not, how could it know whether the hurricane's going to form or not? Right? Now, so from the perspective of the information available to the model, it is also an uncertain random process to some extent. But I, yeah, I think what you're saying is exactly right. So I think the underlying structure of text is random in different ways. It's, it's like, again, I can, I'm going to pause and then I'm going to say a, a word.

• 34:20Speaker 02

  Furniture.

• 34:21Shayle Kann?

  Right.

• 34:21Speaker 02

  Like, weather doesn't work like that, right? Like, it doesn't, I, it doesn't just have something pop into its head that's completely different than what's historic, like present in the historical record, like you pointed out. So I think that the structure of the uncertainty is different in weather. It's, it's more constrained in a way. So in text, it's, you can imagine, you know, if you're watching, same thing with like video, you're watching a movie and someone's going to come through the door, you have no idea what they're going to be wearing, uh, as a shirt, right? It could be wearing blue, red, anything. There's like no way to predict it and whether you can always have some idea, you just don't know the fine details. On the flip side, weather is an extremely complicated process. This fluid, chaotic fluid system is, has, you know, interactions from small scales to

• 35:06Speaker 02

  large scales and it's happening all over the earth at once. So in some sense, instead of just predicting the next word, you're predicting like millions of variables at once. So I think you can kind of like, it's probably better like, you know, to have this as like a debate over beers with your friends in your, in their L L M lab rather than like something that can be adjudicated just on the basis of these things.

• 35:26Shayle Kann?

  I guess there's one other question on this sort of comparison to L L M world. Notoriously, like the, the big L L Ms are trained on the internet, right? Your training, your training data set is like all words on the internet. And so that's one of the reasons that they've been among the first sort of major A I models in this new wave to, to commercialize is because there is this gigantic body of training data that you can draw upon. Now we're hearing lots of folks who are in like robotics world, for example, like facing the challenge of there just isn't an equivalent data set. You can like try to train on YouTube videos or whatever, but it's not quite the same thing. In the case of weather forecasting, it seems to me like in theory, you have an incredible historical, you, you can look at every historical weather measurement, right?

• 36:11Shayle Kann?

  If you had access to that data, you could, if Noah, Noah does, right? Like every input data point they ever took historically and then, and then the subsequent next measurement, which dictated what happened after that, I would think that would be an incredibly rich training data set. Am I, well, two questions, am I right about that and is that actually available?

• 36:32Speaker 02

  Yeah, that's, that's, that's a good question. So the first thing I would say is actually it's not just language, right? Like the, the first big visual neural networks were built, were, it came out of after ImageNet, a big corpus of image data. The first language models, like even a decade ago, they were starting to build large text databases, protein folding, big databases of protein. So actually you see like, you know, A I and, and machine learning are still very, very sensitive to the availability of high quality data and, and large amounts of it. And you kind of like, you know, one of the best ways to advance the field is to go collect high quality data and make it sort of standardized and available. So for weather.

• 37:14Speaker 02

  I think that there's there's sort of good things, there's good news and bad news. So for one thing is we were very fortunate when we started. I think all the all the folks who were working on AI weather forecasting have benefited tremendously from work that was done by the ECMWF, the European Center for Media Range Forecasting. They built this data set called ERA5. And they've been building these, you know, ERA5, I think, was the fifth generation of the ERA data set. It was a record of the of Earth's weather going back for decades. And I think it was originally released going back into the 1979, and then they actually opened it to like back into the 60s. And they weren't, they didn't design this data set to be supporting machine learning. I think it was more to just, you know, have an authoritative record of the climate on

• 37:58Speaker 02

  Earth over, you know, year on year. And it's at like a six hour resolution, 25 kilometer spatial resolution. So it's very, very rich. It just happened to be perfect for machine learning for weather. And it was just a really well curated data set. The folks at ECMWF are just brilliant and sort of organized and systematic, and they had made this available. It allowed a lot of people to sort of, you know, stand on their shoulders and build, you know, great new AI methods. Now, one thing is, though, because we had different satellites and different stations over time, it's not actually all, the data set is standardized, but it's not derived from the same underlying observations. So the quality going back in time actually gets worse in terms of it's not as accurate

• 38:41Speaker 02

  of a record of weather, just because, again, as I said before, the the input data wasn't as good.

• 38:47Shayle Kann?

  Right.

• 38:47Speaker 02

  Now, the other thing that's not, that's not great about weather is that weather takes a while to happen. So like, we have to sort of just wait for more weather data to like, for more weather to happen to get more data, right? Like, the weather data we have so far, we're sort of stuck with it. Now, like tomorrow, we're going to have one more day of weather data. But like when our models are taking six hour steps, we have to just kind of wait. So you've sort of like, we got a lot of data, but at the same time, there's not much room to get more of the same kind of data. And I think we and a lot of others are now looking to more unusual or underexplored sources of data to support building richer, better models.

• 39:30Shayle Kann?

  Right, like, is there like a distributed network data, like, like in theory, if you had access to Uh, I don't know, everybody's cell phone, everybody's iPhone, there's probably a bunch of sensors in the iPhone, presumably you could pull something from that that would have some signal for you.

• 39:48Speaker 02

  Yeah, yeah, you can geek out on all these things. Like, the way, my, my favorite one is like, you know, I have a video doorbell. It sits there and watches weather all day.

• 39:57Shayle Kann?

  Right, right.

• 39:58Speaker 02

  Cars, your car, right? Like, it's, when you're, you got the, you know, it's got a thermometer in it, it's got like, your windshield wipers, some of them now are, you know, they're rain sensing, so it's sensing rain. Or it's like, the lights go on automatically when it's dark, so like, they're sensing whether it's cloudy, or these kinds of things. So, like, I get very excited about the possibility of using all these kinds of things. The other one that's even weirder is like, you know, people go on and they tweet about the weather, or they, like, talk about the weather on, you know, social media. And, like, those types of observations, those are still observations. We don't know if they're very good, but there's actually a very wide range of pretty unusual, under-explored data sources. But even before we get there, like, I think you can start to think about, there's a lot of companies

• 40:42Speaker 02

  that are trying to build very cheap weather stations, people can put them on their roof. Like, these kinds of things could really help, both with the kind of core weather forecast, and probably a lot of the applications that people want to use weather for.

• 40:53Shayle Kann?

  So my takeaway from that is that, despite what I said, there being this, like, amazing historical record of every weather measurement that's ever been taken, you still feel kind of data poor, right? Or like, training data poor, I guess.

• 41:04Speaker 02

  You're always data poor, right? Like, that's sort of the, the story of modern AI is you're basically always kind of data poor. Because, because, one of the most incredible facts about modern AI is just how well it scales with data. More data just means better models. And, you know, I was a person who was very skeptical of this. I didn't think that it was going to scale like this. And I would make my sort of logical arguments, but it turns out I was wrong. And I think a lot of people were wrong. And the folks who really understood that data could really add value even at extremely large scales were right, and pursued that course and brought us to where we are today.

• 41:37Shayle Kann?

  Interesting. Okay, so I guess I want to, I want to finish by talking about what might come. Like, if you, if you draw a line forward a few years into the future, I know you pick, pick your time, three years, five years, 10 years, whatever it is. And, and you and everybody else who's working on AI weather forecasting succeed. Where might we be? Like, what might be possible in a few years that's not possible today?

• 42:04Speaker 02

  Yeah, I mean, I think that, I think there's a lot, you know, weather affects everything. And it's, you know, it has, you know, different things. Energy is obviously a very, very sensitive to weather. Some things are, you know, only kind of, you know, loosely affected by weather. So one thing I would like to see, and I think is very exciting, is a wider range of use cases of weather. So, for example, like, we know that even people make different choices about, you know, what to put in their refrigerator, or like, you know, what clothes, or whatever. These different choices, they're going to go on a trip, what they expect the weather to be.

• 42:45Speaker 02

  I think that you can start to make more subtle and informed kind of guidance and suggestions for people on the basis of what more accurate weather forecasts. And that's kind of like at the consumer level, but I also think that at the kind of, you know, industry level, there could be a huge opportunity. So, for example, in energy, you know, we see there's, you know, if you have a wind farm or a solar farm, you're making forecasts about the weather, and then you're kind of using that to figure out like, how you, you know, if you're going to have energy to sell, and how you price it. But I have a feeling that there's a lot of headroom, a lot more to be gained in how we, you know, plan out our, you know, how to operate our electrical grids,

• 43:27Speaker 02

  how we predict what, what the electrical demand is going to be. Is it going to be hot? Is it going to be cold? Is it going to be humid, or the air is carrying more, you know, mass, which, you know, requires more energy to heat or cool? I think that we just haven't really scratched the surface of the opportunities. I think that supply chains and logistics and, you know, even just like lots of choices that, you know, driving and these type of things, I think. Really could be better informed by better weather forecasts. And I don't think we've even begun to kind of get into this. And we need to see the quality of the forecast get better and more customized to these use cases to start unlocking that. The other one is obviously is just, you know, better crisis handling crises, like, you know, some

• 44:09Speaker 02

  of our recent work on tropical cyclone forecasting, you know, I'm very proud of. And we think, hopefully, with better forecasts, we can warn people earlier, more accurately. We can, in some cases where there's a disaster, like a wildfire, maybe we can even go and intervene and try to stop it before it gets completely out of control. So again, this is not something that we can do today, but I'm, you know, very optimistic that, and in general, maybe this is coming through, but I just, we, you know, my team and I really believe in the power of technology that can have a lot of positive benefits. We often are trying to look for ways that we can, you know, put technology to best use and sort of, you know, leave our kids with a world that's better than the one that, you know, we grew up in.

• 44:53Shayle Kann?

  All right, Peter, this was super interesting and useful for me both in the context of thinking about like weather forecasting and just like understanding like the where AI, how AI is getting applied in various industries, what the challenges are, what the opportunities are. So really appreciate your time.

• 45:09Speaker 02

  Sure. Yeah, it was great to have, to be here and to, your questions were great, too, by the way. It was really fun.

• 45:15Shayle Kann?

  Peter Pataglia is the Senior Director of Research at Google DeepMind Sustainability Program. This show is a production of Latitude Media. You can head over to latitudemedia.com for links to today's topics. Latitude is supported by Prelude Ventures. This episode was produced by Daniel Waldorf. Mixing and theme song by Sean Marquand. Stephen Lacey is our executive editor. I'm Shayle Khan, and this is Catalyst.