How do we predict the weather?

Daniel and Kelly explain how physics predicts the future rain and shine, and all of the incredible science involved. See omnystudio.com/listener https://omnystudio.com/listener for privacy information.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Daniel Whiteson?

  This is an iHeart Podcast. Guaranteed Human.

• 0:04Speaker 01

  Cash flow crunch? OnDeck's small business line of credit gives your business immediate access to funds, up to $200,000, right when you need it. Cover seasonal dips, manage payroll, restock inventory, or tackle unexpected expenses without missing a beat. With flexible draws, transparent pricing, and control over repayment, get funded quickly and confidently. Apply today at ondeck.com. Funds could be available as soon as tomorrow. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 0:35Kelly Weinersmith?

  Granger knows, if you're a procurement manager at an industrial manufacturing facility, you're balancing inventory and budget while sourcing critical supplies across the operation. Too much inventory? Costs climb. Too little? Production slows. Granger's $2 billion in on-hand inventory helps you get what you need, when you need it, so you can help keep budgets balanced and lines moving. Call 1-800-GRANGER, click Granger.com, or just stop by. Granger, for the ones who get it done.

• 1:12Daniel Whiteson?

  When I moved to Southern California, I felt this immediate immense relief. Not just because I was free of the tyranny of outside clothing, but because I was released from the anxiety of not knowing if the weather was going to ruin my plans. Were you planning an outdoor birthday party for your toddler? No need to make backup plans just in case it rains. Do you need to drive a few hours away? No problem. You don't have to worry that a snowstorm might make the roads impassable. Because I could predict the weather myself, since it was the same every single day. But not all of us are lucky enough to live in such calm climates, so it's still very important that we try to anticipate storms so that the less fortunate among us can be prepared.

• 1:55Daniel Whiteson?

  It's not often described as important physics, but predicting the weather is one of physics' great success stories. John Martin, professor of atmospheric noceanic sciences, told me that weather predictions are, quote, "the most unheralded scientific advance of the second half of the 20th century." If you keep score every day, I can't believe how well we predict the weather three to five days in advance. In 30 years, we've gone from predictions from one to two days to now five to seven days. We have made unbelievable progress. So how does that all work? What is the physics underlying the weather? Why has it gotten better? And what can we expect into the future? I talked to Professor Martin and my good friend, Professor Jane Baldwin, here at UC Irvine, about how the weather all works.

• 2:39Daniel Whiteson?

  So we'll dig into all of that in today's episode, dedicated to all of y'all who still experience regular weather. Welcome to Daniel and Kelly's extraordinarily sunny universe.

• 3:03Kelly Weinersmith?

  Hello, I'm Kelly Wiener-Smith. I study parasites and space, and I love rainy days.

• 3:10Daniel Whiteson?

  Hi, I'm Daniel. I'm a particle physicist, and I can predict the weather in California for the next 100 years with my eyes closed.

• 3:18Kelly Weinersmith?

  How boring. How massively dull.

• 3:21Daniel Whiteson?

  How wonderfully, delightfully, predictably, reliably boring.

• 3:25Kelly Weinersmith?

  Oh, you know, one of my favorite weather moments, I have to admit, was a Southern California morning. So I was a visiting scholar at the University of California, Santa Barbara, for a little while. And I had an office that was like, right out on the ocean. It was amazing. And when I was driving in one day, there was just a little bit of water on the ground, and the car tires were kicking up a little bit of a spray. And there were literally rainbows following all of the cars into school. And then I got out of the car, and the rain had stopped. And there was a rainbow over the ocean, and there were hummingbirds. And it was like a Disney movie scene. I expected like, a bunny to hop out and be like, can I help you with anything? And it was, anyway, it was, it was kind of magical. I'll give you that.

• 4:08Daniel Whiteson?

  California is heaven, yes. What happens when you die in Virginia is you end up in California.

• 4:13Kelly Weinersmith?

  Do you know that, that not all California is Southern California?

• 4:17Daniel Whiteson?

  I mean, all of real California.

• 4:18Kelly Weinersmith?

  Oh, I see. Because Northern California's got some weather.

• 4:23Daniel Whiteson?

  You're absolutely right. In fact, I heard Katrina say something really insightful the other day. You know, she's from Northern California, but now we've lived in Southern California for quite a while. And she said to somebody that she's now a complete Californian because she's lived in both Northern and Southern California. And I was like, oh, that's cool. She's like, accepted Southern California, which is hard for Northern Californians, I am aware. Yes. Not everything is Southern California, unfortunately.

• 4:50Kelly Weinersmith?

  Oh, I really like the variability. Virginia weather is amazing for me. But so my question for you is, what is the worst weather situation that you've experienced?

• 4:58Daniel Whiteson?

  Great question. I was on the East Coast

• 5:01Kelly Weinersmith?

  Of course.

• 5:02Daniel Whiteson?

  last year doing a college tour with my son and we were in Massachusetts. I think we were visiting Amherst, or maybe it was Williams, I don't remember. And there was some freak tornado which tore up a bunch of trees and knocked down a bunch of power lines.

• 5:16Kelly Weinersmith?

  What?

• 5:16Daniel Whiteson?

  And there was no power in the whole town for like, almost half a day. It was crazy. And the winds were insane. And it felt a little scary. Like, we saw like, huge branches flying by the window.

• 5:31Kelly Weinersmith?

  Yeah. Yep.

• 5:33Daniel Whiteson?

  And he didn't end up going to school there.

• 5:36Kelly Weinersmith?

  Yeah, I get that. I get that. So we lived in Alabama, Tuscaloosa, and we moved there pretty soon after that giant tornado that like, made the news. And you could see the path of the tornado, because like, you know, you'd be driving through an area with lots of like, you know, Starbucks, Panera, lots of stores or whatever. And then suddenly there would be a like, air, an opening in between all of the stores with nothing. And like, the tornado had just gone through there and just absolutely picked up and thrown everything that was in there.

• 6:05Daniel Whiteson?

  Incredible.

• 6:05Kelly Weinersmith?

  And even after they cleaned it out, there were still, you know, you could tell where the tornado had gone. And we were also in Houston during some pretty bad storms. And we had the kids and our dog and our cats in a little hallway in the interior of the house. And my in-laws were visiting. And my mother-in-law was so sweet. She like looked around and she, she was trying to see, you know, who could get hurt and how. And she gave her glasses to Zach in case there was any like flying glass. And she just insisted that he have her glasses. And I was like, in that moment, I was like, gosh, you are the sweetest person in the whole world. Like you are thinking about the tiny little things you could do to help the people around you. And anyway, she's, she's the best.

• 6:45Daniel Whiteson?

  Yeah. But we've all been caught in surprise weather, right? I remember going backpacking in Arkansas one time and being caught in a snowstorm and the temperatures dropped into the teens and we weren't a hundred percent sure we were going to make it.

• 6:57Kelly Weinersmith?

  Oh, no.

• 6:57Daniel Whiteson?

  And everybody's been like, you know, caught in a snowstorm or a rainstorm or in a heat wave, right? And these things are exciting. They can be dramatic, but they can also be very dangerous, right?

• 7:07Kelly Weinersmith?

  Yeah.

• 7:08Daniel Whiteson?

  People die in these crazy weather storms. And so it's valuable to be able to know in advance what's going to happen, not just so you can plan your picnics, but also so that you can survive the increasingly dramatic weather that we're all facing as the planet warms.

• 7:22Kelly Weinersmith?

  Yeah, that's right. More severe weather is becoming more common. And so today, we're going to talk about how good we are at making predictions and how we go about making those predictions.

• 7:31Daniel Whiteson?

  Exactly. And I wanted to pull back the curtain on like the science of this. How does this actually happen? What are we doing? Why is it hard? What are the challenges? What improvements might we be seeing in the next five or 10 years? What problems are just fundamentally impossible and might never be solved? And so today, we're going to dig into the science of all that. But before we explain to you how the experts do it, I was wondering what everybody knew about how weather predictions happen. How do those numbers end up on your phone? So I went out there to ask our listeners what they knew about how we predict the weather. If you would like to answer these kinds of questions for a future episode, don't be shy, write to us to [email protected]. We will send you fun questions every week

• 8:14Daniel Whiteson?

  in your inbox. In the meantime, think about it for a minute. What do you know about how we predict the weather? Here's what our listeners had to say.

• 8:22Speaker 03

  Sophisticated computer models, which with an understanding of chaos theory, allows us to understand the limitations.

• 8:31Speaker 04

  Predicting the weather is like quantum particles. There are many probabilities, but it is not known until it is observed.

• 8:41Speaker 05

  Meteorologists, they look at the current weather and they try to predict it by looking at the moving clouds and all of that.

• 8:49Speaker 06

  By measuring wind velocity and atmospheric pressure and maybe modeling these data in supercomputers.

• 9:02Speaker 04

  When a cow lies down in the field, it's gonna rain. And when my knee aches, it's gonna snow. Running multiple models. Big computers, really, really big computers. Feed that data to complicated models that run on very powerful supercomputers. I'd say with surface measurements, satellite information and sophisticated models, and perhaps even artificial intelligence.

• 9:30Kelly Weinersmith?

  Observations taken by ships, planes, ground stations, satellites, combined with models built by really, really smart people that run on some of the fastest computers that humans have ever built. There are sophisticated models that use a wide range of observational and predictive inputs. By observing weather patterns and the types of weather those patterns tend to bring. So I don't know if there's actually like scientific evidence that sometimes knees will ache if like a storm front is coming through. But I have to admit that there's a part of me that really hopes that if I get arthritis when I'm older, I do have like the ability to tell when the weather's coming because I'll feel like I'm really intimately connected to my environment. Oh, the knee's acting up again. Storm's coming, get the goats in the barn.

• 10:14Daniel Whiteson?

  I think that really shows your fundamental optimistic nature, Kelly, because you're like, oh, if I get arthritis, there'll be a silver lining. I can predict the weather.

• 10:23Kelly Weinersmith?

  You know, and life is easier when you try to see the silver lining.

• 10:27Daniel Whiteson?

  That's wonderful.

• 10:28Kelly Weinersmith?

  But our audience had great answers and they were, you know, a lot of them said, you know, exactly the right thing, which is you've got to have data. Those are the observations and you feed them into computers.

• 10:39Daniel Whiteson?

  Yeah, essentially. And that's the big picture, not just of weather prediction, but any kind of prediction. There are two fundamental ingredients to how you make a prediction. There's the models and then there's the data. So let's take those each in turn. When we say the models, we mean like we're running a computer simulation or you're calculating things on paper. Fundamentally, this is encoding the rules of the system, what the future can be, given what the past was. And this doesn't have to be some really complicated thing like the weather over Istanbul. Think about a much simpler situation like you're tossing a ball in your backyard. You want to know where does it go? Well, the laws of physics predict the future, right?

• 11:19Daniel Whiteson?

  This is the model. These are the rules that tell you how the past becomes the future, right? In this case, it's simple. It's a parabola. It flies through the air. Things to keep in mind here, though, is that a model like this is always approximate. If I use F equals ma and I just account for gravity, ignore air resistance when I'm describing the ball. I'm going to get a quick answer and it's going to be pretty good. It's not going to be exactly bang on correct. It can't account for everything, all the little wind gusts and the air resistance and the slight change in humidity, maybe the spin on the ball. My model ignores some details and that's crucial, right? If I included every single particle in the backyard, I would never get a calculation. So in order to make this tractable, I got to simplify the problem.

• 12:02Daniel Whiteson?

  I got to pull out the things that are important and ignore the things I think are unimportant. Because I don't think they're going to make a big enough difference in the answer. And this is where the juice is. This is what physics is, right? Physics is taking the universe and simplifying it into a model that represents the bits you're excited about. The bits you think are interesting and relevant. And then you use those rules and manipulate it. That's your model of the universe. And the model gives you an answer. And hopefully, if the model is close enough to your description of the universe, the answer you get from the model is similar to the answer in the actual universe.

• 12:36Kelly Weinersmith?

  Well, so one thing I think that's amazing is that something as simple as throwing a ball up in the air and then seeing where it lands is something we can't completely model because there's so many complicating things. And now you're talking about. Weather, which is so much more complicated and requires so many more inputs. And of course, you can update your model. So you know, if you threw the ball in the air and you were like, you know what? It's a windy day.

• 12:56Daniel Whiteson?

  Yeah.

• 12:56Kelly Weinersmith?

  I absolutely need to add wind. Now you've learned something, you add winds. And so, you know, it's an iterative process where you keep trying to say, what is important? And do I need to include it? And does it make my predictions better? But I also will note that you put predicting weather under the physics umbrella. You, you think you guys get to claim weather predictions?

• 13:18Daniel Whiteson?

  I mean, we're not using economics to predict the weather. What, what else is in the running for taking credit for predicting the weather? Is it chemistry?

• 13:27Kelly Weinersmith?

  I feel like that also is some ecology, you know, like, because you're tracking like.

• 13:31Daniel Whiteson?

  Cow farts or something?

• 13:33Kelly Weinersmith?

  No, no.

• 13:34Daniel Whiteson?

  Yeah, like cow farts play a role, actually.

• 13:37Kelly Weinersmith?

  So do you, do you think that NOAA has cow farts in their weather prediction models?

• 13:43Daniel Whiteson?

  I think the climate models do include bovine methane emissions, yes. So not the daily predictions, but the, the bigger trends, yes, cow farts do help determine the future of our planet.

• 13:53Kelly Weinersmith?

  Amazing.

• 13:54Daniel Whiteson?

  I want to go back to the point you made earlier. You're exactly right that we're always approximating. And not just when we're doing the weather, not just when we're tossing balls, always, every single time, every model is approximation. There's this famous phrase, I don't remember who said it, like, all models are wrong, some of them are useful. Even our description of like the fundamental particles in the universe, as far as we know, these are approximations. Every bit of science we have has boundaries of where it's relevant because there are approximations made when we construct those models. Everything, literally everything. We have no piece of science that isn't an approximation of the universe.

• 14:33Daniel Whiteson?

  Maybe one day we have a theory of everything and it's beautiful and we can do exact calculations on very, very simple situations. But we're not there. We may never be there. And even if we are there, it will be totally impractical for anything useful. Like you couldn't use string theory to predict the path of a hurricane because the complexity would be insane. Right? How many strings you? Modeling the amount of computation required to do it exactly would be impossible. So it's always an approximation. It's just a question of which approximations. And that's where the science comes in, like which ones are important, having a nose for what to approximate and what not to approximate. That's what helps some scientists make more progress than others.

• 15:14Kelly Weinersmith?

  Yeah. And I think another thing to just sort of note is that because this is a human endeavor, sometimes you're limited by what you can afford to get data on. You know, like maybe you do want to know how much cows are farting, but in order to get that data, you would need $70 billion. So that farmers could attach sensors to the rear end of every cow. And so like, you know, sometimes you, you know, there's data you want, but you can't get it because there's not enough money or it's not possible. Maybe one day you can get it. Maybe those sensors will become cheap.

• 15:44Daniel Whiteson?

  Is $70 billion your like fantastical number for some like absurd amount of money for a science experiment?

• 15:49Kelly Weinersmith?

  Uh, yeah, I guess that's, yeah. What is yours? I guess you're a physicist, so it's going to be like.

• 15:54Daniel Whiteson?

  Well, that's embarrassing because our next project is $100 billion. We're like already above the Kelly threshold for like absurd amounts of money.

• 16:03Kelly Weinersmith?

  But, but wait, like, okay, but that's not like your personal project. That's like LHC or like a new particle collider or something, right?

• 16:09Daniel Whiteson?

  Yeah. The new next particle collider budget is about 100 billion.

• 16:13Kelly Weinersmith?

  Oh, my gosh.

• 16:13Daniel Whiteson?

  Exactly. So more than a planet wide cow fart sensor network.

• 16:17Kelly Weinersmith?

  Well, you guys better make some really important discoveries with that money. Otherwise I'm disappointed because I want to know what's happening with the cow farts.

• 16:25Daniel Whiteson?

  But you bring up another point, which is the data. So models are useful. They're a system to tell us how the past becomes the future. But you also need some data so you know which past you had, right? Models describe essentially any possible universe. The rules determine which set of universes we might live in, but the data constrain it. It tells us which past we had. So the rules tell you how the past becomes the future, but you need to know which past we were in. So we know which future we'll have. So in our ball tossing analogy, there's lots of different ways I could toss a ball. I could toss it high or low or fast or slow or east or west. The rules connect the initial conditions, the data, the past. To the future, but you need to know where did I throw the ball?

• 17:10Daniel Whiteson?

  So if I'm writing a simulation of that ball toss, I got to encode in the laws of physics, but then I need a data point. I need to say the ball was here and it was moving in this direction at this velocity, then I can predict the future. Without that, it's useless, right? So you need these two components. You need the models plus you need the data. And then you need more data. Say I'm predicting the ball toss. I want to check in halfway and say, hey, is my model correct? Does it need an adjustment? A way to improve your modeling is to shorten the prediction time to say, I'm not going to predict the whole path. I'm going to predict the second and then I'm going to take a measurement. And if it's off, I'm going to correct it so that if my model has veered off from reality, it doesn't get further off. And so the more data you have, the better your model is going to be.

• 17:54Daniel Whiteson?

  So you need these two elements dancing together, the models and the data.

• 17:59Kelly Weinersmith?

  Yeah, and I checked my weather app today and the prediction for tomorrow was changed. And so I'm guessing we do the same thing with weather. We update. So I think we should talk in a second about what kinds of data we collect to help us inform models. But I guess my first question is, so we've talked about models in general. How long have we been trying to model weather? Aristotle, probably.

• 18:23Daniel Whiteson?

  People have had some crazy ideas about the weather for thousands of years. The first real weather models were conceived of in the 1920s. And remember, we didn't have computers really until the 50s or so. So this was like a conception and somebody did a proof of principle prediction. They tried to predict the weather six hours later. They took a bunch of measurements and said, let's try to do some calculations. We have an early model. That calculation took six weeks.

• 18:50Kelly Weinersmith?

  Not helpful.

• 18:52Daniel Whiteson?

  Helpful, exactly. But they did it and it wasn't terrible. And they sort of proved like, hey, you know, if you could do this calculation more quickly, then maybe you could even know the weather in advance. Oh, my gosh, what an idea, right?

• 19:03Kelly Weinersmith?

  Yeah.

• 19:04Daniel Whiteson?

  It wasn't until the 1950s that we had the first computing models to do these calculations. So we can make predictions in time shorter than the prediction period. You could have enough data and run your model and get an answer before the universe revealed it, right? That's the prediction instead of a post-diction.

• 19:23Kelly Weinersmith?

  That's better.

• 19:23Daniel Whiteson?

  So we've been doing this for decades and the last, you know, 70 years or so have been improving the models and improving the data.

• 19:31Kelly Weinersmith?

  Man, it's exciting to think that we, you know, we're going from slide rules to make these predictions to massive supercomputers. I'm appreciating my weather apps a bit more.

• 19:43Daniel Whiteson?

  And also like six weeks sounds ridiculous. I don't know that I could do that in six weeks.

• 19:47Kelly Weinersmith?

  Oh, yeah.

• 19:47Daniel Whiteson?

  It's an amazing calculation and think about like, not just the ideas, but all the grunt work of doing all those calculations and the human error that's possible. Like, it's amazing they did it in six weeks, you know? So don't laugh at that.

• 19:59Kelly Weinersmith?

  Absolutely. So we've been doing this since the 1950s. Let's talk about what kind of data we're collecting to inform these models when we get back from the break.

• 20:17Speaker 01

  Cash Flow Crunch. OnDeck's small business line of credit gives your business immediate access to funds, up to $200,000, right when you need it. Cover seasonal dips, manage payroll, restock inventory, or tackle unexpected expenses without missing a beat. With flexible draws, transparent pricing, and control over repayment, get funded quickly and confidently. Apply today at ondeck.com. Funds could be available as soon as tomorrow. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 20:47Daniel Whiteson?

  Hey, I'm Nile Rodgers, producer, songwriter, guitarist, and co-founder of the group Sheik. A great song has a way of defining a moment, a moment of joy, heartbreak, or anything in between.

• 21:04Daniel Whiteson?

  This fall, I'll be driving across the country, dropping in on some of the coolest, most fascinating artists working today. We'll go deep into their lives and careers and hear what it was like before the moment that made them. From hitmakers like Miguel. That last year, finishing that record.

• 21:24Kelly Weinersmith?

  It was the hardest experience of my life. Regardless of how everyone else buys into this. I believe in this and I'm gonna push.

• 21:33Daniel Whiteson?

  To rising stars like Alex Warren.

• 21:36Speaker 04

  You know, a song is a story and the verse is setting the scene. And for me, at least, it's allowing me to be personal and vulnerable with the audience. Whether it's about my wife and I'm being vulnerable about her or it's about, you know, my parents and I'm being vulnerable to them. And it's almost like a cinematic piece in a way where each song tells that beautiful story.

• 21:54Daniel Whiteson?

  It's gonna be one hell of a trip. Experience Before the Moment, a Lincoln original series on the iHeartRadio app, YouTube, or wherever you get your podcasts.

• 22:07Kelly Weinersmith?

  Hello, hello. I'm Malcolm Gladwell, host of Smart Talks with IBM. Imagine your business depends on a system you didn't build and don't fully control. For years, it works perfectly. It's fast, reliable, affordable. So over time, more and more of your operations rely on it. Then one day, the terms change. What once felt flexible now feels difficult to leave. Moving to another provider or bringing those capabilities back under your own control could take years, cost millions, and disrupt the business in the process. That's the challenge of digital sovereignty. Businesses today rely on all sorts of third-party systems.

• 22:49Kelly Weinersmith?

  Cloud platforms, data pipelines, AI models, software tools. Most days, these systems operate quietly in the background. But when something changes, like costs go up or rules shift, organizations are forced to confront how much control and flexibility they actually have. So to understand the challenge up close, I traveled to Paris to speak with IBM's Anapaula Assis and AXA's CIO Giovanni Daniele, two people who think about these questions every day. Hear the full conversation at ibm.com/smarttalks or wherever you get your podcasts.

• 23:43Kelly Weinersmith?

  All right, and we are back. So now we're going to talk about the kinds of data that we use to make weather predictions, and I'm going to bet it involves satellites.

• 23:54Daniel Whiteson?

  Always going with space first, right?

• 23:55Kelly Weinersmith?

  Yep, yep.

• 23:57Daniel Whiteson?

  It does involve satellites, but there's an amazing, incredible variety of data sources we have to understand the weather, and yet still it's not nearly enough, right? As you'll hear, our weather prediction would be so much better if we had more data. We're really limited by the data, but we have lots of different kinds. We have weather stations on the surface, and so a lot of these are called like automatic weather stations that are scattered across the country. They're just basically a bunch of sensors with a battery and like either a wind turbine or a solar panel to get power. And they measure things like temperature and pressure and wind speed and precipitation, just the raw measurements you need to know like what's going on out there. What is the state of the weather right now?

• 24:41Daniel Whiteson?

  Because again, if you want to predict the future weather, you've got to know what's going on right now.

• 24:46Kelly Weinersmith?

  So is this like a citizen science thing where like I could purchase one of these weather stations and hook it into what's happening at like the national level?

• 24:54Daniel Whiteson?

  Yes and no. So there are a few sort of official stations. There's a bunch of different networks. The highest quality ones, there's like 10,000 of these scattered around the earth and they're operated by weather services and government agencies. But there's a bigger network of like quarter million of these things. Some of these are personal weather stations that yeah, people just build and publish the data. And there's an amazing network, it's called COCORAHS, C-O-C-O-R-A-H-S, Community Collaborative Rain, Hail and Snow.

• 25:24Kelly Weinersmith?

  Wow, okay.

• 25:25Daniel Whiteson?

  You can just like build your own device and add it to the network and contribute. And I think that's super awesome because it's definitely limited by the data we have. One problem is that these things tend to be where the people are. Like we have a few, you know, top amount, Washington or whatever, but mostly these things are put up by people where people are near. And so like this law- Lots in India, for example, but very few across Siberia. And often the best ones are places like airports. Airports really need to know weather, so they have excellent weather stations. But like the weather at LaGuardia is not the same as the weather in Manhattan. And so often the airport weather stations are very, very precise and used heavily in the models. But they're not giving you the measurements exactly where you want them to be.

• 26:10Kelly Weinersmith?

  Okay, so is that a problem for just the people who are in areas where there's not enough weather detectors? Or is that a problem for all of us? Because what's happening in Siberia is important to what's happening in India.

• 26:24Daniel Whiteson?

  Yeah, what happens in Siberia doesn't stay in Siberia, unfortunately.

• 26:29Daniel Whiteson?

  It contributes to uncertainty and error across the model. And the earth is one big system, which is why you can't just be like, I'm only going to predict the weather in Manhattan. I only need to think about Manhattan. You need to model the whole planet in order to get the weather in Manhattan. So yeah, absolutely. And that's why we have lots of different kinds of sensors, not just these automatic weather stations. We also have things like weather radar. And you might have seen these on your local weather channel, like, let's look at the Doppler. This measure precipitation. It also measures the velocity of those raindrops. And this is a really cool story because it comes out of World War II. It's another example of like reusing military technology and infrastructure after World War II to do some science.

• 27:09Kelly Weinersmith?

  Thank you, war.

• 27:11Daniel Whiteson?

  Ooh, boy.

• 27:13Kelly Weinersmith?

  Hot take, pulling it back.

• 27:15Daniel Whiteson?

  Well, there you are again, finding the silver lining. Tens of millions of people died, but we have better weather predictions. So the way radar works is that it sends these pulses of microwave radiation. The wavelengths are like one to 10 centimeters. That's the microwave region. And it sends a pulse for like a microsecond and then it listens for return signals. So like it sends this pulse and raindrops will reflect. So it gets their signal back and it listens for like a few milliseconds and then sends another pulse. And so it can tell where the clouds are and it can tell the velocity of those clouds by the change in frequency. This is the Doppler effect, right? And this is exactly the same effect. Is like stars are moving away from you. So their light is red shifted.

• 28:01Daniel Whiteson?

  When the radar pulse comes back, if the frequency is shifted, you can tell which direction that raindrop is moving.

• 28:07Kelly Weinersmith?

  So that sounds complicated because like there's not just one raindrop out there. There's a bunch. And so I can imagine like your pulse getting lost as it bounces off of multiple raindrops and doesn't make it back to you. What, what am I missing? This sounds hard.

• 28:21Daniel Whiteson?

  No, it is hard, but you're not detecting individual raindrops. You're detecting clouds mostly like which direction is this cloud going? And, you know, initially this was a problem because in World War II radar operators were trying to use radar to discover like enemy planes. And they noticed like, man, clouds are getting in the way. And then other folks were like, oh, wait, you can use radar to see clouds? Awesome.

• 28:45Daniel Whiteson?

  So then after World War II, they started using this to measure the velocity of clouds and to see them. And there's this moment in like 1961 when Hurricane Carla was approaching the coast of Texas. And Dan Rather went down there to a weather station and they were using radar to see the clouds and to see their direction. And he had them draw like the coast of Texas over this image of the hurricane that showed everybody like, wow, this is a massive hurricane moving fast towards the shore. And probably saved thousands of lives because he publicized this like incoming storm much more rapidly than we could otherwise without this kind of technology.

• 29:20Kelly Weinersmith?

  Wow.

• 29:20Daniel Whiteson?

  So this, yeah, this weather radar is really helpful.

• 29:22Kelly Weinersmith?

  Do you think it still has the same effect or do you think people are just kind of like, oh, there's, there's hurricanes. I've seen them before. They get big and they don't always leave.

• 29:30Daniel Whiteson?

  People don't always leave. There's always somebody who's going to ride out the storm, right? Yeah. And I don't know about the psychology there, but at least now we can inform people further in advance and let them know where these things are likely to go. But there's still always uncertainty and we'll talk about that in a minute. You don't just have one weather prediction. You have an ensemble. You have a, an envelope of predictions because you don't have perfect data and you don't have a perfect model. And so often what you do is you vary your data a little bit within the uncertainties and run the model again. And then you get a different prediction and it'll give you a sense of the spread of the possible outcomes. So you might see when there's like a hurricane approaching the coast of. Florida, they have a bunch of possible trajectories. Those are all like different runs of the weather model, assuming different initial conditions because we have uncertainty. We

• 30:15Daniel Whiteson?

  don't have perfect data.

• 30:16Kelly Weinersmith?

  I personally really enjoy learning about the uncertainty, like in life in general. And whenever I look at those, I have this weird feeling of like security.

• 30:26Daniel Whiteson?

  Yeah.

• 30:26Kelly Weinersmith?

  Like they figured it out and they know what the errors are. We're good. We know what to avoid. Maybe that's, maybe that's a little bit, uh, giving it a little too much credit, but it's still amazing.

• 30:35Daniel Whiteson?

  And another really important source of uncertainty in our models is what's happening in the ocean. Like, how hot is it? How cold is it? How are things circulating? All this kind of stuff. And so we need data about the ocean, but not a lot of people live in the ocean. So we don't have like these automatic weather stations, but we do have buoys. These are like floating weather stations. And around the world, there's a couple of thousand of these, depending on the type, that have these like temperature sensors on the surface. But we also have this hilarious data from what's going on deeper in the ocean that historically has come from people on ships, taking a bucket, dropping it into the ocean, pulling it up, and then measuring the temperature of the water.

• 31:20Daniel Whiteson?

  And it's like really that lo fi. But for many years, that's all we had. We had like no other way reliably to know how cold is it in the ocean? And this is an example of like, it's not just data. You need to take data and interpret it and clean it and correct it. And I spoke to an expert here at UCI, Jane Baldwin, who told me that like, you had to correct for like how long the bucket was out of the water before they dunked the thermometer in it and how Japanese ships and US ships use a different bucket and it had different effects. And like, you gotta really know, you gotta be an expert in how this data was taken and what it really means.

• 31:54Kelly Weinersmith?

  Yeah. So for a while I was doing some water quality work and we had this like tube and you would put the tube underwater and then you'd sort of press a button and like caps would pop into place on both sides of the tube. And then you could lift it up. And so you could get a water sample from specifically different depths. And it was, it was always kind of fun to use that device.

• 32:13Daniel Whiteson?

  Yeah. And you might think like, that's ridiculous. What a risk. Silly system and it's a little bit silly, but if it's the only data you have, it's better than no data.

• 32:22Kelly Weinersmith?

  Yeah.

• 32:22Daniel Whiteson?

  Right?

• 32:22Kelly Weinersmith?

  Yeah.

• 32:22Daniel Whiteson?

  As long as you understand the uncertainties in it. And my friend Jane was telling me that misunderstanding this data might be a cause for some weird pauses in global warming trends. That it could just be like a misinterpretation of this ship bucket dunk data.

• 32:37Kelly Weinersmith?

  Oh, no.

• 32:38Daniel Whiteson?

  I know.

• 32:39Kelly Weinersmith?

  We're so low five.

• 32:40Daniel Whiteson?

  Amazing. These days we have these cool robotic floats that like float on the surface of the ocean and then dive down up to 2,000 meters. Measure things down in the ocean and then come back up and beam it to satellites or whatever. So we're getting better, obviously.

• 32:55Kelly Weinersmith?

  Yeah.

• 32:56Daniel Whiteson?

  But you know, what's really valuable is longitudinal data. Like you want data as far back as you can, so you can understand bigger trends. So you can't just like, say, oh, that ship bucket dunk data is ridiculous. Let's ignore it. It's the only data you have for like 30 years. And so trends in that data do tell you something.

• 33:13Kelly Weinersmith?

  Very cool. Okay, so now we've gone down deep. How do we get data from up high?

• 33:18Daniel Whiteson?

  Yeah, because the weather is not just at the surface, right? And the weather folks call the surface the two meter level because they want to measure the temperature, not on the ground, literally, but like two meters up, like where your head is essentially. But they also need to know what's going on even further. So we take measurements in the upper atmosphere. We use weather balloons and these are literally what you imagine. You put like a bunch of helium in a balloon and you put a weather station on it that can measure altitude, pressure, temperature, humidity, wind speed, et cetera. And you just let it go and it rises because helium rises. And as it goes up, the balloon expands because the pressure in the upper atmosphere is less. And eventually it pops and then the thing comes back down. So these are like one time uses, right?

• 34:03Daniel Whiteson?

  And they can go up like 20 kilometers.

• 34:05Kelly Weinersmith?

  When I visited St. Catherine's University in Minnesota to give a talk, they had a special day where they launched a weather balloon, like, you know, for my visit and, and, you know, did a demonstration for all the students. And it was the coolest thing ever.

• 34:18Daniel Whiteson?

  Super cool, right?

• 34:20Kelly Weinersmith?

  Yeah.

• 34:20Daniel Whiteson?

  These are amazing experiments and I know people who do physics experiments on balloons where they like go the Antarctic. And they let up a balloon and it floats in the atmosphere for like up to a month or something. And like, wow, that's really brave work because you spent like four years building this instrument. And then you're putting it on a balloon and into the atmosphere. And sometimes it's just gone, like it just disappears and you lose your whole thesis. And this seems like kind of bespoke, right? And it is, there's like a couple hundred launches per day in the United States, but it's not reliable. It's not like the place you visited, they do exactly the same balloon launch every single day at the same time. Right. Which is the most useful thing for a weather model is like reliable data. And we don't have a lot of them, but again, this helps you probe the upper atmosphere.

• 35:04Daniel Whiteson?

  We don't have many ways to measure the temperature in the upper atmosphere. This is a really powerful one.

• 35:10Kelly Weinersmith?

  Do we also use planes?

• 35:11Daniel Whiteson?

  We do use planes because every airplane you've been on has really valuable information about weather. Because it samples from the two meter level up to like 30,000 feet. And aircraft, of course, have sensors to measure wind speed and temperature and all this kind of stuff. So every commercial airplane has these sensors, collects this data and then sells it to the government. NOAA buys this data because there's so many flights, like look at a map of airplane flights for a single day in the United States. There are so many flights, they crisscross the country. And it's incredibly valuable data, and this is usually very high quality data because it's very important for these planes to understand the weather.

• 35:50Kelly Weinersmith?

  I had no idea NOAA was getting access to all of that data. That's super cool.

• 35:54Daniel Whiteson?

  It's super cool. Basically any way you can imagine to learn the state of the weather somewhere on earth. Somebody's doing it because the more data we have, the better these models get. But then of course, we can go all the way up to space, right? Because there are places where there are no automatic weather stations and there are no buoys and there are no airplane flights. Yet they still contribute to the weather prediction in Kansas or in Mexico City or whatever. And so we have satellites and since about 1979, we've had weather satellites. We of course had satellites earlier than that, but then devoted to like gathering weather data. And these primarily cover things like storm systems and cloud patterns.

• 36:34Daniel Whiteson?

  They can tell you where the snow is. They can also tell you like where wildfires are, which is an important part of the weather.

• 36:41Kelly Weinersmith?

  Yeah.

• 36:41Daniel Whiteson?

  And so they can't directly measure like, what is the temperature in Houston right now? But they can make indirect measurements, like for example, they can measure the amount of infrared radiation from the surface and that is connected to the temperature, but it's actually connected to the temperature of the surface, not the two meter level.

• 37:00Kelly Weinersmith?

  Oh.

• 37:00Daniel Whiteson?

  Right. So like, how hot is the black top in Houston right now? That's what your satellite is telling you and you have to use that to infer how hot is it two meters above the black top in Houston, which is what you actually want to know.

• 37:12Kelly Weinersmith?

  That sounds hard.

• 37:13Daniel Whiteson?

  It's hard. Yeah, exactly. And so we also don't have a lot of satellites because they're expensive. There's something like 20 satellites, a mixture between geostationary and polar orbits. Eight of them are operated by NOAA, but there's a bunch out there. But my friend, the climate scientist says that we might be on the verge of having a lot more data because launching stuff into space is cheaper. And now we can do like small satellites, CubeSats. These might give us more data, but not as high quality as like the dedicated, you know, super nerd designed billion dollar satellites. On the other hand, we don't know what the future holds for like supporting and operating these satellites. This requires money to fund these things and have people interpreting these things.

• 37:56Daniel Whiteson?

  We don't know how long the government is going to continue to support it. They could just like unfund this stuff or turn off weather stations.

• 38:03Kelly Weinersmith?

  Oh, my gosh.

• 38:04Daniel Whiteson?

  You know, it's more than just like, oh, we turned it off for a year. Having continuous records is super important for these models for predicting the immediate weather, but also for the long-term climate models, which are essentially an average of the weather. And so even turning it off briefly could be very damaging for our ability to do long-term predictions.

• 38:23Kelly Weinersmith?

  And I'm kind of blown away by the fact that we only have 20 satellites. I guess I had assumed since, you know, there's like 5,000 satellites up there or something. I guess most of them are dedicated to like beaming cat videos to us from anywhere in the world, but like, weather seems so important, you know, for, for farmers, for like people who are traveling, just like for everything.

• 38:42Daniel Whiteson?

  Yeah, that's true. But the satellites don't give you a direct measurement of what you're most interested in. They're essentially like really good for filling in the gaps for places where you have no other measurements. So yeah, it would be great, but they're also super expensive. So you'll hear at the end, I asked one of the climate scientists I spoke to, like, if you had a billion dollars, what would you spend it on? And satellites is not their top priority.

• 39:03Kelly Weinersmith?

  Huh. Okay. All right. So maybe 20 is the right number.

• 39:08Daniel Whiteson?

  So you have all these different kinds of data. You have automatic weather stations. You have radar, you have buoys, you have ship bucket data, you have weather balloons, aircraft, you have satellites. What you need for your model are the initial conditions. What you need for your model is a set of what is the temperature and the pressure and the humidity everywhere on the planet right now, so that I can run it and predict it in the future. And that's, there's not a trivial step from like, here I have all this data to what are the initial conditions? Because the data can disagree, right? You have multiple measurements sometimes nearby using different kinds of sensors. How do you incorporate that? How do you clean this data? How do you decide what to use? How do you merge all of this into your best prediction? And so there's a lot of work in this area. It's called

• 39:51Daniel Whiteson?

  data assimilation of running sort of mini models, fluid dynamics to do like physics-informed interpolations between the places where you don't have measurements and to factor in the various uncertainties from the various different kinds of measurements. So sometimes you like back up the model a little bit and feed in some data and then use it to predict the current initial conditions before you go to your full model. And then you do what we talked about earlier, which is ensemble. And you say, well, here's my best guess for the weather like right now, before we even run the model, but I'm going to make like 100 versions of it. And each one, I'm going to tweak my assumptions a little bit. So I get an envelope where I hope reality somehow is described by one of these models or is

• 40:34Daniel Whiteson?

  near one of these models. Or the spread in these models describes my uncertainty in the state of the initial conditions. We haven't even done any predictions yet. This is just like measuring what's happening now, right?

• 40:45Kelly Weinersmith?

  And what's so stressful for me to thinking about this is like your data are coming in constantly. And so it's not like you do this once and then you're like, okay, good. Now we will project. It's like every second more data are coming in. So this has to be like a constant process that's happening over and over again and integrating the information into bigger models. And

• 41:03Daniel Whiteson?

  Exactly.

• 41:03Kelly Weinersmith?

  It's amazing.

• 41:04Daniel Whiteson?

  Exactly. And yeah, we haven't even talked about how the models work.

• 41:08Kelly Weinersmith?

  And so let's take a break. And when we get back, we'll talk about how those models work.

• 41:21Kelly Weinersmith?

  Cash flow crunch? OnDeck's small business line of credit gives your business immediate access to funds up to $200,000 right when you need it. Cover seasonal dips, manage payroll, restock inventory, or tackle unexpected expenses without missing a beat. With flexible draws, transparent pricing, and control over repayment, get funded quickly and confidently. Apply today at ondeck.com. Funds could be available as soon as tomorrow. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 41:52Daniel Whiteson?

  Hey, I'm Nile Rodgers, producer, songwriter, guitarist, and co-founder of the group Sheik. A great song has a way of defining a moment, a moment of joy, heartbreak, or anything in between. This fall, I'll be driving across the country, dropping in on some of the coolest, most fascinating artists working today. We'll go deep into their lives and careers and hear what it was like before the moment that made them. From hitmakers like Miguel...

• 42:25Kelly Weinersmith?

  That last year, finishing that record was the hardest experience of my life, regardless of how everyone else buys into this. I believe in this, and I'm gonna push.

• 42:37Daniel Whiteson?

  ...to rising stars like Alex Warren.

• 42:40Speaker 04

  You know, a song is a story, and the verse is setting the scene. And for me, at least, it's allowing me to be personal and vulnerable with the audience, whether it's about my wife and I'm being vulnerable about her, or it's about, you know, my parents and I'm being vulnerable to them. And it's almost like a cinematic piece in a way where each song tells that beautiful story.

• 42:58Daniel Whiteson?

  It's gonna be one hell of a trip. Experience Before the Moment, a Lincoln original series on the iHeartRadio app, YouTube, or wherever you get your podcasts.

• 43:11Speaker 04

  I'm Malcolm Gladwell, host of Smart Talks with IBM. I spoke with Dr. Laura Jayhai of Cleveland Clinic about how advanced computing is helping shorten the path from medical discovery to treatment. The moment that there is a compound that someone in a lab thinks it would be effective to treat a certain disease, it takes about 10 to 13 years from that moment to when that drug is on a shelf for a patient.

• 43:39Kelly Weinersmith?

  A big part of that gap could be computationally solved. Better simulation of compounds, designing drug trials that are more efficient, that would finish faster. That was the motivation to bring computational tools and technology closer to biomedical researchers. Learn more about how Cleveland Clinic and IBM are using AI and quantum computing to revolutionize biomedical research at ibm.com/smarttalks.

• 44:21Kelly Weinersmith?

  All right, so now we have all of this data and you've got it into an ensemble and you sort of maybe know what's happening right now, plus some uncertainty. How do you now predict what's going to happen next?

• 44:32Daniel Whiteson?

  Yeah, so it's simple. You just break out your pencil and paper and you do a bunch of string theory calculations and that's it, right? It's just like physics it into the future.

• 44:40Kelly Weinersmith?

  Finally, string theory is useful.

• 44:43Daniel Whiteson?

  Yeah, unfortunately not. As we said earlier, like you can't describe everything. You have to make assumptions about what you're going to calculate and what you're going to simplify. Otherwise, you're never going to be able to make a prediction, right? Or your predictions are going to be done in a thousand years for the weather that's happening in an hour and that's not useful. And so it's always a question of how to judiciously make those assumptions. So the current state of the art for weather modeling has basically two big pieces. One is directly model the atmosphere itself as if it's a big fluid. So use like Navier-Stokes equations and think about how it flows and how temperature moves through it. That's the like dynamical core of the model.

• 45:24Daniel Whiteson?

  But there's a bunch of stuff that influences the atmosphere that you don't explicitly include in the model. The clouds, the convection, the ocean, the radiation, the surface temperature, all this kind of stuff. Your model doesn't explicitly include that stuff. We don't have like a complete model of the ocean or the clouds, et cetera. And so we have like various inputs to this core piece that they call parameterizations that like capture the big picture effects of all these pieces that are not included directly in our model but are influencing us.

• 45:57Kelly Weinersmith?

  So I feel like this is a question where you think to yourself, am I about to ask a really stupid question? But I'm going to move forward because that's my job in this podcast. The atmosphere. Is not a fluid, though, right? And so, like, am I wrong about, so why? Why are we do it? Are we modeling it as a fluid because we just can't model it as something else? Because it's too complicated and fluids are a simplification? Or, Daniel, I don't think the atmosphere is a fluid.

• 46:29Daniel Whiteson?

  Well, it depends on what you mean by a fluid. And, you know, when it comes to like how things flow and pressure, et cetera, the fluid dynamic equations do describe the atmosphere. And so, you know, fluid doesn't mean liquid, right? Fluid is about how things flow and move, right? So, for example, like, the mantle of the earth is a fluid. It flows. It's not a liquid, right? It's this weird, solidy kind of state and it moves, but it also flows. And so you can describe it, and it has convection. You can describe it with fluid equations. And so, the Navier-Stokes equations are these famous equations that describe fluid dynamics. And they're pretty good at modeling the atmosphere. They're not perfect, right? They're not perfect, but they're pretty good at it. So, yeah, I think fluid is not a liquid.

• 47:14Daniel Whiteson?

  It's just things that flow.

• 47:16Kelly Weinersmith?

  Okay, in my head, fluid is synonymous with liquid. And so I have learned something today that will probably help me not look silly in the future. That's great.

• 47:24Daniel Whiteson?

  No, it was a great question. And so this is the big picture. You have the dynamical core, and then you have these parametrizations. And we'll dig into that. And we'll describe sort of the US approach to it. But there are three sort of major weather communities. There's the US, the UK, and the Japanese. And they have slightly different approaches, which is good because, you know, different predictions can cross-check each other. But then some people think it's bad because, hey, let's pool all of our resources and make one big global model. And that's awesome, but then you only have the one and you're not sure. Maybe it's all wrong. There's a lot of debate about, you know, how to deal with global questions and global resources.

• 48:00Kelly Weinersmith?

  But who's the best?

• 48:03Daniel Whiteson?

  Oh, I'll give you a ranking at the end. Okay. All right. So the dynamical core, right? Think of the atmosphere. We're going to treat the atmosphere basically like a spherical cow, right? It is a spherical fluid, right? I guess this is physics. Yeah, it's a thin shell around the earth. And you know, the temperature and the pressure. And then you can describe how it's going to flow, how the temperature and pressure are going to change using the Navier-Stokes equation. So Navier-Stokes is a set of really gnarly equations. They're nonlinear, partial differential equations. A differential equation is one where like the value depends on how quickly it's changing. For example, like in ecology, you have differential equations that describe like predator and prey.

• 48:48Daniel Whiteson?

  Right? These two things are coupled. And so these are nonlinear, partial differential equations, which means like they depends on things squared or cubed. All that to say, they're very, very difficult to solve. In fact, differential equations in general are hard to solve. If you've taken a differential equations class, it's basically like differential equations are not solvable, except for these four examples that we have answers to. And we know how to solve them. And so you just got to memorize those. It's a little bit like chemistry, I gotta say. Oh, no. It's mostly unsolved, right? And the Navier-Stokes equations, we've known about them for like 200 years. They were initially developed to try to answer these questions about like, how do things flow? And how does momentum and mass flow through pipes, et cetera?

• 49:30Daniel Whiteson?

  Essentially, people took Newton's second law, F equals ma, and applied it to fluids. And then added terms for like stress and pressure and viscosity. And it's like a real triumph that we can describe this at all. But calculationally, it's a real bear. You can't like sit down and derive a solution and say, here is my pressure and temperature. Let me crunch it through the Navier-Stokes equation. It's going to give me a formula. It's all numerical approximations, which means it takes a lot of computing to go from now to one second from now or two seconds from now. And that computing means approximating things. You're like doing numerical derivatives instead of exact analytical derivatives.

• 50:10Kelly Weinersmith?

  Okay, so some of that got pretty complicated. But what I guess what I want to know is when this is all done, I feel like if we are trying to model fluids, does this just tell us that like the wind is now over here going this fast, but before it was over there? And how many are we going to get to like how you get from that to like, and it's raining? Because that seems like a different problem, sort of, than how fluid is moving around.

• 50:33Daniel Whiteson?

  Yeah, so there's a couple of things to know there. You're exactly right. It takes the current conditions and tries to predict the future conditions. And those conditions are pressure and temperature, wind speed, humidity, right, these kinds of things. But because we're solving this numerically, we can't solve it everywhere. If you have a formula, an analytic description, you can write down for like, where is my ball as I've thrown it? I can write that down on a piece of paper. I have a formula that can tell you where the ball is at any point in time. You ask me any point, literally any value of t, I could plug it into my formula, give you an answer. But if I don't have a formula, that's called an analytic description. If all I have is a numerical estimate, then I've made a grid. I've said, I'm going to sample it at time one, time two, time three, time four. I'm going to make an estimate at those times.

• 51:17Daniel Whiteson?

  And I don't have an answer everywhere. And that's the situation we have with weather is that they put a grid on the planet and they estimate what's going to be the weather, temperature, et cetera, in a grid of points, not everywhere over the planet. And you might think, oh, I bet that grid's pretty small, right? Maybe they measure it down to the centimeter or something. No, the grid sizes are like 10 kilometer cubes.

• 51:40Kelly Weinersmith?

  What?

• 51:41Daniel Whiteson?

  Yes.

• 51:41Kelly Weinersmith?

  That's too big.

• 51:42Daniel Whiteson?

  It's too big, right? They are averaging the temperature and the humidity over cubes of atmosphere, 10 kilometers on a side. It's crazy. And you might think that's way too big. On the other hand, that's still a lot of cubes, right? Like, the atmosphere is a lot of 10 kilometer sized cubes. And then the time steps are tens of minutes.

• 52:02Kelly Weinersmith?

  Okay.

• 52:03Daniel Whiteson?

  Right? And this is awesome that we can even do this. It requires massive supercomputers. We'll talk about it in a minute. But the problem is that it ignores a lot of little details, like how big is a cloud? Usually they're like a kilometer or less. And so you're missing out on a lot of stuff. By making your grid, anything that happens that's sub-grid, that's crucial and important, but is smaller than the size of your grid is not being described by your model. But your question was like, is this directly outputting like, hey, it's going to rain on Kelly's picnic? In a sense, yes, the direct outputs are things like temperature, pressure, humidity, and those are enough to tell you like, okay, it's going to rain because the pressure and the humidity

• 52:44Daniel Whiteson?

  are above some threshold or whatever. So it's not directly outputting like three centimeters of snow. There's another step you have to take after that, but it feeds into that. So those are the inputs you need to the next step, which says how much snow is going to fall.

• 52:57Kelly Weinersmith?

  Okay, so let me see if I can do a super simplified version of this. You get all of the data that you have about a square in the grid, and you do the best job you can to sort of summarize it and ensemble it, and then you put it in the model. The model runs through the equations. Then does the information from the surrounding grids feed into your grid as well?

• 53:19Daniel Whiteson?

  Oh, yes.

• 53:19Kelly Weinersmith?

  Okay, because you would expect there to be similarities between closely related squares in the grid.

• 53:24Daniel Whiteson?

  Yeah, you can't solve one grid at a time. You have to solve all the grids.

• 53:27Kelly Weinersmith?

  Holy cow.

• 53:28Daniel Whiteson?

  Because the grids touch each other and influence each other, and wind flows, right?

• 53:32Kelly Weinersmith?

  Right.

• 53:32Daniel Whiteson?

  And that's why Siberia affects Manhattan over time, because you've propagated these things from grid cell to grid cell. Absolutely.

• 53:39Kelly Weinersmith?

  So does Siberia have bigger grid cells or just the same number of small grid cells, each with poorer data in them?

• 53:47Daniel Whiteson?

  Yeah, great question. So some of these models are adaptive, right? They have bigger grid cells where you have more uncertainty and smaller where you have more data. The most precise ones are the UK supercomputers that go down to two kilometers in some cases. Some of them are like fixed grids and some of them are adaptive. Exactly. It depends a little bit on the model. But you know, there's lots of details that are not described here, and these are called the parameterizations, like especially sub-grid stuff and exchanges with other parts of the system that are not just the fluid. And one important thing, are the clouds. Like, you cannot model every individual cloud because clouds are smaller than your grid size. We do not have the compute to do that. People have tried, and you can like do dedicated runs.

• 54:29Daniel Whiteson?

  On subsets to try to resolve clouds, but then you don't have enough computing to do like ensemble. So you make one prediction, you're like, well, here's a prediction, but I don't know what the uncertainties are on it at all. And so instead, what you tend to do is parameterize the bulk outcomes. You know, the vapor, the clouds, the liquid, the ice, the rain, the snow, et cetera, the condensation, all this kind of stuff. You try to like, grab all of that average over what's happening in that grid cell and use that to inform your Navier-Stokes equation. So things you're not explicitly modeling, you're sort of like averaging over. You're losing all the details and saying like, well, on average, this is going to be the effect of clouds on my grid cell.

• 55:10Kelly Weinersmith?

  And do you think that as, oh, well, I was gonna say, do you think that as we continue to have more and more computing power and more and more supercomputers, at some point, we'll, we'll be doing better here. But, but we were just talking about how Moore's law, we've maybe hit the end of that. So are we like, is this as good as we're ever gonna get at it? This is probably an end of the podcast question, but I'm thinking it right now.

• 55:33Daniel Whiteson?

  No, I think that there's lots of possibilities for making this faster and more efficient. And not just wait till computers get faster.

• 55:40Kelly Weinersmith?

  Okay.

• 55:40Daniel Whiteson?

  There's definitely clever ideas and we'll get there. Yeah.

• 55:42Kelly Weinersmith?

  All right.

• 55:43Daniel Whiteson?

  But there are lots of parts of the weather that are not directly described in the dynamical core. So not just the clouds, but also things like convection, like vertical transport of heat, you know? Especially this is complex boundary mixing near the surface, like the lowest kilometer or so of the atmosphere, where you have like heat from the surface and turbulent momentum exchanges as wind is like hitting mountains and stuff. These things, you can't model all of those details. And so you have like parameterization schemes that model the turbulence and the boundary level mixings. There's radiation from the surface also, right? That changes from day to night. You have models of vegetation and snow, how those things couple.

• 56:27Daniel Whiteson?

  But then the biggest one is the ocean, right? Like, we would love for our models to include also a Navier-Stokes simulation of the whole ocean, right? Might as well do that because the ocean, it plays a big role, but we don't have. Compute for that at all. So we just like use a slab ocean model and just say, let's just assume the ocean is like simple and we have a certain temperature and we assume like how the energy transfers from the boundaries and it's really quite simplified, but that's we're just limited, right? We don't have great data in the ocean and we don't have the compute to also model the ocean as well as the atmosphere. So places where we don't do our best approximation, which is like Navier-Stokes equations, the atmosphere, we have simplified versions,

• 57:10Daniel Whiteson?

  which are called parameterizations, they feed in to the core. But in the end, you got to take it to the computers. And this is why we have like massive supercomputers to make weather predictions. So NOAA in the US has a couple of really big facilities. They're called Dogwood and Cactus. One of them is in Virginia.

• 57:28Kelly Weinersmith?

  You're welcome, everyone.

• 57:29Daniel Whiteson?

  And one of them is in Arizona. And they're huge, amazing computers. Those two have like 12.1 petaflops.

• 57:37Kelly Weinersmith?

  You made that word up.

• 57:39Daniel Whiteson?

  Sounds like a made up word. Peta means quadrillion and flops are floating point operations. So, you know, it takes the computer time to add like 3.91 to 14.42. And floating point numbers, those numbers with a dot in them, right, not integers, are more computationally expensive to add or subtract. And that's what most of these models do. They're like, add this number, multiply by this. And so this is like the way you measure the speed of a computer. And so these computers can each do 12.1 quadrillion floating point operations per second.

• 58:14Kelly Weinersmith?

  Wow.

• 58:15Daniel Whiteson?

  Right? Imagine the guys back in the 1920s, they're like adding two numbers. It probably takes them a minute, right? Or they're super good, it takes them 20 seconds. The computer does 12 quadrillion of these per second, right?

• 58:27Kelly Weinersmith?

  Amazing.

• 58:28Daniel Whiteson?

  So together with all of their computers, NOAA has about 50 petaflops. And that's what it uses to run its model. And so that's the state of the art in the United States. The Europeans have a couple of computers. There's one really big one in Bologna called Bull Secuania and has about 30 petaflops. But the biggest, most powerful

• 58:50Daniel Whiteson?

  Weather computer in the world is in the UK. It's at the Met Office and it's built by Microsoft and it has 60 petaflops. And this is why the UK has some of the best weather prediction in the world.

• 59:02Kelly Weinersmith?

  What?

• 59:02Daniel Whiteson?

  Because they have the biggest computer. They beat us. Yeah, exactly. They just spent more money. They bought more compute. This is literally like money equals computing power.

• 59:11Kelly Weinersmith?

  Those tea drinking bastards.

• 59:13Daniel Whiteson?

  How dare you. Yeah. Well, you know, they got tricky weather over there and so they need it. It's an island after all.

• 59:20Kelly Weinersmith?

  Good job, guys and gals.

• 59:22Daniel Whiteson?

  The Japanese, the Japanese have a big investment in weather prediction computers also. There's one called Prime HBC. It has 31 petaflops. So these are really powerful devices and they run these huge models. And you know, think about what the model does. It predicts the state of the atmosphere on these pretty chunky grids, but it's still, it's a huge amount of data. Like every few minutes, every 10 kilometers. My friend Jane was telling me that sometimes the data is so big that you just throw it away. You run it, you get like a summary number, but you can't keep all of the data because it would just like fill up all of the hard drives everywhere. And this is familiar for me because like at the LHC, we also, we run these experiments every 25 nanoseconds.

• 1:00:03Daniel Whiteson?

  We throw away most of the data from that because it would just fill up all of our storage.

• 1:00:08Kelly Weinersmith?

  Wow.

• 1:00:08Daniel Whiteson?

  And they're in a similar situation. They produce more data than they can store.

• 1:00:12Kelly Weinersmith?

  So are these facilities where the Navier-Stokes equations are being run? Or are these facilities where you have the output from each grid? And now you are translating that into information about where the rain is falling?

• 1:00:27Daniel Whiteson?

  Both, yeah.

• 1:00:29Kelly Weinersmith?

  Okay.

• 1:00:29Daniel Whiteson?

  So these programs do the data simulation, come up with the current initial conditions and then also run the model forward to make those predictions. And from that glean things like weather details, snowfall, et cetera. And so what you're getting on your phone, what you're hearing on TV is not just like what Jane, your local forecaster came up with. She's relying heavily on these central predictions from major resources. Right? So for example, if worldwide governments decide, we don't need these computers anymore. We don't need these satellites. It's not like you could be like, that's cool. I got my local weather forecaster. I don't need you. No, your local weather forecaster is getting that information from these big models that are being run by the government.

• 1:01:12Kelly Weinersmith?

  Oh, wow.

• 1:01:13Daniel Whiteson?

  Yeah.

• 1:01:13Kelly Weinersmith?

  And did NOAA get cuts recently? I'm gonna bet they did.

• 1:01:18Daniel Whiteson?

  There were some talk about cuts. I don't know how much of that is going through. It's all kind of scary. It's hard to know. Yeah.

• 1:01:24Kelly Weinersmith?

  Okay. All right, we won't get into that. Moving on.

• 1:01:27Daniel Whiteson?

  But amazingly, currently, we can pretty accurately predict the weather five to six days in the future. You know, and you mostly remember when the weather prediction is wrong. You mostly don't realize that most of the time it's right.

• 1:01:40Kelly Weinersmith?

  Yeah.

• 1:01:40Daniel Whiteson?

  You know, it tells you it's gonna rain. It tells you it's gonna be sunny. It's mostly correct. It's amazing. But, you know, there's still challenges. Things are not perfect. One of the biggest challenge is just incomplete information. You know, we don't have sensors in enough places and we don't have enough sensors. And sometimes this data availability changes. You know, things go offline or come online. Now your model has to compensate for that. I don't have the data. Do I assume it's similar to the past? Do I try to ignore that kind of data? It's not easy to be running a model if the inputs are constantly changing.

• 1:02:12Kelly Weinersmith?

  The bucket had a hole in it, so now you don't have good bucket data.

• 1:02:15Daniel Whiteson?

  Exactly. Exactly. They got a new kind of bucket. You don't know how to calibrate it. I spoke to John Martin. He's a professor of meteorology. And he said that this might be the biggest challenge is how to combine the data to make a high quality initial state. That's one challenge. The other are these subgrade parameterizations. Can we develop better models for turbulent flow at the boundaries or for latent heat released back into the environment? And another limiting factor is just the computing cost. More computes, more GPUs from Nvidia means smaller grids, which means the effect of these approximations, these parameterizations is less. Another continuing challenge are rare and extreme events. Like, we're pretty good at predicting the bigger picture.

• 1:03:00Daniel Whiteson?

  Like, is it gonna be sunny here? Is it gonna be rainy here? But like, small, rare, extreme events, like there's a tornado right here. That's more challenging because it depend in detail on things that happen within the grid that we're averaging over. And so there's a lot of work being done right now. One thing we're hoping to do is like, let's reduce the grid size, get more computing, more accurate, right? But another really promising area of research is using machine learning. There's this movement in many fields of science to use machine learning to make predictions by essentially skipping the physics. Like, the physics is hard. It takes a lot of time to push the initial conditions through these equations. In the end, you have an input and an output.

• 1:03:44Daniel Whiteson?

  And the idea is, well, can we train machine learning, not a chatbot, not LLMs, right? It's AI, but it's not LLMs, to map the initial conditions to the output? Because in the end, it's just a mapping, and one could learn it. And so we have these machine learning models that are simple functions that take the input and give you the output. And they don't have the physics encoded in them, but they learn from the simulations. They learn the patterns. They learn what the rules are implicitly. And so you don't have to go through all the detailed calculations. So this can dramatically speed up your predictions. We use these at the Large Hadron Collider all the time, so that we don't have to, for example, model every single particle that might hit the detector and create another particle, another particle.

• 1:04:28Daniel Whiteson?

  We can learn to predict the final thing we're interested in and just sort of leapfrog over all the tiny details.

• 1:04:35Kelly Weinersmith?

  And is machine learning being used right now for weather predictions? Or they're just starting to work on how you would do that?

• 1:04:40Daniel Whiteson?

  They're using that now. They're sort of experimental, but there's a guy here at UC Irvine, Mike Pritchard, who is an expert in this kind of stuff, and is very powerful, absolutely.

• 1:04:49Kelly Weinersmith?

  Cool.

• 1:04:49Daniel Whiteson?

  Yeah. So I asked John Martin, if I gave you a billion dollars to improve weather predictions, what would you do? And he said he would spend a billion dollars on ocean probes. Like, he wanted a more substantial understanding of how water is circulating in the ocean, and temperature in the ocean, and how that's all working. Because his suspicion was like, we're right next to this other big fluid that's affecting our temperature, and we don't have much enough data about it. We just knew more about the ocean. And this just highlights like, how little information we have. It's not just a question of like, puzzling out the rules of the universe, but just like, knowing what's happening. If we had more data everywhere about temperature, pressure, about cosmic rays, we would just learn so much about the universe. And we have so few ways to probe it.

• 1:05:34Daniel Whiteson?

  We're really just like, taking the tiniest teaspoon out of this massive river of data, and trying to use that to understand the whole river. It's crazy.

• 1:05:43Kelly Weinersmith?

  How good do you think weather prediction would have to be before people stopped complaining about weather prediction?

• 1:05:48Daniel Whiteson?

  I asked John that question, and his prediction was, quote, "The complaining will never stop."

• 1:05:53Kelly Weinersmith?

  Amazing.

• 1:05:54Daniel Whiteson?

  I think that, you know, weather prediction has improved a lot over the last few decades. It used to be you couldn't get any reliable prediction more than a day in advance. Now, five, six days is pretty reliable. But people expect that and they get used to it. And they're like, what? You didn't predict the weather on my ski trip in two weeks? I'm mad at you.

• 1:06:11Kelly Weinersmith?

  Yeah.

• 1:06:11Daniel Whiteson?

  And so, yeah, the complaining will never stop because we always just get used to the level of technological prowess that we've had. And, and so people want more because it's so important. And it's a hard problem. There's so much physics here. There's instrumental science. There's so many different kinds of science that interface with each other. It's really an exciting field.

• 1:06:31Kelly Weinersmith?

  Yeah.

• 1:06:31Daniel Whiteson?

  And let me throw a special thanks to Professor Jane Baldwin here at ECI, who told me a lot about weather predictions, and Professor John Martin at Wisconsin, who answered a lot of naive questions of mine. Thanks to both of you.

• 1:06:42Kelly Weinersmith?

  Thank you, community. All right. See you all next time. I hope the weather is nice where you are.

• 1:06:46Daniel Whiteson?

  It always will be nice where I am.

• 1:06:57Kelly Weinersmith?

  Daniel and Kelly's Extraordinary Universe is produced by iHeartRadio. We would love to hear from you.

• 1:07:02Daniel Whiteson?

  We really would. We want to know what questions you have about this extraordinary universe.

• 1:07:08Kelly Weinersmith?

  We want to know your thoughts on recent shows, suggestions for future shows. If you contact us, we will get back to you.

• 1:07:15Daniel Whiteson?

  We really mean it. We answer every message. Email us at [email protected].

• 1:07:21Kelly Weinersmith?

  Or you can find us on social media. We have accounts on X, Instagram, Blue Sky, and on all of those platforms, you can find us at D and K Universe.

• 1:07:31Daniel Whiteson?

  Don't be shy. Write to us.

• 1:07:35Kelly Weinersmith?

  With ChatGPT, you can do more than get answers. You can get things done. Whether you're leading a big project, planning an elaborate trip, or trying to balance work and life, ChatGPT can help. Give it a goal, plus things like files, notes, or project plans, develop a trip itinerary with daily updates, or create a prioritized schedule so you can spend less energy sorting out the details and more on bringing your biggest ideas to life. Get started at chatgpt.com. Hey, this is Mario Lopez from all Mario Lopez Daily Podcast. Buy smart, drive for less and trade for more. That's Toyota affordability. And it starts with a great deal on an all-wheel drive Camry Corolla Cross, or a Corolla sedan with the latest features in tech.

• 1:08:18Kelly Weinersmith?

  Or test drive a spacious Grand Highlander or adventure-ready RAV4. And with so many hybrid models available, you can enjoy longer trips without all the stops. While Toyota's legendary resale value means you could get more at traded. So get a great deal today and great value tomorrow. Shop toyota.com for deals and details. Toyota, let's go places. Cash flow crunch? OnDeck's small business line of credit gives your business immediate access to funds, up to $200,000, right when you need it. Cover seasonal dips, manage payroll, restock inventory, or tackle unexpected expenses without missing a beat. With flexible draws, transparent pricing, and control over repayment, get funded quickly and confidently. Apply today at ondeck.com. Funds could be available as soon as tomorrow.

• 1:09:02Kelly Weinersmith?

  Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval. This is an iHeart Podcast. Guaranteed human.