AI’s breakthrough in weather forecasting with Brightband’s Julian Green

In this episode of Gradient Dissent, Julian Green, Co-founder & CEO of Brightband, joins host Lukas Biewald to discuss how AI is transforming weather forecasting and climate solutions. They explore Brightband's innovative approach to using AI for extreme weather prediction, the shift from physics-based models to AI-driven forecasting, and the potential for democratizing weather data. Julian shares insights into building trust in AI for critical decisions, navigating the challenges of deep tech entrepreneurship, and the broader implications of AI in mitigating climate risks. This episode delves into the intersection of AI and Earth systems, highlighting its transformative impact on weather and climate decision-making. 🎙 Get our podcasts on these platforms: Apple Podcasts: http://wandb.me/apple-podcasts Spotify: http://wandb.me/spotify Google: http://wandb.me/gd_google YouTube: http://wandb.me/youtube Connect with Julian Green: @juliangreensf Follow Weights & Biases: https://twitter.com/weights_biases https://www.linkedin.com/company/wandb Join the Weights & Biases Discord Server: https://discord.gg/CkZKRNnaf3

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Lukas Biewald?

  You're listening to Graded Descent, a show about making machine learning work in the real world, and I'm your host, Lukas Biewald.

• 0:11Lukas Biewald?

  Today, I'm talking with Julian Green, who is a serial entrepreneur who's founded at least three companies, Houz, Jetpack, and Headroom, and has also been the GM of AI Moonshots at Google X before founding his current company, Brightband. Brightband's mission is big, is to provide advanced Earth Systems AI to improve weather and climate-related decisions for the benefit of humanity and Earth. And they're taking on a really interesting domain of weather forecasting. This is a cool interview because I get to talk to him about being an entrepreneur in deep tech, as well as a really interesting use case of AI predicting weather. I hope you enjoy it.

• 0:51Lukas Biewald?

  All right, so welcome to Graded Descent. I think one of my favorite kinds of people to have on this podcast is entrepreneurs that are working on a specific, tangible problem that we can all understand. And, you know, we've had people working on industrial farming, we've had people working on designing circuit boards, we've had so many different categories of this genre of person, but we actually haven't had anyone working on climate. And so I wanted to ask you, first of all, what attracted you to the climate space? And why do you feel like this is such a meaningful problem to work on?

• 1:29Speaker 01

  Yeah, well, you know, growing up in the UK, maybe it's just because I miss weather in California most days, you know, so it's often sunny and nice, but even in California, you know, we have atmospheric rivers and all sorts of things. You know, I grew up in Oxford and I had a summer job, or a friend had a summer job, I helped them with there. And we would go and take readings, thermometer readings and sort of rain gauge readings at a weather station in the UK. And I didn't realize it at the time, but I went back and looked and actually this place, the Radcliffe Meteorology.

• 2:06Speaker 01

  Station has had a continuous rain and temperature, you know, stream of data gathered since 1813. And little did I know that, you know, when I didn't want to get out of bed in the summer. I'm glad I did. We would have broken a, you know, a long streak. And then, so it's always been interesting to me. I think what has, with the increasing the extreme weather that we see, I think it's really important that we have the tools to be able to, you know, adapt to that extreme weather. And then if possible, to work out how to mitigate those effects. And I think that's a joint responsibility we all have.

• 2:48Speaker 01

  And I think AI promises to do amazing things there. So, so it's exciting. I mean, some of the stats on the increasing the extreme weather are mind boggling. And we've all seen this summer of various hurricanes. But, you know, back in the 80s, there was a billion dollar disaster every four months in the US. Now there's one every few weeks.

• 3:06Lukas Biewald?

  Is that right? Every few weeks?

• 3:09Speaker 01

  Yeah. And actually, I mean, the ones this summer are not just billion dollar disasters. There are tens of billions of dollars disasters. And these are real lives and real things being destroyed. The World Bank thinks if we could just, one part of weather, if we could just have more timely and accurate notifications for extreme weather, like hurricanes, tropical cyclones, and so on, we could save 162 billion dollars a year in people not dying and assets not being destroyed.

• 3:37Lukas Biewald?

  So what's the state of the art today? And where do you think you could take things?

• 3:43Speaker 01

  Yeah. So the exciting thing happening with AI in weather has been that there's been this amazing research. I guess, I guess backing up slightly, you know, AI has been used for a while in weather, particularly to downscale lower resolution forecasts. And I think of that just like, you know, with computer vision, you know, sort of super resolution, interpolating between.

• 4:15Speaker 01

  The data points you have, and it's been used for some data processing on the front end. But the thing that's happened, which one of my co-founders wrote a seminal paper on in 2022, Ryan Kiesler, is to use AI as a time machine to forecast weather from initial conditions, and to do that in a learned way, rather than the physics approach that numerical weather prediction's been using for, you know, 50 years, really since the dawn of computers. And that turns out to be faster, cheaper, and more accurate. So it's sort of crazy that relatively out of the box AI that we've been using has been able to do such an impressive job. And I think it's very exciting where that will take us.

• 5:00Lukas Biewald?

  But I guess, like, tangibly, I'm not even sure what the state of the art of weather forecast is today. Like, you know, I think everyone's probably experienced opening up an app like Dark Sky or Apple Weather and seeing the weather prediction for tomorrow or the next day. Like, has that gotten gradually better? Is that something that your techniques would help with? Or are you really focused on extreme weather forecasting?

• 5:26Speaker 01

  Yeah, so how good is weather forecasting, right? I mean, I think the easy answer is not as good as we'd like it to be. You know, what has happened is each decade, you know, since the sort of use of computers for numerical weather prediction, we've got one more good day of accuracy each decade, basically, right? So, you know, if you think about, and by good, we sort of mean better than climatology, right? So one way to do weather is you look in the Father's Almanac, and it says today is November 4. And usually, on average, on November 4, it's this temperature, you get this much rain, and you know, sort of a long-term average approach. And you're looking for, for how many days in the future can

• 6:11Speaker 01

  you do a better job than climatology or other, you know, techniques? Another one is persistence, right? So you just say, well, today is this temperature, this, you know, this weather, and so we assume tomorrow's going to be like that, right? So you look for where you can do better than those other methods. And so each day since, you know, since really using computers, we've sort of got one day better. But that means that now-

• 6:31Lukas Biewald?

  Wait, that since 1980 or 1970, where are you starting?

• 6:35Speaker 01

  Well, computers have been, yeah, I mean, used in weather since sort of the 50s, right?

• 6:38Lukas Biewald?

  The 50s.

• 6:39Speaker 01

  Yeah, and the approach has been sort of-

• 6:40Lukas Biewald?

  So we're at seven good days of accuracy? Or seven days that you're better off kind of using this method versus just a baseline average?

• 6:49Speaker 01

  We're using broad brushstrokes here. Obviously, for different things, you can do better. You know, like, temperature, you know, might be easier than exactly how much rain are we going to get wet, right? But sort of roughly speaking, you know, yeah, seven to 10 days, our forecasts aren't bad. Now, there's a nuance there, that that relies upon having good observations and good models. And poor countries, poorer countries, who don't have as many observations and who don't have purpose-built models, their one-day forecast is as good as our seven-day forecast. So that's sort of crazy to think about. So it's very inequitably, you know, distributed, good weather forecasting. But, you know, if you think about all the things you want to do,

• 7:33Speaker 01

  and all the decisions you have to make that are dependent upon both weather and then climate, you know, having, you know, somewhat interesting forecasts only out a week is not as good as we would like. We would like to have much more, you know, local, specific forecasts. We would like to be able to forecast further out into the future. And particularly with things like extreme weather events, where really, you know, things are being destroyed, people are dying. We would like to be able to do a better job.

• 8:06Lukas Biewald?

  And Tor, are you focused on these extreme weather events?

• 8:11Speaker 01

  Well, you know, we're building our approach, and that's definitely one of the most high-value, you know, weather forecasting bits. So yes, we'll be looking at extreme weather. But, you know, we will see what it turns out to be best at. And, you know, AI has been used quite a lot in nowcasting. So sort of what the weather is going to be like in the immediate future, where the initial conditions- Are pretty important. And then, you know, the crazy thing has been that as we develop better AI tools, it's been starting to be better than the physics models, which are run, you know, we've spent 50 years getting good at this with the partial differential equations

• 8:52Speaker 01

  that run on billion dollar supercomputers, you know, but yet AI is now looking, you know, better from a sort of root mean squared error basis, you know, five days, seven days, 10 days, even some signs that it might be better further out than that. Sort of crossing some lines that we thought had been drawn by maths for chaotic systems, where you may not be able to look further out in the future than some time, you know, Lorenz, you know, looked at this decades ago. And so it's, it's super exciting that AI seems to be able to find signal, you know, beyond syllabus limits.

• 9:25Lukas Biewald?

  Okay, you're immediately pulling me off script, but that is actually a question I, I wanted to ask you. I remember, I remember by college partial differential equations class, the example was weather, and I think it was even that butterfly example of how, you know, tiny changes in initial conditions can clearly lead to wildly chaotic output, and therefore it's probably impossible to predict the weather, you know, beyond a certain region. So, so what's, what's changed or what was wrong about that theory?

• 9:58Speaker 01

  The butterfly effect is real, right? And so, you know, that's, that's sort of a mathematical truth. But, you know, ask yourself how many times when a butterfly flaps its wings, you know, does it produce a, does it produce a hurricane, right? And the answer is not that often. And so really, one of the beautiful things about using AI for weather forecasting is, is inherently probabilistic field. And because you can run inference so much faster and cheaper, you could run a lot of scenarios, a lot of ensembles. And, you know, so right now physics laws are sort of constrained to about 50 production ensembles. And so if you're looking for, you know, a one in 100, a one in a thousand storm,

• 10:40Speaker 01

  and you're only running 50 ensembles, like the chances that you capture, you know, that particular scenario are relatively limited. So that again, is, is a lot of promise there that you can run so arbitrarily large numbers of scenarios, really get that whole distribution and zero in on sort of the anomalies that matter.

• 11:01Lukas Biewald?

  Although it's funny, like if you, you know, if you could perfectly know somehow, and I guess this even gets into the theory of probability or something, but you know, if you perfectly knew that there's like a 2% chance of a hurricane, you know, hitting Washington in a week, it's hard to know what you would even do with that information, right? Like, you know, kind of putting median probabilities on stuff, you know, might lead to, um, you know, fatigue of, of, um, emergency forecasting and things like that, right?

• 11:33Speaker 01

  Yeah, I mean, you know, so what can one do? So let me give you a couple of thoughts on that. So, you know, Hurricane Otis hit Acapulco in, in 2023, right? And it did $15 billion worth of damage and it killed 50 people. And they had four hours notice. If they'd had, you know, the World Bank estimates that they'd had more like one or two days notice, they could have saved most of the lives. Cause you can move people to, to safety and you can board up windows and you can move cars and you can move boats and you would have saved maybe a third of the, of the damage, right? So five of the 15, but still gonna hit stuff, but you can sort of choose how to protect that stuff. Um, and so, you know, you can do some things. Now, the, the big challenge, and you know, we see this when hurricanes are circling

• 12:15Speaker 01

  through the Gulf of Mexico, approaching the shoreline, you're like, where's it gonna hit, right? You can't evacuate, you know, all of Texas and Florida or the other states. Um, and so, uh, the more accurate you can be, and actually some of the models for Milton got it within 13 miles, like days out, right? And so then you can say, uh, you know, depending upon how, how things are set up, you know, we have enough time, cause it takes days to get people out of a big city. So we have enough time and enough accuracy to be able to evacuate, um, a manageable number of people in a manageable amount of time.

• 12:54Lukas Biewald?

  Okay, so you've been a serial entrepreneur in a whole bunch of different technical domains, including, I remember Jetpack and Houzz, obviously a well-known company, and then you worked at Google X on various moonshot projects. I'm curious what you're taking from that experience into this domain? And what's already different about this domain where you're operating differently than previous companies?

• 13:22Speaker 01

  Yeah, that's what I'm saying. I mean, this is my sixth startup, right? And so the long-term lessons that apply to all the different things, whether it's books, or bulldozers, or ecommerce tools, or tickets, or home design inspiration, or travel inspiration, or AI video conferencing, or all the different things. But Don, you've got to understand where you're making value for users. And so just really being focused on that. With AI, in the last 15 years that I've been doing AI startups and working at Google on AI.

• 13:54Speaker 01

  You know, AI is amazing at the first 80%, right? And so ChatGPT, like one of the most extraordinary technical demos ever done. And then, how do you turn that into a product? And so the last 20%, when you're using a learned approach, the edge face is clear, right? And so you've really got to think about very, very keenly think about what problem are you trying to solve that the AI technology is going to be well-suited to solve it? And you pick that right. And then it's got to be one that people care about and that's valuable and that's useful. And you've got to work out how you deliver that.

• 14:33Speaker 01

  And I think with LLM, it's extraordinary at generating reasonable-looking language very fast. And so for situations where that is a benefit, where the alternative is slow and painful and not very good quality anyway, then that's amazing. And then more objective. Cases where, you know, we did the started the generative coding stuff at Google. And, you know, thinking about, you know, it's a very structured bit of language. There's a great data set of peer reviewed, you know, data. And you've got to be able to run the code.

• 15:14Speaker 01

  So you have some sort of objective measure of how accurate the language is. That's, that's a great case. Now for weather, you know, it's incredibly data rich. We have, you know, 100 petabytes of cumulatively, you know, gathered historical weather data. And crazily, we're about to get 100 petabytes every year going forward. So the advent of these very high definition, you know, radars and satellites and sort of the scale up of, of observation means that there's tidal wave of data. And you get another day of ground truth every day. Unfortunately, no faster, but you know, you can find out whether your prediction is good and you can go back and you can, you can hink us.

• 15:57Speaker 01

  So I think, you know, it is very well suited. And, and I've talked about how, you know, weather is sort of inherently probabilistic, right? And the techniques that you can use, the statistical techniques you can use, which will give you a full probability distribution rather than just a deterministic binary. You know, is it going to rain? Isn't it going to rain? Those are very useful. So we're excited about, about the, the AI will be able to do a good job in weather. More broadly, I think, you know, the sciences in general, I think, are seeing the dawning of an amazing new, you know, tool with AI to, to work out how to improve, you know, both research and applied research.

• 16:37Lukas Biewald?

  And so with, with weather, you talked about physics based models being the dominant strategy now. I think we've seen lots of domains, things switch from sort of expert based systems or systems with, you know, a lot of knowledge being put into the construction of them to black box end to end trained AIs. Are you also taking that approach? Are you completely disregarding all physics? Or are you trying to insert some physical knowledge into your models?

• 17:09Speaker 01

  Yeah, this is a really interesting area. I mean, I think we, I think, you know, knowing what we bring to the table is I think we're trying to bring, you know, AI sort of top talent and trying to apply the latest AI research to this area. And I think that's sort of what we can bring. And so we're going to see how far the sort of the pure data-driven approach can get us. There have been some really interesting applications. Neural GCM is one where people have taken sort of a hybrid approach using some physics and and the statistics. And, you know, that is very attractive, right? We sort of know that you

• 17:49Speaker 01

  could conserve energy, momentum, mass, you know, you can't make up moisture. What is interesting is in the directly, you know, AI approach, it appears to be learning some of those physical laws just from the data. And so, you know, the trade-off where the AI approaches are so much computationally cheaper or faster, you know, I think that's a really interesting place to advance. But inevitably, you know, I think there'll be some really interesting hybrid approaches. And prediction of the weather is a somewhat practical applied problem. Understanding the, you know, the science of the weather,

• 18:29Speaker 01

  you know, is a different problem and needs a different set of tools. You may be able to inform that with the conclusions of the AI models, but you may also need, you know, physics models to do that.

• 18:41Lukas Biewald?

  So you have this big stated mission right now as a new, a shiny new startup. And I've watched weather startups grow over the years. I'm sure you have too. And it's kind of surprising to me at least, as an outsider to the industry, where the business models eventually land. Could you talk about the different ways that a company like yours could make money? And how you're, how you're thinking about that, where you think things might end up for you?

• 19:14Speaker 01

  Sure, yeah, and it probably helps to sort of understand a little bit, you know, what the weather industry looks like. Please. And historically, it's looked a lot like sort of a vertically integrated, the government does most stuff. And so, you know, you have government satellites, you have, you know, government data assimilation, taking all those observations and getting them into shape for the physics models. And then you have the physics forecasting models, and then they disseminate that. And in the US, you know, there's been a proud history of sort of open weather forecasting data. And then, and then there've been commercial companies who have taken those sort of generic government forecasts and have tried to make them useful and put them on the phone and done all sorts of things.

• 19:57Speaker 01

  So that's been sort of the historical picture. And I think what is sort of crazy about how AI will change that is that for the first time, everyone will be able to forecast. You don't have to be a government with a billion-dollar supercomputer, you don't have to have a satellite, you know, you can take those observations and then you can, you know, use AI to run forecasts on them. So that's going to change the structure. And we think, you know, that instead of just one sort of vertically integrated approach, we think there's going to be more, you know, sort of layers, as you might see in a, you know, like in the LLMs, you know, you have, and you know this very well from your medical background, you have, you know, data providers, you have training tools, you have, you know,

• 20:41Speaker 01

  analysis tools, you have the hardware, the software, you know, various layers of abstraction. So, you know, have these different layers. And so, that's where we see this going a bit. We're very focused on where AI, and not just AI, but sort of AI and data, because, you know, the secret of AI is you really can't do anything if you don't have, have good data. And even, even more, where we have done AI is where we've had the data. You know, the reason we started with cat pictures is because we had a lot of them. And, and so, I think...

• 21:16Speaker 01

  I think that our business model will be to focus on the data and the AI and the metrics. And so this is another area where there's been so a bunch of, you know, there's sort of a temptation to cherry pick, right? It's like, hey, I did great on last Tuesday's storm, so I must be good in general, right? And so what we want to have is sort of a very holistic set of criteria on which to judge AI models and the physics models and to know where AI is not doing well yet and where we need to improve it. So we're focused on the data, the models and the metrics.

• 21:51Lukas Biewald?

  Okay, but where does that go as a business, right? So, you know, I obviously have this, probably everyone, you know, watching this has the experience of like looking at weather applications and forecasting weather. Is that really the best business model here? I mean, I understand that like insurance companies are interested in this. You know, probably there's applications to a broader set of financial companies. I imagine a lot of businesses are impacted by the weather. Like, who really buys the product that you're building here?

• 22:20Speaker 01

  So there's a bunch of end users and it's quite a, it's quite an eclectic group, right? Lots of different end users, lots of different needs. So, you know, you've saw the Climate Corporation doing, you know, farm, farm insurance, crop insurance, you know, you've got energy utilities who need to be able to make sure that their assets will work in whatever weather, that are predicting renewable energy supply and predicting temperature related demand. You've got supply chain people, they need to know whether trucks and planes can go. You know, you've got, you've got all sorts of different end users and they have different use cases for the weather. And I think what has been the case is when you're doing sort of a consultative analytics approach, which has been largely

• 23:01Speaker 01

  what sort of weather, you know, companies have focused on, you sort of get really good at doing one end use case or a couple of end use cases, but it's tough to be everything to everyone in that area. We think as we're a bit for, you know, further back in the chain, that there's an opportunity to provide the tools to be able to do forecasts to everyone. And so including, you know, the weather companies who can then serve the end users, and maybe some end users, but providing tools, and that's our approach. And I, I know that you like that model.

• 23:39Lukas Biewald?

  Oh, me? Oh, that's true. That is true.

• 23:43Lukas Biewald?

  Yeah, I'm not, no judgment. I'm just kind of curious. So, and I guess where I was going with that is, you know, you made this choice to be a public benefit, you know, corporation, which is kind of an interesting choice. Maybe you could describe what that is and then also why that was important to you and your company formation.

• 24:02Speaker 01

  Yeah, so a public benefit corporation is actually a C corp, but with a stated mission. And so the board has a responsibility to not only try and maximize value for its shareholders, but also to maintain the mission. So in our case, you know, democratizing weather forecasting and giving everyone the tools to be able to do that, you know, through our operations and, you know, any acquisitions or changes in control, you know, we need to, as a board, make sure that we're ensuring the success of the mission. There are some people, you know, when I was looking at this and working it out, there's another class of corporation called a

• 24:42Speaker 01

  B corporation, which many people think about because it's been quite high profile. And there is a very sort of long list of criteria and that you have to do. So a public benefit corporation is slightly different. You, you know, state your mission, you sort of pin your flag up on the flagpole, you report out on that mission each year. And then as a board, you have to take that into account when you're making decisions.

• 25:08Lukas Biewald?

  And so how would that affect practically how you make decisions or allocate capital or something like that?

• 25:16Speaker 01

  I mean, hopefully, if, you know, if your mission and your commercial success are aligned, then it wouldn't change it at all. And so, so, and we've set up, you know, what we think is both the right commercial strategy to be relatively open with our technology and to, you know, so interestingly, when you talk to customers, the biggest barrier facing AI weather forecasting is trust. You've got a lot of people who used to highly successful physics approach for a long time and the AI is really interesting, but they it's new, you know, they don't know what it is. And these are important decisions that people are making every day about about the weather and about climate. So the first thing I think we have to do is to get it

• 26:01Speaker 01

  in as many people's hands as possible so that they can play with it and learn to trust it. And that approach and open sourcing sort of the basic forecasting technology, I think will help do that. And you've seen in various different AI areas, so this common task method where you can give people enough data, you know, your model and metrics for them to be able to develop their own models and improve upon that and compare different approaches and really have sort of a sandbox in which to which to play. So that's what we're looking to do here. You know, improving weather and climate forecasting is not only going to be achieved by by us, a bright hand, but our relatively small resources is going to need the whole community to improve.

• 26:47Lukas Biewald?

  But I guess what I I'm curious if if so, public benefit corporation, if what it means is that you need to stay focused on your mission. It seems like, you know, almost any startup probably should be doing that or want to do that unless they're making a massive pivot. Is there something about the weather space specifically that made you want to have this structure? Or is that something that you would now do with like any new startup that that you would potentially start?

• 27:14Speaker 01

  Yeah, that's a good question. I think with weather, it's one of the the rare examples of successful international collaboration. And, you know, one country's weather yesterday is the next country's weather, you know, today. So everyone is very incentivized to share with the data, which they have been doing through the WMO and the different met offices. And and that's been a rare, you know, sort of shiny example of what you can do with with collaboration. And so I think that ethos and and the fact that the weather and climate problem that we face as humanity, I think, was the reason we wanted

• 27:52Speaker 01

  to make it really clear that we're trying to do something for the whole community as well as for ourselves and our company.

• 28:04Lukas Biewald?

  I guess one of the things that I found when I was researching weather and forecasting in general is the company you're coming from, DeepMind, has actually done a fair amount of work in this space. They've published some interesting results, kind of along the lines, I think, of what you're saying, predicting, I think, Hurricane Lee and publishing some papers on this. Are you concerned about competing with a company like DeepMind, which obviously has a lot of resources, especially around machine learning?

• 28:39Speaker 01

  As I get older and do more startups, I'm always wary when there's no competitors. It usually means you're picking a problem that is super interesting. So I'm not afraid of competition. And as a startup, you bring a bunch of things to the table that other companies don't have. And so singular focus, small teams, really working hard on a single problem. So I think we fill a gap. The big tech companies, weather is not their core business. They do amazing AI research and everyone will be great beneficiaries of that. But until it's a sizable business, it probably doesn't make sense for them to

• 29:19Speaker 01

  focus on it and to get deep enough in the domain to deliver what people really need. So we think we can provide some help there. And then the existing weather companies and the incumbents sort of have the classic innovator's dilemma problem, which is they have existing businesses and this is a completely different way to run your business. And so we think there's a chance that we can make a dent.

• 29:47Lukas Biewald?

  Well, I mean, of course, as a tele entrepreneur, I share your instincts here. But I guess where I want to go with that is it seems like deep learning and transformer-based architectures feels like maybe a new world where the capital costs are really high. And when you talk about a petabyte of data being created every year, it does seem like your company's like

• 30:08Speaker 01

  Sorry, 100 petabytes.

• 30:10Lukas Biewald?

  100 petabytes every year. The capital costs here seem like they could be enormous and it could really matter, the access to compute. Do you have a strategy around that? Or how do you think about that?

• 30:30Speaker 01

  Yeah, I mean, that's partly why we resourced ourselves relatively well for first round of 10 million. And on the data side, what's interesting is the data is available and is disseminated publicly by the government. And so that helps. And actually, through the NOD program, NOAA has disseminated its data to the different public clouds. And that actually was sort of what started this boom in AI. Weather forecasting was the this analysis database, the AERIFID database from ECMWF, became available in a cloud form where essentially people who were not weather domain experts

• 31:11Speaker 01

  or weather data domain experts were able to run AI experiments on it. So there's sort of an availability of data. We've resourced ourselves to the extent that we think we need to. And this is, people sort of think of the LLM market of hundreds of millions of dollars to train a model or going into the billions to train a model. It seems as though so far that this is a few orders of magnitude cheaper to do the training and then the inference is also relatively cheap.

• 31:44Lukas Biewald?

  Why do you think that is?

• 31:47Speaker 01

  You know, people love to sort of rationalize this. And I have some rationalizations. I do think it's pretty hard to predict and you sort of have to try it.

• 31:56Lukas Biewald?

  Totally, yeah.

• 31:57Speaker 01

  But one rationalization is with LLMs, you have this tool language and you're trying to capture everything language can do. You're trying to capture history and culture and sport and you know, maths and analysis and sort of everything is being used as a tool in lots of different ways. And to get AI to spot all those different patterns, it pulls it in different directions. So there's a bunch of complexity there.

• 32:25Speaker 01

  With weather, you have sort of a single geometry. You have sort of essentially a moisture in air in a system and it's at its most basic with repeated patterns that repeat every day. And so you're sort of modeling one thing. It's quite complex, it's quite big, but it's a single thing. So perhaps that's why we're finding that AI can do a good job with lower training costs.

• 32:53Speaker 01

  And fewer, you know, based upon a fewer number of parameters that you need in the model to capture that, just Pam.

• 33:00Lukas Biewald?

  Are you more inspired by transformers like language approaches that seem to be all the rage in almost every domain except for numerical time series? Or are you kind of leaning on more numerical time series approaches in your investigation here? Like, what, how do you think about that?

• 33:23Speaker 01

  I mean, transformers, transformers are great because they're getting so optimized, right? And so, you know, if you, and some of these architectures start to look a bit like each other. You know, you can sort of imagine a type of graph neural net that is a transformer and you know, you can sort of, they sort of merge into each other depending upon some of the configuration assumptions. So, you know, transformers have been used for, for the weather forecasting and it's just sort of an encoder, you know, processor decoder type, type architecture and the processor, people have used vision transformers. Some of the other approaches out there use diffusion models, you know, imagine trying to guess the next frame in a video is sort of like trying to guess the next frame in

• 34:06Speaker 01

  a radar picture, but for, you know, multiple dimensions and variables. And so, the architecture is important, but it's sort of not the only thing. And so, we're exploring, you know, different approaches and it will be interesting, as you say, you know, this is, this is a slightly different class of problem. You know, we've seen vision transformers and diffusion models used very successfully for things like, you know, CFD, you know, fluid dynamic, you know, simulations and so on. And really, it's super exciting because there's so much innovation in this area of making these tools more efficient for, for science problems in general, that we and everyone else will benefit from that underlying research.

• 34:52Lukas Biewald?

  Do you, do you, do you work on like trying to predict all the weather in the globe at all times? Or do you kind of specialize to like, okay, we want to try to predict where this hurricane's gonna go? Like, I mean, I could imagine in some sense, those are the same problem, but I would think you would kind of treat those quite differently, if only for kind of computational resources.

• 35:18Speaker 01

  Yeah, so sort of the benchmark task to see if your forecasting tool is any good is sort of the 10 day global weather forecasting and, you know, the resolution of that sort of initial models of being at sort of the one degree level, which is sort of like 100K, you know, at the equator.

• 35:39Lukas Biewald?

  This is one degree of like latitude and longitude, is it?

• 35:43Speaker 01

  Yeah, of arc. So yeah, 100K at the equator. And then, you know, sort of the state of the art with physics models is more like, you know, 10K at the equator for global weather forecasting.

• 35:53Lukas Biewald?

  10K, what? 10K?

• 35:55Speaker 01

  10 kilometers.

• 35:56Lukas Biewald?

  I see. So what's the weather like in a 10 kilometers square?

• 35:59Speaker 01

  An arc of 0.1 degrees. Yeah.

• 36:01Lukas Biewald?

  I see.

• 36:03Speaker 01

  And so, and so, so that's sort of the benchmark task to, you know, where we have really good, you know, long timeframe data and we can run a Heimcast for, you know, a bunch of years and say, okay, on average, you doing, are you doing well or not? We're trying to add to that sort of, you know, minimizing error on average approach, the root mean squared error approach. We're trying to add, you know, how right can you be? How opinionated can you be with a sharp forecast on a specific event? So we've sort of taken, you know, metrics piece, you know, we're taking a bunch of notable weather events and we're trying to see how well do all the different AI models, all the different police models do on those, which I think is really interesting, what the weather people actually care about.

• 36:48Lukas Biewald?

  Right.

• 36:49Speaker 01

  You know, it's, it's nice not to be wrong on average very much, but you also want to be right. And so, so I think that's very interesting. So we're sort of starting with that global benchmark of, of, you know, what are you doing on sort of general out to 10 day weather? But for most of what people care about is what's the weather for me? And so, you know, where I am or where my buildings are or where my fields are or whoever it is that you're talking about. So I think that's really important. And for the extreme weather, what is interesting is

• 37:21Speaker 01

  You know, there's two things you can do. You know, you can get better data and fill in the data gaps. And then you can use that, you know, in more effective ways. Some of the targeted sensing that people are now doing for an oncoming atmospheric river, an oncoming hurricane is to send out, you know, balloons or drones or buoys. And to get additional data, which allows you then to, you know, take those initial conditions and run better forecasts and even take, you know, historic extra observations in the important areas. And so I think that's a really interesting area. And AI's ability to take on much more data than the physics models can take

• 38:04Speaker 01

  and to do so very quickly is super exciting. And so this sort of the existing physics pipeline, first of all, it's very expensive and costly. So you're going to use so much computation. But also it requires this data assimilation step to take all these observations and turn them into sort of an idealized, you know, three dimensional grid with a data cube at each center of each voxel on that grid. But inevitably to do that, you're interpolating because there are parts of the earth we don't sense very well. So our approach, a bright band, is to go back to the raw data. And instead of going from that intermediary, you know, idealized analysis step to go back to all the raw data and forecast from that, that allows you to take on way more data.

• 38:50Speaker 01

  And so that's where you can start using, you know, even things that aren't thought of as having weather signal in them, maybe tweets. You know, people tweet when they feel an earthquake, maybe the pressure re-inks from your smartphone. You know, there's many different sources of information about the weather that are not currently captured in the traditional weather sensors.

• 39:10Lukas Biewald?

  Well, so you include things like earthquakes in your weather forecasting.

• 39:15Speaker 01

  Well, you know, we're largely atmospheric, but you know, we would like to move to the land and the ice and the oceans. And maybe even solar or space weather, they all they all feed into it. And I think, you know, that Earth system model we talk about in our mission, you know, combines all of those things to be able to make decisions. Perhaps even in the future, you know, the biodiversity and human activities and sort of all the other things you may be able to model and include in a picture of the Earth.

• 39:49Lukas Biewald?

  So like thinking about weather all day, are you are you someone where like, how does climate change factor into your models? And how does climate change factor into your personal outlook?

• 40:04Speaker 01

  Yeah, I mean, climate change has unfortunately got terribly political, at least in the US. And so, you know, we're pretty practical about it that there's clearly increasing the extreme weather. And so how do we, you know, how do we deal with that? I think as a as a race, we should have the tools to be able to make decisions about not poaching the planet we're on.

• 40:26Speaker 01

  And currently, the tools that we have are not great. It's a hard problem. And obviously, the time scales are forbidden.

• 40:32Lukas Biewald?

  The time scales are pretty different, right? I mean, like, if, you know, if you could forecast out even 30 days, you probably wouldn't see climate change in that, right? Do you imagine doing like, you know, five year weather forecasts or more?

• 40:46Speaker 01

  The hope is that traditionally, weather and climate have been modeled differently, right? So, you know, weather has been sort of a initial conditions and then auto recursively rolled forward forecast. And then climate has been a long term averages. And then what's called sort of an earth balance model, where the the main, you know, forcing function is the sun's pumping in a lot of energy. And some of that bounces off the clouds and bounces off the ground. And you can make some assumptions about that. And you can make assumptions about how carbon dioxide in the air and changing brightness of the ice and other things are going to change that, you know, balance.

• 41:25Lukas Biewald?

  Although that would require some physics, right? Like, I mean, you're not going to have historical data that would show you that. Or do you think the models could somehow learn that from the last 100 years of of data?

• 41:37Speaker 01

  I remain curious as to where we'll end up on that. I think some of the bets being made are to have some hybrid models, which sort of create a physics spine, hopefully differentiable, so you can actually compute with that. And for that to stop the AI drifting away from some sort of physical reality.

• 41:59Speaker 01

  But I think we will see.

• 42:01Lukas Biewald?

  Interesting. Do you, I mean, I guess climate change is a loaded topic in America, probably everywhere in the world. And I think one of the kind of interesting things happening right now is like,

• 42:16Lukas Biewald?

  AI becoming an increasingly bigger component of our electrical grid energy consumption. Do you have a point of view there?

• 42:27Speaker 01

  Yeah, I mean, clearly, you know, clearly we should be mindful about how we use the world's resources. And, you know, spinning up massive amounts of compute to answer unimportant questions, we might want to think about that. I think weather is an important question. I think, you know, so in terms of priorities of what we use, you know, the world's resources for, I would sort of put it up there.

• 42:54Speaker 01

  The good news is, you know, the research shows that AI is six orders of magnitude faster and cheaper than the current approach we're using with physics. So at least on the inference side, we should be able to save a lot of energy. You can now, so the crazy thing, if there's one thing you take away from this, like you can now, using a laptop in under a second, do a comparable forecast to what it takes a billion dollar supercomputer has to do.

• 43:20Lukas Biewald?

  And wait, why is that? Because, because of the, you're saying with a deep learning approach, you can do that?

• 43:29Speaker 01

  The amazing ability of AI to sort of compress information

• 43:35Speaker 01

  into the size and the amount of compute you need to apply patterns, learned patterns to get good answers is sort of mind blowing.

• 43:44Lukas Biewald?

  Wow, could I download a program that would, that would do that for me?

• 43:48Speaker 01

  Yeah, I mean, you might want a beefy laptop, but.

• 43:52Lukas Biewald?

  I got a pretty good one. It can run in, can run like a 70 billion parameter LLM. So I feel like. Yeah. Yeah. What would like, where would you point me to to?

• 44:02Speaker 01

  Also, I mean, I mean, part of the reason, you know, we're around is a lot of these research models are not yet available for anyone to run, right?

• 44:09Lukas Biewald?

  I see.

• 44:10Speaker 01

  You know, uh, lack of commercial licenses or lack of tooling to make them easy to run. So, so there are some great models out there and you, Lucas, I know could, could work out how to do it, but we want to make it easy for everyone.

• 44:23Lukas Biewald?

  Do you plan to release some of these models so that other people can build on top of them?

• 44:29Speaker 01

  Yeah. Early next year, we're going to open source a data model and metrics so everyone can try this.

• 44:35Lukas Biewald?

  But actually like a, I guess, the, like a model you can run?

• 44:43Lukas Biewald?

  I see. Interesting. Do you expect people to fine tune it? Maybe you could, oh, I mean, do you imagine like a fine tuning use case? Like if I wanted to forecast weather on the moon, maybe I could take some lunar data.

• 44:57Speaker 01

  I had no thought of that one, but that's pretty cool. Yeah. I mean, ideally, you know, you wouldn't want to retrain, you know, all the parameters. You know, one can imagine that you have a model understands enough about weather that if you could put in your initial conditions for your particular place of interest, that that would be a fine tuning, that you could get out a customized forecast for you. So, you know, for the first time, you can run a forecast yourself and it can be customized to where you care about. That's, that's pretty cool.

• 45:26Lukas Biewald?

  Cool. Cool. I've always wished that these weather apps would have a little more of like a probability distribution or be more clear about what they're actually forecasting. I hope somebody takes your model and makes the forecast is a little more clear about the uncertainty and what that really means.

• 45:43Speaker 01

  Yeah, me too. I mean, you sort of look at these weather apps and it tells you what the weather is sort of 10 miles that way, you know, and you're like, well, what about me?

• 45:51Lukas Biewald?

  Totally.

• 45:53Speaker 01

  And you know, part of the, I mean, one of the interesting questions is, you know, once you're able to run, you know, a ton of scenarios and you have all these probability distributions for what the weather could be, like, how do you communicate that to people? And we have an advisor, Amy McGovern, who's a professor at Oklahoma, and she has spent a lot of time trying to work out how to get people to understand and trust weather forecasts. You know, what does it mean to say there's a 30% chance of rain? Is it, you know, is it going to rain in 30% of the area? Is it going to be, you know, like, what? And so we also need to innovate on how we try to communicate weather to people.

• 46:30Lukas Biewald?

  If the butterfly effect is real and you can forecast the weather well, do you think there are applications of a model like this where you could change the weather with some kind of intervention?

• 46:40Speaker 01

  So, I mean, obviously you need, you know, ways to change the weather, and there've been some of those, you know, cloud seeding since, you know, I think the 50s, 60s. And, you know, I think, I think people have been sort of appropriately concerned about large scale application of those and sort of whether we know what we're doing if we go try and change the weather. But I do think when we're changing the weather right now, right?

• 47:07Lukas Biewald?

  Sure.

• 47:08Speaker 01

  You and I are burning, burning fuel to support this, this podcast and, you know, that's going up into the air and that's changing the weather. So I do think that what we need or what, what maybe we can do to help in mitigation areas is for all the different tools that we could use, how do you predict how they're going to change the weather? And there's a great book by Neil Stephenson called Termination Shock, if you haven't read it, which sort of muses on this question and sort of raises the interesting idea that whatever you do, the weather will probably not affect all people the same. And there may be winners and there may be losers and what do we do about that, right? Because we've all been on the same planet, so.

• 47:51Lukas Biewald?

  Yeah.

• 47:53Lukas Biewald?

  Okay, final question. I guess this is more of an entrepreneur oriented question, but you know, my companies have been more software based where you know, $10 million is like a lot of money, you could do a lot with this. I feel like your company is a little more deep tech, maybe. I'm curious how you think about like milestones and where you think the money is going to take you. Like, are you trying to get to like a commercialization stage? Are you trying to show that your technology can work for certain use cases? How do you think about that? Especially in this world of large compute costs.

• 48:32Speaker 01

  Yeah, so I mean, the reason we have to raise, you know, that amount of money is both we're looking for sort of very particular talent, which is pretty sought after. You know, the number of people who can build and run the latest AR, you know, global weather models, who is sort of vanishingly small. And, and then the amount of-

• 48:53Lukas Biewald?

  Although those people would probably be excited to work, work with you. You seem like a great option.

• 48:57Speaker 01

  Well, it's cool, we'd be able to, yeah. And, and you know, fortunately, my, my co-founders, Ryan Kiesler and Daniel Rothenberg are sort of very well regarded. And so it's been fun, you know, people are really, oh, I get to work with Ryan and Daniel. So, so that's exciting. So, yes. But then, you know, so the people and then the amount of compute and data is non-trivial. And so what we're trying to prove with the, with the first money that we've raised is to get to the point where we can demonstrate that we've built something that works, that people want, and that they're willing to pay for. And therefore that it's worth investing more of our time, more of our investors' money to scale that up. So that's, that's our goal for the next two years.

• 49:38Lukas Biewald?

  Awesome. We appreciate your time. That was fun.

• 49:41Speaker 01

  Fun talking to you, Lucas.

• 49:44Lukas Biewald?

  Thanks so much for listening to this episode of Grading Descent. Please stay tuned for future episodes.