A skeptic’s take on AI electricity load growth

The predictions are coming in hot. Data centers could grow to consume more than 9% of U. S. electricity generation by 2030, according to EPRI. That’s more than double its current estimated data center load. AI will increase global data center power demand 165% by 2030, says Goldman Sachs. And billions of dollars are at stake. Utilities, megasite developers, and data center operators are all basing major decisions on predictions like these. But they’re also the kinds of predictions we’ve seen before. In 1999, when the internet was growing fast, a couple researchers claimed it would grow to consume half of all U. S. power generation within a decade — until a team at Lawrence Berkeley National Laboratory debunked it. Jonathan Koomey was one of those researchers. Although today’s predictions about energy usage are tamer than those in 1999, Jonathan still has questions about the current hype around AI power demand. He's is now the founder and president of Koomey Analytics, which has published multiple papers on the topic, including a recent report for the Bipartisan Policy Center: Electricity Demand Growth and Data Centers: A Guide for the Perplexed. So what are the assumptions that go into these new predictions? And how do they hold up to scrutiny? In this episode, Shayle talks to Jonathan about why he questions the hype around AI load growth predictions and why he believes energy constraints will incentivize the AI industry to focus on efficiency. Shayle and Jonathan cover topics like: The time lags and proprietary data that hinders precise data center load estimates, both in historical analyses and future predictions The difficulty of reproducing the predictions of even prominent institutions like the IEA The two basic assumptions that go into predictions: AI demand and AI power requirements Why Jonathan believes conventional wisdom relies on questionable sources, like Nvidia’s business plan The unexplored areas of AI energy efficiency, like computer architecture, software improvements, algorithms, and special purpose computers Recommended resources Lawrence Berkeley National Laboratory: 2024 United States Data Center Energy Usage Report Nature: Will AI accelerate or delay the race to net-zero emissions? Joule: To better understand AI’s growing energy use, analysts need a data revolution WSJ: Internet Hype in the ’90s Stoked a Power-Generation Bubble. Could It Happen Again With AI? Open Circuit: The data center boom: ‘All the cheap power is gone’ Credits: Hosted by Shayle Kann. Produced and edited by Daniel Woldorff. Original music and engineering by Sean Marquand. Stephen Lacey is executive editor. Catalyst is brought to you by EnergyHub. EnergyHub helps utilities build next-generation virtual power plants that unlock reliable flexibility at every level of the grid. See how EnergyHub helps unlock the power of flexibility at scale, and deliver more value through cross-DER dispatch with their leading Edge DERMS platform, by visiting energyhub.com. Catalyst is brought to you by Antenna Group, the public relations and strategic marketing agency of choice for climate and energy leaders. If you're a startup, investor, or global corporation that's looking to tell your climate story, demonstrate your impact, or accelerate your growth, Antenna Group's team of industry insiders is ready to help. Learn more at antennagroup.com.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Hey, it's Executive Editor Stephen Lacey. Many of you were loyal listeners to a show I created and co-hosted for eight years called The Energy Gang. Well, we're back. Jigar Shah, Catherine Hamilton, and I are excited to announce our latest project, a weekly news roundtable called Open Circuit. Every week, we'll break down how major projects come together, how deals and policies get structured, and what it takes to build critical infrastructure at scale, all through the lens of current events. You can subscribe to Open Circuit wherever you listen to podcasts or follow it at latitudemedia.com.

• 0:35Speaker 01

  Latitude Media, podcasts at the frontier of climate technology.

• 0:41Speaker 00

  I'm Shail Khan, and this is Catalyst.

• 0:47Speaker 02

  Usually, what they're assuming is Nvidia's business plan is our growth forecast. They're just saying, well, Nvidia has this projection for the next few years of how many AI nodes they're going to sell. And they just use that and say, that's what's happening. And of course, there's uncertainty in that.

• 1:07Speaker 00

  Coming up, injecting a little bit of skepticism into all the hype around data center energy consumption.

• 1:21Speaker 03

  Catalyst is brought to you by Antenna Group, the communications and marketing partner behind the most influential companies in energy, climate, and infrastructure. For three decades, Antenna has helped ambitious companies stand out, build market leadership, and become category-defining brands. From breakthrough startups to global industry leaders, their team brings the expertise, relationships, and track record to accelerate your momentum. Learn more at antenna group dot com.

• 1:47Speaker 00

  The power plant for the modern grid isn't a building. It's a network. Energy Hub's derm's platform connects batteries, EVs, thermostats, and commercial and industrial sites into virtual power plants that over 230 utilities dispatch as reliable grid capacity day in, day out. VPPs deploy in 60 to 100 days at 40 to 60% less cost than traditional infrastructure, and they perform like conventional generation from day one. Learn about the flexibility platform for the modern grid at energyhub.com.

• 2:19Speaker 03

  Demand flexibility is now an essential part of the data center's speed to power playbook. But how do data centers actually curtail load? Gridbeyond is the end-to-end solution for flexible energy management. Gridbeyond's advanced DERS and VPP platform is tailored to what data centers care about most. Maximizing reliability, intelligently meeting curtailment obligations, reducing costs, and managing all your distributed resources in one place. Ready to transform your distributed energy management? Get started at gridbeyond.com.

• 2:53Shayle Kann?

  I'm Shayle Khan. I lead the Frontier Fund at Energy Impact Partners. New tagline, who dis? Anyway, we've talked about this before, but here is what has been going on in the world. There's a general perception that near and maybe also long-term demand for AI compute is essentially insatiable, at least relative to today's capabilities to deliver it. And crucially also, there's, I think, generally a belief that access to power is one of, if not the, most important bottlenecks in the delivery of that compute and the services that it allows. So we have developers and colos and hyperscalers all out there executing a literal land grab all over the country and much of the world to lock down sites that will be suitable

• 3:37Shayle Kann?

  for tomorrow's data centers. And then meanwhile, forecasters, be they governmental, non-governmental companies themselves, they all keep increasing their load growth projections, painting this picture of an electricity sector that will be essentially bursting at the seams for the foreseeable future. And then DeepSeek comes along and for a moment scrambles that narrative. People ask, will energy efficiency improvements actually slow or stop the rate of load growth caused by AI? And then it seems, at least to me, that everyone kind of collectively decides nah. Jevon's paradox. The less it costs, the more we'll use. And we go right back to seeing gigantic proclamations from the likes of Stargate,

• 4:19Shayle Kann?

  Meta, and many others. I say everyone collectively decides nah, but it's actually not quite everyone. John Koomey, in particular, is not part of everyone. He has been researching data center energy demand for a very long time, starting way before the AI wave. He worked at Lawrence Berkeley National Labs for a long time, and then at Stanford before founding his own firm, Koomey Analytics, which focuses, amongst other things, on this exact question of AI and data center-driven electricity load growth. And as I alluded to, he's more skeptical than most other folks that I've spoken to, at least, on how much electricity load growth we will actually see from this new wave of AI data centers.

• 5:01Shayle Kann?

  It's a perspective I don't think you hear that much these days, so I thought it was worth teasing out and perhaps debating a little bit. So, here's my conversation with John. John, welcome.

• 5:12Speaker 02

  Thank you. Glad to be here.

• 5:14Shayle Kann?

  Excited to talk about data center energy consumption with you and with everyone else in every conversation that I have at the moment, but particularly with you. You're, I think, especially valuable to give a little bit of a history lesson to start here, which is, can you just talk through pre-AI wave, you know, before the last couple of years? We had been on an interesting trajectory with regard to the growth of compute as it pertains to energy consumption overall. So, can you just walk us through the history there? What had been happening for the past couple of decades?

• 5:48Speaker 02

  Yeah, so let's go back to the dot-com boom. So, the late 1990s, there was a lot of focus on computing in various forms, and people were very excited about different things, including fiber optic networks and e-commerce and other stuff. And there was a lot of speculation round about the year 2000 that computers and related technologies were going to end up using a lot of electricity. There were projections widely cited by, you know, people of many different persuasions that, you know, the internet was going to use half of all electricity in the next 10 years, this kind of thing. There were a couple

• 6:29Speaker 02

  of guys running around, you know, pushing that whole narrative. And that led to...

• 6:38Speaker 02

  Pressure on people who are at the National Labs and other places to try to figure out, well, what were the numbers? Well, you know, what was actually going on? I was a staff scientist at Lawrence Berkeley National Lab at that time, and I had done work on computing electricity before that in the early to mid 90s. So people came to me and said, well, you know, do these numbers make any sense? And we ended up doing careful work. We found that those projections were off massively. Like, these guys were claiming that all computing used 13% of all electricity use, and when we actually tallied the numbers, it was 3%. And we've, you

• 7:17Shayle Kann?

  know, figured- That was projections-

• 7:18Speaker 02

  No, no.

• 7:19Shayle Kann?

  -of what would happen?

• 7:19Speaker 02

  No.

• 7:19Shayle Kann?

  That was what they thought was already happening?

• 7:21Speaker 02

  Yeah, so they were saying, they were claiming in 2000 that all computers use 13% of our electricity. And we actually analyzed those numbers and found that, no, it was actually 3%. And that there was no way that it could be half of all electricity use in 10 years. And so it led to a lot of great research, but it made me pretty skeptical about these kinds of projections, because there's always people with an interest in promoting this narrative about, you know, fast electricity use and growth in data centers and all that. So that was, that was the year 2000. We found those numbers, you know, the base numbers are wrong,

• 8:05Speaker 02

  the projections were wrong. And then we looked at historically in 2008, 2010, well, what actually happened? And we did see a doubling of electricity use for data centers from 2000 to 2005. So there was fast growth during that period. And the industry saw that and responded by creating a bunch of efficiency technologies to mitigate that growth. So it took data centers from about half of, 1% of electricity, it was half of 1% to 1% of electricity use by around 2010 or so. And that was about where we were with data centers.

• 8:49Speaker 02

  So this fast growth for a bit started to moderate, and then from 2010 to 2018-

• 8:56Speaker 02

  Growth was actually pretty slow in data center electricity use. There was still an increase in the use of data centers. So we had about a sixfold increase in the compute output for data centers from 2010 to 2018. But electricity use went up a grand total of 6% over that period. So pretty modest growth, even though the service demand for data centers was going, growing pretty fast.

• 9:22Shayle Kann?

  So, so that's interesting. I hadn't thought about this in this, this two periods. So 2000 to 2005, electricity consumption from data centers actually did double from a small base, but doubled. So there we weren't yet seeing, we were seeing a boom in compute. We weren't yet seeing these dramatic energy efficiency improvements. But partially in response to that, in the next big growth cycle for compute, we did see all these energy efficiency improvements. And so the next time compute really took off, we didn't see the same spike in electricity consumption. Is that like the right way to think about it?

• 9:55Speaker 02

  That's right. Yeah, I think that the growth in the use of computing continued. But it wasn't that it stopped. And, you know, it was actually, it was continuing, but the industry focused on efficiency. And they figured out a bunch of different ways to improve the efficiency of computing. Some of that involved redesigning the silicon, right? So you're shrinking transistors and changing hardware architectures and this kind of thing. But also it involves moving from very inefficient corporate data centers to much more efficient hyperscale data centers. And we can talk about why that's important. But basically the hyperscalers have economies of scale and they figured out design tricks

• 10:38Speaker 02

  to make the efficiency of delivering computing and, sorry, electricity and cooling to the computers much more efficient. And they have much higher utilization of their equipment. And so when you think about it in terms of cost and energy use per compute, these hyperscale facilities are much, much better than the kind of traditional corporate facilities. And there was a big movement towards those hyperscalers in that that later period towards 2018.

• 11:08Shayle Kann?

  Right. So then, so it's, I think that's the important, the important lesson here is that there has been a historic, even if you just take that whole 20 year period, starting in 2000, there has been a gigantic growth in overall computing that has resulted in a much, much more moderate growth in electricity consumption, relatively speaking. And so of course, the operative question in the current moment is, are we in for a repeat of history? Either a repeat of the 2000 to 2005 history where compute grows alongside electricity consumption and we do see this big boom, or the, I guess, the bigger part of that 20

• 11:49Shayle Kann?

  year period wherein you saw the 6X growth in computing, but only 6% growth in electricity consumption. So that's like the operative question right now. And I guess before we get into answering it, what are you seeing out there in terms of the variability of projections? This is one of the interesting things. Nobody really knows, right? What's going to happen? And so I think part of the result is that I've seen forecasts that are like all across the board.

• 12:11Speaker 02

  Yeah. So there's huge variability in the forecast. There's even variability in what we think of as history. So for the year 2022, the International Energy Agency did these projections. They did two of them in 2024 and they started by trying to do history and they looked at 2022. And the first study they released in January had a number that was 50% greater than the second study that they released in October. And so even the same institution, looking at a historical year, had this uncertainty. They identified an uncertainty range of 220 to 340 terawatt hours per year for global

• 12:56Speaker 02

  data center electricity. So that's a pretty big. Uncertainty range for a historical year. So there's a lot of things we don't understand.

• 13:05Shayle Kann?

  Is that because of China or something? Like, why is it so hard to estimate historically?

• 13:09Speaker 02

  Well, because the the data are often held as proprietary information. And so we, as analysts, first, we we aren't, we don't have access to the proprietary information. But when there is such information, so these data collection companies, IDC and Gartner and so on, they collect the information, but there is a time lag for releasing the information. So there's like a lag because it has to be collected and processed and so on. And so by the time, you know, we're getting, you know, 2022 data, it's 2025, right? Or 2024. And so there's this time lag and it takes time to analyze and so on.

• 13:53Speaker 02

  So it's both the inaccessibility of certain data and then the time lag associated with real data and real analysis that leads to this uncertainty in the history. And this is distinct from the uncertainty in the projections, right? That you were, that you started the question with. But I think it's important for people to understand that we we don't even know with high precision the historical numbers for 2022 and 2023. And when something grows super fast, like AI, which has been growing gangbusters, it's very hard, like, things happen in six months or a year. And, you know, it's it's two or three years later that we actually get the real data that people have confidence in.

• 14:37Speaker 02

  So so that's a big uncertainty just in the historical data. And then we can talk about the projections.

• 14:43Shayle Kann?

  Yeah, okay, so let's talk about the projections then. What's the what's the range of projections that you've seen?

• 14:49Speaker 02

  Well, well, let's let's actually not start with those quantities. Let's start with, conceptually, what we think is happening. There's two drivers of electricity use in data centers or AI data centers or any data centers. There's the growth in the service demand, the demand for compute, the demand, you know, the AI queries or search queries or something else, some measure of service, right? And if those

• 15:18Speaker 02

  Measures of service demand are growing. That, of course, puts upward pressure on electricity use. Then the other big uncertainty is on the efficiency of delivering the service. And if you can make the service, you know, deliver the service much more efficiently, then you'll be able to meet that service demand, but not increase electricity use in the aggregate. So those are the two forces. And when people make a projection of what future electricity use will be, they are implicitly or explicitly making assumptions about service demand growth and efficiency of delivering that service demand.

• 16:03Speaker 02

  And so that's the kind of the first core level understanding of the, of what's happening here when, when people are making a projection. And they're making assumptions. Usually they're implicit assumptions that are unstated. And some are better at explaining what those assumptions are, like the, the LBNL report to Congress that came out in December of 2024. That was estimating data center electricity use for the U.S. And that's at least reproducible. Like I can go into that report and I can figure out what they did. Now, I was, I was part of that team, but anyone can, can look at that.

• 16:44Speaker 02

  Whereas some of the other projections, even from pretty credible organizations like the International Energy Agency, you can't actually figure out what they did. They don't release the data. They don't release the calculations in any explicit way. And so that's kind of the first question people should ask is can I figure out what these people assumed in making these projections? And usually what they're assuming is Nvidia's business plan is our growth forecast. Right? They're just saying, because for AI, they're just saying, well, Nvidia has this projection for the next few years of how many AI nodes they're going to sell. And they just use that and say, that's the, that's what's happening.

• 17:29Speaker 02

  And of course, there's uncertainty in that. Right? It's, it's not just uncertainty in how many they'll sell, but it's also how they will be used and whether there will be innovations that allow people to use that hardware much more efficiently.

• 17:46Speaker 02

  And so you saw the announcement of DeepSeek a few weeks ago. And that was an example, it was that Chinese AI firm that was able to get similar sort of accuracy benchmarks from their runs, but using much less compute and much cheaper.

• 18:07Shayle Kann?

  So, right, so this is, this is, I think, the core of what we need to talk about here, right? So here's my impression of sort of, to the extent that there was conventional wisdom, conventional wisdom in this space. It was that, prior to DeepSeek, right, for the, for the past year or two, it was, oh my God, there is going to be, we have such a huge need for new data center capacity, all of the hyperscalers, and then OpenAI now with Stargate and all these other things. Everybody's, you know, are planning to spend hundreds of billions of dollars in CapEx and, you know, develop tens of gigawatts of new data centers and we're seeing this happen in real time. And so the load projections, load growth projections keep going up and

• 18:49Shayle Kann?

  up and up. And in fact, I think the conventional wisdom was becoming the limiting factor on load growth from AI will not be demand for AI or willingness to pay for AI, but rather the grid itself. And it's, we're just going to max out the amount of space we have on the grid for these hyperscale data centers, basically. And then DeepSeek came along and shook everybody on that assumption briefly, right? And as you said, DeepSeek's, from an energy perspective, the main thing that DeepSeek was offering was like, we could do the same thing with, with more than an order of magnitude less energy to do the same thing. And so for a moment, everybody freaked out about that, right? And you could

• 19:34Shayle Kann?

  see that reflected in like, the public equity prices of, of the constellation energies of the world, things like that. But it was a blip, more or less. And now, and you know, since then, all the hyperscalers announced additional CapEx investments and, you know, the world sort of resumed back to where it was. And I think it can be boiled down, why is that? I think it can be boiled down basically to The Jevons paradox argument, which Satya Nadella himself made, but many other people have as well, which is basically at the highest level, and then I'll have you explain how you think about it. At the highest level, if you make a thing cheaper, if you make a thing cheaper with sort of endless potential demand for the thing, people will just use more of it in the end. And so even if

• 20:17Shayle Kann?

  DeepSeek and derivatives thereof turn out to be game changers in terms of energy efficiency, that just means we are going to be able to do ever so much more AI. And as a result, from an energy perspective, nothing changed. That's my sense of kind of like the conventional wisdom. I know that is, you have a slightly different view there, but do you think that that is what the world generally thinks? Like, do you agree with me that that's the conventional wisdom at the moment?

• 20:43Speaker 02

  I think that the people who are delivering AI, who are suppliers of AI, they honestly believe that the demand for their product is infinite, as you described. And the Jevons paradox is a complicated thing that we don't even need to get into, but we can because it's fun and interesting. Because that was used by the head of Microsoft to explain why he thought that demand would not be affected, the electricity demand would not, would continue to grow massively, because essentially there's infinite demand for their

• 21:21Speaker 02

  product. And that assumption is something that people should examine. I think there's a lot of people who believe that. And of course, you know, in economics, if something gets cheaper, people use more of it. But the question of how much more they use is a question of, you know, what is the value being delivered by this technology? And if the value of a technology is very, very high, people will be willing to pay more to have it, right? And so costs may come down, and maybe that means they'll use more of it. But the question is, at some point, is there a kind of saturation?

• 22:07Speaker 02

  You know, kind of a leveling off of this. And so, you know, the Jevons paradox is also called the rebound effect. And it's typically used to describe efficiency. Like, the idea is that the efficiency of something like a car goes up. So it's cheaper to operate the car, so people operate the car more. Right? That's the intuition around it. And in reality, what happens, of course, is that, yes, the car is a little bit cheaper. And yes, people drive a little bit more. But generally, these effects tend to be kind of small. They tend to be kind of 10 to 20%. You know, energy is a small part of the economy. It's only 7 or 8% of the GDP, right?

• 22:50Speaker 02

  So in the aggregate, energy is not a huge fraction of total costs. And so...

• 22:55Shayle Kann?

  And that's also true of data centers.

• 22:57Speaker 02

  Yeah, data centers are a little more...

• 22:58Shayle Kann?

  It's a huge portion of OpEx.

• 22:59Speaker 02

  Yeah, yeah.

• 23:00Shayle Kann?

  Right?

• 23:00Speaker 02

  Yeah, there's a huge capital cost in data centers, right? So, energy is important. It's not trivial, like, it's not 1% of costs in data centers. But it's also not the majority. And so tying this to efficiency, I think, is a mistake. But I think the general argument is the one that you're describing, which is there's infinite demand for our products. And even if the products are, you know, the cost of delivering the products go down, people are just going to use more because they want, they have this infinite demand. And that's really the core of the argument. That is, that is what they're saying, is that there's so much demand that

• 23:42Speaker 02

  none of these kind of constraints are going to get in the way. And if we reduce the cost, people are just going to use more. So the electricity demand must happen. Like, that is their core argument. And so, let's examine that question. Because to me, that's really the assertion that there's infinite demand is, should at least be open to discussion. People should talk about that.

• 24:10Speaker 03

  If you work in energy or climate, you already know the company's Antenna Group has helped build. For three decades, they've been the communications and marketing partner behind the industry's breakout companies, category creators, and market leaders. Today, their team is working across the most consequential areas of the energy economy, from data centers and grid modernization to nuclear, geothermal, storage, critical materials, and next-generation technologies. Antenna helps ambitious companies become the names everyone in the market knows, trusts, and wants to work with. They build reputation, elevate executives, drive demand, and turn momentum into market leadership. If your company is ready to lead its market, Antenna is ready to help you get there. Visit antenna group.com.

• 24:54Speaker 00

  When you harness the power of millions of DERs in a VPP and put artificial intelligence behind the dispatch, the outcomes exceed conventional generation. Energy Hub's DERMs platform has been trained on years of real dispatch data from over 2.6 million DERs, so utilities can plan with confidence. That means consistent load reduction across a four-hour event window, even during the widest, most uncertain peak periods. Utilities target, shape, and verify capacity down to the meter. Unlock flexible capacity that costs 40% to 60% less to deploy than traditional infrastructure in a fraction of the time it takes you to break ground on a peaker plant. Learn more at energyhub.com.

• 25:35Speaker 03

  For data centers, connecting to the grid quickly means two things: building distributed energy and having a plan for demand flexibility. GridBeyond helps with both. GridBeyond's DERMs and VPP platform unlocks the potential of distributed assets and dispatches flexibility into the right market at the right time. And this is more than software. GridBeyond supports developers every step of the way, from modeling your highest ROI energy mix to orchestrating the full set of assets around a data center. The track record speaks for itself. GridBeyond manages over 1,400 sites and more than 3.7 gigawatts of load across Europe, the UK, and the US. Learn more at gridbeyond.com or click the link in the show notes.

• 26:20Shayle Kann?

  As it pertains to energy, there's multiple very hard to predict things. The first hard to predict thing is how much of an energy efficiency improvement? How much of an improvement in watts per flop can we expect realistically? As you made a point before that historically, one of the reasons we got such an energy efficiency improvement was the shift to hyperscale data centers. Well, we already have hyperscale data centers, so that's not like another thing. We can't replicate that. So, but then deep sea, because it's own thing, right? Maybe the way to run these AI models can be a lot more efficient. And similarly, I've heard a lot of people talking about, well, look like the hyperscale data centers to run AI right now, it's mostly training. The world is going to shift predominantly toward inference at some point, and maybe

• 27:04Shayle Kann?

  we can, there's a lot more opportunity for new innovation and inference, and we could get more efficiency. So for me, the thing that's difficult about this is that you are, as you said, you're both trying to predict the energy efficiency component and the demand component simultaneously. And, and both seem highly uncertain to me, but I think that if I can reflect the argument of the, of the masses, it is that there is not necessarily literally infinite demand for AI compute, but the demand is sufficiently higher than where we are today. It's orders of magnitude, let's say, higher than where we are today, that the limiting factor on electricity growth is not going to be demand, it's going to be something

• 27:49Shayle Kann?

  else, and it could be.

• 27:50Speaker 02

  It's how fast can we build the equipment?

• 27:52Shayle Kann?

  Yes, and or chips, right? It's chips and, and power and all that kind of stuff.

• 27:57Speaker 02

  And transformers and, you know, backup power generators, this kind of thing.

• 28:01Shayle Kann?

  Does that argument hold water for you? Or like, where would you poke holes in it?

• 28:06Speaker 02

  To me, that it's effect, you know, it's effectively the same thing. They're basically saying that constraints are related to how fast we can supply the AI. So effectively, even if it's not infinite, it's, it's high enough that they will, you know, build as much as they can, given these other constraints. And so the question is, is that actually true? I think most people seem to believe it in the industry. I think that's pretty common. I think most people think that's true, but is it true? That's the conversation that I think people need to have because. You know, there are issues with these models.

• 28:48Speaker 02

  We'll talk about the efficiency side of it because I have a lot to say about things you can do to do this more efficiently. But on the demand side, there are real questions about accuracy, right? Traditionally, what has happened, and this is another set of assumptions that the whole industry seems to make. Historically, they have gotten increases in accuracy by scaling, throwing more computers at the challenge. And they've had, you know, whatever it is, 10 or 20 or sometimes 30% increases for a, you know, a tenfold increase in compute capacity. And when you have, you know, computers improving very rapidly, you can do that over time.

• 29:31Speaker 02

  But now we're reaching the point where the scale of the construction challenge for increasing capacity, you know, computing capacity 10 or 100-fold turns out to be much harder. When it's a relatively small industry, that's not that hard. But now we're getting up to the places where things are a lot harder to do. So that's one question. Is continuing to scale in the way that they have in the past the only or best way to meet this demand for compute? Right? And I think that's an open question.

• 30:14Speaker 02

  I think that there are other ways to meet the demand for compute, and that relates to the efficiency conversation. But then, but on the demand side, accuracy is a critical thing. Liability is a critical thing. And companies are liable for the actions of their agents, be they human or a computer. And if your agents are telling people to eat rocks and put glue in their pizza and doing other things that don't make any sense, ultimately, I think that will probably constrain the demand. So it's really a business question now of there's all this CapEx. People are trying to sell AI and make money from it.

• 30:59Speaker 02

  And the question of whether people are willing to pay for the service being delivered is, I think, an open question. Because they're, you know, they're charging, you know, OpenAI right now is just a major... It's hemorrhaging cash like you can't believe, right? There's no obvious way that they become profitable without a massive increase in the demand for their services and the ability to raise the cost. And that is really unclear.

• 31:28Shayle Kann?

  Yeah.

• 31:28Speaker 02

  Whether that is possible.

• 31:29Shayle Kann?

  Setting aside the model hallucination stuff, because I think there's definitely an argument there, but there's also a counter argument of like, well, these models are improving sufficiently. We're doing better and better at removing the false answers and things like that. But the, the, will the revenue show up to match the capex is a big question. My, uh, my friend David Kahn at Sequoia wrote this piece a while back called the $600 billion question, which is probably now like a $2 trillion question. I don't know what the number is today, but it's basically like, that's the amount of capex that has been announced to spend on, on AI data centers. And so that is the bare minimum of the revenue that has to show up to support that. And it, we're, we're nowhere near that level on that side.

• 32:11Shayle Kann?

  So there is a bit of a leap of faith that the market is taking there. And so I think, you know, again, it comes down to this on the demand side alone. And then I definitely want to hear your view on the energy efficiency question. Of course it is very difficult to predict what is the actual demand. If you remove other constraints from the equation, what is the actual demand for AI compute? It does seem clear to me that at a minimum, we are in for a few years of lots of new applications, model improvement, all sorts of things that are going to lead to lots of additional demand. How high that goes though, is obviously I am not equipped to predict it and I'm not sure anybody really is.

• 32:52Speaker 02

  I don't think anyone can.

• 32:52Shayle Kann?

  Right.

• 32:53Speaker 02

  Yeah, I don't think anybody can.

• 32:54Shayle Kann?

  But what I, I, what I do sort of believe is that again, setting aside dramatic energy efficiency improvements, which we will come back to momentarily. That the infrastructure constraint is a real one. And so you don't need demand to be infinite in the near or medium term in order for in order to sort of hit the ceiling of what is what is really possible. Because I do know that infrastructure is a serious constraint, whether it be transformers, or labor, or power, or chips. Right. And so I guess I've come to believe that side of the equation that like, demand is going to be, demand is not going to be the limiting factor. Maybe that's the way to put it. And at least in the near term and the long term.

• 33:34Speaker 02

  The next few years, yeah.

• 33:35Shayle Kann?

  Right. So if you believe that to be true, then the question is one of how dramatic could the energy efficiency gains be? And there I have no idea, but I want to hear what what you think.

• 33:48Speaker 02

  Okay, so I have a lot to say about that. But let me just first turn back to the question of the hallucinations, because I think there's an overall leap of faith that that is a problem that can be solved. And I think it's at least an open question as to whether this particular technology, which relies on probabilities and past, you know, analyzing past data, whether it's possible to fix hallucinations, given all the weirdness that's in the training data. Right. There's human behavior. And then there's malicious actors. And then there's just general human stupidity, right? That gets reflected in this. And so I'm sure there are ways to improve the situation. But the question, which I don't think anyone knows the answer to, is

• 34:32Speaker 02

  can it be improved enough so that the liability associated with the errors that these technologies always seem to create are worth it? And that I don't know. No one knows the answer, but I feel like that's a question that needs to be asked. Because right now people are assuming, oh, it'll just go away with scale or whatever else. But I don't. Think that that's necessarily true. It could be true. Anyway, so that's, that's the, that's another thing on that service demand side that I think people need to question. So, okay, so we've got the whole service demand question of how much demand will there be? And then there's the question of how efficiently can you meet that demand? And that was where Deepsea came in and seemed to show orders of magnitude improvement.

• 35:17Speaker 02

  There's a whole literature on this, right? And it, until recently, that literature has been around the end of Moore's law. So Moore's law is, everyone knows that, right? It's like the doubling of transistors on a chip every couple of years and so on. And that, that went on from 71, when the first market, well, earlier, from 65, when he first started putting these larger ships together till about the year 2000. So they were able to basically double the number of transistors on a chip every couple of years. And as they did that, they ramped up the clock speed. So that improved the performance. But at the same time, they had to ramp down the voltage.

• 35:59Speaker 02

  And this was called dinard scaling, a very important concept. That worked for 30 years, which is pretty amazing, right? In technology that that, where they were able to do that. Uh, but around about 2000 or so, they ran out of the ability to drop the voltage on these chips. You reach a threshold voltage for silicon, about one volt, you can't really go lower. And so what that meant was that they needed to come up with new ways to to improve performance and efficiency. And in practice, people who used computers around that time will know this. Around about 2005 or so, you saw this proliferation of more than one core on a chip.

• 36:41Speaker 02

  So they, instead of having a single chip with a single core, suddenly people started putting lots of cores on these chips, because that was the way that they could improve performance by basically Allowing the computer to do computations in parallel. And it usually required some sort of recompilate, you know, redesign of software, recompilation. But it allowed them for many applications to really speed things up. And so that was a change that went on. So that was a change in what we call hardware architecture. But it turns out there's a lot of things you can do. Hardware architecture is one thing. But then you can improve software, right? You can move, you know, to a different kind of software.

• 37:26Speaker 02

  You can increase the efficiency of the software. There's also improvements in algorithms. So if you have a much more clever way to do the traveling salesman problem, you can reduce the amount of compute you need to do that calculation, right? And that's an example. And then those are tend to be very domain and problem specific. But a lot of times it's orders of magnitude improvement there as well. And then there's the use of what we could call special purpose computing. So the the long term trends that people talk about, usually these are general purpose computers. They're good at, you know, they can do many things. But, you know, they're they're not perfect at doing specific tasks.

• 38:11Speaker 02

  And so if you design a device or set of devices to accomplish one specific task really well, you can actually speed things up a lot more than what the the Moore's law constraint or any of these other physical constraints would indicate. And so that's another thing you can do. So you design special purpose devices. You optimize, you code, code design the software and the hardware. You optimize them to do this thing really, really well. And so between those various improvements that you can get beyond just shrinking transistors and making things closer together, there's orders of magnitude possibility for improving efficiency.

• 38:53Speaker 02

  And so my sense is that the, you know, the focus has been basically building out AI infrastructure. And while there was some improvement, and you can see it in the Nvidia chips over time, there's some improvement. Efficiency was not the major focus because these constraints that you've been talking about haven't been as proximate. So people weren't really focused on it. And this is very similar to that year 2000 to 2005 situation where people were just putting in as much infrastructure as they could. And then they said, oh, my God, you know, we've doubled electricity use, maybe we better focus on this. And that's what I think is likely to happen here as well, is that the industry will actually really focus on this in a way that they haven't yet.

• 39:38Shayle Kann?

  Yeah, you're making like a necessity is the mother of invention type of argument here where like as the, as the infrastructure constraint becomes more and more clear and makes it more and more difficult to build additional data centers, the industry will turn its attention more toward energy efficiency and we will start to see these more dramatic improvements like what we have seen in the past but have not seen here yet. Do you think that we should be thinking about it? Obviously, there's no way to know exactly, but like, I just think it's interesting that that historical analog, same amount of time plus or minus, like, I don't know, we're maybe two years in, so call it three more years of just like boom, boom, boom, would make

• 40:19Shayle Kann?

  a five-year trend after which, and during that time, the sort of early R and D on dramatically more energy efficiency and the, the curve starts to bend a little bit more horizontal and, you know, plus or minus three more years, something like that, like, or, or is it going to be 10 or one, you know?

• 40:36Speaker 02

  Yeah, I don't know. I don't think anyone can say because it's partly dependent on the demand question. If people, let's say they iron out the problems that we talked about in terms of the reliability of these things, then maybe demand is even higher. So while they might be improving, it's always this struggle between, between improving efficiency and meeting the service demand. And when service demand grows faster than efficiency grows, electricity use goes up. When service demand is, you know, offset. Entirely by efficiency improvements, then electricity use is flat.

• 41:14Shayle Kann?

  So, okay, so without being able to actually predict the timing of it, is your best guess that the trajectory of electricity load growth from data centers is going to look something like what it did from 2000 to 2020, which is a spike for a period of time where it is growing faster, followed by a longer period of time where it is growing much slower, if at all, because of some combination of energy efficiency improvements and possibly hitting a ceiling on service demand? Is that the way to think about it?

• 41:49Speaker 02

  Yeah, I think that's a reasonable way to think about it. I don't think anyone can know for sure because it does depend on the growth in service demand. But assuming the industry does what it usually does, which is it sees a problem and it fixes it. And the reason why I'm optimistic about that is because the industry's incentives are to reduce the cost of delivering their product. Right? And that's been historically true for data centers is that energy and capital turn out to be, you know, huge parts of their total cost. And if they can do things a lot more efficiently, they will because it will save them money and make them money. And so it's a little different than some other sorts of energy using applications where

• 42:34Speaker 02

  you have, you know, you don't have these strong incentives. These players have pretty strong incentives to do things in the most cost-effective way. That isn't always true in a boom time. Right? When it's the two or three years of building things out super fast, they're basically just trying to find enough people and enough equipment to put in enough data centers to meet the demand that they perceive. But, uh, but once they, you know, have a little slack in their... In their building, then they can, they can start to think, oh, how could we do this better? Or they see the constraint, right? There's all these stories, you know, I worked at Lawrence Berkeley Lab for many years and

• 43:16Speaker 02

  there's a story of Steve Chu, used to be the lab director there. And the supercomputer guy said to him, hey, we want to build this supercomputer and it's going to use 40 megawatts. And he said, no, it isn't because we don't have that power. And they sent, he sent them back to redesign the whole thing and it turned out to be what, 16 or 18 megawatts and they made it work. But that's the kind of thing that happens when there are real constraints. When there aren't the real constraints, they just, you know, the world is their oyster. They just build what they can. But then, once things start getting big enough, and the scale of these data centers is very large, we're talking tens or hundreds of megawatts. And that's as much as a small city or an industrial plant.

• 43:59Speaker 02

  So it's individual installations can be quite large. And that leads to constraints in transmission and distribution infrastructure and backup power generators. I know the, the waiting time for backup generators was a couple of years. That was a while ago. It's probably longer now because there's been a lot of building. And it just takes time. It's like in the old Star Trek, Scotty would always tell the captain, I can't change the laws of physics, captain. And that's what we're dealing with. The world of bits moves really fast. They have all the money in the world. The world of atoms is slower. So it's like a year for the world of bits, 10 years or 20 years for the world of atoms.

• 44:43Speaker 02

  And, and so it's like a mismatch.

• 44:45Shayle Kann?

  Right, which is what you're saying provides this enormous degree of uncertainty. But, but if I'm understanding you right, like your high level, I think high level, what you are doing is, is questioning the conventional wisdom that it is certainly going to be basically the trajectory of load growth from AI or from data centers is just going to be up and to the right for the foreseeable future as fast as the grid will allow. And you're saying you don't, you don't believe that.

• 45:11Speaker 02

  Yeah, I think that's not, yeah, I don't think that, I don't think that's right at all. And I think it's, I start from the presumption, based on having done energy forecast for 30 years, that nobody can predict more than, you know, a few years out for computing. And I think it's a mistake, like, nobody has any idea what demand, electricity demand for data centers will be beyond a few years from now. I think, as you were describing, we have, you know, we have an idea of what's in the pipeline, right? We know what people are planning and all this. And some of those may not be built. Like, you start to see, like, Microsoft, I think, started canceling some of their leases and other things like that, planned leases.

• 45:52Speaker 02

  So you might see some shifts, but you have an idea of what's being planned for the next two, three years. But that's about as far out as anyone can have confidence. And even then, I think there's uncertainty. So I think anyone who claims to know what's going to happen in 2035, the data center electricity use, I think is just making it up. And I think people should be very skeptical whenever they see these longer-term forecasts. The problem is that the utilities, they need a longer-term forecast.

• 46:20Shayle Kann?

  That's what I was going to say. Defensive utilities, right? Like, they, their entire business is planning around 20, 30-year cycles. Like, I guess that, this just makes it very difficult for them.

• 46:29Speaker 02

  Yeah, and so, one thing that utilities can do, and the most sophisticated utilities are doing, is they can structure the rate designs and incentives in a way that minimizes the risk to existing customers. Because what, what can happen is a new load comes in, and the utility spends a lot of money to meet that load, and if that load goes away, then suddenly, all the existing customers are stuck with the costs that were imposed. And so they're designing the rates and the incentives in a way that reduces that risk to the existing customers. They make the, the, the big data center folks pay upfront, and they have take-or-pay contracts. They do a bunch of other things.

• 47:13Speaker 02

  And so the most sophisticated people, like, in, in Dominion, for example, in Virginia, they're, they've been dealing with data centers for 10, 15 years. So they actually design things in a better way. But not every place is as sophisticated as they are, and they can minimize their projection on certain key by changing the incentives that they offer to these data centers and changing the rate. That they offer and and making sure that their the existing customers are not on the hook. So in a way, you have a forecasting problem, basically impossible to know 10 years from now what will happen. But you also can at least design some of your incentives in a way that will minimize the forecasting uncertainty.

• 47:57Speaker 02

  Because the people who don't want to put up the money for the take or pay contract, they're not as serious. Right? There it's not necessarily, maybe they can't justify it, maybe they can't get their customers to pay for it. They're just kind of they they're speculating or whatever. But the what basically what happens when you start charging people real money for stuff is that they get a lot more selective. Whereas, you know, a lot of times they they, you know, when there isn't a consequence for for speculating. You end up with with a lot more of a scattershot sort of approach. So.

• 48:32Shayle Kann?

  Yeah, that that's clearly the direction of travel. In in utility world, but it is it's complex and, you know, everything is nuanced and location specific in in that.

• 48:42Speaker 02

  Yeah.

• 48:43Shayle Kann?

  Universe. Super interesting. I'm not going to force you to give me a number, but let me ask you this question. I made a bet a little while ago with a friend. Where, okay, so the current, this might be a few years dated now, so you can tell me if the number is dramatically different today. You probably know, but the current total data center energy capacity in the in the United States is something like 20 gigawatts. It's like in that range.

• 49:08Speaker 02

  I think that was true a few years ago.

• 49:11Shayle Kann?

  So it's probably 25 now or something.

• 49:14Speaker 02

  But so I don't remember in gigawatts, but the the you the US number. From the the report to Congress that just came out from LBL. Was something like on the order of 2% of the electricity in 2020, 2021 was data centers. And now in 2023, now 2023, which is the last historical year we have, it's 4.4%. So there was significant aggregate growth.

• 49:45Speaker 02

  Okay, now that's still, if you look at that, you know, in the, from the perspective of the US as a whole, we're not seeing that kind of growth. Like, like the aggregate number, 2023 electricity generation in the US was lower than in 2022. Now, there's weather, and then there's other weird stuff, but the point is, we're not seeing kind of explosive demand growth at the top level. Right? So some of it is shifting of data centers. Some of it is, you know, other uses are getting more efficient, so it's hard to know. So the bottom-up calculation from LBL is that data centers have gone from like 2% to 4.5% in a few years. Right?

• 50:28Speaker 02

  So that's, that's what they calculated, but you can't actually see that in the aggregate statistics. And so that's, that's one of the tensions that we have to kind of try to figure out, or uncertainties we've got to iron out, which is aggregate generation statistics from EIA, which are like, added up from all the utilities, they're not even, like, 2023 commercial sector electricity use was lower than in 2022. Now, in 2024, there's probably a little growth. But it's, but that's, it's, 2023 was about the same amount of generation as 2018.

• 51:07Shayle Kann?

  Overall, not for data centers.

• 51:08Speaker 02

  Overall, everything, so that includes homes.

• 51:11Shayle Kann?

  What you're saying here is that we have seen data center energy consumption as a share of total electricity consumption double, plus or minus, in a few years.

• 51:22Speaker 02

  We think, we think based on a bottom-up calculation.

• 51:24Shayle Kann?

  Right. However, overall electricity generation has not grown during that time, and so the presumption is that the pie was fixed, and it's got shifted around a little bit, and data centers have taken more of it, and so on. But, okay, so, but that aside, here's, here's the, the bet I made with a friend.

• 51:42Shayle Kann?

  We don't have to know the exact number, whatever the current capacity is of, of data centers in the United States. By 2030, Will it double, more than double, less than double?

• 51:53Speaker 02

  What, what, what side of that bet did you take? Like, what was, how did the?

• 51:57Shayle Kann?

  I was on the more than double side. I made this bet about a year ago. So like now there's even less time for it to double. But I was on the side of we will see total capacity of data centers as measured in megawatts or gigawatts, I guess, really, um, double by 2030. And I was on the side that we will see it more than double.

• 52:16Speaker 02

  Well, I think it doubled from 2021 to 2024. So maybe you're, it depends on what, what base year you're choosing, right? It already doubled.

• 52:26Shayle Kann?

  Right.

• 52:26Speaker 02

  Right?

• 52:28Shayle Kann?

  I'm, I'm, I'm saying, let's say relative to today, though. We have five more years.

• 52:32Speaker 02

  Well, so what that means is going from four and a half to 9%, right? Let's, so let's call it five to 10.

• 52:39Shayle Kann?

  That, that assumes everything else is fixed, right? But it may or may not be.

• 52:43Speaker 02

  Because there's EVs and other things. But let's just say it goes from 5% to 10% over in five years. So that's basically 1% increase in demand growth in the aggregate for the U.S. Driven by data centers. Right? So, so you're saying?

• 52:59Shayle Kann?

  That is what that implies.

• 53:00Speaker 02

  Yeah. So, you know, historically, in the last 20 years, we've had much lower growth in the aggregate. That's certainly manageable. It's not what I would call explosive growth. And that's the narrative that you see in the news is there's explosive growth. And it's like, well, no, there isn't. There's no evidence of explosive growth. There is evidence of regional growth, but in the aggregate, we're not seeing explosive growth. We're seeing, you know, 1%. And maybe if you add electrification of vehicles and industry and heat, that's going to add another one or 2% per year. So we're going to see, maybe we're going to see 3% per year growth versus 0.5%, which is what we've been seeing in the last 20 years.

• 53:41Speaker 02

  And so that to me is the reasonable range here. And so you could, it could be that it doubles again. But my point is simply to argue against the, the facile narrative that we're going to see explosive growth because number one, we haven't seen explosive growth. Number two, there are real constraints coming in to affect.

• 54:04Speaker 02

  The electricity use that are related to the world of atoms and the delivery of power generation, backup generation and building transmission and distribution and all that stuff. And so that, those constraints are real and I could, it would not surprise me if, you know, we saw another doubling, but I would say, you know, probably that's about it. I think that's going to level out and then people are going to be more efficient at what they're doing based on the history. So.

• 54:33Shayle Kann?

  All right. I pinned you down as much as I can.

• 54:36Speaker 02

  Yeah, sorry. I'm squirrely about that. I hate to actually predict the number of five, 10 years.

• 54:42Shayle Kann?

  No one, yeah, no one wants to be stuck with a number that they've stated. Um, John, this was a lot of fun. Thank you so much for the time.

• 54:48Speaker 02

  My pleasure. This was great. Thanks.

• 54:52Shayle Kann?

  John Koomey is the founder and president of Koomey Analytics. This show is a production of Latitude Media. You can head over to latitudemedia.com for links to today's topics. Latitude is supported by Prelude Ventures. Prelude backs visionaries accelerating climate innovation that will reshape the global economy for the betterment of people and planet. Learn more at preludeventures.com. This episode was produced by Daniel Waldorf, mixing and theme song by Sean Marquand. Stephen Lacey is our executive editor. I'm Shayle Khan and this is Catalyst.