The truth about data center water usage

Of all the concerns surrounding the AI boom, data center water usage tops the list. Opponents claim that massive hyperscaler facilities threaten local drinking supplies and drain aquifers. Proponents say data centers use a negligible amount of water, at least compared to agriculture or conventional power generation sources. So who actually has it right? In this episode, Shayle sits down with Tim Shedd, founder of Ebullient Insights and an expert in data center thermal management. They unpack the history and evolution of data center cooling and consider how high-density chip racks have developers ditching air cooling in favor of single-phase direct-to-chip liquid cooling. The two also discuss: - Why modern GPU racks (drawing 220 kW to 500 kW+) have rendered traditional air cooling obsolete - The distinctions between single-phase direct-to-chip, two-phase cooling, and immersion cooling methods - How collecting heat at higher temperatures inside a server allows data centers to reject heat outside with far less energy and water inputs - How developers can better communicate with the public about data centers’ water usage and foster local trust amidst rapid buildouts. - Ebullient Insights - Catalyst: What comes after the data center backlash? - Open Circuit: Can data centers regain their social license? - Open Circuit: A feast of hot takes - Latitude Media: Data centers' hidden water footprint is linked to the grid Credits: Hosted by Shayle Kann. Produced and edited by Max Savage Levenson. Original music and engineering by Sean Marquand. Stephen Lacey is our executive editor. Catalyst is brought to you by GridBeyond. GridBeyond is the world’s leading technology platform for managing distributed and flexible energy resources. Learn more by visiting GridBeyond.com. Catalyst is brought to you by Antenna Group, the strategic communications and marketing partner behind the biggest names in energy, climate, and infrastructure. For three decades, Antenna has helped breakout companies amplify their stories, build reputation, differentiate from the competition, define new categories and accelerate growth. Learn more at antennagroup.com. Catalyst is brought to you by EnergyHub. Peak season puts every grid to the test — and the utilities that pass are the ones that built flexible capacity before they needed it. EnergyHub works with more than 230 utilities to coordinate over 2.6 million DERs and more than 3.6 gigawatts of dispatchable flexibility through a single platform designed to perform when it counts most. See what that looks like at EnergyHub.com.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Speaker 00

  Latitude Media, covering the new frontiers of the energy transition.

• 0:07Shayle Kann?

  I'm Shayle Khan. I lead early-stage investments at Energy Impact Partners. Welcome to Catalyst. So of all the fights over data centers, I think water might be the one where the two sides sort of talk past each other the most. Opponents say data centers are draining aquifers and competing with towns for drinking water. Proponents basically say it's a nothing burger. It's a rounding error next to agriculture and other major water consumers. And actually, new data centers barely use any water at all. I think there are two things going on here. The first is about context. So say a 300-megawatt data center with evaporative cooling uses like 150 million gallons of water per year. Is that a big number or a small number?

• 0:48Shayle Kann?

  Just for context, a combined-cycle power plant of the same capacity would use roughly three times as much water. Or actually, maybe even better, 150 million gallons per year is about what you would use on 550 acres of irrigated soybeans. So there's some context. The second, though, is about history versus future. I think even the data center industry would admit that water consumption was significantly higher in the past. But I'll also tell you that modern designs, which might be closed-loop, consume very little water at all, perhaps down to essentially just restrooms and kitchens. So rather than like trying to argue the numbers, I wanted to understand where the water actually goes. What inside a data center needs cooling? What is the challenge there? How did we do it historically?

• 1:33Shayle Kann?

  And what's changed now that a single rack can draw what used to power a whole row? It turns out that the answer to how much water the data center consumes is actually mostly a design choice. But it's a design choice being made right now at gigawatt scale. And the old numbers don't actually tell you much about the way it's going. So to walk through it, I talked to Tim Shed, the founder of Abullient Insights and an expert in data center cooling. That's coming up next.

• 2:09Speaker 00

  Demand flexibility is now an essential part of the data center's speed-to-power playbook. But how do data centers actually curtail load? GridBeyond is the end-to-end solution for flexible energy management. GridBeyond's advanced derms and VPP platform is tailored to what data centers care about most: maximizing reliability, intelligently meeting curtailment obligations, reducing costs, and managing all your distributed resources in one place. Ready to transform your distributed energy management? Get started at gridbeyond.com. Catalyst is brought to you by Antenna Group, the communications and marketing partner behind the most influential companies in energy, climate, and infrastructure.

• 2:48Speaker 00

  For three decades, Antenna has helped ambitious companies stand out, build market leadership, and become category-defining brands. From breakthrough startups to global industry leaders, their team brings the expertise, relationships, and track record to accelerate your momentum. Learn more at antennagroup.com.

• 3:07Speaker 02

  The power plant for the modern grid isn't a building, it's a network. Energy Hub's derms platform connects batteries, EVs, thermostats, and commercial and industrial sites into virtual power plants that over 230 utilities dispatch as reliable grid capacity, day in, day out. VPPs deploy in 60 to 100 days at 40% to 60% less cost than traditional infrastructure. And they perform like conventional generation from day one. Learn about the flexibility platform for the modern grid at energyhub.com.

• 3:41Shayle Kann?

  Tim, welcome.

• 3:42Speaker 03

  Thank you. It's great to be here.

• 3:44Shayle Kann?

  Let's start by having you at the high level walk me through, within a data center, what needs cooling and relatedly, what needs water and why?

• 3:53Speaker 03

  Okay, so that's a big question. But overall, everything needs cooling. Even the little half-watt devices sitting on the motherboard that you never think about. If any one of those fail, the servers fail,

• 4:06Speaker 03

  the power supply fails, whatever. So everything needs cooling. And as a thermal engineer, this is something we're focused on, making sure that that cooling gets to everything, including the power supplies, including the network switches, every possible thing. When it comes to water these days, it's really kind of important to keep in mind that this push to cool everything with water is really only about three years old. So everything we're talking about is evolving. It's moving very fast. And that's going to be part of this discussion, part of many discussions

• 4:44Speaker 03

  that I have is understanding,

• 4:49Speaker 03

  helping people understand that we just don't know the answers to everything. But back to your point, we're going to focus on those high-load devices like the processors. But other devices that are having a hard time getting cool, like memory and network chips and so on, are now also requiring water. And we'll get into maybe talking about the density, but it's driven by the need to put everything really close together in the racks for networking.

• 5:22Shayle Kann?

  Right. Is the simplified way to think about it just that what is happening over time is that we are densifying everything in the racks? And every individual thing is also becoming more powerful. Like every subsequent Nvidia GPU is more powerful than the previous. And also, we are packing as many of them together as we can. And that generates more heat per cubic meter or whatever you want to use. And so there's more heat to reject and more cooling to be required as time goes on. Is that generally true?

• 5:55Speaker 03

  That is generally true. I think it's important to keep in mind that we're not doing this for fun. This isn't like trying to make a Guinness Book of World Records dense rack. We're doing this because we need to get all of the GPUs to talk to every other GPU in the data center. And that requires network cabling, that requires network switches. And we, those cost a lot of money, actually. And we want to keep them as close together as possible. Also, by having the, as many GPUs as possible on the same motherboard, we can use high-speed networking within the server, the MVLink networking,

• 6:37Speaker 03

  or within the rack in the NVL72 case, or in AMD's Helios.

• 6:43Speaker 03

  And there's just a huge advantage to being able to have that high-speed networking. So that's why we're trying to keep them all in a rack together. But then,

• 6:54Speaker 03

  just as you said, every generation is getting higher-powered. Again, that's not for fun. That's because Nvidia, AMD, Intel, they're all learning how to do more with the silicon they've got and run them harder at higher speeds. To the point where, in some cases, they're running them as fast and as hard as they can, limited only by the cooling. Like, we can move faster electrically, we can move faster from a power delivery point of view, in many cases, not every case. And we're limited just by how fast can we get the heat out? And so that gets back to where do we need water? Water is a very effective way to cool these high-powered devices in a limited space.

• 7:39Speaker 03

  And those two things together have driven us to direct liquid cooling.

• 7:43Shayle Kann?

  Yeah, I think there's an interesting, from the outside, I've seen people getting confused about two things that are happening simultaneously. And I think we can tease them out a little bit. One is, as you said, this pretty recent shift toward liquid cooling, which I want to understand a little bit better. And then simultaneously, the push to have the data center overall consume less water. Right? And those two things on the surface would seem to counteract each other. All of a sudden, we're using water for cooling where we weren't. And yet, how are we trying to use less water in the data center entirely? So I think we should, we should separate out those threads for a second. Let's start with the, how are we doing all this cooling?

• 8:28Shayle Kann?

  Can you just talk, before we get into what's happening today, you said the last three years, we've been focused on liquid cooling. Like, what's the, how do we, how do we cool chips and racks in a data center historically?

• 8:39Speaker 03

  Historically, the average rack density was like nine kilowatts. And with nine kilowatts, the cheapest, fastest, most convenient way to deploy racks was in an air-cooled data center where I can just put, kind of pre-install cooling. I forecasted out the next two or three years, I expect to have this load. I can then install the air conditioners around the sides, the computer room air handlers, and then run plumbing to those devices far away from my compute, so I don't have water near my compute. And then I can, I can use some basic principles to make that efficient by separating the hot and cold air with hot aisles or cold aisles.

• 9:24Speaker 03

  But otherwise, I can put my compute in, whatever, from HPE, from Dell, from Super Micro, it doesn't matter, they can, I can use air, right? Air, air doesn't care. And, and in fact, it's, it's not a bad way to go at all. The problem is that pushing the air from the sides of the room into the center requires a fair amount of energy from the fans and there is going to be mixing of hot air and cold air, which makes it less efficient. And I need to get a certain amount of air to every server for every watt that I'm trying to cool.

• 10:03Speaker 03

  And at some point, that kind of breaks down, at least from that model. So what we did to start to try to make things more efficient as we went from nine kilowatts to some high-performance computing, going to 30 kilowatts and 40 kilowatts a rack, is put an in-row cooler next to it. So we bring the cooling closer to the heat source. That saves me a lot of energy, actually. A rear door heat exchanger or an in-row cooler, they have kind of the same basic concept of just bringing that heat exchanger as close as possible to the heat source. That way, my fans are working very efficiently because I'm only moving heat really closely to my compute and I'm only using fan power that I need.

• 10:45Speaker 03

  And if the racks next to or attached to the rear door heat exchanger don't need to be cooled, the fans dial back and I'm saving a lot of energy, actually. But that requires plumbing now onto the data center floor. Made some people uncomfortable, but overall, you know, it's a good idea. But even that gets limited at about 80 kilowatts. We've, you know, was working for Dell and we shipped racks regularly at up to 80 kilowatts a rack with rear door heat exchangers. But beyond that, it gets difficult to pull enough air through to cool. So that's what brings us to today.

• 11:25Speaker 03

  It's not, you know, rear door heat exchangers are actually really efficient and in many cases, almost as efficient as direct liquid cooling up to that power amount. But beyond that, it just starts to get really difficult.

• 11:37Shayle Kann?

  Okay. So to bring it forward to today, then, you said, you know, you were shipping 80 kilowatt racks at Dell. Where are we today? Like, what are the, in a new state-of-the-art frontier inference or training data center? Like, what's the rack density?

• 11:51Speaker 03

  Everybody probably listens to this podcast is aware of the Vera Rubin MVL 72 or VR MVL 72 racks. Those are rated at 220 kilowatts a rack, including the power supplies and everything else. So that's a lot of power in a rack, but that's actually not the densest racks that are shipping today. As an example, Dell is installing the Dudenberg supercomputing cluster at Lawrence Berkeley National Labs and, and that power level kept creeping up. I want to say it's about 280 kilowatts in a rack, but I, I don't remember that exactly. But my point is that that 220 kilowatts is actually not the limit today.

• 12:35Speaker 03

  There, there are ways to keep packing more and more GPUs into a rack even today. And whether they're 100% liquid cooled, like the MVL 72, or they're a mix of air cooling and liquid cooling, like the, the Dudenberg supercomputer, we can still manage that level. In fact, we can manage, we've, we've mapped out up to 400 plus kilowatts, 500 kilowatts in that rack. So that is what's possible today from a cooling perspective and from a power delivery perspective using ORV3 racks and so on.

• 13:19Speaker 03

  So, yeah, we should expect to see that type of density continuing on.

• 13:25Shayle Kann?

  Right. Presumption would be that density continues to increase. That's the direction of travel. And regardless, it's a, you know, to your point, like we could do 80 kilowatts air cooled, maybe we could squeak out a little bit more than that. Regardless, we're multiple of that now. So clearly, we are forced into doing something other than air cooling, which is liquid cooling. So walk me through the mechanism. What is, what does liquid cooling actually look like in a rack? And then, and then we should talk about the water side of it.

• 13:51Speaker 03

  Ultimately, what we have to do is bring water to every device that needs to be cooled with liquid. So the reason I qualify that is there are different architectures for servers today. Some are like the, the eight-way DGX architectures. Today, those are still typically a hybrid cooled solution. So the GPUs and MVLink and other devices are cooled with water, but storage might still be air cooled, network cards might still be air cooled and so on.

• 14:31Speaker 03

  So I'm going to have some mix of airflow through the server as well as, as liquid. And in other cases, like the MVL 72, it's 100% liquid cooled. So I'm bringing water into the server and having to route it to every device, somehow touching every device greater than about a half a watt. So it's a really intense process once we decide that everything's going to be liquid cooled. So imagine just, it's a plumbing problem. I've got two ports and then a supply and a return on the server and I've got to route that liquid to every cold plate, to every piece of copper, basically with enhanced, you know, with fins on it that I'm going to run the water through to pull the heat

• 15:15Speaker 03

  off of, of all the hot devices. In addition, we typically want to make this serviceable. So in other words, we want to make it so that we can remove one GPU out of eight or four that are in there. That means that all those liquid connections have to have some sort of connector that allows us to remove the GPU without spilling liquid all over the place. So in addition to the plumbing to just connect all the copper together that we're using to cool the, the processors, we also have manifolds and we have quick connects and we have hoses and other things. So there's, it's, the, the liquid cooling infrastructure is actually quite significant

• 16:00Speaker 03

  and takes up a significant volume within the server today. So as far as the, the cold plates themselves, they are essentially, just think of it as a, as a, basically a piece of two millimeter thick copper, maybe it's one, one and a half millimeters, depending. And then we've generally created, people call them different things, but skived fins or fins that are made by shaving a bit of copper off the surface and flipping it up. And the typical spacing is about 100 microns thick for the copper and about 100 micron spacing between. I, historically, we call those micro channels.

• 16:44Speaker 03

  Micro channel cooling today has a different connotation. We can dive into that if we want, but they're typically about three millimeters tall and we're going to try to arrange them such that we're flowing water through every one of those micro channels and pulling heat off. And that's non-trivial, but if we do so, they're very high performance. We can now cool a, you know, 1400 watt, 1800 watt processor using 45 degrees C water. That's phenomenal. That's actually something we couldn't do three years ago. So a lot of progress has been made. And then a lot of additional technology and figuring out how to seal those things so they don't, you know, they don't leak easily.

• 17:28Speaker 03

  They can be shipped across the oceans and, and not burst or have any other problems. There's a lot, a lot of engineering has gone into the cold place in a pretty short time to make them reliable and high performance.

• 17:39Shayle Kann?

  Can you orient me a little bit on, there's different architectures within liquid cooling and you hear these different terms. Just give me like the, the sense of direct-to-chip cooling, immersion cooling people talk about, two-phase, like what, how should I think about categorizing the different ways to do liquid cooling?

• 17:55Speaker 03

  Direct-to-chip means bringing the coolant to the chip. And so that's direct-to-chip. And that can be either a single-phase or two-phase. So the most, the dominant form today is single-phase direct-to-chip cooling using water with 25% propylene glycol mixed in. So it's basically Um crudely speaking, it's basically the antifreeze that you pour into your car diluted to 25%. When you go buy it at the auto parts store, it's often 50% pre-mix. The really important detail that you don't even think about when you go pick up a bottle

• 18:35Speaker 03

  of antifreeze is that that gallon jug that you have has a very small percentage, about 1% to 2% maybe, of a proprietary chemical mix, it's called an inhibitor or inhibitor package that prevents corrosion. So your car has aluminum and brass and all sorts of other materials in it and lasts for years without corroding and falling apart. That's actually phenomenal. That's amazing. So our partners that, you know, like Valvoline and Dow and Ricoh Chem, they've done an amazing job developing these coolants that are very long lasting and allow us to use many different materials in their systems. That's the same thing we

• 19:20Speaker 03

  have to pay attention to in the single-phase liquid cooling system. So that's single-phase liquid cooling. Two-phase liquid cooling is not yet really available at scale. There are companies working on it. I myself had a company that worked on this. So I'm a little, you know, have some history with this. But there are definitely products that are very close to commercialization in this space. And it's where you take a refrigerant, you pump it to the chip, and you literally boil the refrigerant off of the chips, you know, off of the fins. It looks really cool. Bubbles are great. It turns out that the performance is very similar to single-phase. It's not really superior in performance, but there

• 20:05Speaker 03

  are other benefits. I can use much smaller hoses and tubes and so on. So remember I was saying currently the DLC takes up a lot of real estate inside of the servers. Two-phase promises, again, not mass adopted yet, but promises to use a lot less space inside of the servers, leaving more space for innovation on the actual compute. Immersion can take two forms in single-phase and two-phase. In single-phase, we use an immersion fluid, a coolant that is often derived from something like mineral oil, but it's generally much more sophisticated than this and has additives and other inhibitors as well. The problem

• 20:47Speaker 03

  with oil is if, you know, if you can imagine it's a hot day and you're getting excited, you jump in the pool and that pool is filled with oil, that would feel pretty not great. It wouldn't cool you off, right? Because oil is actually itself a pretty poor heat transfer fluid. And so it turns out that it's just not, it's good for picking up low-density heat from a large area. It's not great for picking up heat from the high-performance processors. And so to do that, they're proposing, the immersion community is proposing putting cold plates on the chips themselves and then picking up all the rest of the heat, you

• 21:28Speaker 03

  know, from those small devices with this immersion fluid. And there are pros and cons of that, but that's the, that's kind of the trajectory that they're on. Two-phase immersion is coming, coming and going and coming again, maybe. The challenge with that is I need to be able to have a fluid that boils and doesn't get, doesn't allow the combination of vapor and liquid together in a container to get at too high of a pressure. It's a little complicated here, but that boiling point of that fluid really matters because if the pressure rises even, you know, to a third of an atmosphere and I'm in a great

• 22:12Speaker 03

  big tank, that can create a lot of force on the lid and create a negative, you know, very bad situation. So I have to manage that vapor, I have to manage the expansion. And so it's complicated there. And there are other things, we actually published, me and some colleagues published a paper on this about four years ago, three and a half years ago now, that went into some of the details of the limitations of two-phase immersion. And it's again, going to be kind of stuck at lower powered, still pretty high-powered, you know, like 1,000 watt processors, but it's kind of stuck there in that region for, you know, thermodynamic reasons. So, so really the emphasis in the industry is on the single-phase

• 22:55Speaker 03

  direct-to-chip cooling because it's ready to scale. We have a supply chain that's very robust worldwide. We have liquid coolant suppliers that are now providing, able to provide high-quality coolant worldwide. This is what we need because we're at the state where we're producing and consuming hundreds of thousands of cold plates a month. And by next year, that could climb to maybe close to a million cold plates a month. I mean, this is crazy. You know, in 2022, we might have been talking about 100,000 cold plates a year, right? In the whole industry. So something on that order, right? And so we're just scaling super fast. And right now, the technology that can scale and keep up with it is single-phase direct-to-chip.

• 23:42Speaker 00

  For data centers, connecting to the grid quickly means two things: building distributed energy and having a plan for demand flexibility. GridBeyond helps with both. GridBeyond's DERS and VPP platform unlocks the potential of distributed assets and dispatches flexibility into the right market at the right time. And this is more than software. GridBeyond supports developers every step of the way, from modeling your highest ROI energy mix to orchestrating the full set of assets around a data center. The track record speaks for itself. GridBeyond manages over 1,400 sites and more than 3.7 gigawatts of load across Europe, the UK, and the US. Learn more at gridbeyond.com or click the link in the show notes.

• 24:24Speaker 00

  If you work in energy or climate, you already know the company's Antenna Group has helped build. For three decades, they've been the communications and marketing partner behind the industry's breakout companies, category creators, and market leaders. Today, their team is working across the most consequential areas of the energy economy, from data centers and grid modernization, to nuclear, geothermal, storage, critical materials, and next-generation technologies. Antenna helps ambitious companies become the names everyone in the market knows, trusts, and wants to work with. They build reputation, elevate executives, drive demand, and turn momentum into market leadership. If your company is ready to lead its market, Antenna is ready to help you get there. Visit antenna group.com.

• 25:07Speaker 02

  When you harness the power of millions of DERs in a VPP and put artificial intelligence behind the dispatch, the outcomes exceed conventional generation. Energy Hub's DERS platform has been trained on years of real dispatch data from over 2.6 million DERs, so utilities can plan with confidence. That means consistent load reduction across a four-hour event window, even during the widest, most uncertain peak periods. Utilities target, shape, and verify capacity down to the meter. Unlock flexible capacity that costs 40 to 60% less to deploy than traditional infrastructure in a fraction of the time it takes you to break ground on a peaker plant. Learn more at energyhub.com.

• 25:52Shayle Kann?

  Okay, so now explain the relationship between the cooling architecture and water consumption at the data center level.

• 26:01Speaker 03

  They're actually kind of disconnected. So, and let me explain that. So what, what we want to do is break down the journey of the heat from the chip to the outside into basically three parts, right? We're going to have to collect the heat from the servers and then move that heat through some fluid, air, liquid, whatever, to a exchange point between what we call the TCS, the technology cooling system, which is the part that collects the heat, to the FWS, the facility water system, which is the part that moves the

• 26:46Speaker 03

  heat to the outside, heat rejection system. And so the heat rejection system is going to be whatever devices that the data center designer has chosen for that climate, for that region, to be chillers, cooling towers, dry coolers, whatever's appropriate for that particular region and power level. And ultimately, that interface between the TCS, the cooling architecture that's collecting the heat, and the FWS is just a heat exchanger, regardless of whether I've got two-phase DLC, single-phase DLC, or immersion, there is a, typically

• 27:31Speaker 03

  a plate heat exchanger that interfaces between the two systems. And the FWS doesn't care what was on the other side. What we care about is having the highest possible temperature on the TCS rejecting the heat to the FWS, because I want to move the heat from the processor, which is the hottest point in the system, to the outside. I want to try to move that heat downhill the whole way. Then I can just have thermodynamics carry the heat out without requiring an energy input other than pumps. And that's the most efficient way to move the heat out.

• 28:06Speaker 03

  The challenge is that in many climates, if the outside temperature rises up above, you know, 100 degrees Fahrenheit, you know, 43 degrees Celsius, that's, it's pretty warm. And that's actually warmer than our FWS often wants to be. And so that means we have to push heat uphill and then I need a chiller. So, so what we're doing is we're looking at the climate and we're looking at the heat load and we're looking at how much time are we spending in the year, you know, in a, in a case where I'm having to push heat uphill? And if it's just, you know, a relatively small number of hours, I might use something called an adiabatic cooler, right? It's basically a dry cooler with some pads and I spray the

• 28:51Speaker 03

  pads with water. This is oversimplifying. My colleagues in the, in the heat rejection industry will, will grade at that simplified solution. But basically, it's a, it's actually an elegant solution that allows me on those days when I need to, to dump heat out to on a hot afternoon, I can do so basically with just a little bit of water and do it very efficiently and get that done. Other climates, I'm going to spend half the year requiring, you know, pushing heat uphill. And then I'm just going to need a refrigerated chiller. And I use a lot of energy. So I, I kind of went, I went a long ways from your

• 29:33Speaker 03

  original question, the, but I, I want to kind of disconnect. What our, our goal is to get the best technology on the TCS that generates the highest temperature to feed the FWS, to feed that, the highest temperature to, to the heat rejection system so that I can use hopefully the least amount of energy to push that heat out into the environment.

• 29:55Shayle Kann?

  So that, that all makes sense, but then what does dictate the water consumption, the total water consumption of a data center?

• 30:02Speaker 03

  So most, sorry, I kind of blew past it because most waters, most data centers don't use much water at all. And they historically haven't used much water. And it's kind of frustrating that this whole discussion is even going on. But here's the deal. When we were air cooling everything, hyperscalers who are building out thousands of the same rack over and over and over again, their biggest cost is energy input to the data center. And so they were looking at this and saying, well, the most energy efficient

• 30:43Speaker 03

  way for us to cool this data center is evaporating water. And they were elegant systems, they are elegant systems, that used kind of like I was describing earlier, basically spraying a material, a filler, filler material and blowing air through it, using evaporation to cool that air down. That's actually a smart idea. It uses a relatively small amount of water for the amount of cooling power generated or the amount of cooling that's being done. But for those select data centers, it did consume, relatively speaking, a lot of water because I'm cooling megawatts, right? And so, and for every, I'm going to use a

• 31:26Speaker 03

  kilogram of water for every two megawatts or something like that. So over time, in some communities, they paid attention to this. And, and, and to be frank, the hyperscalers are very transparent about this. Google, Meta, Microsoft, very transparent. They were honest about the amount of water they're using. That has kind of come back to, to, now, I don't want to be over negative. People should be concerned about, people should be paying attention to these things. And I don't want to be negative on that point. But the problem is the data centers we're building today aren't like that. We're not cooling everything with water like that. Even when we're using a cooling tower, we're

• 32:09Speaker 03

  frequently pairing the cooling towers with dry coolers. So we're only using the cooling towers to, to trim the heat rejection on those hottest afternoons. So as far as how much water consumption is there, I don't know that number precisely because every data center is going to be different. Most data centers are built completely closed loop with no cooling towers, even no evaporation. Some are built with, like I said, some evaporation on the hottest afternoons, just for a few hours a year, or maybe tens of hours a year. Others are more dependent on, on cooling towers, but they're using it more for trim cooling or, or just, just the watts that need to be cooled on those hottest afternoons.

• 32:54Speaker 03

  And so what we do know is that the hotter the water, so using a cold plate and direct liquid cooling at the TCS gives me hotter water to the FWS, which gives me more options on the heat rejection, which allows me to use less water if I have to use water at all. So the lesson here is do the best I can collecting heat so that I can be as energy efficient rejecting the heat as possible.

• 33:26Shayle Kann?

  So I guess it'd be interesting maybe to separate out then, you know, clearly what has happened over time is that data centers have, more recent data centers are consuming less water on net. You could make the case that they were never consuming that much water, but it's like all a matter of perspective. Either way, on a relative basis, they're consuming less now. And it seems like you've given sort of two reasons for that to be true. One is a different optimization, basically, where like early on, the hyperscalers were optimizing predominantly for energy consumption. If you're optimizing for energy consumption, then you do adiabatic cooling or whatever, and, and do use some

• 34:05Shayle Kann?

  water. And now if you're optimizing a little bit more for low water consumption, then you just choose a different method of cooling and it, you use less. That's, that's one reason. The other reason being the sort of technical thing that you described of like getting higher temperatures in the TCS so that you have less heat to reject. I don't know if it's possible to separate those two from each other, but like how much of the change in overall water consumption in, in modern data centers versus previous generation data centers, do you think is from this re-optimization versus the technical advantage of a higher temperature TCS?

• 34:42Speaker 03

  I would say that a large percentage is using direct liquid cooling or, or more efficient TCS technology. Because if I'm air cooling and I'm not consuming water to do that air cooling in the data center, I'm requiring cold water there to, to generate the cold air. I've got to have cold water going through the heat exchangers to generate the cold air. And that's going to mean that I'm going to need chillers. And the most efficient way to run the chillers is often, depending again on the climate and many other things, is often with a cooling tower providing cool water to the chillers. So the chillers

• 35:24Speaker 03

  are consuming less energy, but I am consuming some water. So it's, it's all connected to this balance, as you noted, between water usage and energy. But if I start by picking up heat with 45 degrees C water at the chip, and that, then my return is 55 degrees C, I mean, that's 135 or something Fahrenheit. I don't remember the exact conversion. I can reject that heat in most places, except the worst day in, you know, middle of India in the summer. I can reject that outside relatively easily. Right? And so that has

• 36:07Speaker 03

  made it, without the use of consuming water or, or, you know, a whole lot of energy even. So that's just made a huge difference. As we moved into this AI deployment, and it's one of the reasons why Jensen makes a pretty big deal about this in his, his keynotes, is it's enabled this build out to occur with, you know, optimizing the whole supply chain, whether it's chillers, whether it's dry coolers, or whether it's water consumption. So, so that said, I, I do want to come back to the fact that these are really large data centers, and some of them have really huge piping infrastructure. And there's a certain amount

• 36:51Speaker 03

  of water required to fill all those pipes. And during the installation and startup, we have to flush and clean those pipes. And that does use water. Now, it's not all consumed. Initially, I can say we were probably dumping a lot of that water. But now, and just, again, remember, this is happening over a relatively short time period of a few years. Now you'll find that most of the companies are pretty sophisticated about that. They have pumping stations that come out and they're reusing the water. They're re-filtering it onsite often, so that there's very little wasted water. But that, and, and so it, it's still

• 37:32Speaker 03

  likely that there is water used from the local water supply to do the filling and cleaning. But even then, it's not always from the local water supply. It's, it's coming, you know, that, that water may be trucked in from, you know, Lexington, Kentucky, where one plant is and from, you know, wherever, right? And so there's just not a really strong correlation to the water being used to fill and, and operate a data center and actually where it's being drawn from. And so

• 38:07Speaker 03

  Again, it's a, it's a, it's a issue with a whole lot of variables to it. And I don't want to say that a community that's concerned about water and has seen their water table drop, that they're not seeing an issue. I don't want to claim that, you know, that they must be wrong. On the flip side, I think if you dig deeper, there's probably many, many reasons for that, right? And, and it's not, it's possibly not because of the data center solely, or it's possibly not because of the cooling system. It may be that that water is used to build the roads to the data center. So maybe the data center is still responsible, but you, it's, it's not a continuing use. All I'd say is this is a complicated question, but on the whole,

• 38:50Speaker 03

  water use in data centers is pretty low compared to many other sources of consumption. And we are continually innovating and I believe making more efficient power and cooling systems such that we're consuming less water for the power that we're using.

• 39:09Shayle Kann?

  I guess to ask you a final question that maybe steps one degree outside of your area of expertise, because it gets more into this like question of community acceptance. You know, I think this conversation has, has clarified why this is challenging because on one hand, you can look at the raw numbers of like, how much water have data centers consumed historically and then currently. And again, those numbers are changing, like it's, it's falling, the numbers are falling, but you can look at the raw numbers. But, but you know, this, this concern that has arisen and the fact that water consumption, perception of water consumption is one of the two or three big issues right now in the sort of data center backlash. I

• 39:52Shayle Kann?

  think it's difficult to combat, given the nuance associated with it. It would be easy if it weren't nuanced. And if every new data center could just say, I promise zero net water consumption. Right. But in the absence of that, and here's how I'm going to do it. But in the absence of that, it is, it's complicated. And it's like one of these things that like political messaging doesn't do a good job of describing. So I wonder if you have a view on either from a technical and architectural perspective, like should modern hyperscale data centers all do X, Y, or Z? Or from a messaging perspective, what is the right way to communicate the relationship between data centers and water consumption moving forward?

• 40:37Speaker 03

  A tough question for sure, but in, in the past year, I would say, you know, Microsoft has been a great example of messaging. They got out front of this question. They were very aggressive in messaging to the community in Wisconsin, where they were building a large data center. I felt that they were messaging very clearly how they were going to cool and how much water they intended to use and how they were, you know, achieving this. So kind of doing what you suggested there. Unfortunately, I still see a backlash to that data center and, and I still see water being brought up as a, as a, as a point. And,

• 41:21Speaker 03

  and my feeling is that that number one, it, I think it's table stakes to take the approach that Microsoft did in that case. And I think as an industry, I think we're getting better about that. Maybe a little too slowly, but I think we're getting better about being really transparent in the communications. I think we could do better, literally just inviting community in to see how this works. If that's possible, there's a whole lot of safety and other things and, and, and some privacy concerns. But, you know, I think that would help people

• 42:03Speaker 03

  see, okay, I get this. On the, on the other hand, I think being honest about, hey, building a great big building that's going to consume 500 megawatts of power just uses water. It, it does. Whether it's just keeping the dust on the construction site down, or preparing the roads, or, or, or, you know, pouring the concrete, whatever, right? It's going to consume water. And so there's going to be a period where water is consumed, building a huge data center campus. And, and communicating that might help. But overall, I, I, this is an opinion or a belief. I, I, again, this isn't my, my area of expertise, but you know,

• 42:47Speaker 03

  I'm in the, I'm in this, the waters like you are. And seeing that, I wonder if a lot of this just isn't a general feeling of discomfort of this new technology that's being built out all around us. And we're all uncertain about how it's going to play out. And it's pretty easy to take it out on these big nondescript buildings being built and consuming resources. Right? And, and so I think what we can do as an industry is continually perform outreach, try to explain what we're doing, be as transparent as possible, try to get away from all the NDAs. I, I understand sometimes why they're there. But that just

• 43:30Speaker 03

  breeds distrust. And we all know that. Again, I understand the reason for, for that sometimes. But just, just being transparent. And then there, there's not much we can do about communication about AI and its application these days. That depends on a few loud voices that you and I don't have any control over. But, you know, but, but really trying to work as an industry to be transparent and, and hopefully bring focus on, on the applications that people do like and do use. I think, you know, autonomous vehicles and autonomous taxis are, are being used in a huge number of communities and a lot of people really like them, you know, Waymo and CyberTaxis now and, and, and other options. That, those wouldn't be possible

• 44:14Speaker 03

  without these data centers. And, and of course, all the functions on our phones and whatnot. So don't need to go down that path. But, so it's, it's, I think this ends up being more of a, a human issue than, than a technical issue and, and even than a communications issue. And it's going to take a little while to resolve. But transparency, I think, is the first, first step.

• 44:36Shayle Kann?

  All right, Tim, this was really interesting. Very much appreciate your time.

• 44:40Speaker 03

  Thank you.

• 44:42Shayle Kann?

  Tim Shedd is the founder of Abulian Insights. This show is a production of Latitude Media. You can head over to latitudemedia.com for links to today's topics. This episode is produced by Max Savage Levenson. Mixing and theme song by Sean Marquand. Anne Bailey edits the video version of the show. Stephen Lacey is our executive editor. All of our episodes are on YouTube. Subscribe to Latitude Media for episodes of this show and Open Circuit. You can also find the audio version of the show and not have to look at my face anywhere you get your audio podcasts. I'm Shayle Khan. This is Catalyst.