Phaidra’s Jim Gao on Building the Fourth Industrial Revolution with Reinforcement Learning

After AlphaGo beat Lee Sedol, a young mechanical engineer at Google thought of another game reinforcement learning could win: energy optimization at data centers. Jim Gao convinced his bosses at the Google data center team to let him work with the DeepMind team to try. The initial pilot resulted in a 40% energy savings and led he and his co-founders to start Phaidra to turn this technology into a product. Jim discusses the challenges of AI readiness in industrial settings and how we have to build on top of the control systems of the 70s and 80s to achieve the promise of the Fourth Industrial Revolution. He believes this new world of self-learning systems and self-improving infrastructure is a key factor in addressing global climate change. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital Mentioned in this episode: Mustafa Suleyman: Co-founder of DeepMind and Inflection AI and currently CEO of Microsoft AI, known to his friends as “Moose” Joe Kava: Google VP of data centers who Jim sent his initial email to pitching the idea that would eventually become Phaidra Constrained optimization: the class of problem that reinforcement learning can be applied to in real world systems Vedavyas Panneershelvam: co-founder and CTO of Phaidra; one of the original engineers on the AlphaGo project Katie Hoffman: co-founder, President and COO of Phaidra Demis Hassabis: CEO of DeepMind

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  A lot of times, like when we talk about AI, right, both in the valley and elsewhere, I think there's a conflation between AI and automation, right? AI can absolutely automate things. There's no doubt about that, right? Especially like routine things, right? But I think that honestly undersells the real promise of AI, right? I think the real promise of AI is what Demis Sisio of DeepMind calls, you know, like AI creativity, right? It's the ability to acquire knowledge that did not exist before, right? And I, of course, experienced this firsthand. The reason why I'm such a true believer in the technology is because, again, I was the expert who helped design the system, but this very AI agent that we created is telling

• 0:44Speaker 00

  me new things about the system that I didn't know about before, right? And that's a very, very powerful feeling.

• 1:06Speaker 01

  Hi, and welcome to Training Data. Please welcome Jim Gao, founder and CEO of Phaidra. Jim was previously the leader of DeepMind Energy, one of the first and only AlphaGo-style reinforcement learning applications in the wild. DeepMind Energy used reinforcement learning to manage Google's data centers and drove some staggering metrics, including 40% energy savings. We're excited to ask Jim about reinforcement learning in the industrial world and to learn more from him about what other real-world applications are poised to be transformed next by deep reinforcement learning. Thank you so much for joining us. Maybe before we get started, we're going to spend a lot of time today talking about your DeepMind Energy journey, but maybe can you give everyone one or two sentences on your background and what you're building?

• 1:49Speaker 00

  Yeah, of course. So Phaidra is an AI company, of course. Fundamentally, we are an AI automation company. So what we do is we use a type of AI known as reinforcement learning to directly control and operate our customers' very large, mission-critical industrial facilities. So in practice, these AI agents, they act as virtual plan operators, virtual members of the plan operations team.

• 2:14Pat Grady?

  Let's go back in time and talk about the journey that led to this journey. And I believe that you once sent an email with the subject line, "Reinforcement learning plus data centers equals awesome." Can you tell? Question mark, yes, awesome. Sorry, sorry. Reinforcement learning plus data centers equals awesome? Can you, can you tell us, who did you send that email to? Why did you send that email? What was on your mind at the time? And then, of course, what did that lead to?

• 2:45Speaker 00

  Yeah, of course. So the reason why there was a question mark is because it was generally an unknown if the combination of reinforcement learning with industrial facilities would actually be awesome. So that was an email that I had sent to a person named Mustafa Suleyman, who would later become my boss at DeepMind. And really, the impetus was something called AlphaGo. So to set the stage properly, I had been experimenting as part of my famed 20% time at Google with machine learning technologies. And it was actually a very specific course, Introduction to Machine Learning by Andrew Ng and Coursera, that had just come out. This is back in 2013, mind you. I think I was like the second cohort or something.

• 3:29Speaker 00

  And that class had completely changed my life. I taught myself how to program and just started tinkering around with machine learning on the side. It was just, it was a very interesting technology.

• 3:37Pat Grady?

  And your background was mechanical engineering and environmental systems type thing?

• 3:42Speaker 00

  That's absolutely right. So my responsibility at the time was to, one, help Google design and operate their very large data centers. And once these very large data centers, which consume enormous amounts of energy, were built, we, of course, shifted our focus to operating these complex industrial systems in the most energy-efficient way possible because they use billions of dollars in electricity. So that was kind of the background. I was already tinkering around with machine learning technologies on the side to analyze the enormous amounts of data that Google's data centers were generating. In 2016, AlphaGo came out and I was one of hundreds of millions of people around the world, you know, watching. It was like 3 a.m. in the Bay or something and I found

• 4:23Speaker 00

  it absolutely captivating and to the point where I sent an email to Moose, right, describing this idea that if DeepMind could beat the smartest, most intelligent people in the world at complex games like Go, then surely we can train these same AI agents to play a very different game that I'm familiar with called "Let's Optimize the PWE: The Power Usage Effectiveness at Google's Data Centers," right? So that was the context for that email and I remember internally the way I pitched it to Google's leadership, so specifically Joe Kava, who leads Google's data centers and Ours, was I showed a picture of a Go board on one side and a video game controller, like an Xbox controller on the other and I'm like, "Look, there are objective functions

• 5:08Speaker 00

  that we're trying to minimize or maximize. There are concrete knobs and levers, so actions that we can control. There are constraints that we have to stay within and all of this happens within a very measurable environment." I think, you know, reinforcement learning and operating large, complex industrial systems are actually one and the same thing, right? So that was the original kernel of insight, I guess, that inspired it all.

• 5:32Pat Grady?

  And I know Sonya has accused me of going rogue with some of the questions we ask here. I'm gonna go ahead and go rogue for a minute.

• 5:37Speaker 00

  Already? It's been like one minute.

• 5:39Pat Grady?

  We're gonna come back. I wanna skip the story. This is a brief diversion. Bear with me. The three things you mentioned that allowed you to see the parallel between reinforcement learning and control systems or control theory: objective function, actions, constraints.

• 5:54Speaker 00

  Yep.

• 5:55Pat Grady?

  Are those the three key ingredients for where reinforcement learning can be applied to real-world systems?

• 6:01Speaker 00

  Yes, absolutely. That is 100% how we think of it, right? You know, the... Reinforcement learning systems, they need like KPIs to optimize for. They need to know how good or bad an action is, right? They obviously need things to control and they need to know what are the constraints they have to stay within. So really what we're saying is, as long as we can map, you know, the problem we're trying to solve into a reinforcement learning framework, which really from a mathematical perspective, what we're saying is we're solving a constraint optimization problem, right? If you can map the constraint optimization problem, if you can define it and map it to the underlying data, then it should be able to be solved using reinforcement learning, right?

• 6:44Speaker 00

  So that's very much the lens through which we look at things at Fager as well. And, you know, to take it one step further, you know, we often talk about how, you know, reinforcement learning and, you know, controls and optimization are like two wildly different fields historically that have somehow independently converged to the same area, right? Like they're two very similar concepts, well, we've been calling them by different names this whole time. So you've had like almost these independent evolutions, right? A different ways of tackling the same problem. And Fager is really kind of the intersection of both of these.

• 7:18Pat Grady?

  Okay, let's get you back onto the story. So you sent the email, sorry for the diversion. So you sent the email to Mustafa and then what happened?

• 7:24Speaker 00

  Yeah, so we sent the email to Mustafa. Two weeks later, Moose had actually flown out to Mountain View, right? Whereas working at the time on Google campus with a team of DeepMind folks. And we actually started mapping out exactly like how reinforcement learning could be used to control and optimize Google's data centers. So that actually kicked off the original partnership between Google and DeepMind around the application of reinforcement learning for the data center work. It was, you know, it was very, very fascinating. But, you know, most importantly, it's actually also how I met one of my two other co-founders, right? So Veda was one of the original engineers on the AlphaGo project. So he had gone to go to, you know, he went to Seoul, South Korea, right?

• 8:10Speaker 00

  And, you know, he actually got to meet Lee Sedol and Larry Page and all, you know, all this fun stuff. And After AlphaGo, he came back to the States, or rather to the UK, and he was wondering, well, what is my next big thing going to be? And I managed to convince Veda, like, hey, what if we applied self-learning frameworks like AlphaGo to control and optimize Google's data center? So that's actually how I started working with my co-founder, Veda.

• 8:36Speaker 01

  Did people think it was going to work? Or was it like, this is a crazy moonshot, let's just try, but who knows?

• 8:41Speaker 00

  I didn't even know if it was going to work. Like, conceptually, it made sense in my mind, right? I'm like, hey, you're like, you know, this is, it's a... Operating a data center is just a different game to play, right? And there's all kinds of different games in the industrial world, right? Maybe the game is maximize energy efficiency. Maybe the game is minimize water consumption. Maybe the game is maximize the yield of a factory, right? But there's all these games that we're constantly playing, right? So in my mind, it made sense. But to answer your question directly, no, I had no idea if it was going to work. I still vividly remember to this day when we turned on the AI system and we watched the energy just drop. And it was so surprising for two reasons.

• 9:23Speaker 00

  Number one, well, we had designed the system. I played a role in designing that very mechanical system, right? That the AI was now controlling and optimizing. So in theory, I'm literally supposed to be the subject matter expert who knows everything about these systems. But the AI is teaching me things that I didn't know about the system I helped design in the first place, right?

• 9:46Speaker 00

  And two, the moves that the AI was making were just very counterintuitive, right? Like, when we looked at the decisions that were coming out, you know, we looked, the plan operators and I, we were sitting in a giant cornfield in Iowa, right? Where like Google likes to put its data centers. And we were looking at the decisions and we thought to ourselves, there's no way this is right. Like, this AI sucks, it learned the wrong thing. But we're here anyway, so let's try what the AI is saying. And we tried it, and it worked, and we just, we saw the energy plummet. So I think that was kind of when I became a believer in this technology, right? That fundamentally, this technology...

• 10:26Speaker 00

  It's creative. It helps us discover new knowledge that didn't exist before from raw data.

• 10:32Pat Grady?

  Was there a performance trade off? Or was this just straight up Pareto gain? Like, performance health and

• 10:37Speaker 00

  That's a great question. No, it respected exactly the same constraints as the plant operators and engineers had already put in place. So this is pure gain, right? Respecting exactly the same temperature profile is exactly the same constraints around how quickly you can turn on and off a chiller, minimum pump VFD speeds, all that sort of stuff. So this is pure optimization, pure gain, which I think is one of those crazy things. Like, we don't really expect... Like, usually when you think about energy efficiency, for example, right? Like, in the world that I come from, people usually think about expensive CapEx. Like, oh, we gotta rip out the chillers. We gotta buy a bunch of new chillers from Johnson Controls and Trains or whatever, and then we have to install them. So they're like hardware efficiency gains, right?

• 11:20Speaker 00

  But you don't really think about like pure software, like data-driven efficiency gains, right? And I think this part of it was surprising for us.

• 11:28Speaker 01

  Can you walk us through the before and after? Maybe before, what you all implemented? Was this industrial control systems? Was this manual plant operators turning knobs? Like, how did this work before and then after?

• 11:40Speaker 00

  Yeah, it's a great question. So let me set the stage for, you know, for folks who are not as familiar with like, large industrial facilities, right? So, the very... Modern industrial facilities are very, very complex, right? There's all kinds of machines that people are operating and controlling, right? So, you know, I often, you know, tell folks to do like a simple thought experiment. So imagine you have just 10 machines you're controlling. So say they're like pumps, right? And each one of those machines has 10 possible set point values. So 10 modes associated with it. So think something like 10% pump speed, 20% pump speed, 30% pump speed, etc., right? Then, in this very simple toy example, you have 10, raised to the 10, or 10 billion different permutations

• 12:25Speaker 00

  for how you can operate your toy system, right? So then the question becomes, well, at any given point, what is the most optimal way of operating your toy system? And by the way, these are dynamic systems, right? So the IT load is changing, the weather is fluctuating, right? You know, the people operating these systems are changing, the pipes are corroding, the heat changes are fouling, right? So the point is, these are very complex dynamic systems. Real-world systems have a lot more than 10 machines, and each machine has a lot more than 10 set points, right? So you can start seeing why technologies like AlphaGo, which managed to navigate IMAX complexity, are helpful over here. It also helps explain why there's often so much room for optimization in the first place. Because of so much complexity, right?

• 13:09Speaker 00

  Like, if you think about the total action space, right? Like, all the possible actions within a modern data center, for example, right? Because of risk averseness, but also because of hardcoded rules and heuristics, right? We've only ever explored like 0.00001% of all the possible ways that you could operate that system. So then the question becomes, what is in this 99.99999% of the action space we've never explored? Surely, there are more optimal ways of operating the system than what we've done historically. So it's kind of an intuitive explanation, hopefully, of why there can be such large efficiency improvements in the first place that are undiscovered. And the way that we operate these facilities

• 13:53Speaker 00

  is constrained by a mixture of hardcoded controls logic, right? So don't get me wrong, these are automated systems today already, right? They're just not automated intelligently, I would argue, right? And, you know, there is a healthy mixture of human intuition as well, right? Where we have people like myself or plant operators who are constantly monitoring the system, who are like nudging the system, adjusting things, or adjusting the rules, right, for that system, the constraints that the system has to operate within. But fundamentally, human intuition plus hardcoded controls logic is still limited when you talk about this degree of complexity, right?

• 14:35Speaker 01

  Yeah. Can you talk to us about the key results? So you saw the energy levels drop immediately.

• 14:41Speaker 00

  Yeah.

• 14:41Speaker 01

  But, you know, what, what results were you able to drive to, for Google?

• 14:44Speaker 00

  Initially, so there's, there's two types of results, right? You know, for, for Google in particular. There was the results from the pilot. So in 2016, we, we released, we announced like the results of the pilot, right? Now, the pilot was done on a couple of data centers, but fundamentally it was not an autonomous control system, right? So what I mean by this is it was the AI generating recommendations, right? Which for, for human experts like myself to manually review and implement. And of course, you know, we didn't want to jump straight to taking our hands off the steering wheel, right? Cause it's a new novel technology, right? But also like, no one knew at the time, like, is it even possible to use AI from the cloud to control big ass infrastructure, right?

• 15:28Speaker 00

  So, um, so step number one was do the pilot, right? The AI generated recommendations. That's where we saw like really steep, like 40% energy savings, right? Now, that experience taught us like, hey, we think there's something real over here. We should actually just let the AI control things directly, right? To get the value automatically. And also, quite frankly, uh, the pilot operators were getting tired of checking their email like every 15 minutes, waiting for the AI to tell them what to do, right? Or the manual implement things. They had better things to do, right? So, um, so we actually decided and rather, uh, Ours and Joe decided like, hey, it's time to go to a fully automated system, right? This was total uncharted territory.

• 16:11Speaker 00

  At that point, like, forget about can AI control things? We didn't even know, like, is it possible to control machines from the cloud? Like huge industrial infrastructure in the cloud? Cause to our knowledge, no one had done it before, right?

• 16:22Pat Grady?

  Is, is it fair to assume that a lot of the hardware, a lot of those machines are things that Google built from scratch? Or does Google use a decent amount of like commercially available data center stuff?

• 16:31Speaker 00

  It's a mixture of both, right? So, you know, obviously like Google, you know, um, does a lot of things in-house, right? But it doesn't manufacture like chillers and that sort of hardware. So Google does buy like off the shelf hardware, but you know, there's a lot of like modifications and Google specific things. You know that that we did, right? For example, like, you know, programming some of our own PLCs or, you know, making modifications to the building management system. So like the the software control layer looked quite different, right? That was done in-house. But, you know, I still remember like very vividly, actually, to this day. You know, Veda and I, we were standing, you know, in a large, like 90 megawatt data center, right? So it was a fairly large data center. And, you know, Veda's like typing away in his MacBook, right? He he submits the PR, right?

• 17:15Speaker 00

  Gets merged and all of a sudden, this huge, honking, huge chiller that is the size of a bus that we're that we're standing right next to roars to life. And as it's coming to life, right? Like, the ground is shaking vigorously. And we're like, oh, my God, it's like, with a few keystrokes on his MacBook, like, we just turned on this enormous chiller. And that was like the very first data point to us, like, yes, it is possible to control things from the cloud. So now the next question is, how do we control things intelligently from the cloud, right? You know, where all the compute resources.

• 17:50Speaker 01

  What were your biggest takeaways from that experience? You mentioned the creativity of the machine. Any other big takeaways or learnings?

• 17:57Speaker 00

  Yeah, so the creativity is absolutely a big one. I think, you know, just to elaborate on that briefly, you know, a lot of times, like when we talk about AI, right? Both in the valley and elsewhere, I think there's a conflation between AI and automation, right? Like, AI can absolutely automate things. There's there's no doubt about that, right? Especially like routine things, right? But I think that honestly undersells the real promise of AI, right? I think the real promise of AI is what Demis, the CEO of of DeepMind, calls, you know, like AI creativity, right? It's the ability to acquire knowledge that did not exist before, right? And I, of course, experienced this firsthand. The reason why I'm such a true believer in the technology is because,

• 18:40Speaker 00

  again, I was the expert who helped design the system, but this very AI agent that we created is telling me new things about the system that I didn't know about before. Right. And that's a very, very powerful feeling. It's kind of like when, um, you know, if you think back to AlphaGo, right? Like, Lee Sedol was the best in his field at Go. He was the world champion for a decade, right? He was at the top and his Elo rating was just something outrageous. It was like twenty eight hundred or something. It was outrageously high. And, but it had flat, you know, flat line, right? So for a full decade, his Elo rating was the same. And there was no one to challenge him because he was already at the top. So once he hit the top, he just kind of plateaued. And then after AlphaGo happened and he actually got to play against AlphaGo,

• 19:25Speaker 00

  you know, privately a few more times because DeepMind, you know, had, you know, had, uh, uh, let him continue interacting with the system. What happened? For the first time in a decade, his Elo rating started climbing. And so this is what I mean when I say that I think the real power of AI is helping us discover knowledge that we didn't, didn't necessarily know about before. And where are you going to see the most, um, gain from that? It's not going to be in routine automation things, right? Like call centers or whatever, right? It's going to be, I think, in very, very complex areas, right? Like areas where human intuition is insufficient because of immense complexity, but that is yet underpinned by data.

• 20:07Speaker 00

  So that's why you're seeing such things like protein folding, for example. Yeah. That, that's fucking extraordinary, right? And, um, you know, it's those areas are just like massive permutational complexity underpinned by data. That's where I think we're going to see some of the most interesting companies and products. So that was, that was a rather long tangent, but so one creativity is something that I learned. The other one, um, lesson that, that my co-founders and I learned is really around, um, you know, if you want real impact. You got to turn the technology into a product. And this is actually the core reason why we we decided to leave DeepMind and train technologies to start Phaidra, right? Like, over and over again, we were seeing the technologies that we were helping to develop

• 20:52Speaker 00

  at DeepMind were just extraordinary, right? I mean, they were they were achieving crazy things like with protein folding. But the problem is, you know, in order for the technology to make the most impact, you have to get into the real world. People have to actually use it, right? And that fundamentally means we're talking about a product. Turning a technology into a product is like, I mean, you guys would know much better than myself, it's like a hundredfold, a thousandfold more work, right? And, um, you know, and that for us, you know, led us to the conclusion that like, hey, it's, you know, it's time to leave, right? It's time to actually, um, start a company that that creates these intelligent, uh, virtual planner operators, like these intelligent AI agents as a real product.

• 21:34Pat Grady?

  Let's talk more about that. For what you're building now, how much of what you learned at Google DeepMind sort of translates directly into what you're doing now? How much is new because the environments are different, the customers are different, there's something different about it?

• 21:50Speaker 00

  I think the most important thing that we learned from our Google DeepMind experience is that it's possible. Like, this is not a crazy, like, and that isn't to like, you know, like, downplay like what we learned, like, it's it's actually a huge thing, right? We learned that it is, in fact, possible to use, um, you know, closed-loop learning systems like reinforcement learning, right, to drive very large improvements in complex industrial facilities. It hadn't been done before, to our knowledge, right? And that was a massive proof point. I think that the problem, though, is that, like, the real world is quite diverse. Every single customer is diverse, and especially when you talk about industrial facilities, like, every industrial facility is a snowflake, right?

• 22:34Speaker 00

  So for for us, I mean, I mean, the learnings have just been Like massive since you know, since we left Google and DeepMind, right? Because every time we onboard a new customer, we're learning something new about like how equipment are connected or some product gap that we didn't know about before that needs to be fixed, right? Or, you know, new ways that data can break. At this point, I can tell you like a hundred different ways that, you know, data associated with mission critical cooling systems can break. Probably not the most interesting party topic for most folks, but you know, I personally find it quite interesting. But yeah, there's certainly been quite a lot of learnings in that regard.

• 23:09Speaker 01

  Are the folks you're talking to, are they, are they ready to let the technology take over the system and you know, let the cooling system just start, just start going?

• 23:18Speaker 00

  Yeah, I mean, um, yes and no, right? And that actually gets back, you know, to your earlier question, Pat, as well, right? About like the specific learnings from Google. I mean, when I look back, you know, I think, I think what we helped pioneer at Google and DeepMind could only have been done, right, at a company like Google. The, the reason why I say that is because, you know, Google is a very forward leaning company. Yeah. But also like one of the things I've learned, right, is that like, Google is absolutely an anomaly when it comes to like how much data it has and the pristine quality of the data and the ease of access of the data, right? Like, Google is fundamentally a data analytics company, right?

• 24:01Speaker 00

  And, um, you know, as such, you've invested all this like infrastructure in high quality, high availability data, on which you can do things like real time intelligence applications, like what we were doing, and there are many other examples of this within Google and DeepMind. Having left the nest, one of our rude awakenings was, you know, Google is definitely anomaly. And, uh, I mean, gosh, like everyone is in various stages of their AI journey, right? Like, Google's certainly on one extreme. We have customers who've encountered where, like, you know, forget about real time intelligence, they're like, they're not capturing the data in the first place, right? Or, you know, they may be censorizing, in, in the industries we work at, like pharmaceuticals

• 24:46Speaker 00

  and district cooling and especially data centers. Almost always, the customer is sensorized, right? Because these are billion-dollar facilities, so of course it makes sense to throw a million dollars' worth of sensors on it. But that doesn't, just because you censor it, doesn't mean that you're storing the data, right? A lot of customers of ours aren't necessarily storing the data beyond like ninety days, or six months, or a year, or whatever, right? And, you know, and they might cite some reasons like, well, it's costly to store the data, right? Or like, well, we're not, or the more commonly, we're not using the data for anything, which is a true statement, right? A lot of our industrial customers, they aren't using the data, right? It's more like a forensics thing where if something goes wrong, then we go back and we look at the logs to see what happened, right?

• 25:30Pat Grady?

  Yeah.

• 25:31Speaker 00

  And then, you know, so if we think about like Maslow's hierarchy of data needs or something, right? You got your sensorization, you got your storage, then you have to invest in making sure that the data is cleaned, right? There's a lot of effort, as we all know here, you know, around making sure the data is actually cleaned and usable, right? And, you know, that requires you to know what bad data looks like, what good data looks like, and how to convert bad data into good data, so it's actually useful. And then, once you have clean data, you also need to make it accessible in a streaming and like batch historical manner, right? So there's different gradients, I guess is what I'm trying to say of AI readiness. The customers whom we work with are all over this, are all over the spectrum.

• 26:15Speaker 00

  But, you know, like Fager today is at the point where we are autonomously controlling data centers for our customers, right?

• 26:22Pat Grady?

  So I was gonna ask you, if the basic workflow or the basic loop is data goes in, which is a lot of what you just talked about, getting the data into the system, step one. Step two, decision is made.

• 26:34Speaker 00

  Yep.

• 26:35Pat Grady?

  Step three, action is taken as a result of the decision that was made. Step four, action is evaluated against the objective function of the system.

• 26:42Speaker 00

  Yeah.

• 26:43Pat Grady?

  And then the loop continues. So the front end of that process, which is data goes in, sounds like there's a lot of work to get the real-world data ready to go.

• 26:50Speaker 00

  We call it the AI readiness journey, right? So like, if you think about our work with customers, like, there is a chunk of upfront work where it's just like, hey, we're going to get your facility, we're gonna get you and your facility AI-ready.

• 27:02Pat Grady?

  How about, how about on the action is taken piece of that? Are the systems ready to be controlled by some sort of autonomous system, or is there work that needs to happen there, too?

• 27:13Speaker 00

  Yeah, it's a really good question. Yes and no, right? And I'll elaborate on what I mean by that, right? Control systems today were like designed in like the 1980s. So was I.

• 27:27Speaker 00

  Well, me too for that matter. There we go.

• 27:31Speaker 00

  But you know, like what I mean by that is, you know, that was the the seventies and eighties was the third industrial revolution, right? So with that, you know, was the shift from analog to digital and the advent of the first automation systems, right? In order to automate, you fundamentally first have to censorize. But these are simple automation systems, right? The fourth industrial revolution, right? You know, is, you know, we're biased, but Phaidra, you know, we, we think the fourth industrial revolution means intelligent infrastructure, right? Infrastructure that can operate itself and fundamentally get better over time at doing self improving infrastructure, right? But right now, we're shoehorning intelligence into systems from the third industrial revolution, right?

• 28:16Speaker 00

  So they certainly weren't designed for this. But what we do instead is most importantly, we ride on top of the existing control system. So there is a hardcoded layer of rules and heuristics. So millions of lines of if then statements programmed into what we would typically call the BMS, the Building Management System, or a SCADA system, right? That defines how the facility should operate. The problem with hardcoded systems is that because they're hardcoded, they operate the same way today as they did yesterday, or a year ago, or five years ago, more like 10 years ago, because people don't very frequently go into the backend programming, right? To update that control's logic. Now, what Phaidra does is we insert a new cloud intelligence layer at the very top of the control stack.

• 28:59Speaker 00

  So we're not, we don't introduce any hardware, we don't introduce any new sensorization, right? We actually ride on top of the existing control stack. That's really, really critical, right? You can think of it as a general on the battlefield. The general, right, has a global view of everything that's happening across the system, right? And it's issuing command signals, right, to the troops on the ground for actual execution. So the AI is looking in our case at 10,000 trends a minute in real time, right? And it's issuing decisions like which pumps to turn on or what their pumps should be, right? To the local BMS system and or the PLCs for automatic implementation and execution.

• 29:43Speaker 00

  So that's why I said it's a mixture of yes and no. Were they designed for this in the first place? No, right? There is a lot of work that we have to do with our customers to be able to accept this type of external intelligence. There's a lot of work that we do in defining the safety nets and guardrails, right? To ensure that the AI can't do bad things to the customer's system, right? But fundamentally, we are still riding on top of the existing controls architecture. And to be clear, we always want to do that, right? Like, you don't want AI controlling things like how fast a valve opens and shuts, right? Like, that's a terrible application of AI. Like, hard-coded rose and curious six will do great there. So if you were to look at the, you know, like, the overall system, like,

• 30:26Speaker 00

  90% of it is fine with just, you know, hard-coded rose and heuristics, because it's like granular controls logic that doesn't need non-deterministic, crazy-powered intelligence behind it, right? But it's the higher-level thinking and reasoning. That's where you want the AI. It's the global optimization.

• 30:45Speaker 01

  Have you seen any of your customers at Phaidra kind of get the DeepMind or their magnitude results?

• 30:53Speaker 00

  So I'm glad you asked. So the one of the things I'm really excited about is actually just actually literally this week, earlier this week, Merck Pharmaceuticals became our first public customer. So we're pretty proud about that. We've been actually been working with them for two years now. They've been using Phaidra for over two years. The full, like, autonomous AI system to control a massive 500-acre vaccine manufacturing facility in Pennsylvania, right? Like, this is this is the definition of mission-critical complex, right? They've got 62,000 tons of cooling. So they've got four very large shuttle plants interconnected with each other across 500. Miles of manufacturing space, right? Wow.

• 31:38Speaker 00

  Hundreds of machines interacting with each other. Like, this is where the AI really shines. And, yeah, the results that we saw with them were quite strong, right? Like, you know, I think Merck actually just shared some data at a conference we were at. We, you know, we showed 16% energy savings when we first, you know, trialed the system at one of their chiller plants, so. But, you know, what I always tell our customers, right? Is don't over-index on the magnitude of the energy savings initially. Like, we honestly have no idea what the energy savings are going to be, right? And where the liability improvements are going to be ahead of time, right? Because these are non-deterministic systems. And by definition, if I could tell you what things you're not doing

• 32:22Speaker 00

  in order to get energy savings, like, why do you need the AI in the first place, right? So, you know, but what we do know is that the unique thing about this technology, about Fedra and about reinforcement learning in particular, is that it is a closed-loop system. It is a self-learning system. It can learn because it's able to take actions and it can measure the impact of its actions against its predictions, right? And that means it gets better over time. So, maybe we start off at 1% energy savings. Maybe we start off at 5%. Maybe we start off at 10%, right? But fundamentally, it will learn and it will get better over time, right? Not infinitely, because there are still our laws of physics, right? But it will get better over time.

• 33:05Speaker 00

  And once it reaches optimal, it will stay at optimal, right? That's super important, because with hardcoded rules and heuristics, right?

• 33:15Speaker 00

  When you tuned a system as you were commissioning it, so when you're turning it on for the first time, right? That system today no longer performs the same way that it did 10 years ago when you first, you know, commissioned that system, right? Because the pipes have corroded and the heat exchangers have fouled and the cooling towers have scaled, whatever, right? Or you ripped out equipment, so. But the promise of an adaptive self-learning system is that it will change with you, right? As your customers are, for example, now putting in a bunch of H100 and soon H200 GPUs, right? Well, the system will learn and adapt on the fly with you, right? So it can stay optimal.

• 33:51Speaker 01

  I'd love to transition for a minute beyond industrial control systems.

• 33:54Speaker 00

  Yes, totally.

• 33:55Speaker 01

  And get your opinion on, I mean, you were one of the first and maybe one of the only real-world applications of reinforcement learning.

• 34:03Speaker 00

  Yeah, we're definitely not the only.

• 34:05Speaker 01

  Not the only. I'd love to get your thoughts on the not the only. So, I mean, what else are people doing with reinforcement learning in the wild today?

• 34:13Speaker 00

  Yeah, absolutely. So, unfortunately, my knowledge is very heavily indexed on the Google and DeepMind space because that's where we spent so much time, right? But even within Google and DeepMind, there were other, you know, very cool reinforcement learning applications, right? Like, for example, the team that sat right next to us, right? They used RL systems to, you know, to help prolong battery life, for example, right? So you may notice that your Android phone, right, like, the battery life has been increasing. And yes, there are hardware changes associated with that, but there are also intelligent software changes behind the scenes that proactively manage your battery life, right? There were reinforcement learning systems for, you know, for like YouTube, you know,

• 34:57Speaker 00

  video recommendations, for example, right? And a whole host of other things, right? So absolutely, there are, you know, reinforcement learning applications in the wild. To your point, though, I wouldn't say that there are a whole lot of them, right? And I think that it's not a coincidence that you tend to see them at more of like the big tech companies where they've already invested in the data infrastructure, right? So that the underlying infrastructure, so that they can benefit from this technology, right? Like, outside of the big tech companies, there are very few applications of like real-world reinforcement learning, like in production at least.

• 35:35Speaker 01

  Yeah. And do you think that's because of kind of low applicability? You know, you started this podcast by talking about The necessary ingredients for RL to be a good solution. Do you think it's just, there's not that many applications where RL is a good solution? Or do you think it's just tech readiness?

• 35:50Speaker 00

  Absolutely not. I don't know. I think the applications for reinforcement learning are freaking massive. And where Baird-Fedra is one of many examples. And we're just scratching the surface as an industry of what we can do with this technology. Right? Like fundamentally, the power of the technology is that it is a self-learning system. AlphaGo and its successor, AlphaZero, taught itself to become the best in the world at Go, Chess and Shogi, three vastly different games, same learning framework, right? And it taught itself. So I think there's a lot of very interesting application areas, right? I think the data infrastructure is missing in a lot of them. But just to like, you know, like list off a few, right? I mean, obviously, you know, we've already talked about like the protein folding, right?

• 36:34Speaker 01

  Yep.

• 36:35Speaker 00

  But, you know, there's, there's an entire untapped field around logistics, right? Like, that is such a gnarly computational challenge, right? You know, when you start looking at operations research, operations research underlies trillions of dollars worth, worth of, you know, industrial like activities, right? Not just industrial, but other sorts of activities, right? Like shipping, airplanes, FedEx, like driving routes. Like, these are all applications of operations research. Grid balancing, right? I mean, I think grid balancing is probably the single most important way that AI can fight climate change. I generally believe that is where AI will have the most impact on climate.

• 37:14Pat Grady?

  If you had to, if you had to guess, you know, you deploy, the first time you deployed this into a data center at Google, you saw 40% energy savings. If we had just killer AI doing load balancing on the, on the grid, what, what sort of energy savings do you think we could see?

• 37:29Speaker 00

  Oh, my gosh. I mean, that would be wild. I think it's not so much about the, the magnitude of the energy savings per se, but rather about the, the potential cost savings. Because then you could start shifting your loads around, right? To when it's most cost effective to do compute. Or if you had CO2 signals, you could start scheduling loads around. When it's the most, the least carbon intensive to do your non-lackancy sensitive workloads, right? Which I think Google has already been experimenting a bit with, right? But honestly, I think it's really more around the global system level optimization, right?

• 38:08Speaker 00

  We have to keep in mind that data centers already are, but increasingly, you know, are just massive, massive load banks, right? Like data centers, like they were one and a half, two percent of US energy consumption. That's about to increase to four percent, right? Like this year, I think. And then by the end of the decade, it's projected to get up to like nine percent of the US. In Ireland, right? Right now, Ireland is twenty-two percent of Ireland's national energy, electricity consumption goes to data centers alone. The International Energy Agency predicts that that's going to increase to thirty-seven percent by the end of the decade, right? Like just mind-boggling numbers. But the point, the reason why I mention this is because these are massive load banks on

• 38:52Speaker 00

  the grid, right? There is an actual opportunity if you could somehow coordinate the data centers together, right? To, to help balance the grid, right? And that is such a gnarly, gnarly challenge, and it is what is holding the energy transition back. Because, you know, as more and more renewable energy starts coming onto the grid, right? The supply side becomes increasingly stochastic, right? We used to have this perfectly deterministic system, at least on the supply side, where, you know, good operator can call someone who operates a coal-fired power plant and say, hey, ramp up or down your power production, right? It's deterministic. But now- Ramp up or down the sun. Yeah, totally, right? So now, as you get more and more renewable penetration, you know, coming onto the grid, right?

• 39:37Speaker 00

  You have a somewhat non-deterministic demand side, right? It's somewhat predictable, but there are definitely spikes. And a massively non-deterministic supply side, right? And what is the problem with that? The problem is that, you know, we- Because we do not know how much energy we're going to generate, you now have all this wasted excess capacity in reserve, right? So there is a concept of spinning reserves on the grid where, you know, like there are peaker plants, you know, like giant natural gas turbines, right? That as we speak are just sitting there idling, just like your car idles at a stoplight, right? In case we need that power, right? As a buffer against the uncertainty. And as renewable penetration increases,

• 40:21Speaker 00

  ironically, the amount of buffer you need also increases. If you look at Germany's failed energy transition, right? They decommissioned their nuclear baseload while ramping up their renewable energy penetration, right? Good motivation on the surface, although I personally think we need a lot of more nuclear on the grid, but that's another topic. But it ended up backfiring, right? Because Germany actually ended up needing to build more fossil fuel power plants to buffer against all the renewable energy that was coming onto their grid now, right? So that's why I think AI for grid balancing, we need it. And it's probably is the single most impactful thing that AI can do to solve climate change.

• 41:03Pat Grady?

  Let's talk a bit about some of the limitations of reinforcement learning and also where you see it intersecting with transformers.

• 41:09Speaker 00

  Yeah.

• 41:12Speaker 00

  So I should state that, first of all, my co-founder Veda is by far the expert on this topic. He knows way, way more than me. You know, I'm just a simple mechanical engineer, right, who happened to learn a bit about AI.

• 41:27Speaker 00

  I think the intersection is really interesting, like very, very,

• 41:34Speaker 00

  very potentially complementary strengths and weaknesses is how I would describe it, right? It's certainly not mutually exclusive, right? Like what I mean by that is, and I was just talking with Veda about this earlier, right? So Veda will tell you that, like, you know, All intelligence systems have certain hallmarks, right, of intelligence so that we can say they are intelligent. They need to deeply understand the world, the environment that, you know, that, that, you know, that they're modeling, right? There needs to be some element of memory, right, so like remembering things. And very importantly, there needs to be the ability to plan and reason, right, very interlinked. Transformers are clearly quite good at the first one, right, in the sense that like they can take in

• 42:19Speaker 00

  huge amounts of structured and unstructured data, right, to learn quite good models, right, of the world.

• 42:29Speaker 00

  But it is limited in the sense that these models are primarily through correlation and not causation, right? That makes it challenging for, at least for what Fager does, right, because, because we work with real world systems, we have to have causality, like we have to understand why is the AI doing certain things, right? Like, why is it not doing other things? How do we force a certain behavior, right, that we know has to exist in our system, right? So, these are mission critical systems, what I'm trying to say, there has to be causality. So, that's where the limitation is, right?

• 43:02Speaker 00

  With reinforcement learning systems, I mean, the, the power of RL based systems is very, you know, much in the planning and reasoning part, right, where, you know, you're able to plan long trajectories of actions and learn really, you know, like intricate policies, right? I think where it gets really interesting is the intersection, right? Where potentially transformer architectures can learn models, right, you know, like value functions, right? That the, that the, or, or models of the, of the world that the AI can learn policies against. But without that causality piece, right, it's going to be quite tricky to cut it over into at least industrial control applications like what Fager does.

• 43:49Pat Grady?

  Should we move into a rapid fire round?

• 43:51Speaker 00

  Is it?

• 43:52Pat Grady?

  What are you most excited about in the world of AI in the next five or 10 years?

• 43:56Speaker 00

  So in the very near future, right? I'm excited about just the absolute explosion, right? Of AI applications, right? It feels kind of like a pre-Cambrian explosion of sorts where they're like, there's like a primordial soup. And like all these AI startups and services are all of a sudden springing up, right? So it's quite exciting.

• 44:18Speaker 00

  But when I look at where that, where that activity is happening, where that research and that entrepreneurial activity is happening,

• 44:28Speaker 00

  is very clearly focused on like around LLMs and even more specifically around like natural language interactions, right? Text-based interactions, right? And that certainly is a large part of the economy. It is very exciting. But in the five to 10 year frame, to answer your question, I'm most excited about when we can start getting some of this technology into real-world physical

• 44:52Pat Grady?

  Yeah.

• 44:52Speaker 00

  applications, right? It's the intersection of this technology with the real-world infrastructure that we, that we live in, right? Like big industrial systems, cars, homes, like, you know, physical things. I think that's where we're going to see some really interesting things in the future.

• 45:11Speaker 01

  Who do you admire most in the field of AI?

• 45:14Speaker 00

  Gosh, a tricky question. I admire a lot of people.

• 45:18Pat Grady?

  You've worked with some of the greats and so it's gonna be hard.

• 45:22Speaker 00

  Yeah, I mean, of course, my mind jumps immediately to a lot of the people whom I've worked with, right? You know, I admire very much the DeepMind researchers whom we, you know, worked very closely with. I often tell people working at DeepMind, it's kind of like being a kid in a candy shop if you're a technologist like myself. It's like you get to see years in the future. I had all this cool technology on the forefront and it just makes your head spin as to like all the possible applications of that technology.

• 45:53Speaker 00

  I admire Moose a lot, right? My, my old boss who, you know, has of course since moved over to, to Microsoft. You know, I was saying earlier, like one of the biggest lessons I learned, you know, and my co-founders learned at DeepMind is that like, Making a technology like what we did for Google's data centers versus making a product like what we're doing at Phaidra, totally different things, wildly wildly different things. And there are few people as good in the world as Moose at like taking technologies and turning them into real products, right? I remember my co-founder Katie and I, you know, we were sitting down, we were grabbing drinks with Moose at like at some random dive bar in Seattle, right? He happened to be up there.

• 46:35Speaker 00

  And this is before, you know, OpenAI like released ChatGPT two and just like ushered in like a world of craziness, right? And, um, and, you know, he was raving to, to myself and Katie about the applications of LLMs and like, you know, the, um, you know, and how powerful these systems are. And we were like, okay, Moose, but you know, let us tell us, let's tell you about Phaidra. Like, we had no idea what he was talking about, right? But, I mean, he was prescient, like he saw this ages in advance, right? You know, of what the technology, the technology that was being developed and the capabilities, right, that it would usher in. And then, of course, he went off and he started Inflection. So I admire him a lot for the ability to turn technology to actual products.

• 47:20Pat Grady?

  All right, last question. You are building a very ambitious business, very hardened business to build, and you've been at it for a while in the context of the new wave of AI startups. What advice do you have for other founders or would be founders who are trying to build companies here?

• 47:36Speaker 00

  I mean, I'm not sure I'm even qualified because, um, one, I hope, I hope you ask again in, um, you know, in, in order to years when hopefully, you know, Phaidra is wildly successful. We certainly didn't choose the easy path by focusing on reliable infrastructure. Honestly, my mind gravitates towards more like, um, would be founders, right? Like people like my co-founders and I, who were thinking about leaving to start something new, right? And my advice there is twofold. Uh, one, make sure you have co-founders. Like, my guide is so stressful.

• 48:18Speaker 00

  There's so many things that can go wrong and you're constantly on this emotional roller coaster of up and downs. Having co-founders to lean on, both for the workload, but also just for the emotional support and mental sanity, so important, right? Advice number two would be, the risk is less than you think it is. Hmm. Right? I'm biased, but I think people should take the jump, right? A lot of times when I talk with my former colleagues and other people who are thinking about making the jump, they'll say things like, "Well, but I've got a nice job over here. They pay me well, I'm on a rising trajectory."

• 48:59Speaker 00

  But my point to these folks is always, no matter how valuable and successful you are today in the organization, you will only be more valuable and successful for that organization or other organizations or to society in general if you learn new skill sets. Right? Like, take the plunge, go out, start a company, learn what it's like to turn technologies into products, right? And if that fails for whatever reason, hopefully it doesn't, right? But if you fail, then the Googles, the Microsofts, whatever, the world, they will only want to hire you back at an even higher premium. So why not take the plunge?

• 49:39Speaker 00

  The biggest, best investment, and obviously, much smarter people than me have said this for a really long time, but the best investment you can make in yourself is you. Up-leveling yourself, learning new skill sets, that's always the best thing you could do.

• 49:55Speaker 01

  Thank you, Jim. This was a fascinating conversation.

• 49:58Speaker 00

  Yeah. Thank you very much for having me, you guys. Really enjoyed it. Thank you.