Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog

Much of the scientific process involves searching. But rather than continue to rely on the luck of discovery, Google DeepMind has engineered a more efficient AI agent that mines complex spaces to facilitate scientific breakthroughs. Sarah Guo speaks with Pushmeet Kohli, VP of Science and Strategic Initiatives, and research scientist Matej Balog at Google DeepMind about AlphaEvolve, an autonomous coding agent they developed that finds new algorithms through evolutionary search. Pushmeet and Matej talk about how AlphaEvolve tackles the problem of matrix multiplication efficiency, scaling and iteration in problem solving, and whether or not this means we are at self-improving AI. Together, they also explore the implications AlphaEvolve has to other sciences beyond mathematics and computer science. Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @pushmeet | @matejbalog Chapters: 00:00 Pushmeet Kohli and Matej Balog Introduction 0:48 Origin of AlphaEvolve 02:31 AlphaEvolve’s Progression from AlphaGo and AlphaTensor 08:02 The Open Problem of Matrix Multiplication Efficiency 11:18 How AlphaEvolve Evolves Code 14:43 Scaling and Predicting Iterations 16:52 Implications for Coding Agents 19:42 Overcoming Limits of Automated Evaluators 25:21 Are We At Self-Improving AI? 28:10 Effects on Scientific Discovery and Mathematics 31:50 Role of Human Scientists with AlphaEvolve 38:30 Making AlphaEvolve Broadly Accessible 40:18 Applying AlphaEvolve Within Google 41:39 Conclusion

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:05Sarah Guo?

  Hi listeners and welcome back to No Priors. Today, we're joined by two of the key folks behind one of the most compelling developments in AI this year, AlphaEvolve. Pushmeet Kohli and Matej Balog worked on this autonomous coding agent that uses Gemini models and evolutionary search to discover new algorithms. It marks a major leap in AI's ability to contribute to core computer science and math, and perhaps sciences beyond that. It's not just a stochastic parrot or a boilerplate generator. It has shown what you might consider technical creativity in the way that Move 37 did with AlphaGo, something humans hadn't done before, even in thousands of years of play. It might even be a real step on the path to self-improving AI. Pushmeet, Matej, thank you so much for being here.

• 0:46Speaker 01

  Thank you for having us.

• 0:47Sarah Guo?

  It's a pleasure. Congratulations on the success and the launch of AlphaEvolve. Can you give me a brief description of what it is broadly?

• 0:56Speaker 01

  Yeah, so in maybe one sentence, AlphaEvolve is an AI coding agent that is able to discover new algorithms that are able to make new discoveries on open scientific problems. And at the same time, those algorithms can be so practical that they are already deployed in key parts of Google's own infrastructure.

• 1:15Sarah Guo?

  What is the origin story of working on this particular form of coding agent or this problem statement?

• 1:22Speaker 01

  So we are not new to this space of algorithm discovery. As you might know, the mission of all of DeepMind is to build AI responsibly to benefit humanity. And the way our particular team has been doing it for years now is to look for ways how AI can discover new algorithms. New algorithms are everywhere around us, so this is a very, very important question and can have very high impact when we can discover algorithms that solve important computational problems with higher efficiency than what we have been able to do so far. And kind of the first breakthrough we had in this space was in 2022, when we released a system called AlphaTensor. And so that was a system that was an AI system using reinforcement learning that

• 2:05Speaker 01

  for a very specific but fundamental computational task, so multiple

• 2:10Speaker 01

  Like matrices for the first time show that AI agents can discover better algorithms than what humans had been able to do before them. So this was the first system that gave weight to this idea that indeed with AI, we'll be able to go into the superhuman region of algorithms that we as humans have not been able to discover ourselves.

• 2:31Sarah Guo?

  How do you differentiate Alpha Evolve from like Alpha Tensor and Fun Search and some other projects in this sort of lineage of this?

• 2:38Speaker 02

  One way to also describe what we have done is if you look back at the history of DeepMind and see a number of sort of projects that have come even before we started working on computer science. Our earlier sort of work, and if you go back to the project on AlphaGo, where the AlphaGo agent was able to beat the world Go champion in the game of Go. And what was the remarkable sort of thing in that agent was that it was able to explore this amazingly large search space of all possible sort of Go positions in such an efficient manner

• 3:23Speaker 02

  that it can sort of come up with what is the optimal move at that time, right? And that really surprised people, both Go professionals as well as scientists. Scientists believed that that event would come much, much later because it was a very hard problem. And so what that gave evidence for is that is the ability of these large-scale neural network-based systems to be able to reason and do very efficient exploration in these large search spaces and come up with amazing new insights about.

• 4:06Speaker 02

  The particular domain and in the game of Go, I mean, there is this move called move thirty seven, uh, which is a very creative new move that the agent discovered. That was not in the Go literature, right, that really surprised the Go professionals. So in some sense, uh, we asked ourselves the question that if you have an agent which can do very efficient search in the domain of Go, why can't you use the same kind of philosophy to search for algorithms in the space of algorithms? And in fact, that sort of was the underlying basis of the work on

• 4:47Speaker 02

  our our first sort of attempt at that problem, which culminated in Alpha Tensor. So how we structured uh the algorithmic discovery uh problem is we looked at first a very important problem. And that problem was matrix multiplication. It is a problem that is ubiquitous in computer uh science. It's one of the key fundamental operators that underlies not only computer science, but also neural networks and machine learning and and AI. We said, can we find a way to improve matrix multiplication algorithms? So there's a history of matrix multiplication, which is very interesting

• 5:30Speaker 02

  for people who might be interested in it like uh it's even though it's such a fundamental operator, um, people thought that the the complexity or the time it takes to multiply two matrices is order cube. Uh huh. And around fifty years back, more than fifty years back now, uh, a German mathematician Strassen came up with this very counterintuitive construction, which showed that in fact, the complexity was not uh N to the power uh three or what's not cubic, where N is the sort of dimensionality of the matrix, it's lower. And so and that was a very counterintuitive sort of result and uh and but it stayed for more than fifty years and.

• 6:15Speaker 02

  Until sort of Alpha Tensor came up and we said, well, can we actually improve this result? And remarkably, we were able to show that. That Alpha Tensor, while by having this amazing ability to do search in this very large space, even much larger than the space of possible Go moves, was able to come up with this amazingly new algorithm which improved things. But then the question was, well, we now have proved the thesis that you have these super intelligent agents which can go beyond what human computer scientists have been able to do, but can we generalize them?

• 6:59Speaker 02

  This is sort of Alpha Tensor was very smart, but was only sort of purposefully constructed for the matrix multiplication problem. Can we have build an agent that is more general? Both more general in the sense of it can handle more general problems, but can also search in the space more naturally, in the space of programs rather than in the space of very specific sort of operations that were required for matrix multiplication. And that was the origin of sort of the first attempt of us with FunSearch, which was an L L M based agent,

• 7:43Speaker 02

  which for the first time by searching in the space of programs, showed that you can come up with completely new solutions. Like and this and made the first scientific discovery from N L M. And Alpha evolve is basically an extension of that.

• 8:02Sarah Guo?

  I'm very inspired by the idea, I think as many people are, that AI will actually have creativity, does actually have like technical creativity, as you are describing as one way to conceptualize this, where you're, you know, outside of the patterns that we already know as engineers. I want to go back to some of the mechanics here and the limits to generalization and how to think about automated evaluators and a lot of different topics. But, you know, when you think about these problems that are clearly like economically valuable and interesting, like matrix multiplication, the potential efficiency of it, what is your intuition for? Why those solutions have not been found before?

• 8:46Sarah Guo?

  Is it simply like the search space is too large? Or people in this field were complacent in that they believed a certain solution was like the maximum efficiency? Or because clearly there's value to be had here.

• 9:00Speaker 02

  My sort of opinion on this is basically that if you look at the structure of the algorithm what Strassen produced was quite ingenious. It was not a natural thing that you would sort of think of. And that was for only two by two matrices. As you sort of go to larger sizes, the space is so huge. The constructions are not sort of something which is very natural. These are very involved and intricate sort of constructions that would be very hard to discover by chance. So it's quite interesting that it has this very special structure, but it's not something

• 9:44Speaker 02

  that comes naturally or to a human computer scientist.

• 9:50Speaker 01

  Just to add to that, so I definitely agree. The search space is just unbelievably vast. The solutions are maybe not intuitive. And the third thing I want to emphasize is that I really believe the people who worked on this in the past were definitely not complacent. And in fact, the problems we chose to apply Alpha evolve to in the first instance, both on the scientific side and the practical side, we deliberately chose problems which have been worked on for a very long time by the very best people. So on the scientific side, since we're talking about matrix multiplication, this has been a known open problem for decades and many people have been working on it. And similarly, for the practical applications that we mentioned in our Alpha evolve release,

• 10:35Speaker 01

  in key parts of Google's infrastructure, again, like these are things that have been heavily optimized inside Google because they are so important. And so by having a system like Alpha evolve or any other, discover something new on these problems, I think that's as strong a demonstration as I can imagine of the fact that this is indeed something that is new because no one found it before. And also, it is something that was not easy to discover because those results stood for such a long time and have been worked on by such strong people.

• 11:08Sarah Guo?

  I noted that this is not a comment on the, you know, broad efforts of the computer science industry to date on matrix multiplication or data center optimization. I think this is a good moment to try to demystify what's happening under the hood for a broader set of people. Can you, can you walk us through a concrete example of how AlphaEvolve actually evolves code? And say, let's take the example of trying to optimize data center scheduling, right? What does the step-by-step process look like from initial random code to final solution that saves millions of dollars of power?

• 11:42Speaker 01

  I can walk you through that. So the user of a system like AlphaEvolve, they basically specify what is the problem that they are trying to solve. So that's the most important thing. And you specify it by providing what is called an evaluation function. What this function does is whenever there is a proposed solution for solving the problem, you're able to tell how good this solution is. So you basically define what makes a good solution. For discovering an algorithm for scheduling jobs on a data center, this evaluation function could be something like a simulator of jobs in a, in a data center. That given an algorithm for doing the scheduling, it simulates how good this algorithm is. So that's what the user provides.

• 12:23Sarah Guo?

  And this is a simulator you already had?

• 12:25Speaker 01

  Yes, so that's a simulator that we already had. And I would say it's something that is quite natural to have in many, in many domains. Because, you know, whenever you want to innovate on something, you need to have a way of telling, okay, is the innovation actually good or not? So it's a very natural object to have, at least in principle. So you define the what by providing the evaluation function. And then AlphaEvolve fills in the how. So that, that's the job of, of our system. And you can do it in two fairly different ways. One is you tell AlphaEvolve, I have no idea how to solve this problem. Let's start completely from scratch. And let's try to be creative and come up with something completely new. So that's one option you can take. Another option you can take is, actually,

• 13:09Speaker 01

  we have already worked on this problem for a really long time. Here is a very strong initial solution that we can provide to the system. And you can start from here. And that's what we did. For the application to discovering new algorithms for for scheduling jobs in a data center. So Alpha evolve takes this initial solution. And then it on a high level, it combines the creative power of large language models to propose like creative new ways how to improve that solution. The strictness of the evaluation function provided by the user that is able to actually filter out the things that that work from the ones that don't. And then this is wrapped inside an evolutionary algorithm that makes sure that we can discover the whole space of of algorithms in that region.

• 13:53Speaker 01

  So that we don't commit to a very specific type of solution early on. But instead, we maintain a diverse pool of potential solutions. Over time, maybe we combine ideas from different different solutions that are already strong. Until we actually have an algorithm that's so strong that we are happy to happy to deploy to a critical part of Google's infrastructure, let's say.

• 14:13Sarah Guo?

  And intuitively, not in the machine learning sense, but in the evolution sense, you have different generations where you're getting closer to an optimal solution.

• 14:21Speaker 01

  Yeah, that's right. Like you would expect that in each iteration of evolution. What you're doing is you're looking at the previous iteration, looking at the maybe the strongest solutions you have. And then trying to be creative about how can I combine ideas from those solutions? Or maybe bring in completely new ideas to to come up with something even better. And so, yes, each generation gets stronger and stronger.

• 14:43Sarah Guo?

  How much scaling are we talking about? Like, is there a way to predict how many generations it takes? Or how do you constrain the the number of iterations that the model can use?

• 14:53Speaker 01

  So there are two parts to your question. One is about, okay, how does scaling work? And then how can you predict it? So for the first part, this is actually a really nice feature of Alpha evolve that it can adapt to the difficulty of the problem. If you ask Alpha evolve to find a solution to a problem that's actually unexpectedly easy, then it will just do it very, very quickly. Like almost immediately, you will have the solution. But if you ask it a problem that's really, really difficult. And by really, really difficult, I mean like really difficult, maybe an open question that has stood for decades in the sciences or. You want a practical algorithm for a really high value application in Google. Then you would of course expect this is not an easy problem.

• 15:37Speaker 01

  You might need to spend longer time considering different solutions, exploring the space, combining ideas. But what's really nice about Alpha evolve is that it is able to sustain this scaling in a way that it keeps improving over time. And it keeps improving for so long that you can make discoveries on this level of difficulty, like breaking decades old scientific challenges or discovering high value algorithms. Now, I know it maybe sounds trivial that if you wait longer, you get better results. But in practice, that's actually like a really difficult thing to build automated agents that are able to sustain this continual improvement without plateauing quite early. This is, I think, a nice feature. There was a second part to the question about predicting how many iterations you will need.

• 16:23Speaker 01

  So that is something that is actually not so easy because it's like asking a priority. Do you know how difficult this question is going to be? And especially in the sciences, that's something that often has a very surprising answer. Very trivial questions can turn out to be extremely, extremely difficult and vice versa. But the nice thing is that you have continual improvement if you run the system. And as long as you can run it, you can expect to get better and better results. And you just have to see where this gets you.

• 16:52Sarah Guo?

  If you think about the coding agents that general developers have access to and are increasingly using today, one frustration with them is on relatively trivial problems it is set out to do autonomously will get lost and blow itself up or plateau, as you said, in frustrating ways. Like, can you talk about if you think there are implications from Alpha evolve to these other general coding agents?

• 17:21Speaker 01

  While large language models and coding agents are getting much better in their understanding of code, they're not perfect, right? So they do make mistakes. The other sort of element is to think about what is the task that these agents have been assigned.

• 17:41Speaker 02

  Um. Mostly, if you are asking an agent to solve a particular task or write a particular program, you are providing a specification.

• 17:50Sarah Guo?

  Uh huh.

• 17:50Speaker 02

  You are specifying the task either in natural language, or you're saying, well, I'm trying to do something completed, right? So it's not a complete characterization of what you want. It's a a a partial specification of what you want, and the agents then try to solve the problem and might get lucky and or might and might get the right result, or they might hallucinate and get the wrong result. And the issue is, how do you know what uh so whether the result is right or wrong? And that depends on having a good evaluator. That's how Alpha Evolve solves the problem. So in some sense, we are able to leverage the hallucinations for a beneficial purpose, right?

• 18:35Speaker 02

  So the creativity and the wrong answers that Alpha Evolve can somehow come up with, how do we know that they're wrong? They might be very good, we we just don't see them in that way. And which is why the the the role of the evaluator is really important. And how do we even do the evaluation is very important. Because when you come up with a new idea, um, should you try to explore that idea much further, right? Or how deep should you go into stress testing that idea? Should you try that idea out on a few different instances instances? Or a sort of a thousand different instances?

• 19:16Speaker 02

  Or really stress test that the idea actually works for the whole uh thing? This is one of the interesting parts of Alpha Evolve. Getting that balance right is really important, so that you can look at where are the creative solutions? How can you sort of filter out the ones that are promising? And then use them later to refine the search process to get the final solution.

• 19:42Sarah Guo?

  If evaluation functions, automated evaluators are um.

• 19:46Sarah Guo?

  Really like such a limiting constraint here in terms of what we can get agents to do. Any intuition from this project or others on how to overcome that? Like, can models get good at helping us create automated evaluators? Should we imagine simulators that are better for lots of different domains? If I, you know, lame product manager putting in incomplete natural language spec to coding agent, should I work with an assistant to like complete that spec? Do I use traces? How do you, how do you think that gets solved?

• 20:19Speaker 01

  That's a really, really great question. And I think you can view it from two perspectives that I, I think will happen at the same time. So one is that, yes, currently the, the strict evaluation function plays a key role in Alpha evolve. And one takeaway you can take from this thinking about the future is that it, it shows the, the really high value of having these evaluators available. Because in many cases, it might be that you have a really important problem, but you don't actually have a very precise definition of what makes for a good solution. And one takeaway you can have from a system like this is that if you actually do build a very precise evaluation function, then this unlocks the possibility of having an agent

• 21:01Speaker 01

  like Alpha evolve discover something that's way beyond what, what, let's say, humans have been able to discover or your, your best developers have been able to discover. So that, that's one takeaway. But the other takeaway that I'm maybe even more excited about from the research perspective is that we don't actually think this is a conceptual limitation.

• 21:19Sarah Guo?

  Yeah.

• 21:19Speaker 01

  So today we have, this was maybe the, the easiest way to get into this game of discovering new things by looking at problems that already come with these very, very precise evaluation functions. So that's just a natural first step to take. But I do believe that this assumption can be relaxed in very significant ways. And in particular, you, you already mentioned one example where maybe language models themselves will be able to evaluate whether proposed solutions look promising or not, or whether they fail in, in some particular ways. And, and indeed, there is a parallel work from Steve Mideswell called AI Co-Scientist, which demonstrates this. Very clearly that if you if you propose ideas in in natural language, then you can get language models to provide meaningful critiques and

• 22:04Speaker 01

  identify the ones that work from the ones that that don't. So I really do see a lot of hope on relaxing this assumption. And then even in between these two extremes of strict evaluation that exactly tells you how good a solution is on one end and then natural language evaluation by a language model on the other end, there is a continual spectrum of simulators and auxiliary evaluation functions, which are maybe not perfect, but as long as they are correlated with the with the true signal. Then we can build the algorithmic scaffolding of the evolutionary algorithm around this in such a way that we still make meaningful progress. And maybe it will take a few more iterations, but we can still still go really, really far.

• 22:47Speaker 02

  So just to add what Matej sort of mentioned, I think the one of the takeaways is basically that L L M based agents like Alpha evolve, especially when we structure them in this way with population based sort of search, right? With the evolutionary approaches, they are extremely effective in searching. They can search very convincingly and very effectively in very large spaces and come up with very counterintuitive new solutions for important problems. Problems that we have studied for many, many years and sometimes in in some cases decades. So that's one.

• 23:32Speaker 02

  The other sort of element of the evaluator, like as Matej mentioned, there is work on using other sources for evaluation. So you don't have the perfect evaluator. Even for Alpha evolve, even if you have a simulator, that's not a perfect evaluator, right? Because you are sort of going to evaluate things on a specific distribution of problem instances.

• 23:58Speaker 02

  You might want to sort of prove certain properties of the solution, right? You might want to say that the solution always has certain performance. So if you want to prove certain properties of the solution, that might require a sort of other work, right? You might have to have a proof agent which sort of tries to approve certain properties of the solution. While on the other hand, you have these LLM-based evaluators, which can look at the solution and you don't have, nobody has built an simulator, but they can just have a guess on how good that solution is. And in fact, that approach also works very well. And we have shown that this AI coscientist, which we have used for hypothesis generation,

• 24:43Speaker 02

  it basically uses a multi-agent sort of setup and where LLMs themselves are able to sort of figure out that certain hypotheses are better in terms of novelty and significance and impact and should be propagated, right? And that whole process ends up, and this might be surprising and counterintuitive to some, of producing much, much, much better results than the base large language model, right? So you are really able to discover new information beyond what the large language model itself alone was able to produce.

• 25:21Sarah Guo?

  That begs a question, which I think is like one of the biggest meta questions proposed by this sort of work, which is like, do we get self-improving AI, right? One of the things you demonstrated with AlphaEvolve is you can optimize the systems used to train AlphaBall, right? So you have this, you know, 23% speed up in part of the training infrastructure, if I recall correctly. Are we now witnessing the early stages of recursive self-improvement in AI? And, you know, what do you think the implications are if that's true?

• 25:52Speaker 02

  I think in some senses, sort of yes. But at the moment, what we have seen is basically improvements in computation time. So what AlphaEvolve has been able to do is basically make training more efficient. But you can ask the question, can you make the training, you can, can you improve the training process such that the underlying model is not only sort of trained faster, but is actually fundamentally better in certain cognitive tasks? And that is something that has to be validated still, right? But it is a direction that is definitely very appealing and, and something that is being sort of actively sort of looked at by many people.

• 26:37Sarah Guo?

  Do you have a reason to believe it won't work?

• 26:39Speaker 02

  It should work, but as, as we sort of mentioned, that having good evaluators is an important element, right? And so having a sort of evaluator which can say, this proposal that you have just suggested for me to improve the training process will yield a good result. So if you have that kind of evaluator, then it will work, but there is no reason why such evaluator does not exist. But we need to sort of work on building such evaluation functions.

• 27:13Speaker 01

  Maybe just one, one thing to add to it is that, like, I would also agree that we are maybe seeing the first sign of self-improvement, but one also needs to be very specific about what we have shown so far, like as Pushmeet mentioned, it's the speeding up the training of the next generation of the Gemini model. So the feedback loop is, is fairly long, at least currently, maybe on the order of months. But there is, you can call it self-improvement for sure. Maybe the, the big question that many people are curious about is how does this extrapolate into the future? And you can have different types of self-improvement. One is where you get maybe just a one-off benefit, like the model improves itself once and that's it. Another one is, okay, the model keeps improving itself continuously, but maybe the improvements

• 27:56Speaker 01

  get marginally smaller and smaller and smaller and you converge to some limit. Or maybe the, the improvements will, will keep accumulating up and up and up. And that's a big open question that, yeah, we, we don't have an answer to today.

• 28:10Sarah Guo?

  Let's take that projection to other fields and obviously these are all interrelated. But one of the things, Mate, you're really excited about is just how AI applies to these sciences. When you think about new mathematical constructions, improved solutions to, you know, open problems or problems that looked solved, you know, to humanity 50 years ago. What do you, what do you think the implication is in, in different fields? Like, is it a fundamental shift in how scientific discovery or mathematics gets done?

• 28:40Speaker 01

  First of all, yes, I, I'm super excited working in this area of using AI to accelerate the sciences. Because in a way, it's the most exciting application of AI that I can imagine. Like, what could be more valuable or exciting to advancing the frontiers of, of human knowledge? So, so yes, that, that is definitely there. And then, of course, in different fields of science, the speed of progress or the advance you get from AI might be slightly different. So in Alpha evolve, we've primarily focused on mathematics and computer science. Because these are the, the domains where it's the easiest to get these automated evaluation functions. Like, you often get them basically for free.

• 29:23Speaker 01

  That's not to say that you cannot get them in, in other branches of science. But in maths and computer science, it's just, they're just most common. If you think about biology or chemistry, you want to design a molecule, then you can have an evaluation function, again, in the form of a simulator or a predictive model that, given a candidate molecule, will make a meaningful prediction about, okay, is this actually going to work in practice? And then if you are in this regime, then again, Alpha evolve would be applicable. And, and we are only talking about the version of Alpha evolve that we have built today. And these are problems that we can address today. But we don't think that the journey of Alpha evolve finishes here.

• 30:06Speaker 01

  We have many ideas about how to make this system more powerful and more broadly applicable. And I'm, I'm fairly confident that we will see many applications across many branches of science. And then, this is only talking about Alpha evolve. There are many other agents, Bushmeat mentioned, AI coscientist and, and many others that I'm sure will keep transforming how science is being done across the whole spectrum.

• 30:32Speaker 02

  Yeah, so I think broadly, if you look at it, right, science is, a lot of science involves searching, right, searching for the right idea, searching for the right construction, searching for the right sort of solution, the right drug candidate and so on. And in some sense, like what scientists have been trying to do is sort of somehow make that process repeatable, right? There's, at the moment, there is still sort of an element of serendipity to some of the discoveries, but we are, as we move towards sort of rational material discovery or rational drug discovery, you are sort of seeing computational approaches and

• 31:15Speaker 02

  very systematic evaluations playing a much more important role in many areas of science. And I think as that work propagates, you will have systems like Alpha Evolve, which will be able to search in those spaces and use these evaluations much more effectively. So it's like you can sort of see this as a tool that will give scientists a superpower in their ability to search over very complex and sometimes counterintuitive sort of solution spaces.

• 31:50Sarah Guo?

  When I think about one logical extension to this approach, it is, let's say, like automated evaluation in the real world, right? So lab assay, you know, a bunch of robotic arms doing experimentation if you're screening molecules or something. What do you think the role, let's just say like very near term, if that vision is true of the human scientist or engineer is? Is it the problem framing, like determining the evaluation? Is it constraining the, like, giving some intuition for like a starting point or a search space? Like, what should the human scientist be good at from here?

• 32:30Speaker 02

  There are many sort of elements, right? First of all, as we were, as we have been talking about a lot, the role of the evaluation function, right? So that needs to be defined, like what do we really, how do we want to assess these solutions? But then there are many other sort of elements as well, right? When we are trying to find a solution, it has to have certain properties. What are those properties, right? Giving hints, giving sort of, for example, if you're trying to discover a new drug, you want to make sure that that drug sort of treats the disease but does not kill the patient, right? It has sort of, its side effects are low, right? Or it can be, what is, what is the delivery mechanism for it?

• 33:15Speaker 02

  So there are so many different requirements that a solution might want, that might need to satisfy. And some of them are encoded in the evaluator, in function evaluator. And some of them you might want to hard constrain the, them in the solution, right? And so can you specify those so that an agent like Alpha evolve can take that into account? While it is thinking about how it exposed the search space or how it constructs the solutions that it, it will sort of generate? These are all sort of very interesting places where human input might be required. But especially as we look at many different types of domains.

• 33:59Speaker 02

  So yeah, I, I think we should definitely see this as an amazing tool for scientists, for computer scientists, mathematicians, and then this is, in fact, this has been sort of our experience as well. That in the right hands, it is a very powerful tool, right? So like mathematicians who have tried to explore it and, and they, they have been able to specify what are the solutions that, what are the types of solutions that they're looking for? They can be much more productive and much more sort of effective in finding the solutions.

• 34:36Speaker 01

  I just wanted to highlight that even though we have been describing Alpha evolve as this kind of autonomous agent that does things on its own. Actually, in practice, using this agent often turns out to be surprisingly collaborative. And we have seen this in particular with mathematicians that. We have collaborated with and there are a few reasons for this. But one is that AlphaEvolve is an agent that doesn't just give you the solution. It searches for for an algorithm that constructs that solution. And so depending on how you how you set up your problem definition, often it's actually the algorithm that's even more valuable than the solution itself. Because the algorithm, it tells you how to construct the solution.

• 35:19Speaker 01

  So that means you understand what are the what are the ideas that go into building that solution. And maybe especially or definitely it's true in mathematics. That's what that's what people really care about, to understand the nature of our our universe and build up the understanding of fundamental ideas. And so it's actually often not interesting almost at all what the solution is, but what you care about is how you how you build it. And so we had a firsthand experience collaborating with multiple mathematicians. And it's been really fascinating to see where we would would share with them the the output from AlphaEvolve. And they'll they'll be like really fascinated looking at the code that it that it found

• 36:02Speaker 01

  and trying to understand, okay, what is it actually doing? And then understanding, oh, okay, this this is doing this, this is doing that. And now I can see why why if you put it together, then it leads to a really good solution. Yeah, I can also confirm from my own personal experience that looking at the at the code or the algorithms that the system finds, it's it's often a really interesting experience because it's it's code that kind of like looks human like, like it's something that you could have written. But would you have thought of writing it in exactly this way? And then trying to understand of, okay, what exactly is it doing? That that's a really interesting experience. But at the same time, it's one of the key strengths of of the system, not only for scientific applications where you can look at the code

• 36:46Speaker 01

  and get some understanding out of it, but also for many of the practical applications. It's it's hugely valuable that the the artifact you get out of AlphaEvolve is a is a piece of code. And then you deploy that piece of code. And so before you do that, experts and Engineers who have worked on that system can visually inspect that piece of code, understand it and make the final decision of whether it's going to be deployed. So it's in a completely different league from, let's say, considering using a neural network to make decisions in some production system where you kind of need to trust that the neural network is going to always behave in the way that you hope it will. With the code, you can look at it, understand it and make the decision yourself.

• 37:29Speaker 02

  I might add that basically not all code is interpretable by humans, right? The solutions and the programs that AlphaEvolve finds are sort of interpretable by human programmers. So this is going to be a very interesting area of work in the future. As to when you find these solutions, what can we learn from them? This was very interesting, like as Mate was sort of mentioning, this was a very interesting experience that we had working with Jordan Ellenberg in the first, in the earlier version of AlphaEvolve. When you were working on the cap set problem, the programs that it discovered had very interesting symmetries that mathematicians did not know about.

• 38:13Speaker 02

  And so not only the solution was mathematically interesting, but like the actual sort of construction, but the algorithm for producing that construction had the structure of it was interesting in itself.

• 38:29Sarah Guo?

  For listeners who are thinking about accessibility or implications for themselves where they're not professional mathematicians in collaboration with AlphaEvolve, what are the considerations in making some of these capabilities more broadly available?

• 38:43Speaker 02

  We want to make these capabilities accessible to as many people as we can, to the wider community. Now, we have started a trusted tester program where we have asked people to submit proposals. And what we intend to do with that program is to figure out what are the right ways in which people can really leverage AlphaEvolve? So we have internally used it. Across Google, but as you know, it requires certain things, sort of the need for a function evaluator. As part of the trusted test with program, we are going to be evaluating Alpha evolve on a bunch of different

• 39:26Speaker 02

  types of applications and that will inform our future release strategy as to how do we make it more broadly applicable. The second sort of element is that not only you need the evaluator, but you also need a significant amount of computational resources, right? Because it's not just one single L L M call. It's it requires a significant amount of function evaluation depending on the difficulty of the problem, right? If it's an easy problem, then you can do it very quickly. But if you really are going for some very hard problems with a very large extended search space and you want to spend a significant amount of time searching over it, then how do you build the overall system that can

• 40:10Speaker 02

  that people can sort of can use effectively and efficiently? That's the other sort of thing that we'll be thinking about.

• 40:17Sarah Guo?

  Last question for you both, is there sort of practical application within Google that you think will be interesting that you haven't tried Alpha evolve on yet?

• 40:26Speaker 01

  In this white paper, we tried to think about holistically. When we look at the computational infrastructure of Google, what are the the key parts in this infrastructure? To demonstrate that Alpha evolve can make discoveries across the stack, not only in one part of it, and that it can make discoveries that are highly valuable. And so we tried to cover the entire spectrum. So so we showed that Alpha evolve can improve the efficiency of the data center, it can contribute to hardware design, and it can contribute to improving the efficiency of most important pieces of software that that are being run inside Google. And like one intention here was to demonstrate that this is a really versatile tool that you can apply across the spectrum.

• 41:10Speaker 01

  And as Pushmeet was saying, this is a tool that is already available inside Google and it is being being used for for many, many problems. There are quite a few exciting ones. I'm not ready to share about the particulars yet, but as you can imagine, there is so many exciting computational problems in a in a in a place like Google within AI and also outside that, yeah, I'm I'm sure there will be many, many really cool results coming in the future.

• 41:38Sarah Guo?

  I think that's a great note to end on. Pushmeet, Mate, anything we didn't cover? No, I think that will break. Thank you guys so much for being here. Congrats. Okay, great.

• 41:47Speaker 01

  Thank you very much.

• 41:50Sarah Guo?

  Find us on Twitter at NoPriorsPod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.