What Most People Get Wrong About Evolution | Akarsh Kumar

A lot of people in AI treat evolution as a dumb fallback, basically random search for when you can't take a gradient. Akarsh Kumar thinks that is wrong. Selection hangs on to partial solutions, so mutations only need to be useful about 1% of the time for the search to keep making progress. Akarsh is a PhD student at MIT working with Phillip Isola, works with Sakana AI, and is first author of the Fractured Entangled Representation paper with Kenneth Stanley, Jeff Clune and Joel Lehman. He tells Tim Scarfe why the path a learner takes may shape the structure of what it learns, and why that is a different way of looking at intelligence from the statistical one. Most of the conversation is about ASAL, the method he led for searching whole spaces of artificial worlds. Rather than predict what a rule will do, ASAL runs the simulation and asks a foundation model what happened. Mapped across all 262,144 Life-like rules, the most interesting worlds sit on one small island. Along the way: Game of Life, Lenia (with a clip from its creator, Bert Chan), neural cellular automata, Boids, Particle Life and the emergence of persistence.---TIMESTAMPS:00:00:00 Intro: artificial life, ASAL and Core War in four minutes00:04:19 Life as it could be, not just as it is00:05:46 Why the order you learn things in matters00:08:33 No shortcuts: Wolfram, novelty search and regularisation00:10:44 Kenneth Stanley: the path matters, not just the destination00:11:31 Statistical intelligence vs regularity-based intelligence00:13:49 Artificial chemistry and other possible universes00:15:59 Convergent patterns: shadows of the substrate?00:18:02 How Conway's Game of Life works00:20:49 Change one cell, change everything?00:22:31 Lenia (with Bert Chan) and neural cellular automata00:25:26 Boids, Particle Life and cell-like creatures00:28:47 Persistence, entropy and what life is00:30:15 ASAL: a foundation model as the critic00:34:04 Impostors inside the simulation00:36:50 From primordial soup to alien animals00:38:26 262,144 rules and the island of open-endedness00:40:17 From artificial life to AGI00:41:35 Core War: Game of Thrones, Turing edition00:43:22 LLMs as the mutation step: evolving warriors00:46:03 Evolution is anything but random---REFERENCES:paper:[00:04:19] ASAL (Kumar et al.)https://arxiv.org/abs/2412.17799[00:08:08] Assembly theory https://www.nature.com/articles/s41586-023-06600-9[00:09:54] FEP paperhttps://arxiv.org/abs/2505.11581[00:22:38] Lenia: Biology of Artificial Life (Bert Chan)https://arxiv.org/abs/1812.05433[00:23:20] Growing Neural Cellular Automata (Mordvintsev et al.)https://distill.pub/2020/growing-ca/[00:41:35] Digital Red Queen: Core War with LLMs (Kumar et al.)https://arxiv.org/abs/2601.03335[00:43:42] MAP-Elites https://arxiv.org/abs/1504.04909[00:44:55] AlphaEvolve https://arxiv.org/abs/2506.13131[00:47:03] AutoML-Zero (Real et al.)https://arxiv.org/abs/2003.03384book:[00:07:04] Why Greatness Cannot Be Planned (Stanley and Lehman)https://link.springer.com/book/10.1007/978-3-319-15524-1tool:[00:18:12] Conway's Game of Lifehttps://en.wikipedia.org/wiki/Conway%27s_Game_of_Life[00:25:27] Boids (Craig Reynolds)https://www.red3d.com/cwr/boids/[00:27:12] Particle Life (Tom Mohr)https://github.com/tom-mohr/particle-life[00:41:50] Core Warhttps://corewar.co.uk/other:[00:08:49] Computational irreducibility (Stephen Wolfram)https://www.wolframscience.com/nks/p737--computational-irreducibility/[00:10:46] MLST: Why Every AI Model Is an Impostor (Kenneth Stanley, FER documentary)https://www.youtube.com/watch?v=o1q6Hhz0MAg[00:28:30] MLST: Blaise Agüera y Arcas on life emerging from codehttps://www.youtube.com/watch?v=rMSEqJ_4EBk---LINKS:Akarsh Kumar: https://akarshkumar.com/ASAL project page and demos: https://pub.sakana.ai/asal/Digital Red Queen project page: https://sakana.ai/drq/RESCRIPT:https://app.rescript.info/public/share/Qfv3T0EVzqOXL_CYYeRr9Blv8HnNm4JsXBh7ByFjJbc

• 0:00Speaker 00

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com. Not authorized by any candidate or candidate's committee.

• 0:30Speaker 01

  Your heart can tell you a lot about your health. Apple Watch Series 12 measures your heart rate every five seconds with the most accurate heart rate sensing in wearable. So your vitals app, now with heart rate variability, can tell you when something is off. And your readiness score can let you know when to rest and when to push. Hear the story in every heartbeat with Apple Watch Series 12. The features described are for wellness purposes only and not for medical use. iPhone 11 or later required. Based on Apple conducted study of heart rate accuracy, August 2026. Visit apple.com slash Apple Watch Series 12.

• 0:59Speaker 02

  I just want to clarify that evolution is anything but random.

• 1:06Speaker 02

  So artificial life is really about, like, understanding, like you said, life as it could be rather than life as it is. We need to understand the space of all possible intelligences, the space of all possible life forms that can be. Conway's Game of Life is a super famous simulation by John Conway, where you have this grid of cells, this 2D grid of cells, and they can either be on or off, alive or dead. That's it. And from this super, super simple world, it's like possibly the simplest possible thing you can think of. You run this, you get insane amounts of complexity that no one would have thought it could do until you actually run it. You can do counterfactual analysis in other universes.

• 1:50Speaker 02

  In what worlds do we get this open-ended creation of complexity and emergence? And in what worlds do we not get that? Why does this physics, when you run it, it produces this amazing phenomenon? And when you run this physics, it doesn't do that. It produces just random noise. Instead of hard coding a single simulation, you parameterize a space of simulations, which is much, much easier. Just simulate it, see what happens, plug it into a foundation model, and see what the foundation model thinks about what happened. There's 260,000 rules. We plotted all of them. So we found that there's literally like a big island of solutions which are not open-ended. And there's a small island of solutions where all of the coolest simulations lie.

• 2:34Speaker 02

  Some of them actually end up creating little patterns which move around that look like cells. This object, this cell-like object, is exploiting the physics in a way to create a persisting object in this world, right? If you train them on calculus, they'll learn calculus. You don't have to train them on arithmetic first. And it's such a weird, alien way that we train these models. I really like to call them a statistical intelligence. And they're like basically perfect statistically. But this FER idea and this pick breeder line of work, it's basically like a different paradigm of intelligence that we're trying to study. So it's a difference between like a statistical intelligence and a regularity-based intelligence. And you can see this in like the pick breeder skulls.

• 3:17Speaker 02

  It's in the solutions, like the neural circuits that we find. The regularities are baked into the circuits. We wanted to see what would happen if you used LLMs to evolve basically small, little assembly programs that compete against each other. So there's this programming game invented in like the 1980s that people used to play. It's called Core War. What my program needs to do in order to do that is it needs to basically inject an invalid operation in front of your program's execution thread. It's going to halt your program by crashing. Your program will crash. And can we create like this open-ended evolutionary arms race of programs from this?

• 3:56Speaker 03

  It's like Game of Thrones' Turing Edition.

• 3:58Speaker 02

  Yeah, yeah, exactly. That's a good way to put it. The LLM is not good at red code. If you just try to zero-shot this language, it's terrible. But if you do this evolution and you have this verifier, it tends out to be really good. Evolution is probably the most powerful algorithm that we know to solve like discrete combinatorial problems. If you solve like even one part of a puzzle and you hang on to that and you search for the other ones, you can turn your exponential search problem into a linear search problem. We're basically trying to search for cellular automda, like neural cellular automda, which go from self-replicating molecules, so cells floating in primordial soup, to predator-prey dynamics, to alien animals roaming around. You can imagine that if you did this

• 4:43Speaker 02

  with a big particle simulation with like quadrillions of particles, and you had the right foundation model, and you searched for this sequence of events, then what would happen? Maybe you end up with some physics rule for this big particle system, which really does give you this entire sequence. You get animals at them, just like what happened in our universe, right? The field of artificial life, I really think is like a very long-term bet on intelligence.

• 5:19Speaker 03

  The paper is ASL: Automating the Search for Artificial Life with Foundation Models. Akarsh and his co-authors use vision language models to find interesting simulations across game of life, linear, voids, and more.

• 5:34Speaker 03

  Early on in the paper, you said, you quoted, "Artificial life is not just as we know it, but also as it can be." What do you mean by that?

• 5:43Speaker 02

  So artificial life is really about, like, understanding, like you said, life as it could be rather than life as it is. If we really just wanted to understand life on Earth and like how it is right now, we'd be studying biology, right? It's the same reason that we don't just study neuroscience to understand intelligence. We're creating AIs because we want a more mechanistic understanding. And not only that, but me personally, I like to, I'm going to borrow this quote from Jeff Clune that I liked a lot. So we really want to understand, if we want to claim that we understand intelligence or we understand life, we can't just talk about this specific form of life. We need to understand the space of all possible intelligences, the space of all possible life forms that can be. And only then can you talk about general principles

• 6:27Speaker 02

  for what governs intelligence or what governs life, right? You can't just look at one car and be like, this is how all cars are, right? That's the same way. You can't just look at GPT-5 and be like, all intelligences are like this. You can't just look at the human brain and be like, all intelligences are like this. There's a space of things, and we need to study the whole space. If you look at how like humans learn and like animals, and have like a natural curriculum. They're like, I'm going to learn this, and then this, and then this. There's no other way. But if you look at how like our models are trained, you can train them on anything and it'll just work. And the question is, a lot of people think that that's fine and that maybe it's better than how humans learn. And that's a possibility. It could be just be better, right? But there's reasons to believe that there is something fundamentally wrong and the universe doesn't work like that.

• 7:11Speaker 02

  The universe doesn't work like you can just learn anything at any given time. And that's, I think, the fact that humans learn in this sequential order and evolution works in this path-dependent way is the reason that it creates such nice, regular structures in the representations that it uses. And that's the reason it's so robust, adaptable, generalizable. It's for those reasons, I think.

• 7:32Speaker 03

  We see this cashed out in the form of curriculum learning, which is this idea that we learn step by step and we learn in kind of, with an increasing complexity, right? And you were just pointing to when you learn step by step, increasing the complexity, the representations you learn are better. But how would you describe that in a kind of no-nonsense way? What's the intuition?

• 7:56Speaker 02

  The curriculum is not the only thing that matters because people have been doing curriculum learning in RL. And I was obsessed with curriculum learning in RL. I think the key thing is that, I mean, this goes back to the why greatness cannot be planned lesson. You need like a serendipitous curriculum, which is not a pre-planned, like, I'm going to do this, then this, and this. It has to be much more serendipitous and embrace the chaos of our nature, of the universe, right? It has to embrace this. You have to embrace this idea that not just like going down this curriculum, but any possible curriculum which might lead me to somewhere interesting. So that's just a shout out to like, you need serendipitous curriculums. And to answer the question of why curriculums actually end up mattering, I think that's a fundamental question that I'm really interested in answering. And I don't know like the short form answer for that. But it's like a fundamental property of our universe

• 8:40Speaker 02

  that if you want to capture, as you like to say, the car of the universe and understand it by the joints, right? That carve it up by the joints. I think in order to do that, you need to have this process of complexification where you build regularities on top of other regularities in this sequential way, all right?

• 9:00Speaker 02

  I'm not exactly sure why it has to be like that, but I know that natural evolution works like that. And I know that a lot of theories like assembly theory, they work like that, like from, that's like an evolutionary theory in biochemistry and stuff. And human learning also works like that. It works in a sequential way. So I'm not exactly sure why it works like that. But it definitely seems something like it's something fundamental to our universe. And we need, really need to understand that better, right? If someone can answer that question, why that is important or prove that it's not in general necessary? That would be a huge discovery, right? Yeah.

• 9:33Speaker 03

  It would. And the only analogy I can think of is Stephen Wolfram. So what we're talking about here, mapped onto his ideas, is that we need to have a constructivist approach to representations, which means that we, we build them step by step. And he, he noted that there's this phenomenon called computational irreducibility, which is that basically there are no shortcuts. And I guess the way that cashes out in my mind is that it's saying that if, if you want to learn a complex represent, you know, you want to acquire a complex representation. You, you can't actually like jump ahead. You actually need to synthesize these building blocks and you need to take the path.

• 10:10Speaker 02

  Yeah. And one of the things I like whenever Ken explains his philosophy is that whenever Pete, whenever he says like, oh, we need novelty search. We need this thing. And people say, oh, but that's just like searching in the dark. That's like random. It's actually anything but random because randomness is like just doing random actions. Novelty, you know where, what you did before and you want to avoid that. So I guess what you're saying is that if you want to build, if you want to respect the path of where you previously came from, whether it's like by avoiding it via novelty, or because you're respecting the symmetries that you learned along the way in a representation. You do need to respect it and that gives you an idea that get constraints where you can go. And I question is like, how much does that constrain you? There's a good chance it constrains you a lot and it constrains you towards in a right way. Towards the right path, right?

• 10:54Speaker 02

  That's why whenever I think about the F E R hypothesis and this line of work. I really like looking at it through the lens of regularization. What like? Like people talk about regularization and deep learning with like weight decay or some sort of other regularization mechanism. And I love that. I think everything is just regularization. The data is a form of regularization. It regularizes. Um, what like you need to model the world. That's a form of like, it's like, it's like basically molding the clay in a way that you need to model the world. L two regularization, like weight decay is another form of that. Basically it's saying you want to find solutions with low weight norm. Right? And then I think that this curriculum learning, basically you want to respect the symmetries is just another form of regularization.

• 11:34Speaker 02

  But I think it's extremely, extremely, extremely strong regularization that you're subjecting this model to. And maybe it's the right regularization that we really, really need.

• 11:44Speaker 03

  And here's a quick refresher. Our fractured and tangled representations documentary that we published last year explained why the path is more important than the destination when it comes to representations in neural networks. This is Kenneth Stanley. This hierarchical locking tells us something really important about how representations emerge. We think it's about finding the right building blocks now, but weirdly it's about making future discoveries more.

• 12:12Speaker 04

  It matters not just where you get, but how you got there. You know, and that, that's like something missing right now, you know, because we tend to just care where you get. Like we look at the benchmark score, you know, in the field and it's like, that's the result. It's doing really well. It just passed the math Olympiad. But what if it matters how you got there?

• 12:30Speaker 03

  But that kind of structured evolution, I think is a completely different paradigm to doing it the statistical way.

• 12:36Speaker 02

  Exactly, exactly. And I think you said the right word. It's like a paradigm to understand intelligence. But this, this F E R idea and this pick breeder line of work. It's basically like a different paradigm of intelligence that we're trying to study. We're not studying it from the standpoint of statistics. We're studying it from the standpoint of regularities, right? So it's a difference between like a statistical intelligence and a regularity based um, intelligence. And um, I really like the idea of statistical intelligence. I think we did a good job like going down this path. We should still go down this path. But we should also uh, not put all our eggs in one basket and really try to understand this other thing. Because there's lots of evidence that this other thing has lots of um, potential here, right? Um, and it really ties into what all the criticisms of deep learning.

• 13:19Speaker 02

  They're basically talking about regularities. And whenever people talk about like, we need symbolic processing. Whenever we, we need geometric deep learning. We need more inductive biases. And people have like all these different fields, uh, there's whole different fields on like what we need, right? And they're all forms of adding regularities and symmetries to the system. And what I really like about this F E R approach is I like to think about it. It's like, it's like an automatic symmetry learning algorithm, right? Because um, so I have, I love you have an example I like to say. People like C N Ns because they're translation invariant, right? People like transformers because they're permutation invariant and they encode certain symmetries. And that's great. I mean, we need permutation invariant for set operations, right?

• 14:01Speaker 02

  But can you code in like uh, lighting invariance into your model? Like lighting invariance, like if you want your C N N or image model to be invariant to whether it's nighttime or daytime. There's no equation in the world. I'll probably get you there. Maybe there is in the future, but that's impossible for me to imagine that you could come up with a mathematical like inductive bias that's going to guarantee that, right? So what you really need is some sort of data-driven symmetry learning thing that automatically respects it. And it forms neural circuits, which are naturally have that inductive bias built in, right? And you can see this in like the pick breeder skulls. It's like the neural, it's not that the inductive bias is in the hypothesis space of things we're searching for.

• 14:43Speaker 02

  It's in the solutions, like the neural circuits that we find, the regularities are baked into the circuits.

• 14:49Speaker 03

  Well, I suppose another way to look at it is with human intelligence. Um, our knowledge acquisition process is grounded in the world. When, when we build these artificial life simulations, they're not grounded in the same way. Right? So they, they, they just, and but they do have these properties that we're just talking about. So they're actually building these representations. And interesting things happen when you look at the phenomena of how these things evolve. That, and you, you get certain things that just fall out of the fact that you're doing this iterative computation, even in an undirected way.

• 15:20Speaker 02

  Yeah. So I want to touch on one key point that you said, that you said it's not grounded in our world, right? And that's one of the fa- my favorite things about artificial life is that you can ground it however you want to, in whatever artificial world you want to, right? So people often think about artificial life as it's only talking about life. The reason I like it, because there's a subfield in artificial life called artificial chemistry. It's about just changing, instead of using like the normal hydrogen helium, you can change how they interact and see what happens, right? So that's the power of artificial life. You can do counterfactual analysis in other universes, right? And I really like this idea because artificial life is not only about life, but it's also about studying like what happens in general in like artificial physics, artificial chemistries. Can we study the emergent properties in each of these different universes?

• 16:06Speaker 02

  When, in what universe, this is the question that I really want to focus my research agenda around. Is that in what like worlds do we get this open-ended creation of complexity and emergence? And in what worlds do we not get that? And why is it, and this is the key question is why is this parameter, why does this physics, when you run it? It produces this amazing phenomenon. And when you run this physics, it doesn't do that. It produces just random noise or something, right? And that was the key reason I actually uh, worked on automated search for artificial life as ASL work. Is because I want to answer that exact question uh, directly. But ASL was trying to basically parameterize a space of intelligences. Find the good ones, find the bad ones and just do simple stuff like interpolate between them and just see what happens, right?

• 16:51Speaker 02

  Interpolate the parameter space between them and run the simulation for each intermediate point and just see what happens. Just simple scientific analysis like that.

• 16:59Speaker 03

  Yeah. And another thing that jumps out to me is that you get these convergent patterns in rule your space. So even though the rules that, and, and, and we use this term emergence, and maybe we should talk about that as well, which, which is, and um, we, we don't necessarily mean emergence in the strong David Krakauer sense, but you know, weak emergence is just that when you sequentially apply these rules many, many times, you see these very surprising unpredictable phenomena. And these phenomena have characteristics which seem, seem to be shared in rule your space, like certain types of glider and object and pattern. And of course, Wolfram believes that the universe itself is computational, but it's almost like we're seeing shadows.

• 17:39Speaker 02

  What do you mean by that? Like, um, like the, like the, you're seeing shadows of, I guess, the substrate that the universe is running on.

• 17:46Speaker 03

  Well, I mean, it's, it's only a hypothesis. So like, we see these convergent patterns in different parts of Rulio space. And now the skeptical person might just say, okay, well, those are just, they're like designed by us. They are just features of the way that you are doing this computation. They are features of the structure of the game of life, maybe because you're using a grid or you're using a certain type of, um, you know, a program interpreter to do it, that those things are influencing these patterns. But they might also be hints of just something else, you know, they could be shadows.

• 18:22Speaker 02

  I, I kind of agree with you because in my ASIL work, I, I span the space of all possible 2D cellular automda, for example, right? And in my case, I had a lot of diversity. In one world, a glider looked like a Conway's Game of Life glider. In another world, the, uh, the minimal glider looked like a train going down the tracks, right? So, and it like, it left like a long path. So, um, in different worlds, it can be different. But I guess what you're talking about is that when they're all the same in all the different worlds, that points to something fundamental about like, not only the world that you're in, but like, not only a rule that you're in, like the rule that you're operating on, but like the entire substrate that you're working in, maybe all possible rules in that substrate, um, have this type of thing.

• 19:02Speaker 03

  Um, maybe we should just talk a little bit about some of these simulators, just so that folks in the audience understand what we're talking about. Just to bring this to life, can you, can you go through some of these simulators and just explain how they work?

• 19:12Speaker 02

  Yeah, I can go through some of the simulators. So, uh, I can start off with, uh, the linear one. I, I'll start with the simplest one, Conway's Game of Life. So Conway's Game of Life is a super famous, um, simulation by John Conway, where you have this grid of cells, this 2D grid of cells, and they can either be on or off, alive or dead, that's it. And basically it has super simple rules, like if I'm dead and three of my neighbors in like my eight, I have eight neighbors, if three of them are alive and I'm dead, then I come to life, right? As an act of like, um, uh, just coming to life. And there's like rules like that, there's like two or three rules like that. And it's like super simple rules and that's it, that's it, that's, that's, that's all you need to know.

• 19:56Speaker 02

  Then you start off with a, uh, grid of like random, um, dead or alive cells, or you can start off with some pattern that humans have created. And you just run this simulation over many, many time steps. And from this super, super simple world, it's like possibly the simplest possible thing you can think of. You run, you get insane amounts of complexity that no one would have, um, um, thought about, that thought it could do until you actually run it. And now people are like, uh, there's whole Discord servers. Uh, I'm actually in one of them, where people try to, uh, go through and they just share what they found. They were like, oh, I found this cool spaceship that oscillate, that, um, that's a repeating pattern that repeats every 64 time step and it moves at this, um, speed.

• 20:39Speaker 02

  And I really love this community because they call like the glider, how fast it moves, I think it's as like a certain speed, right? And they realized that there's like a speed of causality in this system, which is like one cell per time step. That's like the speed at which information can propagate. So they call that the speed of light C. And then they, they use their gliders and they're like, this goes at C over 16 speed, right? Just like in real physics, how people talk about this kind of stuff. So it really is like an artificial universe. And people, a lot of people seem to think it's like a toy system. But I guess the point of it is that it's toy enough that we can study it. If it were super complex in our real world, then we wouldn't be able to study it, right? And, but I do think it's a very interesting quest, that's one of my quests, is to create more compelling universes, which are,

• 21:25Speaker 02

  uh, capture, like Conway's capture, Conway's Game of Life, they capture some properties of our universe, like emergence and this idea that you can build up global patterns from local interactions. But there's other stuff you want to capture too, right? Like, uh, the idea of like self-organization and, um, open-ended evolution. So a lot of my research is on trying to find simulations which have all those properties that we care about. And that was the point of ASIL again, yeah.

• 21:48Speaker 03

  So then how structured is this space? So, um, you know, like we, we think about, um, you know, many of us have seen visualizations of representation spaces of neural networks, like using Tuzney and something like that. And when we see islands and clusters and whatnot. Is it the same for something like Conway's Game of Life? So for example, if you change one neighboring cell, does it trans, you know, is it like a wormhole where you get transported into a completely different part of the space? Or does, does it make some commensurate small change in the emergent, like high-level domain?

• 22:18Speaker 02

  Yeah, exactly. That's a great, great question, and this is one of the things I really want to study. So in the initial state of Game of Life, yes, it's very, very chaotic. In Conway's Game of, let's talk about that specifically, it's very, very chaotic. I've had examples where you change one cell, like one pixel in like the 64 by 64 grid. And beforehand, I actually use this in one of my talks, it's like, it, uh, basically runs forever. It's just like, it's just, not forever, I shouldn't say forever, but like, for like tens of thousands of steps, it just keeps on moving, right? And it just like keeps on doing interesting stuff. You change this one thing, it stops, it basically halts into an oscillating pattern in like less than a hundred steps. Um, so it's like in the initial state, uh, yeah, it's very, very chaotic. But the interesting thing is that this is not necessarily true for all simulations.

• 23:02Speaker 02

  And, um, in ASIL, we're really interested in searching over the physics, like the actual rules that dictate the dynamical system, like what I call the physics. What we've been talking about is, is the initial state, right? But there's two things, initial state in physics, and if you do it in the physics, uh, Game of Life seems to be a little more robust, as in if you change like one, there's basically 18 possible rules you can toggle on or off. And if you change one of them, it doesn't impact it too much in my experience, but there are some exceptions to that where, yeah.

• 23:30Speaker 03

  Very interesting. So, um, and, and talk about some of the other simulators as well.

• 23:34Speaker 02

  So, uh, one of the, so I can go through a list of the ones we used in our paper. In Linear by Bert Chan. And he basically created this continuous generalization of Game of Life, which generalizes to like three channels and is, you can dictate them to RGB. And they're basically just like a continuous, um, differential equation update rule that dictates how they operate. So I think that one's really cool.

• 23:55Speaker 03

  Here is Bert Chan, the creator of Linear, on what happens when he made the rules continuous.

• 24:01Speaker 05

  But by doing this kind of, uh, making everything continuous, what will happen? When I cook this system in my computer, something creepy happened. Uh, a pattern that, uh, uh, creeping across my screen.

• 24:15Speaker 02

  And then, um, the cool, uh, also another cool one is the neural cellular automda.

• 24:20Speaker 03

  Yeah, by Morv Vincenzo.

• 24:21Speaker 02

  Yeah, are you gonna interview him?

• 24:22Speaker 03

  Uh, hopefully. And, and he famously in 2020 released this paper called Growing Neurocellular Automata. And my God, we talked about this on MLST millions of times, the thing with the Gecko.

• 24:32Speaker 02

  Yeah.

• 24:32Speaker 03

  And, and it was showing this emergentist optimization and morphogenetic engineering, you know, essentially where you train a CNN and then you just, I mean, because you know, you can kind of think of this a bit like a convolution operator. So, you know, you, you, you learn the parameters of stochastic gradient descent of the CNN. Um, I, I guess it was, it was, it was trained, it, it was like emergentistally trained to reproduce an image, so like a Gecko.

• 24:58Speaker 02

  Yeah, you're training the local rules such that they reorganize in order to, um, self-organize in order to form the lizard. Yeah.

• 25:06Speaker 03

  Yeah, and then you can delete parts of the lizard. And then because this thing is running continuously, right? So it's just, you know, convolution, convolution, convolution, and it reconstructs the lizard. So, and, and again, this is like what we want in AI systems is, you know, we don't want imposters. We don't want to learn the lizard. We want to learn the ability to reconstruct the lizard from the building blocks. You know, the path dependency thing, so it's coming up again and again. Like, that's just so important, you know, rather than just learning what the lizard looks like.

• 25:34Speaker 02

  Yeah, yeah, yeah. And, um, to build on top of that, I guess it's like you don't really want to learn, um, the intelligence. You want to learn the process which grows the intelligence, sort of, right?

• 25:44Speaker 03

  Yes.

• 25:44Speaker 02

  It's similar to that. I mean, that's one of the reasons that deep learning took off in the first place, because we're not hard coding these, all this knowledge about, we know about the world, we're letting it learn from the data itself, right? But there's other problems to that maybe, but yeah. Um, going back to the neural cellular automda, um, the, uh, just for the audience, it's basically just like you have the cellular automda. But what if you made this three, basically the dynamics rule is like this mapping from a three by three patch into one cell, right? That's what the dynamics rule tells you to, how do you go from one time step to the next? And basically what they do there is they just replace this with a small neural network that dictates this. So that gives you a whole space of possible physics rules that you can search over. Um, so I really like neural cellular automda. Um, and I can briefly go over the next two, the Boyd's and the particle life.

• 26:28Speaker 03

  Yeah.

• 26:29Speaker 02

  Yeah. Boyd's is like this super seminal algorithm inspired by how fish and how birds flock in nature. Because if you look at, I actually just went to the Osaka Aquarium and I was watching these videos, uh, not these videos, I was seeing this fish, fish in person. And they were like always together and there's like a whale shark coming out behind them and they just like steered to their left a little bit. And I was like, how do they know how to do that? Because each fish, they, and only one of them knew about the whale shark and it tugged the other ones to the left. And the other ones all kind of just realized, oh, we should all go there. It's an example of collective intelligence in nature, right? So Boyd's is basically an inspiration of that, where you have these like bird-like objects which flow through 2D or 3D space.

• 27:09Speaker 02

  And they have super simple rules where they just look at the neighbors, like I'm my neighbor to the left and to the right. And I basically, let's just say I have like four neighbors that I can see, right? I'm just gonna make a decision on whether to go left or right based on where their positions are. That's it. And if you made the right decision and you're all making the same decision, you can get the emergent flocking behavior that we see in nature. And yeah, so what we did is we just like parameterized this with a small neural network and just searched over this decision-making network, this, well, this decision-making. If, if the Boyd's make different decision rules, how does the global behavior of the collective change according to the local decision rules that they're making? Particle life is super cool because in grid-based simulations, it feels like this, it's like this spatial thing. But particle life is basically saying

• 27:54Speaker 02

  that we're gonna have this 3D Euclidean space or 2D Euclidean space, and we're just gonna throw particles at it, just like, just like, just vanilla particles. But their physics rule is not gonna be like, you know, Newtonian mechanics or something like that. It's just gonna be whatever we want it to be, right? So in this case, in particle life, uh, I, I, I was inspired by Thomas Moore. He has this great YouTube videos on this type of thing. Is basically, um, you have this set of particles and each particle can take on one of six types, right? And these are dictated by their colors. And then what you can have is that you have this six by six matrix, which tells you how do different types of particles interact with each other? So it could be that red and red repel, but red and orange attract.

• 28:38Speaker 02

  And orange and green, they, uh, are neutral, right? And you have the six by six matrix, and if you change the values of this physics matrix, this, this is the physics of the world again, right? Or you can call it the periodic table or whatever you wanna call it. As you change this, um, physics, how do the actual, how does the simulation change, right? Some of them are super bland and boring and I don't wanna look at them. But some of them, they're like evolving and moving and doing interesting things. And some of them actually end up creating, um, little patterns which move around that look like cells. So you can literally see them, they have like a cell barrier, which is like this red coating. And inside they'll have like purple and yellow in a triangle. And that's like its insides and it's, it's basically this is a simulation.

• 29:22Speaker 02

  This object, this cell-like object is exploiting the physics in a way to create a persisting object in this world, right? And you just interviewed Blaze, so this idea that, um, life is just the persistence, right? It's just that's, uh, idea that like life is basically about persistence. These organisms are just trying to figure out how to be persistent to the next timeframe. And they've exploited the laws of physics in order to do so.

• 29:47Speaker 03

  Yeah, and Blaze was saying that this persistence is kind of like a natural, is what, you know, I was talking about these natural convergent phenomena. That might be an example of one because even though these systems are basically ungrounded, we're just, we're just running this, this computational simulation. And isn't it fascinating that we see the emergence of like persistence? That things just want to, to, to kind of keep their form over time?

• 30:10Speaker 02

  Yeah, the exist, the emergence of persistence is very fascinating. It doesn't happen in all simulations, it only happens in some. But I think this points to the broader point about what life is and what the second law of thermodynamics tells us. There's a generalization to persistence, which is this, I like, this at least how I have a, in my mental model of how I think about it, is the second law of thermodynamics and life and persistence, they all basically are saying an evolution, I should say. They're all saying the same thing. They're talking about what exists right now and what is likely to exist in the future. So the second law of thermodynamics is saying that you're more likely to go towards higher entropy states, right? That's just like, um, if you look at the macro states and micro states of any simulation, that's how it operates.

• 30:53Speaker 02

  But if you look at it from the standpoint of evolution, what is more likely to exist in the future? It's self-replicators, because they'll naturally exist more often in the future versus non-self-rep, self-replicators. So they're really, uh, people make it out to be like, oh, life is like fighting entropy. There are really two sides of like the same idea of persistence or what is likely to exist in the future.

• 31:15Speaker 03

  Yeah, fascinating. So in, in your paper, essentially you are using foundation models as a, as a critic. Right, so you're saying, I wanna have a principled way for searching this space.

• 31:27Speaker 02

  Yeah, we use it in three different ways. So at a high level before that, I guess the field of artificial life, I think it's a very cool field, but often people like only like, uh, simulate single simulations and they try to draw conclusions from it. And, um, and I, I feel like it's cool, but like you really need to be studying the space of simulations, right? So I think, uh, our algorithm, what it does is basically you, instead of hard coding a single simulation, you parameterize a space of simulations, which is much, much easier. Basically, um, it's a difference between coding up the periodic table, like, well, how does each element interact with each element versus just saying that there's gonna be like a hundred elements, and I don't know how they interact, you figure it out, telling the algorithm to search for it automatically. So it's a much easier problem to just say there's a hundred elements, right?

• 32:11Speaker 02

  So once you have the space of simulations that you have, uh, described, then you can just search for what you want in an automatic way using a foundation model. And the reason I like Isol is because it captures this principle of computational irreducibility. You don't need to know what this physics rule is gonna do and try to predict which one is the best one. Just simulate it, see what happens, plug it into a foundation model and see what the foundation model thinks about what happened, and then go back. And the key thing is that here, you're letting the simulation unroll and you're not having like, you're not trying to predict what's gonna happen, right? You're kind of embracing this idea of chaos and computational irreducibility.

• 32:49Speaker 03

  Yeah, and in a sense, it's a step past, um, Mord Vincet's idea with the neurocellular automata, because he was doing this emergentist optimization where you're saying, okay, when, when the macroscopic thing emerges, I want it to look like this specific thing. And you're now plugging in a foundation model and you're saying, I want to ask the foundation model whether it resembles something I'm interested in.

• 33:08Speaker 02

  Yeah, yeah, exactly, exactly. And, um, there's like two reasons it's better, because one is just easier to specify what you want. You're gonna be like, I want a cat. I don't need to like have a picture of a cat and supervise it in like a pixel-wise manner. I can just say I want a cat in like a general way, right? That's one way. But importantly, it also makes the optimization and the search much easier, because it's really dang hard to find a self-organizing rule that creates this particular image of this particular cat. That's like really, if you do that in Converse Game of Life, that's not gonna happen. But if you just say, oh, I don't want that specific cat, I just want a cat, give me any cat. That's a much easier space of things that satisfy that problem, so it's much easier to search for.

• 33:50Speaker 02

  And we experienced this firsthand that, um, in, like, Con, in, um, the open-ended simulation, we're not searching for particular configurations of states that we want. We're just saying we want some open-ended simulation. It can be open-ended in however it wants to be, as long as the foundation model thinks it's open-ended, it, it'll, it'll do. And the cool thing about, I wanna say, the reason we use foundation models is not just because it's like the newest and greatest thing that you can use, it's because we really care about what a human thinks about these simulations, because it's really hard to describe what complexity is, what emergence is, what does it mean to have a, uh, very interesting system? It's like you can't really mathematize that really easily.

• 34:32Speaker 02

  And if you did and you optimized for those simulations, you'll find like Goodhart's Law, you'll find something that you asked for, but not something that you wanted, right? So what we really want is what a human thinks about this in this subjective vague way. And we can't put a human in a loop because that's like too expensive. So what's the next best thing is you try to find a representation space which matches that of a human. And there's a lot of evidence statistically, at least statistically, that these foundation models are matching human representations from a, in a statistical, um, manner, right?

• 35:03Speaker 03

  Yes, so I'm gonna show a graphic on the screen now. So this is the, the first experiment you did was when you had a supervised target. And we've got a multi-celled, um, organism, uh, a network of neurons, a virus and a fungal colony. And in every single row, we can see what the emergent phenomena was for linear and for Boyd's and for particle life. Um, but it also leads to the question of there must be some degree of ambiguity, because there are so many possible places in computational space where the foundation model would say that this was a match, and it's represented differently in the different simulators. And to the, you know, we had this concept we were talking about, like this, this imposter. So there are many different paths to produce the phenomena. Would you call these things imposters or do you think that they're different?

• 35:47Speaker 02

  So these are, um, that's a good question. So you're basically asking, let's talk about the Boyd's one. You're basically saying, is that decision-making rule of each Boyd, is it an imposter or is it, um, the real thing that gives you the real, uh, thing, right?

• 36:01Speaker 03

  Yeah, and, and what would an imposter mean in, in this simulator?

• 36:05Speaker 02

  Yeah, that's a great question. So I think the idea of imposter is, I like to ground this idea in its performance on future tasks and how, how, and how it adapts to things we want it to adapt to. That's, uh, that's how I like to coin it. I don't really like to ground it in modularity or something specific like that. I like to ground it in set of future tasks.

• 36:24Speaker 03

  And you continue to run the simulation, does it decohere?

• 36:28Speaker 02

  Yeah, in this case, they're pretty robust to that sort of thing. But in this case, I mean, it could be that like, like small, uh, changes. I guess what I'd call, you, you could imagine that if these Boyd's were really UFR or like good, nicely factored, then small changes to their network change the global behavior of the collective in like nice ways, where it goes from a spiral to like triangles or something, maybe not triangles, but some other cool shape. But in this case, they don't do that, they actually degrade. If you sweep the parameters, then they become basically kind of random. So in that way, it kind of is a imposter version of Um, this spiral. Exactly, exactly. And the thing is, in this case, they're evaluated on and they're expected to maintain, persist into the future of this simulation.

• 37:13Speaker 02

  So in that way, they're very, very well structured because they are able to persist indefinitely. You could run this simulation probably for like millions of time steps and they'll still be doing what they're doing. Yeah, but if you want them to do something else, then they like make a triangle by sweeping some weight, then they would fail in that way. And but but to be fair, they're not trained to do that in the first place. They're not, their evolutionary lineage doesn't, there's no reason for them to want to have to do that, right? I want to point out to, we had this figure in our paper, which showed like what the future of this possible direction of searching for simulations via foundation models could look like. And we had this fun example, I have it pulled up right here. Uh, the bottom one is particularly fascinating. We're basically trying to search for cellular automda, like neural cellular automda,

• 37:56Speaker 02

  which go from self-replicating molecules, so cells floating in primordial soup, to predator-prey dynamics, to alien animals roaming around. And this is just like a, like a fun toy example that we used. In this case, the cellular automda just memorized that sequence of frames. But you can imagine that if you did this with a big particle simulation, with like quadrillions of particles, and you had the right foundation model, and you searched for this sequence of events, then what would happen? Maybe you end up with some physics rule for this big particle system, which really does give you this entire sequence of like, you get animals at them, just like what happened in our universe, right?

• 38:35Speaker 03

  Yeah, but isn't it fascinating that you've taken something which you previously argued was a fractured, entangled representation? So the foundation model has fractured, entangled representations. Um, but they produce imposters, but you only need the imposter because all you're doing is you're acting as a critic on the imposter. And then you're using that to bootstrap the creation of something which is a factored representation.

• 38:53Speaker 02

  Yeah, yeah, it's a good point because the, uh, the performance of these foundation models for this type of tasks, it's like, it will be flawless, right? Because this is the exact type of things where you only care about the output behavior. You don't care about its internal representation. You don't care about creativity in a sense. Um, because the humans are the ones kind of injecting those prompts. And you're using this, like, statistical intelligence to bootstrap, like a real, natural, like, um, organic, um, particle simulation, which should have all the properties of our real world, right? Yeah.

• 39:26Speaker 03

  Yeah, exactly. And then, so the other experiment you did was creating a diversity search. You said some parts of the computational space are, are open-ended. So there's this Goldilocks zone you were speaking to before. Maybe, maybe we're in such a region of the computational space where all of this lifelike stuff happened and this open-endedness, um, happened. But did you see similar things in, in the space where these properties that we're talking about, like existed in a sliver of this computational space?

• 39:56Speaker 02

  Yeah, definitely. So if you mean the, let's, uh, in the computational space, I guess we can talk about the laws of physics. That's your space, right? In that way, definitely. I mean, I have this, uh, I have it pulled up right here. Found simulations which are basically in, that look more open-ended than Conway's Game of Life. Yeah, that one. That exact one, yeah. Basically, we found simulations which are more open-ended than Game of Life, right? But we wanted to go further than that and ask, what does this space of possible rules look like? What rules are open-ended? What rules are not, right? So if you scroll down to in the paper, we had this graph right here, where we basically tried to plot all possible rules. There's 260,000 rules. We plotted all of them. We colored them, uh, we, but we did, we, we ran the rules. We put this image into clip

• 40:41Speaker 02

  and plotted this clip space in 2D, but we colored each of the points based on how open-ended they are. So we found that there's literally like a big island of solutions which are not open-ended. And there's a small island of solutions where all of the coolest simulations lie. So in that way, I mean, this graph kind of outlines exactly what you're talking about is the space of possible simulations. And we want to find what part of this space is like that. Yeah. And you can see that there's like this, there's like different islands. There's like one big island and like one smaller island and one tiny island with like one simulation that's very different from all the rest of them and its behavioral properties.

• 41:17Speaker 03

  But then we have the problem of, there seems to be a disconnect between building these artificial life simulations and we want to build a general purpose intelligence which has utility. So how can we leverage this type of technology to build the AGI?

• 41:31Speaker 02

  Yeah, yeah. So that's a, that's the grand question, right? That's like, uh, that's how I like to go about it. So the field of artificial life, I really think is like a very long-term bet on artificial, on intelligence. But there's two caveats. One, they're still interesting to study because they give you lots of insights about our universe and emergence and all these other scientifically, um, the scientific things that we still care about, even if we don't create intelligence. But second of all, um, I think Ken talks about this a lot, is that we don't really need to simulate every possible particle at a lower level and get everything to bootstrap. What we really want to do is create an abstraction of natural evolution and an abstraction of these simulations

• 42:13Speaker 02

  and may, and find an abstraction which is one, ultra-efficient that we can run in 10 years from now. And, uh, two, it gives this abstraction the right way where you get what you want, which is intelligence, general intelligence. And that is an open problem and that's one of my research agendas, is finding the principles that we extract from natural evolution, artificial life in order to create better AI systems. Um, we wanted to see what would happen if you used LLMs to evolve, uh, basically small little assembly programs that compete against each other. So I can give you some background. So there's this, uh, programming game invented in like the 1980s that people used to play. It's called Core War. And basically the idea is, it's, there's like this assembly-like language called red code.

• 42:56Speaker 02

  And how it goes down is I create an assembly program, you create an assembly program, and everyone creates an assembly program. We put them into a virtual machine and we put them in the memory address of this virtual machine in a random location. And we just let the machine go. It's just going to run the programs and they can do whatever they want. It's like a Turing complete language. And the goal is that my program will be the last one to run. So what I need, what my program needs to do in order to do that is it needs to basically inject an invalid operation in front of your program's execution thread. And when it does that, it's going to halt your program by crashing. Your program will crash. And, uh, then, then my program will be the last one running and I will win this game.

• 43:39Speaker 02

  And so in order to do this, you need to basically, your program needs to defend itself against other people attacking it and attack other people. So it's like this really cool game that you can, um, just run. And what we did was we saw that people have been making these war, human warriors for such a long time. We asked, what would happen? Is, uh, if we launched, uh, LLMs to evolve these programs and have them compete against each other? And can we create like this open-ended evolutionary arms race of programs from this?

• 44:09Speaker 03

  That's amazing. So it's like Game of Thrones Turing Edition.

• 44:12Speaker 02

  Yeah, yeah, exactly. It's a good way to put it. Yeah.

• 44:15Speaker 03

  Yeah. Well, well, tell me more. So, um, how, how many generations? How, what was the population size? How did you set this thing up?

• 44:22Speaker 02

  Yeah. So the actual algorithm is really, really simple. So what we do is we start off with a program. This can be a human program. Or you can be like just a LLM-generated program. And then you basically just optimize a program with an LLM as the mutation operator in an evolutionary loop to beat this program. So you just try to do this and we actually ended up using map elites because the space that you're optimizing is very deceptive. It's very non-local. If you just do a vanilla gradient descent, and not gradient descent, uh, greedy, um, genetic algorithm, it won't work. You need to use map elites. But then we end up with a warrior, these are programs are called warriors in Core War. You end up with a warrior that beats your initial warrior. Then what you can do is optimize a third warrior to beat both of these two.

• 45:07Speaker 02

  And then you can optimize a fourth warrior to beat these three. And you just keep on doing this and you end up with a long sequence of warriors, right? And then the question is like, where does this, like, adversarial, like, rounds of evolution, where does it take you? And one of the cool results, I guess we have two main results is, the first one is in this first round of training, we can already beat like 96% of a data set of human warriors. There's like a data set of like 300 human warriors. We can beat 96% of them just by using an LLM to evolve them. And, uh, importantly, the LLM is not good at red code. If you just try to zero-shot this language, it's terrible. If you do best of ends, best of end sampling, it's terrible.

• 45:50Speaker 02

  But if you do this evolution and you have this verifier, it turns out to be really good. This is the same lesson from Alpha Evolve, right? These LLMs are not good at zero-shotting certain kinds of proofs and stuff, but if you let them evolve with a verifier in the loop, they can be really, really powerful. That speaks to the power of evolution and the priors from the model. But the second result that we have that I'm the most excited about, it turns out if you do this rounds of evolution, right? Then these warriors that you end up with, uh, more, uh, further down the line, they become more and more general, as in they're more likely to beat human warriors that they haven't seen before. So I think that's a pretty cool result, but even more than that, the cooler part is,

• 46:32Speaker 02

  is if we, we have this vector describing the behavior of these warriors, just like their behavioral characteristics. And we can get into details of how we, uh, measure the behavior, but we have this vector. The variance in the behavior of these warriors goes down as you increase the number of rounds. So that kind of means that they're getting more general and they're trending towards like, kind of like reducing variance. So they're kind of behaviorally becoming like a single generalist warrior. At least that's what the trend is towards. Exactly. And I think, uh, people in AI have noticed tend to think of evolution in general as this very dumb naive algorithm that you use when you can't do gradient descent, right? And they basically call it just random search.

• 47:14Speaker 02

  And I would just want to clarify, evolution is anything but random. It's anything but random and evolution is probably the most powerful algorithm that we know to solve like discrete combinatorial problems where you need to basically get pieces of the puzzle right before you get the whole thing right. And the selection mechanism in evolution is really, really underrated because if you solve like even one part of a puzzle and you hang on to that and you search for the other ones, you can turn your exponential search problem into a linear search problem. So in that way, evolution is a really, really powerful algorithm, but all it needs is your mutations to be good, like 1% of the time, right? If you just do this with like naively in like program space and just evolve, I think there was a cool paper from

• 47:59Speaker 02

  like the before LLM era of AI, like I think Auto ML Zero, where they did program search, um, but with random mutations. And that's like really hard to get working properly because there's like literally around a 0% chance that any mutation will be, give you some useful code. But LLMs, sure, they can't give you 100% success, but they can give you way more than 1%. So then you can basically, whenever you are successful, you can hang on to that achievement and build on top of it. And that's what gives this combination of evolution plus LLMs a lot, like a significant more amount of power than if you just do LLMs by themselves. It's not even close.

• 48:36Speaker 03

  Akarsh Kumar, it's been an absolute honor having you on the show.

• 48:39Speaker 02

  Awesome. Thanks for having me.

• 48:40Speaker 03

  Yeah, it's been amazing. This was a great conversation.

• 48:42Speaker 02

  Yeah.

• 48:53Speaker 00

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com, not authorized by any candidate or candidate's committee.

• 49:23Speaker 01

  Your heart can tell you a lot about your health. Apple Watch Series 12 measures your heart rate every five seconds with the most accurate heart rate sensing in wearable. So your vitals app now with heart rate variability can tell you when something is off. And your readiness score can let you know when to rest and when to push. Hear the story in every heartbeat with Apple Watch Series 12. The features described are for wellness purposes only and not for medical use. iPhone 11 or later required. Based on Apple conducted study of heart rate accuracy, August 2026. Visit apple.com slash apple watch series 12.