#330 Sebastian Risi: Why AI Should Be Grown, Not Trained

AI has been trained like software. But what if it should be grown like life? In this episode of Eye on AI, Craig Smith sits down with Sebastian Risi, professor and leading researcher in neuroevolution and artificial life, to explore a fundamentally different approach to building intelligence, one inspired by how nature evolves, grows, and adapts. Sebastian explains why traditional AI systems are limited by fixed architectures and one-time training, and how evolutionary algorithms can create systems that continuously learn, self-organize, and even grow their own neural structures over time. They dive into concepts like plastic neural networks that keep updating during their lifetime, AI systems that can recover from damage, and models that develop from a single "cell" into complex structures, similar to biological organisms. The conversation also explores how combining large language models with evolutionary search could unlock more creative and open-ended problem solving, from merging specialized models to building AI systems capable of generating and testing scientific ideas. If you want to understand where AI is headed beyond today's transformer models, and why the future may look more like living systems than software, this episode offers a clear and thought-provoking perspective. Subscribe for more conversations with the people building the future of AI and emerging technology. Stay Updated: Craig Smith on X: https://x.com/craigss Eye on A. I. on X: https://x.com/EyeOn_AI (00:00) Why copy nature's evolution for AI (01:20) What neuroevolution actually means (05:52) How evolutionary search replaces gradients (08:03) Plastic neural networks and continuous learning (11:53) Growing neural networks like living systems (18:08) Scaling challenges and limits of growth (23:16) Can evolving systems replace LLM training (27:28) Continual learning and model merging (30:27) Artificial life, self-repair, and resilience (35:10) AI scientists and evolution with LLMs

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  We can take an example of how nature evolved intelligence and use evolution instead. When you use a static fixed network that is not changing the weights during its lifetime, if you cut off a leg, it will probably fail because it can't adapt. But these heavier networks, they change the weights all the time. It's basically like a continually learning, updating system where you can cut off a leg and oftentimes it will still be able to function even though it has never seen this kind of variation during training.

• 0:27Craig S. Smith?

  Let me jump in with a little explanation before we get started. This is a very technical podcast, but one of the more interesting ones that I've recorded in a while. And I want as many people as possible to benefit from it. I'll begin by explaining in simple terms what gradient descent is, which is used in most neural networks today, as opposed to neuroevolution, which is what this podcast is about. An illustration of gradient descent is standing blindfolded on a mountainside with the goal of finding the lowest point in the landscape. That lowest point is the solution.

• 1:12Craig S. Smith?

  Your distance from it is the error or loss. To get to the solution, you try to reduce that error step by step. So you feel around with your foot to find which direction the ground slopes downward. You then take a step in that direction. You repeat that process until every other direction feels uphill. At that point, you've reached a low point called a minima, though not necessarily the lowest point in the whole mountain range, which would be called the global minima. With neuroevolution, imagine a plane flies over the whole mountain range

• 1:57Craig S. Smith?

  and drops many people with different search strategies in many different places. One wanders to his left, another to his right. One walks in widening circles, another takes big jumps, another small ones.

• 2:14Craig S. Smith?

  After a while, you see who ended up at the lowest point. You keep the best strategies, make variations of them, combine some of them, maybe the jumping and the walking in a widening circle. And then you send out a new group of people with those strategies. Over time, your people get better and better at finding the lowest point, even though none of them ever knew which way was downhill. That is the difference. You have a better chance of finding the global minima, the lowest point in the entire mountain range. With gradient descent, you improve by following the slope.

• 3:00Craig S. Smith?

  With neuroevolution, you improve by variation and selection. You try many candidates, score the results, keep the better ones, and make new variants from them. No one has to know which way is downhill.

• 3:18Craig S. Smith?

  To begin, you have this new book out, NeuroEvolution. So maybe you can start by explaining what neuroevolution is in AI.

• 3:28Speaker 00

  Sure. So neuroevolution is the idea of combining evolutionary algorithms with evolution. So the idea is instead of training networks with gradient descent or reinforcement learning, we can take an example of how nature evolved intelligence and use evolution instead. And so you're applying genetic algorithms, evolutionary strategy, many different flavors of evolutionary algorithms to optimize some part of a neural network. And that can be, the nice thing is it doesn't only have to be the weights, like in standard backpropagation, but it can be the architecture. It can be some learning rule, it can be hyperparameters, it can be many different parts, and it doesn't have to be differentiable.

• 4:13Speaker 00

  So it's quite versatile how you can apply it.

• 4:16Craig S. Smith?

  But what do you mean it doesn't have to be differentiable?

• 4:18Speaker 00

  So, so when you typically train a neural network with like supervised learning, you need to be able to differentiate through the networks. When you use an algorithm like backpropagation, what basically most machine learning is built on, be able to differentiate through the network to know how much, based on the loss, like how well it performs, how much you change each weight in the network. And so the network has the certain properties like this differentiability for it to be able to apply this algorithm. And so if you don't have that, then, then it's a little bit more complicated to apply something like backpropagation to it. Evolution basically still works.

• 4:56Speaker 00

  So you have to have a specific like activation functions that are differentiable. The architecture has to be differentiable. Everything has to be kind of smooth instead of being discrete. And if you want to do something like discrete, discrete actions, things like this, then you have to, you know, use some tricks. Evolution doesn't really care if the, if anything about it is differentiable or not.

• 5:16Craig S. Smith?

  For gradient descent to work well, the landscape has to be smooth enough that wherever you are, you can feel a clear downhill direction. That is what differentiability means. If the terrain is too jagged, covered with boulders and hillocks, the tiny patch under your feet does not reliably tell you which way leads downhill overall. In that case, following the slope is much harder, and methods like neuroevolution may work better.

• 5:53Speaker 00

  If you have a function, you can, you can use, you can take the derivative of that function. And that's basically what you're doing when you train a neural network. You view the whole neural network as a, as a big function. And if you take the derivative of it, like from math, like high school math, then it tells you the slope. It tells you which direction do I have to push the weights for the arrows to get lower? So that's all it does to you. Like it gives you the slope of the function, and it then means that should I take, should I take this weight in this direction? If I take it in this direction, the error increases. And in this direction, the error goes down. So you have basically like a, let's say you have a three dimensional net, a network with three weights.

• 6:34Speaker 00

  What you get is depending on how you vary those weights, you get an error surface. And that tells you, and then if you get the slope, it tells you wh- which way you should go down, right? And if you have a million parameter network, it's a million space. And back from a case is very good. If you can do that, then it's, then it's great to, you know, finding that point, the, the minimum or maximum. But if you can't do it, then it's very difficult to navigate that space. And that's what you can use like evolution for. So, so yeah, basically the, the differences in, if you use evolution is that you don't need to grade it because yeah, you have a whole, you have a population that is basically distributed on this, on this landscape, right?

• 7:15Speaker 00

  And you don't need to have the, this error signal. You can just basically, you kind of sample, like evolution strategy, for example. You have like, you are somewhere on that, that, that surface. And then you, what you do is you sample, you slightly change the weights, right? Create like a hundred different mutations that are like around you. And then you go into the direction of, oh, in this direction. So it's a more, you locally sample. Um, uh, but you have a population, so you're not only sampling here, but in, in many places at the same time. And that can give you a direction to go. And then the nice thing is also that if you, that you're not only need to do mutations, like slightly going in one direction, right? But you can also do big chunks by doing crossover.

• 8:01Speaker 00

  So like, you know, the idea crossover is you take the genes of one parent and the genes of another parent. And, and by combining them, maybe you get the best of both worlds because each one has good building blocks. And by combining it together, you get something even better. But if you want to do that with uh, evolution, you need to take care, you can't just randomly take half of a network and half of another network and assume that it's working well. So that neuro evolution. Um, researchers have developing algorithms that allow like a sensible crossover that you, you know, you don't want to have like suddenly like two left hands, uh, you want to have a left and a right hand. And the same applies to kind of neural networks.

• 8:38Craig S. Smith?

  And then uh, it can combine them uh, in a way that's still executable.

• 8:43Speaker 00

  Right, exactly. That, that makes sense where you don't kind of lose functionality.

• 8:47Craig S. Smith?

  So that's neuro evolution.

• 8:49Speaker 00

  Right.

• 8:50Craig S. Smith?

  As it applies uh, to, to AI and uh.

• 8:55Craig S. Smith?

  You've been working on, and this is an area that fascinates me, plasticity of networks, growing, self-growing networks, and to some extent, self-recurrent, recursive improvement. Are you working on that at all?

• 9:13Speaker 00

  Yeah, yeah, so, yeah, exactly. Like, we have, so we're basically trying to see what are potentially building blocks from nature that we don't have in our current system that might make them hopefully a lot better. And one of those things is this plasticity that you mentioned. So how we learn is to, if, one of the mechanisms that our brains learn is like if two neurons always fire together, then the connection between them gets stronger. So it's like, it's a local learning rule instead of this backpropagation that, that is like this outside thing that changes everything about the network. And so what we have been working on is what if we only train those learning rules for each synapse, we train this local learning rule instead of having a global signal, and we train it through evolution.

• 9:58Speaker 00

  But then we can put the, so we did experiments where we trained those, these Hebbian learning rules. And, and, and they take into account like how much does this, the pre-synaptic neuron fire, like the source neuron, and how much does the postsynaptic neuron fire? And then depending on how much they fire together, we have a learning rule that says, oh, if this fires often, or this one, or them together, then maybe make it stronger, make it weaker. And for every, every connection in the network, we evolve its own rule. And then we showed that if you do that, starting from when the agent is born, we can start from a completely random network. The only thing, it has the learning rules, but otherwise the weights are completely random. And in a few steps, the network can self-organize because it's trained by evolution,

• 10:42Speaker 00

  the learning rules are trained by evolution to self-organize into a network that can, for example, control a car driving around or controlling a quadruped or robot. And the interesting part is that this quadruped, when you use a static fixed network that is not changing the weights during its lifetime, if you cut off a leg, it will probably fail because it can't adapt. Happy networks, they change the weights all the time. There is, it's basically like a continually learning updating system where you can cut off a leg. Um, and oftentimes it will still be able to function, even though it has never seen this kind of variation during training. Um, and so now we're trying to, you know, extending those to also more complicated tasks, more

• 11:27Speaker 00

  more like continually learning tasks. But the, the main idea is that the weights never stop changing, like our, you know, your brain is not frozen at some point, but it keeps, keeps changing. Um, and, um, yeah, so I think this is a very, like a promising direction towards like continually learning agents that are based on their own evolved learning rule that could be, for example, optimized to facilitate continual learning through some kind of meta learning.

• 11:53Craig S. Smith?

  And, and the problem, of course, with continual learning on a fixed network is you overwrite

• 11:59Speaker 00

  Hmm.

• 11:59Craig S. Smith?

  weights and you forget

• 12:01Speaker 00

  Right.

• 12:01Craig S. Smith?

  information.

• 12:02Speaker 00

  Right.

• 12:03Craig S. Smith?

  Um, how do you, how do you deal with that?

• 12:06Speaker 00

  Yeah, so that can still happen with those networks. So, so, so one thing that, that, that, um, people have been experimenting with is so in the brain, we have this happy and learning, but we also have this thing neuromodulation. So, so neuromodulation is like another type of, um, you know, system in the brain that tells some, uh, parts of the brain when they should learn. And like, that's one of the things. It does many other things, but one, one functionality is that it tells parts, when should they switch learning on and when should they switch it off? Um, so us and others have been experimenting with adding another type of neuron to a neural network that can then tell other parts of the network, when should learning be switched on and off? And so that's one way of, um, towards more continually learning system that the system itself learns,

• 12:50Speaker 00

  okay, I should override maybe this part, this, this weights are fine. The other parts maybe shouldn't be changed. And yeah, the, the, so the other thing we have been, um, working on that as this part of this, this E EU project Grow AI, uh, is that we are also trying to learn not just in a, in a fixed network, um, but also learning actually to grow. Network like more taken inspiration from neurogenesis and morphogenesis in nature that we're not given this brain like this brain has been has been growing. Uh. And so that's one thing that we in machine learning skip, we just, this is the the neural network you have you have. But the nature of things are grown from like starting from a single cell.

• 13:33Speaker 00

  So we're trying to replicate that process, trying, starting with one neuron and growing. Uh. And then the hope, the ideas, so we have a system that can do that. And currently it works for for simple task. Um. But ultimately, the idea is to also take into account the environment during the growth process like. Um. To to take advantage, there is already some information in environment. So why not take it into account when you, when the network is is created and and develops? Um. And so that's something we're working on. And also, we often have doing a combination of, and we call this this a neural developmental program. So it's basically like you're learning another small neural network that is a copy of those runs in every neuron of a normal neural network.

• 14:17Speaker 00

  And then that small network can then decide when should another node be created? Or how should the connection between two nodes change based on the activation? So it can learn in principle, any type of learning rule. Um. Which just makes it also harder to to optimize. But it's basically like a, it's like a graph neural network type system, but that this dynamic that can change while the agent is born and interacting with its environment.

• 14:43Craig S. Smith?

  You're growing parameters, right?

• 14:46Speaker 00

  Yeah.

• 14:46Craig S. Smith?

  In effect.

• 14:47Speaker 00

  Yeah, we we're growing, we're growing parameters, but not trainable parameters. So so the trainable parameters is like the the DNA, the the the small program that. And then the the the big network then has also parameters, but those are basically like those it has to learn to change by itself.

• 15:05Craig S. Smith?

  Yeah.

• 15:06Speaker 00

  Our like DNA and the process, but then your brain then is not relying on evolution anymore. But on whatever learning mechanism is running inside of of that system.

• 15:16Craig S. Smith?

  Uh. You have, say, a three-layer network with four nodes in each layer.

• 15:23Speaker 00

  Right.

• 15:24Craig S. Smith?

  So. In the hidden layer, the middle layer.

• 15:28Speaker 00

  Yeah.

• 15:28Craig S. Smith?

  Uh, one of the neurons gets enough information or more information that it can handle and then splits off a new neuron.

• 15:44Speaker 00

  Yeah, e- except that it's, it would be up to the, so, so there are two things, like, one is, it could just grow without getting any information at the start, like, you know, like how be- before the, our cells might get sensory information from the outside, you could just have a fixed program where the nodes communicate with each other and they exchange information and they figure out, okay, you should grow five times and I grow two times. And, and so this is like, without any outside activation, that's one, that's the first, like, process that can run. But then it could be that, yeah, then it could be that they figure out, okay, I have too much, um, so, so the system itself learns to do this. So, so we're not telling the system if you

• 16:27Speaker 00

  have too much information, but the each, because in each node you have a, another like recurrent network basically running, like this genetic program, uh, developmental program, and that could figure out, okay, I've getting all this information, so, so maybe I should split the cells to that, um, you know, you have more capacity. But that is something that we don't program in, but that the algorithm would have to figure that out by itself. And then how it would figure it out is that genetic programs that do that would get a higher fitness than genetic programs that don't do that. And, and so the other ones would selected, be selected out. And, and some that do this a little bit would get initially better fitness and then they would be selected and, and, yeah.

• 17:08Speaker 00

  And so that's one thing that is, that is the little bit of challenge, because the space of what you could learn is so large, right? You could learn any very weird developmental program, so, um, and, and that's we, so it probably would require to evolve very complicated things, have a, a good curriculum of tasks, like first, you know, you have to do some small, um, some small task and, and then you make the task more and more complicated, like kind of. There's also some research that we talk about in our, in the Neural Revolution book, like one system called Poet, where you're evolving the environment and the agent together so that both things get, can like, can scaffold off of each other.

• 17:48Speaker 00

  And so something like this will, is probably required to get that approach to work for really like complicated problems.

• 17:54Craig S. Smith?

  And just on a very concrete level, you're dealing with computer code, right? So does the code, is there a function in the code then that replicates a neuron? And then adjusts that new neuron's weights?

• 18:12Speaker 00

  Yeah, there's basically like the, the small developmental program has an, is a neural network that has an output that says like, create another node. And, and then when it says, when it says, that is over some threshold, we just add another node to that, the parent node. And then there's another output in the network. If you give it as input, the, the state of two nodes, then it tells you how much you should change the connection between them.

• 18:36Craig S. Smith?

  I see.

• 18:37Speaker 00

  Yeah. So it's an extra output to the net, to the developmental program network.

• 18:41Craig S. Smith?

  Yeah. And, and so far, how many nodes have you grown? I mean,

• 18:48Speaker 00

  Right.

• 18:48Craig S. Smith?

  The, the network has grown from what to what?

• 18:51Speaker 00

  Yeah. So, so the networks are grown from like one single node to like, I think the biggest we had is like, we, we tried it on the, on the small, like we tried it on some robotic task. But I think the biggest one was like a small version of MNIST and maybe like a few thousand nodes, something like this. So it's, it's orders of magnitude smaller than like current. But the.

• 19:13Craig S. Smith?

  But it grows by order of magnitude.

• 19:15Speaker 00

  Yeah, yeah, yeah.

• 19:17Craig S. Smith?

  And if you could scale this up, I mean, first of all, are you guys confident enough? Are you at the point in this research that it's? Time to scale it, or are you still working on the base algorithms?

• 19:33Speaker 00

  I think that, I think it's, it could be almost time. I think there's, there's some challenges with, you know, balancing growth and plasticity. Like sometimes it's, what it sometimes likes to do is it grows as much kind of as it almost can, and then uses this whole network and just changes the connection between nodes. So, so, so there has to be some kind of pressure towards being like maybe sparse or not using too much. Because if you don't give it any pressure, like in, in nature, you have like this energy consumption. Like, so, so you can't just grow and, and, and use all that energy. Um, so that's a little bit harder to find the balance, like, and, and, but this,

• 20:16Speaker 00

  in, in neurorevolution, you often also have multi-objective optimization. That means you can say, okay, one thing is important is the fitness. The other thing that's important is, for example, how, how much nodes there are, like some pressure on the, on the, how, how many, how big the structure is. And that helps, but there's still a few things we need to figure out. Like, one thing that's also, um, it's, it's still a little tricky. You wanna learn to grow something and you wanna be able to elaborate on that. And, and nature is very good at it. Like, it learned how to make a butterfly and then it learned how to make a butterfly with like eye spots and different eye spots and, and, and, um, but in, in these type of representations, they're also called like indirect encodings because you have this indirect way of making a network.

• 21:01Speaker 00

  Um, one thing that's difficult is that you, you wanted to grow a certain network and then you want it, that it learns more and adds like another structure to the network. But it should do that without forgetting how to grow the first part. So there is like a kind of continual learning problem in developmental system and we're still trying to figure out how to best the.

• 21:22Craig S. Smith?

  If you can get this to work, um.

• 21:26Craig S. Smith?

  Is there, what are the scaling laws that apply? I mean, can you, could you then grow this indefinitely? Or does something start happening beyond a certain point in the size of the network?

• 21:38Speaker 00

  I think if we really, if we figure out how to grow it, I think we can, we can really, we should be able to really scale it out. And there's actually some interesting work that was presented by NERUBS, where they had a network. They showed reinforcement learning. If you scale up to, normally you have quite small networks for reinforcement compared to like language models. But they showed if you have a network, if you really scale it up, like having hundreds of layers, you actually get much more interesting dynamics out of the system by itself. If you train this in a supervised way, they did this like in some robotics task. And, and that is just scaling up like a typical feed forward network. So, so imagine like, there might be like really interesting dynamics hidden if you scale it up, but let not the structure, like be the structure grown and determined by this.

• 22:22Speaker 00

  And where it's not like a typical feed forward architecture. But I would like to grow something that is more like, I don't know, maybe it discovers how to grow a cortical column. And then it should be able to replicate that cortical column many times or some other structure that is important in, you know, in biological systems. And then I think then we could get really interesting dynamics out of the system. But yeah, the, the difficulty is having a system that can learn to grow important neural motives and able to, to copy them and also maybe slightly create variations of it. Because maybe it wants to use this kind of map, cognitive map for something,

• 23:05Speaker 00

  and then slightly changes to use it for another modality. So, but I think if we figure out how to make this really efficient, then there's a lot of things, interesting things we can, we can do.

• 23:14Craig S. Smith?

  Yeah. And, and once, I mean, growing is one problem, right?

• 23:19Speaker 00

  Yeah.

• 23:22Craig S. Smith?

  Training is, is another.

• 23:24Speaker 00

  Right.

• 23:24Craig S. Smith?

  And then inference.

• 23:25Speaker 00

  Right.

• 23:26Craig S. Smith?

  Is another.

• 23:26Speaker 00

  Right.

• 23:27Craig S. Smith?

  When you grow this network.

• 23:30Speaker 00

  Yeah.

• 23:30Craig S. Smith?

  Then you're starting with a, a trained network, right? A pre-trained.

• 23:38Speaker 00

  We, so what is trained is the program that grows it.

• 23:41Craig S. Smith?

  And right. But, but can you then train that network the way you would?

• 23:49Speaker 00

  A G P T model. So you can also use, you can also, we did some approach, you can also train it in a, in a more supervisory or through reinforcement learning. It's just, it seems to be easier to train it with, with evolution. But the, the, the, the issue is also with this approach. It kind of, there's a, in general, the machine learning and evolutionary computation, there's this kind of issue of like deception, that it's easy to get like a decent score. But if you wanna get all the way to the goal, you might have to first like decrease your, go another way that decreases your performance for it to be then become better.

• 24:30Speaker 00

  Like there's this classical example of like a maze and you can get very close to the, the goal of the maze. But to get actually to the goal, you would have to go all the way around the maze. So getting a decent score is, is okay, but if you want the, the really good score, you have to get worse first. And so likely these kind of problems require like approaches that can deal with this kind of deception. And, and, and that's why also neural evolution, people have been developing these methods of, of

• 25:01Speaker 00

  more open-ended search methods, like methods that don't just go for like one target. But, but methods that are, it's called, it's under this umbrella from like quality diversity. You wanna have an approach that explores much more of the space, but also takes quality of the solution into account. So for this kind of growing approach to really work really well, we have to combine it with these kind of quality diversity approaches, because all of these things kind of work together. Because otherwise it's really, it's quite difficult to explore the space. And we also, we did some, some work back in the day that

• 25:39Speaker 00

  just chose the, just the difficulty of learning to learn and plasticity. Like imagine you're like in a, this typical experiment people use in biology, like this T maze. So you have a maze that looks like a, like a T and the, a, a maze goes to one part of the maze to, has to learn to remember like, oh, was there a big reward here? Here or was it here and when they collected they have you put them back to the start of the maze. And if you train this, we did experiments where we used heavier learning for that. Imagine you learned to always go to the small reward. Like you you go to the small reward, then then you get put back. And then it's the high reward here. So you learn to go to the the small reward. This is like the worst thing you can do.

• 26:20Speaker 00

  It's it's worse than going always to one side of the maze because you would at least get fifty percent right. But in terms of how close is that network to actually learning? It's it's closer than the network that always that is just not reacting and always going to one stupidly going to the the right side, right? So if you use a traditional approach, this will be the worst. This will be directly be sorted out. Um. So you need to have different methods to evolving these more like cognitive skills than just saying, you know, this is the fitness. Because otherwise you will get stuck in that like go fifty percent stupidly to one arm of the maze. So everything has to kind of work together and that's kind of the challenge in this kind of in this where you you want to learn to learn. You want to learn to grow.

• 27:04Speaker 00

  You want to do everything at at the same time and that's kind of the challenge.

• 27:08Craig S. Smith?

  And what I was referring to in in learning, you know, right now, we have large models that have been trained on a tremendous amount of data. And if you do too much training, post-training, you end up overwriting.

• 27:30Speaker 00

  Right.

• 27:30Craig S. Smith?

  The idea here is that you could have a network that's trained. But as it operates during inference in the world, it it learns new things and rather than overwriting, it would grow

• 27:49Speaker 00

  Right.

• 27:50Craig S. Smith?

  new nodes and store that learning in those nodes.

• 27:54Speaker 00

  Yeah.

• 27:54Craig S. Smith?

  Is that right?

• 27:55Speaker 00

  Yeah, I think that that will be the ultimate goal to combine these like smaller scale experiments and seeing what we learn there to to allow ultimately language models to this kind of continue learning. Um. And there is some word by by Sakana that goes a little bit in this direction, which is called this evolutionary model merging where you take. You know, because we have many networks that are already trained, like there are thousands of language models that you can download online. And why not take advantage of all these networks that are already there? And so this, in this model merging approach, you take one network and some layers of that, and you take from another network layers, and then you kind of merge them together.

• 28:39Speaker 00

  And you let evolution figure out how to do it. And then you can, so Collyse and Sakana have done this, that you can then take a model that is good at Japanese. You take a model that's good at math, and you let evolution figure out how to combine them together, have a model that's good at Japanese and math. And so one thing that the next thing could then be,

• 29:00Speaker 00

  could you have a model that's good at Japanese that you can teach incrementally how to be good at math or something else? But I think we're not, we're not there yet, like, but I think this is how where the field is moving towards.

• 29:15Craig S. Smith?

  Yeah, the model merging is fascinating. And so you can take, if you had the source code,

• 29:21Speaker 00

  Yeah.

• 29:22Craig S. Smith?

  G P T five

• 29:23Speaker 00

  Right.

• 29:24Craig S. Smith?

  And Brock, wherever they are now, four

• 29:27Speaker 00

  Yeah.

• 29:27Craig S. Smith?

  And, and combine them, I mean theoretically.

• 29:31Speaker 00

  Theoretically, yeah, those would be probably a little big, like, or like the, if you had the resources, you, you, you could, you could do that. The, the, the trick is like, you have to, to be able to see, is it better, the combination? You have to be able to evaluate it. And you, but you can evaluate on many of the benchmarks that you have, and then you could merge them together. But these really big models, then there could be some other challenges, like the approaches people have done so far, like slightly smaller models, also because then the smaller model, we know that model is not good at math, and we know this one is not good at Japanese. Uh, which is for the really big models, it's a little bit harder to know even what are they not good at. And, um, but yeah, the evaluation is a little more tricky there.

• 30:15Craig S. Smith?

  Yeah.

• 30:15Speaker 00

  But in principle, you could do it, yeah.

• 30:16Craig S. Smith?

  You could, you could distill smaller models

• 30:20Speaker 00

  Yeah.

• 30:20Craig S. Smith?

  from the parents and then merge those.

• 30:23Speaker 00

  Right, yeah, yeah.

• 30:23Craig S. Smith?

  Right, how, how large are the models that you merge?

• 30:26Speaker 00

  Um, they're like a few, I don't know, like, maybe like a hundred million parameters. Actually, I don't remember exactly, but yeah, not, not, not compared to the really, really most.

• 30:37Craig S. Smith?

  Yeah, yeah, yeah. And where do you see all of this going? You're also doing some really interesting work in evolving, what is it called? But life, right? And virtual life.

• 30:52Speaker 00

  Yeah, yeah, artificial life. Yeah, yeah.

• 30:54Craig S. Smith?

  What do you call it?

• 30:55Speaker 00

  Artificial life.

• 30:56Craig S. Smith?

  Artificial life, yeah.

• 30:57Speaker 00

  Right.

• 30:58Craig S. Smith?

  So is that related to this? Or is that completely separate?

• 31:03Speaker 00

  No, no, that is very related. Like, so in artificial life, it's like the idea is that life, the instance we know is like one example of life, but artificial life is like life as it could be. Like, and some, and people simulate things that are life, like properties. And, and one thing of life, like property is, is growth. So, so self-organization and, and growth and self-replication is very like essential to life. And, and those are also things we explore with these growing networks. But we also explore them with what's called this neural cellular automata. Like, it's also basically like neural networks, like copies of it. And, and they, imagine just replacing the traditional rules of, of cellular automata, like of the, the game of life, which has these fixed rules.

• 31:47Speaker 00

  Like, if you have three neighbors, you create a new cell. If you have four, you, the cell dies. And you can replace that with a neural network. So instead, you ask each cell, says, ask the neural network, what should I do? What stage should I be, become next? And we, we changed that to, so we are able to scale that to in, to 3D. Like, we have a paper where we're growing Minecraft structures with, with this. And the fun thing is like, you can have a, you grow a salamander Minecraft, you cut it in half, and then it grows two salamanders. And, and the nice thing is you can train those with supervised learning. So if you have a target, you know you want to grow a house or like a tree, then you can teach it to grow that.

• 32:26Speaker 00

  If you don't know, we also use it to train kind of soft robots, that have like squishy, squishy robots, where we don't know what is a good morphology for locomotion. But there you can use evolution that you tell it, you know, grow a structure, put it in an environment, see how well it works. If it doesn't work, then we throw it out. And through this process, we can grow like structures. That are able to locomote. Um. And then we're also able to damage those, like you can again, you can cut off parts of the structure. And if it's trained to to um recover from it, then it can regrow just only based on the local information. So it doesn't need any other information, it just needs to to sense the the the local part, like a salaman that can regrow its its tail. Yeah.

• 33:10Speaker 00

  Um. And these methods can be used to do that. And there's um this community artificial life that also is um. Few people at Sakana are also working on this kind of ideas of artificial life and and um. Yeah, it's an interesting direction that's a little bit um. Not the mainstream machine learning, but I think there's a lot of promise, like taking some of these properties from biological system. Putting them there, like one is like uh being resilient. So natural biological systems are incredibly resilient. Uh. And uh still deep learning often, you know, you find these like weird examples and it completely fails. So I think there's a lot of um promise in using these systems that can self-organize and based on local communication.

• 33:55Speaker 00

  They have an inbuilt resilience that I think we could exploit to make these deep learning systems also more robust. And and also adaptive um.

• 34:05Craig S. Smith?

  And adaptive, that's another area for evolutionary systems. That use evolutionary strategies to find solutions.

• 34:16Speaker 00

  Mm hmm.

• 34:17Craig S. Smith?

  Right? Find novel solutions to a problem. And then uh.

• 34:24Craig S. Smith?

  Systems that that can uh combine models uh. You know, based on the quality of their outputs and improve uh through generations. It seems like that would be very applicable to scientific research.

• 34:43Speaker 00

  Hmm.

• 34:44Craig S. Smith?

  Because with gradient descent, what we were talking about earlier, you're you're going toward the.

• 34:51Craig S. Smith?

  Local minima.

• 34:52Speaker 00

  Right.

• 34:52Craig S. Smith?

  Hoping it's the global.

• 34:54Speaker 00

  Right.

• 34:55Craig S. Smith?

  But what you were talking about being able to cover a much larger landscape. Can you talk about what the implications are for?

• 35:05Speaker 00

  Right.

• 35:06Craig S. Smith?

  For a scientific research?

• 35:07Speaker 00

  Yeah, that's also something we're exploring at Sakana. It's this kind of idea of like an AI scientist, for example, with AI scientists, but also this thing we call Shinka evolve, which is like kind of alpha evolve.

• 35:23Speaker 00

  And and the idea is that you can, and that's a combination of of evolution and large language models. So so large language models are good at, for example, generating code and and generating ideas. But to explore that that space, for example, it can be really useful to to use evolution. So so like in

• 35:48Speaker 00

  you you can basically, you can you can use a language model as a mutation operator. You you start with like, you know, like one kind of example is this circle packing in the space and you have like a number of circles and you wanna put them in there, like the maximum number of circles you can put into this space. And and so what you can do is you can then, you know, ask a language model to give you a new solution like and and then you and multiple solutions. And then you evaluate those solutions based on fitness, like how how what's the score that it gets packing those circles? And then you you do this again from the best ones and or from the a number of best individuals.

• 36:33Speaker 00

  And you ask again, the language model, give me variations of of the solution. And then you do it over and over again until you find a good solution that packs the most circles into this into this space. So so this is like you use evolution to navigate the space, but you're using the the language model to give you as a mutation operator. And you can do this also for, you know, like.

• 37:01Speaker 00

  Like scientific ideas, for example, you can um uh um start with one idea and and let the model generate you variations of it and then you the the only thing you need is you need to be able to somehow score it based on some fitness function so um which is a little easier if you have the circle packing it's a little harder if you have like some you know scientific um idea which is a little bit more complicated to say if this is this a good uh idea or is it a a a bad idea but um but this is a the direction I mean that a lot of um Sakana is pursuing and and other companies are pursuing where you have this kind of combination of evolution because it's creative in in what it can

• 37:43Speaker 00

  discover but you have it a little bit more grounded and uh because you have a language model that is the mutation operator um and people in evolution have for a long time done things like evolving programs with genetic programming but those were always like very um hand tailored to the kind of problem at hand but now that you can use a language model you can let it output code and you can ask it to modify the code and and navigate in this space and applying all these lessons that we have learned from neuro evolution like more open ended setups using things like quality diversity um to kind of navigate the space and and hopefully not getting stuck in too many of these local automa.

• 38:24Craig S. Smith?

  Yeah.

• 38:24Speaker 00

  Um and I think it will yeah it will change how science is is made that you you have this kind of AI scientist or or like co-scientist that you can exchange ideas um it's navigating some spaces it's giving you some you know hypothesis to the to to test this is kind of where the direction is kind of moving uh towards.

• 38:45Craig S. Smith?

  Yeah and ha- have you guys at Sakana I mean where are you in that research have you?

• 38:51Speaker 00

  Right.

• 38:52Craig S. Smith?

  Are you still in at the architecture level or have you done run experiments to see if it will output some useful in?

• 39:01Speaker 00

  Yeah so so I think it's it's going this way so there is this the AI scientist from Sakana that can and and the recent version the new version was able to actually generate um some like a paper that got accepted at at a Zoc show yeah so so so it kind of shows that there is some it it can generate some generally. Interesting things. Now's the now's the question, how far can you push this approach to generate something like truly groundbreaking? So so I think that's the kind of the holy grail of the the field like and still like in an open question like it can definitely generate some new things, but how far from the trainee distribution, how creative can it be?

• 39:46Craig S. Smith?

  Yeah.

• 39:47Speaker 00

  Uh, and do we need to um, is it just about prompting it and getting those ideas out? Or do we need to, does the need to run its own experiments and we need to fine tune it on those results? And then like iteratively making it better um, and also this idea of um, the the self improvement, so so that the model itself gets better and better and it gets better at getting better. So so so those are like ideas that uh also exploring.

• 40:18Craig S. Smith?

  Yeah, on the paper that was submitted, I think that I can clear, I mean, it got accepted.

• 40:23Speaker 00

  Right, right.

• 40:23Craig S. Smith?

  Was, as I recall, was kind of proving a negative, right? How did that start?

• 40:29Speaker 00

  Yeah, yeah, I wasn't part of that paper, but I think the main thing is that that's probably the worst it will ever be. So I think that's kind of the the idea that this was an um, with an older model, you get this paper, right? But if you would replicate it now using like um Gemini or some some other model, it would probably push further on on this. So so the so the better, the the nice thing is about the framework that you can use the same kind of framework and you can switch out the the language model that you're using. So the better the language models become, the better, you know, papers they should also be able to to write. Um, and and I think that's kind of the main thing, not necessarily that there was this what ideas to generate then,

• 41:12Speaker 00

  but showing that you can kind of automate the whole pipeline um, and it will get better and better with better models. Um, but also for me, I think the the interesting part is um, how can we use this also as a kind of like a co-scientist? Like, because at least for some time, there will be humans and the I's like working. Closely together and and I think it's very interesting. How can you make sure that it can take into account both ideas? How can you make sure that AIs and humans talk in the same language? Um, there was an interesting keynote by Melanie Mitchell. Um, and where where she showed that basically like even the models that come up with it looks

• 41:53Speaker 00

  great on the benchmarks and it looks like the right solution and it goes it was cool, but it solved it in a very different way that was not even intended by the humans. Like it exploited some kind of feature about the domain that wasn't even built in. So how do we kind of, we have to find kind of like a language that we we talk in the same way. If we want to collaborate, then we have to find a common ground to being able to to do that. And and um, I guess it's, there's already some common ground. It's natural language. It's trained on text. You can communicate with it. But you you might not be sure about intentions or so is I think to collaborate well. We need to do a lot of work that goes beyond it just being able to write its own papers. But how do we best kind of combine it with what humans are good at and what machines are good at?

• 42:37Speaker 00

  And I've always been interested in this kind of co-intelligence or hybrid intelligence. Like, how can we combine the best of of both worlds? And before it was a little more easy, like it was a little bit more clear what what are humans and computers good at? Now it's becoming a lot less clear. Uh, so I think that's something we need to kind of figure out.

• 42:55Craig S. Smith?

  Yeah, you said something interesting before, not when you said its creativity is constrained by uh the training data, like, is it really going to come up with?

• 43:07Speaker 00

  Yeah.

• 43:07Craig S. Smith?

  Ideas that aren't in some way embedded in the training data. Two questions on that. One, maybe that's the case, but the training data can be very rich in

• 43:18Speaker 00

  Right.

• 43:18Craig S. Smith?

  Scientific ideas that have never been explored.

• 43:21Speaker 00

  Yeah.

• 43:22Craig S. Smith?

  I, I mean, even evolutionary AI has

• 43:26Speaker 00

  Yeah.

• 43:26Craig S. Smith?

  Not gotten the attention until very recently that it probably should because people get attracted by other things.

• 43:33Speaker 00

  Yeah.

• 43:34Craig S. Smith?

  AI is

• 43:35Speaker 00

  Yeah.

• 43:36Craig S. Smith?

  Not like that. It'll look at everything.

• 43:38Speaker 00

  Yeah.

• 43:39Craig S. Smith?

  So aren't there insights to be discovered within?

• 43:44Craig S. Smith?

  The training data within the body of scientific researches that exist.

• 43:48Speaker 00

  And I think it's still, I think there's some results that show it can, it can generate something new. It's just the question, how far can it be pushed beyond what was in the training? So I wouldn't say that it cannot produce anything new. I just, for me, it's less clear. How far is that outside of what it has seen? And I mean, there's like mathematicians, I think Terence Tao that used it and it found a proof that humans forgot or didn't know about. And so it can certainly be helpful. It can, I mean, people have been applying it to also optimize, you know, robot morphologies. And it came up with new morphologies that didn't exist before.

• 44:31Speaker 00

  So, but the question is, yeah, how far can we, can we push it outside that?

• 44:36Craig S. Smith?

  And what you're saying is that, you know, the human knowledge contains a finite amount of all possible knowledge.

• 44:45Speaker 00

  Right.

• 44:45Craig S. Smith?

  And how do you go beyond?

• 44:47Speaker 00

  Yeah.

• 44:47Craig S. Smith?

  Current human knowledge.

• 44:48Speaker 00

  Like, let's assume like, I don't know, like 50 years ago, let's say you had a model that was just trained until this point, would have had then invented the iPhone at some point.

• 44:57Craig S. Smith?

  Yeah, I've heard this.

• 44:58Speaker 00

  Like, yeah, and so this not, I don't think the current ones would. They probably would invent other things. And then you could imagine, okay, they invent something and then they could be trained on, on those inventions. And then maybe ultimately they would get there, but.

• 45:13Craig S. Smith?

  Maybe they would come up with something.

• 45:15Speaker 00

  Something else, yeah.

• 45:16Craig S. Smith?

  Yeah, I mean, the iPhone is, is, it certainly was revolutionary, but it was, there was no groundbreaking tech in the iPhone.

• 45:26Speaker 00

  Right, right.

• 45:27Craig S. Smith?

  It was an engineering exercise. We brought together MP3 player and telephony and.

• 45:33Speaker 00

  Yeah.

• 45:34Craig S. Smith?

  So, I mean, given the right target, it might

• 45:38Speaker 00

  Yeah.

• 45:39Craig S. Smith?

  evolve an iPhone, but it might evolve something that

• 45:42Speaker 00

  Right.

• 45:42Craig S. Smith?

  is better than an iPhone.

• 45:44Speaker 00

  I think the the thing is also, it probably has to be combined with, it has to be able to run like its own experiments and stuff. I think if you just like fifty years ago, and then you just only in your head and think about what could you invent? Like, I don't know how far people would have gotten, but you have to be able to either run an experiment, or you have to be able to manufacture something and then see how does it do? Or, uh, so, so I think, and some people are are working on combining this now, like this is like Leila Science, uh, that um, they combine these ideas and they have robots and manufacturing things to then, I think ultimately try to see what the model makes and then use it to automate kind of science or like, uh.

• 46:28Craig S. Smith?

  What kind of a factory like, because there are

• 46:31Speaker 00

  Yeah.

• 46:31Craig S. Smith?

  you know, I'm thinking of, uh, Incilico, the drug discovery company.

• 46:36Speaker 00

  Yeah, yeah.

• 46:36Craig S. Smith?

  That has, is using A I for, you know, compound.

• 46:41Speaker 00

  Right, yeah.

• 46:42Craig S. Smith?

  discovery, but then they have it attached to an automated wet lab that synthesizes the molecules.

• 46:49Speaker 00

  Yeah.

• 46:50Craig S. Smith?

  Uh, but in this case, what what kind of a manufacturing?

• 46:54Speaker 00

  Yeah, I actually, I I don't know, like, um, uh, exactly how it looks like, but, um, I guess like also trying like, uh, I could imagine this like, yeah, robots that try to mix things together and and like seeing what happens. I think they're doing many, many different things, but I guess there are a few labs, like companies that that moving towards that direction, like automating also like material science. And I think that what probably has to happen, because if it's just only being able to output language, I don't know how much it will be able to do something. I think it has to be able to affect the the world or or run experiments somehow to, because that's how yeah, how we humans at least learn, like, um, uh, so, but but I think that's also moving in this in that direction.

• 47:38Craig S. Smith?

  Yeah, and, uh, so we talked about artificial life, we talked about

• 47:43Speaker 00

  Hmm.

• 47:44Craig S. Smith?

  uh, growing or plastic

• 47:47Speaker 00

  Yeah.

• 47:48Craig S. Smith?

  neural networks, uh, about.

• 47:52Craig S. Smith?

  Evolutionary strategies. What are the other areas that you're focused on?

• 47:58Speaker 00

  Yeah, a lot about like also open-endedness, like

• 48:02Craig S. Smith?

  That's right.

• 48:03Speaker 00

  How can you create a system that can? And that's very much tied to the, for an L L M's like, how can you create a system that can keep innovating, keep producing interesting new things? And, and that's why, yeah, people use more and more like language models in, in depth as well. But there is a lot of like techniques in the neuroevolution community that people have already developed that now getting kind of augmented using, using language models. Like one is, for example, like evolving both the environment and the agent at the same time.

• 48:39Craig S. Smith?

  Yeah, yeah, you mentioned that. What does that mean?

• 48:41Speaker 00

  Right, like, so one of these examples is like this algorithm called Poets, like where basically like the agent might be like a bipedal robot.

• 48:50Craig S. Smith?

  Yeah.

• 48:50Speaker 00

  And the environment is the, the terrain in this case, like, and it's easy to, if you have a flat terrain, it's easy for the robot to walk, but then you can choose like gaps or like obstacles. And so the agent has to learn to deal with those. And so in this, in this approach is initially you start with a very simple environment, but then over time you make it more and more difficult. And, and it, at the end, it's solving like crazy environments that it goes down or it has to jump over things like it. It is really impressive what it can do in the end. And the interesting part is if you would have started with the, the really complicated environment, at the end, it wouldn't have been able to solve it. So it needs, you need to go through these stages for it to

• 49:33Speaker 00

  discover these kind of stepping stones in the behavior to be able to do the final thing. And then people have also extended that to, which we also talk about in the book is this, one is this approach called

• 49:47Speaker 00

  Omni where you, you can extend this to, to things like environment generation in Unity, for example, like where you have not just this two dimensional, flat landscape, but now that we can have language models, you can have the language Model produce code that creates an environment and initially it might create an environment that's very simple and then it creates more and more complex environments um and and so that's I think really really interesting and could allow also like neuro evolution to scale up to really complicated task and I mean now you could even imagine combining this with um you know we now have also neural networks that can simulate whole three D worlds like um uh and if you combine I think neuro evolution

• 50:29Speaker 00

  with a controllable world that you can just prompt how should this world look like should it be a simple world so to be very difficult worlds more predators more this and that then I think we could really get like um um yeah in in increase in what the the the agents we can kind of evolve this way.

• 50:51Craig S. Smith?

  Yeah, uh and and Sakana's building world models, right?

• 50:56Speaker 00

  Yeah.

• 50:56Craig S. Smith?

  And are are they explicit, you know, Faye Faye Lee I had on recently.

• 51:03Speaker 00

  Right.

• 51:04Craig S. Smith?

  And her marble world model played with that, you know, it it uh it expresses the output is an explicit representation of

• 51:15Speaker 00

  Right.

• 51:16Craig S. Smith?

  A world.

• 51:19Craig S. Smith?

  I'm I'm interested in how

• 51:23Craig S. Smith?

  you you use the internal representation of a world.

• 51:30Craig S. Smith?

  For the for the model to learn uh as it interacts with the external environment so it it it understands physics it understands you you know whatever it it it learns from the external world.

• 51:47Speaker 00

  Right.

• 51:48Craig S. Smith?

  Is is which is Sakana working on?

• 51:51Speaker 00

  One of the the David Hao, our CEO, he he he made kind of one of the first papers on on world models like uh there was maybe I think two thousand seventeen when he he trained a simple word model at the time and then he could train inside of this dream you could have the agent get better and better. Uh he trained us on like um. Uh, this kind of three D doom task. And yeah, I think one of the most interesting things was that you can even back then, it the the simulation wasn't perfect. You could see that it was like a hallucinated.

• 52:26Craig S. Smith?

  Yeah.

• 52:27Speaker 00

  World, but you could already then use the agent to train inside of the world model to then be better in the real world. Um, and so that's something um that like David has explored in the past. Something where we looked a little bit into is um, when do you need to have a world model? And when can you just use a a lang- when can you just ask a language model the the answer like if you if there's something about like physics, when you need to, you know, when you need to ask and when you actually need to run a simulation. So so so those are kind of some things that we have been looking at a little bit, but um, yeah.

• 53:09Speaker 00

  Uh, like like basically what you said, like answering the question, when is a world model useful? Because you might not, I might not need to to ask a world model every little step I do, right? But if I thinking about I'm jumping over the table, maybe I should ask the world model, will I likely get to the other side or or or not? Yeah. Can you learn to when to run kind of which process?

• 53:31Craig S. Smith?

  That's interesting. And and so you're not using generated environments like that.

• 53:37Speaker 00

  Yeah, no.

• 53:38Craig S. Smith?

  Evolutionary AI has not been front and center. I mean, there's, you know, the

• 53:44Speaker 00

  Yeah.

• 53:44Craig S. Smith?

  Transformers took over and

• 53:46Speaker 00

  Yeah.

• 53:47Craig S. Smith?

  Everything's about optimizing transformer architectures or looking beyond transformer architectures. Is there a reason why evolutionary AI is swinging back into focus?

• 54:01Speaker 00

  Yeah.

• 54:01Craig S. Smith?

  Maybe it always has been, and I'm just not attending the right conferences.

• 54:05Speaker 00

  No, I think it definitely wasn't in the the the main focus. Um, but uh, I think now that I think it's just a very good pairing, for example, L L M's with evolutionary algorithms. I think that's a really good match, and that's why I think more people also now interested in combining generative AI. With evolution, because it just before, basically before an evolution or revolution, you always had to think about what does your representation look like? And then you kind of with a particular representation, you might restrict the type of solutions you're getting. And now that you can use language model, you don't have to worry so much about, you can just have that come up with the representation or be the representation.

• 54:46Speaker 00

  In that instance, it's quite, it's difficult to differentiate or use gradient descent like, it's so searching that, you know, you can use the language model to give you ideas or like generate you artifacts and then searching that space, it's beneficial using evolution because it's hard to back propagate using gradient descent to that space. So I think that's one reason why it's becoming also more popular now, like the approaches like Alpha evolve and this model merging, I think fit really well together. And then that's also something that Sakana is very much looking into, like we want to go beyond the transformer. Like one of our co-founders, Anjo C, is one of the authors on this transformer paper.

• 55:29Speaker 00

  And we think this is not where we should kind of stop and go beyond what the transformer can do. And, and so one of our approaches is this, this continuous thought machine, which is an approach where the network actually, it's not just like input outputs, like you get something in, you put something out, but the network itself can decide to think about a problem

• 55:56Speaker 00

  for, for longer time periods. So where the, the external input is actually not as important as like the internal thought process itself. And, and so we made an, an architecture that, that allows the model to kind of do that. And again, like incorporating some more.

• 56:14Speaker 00

  Biologically inspired algorithms, like an activation memory of what comes into each neuron, making each neuron more complex, so not just the simple neural models that are most times used, but slightly more complex, that each neuron is its own neural network. And also this idea of the brain, like biological brains often use this kind of synchronization and oscillations to do like processing. And they seem to play a really, I mean, I'm not a neuroscientist, but they seem to really important role in biological brains, like how neurons oscillate together or synchronize together. And so that's one of the key components in this continuous thought machine,

• 56:55Speaker 00

  that you see how much the neurons synchronize and you kind of use that as a representation.

• 57:02Craig S. Smith?

  And synchronize, you talking about the, what's the hemian, fire together?

• 57:08Speaker 00

  Actually, in this case, it's more like, yeah, basically looking how much they fire together, like how much they correlate together, so very related to the to the hemian one. And in this case, so those systems, you can also train with, this whole thing you can train with gradient descent. So it's not, you should only use evolution. I think it really depends on the type of of problem you have. Where is evolution useful? Where is gradient descent useful? There's also some other work we published where we used traditionally neural cell automata, where we used gradient descent to optimize them. So I think kind of both techniques, I think, are very complementary and useful for like different aspect. And we also talk about that in the in the book.

• 57:49Craig S. Smith?

  Yeah, the thought machine and the the data it's taking in is,

• 57:54Speaker 00

  So we applied it-

• 57:55Craig S. Smith?

  Is it a continuous stream of data or is it?

• 57:57Speaker 00

  Right, so the benefit of it is it can operate on continuous stream of data, but it can also be used for domains that don't inherently have like a sequential dimension. Like one thing we applied it to is is image classification. And what happens there is that it it kind of it gets the image and it learns to look around on the image and to based on what it sees, it slowly builds up confidence like, oh, I'm seeing like an image of a dog, I'm seeing this image. And the other domain we used it in is like reinforcement task, but also like maze navigation.

• 58:37Speaker 00

  And maze navigation is and there it's it's quite a lot better than like a standard like L S D M long short time memory. Network in kind of yeah, learning to navigate out of a maze. So these are the tasks we applied to, but in the future, we want to also apply to more complicated tasks. But in this one, it's it's really interesting. I think one of the very interesting parts for me is like that it looks very eerily human where it looks like it looks sometimes at the eyes or it looks at these parts like if you would compare it to, I think, how a human, if you track the eye tracking of how what humans look at. I think it looks, it seems pretty similar to that and and

• 59:21Speaker 00

  and then you can also, you can kind of track how confident it is that it sees a certain image and it's interesting to with some images, it's more difficult like I don't know, a dryer, it might take more time to look around to be sure. And then some other images, it looks very briefly and it knows what it is. So and then you could take that into account to give it, you know, say, okay, when you're confident enough, then you stop computing. And if not, then you just, you know, you can go on for a while. And that's one thing that I think it's starting to, I mean, in language models, people added this kind of reasoning.

• 59:59Craig S. Smith?

  Yeah.

• 1:00:00Speaker 00

  But it's often also like kind of reasoning that you elicit through reinforcement learning and you're kind of reasoning in text, like you can see the models like, oh, I think this and that and then but this is another type of reasoning that's happening in the in the neural substrate itself. That and so there are some interesting differences. And the question now is how can we take that and apply it to language models and how that would look and yeah.

• 1:00:25Craig S. Smith?

  Wow, that's fascinating because it seems like all of these threads could converge.

• 1:00:30Speaker 00

  Right. Yeah, that's the hope.

• 1:00:32Craig S. Smith?

  Yeah, yeah. Okay, Sebastian, I know you've got a plan to catch. So I'll leave it there.