Computational Life: How Self-Replicators Arise from Randomness, with Google’s Ettore Randazzo and Luca Versari

Nathan explores the emergence of computational life with Google researchers Ettore Randazzo and Luca Versari. In this episode of The Cognitive Revolution, we delve into their groundbreaking paper on self-replicating programs arising from simple interactions. Join us for a fascinating discussion on the implications for AI development, the origins of life, and the potential future of artificial intelligence. Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj RECOMMENDED PODCAST: 🎙️ This Won't Last - Eavesdrop on Keith Rabois, Kevin Ryan, Logan Bartlett, and Zach Weinberg's monthly backchannel. They unpack their hottest takes on the future of tech, business, venture, investing, and politics. Apple Podcasts: https://podcasts.apple.com/us/podcast/id1765665937 Spotify: https://open.spotify.com/show/2HwSNeVLL1MXy0RjFPyOSz YouTube: https://www.youtube.com/@ThisWontLastpodcast SPONSORS: Building an enterprise-ready SaaS app? WorkOS has got you covered with easy-to-integrate APIs for SAML, SCIM, and more. Join top startups like Vercel, Perplexity, Jasper & Webflow in powering your app with WorkOS. Enjoy a free tier for up to 1M users! Start now at https://bit.ly/WorkOS-Turpentine-Network 80,000 Hours offers free one-on-one career advising for Cognitive Revolution listeners aiming to tackle global challenges, especially in AI. They connect high-potential individuals with experts, opportunities, and personalized career plans to maximize positive impact. Apply for a free call at https://80000hours.org/cognitiverevolution to accelerate your career and contribute to solving pressing AI-related issues. The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist. CHAPTERS: (00:00:00) About the Show (00:00:22) Sponsor: WorkOS (00:01:22) About the Episode (00:04:02) Introduction and Paper Overview (00:05:38) Self-Replication and Life (00:07:59) Complexity and Information Theory (00:17:09) Sponsors: 80,000 Hours | Brave (00:19:44) Experiment Setup (00:24:27) Evolution of Self-Replicators (00:34:41) Sponsors: Omneky | Squad (00:36:07) Types of Self-Replicators (00:38:23) Symbiosis and Parasitic Behaviors (00:46:17) Implications for Life in the Universe (01:05:47) Intelligence and Complexity in AI (01:21:03) Concluding Thoughts

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Nathan Labenz?

  Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together, we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. Hello and welcome back to The Cognitive Revolution. Today, my guests are Atori Randazzo and Luca Versari from Google's Paradigms of Intelligence team. Atori and Luca are co-authors on a fascinating recent paper called "Computational Life: How Well-Formed Self-Replicating Programs Emerge From Simple Interaction."

• 0:43Nathan Labenz?

  This paper explores how self-replicating programs can spontaneously emerge in an extremely simple computational environment. And while the paper itself is focused on a narrow set of technical experiments, I found it to be deeply thought-provoking about the nature of life and the potential future of artificial intelligence. By demonstrating how complex lifelike behaviors can arise from very simple setups and basic rules, it helps develop our intuitions for the origins of life on Earth and the possibility of artificial life emerging in our increasingly complex digital ecosystems. In this conversation, we go deep into the esoteric details of their experimental setup, which uses a minimalist programming language to process random interactions between random

• 1:27Nathan Labenz?

  short strings of code, and yet surprisingly quickly, gives rise to complex phenomena, including multiple classes of self-replicators, competition, evolution, and even extinction, all without any selection mechanism or optimization function whatsoever. It really is profound to behold just how much complexity emerges from randomness in such a tiny, digital toy universe. Toward the end, we zoom out to consider the implications of this work for AI development and safety. While Atori and Luca are cautious about drawing too many conclusions, I can't help but understand this work as yet another reason to expect the unexpected. If such simple digital life can form from pure randomness, I have to expect that the

• 2:10Nathan Labenz?

  optimization power of the economy as a whole will give rise to meaningful forms of self-replicating AI systems. And yet, even with determined effort, I find it extremely hard to imagine what those new forms might be like. This episode exemplifies why I love making this show. We start with a seemingly niche technical paper and end up grappling with some of the biggest questions facing humanity and all the other life forms with which we share planet Earth as we collectively hurdle toward an AI-altered future. As always, I appreciate it when listeners share the show online. Feel free to tag me and of course, we welcome your feedback via our website, cognitiverevolution.ai, or you can DM me anywhere you like.

• 2:52Nathan Labenz?

  Now, I hope you enjoy this horizon-expanding conversation with Vittori Randazzo and Luca Versari from Google's Paradigms of Intelligence team on the emergence of computational life. Vittori Randazzo and Luca Versari from Google's Paradigms of Intelligence team. Welcome to the Cognitive Revolution.

• 3:10Speaker 01

  Glad to be here.

• 3:12Nathan Labenz?

  Nice to be here. Yeah, I'm really excited about this conversation. It's prompted by a recent paper that you guys put out that I have found really super interesting and quite thought-provoking. It's called "Computational Life: How Well-Formed Self-Replicating Programs Emerge From Simple Interaction." And that's a pretty good summary of everything that we're going to be unpacking over the next hour and change. I would love to start with some real kind of basic stuff and we'll get into a lot more detail. I'm coming to this from a very AI-obsessed perspective. I assume you guys think a lot about how your work connects to the future of potentially

• 3:55Nathan Labenz?

  many AIs and potentially AIs becoming a new sort of life form as well. I'm always trying to figure out where is this going in the big picture? Is it crazy to think that AIs could be at some point properly viewed as a new life form? Or is that in fact realistic? And what would, what criteria would it need to satisfy? So I thought this paper was really enlightening and again, really thought-provoking in a number of different ways, but one... Kind of simple early argument that you make in the paper is that the capacity for self-replication is really fundamental to the notion of life. So I'd love to just give you a chance, first of all, to start with that high-level observation and unpack that argument for us.

• 4:40Speaker 02

  Okay, so from my point of view, of course, getting into the business of defining what life is, is pretty complicated. But one thing that I think people can agree on is that life is like something that has some inherent order, let's say. It's not like a random collection of molecules or whatever, it's probably not life. So from that point of view, one can think of things like the second law of thermodynamics that would say that it's hard to have order effectively. And in that way, then basically a self-replicating system is one of the simplest systems in which you have this force that acts against the second law of thermodynamics.

• 5:23Speaker 02

  Because copying oneself introduces some structure, some shape, some coherence into what otherwise would be just completely sorted. So that's why I personally think that self-replication is one of the first steps towards life.

• 5:41Speaker 01

  So in the field of origin of life, and also biology and artificial life, I think self-replication appears always at the very beginning because it looks like it's the best way that nature at the very least has figured out to preserve and accumulate information over a long time. You can essentially see that we try to explain in the paper as well and show that it happens there. Some things can happen even before there are self-replicators, but until there's not a self-replicator, it's just a dynamical system. And when there's the first self-replicator arising, then you start accumulating information that gets repeated over time. And then you can think about biology and nature as an evolution, essentially as a

• 6:25Speaker 01

  fight against many different self-replicators that try to compete for limited resources. And what you observe in the current world is essentially the most efficient self-replicators for any given niche that you observe in the planet Earth.

• 6:38Speaker 02

  Maybe I am overextending my claims here, but one could even claim that intelligence evolved as a way to self-replicate better.

• 6:49Speaker 01

  It doesn't mean it's the only way that you can get to intelligence, but at the very least, we know that nature did it this way. So it would be, would say like unwise to not explore this direction.

• 6:59Nathan Labenz?

  Cool. So I also came away from the paper with a, maybe one other sense of why it matters, which is that it seems that prior to the appearance of self replicators, the systems as a whole are in some ways just noisy. And the most kind of interesting, complex dynamics only seem to emerge in the presence of multiple self replicators. Maybe I'm over reading that aspect of the paper, but that seemed to be pretty important. And it almost seemed like there was a, almost like a value judgment implicit in that. And I don't know how you guys feel about that,

• 7:40Nathan Labenz?

  but it seemed like there is something that we value this sort of higher level of complexity on some level as opposed to the more noisy reality before self replicators.

• 7:55Speaker 01

  I think that one way that you can think about what happens before is that it's almost like noise essentially. Like there are things that happen, there are complex behaviors that are going on, but if they cannot be preserved and continued, they are just transient. And when you have some way to preserve information and also to, let's say, reproduce, so to have more of you that do the same thing, then you start observing it with more and more frequency. And then, of course, there's the second part of it, which is once you've got separation and once you are competing for space, evolution kicks in and then you can really start to have more complexification of behaviors because competition and collaboration really helps with that.

• 8:35Nathan Labenz?

  Yeah, gotcha. Okay. So the study that we're going to go deep on here is. If I were to describe the motivation to someone, I would say these guys set out to find the very smallest, simplest environment that they could possibly come up with that could still give rise to self-replicators. Is that a fair characterization? Is that the actual motivation that you had coming into this?

• 9:03Speaker 02

  So on one hand, for sure, simplicity is a motivating factor. In fact, what I consider one of the selling points of this paper is that we have an environment that doesn't explicitly favor the existence of self-replicators. It's just self-replicators appear, but we're not looking for that. So it's really hard to talk about what's the simplicity of an environment is. There are many ways to define that. What I think is the, what actually got us into this was when one of our co-authors did the first experiment. And so this phenomenon

• 9:44Speaker 02

  of just things suddenly getting into, very suddenly without asking for it, without anything. And we found that fascinating. And that's what I think is the most important part where, you know, how there are systems in which we don't ask for a replicator. It just, there is some spontaneous evolution and that creates structures that are complex enough to copy themselves. And that's, that's the part that is interesting.

• 10:14Speaker 01

  One reason that we don't want to focus too much on the simplicity, although simplicity really matters, of course, because you don't want to study two complex things. But you can think about a simple system where you've got a copy operation and then everything can be thought as a self-replicator. But that's interesting. At least it's not interesting to think about how did we get from nothing to self-replicators. I think the most important point on what we are trying to show here is how we can pass from a period that, which has, let's say, a random soup of programs that do nothing and are random initialized, that they have a transition that goes into a period where there's self-replicators. How this happens and why is part of what we are trying to explore here.

• 10:55Speaker 02

  Yes, and this work doesn't go into the why yet. We are doing follow up work that tries to figure out exactly what is going on. But for now, we just thought it was so interesting that we couldn't not show it to the world that we have this kind of phenomenon.

• 11:12Nathan Labenz?

  I think this was quite an interesting, almost philosophical note in the paper. Where on the one hand, you're looking for some measure, right, to characterize these systems. And the extremes are both uninteresting. Noise, pure noise is like hard to compress, but that doesn't make it like fundamentally interesting. And then on the other extreme, pure order, a pure crystal lattice or whatever that's totally regular is high order, but also like not super interesting. And so you end up looking for something in the middle that has the Goldilocks properties of interesting complexity.

• 11:54Nathan Labenz?

  Maybe you can give more kind of motivation or theory or philosophy behind this. But I didn't have a great sense for kind of exactly how principled that idea is. Or maybe you can, first of all, just define the metric that you ended up using. And then give a little more color on could reasonable people disagree on whether that's a good metric? How should we think about this metric of complexity?

• 12:15Speaker 02

  Okay, so first, let me just note that I have a bit of a background in information theory and compression, which has colored some of the ideas behind this specific definition of complexity. And of course, another important note is that I think you could have discussions for years about what complexity actually means. So what I wanted to capture when using this definition of complexity was basically trying to have a somewhat mathematical definition for structure, like amount of structure that is present in string or something else. Of course, as you mentioned, this notion of structure has some issues

• 12:58Speaker 02

  because too much structure is also not too interesting. And I think that the metric we ended up defining has a reasonable trade-off between overemphasizing structure and ignoring structure entirely. Anyway, so the metric we ended up going with is, at least theoretically, and I'm not actually giving an exact definition, because there are a lot of details that one needs to get right. But intuitively, the idea is to consider the difference between the Shannon entropy and the Kolmogorov complexity of the data that we have. Now, why would someone do this? Intuitively, one can think of Shannon entropy as how hard would it be to describe, let's

• 13:42Speaker 02

  say, this string of characters if I were completely ignoring any kind of structure that it may possibly have? For example, a sequence of half A's and half B's has the same Shannon entropy regardless of whether it first has all the A's and then it has all the B's, or whether they are just mixed together completely randomly. Shannon entropy is a measure that completely ignores structure. Kolmogorov complexity is basically sitting at the opposite end. It's saying, what's the shortest description you could give of this string? You can do whatever you want. Any Turing machine, what's the shortest Turing machine that will produce this string as output?

• 14:23Speaker 02

  So a Kolmogorov complexity absolutely can tell the difference between first all the A's and then all the B's, or alternating A's and B's. So then, the idea of taking the difference is that one ignores the structure, the other one lets you take any amount of structure in consideration. So what is left is basically only the structure. So that's the main motivation for the definition. I think also that may be a good idea to make an analogy with the real world, that maybe will help to clarify the definition a bit. So imagine you have a building. A building contains a lot of molecules, a lot of different materials, a lot of different things.

• 15:07Speaker 02

  And you can imagine the equivalent of Shannon entropy would be something that tries to define the position of all the molecules independently for the context of a building. The equivalent of Kolmogorov complexity would basically be here's the blueprint of the building and given the blueprint, you know how to build this building. So that tells you what the building is. It's a description of the building. Now imagine that you take the same building and you completely destroy it and it becomes just a pile of rubble. For a pile of rubble, it has the same molecules, so it has the same entropy, but it doesn't have a simple description like a blueprint. If you wanted to get the same pile of rubble, you would need to know where all these random small pieces sit.

• 15:50Speaker 02

  So because of that, in this analogy, the pile of rubble would have a significantly lower complexity or structure. And that's more or less the intuitive reason why I think that defining complexity in this way makes sense for, I want to emphasize the amount of structure that is in it.

• 16:11Nathan Labenz?

  Hey, we'll continue our interview in a moment after a word from our sponsors. Is there a philosophical extension to that or implication of that? When I read this paper, I can just look at the graphs or these graphs where it shows complexity rising through time as the dynamics of the system evolve. And we'll get to the actual details of the nature of the system in a second. But I really was left with this feeling that there is some implicit value judgment over these states. And I don't know if that's something you would want to sign on to or not, but it brings to mind these like various thought experiments for me where if I understand correctly, a

• 16:55Nathan Labenz?

  Tiled universe of happy people. If you were a utilitarian, you might say, oh geez, if I had this one very happy person and I made a zillion copies of them and filled the whole universe with this same thing over and over again, that would be great because all these people would all be happy and we would count up their utility and have very high utility. So fantastic. And then somebody else might say, doesn't that sound kind of boring? Shouldn't there be some sort of discount for the fact that this like same thing is taking up all the space? Don't we want more diversity for its own sake? It seems like this measurement that you've defined implicitly suggests that second frame. Is that fair or would you not go that far?

• 17:37Speaker 02

  I have to say, we did not assign any value judgments, philosophical implications of this definition of complexity. We did not consider that. In fact, I would be somewhat surprised if this definition of complexity becomes useful outside of this paper. Pleasantly surprised, but still surprised. So one thing I want to say is that this is a measure of structure, right? It's not a measure of value or it doesn't take into account diversity. It is not a measure of utility, right? So it's just a measure of structure. Because of that.

• 18:12Nathan Labenz?

  Wait, just quick interjection. There is some way in which it counts diversity, right?

• 18:18Speaker 02

  To some extent, yes. So it's a bit hard to explain exactly how repetition gets taken into account by this measure. This middle ground is the best point, as in if you have no repetition at all, then very likely this complexity will be low. On the other hand, if you have a fully repeating and copies of the same character, for example, then the measure of complexity will also be very low. So you do want some amount of repetition and ideally repetition that itself has some structure, but not too much. But that doesn't imply that you can still have repetition of.

• 19:01Speaker 02

  One relatively complex structure that's just the same but repeated a lot of times. And that will have high complexity. So it's a bit of a trade off. If you have too much repetition, complexity is low. If you have too little repetition, complexity is high. In the experiments we have been doing since programs at 64 bytes. That turned out to be a reasonable trade off and high complexity effectively correlates extremely well with the soup that's full of certificators.

• 19:32Nathan Labenz?

  And to be clear, this is also purely a measure, right? This measure of complexity is not being fed back into the system.

• 19:40Speaker 02

  Not at all.

• 19:41Nathan Labenz?

  In any way, shape or form, yeah.

• 19:42Speaker 02

  Yeah.

• 19:42Nathan Labenz?

  Okay.

• 19:43Speaker 02

  Also, just I think I have to make the disclaimer. Kolmogorov complexity is not something you can actually compute. So we're not actually using Kolmogorov complexity. It's just mathematically easier to reason about Kolmogorov complexity than it is to reason about whatever other proxies for Kolmogorov complexity you may end up using.

• 20:04Nathan Labenz?

  Gotcha. Okay.

• 20:06Speaker 01

  About the complexity metrics that we used, we have also used something else once, which is counting the number of unique tokens that rise into the soup. And this would mean that if you got a certificator that takes over, we don't count the reproduction of this certificator to another string as creating new tokens. And this shows that, like you see when certificators arise, very stark decreasing the number of unique tokens that there are into the soup. And then we checked, and this is very well correlated with this other complexity metric. So you can think about us using this higher order entropy metric for, as a proxy for this unique token count, because of course it's much easier to compute.

• 20:45Nathan Labenz?

  Gotcha. Yeah, that's a very good point. One of you guys can take, just setting up, what is the little micro self-contained universe that you've created? And I'll ask any clarifying questions that I have.

• 20:58Speaker 01

  So the very simplest experiment that we performed is as follows. Imagine that you have a lot of, a soup of random strings. The random strings represent code, but they can also represent data. Those are essentially what you would pass to a compiler to execute some code, to a computer. And what we say is you randomly take a pair of strings, concatenate them, and execute the code, starting from the very bottom left. And you execute the code for a long time. And then you just split the strings again and put them back into the soup. Now, repeat this many times and have a lot of strings do that in parallel.

• 21:41Speaker 01

  So this is the only thing that happens in the simplest setup. You just take two random strings, concatenate them, execute them, and then split them again. So there's no objective function whatsoever. There's nothing else that happens from our side, at least in one of the possible setups. And the only thing that happens is that the strings modify themselves by reading each other's code and modifying the code.

• 22:05Speaker 02

  Let me show you an example, which is also the example that captivated us into digging deeper into this. Let's see. Okay, so here you can see a few programs from the soup. Each program is 64 bytes. This is one program. There are two columns, not for any particular reason, it's just that it makes better use of space. And you can see that most of the bytes are random. And there are a few bytes, you can see the ones here that are highlighted, that actually represent instructions, and everything else is just nothing.

• 22:47Speaker 02

  There is also a special character that represents the byte zero, which is special. And you can probably see that here, I'm not sure how well it's visible across the video, but it's a null byte. Okay, so if we let these programs randomly execute, you can see that they start modifying themselves. And sometimes some strings that appear, like copies of the same character, or maybe the ends of the string, gain some characters, lose some characters. There's also a few mutations that just erase things from here and there.

• 23:23Speaker 02

  This evolution continues. There's a lot of interesting behaviors. There are things that extend, things that... For now, there's nothing too interesting that is happening. By the way, this is the same example that is explained in the paper in Figure 2, I believe, where there is like a story of the replicators that appears here. So somewhere around now, you notice that there was a lot of activity, and that activity left a lot of null bytes around. Now, you can see here we have a count of how frequent is each byte. And you can see that the null byte now is way more common than it was before. And if you just go back a couple of seconds, okay,

• 24:04Speaker 02

  if you go back a couple of seconds, there are basically no null bytes are not special, but in the time of just a very short number of executions, there are a lot of null bytes. After that, there is another little bit of nothing too interesting happening. And then, somewhere around now, we get actual replicators. And this explodes in a lot of activity. And at the end, like now it's posed, but at the end you can see that there is the same structure that repeats itself basically everywhere in the soup with some minor modifications. So seeing this for the first time is what gave us the intuition

• 24:48Speaker 02

  that there was something interesting happening here. This kind of weird evolution of a time, different behaviors, and all of this.

• 24:55Speaker 01

  So what happens before seplicators arise is actually not completely random. There's a distribution of characters that changes like from the very beginning. You can think about it as just a complex dynamical system where there are operations that are growing inside, but in this transient way that is not stable until there's seplicators arising. And this is, if you were to talk about it in the origins of life terminology, you could think about this as being the pre-life period. A period where there's dynamic interactions, usually in the case it would have been with some catalysts that catalyze each other or create... Some complex dynamics until to the point where self replicators arise.

• 25:37Speaker 01

  And then this moves into what sometimes is called the life period where self replicators arise. And the important part there is that things happen also in the pre-life period. But whenever there starts a replicator, this change is very stark. There's a very big transition in dynamics. And that's why it's useful to think about it in these terms. So our paper is mostly focused on the how do we get to this transition? How do we get from a random soup to something as complex as a replicators?

• 26:06Nathan Labenz?

  So for the uninitiated, which included myself up until very recently, let's take just a little bit more time on what we were just looking at there. And you can maybe even pull it back up again, just to have the thing on the screen. So you start off with just this totally random set of strings, each of which are 64 characters. So that's worth just pausing on for a second to say that is a remarkably small number of characters for anything really interesting to happen. But that's the whole thing, right? You've got 64 characters. And if I understand correctly, you're literally generating these out of ASCII characters at total random?

• 26:45Speaker 01

  Not ASCII.

• 26:46Nathan Labenz?

  So literally just?

• 26:47Speaker 01

  Just bytes, but yes.

• 26:49Nathan Labenz?

  Okay.

• 26:49Speaker 01

  Random, independently uniformed by every character selected independently from everything else.

• 26:55Nathan Labenz?

  Gotcha. And then the way that this is going to be interpreted, I'd never actually heard of this before, is through this extremely minimalistic programming language with the charming name of Brainfuck.

• 27:09Speaker 01

  Yes.

• 27:09Nathan Labenz?

  And like this is a prior art, right? But this is something that somebody made to try to say, what is the smallest possible computing paradigm that we could imagine? Didn't stop me at any point here if I'm wrong. But what actually happens is you have a two-head computer that both start in the first position, if I understand correctly. When you take two strings, you put them together. Now, why do you take two strings and put them together? That's essentially simulating the randomness of the universe, right? We've got molecules bouncing around, Brownie in motion, in solution, whatever, in a gas, whatever they're doing. They're bouncing around and they're bumping into each other.

• 27:52Nathan Labenz?

  At essentially random, right? Or at least we're modeling it as being random in this case. So take two of these strings out and just simulate an interaction and just arbitrarily tell me if there's more to it. But seem arbitrary that we just concatenate them and then we'll treat this as if it were brainfuck language. There are only, looks like 10 different commands in the entire language. And all the other characters out of the 256, right? Possible bytes, all the other things are just considered to be neutral data. And then I'm, so the commands that can happen are like, okay, you've got these two computer heads that both start at zero. One thing to do is move ahead from place to place.

• 28:37Nathan Labenz?

  And you literally just move them one square over, right? So it's move one to the left, move one to the right. There is increment and decrement of value. So you have a value, I guess that assumes that there's like an order to the bytes as well. Or you just increase the bit value by one. So the heads can only move one position, one step at a time. And the bytes can only move one value at a time.

• 29:03Speaker 02

  Yes.

• 29:04Nathan Labenz?

  And then you can also copy from one position to the other?

• 29:07Speaker 02

  Yes. And there is the most important instruction, the one that actually makes this language thing complete. And just for full disclosure, this is not exactly brainfuck. This is very close to brainfuck, but the standard brainfuck language only has one head. And then the input and output, like the copy operations, read and write things to the terminal for humans to interact with. But in this setup, there are no humans, so it makes sense to have two heads that modify things themselves. So let me show you, this is a representation of an execution of a program. There are actually three cutscenes, not two.

• 29:51Speaker 02

  The two heads have to do with the position that you read and write from. And the third head is the position of the next command that you execute, which is fundamental. In particular, the next command that you execute head is the only one that can move by more than one position per step. And those are the square brackets. What the square brackets do is they look when you execute that instruction. If the character is on the, I think the blue hat, I'm not sure if I remember correctly, is zero. And that's why zero has a special representation.

• 30:33Speaker 02

  Then you skip that execution, like you skip those brackets if you're going in one direction or or just go through. If it's not zero, you jump to the other bracket. So that's how you can implement things like conditions and for loops and all of those things, which is what allows for this language to be Turing complete.

• 30:54Speaker 01

  Just one note. So Brainfuck is not necessarily the even smallest language that would be Turing complete. It was created as far as I understand to make it very hard for humans to code in it.

• 31:08Speaker 02

  Yes.

• 31:08Speaker 01

  You can have even simpler instruction sets that are still Turing complete.

• 31:12Speaker 03

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 31:17Nathan Labenz?

  I just want to make sure that people have a very, maybe not full understanding, but at least a intuitive sense for just the sort of universe that we're operating in here. So maybe to try to say it back to you one more time. There are three computer heads on this that operate over this ultimate space when two different strings are attached together and they go from 64 to now combined, they're 128. So you've got 128 bytes. Yes. Three positions, two can read, write and be moved left or right and can increment or decrement. And the other one is the one that handles the commands. That one is the one that can bounce around when it hits these open and close parentheses

• 32:01Nathan Labenz?

  type operators. And I guess there's maybe two more things that that matter here. One is the definition of a self replicator. That is when the first 64 bytes manage to copy themselves over to the second 64 bytes exactly.

• 32:21Speaker 02

  Mostly. It's slightly more complicated than that. Because they need to copy themselves to the other 64. The, yeah, the thing that could do is have the other sixty four bytes be the copy of the original program, but destroy the original program while doing so. And that would not be a self replicator, it's just something moves.

• 32:41Speaker 01

  Also, to make things more complex, a self replicator doesn't have to be only with sixty four bytes, only if it's entirety of sixty four bytes. You could copy a smaller part of the, of this substring, let's say, of the sixty four bytes many times, and this would still count as a self replicator. So it's actually quite complex to tell whether you created a copy of yourself as a substring, because it's not just saying sixty four bytes on the left need to be equal to sixty four bytes on the right. It could be that it's like thirteen bytes on the left that replicate on a very random position on the other side, or some of them that get replicated many times in this hundred twenty eight bytes concatenated string.

• 33:23Speaker 01

  And this makes it even more important to figure out a complexity metric to understand what's going on, because it's very hard to detect explicitly for a perfect self replicator otherwise.

• 33:33Speaker 02

  Yeah, the, the way I like to think about it is that saying that it's hard to say whether one program is self replicator, but it's a lot easier to say whether the soup is full of self replicators.

• 33:46Speaker 01

  And it's even more complex than this, because in some experiments, we have one single long tape, where there's no concept of individual strings. So determine whether there's a self replicator there, you cannot avoid it. You really need to think about, there are sides of which can be anything for a substring which is a self replicator, and how do you tell whether a self replicator is replicating in any given offset?

• 34:09Nathan Labenz?

  And that's basically just an experimental variant where instead of having these discrete units and merging them randomly, you just literally run the program indefinitely over the one super long string of characters?

• 34:24Speaker 02

  I believe the way that setup works is that theoretically you stop the program and you start it again from a random point.

• 34:31Speaker 01

  Yes.

• 34:31Nathan Labenz?

  Gotcha.

• 34:32Speaker 01

  But I think that thinking about this example is the simplest, and there are also extensions which are putting one dimension or two dimensions into the equation, so it's no more uniform distribution, but it's a uniform distribution among your neighbors. But it doesn't change the concept. Thinking about just having uniform distribution of random strings concatenating should work.

• 34:52Nathan Labenz?

  Gotcha. Okay. And then the one other thing that I wanted to note is that there is the possibility for errors in these programs, right? Since the data was all generated randomly in the first place, when you try to run this random string of characters as a program, it can error out. And specifically, it seems like it errors out if the sort of control logic breaks. If it hits one of a particular special character, which is the zero character, and it's meant to look for the next or the last parentheses to handle control logic, and there isn't a corresponding proper place for it to go to, then that's basically just the end of that run. Is that right?

• 35:37Nathan Labenz?

  And everything that happened up until that time, all changes that are made, those are just left in place and we move on to the next step in the simulation?

• 35:44Speaker 02

  We tried a few different setups. The overall result doesn't change that much anyway. But yes, I believe in this setup. If you encounter a jump instruction that doesn't have anything to jump to, it just stops execution. And there are a few other things, like if you end up trying to execute past the end of the string, it also stops execution because what did execute anyway? But in general, one important thing is, and also one of the reasons why we are using Brainfuck and not C++, is that we want the interpretation of programs to be as permissive as we reasonably can. If you don't do that, then, you know, then basically you never actually get anything interesting

• 36:29Speaker 02

  because you just error out at this single time.

• 36:31Nathan Labenz?

  Okay. So I think that's a pretty good overview of the setup. It's both remarkably simple in some ways and obviously has some nuance. Well, let's talk about what happens as you run these things. I think you've alluded to it a little bit already. But essentially, you have this pre-life, or at least pre-self-replicator period in which everything is still random and these things are being concatenated, they're being run, data's changing. You can maybe tell us a little bit more about what evolution of the overall system you can see during that phase? And then there's a very sharp Phase change, and I think this notion of phase change is a really important one

• 37:15Nathan Labenz?

  that happens when the first self replicator pops up. And again, probably can't stress enough, there's no incentive for this self replicator to pop up, right? No loss function here like we're used to thinking about in AI systems. Literally just blind execution of the program, stumbles on through random interaction between two strings, something that is a self replicator. And now when that happens, very quickly, it starts to replicate a lot. And now we're into like a different regime in the overall story of this little universe.

• 37:51Speaker 02

  So since we initialize everything with random bytes, in the beginning, everything is random. Right? So you can see, okay, so this is already after a few executions. It's not exactly quite uniformly distributed, but here we have the least common character and here we have the most common character. And you can see they are pretty close to each other. No, no character appears more than half of the time. So the first thing that happens in the first few thousand executions is that distribution shifts. And you can see here, after a few thousands of times that programs interact with each other, we start having a few characters that appear like two percent of the time. We are still investigating exactly what is happening here.

• 38:36Speaker 02

  There's clearly something that is important. Here we can see that this character, this is one of the characters that moves the heads, for example, is getting particularly popular. And that some characters also become particularly unpopular more than others. It's not like there's just one character that becomes significantly less common. It's also that other characters become less common. And then we have some of these state transitions, as we were saying, where the behavior changes significantly. Here, there's a lot of activity for a really short time. And that leaves you with a completely different distribution. And some characters almost completely disappear.

• 39:17Speaker 01

  And there's one interesting thing that that one of our co-authors observed that you can think about the initial distribution where you got some operations which are code that you can execute and other operations which are just data. So if you try to execute there, nothing happens. So even at the very beginning, the only thing that can generate more code is code itself. And since the very beginning, in this kind of system, the number of instructions that get executed and that are present in the soup starts increasing immediately. So even before having separate indicators actually appearing, we still have some one thing that could be very similar to the concept of auto-catalytic sets, where those codes, strings of codes or pairs of codes,

• 39:57Speaker 01

  generate other pairs of codes that then can have a chance of generating themselves again.

• 40:03Nathan Labenz?

  Yeah, that's really interesting. I'm noticing, just looking at the top of this, that it seems that both the highest frequency characters, although maybe not that equals sign.

• 40:16Speaker 02

  No, the equals sign.

• 40:17Nathan Labenz?

  If I'm reading this right, I'm seeing the open arrow. The open arrow is the 4.94%, right?

• 40:22Speaker 02

  Mm-hmm. Yes, so here we have the 16 most common characters and the 16 least common characters. It is quite interesting indeed that plus and minus tend to be not that interesting, but I think perhaps makes sense because if you execute a plus, that that will likely make an open parenthesis into a completely different character. So that's likely to ruin meaningful code. And the pieces of code that are useful for to copy, like the instructions that are useful to copy things over, like the loop instructions, the instructions that move the heads, and the copy instruction are super common. At least we believe that because if you have a lot of copy instructions, then...

• 41:07Speaker 02

  It's easy to produce more copy instructions because you just will read something and that something is probably a copy instruction itself, so you will copy that over. And this is a self-reinforcing effect. Like, having many copy instructions just gets you even more copy instructions. And probably there is something similar happening with loop instructions.

• 41:27Nathan Labenz?

  Yeah, cool. That's fascinating unto itself. The instructions that enable copying tend to get copied and the ones that scramble data tend to scramble themselves out of existence. And this is before we've even got two self-replicators. Now, I guess one could ask, is there a sharp line between this notion of, even you raised the concept of an autocatalytic set, is there a bright line between that and a self-replicator? It seems like there is because there's a phase transition, but when I just look at a pair of characters could be, in a sense, enough, right, for it to be copying itself over, I feel like I'm missing some critical distinction there.

• 42:11Speaker 01

  Yes, I think I can help here. So there are also people that would very easily call autocatalytic sets set replicators. It's a matter of how efficient they are, because the biggest difference between the pre-life and the life period is the speed of, let's say, information to be replicated and be copied into their neighborhood. When you have got the pre-life period, you've got some autocatalytic sets that happen, but they're very slow in copying each other, also because they need to do many more steps before they become multiple of what they were before. And it's also by chance. But when you've got a function set replicator, in biology, the best example is like something

• 42:52Speaker 01

  that people suspect being the RNA and with the polymers and the ribosome, then this just sped up very quickly and then in much shorter timelines, it takes over the entire soup.

• 43:04Speaker 02

  So, looking at the paper, there is a figure one that shows us the number of unique tokens.

• 43:12Speaker 02

  Or even the complexity. And you can see that the number of unique tokens slowly decreases in the initial phases, because...

• 43:19Nathan Labenz?

  Quick clarification. A token here is a sixty-four length string, no?

• 43:26Speaker 02

  A token is a marker that basically identifies, is what Ettore was saying before with the tracking of copies. So, number of unique tokens is the number of bytes that have not been copied.

• 43:43Speaker 01

  So, in a string, there will be at the beginning sixty-four unique tokens, because it's sixty-four bytes.

• 43:49Speaker 02

  And then if you copy half of the string over itself, then you have thirty-two unique tokens, because half of those bytes come from the other half of the string. So it's just keeping track of where the bytes originally came from.

• 44:04Nathan Labenz?

  You're actually keeping a full log of every copy event?

• 44:09Speaker 02

  We did it once, let's put it that way. But yes, this is, this number of tokens takes into account the history of every byte and looks at where it came from. And if the token came from the same byte, then it's the same token.

• 44:23Speaker 01

  There's one caveat which here becomes quite important. So far, we talked about separate modification being the driving factor of certificators to arise. And this can be proven even by just having experiments where there's absolutely zero background noise. So, the only way to modify some characters is to execute the code. But we usually also add background noise. So, every now and then, some characters get randomly modified. This helps to speed up the process. And so you need to find certificators which are more robust. But it's not necessary. We still observe everything happening even without background noise. What you're seeing here in this graph shows that unique tokens decrease.

• 45:04Speaker 01

  And that's also because, sorry, the decrease, but there's the impact also of adding unique tokens every now and then.

• 45:13Speaker 02

  Yes, if you look here... Once we do have a self replicator, the amount of unique tokens increases. That's obviously not possible if you don't add new tokens every now and then. So this increase here is the effect of mutation. So what is interesting, tying back to the conversation we were having before, is that here, the number of unique tokens, even before we have anything like a full form self replicator, is steadily decreasing. So that's basically saying that there is a fair amount of copying that is happening even before we have actual self replicators.

• 45:53Nathan Labenz?

  So that first line, when the first self replicator pops up, is obviously a big event in the history of the system. You've got a major drop in unique tokens. You've got a huge rise in top 32 tokens. And that is, should I understand that as the sourced back to an original initial token? Like this is the

• 46:15Speaker 02

  So I believe that

• 46:15Nathan Labenz?

  A character that has been copied the most?

• 46:18Speaker 02

  Yeah, basically, top 32 tokens is basically saying, how many copies exist of the 32 most common bytes? So as you can see here, there is a sharp increase because at that point, we have a self replicator that copies itself a lot. And by byte, I don't mean like byte value, I mean original byte.

• 46:39Nathan Labenz?

  So then the complexity also spikes because we go from a noisy environment to a more ordered environment. And then it drops back down. And should I understand that as being a period of tiling? Is this, is that where we've overshot and gone to the other extreme in complexity? Where the self replicator is everywhere for a while?

• 47:04Speaker 02

  Actually, no. So let's see if I can find the moment corresponding to that point here in the video. I believe it was here. So that's that's basically an extinction event. I'm I'm still on the um on the graph. Yeah, okay, so in in a few seconds, you will see there is going to be a lot of movement and then nothing will happen after one second of a lot of movement. And that's the spike. See, here, there was a lot of changes. And now everything is stopped. Yes, there is still some modifications, but it's it looks like it would look to begin, right? It doesn't look like the self replicator regime here.

• 47:46Speaker 02

  When you do have successful self replicators, there is a lot of movement around. And you can see that, but only for a really short time. And that's the time that corresponds to this spike. Because there we have you have replicators take over. But they are this initial kind of replicator that we have here as an example. And the interesting property of these replicators is that if the bytes that they are trying to write themselves over as zero, they just don't do anything. If they try to write a set of bytes of any kind, it works. You can see here that it's just slowly copying itself

• 48:31Speaker 02

  in reverse. But if if the the bytes you had there are actual zeros, then it doesn't work out. So it just doesn't do anything. And so what this does, and you can see this in the soup here. What this does is basically very quickly you have every single byte at the end of the strings be a null byte. You see here? And now the replicators cannot replicate themselves anymore. And so basically, the replicators go extinct. Nothing happens. And then afterwards, we find a different kind of replicator that doesn't have this problem. Like manages to copy itself over null bytes without any problem.

• 49:16Speaker 02

  And because of that, you you have the second state transition here. The soup becomes entirely self replicators and the complexity just jumps up. So this is basically just a short extinction event. And it doesn't happen always for that matter. Like in most of our experiments, we just see a sharp jump in self-replicators and that's it. It's just that we were lucky enough that the standard setup ended up having this particularly interesting example, so which is the one that we ended up showing here. But it doesn't always happen. What's more typical even is just that as soon as you have one replicator, you, the soup takes over and there's no extinction event in the middle.

• 49:56Speaker 01

  Not one replicator, in the sense that we have actually also tested the hypothesis of if you had one self-replicator initialized in the soup, what's the chance that it takes over? And it's actually quite low. It's what, 23%?

• 50:10Speaker 02

  Twenty-two.

• 50:10Speaker 01

  Because there are many things that can happen that destroy a self-replicator. If there's background noise, why they self-replicate, they may just be destroyed by the noise. And if there is no background noise, you might be in the wrong order when you try to replicate. So if another string is the one on the left and it starts from this random code, it might destroy you and therefore you will not self-replicate. So there's actually low chance for one self-replicator to take over.

• 50:37Speaker 02

  In particular, we don't, we did not test this because as we said, it's hard to see if one program is a self-replicator. But what we expect is that there actually have appeared other self-replicators here that just died. What is remarkable about this specific evolution is that our self-replicators took over a fair bit and then still died.

• 51:01Nathan Labenz?

  Gotcha. So can you give a little bit of, I don't know if a taxonomy is too much to ask of like the different kinds of self-replicators that arise. I might be overinterpreting the graph, but I came away with the sense that there are some primitive ones that pop up first, but then they have these problems with zero. And so they sometimes fizzle out and then like more robust ones come later, but maybe they take longer to emerge. But maybe I'm telling more of a story there than I, I should. Maybe it's more random actually than all that narrative that I just laid out.

• 51:39Speaker 02

  So certainly have observed different kinds of self-replicators. So here you can see the first replicator to appear in that setup. And it has this structure where it starts a loop. And then it first copies a character, then moves the two heads and then looks back. So you can see here the execution, right? So when it's executing this character, it copies this space to here. Then it moves the green head back, sorry, the blue head by one position. It moves the red head forward by one position. And it says, this is not zero, let's look back. Sorry, this is not zero, but that's the bytes that gets checked.

• 52:21Speaker 02

  And so here it repeats, and so now it just copies itself over. And as I was saying before, this has the problem that if it tries to copy itself over when it does this check of, okay, should I go back here? And says, this is zero, so no, I will just go forward and then nothing interesting happens. The second replicator that appears is somewhat more resilient. And it's fundamentally still the same thing, right? It still does a loop, it still moves the two heads. But the difference is, I don't know if you noticed this, but the order of the heads is swapped. So here you write to the blue head and here you write to the red head.

• 53:03Speaker 02

  So what this means is that having zeros in the string you're trying to override doesn't give you problems. You do get problems if you have zeros in the string you're trying to copy. If this were a zero, then you fail at copying yourself because you just copied the first part of the program, you don't copy the entire program. And note that the program is copying itself in reverse order. When execution starts from the next program, it starts from here. So it's important to copy these first few bytes. The damage doesn't go to the end of the execution, it just shows the first few steps of execution because it takes a while.

• 53:44Speaker 02

  Yeah, the other duplicators will eventually manage to copy the this part over to this part and so you get an actual copy of the full replicator. But with this one, there is, as I said, this problem that if you have a zero byte in between the two halves of this replicator, then it doesn't actually copy itself. Now, that's not as much of a problem as it is for the other replicator. Because there is a difference between I need to have a null byte myself and the program I copy myself to has a null byte. If you have even just one self replicator that doesn't have a null byte, then it will succeed at copying itself over and so it will survive way more easily.

• 54:27Speaker 02

  But the problem is if the, let's call it food or rest of the soup is full of zeros, then this self replicator cannot copy itself.

• 54:38Speaker 02

  Uh, but sometimes appears in after a longer evolution is, or this one specifically is handcrafted. But there are other replicators that sometimes appear that are resistant to zeros, both in the destination and in the source. Usually they contain nested loops. So it's a bit harder to explain clearly what is going on. So this is still the same replicator that we had in the previous example, like the most successful replicator. But around it, there is an additional loop. And what this loop does is just suppose I exit from the first loop because I happen to have found a zero byte.

• 55:20Speaker 02

  Then let's just take the zero byte, decrement it or increment it, whatever. Now it's not a zero byte anymore. So let's jump back and get back into the previous loop. So this is one of the reasons why I was saying that it's hard to define what a self replicator is. Because at the end of the execution of this program, neither of the halves of the self replicator is the same as the original program. Because this null byte is not a null byte anymore in the original program and in the new program is also not a null byte. But still, you would say that this program copied itself, right? Because the important parts copied themselves.

• 55:56Speaker 01

  And this program is also very robust to noise, by the way. So in noise environments, this is particularly good. Because even if you can copy yourself perfectly, you can have a random mutation that could destroy other programs. But for a random mutation to destroy you, it has to be in the few operation that you have in this case. We also observed that in other languages, that there's this typical behavior of self-propicators, that if there's nothing else that has a pressure, they're trying to become shorter and shorter. In this specific one, it's tricky, because actually, it's a speculative self-propicator for the nuances that this Brainfuck variance has. But in other languages, that would be relevant. And we've also observed that different kinds of self-propicators

• 56:40Speaker 01

  that use different operations take over in other languages that, for example, are CPU instruction-based. But I want to stress out that the main focus of this paper was to focus from the how you get out of the pre-life period. So we didn't explore nearly as much as what happens after the first self-propicator arrives. This is something we very much care about, but for future work.

• 57:04Speaker 02

  Yes, we just had some observations of more resilient self-propicators appearing over time.

• 57:10Nathan Labenz?

  So there were at least three classes. Tell me if I missed any. There's an initial one that... Maybe I shouldn't order them. But there's working from least to most resilient. There is one that can copy itself, but if it hits a zero where it's trying to copy two, it fails. There's one that is highly susceptible to mutations in its own code and can fail as a result of that. And then there's the robust kind that can both write over the zeros and is much more tolerant of...

• 57:50Nathan Labenz?

  Mutations in its own space as well.

• 57:53Speaker 02

  Yes.

• 57:54Nathan Labenz?

  And we know that all these things can arise. How often do multiple of these things arise? I guess you just said there's more future work to do to play out from the emergence of these things to the long term dynamics. But how often do we see multiple different things battling it out versus is this like one replicator's lonely struggle for survival? Or are they doing battle with each other frequently enough?

• 58:17Speaker 02

  It's a bit complicated to evaluate, especially in the multidimensional setups. Interactions are a lot more sparse. And since interactions are a lot more local, there is space for niches to appear and going into different directions in different places. And then they start fighting with each other at the border and trying to take over other self replicators. But even in this like back dynamic setup where everything can interact with everything else, you can see that the structure is similar, but not that many self replicators are identical. Because the effect of mutations gives you this, right?

• 58:58Speaker 02

  Where now everything is equal, but then one self replicator mutates a little bit. And then if that one starts getting copied a little more, then you have replicators that fight with each other. Even if one self replicator became a lot more efficient than the other one, then it probably would win. But in this setup, if you have multiple replicators that are similarly efficient, for example, if you look at these two, they are identical, except like some of this random unused bytes here are different. So the efficiency is the same, but you will see a bunch of both kind of both kinds.

• 59:34Nathan Labenz?

  Yeah, interesting. And that's if I understand correctly there, it's like you can have the same mechanism, but you can have different like filler data. This is almost like junk DNA. I don't know that there is such a thing as junk DNA, but if you believe in the notion of junk DNA, this is the junk DNA of your universe.

• 59:50Speaker 02

  Yes.

• 59:51Nathan Labenz?

  So it's amazing how many concepts like that we are seeing in just this, you know, very small thing, right? We've got different mechanisms for self replication. We've got mutations that can be

• 1:00:07Nathan Labenz?

  deleterious or that can be survivable depending on the mechanism. We've got some notion of pollution with the zeros popping up that can become so common that they make it difficult for the self replicators to succeed. We have the junk DNA, we have competition for niches and then fighting at the border. It's really an amazing, this is what kind of just captivated my imagination. Looking through these results because it's like, that is such a small universe in which to find so many of these advanced concepts. Is there any other kind of recognizable concept from

• 1:00:53Nathan Labenz?

  the study of macroevolution that you have observed that we haven't touched on yet?

• 1:00:56Speaker 02

  Actually, yes. Please one, so we have some interesting cases of something similar to symbiosis. This is in a different language and I don't think I should go into too many details because it would take a while. But the idea is that we have like a different language, which is a variation of FART in which there is always one byte self replicator that just copies, that's just that one byte. So if a program starts with that byte, then that byte copies itself over. Maybe it's not symbiosis, it's not the correct term, I don't know, I'm not a biologist. But what happens is that there is evolution that happens that starts to exploit that one byte self replicator.

• 1:01:41Speaker 02

  Until you get, self replicator is able to copy the entire string that uses that one byte self replicator. And because of this, because of the existence of this one byte self replicator and using this one byte self replicator, evolution is a lot faster than in a lot of other FART variants where you don't have this one byte self replicator. So it's like enabling further evolution just by being there. Then it's not a parasitic kind of relationship because the self replicator still survives. But it's a sort of cooperative.

• 1:02:16Speaker 01

  But some of our colleagues have also observed some early examples of parasitic behaviors in 2D environments. And this is very consistent with other research done in artificial life where if you got to the environments, you can have these waves of parasites that try to destroy some self replicators that replicate in the other direction.

• 1:02:36Speaker 02

  Yeah, it's really remarkable. Basically, a lot of things we see from the world and like biology appear in this completely different setup. In fact, for me, it would be really interesting to manage to prove that this kind of evolution that we observe in our world is not something that is special to our world, but just happens naturally. It's, you know, it's a very natural thing to occur. And it's not that the world is special and that had a lot of coincidences that caused evolution to go on the way it did. It's just even in simple setups or whatever, it would be really nice to prove that we managed

• 1:03:16Speaker 02

  to have this emergence of life, self-replication, whatever way you want to call it.

• 1:03:23Nathan Labenz?

  That is maybe a good transition to some big picture philosophical questions that I have for you. One thing is just, it seems like most things go extinct. Should that be my understanding? You said even when you put in a self-replicator purposefully, there's still only like a 23% chance, I think you said, of it taking over. Do I understand correctly that the other 77% of the time, it's going extinct?

• 1:03:50Speaker 02

  Yes.

• 1:03:51Nathan Labenz?

  And if I think of this through like a Robin Hanson, great filter sort of lens, this sort of work would make me update my worldview away from the idea that the origin of life was the super hard step, or that the formation of self-replicators maybe to be a little bit more generic in my, or abstract in my terminology. And it makes me more think that a lot of things are probably popping up in the universe and fizzling out before they get to the point where they are putting out the sort of radio signals that we're putting out, or certainly like starting to colonize space. Does that seem like a justified interpretation?

• 1:04:28Speaker 02

  My way to think about that is, yes, once you have a self-replicator, you still have only one chance in five of actually doing something. But the chances of having a self-replicator, in my opinion, are even lower. It's just all steps are hard. And maybe if, what we think of what's the simplest step is, okay, there is a self-replicator, now it just needs to take over. That's already hard, then maybe just, on the other hand, it's just that the, all the other steps were even harder.

• 1:04:58Speaker 01

  But I want to be more of an optimist right now. Even in our experiments, even if it's only 23% the chance that one single self-replicator takes over, we observe a much higher than that chance that in 16,000 epochs, self-replicators arise. So I think it's about 40% in the first 16,000 epochs and you can go for much longer. So even if the individual one might not be the trigger for this state transition, it's still quite likely to happen.

• 1:05:25Speaker 02

  Yes.

• 1:05:26Speaker 01

  The real question that I think matters more is, how close is this to what you observe in the real world, if you're interested in the biological implications of this? And there are many things that this thing doesn't have yet. For example, a clear way to accumulate complexity, because as you pointed out, we only have 64 bytes per string and it's very difficult to go beyond that number, because there's no easy way for strings to connect to each other in a stable way. And biology has that all the time, it can accumulate more and more complex strings. So we would want to explore directions which are similar to that as well, if you want to try to assess how likely it is for life in biological worlds to happen.

• 1:06:09Speaker 01

  But there's still so many similarities that I think it's an interesting starting point.

• 1:06:13Speaker 02

  You can see here, there is representation of how often you actually get a self-replicator and within the first 16,000 steps, it's, it happens like 40 to 60% of the time, which is quite remarkable, because it's still order appearing from chaos, right? So you would think that's less likely. Again, this is not the probability of self-replicator appearing, it's the probability of self-replicator taking over. This is even if the first one doesn't succeed, then someone else will afterwards, the reasoning that you can have. So NA here means that it didn't happen? It didn't happen, and you can see that- At any point? It didn't happen like 40, 60, 65% of the time.

• 1:06:55Speaker 02

  In this case, there's just so much noise that nothing could possibly happen. But in this setup- This is the setup in which you observe ourselves would be getting dying in eighty, seventy-seven percent of cases. But even if it does die, then something else takes over anyway afterwards.

• 1:07:14Nathan Labenz?

  That bit about the frequency of mutation is really interesting, too. When you get up to that one percent mutation rate, then it becomes very hard for anything to persist. I was thinking that when I look at so much of the, at least the popular science discourse around life out somewhere in space, sort of says, oh, this planet might have water and therefore it might be conducive to life. And unsaid there is as we know it. And this work also makes me think that there's probably just a lot more space for life that, who knows, right? It could be extremely weird, but I no longer feel like, oh, if you don't have water, you couldn't have life on that planet.

• 1:07:59Nathan Labenz?

  Now I'm thinking you could have a gas giant that could have life in it, right? Who knows, right? It just seems like there's, if you can see this much complexity in this simple of a system, then I don't really have any reason to be prejudiced against what's going on at some depth in Jupiter. Maybe I would have some reason to think that like planets need an atmosphere or they need some sort of shield because they can't be just totally bombarded with crazy radiation that's going to cause too much noise in the system. But I feel like even at the level of substrate, I have to be like pretty open-minded now in view of these results. You're nodding. Sounds like you, it looks like you interpret that similarly.

• 1:08:41Speaker 01

  I mean, even with very high levels of radiation, you can still have life happening. It's just that it's going to be probably smaller and even that would be interesting to observe. One extra thing that this shows, at least I think is an interesting direction, is sure you can think about what can happen in our universe, but we are not restricted to our universe when we do simulations. So what we are showing here is a completely different computational substrate, which is as similarities to what biology can show, but we can do many different things and we can really be creative here for trying to figure out how to create interesting and complex life in those directions and ideally ultimately intelligence as well.

• 1:09:23Nathan Labenz?

  Perfect segue to one of my big questions, which is, what should we take away from this for the purposes of understanding what the future of life and artificial intelligence turned to life might look like on our planet? My gut says, like, again, I should just be very humble in terms of what my expectations are. If you can create something that looks this much like life in this tiny little space, then I think we're headed for a world in which some form of AI begins to self-replicate. We obviously have a very large digital ecosystem, computational environment for possible things

• 1:10:06Nathan Labenz?

  to self-replicate through. And here, obviously, there's not intelligence involved, but if you add on intelligence and these sort of interesting primitives and put them in a big, open space, like we seem to be doing right now with open source models, this significantly increases my sense that whether it's Lama 3 or 3.1 or Lama 4 or whatever, that some version of this sort of thing pops up that has the recipe for self-replication. Possibly a mix of smart and dumb, but it just seems like the barrier to getting there is not nearly as high as people might be inclined to intuitively think.

• 1:10:51Speaker 02

  So I think one thing that is important to keep in mind is that here we are getting this spontaneous emergence of self-replication, but that's through a lot of interactions. The time skip, and at the end of the execution, when we have the first self-replicators, we have seen 50 billion instructions executed.

• 1:11:15Speaker 02

  We run, I believe, three million pairs of programs. And yes, you can see that running in one million. But imagine i- i- if open source AI models gain like a chance of the capability to self replicate. And even if that were to happen, it would probably be quite unlikely for it to happen at the frequency and at the speed where we have time scales that are compatible with what we are observing here. Which of course doesn't mean that some of this we shouldn't take into account. In particular, what I would be more concerned about is the risk of subsequent modifications,

• 1:11:57Speaker 02

  like imperfect self replication, just taking things farther away from its original internal limitations or whatever it is that that were incorporated. But I don't think that the insights we see here are particularly applicable to larger scale models that would execute at much slower speeds and with restrictions. And we still also, the space that we let them execute in is significantly more guarded in some sense. Here we have a world in which you can just copy yourself over on any other string. And I do not know this, but I would expect that for even for open source models would

• 1:12:40Speaker 02

  be quite hard for them to be able to just take over any random computer for the foreseeable future. If if that were to happen, I would not be, I would be concerned by other things first and then by the subsequent ability of fundamental evolution happening. I think we would have other problems.

• 1:13:01Nathan Labenz?

  Can you put a little more detail on those other problems too?

• 1:13:05Speaker 02

  I think, you know, that would imply that all of our computer infrastructure is utterly broken and security infrastructure and any malicious actor.

• 1:13:14Nathan Labenz?

  Oh, that's true though, right?

• 1:13:15Speaker 02

  Yeah, that's why I'm not like that confident that it will not happen.

• 1:13:18Nathan Labenz?

  It's not all broken, but it's like a lot broken.

• 1:13:20Speaker 02

  Yes, that's fair.

• 1:13:21Nathan Labenz?

  Yeah, you guys at Google live in a world of the best computer security infrastructure in the world by perhaps a significant margin. And the rest of us out here are barely updating our passwords. So I would say. If the problem hinges on, or if the upshot would be that that would mean our security is bad, look no further than just how many different branches of the US government have been hacked at the highest levels. Like, the White House has been hacked, Congress has been hacked. It's all been pretty, my working assumption is that we're all pretty well and thoroughly hacked.

• 1:13:53Speaker 02

  That could be. I think that the main biggest takeaway is not to underestimate the effect of subsequent modification, like, over time. Like, I don't think that open source models coping themselves over are even just the biggest risk of open source models themselves. But my area of work is not AI risks, and I don't think I'm qualified to make an actual statement on what I should be working, but people should.

• 1:14:21Nathan Labenz?

  There's a lot of different vectors of attack and a lot of different ways that things could get pretty crazy. This just, to me, feels like one that people haven't considered nearly enough. And I'm not sure, I couldn't even begin to put like a probability estimate on it. But whatever my implicit, non-explastically calculated sense previously was, this does pull it in for me, because I'm just like, man, there's just a lot going on here. The possibility of a sort of ecology rising in such a limited space. Then I just look at the complexity and the vastness of the digital ecosystem that we have today. And I'm like, there's a lot of green field out there. Even if Google's systems are super

• 1:15:03Nathan Labenz?

  secure, like, plenty are not. And it seems like there is at least some non-trivial possibility that you could have a digital ecology start to, or start to come online, or an AI ecology start to come online. And maybe that's even good. One of the things I always think about with life in general is just the fact that it is, like, homeostatic, and that there's these sort of many buffers that exist within any given system. Those things may be really good for us long term. We might be in a better situation if we have a more... Ecological model of open source AI versus a sterile planet that suddenly has something

• 1:15:44Nathan Labenz?

  super powerful pop onto the scene and have a much larger chance of going to dominance. I have no idea, really. It's very speculative, but this work has at least got me reconsidering with a more open mind, which I think I might be totally going in the wrong direction, but it still feels like we're in this phase of time where these sorts of thought provokers, for those of us that do sit around trying to think about what's going to happen with AI, they're valuable regardless, because we need to be more open minded. We need to entertain the weird, I think, a lot more than we're naturally given to. So for me, this is a great spur in that direction.

• 1:16:21Speaker 01

  I mean, this goes back to the point where Luca was trying to say, yes, we shouldn't underestimate the cascading effect of interactions. So if you have a seperticater with a certain amount of mutation, and then it mutates many times, the results can be unpredictable. But at the same time, we need to highlight that, first of all, if you're just copying something which is equal to something else, if you have a perfect copy, nothing interesting would come up. And only when you got mutations and variants, then something interesting can happen. And this is very much an open problem that the field of artificial life in particular, but also deep learning and evolutionary strategies, has been studying for a long

• 1:17:00Speaker 01

  time. We still don't know how to create a bounded complexity of behaviors. Right now, one very open question, even in this very small experiment that we have done, is would this have a limit in amount of complexity that it can show? Or how could we make it not have a limit in the amount of complexity? Most experiments that I'm aware of did eventually plateau at a certain level where they couldn't escape. So before ringing alarm bells and saying this might go to all the way to a high level of intelligence, or like copying a LLM many times with variation would go to a high level of intelligence, we would need to have a much stronger belief and evidence that we can accumulate complexity in this way.

• 1:17:42Nathan Labenz?

  Yes, although we are making something that is extremely complicated and capable in a lab and putting it out there in a way that it can potentially interact with other things, right? The way I'm thinking about the open source AIs now, and I don't mean this to be in a prejudicial way of good or bad, because I'm increasingly more open-minded to this notion that an AI ecology, should such a thing exist, might be protective or might be good in its own way. But this seems like a, we're certainly giving some future AI life forms like a major shortcut by building their brains for them and then releasing them into the wild. It seems like we've done one of the really hard parts of creating this future in a way

• 1:18:27Nathan Labenz?

  that was not random, right? Obviously, we engineered these systems and figured out how to train these things. And so my kind of mental model, I wonder if you guys could attempt some simulations like this. I don't have nearly enough competence in this domain to know exactly what this would look like, but I wonder if there would be a way to combine something that has intelligence with something that's really simple or primitive in the self-replication sense. Is there a way to look at, people are doing these sorts of things with, can an LLM, can GPT-4, can Gemini or whatever find day zero exploits in software? And I'm trying to close in on something that's, man, you have these like pretty simple self-replicators

• 1:19:12Nathan Labenz?

  that seem to work here. And then separately, we have these like language models that are able to code and able to assess code and debug code to varying degrees, of course. Is there a sort of symbiosis type model there that could make sense? Is there something in that mix that that could be modeled in a closed environment? That could shed light on what the future of open source, just as it evolves through time, might look like. I don't have the answer to that question, but I'd love it if you did.

• 1:19:44Speaker 01

  No, we don't have an answer, of course, but I think that the closer we get to high level of intelligence, the more relevant like trying to have a setup for understanding the risk becomes and how we can control it. In our simple experiments, there is little to no risk that they can take over, especially because they cannot do more than 564 bytes. But if you get to more and more levels of complexity such as an LLM, then you really need to think about what you're doing. And I think this should be taken quite seriously.

• 1:20:11Speaker 02

  I have, and maybe an interesting point of view here. So first of all, if defining life is hard, defining intelligence is harder. I will not claim to have a definition of intelligence. When you say these things about can you find zero day exploits with LLMs and things like that, that makes me think I have a bit of a background in computer science olympics. And for that matter, there are actually sponsored experiments you've probably heard of. There is this AI MO. I don't know if that's something you're familiar with. I'm not immediately sure. This is, I think, extremely fascinating experiment. I am aware I'm decreasing significantly

• 1:20:53Speaker 02

  from the main topic of the paper at this point. There is an extremely fascinating experiment where there is this international mathematical olympics.

• 1:21:04Speaker 02

  And there are these efforts that try to get AI to just solve half of the problems in the mathematical olympics. We are not there yet. Like, I don't know if you want to classify ability of doing mathematical olympics as intelligence.

• 1:21:24Speaker 02

  My opinion is correlated.

• 1:21:26Nathan Labenz?

  Certainly some form of intelligence.

• 1:21:28Speaker 02

  Exactly. You can be intelligent if you, even if you cannot do math on Peds, I would claim. But if you think of this kind of questions, then I'm not even sure if we can actually say that the system is intelligent now. So when you say, okay, start mixing in self replicators and intelligence, I don't think I even know how something like that would look like, if that makes sense.

• 1:21:50Nathan Labenz?

  Yeah, that's fair enough. I tend to use a pretty functional definition of intelligence, one that I've been using in very different types of conversations where I'm talking to business owners about using language models to automate tasks, is that intelligence is the ability to perform useful work without fully precise instructions. So your like number recognition is a great, you know, toy example that I tend to start with, the MNIST dataset of you can recognize these numbers. Nobody knows how to write fully explicit code that can recognize numbers, but we can definitely train an AI to do it. That sort of ability

• 1:22:31Nathan Labenz?

  to succeed when we don't have the full algorithm is, I think, an interesting functional definition. I'm not sure how much the definition really matters. I think what would matter maybe most in, in the context of the work that you guys are doing is, is there a way that you could mix in a language model, or any sort of model, I think most about language models, into a simulation like this in a way that makes the self replicator like much more likely to take over?

• 1:22:58Speaker 02

  In practice, no, for a very simple reason, that you're not going to be able to run a million iterations of a language model in any reasonable amount of time.

• 1:23:11Nathan Labenz?

  Yeah, you need real compute for that.

• 1:23:12Speaker 02

  Probably you could do that experiment if you start using half of the computers on the planet, but arguably there are better things to do with half of the computers on the planet than just, you know, saying if you get self replicators is likely faster.

• 1:23:25Nathan Labenz?

  Yeah, although it depends on how much you think the AIs can help succeed, right? If they're really effective, then you might not need that much compute. We imagine ourselves like taking over sterile planets, not through random brute force search of the ways to do it, but like by actually being smart enough to show up and get it right the first time. And I don't think Llama 3.1 is probably there today, but you could imagine a level of effectiveness that might like greatly reduce the compute requirements, right?

• 1:23:54Speaker 01

  Yeah, I think it makes sense to think that a variant of our experiments with superplication can be used to find better ways to find, let's say, evolved agents. This is part of what we want to explore, but of course, like we are starting from literal strings of code and getting to something as complex as an LLM will take quite a while. And I think that we should take account for the risks at every level and increasingly so the more complex our systems become. It's about Luca's point is very well taken that the bigger the model, the more costly this would be. So to have something like this work, we would need to have much more efficient models to start with.

• 1:24:35Nathan Labenz?

  Okay. Cool. This is a fascinating discussion. I love the work. I think I should probably let you guys get back to it. Anything else you wanted to touch on that we didn't get to before we break?

• 1:24:46Speaker 02

  I think we discussed pretty much all the aspects of the paper in quite in detail.

• 1:24:52Speaker 01

  Yeah.

• 1:24:53Nathan Labenz?

  Fascinating work, thought provoking, maybe more thought provoking for me than you intended to be for the average reader, but that's the business that I'm in is trying to go grab these random things and use them to at least stimulate some creative thoughts about where the future might be headed. And you've certainly helped me do that. So I really appreciate it. Appreciate the time today. Ettore Randazzo and Luca Versari from Google's Paradigms of Intelligence team. Thank you both for being part of the cognitive revolution.

• 1:25:22Speaker 02

  Thank you.

• 1:25:23Speaker 01

  Thank you very much for having us.

• 1:25:25Nathan Labenz?

  It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at [email protected] or you can DM me on the social media platform of your choice.