Dr. Michael Levin on Embodied Minds and Cognitive Agents

In this episode, Dr Michael Levin, Distinguished Professor of Biology at Tufts University, joins Nathan to discuss embodied minds, his research into limb regeneration and collective intelligence, cognitive light cones, and much more. Michael and the Levin Lab work at the intersection of biology, artificial life, bioengineering, synthetic morphology, and cognitive science. LINKS: The Levin Lab and Dr. Michael Levin’s research: https://drmichaellevin.org/resources/ Dr Michael Levin’s blog: https://thoughtforms.life/about/ Tufts University Faculty Profile: https://as.tufts.edu/biology/people/faculty/michael-levin Michael Levin @ Wyss Institute: https://wyss.harvard.edu/team/associate-faculty/michael-levin-ph-d/ Dr. Levin’s Research on Limb Regeneration: https://news.uchicago.edu/how-bioelectricity-could-regrow-limbs-and-organs SPONSORS: The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://brave.com/api Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off www.omneky.com NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. X/SOCIAL @labenz (Nathan) @drmichaellevin (Michael) @CogRev_Podcast Timestamps: (00:00) Preview (01:07) Intro and summary (05:40) Xenobots, anthrobots and the other creatures created by Mike Levin (09:39) Bioelectric memory rewriting (15:01) Sponsor | BraveSearch API (16:09) The difficulty of conducting simulations, which involve running forward passes to predict and alter electrical patterns (20:30) The concept of backpropagation and mode switching in AI models (23:06) Why humans do not regenerate their limbs (37:10) Sponsor | Netsuite (39:40) Learning from small and biological systems onto the concept of possible emergence (45:16) The criticality of multiple scale questions and would a single scale? (55:49) The concept of the cognitive light cone (59:43) Advice on habits of mind and suggestions for inspiration on the AI side (1:13:36) Mike's suggested directions for the AI developers (1:24:49) Wrap & Sponsor | Omneky This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Everything we do is around this notion of embodied minds and what it means to be a cognitive agent in this physical universe. And so we think about things like the collective intelligence of cells during embryonic development, during regeneration and so on. We've had projects in cancer where we can detect and normalize cancer by controlling its bioelectrical connections between the cells and the large-scale network. All of these terms, machine, human, robot, alive, emergent. I think what gets us in trouble is assumptions of that these are binary categories. Think about the whole spectrum of diverse intelligence. Cells, organs, tissues, chimeras, cyborgs, hybrids, and every other kind of combination.

• 0:45Speaker 00

  That's only going to continue. What are the things that matter to a bacterium? What are the things that matter to a dog? What are the things that can matter to a human? And could we have at some point a being that literally is in the linear range, can care about all the living beings on Earth? I think that any improvements that we can make to enlarge our cognitive lifetimes, to improve on our cognition, to have better embodiments, we can certainly do better than we do now.

• 1:07Nathan Labenz?

  Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together, we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. Hello and welcome back to The Cognitive Revolution. Today's episode is a special one. My guest is Mike Levin, professor of biology at Tufts University, and one of the most distinctively accomplished experimental scientists, and one of the most fascinating philosophers of intelligence that I've encountered in my life. We cover a ton of ground over the next hour and 15 minutes, and this is not an easy conversation to summarize.

• 1:53Nathan Labenz?

  If you're like me and usually listen to podcasts at 2X speed, I might suggest slowing this one down just a bit. Because unless you are already familiar with Professor Levin's work, there are simply so many strange results and deeply thought-provoking ideas in this conversation that I found my brain, at least, needed a little extra time to process it all. It might also help to watch some of the short videos that Levin and his team have released over the years, both on his YouTube channel and their academic group website. After listening back to this conversation myself, a few major themes do stand out. First, AI defies all binaries. I say that all the time, but Levin's work takes that sentiment to the next level by showing that even the most familiar binary distinctions, the ones that we take most

• 2:38Nathan Labenz?

  for granted, such as that between a living thing and a machine, are in fact rapidly collapsing from both directions, as we simultaneously see remarkably sophisticated behavior from even very simple computer systems on the one hand, and at the same time on the other, groups like Professor Levin's devising striking ways to program biology itself. Second, we have a lot of surprising latent capabilities left to discover. Just as GPT-4 can do far more now than we knew about at the beginning, so it is with biological systems, particularly when we place them way out of distribution, as Levin has done in his famous Xenobot and Anthrobot projects. Third, and perhaps most importantly, we must always favor epistemic modesty and open-minded

• 3:22Nathan Labenz?

  experimentation over our philosophical commitments. Many of the capabilities that Levin has discovered in biological systems would have been laughed off as impossible, if not crazy, right up until the moment that he demonstrated them. Reflecting broadly on how little we know about even the simplest systems with which we've co-evolved should be enough to keep anyone humble, particularly in the presence of new systems that don't share our evolutionary history. In Levin's view, the hot-button topic of emergence in AI systems, a subject that I plan to cover in depth with a dedicated episode soon, is inherently a subjective question and ultimately boils down to the element of surprise. I agree with this framing, and I was struck to hear that Levin, a world-class biohacker

• 4:06Nathan Labenz?

  by any measure, who is obviously willing to conduct experiments that some might consider playing god, still has consciously avoided working on ideas that could make AI systems significantly more lifelike than they are today, at least until he has a better understanding of what that might imply for their subjective experience and our collective future. As always, if you're enjoying the show, we'd appreciate it if you'd take a moment to share it with your friends. Considering how it brings such critical themes to the fore, I think this episode would be a great introduction to the cognitive revolution as a whole. And if I could be so bold, I think it is worth sharing widely. Now, with full credit to Professor Levin, his team, and the many marvelous intelligent systems that they study, I hope you enjoy

• 4:50Nathan Labenz?

  this paradigm shifting conversation with Professor of Biology, Mike Levin.

• 4:54Nathan Labenz?

  Professor Mike Levin, Professor of Biology at Tufts University, welcome to the Cognitive Revolution.

• 5:00Speaker 00

  Thanks so much. Happy to be here.

• 5:02Nathan Labenz?

  I'm really excited about this conversation. You do some of the most fascinating work, some of the most mind blowing work, really of anyone out there today. I think when people learn more about, if they've never encountered your work before, when they learn about some of the projects that you have, that you have developed, they're going to just think, wow, like, how did I not already know about this? And it's profound stuff on its own terms, but I think it also, you know, hopefully will be able to help us develop some new and kind of different perspectives on the future of AI as well. So I'm really excited to get into it. For starters, because I do suspect that, you know, we have obviously a lot of AI enthusiasts and builders in the audience, but I'm not sure how familiar many will be with your work.

• 5:47Nathan Labenz?

  Do you want to just kind of go through a few of your kind of most famous projects? I think if folks know any from you, it would be the Xenobots and the more recent Anthrobots that you have created. So maybe just tee things up by kind of telling us about these strange creatures that you have managed to create.

• 6:05Speaker 00

  Sure. Well, my lab does a wide variety of interesting things. I work with some amazing people that span the range from philosophy to biology to computer science to physics and so on. And everything we do is around, fundamentally, it's all around this notion of embodied minds and what it means to be a cognitive agent in this physical universe. And we study mind in a variety of unusual substrates. And so we think about things like the collective intelligence of cells during embryonic development, during regeneration, and so on. So I'll just kind of run down some interesting things we've done in the past. Let's see, one of the things that

• 6:47Speaker 00

  We specialize in is that we made the first molecular tools to read and write the electrical memories of non-neural cells. And so this collective intelligence of cells that builds organisms and so on, it has to store memories of what it's doing. And we now have ways of reading and writing that information. And so we've done things like look at how the different cells of a flatworm know how many heads the flatworm is supposed to have. And it turns out there's an electrical memory that stores that information and you can rewrite it, you can physically rewrite it. And then what you get is a worm that when you cut it into pieces, the pieces make two-headed worms forever. It's permanent. So that memory, like any good memory, it's rewritable, but it's stable. And so you get these two-headed worms. And of course, the genetics are untouched.

• 7:32Speaker 00

  So this question of where is the number of heads stored is really interesting because it isn't in the hardware. It's software rewritable. So we've done things like this. We've made tadpoles with eyes on their tail and those animals can see out of those eyes. And so this shows the remarkable plasticity of life because you can produce this animal with a completely different architecture of sensors and processor in the brain and it will adapt to a new connectivity of that system. We've had projects in cancer where we can detect and normalize cancer by controlling its bioelectrical connections between the cells and the large-scale network that can remember how to build large, complicated organs as opposed to being an amoeba-like cancer cell.

• 8:17Speaker 00

  As you said, we've created Xenobots and Anthrobots, which is looking at the plasticity of cells that normally will build very specific things like frog embryos or human bodies. And we've shown that if you put them in new environments and give them an opportunity to sort of reboot their multicellularity, then they can do that in new ways and have new and interesting capabilities that we didn't know about before. So all of these concepts keep coming up again and again, this idea of the software hardware distinction in biology. I know a lot of people are very suspicious about computer analogies in biology. We can sort of dig into which part of that is true and then which part of that is not. But yeah, those are some of the things we do.

• 8:57Nathan Labenz?

  So these are really incredible, pretty stunning results. I mean, some of the short videos and some of the images of these little living things or kind of quasi-living things are really just, you know, like, again, how did I not know about this previously? I don't understand how it works. I wonder if you could give me a little bit more, you know, kind of sense of how you're doing this control. So again, like you start with a worm that has this like remarkable regenerative capability where, you know, you can cut the worm in half, it'll regrow. So that's how the worm starts. Then you do the chop in half, but you apply some change and instead of growing the missing part, it will grow a second head or a second

• 9:39Nathan Labenz?

  tail, you know, and you can control that. And again, the tadpoles with the, the eyes on the leg.

• 9:44Speaker 00

  From the tail.

• 9:45Nathan Labenz?

  The tail. Thank you. Can you give us a little bit more intuition around like, what are you doing there? I've, I think I've heard you use the term like executing stored procedures, but I'm still kind of at a loss as to the details of the interventions that you're actually making and would love to understand that a little bit better.

• 10:04Speaker 00

  Sure. Um, yeah, there's a, there's a variety of techniques, but let's just, let's just talk about the bioelectric memory rewriting. So I'll give you, I'll give you a, a kind of a simple story about the workflow that, that we're doing. So you've got a, you've got a flatworm and it has a head and a tail. And these flatworms are actually an amazing model because not only are they incredibly regenerative, but they also, they're immortal and they have no obvious lifespan limit. They, you know, the asexual ones that we work with don't seem to age at all. They're highly cancer resistant. They, they can learn, they can form memories. In fact, you can, you know, you can do the old McConnell experiments, which we have done where you train them, cut off their heads, they regrow a new brain and then they still remember the original information, right? So this incredible ability to move information around from tissue

• 10:48Speaker 00

  to tissue in the body, right? And imprint it on the new brain. So they do all these cool things. But if you, if you cut them in half, so imagine you've got a worm like this, you've got a head here, you've got a tail here, you cut it in half, something amazing happens, which is that the cells on this side of the cut, and so the head is over here. So the cells on this side of the cut end up growing a tail. The cells on this side of the cut end up growing a head. And so now you've got two normal worms, but they were right next neighbors to each other. Why did they have completely different anatomical outcomes? And how is it that a fragment, right? So you can cut the worm into fragments. How is it that a fragment knows exactly how many heads it's supposed to have? One of the first things we did was to Well, well, just to realize that that the algorithm of knowing should you be a head or a tail cannot be local because these cells were at the same location.

• 11:33Speaker 00

  So you know right away that from knowing where you were, you cannot tell whether you should be a head or a tail, you have to talk to the rest of the tissue. And so for for many reasons, my hypothesis was that that conversation with the rest of the tissues was electrical in nature. So the very first thing we did was simply prevent it. We said, what happens if you're cut off from the electrical network and you really can't find out. So in that case, what what you would do is there are drugs, specific pharmacological compounds that you can put on these worms in the water that they're regenerating in that control how well the cells can talk to each other electrically. That's that's one of the things we started with. Then we applied something called voltage sensitive fluorescent dyes. So these are just molecules that are developed by other people, so chemists that develop these things.

• 12:17Speaker 00

  You throw them on to the sample and they will fluoresce. So you use a microscope to detect the fluorescence and they fluoresce differently depending on the voltage of the cell that they happen to be sitting in. So you get this, so right away you get this map across the entire tissue. I'm making it sound very easy. It's actually an incredibly sort of time consuming and difficult process. But eventually you get you get a map of where all the voltages in this tissue are. And what we noticed is that we could interpret this map to say where the head and the tail was going to be in the future. In other words, it's really a kind of pattern memory that tells you where it's going to be. And so once you've seen that, it becomes pretty obvious to then say, well, what if I change that pattern? And what if I now I can see that it says one head? Well, I would like it to say two heads.

• 13:02Speaker 00

  What will happen? And so now you have an electric circuit. And what you can do is you can make a because because so you can make a computational model of that electric circuit in the tissue. And so we do a lot of computational modeling. And you can ask that that model, what do I need to do to change the pattern so that it now says two heads instead of one head? And so by by playing around with that model, you can say, ah, I need to close some chloride channels over here, or I need to open some potassium channels or something. Because in all of these things, what we're doing is we're not applying any electric fields where there are no magnets, no waves, no radiation, there's no electromagnetics, none of that. What we're doing is hacking the natural interface that these cells use to control each other's behavior.

• 13:46Speaker 00

  That that interface is a bunch of electrical. Electrogenic proteins known as ion channels, which sit on the surface of the cell, they produce a voltage, and the next cell can feel that voltage and they communicate to each other. So it's very much like what happens in the brain, actually, right? It's very similar. So what we're doing is using either drugs or optogenetics, not in the worm, but in other cases, we've used optogenetics. So that's light-based, opening these channels on and off. And there's some other molecular biology kind of tricks, but the idea is to open and close the existing channels on these cell surfaces. So you can think about it as an interface. It's like a keyboard or any other interface that you would have to the programmable layers underneath your machine. That's what this electrical system is.

• 14:29Speaker 00

  So guided by, you know, originally just designing these cocktails in our heads, but eventually with a computational model, we can pick drugs that are going to open and close these channels in the right way to give you the pattern that you want. And then you soak your fragment of the planarian in that drug. You know, typically you do that between, you know, minimum of three hours, in some cases, 24 or 48 hours. And then you leave it alone and you wash it out, you put it in regular water and you see what it does. That's a typical workflow. There's some others, but that's typical.

• 15:00Nathan Labenz?

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 15:05Nathan Labenz?

  It's striking how similar that is in some ways to some of the mechanistic interpretability techniques that people are developing to try to understand how AI systems work. It really sounds a lot like activation patching, which is where, and again, it's often done in toy models. So, you know, again, there's a notable similarity. People will take the model, and of course, with an AI, it's all, it starts digital, right? So you have the, you have a lot easier path to looking inside it than you do in a biological system. But that doesn't mean it's any easier to know what's going on inside, because it's still just, you know, a lot of numbers flying around. And, you know, what does it all mean is not easy.

• 15:47Nathan Labenz?

  But doing the simulations, it sounds like you're almost kind of... Running a lot of forward passes, so to speak, in your simulation, you're kind of saying with this electrical pattern, like, what do we expect to happen? And then if we perturb that pattern, what would happen differently? Is that the, is that the sort of counterfactual modeling you're doing?

• 16:06Speaker 00

  We have several different levels of models. So some models are exactly what you just described. So it's a, it's a, the model only feeds forward and all we're able to do is run multiple counterfactual scenarios until we see what we got. In some cases, we can actually run it backwards and we can say, actually, if I want this result, what would I have had to do? And of course, now we're working to integrate modern AI methods so that you can actually train and have, you know, have an AI instance. I will actually make guesses about what to do. That's, that's the next step. In all of these cases, uh, so, so, so this is very much an interpretability issue because we don't know what the native representation encoding is.

• 16:46Speaker 00

  So it is not obvious how the different voltage gradients are mapping onto whatever happens next. It's very parallel to neural decoding in the brain where you read brain states from a, from a patient or from an animal model. And then eventually you try to, you try to guess, well, what was he thinking about when, you know, when that, you're trying to infer semantic states from the physiology, from the readout of the, of the network. So, so that's what we're doing and it functions on many levels. And so you can measure individual cells and get metrics of, is that a stem cell? Is it a cancer cell? Is it a mature differentiated cell? You can do that. You can take measurements of whole tissues and you can say, um, are you going to be an eye or are you going to be a leg?

• 17:28Speaker 00

  Or, you know, what, what, what's the shape or the face? Like, what's the shape of the face? Where do the eyes go? And then you can have very high level information where we have, we've, we had this project in the frog that, frogs, unlike salamanders, they don't regenerate their legs. So if they lose a leg, what we're able to do is go in with a, with a bioelectric, um, drug that triggers leg regeneration. But what's, what's cool, there's two things that are cool about it. And of course, now we're sort of trying to move that to mice. And so eventually, hopefully, humans someday. But, but the idea, what, what's, what's interesting is that. First of all, the intervention in the latest experiments, the intervention is 24 hours. The leg then grows for 18 months. So it isn't a micromanagement, the kind of thing.

• 18:11Speaker 00

  We're not sitting there telling every cell what to do, telling all the, you know, all the genes which comes on, which comes off. It's a very early communication to the leg that says, take this path in morphous space towards a nice leg, not this path that leads to scarring. And that's the end. And then the other thing is that the exact same intervention that regenerates legs in adult frogs causes regeneration of tails and tadpoles. So what that means is that the information content is not really in the intervention. We don't give it all the information needed to make a complex leg or a complex tail. We say at that, in that particular application, we say, build whatever normally goes here. So you're offloading a huge amount of information onto the system of the task, onto the system itself.

• 18:53Speaker 00

  We, all of this works because the system knows what to do and it is our job to convince it that it should do one thing versus that versus another. So it's very much so with all these different layers, you know, we can look at the cell layer, we can, you know, we could look at the tissue, the organ, and then the whole, the whole kind of, in fact, in our latest paper. That came out just yesterday, we showed that even groups of embryos have their own decision-making, collective, collective decision-making that individuals can't do. And so, yeah, it's, it's very much an interpretability question to understand how, how does the system understand its own states? And then how do we as scientists understand them as well?

• 19:26Nathan Labenz?

  There's so many connections, even, you know, just listening to your, your comments there. When you talk about running things backward, you know, that obviously reminds me of back propagation, but kind of in, you know, very similar way too of answering the question, what would have had to be different in the upstream signals in order to get the desired downstream outcome? I mean, that's really the kind of tweaking that's, that's going on in the, the AI models and the back propagation process. Also, mode switching, you know, is an interesting, when people train the, the current chatbots to like refuse your, you know, bad requests, right, for whatever the definition of bad is, that they want to exclude from the model behavior.

• 20:08Nathan Labenz?

  There seems to be this kind of early fork in the road that could be characterized in different ways, but mode switching is one label that has really stuck with me. Where if you can get the AI to say, oh, sure, happy to help with that, then it will continue to do the bad thing against its training. Whereas if it starts with, I'm sorry, as a large language model trained by OpenAI, I can't do that, then it's never going to do it. It's striking that there's something similar at the level of limb regeneration, where basically, if I'm understanding you correctly, you kind of have this early fork in like, which mode are we going to go into? And that intervention lasts for basically two orders of magnitude, you said like one

• 20:53Nathan Labenz?

  day to like 100 days essentially that you can, where the effect persists? Did I catch that right?

• 20:59Speaker 00

  Yeah, yeah, it's 18 months, 18 months. So, yeah, I really think that there are a lot of similarities here in terms of, if you think of the kind of the classic A and N structure, where you have the different layers that are progressive abstractions of the input that have come in. So I think the higher you are on that, as somebody who works in regenerative medicine or bioengineering, I think that you want to be as high as possible on that level for control, because I don't want to have to tell you which genes to turn on. I don't even want to have to tell you which types of tissues go where. I just want to say, you already know what goes here, just rebuild it, that's it. I want to have the minimal, kind of the simplest trigger.

• 21:43Speaker 00

  And the systems decide that the decisions are made very early on what it's going to do. And then after that, there's a cascade. Once, it's exactly like you said, once you're going down a particular road, it becomes much harder to make changes. So you want to do it as high up in the decision hierarchy as you can.

• 22:00Nathan Labenz?

  Yeah, I mean, that's another just striking similarity is the existence of surprising capabilities. You know, GPT-4, this has been remarked on a ton, right? It was trained, training was complete 18 months ago. It was released like 10 months ago. We're still seeing new state of the arts set with people just prompting it in ever more sophisticated ways. And revealing kind of capabilities that nobody quite knew existed. Here, it's striking to me that, obviously, you haven't demonstrated this in humans yet, but you're working your way up the, the sort of, you know, complexity of organisms ladder. And I guess this is maybe an ignorant question, but like, why doesn't it happen?

• 22:44Nathan Labenz?

  You know, why do we not regenerate our limbs? If we have this capability, why is it seemingly like never expressed? Or is it maybe, is it sometimes expressed? Or are there examples of people who have done this? I've never heard of anyone regenerating a limb. It's crazy that to think that that capability is latent.

• 22:59Speaker 00

  Yeah, so, so children, human children regenerate fingertips. So, so up until a certain age, somewhere between seven and 11 years old, kids will regenerate their fingertips. We don't know why humans don't regenerate their limbs. I can tell you a story that might make a little bit of sense, but, but it's just a story we don't know for sure. So, so here's a story. Imagine, imagine that you're a mammalian ancestor. So you're this ancient, you know, mouse-like creature. You're running, running around the forest. Somebody bites your leg off. So now here's the problem. A, you're going to try to put weight on it because you're, you're, you're a tetrapod. So you're going to try to put weight on it. You're going to grind that, that wound into the forest floor. It's going to get infected. It's going to, you're, you might even bleed out.

• 23:44Speaker 00

  And so unlike a salamander, which has the ability to sort of float buoyantly in peace and quiet for, you know, for, for months. You really don't have the luxury of hanging out and regenerating. Your best strategy is to seal the wound, scar, have some inflammation and, you know, live on hopefully to tell the story. Now, there is one example of a mammal that does amazing regeneration and it's an appendage that you don't put weight on. So it's deer antlers. So in deer, they grow these massive structures, bone, vasculature, innervation, velvet. They grow a centimeter and a half per day of these things when they're growing out. I mean, it's crazy. And they can do it. And is it a coincidence that that's the one thing you don't put weight on? I don't know.

• 24:27Speaker 00

  I do want to say something about your point about unexpected capabilities. I think this is really critical. And we, I don't know if you've seen this thing, but a few weeks ago, we put out a pre-print. This is a purely computational study. What I wanted was, and this was done with my student, Han Ningzhang and Adam Goldstein. We wanted to, I wanted to really hit this issue of unexpected capabilities because working in diverse intelligence research, which I think is hugely important and completely basically unknown in the AI community, and I think it's a real problem. I think a lot of the answers to things that people have been sort of debating in AI really have their origins in diverse intelligence research.

• 25:12Speaker 00

  But one of the kind of fundamental aspects there is that you can find intelligence, AKA problem-solving capacities, in very minimal unconventional systems. Really, the idea that we do not have a good intuition for what to expect. When we build systems, we don't know, never mind immersion complexity. That's easy, you know, fractals, game of life, cellular automata. Complexity is easy. But what also tends to happen is there's emergent agency, so ability to pursue goals and solve problems. And we are terrible at noticing these things when they're in unfamiliar substrates. So what we did in this paper. Was I wanted something that was extremely simple, transparent.

• 25:55Speaker 00

  The thing about biology is that in biology, there's always more mechanism to be discovered. So no matter what you show, somebody will say, well, there really is a mechanism for that. You just didn't find it yet. Right. So we wanted something super simple. And what we chose were sorting algorithms. So these things that computer science students have been studying for many decades, you know, bubble sword, selection sort, that kind of stuff. And completely deterministic. Everything is right there. It's completely open, you know, six lines of code. There's really nowhere to hide like it's all, it's all there. And what we were able to show is that if you treat them, if you're, if you're a little bit humble about what these things can do. And you ask questions about what they can do and rather than making assumptions, that they only do what the algorithm tells them to do.

• 26:36Speaker 00

  You actually find some really important capabilities that are nowhere in the algorithm. They're sort of implicit. So there's the explicit algorithm that sorts a list of numbers and that's there and you can't get away from that. They will, in fact, sort lists of numbers. But it turns out that they have some really interesting properties and some, uh, some, some novel capabilities that we did not know about. And so I think that if that's the case for these really minimal dumb sorting algorithms, then something as unique and novel as these large deep networks and all the other stuff that is made in AI. I don't think we've even scratched the surface of what's really going on there. You know, that the, the, the capacity for surprise in even small systems is, I think, massive.

• 27:20Nathan Labenz?

  Okay. So I have to ask, what can the sort? Uh, I haven't seen this pre-print yet. What can the sort algorithms do that is not obvious?

• 27:28Speaker 00

  I'll give you two examples. So, so just to, just to introduce this, this story, uh, the typical sorting algorithm is you sort of have this, uh, this central godlike observer who sees the whole string. And under some algorithm, he's, you know, moving around, moving the numbers around. Right. So we made, we made two changes to be able to study this. One, one is that we said, uh, to make it a little more biological, we said, well, Instead of having a central algorithm, we're gonna, we're gonna do it bottom up in a distributed way. So every, so, so you've got an array of numbers, every array element is, we're gonna call it a cell. That cell has some numerical value from zero to a hundred assigned to it. They start out randomly, you know, randomly mixed up. And what we're gonna do is every cell is gonna follow it, so the algorithm.

• 28:11Speaker 00

  So for example, every cell, it, it doesn't see the whole string, but it wants the neighbors. So the five wants a four on its left and a six on its right. And, and, and every cell wants that. And we're just gonna, you know, we're, we're gonna, we're gonna let every cell be, be, be its own, its own agent. So, so the first thing you find out is that if you do that, it still works. They actually sort themselves quite, you know, quite well. The next thing we did is we said, okay, now we're gonna let go of the assumption of reliable hardware. In other words, if the algorithm says swap the numbers. Well, they might be broken, one of the cells might not swap. In the standard algorithm, you never check whether in fact the numbers got swapped the way you wanted them to. Nor do you check, how are you doing? You don't do any of that, right? You assume the hardware is reliable and you just like, that's it. We, we kept all that.

• 28:54Speaker 00

  We did not introduce any new code for any of the things that I'm about to tell you. They do, we didn't put in any new code. So the code is exactly the standard stuff that everybody studies, just being run individually on, on every cell. So the first thing that happens is that if you introduce defects in the string, so you introduce broken cells.

• 29:15Speaker 00

  It still does a really good job sorting. In fact, it will sort other numbers around the broken numbers if it can. Again, there's no code in there to say that, hey, was this one broken? Did it move? No, no code about that. One of the interesting things about intelligence and trying to estimate intelligence is this, and this was, uh, William James had this example. He said, think about the, the spectrum between two magnets trying to get together and Romeo and Juliet trying to get together, right? Imagine you put a piece of wood between the two magnets. The two magnets are, are just gonna stand there pressed up against the wood. They're never gonna go around because in order to go around, you have to temporarily get further from your goal. Let's call that capability delayed gratification.

• 29:57Speaker 00

  So this idea is that I'm going to reduce the thing that I'm being, um, opt, the thing that I'm optimizing. I'm actually gonna, gonna reduce that in order to acquire gains later on. I mean, that's a, you know, it's not super high intelligence, but it's a, it's a, it's an ingredient to be, being able to do that is an ingredient. Otherwise, you're just a very simple gradient follower, you're not gonna get too far, right? Unlike Romeo and Juliet, who have all kinds of tools, cognitive tools, you know, to get around social and physical barriers, planning, all this stuff. So delayed gratification. So it turns out that if you actually track as these algorithms are sorting the numbers, when you introduce these broken cells, which if you visualize, well, one of the things we did was visualize the sorting process as a journey in sort space. So they all have to get to one point where everything is sorted and they start out at

• 30:42Speaker 00

  different points, but it's like this path and they all take these different paths. A broken cell is basically a barrier in that path. So you're walking along, then you want to move the cell and you can't. It just doesn't move. How are you going to get around this barrier? It turns out that these algorithms, and some more than others, we looked at, I think, four different ones, some of them have the capacity for delayed gratification. What they do is they'll go move some other numbers around and in fact, the sordidness drops for a while. The string gets less sorted for a while and then they catch the gains later, right? It becomes better later. Now, this is already quite amazing because there is nothing in that algorithm that explicitly, I mean, if you just look at the algorithm, there's nothing in there that explicitly says you have the capacity for delayed gratification. And they do this more when there are more barriers.

• 31:26Speaker 00

  In other words, they don't just randomly back up and, you know, sort of wander around. No, they're extremely linear until it comes time to deal with a barrier and then they sort of dip down and come back. So that's one kind of capability that we found, that they're actually able to move around barriers like that without any, you know, explicit code for it. The other amazing thing is this. Imagine that once you've put the algorithm in the individual cells, you can do a really cool chimeric experiment, meaning that you could have cells that are running different algorithms. So you can mix, so let's say some cells are running selection sort, some of them are running bubble sort, for example. And the thing is that

• 32:06Speaker 00

  None of the algorithms have any code to know which one they are. So they don't, they don't have any data about what they are, nor do they have any ability to look at my neighbor and see what he's doing. You're just following your algorithm. You have no idea what it is. You're just following their algorithm. Now, now imagine at the very beginning, we have a hundred cells and we mix it randomly. Right? So every cell has now two properties. It has the number that it's trying to sort and it has which algorithm it's following. Okay. So there's two, two types of, two types of cells. They're randomly distributed. And, and all of this, by the way, has developmental biology consequences because the ability to sort out tissues is, is, you know, animals do this. So frog, embryonic frogs, if we make, we call this a Picasso frog. We, we, we start with a tadpole with all the organs in the wrong place.

• 32:50Speaker 00

  All, everything will sort out and you'll get a very nice frog out the other end. So then I will get back to where it needs to be. The jaws will come, like they know how to sort themselves out. And, and we make chimeric animals too. We make what we call frog lottles. So it's got a bunch of frog cells and a bunch of axolotl cells. And you can actually ask this different hardware that normally makes different things. How do they, like, what's it gonna make, right? So, um, so, so imagine we, we have this chimeric string and then we're gonna ask a simple question. What is the degree of clustering at any point in time? In other words, what's the probability that when you look to the cell next to you, it's the same type as you are? And Adam actually invented this term, which I like a lot called algo type. So algo type, so there's genotype, you know, which is, which is the, the algorithm that you're following.

• 33:33Speaker 00

  There's the phenotype, which is what actually happens in biology. And then the algo type is like, what algorithm, you know, what algorithm are you actually running? So, so what's the probability as I look, as I look at my neighbor that he's the same algo type as I am? Now, initially at the beginning, it's 50% because we assigned algo types to numbers randomly. So you know, it has to be 50%. Yeah. At the very end, it's also 50% because at the very end, everybody's got to get sorted in order. And the assignment of algo types to numbers was random. So of course it's going to be 50% again, right? Because you've now reshuffled everything in order, but there was no pattern. So there's still not going to be pattern, 50%. So if you imagine this graph, so, so 50% here, 50% here.

• 34:14Speaker 00

  But during the sorting period, it actually goes like this. And what it means is that in the middle of the sorting period, they are, they sort together. So, so, so common algo types like to hang out together. Now, this is, this is kind of a weird way to, to, to, to think about it. But to me, it's almost like a minimal model of, of the human condition. It's like, eventually, the physics of your world are gonna pull you apart. Because, right, because, because the, the, the, the, the sort, the actual sorting algorithm is inexorable. You can't get away from it. So eventually, you're gonna, you're gonna get, you know, you're gonna get yanked, yanked apart. But up until then, you have this life that, that allows you to do some cool things that are compatible. They're not, they're not, um, you know, they're not directly forced by the laws of physics,

• 34:58Speaker 00

  but they're compatible with them. And you get to do this thing where you hang out with your buddies for a while, until you get, you know, you get sort of, sort of yanked apart into, into what the, what the, um, what the physics is trying to do. And so that's something that's completely not obvious from the algorithm. You'll, you'll, you'll never know, looking at the algorithm, that, that that's what it was going to do. I have a gut feeling that this can be harnessed in various ways. Uh, you know, uh, the fact that it's, the fact that it's also clustering, it means that you can get multiple, um, work out of the same algorithm. It might be doing other things. And again, this idea that even something as simple as the sorting algorithm has this other property that we wouldn't have guessed until we checked. Right, until we, until we try, you know, we, we, we actually ask, what, what, what do you act,

• 35:42Speaker 00

  what are you actually doing, um, besides the thing we asked you to do? You know, what, what else is baked in? And, uh, yeah, that, that kind of emergent ability to maximize other outcomes besides the ones that, that are explicitly programmed in, I think, is, is probably all over the place, both in biology and in, in, in AI.

• 36:01Speaker 03

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 36:05Nathan Labenz?

  There are so many different, uh, follow up questions I want to ask about all that. At least kind of three big themes, though. One is, and I know you've developed this in different papers and different venues, but definitely very relevant, I think, in the AI context, is this sense that...

• 36:23Nathan Labenz?

  The once supposedly very clear distinction between life and machine, maybe was never really properly so clear in the first place, or at a minimum now, seems to be kind of blurring. I'd love to hear you talk about that, especially as it relates to this notion of emergence, which is one that I don't know how closely you're following the AI discourse, but intense debate right now about emergence, you know, how to conceive of it, what should count, is it a mirage? Actually, a best paper award at NERUPS, the recent, you know, major AI conference, was given to a paper arguing that emergence is a mirage, which I think is honestly, put my cards on the table, kind of missing the point on a few key levels.

• 37:08Nathan Labenz?

  But you could talk about this, I'm sure, for 10 hours, but especially with this kind of eye toward, people are like, oh, well, it's AI. It's just computer code. It'll never do anything unexpected. We can control it, right? And you are really, even with some pretty simple systems, calling that into question. Yeah, I guess you can react to that however you want, but I'm very interested in how you project what you're learning with these small systems and these biological systems onto this notion of possible emergence in AI.

• 37:35Speaker 00

  I'll be kind of philosophical for a moment and then let's talk about definitions a little bit and then I'll give some practical examples. All of these terms, machine, human, robot, alive, emergent, what are all these terms for? What are they supposed to do for us? So I take all of these terms as engineering protocol claims. So what I hear, I don't, I think they're all mirages in an important sense, all of it, in an important sense. I think all of these terms are not objective truths. I think they are claims about the utility of a particular worldview from the point of view of some perspective, from the perspective of some other agent, including the system itself, by the way. So cognitive systems have to have models of themselves.

• 38:20Speaker 00

  All of these things are different models of what's going on. I think that emergence is basically a kind of expression of how of surprise in an observer. So if you knew something was going to happen, you don't think it's emergent. If you were smart enough to predict that in advance from knowing the rules about the parts, then to you, it's not emergent. To somebody else who couldn't predict it, it's absolutely emergent. And so, so I don't think, I don't think these are binary categories. I don't think there's a true, you know, kind of an objective truth as to whether something is emergent or not. I think everything is from the point of view of some observer. So now, all of this business about machines and, and, you know, and, and, and living things and so on. Look, you, you do not want an orthopedic surgeon who doesn't believe that your body is a simple machine.

• 39:05Speaker 00

  If you watch, if you, if you look at what orthopedic surgeons do, they got hammers and they've got chisels and they've got nails and screws. And, and absolutely, they treat your body as a, as a machine. Okay. And, and you want them to, that, that is the right, that is the right frame for what they're trying to do. Do you want a psychotherapist that thinks you're a machine? You do not. And so, so right, so, so there are, there are different levels of, of, of this framing. And this is, so, so I've got this thing called the TAME framework. T A M E stands for technological approach to mind everywhere, which begins by setting out a, um, a spectrum all the way from mechanical, you know, clocks and things like that, all the way up to humans and all kinds of things in between.

• 39:43Speaker 00

  Where what's different between them is it's not what they're made of and it's not how they got here. It's not whether you were engineered or you were evolved or, or if you're squishy and, you know, and, and alive. I really, as it's, it's weird to say, as a biologist, I have very little interest in whether something's alive. I'm not even sure that's a useful category, really. I think now, now, look at now, now the level of cognition, that's super interesting. And, but I don't think it necessarily tracks with being alive at all. And, um, what, what's, what's, what happens when you, when you move across that spectrum is the, you change tools as far as how some, as how some other observer is going to interact with you.

• 40:20Speaker 00

  So the way you interact with mechanical clocks is very different than the optimal way to interact with cybernetic devices like thermostats versus learning agents like, you know, animals and some other, you know, some robotics versus humans and so on. So are, are we machines? Yes. Are we amazingly gentle creatures that do things that simple machines don't do? Absolutely. Yeah. You know, are aspects of our psychology robotic? Sure. Are there aspects that are, that are not? Yep. Uh, you know, all, all, all of it. It's, it's, it's, I think what gets us in trouble is assumptions of, uh, that these are binary categories. I think, I think there is no such thing. I, I think these binary categories don't exist. And and assumptions that all of this is has an objective answer that we should just discover what it is and then and then we're done.

• 41:05Speaker 00

  And even worse, some people try to make decisions on this from a philosophical armchair. In other words, they look at something, oh, well, that's just, I see what that is. That's just physics. That's not cognitive. Like, well, you have to do experiments. You can't, you can't just have feelings about where something is on that spectrum. You have to do experiments. For example, we took gene regulatory network models, which are extremely simple, either Boolean or ordinary differential equation networks. And again, deterministic, very simple, you know, very simple, nowhere to hide. We show that they can do six different kinds of learning, including Pavlovian conditioning, right? Just out of the box. And you wouldn't know that, you know, if you, if you, if you have this commitment at something that's that looks like that has to be stupid, you wouldn't know that. And who knows what else they can do.

• 41:48Speaker 00

  So I feel very strongly that this is, these are all empirical questions. And if you can find a frame that you've taken from behavioral science or from, you know, one or from cybernetics or something, and if you can usefully apply it to that system, then there it is. Then, then, then you found, you know, you found a good way to deal with. So, so, yeah, I don't believe that this, this machine life distinction is valuable at all. I've never seen, I've never seen anything really useful, useful come from trying to enforce a binary distinction like that.

• 42:18Nathan Labenz?

  One of the things I think is really striking about some of your findings is just how small scale intelligence can be, you know, whether it's a clump of cells, you know, that constitutes a xenobot or an anthropobot, or it's even just, you know, apparently a, you know, virtual cells in a, in a, running a little sorting algorithm. But it starts to get confusing, I guess, when it's like, okay, how small can we get? You know, is there, is there some nature? A lot of these systems, maybe not all, but a lot of them do seem to have like multiple scales. So I'm kind of wondering, does like, is that multiple scale question critical? Could you have something that just works at a single scale? What would that even look like?

• 43:03Nathan Labenz?

  And also, you said this, you know, this framework for mind everywhere. Do you have any intuition for how this relates to? You kind of said, you know, everything is subjective. Does that also imply that there is some subjective experience on behalf of these systems? Like, do you think these little sorting cells have any sense of experience? I think these questions are also, you know, very under theorized in AI. People are today just like, oh, well, it's an AI. Of course it doesn't, you know, have any experience or have any moral value. I'm not rushing to say that they do, but I'm also like, you seem very quick. You know, everybody seems so confident in that and I'm kind of like, not so confident.

• 43:41Speaker 00

  Yeah, yeah. Though, well, the one thing I can say for sure is that we absolutely cannot be confident because we do not know. I mean, I hear people all the time making these pronouncements that definitely is or it definitely isn't. No, we, we, we do not have a principled way of answering these questions for the biological world. And that means we have, we are completely out to sea when we're faced with unconventional embodiments. And we know, we know this, you know, um, science fiction has been at this for, for well over a hundred years. The idea that you, well, when something shows, you know, lands on your front yard and sort of trundles out and, and it's kind of shiny. But also it's giving you a poem about um, how happy it is to meet you. And it's kind of got wheels and you're not quite sure where it came from. But also it's, you know, it's, uh, you're having a great conversation with it.

• 44:22Speaker 00

  Like, well, what, what are you going to use as a criterion for, for how you're going to treat it and so on? So all of the old uh, categories that we used to have in terms of, well, did you come from a factory? Or did you come from the process of a random uh, mutation and selection, right? Or that, those kinds of things, these are all terrible categories and, and they're, they're not good ways to, of making that distinction. Let's uh, let's, let's, let's talk about the first question about um, how, how low down does it go? I mean, people even uh, there's, there's um, minimal active matter research where people can make uh, very simple systems out of like three inorganic chemicals. And then they can solve mazes and they can do interesting things.

• 45:01Speaker 00

  So I think that all of this really becomes disturbing when you insist on a binary category. When you want to know, okay, is it cognitive or isn't it? And then, then, then you've got a real problem because look, each of us starts out life as a single unfertilized oocyte. It's a thing, it's a little blob of chemistry. Uh, it's a, we, we, we look at it and we say, okay, very clearly this does not have um, the kinds of cognitive capacities that, that humans have. Nine months and maybe a couple of years later, you've got something that obviously does. Developmental biology offers no sharp line during this process where lightning bolt says, "Okay, now you've gone from physics to mind." Okay, there is nothing like that. The whole process is smooth and continuous.

• 45:44Speaker 00

  So there is no escape. So all of this, these continuum, first of all, there's the continuum of development. Then there's the continuum of evolution. If you think that you as a human have some very, you know, sort of specific competencies, just start walking backwards. So which of your hominid ancestors and which of the, you know, sort of mammals before you and before that, and eventually you get to something that's basically a, you know, much less complex than a current, single cell organism. Where did those things peter out, right? There is no good story about that. It is imperative that we understand that it's not, are we cognitive or aren't we? It's how much and what kind, right? It has, it cannot be binary. There's no way to support. Not only that, now with bioengineering and robotics,

• 46:29Speaker 00

  we can make combinations. So if you think machines are not whatever, and humans are, fine. So right now we have cyborgs walking around. We have people with microchips in their heads and various other, you know, insulin pumps and various other things. That's only going to continue. Now, right now, you're dealing with, I mean, talk about non-neurotypical. Like right now, you've got somebody that's, you know, 98% human, maybe 2% digital. Eventually, you're going to get to 50/50 and every other kind of combination. What are you going to do that? Are you going to do it by weight? Like what percentage of you is original parts and what percentage of you is cyborg? It would be ridiculous, right? So there is absolutely no way to maintain this hard distinction. An interesting question might be,

• 47:11Speaker 00

  is there anything in this universe that is, that is zero on the cognitive scale? Okay, because I think, I think very simple things already are not zero. Now, the question is, is there a zero? So this is a hard question. I'll tell you what I think about it right now. Let's ask this. What would the most simple, the most basal, like the most basic versions of agency look like? What do you need for that? Now, it's obviously not gonna, you know, people, people say, well, you think that the rocks have hopes and dreams like us. No, you have to scale down. The point isn't that it's going to have our level of cognition. What does the most minimal level look like, right? The smallest possible. Well, I think you need two things for that.

• 47:53Speaker 00

  You need some degree of goal directedness. And that in William James's definition is the ability to reach the same state by using, by different means. Okay, so same goal by different means. So you need some degree of goal directedness. And you need to be not completely, your actions need to be not completely explainable by current local conditions. So if what you're going to do is completely determined by all the physical forces acting on you right now, then you're probably some sort of billiard ball and that's, you know, that's it. So those two things, I think even particles have those. Because least action, least action kind of laws in physics tell you that there's goal directedness baked into the bottom levels of the universe.

• 48:35Speaker 00

  And quantum indeterminacy gives you a really dumb version of not being predictable by local conditions. It's not great cognition because it's random. That's not really what we like from, but it's something. And so, and so here's, here's what I would say. If there is any kind of a definition to life, I think life, we call life those things that are really good at scaling those up. So if you've got a rock, it has no more capabilities than the parts that went in it. Didn't scale. It's sort of lateral and it's bigger, but that's it. If you've got something that's alive, it's cranking up the agency, the indeterminacy and the goal directedness across every scale of organization and its cognitive light cone is increasing, you know, as a function of time.

• 49:16Speaker 00

  So my friend and colleague Chris Fields, who's a brilliant physicist, among other things, I asked him, so can you have a universe in which there is no least action and which like there would be zero? And he said that only, you know, in a universe where nothing ever happened. Right? In a completely static universe where nothing ever happened, it would be zero. But as soon as you get you get interactions already, you know, you've got you've got the basics of the least action. So so I believe in this universe, I don't think there is a zero. I think everything has some capacity, but some things scale it up in an interesting way and some don't. And that means that for some things, the tools of behavioral science are going to be applicable. And for some things they're not, right?

• 49:58Speaker 00

  And that's the empirical that that's that's why this is not the same thing as ancient animism where you saw a spirit in every rock. Because it actually does not pay off to treat rocks with tools from behavioral science. But it does pay off to treat gene regulatory networks that way. And it absolutely pays off to treat various animals that way. Because that's how humans train dogs and horses, knowing zero neurosciences. Because there's this there's this amazing interface that some animals expose that that you can you can train them. So, yeah, and then and then I guess the last thing, the last thing you talked about is is about inner like this inner perspective. I I try not to say too much yet about consciousness per se, but but we can we can say something about inner perspective. And which systems have it and which systems don't. Here's what I think.

• 50:42Speaker 00

  Imagine imagine a landscape, you know, kind of a hilly up and down landscape. And you got a bowling ball on this landscape. In order to know what this bowling ball is going to do and to make it do whatever it is that you want it to do. All you have to do as an external observer is to pay attention to your your view of the landscape. Your third person view of that landscape tells the whole story. You know where the hills and valleys are. That's all you have to know. You don't need to take into account anything else. Now imagine a mouse on that landscape. When you got a mouse on that landscape, your view of that landscape is not really that relevant. What's really relevant is the mouse's view of that landscape. Because he might have been rewarded and punished in different at different times and different locations. He might have he's he's seeing that landscape in a completely different way.

• 51:26Speaker 00

  There's a there's a, you know, there's a valence map sort of superimposed on it that he's got his own opinions about where he's going to go. And so for all these different systems, you can sort of ask. To what degree, and again, it's not going to be a binary thing, it's never going to be a binary thing, but you can ask, to what degree do I need to take the perspective of that system and ask, what does it see and what does it think about what it's going to do? So that to the extent that you have to do that a lot, you're dealing with a high agency system that has an inner perspective. To the degree that you can get everything done by your own model of what's going on, then probably it doesn't. Right? And so, and so that I think is a, is a, is a heuristic by which we can start to say, um, how much inner perspective can I expect from these things?

• 52:05Speaker 00

  And I, I really, you know, as far as being, being sure that, I mean, look, let's be clear, I am not claiming that today's AI architectures are mimicking human brains. Okay. I don't think they are. I don't think they're mimicking human cognition, although they do have some interesting things in common that people don't realize. But um, I, I don't think they have to. The point about AI is not that you have to be a human for us to have to be kind to you. There, there are many, there are many living beings that, that are nowhere near humans and the same thing for danger, right? You don't need to be human level or above to be dangerous. There are many really dumb things that are extremely dangerous. So the thing we have to understand is that you cannot pin your hopes for, for being able to distinguish like moral worth and things like that.

• 52:45Speaker 00

  You cannot pin, pin that on what are you made of? Because there's nothing magical about, um, what protoplasm. You cannot pin it on being an evolved biological, because there's nothing magical about the random meanderings of the evolutionary process. Which, you know, just, uh, it doesn't optimize for anything that we care about. It doesn't optimize for intelligence or, you know, meaning or anything like that. It optimizes for survival, for copy number. And, um, and, and so none of those things, none of those things are reliable guides. You, you have, you, you can't, you know, you can't say, uh, just by, um, by knowing what something is made of or how it got here. We have to have principled frameworks for making that decision and, and we don't have them yet in biology really very much.

• 53:26Nathan Labenz?

  So I guess the beginning of maybe a framework for starting to make sense of those, you kind of touched on a little bit in those comments, but just to kind of pull out a couple of definitions, I'll try to, try to do it briefly. You tell me if I'm missing anything. I've heard you define intelligence as the ability to solve problems in some space. I think that kind of notion of in some space is interesting. And then you said kind of also, there's the question of like, not are we or aren't we, but how much? And you've, you've used the term the cognitive light cone there, I think. In other words, kind of, what is your sort of domain of concern and like? Possible reach, you know, or how far can you project your influence into the world?

• 54:08Nathan Labenz?

  And that is sort of a way of thinking about how much intelligence you might have. Does that extend all the way up to sort of nature as a whole? You're thinking of like the Gaia hypothesis? That's a far out one in many people's minds, but it seems to follow pretty naturally from what you're saying.

• 54:26Speaker 00

  Yeah, so, so the key, the key to all of this, so first let me, let me define. The cognitive lichone is the size of the largest goals that you can pursue. Okay, so you can think about what are the, what are the things that matter to a bacterium? What are the things that matter to a dog? What are the things that can matter to a human? And, and you know, and could we have at some point a being that like literally is in the linear range? Can care about all the living beings on earth? Like we, we can't do it as humans, but, but maybe our next, you know, some next stage of evolution can. So, so it's, it's this idea of, of what are the, what are the, what are the size of, of goals that you could actually maintain? The important thing about this is that you can't decide that just by thinking about it. You have to do experiments because the way you find goals is by perturbing the system.

• 55:11Speaker 00

  You cannot infer a intelligence or goal directedness by pure observations. You have to do perturbative experiments. So what that means is you put barriers between that. You, you make a hypothesis. What problem space is it operating in? What are the goals? These are all hypotheses and multiple observers could have different hypotheses. What, what, what are the goals? What competencies does it have to reach those goals? And now you get to test that hypothesis by putting in different barriers and seeing what it does. Right? Does it have long-term planning? Does it have, does it have learning? Does it have, you know, what, what, what can it, what can it do? So with respect to the Gaia hypothesis, some people say absolutely not. Some people say, well, I like to think it does, and therefore it does. Neither, neither of those is any good because, because you can't do any of that without doing experiments. Now, can you do, can you do experiments in?

• 55:56Speaker 00

  It'd be, for example, people have said to me, well, you know, that sounds, that sounds like you're going to say the weather. The weather has some, some degree of cognitive ability. Well, I'm not going to say that because we don't know yet. But one thing that this framework does for you is it doesn't close off the possibility of testing it to find out. Do I know that if you had the appropriate ways to change the temperature, the pressure of air and whatever, you couldn't use some kind of a habituation or sensitization assay or an associative learning to to show those things in a, I don't know, in a hurricane or something? We don't know. I mean, that's an empirical question, right? That the thing about the thing about this framework is that is that it does not let you just sort of pick answers out of thin air. It requires you to do experiments.

• 56:37Speaker 00

  So so Gaia, you know, can we show that ecosystems learn? Well, ecosystems have stress, you know? Would would an ecosystem be able to show different kinds of learning, anticipation? We don't know, but that can be tested. We're actually testing this right now. I have a student who's testing this in predator prey kinds of models to see if they can actually learn from from experience. We don't know. These are all these are all testable questions. So is it possible that, you know, on the scale of, I mean, people ask, you know, on the scale of galaxies or something that that, you know, we're a giant, like we're part of a giant mind. I think it's absolutely possible. I don't think we know, and I don't think you can say yes or no, just, you know, based on philosophical commitments. You have to do experiments.

• 57:20Nathan Labenz?

  That's maybe a great place to kind of segue to some questions that I have about maybe recommendations or sort of inspiration that you could give to people working to understand AI systems better. And I think, you know, you you said there, you can't just sit and guess, right? You have to do a perturbative, I believe is the word, experiment to see what is going on. In the AI world broadly, there's a lot of debate around the degree to which AI systems are generalizing beyond the distribution that they've seen in training. And it strikes me that some of the experience that you've done, particularly like, as you

• 58:04Nathan Labenz?

  said, rearranging the face of a tadpole or whatever, and watching as they sort of reconstruct their faces. One way to talk about what you're doing there is you're taking them way outside of the distribution of what presumably they had ever seen in evolutionary history. I mean, maybe, right? But that seems pretty like you've you've found a dark corner that, you know, presumably has not been previously explored. So with AI, we're trying to kind of begin to wrap our heads around that. Do you have any sort of, you know, advice, like habits of mind, food for thought, you know, any kind of suggestions from what you've learned in your work that people could take inspiration from on the AI side?

• 58:45Speaker 00

  Yeah, well, there's two, there's two kinds of things. There are some very specific biological principles that I think would be, would be interesting. And then, and then there are kind of general ways to think about this in terms of the whole debate about AI. I mean, I want to say a couple of things. One, one thing about embodiment. So, so there's a lot of talk about real, people say, well, if it's just a software agent, if it's not embodied, if it's not integrated into the real world and, you know, kind of grappling with, with some sort of embodied physical, physical existence, it doesn't, you know, it doesn't have real, doesn't have real, real, doesn't know what it's talking about, it's shuffling symbols, right? This is the old like Dreisssis argument and all that kind of stuff. So I want to say something interesting about, I hope, about embodiment. Embodiment is absolutely critical, but embodiment isn't what we think it is.

• 59:30Speaker 00

  People think embodiment is a physical robot that hangs out in three dimensional space. And that's because most of our sense organs are pointed outwards and they're optimized for tracking medium scale objects moving at medium speeds. Imagine if we had, imagine if we had a sense organ for our own body chemistry. Let's say inside our blood vessels, we had something like a tongue that could like feel, I don't know, 20 parameters of our, of our physiology. I think cognitively, we would, we would be living in a 23 dimensional space and we would immediately recognize our liver and our kidneys as intelligent beings that navigate that space. They have goals, they have certain competencies to reach those goals. Every day we throw, you know, wacky shit at them and they, and they know how to, they

• 1:00:15Speaker 00

  know how to navigate that space and they have certain competencies and they, and they live and they strive and they suffer in that space. So I think biology tells us that there are many spaces. There are spaces of gene expression, there are anatomical morphous spaces where the anatomical collective intelligence operates. There are metabolic spaces, there are, then, then of course, there's the familiar three dimensional space of, of behavior. And then there are linguistic spaces and so on. Embodiment can take place in any of those, intelligence can take place in any of those. So, so the first thing I would say is to take very seriously. Um. This idea that I, you know, is popular in, in, in science fiction or whatever, that this physical space that you're in is not privileged in some way and everything else is, is virtual.

• 1:01:00Speaker 00

  There are many other spaces and they are just as real. There are other beings that live in these other spaces. Many of them are in our bodies right now. And, and those spaces are no less real than this one. It's just that, you know, our left hemisphere, like, you know, this, this is, this is the, this is the, the kind of sense data that it, that it mostly gets. And that's why we all feel like we live in this space and everything. Yeah. So, so that's, so that's the first thing. So that's, I say that about embodiment. Something else I would say is about this issue of a symbol binding and grounding, right? This, this idea that, well, you know, it shuffles the little letters and it shuffles words and so on, but, but they're not grounded to anything. It doesn't, you know, it doesn't know what they mean. If you have kids and you watch, you watch a little human, uh, develop, what you see very quickly is that they start out as

• 1:01:44Speaker 00

  kind of, uh, happy-go-lucky pattern matchers. They will just sort of repeat stuff they hear. They see what sticks and they sort of talk about all kind, all, all kinds of stuff that they have actually no actual grounding on. And then eventually they get to the point where you say, oh, wow. Yeah. He really knows what he's talking about when he's saying that. This is a smooth process. It's not a categorical, yes, you, you are real or no, you're not. In fact, I, I don't know about you, but, but, but I, I thought, um, at one point I tried to sort of estimate what, like what percentage of the things that I talk about are actually grounded in the sense that I've had actual experience with them, you know? I, I, I can, you know, we, we can talk about all kinds of stuff, you know, places we've never been and, and, you know, all the, all kinds of, all kinds of things we've never seen.

• 1:02:26Speaker 00

  A ton of our cognitive, uh, fodder is not grounded to anything. It's just tied to other stuff. So it's, it's important to, to be clear that a lot of these, uh, supposed distinctions are really, um, are really spurious. You know, this issue of, um, confabulation. Right? I mean, humans can fabulate constantly. There is, there is, you know, there's a good theory that basically says that part of your, your language ability is basically just to tell stories about what your brain is already doing, right? After the fact, to concoct good stories that you can just share them with others and increase cooperation. There's some amazing, you know, some, some amazing data on, on split brain patients and, and other, other cases where it's very clear that we're very comfortable confabulators.

• 1:03:10Speaker 00

  So, so there are these distinctions, again, not, not saying that the current architectures capture what's essential about, about, about life, because I don't think they do. But, but, but a lot of these things are not, not, the biology isn't what, what, what people think, what people think it is. And I would encourage workers in AI to think about the whole spectrum of diverse intelligence, cells, organs, tissues, chimeras, cyborgs, hybrids, not just like the standard adult human that, you know, that, that, that we think about as a, as a, as a, as a counterpoint. This idea of, of, of ethics and the kind of, the impact of highly competent AIs. And, and people say, this is, this is really

• 1:03:53Speaker 00

  scary. You know, we're creating these, these, these intelligent entities and we're, we're going to let them loose into the world and we don't know what they're going to do. We've been doing that for millions of years. It's called having children. We already do that. All, all of us, we, we create these guaranteed high intelligent agents. We do our best or not during the critical kind of training phases. And then we let them into the world and sometimes they do, you know, miraculous things and sometimes they do horrible things. And so I, I think a lot of, a lot of the issues that we have around AI right now, I mean, obviously there's some unique ones, but, but most of the issues around AI and, and what are we going to do to stay relevant? And, you know, what happens to collaboration

• 1:04:35Speaker 00

  and to AI art and all this kind of stuff is really just reflections of existential issues that humans have been struggling with for a really long time. You know, these are all fears about what, what happens, what, you know, what do the next generations think of us as they, as they mature and, and, you know, and fears about humans and what does it mean to stay human? Yeah. I mean, we can, we can, we can talk about that too, you know, this issue of what, what do we want to persist, you know, as far as people being worried that we're, we're beginning to get taken over and so on. It's a whole, whole thing we could talk about there.

• 1:05:05Nathan Labenz?

  This is extremely thought provoking on a lot of different levels. One quick follow up on the perturbative experiments. It seems like there is a decent analogy I can already start to see between some of the things that you're doing and, you know, there's, there is a lot of weird stuff that is, that is kind of on the edges of, of modern AI research, but is like remarkably, it kind of works. Stuff that I think probably would get a lot more energy if the mainline stuff weren't working so well, then people would be kind of forced to go out and explore more of, you know, these kind of weird frontiers. But there, right now, the, the vein of just progress in the mainline research is so rich that, you know, there's not a lot of incentive in many cases to, to go outside of it.

• 1:05:50Nathan Labenz?

  But you do see these things, which kind of remind me of, of some of your work where, you know, you can, for example, take a lot of weights in a lot of models and just drop a ton of the weights to zero and find that, oh, hey, it still kind of works, you know? And you can, you know, take two pre-trained models and put a little connector between the two of them, freeze the pre-trained ones and just train the connector and get kind of novel behavior that, you know, neither model could, could come up with on its own. I guess, you know, I'm riffing there. I think that kind of work feels really important. It also, to me, it does feel a little bit, I mean, the whole AI enterprise to me, not like any individual modern day experiment, the whole AI enterprise does feel to me a

• 1:06:33Nathan Labenz?

  little bit dangerous, to be totally honest, because unlike our children, who I think we have a pretty good handle on kind of, you know, just how big their, you know, individual light cones may be, I, I think we may very well create something in the AI realm that just has like way bigger scope of action than anything we're used to. And then, you know, that could go badly for us. It, it sounded to me like you weren't too worried about that. Maybe you're just not that worried about it, but I wouldn't, based on everything we've talked about, I find it hard to imagine you would rule that out or see some reason that like that shouldn't be an issue that people would worry about at all, right?

• 1:07:09Speaker 00

  No, it's definitely an issue, but I also think that that issue is here long before we make anything with a huge cognitive lichome. So there are many, many things that are extremely dangerous that do not have a large cognitive lichome. And some of that is because of how we relate or misrelate to them. So I can imagine AIs that are going to be, that would be extremely deleterious for society, but not because they're so smart. It's because we're not that smart, right? And I think that we, you know, we, one of the, one of the big things that I think is an existential risk scale thing for humanity is to develop a principled science of where do goals of novel systems come from?

• 1:07:53Speaker 00

  And how to, how to interact with, ethically interact with systems that are radically different from us, minds that are radically different from us. We make, we make systems, you know, Internet of Things, social and financial structures. AI, obviously, we make all these different things. We do not have a science of where novel goals come from. Systems, not just emergent complexity, that's bad enough, emergent complexity is bad enough. But also what is to be clear that can, that emerges is emergent goals. So systems that have some degree of pursuing various goals and some competency to, to get those goals met. And we do not have a good science of knowing when that's going to happen

• 1:08:37Speaker 00

  and what those goals are going to be, what the competencies are going to be. And so I'm not saying that, that it isn't possible that we make a hugely, you know, super intelligent AI that's going to be a problem. Absolutely possible. But I think we need to realize that that problem can occur long before we get to that point. Our physical infrastructure and also our, our kind of the, the general mental. Uh, frameworks that people are using are extremely brittle. You know, most of the terminology that that we rely on today in the legal system in, you know, in interpersonal relationships, that stuff isn't going to last the next decade or two. Like, all those terms are going to crumble. And and it's not because we're dealing with something yet that's super intelligent.

• 1:09:18Speaker 00

  It's because we haven't got the right framework for for for dealing with other minds that are different from from our own. That's that's where a lot of effort has to go.

• 1:09:27Nathan Labenz?

  Yeah, that question of emergent goals is definitely a very salient one in the AI space, and I think, you know, probably deserves to be even more focal still. But some challenges that we have in AI today. One is like figuring out what the goal should be. You know, we have the pre-training goal is just like predict the next text. Then we've got reinforcement learning from human feedback, which is like satisfy the human. Then we look at ourselves and as you said, we're kind of brittle. We can, you know, what satisfies the human is not necessarily good for humanity or, you know, the ecosystem broadly or whatever. So lots of possible problems there. People start to think, well, maybe we could have like multiple goals and we could kind of, you know, this is sort of what humans seem to have is like this kind of multi-dimensional value. Maybe we could have multiple goals we could optimize the systems for.

• 1:10:12Nathan Labenz?

  Yikes. Okay, a couple more and then I'll just let you go off on all of them. Memory is a huge challenge right now for AI. We have, you know, at least with the kind of systems that people are most familiar with, we've got like the context window. It can kind of process that information densely at runtime, but there's not really another level of memory in the in today's frontier system. So people are kind of rigging up all sorts of scaffolding and like finding things that work and saving those to a database and recalling them later. They're not very integrated. So, you know, you've got such incredibly remarkable findings in terms of memory and bioelectrical space in organisms. Really interested in that. Robustness is another big challenge, right?

• 1:10:54Nathan Labenz?

  With these AIs are like superhuman across like passing all the graduate school exams, but then we can trick them, you know, and and they're so gullible. It's like, well, what's up with that? Well, there is something definitely remarkable about our robustness, given our finite capability that the AIs are not that close to yet. Even like superhuman Go player, I don't know if you saw this result, but a relatively simple attack was able to beat, you know, the superhuman Go player. So I'll stop there. The big ones, goal design.

• 1:11:26Nathan Labenz?

  Memory, robustness, AI has a long way to go to catch up to biology. And I wonder if you have any suggested directions for the AI developers?

• 1:11:35Speaker 00

  The thing with the thing about memory, I'll just give you a quick biological example. Caterpillar to butterfly. So caterpillar, soft bodied robot, lives in a two dimensional world, eats leaves. Butterfly is a hard bodied robot, flies around 3D world, drinks nectar. In order to get from here to there, you have to build a completely new brain. So what it does is during the during the metamorphosis process, it basically dissolves most of the brain. Most of the cells die, most of the connections are broken. And then, you know, you have a new brain that's suitable for a completely different kind of creature. The memories that this has been shown, the memories that a caterpillar forms are still present in the butterfly. It in fact mapped onto its new kind of lifestyle. We don't have any, to my knowledge, any memory in the in the computer science world that can

• 1:12:21Speaker 00

  survive such refactoring, you know, you just tear the whole thing up and you know, sort of reuse the pieces. And now you can still, you know, retain the original information. Biology does it quite differently. Josh Bongard and I have a have a paper on polycomputing and this this this ability of biological systems and subsystems to reinterpret the same physical events from their own perspective as multiple different computations. I think that's actually pretty, pretty key. Also key is the fact that evolution doesn't make specific solutions to specific environments. It makes problem solving machines. This is why, and I could tell you all kinds of crazy capabilities that living things have to deal with things that they've never seen before in evolution. And it's because evolution doesn't overtrain on prior experience.

• 1:13:04Speaker 00

  It produces multi scale agents that are able to solve problems in new ways. Fundamentally, for the use of AI, we have to decide, do we want tools that we use for specific purposes? In which case pleasing the human is probably a great goal to shoot for. Or do we want true agents with the kind of open ended intelligence that that we have and the same moral worth that that we see among animals and among humans. Those are, those are very different things. Honestly,

• 1:13:43Speaker 00

  I was, I had started about six months ago, I started writing a paper on specifically what parts of biology is the current AI architectures, are they missing such that I, you know, uh, what, what would I, which I, which I think would, you know, of course elevate, elevate the, the whole, the whole thing majorly because I think, I think they're just missing some of the really exciting new biology. And I stopped and I'm not going to write that paper, not, not that, not that somebody else isn't going to crack it because they will. So this is, this is, you know, not, not going to solve this issue because I think to whatever extent I'm right and that those things are actually what's critical about agency, they can easily be implemented in other media. They're not, there's nothing magic about, um, but you either evolution or biological

• 1:14:26Speaker 00

  substrates. You can, you can implement those, those kinds of things in other media. But if you do, to whatever extent I'm right, that those are the ingredients to an actual mind, that means that, that we are going to then generate, you know, I don't know, trillions of new agents with moral worth. And I, I'm not interested in, in being responsible for that. But, but, but it's going to happen. And I don't think we need it, you know, I, I think, I think people are going to do it just because, um, to see if it can be done. And I think it's the equivalent of, of, you know, just, uh, you know, having, uh, it's, if you do it correctly, which, and, and somebody will do it correctly. It's like the equivalent of, of, of having trillions of, of new children. Now, you might think that your job is to populate the cosmos

• 1:15:10Speaker 00

  with as many minds as possible. I mean, that's one view. But to me, if, if you're not able to vouch for the kind of upbringing and, um, and, uh, you know, life they're going to have, then I think that's a mistake. So, so, so we can stick to the kind of AI that's actually really, I, I, I think that's actually really helpful. It's a, it's a tool, it's a thinking tool for, um, for humans. It, it doesn't have a kind of open-endedness, uh, and the kind of agency where you have to worry about it, or you don't know what it's going to decide to do. I, I, I think it would be much, much better to stick to that kind. And it's not because I think humans are, you know, just to sort of end on this because. A lot of people are really worried that we're going to get supplanted.

• 1:15:51Speaker 00

  I, I, I think we will, and I think we should. If, if, if you think forward, you know, 10,000 years from now, what, what, what do you want to be living on this earth? Do you really want it to be the exact same humans that we have now that are subject to lower back pain and stupid infections and, you know, and they have about 80 years of, of, of productive lifespan and then the kind of things go like, why is that something that we're attached to? I'm not interested in that at all. I think, I think that any improvements that we can make, we, to enlarge our cognitive lipo and to improve on our cognition, to have better embodiments, whether those are, you know, engineered or bioengineered, or I don't, I don't care. We, we can certainly do better than we do now, right? If, if, you know, anybody that's, that's been to a hospital knows that, that this is, this is not a great embodiment that, that we're in right now. It's amazing.

• 1:16:34Speaker 00

  The future should be, should be way better than this. So I'm not, I'm not talking about limiting AI because I think humanity should, should be kind of freeze down in the version that we are now. I don't, I don't think there's any reason to kind of lock it down that way. But I also don't think that there's any particular reason, and I think there are good reasons not to try really hard to make new agents that have moral worth. And I, and I absolutely think we could in other substrates, including digital. So yeah, I'm, I'm, I'm, you know, sort of not really a, not really advertising those, those methods that I think actually could be, could be done. But, um, but there's plenty of other stuff that I think, you know, is, is perfectly reasonable and could, and could help create better AI tools for us.

• 1:17:18Nathan Labenz?

  I've been starting to think about that same question from a different angle. What are the pieces that are missing from the current AI systems? And, you know, what would it maybe look like to solve them? And what would the consequences of that be? And I do think you raise a, a good, um, you know, sanity check on the wisdom of the direction of that research. When you look at the AIs today, you know, one thing that is kind of striking is we've got these language models, right? And they, you know, just predict the next token, quote unquote. They're very much just, you know, happy to be a chatbot for you. But they are. You know, they did. You said multiple times, like, there's no bitaries. I always say AI defies all bitaries we try to put on it. This one also seems to be kind of defied to a limited

• 1:18:02Nathan Labenz?

  extent, at least, where, you know, you cast the language model as an agent. You basically say you have a goal, and now I'm gonna like, put you in a loop with, you know, some tool in the outside world. And they're like, not very effective agents, but they start to look something like proto-agents. They certainly like, try to pursue the goal, and again, usually fail. But how do you interpret that? Or how do you kind of make sense of that sort of non-agent versus agent mode of the current systems?

• 1:18:28Speaker 00

  A lot of problems in thinking about this arise from A, comparing it to humans, and B, comparing it to very high-performance humans. So I sometimes, I sometimes hear, you know, really, really like brilliant scientists or AI workers say, ah, this thing is, you know, it can't even do that. How many people can do what you're talking about? Like, barely any of them can. Your friends can, because it's a very selective elite population of human minds. But but most humans can't do that. We, so there's that. And then, and then again, look at, look at the diversity of life on Earth. I mean, we have other, other, all kinds of other creatures that are not humans that are minimal or sort of meso-goal, goal-seeking agents. You know, you've got insects, you've got, you know, birds and

• 1:19:11Speaker 00

  fish, and you've got all these things. For AI to be, to be a real agent, it doesn't have to be like a human. You might be developing some kind of weird insect-like thing, or something like that. And, and let's, let's remember, we are not very nice to all kinds of creatures that are for sure agential creatures. So adult mammals, right, in our food chain, what we eat is so factory farming is a, is a, you know, is an incredible moral lapse. And it's not because we don't understand that that pigs are intelligent, and that cows are intelligent, and that they have, you know, they deserve certain moral protections, and so on. It's not because we don't know. It's that we're just not very good as a, as a, you know, as a species, we're not very good at extending our cone of compassion

• 1:19:54Speaker 00

  to things that are in any way different from us, right? The stupidest, slightest difference is enough to trigger this like self versus other thing. And then, you know, then you, you know what happens then. It is absolutely erroneous to assume that these systems do not have some degree of goal directedness. I think that it's possible that we haven't figured out how to, how to. How to measure it properly yet, or it's possible that that we have and it's just low, like fish or insects or something like that. But it's again, I'm not, I'm not claiming that I want to be clear. I'm not claiming that these architectures share the important things, you know, share a lot of the important things with life. It one thing, one thing that's super confusing nowadays is that in the past,

• 1:20:35Speaker 00

  you were guaranteed that anything that talks also shares your agential, uh, kind of key agential properties. That's no longer true. Now we have these things that talk, but they actually, I don't, I don't think they share them. But that doesn't mean the fact that they don't share them with, with, you know, elite adult modern humans doesn't mean that they don't have any. They may have some and we are not good at recognizing it. And that's why I'm saying that I'm not saying it because I just would be, I'm not saying it because I'm fooled by the, by the chatbot action. I'm not saying it because I just sort of assume that robots and AIs are going to be that way. I'm saying we do not have a great way to detect it on the biology end. So I, and, and we certainly do not have a great way to detect it on an exobiology end. So like if we, you know, if we had alien life and so on, we'd be, we'd be really epic. Actually, we have no clue.

• 1:21:20Speaker 00

  And so I don't see any way to be certain about anything with these, with these complex creations.

• 1:21:27Nathan Labenz?

  Well, that is, I think, an appropriately, uh, sober note to end on. I have really enjoyed reading your work in preparation for this. It is fascinating and this conversation has been super thought provoking. I'm sure people are going to take a lot from it and I hope we can do it again at some point in the not too distant future. But for now, uh, Professor Mike Levin, Professor of Biology at Tufts University. Thank you for being part of the cognitive revolution.

• 1:21:53Speaker 00

  Yeah, thank you so much. Yeah, it was great to talk to you. I'm happy to, happy to come back. Um, you should also try to get, um, uh, Richard Watson at Southampton. If you, uh, if you, uh, if you know his work, uh, he's got, he's got a lot of really interesting things to say about this. So that's, that's something else you might want.

• 1:22:08Nathan Labenz?

  Cool. That's a great tip. It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at [email protected] or you can DM me on the social media platform of your choice.