Karl Friston - Why Intelligence Can't Get Too Large (Goldilocks principle)

In this episode, hosts Tim and Keith finally realize their long-held dream of sitting down with their hero, the brilliant neuroscientist Professor Karl Friston. The conversation is a fascinating and mind-bending journey into Professor Friston's life's work, the Free Energy Principle, and what it reveals about life, intelligence, and consciousness itself. **SPONSORS** Gemini CLI is an open-source AI agent that brings the power of Gemini directly into your terminal - https://github.com/google-gemini/gemini-cli --- Take the Prolific human data survey - https://www.prolific.com/humandatasurvey?utm_source=mlst and be the first to see the results and benchmark their practices against the wider community! --- cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++ Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst *** They kick things off by looking back on the 20-year journey of the Free Energy Principle. Professor Friston explains it as a fundamental rule for survival: all living things, from a single cell to a human being, are constantly trying to make sense of the world and reduce unpredictability. It’s this drive to minimize surprise that allows things to exist and maintain their structure. This leads to a bigger question: What does it truly mean to be "intelligent"? The group debates whether intelligence is everywhere, even in a virus or a plant, or if it requires a certain level of complexity. Professor Friston introduces the idea of different "kinds" of things, suggesting that creatures like us, who can model themselves and think about the future, possess a unique and "strange" kind of agency that sets us apart. From intelligence, the discussion naturally flows to the even trickier concept of consciousness. Is it the same as intelligence? Professor Friston argues they are different. He explains that consciousness might emerge from deep, layered self-awareness—not just acting, but understanding that you are the one causing your actions and thinking about your place in the world. They also explore intelligence at different sizes. Is a corporation intelligent? What about the entire planet? Professor Friston suggests there might be a "Goldilocks zone" for intelligence. It doesn't seem to exist at the super-tiny atomic level or at the massive scale of planets and solar systems, but thrives in the complex middle-ground where we live. Finally, they tackle one of the most pressing topics of our time: Can we build a truly conscious AI? Professor Friston shares his doubts about whether our current computers are capable of a feat like that. He suggests that genuine consciousness might require a different kind of "mortal" computation, where the machine's physical body and its "mind" are inseparable, much like in biological creatures. TRANSCRIPT: https://app.rescript.info/public/share/FZkF8BO7HMt9aFfu2_q69WGT_ZbYZ1VVkC6RtU3eeOI TOC: 00:00:00: Introduction & Retrospective on the Free Energy Principle 00:09:34: Strange Particles, Agency, and Consciousness 00:37:45: The Scale of Intelligence: From Viruses to the Biosphere 01:01:35: Modelling, Boundaries, and Practical Application 01:21:12: Conclusion

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Progressive's Accident Response can contact emergency services if you're in a major accident. The only thing worse than someone you love being in an accident is not being able to help them. So Progressive is here to help when you can't. So get peace of mind knowing your loved ones are protected with Progressive's Accident Response. Learn more at progressive.com. Progressive Casualty Insurance Company and Affiliates. Limited availability in California. Driver is responsible for emergency and towing charges not covered by the policy. Coverage subject to policy terms and conditions. Hey Spotify listeners, when you travel with American Airlines, you'll enjoy comfort and convenience at every altitude. Before takeoff, unwind in the Admirals Club Lounge with curated self-serve offerings of hearty bites and refreshments. In flight, stay connected with high-speed Wi-Fi.

• 0:45Speaker 00

  Sponsored by AT&T. Your journey is designed to feel easy from the moment you check in to the moment you arrive. No matter where you're going, American gets you there. Book now at aa.com.

• 0:54Tim Scarfe?

  Keith has been talking about this moment where we had a glass of sherry with Professor Friston for about the last four years. Right. And our dream has been realized today. Absolutely, so a pleasure to meet you. It's absolutely been a pleasure.

• 1:08Speaker 02

  Cheers.

• 1:09Tim Scarfe?

  Cheers.

• 1:10Speaker 02

  You mentioned consciousness, which you shouldn't really do with me, but if we stay here for long enough, for more than five minutes, we will ultimately become completely entangled. We will not become one, we'll still have a, we'll still have, well, actually Chris Fields thinks you would become one, but is evolution an intelligent process? It's certainly a free energy minimizing process, you know, it is just basic model selection.

• 1:39Tim Scarfe?

  Well, it's like, is the planet intelligent?

• 1:41Speaker 02

  Yeah. I don't see the weather planning. I don't see evolution planning. It doesn't think about its future.

• 1:47Tim Scarfe?

  Life is the intensive property of matter and intelligence is the extensive property.

• 1:52Speaker 02

  It's an ensemble of universes. Well, of course, for the purpose of this argument, there is only one state of the universe at any one time. I'm not sure this is correct, but I don't want to disappoint you, so.

• 2:07Tim Scarfe?

  Don't worry, it's well beyond my lifetime when that will happen.

• 2:10Speaker 02

  If you're going to make a difference, it's really understanding how to use all the marvelous engineering that we've witnessed in terms of generative AI and large language models and like, in the service of understanding the principles of right natural intelligence and deploying that understanding either to make life better, fun, or to underwrite sustainability in a slightly more sort of political way, or to deploy it, which is why I got into this game, in the context of mental wellbeing and computational psychiatry, to quote Feynman, that which I cannot create, I do not understand. That means we have to be able to create things

• 2:54Speaker 02

  that suffer, which means that we need to understand the principles of natural intelligence.

• 2:59Tim Scarfe?

  I don't know if you saw, I interviewed Yoshua Bengio.

• 3:02Speaker 02

  All right.

• 3:02Tim Scarfe?

  And I used the term epistemic foraging a few times, and he said, I love that term. I love that term. Where did it come from, Professor Christen? Absolutely. Epistemic foraging, it's what we do. Yeah, it's epistemic foraging.

• 3:15Speaker 02

  Epistemic foraging. There are certain causal forces in our universe that are at a distance, those usually mediated by electromagnetic radiation or magnetic fields in this particular instance. And that really sort of complicates the way in which we construe all the cause-effect structures that we have to model in our brain. Because if I was a simple little virus or a little bacterium, I wouldn't have to worry about electric fields and seeing things or hearing anything. My world would just be that world that I could touch, literally. It would just be my next door neighbors, which leads to a very particular kind of

• 3:56Speaker 02

  Markov blanket structure, an ecosystem of Markov blankets, that precludes, oh, certainly does not license some of the deep structures that we were talking about earlier on in terms of strange things. So strangeness of a beautiful sort, beautiful loops, to use.

• 4:21Speaker 02

  One of my colleagues' notion rests upon action at a distance of a very non-spooky sort that is nicely exemplified by this compass. So the moment, I don't know of any good maths that allows you to work out what is the best thing to do to disambiguate between this structure and that structure.

• 4:42Tim Scarfe?

  Yeah.

• 4:43Speaker 02

  So if you can do that, that'll be good.

• 4:45Tim Scarfe?

  Structure learning.

• 4:46Speaker 02

  Structure learning.

• 4:47Tim Scarfe?

  The structure learning.

• 4:48Speaker 02

  Structure learning.

• 4:49Tim Scarfe?

  Somebody out there, help us out. Yes.

• 4:57Tim Scarfe?

  Human data is shaping the direction of frontier AI, yet there's little visibility about how teams are actually using it. Our sponsor, Prolific, are putting together their first report on human data in AI, and they need volunteers. It just takes a few minutes to fill out, and you'll also get early access to their findings so you can see how you compare.

• 5:18Speaker 03

  This podcast is supported by Google. Hey folks, Taylor here, creator of Gemini CLI. We designed Gemini CLI to be your collaborative coding partner at the command line. Think about those tedious tasks like fixing a tricky bug, adding documentation, or even writing tests. We built Gemini CLI to handle all that and more. It iterates with you to tackle your most challenging problems. Check out Gemini CLI on GitHub to get started.

• 5:47Tim Scarfe?

  MLST is supported by Cyberfund. Professor Freestone, it is absolutely amazing to have you back on MLST. As you know, Keith and I are incredibly fond of you. You've been a big hero of ours for many years. I think this is the fourth time. Thank you. Welcome, Professor Freestone.

• 6:06Speaker 02

  I've lost count. But it's lovely to see you both in person, and congratulations on your recent marriage.

• 6:13Tim Scarfe?

  Thank you very much. Thank you very much.

• 6:16Speaker 04

  Yeah, so you know, we were just thinking about this. We've done these interviews. The Free Energy Principle originated around about 2005, right? And I just kind of wanted to get a bit of a retrospective from you on Let's say like, what have been the successes so far of the free energy principle? How's it been going? What could have gone better over the last, you know, 20 odd years? You know, really, just what's your, what's your take on the progress, I guess, of the free energy principle?

• 6:44Speaker 02

  It's a difficult question to answer, in the sense that, what are you, if you asked me, if I'd made a meal, if you've asked me, how's that meal going? And did you really enjoy it? When you're doing it, you're sort of in the middle of it, and it's very difficult to assess. So, it's been a journey. I, I do catch myself from time to time, thinking, this is the right way to think about things. Whenever you come across, you know, a phenomenon, or a problem, or an application, that seems to just slip in quite neatly to the overall theoretical framework.

• 7:25Speaker 02

  Every time that happens, I get a little buzz of dopamine, and that you're, that consolidates and comes, yeah, I'm on the right track, I haven't wasted so much time. There have been times in my life when I've realised I'm on the wrong track, so I'm used to that. But for the particular application of the free energy principle, I have yet to have that, oh, well, that doesn't work then, think about it, do something else. What could have gone better? I'm not, I'm ambivalent about this, but I am sometimes, well, I am ambivalent when I read, the free energy principle is notoriously difficult to understand.

• 8:07Speaker 02

  Now, half of me thinks, good, because it's always important to have a slight degree of magic and mysticism to engage people. You know, if people don't think there's a challenge, that they have accomplished something by understanding it, then they're not going to be motivated to inquire and to think about it, or indeed debate it. On the other hand...

• 8:32Speaker 02

  The free energy principle is not meant to be complicated or difficult to understand. It's actually, you know, almost tautologically simple. So that being able to communicate the free energy principle in a way that people find it a useful and obvious sort of tool or method to apply could have, I think, gone better. And that may be because I'm not very good at keeping things simple or intuitive.

• 9:00Tim Scarfe?

  We can attest to that.

• 9:02Speaker 04

  You know, one thing that's interesting is I feel the same way about probability theory. Like, you know, conditional probability theory is very easy to write down. There's really, you know, you end up with the two fundamental rules, the sum rule and the product rule, but the consequences of it are very profound and probability is notoriously misunderstood, misapplied. There's many errors of reasoning, you know, that happen when people try to reason through statistical correlations or whatever. So I think there are things like that in life that are very simple and yet very hard to to kind of understand the full extent of their impact, right?

• 9:39Speaker 02

  Yes. I mean, it's interesting you picked up on conditional probabilities. I mean, that is the heart of the free energy principle at many, many different levels. So the classical formulation of free energy principle just starts off with a partition of different states of being. And that partition suddenly means you've got the probability distribution or density of a one set of states of being that now can be conditioned upon another. So the whole free energy principle is just basically a principle of least action pertaining to density dynamics, the dynamics or the evolution of not densities but conditional densities. That's just it.

• 10:20Speaker 02

  This is before thermodynamics, it's before quantum mechanics, it's just about conditional probability distributions. So it's interesting you picked up on that as something that is so simple yet so absolutely powerful.

• 10:33Tim Scarfe?

  So Professor Preston, we were reading your paper earlier, it's from 2023. Path integrals, particular kinds and strange things. And it introduced a categorization of particles. So for example, inert to active and ordinary to strange. The strange categorization was particularly interesting because you were giving an account of how these particular particles could give rise to phenomenal, you know, conscious states or even agentic states. And we felt that this was almost an account of panagentialism. So you know, historically, we've done interviews about the free energy principle and we've

• 11:13Tim Scarfe?

  spoken about self-organization and emergence and the agency and the phenomenal states are something which emerge when you have this sort of temporal and counterfactual depth. And maybe we're misunderstanding, but it feels like this is an account of low-level agentic potential and phenomenal potential.

• 11:33Speaker 02

  So to set the scene, just to come back to that partition that we were just talking about, there are many ways of arranging the conditional independences, you know, to sort of disconnect various partitions, you know, external states, internal states, my sensory states, my active states. And the combinations of disallowed influences give rise to an ontology of different kinds of things and you could call them natural kinds. I've been told by Maxwell I shouldn't use that, but I like natural kinds. The things in nature that just prescribe themselves just by being special instances

• 12:16Speaker 02

  of a lack of causal influence. So if we start off with, you know, the simplest case where there are no internal states and there are no active states, and what are we talking about? We're talking about some kind of causal black hole, something that's, you know, quintessentially inert. And you can never see it because you can only see the active states that act back on the environment in which it is embedded. But then we get to more interesting things that have a full complement of internal states and external states and a bidirectional coupling between the two, which of course is mediated by the sensory states and the active states.

• 12:58Speaker 02

  And if you disallow action at a distance, by which I mean I have to be in some metric sense next to you in order for me, my active states to become your sensory states and your sensory states to be my active states, then you have a very simple kind of Markov blanket, which would be fine for describing active matter, for example, or any medium where I have to touch, to feel, to influence or to sense. And this kind of particle, interestingly, has its, or this kind of system, partition, has its active states on the inside, which is, you know, I had to think about, you know,

• 13:38Speaker 02

  and it can be no other way if you, you know, just try to arrange all the dependencies. So what would that look like? It would look like something like a cell that had on the outside its sensory states, that was supported by a layer of active states that surrounded the internal states. So you're complying with the laws of the Markov blanket of thingness, that, you know, you need to have that conditional independence to separate the thing from everything else or the self from the non-self. And everything's fine, because the active states are hiding behind the sensory states, so the external states can't influence the active states, so that's tick one, that's, you know, what we need.

• 14:22Speaker 02

  But we also have to ensure that the internal states symmetrically cannot act or influence the sensory states, and that's fine, because they're hiding behind the active states. So you've got, you know, a nice mathematical image of, you know, a cell, if you like. But that doesn't work for things like you and me. Things like you and me have a hierarchical structure. And what that basically means is that the active states, which were hitherto in very simple organisms, immediately juxtaposed to the internal states, now become sequestered.

• 15:03Speaker 02

  So now, effectively, the active states are no longer seen by the internal states. And then something quite remarkable happens mathematically. It's really simple, and it's just another aspect of conditional probability distributions.

• 15:18Speaker 02

  Because you can't see your active states, so you can now only see your sensory states, it looks as if, from the point of view of the internal states, that the active states are now become causes of sensory states. So now, your world is caused not just by the external world, by the environmental states, by my heat path, by my external milieu, but also my own actions. So now, I'm inferring the causes of my sensorium, where I'm actually a cause, so I'm inferring myself. So there's this beautiful recursion, which is, license is this sort of strange loop analogy,

• 16:02Speaker 02

  with a bit of poetic license. That, another way of putting that is that, for these simple structures, for these natural kinds that would be, say, single-celled organisms, the internal states can be read as modeling the causes of their sensations that just are the external states. So they have direct access to the active states, so there's no conditional independence that licenses the notion of description as, in terms of inference or sense-making. Whereas now, the Bayesian mechanics that attends, the internal activity, internal machinations,

• 16:45Speaker 02

  now covers both my action and the things that I'm acting upon. And of course, this is a nice metaphor for planning as inference, that I'm now thinking about and trying to infer what am I actually doing. And that, to my mind, produces a very unique and special kind of thing. You know, things like you and me, basically, which are pretty unique. We look at all the different kinds of things that could be around.

• 17:13Speaker 00

  Progressive's Accident Response can contact emergency services if you're in a major accident. The only thing worse than someone you love being in an accident is not being able to help them. So Progressive is here to help when you can't. So get peace of mind knowing your loved ones are protected with Progressive's Accident Response. Learn more at progressive.com. Progressive Casualty Insurance Company and Affiliates. Limited availability in California. Driver is responsible for emergency and towing charges not covered by the policy. Coverage subject to policy terms and conditions. Hey Spotify listeners. When you travel with American Airlines, you'll enjoy comfort and convenience at every altitude. Before takeoff, unwind in the Admirals Club Lounge with curated self-serve offerings of hearty bites and refreshments. In flight, stay connected with high-speed Wi-Fi.

• 17:58Speaker 00

  Sponsored by AT&T. Your journey is designed to feel easy from the moment you check in to the moment you arrive. No matter where you're going, American gets you there. Book now at aa.com.

• 18:09Speaker 04

  Well, it's another example of how, as soon as you had this recursion, right? As soon as you had these strange loops, especially for systems that are computational. And I forget who it was that said, "Life is the computational phase of matter." But I mean, when you're at this level of complexity, where you're having some kind of information processing and you introduce this strange loop, right? That unlocks a completely different, like, category of behavior, you know, tier of computation, right?

• 18:39Speaker 02

  Absolutely.

• 18:40Tim Scarfe?

  It was David Krakauer. He said, "Life is the intensive property of matter and intelligence is the extensive property."

• 18:46Speaker 04

  No, this is a different one, though.

• 18:47Tim Scarfe?

  Oh, very similar.

• 18:48Speaker 04

  It may have been Wolfram who said, "Life is the computational..." I don't remember. I'd have to look it up. But yeah, so it unlocks a completely different category of behavior.

• 18:58Speaker 02

  Yeah. Well, you alluded to that earlier on in terms of phenomenology. And you mentioned consciousness, which you shouldn't really do with me, but...

• 19:08Speaker 02

  We'll read that as an elementary kind of sentence. So I think that's really important, you know, that point that you do unlock or you do manifest, or now at least you have a mathematical calculus that allows you to talk about things that model themselves. And as soon as you're modeling yourself, you will become an agent or you have an authentic kind of agency. Because now, in order to model myself or to model the... Me as a cause of my environment, I have to infer what I am doing, in particular, the consequences of what I'm doing. And because the consequences have not yet appeared, this has a quintessentially future-pointing

• 19:53Speaker 02

  aspect. So we're now moving from a thermostat, or a virus, or a single-celled organism, to things that actually have a future on the inside, their private future, that just exists for them. And of course, if you can get suitably far into the future, where you have a divergence of particular paths into the future, just as a nod to the pathological formulation, then you have to select one. So now you've really unlocked, again, a calculus of selection of your paths into the future, which I think, you know, is not a definition of agency, and certainly not consciousness, but certainly would be, I would imagine, arguably necessary to support something that

• 20:38Speaker 02

  had true agency and possibly even consciousness, you know, of an elemental kind.

• 20:44Tim Scarfe?

  Could we press on the consciousness a bit? Because the account that you just gave is a beautiful account of agency, and it's very plausible. So as you increase this reflexive layering, this recursion, you get these multiple paths into the future, and you become the cause of your own actions, so you become more of an agent. So it's almost an account of strength of agency with more and more layers. But in the abstract of this paper, we were quite struck with the language that convicts consciousness with agency. And intuitively, I feel that that phenomenal experience is quite orthogonal to agency. So why were they kind of convolved together?

• 21:29Speaker 02

  I'm desperately trying to remember how I referred to consciousness. You've got to be very careful when using that word, depending on who you're talking to.

• 21:40Tim Scarfe?

  Yes, the C-word.

• 21:41Speaker 02

  The C-word, oh, naughty. So, yeah, and I think, you know, people like Neil Seth make a very similar point quite earnestly. Intelligence and consciousness are completely orthogonal, that you're making agency and consciousness are completely orthogonal, and I would agree entirely. So your consciousness is not implied by being agentic, having agency.

• 22:12Speaker 02

  You would, first of all, well, there are many stories you can tell here. And, you know, my mind goes to the people who might be watching this to make sure I don't offend anybody for not mentioning it.

• 22:26Speaker 02

  From the point of view of three energy principle, the way that you'd look at consciousness is

• 22:36Speaker 02

  in terms of a dual aspect monism, which you get for free from the treatment of the density dynamics in exactly the way we were talking about before. But, you know, so now my internal, say, brain states

• 22:56Speaker 02

  have a thermodynamics, they have, that can be written in terms of an information geometry that itself is just predicated on conditional probability distributions. But they also have an information geometry that inherits from the fact that my internal brain states represent the outside world, including my own actions. So there's both an information geometry and by implication a thermodynamics of my brain activity, which supervenes on exactly the same substrate as does the information geometry, which is representational, which is the inference, the basic mechanics side of things.

• 23:41Speaker 02

  So that gives you license to talk about a sort of dual aspect monism. But what is it, what does it really mean to be conscious? Some people might believe that it's just having a posterior belief, having a basic belief, which is literally just a conditional probability distribution that is encoded or parameterized by some physical state of being. And in this instance, in the three energy principle, it's the internal states of being.

• 24:06Speaker 02

  Other people say, "Well, no, just having, you know, just be able to read a thermostat, for example, as having post-traumatic beliefs about the temperature of his external world does not licence you to ascribe consciousness to this thing." So the next step would be, "Oh, it has to be ignited in some way, it has to be realised, it has to be emergent." And I'm sort of paraphrasing Jakob Howey here, and to a certain extent Mark Soames. So they emphasise now it is when these beliefs become sufficiently precise in a dynamical way that they are realised and influence other aspects of belief updating in a hierarchical or distributed system.

• 24:47Speaker 02

  So Mark Soames would call this felt uncertainty, so he would emphasise that the feeling part of consciousness just is, or is mediated by, the representations not of the content, but of the precision or uncertainty or the confidence with which these beliefs are currently in operation. And, you know, he would bring to the table all sorts of very compelling neurobiological evidence as to why this is, how you can switch off consciousness, literally by, I think he says it's, you know, a three-millimetre cube part of the brainstem that is the cells

• 25:28Speaker 02

  of origin, that these ascending systems that encode and represent, I repeat, not the content, but the confidence or the uncertainty or the precision of these conditional probability distributions. Jakob would, I think, say, "Well, to be aware is to equip all those messages that are providing evidence for your current explanation with precision," which has, you know, these physiological and, you know, bioelectric sort of mechanisms behind it.

• 26:04Speaker 02

  Other people go further, people in the world of phenomenology of the kind that Thomas Metzinger pursues, would say that it would be necessary to have control over the precision. And furthermore, to be aware that you are rendering that which was once transparent opaque, you actually have to recognize that you're attending. So again, not only do you have this very simple recursion of self-modeling in terms of planning as inference, but now you've got this hierarchical recursion where you're now recognizing that you're attending to something. And then to be, if you're somebody like Lars Sandvad Smith, you'd say, well, to actually be self-conscious, I now have to recognize

• 26:49Speaker 02

  that I was recognizing that I was attending to something. And you get layer upon layer upon layer. And to join the dots with Chris Fields, who has brought to the table the inner screen hypothesis, have you come across this yet?

• 27:05Tim Scarfe?

  No, no, tell us about that.

• 27:06Speaker 02

  Oh, well, strictly speaking, you should get Chris to tell you about that. But I'll give you a quick preamble. So Chris Fields is the theoretician who has formulated the quantum, quantum information theoretic version of the free energy principle. So for him, the Markov blanket that separates self from non-self becomes a holographic screen upon which classical information is, or to which it is written and from which it is read by some internal bulk and some external bulk.

• 27:43Speaker 02

  And in the sense that strange loops and strange recursions are only allowable when you have this hierarchical structure on the inside.

• 27:55Speaker 02

  You've now got lots of inner holographic screens, inner Markov blankets. I mean, Markov blankets, you know, in their pragmatic sense, just define things like hierarchies, for example, and, you know, they define the architecture of, you know, of any network, factograph or computer. And when you've got lots of them, you've effectively got lots of inner screens. So the idea, well, my reading of that idea is that there is one irreducible inner screen. So there's one inner screen that within it has no other screens. And this is an interesting screen, Markov blanket, from the classical perspective.

• 28:36Speaker 02

  Because the only way that the internal states of this irreducible Markov blanket can know themselves is by acting on the exterior, which of course the exterior is the rest of your brain. So you've got this metacognitive, self recursive aspect to this inner screen hypothesis.

• 28:55Speaker 02

  And that, that has, I think, a lot of mileage, that particular notion has a lot of mileage in relation to other people's theories of consciousness. So we're talking about higher order thought theory. So the very notion of higher order thought and its implicit appeal to metacognition, and meta metacognition, and meta meta, is again, this notion of looking at oneself and inferring oneself, but on the inside. So looking at, looking at the looking at the looking, which is of course, you know, a natural consequence of having this hierarchical structure. You could also argue it's completely compatible with global neuronal workspace theories that,

• 29:40Speaker 02

  you know, there is this sort of precision dependent ignition of certain sources or messages.

• 29:51Speaker 02

  Sufficient statistics of conditional probability distributions that gain access to, you know, to all of, across all of these inner screens in a dynamic way, because you're controlling the access by attending, by encoding and representing and optimizing the precision or the uncertainty or the confidence. So you've got this notion of ignition and penetration through to the global workspace, which, you know, you could read as this sort of irreducible inner screen, the core, the deepest part of your sense-making, your brain. So I think this notion, not only is it consistent with the, you know, the conditional independences

• 30:33Speaker 02

  that are implicit in the free energy principle, as applied to strange things that have this hierarchical and recursive aspect, but it also, I think, relates very comfortably with extant theories, all coming from different perspectives, seeing different parts of the elephant, as it were. But it's, you know, it is, I think, a comfortable accommodation of things that we presume, or things that people have brought to the table in order to explain consciousness, you know, from their direction of theorizing.

• 31:08Speaker 04

  It seems that if you, if we kind of split, let's say, the camps or the thought, the groups of thought about this, there's, almost all of what you just talked about now are, there is a minimal complexity required for there to be consciousness, and then we can, we can talk about what the, what that level is. Well, it requires this, or this, or this. But then there's also people who say, no, there's no, there's no minimal complexity. It's like electrons have some kind of small amount of consciousness. And I fall much more into the category of people who say, there is a minimum, like you have to reach a certain something. I don't really know where that boundary is, you know. Maybe it's one of these that we've mentioned, maybe we'll figure out a more elegant kind

• 31:51Speaker 04

  of categorization, but, but it's just like you need a certain number of pixels in the game of life before you can have a glider. You can't do it with three. Like you need, there's some minimum number. And so there's some minimum amount of brain material, you know, three cubic millimeters is still a lot of neurons. I don't know how many are in there, but, you know, hundreds, thousands, or whatever it is. You need some level of complexity before you have. Consciousness or before you have, you know, let's say recursive agency or something like or intelligence or whatever, would you agree with that? Like there is some, some minimum. We don't know where it is yet. It's hard to place that, but there is a minimum or not.

• 32:29Speaker 02

  No, I would agree entirely.

• 32:30Speaker 04

  Okay, so there, there is a minimum and not only that, it has to have a certain causal structure, right? And we can, we can kind of debate that. And that's really in line too with Searle, you know, in the Chinese room argument, which is like, look, a dictionary is not, doesn't have understanding because it doesn't have the right causal structure. You have to have a certain causal structure or a certain minimum complexity. And then you reach this, whatever it is, whether it's consciousness, we're talking about understanding, agency, all of these things, right? So I guess my question to you is, will we be able to build machines based on our current computer architectures some, someday, whether it's a hundred years from now, 200 years, doesn't matter. But in principle, can we build machines

• 33:13Speaker 04

  that have understanding, that have consciousness, that have all these capabilities?

• 33:19Speaker 00

  Progressive's Accident Response can contact emergency services if you're in a major accident. The only thing worse than someone you love being in an accident is not being able to help them. So Progressive is here to help when you can't. So get peace of mind knowing your loved ones are protected with Progressive's Accident Response. Learn more at progressive.com. Progressive Casualty Insurance Company and Affiliates. Limited availability in California. Driver is responsible for emergency and towing charges not covered by the policy. Coverage subject to policy terms and conditions. Hey Spotify listeners, when you travel with American Airlines, you'll enjoy comfort and convenience at every altitude. Before takeoff, unwind in the Admiral's Club Lounge with curated self-serve offerings of hearty bites and refreshments. In flight, stay connected

• 34:03Speaker 00

  with high-speed Wi-Fi. Sponsored by AT&T. Your journey is designed to feel easy from the moment you check in to the moment you arrive. No matter where you're going, American gets you there. Book now at aa.com.

• 34:16Speaker 02

  Yes, that's a good question. And the answer, I think, in principle, yes.

• 34:23Speaker 02

  Can I just come back? Yeah, absolutely. And qualify that answer by reference to that wonderful example of, you know,

• 34:31Speaker 02

  what I would read as vagueness, in a technical sense. So vagueness, you know, how many grains of sand constitute a pile? Yeah, so it's not well-defined. It is in philosophy, but not in mathematics. I think that's absolutely the right way to think about these, you know, the bright lines. And if I had to commit to the dimension, the number of grains of sand that you get before you have consciousness or there is a pile, I would say, it's something you actually referred to earlier on. I think it's the depth of your future.

• 35:12Speaker 02

  Or your future in your head. So if you're talking now about an algorithm or some artificial intelligence that is equipped with a generative or a world model of the consequences of its action, there will be a time horizon associated with that component of its generative model. And I think that it's the depth of that time, what Anil Seth would refer to,

• 35:41Speaker 02

  not as counterfactual breadth, which is the number of divergent paths one could take into the future, the options that you select among, but the counterfactual depth, the temporal depth. And so that means that you can say, you can be panpsychic. You can say that, you know, a thermostat has a notion of the future in the sense that it operates of, you know, sort of path integral control and differential equations. So you put a differential equation in play, you've got, you know, an instantaneous future because you've got some gradient with respect to time. That's not, though, the kind of depth that you and I enjoy.

• 36:25Speaker 02

  So I would imagine it is really just the depth. You are very, very reflexive, and I think people like Maxwell Rammstead talk about this as merely reflexive active inference, with, you know, very myopic, very short-term self-models, right through to fully, well, to be conscious in the way that we've been talking about you, I think you'd need to have, you know, a long depth. So what does that mean for building AGI or conscious artefacts or machine consciousness? It means, first of all, they have to be agentic, because we've just said that having, you know, a world model of the consequences of your action

• 37:08Speaker 02

  would be necessary to be an agent. But more than that, you'd have to look quite a long way into the future. So your generative model, your world model, would have to go quite a long way into the future. And I repeat, have that both counterfactual breadth and counterfactual depth at hand. Would that be sufficient? And I'm now remembering that I forgot to mention a Neil Seth in the list of people not to upset when reviewing theories of consciousness. So this is an opportunity just to say that there are people out there who would say, "Well, you know, okay, you can write down the maths of all this, and you can write down industry hypotheses,

• 37:53Speaker 02

  or indeed simulate global neural workspace theories and try to produce things that look as if they have consciousness." That's not going to work unless you actually embody it, unless you are, in a Neil's words, "a beast machine." I think that that sort of coheres with the argument for mortal computation.

• 38:20Speaker 02

  So when you may ask the question, can't we build it on our computer architectures? I would have to ask you, do you mean a von Neumann architecture? Or do you mean a memory, processing in memory or in-memory processing architecture? Which I would take as synonymous with a neuromorphic architecture. Not spiking neural networks, you don't need those. But you do need the processing in memory to be, you know, to be mortal. You need that substrate dependence, you know, read, you know, in terms of Jeff, Jeffrey Hinton's definition of mortal computation and, you know, Alex's subsequent elaborations of that. So if you've got, I think, I would subscribe to that.

• 39:05Speaker 02

  I think largely to keep a Neil happy, but I would subscribe. I don't think you can do this on a von Neumann architecture. Because the Markov blankets of a von Neumann architecture, where you're reading and writing from memory, make it very difficult for the memory to self-organize. I see. And people have written about this philosophically. I think Vanja Weiss has written a paper about this. And I think Anil Seth speaks to this argument in a recent behavioral and brain science paper.

• 39:40Speaker 02

  You know, it may not be possible. And certainly, you know, from the point of view of efficiency, it's highly unlikely that sort of von Neumann architectures are the kind of things that would conform to. Let me reverse. To be is to pursue a path of least action in accordance with the free energy principle. To be conscious does not excuse you from that. So if, if you want a conscious machine, you have to have a machine that pursues a path of least action. Via the Janinski equality, that has to be true both themodynamically and informationally in terms of the conditional probability distributions.

• 40:22Speaker 02

  If you don't pursue that path of least action, you can't be and you can't be conscious for a non trivial amount of time.

• 40:30Speaker 02

  And I can see you want to ask a question.

• 40:34Tim Scarfe?

  Well, I need to say Maxwell pointed me to Anil's, where he defends biological naturalism. And of course, even Seale said that the biological substrate is an existence proof. It is not saying it has to be biology, but Keith and I certainly agree that there's something about the substrate which is which is very important.

• 40:54Tim Scarfe?

  I wanted to talk a little bit about viruses. As you know, I'm a bit of an externalist. I've never been able to completely pin you down, Professor Friston, because there have been so many interpretations of the free energy principle that that lean internalist and externalist and even the hybrid version, which Maxwell also wrote a paper about. But I'm fascinated in this idea of diverse intelligences. And for example, could a virus be intelligent? And I spoke with David Krakauer and he said intelligent things do inference and they have representations and they're adaptable. And we are a little bit, you know, we're a bit chauvinistic about our brains, aren't we? Because our brains seem to have a privileged status. But what say you of viruses?

• 41:37Tim Scarfe?

  Or actually, we read your paper with Calvo predicting green, really radical plant predictive processing. And in that paper, you gave a beautiful account of how even plants could be doing inferencing. And that to me seems, it seems incredible because I'm amenable to the idea. But it seems to me intuitively that plants are not as sophisticated as we are. So it comes back to this line that Keith was talking about before, that if you go too far down the stack, it's an account of, let's call it pan-intelligence, where you have a tiny little bit of intelligence, even if you go all the way down the stack.

• 42:19Speaker 02

  Have you spoke to Mike Levin?

• 42:21Tim Scarfe?

  Yes, relatively recently, about a year ago.

• 42:25Speaker 02

  Right. I mean, you know, it would be nice to.

• 42:31Speaker 02

  Revisit him, I mean, that is exactly his big question at the moment. And of course, he has as an intellectual accomplice, Chris Fields, with him as well. So that, you know, the two of them are really pursuing this notion of basal cognition, that there is intelligence everywhere. And it's just a question of, you know, how we conceive of it and how we test for it. And indeed, his argument, Mike's argument, is that, you know, you have to design the right experiments. It's an empirical question. Is this virus intelligent or not? Well, you have to, you know,

• 43:12Speaker 02

  design the right experiments to disclose or evince its intelligent behaviour. And Mike would claim that, you know, by many metrics, many rules, yardsticks that we use to measure adaptive intelligent behaviour. Yeah, slime moulds and viruses and xenobots are incredibly intelligent. And, you know, I think he's fighting against the chauvinism that you mentioned. That intelligent things are just properties of things like you and me, creatures like you and me and our brains. And that that kind of cognitive capacity and competence can be found everywhere. So I, you know, and I'm very sympathetic to that.

• 43:54Speaker 02

  But it does tread on the toes of the vague argument. You know, I certainly don't think that viruses have the same expressive kind of agency that we were talking about. And indeed, you know, the counterargument to the vague notion of intelligence and/or consciousness is of course hitting you in the face when you, you know, when you read that Strange Things paper. You know, I'm talking about categorically different natural kinds that do and do not have, for example, a causal power or influence of active states or internal states. You're either one of these or you're one of these.

• 44:38Speaker 04

  I'm not so sympathetic to the view. You that, you know, a virus is intelligent, right? Because I think this falls into the category of things like where you're in a biology class and you learn how to define life. And then somebody's like, what about fire? Then like, it seems to meet all these criteria here, you know? It grows, it expands resource. You know, so for example, when a virus quote mutates, the virus doesn't mutate itself. It is mutated by a gamma ray hitting it or a, you know, transcription error or whatever. And and oil, you know, vinegar and baking soda undergoing a reaction is not intelligent. Right? So there's, I think there's some level of complexity and, you know, whether it's recursion or these other types of causal structures that have to be there

• 45:22Speaker 04

  before I'm even interested in talking about intelligence or entertaining intelligence. You know, other things are just kind of dynamics that unfold in certain ways, chemical reactions, you know, that that sort of thing.

• 45:34Speaker 02

  And I think that that point is a nice opportunity to introduce the notion of scale freeness, or scale invariance. Because I, you know, I can see you could also take this towards collective intelligence and federated learning and federated inference.

• 45:53Speaker 02

  So it's not, it's not the single cell. It's a single cell with its neighbors and its neighbors of neighbors and neighbors of neighbors. And so what I was looking for is some conservation of intelligent dynamics that is preserved over different scales. So, you know, the single virus is probably not interesting. It's probably the colony, which is, which, which is, which is interesting. So I thought that was, you know, a nice point to introduce the notion of that kind of scale invariance. And of course, as a mathematician, you would be looking at the renormalization group to see how that unpacked mathematically. And interestingly, it also speaks to this notion of a mutation.

• 46:36Speaker 02

  Is evolution an intelligent process?

• 46:41Speaker 02

  You know, it's certainly adaptive. It certainly has the level of complexity that you would require in order to pass those vague thresholds. But is evolution in and of itself an intelligent process? It's certainly a free energy minimizing process. It is just basic model selection. Natural selection just is selecting those things that have the highest model evidence of marginal likelihood of being that phenotype in this kind of environment.

• 47:13Speaker 04

  Well, it's like, is the planet intelligent? You know, it certainly contains eight billion or so of us, so does that count? I mean.

• 47:22Speaker 02

  Yeah.

• 47:23Speaker 04

  Why not? Well, I mean, that's where, well, I would say like, and we talked about this way back when we talked about, you know, is a flotilla an agent? Or is it the pilot who's, you know, kind of, kind of controlling the convoy or whatever? I would say like, if there's a more minimal boundary that contains the intelligent, then the larger one is not. Like I'm talking about, so the planet, for example, I would say, since I can draw a smaller boundary, which is an individual person, the individual person is intelligent. Anything that contains that person is just more stuff.

• 47:57Tim Scarfe?

  Keith, what is your, the issue with viruses? Is it because the individual virus is inert? Whereas a cell, for example, you can partition down to the individual cells and the cell still has some degree of agentic property, as Professor Friston was describing? Is it that you see them as inert and being carried by something else?

• 48:18Speaker 04

  Well, I think it's too much of their causal structure and machinery is basically outsourced to other things. It's like they, you know, they're kind of just these little particles that go around and if they happen to stick on a cell, they're like a mousetrap that activates and just injects some DNA. But, you know, the cell does all the rest of the work, right? It does the transcription, it has all that machinery. By themselves, a virus doesn't change over the course of its lifetime. It doesn't really. Other than this simple mousetrap kind of activation that does nothing, you know, it doesn't, doesn't have any machinery to learn and adapt by itself, like over the course of the lifetime of a virus, you know, so I might be more amenable to like if somebody

• 48:58Speaker 04

  says, well, like, an ecosystem of, you know, viruses or something has some degree of, you know, intelligence, but it still doesn't have the, the type of processing and causal structure and minimal complexity that at least for me is the useful concept of, of intelligence.

• 49:15Speaker 02

  So I think we're coming back now to the, the number of inner screens and the counterfactual depth and the definition of agency in the sense of strange things that we're talking about for this particular scale. So if we just read the scale as, if you like, the size of the Markoff blanket, if within, at this scale, for this kind of thing, there is a sufficient degree of complexity, read specifically though, in this instance, as the counterfactual depth and breadth of your world model about the consequences of your

• 49:55Speaker 02

  action, that particular part of, of, of, of your implicit generative model. And that would happily accommodate the fact that when you get something, you know, something of the size or the scale of a virus, there just isn't the machinery or the space to entertain that in any non-trivial, in a non-trivial way. One could also argue, and I got a sense that you were arguing yourselves towards this, that if you go too big, you also lose that. So, you know, this, you know, coming back to this wonderful question, we have eight billion intelligent, really intelligent, I sounded like Trump there, didn't I?

• 50:38Speaker 02

  Entities constituting our biosphere, but is the biosphere, is, you're from the point of view of saying the Kahar hypothesis, is the biosphere in and of itself intelligent? And I would say no, not because, it's simply because at that scale, the elemental, all the complexity of the constituent elements disappears at that scale. So, you know, it'd be a little bit like saying, well, let's just take it to the limit, let's just take it to the astronomical limit, you know, of the motion of heavenly bodies.

• 51:18Speaker 02

  Now, the motion of heavenly bodies is completely described by the position of the planets and the moon, right, and the position of the earth. So, at that scale, you've averaged away the fact that the earth contains a biosphere, and the biosphere contains human beings, and human beings contain cells, and the cells may or may not contain viruses, depending upon who you've been exposed to. So, there will be no intelligence at that level. So, it's perfectly possible to have intelligence at a particular scale that disappears when you get too big and when you get too small.

• 51:57Speaker 02

  Another way of looking at that is the, going right back to things like Bragg-égin and dissipative structures, you know, you're talking about complexity, if we just think about, you know, how have people tried to understand complex systems and self-organizing systems that are open? So, we're not talking about 20th century physics and equilibrium physics, we're talking about the physics of non-equilibrium, and things that are open, in open exchange with each other. And of course, you get to the notion of dissipative structures. What does that tell you mathematically? Well, what's not dissipative? That's a schoolboy question. Can you remember? You've probably forgotten. So, what is not dissipative is conservative.

• 52:42Speaker 02

  Oh, got you. Yeah, so I'm teasing. So...

• 52:51Speaker 02

  Another gift of Helmholtz, of course, is that any dynamics can be partitioned into a dissipative part and a conservative part. And the dissipative part is that which rests upon very fast, random, complicated fluctuations of the kind you might find in, say, quantum mechanics or thermodynamics, whereas the conservative part does not rely upon that and just goes round in circles, basically. It's, you know, literally it's called solenoidal flow, or conservative flow.

• 53:30Speaker 02

  So that basically means that dissipative structures have to have an admixture of this circular aspect, this conservative, classical life cycles, reproduction, oscillations, plus the random dissipative part that you'll find in things like thermodynamics and quantum mechanics, and that is definitional of dissipative structures. As you go too small, then everything becomes quantum and random. It all becomes probabilistic.

• 54:12Speaker 02

  There is no conservative stuff, other than a scrotting of potential,

• 54:18Speaker 02

  at which point I think you'd find it very difficult to find something that was intelligent in the sense we're talking about, because we have to have this recurrence, this solenoidal aspect, in order to revisit the states, so you know a virus is a virus, in order for there to be an equilibrium steady-state distribution as a solution to that. So, but as you get bigger and bigger and bigger, you get to viruses, and you're still not quite, they're still not quite sort of complex enough to be intelligent in the way that we mean. And then you get to our size, and that's perfect, because we both have both this dissipative, we deal with a random world, with an itinerant world, and yet we keep revisiting states of being.

• 54:58Speaker 02

  So we have this sort of conservative, biomimetic kind of self-organization. But then you get bigger and bigger and bigger, you get to the level of the biosphere, or, you know, the size of the...

• 55:12Speaker 02

  The moon or the sun. And of course, by averaging, all the random fluctuations go away. So you're just left with the solenoidal part. You're just left with the solenoidal motion, the Newtonian motion of classical mechanics.

• 55:27Speaker 04

  Because there's, so one thing that really fascinates me about this is, however, it is possible for us, and we do, construct large-scale things that are intelligent, like a corporation. You know, a corporation is seen as like a superintelligence. But we're only able to do that by putting in structure, right? That maintains this balance of the dissipative and conservative flow, right? Like, we have to put in organizational structure so that it continue to function at these larger scales, right? Like, isn't that pretty, pretty interesting? I mean,

• 56:04Speaker 04

  it, it's, it's again this yin and yang thing that we run into all the time, right? Where it's the balance between two, two forces. Like, either between, say, complete order and complete noise, or between dissipation and conservation. And you have to be almost on the edge of chaos. And it has to have a certain causal structure in order for it to, to be intelligent.

• 56:26Speaker 02

  No, no, I agree entirely. And, and, you know, that, that, that sort of Goldilocks regime, where you are on the edge of chaos, I think is quite specific to a particular scale. Certainly, you could, you could invoke a sort of strong anthropomorphic principle here and say that, yeah, the kind of intelligence that we will recognize has to be at our scale. But I think there's something more fundamental than that. I think, I think the very existence, if you subscribe as an externalist to quantum physics and Newtonian physics, or Lagrangian classical mechanics. I, you know, I, I, I think that there is a Goldilocks regime, which means that we can only exist at this scale with, with, as you say, this sort of yin yang,

• 57:08Speaker 02

  this admixture of dissipative dynamics and conservative dynamics. You know, just to reinforce this, because the conservative dynamics is absolutely essential, because it is that which causes this Poncaré It defines these strange attractors, which is the other way of licensing the notion of strange things. You always come back to somewhere near where you started, like Red Queen Dynamics, you know, in theoretical biology. So you have to have the conservative, the circular motion, just to have a routine, have biorehythms, replicate, reproduce, many, many different levels at many different scales. But it's not, it's always remarkable in the

• 57:52Speaker 02

  face of a dissipative, itinerant world. Everything is changing all the time. And yet, we somehow sort of resist that change by being at the edge of chaos. Just, oh, no, you're more anxious than I do to say something. Carry on.

• 58:06Tim Scarfe?

  Please go on, Professor Preston.

• 58:08Speaker 02

  No, no.

• 58:09Tim Scarfe?

  Well, I mean, I'm just so fascinated by your observation that with Gaia theory, for example, when we zoom out, the apparent phenomenon seems less intelligent. And as you said, maybe intelligence is just what we recognize. And when we spoke with Wolfram that time, he was talking about how we are computationally bounded as observers. And we do this thing called abstraction, where we ignore details, or even idealization, where we deliberately distort the truth. And David Krakauer said that intelligence is doing more with less, and emergence is more is difference. And, I mean, the Earth is not doing, it's not doing less, we just don't see what it's doing. But if it's, if emergence

• 58:54Tim Scarfe?

  is more is different, then that, it licenses a fundamental reorganization of the underlying substrate, which means it isn't actually doing the thing at the lower level anymore. It's a fundamental coarse graining, and it's actually changing at the higher level. So which is it?

• 59:14Speaker 02

  I didn't realise there's a choice there, because I agree with everything you've said.

• 59:20Speaker 02

  But I like the notion of coarse graining, because of course that is exactly what you get from the renormalisation group treatment of these things. And you can simulate this. You can simulate free energy minimising processes that are running at different scales. And then you have to ask the deep questions of how you couple between the scales. And there you get to supervenance and emergence mathematically so defined. But it all boils down to exactly where you started and where you ended up, which is a coarse graining, in the right kind of way. So, you know, to actually write down a renormalisation group, you have to have an RG operator. What is an RG operator? Well, it has two parts to it. It has a sort of dimension reduction, the R part, if you like, and a grouping operator.

• 1:00:04Speaker 02

  And they basically do the right kind of coarse graining. They reduce dimensionality and they group together in the right way. Those states of being at the highest scale, and so on, ad infinitum. So I think the notion of coarse graining is absolutely essential here. And on that level, you could argue both ways. So I'm getting a sense that your question is, is there a true emergentism here, or not? Again, I don't have a philosophical train, so I'm not sure I can really answer that.

• 1:00:43Speaker 02

  But certainly, the intelligent dynamics read as self-evidencing perspective on a free energy minimising process are recapitulated at a completely different scale, in a different way, at each scale, through the recursive application of RG operators. So, yes, there is something brand new going on. And yet, in the spirit of, say, Hakenzenergetics, these RG operators are just taking functions of stuff that's happening at the finer scale. So you're not inventing anything new.

• 1:01:26Speaker 02

  It's just predicated or emerging or supervening on finer scale stuff. But what emerges at the high scale has all the attributes of self-evidencing, and you could argue consciousness or intelligence at some level. But coming back to...

• 1:01:43Speaker 02

  The point about building bigger organizations. If the argument that as you get bigger and bigger and bigger. Means that you are effectively to exist, you have to be effectively conservative. So a company, for example. What is its metric of goodness is how long it survives. You know, can you get from series eight as series whatever. In some recognizable form. So as you get bigger and bigger and bigger, I think there's less opportunity for this complex scale free within it's not hierarchical recursive structure within the scale. So I do, I don't see the moon thinking or planning.

• 1:02:28Speaker 02

  I don't see the weather planning. I don't see evolution planning. It doesn't think about its future. It's too big. And I would imagine that you could apply the same arguments to globalization and institutions that get too big for their own good because they can't plan anymore. So we're coming back to the pilot who's really the intelligent person, not the flotilla.

• 1:02:52Speaker 04

  Right. Yeah, it's in a way it's. I guess maybe it's slightly depressing to me because it means we can't build an intelligent intergalactic, you know, civilization. It's it's almost like beyond a certain scale, we just have to. Leave it up to kind of distributed emergent. You know, processes that that may or may not end up doing something intelligent as a whole. You know, because because at some point you've got the speed of light. And that's that's basically going to be a barrier to organization right across across light years, right? So so maybe there is an ultimate Goldilocks limit, right, to to intelligence can't it can't get too big.

• 1:03:38Speaker 02

  That's certainly what the, you know, the arguments from the physics part, you know, so there is a golden locks, a golden locks of scale or zone in scale space. I'm not sure this is correct, and I don't want to disappoint you, so.

• 1:03:54Speaker 04

  Don't worry, it's well beyond my lifetime when that will happen.

• 1:03:59Speaker 02

  There's a certain beauty, though, in federating and distributing and, you know, thinking in terms of ecosystems of things at a particular scale. I mean, evolution is a beautiful thing.

• 1:04:10Speaker 04

  Yeah, yeah.

• 1:04:11Speaker 02

  But, you know, it doesn't have to be intelligent to be beautiful.

• 1:04:15Speaker 04

  Correct, correct. Well, good point. So it could be, it could be beautiful but not intelligent. You could have a beautiful distributed civilization. That's fine. I'm happy with that.

• 1:04:24Tim Scarfe?

  On the internalism and externalism debate, I'm a huge fan of Andy Clark, as you know, and he famously argued in his, was it 1999 paper with David Chalmers, that the phone extends your mind. And even when we were talking about the plant example before, when we were talking about plants doing modeling, there's an inactive interpretation there, and Keith and I were arguing about this, he doesn't agree with me, but is the model the, the physical morphology of the plant, you know, it fits the environment like a key in the lock? Or is the model the software? Is it the DNA of, of, of the plant? And indeed, it's just fascinating, isn't it, that, that you can think of cognition not

• 1:05:08Tim Scarfe?

  being entirely inside our heads, but it's just this unfurling causal bidirectional process of, of, of so many things around us. So how do we actually draw boundaries around things where we say, okay, well, here's the, the principal modeling in cognition is, is in here. This is where bona fide beliefs are happening. This is where bona fide thinking is happening. Is it even possible to make that distinction?

• 1:05:36Speaker 02

  I think it is, I'm answering in an almost trivial way. If you want to talk about something, you have to be able to define its Markov blanket. So yes, you have to be able to draw lines, bounds and lines around things to talk about them. I think that your question though speaks to something that we were, that we'd just been covering, which is the separation of scales and that everything, everything, literally, from the point of view of the free energy principle, that is equipped from the Markov blanket, is contextualized by a scale above. And the same rules apply to the scale above as to the thing itself.

• 1:06:20Speaker 02

  So you have to have a context in which everything is operating. So the virus may not be intelligent, but it certainly has to be living in a world which is conducive to its existence. And that world and the states of that particular world at the scale above, for example, the host cell and the host organism, has to comply and be intelligent in some sense in order for the virus to be there, even if the virus itself is not intelligent. So applying that to the plant, but we should come back to the extended cognition and Andy Clark. But to the plant thing, you made an interesting point, is it in the DNA?

• 1:07:02Speaker 02

  Or is it in the morphology and the phototactics? Everything else that plants possess and we infer goes on on the inside. First of all, I think you're absolutely right to say, yes, it is in the morphology. And in a sense, that's what I was getting at when talking about the importance of mortal computation for machine consciousness, that you have to...

• 1:07:34Speaker 02

  To realise a basing mechanics or self-evidencing, you physically have to parameterise your conditional probability distributions, your basing beliefs about the environment with, you know, to which you are coupled. So it is, the substrate is the parameterisation. So by definition, it's substrate-dependent computation, which means the structural form, the morphology, is the structure of the genetic model in the spirit of structural learning. So that to be part, and very much in the spirit of the good regulator theorem, you know, to be in my world,

• 1:08:14Speaker 02

  I have to physically encode and embody a model of the structure of my world, which means that I have to have, if you like, if you think your world is scale-free, that means that my brain must have some scale-free hierarchical aspect, and indeed it does. And you can actually write that down and model it with a renormalisation group. And that's just a reflection of the fact that I'm immersed in a world that has some scale invariance in it. So that is certainly true for the plant. And just one little aside here, that paper was written before David Attenborough's series on the life of plants,

• 1:08:58Speaker 02

  where he was able to speed up by a factor of 10 or 100. And of course, if you look at plants doing stuff, they're conspecifics by sending their little roots and off in a particular direction, or you're eating insects or whatever. If you speed it up, these things are very animalistic and they look, you'd be hard-pressed to say that they weren't intelligent, irrespective of whether they're conscious or not. But they certainly start to look much more like you and me when you speed things up. So, yeah, their morphology matters, matters more than one could possibly imagine.

• 1:09:38Speaker 02

  Because this is how you become a good regulator. You model your environment. You are, you install that cause-effect structure into your computer architecture, which is again the argument against von Neumann architectures, which is, you know... Which is why um one might, I think, um look to all the um processing in memory, neuromorphic photonics, um possibly quantum computation, but I think that's gone off the boil recently, uh but the um you know, all that processing in memory stuff, uh memory stuffs, for example, I think that's where where the answer, the answer will be. Is it

• 1:10:23Speaker 02

  in the DNA? So is it, does the DNA cause a structure? So.

• 1:10:29Tim Scarfe?

  Okay, your your comment, the expression, to make that point.

• 1:10:32Speaker 04

  Yeah, so I think um I I think some of the hang up or crux may actually be like what we mean by model, and I'm sure this is a whole philosophical debate that I'm probably not qualified to, you know, uh decide. But I think, for example, um you know, if you're flying an airplane, like the airplane internally has a computational model of the airplane and you're interacting with this computer and through controls that then are enacted through all kinds of gears and cables and levers and things like that. But I think of the model as being the thing that's in the computer, and then it's modeling the the airplane, right? And so in that sense, um you know, the the DNA of of a plant species

• 1:11:13Speaker 04

  must incorporate the code for the model, right? Because because that's what unfurls is the plant, it's the same DNA in every cell that's enacting a program that's essentially happening through gene expression, you know, and so a particular cell, maybe some salt content is in there which causes it to release a hormone that binds to all the other cells in the plant, which activates certain genetic pathways. So I kind of think about in that same analogy, you know, the the core of the model is is the code, you know, that's running and operating and controlling these processes. And then I think of the embodiment of the plant as the stuff that the model is modeling, right? But I guess, you know, there's some some vagueness there too, so.

• 1:11:57Speaker 02

  You could, um if you wanted to simulate these kinds of things, um you could read the DNA as the code that specifies, the prize that specify the structure. Um and, you know, I think that um if if if you think of DNA.

• 1:12:17Speaker 02

  As prescribing the structure of your generative model or your world model or your factograph that will be fit for purpose and is learnable. So you start off with, say, DNA. DNA does not tell you what particular kind of plant you're going to be, where it's not going to tell you how to engage in your phototaxis and point towards the sun or compete with other plants that are trying to deny you sunlight. That is something that you have to update and learn during your particular lifetime. But you're equipped with the basic structure, the prior on the structure of your generative model.

• 1:12:59Speaker 02

  And of course, that is most gracefully accommodated, I think, again, with respect to the renormalization group, we're just talking about two scales. So you've got a slow scale where you've got the you mentioned before viral mutation. So the viral DNA, you should have RNA or DNA,

• 1:13:20Speaker 02

  is changing very, very slowly on a time scale that is greater than or equivalent to the lifespan of any given virus. And that's, if you like, specifying the initial conditions, the structure for the specification of a particular instance of a virus.

• 1:13:40Speaker 04

  Maybe just jump in for a minute because I think actually it's not even specifying the structure. I think, and this is what you pointed out before about the difference between, say, active inference versus modeling, is actually the DNA is specifying a policy.

• 1:13:54Speaker 02

  Yes.

• 1:13:55Speaker 04

  It's effectively just specifying a policy that every single cell in the organism follows. And so this policy is not instructions for a structure, but it's just instructions on what to do in response to a certain set of sensory states. Right? So it's almost, it's the policy that the DNA specifies and this policy applies to every cell. And then somehow miraculously, it works out to create a structure that is fit for purpose in its particular environment.

• 1:14:25Speaker 02

  Yes, yeah. And of course. To work, that specification has to change at the same rate at which the environment changes, which is varies very slowly. I think that's absolutely right. I mean, just practically, that hits you in the face when it comes to thinking about how you commit to one policy or another policy. So in my world, that would be the expected free energy. The expected free energy comes in two parts. It has a sort of expected information gain and the expected cost or constraints or utility. And that is at the point you need your DNA. You need to know what it is to be the kind of thing that I am. What do I not do and what do I do?

• 1:15:07Speaker 04

  Well, the fascinating things in organisms is this is different even on a cell by cell level, right? Because morphologically, the same code evolves into all the different organs and parts and roots and, you know, bark and whatever else, which is pretty fascinating.

• 1:15:22Speaker 02

  Yeah, and it's exactly the kind of fascinating issue that preoccupies Mike Levin. And how do you?

• 1:15:32Tim Scarfe?

  I spoke with our mutual friend Maxwell Ramstead the other day, and he actually gave a wonderful description of the free energy principle. Because you know, what you want in a good pitch is for it to be replicable, easily understandable. And I'll try and recapitulate it to you just to test how replicable it was. He's, you know, second law of thermodynamics, closed systems. What if we have open systems with boundaries? And he was saying, when things can't merge together, they instead share information with each other. So they kind of, there's this informational synchrony. And that was his way of describing the free energy principle. But first of all, is that a good description? But number two, the boundaries.

• 1:16:16Tim Scarfe?

  Where do the boundaries come from? Because we want to have practical implementations of the free energy principle. And of course, we can implement it with computer games or contrived environments where the boundaries are clear. But what if I want to build a robot and I have a camera and I see all of these pixels on the screen and I need to divvy that scene up into objects so I can start doing the free energy principle? How do we do that?

• 1:16:40Speaker 02

  Right. Five minutes, five questions.

• 1:16:43Speaker 04

  It's got to be practical. That's why we're only giving five minutes.

• 1:16:48Speaker 02

  Right. First of all, the notion of

• 1:16:55Speaker 02

  Being separate but part of a universe through synchronization, I think is absolutely correct. And could be unpacked in two ways. One, you could say that, you know, to find a free energy minimizing solution to your dynamics, namely the path of least action, just is to evince a generalized synchrony between the inside and the outside. Generalized synchrony is also a synchronization of chaos. So it's basically the two systems that are loosely coupled, in the sense of dynamical systems, will ultimately converge on a synchronization manifold, and they will show chaotic dynamics on that manifold.

• 1:17:39Speaker 02

  That manifold is a synchronization manifold, and it is being on that manifold that is the minimization of free energy. So you can talk about the free energy principle without mentioning self-evidence, without mentioning Bayes, without mentioning predictive processing, or even extended cognition. And you can just talk about it as a variational, specifying a variational bound on the Lagrangian that specifies generalized synchronization, or synchronization of chaos. And, you know, what you're talking about is exactly this sort of separate but the same.

• 1:18:17Speaker 02

  I think quite nicely, although I don't fully understand it, recapitulated in Chris Field's quantum treatment. So what he would talk about there is not the synchronization between the inside and the outside, or me and you and everything else like me and you, but entanglement. So the principle of unitarity is this basic, well, you can read the free energy principle as just the principle of unitarity, which just means if we stay here for long enough, for more than five minutes, we will ultimately become completely entangled, which means classically, we will be engaged in a generalized synchrony. There will literally be a synchronization of our antinerrant dynamics.

• 1:19:02Speaker 02

  We will not become one, we'll still have a... Well, actually, Chris Fields thinks you would become one, but we will be indistinguishable because we're just playing out on exactly the same synchronization manifold. So I think that's absolutely right. Your last question, and you had about three in between, was how would you practically use the free energy principle, the notion of thingness that inherits from Markov blankets practically if you're building robots? I think you're talking here about physics discovery or Markov discovery algorithms, which of course Maxwell and Jeff Becker are furiously working away on to try and...

• 1:19:42Tim Scarfe?

  Assuming you didn't know anything about that stuff, I mean, how would you approach that problem? From first principles, if you had a robot and a camera, how would you partition the scene into boundaries?

• 1:19:54Speaker 02

  You would be appealing to the notion of Markov boundaries. And you can turn it on the head and look at image segmentation as just what has emerged in computer vision as a way of identifying things. So what you're doing is you're committing to an implicit world model or generative model for your robot. You're assuming that it is going to be operating as a good regulator, as a good model of its environment, in an environment that is composed of things.

• 1:20:26Tim Scarfe?

  Yes, but I think my intuition is, we've been talking a lot about understanding and creativity recently, and I intuitively feel that understanding is about knowing the history of something. It's about knowing how you got there. It's not about knowing the state. So it's a little bit different from segmenting an image. To understand the history and the dynamics of a system seems to be much, much more powerful.

• 1:20:49Speaker 02

  Absolutely, yeah, yeah, no, absolutely, which is why, you know...

• 1:20:56Speaker 02

  Markov Blanket Discovery is not image segmentation because to do Markov Blanket Discovery, you have to look at the dynamics and the history.

• 1:21:04Speaker 04

  Yeah.

• 1:21:05Speaker 02

  Yeah, I mean, there's quite a fundamental point to be made here that, you know, we're talking about, we have been talking about from the beginning right through to the end about conditional probability distributions. Probability distributions over what? Because there's only one state of the universe at any one time. There are not many worlds, there's not an ensemble of universes. Of course, for the purpose of this argument, there is only one state of the universe at any one time. So you can't have a probability distribution unless you invoke some sort of ensemble, canonical ensemble assumption, as they do in thermodynamics, where there's some exchangeability, you can swap around universes and have a distribution.

• 1:21:45Speaker 04

  You can have an epistemological probability distribution, right?

• 1:21:48Speaker 02

  I would argue, though, that what you're implicitly doing is actually doing that over time. There's a history, a past and a future, that that epistemological notion really has to refer to what are, what is, what is the background over which these multiple realizations occur, from which you can select to build a probability distribution. And of course, from the point of view of the Freihenig principle on also building robots, this has to be time. So by definition, it has to be in the dynamics and the history, and not just the short-term dynamics, but this recurrence, the characteristic states that I keep returning to as a robot, as a good robot.

• 1:22:31Speaker 02

  So, yeah, absolutely. So segmentation algorithms are, you know, a good start, but they're going to be completely useless when it comes to autonomous vehicles, for example, unless you've built in the fact that there is conservation of this particular Markov blanket over time, which of course they do. But it's probably better to start with the generative model, that this...

• 1:22:58Speaker 02

  Two dimensional RGB feed has been generated by Markov blankets that by definition persist over time because that is the time is the support of the condition distributions that define the existence of the Markov blanket. And then it's a question of how many, what kind of priors do you put on these things? Are they things or this stuff, you know? What's the Markov blanket of water? Or perhaps fog, you know, if we're doing autonomous. And do they, to what extent would I now apply my priors to these things?

• 1:23:40Speaker 02

  And you can go right through to sort of object centric priors, you know, of the kind, you know, sort of physics engine based stuff of the kind that say Josh Tenenbaum pursues, assuming that you've got some Newtonian behaviour. Or you could be much more relaxed. I would relax, but not beyond the renormalisation group.

• 1:24:03Tim Scarfe?

  Professor Friston, it's been an absolute honour. Thank you so much for joining us today. It's been amazing.

• 1:24:07Speaker 02

  I'm really enjoying it. It's a lot to speak to you both again.

• 1:24:09Speaker 04

  Yeah, and it's really been an honour for me because this is the first time I've met you in person. So, and you know, I've been thinking about you for many years. So it's been, it's been really a pleasure to meet you and thank you for coming to the studio today.

• 1:24:23Speaker 02

  Thank you for coming to England.

• 1:24:25Speaker 04

  Absolutely, anytime.

• 1:25:01Speaker 05

  But you're running to get game day deals on Uber Eats. Keep everyone at tonight's watch party fed with deals from your go-to restaurants. Or stock up with deals on your favorite snacks and groceries. Whatever the occasion, get game day deals on Uber Eats all season long. It's not football without food. Uber Eats. Order now. Terms apply. Product availability varies by region. See app for details.