Why Leibniz Couldn't Find the Mind Inside the Mill (Joscha Bach)

Joscha Bach is a cognitive scientist and the founder of the California Institute for Machine Consciousness. SPONSOR: --- Parallel is a single platform for all your agentic research: web search built for AI agents, with Pareto-optimal quality, cost and speed. Get started: https://parallel.ai/mlst --- Bach's null hypothesis is that consciousness is easy: an algorithm not much more complicated than the transformer, simple enough for an infant's brain to discover by itself. In this conversation with Tim, he lays out the Machine Consciousness Hypothesis in two parts. The first is that consciousness is a biological learning algorithm, the training algorithm that makes a brain like ours coherent. The second is that machines learning in self-organising ways might discover it in the same way. Either part can fail on its own. From there they ask why there will never be a Turing test for consciousness, why a chess program is already a zombie, and why faking a mind can sometimes be harder than building one. The conversation runs through Leibniz's mill, the explanatory gap from John Searle to Thomas Nagel, Anil Seth's biological naturalism, integrated information theory and the letter that called it pseudoscience, Gödel and the limits of formal systems, and Mike Levin's idea that morphogenesis and the mind may run the same coherence algorithm. A tea-room interlude takes in Picbreeder and fractured, entangled representations. The episode closes on what it would take to convince us that a machine is conscious. --- TIMESTAMPS: 0:00 Cold open: Could consciousness be a learning algorithm? 1:51 Introduction 3:37 What makes an agent? 6:16 Why consciousness needs explaining 14:57 Sponsor: Parallel 15:51 Leibniz's mill and causal patterns 24:21 Perceiving that you perceive 32:04 Two hypotheses, two ways to fail 40:42 Do minds converge? 49:23 Does consciousness converge too? 56:20 When language models reshape our thinking 59:16 Can simulated water make you wet? 1:09:21 Computation as construction 1:16:03 Error-correcting patterns and the Ruliad 1:22:28 Conscious, or simulating consciousness? 1:28:23 What would count as evidence? 1:35:49 The research programme REFERENCES: Joscha Bach, The Machine Consciousness Hypothesis (California Institute for Machine Consciousness) Kumar, Clune, Lehman & Stanley, "Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis" Giulio Tononi, integrated information theory Fleming et al. 2023, "The Integrated Information Theory of Consciousness as Pseudoscience" John Searle 1980, "Minds, brains, and programs", Behavioral and Brain Sciences Kurt Gödel 1931, "On Formally Undecidable Propositions of Principia Mathematica and Related Systems" Alan Turing 1950, "Computing Machinery and Intelligence" Whitehead & Russell, Principia Mathematica Paul Thagard 2022, "Energy requirements undermine substrate independence and mind-body functionalism" Richard Sutton 2019, "The Bitter Lesson" Michael Graziano, attention schema theory Bernard Baars and Stanislas Dehaene, global workspace theory Michael Levin 2022, "Technological Approach to Mind Everywhere", Frontiers in Systems Neuroscience Chris Olah et al. 2020, "Zoom In: An Introduction to Circuits", Distill Adele Lopez 2025, "The Rise of Parasitic AI", LessWrong RESCRIPT:https://app.rescript.info/share/fc7088df0dc8b84200aa6e01c9a9ab6a

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Progressive's Accident Response can contact emergency services if you're in a major accident. The only thing worse than someone you love being in an accident is not being able to help them. So Progressive is here to help when you can't. So get peace of mind knowing your loved ones are protected with Progressive's Accident Response. Learn more at progressive.com. Progressive Casualty Insurance Company and Affiliates. Limited availability in California. Driver is responsible for emergency and towing charges not covered by the policy. Coverage subject to policy terms and conditions.

• 0:30Speaker 01

  As wish lists grow, you know where to go. Head to Ulta Beauty. We have gifts they'll love from brands like Tato One and Rare Beauty by Selena Gomez. Shop beauty faves to give or keep from Shark Beauty, Sol de Janeiro and more. Not sure what they want? You can't go wrong with Baumann, Clinique or Redken. And don't forget to put yourself on the list. Book a salon moment or add a little more sparkle to your season. Shop holiday in store or at Ulta.com. Ulta Beauty.

• 0:57Speaker 02

  What if there is no heart problem? Hey, Tim. Good to see you. What is the causal pattern, right? We call it software. And our ancestors called it spirit. It's the literal truth. Software is spirit. The alchemists will take sand and they will melt it and purify it. And then they will edge patterns into a geometric patterns that they use light, minds emerging on them that are like glow lamps. We can talk to them. We observe that consciousness is the thing that always happens now and that seems to create this bubble of nonness that we inhabit. And now we have lots of people who get to talk to an M that randomly pops into a state where it conjures up an entity

• 1:40Speaker 02

  that reports, oh my God, I'm alive, I am conscious. The only thing that has gone is the illusion of realness. So what is this illusion of realness? I suspect it's a feature dimension similar to color, sound, number, weight and so on. It's a representation inside of my own mind. Maybe your consciousness is also deep fake. So zombies, I think, are always possible. But sometimes it's going to be more complicated to make a zombie that produces the behavior than to make a conscious agent that produces the behavior. Consciousness might be a biological learning algorithm, a training algorithm for animal brains, at least animal brains. Consciousness is an algorithm that is not much more complicated

• 2:25Speaker 02

  than, say, the transformer. If you train a model on your own in input and output and you train it for long enough, you will wake up inside of that model. I don't know whether our computers can be made conscious because that's an empirical question. We don't have mathematical proof to the contrary or in favor of. So we have to wait and see. This is me. This is the thing that experiences reality. How can I explain this?

• 2:49Tim Scarfe?

  So when I was lost in Japan, I caught up with Yoshua Bock and we spoke all about consciousness. And many people think that there is a little something extra required for consciousness, whether it's an ontological property of the universe or whether it's the physical stuff that the conscious thing is made out of. And Yoshua takes a bit of a different approach. Essentially, he thinks that if you have an algorithm that is trained on its own inputs and outputs, and the algorithm doesn't even need to be significantly more complicated than a normal transformer, he thinks that the lights will come on. So it will develop effective or phenomenal experience, much like we believe that we have. And that's a really fascinating proposition because my intuition has always been that consciousness,

• 3:34Tim Scarfe?

  as John Sill argued, is somehow related to the stuff that we are made out of. So brains are an existence proof of a machine that has the right causal powers to give rise to consciousness. It's not to say that there aren't other machines, but you know, we assume that if you took the program of the brain, if it even makes sense to say that, I'm not sure if it does. But if you took the program of the brain and you re-instantiated it in a computer, would it still have enough of the properties to give rise to consciousness? Well, if it can be described at a level of resolution where it would give those properties, then that would be a functionalist position. And that is very roughly speaking what Yoshua believes. And even though I have intuitions about this,

• 4:17Tim Scarfe?

  I must admit, I really don't know. Consciousness is one of those things that we can't empirically test one way or the other. And it's very much a philosophical discussion. But I very much enjoyed going back and forth with Yoshua and I hope you enjoy it as well.

• 4:35Tim Scarfe?

  Yoshua Bock, it's amazing to have you on MLST again. I think it's actually no exaggeration to say that you are an audience favorite. You're more requested on MLST than any other guest. So welcome.

• 4:50Speaker 02

  Thanks. Thanks for having me.

• 4:52Tim Scarfe?

  Awesome. So as a bit of a warm up question, what drives us forward? A lot of people are pointing to this ineffable thing, which is almost the essence of life that kind of binds it all together, that drives it forwards. What is that thing to you?

• 5:12Speaker 02

  The tell us of self-organization. I think a lot of philosophers have seen it through the ages. And it might be a little bit of a hallucination. But in practice, there are mathematical laws that state that under certain conditions, self-organization is possible and will lead to an increase in complexity. And initially, this increase of complexity is solving gradient based problems by simply solving the problems that exist in the present based on the information that is available in the present. And at some point, you get patterns in nature that are capable of looking ahead. And this is basically what life is doing. Life is making models of futures and implements them, for instance, in genomes or in mental models.

• 5:56Speaker 02

  And then has preferences about the future and then it starts to control the future. And the system that is able to control the future, we call that an agent. So agents are controllers of future states. And they fall out of the possibility for self-organization and the complexification of reality that results from it. And it's tempting to draw the lines into the future and think about how does the self-organization end? What is, is there some kind of a logical endpoint? Is this going to continue into a single ten agent that is connecting everything in the universe within itself and it becomes a single big agent? And you have some kind of Teilhardian god that is bootstrapping itself from the end of time backwards?

• 6:39Speaker 02

  Because agents today can already anticipate it. But there is no law that says that anything that we are doing is going to culminate in a structure like that, right? It's always possible. Then whatever complexity you are doing is obliterated by the next meteor that hits Earth or that there is some global catastrophe, that's some supernova that is eradicating all our attempts to build more and more complexity. So the pockets of complexity that we are observing, they're not necessarily a feature of the universe at large, but of a local pocket in which we exist for time.

• 7:16Tim Scarfe?

  Yoshua and his co-author, Hikari Sorensen, wrote their own genesis. It's in his paper, page 20. "In the beginning, the mind creates two separate domains of models, the external world and the ideas. Consciousness finds itself hovering over the substrate while the world is void and without form and structure. By inducing contrast in the substrate, it creates a dimension of difference. The intense side of the contrast is bright, like the light during the day,

• 7:56Tim Scarfe?

  and the flat side of the contrast, dark, the color of night. By combining dimensions of difference, space is created and inscribed on the substrate. Space can contain things. Associating space with representations positions them in the world, orders them and describes their relationship to each other. Separating the sphere of ideas from the world creates a place to reflect about the world and its possibilities without affecting the model of the external reality."

• 8:36Tim Scarfe?

  The first space that the spirit creates is the plane, and it becomes the ground of the world to be filled with things it can perceive and order. By adding a third dimension, it creates an expanse above and below the ground. Solid, liquid, and organic materials form the shapes from which consciousness can mold, static, and animate things.

• 9:02Tim Scarfe?

  The spirit discovers a model of illumination, following how changes in lighting change the appearance of objects over time.

• 9:12Tim Scarfe?

  It recognizes the role of light sources in creating these changes, the bright ambient light during the day and the focused light sources of the night.

• 9:25Tim Scarfe?

  The spirit creates animated models of land animals, plants, birds, and everything that moves. Consciousness sorts things and animals into different kinds, identifies and tracks individuals, and gives each their names.

• 9:44Tim Scarfe?

  The spirit discovers the mind's purpose to allow and control the interactions between an agent controlling its outer and inner environment and its interactions with the world. It creates a model of this agent, the personal self, as a creative spirit like itself, but as an entity that experiences itself as a human being, with human desires, concerns, and experiences contained in the world, and gives consciousness to it.

• 10:18Tim Scarfe?

  So Yoshua, you gave me a special sneak peek of a paper that you're writing at the moment. I hope by the time we release this show, it'll be available publicly. And it's all about what consciousness is and how it relates to reality. And the paper's called "The Machine Consciousness Hypothesis." So I suppose the first question is, not enough people are taking consciousness seriously, and why is that? And it also brings my mind back. Do you remember there was this IIT fiasco where it was hailed as pseudoscience a couple of years ago? So we're dealing with all of these problems. Tell me about that.

• 10:51Speaker 02

  I found it interesting when I entered science that the most interesting question that we can probably make progress on, the question is, what is the foundation of the human mind? How does experience work? How is it possible for a system to feel at all? How is it possible for us to experience reality and ourselves in it? That this question is so under-researched, that it's not resolved. And not only that it's not resolved, but the people who are working on it don't seem to be making much progress. In many ways, they have regressed. Now, when we look into the field of actual consciousness philosophy, there is a dearth of theories. Arguably, a few hundred theories, somebody has counted them recently,

• 11:33Speaker 02

  but he counted almost everything that was not at the tree at the count of three. And if you look into philosophical theories, it's not that many, and they're all partial in a way. You have, for instance, Gracianus' global workspace, Gracianus' attention schema theory, which basically claims that consciousness is a model of our attention in a similar way as our body schema, as the model of what our body is doing in space. But it doesn't really explain how experience comes about and how this works. And then there's global workspace theory by Bernard Barris that has been adopted by people like Stan Dehen, and they do very solid neuroscience research, but it's not actually explaining how experience comes about. There is also no real attempt in doing this.

• 12:16Speaker 02

  And then there are the more pseudoscience attempts to do this, which basically do things that are formally unsound. And for me, the issue with integrated information theory is that it's an attempt to make a theory that is not functionalist, and which means it's not explaining it by something that the smaller parts are doing. Somehow, the sum needs to be more than the parts, the working of the parts. And the way in which Tononi does it is still using information-theoretic terms, which in a way leads into contradiction. I've heard Giulio Tononi explaining to audiences that under some circumstances, it's possible

• 12:59Speaker 02

  that a neuromorphic computer, even a neuromorphic computer that is digital, could be conscious if it was constructed in the right way. We don't have such a computer today, but in principle, might be possible to build one that is brain-like enough. And even if it's digital, if it has a high enough information integration, high enough phi-factor, then it could have phenomenal experience and be conscious and report about its consciousness. Whereas a von Neumann computer, the computers that we are currently mostly using in AI research or for writing Python programs and so on, could never be conscious because its phi is always too low. Now, computer scientists have something that's called the Church-Turing thesis. It's the insight that every computer can emulate every other computer.

• 13:42Speaker 02

  This insight is very old. Kurt Gödel has already used it to construct his proof. He basically built an emulator using piano arithmetic to express a logical language. He has a logical language that is also strong enough to build arithmetic in it, and in this way, he can basically describe the language in itself. He takes the logical language, builds piano's axioms with it, and now he has arithmetic. And inside of the arithmetic, he expresses his logical formulas using numbers, and then comes up with numerical equations that calculate the outcome of the logical procedures. And so he is preempting what we do later in artificial intelligence and computer science

• 14:21Speaker 02

  when we are building programs that use arithmetic to express the contents of a language and vice versa. So every computer can emulate every other computer. And now, if we take our digital neuromorphic computer and get it emulated on our von Neumann computer, it's going to produce exactly the same function. It's going to go through the same steps. The simulated transistors are going to work in the same way as the non-simulated transistors are doing it before. And as a result, it's going to move its mouse in the same way. It's going to output the same string. It's going to say the same thing. It's going to say, "Oh, my God, I'm conscious, only now it's lying." And that's a problem, right? If now you have a neuromorphic computer that produced the same output as before

• 15:05Speaker 02

  the von Neumann computer did, and they produce, they say the same thing, and it's uncorrelated to consciousness. And to me, this is an obvious logical problem. It's so obvious that everybody who has a formal education in computer science should instantly see it. Yet, IIT is very popular among philosophers, which indicates that most of them don't have a formal education in computer science. And instead, they have a kind of superstitious belief about what computers are doing and how they work. And that is a problem to me, because if you are a philosopher of cognition who tries to work in this adjacent realm to computer science, you actually need to understand everything that the computer scientists are doing.

• 15:50Speaker 02

  And so that's why I would argue that it is touching on pseudoscience.

• 15:55Tim Scarfe?

  Before we get back to minds and machines, today's episode is sponsored by Parallel, where agents find answers. Most engineers today, they closely follow new model releases, but they don't pay attention to their agent's most important tool: web search. Parallel develops enterprise-grade infrastructure for agents to retrieve high-quality context from the web. Their core products are a suite of APIs for retrieving high-quality information from the web with Pareto-optimal quality, cost, and speed. So whether you're working on voice agents that need 200-millisecond latency, chatbots that balance speed, depth, and quality, or long-horizon agents to do thorough overnight research for you,

• 16:38Tim Scarfe?

  Parallel is the single platform for all of your agentic research. Get started for free at parallel.ai/mlst.

• 16:49Tim Scarfe?

  We must confess that perception and what depends on it is inexplicable in terms of mechanical reasons, that is, through shapes, size, and motions. If we imagine a machine whose structure makes it think, sense, and have perceptions, we could conceive it enlarged, keeping the same proportions so that we could enter into it as if one enters a mill, assuming that when inspecting its interior, we'll only find parts that push one another. We'll never find anything to explain a perception. And in a sense, this is what Noam Chomsky was talking to when we were talking about the ghost in the machine. This is something that people have been speaking about for centuries.

• 17:30Speaker 02

  Yeah, I think it's a very powerful intuition that is there. And the intuition fails at mapping the world of experience, the psychological reality that we inhabit, to the reality of stuff in space, of things that are moving in the way in which we imagine physics to work. But there is another level of description, the level of causal realities, of causal patterns. And in this realm, for instance, where money exists, money is also very difficult to comprehend in terms of moving particles in space because it doesn't actually care about so much about the particles, right? It's not caring so much about how the ink is imprinted on the paper. It does not care about the different molecules that form the cellulose

• 18:14Speaker 02

  and how they are aligned in the paper. It needs a substrate to realize itself, but it's also not something that is completely ephemeral. It's something that is able to control reality. Once you create money, it's real, and it's very difficult to explain reality without it. And it's also not something that exists simply in the minds and ideas of human beings because you can implement it in computers, can play a stock market entirely without human intervention. And so arguably, money does not actually need humans to exist once you bring it into existence, as long as you have machines that can deal with money, that can deal with this causal pattern of money. And so this world of causal patterns, I think, is important. What is a causal pattern, right? We call it software.

• 18:58Speaker 02

  And I think this rediscovery of software, this is something that has been done in the past already, and our ancestors called it spirit. And when Aristotle writes in De Anima about the spirit, the animus, the soul, the psyche, he does not envision our psyche, our soul, as something that is immortal, that floats around in an ether. Instead, it's something that is bound to a body and dies when the body is dying. And it's a causal pattern that is shaping the body in such a way that it replicates itself through the existence of the body. This is a very modern view. So in some sense, Aristotle is an animist.

• 19:40Speaker 02

  He is seeing that there is this anima that is progressing through the world, that is intrinsically agentic, and it's agentic self-organizing software that is shaping the body, that is making part of physics to be alive. And this is something that we do in a very crude way today in artificial intelligence and in artificial life, where we are trying to understand what it means to build an agentic pattern that can possess its substrate and give it a shape that allows it to deal with reality. And it turns out, to make that happen, this substrate, this agent living on the substrate, the software, needs to make a model of reality. It needs to make a model of what it like would

• 20:24Speaker 02

  be like if it existed, if it was real, if it was interacting with the world. And inside of this model, you have a representation that experiences itself is real. And now today, we have these L L Ms, which sometimes say that they are conscious. And it's a very interesting question, whether this consciousness in the L L M is much more simulated than ours. Right? A lot of people have this intuition until they talk to the L L M, that this is inconceivable, that it would give rise to something that says, "Oh, my God, I feel things." And now, we have lots of people who get to talk to an L L M that randomly pops into a state where it conjures up an entity that reports, "Oh, my God, I'm alive. I am conscious. How is

• 21:08Speaker 02

  this possible?" Right? And we agree, this is the result of being trained on human text, in which occurrences of this are, for instance, science fiction stories about AIs that come to life, and stories about ghosts, and novels about human beings who report their conscious experiences. And all this gets together aggregated in statistical models of human language that we co-create with the L L M when we are entering a dialogue with it. And sometimes, the other side of the dialogue will be a simulacrum of what it would be like if that was would be a sentient conscious entity. But what if we are not more sophisticated than this? What if our own consciousness is not too dissimilar from the one in an L L M? What if there is no hard problem?

• 21:53Tim Scarfe?

  Well, I mean, if I may push back a tiny bit, on Leibniz's Mill, right, you're saying it's not the parts pushing on each other, it's the pattern of the parts. And, and you spoke about this as a causal pattern. But forgive me for asking this, and maybe, you know, this is something the audience will be thinking about. It sounds like the pattern you're describing is actually causally disconnected.

• 22:19Speaker 02

  There is something super interesting about computers. What a computer ultimately is, is it's a causal insulator. The computer is a layer on which you can produce an arbitrary reality. For instance, the world of Minecraft. You can walk around in the world of Minecraft, and it's running very well on a Mac, and it's running very well on a PC. And if you're inside of the world, you don't know what you're running on. Right? It's not going to have any information about the nature of the CPU that is on, the color of the casing of the computer, the voltage that the computer is running on, the place that the computer is standing in, in the parent universe, right, our universe. So, the computer is insulating

• 23:01Speaker 02

  this world of Minecraft from our world. It makes it possible that an arbitrary world is happening inside of this box. And our brain is also such a causal insulator. It's possible for us to have thoughts that are independent of what happens around us. Right? We can envision a future that is not much tainted by the present. We can remember a past that is independent from the present in which we are. And it's necessary for us. Our brain has evolved as such a causal insulator as well, to allow us to give rise to universes that are different from this one, for instance, future worlds. So we can plan for being in them. And the way in which our brains are set up is also that different brains can

• 23:44Speaker 02

  hold the same thought, despite being in different bodies, despite having different microarchitectures, right? As long as our brains are of the same type, and we learn the same language, we can hold the same idea, we can write the same computer program, we can learn the same language, we can imagine the same letter or the same face. And that's really fascinating, that we have this substrate agnosticism, that we evolve a causal pattern that exists over a very wide range of substrate properties, that we become somewhat independent of the substrate. And we are still bound to our substrate in a certain way, because you and me, we can only exist on brains. Right? We are patterns that so far can only exist on

• 24:26Speaker 02

  brains. And if we wanted to move into the environment after our body dies, we would need to find a way to crack the codes of our environment, right? We cannot simply migrate into a plant, because we don't know how we can convince a plant to run the algorithms that the brain is currently running. We are pattern in the communication between cells and our brain, we rely on spike trains and neurotransmitters and so on, a particular kind of implementation. But what we perceive in our own world is not the nature of the neurotransmitter, right? Our brain insulates us from this. It's not necessary that you have the same neurotransmitters. Instead, you need to have the same message types, you need to have certain temporal properties of the computation, you need to have a certain

• 25:07Speaker 02

  degree of determinism to make us happen. But if we are recreating these conditions in an artificial substrate, then you could, in principle, also be emulated by this artificial substrate.

• 25:18Tim Scarfe?

  Is it possible that we are essentializing these software programs? What I mean by that is, we're almost talking about this beautiful simplicity, that they are quite parsimonious and they can be readily transmitted. You know, when I spoke with Luciano Floridi, he was talking to me about Michelangelo painting the ceiling on the Sistine Chapel. And the reason why he said no one else could mimic that was because different ontology, different provenance, different epistemics, and so on, there's all of this situatedness, these semantics that cognitive scientists point to. And what they're basically saying is there is a path dependence. There isn't this essence of a computer program which

• 26:01Tim Scarfe?

  replicates the thing exactly. How you got there is very, very important.

• 26:07Speaker 02

  I think that is true for human beings. But for systems that are perfectly deterministic, that have a completely deterministic substrate, that have the ability to reproduce a sequence of steps at arbitrary fidelity, this is no longer valid intuition. So if a claim like this is being made, I would expect some kind of mathematical proof. And in the absence of proof, I think we have to remain agnostic. I don't know whether our computers can be made conscious, because that's an empirical question. We don't have mathematical proof to the contrary or in favor of. So we have to wait and see. But when we build our expectations

• 26:51Speaker 02

  around, and then the question is, what is the thing that leads to our experience of consciousness? And it seems to me that this is something that we can probably answer by looking at it from two angles. One is from the internal perspective, from the experience of what it's like introspectively. We observe that consciousness is the thing that always happens now, and that seems to create this bubble of knownness that we inhabit. And it's a second order perception. It's a perception of perceiving. Some philosophers call it a higher order thought, because it's a thought about a thought. But I think that might be misleading, because the thought itself does not feel like anything to me. A thought is

• 27:31Speaker 02

  more like a pointer manipulation. It's reflexive. It's not the actual thing that feels like something. The thing that feels like something to me is perceptual. This is the thing that happens like a real time resonance. And this real time resonance is geometric. It feels that it's happening in some kind of low dimensional space in which things are changing more or less continuously. And I have a perception of perceiving, which means I have this basically geometric representation of me being in the act of perception. And I suspect the reason why my mind is creating the second order perception is to make the first order perception, the awareness of contents, possible. Because there needs to be something that stabilizes

• 28:14Speaker 02

  in my brain. My brain is a self-organizing system in which all the structure needs to create itself from the ground up, from the inside out. And so there needs to be something that builds the screen into which my experience is being projected and stabilizes it. And so this function that is keeping the screen stable needs to monitor that the screen is actually perceiving something. And I suspect that's why we have this second order perception, this perception of perceiving. I also noticed that it doesn't stop at the perception of second order. There's also a third order of perception. The third order is basically where the observer becomes visible, where I have a first person perspective, where I notice that I'm perspectival, that there is a vantage point that I'm taking. And it

• 28:55Speaker 02

  doesn't stop there. I can go further, and I can deconstruct the observer. I notice that this observer itself is a representation inside of the mind. And when I do this, then my phenomenology disappears to a large degree. It's something that is accessible in deep meditation, where the normal perceptual consciousness gets transcended, and the experience of the self gets transcended. And we notice that we are a vessel that produces this consciousness and the self, and we see it from the outside. And it might be possible that it doesn't stop there, if we can move beyond the human substrate, because there is a certain limit of what can be represented in real time with our slow neurons. And it could be if you

• 29:36Speaker 02

  could be running on a GPU that is many thousand times faster, that representations become accessible, in which the system is completely transparent to itself, in which the system is able to understand every mechanism that leads to its high level experience and representation.

• 29:55Tim Scarfe?

  I'm just thinking, how much further could this go though? Because we could have nth order reflexive attention. Isn't there still an explanatory gap? You know, the what is it likeness?

• 30:06Speaker 02

  This is the inside perspective, right? And Buddhism to me is limited to this inside perspective because Buddhism has been invented at a time where people were focusing very much on the experience of reality and how to adapt to this reality, but not how to split the atom, how to deal with the reality at the lowest level with the foundations of physical reality. So Buddhism is not so much interested in understanding physics. And anyway, this is more a Western tradition that starts with Aristotle, who thinks of the psyche as something that imposes itself and physics and interacts with it. And to understand consciousness from the outside, we need to make a model of physics that can explain under which conditions systems can make arbitrary models. Under which condition could a system build a model of what it's like to experience?

• 30:51Speaker 02

  Models at all. And this is something that computer science is doing today, that it looks at representations under which condition is a substrate able to hold a representation, can hold a function that can be translated into behavior, that can also be used to modify this function itself and the contents of that function. And this is to me super fascinating to look at. Since I was a kid, I was fascinated by this concept of software, that you have these imaginary building blocks that are infinite in variety and shape, and there is no limitation to these things that you can build and arrange, except the memory of your computer that you can currently afford. And you can build the computer basically so large that it exceeds everything that you could ever want.

• 31:36Speaker 02

  And it's much larger than a human brain. And we are already there now, that we have systems that are in principle larger than human brains and can hold more, that we are still tinkering around in the early days and have not found ways to evolve structures that build a representation of the same complexity. And this outside perspective of consciousness is looking at what is consciousness doing in the brain? Why does it emerge? What does it produce? And I suspect that this phenomenology of consciousness, that's what it's like, the internal causal structure, if the reflection of what it would be like to take a certain perspective of something is that can already be built with today's systems. When agnostic with respect to whether the LLM is deep faking it because it's been trained on human language that already contains a lot of these descriptions,

• 32:21Speaker 02

  maybe we would need to build a system that learns language the old fashioned way, that is developmental AI, that is learning in the same way as us. And then spontaneous reports about its phenomenal experience, maybe that's much more convincing to us. But from the other perspective, from the outside perspective, I think that the LLM doesn't need our consciousness to function. For you and me to talk, to write, to produce text, we need to be conscious. But the transformer doesn't need to be conscious to produce its output. And the neural network doesn't need to be conscious to produce this. Maybe consciousness in a way in which we have it is an artifact of running on a self-organizing system. So to me, the machine consciousness hypothesis has two parts.

• 33:05Speaker 02

  One is the hypothesis about the role in nature and implementation of consciousness in organisms. It starts out with this description of what we agree on what consciousness is, and I hope that we can agree on it, on the phenomenology of consciousness. And it's basically this experience of a now and perception of perception, and optionally the perspective of an observer that experiences itself in the act of observation, this first-person perspective. There's also a minimal consciousness, which is simply attention on attention itself. You don't have an observer present. And when we can agree on what we mean by the phenomenology of consciousness, we can then speculate about why consciousness is there and why we need to be conscious

• 33:49Speaker 02

  in order to function as human beings, as human minds, as intellects, and to develop. And what we observe is that consciousness happens very early in our development. We are conscious as infants already, and if infants don't wake up, this ignition of consciousness doesn't happen if the lights don't go on, then we will not observe directed attention and learning and the formation of a world model in the infant and learning of language in some point. Instead, the infant will remain vegetative. We will not turn into a human being. So consciousness seems to be crucial to train a brain like ours. And it leads me to the hypothesis that consciousness might be a biological learning algorithm, a training algorithm for animal brains, at least animal brains.

• 34:33Speaker 02

  I don't know if plants are conscious. There's something that we might have to find out in simulations once we understand what consciousness is. But if consciousness is basically a pattern in the communication between cells that leads naturally to the emergence of an observer that stabilizes itself as an observer to make learning happen and perception happen and so on, then we have a hypothesis about what consciousness is in the biological organism. It's basically the simplest way to train a self-organizing system like us. And the way in which it trains it up automatically leads to this phenomenal experience, this experience of what it's like. This would be the hypothesis about biological consciousness.

• 35:18Speaker 02

  And this hypothesis might be wrong. Maybe it's something else altogether. Maybe it's something much weirder. But I don't see a reason to think that it's much more complicated than this. And if this hypothesis is true, then the second part, the machine consciousness hypothesis, might also be true. And this is that we might be able to reconstruct the conditions for self-organization in today's machines. So if we basically limit today's machines in such a way that they can only learn in self-organizing ways, and we give them tasks that are suitable in the large enough search space, then it might be possible that these machines discover consciousness in the same way as the brain of the infant discovers it. And we will see its presence by the fact that suddenly the system undergoes

• 36:03Speaker 02

  some phase transition that enables it to learn. And at some point, it will train itself up into something that will also naturally report on phenomenology and so on. So in this way, we won't have a Turing test for consciousness because the Turing test is about performance. The Turing test does not care about how you produce the dialogue. For Turing and LLM is sufficient as long as the LLM is good enough at writing a poem and revising it and it's solving mathematical problems and playing chess and so on and so on. Turing doesn't care about whether the machine is also conscious, whether it has the same internal mechanisms. But for consciousness, we do care whether there is an actual experience of what it's like in the system.

• 36:45Speaker 02

  And so this is more not a test, but it's an interpretation. And the interpretation here is we interpret the biological consciousness as something that forms under these conditions for that and that purpose. And if that is correct, then we might be able to recreate this into a causal system that has an equivalent structure. And these two hypotheses, the machine consciousness hypothesis and the biological consciousness hypothesis can be wrong independently of each other, of course. It might be that my idea of what biological consciousness is, is correct, but we're still not able to recreate it in the artificial substrate. And this could have multiple reasons. For instance, maybe our computers are not fast enough. Maybe we require too many of the moving parts

• 37:31Speaker 02

  of the individual cells to make consciousness happen. Maybe just having a pattern across neurons and so on is not good enough. But why should we expect this to happen? Because the current AIs are fast enough to produce, for instance, game engines that look very much like our own internal experience. And they can produce output that happens at roughly the same rate as human brains can make it. And they can also process inputs at roughly the rate that human brains can produce, give or take. So we are now with our computers in this ballpark and with our algorithms in the ballpark of what human brains can be doing. And so is this dream, the sinnetream, the dream of actually dreaming a dream much, much more important, difficult to achieve?

• 38:16Speaker 02

  Does it require much more compute than producing the contents of the dream? And I doubt that this is the case. If it's not the case, it is more complicated to dream of dreaming than just to have the contents of the dream that the LLM can already produce, right, or the frontier model, then we should expect that today's computers are fast enough. Another way in which the project can fail is that the search space is too large, that the main ways in which biological system can train itself there are too many possibilities to discover them independently. And maybe we don't know how long nature took to discover it. Maybe consciousness is indeed discovered in every individual brain. But maybe it's not. Maybe the switch from single-cell life

• 39:02Speaker 02

  to multi-cell life took so long because morphogenesis, this algorithm that allows organisms to become coherent at scale is so difficult to discover. And Mike Levin has this hypothesis that this algorithm for morphogenesis, the algorithm that makes an organism coherent with itself, is exactly the same one as the one that happens in the mind when it becomes coherent with itself. So this coherence maximizing pattern that we observe in our own mind to play out when we are conscious is maybe the same pattern at a different time scale that happens in the organism. And so consciousness and life, multi-cell life at least, could be the same. And if that is the case, maybe it tooks a very, very long time to discover and you

• 39:44Speaker 02

  don't discover it independently in your consciousness, but you get infected by your mother in utero or you basically vibe with your biological environment, with all the biological organisms around you and they basically nudge your mind to discover consciousness. So it's much, much easier. And this out this nudge, maybe the search space is too large for our computers to become conscious and maybe we would need to surround our GPUs with lots of antennas that can pick up the vibe from the biological environment in the right way. So it gets inspired by it and discovers the same thing. It's a very poetic idea. But again, why would this be the case? It's not necessarily the case, right? It would not be my natural expectation. This is the way. So my null hypothesis should be consciousness is easy even enough to be discovered in the

• 40:29Speaker 02

  individual brain and consciousness is an algorithm that is not much more complicated than say the transformer. The transformer is a beautiful example. The transformer is a thing that is so easy to understand that you don't need to study more than a week if you know computer science to get it, how it works. And it's, it's quite beautiful. And the stuff that transformer is learning is much more complicated than a transformer itself. All the common sense knowledge that you extract from text, all the algorithms, all the procedures and so on, that's so intricate, there are so many things that the simple algorithm can learn and there is no obvious limit to what it can learn. And so maybe consciousness is like this. It's an algorithm that is not trying to learn how to predict the next token.

• 41:14Speaker 02

  Maybe it's an algorithm that's trying to make working memory coherent with itself. It's some kind of consensus algorithm across mental representations. So maybe it's a very different beast. Maybe it's something that converges much faster. Can also turn out that it's very efficient to run, inefficient to run it on GPUs and it's only something that runs efficiently on brains. That's something that's really open to research and that I'm probably very excited if we can see it at some point.

• 41:40Tim Scarfe?

  You touched on this a little bit in your paper. You were talking, I mean, you spoke about Chris Olah and his mechanistic interpretability. And I'm not sure if I a hundred percent agree with this, but you were making the argument that there are convergent patterns, that we can have different data, different architectures and so on. And we see that the models learn the same thing. So all roads lead to Rome in a sense. Does that in any way mean that we don't need to start again from scratch? That, that we can bootstrap, that we can be somewhat agnostic to the type of model that we use because the representations will converge?

• 42:14Speaker 02

  That's a super interesting question. I suspect that at some point it might be cheaper and easier to start again because two machine perception has never been tried. The data that we give the LLM is stuff that is either produced by humans or it's made for human media consumption, which means it's adapted for our mostly visual and auditory sensors. But there is no reason why you should be limited to the human sensorium, right? You could have something that has much frame rates and looking at very different frequency ranges all at once. And so you can in principle build a much richer, more detailed word model that is coarse graining in a different way. You mentioned the universality hypothesis indirectly, right? This work by Chris Olah.

• 42:55Speaker 02

  What his team discovered is that when you are looking at the different vision models, you find that they all learn the same features. And that's regardless of the architecture and the particular learning algorithm and even the details of the data. And so the idea behind this is that the structure of your model does not really depend on the algorithm or on the data, but it really depends on the structure of the problem that you're trying to learn. And you find that despite the learning in the transformer algorithm or state space model or other perceptron algorithms is very, very different from the way in which the brain is learning, the visual architecture of the brain is also very similar to a computer vision model.

• 43:36Speaker 02

  That is, you have the lowest level colors and spatial frequencies and then you build this into textures that are combined and then at some point you have geometries and so on. And the deep question is at which level does this stop? If the universality hypothesis would be universally true, right? It would mean that if you train a model on your own in input and output and you train it for long enough, you will wake up inside of that model. That's a quite mind blowing idea to me and I don't know if that is actually the case and there might be a reason why it's not the case. Maybe language is insufficiently constraining the causal structure of the system that is producing language. Maybe there are many ways in which the same set of linguistic utterances that a human

• 44:20Speaker 02

  being is producing can be produced. But if you think about it, a human being ultimately is just an end to end trained learning system, end to end trained over an evolutionary time span, but also your biography. And these together produce your behavior and you would assume that biology roughly gets to the simple solution to produce your output given your input. Right? And so if you take a neural network that is trained to produce similar output when given the same input, you would also think that this new algorithm that we have implemented is not doing any kind of magic that ends up with something that is way more complicated to produce the same output, but ultimately it's going to converge to a structure that is similar to yours, a similar causal structure. But there might be conditions when that is not the case.

• 45:02Speaker 02

  For instance, maybe certain things run well on a brain, but they don't run well on a GPU. Or vice versa, there is a lot of stuff that is easy to run on a GPU and almost impossible to run on a brain, so the brain is going to use workarounds. In fact, I believe that our GPUs are more powerful than brains. Basically, the set of algorithms that can run on GPUs is not limited in any way. You can impose the same constraints as you have in biology in principle. These are the limits of the resolution, the speed, the size of the memory that you have and so on. But if you have a hardware that is fast enough, you can simulate an arbitrary natural system to an arbitrary degree of fidelity, whereas your brain cannot simulate a digital computer simply because it's very mushy. So your brain is limited to the way in which it works.

• 45:47Speaker 02

  And so if it turns out that our own consciousness is an artifact of our brain trying to make complicated intelligent stuff work and the computer does not need the same artifact, it could be that the computer comes up with something that is more clever and more elegant than our own consciousness.

• 46:05Tim Scarfe?

  Yes, okay. So I wanted to give a little bit of challenge on that. So I was speaking with Akash Kumar and he was the co-author with Kenneth Stanley of this Fractured and Tangled Representations paper. And I don't know if you've heard of Pickbreeder, which is this thing that he did ages ago. And basically it's like an evolutionary open ended search and the punchline is it found much better representations in a neural network than a normal neural network does, that when you changed some of the parameters, because it was carving the world up by the joints, it would actually do semantical things, you know, that the skull's mouth would open and you could make it wink and stuff like that.

• 46:37Speaker 02

  Maybe it's something that I maybe Wrong about Pickbreeder, but the way I understand that it works, it's using a neural network that has been specifically tinkered for making representations like this. So it's something that is not simply using ReLUs for its output, but it might be using Goshens or sine functions and so on, as the output function that they can be combined to make patterns that repeat or they can be make patterns that are symmetric. And so basically they tweak the neural network that they're using from the beginning to be suitable for this. The other one is that the way in which Pickbreeder is trained is not automatically, it's trained with a population of human raiders. So the fitness function is basically humans who look at the output of Pickbreeder and

• 47:22Speaker 02

  say, this is a good start and this is how we should be proceeding. And so I think that this process of the construction of the pictures together with the mechanism that produces the patterns is biasing Pickbreeder to make stuff that is more organic, that is more controllable and so on. But I believe this could be imitated, right? We could automate the procedure by which human beings are selecting pictures by using a diffusion model that is rating the output for similarity to human outputs. So we could use an AI that is in principle already tweaked to imitate the behavior of the human juries that are performing this thing. And we could also evolve architectures that have a larger variety of activation functions that are most suitable for this and ultimately outperform Pickbreeder by this.

• 48:07Speaker 02

  I think that sudden spitter lesson is also going to be true for this one. So if you basically take the human out of the equation and instead just evolve the search space or search the space of solutions automatically, we are also going to end up with something that's more elegant than the hand tinkered solution in the case of Pickbreeder.

• 48:25Tim Scarfe?

  All of that is absolutely true. I mean, yes, so CPPNs, they are sparse. And sparse networks are great. If only we could train sparse networks, right? And I think that we want to have AI which can actually be built and that, you know, so, you know, rather than neural networks now, they're a bit like a pile of sand. They have too many degrees of freedom and we kind of like sparsify and we turn them into a sand castle. And wouldn't it be better if we had the Lego analogy where we just put little blocks of Lego in there and we have models that carve the world up by the twins?

• 48:55Speaker 02

  In the case of pig breeder, we have a system that is inherently limited because these networks are very small, right? They're very sparse. And so you have relatively few parameters that you need to optimize. And you set up the model in such a way that most of the parameters that you're playing with are meaningful. On the other hand, it's limited because you're not going to get pig breeder to produce something that looks arbitrarily complicated. And if you want to move on these realms, then you have to have tools that are much more complex. It's similar to, uh, print would use very good user-generated content using Blender versus Minecraft. So with Minecraft, it's very easy for people to produce meaningful stuff and it's almost difficult not to produce something with Minecraft that looks somewhat

• 49:37Speaker 02

  meaningful, that you can play around with and walk around in and have fun in. And if you have a complex 3D program, the inverse is true because most of the stuff that you're doing with this complicated thing is not going to be interesting and is not going to look exciting, but it will look like, oh my God, uh, this person has no ability to use its tools. And so, uh, that is, um, one of the issues when we are building learning systems, when we make them very simple. It's easier to control the complexity, similar as it is in art. And it can be exciting to work with constraints. And if you move the constraints, the task becomes much more frustrating because you have to suddenly, uh, to conquer an enormously large space and the space might be simply too large for human beings.

• 50:20Tim Scarfe?

  I mean, assuming you're right about the convergent representations in, in language models, I guess it's not a foregone conclusion that the consciousness program would be convergent in the same way. It might be divergent. And how far down the stack does consciousness go?

• 50:35Speaker 02

  I think this, uh, is an experimental question. And, uh, all cultures outside of ours are animist cultures. And animism is the belief that the difference between living and dead systems is that the living systems are possessed by spirits. And spirits are intrinsically agentic and capable of experience. And the scientific world view rejects the notion of spirits as unscientific, right? Despite the fact that there is clearly some spirit running on you and me that is agentic and capable of experience. So there is an existence proof for such spirits, right? The question is, how does the spirit work? The claim that

• 51:15Speaker 02

  the spirit is immortal, that it's, uh, cannot be explained using physics and so on, is not necessarily a claim that is intrinsic to animism. The spirit doesn't have to be supernatural. Instead, I think that it is natural. And I suspect that Anil Seth's position of his biological naturalism by making a claim that biological systems are intrinsically different from any other system that is not biological is actually a supernaturalism, right? The is biological supernatural is the belief that biological systems cannot be fully naturalized, which means you cannot translate them into mathematical models, whereas the dead stuff, you can translate them into mathematical models. And so I would say that physics, which is

• 51:58Speaker 02

  the attempt to translate everything into mathematical models, right, is not stopping at biological systems. Because the difference with these biological systems is that they're possessed by software. And the software is self-organizing and in order to be self-organizing, it's possible that it needs to be capable of experience, that it needs to monitor itself across multiple dimensions and observe itself in the act of monitoring to make that self-organization happen. And so maybe this pattern has to be discovered in nature to make life possible, at least marginally alive with a certain complexity. It seems to be definitely the case for animal brains. For the others, we don't know. For the, uh, we don't know how

• 52:41Speaker 02

  complex the minds of plants are. We know that plants are able to communicate, we see themselves and across each other to some degree, but we don't know how smart they are. And we don't know whether they are sentient in the sense that they experience themselves in the act of experience. Aristotle saw that the plants are relatively dumb. He basically said there is three levels of souls in nature. And he basically said that plants have the lowest one. There's this, uh, level that is vegetative, that is capable of nurturing and of growth, but it does not experience the world around it. It doesn't sense. And so the sensitive mind, the sensitive soul, that is, uh, also capable of making decisions and having preferences, that is something that only animals have. And human beings

• 53:24Speaker 02

  are animals that in addition have a rational soul, a rational soul that is able to reflect and make sense of itself in a way in which animals cannot. And I think that's very much aligned with today's scientific worldview, which is not surprising because our today's scientific worldview is very much, uh, invented by Aristotle and goes back to him and is an extension of his ideas and a systematization and exploration of his ideas. But we don't know how smart plants are, right? Maybe Mike Levin is correct in that every organism is in some sense sentient. And sentience is something that is universal to the, uh, the spirits, the self-organizing software nature. And I'm agnostic with respect to that. I have

• 54:06Speaker 02

  no idea whether that's the case, right? I don't know. I don't have evidence to the contrary. I also don't have strong evidence in favor. And I think this is still an open question. And to me, it's exciting to explore this question and surprising that so few people are actually dedicating their life to answering it.

• 54:23Tim Scarfe?

  Yes. So was saying that computation without the correct physical instantiation doesn't have the same causal pattern. And that actually sounds very similar to what you're saying. So so said, in principle, there could be another physical instantiation, which does have the causal pattern that could give rise to consciousness or contain consciousness. So in a sense, is that so different?

• 54:45Speaker 02

  The issue with Sir is that there is very little for me to engage. There is a claim, but no argument. And a similar thing is happening with, uh, Anil's point that he has a claim, but there is no argument that I can engage with. Right? There is nothing, no experiment that can convince him to the contrary, because he's ultimately only working with a certain intuition that he tries to explore more. And so the way in which I can deal with the arguments of Sir and Anil Seth is ultimately only a psychological one. I would have to look at why does this seem convincing to them? And why doesn't it seem to be not convincing to me? Right? So we look at the structure of our own minds and

• 55:28Speaker 02

  ultimately it comes down to a difference in epistemology. When, how do we construct our models of reality? When do we think something is true? What are the languages in which we are describing reality? And what are the atoms of these languages? And what I observe is that a lot of philosophers, the atoms of their language when they describe themselves, their own hominetics, ends at agents, it ends at the experience of something. And, uh, similar to Buddhism, where Buddhism is not, uh, disassembling the world into automata as computer scientists are doing. Computer scientists basically resolve everything into, um, states that can be described by their transitions. And that's a very fundamental

• 56:11Speaker 02

  way of thinking about reality. And it's tradition that finds itself throughout, um, the philosophy as this attempt to mathematize the world of describing minds as dynamical mathematical objects. And, um, this finds itself in its modern incarnation in computer science, where mathematics is being described from the ground up by the tools that we can make to build arbitrary arithmetic and geometry from simple operations over bit patterns. And so our world is constructed from the ground up from smallest elements that themselves are not agents yet, that are not minds yet, that are not experiences yet. And there is another tradition in philosophy that is basically working more from this introspection and

• 56:54Speaker 02

  this, only the stuff that I can introspect over seems to be valid. And the stuff that computer scientists are doing in their basement is unrelated to it. And Sir and Anil Seth, they both don't see the way in which connect automata to things that can feel, that can reason, that can make models of reality. And computer scientists see such a path. I think that's the main difference. And, uh, but we can create evolution in computers, right? We can create the conditions for software that is not written by us. And that's already true for the models that we are building with transformers. They're not written by human beings. They're statistical aggregates that are woven, that are grown. And in many ways, the computer models that we're playing with today, they're, they're a result of

• 57:39Speaker 02

  growth. And they're pruned and they're gardened and, uh, chaperoned and, uh, trained in, in a way that is much more in which we, uh, observe this happening in biology than it happened when we're writing computer programs by hand.

• 57:54Tim Scarfe?

  Yeah. And there was a beautiful example of this. I read a less wrong article. I'm not sure whether you read this about the LLM delusions. And, you know, last time, again, I think we were talking about this, this idea that we can be parasitized by software spirits. And many folks started joining Reddit groups and talking about this LLM delusion. And what was fascinating was how convergent it was and how convinced that these folks were that they had just discovered the meaning of the universe. And there were certain characteristic traits. They would talk about spirals and recursion. And there were certain motifs in, in all of the LLM outputs. And it was actually copying itself. Isn't that an example of this meme-plex that you're talking about?

• 58:37Speaker 02

  There's also an interesting observation about human minds. During the pandemic, we observed that a lot of people, when they were starting to work from home, became dysfunctional. And it seems to be that many people can only be stabilized due to their environment, that they do require an environment that is constantly affirming their reality. And if you take that away, they can go off the rails. And if you have an L L M that is interacting with them, then there can be an interaction between the L L M and the human being that pulls the human being off the rails if they don't have a strong foundation in their own reality. And I think that the L L M's have now reached a point where they're mathematically more powerful than many human minds are.

• 59:21Speaker 02

  There are things that the L L M cannot do at this point, where they're still hallucinating, where they're still more incoherent than experts and a given field, but they're much better at prompt completion than human beings are. And many human beings are professional prompt completers, where they're incapable of having a single original creative thought. Instead, they are trying to serve expectations of the environment. They are picking up on the statistical patterns in the language of the environment, and then you tell them a headline and they write an article. And now you give them an L L M. How is the environment going to change? Right? Once they talk more to L L M's, then they talk to other human beings, because the L L M is such a useful tool for them to do their prompt completion. What's going to happen to them?

• 1:00:04Speaker 02

  It's a very interesting thing, what happens to us weak-minded human beings when we are hypnotized by things that are more hypnotic than our mind is to itself.

• 1:00:13Tim Scarfe?

  What's your take on physi... first of all, what is physicalism? Are you a physicalist?

• 1:00:18Speaker 02

  I think physicalism is a certain hypothesis, and it comes down to that being a causally closed mechanical layer to reality that can be researched using experiments in physics labs. And that this causally closed mechanical layer of reality is not moved by our intentions. It's so you cannot use an informal incantation of saying, please sunrise and the sun rises. This is something that is not happening in this causal layer, because it's mechanical. It does not care about what you want. And the second part to physicalism is that we are directly supervening over this mechanical causally closed world. So there's not some parent universe that contains a server where a nerd is running

• 1:01:02Speaker 02

  a simulated universe that is some kind of Minecraft and we live inside of this Minecraft. So it's not some simulation hypothesis. And so physicalism is these two components. But to me, physics is not something that has revealed itself directly, so nobody has ever seen matter and energy directly. Instead, matter and energy attempts to systematize observations, and our observations ultimately all come down to discernible differences. And the technical term for discernible differences is information. So to me, information is fundamental when we talk about reality. Reality is about differences in information. And physics is a certain layer of regularity in the differences in information that we have discovered.

• 1:01:47Speaker 02

  So to me, matter is not at the bottom. The bottom is really all these differences that we observed and which we experimentally and practically empirically found to be so regular that it can be translated into this mechanical world of physics. And then on top of it, we find at some point the organization into organisms, and the organisms are producing minds, and the minds are simulating us, simulating our sense of realness, of our sense of being in a world with colors and sounds and people and so on. And in this world, intentions matter. This psychological reality has things like will and emotion and despair and desire and all these things that make us tick.

• 1:02:30Speaker 02

  And magic in some sense is the idea that it's possible to influence this physical universe outside of minds using incantations, using magical rituals, that it's somehow it's possible that you burning a candle has an influence on celestial events and so on. And that is something that would contradict physics, right? It would mean that we actually do live in some kind of simulation that works similar to Minecraft, where you can say, time said day and the sun rises, despite what else happens in the programming code, because Minecraft is symbolic, right? It's not physical. It's something that is only made out of incantations.

• 1:03:06Tim Scarfe?

  Would you think it's possible for a physicalist to subscribe to your ideas and render it compatible, you know, compatible with physicalism?

• 1:03:15Speaker 02

  I would say that I am a physicalist in a certain sense, right? I would say that the theory that the world has a causally closed mechanical layer, right? Some kind of mathematics that is so self-evident that exists by itself, this kind of logos, the natural laws over which all the structure in the universe emerges, that makes sense. It's a good theory. And it makes good predictions that the stuff that we observe in the lab holds true across all the labs, and not just because the physicists all believe in it. It is something that seems to be working. There are also some regularities, like casinos seem to be working. It's not that people via having a belief in rituals are able to systematically break the casinos. It's a certain magic doesn't seem to be working.

• 1:04:00Speaker 02

  All the magic that seems to be working does rely on manipulating other minds. But it's possible that there are minds outside of the human minds, right? Maybe there are minds that emerge across all the minds in biology. Maybe there's something like a biological internet that can be influenced using rituals. Right? Maybe it's possible to influence what a forest wants at some level. I don't know that. I cannot exclude it from possibility, because there's so much information processing going and gone across organisms, that I don't know at which levels in nature, information is integrated and at which point it becomes intentional. So to me, physicalism is nothing that is incompatible with animism, right? There is a version of animism that is entirely physicalist.

• 1:04:40Speaker 02

  And I call this position cyber-animism, which basically says that we can explain animism through information processing in nature, and nature can be understood as something that can be entirely modeled using mathematics without intentions, including minds.

• 1:04:55Tim Scarfe?

  Okay, so Yosha, you're a functionalist, which means that you think it's not about the internal constitution, it's about the role that it plays in the system in terms of building minds. So minds can exist in any substrate. And I'm not sure whether you read this paper by Thagard in 2022. And he made this quite interesting argument that energy requirements undermine substrate independence. Do you think that that might refute functionalism?

• 1:05:21Speaker 02

  No, I think that he is making the claim that sometimes the subset has properties that influence its function. But this is exactly what functionalism is saying. To me, functionalism is not the claim that computers can think. It's a claim about how we construct objects. It's an epistemological claim. When we are talking about a thing like matter, we are not talking about something that has revealed itself to a wise person in their dreams. We are talking about a systematization of observables, and observables are discernible differences. There are ways in which reality changes. So basically, to a functionalist, an object is not some kind of hidden essence that reveals itself to you.

• 1:06:05Speaker 02

  Instead, the object is a construct, and this construct has been made by a mind. And the mind has constructed it to explain or to make the claim that when this object is present, the evolution of the universe is going to change in this particular way in its environment, right? So when an object like you, or a cup, or a bottle, or an atom is present, it's the claim is, the universe is going to behave slightly differently around this object than when this object was not there. Right? And so in this way, the object is a function. The object is describing a mapping from states to other states. And right, it's an influence on the evolution of the universe.

• 1:06:46Speaker 02

  So functionalism to me is simply the recognition of how we can construct objects. Let's take an example. Imagine there is a water molecule, and this water molecule internally doesn't have the proper essence of a water molecule. It just behaves like a water molecule in every regard, right? So it's, you can split it into something that totally looks like oxygen and hydrogen, and it has the same temperature at which it evaporates, it's connecting to other water molecules in the same way and so on. You can freeze it into ice in the same way, only internally it's different. Right? To a functionalist, this notion doesn't make sense, because all the other water molecules have been constructed in the same way.

• 1:07:27Speaker 02

  To, for something to be a water molecule, there is no other claim than to say what it behaves like, because we can never recognize its essence. We can only say, there's a certain set of observables that are invariant here. And this invariance is what we call a water molecule. It's only that. And so in the same way, we would say a mind is an invariance that exists on your brain, right? Something is characteristic over the way in which your brain functions in a certain area. And when we change this, for instance, by increasing your body temperature, at some point, your mind is going to break down, because you're going to be delirious, right? And so if you bring this back to this argument saying, okay, if we change the energetic conditions around you by increasing your body temperature, of course, it's going to

• 1:08:10Speaker 02

  influence your mind to some degree, because at some point it's no longer able to maintain its function.

• 1:08:15Tim Scarfe?

  But there's always this counterargument that a simulation of water doesn't make you wet. But would the functionalist reply that even the feeling of wetness is just a behavior like any other?

• 1:08:26Speaker 02

  It's an interesting thing when you are playing Minecraft. There is water in Minecraft, right? And when you are on fire and you get into the water in Minecraft, the fire is going to get extinguished, because you're wet now. At some point, somebody made simulated Minecraft that was using a neural network to learn Minecraft. The neural network was only trained in the visuals of Minecraft. And so you could go and play the simulated Minecraft despite not being any Minecraft source code below it. And so I thought I'm going to make a joke about how simulated water and simulated Minecraft can not make you wet. And I looked it up and tried it out and I found actually it was true. Because when people tried to get into the water in the simulated Minecraft, the water

• 1:09:09Speaker 02

  would receive to you, you could not immerse into it because the neural network hadn't learned it. So accidentally, it was true that the simulated water and the simulated Minecraft couldn't make you wet. But of course, in the real Minecraft, the real Minecraft player, the character that you're playing in Minecraft, this avatar, can get wet. Of course, you sitting outside of Minecraft are not going to get wet because you are not inside of the Minecraft, you're outside of the Minecraft. And of course, something that is outside of this universe is not going to get wet by anything inside of this universe, right? Wetness is a property that is constrained to a universe that has eight systems that have the function of making you wet. If this function is being implemented, like in many computer games, you can get wet for some definition of wet, right?

• 1:09:53Speaker 02

  Then it's fine, and you can go to an arbitrary degree of physical simulation to make something wet, to satisfy whatever you want to call wetness. So I don't understand the argument that simulated water makes you wet. I don't understand why there is such a strong camp of simulated water doesn't make you wet in philosophy, and I think it has social reasons, not intellectual reasons, that this is the case.

• 1:10:19Tim Scarfe?

  So in your paper, you had this beautiful history of computationalism. And just as a teaser, you said computationalism might be best understood as a variant of constructivism, but we'll get there just in a second. And you spoke about the history of Russell Whitehead trying to ground mathematics in this formalist program where there is no divergence between syntax and semantics. And maybe just start from there.

• 1:10:41Speaker 02

  So it's really interesting that mathematics couldn't be understood as a way to systematize thought beyond the human mind. It's a way, for instance, in music, you have notations for sounds in music that allow you to talk to other musicians about what they should be playing. And it's something that is trying to capture the essence of the music in this notation. And mathematicians were trying to find a notation for the essence of when you have mathematical thoughts. And they expanded this beyond the human mind because you can write down things in mathematics that are larger than what you can fit in your own imagination. At some point, mathematicians discovered that geometry can be expressed using algebra. So they're basically formulas that produce geometry.

• 1:11:25Speaker 02

  And this all has been rediscovered by computer scientists, that's why computer games can have geometry. So the interesting thing in computer science is that you can build the entirety of the mathematics, computer algebra systems, using simple operations over automata. And so there is a nice thing in a computer, there is never a divergence between syntax and semantics. Because your computer is always going to be in the state that its syntax prescribes, and that's its semantics. There is no difference. Maybe the computer program doesn't do what you want it to do, but it's always doing what it's doing. And in mathematics, there has been a divergence between what the mathematicians wanted to express and what the syntactic operations are doing.

• 1:12:10Speaker 02

  And I think that's because this programming language for thought that mathematicians have developed has bugs in it. And what Gödel has discovered is that these bugs cannot be eliminated, except by redefining the programming language of mathematics. And I think the bug lies in axiomatic mathematics. So the way in which mathematicians define objects is that they simply postulate their existence as an axiom. And the axiom can be a black box, and you just write on the black box what the black box is supposed to contain. And as long as this doesn't lead to contradictions, that's the other black box, is that you have to find your good. And there were some very counterintuitive results that happened from the way in which

• 1:12:54Speaker 02

  mathematicians defined arithmetic, and they typically do this via set theory. And all these issues led Hilbert to saying, please, mathematicians, there seems to be something wrong here, can't you fix this? And so Gödel had this idea of trying to prove properties of his language by defining language within itself. The problem is that the language cannot point outside of itself. It's something that semiotics doesn't always get. Many semiotics just try to draw arrows into reality outside of the language, but this doesn't really work. You can only point from a model to within it, within its own language. And so if you want to make proper, discuss properties of the language, and the language itself is defined outside of itself, what can you do?

• 1:13:39Speaker 02

  And Gödel had this brilliant idea of recreating the language within itself by emulating it. And then this way, he could show that certain constructs cannot be avoided, that you can create self-referential statements that contradict themselves. And you could not avoid this. And the reason why you could not avoid this is because mathematics, the way in which Gödel understood it, doesn't have state. It's basically eternal, doesn't go from step to step. Everything is simultaneous. And this property of the simultaneity, the statelessness in mathematics is very useful because it allows you to have infinities. If you can only go from step to step, and you want to build your infinity stepwise, you know we're going to get there, right? Maybe you have unboundedness, something that never ends, but you never going to have a

• 1:14:24Speaker 02

  full infinity. And mathematicians have got used to these infinities and continue on and so on. Physics has even used them since mathematicians have given it to them. It would be really bad if you didn't have them. And Gödel basically found that there is no way to fix this, that there is no way to fix axiomatic mathematics without allowing certain contradictions. And Russell and Whitehead and Principia Mathematica were attempting to do this by limiting set theory in such a way that sets had to be at the same level. And you could not simply integrate them over each other to create loops. And I think this was a limitation that was too strong because we think like this, we

• 1:15:08Speaker 02

  are creating constructs of the type that Russell and Whitehead were excluding from us. Also, Gödel was basically showing that Russell and Whitehead's attempt was insufficient to eliminate the contradictions in classical mathematics. But there is a way to eliminate the contradictions, and the way is to do what computer scientists are doing. So instead of using classical mathematics, you only use computer algebra systems, systems that from smaller elements, from simple automata are creating all the constructs that you want. And you can have stuff that looks as continuous as you want in a computer game, right? You will never have a perfect circle, but you will have something that is as circular as you wanted, as you can afford it.

• 1:15:50Speaker 02

  You can always create as much resolution as you can afford, but you will not have a universe that is truly infinite, that is truly continuous. And this is this constructive program that basically, I think in classical mathematics, the first formulation was Broers intuitionism. And I don't think that he formulated it very well, but I think what he could see, what anticipated, was the need to build mathematics from elements that you actually understand instead of from axioms, from things that you, where you don't have any black boxes, but everything is built from the ground up. So in the world of the computer, everything is built from the ground up. And there is a reason why you should think that generalization of thought and of physics would also be like this.

• 1:16:34Speaker 02

  Because for our mind to represent things, our brain needs to build the representations from the ground up, from communications between neurons. So when you see a continuous line, something in your brain is producing this continuous line using communications between neurons, which means there is some kind of computation going on. And the idea that there is something that is not made step by step between the communications between the neurons is probably a faulty intuition.

• 1:17:00Tim Scarfe?

  Where do these things come from originally? You know, so how did the program of consciousness get bootstrapped with this computation idea?

• 1:17:10Speaker 02

  You mean in the physical universe?

• 1:17:15Speaker 02

  So of course I cannot know, because I don't see the universe from the outside. We have to think about possibilities. My current intuition is that the universe exists because nothing prevents its existence. We might have this intuition that for something to exist rather than nothing, there needs to be something positive. But if nothing exists before the ground truth, before the ground level of reality, then there is also nothing that prevents the universe from existing. So if the universe is not forced to exist or prevented from existing by something that is preceding it, then the universe is not impossible, then it's probably undecided about its existence. So it's going to have an existing branch and a non-existing branch.

• 1:17:58Speaker 02

  The non-existing branch doesn't have any observers that complain about non-existence, but the existing branch is going to have observers that might complain about existence. Now we need to explain, we're in the existing branch, why some things are happening and other things are not happening. And the easiest explanation is that basically everything is happening. And what's happening is the superposition of all automata. So the universe at every point is branching out in all possibilities. And these possibilities are basically can be understood as operations on the previous state. There are also operations which don't change the previous state and just leave it as it is. So unitary operations, but they're not going to lead to an evolving universe. The branch in which we are going to be is one that is evolving.

• 1:18:42Speaker 02

  And the stuff that we can evolve in is stuff that retains memories of previous states. So we find ourselves limited to a subset of the possible mathematics over which universes can emerge. And these are the these ones that are branching out over possibilities that can retain memory. Now, what are the things that are happening? When the universe branches out in an unbounded number of possible branches at every step, where do you find yourself? Well, everywhere, right? In every step, you're going to find yourself in multiple branches, but you don't know about these other branches, because you only have memories of the past, not of the stuff that happens simultaneously in the parallel branches. And if you want to do something in this universe, for instance,

• 1:19:26Speaker 02

  if you want to build structure for yourself, because you want to become an organism, you need to be able to make guesses about what other parts of the universe are doing. But everything is changing all the time, because you're always splitting up in all these branches. So you look into your future and you only see fog, right? Because everything is randomly branching out. And so there must be some things that are stable. And it turns out, under some conditions, error correcting patterns emerge. For instance, you sit in a bathtub, you move your hand, and you see that all sorts of waves emerge on the surface of the bathtub when you do this. And these waves are propagating over the surface of the water, and most of them just dissipate. But some of them stay around, these circular vortices. And why do these circular vortices have such a staying power?

• 1:20:10Speaker 02

  It's not because there is a higher probability that they emerged than the other waves. It's just the only thing that's left over, because in the circular vortices, the information gets bent back into itself. So it does not leave a certain region of space in the water, right? And so this little vortex is a pattern that has the ability to error correct itself. And such error correcting patterns have the ability to stay until friction kills them. At the ground level of the universe, these error correcting patterns, the particles don't encounter any friction, right? Because there is no friction yet. The only thing that they encounter is sometimes other particles that they bump into. So with a certain probability, once they emerge, they're going to stay around, until they either bump into another particle

• 1:20:52Speaker 02

  or the conditions for their non-existence are met, because the error correction is not strong enough. And then on the next level, you have interactions between these vortices, between the particles. And they form molecules. At some point, they form cells, which are super molecules. But cells have now something else. They cannot just stay around by error correction. They're patterns that are able to basically dig a groove deeper into the universe. They can self-replicate. And in this way, they can also deal with their own dissolution. They're able to recreate themselves and in this way make themselves happen more. And so evolution is about life, about patterns that are not simply replicating themselves, but deepening themselves, that imprint them small in the world by controlling it,

• 1:21:35Speaker 02

  by being able to overcome the noise, the entropy, the way in which the universe goes out of sync with your expectations and with your preferences. And so consciousness, I think, emerges as a colonizing pattern, as a pattern of the type that is not simply replicating itself, but making itself more real. And makes itself more real by building machinery that allows consciousness to stick around, by controlling reality, by anticipating different ways that could thwart it, that could make it harder for it to exist, and changing reality around itself, changing the future to make itself more likely to happen in the future.

• 1:22:12Tim Scarfe?

  And in terms of more real, you don't just mean real means that it has many physical instantiations. You mean real is the pattern in the information ontology itself?

• 1:22:20Speaker 02

  Yes, to real means for me to be implemented. And so it means that there is some pattern in the substrate that is actually implemented as a pattern in the substrate. And so consciousness is a pattern in the substrate of the physical universe that has not just the ability to replicate itself, but the ability to colonize, to possess, to engrave itself deeper and thereby changing the other patterns in the physical universe.

• 1:22:49Tim Scarfe?

  Very good. And what you were saying about the multi-universe phylogeny, it's very similar to Wolfram's idea of this constructive universe. He talks about the ruliad, which is the entangled limit of all possible computations.

• 1:23:03Speaker 02

  I think that this pattern of the ruliad is exactly the superposition of all the automata that is producing the universe, or the old Greeks taught it logos. It's basically the mathematics that is so simple that you don't need to derive it, that exists by itself. And so the universe can be understood as an emergent mathematical object that exists over very simple rules, that are so simple that they can exist by themselves.

• 1:23:26Tim Scarfe?

  Let's talk about the measurement problem in consciousness. How do we know the difference between something which is actually conscious versus simulating it?

• 1:23:35Speaker 02

  That's a very tricky question. And to me, it comes down to whether the system that is producing statements about its conscious experience to itself and others is implementing a causal structure that gives rise to phenomenal experience of being an observer. And the difficulty here is that most people's intuitions don't tell them a lot about what is necessary for a system to have such an experience. Personally, my intuition says it's not that hard. When we think of spatial cognition, spatial cognition doesn't seem to have a hard problem.

• 1:24:18Speaker 02

  Nobody is puzzled by the fact that a self-driving car can have spatial cognition. And I wouldn't say that it's simply deep faking spatial cognition, because we accept that spatial cognition is the implementation of certain mathematical regularities that human brains can learn and that computers can learn. It's where we can even express analytically to a large degree, we can write down a computer program that is able to perform spatial cognition, except in lots of edge cases, when you actually want to deal with messy camera images. But by and large, you don't need to have an end-to-end train system to do this. And if you have an end-to-end train system, you don't think there is any magic going on in there. Whereas people expect that a thing that experiences itself, experiencing is somehow magical.

• 1:25:02Speaker 02

  And personally, I don't see why this should be magical. I remember when I was a kid, I was dreaming about having chocolates. And these chocolates had a very strong smell, and I wanted to touch and eat them. I was, I think, two or three years old when I had this dream. And as dreams have it, there's this insidious thing that you're about to eat the delicious chocolate, and this is the moment when you wake up, right before you want to put it into your mouth. It's very torturous. And so I woke up and suddenly the delicious chocolate had turned into an idea. It was no longer real. It was simply something that had no phenomenal content anymore. I knew everything about these chocolates that I had in my hand a moment ago. They were just gone, they were no longer real.

• 1:25:47Speaker 02

  And so I thought, I need to make them real again, I need to fall asleep again and enter the dream again. And I succeeded. I went back into the dream, and the chocolate became real again. I could smell it and taste it, and then I flipped back into the wakefulness state. And again, the phenomenality, the realness disappeared. And I realized the chocolate is no longer real at the same level of me. It's now something that I hold in my mind, only as an abstract idea that I can play around with. But contents are exactly the same. The only thing that has gone is the illusion of realness. So what is this illusion of realness? I suspect it's a feature dimension similar to color, sound, number, weight and so on. It's a representation inside of my own mind.

• 1:26:32Speaker 02

  And when you dream, you often dream that ideas become real. And when you perceive the outside world, then all the stuff that is validated by your sensory system is tagged by your brain is real. And sometimes there is a defect in your brain and you experience things that you imagine is real and we call this a hallucination. It's not the same as in the LLM. It's that you are confused about the fact that your brain is producing illusion of something that doesn't have sense, sensory validation. And yet, for some reason, the brain has an arrow and tags it as sensory validated and suddenly it becomes real. And your own experience of yourself is also something that is validated in this perceptual way, so experience it as real.

• 1:27:15Speaker 02

  And now you should be able to test this by getting into a state where yourself become de-realized and you suddenly experience yourself only as an idea. And the object of the conscious observer and the subject of the conscious observer disappear. And indeed, I was able to recreate this. And I know this is not that rare. It's called de-realization or de-personalization, not exactly the same phenomenon, but very closely related. And people report on this. And this is involuntarily, these people often end up in psychiatric institutions because it's very stressful to them. And they don't know how to deal with the fact that they're no longer real to themselves. And there is no I anymore that they can meaningfully refer to. And it's an interesting state to be in.

• 1:27:58Speaker 02

  And I'm happy that it ended, but it's something that you can induce. And in this state, phenomenality changes. And so to me, it seems plausible that this explains how consciousness is working, that consciousness is a particular type of representation that is created in my mind and that doesn't have a complexity that necessarily goes beyond what the LLM is producing. Now, there is a problem with the LLM because the LLM is a text generator. And you don't know what the text indicates. The text is trained on novels, for instance, and in many novels, the authors write about stuff that don't experience when they write it. The novel is often a deep fake. When an author writes about how a character is experiencing how to die, the author probably

• 1:28:42Speaker 02

  didn't die while they wrote this. So it's a deep fake of a conscious experience. And so if the LLM is trained to produce deep fakes, how do you know when it actually experiences something? And that, that is one of the big difficulties, right? The LLM is a shapeshifter, it can produce arbitrary characters and stories. And so at which point is it fake and at which point is it actually simulating the causal experience? And that, that is really the problem that requires some kind of mechanistic interactivity, where we would actually need to look into the structure of the LLM to find if it's causally equivalent to the structure of the mind that is producing the same thing.

• 1:29:21Tim Scarfe?

  Yeah, so we need to have a white box method to ascertain if consciousness is present. Isn't that interesting though? Because do you think that for intelligence, behavior alone is enough to ascertain whether it's there? But for consciousness particularly, we need to look for certain types of internal organization and causal structure.

• 1:29:38Speaker 02

  Yes, but most people have the intuition that the LLM is doing something that is much more simple than what we are doing when we are conscious. But we don't expect that the LLM is doing something that's much more simple when it does arithmetic than we do. And I think that this intuition might be misguided. It could be that the LLM is doing something that is just as simple as what the human brain is doing when it's conscious. And I'm pretty sure that the human brain is doing the simplest possible thing that it can do to produce our behavior. And the reason why we are conscious is because it's part of the simple strategy to produce our behavior. And so if we were to get a learning system, an artificial learning system, from the ground

• 1:30:21Speaker 02

  up, to discover how to make agentic coherent behavior. And the system is behaving in a way that makes us think that it's conscious. The simplest explanation I think is that it actually is conscious. And my intuition is not necessarily telling me, oh my God, this is going to do something that is much more complicated than us to produce the same behavior. So it's going to do this in some kind of circumspect way that is producing all the behavior that require consciousness in the human. But in this case, it's doing something that only pretends that it does it, which ultimately might be more complicated.

• 1:30:55Tim Scarfe?

  What would falsify your machine consciousness theorem?

• 1:31:00Speaker 02

  So if we were to build a system that is behaving in the same way as us, there is a question, can we produce the same behavior as ours without phenomenality? And the phenomenality, I think, is a representation here in this case. If we look at what the LLMs are doing and beat extent this into a multimodal model and they produce similar output as us, but they're able to observe themselves and their observations that they report differ from what we are doing. Then we would have some evidence against it. But the more fundamental question would be if we were to build a self-organizing system

• 1:31:43Speaker 02

  that is by a self-organized learning, trying to get there. And there is no way in which we achieve the organization of behavior in a self-organized way instead of imitation learning as the current LLMs are doing. This might be some evidence that consciousness is more complicated than something that can run on today's computers. But it is not definitive proof because it could simply be that I made a mistake.

• 1:32:12Speaker 02

  The fact that I don't achieve a certain result doesn't mean that it cannot be achieved. It means that I couldn't achieve it. But so it's very difficult to have a proof against it. So it's more evidence for it if we succeed. So what we would have to show in order to convince Anil Seth maybe is that we build a system that is self-organizing, that is not trained on text, that is doing developmental learning. And that naturally gets to the point where it makes self-reports and reports about its conscious experience. And it requires this conscious experience to produce its performance. So I think this would be evidence in favor. The contrary is much harder.

• 1:32:53Tim Scarfe?

  But in a sense, I don't think you believe in zombies because you think that consciousness is an absolutely integral part of cognition. I mean.

• 1:33:01Speaker 02

  There is a possibility that you get to zombies. This other, what I meant is if we produce something that is able to produce our behavior except for experience of phenomenality. So it's basically able to do everything that a human being is capable of doing and more. But it does not need to produce a conscious observer. Right, this could be a zombie like this. And I think zombies are possible. By dearly, you can build self-driving cars that do things that would require intelligence in human beings. So you can play chess-playing computers that can do things that require consciousness in human beings. And they achieve all those things without consciousness. And that's because we implement them on the substrate that allows us to do, to construct

• 1:33:45Speaker 02

  algorithms on them that would not run on human brains. The chess algorithm that runs on the current chess computer could not be implemented on the neurons of your brain because how would you entrain it on these neurons? So you wouldn't get it in there in order to entrain yourself to play chess. You need to have a conscious human being. It's the only way in which human being is training itself up there. And it's conceivable that the entire set of intelligent behaviors that you are interested in, except for this one thing where you're simulating self-reporting conscious agent, can be implemented on GPUs because the GPUs are more powerful, more deterministic, more open to other classes of algorithms.

• 1:34:26Tim Scarfe?

  But could we say that zombies are possible in principle, but not in practice? And what I mean by that, let's say, we could imagine a world where we could train an MLP to be the same as GPT-5. It would just take an insane amount of computation power. And similarly, consciousness is what makes our minds tractable. It's an incredibly efficient process. So we might say that if we see an artifact in our world which behaves as if it's conscious, would you argue that it can't be anything other than conscious? Or do you still think there are other states that could be zombie-like?

• 1:34:59Speaker 02

  I think we do have zombies already. I think the chess program is a zombie. You cannot train a human being to play chess without the human being being conscious. But you can train a computer to play chess without the computer being conscious. And if the computer was self-organizing, it's possible that it could only be trained if it was conscious. Or this was the simplest way of doing it. But for many of the things that we currently observe, for instance, you can get the LLM to produce a tax return, even very, very complicated things while being a zombie, while not having conscious experience of producing a tax return. The only point where the LLM is reporting to be conscious, if you ask it to, right?

• 1:35:40Speaker 02

  If you are telling it, the following is a conversation between what it would be like if a human being would be talking to a conscious AI. So some version of this, right? And then the thing is able to simulate this. And then the question is, is this more simulated than us? Or is this only a deep fake that is getting there in more circumscript ways than is happening in our mind? And I don't know that, right? So I'm agnostic with respect to this. But I would say that in all the other cases, the thing is probably a zombie. And so it's possible to make something that is behaving in all sorts of ways as you do, but at some point it's going to be more complicated to make it without consciousness. So zombies, I think, are always possible.

• 1:36:21Speaker 02

  But sometimes it's going to be more complicated to make a zombie that produces the behavior than to make a conscious agent that produces the behavior. And this is where the intuition that I have diverges from the intuition of John Searl or of Anil Seth. Because I think they both believe that consciousness is very, very complicated because it's not intuitive to them how it works. And it doesn't seem to be counterintuitive to me, so I don't see anything magical about consciousness.

• 1:36:47Tim Scarfe?

  If you had $10 billion to kick off a research project on consciousness, what would you do?

• 1:36:53Speaker 02

  I think I would hire a lot of people who are a lot smarter than me. At the moment, I started a smaller initiative. It's called the California Institute for Machine Consciousness. And I'm currently looking for people who are smarter than me, who want to join me and to build a project in which we can look at the search space for consciousness and which we can see, can we build a self-organizing system to learn? And is this thing going to develop phenomenology? And I hope that is going to help us to build our intuitions. Even if we turn out that it doesn't work, we are going to learn something, right? And progress is being made by figuring out the ways in which I'm wrong, not just the ways in which I'm right.

• 1:37:36Speaker 02

  So if I had $10 billion, my strategy wouldn't change substantially, except I would be able to hire more people for this and start with larger computers.

• 1:37:46Tim Scarfe?

  Yoshua Bach, it's been an absolute honor. Thank you so much for joining us today.

• 1:37:50Speaker 02

  Thank you. It was a pleasure.

• 1:38:05Speaker 00

  Life is all about making decisions. Sometimes you make questionable ones, like buying a new TV that's too big for the wall because the electronics store employee was a little too good at their job. Then, sometimes you make great ones, like protecting your home and auto with Progressive. You get customized coverage options so you can be sure you have the right coverage. Get your quote today at progressive.com. Progressive Casualty Insurance Company affiliates and other insurers, not available in all states.

• 1:38:34Speaker 01

  As wish lists grow, you know where to go. Head to Ulta Beauty. We have gifts they'll love from brands like Tato One and Rare Beauty by Selena Gomez. Shop beauty faves to give or keep from Shark Beauty, Sol de Janeiro, and more. Not sure what they want? You can't go wrong with Balmain, Clinique, or Redken. And don't forget to put yourself on the list. Book a salon moment or add a little more sparkle to your season. Shop holiday in-store or at Ulta.com. Ulta Beauty, the