Bayesian Brain, Scientific Method, and Models [Dr. Jeff Beck]

Dr. Jeff Beck, mathematician turned computational neuroscientist, joins us for a fascinating deep dive into why the future of AI might look less like ChatGPT and more like your own brain. **SPONSOR MESSAGES START** — Prolific - Quality data. From real people. For faster breakthroughs. https://www.prolific.com/?utm_source=mlst — **END** *What if the key to building truly intelligent machines isn't bigger models, but smarter ones?* In this conversation, Jeff makes a compelling case that we've been building AI backwards. While the tech industry races to scale up transformers and language models, Jeff argues we're missing something fundamental: the brain doesn't work like a giant prediction engine. It works like a scientist, constantly testing hypotheses about a world made of *objects* that interact through *forces* — not pixels and tokens. *The Bayesian Brain* — Jeff explains how your brain is essentially running the scientific method on autopilot. When you combine what you see with what you hear, you're doing optimal Bayesian inference without even knowing it. This isn't just philosophy — it's backed by decades of behavioral experiments showing humans are surprisingly efficient at handling uncertainty. *AutoGrad Changed Everything* — Forget transformers for a moment. Jeff argues the real hero of the AI boom was automatic differentiation, which turned AI from a math problem into an engineering problem. But in the process, we lost sight of what actually makes intelligence work. *The Cat in the Warehouse Problem* — Here's where it gets practical. Imagine a warehouse robot that's never seen a cat. Current AI would either crash or make something up. Jeff's approach? Build models that *know what they don't know*, can phone a friend to download new object models on the fly, and keep learning continuously. It's like giving robots the ability to say "wait, what IS that?" instead of confidently being wrong. *Why Language is a Terrible Model for Thought* — In a provocative twist, Jeff argues that grounding AI in language (like we do with LLMs) is fundamentally misguided. Self-report is the least reliable data in psychology — people routinely explain their own behavior incorrectly. We should be grounding AI in physics, not words. *The Future is Lots of Little Models* — Instead of one massive neural network, Jeff envisions AI systems built like video game engines: thousands of small, modular object models that can be combined, swapped, and updated independently. It's more efficient, more flexible, and much closer to how we actually think. Rescript: https://app.rescript.info/public/share/D-b494t8DIV-KRGYONJghvg-aelMmxSDjKthjGdYqsE --- TIMESTAMPS: 00:00:00 Introduction & The Bayesian Brain 00:01:25 Bayesian Inference & Information Processing 00:05:17 The Brain Metaphor: From Levers to Computers 00:10:13 Micro vs. Macro Causation & Instrumentalism 00:16:59 The Active Inference Community & AutoGrad 00:22:54 Object-Centered Models & The Grounding Problem 00:35:50 Scaling Bayesian Inference & Architecture Design 00:48:05 The Cat in the Warehouse: Solving Generalization 00:58:17 Alignment via Belief Exchange 01:05:24 Deception, Emergence & Cellular Automata --- REFERENCES: Paper: [00:00:24] Zoubin Ghahramani (Google DeepMind) https://pmc.ncbi.nlm.nih.gov/articles/PMC3538441/pdf/rsta201 [00:19:20] Mamba: Linear-Time Sequence Modeling https://arxiv.org/abs/2312.00752 [00:27:36] xLSTM: Extended Long Short-Term Memory https://arxiv.org/abs/2405.04517 [00:41:12] 3D Gaussian Splatting https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/ [01:07:09] Lenia: Biology of Artificial Life https://arxiv.org/abs/1812.05433 [01:08:20] Growing Neural Cellular Automata https://distill.pub/2020/growing-ca/ [01:14:05] DreamCoder https://arxiv.org/abs/2006.08381 [01:14:58] The Genomic Bottleneck https://www.nature.com/articles/s41467-019-11786-6 Person: [00:16:42] Karl Friston (UCL) https://www.youtube.com/watch?v=PNYWi996Beg

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 0:31Speaker 01

  Low energy, unfocused, foggy? You might be dehydrated. Whether it's hot yoga, over 100 degree weather, or too much sun. Gator Light, with a specialized blend of five electrolytes, was scientifically designed to help move fluids into the body faster for rapid rehydration. Shop now at retailers nationwide. Gator Light: Hydrates Faster Than Water. Is it in you? Symptoms noted are signs of mild to moderate dehydration. Consult a healthcare professional if symptoms persist.

• 0:59Speaker 02

  So my PhD is in mathematics from Northwestern University. I studied pattern formation in complex systems, in particular combustion synthesis, which is all about burning things that don't ever enter the gaseous phase. Bayesian inference provides us with like a normative approach to empirical inquiry and encapsulates the scientific method writ large. All right, I just believe it's the right way to think about the empirical world. I remember I was, I was at a talk many years ago by Zubin Garamani, and he was explaining the Dirichlet process prior. This is when the Chinese restaurant process and all that stuff was like relatively new. And his explanation of it, it so resonated with me in terms of like, oh my gosh, this

• 1:43Speaker 02

  is the algorithm that summarizes what, what the science, how the scientific method actually works, right? You get some data, right? You get, then you get some new data and you sort of say, oh, how is it like the old data? And if it's similar enough, then you sort of lump them together and then you sort, and you build theories and you properly test hypotheses in the fashion. That's, that's, that's the essence of the Bayesian approach is it's about explicit hypothesis testing and explicit models, in particular generative models of the world conditioned on those hypotheses. It, I believe it is, it is the only right way to think about how the world works and see, and, and it encapsulates the, the structure of the scientific method.

• 2:25Speaker 02

  I mean, if I'm being perfectly honest, what actually convinced me the brain was Bayesian had a lot more to do with behavioral experiments done by other people. My principal focus was on, well, how does the brain actually do this? So I'm referring to experiments, you know, showing that like humans and animals do optimal cue combination were surprisingly efficient in terms of like the information that comes using the information that comes into our brains with regards to, again, these low-level sensory motor tasks.

• 2:51Tim Scarfe?

  Oh, interesting. So it's almost like we we're so efficient that the only explanation that makes sense is that we must be doing Bayesian analysis.

• 2:59Speaker 02

  More or less. I mean, it's a bit more precise than that. It's not just efficiency. It's, you know, like the cue combination experiments, I think, are really compelling. And so the idea behind a cue combination experiment is that I give you two pieces of information about the same thing. And one one piece of information is more reliable than the other, and the degree of reliability changes on a trial-by-trial basis. You never know a priori that, like, say, the visual cue as opposed to the auditory cue is going to be the more reliable thing. And yet, nonetheless, when people combine those two pieces of information, they take into account the relative reliability on a trial-by-trial basis. And that means that they're optimal in a sense. Now, we have to, like, be super careful with our words. They're relatively optimal because they're not actually using 100% of the information

• 3:44Speaker 02

  that the computer, like, the visual information that you use, you don't use 100% of the information that the computer provided you, right? But, you know, there is some loss between the computer's screen and your brain mediated in principle by, but the system behaves as if, right? It has optimally combined those two cues. It has taken into account uncertainty. This also is because it's like how we really do think about the world. Like, we take into account uncertainty all the time in our decisions, right? You know this, if you're driven in the fog, you're aware of this. Ninety percent of what the brain does is decide what to ignore, right? And because if if we didn't... Right, we'd be screwed, right?

• 4:26Speaker 02

  We receive an insane amount of information, most of which we don't even bother to process. Right, so.

• 4:33Tim Scarfe?

  Yeah. Is that definitely the case, though? Do you think that we could actually be processing more information than we know?

• 4:41Speaker 02

  We are definitely processing more information than comes out in behavior.

• 4:45Tim Scarfe?

  Yeah.

• 4:45Speaker 02

  And a lot of that is because we are continually learning. And like learning, you know, you have to, you know, you close your eyes for five years and your visual system decays. Right, you lose fidelity, right? It forgets. It requires constant input simply to maintain this understanding of like the low-level statistics of the visual world. Right, without input, like you're, you know, you're W. So the question is, is that, is that using all the information? Or is it just using like the low-level information? And it's information that we don't like directly perceive, but it's still definitely being used in a sense. When it comes to, you know, but what is it being used for?

• 5:28Speaker 02

  It's being used to track these sort of low-level statistics that we sometimes need, but don't always need. And so this is why I say that like, you know, when we say context matters, you know, you can think of that in terms of like we were able to flexibly switch between tasks, which means having a lot of resources and having a lot, you know, maintained and having them still be in good working order just in case we need them. Right, and this is why like the self-supervised or unsupervised learning approaches that are ubiquitous for like starting, you know, getting your LLMs to give you, you know, sort of your reasonable prior over language is the sort of stuff that your brain is definitely doing.

• 6:08Speaker 02

  So in a sense, it is using everything, but it's not really using all of the information that's present. Right, and that's sort of, I think, the argument that I want to make.

• 6:17Speaker 04

  The idea of having to traffic in squishy people in order to make our systems go is not immediately appealing, let's put it that way.

• 6:26Tim Scarfe?

  This episode is sponsored by Prolific.

• 6:29Speaker 05

  Let's get few quality examples in. Let's get the right humans in to get the right quality of human feedback in. So we're trying to make human data or human feedback. We treat it as an infrastructure problem. We try to make it accessible. We make it cheaper. We effectively democratize access to this data.

• 6:46Tim Scarfe?

  What do you think about these broad sort of metaphorical idealizations? You know, the big one is that the brain is a computer. The probably the more popular one is that the brain is a prediction machine.

• 6:58Speaker 02

  It will always be the case that our explanation for how the brain works will be by analogy to the most sophisticated technology that we have. Is that, how's that for a non-answer? Right? So, so, you know, you know, a couple thousand years ago, right? How'd the brain work? It was like levers and pulleys, man. I mean, duh, don't be ridiculous. Why? That was, you know, at some point in the Middle Ages, it became humors, right? Because fluid dynamics was like the, you know, was the kind of techno, you know, the technology that was like the most advanced or technology that took advantage of of water power was like the most advanced technology that we had. Now the most advanced technology is computers. So duh, that's exactly how the brain works.

• 7:40Tim Scarfe?

  Philosophers used to think that the universe was a machine.

• 7:43Speaker 02

  Mm-hmm.

• 7:44Tim Scarfe?

  And Winfrey Chomsky about this as well, you know, because he talks about the ghost in the machine, you know, and the ghost is all of the bits in the machine that we don't understand. But do you think now that we can think of the universe as a machine?

• 7:56Speaker 02

  I think that that is a very convenient way to think of the universe. Right? So when we model the universe as having like causal structure, right? Do we do so because it actually has causal structure? Or because that's a really convenient class of models with which to work? I think that it's, you know, it has causal structure, right? But also it's a convenient class of models, right? So like a good example is large language models, right? So they're all, you know, the most, but not all are autoregressive in terms of their predictions. All right, well, why? Like, why is it auto? Oh, it's because it's mathematically convenient. It's a compact way to like take the past and make a prediction about about the future. Does it mean that that's actually the way language works?

• 8:40Speaker 02

  No, I don't think it's actually the way language works. But it's a computationally convenient model. In physics, we have, like, there are, in fact, like, momentum is a good example. Like, why do we need momentum in order to describe? We don't, you know, we don't observe momentum directly, right? We only, you know, you're just looking at videos, you just, you know, the position of the ball, right? You know, you want to infer the velocity, well, you just take the difference between two adjacent positions and then that gives you momentum. But you don't ever directly observe, like, the momentum. And this is, you know, in a mechanical, in a mechanical setting. So why did we choose momentum? Well, we chose momentum because that's the variable that, that, you know, that if we

• 9:22Speaker 02

  knew what, if we knew what momentum was, now everything is Markovian, right? Everything is, it's a, now there's like a simple, like causal model that describes how the world works. We picked that model because, we picked that particular hidden variable because it's what rendered the model causal. Does that mean that's how the universe works? Or was that just a computationally convenient choice? I'm going to stay agnostic on that one. But I do like that it's a computational, that ended up working out, right? So.

• 9:52Tim Scarfe?

  And just quickly riff on the benefits of having models that preference causal relationships.

• 9:58Speaker 02

  So the nice thing about causal, so when you have a causal relationship, it reduces the number of variables you have to worry about and track. That's the beauty of having a causal, it's like, it's like a Markov, it's the same, it's the same argument with, with momentum and Markov models. We chose to have that hidden variable because it's the thing that made the model simpler, right? It made the calculations easy. Now we can just, like, go forward in time, just make predictions in a, in a totally, like, iterative fashion. That's what makes causal models great. The other thing that makes causal models great is if you do ever intend to serve, you know...

• 10:32Speaker 02

  Act or behave, right, then you still need to be, you know, you need to be able to predict the consequences of your action. The more tightly linked your actions or your affordances are to the things that causally impact the world, the more effective those actions are with respect to your model, but hopefully also with respect to reality. And so we prefer causal models, you know, in part because they are relative, relatively speaking, simpler to execute, right, in a simulation form, but also because they point directly to, well, where should I intervene? You know, where should I go in? And, you know, and how should I choose my like, series of actions that will give me the desire, that will lead me to the desired conclusion or goal?

• 11:14Tim Scarfe?

  What's the difference between micro causation and macro causation?

• 11:18Speaker 02

  I think the difference between micro and macro is a single letter. No, um, so.

• 11:25Tim Scarfe?

  We could just model the light cone at the particle level.

• 11:28Speaker 02

  Oh, yeah.

• 11:29Tim Scarfe?

  So that's how physicists think.

• 11:31Speaker 02

  Yeah.

• 11:32Tim Scarfe?

  Yeah, I mean, that's the way physicists see the world. And, and we see the world in terms of populations and people and all these macroscopic things. And, and we still reasonably do experiments and we do interventions and we do randomization.

• 11:43Speaker 02

  To truly identify a causal relationship, you have to do an intervention, right? Um, you know, the classic example, this is also in lung cancer, right? It's like, I forget how long ago this was, but at one point, there was this belief that alcoholism caused lung cancer, but it was actually because they were in poor health because they were alcoholics and they smoked a lot more than the rest of the population, right? So you do need to do that kind of intervention to discover a causal relationship. However, right, the causal relationships that we care about are the ones that mesh with our affordances, right? If, you know, identifying a microscopic causal relationship is super, that's great, right? But unless you have really tiny tweezers, it's not very helpful, right?

• 12:25Speaker 02

  What you need to do is you need to identify the causal relationships that are present in the domain in which you are capable of acting. We care about the causal relationships at the macroscopic level because that is where we live. We live in the macro, at the macroscopic level. Most of our actions are at the. Now. One of the best things about humans is our ability to extend the domain of our affordances with technology, right? We have like nuclear power because what we did was we acquired the ability to take tweezers, you know, at that, at that scale and like, you know, make these things happen, right? We figured out how to take advantage of causal relationships at that level, not because

• 13:07Speaker 02

  we have those abilities, but we were able to create the tools that, that gave us access to that space. It all depends on what the problem it is that you're trying to solve. And the causal relationships that you always care about will be the ones that are related to the actions that you are capable of performing. Now that said, there's clearly a great advantage in understanding the microscopic causal relationships, right? If for no other reason, then that might lead to us discovering a way to expand our affordances into them, into, you know, into another aspect of the microscopic domain.

• 13:38Tim Scarfe?

  Is, is this just instrumental? You know, is, is this just something that, it's a little bit like we, we say that agents have intentions and representations and it's just a great way of understanding things, but for all intents and purposes, it's not, it's not actually how it works.

• 13:52Speaker 02

  Well, I, I think that that sentence ended on a rather definitive statement with which I don't think we could, I would agree, but the rest of it, is it all, you're asking like the, the, you know, the scientific anti-realist if it's all instrumental. So yeah, yeah, it's.

• 14:36Speaker 06

  Low energy, unfocused, foggy, you might be dehydrated. Gatorade Zero powders have been scientifically designed to help improve hydration balance in the body with a clinically proven electrolyte blend and no artificial colors or flavors. Great tasting hydration, no matter where you're going. Shop now online or at retailers nationwide. Gatorade hydrates better than water. Is it in you? Symptoms noted are signs of mild to moderate dehydration. Consult a healthcare professional if symptoms persist.

• 15:05Speaker 02

  It's all instrumental, right? I mean, we, you know, we, the, the things that we care about are the things that, that, you know, again, back to affordances, right? So, you know, we need to understand causal relationships at the scale that we can manipulate, right? That's what, that's what matters most, right? Because that allows us to have effective actions in the world in which we actually live. To the extent that we care about other scales, right? It's, it is because we simply wish to expand, you know, our domain of influence, right?

• 15:37Tim Scarfe?

  The mind is quite an interesting example. So let's say I wanted to move my hands and my, my mind wielded, so it's top-down causation. Now, I can't act in the world of my mind, but it seems, it seems macroscopically intelligible. You know, we think about our minds. So maybe the mind is a special case, I don't know.

• 15:56Speaker 02

  Well, the mind is a special case. I'll agree with that. I think of like downward causation from, well, I guess from an instrumentalist perspective, right? It's like, I'm not saying downward causation is the thing. I'm saying that downward causation is one of, is like how it all works. I would take it from more from the perspective that downward causation, if discovered, downward causation is what justified your macroscopic assumption. So what do I mean by that? I mean that like, suppose I'm in the following situation, I got a bunch of microscopic elements and they're all doing stuff and I'd like to draw a circle around them and call that a macroscopic object. Now, I am justified in doing so if that particular description of the macroscopic, at the macroscopic

• 16:41Speaker 02

  level, right, has the downward causation property, right? It's, it's, it is a way of sort of saying, oh, that was a good, you drew, that circle you drew, that was a good circle, right? Because it summarized the behavior of the system as a whole, right? In a way that rendered the microscopic behavior irrelevant to further, for further consideration.

• 17:04Tim Scarfe?

  Yes, I can think of some situations where we do this. I mean, we might identify an aspect of culture or a meme, and we might say that is responsible for violence or something like that.

• 17:15Speaker 02

  You still have to show that it has that property, right? And I think, you know, intentionality is a tough one, right? Because, you know, it's a variable that has a lot of explanatory power, but it's not, but it's not one that evolves. So when I think of a good macroscopic variable, it's one that I understand how it evolves over time. That's what makes it a good macroscopic. I can just write down a simple equation, and it says, you know, pressure, volume, temperature, right? They are going to do this over time. And like, taking any little microscopic measurement becomes like totally irrelevant, right? But what made it useful wasn't just that the microscopic measurements

• 17:59Speaker 02

  are irrelevant, right? It's that I had an equation that describes how it would have behaved, you know, that's also fairly accurate. So I have a nice, you know, relatively deterministic model that, you know, that is at the macroscopic level, right? And so when you talk about like intentionality, I think it's, you know, yes, it can be used as an explanatory variable, but it's only good to the extent that we understand how that intentionality changes over time, right? It's a long-term prediction. And this is why, like, you know, the jurisprudence example made me really uncomfortable, because it's sort of like saying, well, you know, what you're kind of doing is you're saying, this is a bad person, right? And I don't know how we would necessarily like identify like that intentionality, except in a very

• 18:43Speaker 02

  indirect way, right? That is that then they're stuck with, but then, you know, because it's only good as a macroscopic variable, if we can make predictions about how that variable changes over time, and we're not doing that. We're saying that you're stuck with it, right? And I just, that's why it sort of makes me a little uncomfortable.

• 18:59Tim Scarfe?

  I did actually notice that the active inference community has quite a ragtag, it's got very diverse.

• 19:07Speaker 02

  Yeah.

• 19:08Tim Scarfe?

  So in a way, you see people rubbing up against each other that you normally wouldn't. And that can create...

• 19:15Tim Scarfe?

  Arguments, I suspect.

• 19:16Speaker 02

  Yeah, well, I think, you know, this was Carl's influence. So what did Carl actually discover, right? He's got this link between information theory and, you know, and statistical physics that in some way gives you this sort of uniform mathematical framework that's widely applicable to a huge number of situations. It has a lot of sort of things that are baked into it. The how we think about the world is kind of like baked into it. And so it can be applied in a whole bunch of different areas. And Carl spent a lot of time, basically, evangelizing various different aspects of the scientific community. It's like, oh, look, you can apply this to epidemiology. You can apply this to the social sciences. You can apply this to physics. You can apply, you know, and just sort of, and, you know, wrote a series. This is one of the reasons I think he's so prolific is because he's basically, you

• 20:00Speaker 02

  know, written variations on the same paper, right? But just applied in different domains. And he did this, and this was intentional, right? Because he wanted to show that this is a non-uniformly applicable mathematical framework. And I think he's largely right about that. As a result, right, there's all these people from all these different communities that have been pulled into his sphere that think about the world very differently. And it makes for some very entertaining conversations at the pub.

• 20:30Tim Scarfe?

  Yes. Even in our Discord server, you know, we've got people thinking about it in terms of crypto, even in terms of Christianity, phenomenology, psychology. It's really interesting. But yeah, it's.

• 20:44Speaker 02

  But that's the beauty of constructing like a nearly uniformly applicable mathematical framework, right?

• 20:49Tim Scarfe?

  Yes, exactly.

• 20:49Speaker 02

  You get to suddenly, this is one of the things I love. I mean, this is what I love about the community, in fact, is that we now have a relatively common language to discuss a huge variety of different things.

• 21:00Tim Scarfe?

  Yeah.

• 21:01Speaker 02

  Now, of course, that means we often end up talking cross purposes, but that's half the fun, right? So I often ask people in the business, like, what changed? Like, what's, you know, what, you know, why did we have this like massive explosion in, you know, in AI development over the last several years? And I get three, there are three common responses and I agree with every single one of them. Autograd, right, the transformer, but why the transformer is something that I often disagree with with you about. Transformer architecture and just the amazing, the, the, the ability to scale things up in a manner that we haven't, haven't really seen before. I actually, the reason why I say transformer comes with an asterisk is because a lot of the

• 21:43Speaker 02

  things that transformers have, that, that people believe that the transformer enabled, I think really resulted more from scaling. And my, the point, you know, the point of evidence that I like the site is like Mamba. Mamba, which is a state, which is a traditional state space model. It's basically a common filter, but like on steroids. They scaled it way up and yet, and now it's, you know, got, they've, you know, Mistral has their very nice like coding agent and it works pretty darn well. Right. They got a lot of the same functionality with a completely diff, with a, you know, a completely different architecture simply by virtue of scaling. So transformers get a, get an asterisk. I think the biggest thing was autograd, right? And autograd turned the development of artificial intelligence from being something that was

• 22:26Speaker 02

  done by like carefully constructing your neural networks and then writing down your learning rules and going through all that painful process that was tick, tick for. And they turned it into an engineering problem. It made it possible to experiment with different architectures, different networks, different nonlinearities, different structures, different ways of like getting your memory in there and differently. And all this fun stuff that allowed people to just start trying things out in a way that we couldn't do it before. And then what did we did? We, we suddenly discovered, oh, it turns out backprop does work. I mean, when I was a young man, like backprop was considered a non-starter for two reasons, right? One is it is not brain-like, which is true, right? The brain does not use backprop. And the other one was the vanishing gradients. Oh, you'll never solve the vanishing gradients problem.

• 23:11Speaker 02

  And so, oh, it'll always be unstable. And yet nonetheless, once we turned it into an engineering problem, started playing around with tricks and hacks and certain kinds of nonlinearities and relus and this and that. We discovered that, oh no, in fact, like there are ways around this. You just, you know, we just, you know, weren't going to discover them by like playing with equations. We had to actually start, we turned it into an engineering problem. As soon as it got turned into an engineering problem, you know, that's what enabled the hyperscaling, which is what led to all of this, all of this, you know, these great developments over the last several years. What got lost in the mix though? Was the notion that that that that there's more to artificial intelligence than just like function approximation. We got really good function approximators, but that's not the

• 23:52Speaker 02

  only thing you need to develop like proper AI, right? You need models that are structured like the brain is structured. You need models that you need. You need models that are structured like how we conceive the world is structured. Certainly, if you want to have models that think the way we think. And that that got lost in the shuffle. And we're starting to see, you know, as as we're starting to see the limitations and the faults and flaws of of these approaches, and starting to see them not living up to the hype, which I think is like now it's standard that like, like AGI is no longer, I don't know if you read the other day, at least according to, you know, the experts in the field at the top

• 24:32Speaker 02

  of the best companies in the business, like AGI is no longer like a huge priority, right? And they're they're they're dialing back the rhetoric surrounding that. In part because I think that they've begun to realize that like, just function approximation isn't going to deliver. That was just hype, right? We do need to do something different. We do need to start getting, you know, bringing in what we know about how the brain works, right? If we're ever going to get to something that is a human like intelligence. And that was the starting point for us, you know, about a year or so ago, is that we were sort of like, yes, let's do the same thing for cognitive models. Like, let's talk about, let's take what we know about how the brain, the brain actually works. Let's take what

• 25:16Speaker 02

  we know about how people actually think about the world in which they live, and start building an artificial intelligence that thinks like we do by incorporating these principles. And this means, this means basically creating a, you know, a modeling and coding framework for building brain like models at scale. And that's like the critical element, because obviously, scaling was a was a big part of the solution. And right now, most of the work in the active inference space, as I'm sure you're aware, is not at scale. There's very little like active inference work that is active inference at scale. Most of the models are all Relatively small toy gridworldy type models.

• 25:57Speaker 02

  And part of the reason for that is that, you know, it is in fact difficult to scale Bayesian methods. Now that also has now begun to change, right? We now have a lot of great mathematical tools and a lot of great frameworks for approximating Bayesian inference. You'll never do it exactly, or approximating Bayesian inference, which I believe is how the brain works, right? Bayesian brain and all that. That allows us to build these kind of structured models that are structured both after the brain, how the brain is structured, and how the world that we live in is actually structured. Hence, this notion that what we need to get to the next layer of AGI,

• 26:40Speaker 02

  and I also don't like that term and don't intend to use it very often. What we need to get to the next level, right, is this framework that allows us to build the kinds of models that we know people actually use and just make them bigger and more sophisticated and so on. And then take advantage, like hyperscaling Bayesian inference is part of it, but also like it's, you know, constructing models of the world as it actually works. The way the world actually works, right, is what is, you know, provides us with the structure of our own thinking, right? The atomic elements of thought is how I like to phrase it,

• 27:24Speaker 02

  are models of the physical world in which we live. And the physical world in which we live is a world of macroscopic objects that, you know, that have specific relations and interact in certain ways that we understand, right? You know, looking around the room for a good example, right? You sit on a chair, right? That's an example of a relationship. It holds you up and all that fun stuff. And those are the kinds of, you know, that understanding of the physical world was necessary for us to have in order to survive. Dogs have it too, right? It's language isn't what make, you know, isn't all that special, right? Well, it's actually quite special. But those are the models that form, that understanding of the world in which we live

• 28:05Speaker 02

  is where we get our, the models that form the, the models that form the atomic elements of our thoughts, out of which we have composed more sophisticated models that have allowed us to do all this great systems engineering, build this great technology that we've got. So that's what we want to do, right? Is we want to, is we're focused on building cognitively inspired models that are based on our understanding, on the way the world in which we live actually works, because we believe intelligence must be embodied. Building a framework for putting those models together and experimenting with them at scale, all in an approximately Bayesian way, because we believe that's how the brain works. It's not just about putting your AI into a robot.

• 28:47Speaker 02

  It's about giving the robot a model of the world that is like our model of the world. A model that is object-centered, it's dynamic, it's largely causal, right? It's, you know, that's the big difference. And I think that the sort of sparse structured models is another sort of key differentiating component. Like, when you think about how like a transformer and LLM work, a transformer takes every word in the document and says, now, how does this word relate to every other word? And it does it many, many, many, many times, right? It's a, you know, it's very much, it's the same thing with like your generative vision language action models. They operate in pixel space. They are microscopic models.

• 29:32Speaker 02

  Now, yes, do they have an implicit notion of sort of macroscopic? Yes, they must, because they work, right? But it's implicit, and it's not implemented with the kind of sparse structure that actually exists in the real world and in our conceptualization of it. And that's the thing that we are, that we are saying, no, no, look, like, if we want an AI that thinks like us, right, then we are going to build models that are structured, both like the real world is structured. They have this sparse causal macroscopic structure to it. And so should our models, and so should, and the only way to do that is not just to like put a robot in the real world, but to put a robot with a model that is structured in that fashion into the real world.

• 30:11Tim Scarfe?

  No one's using the X LSTM. Not many people are using Mamba, because why not? All you need to do is just scale the transformer as much as possible. So, you know, many people just really think you just magically get these things for free, right?

• 30:25Speaker 02

  So I think you could argue that with enough data... That's the right kind of data. One of these like really big, super scaled models will obtain an implicit representation of the world that is more or less correct. Now, having an implicit representation is great if your only goal is to just represent the world. If your only goal is to just predict what's going to happen. But it turns out, people do something which is very different. People are creative. People can solve novel problems. They can't, it's not just about mining old problems and figuring out where I can move some words around and get a, and get an answer that looks more or less right. Right?

• 31:08Speaker 02

  We actually are capable of creating, we're capable of inventing new things. The way that we invent, I think, is exemplified by, via like systems engineering. Right? How does systems engineering work? Well, I, you know, I know how, you know, I know, I know how an airfoil works to create lift, I know how a jet engine works to create thrust, right? And I can take those two bits of information to invent something brand new, which is an airplane, right? That kind of systems engineering was predicated upon having this sort of model of the world that was relational, right? Here's the wing, I can put a jet on, you know, I can like, I don't know, you don't staple it on, I'm sure you use rivets or something, right?

• 31:50Speaker 02

  I know how to put things together, I know how to construct new relationships and new objects. An AI that, that is designed for systems, that is designed to do systems engineering, will have a object-centered or system-centered understanding of the world and will know how those, all of the objects relate so that it can sort of start experimenting with different ways to combine them. That's absolutely, you know, without that, the only thing you will ever be able to do, right, is just retool old solutions for new purposes. And it won't even, and even that is, I think, is a generous interpretation of what a purely predictive model is gonna do, right? So this is how I like to think about, like, you know, the, the principal advantage of

• 32:35Speaker 02

  taking this object-centered approach, right, is that it enables systems engineering.

• 32:41Tim Scarfe?

  What is a grounded world model?

• 32:43Speaker 02

  That is a, so, so I, I feel like that's a trick question.

• 32:49Speaker 02

  I actually had this conversation with one of my friends Max and co-conspirators the other day. In some sense, every model is grounded. It's grounded in the data that it was given. Now, okay, so that's like a true statement. It's like, okay, but that's not what we want. And when we often use the word like a grounded world model, it's we say that it's grounded in something. And that something is not just the data that it saw. So example, vision language models. A vision language model is like is a way of grounding the visual model in the linguistic space. And this is the approach that we're taking. This is what Langchain does, right?

• 33:33Speaker 02

  It's all about taking, you know, models and everything becomes a blank language model, right? You know, vision, you know, whatever, everything becomes. And what you're when you do that, what you're doing is you're saying that that you're grounding all of your models in like a common linguistic space so that they can communicate with one another, right? Via language. Now, why did we choose language? Well, we chose language because like honestly, I think it's because we wanted models that we could talk to, right? We wanted, we wanted a model that like, you know, it was really all about the making the interface convenient for us and which is great. That's totally something you want. But it begs the question, what's the right domain in which to ground your models?

• 34:17Speaker 02

  Now, I like grounding models. Like, so we also use the phrase like, you know, one of those ground truth. And of course, ground truth is the thing you made up and a priori said was ground truth, right? So what's ground truth? What is the, what is the right, you know, domain in which to ground models in order to get them to think like we do? That's the relevant question. And so my, my view is, is that if, you know, again, if you want AI that thinks like we do, you need to have it grounded in the same domain in which we are grounded. And we are grounded in this domain, right? This is why the embodied bit is such an important thing. We want models that are grounded in the physical world in which we evolved.

• 35:01Speaker 02

  And the reason for this is because that is the world that provides us with these atomic elements of thought. A single cell, like, lives in a soup, right? And it has, you know, and it, it, it's, you know, whatever model it has of the world to the extent that it has one, or it behaves as if it has one, that model is, is the model of its environment, right? If it didn't understand the environment in which it lived to some extent, right, then it wouldn't, it wouldn't be able to continue to exist and function in that environment. So you can sort of say that a cell has a model that's grounded in chemistry, right? Of the chemistry of the soup in which it lives. You know, when we talk about, like, that, that is a prerequisite for its survival. Now we talk about like mammals and bigger animals and things that live in the macroscopic

• 35:46Speaker 02

  world that includes other animals, right? And, you know, and all that stuff. So what's that model? What's, what's, what's the world, the, the, you know, well, at the very least, we can say that whatever models we have, a significant subset of them are grounded in that world, right? And that world we know has properties that we can understand. It is object-centered, it's relational, it's all this, you know, all this stuff. And so the ground, the, the, the, the grounded bit is more about like properly grounded, grounded in the domain in which, in which we are grounded as a route to creating, you know, AI, you know, and AI models that in fact think like we think, right?

• 36:29Speaker 02

  That's the grounding that we, that we're particularly focused on. If you had to choose the domain in which to ground your models, what would you choose? Right? I don't think language is the right one. Language is an incredibly poor description of both our thought processes and reality. I tell the story all the time, right? So. You ever, you ask any cognitive scientist or psychologist who's done some experimental work with humans, right, the, you know, you, you put them in a, in a chair, you make them do some tasks, you carefully monitor their behavior, you look at what they did, right? And then you have a nice way of, and then you, you know, that informs your theory of that behavior or however that works. Then you, and you know, and if you do the experiment well, you have a very good model of how they made whatever decisions they made throughout the course experiment.

• 37:12Speaker 02

  And then you go back and you ask them, what, why did you do what you did? And they give you an explanation. It sounds totally reasonable. It also is completely inconsistent with an accurate model of their behavior. Self-report is the least reliable form of data, right? That one gets out of a cognitive or psychological experiment. And so we don't want to rely on that. We don't want to ground our models in what we know is an unreliable representation, both of the world and of our thought processes, right? We want to ground it in something that's a good model of our world. And that's why we, we've, we've chosen to focus on like, you know, models that are grounded in the domain of macroscopic physics as opposed to language.

• 37:50Tim Scarfe?

  Can you speak a little bit more to the, the limitations with current active inference?

• 37:55Speaker 02

  A nearly uniformly applicable information theoretic for describing objects and agents, right? It's, it really is inspired by statistical physics and its links to information theory. And when you take those two mathematical structures, throw in a little like Markov blankety thing, so you can talk about macroscopic objects, you kind of have a very generic, widely applicable mathematical framework that you can throw at many problems. And a lot of what has gone on in the active inference community over much of the last 20 years has been demonstrating that it's, it's, it's like uniformly applicable. So there's been a lot of breadth and not a lot of depth, right? And part of, you know, and I, and I think that, you know, of course, like,

• 38:39Speaker 02

  That's appropriate, like, given, you know, if you really want to make the argument that everyone should be using this. You show, see, in this domain, it works on your toy examples. But the people doing that, right? It's kind of, you know, the active inference community has this habit of showing like, like, see, like, oh, this basically, like, I can handle this psychological phenomenon. I can model this cognitive phenomenon. Oh, and look, like, it's a good post hoc description of this neural networks behavior and things like that, right? They're, they've been showing that, but they've, they've never really sat down and, and, and like, tried to tackle any really big, really hard problem. Because the emphasis has been on evangelism. You couple that with the fact that there is this strong bias within the active inference community

• 39:23Speaker 02

  towards being as Bayesian as possible. And so, of course, they also like shun the really hard problems because Bayesian inference, you know, is, has been historically challenging to scale. There have been a lot of developments over the last few years that, you know, that have come, you know, out of the machine learning community as well, you know, but mostly out of the Bayesian machine learning community that, that have really made it possible to start scaling Bayesian inference in ways that we, we, we really weren't able to do it before. And you couple that with a desire, you know, to sort of stop the evangelizing and start solving really hard problems with these methods. And you've got a way to prove that active inference really can live up to its promises.

• 40:03Tim Scarfe?

  Yeah, it was a similar thing with constraint satisfaction. You know, but in the 1970s, there was that Light Hill report and people said symbolic AI will never work. And they wrote it off. Apparently, just that there are all of these empirical methods that have been discovered in the last 20 years that just make it massively more scalable and tractable. And is it the same thing here, right? Are there some specific techniques that have dramatically improved the tractability of active inference?

• 40:25Speaker 02

  Well, I would just sort of, I would lump it all into the Bayesian inference category. There have been a number of developments over the last, I would say, eight. Eight, yeah, eight years or so that have made Bayesian inference significantly more tractable than it used to be. Some of it had to do with, you know, work in the sort of Gaussian process space. My, my, my current favorite trick is, is, you know, normalizing flows, which is a great way of ensuring that you have like access to sophisticated likelihoods, but nonetheless result in tractable probability distributions.

• 41:03Speaker 02

  There's the work, you know, I mean, I've been using,

• 41:08Speaker 02

  I've been using like natural gradient methods for a very long time, which allow you to like massively speed up gradient inference and in some situations completely eliminate the need to do gradient inference and instead like, you know, do coordinate descent and allowing you to take massive jumps in parameter space and not actually lose the, the ability to do learning of sophisticated model in a sophisticated modeling scenario. I also like the fact that like the, the natural gradient stuff has been getting some great acronyms recently, like Bayesian online natural gradient for BONG for short. I just think these guys, these guys get me every time. I, I wish I was that clever, honestly, but like it's, it's, but there've been a lot of developments in that space as well. You know, making, you know, in addition, additionally,

• 41:53Speaker 02

  there's been a lot of developments in like rapid sampling methods, conditional sampling methods, constrained methods like that, that, that, that, that have really improved things. And I think that like one of the problems again with the active insurance community, you know, historically that, that I think is now starting to change has been a hesitance to use these sort of certain approximate methods. There's been this, this focus on like straight up old school message passing. And, you know, as soon as you sort of, you know, you know, if you relax the desire to be as Bayesian as possible, it opens up a lot more possibilities for, for scaling this stuff up.

• 42:29Tim Scarfe?

  When we're now talking about agents that are, you know, interacting with the world around them and that, that still presumably needs a lot of data.

• 42:39Speaker 02

  So we've got a couple of tricks. One of the nice things about taking an explicitly object-centered approach is that you can, you don't have to train all of your models. You don't have to train just one model at a time. This is my favorite trick, and I think that, you know, this is one of those things I think we're going to be seeing a lot more of in the near future. You know, so if you want to train, you know, a vision model to understand like YouTube videos or something, you know, really complicated like that, you basically take one big model and you train it on a shit ton of data, right? You just keep training, keep training, and eventually it sort of gains this implicit, and it does get an implicit sort of, sort of object-centered understanding.

• 43:20Speaker 02

  Another way to go, right, is to, is to, you know, train objects in specific domains that are, so these are smaller data sets, like I'm only going to worry about like the Zillow problem, like the inside of people's houses, right? And that's going to have a much smaller set of objects, right, that it has to, that it has to learn an implicit distribution over. And you can do this with one big neural network, right? You can train, and, you know, there's a really great, like, you know, Gaussian splatting paper where they trained a massive neural network that is able to like, you know, to make predictions about what's going on inside people's houses and some nice language models. But obviously, it has an understanding that's limited to a house and the objects that are inside a house.

• 44:03Speaker 02

  If you have an explicitly object-centered model, then you end up not just with one model that understands a house, you end up with one model that's actually thousands and thousands of little models, each of which, right, you know, sort of explains like a single object or object class within the house, right? So you got like a book model, so all books come in different shapes and colors, right? But there's just one like book model. And the beauty of doing this is that, is that that book model... You have to be a little clever about the how you structure the interactions between these things. But if you're a little bit clever about how you describe the relationships between objects

• 44:44Speaker 02

  within this modeling framework, you gain the ability to train a model just on the insides of houses, a model like just on, you know, just on like, you know, parks and park benches, and take the objects that were discovered in this space and the objects that were discovered in this space, and put them into a combined environment that has objects of both of those kinds, and it still works. Right? That's the advantage of the of taking an object-centered approach, or a what I like to refer to as the lots of little models approach.

• 45:11Tim Scarfe?

  Some of these things are a little bit weird, you know, some cultures have, you know, maybe one culture doesn't have the notion of time, and some cultures might see two objects as one. So is there a potential problem here that there's some ambiguity that we need to overcome?

• 45:27Speaker 02

  I'm not going to say that there's not the potential problem for ambiguity that we need to overcome. What I will say instead is that is that the additional constraint that we're imposing, it's not just about objects, it's also about their relationships. Now, think about physics. This is why we this is why the physics discovery stuff is such a big part of it, right? In physics, in particular, like, you know, Newtonian mechanics, you can, you know, let's pretend we're living in a world of rigid bodies, right? So all I need to worry about is like weight and shape of things. And that defines a particular object type. But I also need to know how they interact. And so in in Newtonian mechanics, we have like, you know, what we can do is we can take these objects, we can watch them like bouncing off of each other and doing all these sorts of things.

• 46:11Speaker 02

  And we can quickly infer that like, oh, like their interactions are all governed by a in a single language, which is the language of forces and force vectors, right?

• 46:22Speaker 02

  That that language of interaction, right, is really what makes it work, right? Otherwise, we just have like, you know, pictures of things. That's all we would have got. What you're empirically discovering is sort of a generalized notion of forces that describe the relationships between things. And you can the constraint that you place in order to To avoid the problem of things being too brittle, right, is that well, they all have to use the same class of forces together in order to interact. We're stuck with that. But by being flexible about our definition of what a force is, and the having the ability to discover new kinds of forces, not just like literal force vectors, right?

• 47:04Speaker 02

  Gives us the ability to sort of generalize without be without becoming too brittle.

• 47:10Tim Scarfe?

  You're talking to this interaction dynamics. So there's a graph of interactions, which might possibly represent affordances in the macroscopic domain. And by doing analysis on the interaction graph, you and and sort of simplifying the the analysis as much as possible, you get a principled way to partition the world up.

• 47:29Speaker 02

  That's right. And so so there's there there it's it's a it. It's all about having interactions and interaction classes. So it's not like there's not just one adjacency matrix, right? There's an adjacency matrix that also specify there's there's one for every type of interaction that that that's possible. That's what gives you the additional flexibility. The other thing that gives you the additional flexibility is being is being a little bit Bayesian about things, right? It may very well have been that all of your observations of this object, you know, when it was in a house, were like really simple. It was all just it sits on a shelf, right? And so what do you know? Well, what you know is that that object sits on a shelf. But you have to be, you know, which is one kind of interaction, right? That's just the, you know,

• 48:14Speaker 02

  it has a force pushing down, there's a force pushing up. You don't know anything about like the weight on you have night, but but if you keep error bars about that, if you keep error bars about the other kinds of interactions that you have seen, but are agnostic, right, about the specific deals for this particular object, it gives you the flexibility to say, well, I'm going to put it in this environment. I can make some predictions about how it's going to behave, right? But if I throw a bowling ball at it. I'm gonna be making some, you know, assumptions about how it might behave. But, but once the bowling ball hits it, right, I might have to revise those assumptions. This is the other critical elements of the approach we're taking, which is continue, which is you have to have some kind of continual learning element. This is something that really doesn't exist in contemporary AI, right?

• 48:58Speaker 02

  And, you know, when you build your big model, you've spent millions of dollars training it, and then you're done. Right, yes, someone else can come along and fine tune it a bit for a particular task, right, which is great. But at the end of the day, when you're at the deployment phase, you turn learning off. Whereas in this approach, we're saying, no, no, you've, one of the things that's critical, a critical aspect of the way we think about the world, the way we learn about the world, is that it's continual and it's interactivist, right? So there's, you know, and that needs to be true of the objects that we're discovering as well. We've learned classes of interactions, but just because we haven't seen a particular class of interactions previously, doesn't mean we say the others never happen, right? We still allow for that possibility, right?

• 49:43Speaker 02

  And then do continual learning, the quick with, with rapid updates when we see something happen, we see, you know, a new interaction. Now, the what makes that work, right, is the fact that you've specified that there are certain set of kinds of interactions, some of which you previously observed, some of which you still don't know about and might observe soon, and then you could update your posterior beliefs about whether or not that object interacts in that way.

• 50:06Tim Scarfe?

  What would the architecture of such a system look like? I'm imagining it'll be distributed, right? So, you know, have all of these, have all these different agents. And, and then we have the consistency problem, because maybe this agent has empirically learned that these two things are a book, but the agent over there, you know, just thinks this one thing is a book. And then there's, how many objects are there? Would it become intractable, you know, like realistically, I don't know what you're...

• 50:25Speaker 02

  So, so from a simulation perspective, the way that this gets simulated is remarkably the way, remarkably like the way a video game engine simulates the world. The only difference being is that this abstract notion of forces. So, how does a video game represent the world? Well, you have all these assets, right? And each asset is basically a shape. Maybe a texture, color, something like a fork is an asset, right? Or a little like three-legged stool is an asset. And it has a bunch of properties, but it's basically, you know, that, you know, that are associated with its shape, color, mass, all of this stuff. And then it has a set of interaction rules, which are like Newtonian forces, force vectors.

• 51:07Speaker 02

  Then you've got other things like water and sand that have like special rules for them. Because if you just try to, because otherwise, you know, you need a macroscopic rule to describe them. Otherwise, the compute would be insane and stuff like that. So it's very similar to that, right? We, you know, what we do, when you take this lots of little models approach, what you end up with is the moral equivalent of a giant list of video game assets. Right? And then when it goes to modeling a particular environment, right? When you find the agent that you're talking about that has this lots of little models model in its head, what it does is, is it, it sort of looks at the scene and says, oh, okay, I need to worry about these 10,000 little models right now. And that's it. I don't need the rest of it. Right? And then it just sort of operates in that space,

• 51:52Speaker 02

  running something that looks a lot like a video game simulation. Right? So it's, it's that sparsity is what makes, is what makes this little, this lots of little models approach, right? You may have a million little models, but at any given time, you only need a tiny fraction of them. And you just instantiate those.

• 52:10Tim Scarfe?

  The thought occurs though, that in a game engine, all of these particles are, they're in, they're in the engine. I can say, what, what are the forces between these two parts?

• 52:18Speaker 02

  Yeah, it's called cheating.

• 52:20Tim Scarfe?

  Well, yeah, because, you know, when you deploy an agent in the real world, you can't just ask, well, what's the force vector between Jeff and the light?

• 52:27Speaker 02

  That's right. Yeah, you have to learn those. You know, does this model, you know, if you take a video game engine as ground truth, are we capable of discovering the video game, the assets and their properties that were in that, in that, in that game engine?

• 52:43Tim Scarfe?

  So what would your input be? Would it just be the pixels?

• 52:46Speaker 02

  Yeah, why not? Make it hard. Like if, you know, it would be cheating to sort of like, start out with something that already segments the image for you. Right? If you can't solve the hard problem from, you know, from, from the bottom up, then like. It's not a hard problem. Why'd you do it?

• 53:03Tim Scarfe?

  If I understand correctly, a successful implementation of the technology you're talking about would be, let's start with a game engine and we almost treat the AI like a black box. So it has input, you know, like I can move left, I can move right, pan up, down, I can interact with objects. And then maybe there's some kind of a score function, I'm not sure. But, you know, it can learn inside the game engine and it will build up this internal model library that represents things in the world in the game engine. And if it's learned a sparse, robust model library, you could in principle take the same learned model and apply it to a robot in the real world and it would generalize.

• 53:39Speaker 02

  That's the idea. And that's the problem that we're trying to... This is like one of the critical missing elements in the robotics space. Is that, you know, if you, you know, training models in simulated environments does not translate really very well to real world environments. This could have, this could be a result of a situation where the, you know, the simulated environment is just too impoverished. But it could also be a result of a situation where the artificial environment just isn't actually a very accurate representation of the real world. Right? And I think it's, I think it's largely the latter, right? Is that, is that these, you know, these, I mean, also coupled with the fact that the robot, that the artificial agent's internal model looks nothing, is not structured

• 54:23Speaker 02

  like the world that it actually is being trained to function in. Those are the two big, those are the two biggest problems. But it could also be the case, you know, what do you need in order to address those? Well, one is you need a good model for the robot's brain that has the structure of the world in which it lives. The other thing is, is you need a mapping from real world data to simulated data. And right now, what we're typically using is video game engines. Now, video game engines are great. I know I certainly enjoy them on a 10-hour-a-week basis. The, the problem with them though is that they weren't trained to be realistic physics. Right? Most of them were designed to be plausible. They were designed to look good to the user.

• 55:04Speaker 02

  Part of this has to do, you know, and there's a lot of like tricks and hacks and things that are thrown in to deal with the fact that the equations of Newtonian mechanics are very stiff, right? When collisions happen, if you're just a little bit wrong about that, you know, then things can, you know, then weird stuff can happen and and non physically realistic things can occur. So if if you if you had the ability to construct an environment that had good enough physics, that accurately represented the real world and trained your robots in that domain, where they have these models in their heads, so they're actually capable of learning, you know, the quote unquote ground truth that you've implemented in the in the simulated world,

• 55:46Speaker 02

  then I I I believe that they will generalize better to functioning in the real world. And this is this is absolutely critical, I think, for robotics going forward, if for no other reason than right now. I mean, like large language models, all these self-supervised models, the way that we're currently training robots to like put your put your groceries away and things like that, is all by training them to mimic human behavior, right? It's expert trajectory learning. They're not really learning the physics of their environment. They're learning to mimic human behavior without like crushing the eggs, right? And so with, you know, you know, if you want them to be able to generalize across domains, across tasks, you need to get rid of reliance on expert trajectory learning.

• 56:28Speaker 02

  And so that's and that only happens when you move to something that is explicitly model-based, with a model that accurately represents the world in which they live.

• 56:37Tim Scarfe?

  Once you've got a core set of models that work in the world, is that the value of the AI?

• 56:42Speaker 02

  Yeah, so once you have a core set, then then you have the ability to like deploy your agent out there in the real world, and it can handle situations that it couldn't previously handle. One of my one of my co-conspirators likes to talk about the cat in a warehouse problem, right? So what do we have? So now we've we've we've we've got an AI agent. That has been trained to like manage a warehouse, right? And so it understands things like forklifts and boxes and workers, hopefully, you know, and all and it knows how to do it. And then one day, one day, something comes along that's never seen before. It's a cat, right? Cats don't belong to a warehouse. The cat comes along, right? And so the the model has no eye, has never seen a cat before, right? Because that's the environment in which it was trained.

• 57:27Speaker 02

  This is the one of the beauties of this approach. So it's, the cat comes in the warehouse and it's, what the hell is this? And like, it's, you know, it's screwing with my system. And because we're taking this sort of like, you know, free energy based approach, right? One of the critical elements is that is is tracking surprisal. So when a cat comes along, doesn't know what a cat is, the surprisal signal goes crazy. And then it says, okay, stop, right? Don't run over the cat, right? Let's figure out what's going on. And what it can do is it can take a picture of the cat and it can fire it off to a server somewhere that has a huge bank of models and has been pre-trained on on model selection to to a small extent. And so and it says, what the hell is this?

• 58:09Speaker 02

  And then the big bank of models says, oh, I think it's, you know, here's like seven or eight things it could possibly be. And it's different kinds of cats. Maybe there's a dog thrown in, whatever. And then it ports those little models over to the to the to the warehouse model. And then it sort of does some proper hypothesis testing, watches the cat behave for a little bit. Ah, it's a cat. Puts the other models, sends the other models back because it doesn't need them anymore, right? It's figured out that this is what it is. And now it's incorporated and understanding the cat into the system. This is the beauty of taking an explicit, this is another beauty of taking an explicitly object centered approach. It gives the model the ability to be to to to to know what it doesn't know. That comes from the active inference component.

• 58:52Speaker 02

  Know what it doesn't know, when it doesn't know it, it can go phone a friend. That's another way to describe it. The friend will respond by saying, oh, it's a cat. And then and it can take the model, the cat, incorporate it into its warehouse model, right? And now it understands that. This is really great from a compute. There's a huge compute advantage to this, right? If we had started with one big model that already knew what all a cat was. Think how many parameters that would have. It'd be huge. This model is very frugal in a sense that it only knows, it only needs to know two things. What it needs to know about the environment in which it, in which it exists, right? And when it sees something it doesn't know, and then it can just go pull. So that's the idea is that you have this massive bank of models. But when you instantiate a particular, if for a particular use case, you don't need them all.

• 59:36Tim Scarfe?

  Yeah.

• 59:36Speaker 02

  Right? You just need the ones that are relevant to that environment. But you could, but these, these models are continuously tracking surprise or uncertainty. And when it sees something it doesn't know before, it's smart enough to say, I don't know what that is.

• 59:49Tim Scarfe?

  How and when should deep learning be combined with this? I mean, my, my naive perception of Bayesian inference is right now, if you have a photograph from a camera and it's like, you know, 300 pixels squared or something, you know, that, that would be a challenge for Bayesian inference. Something, and could you, could you just use a, like a vision language transformer or something? And use that as part of the Bayesian framework? Or could you even use deep learning models as a way of bootstrapping the knowledge acquisition in the, in the Bayesian framework?

• 1:00:17Speaker 02

  So the reason why I mentioned normalizing flows is because that's technically, that's a deep learning tool. It just happens to be a deep learning tool for which the output that, that takes in an image and turns it into something that is easy to deal with from a probabilistic reasoning perspective. Are we going to use deep learning tools? Yes, the ones that are fit to purpose, for sure. Um, and you know, that's, that's, that's a, that's a great example of, of, of one where we're taking sort of like, oh, well, like, why wouldn't we use this if it's compatible with our framework?

• 1:00:48Tim Scarfe?

  Many folks in the audience won't know what a normalizing flow is. Can you just give us a quick update on that?

• 1:00:53Speaker 02

  Well, so, okay, so, well, we all, we, we've got a pretty good handle on how diffusion models work these days, right? It's, you know, you take your image, you just add a bunch of noise to it, make it Gaussian, and then you learn an inverse transformation. It's the same thing, right? What you're doing is you're learning a mapping from a probability distribution that is easy to deal with, like a Gaussian distribution. And you're learning a mapping from that distribution onto the thing you actually are observing, the thing you care about. So in this case, it could be an image. In fact, I actually don't think we should call them diffusion models. It's a normalizing flow. The diffusion is, it's, it should be referred to as a diffusion training protocol for a normalizing flow.

• 1:01:33Speaker 02

  So to some extent, we will be using some of those tricks as well. You could say, yeah, we're going to use diffusion models. It's going to make me roll my eyes and say that.

• 1:01:40Tim Scarfe?

  Jeff, what is your approach to alignment?

• 1:01:43Speaker 02

  Well, I typically like to talk to people about their beliefs and values and figure out what it is, how it is that they came to form them and then try to convince them to adopt my values. The beliefs that these systems have, right? The belief that our artificial, artificial systems have are not the same as our beliefs. And they're, and they're, the reward functions that we specify for these artificial agents are definitely not the same as our reward functions. Now, there's a few exceptions, it's like, like, like Go, chess, right? Any game where you either win or you lose, the reward function is obvious.

• 1:02:20Speaker 02

  But in general, like, in complicated situations, reward functions, it's not so obvious what the reward function should actually be. You know, I know that there's this definite belief that like reward is all you need and there's some truth to that, but the question is, well, where'd your reward function come from? Now, from a philosophical perspective, you know, there, there is no normative solution to the problem of reward function selection. I always say barring divine intervention. And that, which is just another fancy way of saying that like, your values and my values might be different and it's really difficult to say whose are better, right? From a practical perspective, right?

• 1:03:02Speaker 02

  A situation that I like to point out is like, if you're talking about self-driving cars, obviously you'd like to penalize your self-driving car if it drives over a squirrel, right? But if it had to choose between a squirrel or a cat, you probably, most people would want it to choose the squirrel. And the way you would do that in RL model is you say minus 10 points for a squirrel, minus 50 for a cat. Where do those numbers come from? It's completely ambiguous, right? It's relatively arbitrary. They're kind of sort of made up. And so relying on arbitrarily selected reward functions like seems like a terrible idea. We also know that like things can go horribly wrong. And I know everyone's sick of this example, but like, you know, you all, you know, when

• 1:03:43Speaker 02

  you rely on reward, you're effectively like, you know, making wishes from like a malevolent genie. Or, you know, you run the risk of saying, hey, hey, Skynet, end world hunger. And it's like, no problem, kill all humans, right? Those are, if you don't specify your reward functions very carefully, you can get very degenerate behavior. So the goal of alignment in an RL setting is to get, is to get, it would be to somehow get my reward function, or perhaps humanity's collective reward function, right? Into the AI agent. This is really, really, really hard. It's really, really, really hard because measuring reward functions is really, really, really challenging. The approach that we're taking is we're saying, well, so how do people actually do this?

• 1:04:27Speaker 02

  How do, how do we as humans construct alignment? Well, the first thing we do is we try to figure out what other people's reward functions are. The problem of reward function identification is conflated by the fact that people have different beliefs. Action, which is what we can observe other people doing, is a combination of their beliefs, right? And their, and their, and their reward function or their values. And so just sort of like taking those, you know, so the problem, of course, is that you only observe people's actions. Like, there's a difference of opinion about what to do, right? And so you want to figure out why. And it could be because your beliefs are different, or it could be because your values are different. The only, but it's ambiguous. You can't tell. It's mathematically, it's not even possible to separate these two.

• 1:05:11Speaker 02

  Belief and value are fundamentally conflated when all you observe is action or decision.

• 1:05:17Speaker 02

  The way that we solve this problem as people is we talk about our beliefs. Right? I tried to, I ask you, well, why do you think this is the action? And then you tell me like, oh, well, it's because, you know, this fact, this fact and this fact suggests that if I do this, then this will happen. Right? And then I can go in and say, ah, I see. So maybe the reason for the disagreement in our beliefs or in our, in our decision. Right? Is because you're not aware of this fact. And I'd forgotten about this fact. And so what we do is say, well, let's incorporate all of these things together and then sort of see. And then, and then you would still say like, well, I still think we should do X. And I'm like, no, it's still definitely Y. And we continue this conversation until we both, until each of us has a very reasonable model of the belief formation mechanism that the other person has.

• 1:06:03Speaker 02

  At which point the only cause for disagreement is a disagreement about the reward function.

• 1:06:10Tim Scarfe?

  AI systems are completely illegible. And that's almost a good thing. Because if we actually understood how flawed they were, they would be banned. Right?

• 1:06:20Speaker 02

  Well, they're amoral. It's, we have no idea how to put morality into them. The, the, the smart, safe thing to do is, is, is to remove decision making from their capabilities and to simply use them as oracles or prediction engines. Right? And then we can just say, hey, like, you know, what would happen if I did X, Y and Z? And then it just sort of tells you, well, this is the, this is the ultimate outcome. And then we're like, oh, okay. Well, then maybe A, B and C were better choices. Right? And things like that. That eliminates them from the consider, from the consider, from, from participating in the, in the, you know, in the actions, it's all right, that, that prevents them from using their reward function. Right? And you can get that just by like training them to just, just do good prediction. That's totally great.

• 1:07:03Speaker 02

  But that doesn't give us the kind of automation that we really want. Right? What we really want are decision, you know, artificial agents that are decision makers that can act on our behalf. And so it's either going to be human in the loop, or it's going to be something like what I propose, where we figure out how to like solve the alignment problem in that fashion.

• 1:07:24Tim Scarfe?

  But Jeff, you're an old school cognitive guy. So for someone like you, would you always think that in the absence or in the lieu of explicit cognitive models, that we would never be able to say that these things actually had beliefs or intentions?

• 1:07:39Speaker 02

  I think that what allows us to currently say that they don't have beliefs or intentions actually stems a lot from our knowledge of how they actually work. Right? I, for example, have no problem concluding that you have beliefs and intentions, though it may very well be that that conclusion is drawn from the fact that I really don't know how you work. I have an intuitive feel for it. I assume you work the way I work. I have beliefs and intentions. You know, that's my perspective of myself. And so I conclude the same about you. It's kind of like emergence. Emergence is such a funny concept, right? Is that there's this whole class, this whole like branch of the emergence literature that defines an emergent phenomenon as anything that I didn't predict, right? Which is a remarkably anthropocentric and I would argue ignorance based definition of emergence. And I don't like it.

• 1:08:25Speaker 02

  That's for those reasons. The same sort of thing goes with, you know, I think the sort of converse of that is what's going on here. It's like we know that these algorithms do not have like the capability to do anything other than predict. And so we don't believe they have intentions.

• 1:08:45Tim Scarfe?

  But something like strong emergence usually means like, you know, causal irreducibility.

• 1:08:49Speaker 02

  Whatever definition of emergence you end up going with, it shouldn't be ignorance based. It shouldn't be based on like, oh, well, there's, you know. And so, you know, that includes explanations of emergence

• 1:09:01Speaker 02

  that involves things like, well, the only way I could have discovered this was by simulating it. Therefore, it is an emergent phenomenon. I don't even like that. I don't even like that. I am more sympathetic to that. But I prefer definitions of emergence that are sort of more pragmatic, right? That are sort of like, oh, no, an emergent phenomenon. This is why I like downward causation as like a fundamental feature of emergence. of behavior, mostly because downward causation, it's not only a nice explanation of when you can, you know, a nice sort of fairly rigorous definition of when you can say a phenomenon is emergent. It also comes with a practical tool. It tells you you don't need to model the microscopic phenomenon.

• 1:09:39Tim Scarfe?

  Last time we spoke about linear in Game of Life, didn't we?

• 1:09:41Speaker 02

  Yeah, that was, oh, that's, I'm still playing with that, by the way.

• 1:09:44Tim Scarfe?

  Are you?

• 1:09:44Speaker 02

  Some of the Lenias, one of my, one of my favorite Lenia simulations, this is not particle Lenia, this is the traditional Lenia. And so what they do is they have a field and they're obstruction, and they're squares and circles and things like that. And then they have these little creatures that are sort of like a little amoeba-like swimmers, like they've got like fins in the back and everything. And they swim and they'll hit one of these obstructions, which will cause them to deform and kind of look like, oh, it's gonna die, that's so sad. And then it reforms and becomes itself again. And so we thought of this as like a really nice abstract environment in which to, to test the, you know, some of the properties of the physics discovery algorithm. Because one of the nice things about the approach we've taken is, is that as the thing, as a little swimmer goes and hits, hits something, it's possible that it loses its identity

• 1:10:28Speaker 02

  when it like deforms into something new and then reforms into itself. And we wanted to see if the approach we've taken captures that. And it more or less does, right? It hits the obstruction, right? The, you know, it, it, it changes its identity into an object of a different type, and then reforms and comes out the other side and then regains its identity back.

• 1:10:49Tim Scarfe?

  Fascinating. Quick aside, you know, we spoke last time about Alex Mordvinserv and he had this, you know, convolutional cellular automata with the Gecko, you know, the self-healing Gecko. And he has now written a new paper with his friends at Google, and it's using logic gates. So it's like an emergentist logic gate thing that draws a Google logo. And I haven't read it in detail, but it looks amazing. So definitely look at that. And now you're taking your system and you're applying it in like something like a game of life, basically. But you still expect it to work.

• 1:11:21Speaker 02

  Yeah, well, so I, there are forces in, in, in lenia, right? It's there, you know, there's the, the, the, the rule that causes the pixels to change, right, has a few properties, right? It's, it's radially symmetric.

• 1:11:37Tim Scarfe?

  Yeah.

• 1:11:37Speaker 02

  Right? And, um, and it can flip sign, but it's like any radial symmetry can work. Um, and then, and so it has like a polarity, it has like, you know, and you can think of that, I mean, it is a force, you know, in a sense, right? And it's even a force that's like, kind of like real forces. It's like a weird kind of charged particle thing. Um, and so I, I still think that you can, you know, it's, you know, the, the approach that we're taking is basically just discovering what are the effective forces between, it's, you know, we're not worried about the microscopic forces. We don't care, right? That's the whole point of a macroscopic physicist. You know that there are microscopic forces that govern the behavior of the system as a whole, but what you're interested in are the things that, that, that make predictions on the scale you care about.

• 1:12:20Speaker 02

  And so what you're doing is you're discovering the effective rules that describe the interactions between not just the particles that make, or the pixels that make up the, the, the little floater or whatever flyer, um, but the rules that govern its interactions with other floaters or physical objects like obstructs, they put obstructions in the domain and things like that.

• 1:12:39Tim Scarfe?

  And Keith really loves cellular automata because they are too incomplete and they have this miraculous ability to arbitrarily expand their memory. So you can have a grid size that's this big and you can just add more memory and you can add more memory and you don't have to train the thing from scratch. And using some of these approaches we've just been talking about, you can actually train the update rules, learn the update rules of stochastic gradient descent. So do you think in the future we might actually have an AI system which is running inside a cellular automata?

• 1:13:07Speaker 02

  That is a very good question. So the snarky response is to say, don't we all, don't we already? I mean, what's, you know, we got it running on a computer and at the end of the day, a computer is just a whole bunch of logic gates. So isn't it already? It's all right, Tom.

• 1:13:22Tim Scarfe?

  Well, it's in the same class of algorithms.

• 1:13:24Speaker 02

  Yeah.

• 1:13:25Tim Scarfe?

  But there seems to be, you know, a cellular automata that has this emergentist thing. So what it does is not how it's programmed. And it feels that, it feels like there's a trick that the, the way it's programmed is an order of magnitude less complicated than the thing it does.

• 1:13:43Speaker 02

  Right.

• 1:13:43Tim Scarfe?

  So it feels like a magical bridge to do stuff which is more complicated than we could explicitly program or learn.

• 1:13:51Speaker 02

  I agree with that. Also sounds a lot like a computer.

• 1:13:56Speaker 02

  You know, it's like, well, what can you do with like a lot of, now, I guess the difference between a computer and cellular automata is that a computer is, you know, you program it, you tell it exactly, like you're, you specify something. Whereas in a cellular automata, the way that you're training, if you're training to do something in particular, like so, for example, find a bunch of discrete objects that go in a certain direction and then you're allowed to like tweak the rules that govern the local interactions until you get something that more or less does that. That's just programming in a sort of backhanded way. But I think that's, I mean, it, those systems are very interesting because it is remarkable that really dumb, simple rules, right, can lead to like really interesting sophisticated behavior. But I, the thing that I find interesting though, isn't the fact that like, like complicated,

• 1:14:40Speaker 02

  you know, complicated stuff can result from like simple local rules. What I find interesting, what I'm more interested in rather is like, well, what are, what are the properties of the resulting large scale objects, right? Is that, how is that related to the small scale objects? What's the mathematical description of those big things? The things that, the things that are, that have emerged? I'm less concerned, less interested in how they precisely emerge. This probably is because of my bias towards taking a, a human cognitive approach. Most of people don't act, you know, when you look at the game of life, when most people think, oh, that's really cool, like, look at these pretty pictures and all these little creatures and they're doing fun things, they don't really care about the low level rules, right? The thing that captures their imagination is the macro, the, the high level, the macroscopic level behavior of these things.

• 1:15:26Speaker 02

  Though it is cool that you can get them from simple rules.

• 1:15:29Tim Scarfe?

  Yes, yes. No, as you say, we can program computers, but there's the legibility ceiling. We can do program synthesis, doesn't work very well.

• 1:15:40Speaker 02

  It will. They're gonna, they're gonna, I have confidence that that's, that's not one of those things that I'm gonna like outright poo poo.

• 1:15:45Tim Scarfe?

  Yeah.

• 1:15:46Speaker 02

  I do have confidence that, that there's a lot of, that, you know, that is a rich, that is a new area. It's a, you know, a relatively new, and they haven't really, you know, there's a lot, and there's, you know, there are a lot of, there's a, it has a lot of promise. That's what I'm gonna say. And to some extent, the approach that we're taking is compatible with program synthesis, right? We're taking this object-centered description of the world. And the reason we're doing that is because we want to automate systems engineering. Well, what's systems engineering? Oh, that's like taking this object and attaching to this one, attaching to this one, until you get something that does something really cool. Right? Program synthesis, right, is an abstract way of doing that, right? Is that you start with one program, you attach it to another program, attach it to another program, and so on and so forth.

• 1:16:28Tim Scarfe?

  There is this problem of just understanding the program. I mean, going back to Dream Coder, and I'm sure Kevin and Josh have put other ones out more recently. Some of the programs which I learned are just really complicated. You know, they had examples of like, I think, drawing towers and drawing graphs and stuff like that. And you just saw this, you know, huge confection of rules that are being composed together. And it, it's great, it has many good properties that it's a program, but it doesn't really make sense to us.

• 1:16:57Speaker 02

  Yeah. To a large extent, I suspect that there are ways around that that are related to how it is that like your AI coding agent actually works. So for example, right, when they, when they're doing this program synthesis, what they don't currently have access to is, is, you know, is the kind of data set that like GitHub has access to. They don't have access to a whole bunch of really well-written programs that were, that do exactly what they were intended to do. There was a paper in Nature, this was actually one of those situations where, you know, neuroscience is, is making interesting statements about machine learning. From Tony Zadore, and what he had done is they'd taken a whole bunch of neural networks that did a variety of different things.

• 1:17:40Speaker 02

  And then they came up with a way of genetically encoding them. For the purposes of seeing if like, okay, so what's this? So it's like, oh, I had to have a layer that did this and then a layer that did this. And then, and then what I'm going to do is I'm going to like, compactly represent the weights in each layer and come up with a representation of that. And then I'm just going to look at a whole bunch of different neural networks and solve a whole bunch of different problems and say, are there any like patterns that are present in these neural networks such that when I have a new problem I'm interested in, I can sort of just, you know, take something that understands this genetic code, maybe mutate it a little as a way of sensibly, you know, traversing the space of possible neural networks until I find the best one. Program synthesis could in principle exploit the same trick. They just need a data set to do it.

• 1:18:25Tim Scarfe?

  Yeah, yeah.

• 1:18:26Speaker 02

  What are, what are humans in a world where everything can be done by a robot?

• 1:18:31Tim Scarfe?

  Yeah.