How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)

Prof. Kevin Ellis and Dr. Zenna Tavares talk about making AI smarter, like humans. They want AI to learn from just a little bit of information by actively trying things out, not just by looking at tons of data. They discuss two main ways AI can "think": one way is like following specific rules or steps (like a computer program), and the other is more intuitive, like guessing based on patterns (like modern AI often does). They found combining both methods works well for solving complex puzzles like ARC. A key idea is "compositionality" - building big ideas from small ones, like LEGOs. This is powerful but can also be overwhelming. Another important idea is "abstraction" - understanding things simply, without getting lost in details, and knowing there are different levels of understanding. Ultimately, they believe the best AI will need to explore, experiment, and build models of the world, much like humans do when learning something new. SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT: https://www.dropbox.com/scl/fi/3ngggvhb3tnemw879er5y/BASIS.pdf?rlkey=lr2zbj3317mex1q5l0c2rsk0h&dl=0 Zenna Tavares: http://www.zenna.org/ Kevin Ellis: https://www.cs.cornell.edu/~ellisk/ TOC: 1. Compositionality and Learning Foundations [00:00:00] 1.1 Compositional Search and Learning Challenges [00:03:55] 1.2 Bayesian Learning and World Models [00:12:05] 1.3 Programming Languages and Compositionality Trade-offs [00:15:35] 1.4 Inductive vs Transductive Approaches in AI Systems 2. Neural-Symbolic Program Synthesis [00:27:20] 2.1 Integration of LLMs with Traditional Programming and Meta-Programming [00:30:43] 2.2 Wake-Sleep Learning and DreamCoder Architecture [00:38:26] 2.3 Program Synthesis from Interactions and Hidden State Inference [00:41:36] 2.4 Abstraction Mechanisms and Resource Rationality [00:48:38] 2.5 Inductive Biases and Causal Abstraction in AI Systems 3. Abstract Reasoning Systems [00:52:10] 3.1 Abstract Concepts and Grid-Based Transformations in ARC [00:56:08] 3.2 Induction vs Transduction Approaches in Abstract Reasoning [00:59:12] 3.3 ARC Limitations and Interactive Learning Extensions [01:06:30] 3.4 Wake-Sleep Program Learning and Hybrid Approaches [01:11:37] 3.5 Project MARA and Future Research Directions REFS: [00:00:25] DreamCoder, Kevin Ellis et al. https://arxiv.org/abs/2006.08381 [00:01:10] Mind Your Step, Ryan Liu et al. https://arxiv.org/abs/2410.21333 [00:06:05] Bayesian inference, Griffiths, T. L., Kemp, C., & Tenenbaum, J. B. https://psycnet.apa.org/record/2008-06911-003 [00:13:00] Induction and Transduction, Wen-Ding Li, Zenna Tavares, Yewen Pu, Kevin Ellis https://arxiv.org/abs/2411.02272 [00:23:15] Neurosymbolic AI, Garcez, Artur d'Avila et al. https://arxiv.org/abs/2012.05876 [00:33:50] Induction and Transduction (II), Wen-Ding Li, Kevin Ellis et al. https://arxiv.org/abs/2411.02272 [00:38:35] ARC, François Chollet https://arxiv.org/abs/1911.01547 [00:39:20] Causal Reactive Programs, Ria Das, Joshua B. Tenenbaum, Armando Solar-Lezama, Zenna Tavares http://www.zenna.org/publications/autumn2022.pdf [00:42:50] MuZero, Julian Schrittwieser et al. http://arxiv.org/pdf/1911.08265 [00:43:20] VisualPredicator, Yichao Liang https://arxiv.org/abs/2410.23156 [00:48:55] Bayesian models of cognition, Joshua B. Tenenbaum https://mitpress.mit.edu/9780262049412/bayesian-models-of-cognition/ [00:49:30] The Bitter Lesson, Rich Sutton http://www.incompleteideas.net/IncIdeas/BitterLesson.html [01:06:35] Program induction, Kevin Ellis, Wen-Ding Li https://arxiv.org/pdf/2411.02272 [01:06:50] DreamCoder (II), Kevin Ellis et al. https://arxiv.org/abs/2006.08381 [01:11:55] Project MARA, Zenna Tavares, Kevin Ellis https://www.basis.ai/blog/mara/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  You're building a model on the fly from very small amounts of data.

• 0:07Speaker 00

  But you are not passively receiving the data. You have to go out there and poke things and push things and try things out. Compositionality is a kind of double edged sword. We saw this in the first wave of symbolic AI, when people tried to build these kind of compositional production systems to solve problems. So the issue is you immediately encounter this combinatorial explosion in the number of things that you could represent. You don't necessarily have any way of steering yourself through that space toward the kinds of concepts that are probable or which make sense in your current situation.

• 0:44Speaker 00

  So you can kind of think infinitely many things. Those things might be very out of distribution. You're both immensely powerful, but immensely overwhelmed by possibilities.

• 0:52Speaker 01

  It's coming both things from first principles.

• 0:56Speaker 01

  You know, I think it's hard to say what is thinking and what is not thinking. It does seem that some core part of thinking is, you know, a kind of a step by step process.

• 1:07Speaker 00

  Where if you ask people to think carefully and verbalize a solution, they actually get worse.

• 1:19Speaker 01

  Something obviously right about scale, right? And there's something obviously right about learning.

• 1:27Speaker 01

  I think it's kind of true and like obviously true that programs, programming languages are compositional, right? We build more and more complex programs by understanding the parts and then combining them together to build more and more complex parts.

• 1:47Speaker 01

  And most of the kind of hypothesis development that we do as humans is of this form, both as an individual and then of like science as a whole, right? Science as a whole collectively finds hypotheses and then we revise them communally in conferences like this. So it seems natural to me to say that, you know, we should build systems that have this kind of refinement process. One of the hard things to do in trying to mechanize this is like, how do you kind of guide or reinforce good refinement paths? And another axis I think is important, you know, Kevin said this kind of tower of abstractions. I think a key thing which is somewhat overlooked in the current discourse on world models is that there isn't a single world model. You can understand things at multiple different levels.

• 2:30Speaker 01

  There's multiple different models you can build of pretty much anything. There's a camera right here. I've got a model of how this camera works at the level of I press this button and an image is taken. But I can also understand the internal structure of the wires and the circuits. I can go down to the sensor in the camera. All of these are different models that are useful for different things that you need to do with that model.

• 2:53Speaker 00

  We want to avoid building these Frankenstein systems where we are hard coding a whole bunch of different null interpretations and heuristics.

• 3:03Speaker 01

  In those cases, as you're alluding to, it's required smart people to look at the world and say, here are the inductive biases. Let me encode these into the system. And we think that the principles underlying this everyday science, like how do we form hypotheses? How do we revise those beliefs? How do we take actions to learn about how the world works? The same principles in everyday science that apply to real science. And so we want systems that can do that, that can learn how to interact with new toys and devices and interfaces in a way which isn't large-scale limitation learning, which is like thinking.

• 3:39Speaker 02

  I'm Benjamin Cruzier. I'm starting an AI research lab called Twofal Labs. It is funded from past ventures involving machine learning. So we're a small group of highly motivated and hardworking people. And the main thread that we are going to do is trying to make models that reason effectively and long-term trying to do AGI research. One of the big advantages is because we're early, there's going to be high freedom and high impact as someone new at Twofal Labs. You can check out positions at tufalabs.ai.

• 4:12Speaker 00

  I want to make machines that learn in more human-like ways, especially learning more abstract knowledge from fewer examples. And we've been working a lot recently on world models and on things that try to actively discover symbolic knowledge. And I'm interested in like, everything people do. I'm interested especially in the kinds of things that people are especially good at, but that the kind of AI that we're building today tends to not be so good at. So learning from a few examples, generalizing to situations that are very different, learning knowledge you can communicate as opposed to being just embedded inside of a weight matrix, and also learning things that can cooperate between weight matrices and symbolic

• 4:57Speaker 00

  kinds of knowledge.

• 4:58Speaker 01

  Yeah, so hi, I'm Zenna. I'm a co-founder and co-director of Basis. Kevin and I went to grad school together. We share a lot of the same interests broadly in trying to understand and build intelligence. Within Basis, we have that as a goal, but we also care about kind of just like scientific and societal problems. And so I've got many, many interests and in some ways I've built an organization so I can do all the things that I find enjoyable.

• 5:23Speaker 00

  So I think there's a lot of good lessons from the scaling round. We do actually want algorithms which can make use of the new hardware that we have. At the same time, it's not the full story. I think our personal belief is that that's not going to carry us all the way. And also pragmatically, there's other people that are clearly trying to carry that torch. And if it does get us all the way, then maybe we should kind of hedge and do things that draw on the amazing GPUs, amazing pre-trained models we have, but which also bring to bear ideas from cognitive science, classic AI, and so on.

• 5:56Tim Scarfe?

  Zenna, how do humans learn from examples?

• 5:58Speaker 01

  I think there's many possible answers to that. You know, I somewhat believe in the Bayesian paradigm, right? Which is that you have beliefs about how the world works. And when you kind of observe examples, you incorporate that knowledge into your kind of current hypothesis about how the world works. But that's a very abstract theory and the question is how was that actually implemented? And you know, the real, the reality is it's probably approximate. It's probably, you know, we're not doing exact Bayesian inference. So there's a nice kind of foundation to think about how you should, you know, how an ideal agent should incorporate knowledge. And then there's kind of practical questions about how to build systems that do that, right? And so as Kevin said, We're very much like not anti, you know, large scale deep machine learning. We're building these tools, we're using these tools.

• 6:42Speaker 01

  Uh, I would say like the main kind of distinction between how I think of the work that we do and what we want to do is can we build things from first principles, right? So a lot of, you know, current mainstream machine learning is basically like large scale limitation learning. And that's because it's so effective, right? Uh, and it's, you know, producing all of these amazing tools. But both from a scientific perspective and also from an engineering perspective, can we understand some of the principles about how to build intelligent machines kind of from the ground up? Uh, and I think there are some of the pieces, right? Uncertainty, causality, reasoning in general, but there's many things that we, we don't know, right? And so part of having a research program is to try and figure out those missing parts.

• 7:21Tim Scarfe?

  Kevin, um, you know, what is the importance of compositionality and, and what is compositionality?

• 7:27Speaker 00

  So, uh, compositionality has a lot of different meanings in different situations. The way that, um, I think that we're using it in the work that we're doing right now is that you have, um, atomic knowledge that you may be learned in one situation, but then you can build bigger structures out of those little pieces of knowledge in order to extrapolate to new situations that might even be out of distribution. So I think that's really important for being able to learn in environments that are not like totally new. It's not like you can plop yourself in a totally new world and immediately be fully competent. But if you're in a world that's, um, is changing, but where the building blocks are the same, the causal mechanisms are being reused, then compositionality and, um,

• 8:09Speaker 00

  ways of having knowledge that can be broken into pieces and then recombined for new situations is really crucial.

• 8:15Tim Scarfe?

  Very cool. I mean, just as an extension to that, Kevin, um, I, I watched your amazing YouTube talk, which you published about three months ago, and you quoted Elizabeth Spelke from Harvard. Yeah. She's a psychologist and she said, "The possession of infinitely many concepts that were, uh, that were expressible in an innate language would lead to a curse of a compositional mind." Can you explain what she meant by that?

• 8:36Speaker 00

  Yeah, so, uh, so Spelke is wonderful.

• 8:40Speaker 00

  And what she's pointing out is that compositionality is a kind of double edged sword.

• 8:47Speaker 00

  And we saw this in the first wave of symbolic AI, when people tried to build these kind of compositional production systems to solve problems. So the issue is you immediately encounter this combinatorial explosion in the number of things that you could represent. And you don't necessarily have any way of steering yourself through that space toward the kinds of concepts that are probable or which make sense in your current situation. So you can kind of think infinitely many things. Those things might be very out of distribution. And so you're both immensely powerful, but immensely overwhelmed by possibilities. And so I think what we're seeing now is that you can address this curse of compositionality

• 9:31Speaker 00

  through ways of learning to guide searches over program spaces. And also by kind of learning the basic atoms of a compositional language and treating it as how that works.

• 9:44Tim Scarfe?

  When we design AI algorithms that kind of navigate compositional spaces, how does that work? And maybe how does that compare to how humans do it?

• 9:52Speaker 01

  One of the, in my view, most interesting examples of compositional languages are just programming languages. Right? And so I think it's kind of true and like obviously true that programs are, programming languages are compositional. Right? We build more and more complex programs by understanding the parts and then combining them together to build more and more complex parts. One of the differences between programming languages and natural languages that programming languages are like strictly compositional. Right? Like whereas in natural languages, all these kind of heuristics, you know, phrases where you have to just like learn the thing. But you know, how, you know, how do we build systems that build compositional structures? You know, there's many ways. A lot of old fashioned ways for like, you know, start with a grammar, expand that grammar, search through it,

• 10:36Speaker 01

  and try and find a program by kind of, you know, search through a grammar space. Now people are using obviously language models to, you know, generate, you know, programs and compositional structures. And so there's this kind of interesting. I would say like spectrum of approaches where you know, there's a question of like, how much semantic knowledge about the compositional structure you incorporate into your model versus it's kind of purely like data driven from, from examples. And you know, we've both worked on both and continue to work on both. But as Kevin said, like the, you know, the challenge is if you've got a compositional structure, it's often, you know, you know, of kind of unbounded dimension, a very hard space to search through. And so you need, you know, smart methods to try and find, you know, the, the structure

• 11:20Speaker 01

  or the program or whatever it may be within this vast space.

• 11:23Tim Scarfe?

  On, on that then, Kevin, I mean, is there a principled way to find a set of primitive abstractions which can be composed together to build these higher level abstractions?

• 11:32Speaker 00

  I, I think that, that some kind of basis sets of primitives might be strictly better than others. So as a easy example, so as Anna said, one of the best examples of compositionalities is programming languages. I think programming languages have gotten better over time. And also it's really easy to make a language worse. Right? I could break Python in a million ways. So obviously there must be kind of a partial order between compositional systems where some are better than others. But I think in the fully general case, it, it does depend on the kinds of problems and environments with which you're going to be confronted. And I think you see this reflected in the fact that we have tons of different programming languages. We don't actually have a single best programming language. We have a kind of parry to frontier.

• 12:17Speaker 01

  You know, a neural network or a class, you know, particularly in neural networks is an architecture, right, which implements some algorithm. You could implement an algorithm in, in Python, right? The Bayesian paradigm is a normative model. So it says like, what should an ideal system do? You know, given essentially, you know, without consideration of compute. You could also say, what would an ideal system do if you had bounds of, you know, bounds on top of compute? Or like if you had to reason about how much compute you should use. And so I don't think of it necessarily as like bottom up versus top down, at least on this, on this axis. But you can think about kind of composite methods which combine different kinds of systems, right? So the paper that Kevin started and we kind of joined forces to work on and complete,

• 13:02Speaker 01

  you know, use the combination of different methods, right? And so Kevin will talk more, I'm sure, but we have like an induction and a transduction model, right? And the induction model is then in a program, in the case of ARC, right? Like a program that transforms the input to the output, like a Python program, and the transduction model is directly outputting the output grid given the input. And so I think it's important to consider, like, again, different axes. One is like, what is the representation of the function? Like, is it a Python program? Or is it a neural network? Right? And then the other is like, in some sense, like, what is the type signature of the thing that you're finding, right? Are you, you know, directly outputting the final solution? Or are you outputting a function which you can then apply to the to

• 13:45Speaker 01

  the input, right? And so, you know, we've explored a little bit of this kind of, you know, grid of possibilities, but I think there's, you know, more we could think about. I don't, do you want to kind of?

• 13:55Speaker 00

  So I like how you described it as like the type signature. So in a sense, the inputs and outputs of the symbolic and neural ways of solving problems are often very different. So if you try to solve something which is a pure transformer, like implicit outputs, like as we do within the transduction model, you're not constructing these intermediate hypotheses. And when you have these symbolic compositional languages, like when you're doing program synthesis, the actual type of the thing that you're trying to learn is different. So when we think about comparing neural and symbolic methods, they often kind of confound these different factors. When we did this work, we were partly comparing

• 14:38Speaker 00

  neural and symbolic, but also partly comparing these two different styles of problem solving. One where you look at a problem and think hard about an explicit way of solving it, a way you could maybe verbalize in a symbolic language, either in code or in language, and then contrasting that with this more intuitive, implicit, more transductive way of making predictions. It just so happened that was convenient to map on to this neural symbolic divide. But you can totally imagine inductive methods that kind of search for like a vector that describes how to solve a problem. And in fact, there were art teams that did this, and it actually kind of works, which is really cool. So as I was saying, there's really kind of a grid of possibilities.

• 15:21Speaker 00

  And we're trying to do a very careful study of these different ways of solving these inductive learning problems.

• 15:31Tim Scarfe?

  There was an amazing paper, by the way. So as I understand it, there's two LAMA 8 billion models. One's an induction model, one's a transduction model. In one case, you produce an explicit function, you do like, you know, test time inference, you greenblat it. This is my verb, I use Ryan Greenblatt. I generate loads of example programs and take the good ones. And then you've got the transduction approach where you directly compute the solution, but still with some transductive active fine tuning by augmenting the test examples. And the ensemble approach is you try and see if one of the induction functions works, if not, fail over to the to the transduction. There's this beautiful Venn diagram in the paper where you show that transduction works for some types of problems, induction works for other types of problems. Help me understand.

• 16:16Speaker 00

  So interestingly, this does relates to some classic findings in cognitive science where we know that there's certain kinds of problems where if you ask people to think carefully and verbalize a solution, they actually get worse. So for instance, if you try to have people infer a rule with exceptions, or if you have people try to do more of a statistical learning task, like learn kind of associations between different symbols, having them think a bit and explicitly verbalize a solution can degrade performance. So we know empirically this is true, and there's a really nice paper from Tom Griffiths showing that LLMs also have similar kind of splits.

• 16:59Speaker 00

  And I think we were kind of rediscovering this within the abstraction reasoning corpus. So we are finding that there are some kinds of problems where if you have the system kind of think hard and churn through many thousands of different possibilities and test them systematically, that's actually worse than having the system just blur down an answer. And so because transduction induction, so systematically sweeping through possibilities versus just blurring out the answer, because you can check the correctness of induction, this means that you can think systematically for a while. And if you fail to find anything that seemed like a good explanation to you, you can just fall back on your intuition. So there's a very natural way of ensembling or combining these two methods.

• 17:44Speaker 00

  We don't really need a way of looking at a problem and deciding if you should think explicitly in symbols or if you should use your intuition, because we can validate the correctness of symbolic hypotheses.

• 17:56Tim Scarfe?

  Is one thinking and is the other not thinking? So for example, if I do lots of transductive active fine tuning and augmentation and I give a sub symbolic intuitive solution versus doing thinking by generating lots of explicit you know, and functions, is one thinking and the other not thinking?

• 18:12Speaker 01

  I guess it depends on what you mean by thinking.

• 18:14Tim Scarfe?

  What do you mean by thinking?

• 18:16Speaker 01

  Well, now people say thinking is what I wonder. So maybe that's thinking. You know, I think it's hard to say what is thinking and what is not thinking. It does seem that some core part of thinking is, you know, a kind of a step by step process where you kind of go through some kind of internal mental computations, right? Revising your beliefs. You know, I think it's hard to put a strong kind of circle around it. Again, I do think there's a distinction between representations of knowledge. In this case, right, you can have a symbolic program, right, or you can have a purely neural connection

• 18:59Speaker 01

  or system output and answer versus kind of the kinds of procedure that you produce, right? And again, as you can kind of combine and compose all the different possibilities. Uh, we just talked about the fact that when you have an inductive model in our, in this kind of terminology, right, we have a system that outputs a Python program. We can then use this Python program to check whether it's kind of consistent with the training examples. And this is a good, a very strong, you know, signal, whether it's going to be a valid solution for the, for the test example, right? And that is not currently true for our transduction model, right? But you could imagine a system where that is true, right? Where you could have the messiness of a neural network, but you still get the ability to

• 19:42Speaker 01

  go through and check, is it consistent with the, with the examples? Uh, but on the question of like, you know, is that thinking, I think, you know, all of these are different versions of, of thinking, right? I think there is a kind of a slow deliberative hypothesis forming notion of thinking, which probably isn't quite captured fully within any of the methods that we use in ARC and probably is something more like what, you know, O1 is doing, where you've got a kind of a, uh, you know, a variable time computation, uh, where, uh, you're, you're not kind of blurting out things, right? In one step, uh, but you're understanding how much you have to do to answer a question, uh, which is something like thinking. Maybe a more narrowly constrained version of thinking is reasoning, right? Where you have the kind of the purely classical forms of reasoning, where you've got some

• 20:27Speaker 01

  axioms and you kind of go from that to some conclusion. And then you've got the common sense reasoning that we do every day as humans, right? To figure out how to, you know, take, you know, carry out our everyday lives. And I imagine that there's some way to think about, you know, reasoning in general, which is, uh, you know, broadly construed as forming beliefs, given our, our knowledge, but allows for the messiness of, of the world as, as we see it in a way that kind of classical formalisms don't quite allow.

• 20:52Tim Scarfe?

  In the ensemble approach, you biased the induction. So you, you wanted to find an inductive solution first and then failed over to the, to the transduction. Does that hint that you kind of think there's something special about the, the, the functional version?

• 21:07Speaker 00

  I think that the functional version is more regularized, like it's harder for it to overfit. It's not impossible, but it's pretty tough to overfit with these modern, high-level programming languages. They're just designed and they just want to express general-purpose computations.

• 21:26Speaker 00

  Neural networks, they can do that. Particularly, as you were saying, in the kind of large data limit, they tend to eventually learn representations that start to really capture the kind of stuff you want. But that's not always true, and often they kind of interpolate around their data points. And so I think if you can come up with a clear, explicitly verbalizable description that actually works for the problem you're trying to solve, then that's likely to generalize a lot better than

• 21:55Speaker 00

  just going with an intuitive kind of vector-based interpolation.

• 21:58Tim Scarfe?

  Zenner, is there a better way that we could, you know, maybe in a principled way, combine transduction and induction?

• 22:04Speaker 01

  Several different ways. So again, I think one thing you could think about is could you have a kind of a transductive-inductive model where the underlying transformation representation is something like a neural network? But you can apply it kind of point-wise to each ARC instance as an input, right?

• 22:24Speaker 01

  And this would allow you to get perhaps this maybe stronger inductive bias about is this the correct example by testing it on the training set? Maybe a more interesting kind of, I don't know if this is more principled, but a more interesting space of things to explore are more on the representation side. Like what is it that a kind of a a normal program gives you, like fundamentally? Like what is the benefit of using a Python program in some cases over a neural network? And what is the fundamental benefit of a, you know, of a neural network in terms of encoding transformations, right? So we talked a little bit about maybe the neural network can capture these messy parts of of the, you know, of the transformation. In the case of ARC, you know, there are some things that are hard to describe with Python programs, right? They're hard to actually, you know, if you sit down and try and write a program,

• 23:09Speaker 01

  it's actually quite difficult because of, you know, just weird, complicated things that happen.

• 23:15Speaker 01

  And so, you know, there's the whole paradigm of like neuro-symbolic programming, which is, okay, let's take some neural components and let's take some kind of classical programmatic components and combine them together. But you might try and think like from the bottom up, like, is there a, you know, way to restructure programming languages in general, right? If you think of them both as kind of different programming languages in a sea of all possible programming languages, and we've kind of touched on a very small number of, you know, languages in the space, what other things exist, right? And I don't know, but I think, I think we really haven't scratched much of the surface of different representations of computation and programs in general that we could. And part of what we're thinking about now is like, can we find not just deeper integrations,

• 24:00Speaker 01

  but just like, you know, more fundamental things about like, what in principle do we want from a programmatic representation versus what we don't want, right?

• 24:08Tim Scarfe?

  That's fascinating. Maybe you could expand on that, Kevin. You know, in the Dreamcode paper, and we'll go into that shortly, it was using a DSL. And of course, we could use something like, you know, like Python. And maybe you could argue that there's some kind of computational equivalence between high-level programming languages because they're too incomplete. What is the difference between these different expressions?

• 24:32Speaker 00

  Yeah, so in the first generation of AI, there's a saying, you can't learn what you can't represent. And I think that, you know, to tack on that like spokey quote earlier, if your computational language explodes too bad, maybe you could represent it, but it's not really in a practical sense learnable. So in Dreamcode and a lot of other languages that use domain civic languages, often you're just not able to represent all the stuff you could do in Python. Now, there's a simple fix to that. You can introduce a kind of escape hatch, which is you have some primitives that upgrade your DSL, your domain civic language, to actually be Turing completes. And then in some formal sense, you actually do cover all the stuff that you could do

• 25:16Speaker 00

  in Python, but then this curse of compositionality bites you. You couldn't have to learn in a practical sense. So in my opinion, having worked with these DSLs with lambda calculus,

• 25:30Speaker 00

  Python is just a lot more practical for the kinds of problems that we really care about. This is true in ARC. I think it's true when we're building LLM agents. I think it's true when we're having like visual question answering systems.

• 25:43Speaker 00

  Like there's been interesting kind of convergent evolution between the design of software engineering languages, which has caused them to converge to something that's actually really great for a lot of problems we care about in AI. They're not perfect, and I think that in this transduction induction paper, we do see that Python is not actually covering all the stuff that we care, but it's miles better than lambda calculus.

• 26:05Speaker 01

  Yeah, I think it is really important to to recognize that there's an evolution of programming languages. And Kevin said this at the start, right? The original programming languages were basically just like, you know, long streams of instructions, right? And then we built in more and more forms of structure. There was a whole structured programming movement. And then we have like, you know, a variety of different families of languages that we use in modern software engineering today. You know, kind of very mainstream languages like Python, you know, maybe slightly less mainstream languages, but popular in other circles like, you know, high school and functional languages. And so I guess the key thing for me is that beyond just expressing computation, which is, you know, not that hard to do, we can express computation in many different systems. We've started to build more and more kinds of systems to help us build better programs, right?

• 26:50Speaker 01

  Class structures, type structures, right? Different modular structures. And they allow us to encode more about the world within our programming languages and more, you know, just more useful, right? And so if you think of this as an evolution,

• 27:05Speaker 01

  one, as Kevin said, right, like we've got to a point where they're pretty good, but also there's like room for development, right? We probably wouldn't expect that in, you know, a hundred years, everything would look exactly how things are now. And so I think an interesting question is like, how do we expect the evolution of programming languages to develop? And then when you add in just like, you know, AI and general into the picture, it becomes like a lot harder to predict, right? Like how, you know, now we're seeing like these interesting compositions of, you know, ChatGPT calling out to Python and Python calling out to ChatGPT. It seems to be in this kind of moment of flux of the design space. And this kind of, for me, it's interesting to see like how this will evolve over time.

• 27:42Tim Scarfe?

  Yeah, I guess, should we have some kind of meta-programming built into the algorithm? So, you know, start with maybe an iterative process of rewriting rules and improving hypotheses, rather than trying to immediately happen on the final?

• 27:57Speaker 01

  The answer is yes, right? I think it's hard to like, kind of provide an argument why that has to be true, right? But at least intuitively, to me, it seems to be quite hard to like, write exactly the right program in one shot all the time, right? And so there's some value in, you know, constructing a model, a hypothesis, or a program, depending on the context, and revising that with more information. And that more information could be, oh, it doesn't work in some particular use cases, or it's like, doesn't support the evidence. And most of the kind of hypothesis development that we do as humans is of this form, both as an individual and then of like, science as a whole, right? Science as a whole collectively finds hypotheses and then we revise them communally in conferences like this.

• 28:39Speaker 01

  So it seems natural to me to say that, you know, we should build systems that have this kind of refinement process. One of the hard things to do in trying to mechanize this is like, how do you, how do you kind of guide or, you know, kind of reinforce good refinement paths, right? Like, how do you say, okay, this mode of reasoning or this mode of refinement or this path of refinement is the good path versus this other path, which is bad? Right now, you know, in, let's say, like, modern machine learning, people are doing this mostly based on either reinforcement from human examples, right? Or if you've got a kind of a well-defined objective function, you can like, back propagate the signal from the, you know, correct solution, if you find it, through the reasoning paths.

• 29:24Speaker 01

  But that's not always the case, right? That's not always possible. And so I think one of the big open questions is, how can we build systems that reason? And part of reasoning is refining hypotheses, where we don't have kind of a well-defined, easy-to-compute objective function that we can evaluate at the end? I think if you can solve that, you can, you know. Do pretty well.

• 29:46Tim Scarfe?

  Very cool. And any thoughts on that, Kevin?

• 29:48Speaker 00

  Yeah, I just want to reinforce exactly what Zenna said right there, which is a lot of this really hinges on being able to check that you're heading in the right direction.

• 29:56Tim Scarfe?

  Yeah.

• 29:58Speaker 00

  So we see this in like our own work that we're doing together with when we're trying to learn world models or programs or ARC tasks, because we have some data we're trying to fit and we can check our fit to the data. And I think you also see this in things like O1, where to best my knowledge, they're mostly training it on like math problems where you can check if you got the answer right. And it is a big like open puzzle, and I'm not sure I have like great ideas here, on how to do this when you can't really check if you got it right. So I'm not really sure to do that, but it does seem very important. Interesting fact that when you're trying to learn a model of the world, you actually can check if you're getting it right. You can look at your data, what you seen earlier, and you can say, okay, does my model like correctly predict that data?

• 30:43Tim Scarfe?

  Kevin, in the Dream Code paper, you helped pioneer this wake-sleep fine-tuning strategy, you know, essentially where a model can dream, it can it can kind of expand what it knows, and then in the waking phase, it can reincorporate those dreams as hypotheses. It's absolutely amazing. And of course, you're using it in a slightly different way in this new transduction paper. But can you just tell us about the philosophy there?

• 31:07Speaker 00

  Yeah, so the philosophy there is tightly connected, actually, to what you see, I mean, in a lot of machine learning, where you train a bottom-up model based on synthetic data produced by a top-down process. So often, we're trying to solve in AI some kind of inverse problem. Like in vision, we're trying to look at a image and then infer the three-dimensional structure. But the forward process of inferring the image from the three-dimensional structure is a lot simpler. That's a rendering function. So in wake-sleep, you're taking some forward process, you're imagining or dreaming possible ways it could run forward, and then you're learning how to go backward.

• 31:49Speaker 00

  So for learning a function or a program, this means you're imagining programs, you're running them, and then you're saying retrospectively, well, I just imagined this thing, I did this thing, so when I see this kind of behavior, I should infer this kind of program. What wakesleep brings to the table on top of that, on top of just training on top of synthetic data, is that it's a kind of back and forth between learning from your own synthetic data and then learning how to make better synthetic data. So it's not just this big batch mode thing of lots of dreams and then some money from dreams. It's that you actually wake up, you go into the world, you try to solve problems, and then you adjust your distribution of synthetic data based on the problems that you're solving. And this allows you to adjust to distribution shifts.

• 32:34Speaker 00

  So you might think the world works a certain way, you have certain kinds of dreams. You learn from those dreams, but then you wake up, see the world is different. And then during the next sleep cycle, your dreams shift to better match the world. So there's really a kind of cooperation between the wake and sleep cycles. And they're interlead inside of this kind of big batch of synthetic data.

• 32:53Tim Scarfe?

  I also think there's an abstract component here in intelligence systems, certainly in all AI models. There's this iterative expansion and compression, expansion and compression. We see it even in neural networks.

• 33:05Speaker 00

  You actually want to be able to cover a little bit more of the distribution. And I think this is where compositionality comes in and why it's good to do wakesleep with programs. Because when you have a compositional language, you can take two atomic pieces of knowledge you learned earlier, you can glue them together in a way that you never actually saw, but is kind of plausible and which you might see. And you should equip your neural network to be prepared for that composition. So I don't think this is really a thing you want to guard against. You don't want to say, oh, I need to tightly fit my dreams to match my waking experience. You actually want them to go a little bit beyond that.

• 33:43Tim Scarfe?

  And of course, it's interesting that in DreamCoder, it was an explicit process. And I suppose you could actually think of the induction transduction paper as being the same thing, but it was using remixing and retrieval augmented generation. And because in DreamCoder, there was this notion of a library and kind of like expanding knowledge. And now the process is becoming slightly more diffused. I mean, can you sketch that out?

• 34:05Speaker 00

  Right, right. So we're moving more and more toward ways of implicitly doing the kinds of things DreamCoder was doing.

• 34:11Tim Scarfe?

  Yes.

• 34:12Speaker 00

  But it's building them on top of large foundation models. So instead of having an explicit library of symbolic concepts that the system is learning over time, instead it has code that it knows was good in the past and it uses a neural network to produce similar code. So this is effectively what library learning was doing. It was always saying, you know, I've written some code in the past. I'm going to learn some functions and that will allow me to write similar code in the future, except now it's done in a softer, more probabilistic way. So what you, so I think actually you want to do both. There's something really valuable about how software engineers write reusable

• 34:55Speaker 00

  libraries, but it's tough to really write the correct library right now with AI systems. It's tough to automatically debug not just a program with AI, but a full library of programs. And so as a kind of halfway measure, you can use in context learning to approximate the kind of abstraction learning you would get from a library. But in the end, I think they are complementary and we're not, we're not ditching libraries. We're just saying there's a kind of middle ground that's easy to implement and works like reasonably well.

• 35:29Tim Scarfe?

  We want to build a library of knowledge and there's an exploration, exploitation dilemma. How much entropy, for want of a better word, you know, how much of that library do we, do we, do we keep hanging around?

• 35:42Speaker 01

  Yeah, I think one way to think about it is like, yeah, just as a programmer, you know, why ever build a library function? I think there's a few different reasons. One, by having a function that can allow you to kind of express your current program that you're trying to write more compactly. Right? So I've got some kind of shared structure and I can reuse that in a variety of different ways within my current kind of task that I'm trying to solve. But another related reason is more future looking, right? So people build libraries not for their current program, right? But in the expectation that, you know, them or other people are going to use that functionality into the future.

• 36:26Speaker 01

  So I think you might be able to think about and perhaps even formalize.

• 36:30Speaker 01

  Uh, you know, when and why you want a library? And it's something like, you know, in the expectation of current and future uses uses, it's gonna make my life easier, right? Like, I'm gonna be able to use this thing that I'm kind of cashing now, right now, but perhaps sometime in the future, right? And we do this informally, again, as, you know, as software engineers, where we build, we build things that we think other people will eventually use. And I think the mind probably does, you know, you know, something like that in a kind of very hand-wavy way, right? You know, we're building structures that we expect we're gonna be able to use and that we'd like to use sometime in the future. And we have this kind of, you know, kind of cached computation that we can reuse at a future point. So that's like broadly how I would think of it, and you can, you know, perhaps even

• 37:15Speaker 01

  try and formalize that and cash that out in some kind of, you know, rational decision theory.

• 37:20Tim Scarfe?

  Kevin, how can, how can we test that these abstractions that we infer kind of represent the causal relationships in the real world?

• 37:29Speaker 00

  You need to actually go out into the world. You need to have an agent that's in an environment, and it needs to be testing that its model

• 37:38Speaker 00

  faithfully describes real causal mechanisms. So it needs to have an action space, be able to do interventions, and so on.

• 37:45Speaker 00

  So when we're just trying to learn functions, it's kind of hard to tell that your abstractions are the right ones, right? In a sense, there might be lots of equivalence ways of describing the same function space. But as soon as you put your agents in a world where it has to achieve a range of goals, where it has to plan and intervene on things, then you could actually kind of falsify these hypotheses. In contrast, in, in program synthesis, you don't falsify a library. You just say, this is not very useful right now. Maybe useful later, but like, it's not very useful. In contrast, if your program is making hard claims about how the world works, that's where you can actually say, you know, this is not faithful to the true causal mechanisms.

• 38:29Tim Scarfe?

  Very interesting. I mean, we should bring in your autumn paper, Zenna, but I suppose you could say that something like ARC is non-agential.

• 38:36Speaker 01

  ARC is not agential in the sense that there's no, you know, it's a regression problem, right? There's no like interactivity. You can try and solve it in a kind of internal agential way, right? I intuitively feel when I'm solving ARC, I'm doing little mental experiments to try and, you know, look at that, test that, you know, form this hypothesis. But the actual domain itself doesn't require me to take any kind of sequence of actions, right?

• 39:02Speaker 01

  In contrast, like, most of the things in the real world are not like that. Most of the things in the real world require you to like actually interact with the system through time. The canonical example of just like games, right? If I'm playing in some video game, I'm taking some sequence of actions to try and explore that. So the autumn paper, which was, you know, the first student was a graduate student called Ria Das, it was a joint paper with Ria, myself, Josh.

• 39:27Speaker 01

  And the goal there was to build a system that could synthesize essentially the source code of a video game after observing interactions with that video game. And, you know, the background idea or the background proposition is doing so is a kind of science, right? If you can observe some dynamics and you can infer the source code of the underlying world behind the dynamics, that's something like, you know, understanding, that's something like building a model of how that world works. And so the paper that we kind of, you know, did there was to build a little DSL for a certain class of interactive games, if you will. The way in which they're not games is that there's no external reward function.

• 40:07Speaker 01

  It's just like an environment which a system can interact with. And from kind of traces provided by human of these games or these environments, the autumn, you know, synthesis method infers your underlying code. And one of the key things we wanted to explore there was what you might call latent state, right? So things that are true about the world but you can't directly observe. And the real world is full of things like this, right? Pretty much everything has some hidden state that we can't observe. And this hidden state is often very complicated, right? And dynamic, right? And it's probably best described by some kind of Program evolving over time. And so kind of a key contribution there is like, can we infer this hidden latent state in addition to the entire kind of full program? But one thing like maybe going back to the previous point about abstraction is

• 40:52Speaker 01

  in that work, you know, the programs or the models that we kind of inferred are not abstract, right? They're kind of ground truth models of the world. And we don't really think this is how, you know, thinking, human thinking works or even could work, right? Like, there has to be some abstraction error. There has to be some parts of reality that we omit or discard from our models. And so, you know, a kind of a big open question is like, how can you infer abstract models, right? How can you infer models that kind of omit the right parts of, you know, of the, of, of the world in order to, you know, be a kind of a practical and useful thing? So we didn't really explore that within Autumn, but this is, you know, kind of very high on our minds of things that we want to explore and not just us, other people in the community too.

• 41:36Tim Scarfe?

  So we, we think in abstractions, you know, she's on top of the world. And there must be some kind of a hierarchy of abstractions. And when we're dealing with perceptual input, for example, I mean, how, how, how do we kind of navigate the abstraction hierarchy?

• 41:49Speaker 00

  So one thing that's interesting about how people think about problems at different levels of abstraction is that the abstractions are often defined on the fly for each problem. There's not one ground truth abstraction. The world gives you data and you could do the kind of Sora or genie type of world model where you truly model the full data, you capture all the pixels. Or you could do the kind of thing that we're advocating for, where you deliberately discard piece of information. And when you do that, I think the problem just becomes under constraint. In order to introduce those extra constraints that are needed to tell you what abstractions are valuable,

• 42:34Speaker 00

  the easiest thing to do, and something that certainly works, is to introduce reward. So if you give a reward signal in the environment, then you can say that a good tower of abstractions, even if it's on multiple levels, is one which allows you to plan to achieve reward.

• 42:51Speaker 00

  You see this in MuZero, where they're learning kind of an abstract world model that doesn't, it's not fully generative. It just predicts reward, policy, value, and so on. In some of the work we've done recently, we had these kind of simulated robot environments where robots interacting with an environment to use like a tool or a mechanism. And it sees pixels, but then it tries to define some abstractions on top of that. And the abstractions deliberately ignore a bunch of details. So this was work led by Yichai Liang. It's called Visual Predicator.

• 43:24Speaker 00

  And it's, you know, I was taking this kind of MuZero perspective that someone gives you a reward signal. What I think is really interesting is that even if you don't tell someone what the reward is, someone can still play with a new object or a new web app or a new appliance and form an abstract model. We're still thinking about exactly how that would work in a program synthesis context. A lot of the cleveries that we're planning right now are kind of trying to answer that question. But I think it's very open. And if you don't have reward, clearly, humans can still figure out abstract models that, as Zenna was saying, omit a bunch of details. But it's trickier and it might connect to certain normative theories of like intrinsic motivation or it might be something like you want to be robust to a wide range of possible reward functions.

• 44:10Speaker 00

  There's a bunch of possibilities, sir.

• 44:11Tim Scarfe?

  Is there a principled way, though, of kind of, in the situation, detecting which is the best level of abstraction?

• 44:19Speaker 01

  Maybe. The framework that I find to be quite compelling is the framework of resource rationality. And this is basically saying that you should try to do the best you can with the resources that you can, right? And so, you know, maybe that's a slightly convoluted statement, but the idea is that you have some kind of belief, right? Some distribution of a possible tasks or uses of a model.

• 44:47Speaker 01

  And you have kind of computational constraints, right? You can't run things forever. And so I think a way in which you can kind of cash out this question of like, when should you choose the right abstraction? Or what kind of abstraction should you construct? Is to say, well, again, I've got some beliefs about how I'm going to use this model. The question is, it needs to allow me to answer. The task it allows me to do needs to incur computational costs. And so I should consider all of those things and do like the optimal thing.

• 45:19Speaker 01

  And another access I think is important, you know, Kevin said this kind of tower of abstractions, that I think a key thing which is somewhat like overlooked in the current discourse on world models is that there isn't a single world model, right? You can understand things at multiple different levels. There's multiple different models you can build of pretty much anything, right? You know, there's a, there's a camera right here. I've got a model of how this camera works, right? At the level of I press this button and, you know, an image is taken, right? But I can also understand the internal structure of like, you know, the wires and the circuits, or I can go down to like the sensor in the camera. All of these are different models that are useful for different things that you need to do with that model. There's no single correct answer, except, you know, physics, right?

• 46:03Speaker 01

  And so I think a key goal for us is to say, well, let's embrace that plurality, right? And try and find representations of models which incorporate a plurality of different models within them, right? So I've been using this term polystructural to kind of capture this idea. We'll see if that, you know, if that term, you know, sticks. But whatever you call it, like, we need to encode multiple different models of reality and the relationship between those models, right? As a human modeler, you know, let's say like a formal scientific modeler, let's say I'm modeling COVID, I can say, well, I don't know, like hair color doesn't matter, right? In my COVID model, right? But this relationship between the model and reality is encoded in my head as the, as a scientist.

• 46:46Speaker 01

  We want that relationship to be within kind of the computational formalism itself. And that is, in my sense, like a hard, just like scientific computer science question, which I think hasn't fully been explored. Maybe, maybe it will just emerge from scale and data. I don't know. It's kind of a question of whether we have to build these things in or whether they'll emerge.

• 47:04Tim Scarfe?

  How can we, how can we automate this process of epistemic foraging?

• 47:09Speaker 00

  So I think what we want to avoid is we want to avoid building kind of like these Frankenstein systems where we are hard coding a whole bunch of different non-representations and heuristics for reasoning with those representations.

• 47:26Speaker 00

  It's everyone something which looks more like rational analysis from first principles. And when you do that, you do immediately run into hard computational problems. You get a big search space, it might even be hard in the inner loop to evaluate how good a model is, or how good an abstraction is, because you need to then retrospectively say, well, would this be good for the kinds of reasoning tasks I expect? So that computational problem is a place where I think it'd be good to insert learned neural networks that have good intuitions about, you know, everyday common sense of factions and so on, which can propose them.

• 48:09Speaker 00

  It can say like, you know, this code would be valuable probably in this situation. But whereas proposing much even alternatives, as you were saying, the kind of green bot style, like, whereas a heuristic, so that we can still have this first principles way of saying, you know, this would be a good collection of abstractions for the kinds of stuff I expect. But then we can still take advantage of the kind of prior we get from pre-trained neural networks.

• 48:33Tim Scarfe?

  I suppose the broad question is how much human engineering and seeding is required?

• 48:37Speaker 01

  Yeah, so I think if you look at kind of the history of, let's say, Bayesian computational models of cognition, a lot of which was done by psychologists and cognitive scientists like Josh Tenenbaum and others, there's a really compelling and strong history of, you know, expressing some prior knowledge and showing that humans do something like approximate posterior Bayesian inference conditional on the data. But often in those cases, as you're kind of alluding to, it's required, you know, smart people to look at the world and say, okay, here are the inductive biases, let me, you

• 49:21Speaker 01

  know, kind of encode these into the system. And, you know, I think the kind of the bitter lesson of AI is like, when you can, like, you shouldn't encode explicit inductive biases to the, you know, this will lose out relative to learning these from the data, if you've got lots and lots of data and lots and lots of compute.

• 49:41Speaker 01

  So in my mind, there's something, you know, kind of obviously right about the fact that you need prize and you can incorporate data to revise your beliefs. And again, I think Bayesian theory is a good normative theory for that. But that doesn't mean you have to, you know, adhere to the classical tradition of explicitly encoding these inductive biases in. And so I think there's a potential paradigm of saying, well, let's encode prize, but let's try and learn these as implicitly as as possible. Where do you learn these from? Well, it could be something like, you know, the standard paradigm here in in modern machine learning where you're you're learning these biases from, you know, large corpuses of data.

• 50:22Speaker 01

  They could be kind of richer corpuses or corp, corp, corpuses of of data than just internet data, right? You know, there's all of the things that you do as a human. You know, I I can, you know, observe you and and infer some of your beliefs. And so you could imagine like, you know, richer sources of data than just internet data that would allow us to get closer to like the inductive biases that that humans have. So that's, you know, I think it's it's it's tricky, as Kevin said, to actually implement these systems, like you kind of face hard computational problems. But I do think, you know, broadly speaking, you know, doing inductive inference over

• 51:02Speaker 01

  large corpuses of data to learn implicit inductive biases as opposed to explicit hand-coded is a promising path to pursue.

• 51:10Tim Scarfe?

  And Kevin, maybe we should have started with this. I mean, what is an abstraction?

• 51:13Speaker 00

  Well, it means different things in different contexts, for sure. But there's always an element of hiding details. So in programming languages, abstraction is often synonymous for a lambda expression. So it's a function, it has variables, and it's ignoring what values those variables take on. And that is a sense in which a lambda abstraction is an abstraction. In the kind of stuff that sometimes called like causal abstraction in causality, there's also, it's, as I was saying, like kind of analogy or relation between two different causal models. And the more abstract one is the one that's ignoring details but still preserving some kind of essence of what the underlying causal model is.

• 51:57Speaker 00

  So it's a word that means it for these different situations. But the kind of analogy we hear all of them is that there's some hiding of details, but some retention of the essence.

• 52:07Tim Scarfe?

  What if we had a richer ontology to start with? So you know, we're using like these, you know, like asymmetries, rotations, translations, and so on. You know, what if we started doing some galaxy brain stuff like causality and time? And just put some different basis functions in there? Do you think that could have an uplift?

• 52:25Speaker 01

  I guess one thing that's true at the moment about the primitives that we put in is that they correspond, you know, directly to transformations to the ARC grid.

• 52:34Tim Scarfe?

  Yeah.

• 52:34Speaker 01

  Right? So conceptually, I think it'd be cool to add, you know, as you said, these galaxy brain principles, but what do they correspond to in terms of the actual transformation that, you know, that we're trying to construct? I think one actual promising area for kind of newer ARC approaches is precisely abstraction in the model. Let's say the program, the transformation program itself. And so what do I mean by this? So right now, you know, our approach and many approaches, you know, synthesize something like a Python program or literally a Python program.

• 53:09Speaker 01

  And then we apply this Python program to the input to get the output. Right? But that Python program is fully formed, right? It's not abstract. It has all of its details there, right? You can run it and that's great because you can run it, right? You can see if it, if it works. Intuitively, when you solve an ARC problem, at least for myself, I can't speak for other people, you first find some abstract part of the rule, right? You're like, well, I know that this object translates into this other object, but I'm not quite sure like what the actual color transformation is, right? For example. And then I can go from there and say, okay, what could be the actual color transformation? So you know, conceptually, you can think of it almost like you write a sketch of a program in your mind with some holes.

• 53:54Speaker 01

  And then this gives you a direction to try and fill in these holes, right? You might fill them in one way, evaluate, and then go back and say, that's not quite right. We don't quite have at the moment, in just like in the terms of the actual methods that we're producing and other people are producing, these abstract program representations, right? And I think that's something we could actually build, like a representation of arc transformations, which has not all of its detail filled in, but it's still useful as a partial solution on the way to a full solution. And so I think a lot of, I think there's a lot of potential approaches of that form, where we've enriched the knowledge representation. And this isn't quite what you're saying of like building a new kind of ontological ideas.

• 54:38Speaker 01

  It's like saying, well, we could actually abstract our current representations, and that could be a powerful thing to do.

• 54:43Tim Scarfe?

  Kevin, when you solve arc puzzles, can you talk through your kind of conscious strategy?

• 54:51Speaker 00

  Yes, sometimes it's very intuitive, and I can't quite describe in words exactly what I'm doing precisely. And it might be something more like, well, I just kind of denoise the input and imagine like what it should be in the parts that I can't really see. So there's some things that are definitely just perceptual and difficult to say precisely, except saying, well, you know, you just kind of see, kind of denoise it. Other times, it is a very systematic thought process. I kind of jumble up in my head different ways of seeing it. I see if it looks like it's on the right track. I have sort of half-formed hypotheses.

• 55:33Speaker 00

  It's a much more perceptual and much more dynamic process than just the Greenblatt style of spamming out thousands of programs, which, I mean, to be clear, we do that also. So I do think it's sometimes a bit dangerous to introspect too heavily, a little misleading. But I think even if you look at the kinds of mistakes that people make, they don't exactly like the kinds of mistakes that these AI systems make. And that means that maybe, you know, there's something about the dynamics of how we're constructing the solutions that we're not really capturing with any of these approaches.

• 56:09Tim Scarfe?

  Well, I suppose like just comparing transduction and Induction. One thing I I think is good about the induction is that it's more compositional. So I could, you know, mix and match programs together. It does. It doesn't intuitively make sense to me what would happen if you composed the transduction model. It kind of feels like it wouldn't compose very well, but but we could take this this convolved program. And instead, just think of it as as functional programming, so like a data flow graph or something like that. And having the those data flow filters as first class in the algorithm seems like a good step to me.

• 56:40Speaker 01

  Yeah, so you could imagine

• 56:43Speaker 01

  almost like iteratively applying, you know, the induction model and the transduction model, you know, in sequence, you know, all parts of it. Assuring that, you know, the types match, you know, essentially.

• 56:55Speaker 01

  And you can also imagine almost like a like a REPL style approach to trying to solve ARC, right? So I suppose that, you know, you were given an ARC solution, also an ARC problem, and you had a, you know, a REPL, like a, you know, an interpreter environment. And then you could like write, write code, right? Evaluate, check, write more code. So instead of like creating like one big transformation that you run, you do it in a more step by step process where you're kind of continuously analyzing your current solution, writing some more code, checking. So I imagine there's, you know, some approaches to ARC and we're pursuing some of these, which look a little bit more like that. Where you're doing a step by step process where each step is producing code and that code could be

• 57:36Speaker 01

  normal Python code, or it could be a transduction like transformation, ultimately to get to a solution at the end, right?

• 57:43Tim Scarfe?

  Yeah, I I love that idea. I mean, I um, I think there's something powerful about iteration. Something really magic about sort of like refining a solution over time. What do you think about that, Kim?

• 57:54Speaker 00

  I strongly agree. So I think that in to some sense, in some sense, you don't need it maybe so much in ARC because you're solving just one problem at a time. But if you think about it, agents in a world that's learning how to interact with many different causal mechanisms, then your agent needs to accumulate knowledge over time. It needs to revise its beliefs. And so if you had something which was more factored, like the DAG that you were saying, or just anything that breaks up the knowledge even more compositionally. So it's not just one program. It's lots of little programs that are cooperating. Then I think it would also be, it would probably be better at ARC. And also would be closer to what you need for something that can grow its knowledge over time.

• 58:39Tim Scarfe?

  So Zen, some of these solutions to ARC, you know, we had, we had Greenblatt's and of course we had other paper with Wending and Kevin. Essentially, there's this expansion where we do loads and loads of test time computation. And I think in your paper, Kevin, you kind of justified it as, you know, like amortized dreaming. I guess the question is, is it in the spirit of ARC, right, to be doing like this, this massive expansion and all of this computation? Is that, is that what Shalay wanted?

• 59:09Speaker 01

  Well, I think we'll have to ask Shalay what he wanted.

• 59:13Tim Scarfe?

  Maybe we should. But what, what would you think he would say?

• 59:16Speaker 01

  I think, you know, Shalay said several times that ARC is an imperfect benchmark. I think there are ways to try and solve ARC, which don't necessarily lead to the fundamental insight that you might want or that Shalay might want. And there are ways which are more fundamental. And I think the approach that we've taken is like a mixture of both, right? We've certainly got some ARC hacks in there to try and make it work. And there also certainly is some fundamental ideas in there that we're trying to pursue. And so, you know, in terms of this kind of expanding horizon, I think, you know, the more that you're trying to specialize to the particularities of ARC and, you know, build a DSL by, you know, going through and saying, okay, this is a, this is a useful element of ARC problems.

• 1:00:01Speaker 01

  Just include that. This is another useful element. I think the more you're diverging a little bit from the essence of the intention of ARC. But there's an open question of like, how much of that is necessary, right? Like you need inductive biases to solve ARC. They have to come from somewhere, right? And so, you know, I asked Franz Schalay this several months ago. Like, do you think that a kind of a tabular raster approach? Could solve ARC just from the, just from the examples within ARC, right? Is there enough knowledge, information in the ARC dataset as a whole? He thinks there is, right? He thinks that, you know, if you condition on all of the ARC problems and that's sufficient to solve ARC,

• 1:00:45Speaker 01

  like you don't need to kind of pre-train on the Internet of Data. But it's kind of empirically true that the best solutions are at least partially kind of pre-trained on the, you know, the Internet of Data.

• 1:00:59Speaker 01

  So, yeah, I, you know, I think that the ideal solution to ARC would be kind of simple and elegant and, you know, and wouldn't require lots of ARC-related hacks and tricks. And I think a robust way to try and get there is to kind of introduce other problems which are kind of related or in the spirit of ARC or capture some of the same things, but are not quite ARC and kind of force solutions which work on ARC. And this, and that, to kind of push against kind of ARC-specific kind of domain hacks that humans might encode.

• 1:01:36Tim Scarfe?

  But Kevin, if you could design a better ARC, what would you do?

• 1:01:40Speaker 00

  Well, that's what we're trying to do.

• 1:01:44Speaker 00

  Yeah, yeah, so this is part of this new project, Mara, that Zen and I are doing.

• 1:01:51Speaker 00

  So we want to have something which is, in a lot of ways, in the spirit of ARC. So we're trying to learn something from very few examples and then generalize new situations, but where you get to interact with something. So it's not quite an MDP, it's not like reinforcement learning, it's more like a model-building exercise.

• 1:02:12Speaker 00

  And I think that this makes it a lot harder, in some ways, to just generate synthetic problems, which we did, because it's a little trickier to generate lots of synthetic, interactive environments. You could do it, and I'm sure that we will try to do it.

• 1:02:31Speaker 00

  But it's at least one way of introducing a forcing function that causes you to not overfit so much dark.

• 1:02:39Tim Scarfe?

  So philosophically, transduction and induction seem like jewels to me. They seem like, certainly from an expressibility point of view, the function space is the same. So why do we see empirically differences between the two classes?

• 1:02:53Speaker 01

  A lot of it just comes down to representations of models or transformations. And again, I think programming languages are a good way to think about it, right? Where a neural network is a program in a class of programs, which is the class of neural network programs, right? And a Python program is obviously a program. And for any particular representation, there are some things that are going to be easier to encode and some things that are going to be harder to encode. If the languages are universal, then everything's possible, right? But some things are easier and some things are harder. And it turns out to be the case that, at least in some class of ARC problems, like some are easier to encode as Python programs

• 1:03:37Speaker 01

  and some are easier to encode as like direct neural transformations. But again, I think it is important to like separate two different distinctions within transduction and induction. One is what I'm just talking about now, right? The kind of programmatic representation. And the other is, you know, what is the kind of the type of the objects, the type signature of the object that you're constructing? Is it a function from, you know, is your system producing a function which takes as input an ARC instance and outputs an ARC instance? Or is it a function that takes as input the entire ARC problem and then directly produces the solution, right? And I think, you know, we have somewhat confounded those two things within the submission, but you could separate those things

• 1:04:20Speaker 01

  and explore the different kind of combinations. And I think that would lead to different trade offs in different ways. So, so, yeah, you know, I. I think maybe to be a little bit more concrete, I think obviously Python programs are good at expressing loops, deterministic computations, things where you've got like maybe even unbounded or variable bounded number of computations, whereas a transformer model is like a finite model, right? It has, you know, it has one pass, right? And it produces an output. And there's a lot of interesting work showing that this corresponds to a particular class of computations, and there are things that you can express within that and things that you can't. So yeah, I think a lot of it comes down to representation of programs.

• 1:05:00Tim Scarfe?

  Yeah, interesting. Because I suppose Kevin, like philosophically, they're jewels, but as Dino was just saying, from a computational point of view, one is a transformer. So it's a finite state automata in the in the class of automata. And a Python program is Turing complete. But they are fundamentally different. But there's this weird kicker that the types of things that this, you know, limited form of computation and neural network can do can express programs that no human knows how to write.

• 1:05:28Speaker 00

  Yeah. So that I don't really have a theoretical handle on. I can't justify theoretically why a neural network should be able to do computations that are, you know, really tough to do in Python. It's empirical fact. You know, obviously, Python can do things that neural net is going to really struggle with. So as I was saying, like in the paper, we found things like, you know, counting, or we need to systematically do the same thing to every single object. Like Python's great for that. And then the other stuff, it's hard to really theoretically justify that. I can just say it's empirically, it's definitely true. Like, we ran the induction transduction model with different random seeds, we learned

• 1:06:13Speaker 00

  to make sure this was not just an artifact of randomness. It really is just empirically the case that there are certain kinds of art problems, I think many other kinds of problems, where Python in principle could do it. But a neural network is just much better for the job.

• 1:06:29Tim Scarfe?

  I think it's both of you. I'm not sure. But there's this program induction by example paper, which was at NERIPs, spoke to Wendy about it. Are you both on that? Or was that just you, Kevin?

• 1:06:38Speaker 00

  It was me and Mading.

• 1:06:39Tim Scarfe?

  Oh, amazing. Tell us about that.

• 1:06:42Speaker 00

  So that paper, in some sense, was our first attempt at trying to do something that was like wake-sleep in the DreamCoder style, but in a modern LLM setting. So what it does is it starts with some human-written programs on the order of 10 to 100. And this defines implicitly a generative model over code, because you can prompt an LLM with these example programs, and then it will make up similar programs. So this means we have essentially a forward model. We can imagine programs. And this gives us the dreaming phase of sleep, because we can imagine these programs,

• 1:07:23Speaker 00

  run them, see what they do, and then train a program synthesizer based on these programs that we generate. And we found that just doing this was pretty good at program synthesis problems. It was frankly a lot more effective than most of the kind of lambda calculus stuff that I did in my PhD. And it substituted a lot of symbolic machinery with neural machinery. And in doing so, it just was able to take advantage of a lot of the advances in scale that we've had in the recent years. So that thing that I've described so far, where we make synthetic data and then train a synthesizer on it, that was what gave us the induction model for the ARC paper.

• 1:08:06Speaker 00

  In the paper with Wangding, we also introduced a full wake-sleep cycle, where it then tried to solve problems, and then it remembers those solutions, and then it dreams about variations of those solutions. And this means that if your prior is mismatched to what you really care about, so imagine you only think you need to write short programs, but the real world has long programs, then you can pre-train on short dream data, go out and encounter the harder problems, solve a few of them, and then during the next sleep cycle, you'll fine-tune your model to this new data distribution. It doesn't have the library learning component of DreamCoder. It has this kind of softer neural in-context learning type of turn-to-model.

• 1:08:51Speaker 00

  But we're trying to bring this like wake sleep cycle into program synthesis, but in a more kind of modern, more scalable setting.

• 1:08:58Tim Scarfe?

  Sin, and one thing I'm seeing here is pragmatism prevailing. So the connectionists are embracing hybrid models and neurosymbolic and so on. And maybe even like folks from your camp, you're kind of embracing connectionism as well. What are your reflections on that?

• 1:09:15Speaker 01

  So I think there's, as Kim said at the start, there's something obviously right about scale, right? And there's something obviously right about learning, right? Like a connective architecture allows you to learn from large amounts of data.

• 1:09:30Speaker 01

  And there's something obviously also right about, you know, symbolic architectures in the sense of most of our modern society is built on top of them, right?

• 1:09:40Speaker 01

  And I think there's also something right about, you know, you know, general normative principles of intelligence.

• 1:09:47Speaker 01

  So given that, then the question is like, what should you do?

• 1:09:51Speaker 01

  And can you take the things that we know to be right and kind of compose them in a way that makes sense? And I think you're seeing that from a variety of different ways, right? So from, you know, let's say, you know, the large labs, right? Like, you know, OpenAI, they have, you know, large models that they've trained on huge amounts of data. And again, but they have these kind of scaffolds around them that these models call out to Python to do some computation. And why is that? Why is that a smart thing to do?

• 1:10:22Speaker 01

  Because Python's an effective language for doing a certain class of computations, much more effective, you know, than a neural network for a large class of things, right? And so they've come to a kind of business decision, right? That is, you know, it's useful to have this kind of hybrid system, right? From the other perspective, right? You know, there's a kind of a history of symbolic systems. And it's, you know, from a kind of a different perspective, I've realized that, you know, you can't model the entire world through a list of propositional formula, right? The real world is a lot more messy and complex than that. And so we want to do kind of nice reasoning, but we have to handle the complexity of reality. And so this kind of immediately leads to architectures which can do both of these

• 1:11:06Speaker 01

  two things. And so I think just both the complexity of the world and what we want from, let's say, reasoning and learning systems is leading to kind of a convergence of ideas. Right now, I think that convergence is kind of composition, right? Taking these systems, kind of plugging them together. And I think the real question is, can we do better than that, right? Can we go from the ground up and re-engineer systems that have the functional things that we want, but may not look architecturally like the systems that we have today?

• 1:11:37Tim Scarfe?

  Yeah, I'm very excited about that. Building, building systems with L L M's with the best components of everything. Zenna, you're Building Basis. Can you tell me about the Everyday Science project?

• 1:11:49Speaker 01

  Yeah, so Project Mara is a project within Basis. We have a few projects within Basis, but Project Mara is a new project and it's led by myself and Kevin. And so this is a three-year program where we're trying to build upon many of the ideas that we've been talking about today. Really focused on, I guess, four components, right? The, which is what Mara stands for. Modeling, abstraction, reasoning, and agency or acting, right? And so, you know, as a kind of a first approximation, you could think of it as like active ARC, right? Like, can we build systems that model the world, abstract the world, reason with these models, or reason to find these models, but they do it in an interactive

• 1:12:33Speaker 01

  way, right? They have to kind of be part of the environment to, and take actions to learn about how that environment works. And so the way that we structured it is to do two major class of things. One is to develop new benchmarks and new kind of problems to solve. And the other is to develop new algorithms to solve those, those problems. And the first thing we've done is to take an existing benchmark, which is ARC, and do our best effort to try and solve that, right? And we'll continue to do that, most likely. But we're also developing new benchmarks. And so within, you know, this kind of high-level goal of building what you might call like a general Mara system, right? A system where you can plug it into the world and it takes actions to learn about the world, build an internal model.

• 1:13:15Speaker 01

  We're focused on a narrower subset, which is what we call everyday science. And again, we've discussed this today in various contexts, but the intuition is, you know, there's real science, right? That's what we do as, you know, chemists and biologists and computer science, you know, learning about the kind of the physical and the artificial world. But then there's also everyday science, what we do as just like normal humans, adults and children, right? We learn about, you know, a new AC system in my hotel or a new microwave, right? Or a new toy. And we think that the principles underlying this everyday science, like how do we form hypotheses? How do we revise those beliefs? How do we take actions to learn about how the world works? Are the same principles in everyday science that apply to real science. And so we want systems that can do that, that can learn how to interact with new toys

• 1:14:00Speaker 01

  and devices and interfaces in a way which isn't, you know, large scale imitation learning, right? Which is like thinking, going to your earlier question, right? You know, as part of a kind of a, well, I would say like a real approach to AI to science, even if it's not actually discovering, you know, new, you know, new kind of useful science. And so, yeah, so joint project and ARC was our original or our initial kind of output, this ARC solution, this transaction induction model. And we're just at the start of kind of planning a whole, you know, program and building a whole team. And maybe Kevin can kind of give his, his flavor and interpretation of what we're doing.

• 1:14:40Speaker 00

  This is very ARC-like in the sense that you're building a model on the fly from very small amounts of data, but you are not passively receiving the data. You have to go out there and poke things and push things and try things out. So Zen and I both did grad school in the cognitive science departments. And I think as a cognitive scientist, well, cognitive science adjacent, it's very exciting from that perspective. Like this is the kind of thing that is science-like that humans do in everyday life. But I think it's also practically important because increasingly we're building these AI agents, both in digital and physical worlds.

• 1:15:25Speaker 00

  And they, they work well as long as.

• 1:15:28Speaker 00

  You know, their prior is well aligned with the kind of environment they're already in. But when they're faced with a new kind of webpage, or if you imagine like a robot that has to like, figure out how to use like a new kind of dishwasher and has to experiment with different buttons. That's actually quite hard. Like, the more abstract kind of knowledge you need to learn, that, I think, is a really exciting challenge.

• 1:15:49Tim Scarfe?

  Before, before we go, I mean, are you looking for investors? Are you looking for researchers?

• 1:15:54Speaker 01

  We're certainly looking for researchers. If you, if you're a research scientist, a research engineer, and you want to work on hard and interesting problems that are, you know, a little bit outside of the mainstream of what some of the larger labs are doing, then, you know, we're, you know, get in touch. Investors, that's a little bit more complex. Certainly, you know, Basis is a nonprofit, and so if you want to donate, feel free, you know, feel free. But, you know, the project is funded and it's, you know, we have a, we have a kind of an ambitious three-year program. We can obviously make it more ambitious, but I think that would already be quite hard. But yeah, we're just excited to get things started and move, move quickly and find like the best possible people to, to work with. So also, you know, other collaborators.

• 1:16:38Speaker 01

  If, if, if you're doing adjacent things, that'll be a cool, cool area to connect with.

• 1:16:43Tim Scarfe?

  It was such an honor to meet you guys. Thank you so much and keep doing the great work.

• 1:16:47Speaker 00

  This was fun. Thank you for having us.