AI Is Learning at the Wrong Level of Abstraction — Matthieu Wyart

This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlstWhy can deep networks discover abstractions that shallow models miss? Statistical physicist Matthieu Wyart joins Tim Scarfe to argue that the answer lies in the hidden hierarchy of data. Language and images are built from parts within parts; depth lets a network recover those coarse-grained variables and escape the curse of dimensionality. The conversation moves from jamming transitions and rough loss surfaces to Chomsky, context-free grammars and machine creativity. Wyart explains why next-token prediction can still recover compositional structure, where current systems fall short of genuine scientific invention, and why predicting latent representations rather than raw tokens could make learning far more sample-efficient. They also examine diffusion models, neural scaling laws and the limits of physics-inspired theory. The final question is on a personal note: if mistakes are the price of leaving the beaten path, how much scientific risk is worth taking?---TIMESTAMPS:00:00:00 Can machines learn abstractions from data?00:02:00 Notion agentic workspace00:02:49 From statistical physics to machine learning00:06:40 What physics can explain about learning00:16:37 From Carnot to Chomsky bulldozer00:21:21 How deep networks recover hidden hierarchies00:32:43 Where machine creativity still falls short00:40:48 How deep nets escape the curse of dimensionality00:52:19 Why predict latents instead of tokens01:02:49 The sample-efficiency case for latent prediction01:08:31 Diffusion, scaling laws and text entropy01:16:40 The scientists we learn from and the mistakes we make---REFERENCES:person:[00:00:43] Noam Chomskyhttps://linguistics.mit.edu/user/chomsky/tool:[00:02:08] Notion Developer Platformhttps://www.notion.com/en-gb/blog/introducing-developer-platformpaper:[00:04:43] Mastering the game of Go with deep neural networks and tree searchhttps://www.nature.com/articles/nature16961[00:05:52] Reconciling modern machine-learning practice and the bias-variance trade-offhttps://arxiv.org/abs/1812.11118[00:25:54] How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Modelhttps://arxiv.org/abs/2307.02129[00:42:12] Efficient Estimation of Word Representations in Vector Spacehttps://arxiv.org/abs/1301.3781[00:52:46] Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturehttps://arxiv.org/abs/2301.08243[00:52:54] Learn from your own latents and not from tokens: A sample-complexity theoryhttps://arxiv.org/abs/2605.27734[01:08:31] A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Datahttps://arxiv.org/abs/2402.16991[01:11:39] Scaling Laws for Neural Language Modelshttps://arxiv.org/abs/2001.08361[01:12:17] Deriving Neural Scaling Laws from the statistics of natural languagehttps://arxiv.org/abs/2602.07488[01:13:34] Prediction and Entropy of Printed Englishhttps://ieeexplore.ieee.org/document/6773263---LINKS:Download PDF transcript: https://app.rescript.info/share/f7644cdaa86c5cc1e41e484e290f2bd4

• 0:00Speaker 00

  I am Matthew Wyatt. I'm a full professor at John Hopkins University in the US and at EPFL in Switzerland. Those machines that can build new images that we've never seen before, or say new sentences that were never heard before. Our brain seems to, you know, to learn languages with 100,000 times less words than machines. Why is it so? Are we doing the wrong thing? And I'm very interested in, you know, should we predict in token space at a very low level? Or more, should we train machine to predict abstractions? And so what we've been doing over the years is trying to build a framework based on physics that's really tried to answer those different questions in a unified manner.

• 0:43Speaker 00

  Chomsky gave this poverty of stimulus argument, you know, arguing that it was actually impossible to learn, to become creative from example. But if you have a deep architecture, there is a huge implicit bias to build those coarse-grain variables. And so if you think about LLMs or diffusion models, the way they build concepts, they emerge from statistics alone, this abstraction, they emerge, they are there in the data. They emerge. And those concepts emerge if you group together configurations that predict similar context around them.

• 1:25Speaker 01

  And this is very pertinent because you've got a paper out basically saying that we should predict in the latent space, not the token space.

• 1:32Speaker 00

  So again, in those models, what we found is that those algorithms that are introspective, that learn from their own latent, are much more powerful in terms of sample complexity. They will eventually, they will learn the same abstraction, but much faster. If you never do mistakes, maybe it's a sign that you're staying a bit on the beaten path in science. And some of us want to explore the jungle. And the jungle, you can be wrong, I mean, yes.

• 2:00Speaker 01

  Quick pause. Agents are getting smarter every day. But even the smartest agents get stuck without the right context and the right tools. That is where Notion comes in. With the recent launch of custom agents, Notion became the collaborative AI workspace where teams and agents work side by side. And now their new development platform is turning that workspace into infrastructure developers can build on. Now, this is exactly how I run MLST. The whole show lives in Notion. My guests, the publishing calendar, the commercial side, everything is in there. But what's changed is that it's now agentic. I just talk to my agent. It can be Claude or any agentic harness. And then it then talks to Notion via the MCP or the CLI.

• 2:45Speaker 01

  And it's just done. And then I can access it on my phone. It's an absolute game changer.

• 2:49Speaker 00

  So yeah, I'm a physicist. Actually, I really, I really liked learning physics because you have to deal with, you know, nature at all possible scales. And I focused on one specific field in physics, which is called statistical physics. And statistical physics is essentially the field where you try to understand how many entities, particles, interact together to do collective phenomenon. And so a classical example is, you know, you take water, you cool down the system, and at some point, boom, it freezes, completely changing its organization. And so I started to do, to work on that initially on the stock market, where you have interacting agents that influence, you know, the evolution of the price,

• 3:31Speaker 00

  which is a very interesting sort of random walk. And then I went to study complex systems. So complex systems are physical system with a rough energy landscape. It means that if you're, it's a bit like if you're flying above the Alps, like you just did. If you are throwing a ball in those mountains, it could, you know, stop at many different points. So the energy landscape has many metastable states. And those systems are really intriguing. They're a physical system with memory. I worked on several of those, for example, sand. So what's beautiful about sand is that it is a complex system. If you prepare 10,000 piles of sands, each of them is different, you know.

• 4:17Speaker 00

  But it has intriguing, again, phase transition. So as you know, if you tilt a layer of sand, at some point, it's going to flow. It means that the energy landscape was rough and you are in a metastable state, but you tilted this energy landscape, you had a phase transition, and then the entire system flowed, although it's very dense, particles managed to avoid each other. And so I've been very interested in, you know, understanding geometrically those questions. But then like nine years ago, I'm a Go player, a pro Go player, but I enjoy playing. And I was mesmerized by, you know, AlphaGo and so on. And so I started to think about machine learning. And I started to think of it as a complex system.

• 5:02Speaker 00

  And it is because when you train a machine, you know, you build a function that is low if you fit well your data. It's called the loss function or cost function. And so we are very intrigued by what is the geometry of this landscape? And so what we discover is actually that this landscape has exactly the same phase transition as sand. It means that when you're actually underparametrized, when you don't have enough parameters, you have a rough landscape with many metastable states. And if you train your machine and you train it many times, it will end up in different position where it's actually stuck. But if you have enough parameters, then suddenly the system can flow.

• 5:42Speaker 00

  I mean, your landscape has many flat valleys and you can, which have essentially zero energy. So there is really a close analogy. We discovered, we discovered that like nine years ago. At the same time, others find a very similar, I mean, the same phenomenon and called it double descent. So now that's name has stuck. But this double descent, this peak of the double descent is really for physicists, a jamming transition. So yeah, so that brought me to machine learning. And just maybe to, to finish with that, I mean, we, in the last four years, we've been very much interested in another landscape that I think is even more interesting. It's a landscape of data.

• 6:25Speaker 00

  So if you think about, you know, an image, let's call it X, an image, it's a vector. You could ask, what is the density of those images, rho of X? And this question relates to what is the structure of the world? And we think it's key to actually understand how machine work.

• 6:40Speaker 01

  Does it make sense to talk about, because obviously, you know, you're a physicist and you're applying this lens of analysis to large language models? And like naively, I'm looking at this and saying, well, that there's, it doesn't feel like a material substrate. It doesn't feel like it has the same type of dynamics as things do in the real world. But, but indeed, when we look at the training dynamics of LLMs and when we look at the types of representations they learn, we could adopt a physics lens and say there are coarse granings and there are phase changes and whatnot. Do you think it's coherent to make that analogy?

• 7:13Speaker 00

  Yes, I think it's a sort of mesmerizing fact of science that some concepts can be applied in so diverse phenomena. I think, I mean, science is essentially built on those kind of analogies. I mean, if you think about Wriggans, who is one of the first to propose that light was a wave. How did he propose that? I mean, he noticed that waves on the ocean could cross each other without interacting. And he noticed it was the same foray of light. And so he made this analogy. I think it's even hard for me to talk about it because I think it's so fundamental that we are always, you know, building our understanding in terms of analogies.

• 7:53Speaker 00

  So for the specific example I gave you about the sand and the loss landscape of machines, I think the analogy is very direct in this case. Because in both cases, what you have are essentially degrees of freedom. In one case, those are the particle of sand. In the other case, there are the parameters of your large model. And in both cases, the system are trying to satisfy constraints. So for sand, essentially, the particles are just trying to avoid each other. But for the parameters, what they are trying to collectively do is to fit data. So the more data you have, the more constraint you have. And at the end, the universality there is what we've argued from physics,

• 8:37Speaker 00

  but applies there, that if you have a problem of satisfability of constraint and you have continuous degrees of freedom that can change continuously, then boom, you have a universality class. So in this sense, yes, there is something universal about those kind of problems. And so, but that's a very, I, that's a very specific example. I don't want to say that everything is always the same. But, but this specific problem of jamming of sand and the one of, you know, loss landscape of machines is very much the same, yes.

• 9:08Speaker 01

  It's such a tantalizing idea because I think it is constraints all the way down. And in evolution, we have like naturally convergent patterns, reoccurring patterns like carsonization. And I guess the only critique to this is it feels like in neural networks or just in computers, we don't have the same kinds of physical constraints. You know, we don't have, you know, two objects can't touch each other at the same time and the laws of physics and so on. So the constraints are there by dint of, you know, statistical patterns in the data, but they still apply pressure on the training process. Are those still valid constraints?

• 9:50Speaker 00

  Yeah, so here I was really not talking about any sort of constraint of the computer itself. I was really thinking some abstract way about the algorithm, what the algorithm is doing. And the algorithm is doing some sort of gradient descent flowing down an energy landscape in both cases. And that's where the analogy is. The analogy is not really related to the material aspect of it. It's true that in one case it's a material, it's sand. In the other case, it's an algorithm. But it is at some levels the same if you think about it correctly, yeah.

• 10:22Speaker 01

  It's so interesting that you're thinking of the constraints as being the algorithm rather than the energy landscape itself. And I get it. I think you're leaning towards there being some kind of a universal learning algorithm. But the way I intuit it is it's almost like the data and the world are more meaningful as constraints. Is that, is that legible?

• 10:43Speaker 00

  Okay, so I think we will be talking about creativity. And then I will, we'll also be discussing a lot about constraint. But they will be, in my way of thinking, in another space. So I started to tell you, we were discussing loss landscape. And the constraint here was just to fit data. And later on, something I'm really interested to discuss is, you know, if you think about the world itself, you know, sentences, the data itself, forget about the algorithms that's actually learning it. The data itself is very constrained, you know, all possible sentences are not valid in terms of syntax.

• 11:24Speaker 00

  So, uh, so I think thinking of constraint is very useful in both cases, but I think of them as very different kind of constraints.

• 11:32Speaker 01

  And do you, do you still think of yourself as a physicist first? I mean, because now we're talking about the physics of learning, we're talking about machine learning. I mean, how does your instinct as a physicist come across into this world?

• 11:45Speaker 00

  That's a very interesting question. We have to debate it all the time in physics departments because, you know, we need to hire people and what is physics? And, and we always ask this question. And, uh, and it's true, I started initially when I started to think about machine learning, it was closely related, it was more closely related to complex system and disordered solids and things like that. And now I'm rethinking, now I'm organizing conferences with linguists and neuroscientists and so on. But I still think, deep down, it's physics. And I think we need to bring physics to this field and do the physics of linguistics. And, uh, and okay, that's a long discussion, but maybe I, I think what, what, what is

• 12:28Speaker 00

  very special about physics, I mean, other fields, natural sciences do it too, but we're really trying to do that. I mean, first is to build a dialogue between theory and experiments. And so, you know, for example, when there is, uh, and a new technology, it's, it's, you know, a new technology is asking, you know, a huge number of, of novel questions. And, uh, and we can start thinking about it, making theory, and then the theories, we make simple models, and then the theories have to be predictive on what's going on. So first of all, we are very good at building this dialogue, I think. So building some empirical science. I mean, the second aspect is modeling.

• 13:11Speaker 00

  Uh, so the world is super complicated. You know, if you try to make a map where one mile is one mile will never help you. So you need to build caricature of the world. Uh, and it's an art that physicists have done. I mean, to paraphrase Einstein, a model should be the simplest, but no, not simpler than that. So it means there's tension, it's really actually difficult to describe the, the problem at the good, you know, level of, of complexity. And it depends also specifically what question you're asking. So, so I think physicists are also good to, to develop this sort of models. I mean, um, and, I mean, just to give you an example, I told you about phase transition.

• 13:54Speaker 00

  I mean, one century ago, Pierre Curie was thinking about magnetism and the fact that when you change the temperature, you know, suddenly, uh, those materials become magnet and they stick to your fridge, but at higher temperature they don't. So what's going on? If you think of it at the microscopic level, it's awfully complicated quantum mechanics. But you know, the description that stuck and that made huge headways in terms of phase transition and led to Nobel prizes, meta fields in math and so on is very simple models, the Ising model, where you have essentially arrows on the lattice that are interacting with their neighbors to, to try to align. So a very crude description, uh, of the phenomenon was a good one to essentially think about

• 14:39Speaker 00

  this problem. So, so I think that's also what we can try to bring to that, to, to those questions. I mean, um, and then lastly, I think what you are describing analogies like Wigan's. I mean, we are, we're physicists that try to think about problems at all scales and to do it fast, they had to build analogies between different fields. So, so I think that's, that's what we can bring.

• 15:05Speaker 01

  I mean, when we spoke with Chomsky, he was quite disparaging about the enterprise of physics. He, he was talking about, you know, the original hard problem, motion. And he said that Newton exercised the machine, you know, the, the mechanical universe view, uh, but he left the ghost intact. So, you know, we still don't know how mind and consciousness works and, and all of that kind of stuff. But he was rather pointing to this notion that, um, a lot of physics is, is idealization. And, and then there's an interesting question about whether, um, our theories are really to actually understand how the universe works, or are they more to help, like, to help us understand in terms that we can understand? So are we intentionally leaving something out?

• 15:43Speaker 00

  Yes, I, I, uh, I think both, I mean, both. I mean, certainly they are interacting with the world because those theories, theories in physics are super useful to build technology. I mean, think about the laser, I mean, I mean, half of the technologies are, I mean, a big fraction of them are coming from, from theory. Um, but another big portion of it is coming from, I mean, it's the other way around. Technology, you know, is asking, uh, immense questions. So, so yes, I think we need theory also to have a sort of highway, build your highway of, of thinking super fast about problems.

• 16:25Speaker 00

  Um, and then, I mean, then the, the difficult question is at which, yeah, level of precision do you need your theory to be? And that depends the question you're asking.

• 16:37Speaker 01

  And who, who were your mentors? Who inspired you? You know, what books did you read? How, how did you kind of land on your current trajectory as a physicist?

• 16:45Speaker 00

  Well, yeah, that's a complex question for me because I have, uh, my two, my two parents are physicists actually. And, uh, and when I started to do my PhD, I, I tried to escape, uh, to escape them by going to econophysics. I mean, doing more finance and economy. And then already during my PhD, I started to be fascinated again by physics and how sand flows and, and things like that. Um, and then when I was a postdoc, actually, I was always mesmerized by our brain, how we think. Uh, and, uh, and I, I tried at that time to, uh, you know, I even spent one year in Geneva Farm, a place where, you know, a neuroscience institute. And, and I met lots of fantastic people and I learned a lot.

• 17:28Speaker 00

  But I felt at that stage that a lot of the theory, you know, how, you know, you have many connected neurons, what their dynamics were, a bit applied math and detached from really function and then the question of how do you learn intelligence or rules, constraint language and so on. And so I was not, I was not at the, it was not at the level that I really wanted to operate. And so I gave up, I, I went back to physics. And then, uh, I mean, I think it's essentially the development of the technology. It's, uh, it's facing us. So, so I think the analogy I, one analogy I like is, uh, you know, the industrial revolution

• 18:09Speaker 00

  when, uh, when actually, um, heat engine emerged before. Again, one case where technology was first and, uh, and then you, you had to understand, you know, what's behind them, how efficient can they be, the limit to their efficiency and so on. And so then Carnot, actually a French physicist, uh, came up and wrote, you know, beautiful theories like philosophies, essentially no math and interesting concept like entropy. And it was the beginning of thermodynamics. I mean, uh, very deep ideas coming from some, uh, technological facts. And here I think it's the same. I mean, with those machines, it's amazing. You look, they are creative. You, you give them a bunch of images and those diffusion models suddenly like a painter,

• 18:52Speaker 00

  they, you know, they build new faces, they compose new faces. How can it be? Or they create sentences that they've never heard before. And now Chomsky and others said that it would be extremely hard to do. Uh, they do it. So how, why? So yeah, it's being fascinated by questions. Yeah, that's, that's what drives me, yeah.

• 19:13Speaker 01

  I mean, you know, I'm sorry to bring Chomsky back again, but he, he said that LLMs are like bulldozers, you know? He, he says, I love bulldozers. They're great for clearing the snow, but they're not a contribution to science. And he, and he said, you know, he's, I've, I've got a theory, anything goes, right? You know, it explores all the laws of nature, anything that can be. And, you know, he, he says that when you've got a scientific theory, you have to explain why are things this way, why are things not that way? But you were just saying, you know, when, when we discovered the steam engine, I, I think you believed that that actually was the stepping stone to building theory. But, but for Chomsky, there's a huge difference between competence and performance. It's possible, he, he had this wonderful expression about, um, deep blue, you know, the chess thing.

• 19:58Speaker 01

  And he said, that's a little bit like a bulldozer winning a weightlifting competition. So it's, it's almost inconsequential, it's, you know, it's incoherent.

• 20:07Speaker 00

  Yeah, so, so I, I respect him a lot. A lot of our work is actually inspired by him. The way we model data is, uh, is sort of part of the classification he built. So, but, but, uh, but I think what he's saying is not, is not wrong. It is not a theory, but it's obviously not wrong, but it's not the point. I mean, to me, the point is that it's an amazing observation. And this is raising a bunch of questions. You have a machine facing you that can learn to be creative. You can open it up, you can look at its neurons, artificial neurons, and you can ask, how is it encoding syntax, semantic and so on? How is it doing it? You know?

• 20:48Speaker 00

  Even if ultimately the brain doesn't work like that, I would argue, in some sense, I, I don't, I, I would love to, to think about the brain and to have something to say about the brain, but I find intelligence in machine also extremely interesting, you know, in itself. So, uh, so yeah, so it's just like at the time of...

• 21:09Speaker 00

  Of Karna, okay, you know, we're facing with new technology, it's asking many questions, we should, I think, I hope physicists, you're asking about physics, we're pushing physics departments to also, you know, invest in those directions, so.

• 21:22Speaker 01

  Unfortunately, it is possible to make a system perform a function in many, many different ways. Right, so, for example, L L M's, they apparently have linguistic competence, but it's different. And then you might just say, oh, it doesn't matter if it does the thing I want it to do, it doesn't matter. And, but I think we can be a little bit more rigid here. One thing that seems to be missing is the acquisition of abstractions. Now, you've done some amazing work on this, but certainly when I use language models, what's abundantly clear to me, even when we do hill climbing and we we solve these mathematical problems, they traverse the spaghetti monster and they get the right answer, but for the wrong reasons. And they they seem to be low down the abstraction mountain.

• 22:06Speaker 01

  And and what we do is we have the ability to do this coarse graining or to use metaphor and and work higher up the abstraction mountain. And it just feels to me that when we use stochastic gradient descent and we just learn from data, the models aren't acquiring these high-level abstractions.

• 22:21Speaker 00

  That's the opposite, we, I will, so essentially, I think those questions are super interesting, super deep. And we are approaching them as physicists, and so,

• 22:33Speaker 00

  and so, yes, so so the way we were thinking about this question is that, yes, the world is very complicated. So let's stick to language, maybe, and,

• 22:43Speaker 00

  and I, so I think to understand, you know, how machines work, we first need to understand very well, we first need to understand very well,

• 22:52Speaker 00

  what is, what are we talking about, what is the data that needs to be learned? And on that, I would say the linguists have been one of the most impressive field in terms of characterizing the data they are dealing with. And in particular, Shamsky and others, I mean, it has been argued for centuries that underlying tags are trees, you know, a different level of abstraction. I mean, it's also also been argued for images, it's called pattern theory, you know. We are very used to that in physics, in physics, let's say you take a liquid, I told you about liquid before. You can describe it at the level of atoms.

• 23:30Speaker 00

  But if you have, you know, billions of billions of billions of atoms in your glass, it's not going to help you describe the glass so well. And so what we did as physicists is to build coarse-grain variables, you know, like pressure field, velocity, density, things like that. For this system, it's sort of simpler, because there is a single, I mean, there are two levels of description, if you want, essentially, this is very microscopic and microscopic. Real data have layered, multi-scale, you know, level of description. I mean, if you think about an image, you can think of it at a pixel level, at very low level, and at very high level, you could think of it as, you know, the caption that's describing what's on the image.

• 24:10Speaker 00

  And you have very intermediary steps, I mean, at low level, you could start to make from pixel edges and little geometrical figures. And at some point, you could make eyes and nose and ears and understand that it makes a head, that, you know, so so you have many different levels of of description. And so the question you are asking, so that's the one we wanted to understand, and so so the first thing was, how do we model this? Because if you look at the most complicated context-free grammar, so those are the sort of models where you, which are based on trees that Shamsky introduced.

• 24:48Speaker 00

  Essentially, the idea behind those models is that if you want to describe sort of linear objects that language text is, you can describe it by some underlying tree. And you describe, you know, you have hidden variables living on those tree. And you describe the way whereby those hidden variables can give rise to strings of hidden variables. So essentially, you're describing a generative way to make sentences in a recursive fashion. But again, if you want to sort of feed those models to English, it's lots of, it's very complicated. And so so what is a good model? And and again, it depends on the question you're asking, it's not an absolute, there is no absolute answer to that, but for the sort of question we're asking, like the one you're you're

• 25:33Speaker 00

  you're asking, which is, in which sense do we build those coarse-grain variables? Well, we build context-free grammar, so models of data where you have those trees underlying it. But like you, as a physicist, it's a synthetic world, we build it. And then, once we build it, the game we're playing is that we we have to believe that it's rich enough, so in our case, we want it to capture the fact that there is some hierarchical hidden structure to the world, so this we capture. But we want to make it tractable, so maybe it's a bit technical, in this case, we started the the first models essentially had a tree that was frozen in geometry. And production rules, which is how latent this rise to a string, were randomly chosen.

• 26:18Speaker 00

  Finally, randomness, although it's counterintuitive, in physics often make things simpler, allowed us to compute any correlation in this model. And from that, we could actually understand how they are learned by machines. And and indeed, if you, you know, if you have a poor machine, like a very shallow network, first of all, even those models would be essentially unlearnable, okay, so so there would be a lot to say, and maybe I can come back to that in high dimension, it's it's very hard to learn. But if you have a deep architecture, what we find is precisely the opposite of what you're saying, the the reason why deep architecture can solve those tasks

• 26:59Speaker 00

  is precisely because they understand, just like the physicists understood about pressure, velocity field, and so they understand from data, from the statistics of data, this hidden hierarchy. Otherwise, they would never be able to do this job, so they understand that there is this hidden hierarchy. And and from it, they can perform the task that that you want.

• 27:20Speaker 01

  Just just to kind of play that back, just so that everyone understands, the idea is that there is, I mean, we're talking about grammar here, but, you know, more more broadly, we think that there are structured generative processes in the world.

• 27:34Speaker 00

  Yes.

• 27:35Speaker 01

  So so we can think of those as being some kind of a constrained generative model, so we're talking about syntax here. And when when we do machine learning, we we look at the output of that generative model. And the the learning process ideally should be not to memorize the raw output, but it should be to understand abstractly the model which generated it. Because creativity is about respecting the deep structure and the constraints. If you have the structure, you can go on and generate many, many more things, and you obey the rules, you're you're creative and and all the rest of it. So you you're saying that you've done experiments, so you you've created a mathematical generative model, which is parameterized, you can have as much depth as as you want. And you found on shallow networks

• 28:19Speaker 01

  that basically it wasn't really learning any of the the abstract structure, but when you have deep networks, it it was.

• 28:26Speaker 00

  Yeah, exactly, so maybe I can put it into the the context of creativity and this discussion we just had with

• 28:35Speaker 00

  with Shamsky, so so, you know,

• 28:38Speaker 00

  so so there is a question, creativity, I will use this term in a very narrow sense of being able to generate new sentences that satisfy hard constraints, syntactic rules, that, you know, the child would never have heard before. And and Shamsky gave this poverty of stimulus argument.

• 29:01Speaker 00

  You know, arguing that it was actually impossible to learn to become creative from example. And essentially, okay, this would be a very crude way of of summarizing his argument, but I describe the fact that you have the sort of generative tree-like, rich, context-free grammars. You know, assuming that that, and really capturing the fact that the the world has a hierarchy of abstract concepts. And but you have other possible generative grammar, some are much simpler, one is called regular grammar. So this would be a caricature, but essentially, the idea that maybe a group of words will fix the probability of the next word, essentially.

• 29:44Speaker 00

  And Shamsky's argument is to say that, well, even if you give me one million sentences, okay, I can fit those, you know, those sentences by a context-free grammar, but I can also fit them by a much simpler, I mean, a regular grammar, simpler in its classification, but to fit those sentences, it would have to be awfully complicated, you know, many, many, many rules. And and that, you know, nativism and empiricism, big debates on this question, and again, we we we felt like we want to address this question as physicists, so in our idealized world, where the true world is hierarchical and has this hidden hierarchy,

• 30:27Speaker 00

  can machines learn to be creative or not?

• 30:32Speaker 00

  And what we find is that if you have a shallow network, what Chomsky worried about is completely true. You don't learn this sort of interesting generative grammar. You essentially memorize and you can't do anything. But if you have a deep architecture, there is a huge implicit bias to build those coarse-grain variables. This hierarchical architecture leads very easily to some iterative calculation. And so what we found is that indeed, you can learn to be creative by having been exposed to a very small number of sentences.

• 31:17Speaker 00

  So let's say in our model, if D is the size of the sentence, the number of sentences is huge with D, exponential in D. But the number of sentences you need to see to be creative is only polynomial in D. So these models are really a counter example to his argument. And ultimately, it comes from the fact that, you know, machines have strong implicit bias. They are not comparing equally, in an equal fashion, all hypotheses. And if you're deep, you learn hierarchies. So that's a counter example to that. So in some sense, I would argue that

• 32:02Speaker 00

  in terms of what needs to be innate, if you have deep architecture, that does lots of that. This is not to say that, you know, so I'm arguing against Chomsky's argument. It doesn't mean that what he inferred is incorrect, right? It's not because I think an argument is incorrect, that the statement is incorrect. I don't want to imply that our brain is just a deep net and that there are not, you know, much smarter mechanisms to learn, you know, much better. And actually, the brain can learn with 100,000 times less exposition to words than those machines. So I think there are lots of questions about how the brain works, and they are fascinating.

• 32:43Speaker 01

  I think it's possible that you're both correct. So, you know, Chomsky, even back in the 50s, and he wasn't the first to do this, but he came up with these very basic, you know, transformative rules that could be composed together. And that went on for quite a while, but they realized there were lots of problems and edge cases. And then eventually, the minimalist program came out, and it was even more parsimonious. It was like move and merge. And that's very, very abstract. Because I agree with you. These networks clearly have syntactic competence, which means they absolutely are creative because they absolutely can create novel grammatical sentences. But in a broad setting, though, they're not creative because they don't understand many other abstractions in the world

• 33:27Speaker 01

  that we do, which is why we need to prompt them to be creative. You know, they can render an image of a dog, but they're not creative enough to know what, you know, an interesting and worldly coherent image of a dog is. But another thing I wanted to get to is this is touching on the idea that networks, as we train them for longer and they get bigger and deeper, they start to factorize. So, because we have this notion that they have these fractured and tangled representations. They understand things at a shallow level. They understand some things and not other things. I spoke with Tom McGrath at Goodfire. It's a big mechanistic interpretability company. And he's been studying networks for years. And he says, as they get bigger and bigger, they become more factorized. And he actually believes they're converging

• 34:11Speaker 01

  towards some kind of natural factorization. But at the moment, it's a bit weird that they have some factorization and then they have some fractionation in other areas. But you kind of see where I'm going with this? Because we know that a neural network is a finite state automata. And language, in Chomsky's sense, it's somewhere between context-free and context-sensitive. He wasn't super specific about that. But we know as a mathematical fact that it's not a generative grammar in the abstract way Chomsky was describing. But it is still a coherent generative grammar in some lower sense.

• 34:43Speaker 00

  In those models, first of all, what we find is that as you train the machine more and more, those factorization or abstraction are created progressively. And if you have an immense machine and you give me more and more data, then you start to play with more and more abstract concepts. Those are the hardest to learn in this viewpoint. So that's my take on the being of your question. The second question was, yeah, I think you're referring to the fact that transformers have a finite depth. It's a finite depth. And so, I mean, if you are thinking about sentences where you have 50 complements of sentences and that are looping like that, it may be very hard to reproduce and so on.

• 35:26Speaker 00

  But I think those are more like academic. My take is that those are more academic problems that you never encounter in practice because, you know, sentences that loop for 50 times are extremely rare. So I'm not sure it's really, you know, I know that some people put a lot of attention on that. But I'm more, you know, as someone who likes to do empirical studies and so on, I don't know if those worries are actually relevant in practice.

• 35:55Speaker 01

  But would you agree, though, that there is a spectrum of abstraction? So yes, it's a different type of syntactic competence, but maybe it doesn't matter. There's the performance competence type thing. But an interesting example is I feed my entire code base into Claude code. And isn't it interesting that it doesn't really understand what my intentions were? So if I put it in a loop, I put it in an agent, and I say, just fix the bugs and just keep evolving this software. And it doesn't respect that, you know, the deep constraints now are my mental constraints. Like, what was I trying to achieve with this, right? What would I have done? So it has the syntactic competence. It knows how to write the code. And are you saying this is just a matter of the networks aren't good enough yet? When they do understand, you know, when they have a theory of mind

• 36:39Speaker 01

  and they understand how the world works even more abstractly, eventually, we could just autonomously create coding agents that will just make Microsoft Word or something? And there is a path to that level of competence?

• 36:50Speaker 00

  Composition. So essentially, what you're saying is that imagine that you have this diffusion model that learns to compose new faces. Essentially, what it's doing is that when it has seen enough of low-level features, like nose, eyes, and mouth, it understands the rules of the games and it composes them together. And what we also liked about this description is that we can make non-trivial predictions that, you know, we test with real images or with real text. And maybe we'll come back to that because I think it's a really important part of physics. It's not, creativity is not just putting pieces together that satisfy constraint. Although, when you have a new idea, often it's putting existing ideas together into a new hole.

• 37:34Speaker 00

  But I think creativity can be much more than that. I mean, if we think about what we discussed, about what it means to be a physicist and how science proceeds, I mean, it's an example of creativity. You know, if you think about creativity, like Newton understanding the motion of planets and things like that. I mean, we talked about dialogue between experiment and theory. We talked about building models at a good level of description. We talked about, you know, analogies. And I don't think, I agree with you that, for example, all that, I don't think it's in the machine. I think there is no reason why we would not be able one day to build machines that can do that. I'm not sure if just scanning up things will lead to that. I think maybe we need to do more introspection of how we function as scientists to come up with a good data set

• 38:19Speaker 00

  and the good procedures to teach machines to be good scientists. So just an example, like creativity in science requires a lot of abilities to create something really new and how to interact with the world around us that, you know, I don't think machines have and yet. I mean, so I think I agree with you that just scanning up, I don't think will lead to a total success. We need to, you know, have, develop other abilities in those machines.

• 38:48Speaker 01

  Yeah, I think I'm just trying to understand what the gap is because it would be consistent with your argument. If you're saying that we can learn the abstract structure of the world and be generatively competent in one domain, why would we not? Because for me, creativity is not just about coherence and respecting the constraints. I mean, transformative creativity, in my mind, is about discovering interesting new subspaces. So, you know, we can traverse these constraints collectively. And serendipitously, sometimes we happen upon these fascinating new subspaces and we go on to explore them. And when we look back after discovering them, we think, oh, that was a very transformatively creative stepping stone.

• 39:26Speaker 00

  Yes, I think if you look at the history of science, it was very creative. I mean, like, think about mathematicians inventing imaginary numbers or Newton describing the motion of planets or Wigan's and something about waves and diffraction, you know? I mean, humans forever have seen waves entering a port and being diffracted. So, you know, starting to make more circle-like patterns. But, okay, if you're just showing those patterns to a machine, it would sort of stupidly predict the next frame because it understands, I mean, velocity will propagate. But what do we do as physicists? We have to, first, some people are really good at observing that something is intriguing.

• 40:10Speaker 00

  Then you have to simplify the geometry. So maybe you put it in a very simple geometry. And then you have to, then an aspect of modeling that I was describing, how do we model this and so on. And so I think all that is very much needed. And I don't see how you could, you know, it's our interaction with the world and simplifying the world. And that requires an interaction. So I don't think you can learn it just by looking at everything that was ever written without enforcing those interactions.

• 40:48Speaker 01

  And to recap, so we were contrasting before that Chomsky has this poverty of stimulus argument. So he was essentially saying that it's not really possible, you know, realistically, with the amount of data, sense data that children have for them to learn this grammar. And your paper demonstrated that actually it is because you created this generative function and deep networks could learn it. But I want to understand how. So you said, for example, that the networks, that they encounter ambiguity. And when there's a sufficient amount of data which has a bias in it, then the network can suddenly grok it and it can learn this invariance. Tell me about that.

• 41:27Speaker 00

  All right, so this is about how does a machine actually build those coarse-grain variables or abstractions?

• 41:36Speaker 00

  And we looked at it in various cases, supervised learning, where you're trying to classify cats and dogs. And then we went to generative models like next token prediction or diffusion models. And very recently, we went to maybe smarter algorithms that are trying to predict in more abstract spaces. So maybe I can do, start this discussion in the middle. So think about models like diffusion models or L L M's that are trying to predict very low level tokens or pixels or low level features.

• 42:08Speaker 00

  So essentially, what we argue is that, maybe there is an analogy again with a simpler algorithm, which is the word to vec introduced 10 years ago and a beautiful idea.

• 42:23Speaker 00

  And so, and so the idea there was how can we build an interesting vectorial representation of word? To each word, I want to associate a vector. And that was a beautiful idea. What you can do is take this word, make a little machine, a one hidden layer. So you have, you know, neurons, one layer of neurons and train this machine to predict, you know, the words nearby. So essentially, it's a machine that's trained on coherence. How often, how often two words co-occur in the same sentence, let's say, to say it simply.

• 43:02Speaker 00

  And what you realize here is that, you know, if two words are synonyms, they will have a similar context. And so this machine will represent those two guys with the same vectors. So this is, and so you will have, instead of the incarnation of those, you will just have the meaning. So this is for me an example of cause grain variable, right? So, and essentially it's key, because we talked about the fact that.

• 43:33Speaker 00

  Okay, so, so learning in a large dimension should be extremely hard. So, so essentially, it's really important that this machine managed to, in some sense, get rid of a lot of things they don't care about. And to do that, they have to breed those cause grain variables. So word to vec is doing that at one level, at one level of abstraction, at a low level of abstraction. So essentially, what we're saying is that deep architecture diffusion models or large language models, they do exactly that, but in a recursive fashion. So once they, let's say, they have understood the meaning, they will group those meanings into supra meanings. So maybe, let me give an example.

• 44:15Speaker 00

  So think about streets, the concept of streets. So you have, you have passerby, you have cars, you have sidewalks. You have an immense number of possible streets. It would be very useful to have a concept for that, that group all those different configurations together. And that's the concept of street. And so if you think about L L M's or diffusion models, the way they build concepts, they emerge from statistics alone, this abstraction, they emerge, they are there in the data, they emerge.

• 44:50Speaker 00

  And those concepts emerge if you group together configurations that predict similar context around them. So maybe if you have a street, typically you have houses nearby and maybe the houses have colors or edges and so on. You would predict color and edges.

• 45:09Speaker 00

  And with that, in those models at least, you find that if you have enough data, you can learn all the abstractions. But as you get more and more abstract, you have a problem. Because you're always trying to build those abstractions by saying how they are predictive, but at a very low level. And when you're very abstract, you know, how you predict pixels or colors and so on, is a super noisy signal.

• 45:35Speaker 00

  So essentially, that's why in those models, we find, and we have empirical evidence, and I'm happy to talk about empirical evidence, that, that the more abstract concepts are, you know, the toughest to learn. Because essentially, your signal, as you get more and more abstract, your signal gets diluted.

• 45:55Speaker 00

  So this is, yeah, so this would be the mechanism whereby we think you build those latent variables or abstractions.

• 46:04Speaker 01

  Yeah, it's such, it's such a tantalizing idea. And we'll, we'll get to the latent stuff just in a minute, because that's also a great, a great thing to talk about. But are you suggesting that there is some kind of natural factorization? So do you think that different networks, perhaps with different architectures, given the same data, would almost converge towards the same logical factorization of the data? This hierarchical factorization?

• 46:33Speaker 00

  So in our dream world that we invented mathematically, it's true as long as the network are deep. So shallow networks, they don't do anything, but if you have a transformer or C N N and so on, they build essentially very similar abstraction. Actually, in this, those arguments also predict how many data you need to build them. So this is a sort of evidence, we make prediction and that's what we play as faces. We make prediction and we test them. So we predict how many data you need to learn how many different level of abstraction. And we find similar, in terms of scaling, similar number of data independently of the architecture.

• 47:14Speaker 00

  So I, I do, I still do think that different architecture are going to lead to, you know, slight difference and so on, different circuit. But the big picture, I think, is yes, those abstractions actually really exist. And again, they are a set of configurations that predict a similar surrounding. And if they have a strong signal, if they have a strong predictive power, they are formed earlier.

• 47:39Speaker 01

  It's, it's so tantalizing to think of them as being the one true abstractions. But we know, as we said earlier, it's not like the, the merge operator that Chomsky was talking about. When we do mechanistic interpretability and look at how networks do addition, for example, that it's super weird that they're composing trigonometric functions together, they're not doing it the way we can. And maybe that's just a limitation of the architecture. Maybe if we had proper learnable Turing machines, that they would converge higher up the abstraction tree. But, but I suppose a tangential thing that we need to talk about is this curse of dimensionality. So, you know, there's always been this statistical law essentially, that when we have high dimensions,

• 48:18Speaker 01

  the number of data that you need, you know, to, to, to make it tractable increases exponentially. And there were all of these theories about why that's not the case. There's the manifold hypothesis, so you know, the intrinsic dimension is, is lower. We spoke with Randall Bellistiero about this. He's got this blind theory of neural networks. And he said that, you know, in high dimensions, all data is extrapolation, there's no manifold, that's bullshit. It's actually doing this splined decomposition in an input sensitive way. Lots of, lots of people have different ideas about this.

• 48:48Speaker 00

  Yes.

• 48:48Speaker 01

  But you're saying that, that this kind of emergent behavior is actually how it is tractable?

• 48:54Speaker 00

  Exactly. So actually, that's, before we started thinking about creativity, our first work was really trying to understand what, what, you know, sort of data structure allow deep net to actually perform. So it's exactly as you said, maybe I can say it again. So in physics, we know that volume goes like a length to the exponent of the dimension. So in 3D, L cube, in 2D, L square, if L is a length. So think about a large dimension. So if you think about an image, D may be the number of pixels, at least naively. If you think about text, it may be the number of words in your sentence, again, naively. So those volumes are huge, they are exponential, you know, they are exponential in large in the dimension.

• 49:39Speaker 00

  So what it means is that even if you give me one trillion points, because the volume is so huge, they are extremely far away from each other, extremely far away. And so if you have a machine that's just interpolating, and now you ask a question about a new test point, and you can prove mathematically that if the data has little structure, like you're trying to learn to regress some function that's smooth, it's hopeless. I mean, the only way you will extrapolate and have, you know,

• 50:12Speaker 00

  power to, to generalize, is if, you know, you bring those points together. It means you have an exponentially large number of data, you have more data than atoms in the universe. So it's just impossible. So to me, this is completely fundamental question. And, and it's true that sometimes in the literature, it's tossed aside by saying, okay, it's super naive to say that the dimension is a number of pixel in an image. In fact, it should really be that the data lie in a lower dimension manifold. And if you try to measure it, it's true that it lies in a lower dimension manifold. But this dimension is still hard, large in dimension. And, and to me, the big problem is that if this was the answer to this question,

• 50:56Speaker 00

  it would mean that very simple algorithm, like kernel methods are ancestor of deep net or even shallow network, I mean, they do it perfectly. And if you give them, you know, a low dimensional manifold, you don't need to have any interesting architecture. But if you use those on text, I can tell you, it fails lament, I mean, it's complete, does nothing. So, so the question is, why do you need deep architectures? You know, I mean, and, and some of the thing you said is, does not answer that, that question. So that really is a question we are, we are looking after. And so essentially.

• 51:35Speaker 00

  The answer to that is that if the world is hierarchical, if it has those hidden coarse-grained variables, those machines are super good to discover them. And they can discover them with a generically with a number of data that's not huge, polynomial in the dimension, once again. And once they discover them, it's a sort of summary of what the data is. You know, instead of describing pixel by pixel, it's always there's a nose, ears and so on. So you're reducing the dimension of the problem essentially, and you can solve the curse of dimensionality. So I think this explanation has the advantage that it, I mean, whatever explanation you come up with, it has to explain, it needs to explain why you need deep networks.

• 52:18Speaker 01

  Yes, yes. You know, when I was speaking with Tom from Goodfire the other day, he was saying that so much of interpretability is going from essentially neural representations to text, you know, to try and introspect about them. And he thinks we could have a new type of training method where we go from text back to neural representation. So we discover these, you know, emerging modular structures. And during training, we kind of encourage them to be even more pristine, even more evolved. But there are other folks talking about similar ideas as well. So Jan LeCun, for example, he's got this idea called a joint embedding prediction architecture. And this is very pertinent because you've got a paper out basically saying that we should predict in the latent space, not the token space.

• 53:00Speaker 01

  And his idea essentially is that if we actually predict in latent space, then we can be significantly more sample efficient than if we predict in the ambient space. Tell me about that.

• 53:12Speaker 00

  Yes, so I, okay, so that's a question we've been fascinated by in the last year or two. As we just discussed, the brain, you know, learns languages with much less data than machines. So machines are amazing, they speak better English than me for sure. But in some definition of intelligence, they need many more data than us to do those tasks. So, so what's, why are we so different? And so something, I mean, there are many hypotheses, but one thing that's discussed in the field is the fact that, you know, those large language models, at the end, they do something that's seems a bit trivial.

• 53:54Speaker 00

  It's like you mask a token and you try to discover it. Even to do that, in our models, you find that you need to, if you want to do it well, you need to understand the full, the fully hierarchical abstraction of the world, even to do that well. Actually, so that we started to work on next token prediction, because I always had for at least one decade, this sort of question related to the curse of dimensionality, which was, you know, how can when we produce speech, think about the end of a sentence, maybe I said 30 words before, the number of possible sentences is huge. How do I, do I need to memorize the 30 words to do that? I mean, how is it possible?

• 54:38Speaker 00

  And actually, those models gave a sort of elegant answer to that, because what happens when you try to predict the next token is that you will, if you said a long sentence, maybe you would have, you would have a coarse-grained variable that describes a coarse meaning of the first half of the sentences. And as you approach what you're going to say, you have a finer and finer, more precise description. So at least to me, this sort of way of thinking led to a possible solution for my paradox. But so, in any event, even if you try to learn the next token, you need to build those very, those abstractions. But I told you that one problem with doing this is that if you're very abstract,

• 55:20Speaker 00

  it needs a lot of data, because you build those abstractions by bringing together configuration in the data that predict a similar surrounding, but at a low level, like similar pixels around. So going back, so, so what has been proposed in the literature, actually, it's interesting, also in neuroscience, there is this notion that maybe the brain is doing some sort of very interesting self-supervised learning, where instead of just predicting what's going to be the next frame on its eyes, it's trying to predict the next activity of its cortex. So predicting in some sort of abstract space. And this idea has also emerged in machine learning.

• 56:03Speaker 00

  And you talked about Jan LeCun, and there are also other models. And they're extremely interesting. And again, the idea is instead of predicting at the level of token, can I predict in more abstract space? And they developed this field, developed very interesting machines to do that. I mean, you can think about twins, so you have one machine, you duplicate it. And one machine is shown the entire data, it's a teacher. And one machine is shown some occluded version of the data. And your student has to predict not the tokens that were occluded, but how those tokens were represented by the teacher that could see them. It's beautiful, right?

• 56:46Speaker 00

  It's some, like those networks are doing some kind of introspection. And, and there's been, you know, a big debate, is it better or not? Because after all, those LLMs are doing fantastic things. And, and so we felt that, and there was essentially no theory on that, on sample complexity. And, and so we felt we needed to think quantitatively about this question. And so, again, with the same kind of model, so the game we're playing is to develop a framework where with a single viewpoint, you try to engage with many different problems. So curse of dimensionality, you know, creativity, and now learning from your own latent. I mean, also scaling those, maybe we'll talk about those.

• 57:30Speaker 00

  And so, again, in those models, what we found is that those algorithms that are introspective, that learn from their own latent, are much more powerful in terms of sample complexity. And they will eventually, they will learn the same abstraction, but much faster. To build abstraction, you need to bring configuration. Think again about the street. All this configuration, you need to understand it's one entity, a street. And what diffusion or next token prediction do is that they, the signal they have to bring those together has to do with pixels around, very low-level features. And I told you that this correlation between abstract things and things that are not abstract,

• 58:14Speaker 00

  it's there, it exists, but it's very noisy. But imagine instead that you, when you start, when you understood the concept of cars and passersby, and you also understood the concept of house, houses. Then essentially, what those methods do is they can build the concept of street by predicting that those configurations have not pixels, the painting of the house, but just the concept, houses nearby. And then the signal is much larger, and so you need much less data to extract the signal from noise. So, yes, we do find that those, to understand the hierarchical structure of the world, those, at least in those simple models, it's much more efficient.

• 58:59Speaker 01

  Yes, I mean, many folks will know LeCun's work in the vision space, so Ballo, Twins, and, you know, all of these joint embedding prediction architectures, roughly speaking, where you have something like a Siamese network, and then you might do some kind of mask prediction. So you might, you know, occlude tiles from one side, and you're learning this prediction function over, over the, you know, the embeddings, the latents, rather than the ambient space. But, you know, this also goes into his broader philosophy about energy-based models as well. So the rough idea is that you can, you can imbue domain-specific knowledge into a prediction architecture, and energies are composable. So you could be ridiculously specific and actually have variables that represent things in the domain.

• 59:44Speaker 01

  But what we're talking about here is something which is quite generic. It's a little bit like an inductive prior, which is not really domain-specific. So it could work for any type of vision, or it could work for any type of language model. And it's significantly more sample efficient, as you just said. But do we still have this issue that it is learning really good general abstractions? It's more efficient, but is there still something missing? You know, like, we have these galaxy brain abstractions. We can just select these meta-relations between things from a seemingly infinite set of possible relations. And is this just one step in that direction, but not all the way?

• 1:00:24Speaker 00

  Maybe I should indicate a word of, as physicists, what I've been discussing so far was the same sort of theoretical approach, but we could test it empirically and make non-trivial prediction and test them. This last part, which is a paper that's one month old, is a theory that we are now testing. And so when I talk about it, I talk about it with caution. You know, I think it's nice, that's what we want to do as theorists, is to have theories that are predictive, so make non-trivial prediction. Then being rigorous for us is not, doesn't mean having a theorem. It's us going back and testing those predictions. So we're in the process of doing that.

• 1:01:03Speaker 00

  So, yes, I mean, so, so I think that there is a very deep question there. What we find in those simple models is that indeed, you learn those abstractions with much less data. But now there is a question of, so those abstractions are there, represented in your machine? And what do you do with them?

• 1:01:35Speaker 00

  And there are things like from those representations, you can do tasks like segmentation in images or classification. And now you can, you know, be competitive with supervised method and classification. So there is evidence that it's doing a very, very good job. But for example, if you want to compare them to large language models, I mean, large language models, we like them also because they are generative. We can talk to them and then they... So for example, that's a question we're working on. And I still don't know the answer is, you know, once I have discovered those variables, essentially I created some sort of encoder of the world. Can I create with not so many data as a decoder and build a generative model from them? So can I really go back and say I can compete with those,

• 1:02:20Speaker 00

  you know, next token prediction and build something generative? I don't know. So this is completely open to me. So yes, so you can build this sort of very interesting representation of the world. Now, it's in a sort of, in our model, we know what should be there and we can check that it's there. But if you don't know where things are, how do you use this information to do specific tasks most efficiently? So I think to me it's a fascinating, you know, field of study for the years to come.

• 1:02:49Speaker 01

  And in this recent paper that you just released about a month ago, the, you know, the predict latence not tokens, you should explain figure one. We'll put it on the screen now, but you actually visualize and have a kind of analytical explanation for why it's more efficient using latence and not tokens.

• 1:03:08Speaker 00

  Exactly. So yeah, what we like to do as physicists is also to have models where we can vary parameters and then we make scaling prediction. And then it's very easy to test that, to test your prediction. You plot it in the log log and you see. So we like to have those kind of parameters to test our ideas. But in terms of the conceptual picture, this figure, as I think three networks, the first is supervised learning. So in our models, the data are a tree, a tree like. There is a top route. Maybe if you think about images, this is maybe saying if you, your image is a cat or a dog or whatever.

• 1:03:47Speaker 00

  And you don't see anything, those are hidden fibers. And then you see just what's, you know, the data, which is the input.

• 1:03:56Speaker 00

  And so one question would be how many data do you need to, from the input, be able to classify the root of your tree? That supervised learning.

• 1:04:07Speaker 00

  And then, and maybe I will skip that for reasons of time. I mean, the central figure is more like diffusion models or next token prediction. And what this figure is showing is really the concept I was trying to tell you. It's, you know, this concept of streets and below streets, you would have passerby, car sidewalks. So the nodes below, think of them as passerbys, you know. And the node above is a concept of street. And really what matters in those models is how do you correlate this concept with very low level aspect of your data?

• 1:04:53Speaker 00

  And what's in those models, something that's quite clear is that as you go away along this tree, the correlation decrease.

• 1:05:06Speaker 00

  Because you have to make, every time you have to make several choices and that leads to decreasing correlation. So what the middle panel would show is that when you try to approach the top route, so when you're trying to build, you know, abstract concept, you're correlating with very low level. It's a long distance along the tree. And so we know that every time we move along this tree, we have to pay a cost, a multiplicative cost in number of data we need. And so that's why actually the sort of,

• 1:05:41Speaker 00

  the number of data you need to learn your hidden hierarchy is exponential in the depth of the tree. Which is still good because if you think about it, the dimension of the input is also exponential in the depth of the tree. It means you can learn that polynomially in the dimension of your problem, which is much better than exponential, which would be impossible. Okay, so next or completion works, but it's still exponential in the depth of the tree. Now, if you think about the last panel, what you would really see is that you can do something very different. And again, when you try to build the concept of street, you will just predict what's nearby, you know, houses.

• 1:06:25Speaker 00

  And you've already understood this concept of house. And now the correlation, it's much closer on your graph. So it's much more correlated. The correlation being much stronger, you always have a signal to noise, you know, you need enough data to measure correlations. But if the signal is strong, you need much less data to measure it. And once you measure it, boom, you can build this abstraction. So this is essentially what this figure shows. It's a summary of what we've been discussing in this example of street and houses.

• 1:06:57Speaker 01

  It's so frustrating that we know so many things that could advance the frontier. But OpenAI and Anthropic, they're still training old school transformers. And I spoke to Cleon Jones at Sakana about this. He was one of the inventors of the transformer. And he said, any new method has to be crushingly better because we've invested so much time in hardware and optimizers and compilers. There's an entire ecosystem about this and it's actually very difficult to steer the ship. Lacan does have a couple of new startups. But the read I'm getting is that it's, he's focusing on vertical domains. We haven't yet done the moonshot where we try these new models en masse.

• 1:07:38Speaker 00

  Yes, so I agree. I will still say a word of caution that those LLMs are generative models. That's very important for them to be because you can interact with them and they produce reasoning and so on. And I still don't know even theoretically, if even though I understood all this hierarchy hidden in the world, I can use it efficiently to bring, to go back to prediction at a token level or not. If we can, then, no, if we could, if we could, then even, it would mean at least conceptually that we could do a much better generative model. But you see, so it's subtle,

• 1:08:21Speaker 00

  but there is a distinction between understanding the structure of the world, which is like building an encoder, and then decoding it for a very low level aspect of the data.

• 1:08:31Speaker 01

  You mentioned diffusion models and you had a great paper out about that. But just conceptually, how do you think they are different from something like a transformer?

• 1:08:41Speaker 00

  Well, some, often they are based on transformer architecture. So it's more the objective you mean. And in one case, you are masking, you are masking the future, like you're predicting the next token. Whereas in diffusion models, you're actually masking randomly at random position. I think it's very similar. And in our, essentially in our theory, it's the same sample complexity for both. So it's not like one has a huge advantage on the other. In some sense, the only difference is the order in which you're filling up what is being masked.

• 1:09:23Speaker 01

  Well, that's interesting. I mean, because intuitively I think of it as, first of all, you have an arbitrary number of diffusion steps. And maybe you would say that's analogous to just doing more back passes during training on a normal network. And there might be some difference in terms of the diffusion model seems to learn global relationships first and then tightens up, whereas transformers and CNNs seem to have a locality bias and that.

• 1:09:50Speaker 00

  I like to simplify the discussion by focusing on sample complexity. How many data do you need to learn? That's where we find analogous quantities because you are going into sort of compute of going back with this forward and backward process. So I'm really talking about sample complexity. And then what we find in both cases is that as you increase the number of data, you learn those constraints, those grammatical rules, bottom up. So first low level and then higher level in both cases.

• 1:10:28Speaker 00

  And we could, okay, so actually we have some confidence on those statements also because they make non-trivial predictions. So for example, you would predict that as you train a diffusion more and more to generate text, that initially...

• 1:10:48Speaker 00

  It would be random if you don't have data, it's regenerating crap. As you increase the number of data, it should start to form coherent words. Then later on with more data, coherent group of words and then coherent full sentences. And this is a prediction of those models that coherence, the sort of coherence of the context should steadily increase as you increase the number of data that we could actually check for diffusion models and also check in, you know, a theory of scaling laws of next token prediction.

• 1:11:23Speaker 01

  About a year ago, you had a paper about scaling laws as well.

• 1:11:26Speaker 00

  Yes.

• 1:11:26Speaker 01

  Tell me about that.

• 1:11:27Speaker 00

  As you increase the number of data or you increase how much compute you put or you increase the number of parameters, your performance steadily improves.

• 1:11:38Speaker 00

  And this observation by Kaplan and others had a huge impact for all of us because it drove, you know, the tech companies to just invest more and build, you know, maybe nuclear plants.

• 1:11:50Speaker 00

  So it has a huge technological impact, but it's a bit embarrassing for us theories that essentially it's not understood at all, you know, those scaling laws quantitatively, they have exponents in them. For example, describing how well you perform better if you multiply the number of data by 10. And there was very limited understanding on that question.

• 1:12:15Speaker 00

  And so, yeah, so just a few months back with Francesco Cagnetta, Alan Ravento and Soya Ganguly, we proposed a theory for this problem, inspired by this synthetic world I told you about, but detaching sort of essence of the lesson we learned from those models to really make quantitative prediction for natural languages. And essentially, we, the theory predicts that there is a simple recipe for those exponents to extract those.

• 1:12:47Speaker 00

  And essentially, this is saying, again, what's underlying it is the fact that if you give me more data, I can learn more abstract concept and that's longer range, leads to longer range correlation. But at the end of the day, the two quantities you need to measure is one, the fact that words or tokens are correlated and that this correlation, it was well known before us, that this correlation decreases as a polo of the distance between those two words.

• 1:13:19Speaker 00

  From that, you can measure exponents and they depend on the language you look at, your dataset, you can measure them. And then there is another key quantity, we argue, which is related to the entropy of text.

• 1:13:37Speaker 00

  Entropy of text has been discussed already by Shannon in the 50s. It's a beautiful question. So essentially, it's the entropy is a log of the number of possible words that you would have at one location in average. What we argue is very important to look at and that we could finally measure with LLMs or other architecture and we find consistent result is what is the entropy left after a sentence of N token? If you see N token, the more token you see, the least possibility you have there. What is the entropy of that? And in the toy model of it's a polo and in real life, it's also a polo that is found.

• 1:14:18Speaker 00

  And so essentially, what we argue is that with those two exponents, you can combine them in a way that we specify to get the training curve exponent of LLMs acting on those natural languages. And it works very well. So we got very excited with that. Also, I have to say that in addition, it's making, it's making non-trivial prediction in terms of how the loss should depend on the context that you give it, the context and also the number of data. So it's a function of two variables and we predict that it should bend in a very specific way and we could test it and we also observe it.

• 1:14:57Speaker 01

  And can you give me some interest from that? I mean, it's often, you know, when we speak about Shannon, there's a graph that people often see that, you know, during, during a sentence, every single word reduces entropy. And we're now talking almost at the population scale that when we have a huge corpus of text, entropy is just going down and down. What does that mean? You know, does that mean that the problem is getting easier over time? Does it mean that the models will just continue to get better or maybe there'll be some phase change? What's going to happen?

• 1:15:28Speaker 00

  So I should be extremely careful first of all, because we could test our theory at the sort of academia range of, so it means 1 billion of parameters, 1 billion of tokens. Okay. And I told you that as you increase the number of tokens, this machine starts to use a context that's larger and larger and we can visualize this context. So I can tell you for which context scale our theories was tested. And it's about, you know, 50 tokens or two or three sentences. I think it's great because there's all syntax and so on there. So there's a lot of stuff, but that's where I'm, I'm confident that, I mean, confident,

• 1:16:12Speaker 00

  I think we have a very robust that story that I think will hold true. I mean, you know, the field has to investigate further and decide. But what really we have not done, because it's not possible with our means, is to test this theory for, you know, much beyond three or four sentences. And so I don't know if the mechanism we put forward still applies there or if it's something completely different.

• 1:16:40Speaker 01

  If you could explain deep learning to any scientist, dead or alive, who would it be?

• 1:16:47Speaker 00

  Oh, if I could explain it to any scientist, dead or alive, who would it be?

• 1:16:53Speaker 00

  Oh, that's, that's, I never thought about this question. I had a very strong bond to my dad, was a physicist and with whom we discussed a lot about science. And at the end of his life, he was actually very interested by neuroscience. He thought about olfaction and things like that. So it would be him, yes.

• 1:17:16Speaker 01

  Are there any things in your career that you've been completely wrong about and you've changed your mind?

• 1:17:20Speaker 00

  I think it's completely fine to do mistakes in science, but I think what's very important is that once you're convinced that you did a mistake, you, you state it and it's obvious for everybody and you don't sort of encroach to your, even if you don't believe in it anymore, you don't encroach to it. I mean, I, so, so yes, I mean, it, it, it happens for sure. You know, the game we're playing as physicists is to propose model of the world and then make predictions. Already that we feel is doing our job, then it needs to be tested by us or by others. I mean, sometimes your prediction does not hold because it was not the good model,

• 1:18:02Speaker 00

  but that's how we build, you know, understanding hypotheses and so on. So, so in this sense, yes, it often happens, but, but it should. I mean, that's how, you know, that's how science should work in some sense, like you're, you're, you're making hypotheses and then you, then you really test them. So, yeah, I, I think it's important to, if you're creative, if you take risk, I think you should do mistake. If you never do mistakes, maybe it's a sign that you're staying a bit on the beaten path in science and some of us want to explore the jungle. In the jungle, you can be wrong. I mean, yes.

• 1:18:46Speaker 01

  I think it's been an honor having you on the show. Thank you so much for joining us.

• 1:18:49Speaker 00

  Thank you so much. That's fine. Thanks.