Deep Learning is Not So Mysterious or Different - Prof. Andrew Gordon Wilson (NYU)

Professor Andrew Wilson from NYU explains why many common-sense ideas in artificial intelligence might be wrong. For decades, the rule of thumb in machine learning has been to fear complexity. The thinking goes: if your model has too many parameters (is "too complex") for the amount of data you have, it will "overfit" by essentially memorizing the data instead of learning the underlying patterns. This leads to poor performance on new, unseen data. This is known as the classic "bias-variance trade-off" i.e. a balancing act between a model that's too simple and one that's too complex. **SPONSOR MESSAGES** — Tufa AI Labs is an AI research lab based in Zurich. **They are hiring ML research engineers!** This is a once in a lifetime opportunity to work with one of the best labs in Europe Contact Benjamin Crouzier - https://tufalabs.ai/ — Take the Prolific human data survey - https://www.prolific.com/humandatasurvey?utm_source=mlst and be the first to see the results and benchmark their practices against the wider community! — cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++ Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst — Description Continued: Professor Wilson challenges this fundamental belief (fearing complexity). He makes a few surprising points: **Bigger Can Be Better**: massive models don't just get more flexible; they also develop a stronger "simplicity bias". So, if your model is overfitting, the solution might paradoxically be to make it even bigger. **The "Bias-Variance Trade-off" is a Misnomer**: Wilson claims you don't actually have to trade one for the other. You can have a model that is incredibly expressive and flexible while also being strongly biased toward simple solutions. He points to the "double descent" phenomenon, where performance first gets worse as models get more complex, but then surprisingly starts getting better again. **Honest Beliefs and Bayesian Thinking**: His core philosophy is that we should build models that honestly represent our beliefs about the world. We believe the world is complex, so our models should be expressive. But we also believe in Occam's razor—that the simplest explanation is often the best. He champions Bayesian methods, which naturally balance these two ideas through a process called marginalization, which he describes as an automatic Occam's razor. TOC: [00:00:00] Introduction and Thesis [00:04:19] Challenging Conventional Wisdom [00:11:17] The Philosophy of a Scientist-Engineer [00:16:47] Expressiveness, Overfitting, and Bias [00:28:15] Understanding, Compression, and Kolmogorov Complexity [01:05:06] The Surprising Power of Generalization [01:13:21] The Elegance of Bayesian Inference [01:33:02] The Geometry of Learning [01:46:28] Practical Advice and The Future of AI Prof. Andrew Gordon Wilson: https://x.com/andrewgwils https://cims.nyu.edu/~andrewgw/ https://scholar.google.com/citations?user=twWX2LIAAAAJ&hl=en https://www.youtube.com/watch?v=Aja0kZeWRy4 https://www.youtube.com/watch?v=HEp4TOrkwV4 TRANSCRIPT: https://app.rescript.info/public/share/H4Io1Y7Rr54MM05FuZgAv4yphoukCfkqokyzSYJwCK8 Hosts: Dr. Tim Scarfe / Dr. Keith Duggar (MIT Ph. D) REFS: Deep Learning is Not So Mysterious or Different [Andrew Gordon Wilson] https://arxiv.org/abs/2503.02113 Bayesian Deep Learning and a Probabilistic Perspective of Generalization [Andrew Gordon Wilson, Pavel Izmailov] https://arxiv.org/abs/2002.08791 Compute-Optimal LLMs Provably Generalize Better With Scale [Marc Finzi, Sanyam Kapoor, Diego Granziol, Anming Gu, Christopher De Sa, J. Zico Kolter, Andrew Gordon Wilson] https://arxiv.org/abs/2504.15208

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Speaker 00

  Hey, listen!

• 0:02Speaker 01

  Power up your play with Nintendo Switch 2 and experience Link's legendary adventure, now reborn in vibrant detail on our fastest and most powerful system yet.

• 0:14Speaker 01

  And harness the power of the Ocarina with the built-in microphone.

• 0:22Speaker 01

  The Legend of Zelda: Ocarina of Time, a Nintendo Switch 2 exclusive. Game and system sold separately. Game rated everyone 10 and up.

• 0:30Speaker 02

  LinkedIn is pretty amazing at helping you grow your small business. We cannot stop your new clients from emailing you at 3 a.m.

• 0:39Speaker 02

  We can help you sell, market, and hire in one place. We cannot help you be in three places at once. And while we can't help you organize your calendar, LinkedIn can help you land more clients so you have a calendar to organize. Grow your small business on LinkedIn. Learn more at LinkedIn.com/smallbusiness.

• 1:00Speaker 03

  So when I say deep learning is not so mysterious or different, I'm not saying that deep learning is not mysterious or not different. I think it's actually both. It's rather that the ways in which people often think it's mysterious can be relatively well-understood, both intuitively through a notion of soft inductive biases, but also formally in terms of rigorous generalization frameworks that have actually existed for many decades. And a lot of these phenomena can also be reproduced using other model classes. I think deep learning really is distinguished in its relative universality, how broadly applicable it is relative to other model classes. That doesn't mean that it's anywhere close to being completely universal, but it's sort of a movement in the direction of universality,

• 1:45Speaker 03

  a significant movement. It also does representation learning incredibly effectively. It has properties of its optimization objective, its loss landscape, which are relatively different and surprising, like mode connectivity. And so I think deep learning is certainly different and mysterious, but often not in the ways that people might believe. So I'm very excited to be talking to you both, even though it can be challenging knowing that there are so many people watching.

• 2:17Speaker 03

  But I think there are so many fundamental misconceptions in the way that people understand generalization and model construction and artificial intelligence. It's really important to hear a different perspective around, for example, how it's completely fine to build a huge model that will also have a stronger bias for simple solutions, have more of an Occam's razor-like behavior than even smaller models. And these sorts of perspectives actually help us understand phenomena that are often seen as very mysterious, like double descent and benign overfitting and overparametrization, and give us a principled approach for thinking about how we're going to build our own models for whatever properties we're interested in.

• 3:01Tim Scarfe?

  There's this fundamental trade-off between bias and variance. And it feels like you're saying you can have your cake and eat it, and you can keep them in the mixture, and you still win. And that just goes against most people's intuition.

• 3:14Speaker 03

  So I think the bias-variance trade-off is an incredible misnomer. There doesn't actually have to be a trade-off.

• 3:21Tim Scarfe?

  Folks, that interview with Andrew was absolutely amazing. Keith came over to the UK and we did it in my home studio together a few weeks ago. It's been on Patreon for a little while. And I updated so much based on that interview. Andrew was absolutely brilliant, so I know you're gonna love it. But before we kick off, you've probably heard that human data is kind of the dirty secret of Silicon Valley. You know, human data is the reason why these AI models work so well, because the OpenAIs and the Anthropics, what they do is they hire humans to do things like dataset curation and evaluation and post-training. And there is a ridiculous kind of uplift from using this human data.

• 4:05Tim Scarfe?

  Our sponsor, Prolific, what they want to do is produce the first survey on how human data is being used in AI. And if you volunteer and fill out this form for them, they will give you first access to see how you compare. So I'd really appreciate it if you did that. There's no personally identifiable information. Link is in the description. And we are also sponsored by Tufa AI Labs. They are an incredible research lab based in Zurich. They've just upgraded their office. They've got an amazing new office. They've hired thirteen research engineers in the last year, doing things like reasoning and the ARC Challenge. You've probably seen some of the papers that they've published on that. But they have ambitions to build their own foundation models from scratch. They've got an amazing culture.

• 4:50Tim Scarfe?

  And Benjamin Cruzier, the director, is also very interested in AI safety. So he's going through the Yudkowsky book at the moment. So if that seems like a fit for you, please get in touch with Benjamin Cruzier. Go to tufalabs.ai or look in the description. And also MLST is sponsored by Cyberfund. Enjoy the show, folks. Well, Andrew, much of your work challenges conventional wisdom. Is that hard to do? Is there resistance in challenging strongly held beliefs?

• 5:20Speaker 03

  So, yes, I mean, this is what happens when you challenge conventional wisdom. But in some sense, I think that we should always be trying to do that. Because otherwise, we're just preaching to the choir and then what's the point? But if no one, if you're not changing anyone's beliefs about anything, then maybe it doesn't make a difference. And so I think it's important to really try to understand what do a lot of people believe that might be wrong? And then just unpack that. And it's also very exciting and fun. But it's challenging because, of course, the initial instinct will be to resist whatever you're saying. But then over time, and if you try hard enough and if you talk to amazing communicators like you two, then you can start to have an influence.

• 6:00Speaker 03

  And I think that's really important because so much progress has been stalled, I think, by just getting stuck on misconceptions. Like, once a certain number of people believe something, it's very, very hard to change their minds, no matter what you say. And I think as a consequence, we've been in all sorts of local minima in machine learning and AI research because we haven't been able to get unstuck from these erroneous beliefs. And there's a whole roster of things like this, like the role of implicit biases, of stochastic optimization and generalization, I think, is significant but also significantly overstated. How we can have really large models that also generalize well,

• 6:45Speaker 03

  even when there's a small number of data points, is something that is not very well recognized. And in fact, I think is one of the primary drivers of scale. Being important for achieving good generalization. So not just flexibility, the simplicity bias that comes about through scale. I think another misconception is this idea that we should change our model depending on how many data points we happen to have available. And this might even be the most controversial one. The reason I don't think we should is because we should always honestly represent our beliefs. And our beliefs about the process that generated our data typically shouldn't change depending

• 7:28Speaker 03

  on how many data points we happen to have access to. And you can actually demonstrate that these principles work in practice. You can have models that will be very good when you have a small number of data points and also very good when you have a very large number of data points. And so this relates to not necessarily needing to have hard constraints, but instead combining expressiveness with Occam's razor.

• 7:50Speaker 05

  Yeah, and you know, you're only slowly starting to convince me to give up this 10,000, you know, degree polynomial is bad. And I'm only starting to change because you value simplicity, bias towards simplicity as much as I do. And the real key for me was understanding that somehow scale has a bias towards simplicity. And I don't know why or where it comes from, but I believe it and it's weird.

• 8:17Tim Scarfe?

  And can we just set this up? So in your talk last year, your first slide, so you had a bunch of students in the room and you showed like a fairly sort of simple linear correlated, you know.

• 8:28Speaker 05

  Airline passenger data.

• 8:29Tim Scarfe?

  Airline passenger data, okay. It was pretty linearly.

• 8:32Speaker 05

  Some seasonality.

• 8:33Tim Scarfe?

  And you said, here's three models. One is basically y equals mx plus c or something like that, just a straight line. And I think was the second one, something like 10 parameters and 10,000 parameters was the third one. And almost everyone in the room said they preferred one or two. And you said, at the end of this conversation, I'm going to convince you to prefer three, which was 10,000 parameters.

• 8:55Speaker 03

  And I took a pull at the end and it did shift. And so that was promising. Yeah, I I sort of joke sometimes that if I hadn't met the airline passenger data set, I don't know really what my life would be like now because it's it's driven so much of my research. And it's just amazing how people are biased towards choosing the linear function or the cubic polynomial, even if in practice they're not making that choice. Like on CIFAR, for example, it's not uncommon to use a neural net with tens of millions of parameters to fit a training set with tens of thousands of data points. And even before deep learning was popular, we were doing nonparametric statistics where we were working with models like Gaussian processes that were inspired by taking infinite

• 9:37Speaker 03

  limits of neural nets that are more flexible than any neural net you can fit in memory. And other popular covariance functions as well, like the RBF kernel, really are like saying, I want to use an infinite order polynomial. And so even in these kind of classical statistical models, we're implicitly saying, well, if we're unhappy about the third choice, it's actually because it doesn't have enough parameters. We want infinitely many parameters, not just 10,000 parameters. And I think another way to say this is that parameter counting is a very bad proxy for model complexity. Really what we care about is the properties of this sort of induced distribution over functions, rather than just how many parameters the model happens to have. And so you can have a distribution over functions, which is very flexible.

• 10:19Speaker 03

  It can represent many different solutions to a given problem, but it can also have very strong preferences for certain types of solutions over others. And strong preferences doesn't mean saying that certain things are impossible necessarily. They can just have epsilon probability. And I think that that is really meaningfully different than saying, okay, we're we're going to have a hard constraint and we're not going to represent those solutions. A, because it's not an honest representation of our beliefs to have those hard constraints. And B, because we see in practice that when we do have these expressive models with simplicity biases, they're much more adaptive, they're much more automatic. So when you have a small data set, it sort of does the right thing. When you have a large data set, it also

• 11:02Speaker 03

  does the right thing. And so you don't need as much human intervention. And arguably, that's the definition of what really machine learning is trying to achieve. It's trying to build an intelligent system that doesn't require manual intervention. And so I think this is an important principle towards that goal.

• 11:18Speaker 05

  You know, you had a paper kind of removing some of the mysteries of deep learning, but I think it's still fair to say it's a bit mysterious where the simplicity bias comes from at scale, isn't it?

• 11:30Speaker 03

  It is. So there is some handwavy intuition around loss landscapes. And I think this is borne out empirically. So we can sort of understand, for example, that the solutions that we're finding when we're building larger models are more compressible and are flatter, etc. But this is really ongoing research. And I think this is one of the most important questions to understand right now is like, why does scale, rigorously speaking, produce a simplicity bias? And can we get that bias in a more elegant way than just building bigger models?

• 12:05Tim Scarfe?

  I was going to ask you, are you a theory, because I know you're a theory guy. Are you a theory guy or an engineer? Presumably, you're both. But, you know, deep in your bones, are you an engineer or are you a scientist?

• 12:16Speaker 03

  It's really hard to choose. In some sense, both. I'm mostly driven by trying to understand things. And so this can be done in a variety of ways. A lot of our papers empirically try to understand model behavior. And so I feel like this is a scientific approach to machine learning. And one thing that really motivates me about this type of approach is whatever you learn will never go obsolete. So quite often, newcomers to the field and even, you know, very experienced researchers. Feel distressed at the rapid pace in the discipline where you see methods getting published at a conference and then becoming obsolete within a month, or when you go to

• 12:59Speaker 03

  the conference, everything you're seeing is sort of, you know, been replaced by some other algorithms and wonder, okay, well, is there any point to me sort of investing myself significantly in building a model if I know that it's not going to be used by anyone for any long period of time? And I think one way to address that is really to try to combine what you're doing with an understanding of why things are working. So if you're building a model that gets better performance on some problem, there's a reason for that. And if you can understand the reason for that, that understanding will outlive that specific model and how widely it might be used. And I'm really hoping to do research that will be relevant in hundreds of years from now.

• 13:41Speaker 03

  And so I think these questions around model selection, for example, in Occam's razor, people will never stop asking, like, hopefully they'll be able to go back and read not just my work, but like work that's been done in this space and think, okay, this is useful to me in thinking about how to approach some of these questions. And so in this respect, I would say I'm a scientist and I try to combine classical theory with empiricism towards understanding model behavior. And I think if you really understand something, hopefully it's something that you can demonstrate in practice. And so I also try to combine some sort of practical demonstration with a lot of the work that I do. And this process also involves engineering and sometimes understanding those low-level

• 14:25Speaker 03

  engineering details becomes really fascinating and leads to unexpected intuitions about the principles behind model construction. I think this is something that's perhaps underappreciated. Like, quite often, you can have a great idea and whether it works or not depends

• 14:44Speaker 03

  Very significantly on all the low-level details like numerical stability and other things like that. And when you get really deep into those details, sometimes you can discover things at a higher level that are also very significant in how we should think about model construction and algorithm design.

• 14:59Speaker 05

  Very cool. Maybe, well, I was just gonna share with you, there's a duality that Shannon pointed out that you, they may like, and, you know, it's a duality between past, future, knowledge and control. He said, he said, "We have no knowledge of the future, but we can control it. We have knowledge of the past, but we cannot control it." And, and I took that and related it to science and engineering. So the way I look at the two sides of that coin is scientists leverage control to gain knowledge, engineers leverage knowledge to gain control. So you can be both, because the, the new knowledge that you collect allows you to better control the environment, the future of the world, and collect even more learning.

• 15:44Speaker 05

  Right?

• 15:45Tim Scarfe?

  Absolutely beautiful. Andrew, we haven't even introduced you yet. Can you, can you, can you tell the audience about yourself?

• 15:51Speaker 03

  So I'm Andrew Wilson. I'm a professor at the Courant Institute of Mathematical Sciences and Center for Data Science at New York University. My work focuses on having a prescription for how to build intelligent systems, what are the key principles involved in model construction? I think although the field has made an extraordinary amount of empirical progress towards building more performant machine learning systems, we're still at early stages of understanding, you know, what principles should we broadly embrace when we're approaching our own problems? And so this involves work on understanding inductive biases, so what assumptions we should be making. And so this relates to symmetries, like equivariances, maybe we're modeling molecules, their rotation invariant images could be translation invariant, how do we represent those invariances?

• 16:36Speaker 03

  How do we learn them automatically? How do we discover interpretable scientific structure in our data that tells us something maybe surprising that we didn't know before, that will go beyond a particular application? How do we represent uncertainty towards decision-making? Arguably, a prediction that's just a point estimate without any kind of error bars associated with it, isn't really actionable in the real world. You know, if you have an autonomous car and it says there's a stop sign five feet ahead, plus or minus 10,000 feet, you can't really do anything with that information. But if it's plus or minus one foot, then you can really act on that information and, you know. Observing that almost makes you paranoid, like, okay, now I really need to represent uncertainty, because if I don't have that uncertainty, then, you know, machine learning

• 17:19Speaker 03

  can't meaningfully engage with the real world. And Bayesian methods, I think, are a really great way of reasoning about uncertainty. And so that also forms a big part of my research program.

• 17:27Tim Scarfe?

  Well, welcome to MLST. We have Dr. Dugard in the house. The first time we've met in person for, we've been doing this for eight years. Yeah, yeah. Five years on this channel, but we had the previous channel as well. Yeah. And Keith came to my, to my wedding on, on Friday. It's good to have you here, man. Yeah, it's pretty crazy. We haven't met until now. It's, it's gonna be a blast. Absolutely. Yeah, we've got some good stuff lined up. Well, I guess, just to kick this off, Andrew, I was inspired by geometric deep learning. I interviewed, you know, like Michael Bronstein, Tako Cohen, Joanne Bruner, Peter Vilejčkovic. They had this geometric deep learning blueprint. Our video on that did half a million views. It was an amazing video. And the basic hypothesis, as Joanne outlined, the three curses in, in machine

• 18:12Tim Scarfe?

  learning, right? So there's the statistical curse, which is that you only have so many data points, and the distance to those data points, you know, or the density of those data points is kind of cursed by the dimensionality of your data. There's the, the optimization curse, which is that you get stuck in these local minima. And there's the approximation curse. And this is kind of where they were driving to, that you, you have this function class, and you can be quite opinionated in how you structure that function class. But if you make it too small, you incur approximation error, where the actual test sample is, you know, some epsilon distance from, from the approximation class. And they said that all of these things are cursed. And I guess their prescription was this Platonistic idea that if we constrain the models, so if we add bias to the models with these symmetries,

• 18:56Tim Scarfe?

  because the generating function of the universe is using these symmetries anyway, then there's no apparent approximation error in doing so. So why wouldn't we do it anyway? I think your, your ideas are kind of tangentially related to that. You still think we should have biases, but you, you also think we can have our cake and eat it, so to speak.

• 19:17Speaker 03

  So this is a wonderful question. And I've also done a fair amount of work on geometric deep learning, particularly in terms of How we should represent equivariant symmetries in scientific domains. And I agree that it's very appealing to say, well, if we know that some constraint applies to our problem, then we should encode it in our model. And I don't think that's particularly controversial. There are some perhaps surprising results that in some instances, even when you know what the constraint is, you can do as well or better when you don't represent that constraint. And this can be due to the dynamics of how you train your model, et cetera. But generally, the way I would respond to this is really to say two things.

• 19:59Speaker 03

  The first is there aren't that many instances in practice where we know exactly what constraints we want to have, even when it comes to things like physical conservation laws. So rarely are we modeling, for instance, closed systems. You can have a dynamical system and maybe you have like a pendulum with like wind in the air or whatever, and now you have some sort of violation of conservation of energy. And so you want to perhaps instead build models that are just biased towards these constraints without being exactly constrained. Secondly, when you do this, when you represent so-called approximate constraints or you have soft constraints, so you have a model which is very flexible, but it says, well, if we can fit the data in a particular way, then we want to do that. Quite often, it will just collapse down onto those constraints if it provides a consistent

• 20:44Speaker 03

  explanation of what we observe. And so if you're paying any kind of penalty for deviating from those constraints and you can perfectly explain your data with those constraints, you'll just collapse down onto that. And so I think as a prescription for model construction, it's often going to be fruitful to try to embrace expressiveness, but at the same time have a simplicity bias, which can be formalized in terms of compression. And this is really an honest representation of our beliefs in many cases. Like another way to describe my philosophy for model construction is just honestly represent your beliefs. And we believe the real world's a complicated place. And if we combine that belief with the idea that simple solutions that are consistent

• 21:28Speaker 03

  with our observations are more likely to be true, then we can often see desirable behavior in quite a variety of different settings. And so in terms of like having cake and eating it too, I think one of the most surprising Findings that we and others have had is that quite often you can increase model expressiveness while simultaneously increasing its biases. So larger models are often more inclined towards simple solutions. And there are sort of demonstrations of this that have been just hiding in plain sight, like double descent, this idea that as you increase model flexibility, your generalization error first gets lower, so it improves.

• 22:12Speaker 03

  As you capture more structure in the data, it gets worse as you start to overfit, and then it gets better again. And in that second descent, typically, all of the different models that you're considering are fitting the training data perfectly. So the only possible way that larger models could be generalizing better is because they have some other sort of bias, like a simplicity bias, rather than being more expressive. And so I think, time and again, researchers express surprise at the fact that they can have these massive models, billion parameter-plus models, trained on relatively small data sets that aren't overfitting. But in fact, actually, if they've made the models even bigger, they would be less likely to overfit.

• 22:52Speaker 03

  And so quite often, expressiveness and soft constraints can be aligned.

• 22:58Speaker 01

  Hey, listen! Power up your play with Nintendo Switch 2 and experience Link's legendary adventure, now reborn in vibrant detail on our fastest and most powerful system yet. And harness the power of the Ocarina with the built-in microphone. The Legend of Zelda: Ocarina of Time. A Nintendo Switch 2 exclusive. Game and system sold separately. Game rated everyone 10 and up.

• 23:28Speaker 02

  LinkedIn is pretty amazing at helping you grow your small business. We cannot make your email response time faster. We can help you sell, market, and hire in one place. We cannot help you find space for your three desk drinks. Why do you have three? And while we can't help you find the perfect volume for your presentation video, LinkedIn can help you find the perfect audience for your business. Grow your small business on LinkedIn. Learn more at LinkedIn.com/smallbusiness.

• 23:57Speaker 05

  So if parameters don't solve your problem, you're not using enough of them.

• 24:02Tim Scarfe?

  We always want more.

• 24:05Tim Scarfe?

  Prediction with expert advice, we know theoretically and for me empirically that by keeping all of these other experts around in the mixture, you pay a cost for that. So in my particular case, having the historical experts in the mixture, every single prediction, they still had some weight. You have to give them some epsilon weight, because otherwise they would die. And that actually harms your performance. So the question is, is it better for you when the new regime comes to pay the cost of learning the regime versus the switching cost of bringing the old regime back? And that's kind of the same with any ensemble or any set of restrictions on biases, really. There's this fundamental trade-off between bias and variance. And it feels like you're saying you can have your cake and eat it, and you can keep them

• 24:49Tim Scarfe?

  in the mixture and you still win. And that just goes against most people's intuition.

• 24:55Speaker 03

  So I think the bias variance trade-off is an incredible misnomer. There doesn't actually have to be a trade-off. So the idea, I guess, behind the classical bias variance trade-off is that your generalization error can be compartmentalized in these two terms. So bias, sort of how well you are fitting the data essentially, and variance, like how your fits vary depending on, like, if you sample different points from this distribution that you're trying to model. And it's true that sometimes if you naively build like a really large polynomial, for example, you can have low bias and high variance, whereas if you build a small polynomial, maybe

• 25:40Speaker 03

  you have low variance and high bias. However, approaches like ensembling are actually a good way of getting low bias and low variance. And it turns out, actually, building large neural nets are another way of getting both low bias and low variance. You actually have flexibility combined with a simplicity bias. And this is what's leading to good generalization, and it's sort of another perspective on double descent.

• 26:07Speaker 05

  I think here's a way I'll put the question is, and I ran into this as a practitioner, you know, back in the day. So maybe I was just stuck in the hump of having like too many but not enough parameters, you know, to get to the double descent phase. I'm not sure, but I mean, you know, what I experienced is that... You know, having parameters in a model, even if they're very, very small, because some, you know, I put in some term in the objective function that force them to be small is not the same thing as actually the simpler model that just didn't have them at all. Right? Like, I mean, like we kind of, you know, we'll talk about marginalization and kind of Bayesian, Bayesian perspective on that. So like, overfitting can be a real problem. So, for example, you brought up, you know, conservation.

• 26:48Speaker 05

  It's like, well, if I'm doing a model and I don't enforce conservation of energy, and then as a result, I end up with some small parameters that cause like a little bit of feedback and increasing, you know, energy every single time a robot, you know, takes some action, it can cause it to like spin out of control. Right? Where I actually did need it to conserve energy and not have that, that positive feedback. So I guess, I guess maybe we're still struggling with, you know, overfitting can be a problem. It's a real problem. It's a known problem. How do we know if we're overfitting in a bad way? We maybe haven't, don't have enough parameters, we're stuck in kind of the, the area before we got to double descent, or like, how do you in practice avoid the actual consequences of bad overfitting?

• 27:33Speaker 03

  Right. So, overfitting is real, absolutely, but the conventional wisdom about how we should approach it, I think, is fundamentally misguided. So, and this is rooted in things like the bias variance trade-off. Like, let's constrain our hypothesis space so that we can't have a bad fit to the data that will make bad predictions and so on. Whereas, instead, I think if we just embrace the honest belief that there are many possible solutions, even if they're not probable for any given problem, combined with this sort of simplicity bias, we won't tend to overfit. And interestingly, the prescription is almost the opposite of what people think it perhaps should be in principle. Like, build a smaller model is usually the, the prince, the, the sort of, the... The prescription for avoiding overfitting.

• 28:19Speaker 05

  Or to enforce of simplicity.

• 28:20Speaker 03

  Exactly, yeah. Whereas, in fact, as we build bigger models, we often actually start to alleviate overfitting. And double descent is just a great example of this because that first ascent is from overfitting the data, but then it gets alleviated as we start to make our models bigger and bigger.

• 28:38Speaker 05

  By some phenomenon that we still don't really fully understand, like this ability of simplicity to start to come back into the picture as you just make it even bigger.

• 28:48Speaker 03

  Right. So in that second descent, the models are typically fitting the training data perfectly. And so the loss is not really the decisive factor anymore between which models we're selecting. It's something else. And so those other biases start to dominate. And so this is actually really important, I think, when people talk about phenomena like flatness. So flatness is this idea that if you perturb your parameters, that you can still get a relatively low value of the loss. And there are all sorts of debates about like the role of flatness and how relevant it ought to be and understanding generalization, etc. But I think what a lot of these discussions miss is it's just one of many properties that control generalization. So if I had to choose between a model that has very high loss,

• 29:31Speaker 03

  but it's very flat versus like a very flat solution versus a model that finds a solution that's low loss, that's relatively sharp, I would almost certainly choose the low loss solution. And so when you're in that kind of second descent regime, you now are controlling for the value of the loss. And it's just the flatness of the solutions that's increasing. And so that's sort of one way of knowing maybe like what side of the curve that you're on and thinking about how big should you make your model. But I would also just say, make your model always as big as possible. Just try to combine what you're doing with some sort of simplicity or a compression bias. And there's a question of like how you do that. But I think there are good, good ways that we have of thinking about how to do that.

• 30:15Tim Scarfe?

  Trick question. Is predictive power the same as understanding?

• 30:19Speaker 03

  I agree with the idea that representation matters, representation meaning sort of how you're solving the problem, even if you're getting the same performance in a particular application. But the reason it matters is because

• 30:34Speaker 03

  Different representations that are achieving the same performance might give you different performance than on different problems. And so if we're trying to build more general agents, we want to understand what sorts of representations are going to provide a better general description of the real world. And so that means we want to avoid things like shortcut learning and so on. If it's just going to lead to good predictions in some contrived problem and not really in the real world. And so there's this question of like, can we understand what sorts of distribution shifts we might typically encounter? And can we build methods that have broadly more robustness to a variety of different types of realistic distribution shifts? And I think this connects to things like no free lunch sort of thinking.

• 31:17Speaker 03

  So the no free lunch theorems say that every model is equally good in expectation over all problems sampled uniformly from a distribution over all problems. And there are other no free lunch theorems that say no single learner can be good on all problems. The issue, I think, with these theorems is not their mathematical validity. What they're saying is correct under the assumptions they're making, but rather that the assumptions they're making are not a good description of the real world. So the real world is a small corner of all possible data sets. It's not drawn uniformly from a distribution of all possible problems. If we were to do that, we would mostly just get noise. And so the question then is, to what extent is the structure across real world problems shared?

• 31:59Speaker 03

  And at what level of abstraction can we represent that shared structure? And so my contention is that the distribution of a real world data is biased towards local McGraw complexity. And so are some of the the models that we started to to develop.

• 32:15Tim Scarfe?

  Yeah, but can you give an example of where it was hard to confront a misconception? Why it mattered and what the process involved?

• 32:24Speaker 03

  There had been this sort of Approach where you would take some approximate Bayesian inference procedure and pit it against deep ensembles as the non Bayesian alternative. And normally, I actually don't care that much about what's being called Bayesian or not. Like, it's same with like intelligences, what is intelligent, like whether something's a good representation, all these things. Let's just connect this to whatever problem we're trying to solve. But this was actually quite problematic because the takeaway seemed to be that if deep ensembles were working better than some Laplace approximation or some MCMC procedure, then the answer is to be non Bayesian, to be less Bayesian than we have been historically.

• 33:04Speaker 03

  And that's turned out to be exactly the wrong sort of directionality for how we should think about model construction. Because for a given computational budget, those deep ensemble procedures were actually doing a much better job of approximating the posterior Bayesian predictive distribution, so doing marginalization. And so, in fact, the prescription should have been, well, we actually need to be more Bayesian. And so, like, this was sort of a frustrating thing. But it was also very difficult to approach because there had been so many papers where people had just written, basically, that these deep ensembles are the non Bayesian alternative. And so, kind of coming out and saying, well, actually, they're doing a better approximation of the Bayesian ideal than all these methods that are being called Bayesian is sort of, you know, confronting, you know,

• 33:49Speaker 03

  just sort of hundreds of papers in some sense at once. But it felt like a very important thing to do. And we had actually done it subtly in a lot of papers where there'd be some subsection of some paper that mentioned something like this. But that was never really internalized because it was never front and center. So I thought, okay, a blog post is the right way to do this, and let's make it all about this. And I think, in the end, actually, it was the blog post that changed people's minds. I never saw a paper after that where people were making this separation. But also, I think, you know, it did, it did sort of strike a nerve a little bit.

• 34:24Tim Scarfe?

  Yeah, but how should we approach model construction? How can we embrace expressiveness without overfitting?

• 34:31Speaker 03

  And so when I say I want to embrace expressiveness, there is some subtlety associated with that idea. And that basically means that in some cases, maybe we're wanting to represent lots of solutions, but we're assigning them almost zero probability, but not zero probability. So they're possible, but not plausible solutions in our view. And then if the data is telling us something that like, well, actually, we really should be paying attention to certain type of structure that maybe would surprise us, the model can actually respond to that. And if that structure isn't actually there, then your model isn't going to perform a lot worse than the model that is exactly constrained in those ways. In terms of how we should approach model construction in general,

• 35:15Speaker 03

  if you have a soft bias, a gentle encouragement towards certain types of constraints over others, quite often you can do as well as the perfectly constrained models. And the reason is you're paying some sort of penalty, even if it's small, for deviating from that constraint. So if you can fit the data perfectly with the constraint, you'll just collapse down onto that model. And we noticed this in a work we had called residual pathway priors for soft equivariance constraints. And so this was a Bayesian mechanism essentially to have a distribution over solutions, which would be concentrated in some way around certain types of equivariance constraints. Just for the sake of the audience, equivariance is a generalization of invariance. It basically means if you have some transformation T, F of T X equals T of F of X,

• 36:00Speaker 03

  rather than F of T X equals F of X.

• 36:03Tim Scarfe?

  Like a CNN, for example, so it commutes in that case with the translation.

• 36:09Speaker 03

  Exactly right. So if you translate the image in some way, the pattern of activations will translate in the same way across the different layers, rather than them just staying exactly the same, which would be invariance. So.

• 36:24Speaker 03

  In this paper on residual pathway priors, we were interested in the strength of this soft bias. So we basically had a distribution over neural net parameters that had a covariance matrix that lived in some equivariant subspace of our choice, plus some orthogonal complement. And so there'd be these weighting terms, basically, that would represent, you know, how strong do we want this bias to be for equivarians? And surprisingly to us at the time, it didn't matter very much. Like, as long as you had a very soft bias for the constraint, it would often be as good as even a perfectly constrained model. Basically, for this reason, that, you know, you're still paying some penalty for deviating from the constraint. And so if it is a good description of the data you've fitted perfectly, then you often collapse down onto that constraint.

• 37:09Speaker 03

  Although I wouldn't want to dismiss the importance of trying to calibrate these biases. It can matter in certain instances. But in a lot of instances, a very gentle bias is sufficient.

• 37:19Speaker 05

  So maybe, maybe just to put this into perhaps more familiar territory for other Bayesian's out there, you know, like the assignment of priors just for parameters. It's always the goal to try and find a prior that's relatively ignorant, but encodes some very soft, you know, type of constraint. Like, maybe it's a scale invariant parameter or a scale invariant prior or a location invariant, you know, kind of prior. And overall, a lot of times, these priors are maybe worth like one data point or two data points. But they're small enough that they can be overridden quite easily by enough data. But even that small amount is enough to avoid sort of stupid answers.

• 38:02Speaker 05

  Like, you know, that, like, the chance of a head is infinity or something like that. Is it kind of analogous to that? Or?

• 38:09Speaker 03

  I think that's a reasonable analogy. I would also add that we can't get away from making assumptions. So even though I'm in favor of embracing expressiveness and having relatively soft biases for certain types of solutions as opposed to hard constraints, machine learning means learning by example. And we can't do that without making assumptions. The question is just what assumptions should we be making and at what level of abstraction? And perhaps it's enough in a surprisingly large array of different problems to embrace expressiveness in combination with some sort of simplicity, some Occam's razor bias that can be formalized in terms of compression.

• 38:48Tim Scarfe?

  Empirically, simple models work better.

• 38:51Speaker 03

  Ironically, given the surprise people often express at this idea of wanting to always embrace flexibility is that like before deep learning, the community had started to come on board with this notion that we want arbitrarily flexible models. In fact, the class of models that I was working on in my PhD Gaussian processes in machine learning were kind of inspired by this idea that we want really, really large neural nets. And Radford Neal, a statistician at Toronto at the time, working in Jeff Hinton's group, was saying, okay, we want to build models the size of a house. And I'm a Bayesian and I'm going to really embrace expressiveness. So I'm going to take an infinite limit of a neural net with an infinite number of hidden units. And this is going to converge actually to a Gaussian process using a central limit theorem, argument with a particular type of covariance function.

• 39:36Speaker 03

  People thought, oh, well, that's amazing. Let's just use Gaussian processes because they're so much more principled in a lot of other ways. Like, they're less sensitive to a bunch of design decisions, etc. They can be written in, you know, a very small number of lines of code and everyone, anywhere in the world is going to get basically the same answer, etc. And so people sort of moved in this direction of just embracing expressiveness. But then, once we started working in these kernel formulations, I think people perhaps started to forget that there was this dual space correspondence. And we actually were working with models that were more flexible than any neural net you can fit in memory and finding we were achieving very good generalization, especially on problems with a relatively small number of data points. So now to get back to your question, Radford Neal also had an interesting quote, I believe, in his PhD thesis, that whenever you have

• 40:21Speaker 03

  a simple model that performs well, you can always build a more complicated model around it that will perform even better. And he gives this example of handwritten character recognition, where you might have irregular writing styles, weird ink blots on the state, on the page, just some sort of structure that you probably haven't already accommodated in your model, that you can try to accommodate and you'll achieve better and better performance. So what I would do in this situation you described is really try to understand what's the inductive bias there that's leading to good performance? And how can I soften it in some way? How can I generalize this in a way that still honestly represents my beliefs?

• 40:58Speaker 05

  So but there, there seems to be something wrong with, with this quote, that you can always build a more complex model because Maybe you can always build a more complex model that has a higher likelihood on the training data, but overfitting is a real phenomenon. Like, nothing in your work says that overfitting doesn't occur and can be harmful. So it's like there have to be situations where if you diverge from the ground-truth model, I mean, I'm sure I could just build a system that has a ground-truth model, simulate data, and provably show that a more complex inference doesn't generalize as well. So that has to be true. So how do we know in reality when we've stepped too far?

• 41:38Speaker 03

  It's a great question. I think we have to be careful about what we mean when we say a complex model. So I think most people would not consider Gaussian processes with an RBF covariance function, just the standard covariance function kernel that's often used, to be a complex model. But it's highly expressive, so it's more expressive than any neural net we can fit in memory. It just has very, very strong preferences for certain types of solutions over others, and this enables it to be extraordinarily data-efficient. So one of the main use cases these days for Gaussian processes is in something called Bayesian optimization, where you're trying to maximize some sort of black box objective, so it's not something you have a closed-form expression for. Like, it could be generalization performance of a neural net as a function of some of

• 42:21Speaker 03

  its hyperparameters, for instance, or some really costly physical simulation as a function of some parameters, and you basically want to query this objective as few times as possible in order to achieve a good result. Gaussian processes are an amazing surrogate model for this objective, and you use the uncertainty to do the exploration efficiently. So I think that you can have expressive models that aren't necessarily complex. They still have very strong simplicity biases, very strong preferences for certain types of solutions over others, but at the same time, they're representing a wide array of possible solutions to the problem. And I think that's how you can kind of reconcile what Radford-Neal was saying with what we see in practice around things like overfitting.

• 43:06Speaker 03

  So I think there's a common misconception that the expressiveness of a model and its inductive biases are at odds with each other. The more expressive the model, the weaker its assumptions in some sense, like the fewer inductive biases it has, the less data efficient it will be, et cetera. And what we found, which has been quite exciting, is that the larger you make, say, big transformers, actually, the stronger its inductive biases. Like, the models get both more expressive and they have a stronger simplicity bias. And so I think you can expand these two things together in some sense, and this is how you

• 43:48Speaker 03

  can avoid, say, overfitting and other sorts of issues with not achieving very good generalization. And I think one of the clearest demonstrations of this is in a phenomenon called double descent. So double descent is this phenomenon where typically on the horizontal axis, you have the expressiveness of the model, the number of units, for example, in each layer of a residual neural network. And on the vertical axis, you have generalization error. And so initially, generalization error decreases as you increase the expressiveness of the model and it's able to just fit the data better and capture more structure. And then it starts to decrease, it starts to go up, and that corresponds to some sort of overfitting. And then it decreases again, and that's why it's called second double descent, because

• 44:33Speaker 03

  of that second descent. And in that second descent, all of the models typically have about zero training loss. So the training loss just keeps going down as you increase the expressiveness of the model until roughly the number of parameters equals the number of data points. And what that means is that the larger models in that second descent cannot be generalizing better because they're more flexible, they're all fitting the training data perfectly. It has to be that the larger models have some sort of bias which is enabling better generalization. And it turns out that this is a simplicity bias, a compression bias that we can measure.

• 45:07Speaker 00

  The best part of waking up? A full cup of Folgers coffee and music on full blast.

• 45:34Speaker 06

  Shop Folgers Instant Coffee and more at your nearest retailer.

• 45:38Speaker 07

  Now streaming on Paramount Plus. I'm Dexter Morgan. I've been killing killers for half my life. It gets to you. You're having a midlife crisis. I'm not ready for that. He's back. I'm older. I'm wiser. And sharper than ever. Things are gonna get messy. But if I can't believe in myself, I can't believe in anything. Dexter Resurrection, new season now streaming on the Paramount Plus Premium Plan.

• 46:09Tim Scarfe?

  Yeah, let's talk through that a little bit. So, as I understand, your thesis is that there is some kind of generating function of the universe. And in some sense, it's quite simple. Francois Charlet talks about this. He talks about the kaleidoscope effect, that, you know, we have the generating function and then it gets composed together in a myriad of different ways. And we see the kaleidoscope and intelligent people can decompose the kaleidoscope back into the original generating function. And you've also said that natural data in particular is quite low dimensional, quite simple, and that neural networks prefer simple data. And even that, I want to take a slight issue with that because what I find is that neural networks in the early stages of training prefer simple data.

• 46:52Tim Scarfe?

  And when you continue to train them, they seem to complexify and complexify and learn more high-frequency data. So how do you think about that?

• 47:01Speaker 03

  It's a great question. So it's a really important observation that neural nets tend to learn structure before they learn noise and things like this. There's a question. Around to what extent being able to fit noise is hurting their generalization capabilities. So there's this other phenomenon called benign overfitting, where the model fits typically a mixture of signal and noise, but the noise, fitting the noise doesn't significantly degrade its generalization performance. And this is often seen as something that's specific to deep learning and in the face of everything that we know about generalization. And that's partly because classical frameworks for trying to understand generalization, like VC dimension and random macro complexity,

• 47:46Speaker 03

  are essentially measuring a model's ability to fit noise. However, there are other generalization frameworks, like PAC Bayes and countable hypothesis bounds that we explore, which don't penalize an expressive hypothesis space and instead try to understand what sorts of soft preferences the model has for certain solutions over others. And we've been able to achieve fairly tight bounds on the generalization performance of these large models using something called a Solomonov prior. And so a Solomonov prior says that we actually have a maximally overparametrized model. We can represent every possible program on a computer, that we have exponentially stronger preferences

• 48:29Speaker 03

  for solutions with, that have what are called low comograph complexity. And so they're very compressible. The comograph complexity is the shortest possible program that can generate our hypothesis. And the fact that we're able to get these tight generalization bounds for these very large models, and in fact, the models get sort of better, sorry, the generalization bounds get better as we make the models larger, suggests that this is not a bad description of how these models are actually behaving. So doing induction with a Solomonov prior is called Solomonov induction. And so it seems that when we make these transformers, for instance, very large, we're combining this expressiveness with this strong preference for low comograph complexity solutions.

• 49:14Speaker 03

  Another observation that's been made is that models are becoming increasingly general purpose. And so 20 or 30 years ago, the

• 49:26Speaker 03

  Typical prescription was to encode as much expert knowledge as possible into the model you're constructing and tailor it very specifically to the problem that you're considering because of results like the no free lunch theorems that say that every model is equally good in expectation over all problems sampled uniformly from this distribution over all problems. There are several no free lunch theorems. Another one says that a single learner is not going to be good on all problems. Now, these results are mathematically correct, but they don't really correspond to the real world data generating distribution. The real world is a small corner of all possible data sets. It's not drawn uniformly from that distribution. If you were to draw data sets from that distribution, you would mostly get noise. And so I guess the question is, well, what

• 50:11Speaker 03

  is the real world data generating distribution really like? And it seems like there is a bias towards generating data with low comagrof complexity, and our models share that bias. And this is why we've seen increasingly general systems. So we've moved from feature engineering, basically hard coding structure into our models, to more modality-specific models and architectures, so convolutional neural nets for vision, recurrent neural nets for sequences, language, etc., MLPs for tabular data and regression, to transformers for almost everything. And so this isn't to say that transformers have achieved general

• 50:53Speaker 03

  intelligence, absolutely not, but they're relatively speaking more general than the predecessors. And we have seen this kind of movement towards increasingly general models. And our contention is that this has been made possible by aligning with the real world data generating distribution, which seems to have a bias for low comagrof complexity. And we had this paper on no free lunch theorems.

• 51:20Speaker 03

  Inductive biases and Kolmogorov complexity. And I think one of the most surprising findings in that paper was that convolutional neural nets, which were clearly designed for image recognition, so they have locality and translation equivariance and so on, provably have inductive biases for tabular data shaped as an image. And the only possible reason that could be the case is because they both sort of share this bias for low Kolmogorov complexity. And that bias gets stronger as we make the model bigger.

• 51:48Speaker 05

  Okay, so I have actually two questions about this. And it's about the Kolmogorov complexity. So I think if I heard you correctly, on the one hand, you're saying that just stock neural network training of today, so just transformers, SGD, batch norm, whatever people are doing, seems to empirically exhibit bias towards lower Kolmogorov complexity models. And this shows up by their generalization kind of falling within, you know, this bound, you know, that you found. And then you also mentioned Kolmogorov induction, which I believe would be explicitly introducing, you know, a kind of penalty term or a risk term and objective, part of the objective function that has to do with Kolmogorov complexity and kind of maybe pushing the model a little

• 52:33Speaker 05

  bit further towards, you know, simple. I think you're talking about both. So maybe if you could elaborate and also, you know, where is this bias towards simplicity coming from? Like, everybody knows the algorithms. Like, which part of the algorithm is inducing this simplicity bias? Is it because we're using floating point numbers, IEEE, or what? Like, where's it coming from?

• 52:55Speaker 03

  This is largely an open question, although there are some intuitions, and this is something I'm really excited about pursuing further in my research. Where does this simplicity bias, especially from scale, originate? There are some intuitions, so geometric intuitions around the loss landscapes for the objectives that we use to train these models. And so when we're sort of minimizing training loss, we can try to geometrically understand the properties of this landscape that we're minimizing. So it's been observed, for instance, that flat solutions, meaning solutions where you can perturb the parameters by some amount but retain low training loss, tend to generalize better than sharp solutions that have the same value of the loss. And you could make a compressibility argument for why that's the case.

• 53:37Speaker 03

  Flat solutions don't need to be represented with as much precision, and so they're more compressible.

• 53:43Speaker 03

  As you grow the size of these neural nets, the relative volume of these flat solutions starts to exponentially dominate the volume of the sharp solutions. And so you can imagine heuristically.

• 53:54Speaker 05

  Why is that? I mean, do we know why? Or it's just an empirical observation?

• 53:59Speaker 03

  Yes, so we know why to some extent, but our understanding is somewhat heuristic. And so you could imagine, for instance, having a region of the loss surface with radius RA that's flat. So you perturb your parameters within that radius, and you have a low value of the loss. And another region with RB that's, you know, where RB is much less than RA. Now, as you grow the number of parameters in your model D, RA to the D starts to really dominate relative to RB to the D. And we seem to observe this in practice. So there was a result, not from my group, from Tom Goldstein's group at the University of Maryland, where they were trying to understand

• 54:43Speaker 03

  the role of the implicit biases of SGD and stochastic optimization in generalization. And so quite often, as you perhaps have alluded to, SGD is often thought to be a really integral component of achieving generalization in deep learning. There's this idea that we have these very complicated objectives that we're minimizing that are very non-convex, et cetera. And SGD somehow saves us, that the implicit biases of SGD navigate our procedure through some region of the loss landscape that represents low-loss solutions that do generalize, rather than low-loss solutions that don't generalize well. Well, it turns out you can actually do full-batch gradient descent and achieve pretty comparable generalization

• 55:27Speaker 03

  to what you would get if you were using SGD, even if you don't try to make that implicit regularization explicit in the loss. And there was another paper that showed that if you even do guess and check, so you just randomly sample your solution vector and then stop when your loss is below a certain threshold, the generalization will also be fairly comparable to what you get if you use SGD or Adam. And so that corresponds to this geometric intuition. If you're just sort of throwing darts at the loss landscape and then you stop as soon as you have loss below a certain threshold, you're much more likely to be in this region of low loss and good generalization than low loss and bad generalization as you increase the number of parameters in the model. Now, this is not an airtight argument, so there are lots of ways that you can increase parameters in models

• 56:12Speaker 03

  and not really influence the geometric properties of the loss landscape. However, it does seem to correspond to what we observe empirically. And not just in terms of results like guess and check. If we look at something like double descent, in that second descent, we can measure something called the effective dimensionality of these models. So that's the number of relatively large eigenvalues of the Hessian, which is sort of the number of sharp directions essentially in the loss landscape. And so that decreases as we make the model bigger.

• 56:41Speaker 05

  So interesting. And to the second question about explicitly introducing, you know, a penalty term for a Kolmogorov complexity. Have you done much of that, looked into that? Is that useful or just not necessary?

• 56:53Speaker 03

  It's something I've been thinking about. It's very hard to operationalize. So you can evaluate these bounds by computing an upper bound on Kolmogorov complexity, measuring the compressed file size of your model after training. But in order to use that prior as some sort of regularizer, you would need to be considering sort of compression of some whole set of different hypotheses, not just a single hypothesis that's found by the model. And so there's an open question of how you could try to operationalize that. Like Solomonov induction is sort of, represents an idealized learning system. It's not something that we can really do exactly in practice. It seems that neural nets are sort of approximating it.

• 57:38Speaker 03

  And that's evidenced by these bounds and how they're able to tightly characterize generalization behavior of these models. But it's hard to sort of turn into some kind of regularizer. I'm also interested in how we can go beyond things like Kolmogorov complexities. So.

• 57:56Speaker 03

  Comlograph complexity doesn't distinguish between incompressibility due to randomness. So like noise, for example, in our data versus structural complexity. And Scott Aronson actually had a really interesting blog post related to this about 10 or so years ago, where he was imagining a system, he had this kind of physics analogy, where you have coffee and cream and initially they're separated liquids and you start to stir them together. And as you do this, the entropy of this system is increasing.

• 58:25Tim Scarfe?

  I've seen that one. That was shared on our Discord recently.

• 58:28Speaker 03

  Yeah, very nice. Okay. Join our Discord. I'd love to, yeah. We actually have a, yeah, I don't want to get ahead of myself. We have an idea, of course, for how we can do this. But so the blog post sort of compares this to, so the entropy of the system is increasing, the comlograph complexity is increasing. But the intuitive sophistication of that system is kind of non-monotonic. Initially, it has low entropy, low sophistication, then sort of intermediate sophistication and entropy. And then again, sort of like, I'm sorry, and then high entropy and kind of low sophistication. And so you can think of this in machine, in a machine learning context in terms of reasoning about the value of data. So if I sample data from like a uniform, random uniform distribution,

• 59:12Speaker 03

  that's going to be very incompressible. I'm going to need to memorize it. It's sort of uncorrelated. This could be useless for learning a representation, for training my model. I could alternatively imagine some sort of sophisticated cellular automaton problem, some sort of game of life problem with very sophisticated generalization or generation rules. That data actually could have an extraordinary amount of value for learning a representation. There was a paper that looked at something briefly like this called Intelligence at the Edge of Chaos. And I think there are other results like that that are coming out that like, you actually might want to train your models on this data with a lot of structural complexity, even if you really want, you know, in the end, the model has some kind of Occam's razor bias, et cetera. And so we've been thinking about kind of measures of information that might compartmentalize structural complexity and random complexity.

• 59:58Speaker 03

  And this will help us reason better about the value of data and developing priors sort of that are like Solomonov priors, but might actually be more directly addressing the type of incompressibility that we're interested in.

• 1:00:10Tim Scarfe?

  Yeah, I mean, so you said you said so many interesting things. I mean, first of all, to Keith's point, you were saying that this isn't a penalty term yet. Your hypothesis is that neural networks implicitly do this kind of compression, which might be correlated or related to this kormagra complexity. So many folks just kind of, they equate intelligence with compression. And when I spoke with David Krakauer, he took umbrage of that. He said, you know, compression is a component of of intelligence, but there are so many other things going on as well. And certainly, when we look at things like the ARC challenge, there are so many possible solutions. So a naive heuristic of just selecting the simplest program isn't always the best thing to do. There are many possible selections you could

• 1:00:52Tim Scarfe?

  make. And, you know, so you demonstrated this upper bound, which used this complexity term. And in a sense, that's saying that it could be no worse than this, rather than it could be, but it could actually be so much better. And our empirical experience of deep learning models is that they they seemed, you know, we call it shortcut learning, basically, you know, they seem to have found some superficial generalizing thing which does all the things you said when you mentioned the no free lunch theorem. So you could take a CNN and you could use it on tabular data, you could take a transformer and you could use it on audio data. So it's almost like what we've seen is that we've we've hit this, for want of a better word, local minimum. And it feels like we need something more

• 1:01:35Tim Scarfe?

  to get to the real understanding of some of these problems. Does that make sense?

• 1:01:40Speaker 03

  So is compression intelligence? I think this is a big debate right now. I think it is very closely associated with intelligence in a lot of ways. If we can compress our data really effectively, then in order to do that, we're discovering regularities that are typically going to help enable generalization. And in some sense, physical laws, for example, are a great compressed representation of reality. And so I think that compression is really intricately connected with what we mean when we talk about building intelligent systems. There are instances in which this can go very wrong, like in shortcut learning, where, for instance, you might have some spurious correlation, like maybe every time there's a blue pixel in an image, the label is a bird or something like this. And so

• 1:02:24Speaker 03

  the model, you know, forgets about the foreground and it just looks for some feature in the background. In terms of generalizing on that distribution, that's actually not a bad idea. That actually is the right thing to do. So Occam's razor is really a good principle. But if we go out of distribution, so we see like a bird in, you know, with a volcano or something behind it, or in a room, then, you know, it's not going to arrive at the right label. So there's this question of like, are there other principles of induction that will lead to greater robustness under distribution shifts? I think in general, Occam's razor still is the right thing to do. Like, there are many cases where a compression will actually not give you what you want more broadly if

• 1:03:09Speaker 03

  you move beyond that distribution. But in an absence of additional information, it seems like you can't really do better as a guess as to what's the right strategy. And so I still think Occam's razor is a very robust principle of induction. And perhaps one answer is just more data. I think this will work well in some cases and not in others. So we have some work in progress on trying to build transformers for matrix operations. So this is actually kind of analogous to the work that people have done on transformers for multiplication, addition, things like this. So I guess, you know, these systems,

• 1:03:49Speaker 03

  L L M, seem very impressively effective in some instances in being able to write and code and solve a variety of different problems we didn't really expect them to be able to solve. In other instances, they're just shockingly bad. So, you know, how many Rs are in strawberry, like reversing strings, counting, and just basic addition and multiplication, etc. And so there's this debate, I think, about whether we should be giving them tools to do those things. Like, why not just give them a calculator? I mean, whatever they learn to do in terms of adding and multiplying numbers is still not going to be as efficient

• 1:04:30Speaker 03

  as giving them access to a calculator. So why don't we just do that? Or like, you know, is there some sort of greater value in having to sort of learn a representation that can do some of these things? Because maybe even though we'll always want to use a calculator if we're multiplying or adding numbers, being able to do something like that reasonably well might transfer into other settings that we don't really anticipate. And I'm probably more in this category. Like, I just think intellectually, we should try to figure out how to do this without tools. And so This work that we're doing on transformers or matrix operations, I think, is quite analogous. So you could argue that addition and multiplication are just fundamental primitives for trying to build intelligent systems.

• 1:05:10Speaker 03

  We're not as good at it as a calculator is, but it might be important for us to have some ability at being able to do these things. The same could be said of matrix operations. This is really the backbone of all sorts of different learning algorithms like Gaussian processes, which we discussed a little, involve solving linear systems with a covariance matrix, computing log determinants, et cetera. Degenerative models like normalizing flows involve log determinants. Dimensionality reduction like PCA, et cetera, involve other matrix operations. And so they're just really ubiquitous as kind of a primitive for trying to build learning algorithms and intelligent systems. And so I think if transformers are ultimately going to become some sort of general intelligence, then they ought to be able to be competent at these types of operations.

• 1:05:55Speaker 03

  And so we were kind of representing then matrices as sequences of numbers and then having the outputs be things like the maximum eigenvalue or, you know, a solution to a linear system or whatever other operation we were considering, the spectrum of eigenvalues. And we found, interestingly, that when you train this approach on Gaussian random matrices, you just sample every entry of the matrix from a standard normal distribution, it will do fairly well in distribution matrix operations. So other matrices sampled from that distribution that it hasn't seen before, but extraordinarily poorly, even if you go slightly outside of that distribution. So given an identity matrix, all ones on the diagonal, zero everywhere else,

• 1:06:39Speaker 03

  that'll have pretty low density, has support under this Gaussian distribution of matrices, but pretty low density under it. Hasn't seen something very much like that. It will just completely fail. It won't do anything reasonable. And so we considered a number of interventions like looping, sort of adaptive test time computation, enriching the training distribution very significantly. So having sort of this like einsome space of structured matrices that we were sampling from. So all sorts of different matrix structures, topless, chronicer, et cetera, block diagonal, low rank, and a variety of other interventions. And interestingly, we found, once we had done that, This approach actually was able to generalize even to matrices that were

• 1:07:23Speaker 03

  out of distribution for this fairly rich sort of training set that we've created. And so it seemed to move more towards learning an algorithm rather than just doing statistical interpolation on the training data. And so this is, I'm an optimist by nature, so I am very happy about the fact that we can use data to move towards things like algorithm discovery. But now I remember why I started talking about this. So I think there are instances where it's going to be hard. So autonomous driving is an example where like you have outliers, but they're different each time. So just training on the outliers isn't going to be useful because there are going to be new outliers that look very different from those outliers. And I just have the intuition that more data alone is not really the answer to building robust autonomous driving systems.

• 1:08:07Speaker 05

  Yeah, I just want to say, as far as why not just give them tools? I mean, my answer, folks, is because those have to be programmed and built by people. And the whole point here is to allow machines to do their own programming, right? Machine learning. And if they can't, if they can't even learn to do multiplication reliably and to generalize from decimal multiplication of binary or hexadecimal and nine digits to 36 digits, you know, what hope do we have that they're going to discover relativity or non-Newtonian mechanics or any other kind of frontier, frontier things, right? I mean, isn't that part of the goal here?

• 1:08:42Speaker 03

  Absolutely. And I think being able to do certain things while we're finding might surprisingly relate to doing other things very well. And so I think we have seen this to a large extent with LLM. So we had a paper where we just took a text pre-trained LLM off the shelf and applied it to time series forecasting. So we did everything naively. We weren't even really intending for this to be like a proper project or a paper. We were just curious, you know, if you just give GPT like a sequence of numbers naively encoded to strings and have it extrapolate the next sequence of string tokens, how would it compare to purpose-built time series forecasting procedures? And it just worked way, way better than we thought it could possibly work. It didn't even really make sense. And so we did sort of turn this into a proper project.

• 1:09:26Speaker 03

  We did a little bit of work on trying to improve the tokenization and think about uncertainty representation, et cetera. But most of that paper, it's called Large Language Models or Zero-Shot Time Series Forecasters. Was focused around trying to understand how this is even possible. And in the end, it did start to feel a bit more like, maybe it's not just that you can do this, maybe you should do it in some instances. Like, they did quite well on a variety of benchmarks. And to me, this suggests with like, a proper dedicated research effort on LLMs for time series, you know, we could see these systems actually working a lot better than the purpose-built models. And so what that shows is being able to predict the next words in sentences can actually transfer to being able to do other things, like time series prediction really well.

• 1:10:10Speaker 03

  And we had another paper on LLMs for materials generation, which was kind of similar. So we took a text-pretrained LLM off the shelf, like a Llama-2 model at the time, and we fine-tuned it, in this case, on atomistic data represented as text, so locations of atoms and energies and things like this. And the resulting system was able to generate inorganic crystals with favorable properties better than these purpose-built approaches, and even foundation model approaches that had been trained on that domain-specific data. And so one of the takeaways from that project was like, the text-based pre-training was an indispensable component in being able to achieve good results on materials generation. And we tried to understand also in that paper why that was the case.

• 1:10:54Speaker 03

  We were collaborating with some chemists at Fair in California, and they were just very curious about LLMs and foundation models, so they were willing to sort of humor us a bit and help us sort of see what we could do, but they were just like skeptical throughout the project until we saw the results. And it's like, well, you can't deny the results are great, you know, why is this happening? And part of it was that in being able to predict the next tokens in strings, you are learning principles of induction, like Occam's razor. And so how do those sort of manifest themselves in context learning? So what it means is if, or and in fine-tuning. So what it means is these models are going to be predisposed to discovering Compressible representations. And that means, for example, salient symmetry. So this was a problem where there was a rotation invariance. And these models actually were very quick to learn these kinds of invariances

• 1:11:39Speaker 03

  because of the text-based pre-training. And it sort of instilled this principle. And so I think this is also an example of how compression can be a broadly applicable principle for induction. It's at sort of the right level of abstraction that you can start to see more relatively universal behavior. So for instance, there are some theories on the successive foundation models that suggest that different problems are just different projections of some underlying realities. So like platonic representation hypothesis is an example of this. And so you can represent an image with pixels or with words and, you know, train the respective models on those different modalities, and they learn similar representations. I think that can be true in some instances.

• 1:12:21Speaker 03

  But I also think different problems are often truly quite different from each other in terms of their low-level structure. So like if we have molecules, there's rotation invariance, maybe some other problems, some image recognition problem, like character recognition. We might have translation invariance. Doesn't matter. If the 2 is on the left of the screen or the right of the screen, the label is still a 2. These are very different low-level feature representations. So like the architectures that you would typically use for each of those modalities and to respect those different types of invariances would look very different. But what they have in common with each other is they're both ways to compress the respective problems that they're being applied to. So if you have a mo-

• 1:13:00Speaker 01

  Hey, listen! Power up your play with Nintendo Switch 2 and experience Link's legendary adventure, now reborn in vibrant detail on our fastest and most powerful system yet.

• 1:13:14Speaker 01

  And harness the power of the Ocarina with the built-in microphone.

• 1:13:22Speaker 01

  The Legend of Zelda: Ocarina of Time, a Nintendo Switch 2 exclusive. Game and system sold separately. Game rated everyone 10 and up.

• 1:13:30Speaker 02

  LinkedIn is pretty amazing at helping you grow your small business. We cannot make your email response time faster. We can help you sell, market, and hire in one place. We cannot help you find space for your three desk drinks. Why do you have three? And while we can't help you find the perfect volume for your presentation video, LinkedIn can help you find the perfect audience for your business. Grow your small business on LinkedIn. Learn more at LinkedIn.com/smallbusiness.

• 1:14:00Speaker 03

  Model that has this compression bias, then it can discover those salient symmetries. And we had this really surprising finding that vision transformers actually can be more translation equivariant than convolutional neural nets after training, which sounds impossible because conv nets by design are translation equivariant. But they're not exactly translation equivariant because of aliasing artifacts and edge effects and things like this. And so this other model, this transformer with no explicit constraint whatsoever, just a soft bias that manifests itself increasingly at scale, is able to discover a solution that has lower equivariance error than the convolutional neural net, which is just absolutely remarkable. So to come kind of full circle to your question, I think that we're discovering more and more

• 1:14:45Speaker 03

  that being able to solve certain problems really well will translate in perhaps unexpected ways to being able to solve other problems well. And so I think especially when it comes to things like addition and multiplication and matrix operations, like these are just even like obviously like fundamental primitives for building intelligent systems. Like even if we can use tools for those things, we want our representations to be somewhat competent at them because it's going to be useful for all sorts of other things we probably haven't anticipated.

• 1:15:13Speaker 05

  So Occam's razor has been mentioned multiple times in the last few segments. So I want to dive into that bit because I love Occam's razor. And I first came to really understand it when I became a Bayesian. And it's nice to have a fellow, you know, arch-Baysian to talk with here because... In Bayesian inference, if you will, Occam's razor has a very explicit form, like mathematical form, and it's marginalization, right? It's saying like if you have all these parameters around and you you compute the average, you do this integration over all these parameters, that really that's telling you like the probability of the model, like given your, you know, full parameter space having been integrated away.

• 1:15:58Speaker 05

  And that's really where you get, you know, let's say a penalty for complexity comes into play with with marginalization. It's like if you make if you expand your model's flexibility or you make it more complicated without any gain in inference, well, then that kind of like counts against you or without any gains of some weird simplicity in the form of, you know, less curvature, you know, things like that. And I was looking at some of your talks, which were really great from from five years ago, these Bayesian tutorials, Bayesian deep learning tutorials you had, I think, when you first when you first went to NYU. I really enjoyed them. And there was a lot of talk about marginalization and the importance of it and the intuitions that you gain from that.

• 1:16:39Speaker 05

  And I think but more recently, that's played less of a role in the conversation or people have just given up on trying to do the integrals and they're just back to doing maximum likelihood and maybe with some hacks and the objective function. So I'm just kind of curious, you know, as a Bayesian, your journey going from understanding like the beauty of marginalization and the Occam's razor built into Bayesian inference versus what you do as a practitioner, what you see people doing in practice, how it's playing out in the field or not. And maybe is there a future in which marginalization and doing these computational intractable integrals or something could play a role in driving like even better, you know, machine learning and inference?

• 1:17:22Speaker 03

  I'm so glad you asked. So there's almost nothing I like more than talking about Bayesian inference. And I'm so glad you said marginalization because I feel like that's often overlooked. Like when someone says Bayesian, probably the word that comes into your most people's mind is prior and is the prior good? And how do we know what would a good prior be, et cetera? And they get worried about that. But really, the prior is not the defining feature of what it means to be Bayesian. Instead, being Bayesian means that you want to represent the honest belief that you have uncertainty over what solution is correct, given a finite data sample. And so if I'm trying to estimate the bias of a coin that I'm flipping, if I flip it once or twice,

• 1:18:04Speaker 03

  regardless of whether it comes up heads twice or something like this, I ought to have some uncertainty over the bias still. And that uncertainty is manifested through this procedure called marginalization. So I think we can think of regression to get some intuition of this. So we could imagine like a bunch of different points on a sort of Y-X plot. And maybe the points look roughly like they're on a straight line, but not quite. We could also imagine that there are many different curves that will perfectly run through all of those points. They all look different from each other. Some of them will be very wiggly, some of them will be slowly moving, et cetera. Some of them will basically just be like a straight line. And we wouldn't be able to say, well, we know with 100% certainty, it's this curve that's the right description of our problem.

• 1:18:49Speaker 03

  But that's exactly what we're doing 99, 100 minus epsilon percent of the time when we're training models in deep learning. That is not an honest representation of our beliefs. And it's going to become a bigger and bigger problem the more expressive our model actually is. Because that means there are going to be many more different settings of parameters that are consistent with what we observe. And we're just betting everything on one of them. And probability theory says, no, that's just wrong. That's not what you should be doing. The sum and product rules of probability say you should be doing marginalization. And so that's all to say that Bayesian marginalization, basically looking at all possible solutions that can be expressed by your model class, weighted by their posterior probabilities, is going to be most important when we have a model that's very expressive, has a lot of parameters.

• 1:19:34Speaker 03

  So deep learning, for example, especially relative to the number of data points that we're considering. And I think often people think of it in the opposite way, that like, oh, maybe Bayesian methods are most relevant in the realm of classical statistics. Like if you're doing some logistic regression and maybe you want to represent some uncertainty, et cetera. But it's not really suited for deep learning. It's really the opposite. The challenge then is how do we do this in a way that's computationally tractable? And I think like with most things in life, the answer, of course, is nuanced. So. It's not that like we can either be fully Bayesian or not Bayesian at all. Let's just try to do what makes sense, given the resources available to us. And so like if we're just doing standard training, you can actually view that as a very crude

• 1:20:18Speaker 03

  form of marginalization, where you're saying, okay, I'm representing the posterior as a point mass around the most likely setting of parameters. And then we can say, okay, well, maybe the posterior, which is really like, so the loss functions that we're minimizing are basically negative log posteriors. Maybe the posterior looks nothing like a point mass and it looks nothing like a Gaussian. It's very multimodal, it's very messy, but we can still do a better job of representing it with a Gaussian than with a point mass. And so let's use a Gaussian approximation. And indeed, when we do that, we often see better generalization because we're representing all these other complementary and compelling explanations to our problem. And then we can keep taking it from there. Okay, maybe we can develop some MCMC procedure that will explore the loss landscape and

• 1:21:02Speaker 03

  capture something much richer than just unimodal Gaussian structure, et cetera. And again, we see improvements in performance in doing that. And so I actually think that Bayesian methods have been an extraordinary success story in deep learning and beyond. But we don't hear as much about them now as we did maybe 10 years ago. And I think there are a number of reasons for that. So to some extent, they're a victim of their own success. So I think there was some low hanging fruit in being able to do better approximate marginalization in neural nets around 2015. And there was really a lot of progress between about 2015 and 2020 in achieving increasingly

• 1:21:43Speaker 03

  better results that will also be kind of computationally efficient. So there are procedures, like one we developed called SWAG, for instance, which was based on some insights we had into the structure of these loss landscapes that will allow you to do Bayesian marginalization without really any significant additional cost on training. It's a bit more expensive at test time and so on. It's also deep kernel learning, which basically just requires a single forward pass through the model and you get sort of some representation of epistemic certainty. And these procedures were adopted in practice. And so if I go like to some more domain specific conference, like some workshop or conference on materials engineering and things like that, I'll see lots and lots of talks using Bayesian

• 1:22:24Speaker 03

  optimization, Gaussian processes, neural networks with epistemic uncertainty, representation, etc. So this is really useful in practice, but it's very hard to kind of go beyond the useful approximations I think we developed without doing a significantly, making sort of a really significant, you know, 10 year kind of style, moonshot landing kind of investment in those directions, which I think is worth making. Secondly, there's the advent of LLMs, and that did sort of change things in practice. So like, we went from million parameter models to billion parameter models. And the role of epistemic uncertainty is also a little

• 1:23:05Speaker 03

  bit less clear, I think, in some instances when we're working with LLMs. And so this is just a real technical challenge, and I think we're just sort of barely starting to scratch the surface of how to think about it. But the motivation is absolutely there. I think it's really greater with these big models than it is for virtually any other model class. And in this sense, I don't know how anyone could not be a Bayesian. Like, how could anyone say, I don't want to represent epistemic uncertainty. I don't like using jargon. So there's like, there's aleatoric uncertainty, which means irreducible uncertainty, often associated, if we have given a data set, we have some noise on that data set, there's nothing we can do about it. That's aleatoric uncertainty. Epistemic uncertainty is uncertainty that's reducible with more information. And we believe that uncertainty

• 1:23:47Speaker 03

  is there. And there's even an argument that that might be the only uncertainty in the world, right? Like, things that seem intrinsically random, like I could roll a dice, and it might seem like, okay, there's a 1/6 probability that it'll land on any one of the sides. But if I had enough information, like if I could sort of exactly know the strength of the throw, the wind in the air, the friction on the table, and I had the right physical model, I should be able to predict exactly what side it's going to land on with each throw. So that's just saying, okay, the more information we gather, the less uncertainty we have, even for this process that somehow seems intrinsically random. And physicists, modern physicists now, you know, probably do believe that some aspects of the universe are intrinsically random, radioactive decay and things like this. But Many others have not historically, like Einstein was a determinist.

• 1:24:30Speaker 03

  And so he would have believed that the only uncertainty there is is epistemic uncertainty. And I think for practical purposes, that's not an unreasonable belief. And so it's absolutely crucial that we try to model it in some way to not do it is just to do something that's mathematically incorrect and could come at a huge cost. There's also a change, I think, that's happened in this movement to L L M's and the way that people think about data. So it used to be the case that you kind of have a fixed data set and then you throw everything you can at it to get the best possible performance. Maybe you have some scientific problem and you just really want to do well and you're limited by computation and other things to some extent.

• 1:25:09Speaker 03

  But it's not the case that you have this sort of trade off to manage for a given computational budget between the size of the data and the size of the model. And so that's described by these scaling laws, like Chinchilla scaling laws. And that is starting to become the assumption in mainstream machine learning, that you have almost like an arbitrary amount of data and you have a fixed computational budget and you want the best possible performance under that computational budget. In that case, you know, rather than trying to be more Bayesian or something like this, it could actually make sense just to use more data.

• 1:25:43Speaker 05

  But the other huge benefit though is because you mentioned, you know, that for whatever reasons that we're still trying to understand these large, you know, large neural networks, deep neural networks have this kind of simplicity bias towards simplicity, right? And marginalization is the ultimate Occam's razor. You know, it's like Occam's guillotine or something that really can force models, you know, to force you to select, let's say, the simplest model that's consistent with the data in a sense. So I think there's possibly more to gain there, like just, you know, trying to understand in computationally tractable ways, how to pull in more of those effects of marginalization, like on inference. I mean, is that true?

• 1:26:26Speaker 03

  Absolutely, yes. So Bayesian marginalization has this automatic Occam's razor bias. And I actually started working on Bayesian deep learning ironically when I saw a talk on optimization, which might be viewed as almost the opposite of being Bayesian. So Jorge Nocedal gave a talk at Cornell where I was initially a faculty and he was presenting on small batch biases, basically of stochastic optimizers. And he had this figure where he was contrasting flat optima with sharp optima and he had this horizontal translation of the loss landscape. And under that translation, the flat minimum still had reasonably good test loss, but the sharp minimum had very bad test loss. And so he was saying, oh, this

• 1:27:09Speaker 03

  is why we want to do stochastic optimization with small batches, because it's going to be more likely to find these flat solutions. And in some sense, like the other optimizers, like his LBFGS and so on, were like too good. They were like converging to these sharp minima. And if like you just care about minimizing loss, then maybe that's not bad behavior. But generalization depends more than on just getting a low value of loss, at least with the loss functions that we're using. And so I saw that figure and I thought, well, this is actually really great motivation to be Bayesian, because if you're being Bayesian, you're not just betting everything on one solution, you're integrating under a sort of a flipped version of that curve. And so automatically, most of the volume will be in these flat regions. And that's so much more elegant than approaching this from the optimization perspective, where you have to say then, okay, I need to rigorously define

• 1:27:53Speaker 03

  what it means to be flat. Like, maybe it's going to be like the largest eigenvalue of the Hessian, maybe it's going to be

• 1:27:59Speaker 05

  HCG tensor or something like that.

• 1:28:01Speaker 03

  Exactly, yeah. Maybe flatness in random directions versus sharpest directions. Maybe it needs to be parametrization invariant. Like, that's been often a criticism of discussions around flatness, etc., that it's most ways of measuring flatness aren't parametrization invariant. I have thoughts about that separately. But anyway, you might then look at Fisher matrices or something. Okay, so then, once you've decided on what it means to be flat, and no one will completely agree with you, or most people will strongly disagree, whatever you choose, then you have to decide, like, how much am I going to penalize sharpness? No one will really know what to do about that either. And like, the idea is you want to build a loss function that is a better proxy for generalization by accounting for things like flatness. But it's just this really messy rabbit hole. Whereas if you're just doing marginalization, this is all happening under the hood. You don't need to worry about

• 1:28:46Speaker 03

  it. And so that's really elegant. And then there's Occam's razor in terms of model selection. And I'd recommend that everyone check out chapter 28 of David Mackay's Information Theory and Inference Learning Algorithms book. It's titled Occam's razor and I don't agree with some of the stuff in that chapter. And in fact, we wrote a paper about this on Bayesian model selection and marginal likelihoods, but it's a very beautiful description of automatic Occam's razor. And he has these just extraordinary visualizations to try to demonstrate how that's possible. So he has like these also, you know, everyday examples as well, like this idea that maybe you have like a block behind a tree, but if you had X-ray vision, it would appear to be like two blocks of equal height and color or 10 blocks or something like this. But given that you don't have X-ray vision and you don't think this is a trick question,

• 1:29:30Speaker 03

  you'd be pretty confident it's got to be one block. And this seems like some manifestation of Occam's razor. And you might try to rationalize this by saying, well, it would be just a remarkable coincidence to have two blocks standing next to each other of equal height and color. But he argues that this is actually a quantifiable consequence of doing Bayesian marginalization. And so he basically has this conceptualization where you have all possible data sets on a horizontal axis and the probability of generating a particular data set under your model on the vertical axis. And that's the marginal likelihood, the probability that you would generate your training data under your model prior. And so if you have the one block model, you're not going to be able to generate very many different data sets.

• 1:30:12Speaker 03

  So most of the data sets on the horizontal axis have no support, no P D given M. But the ones that it can generate, it's going to have to give a lot of probability for it because this is a proper normalizable probability density. Similarly, if you have the 10 block model, you can generate all sorts of different types of observations, but you're going to have to sort of spread that mass more thinly because this is like a proper normalizable probability density. And so for a particular data set that's consistent with both of these models, like the block behind the tree example, the one block model is actually going to have significantly more probability. And this is even not factoring into account the idea that maybe we have some prior preference for simplicity and that there's like, we think that one block is more likely than 10 blocks or something like that. He's just like, just forget about this ratio of prior odds and only consider this ratio

• 1:30:57Speaker 03

  of marginal likelihoods. And so it's just a beautiful demonstration of how Bayesian inference automatically encapsulates a notion of Occam's razor. And this is just a fundamental question in science, like. If you have arbitrarily many hypotheses that are consistent with any number of observations you can ever record, how do you choose between them? What's the principal way of approaching that problem? And the answer, to some extent, I think, is given by Bayesian marginal likelihoods. There are very subtle issues, I think, with some of the ways that this can be done. And so we wrote a paper all about that. But largely speaking, it's something that I think people should acquaint themselves with because it really is sort of getting at something very fundamental and it has extraordinary practical value.

• 1:31:40Speaker 05

  What was that paper?

• 1:31:41Speaker 03

  The one that we were sort of looking at. So we have a paper called Bayesian Model Selection, The Marginal Likelihood and Generalization. And so that paper is really trying to sell two sides of a story. On the one hand, it's trying to convince the readers that the marginal likelihood is something quite extraordinary and that there's a reason we should be interested in it. Because if it isn't, then if you haven't heard of it, like, why even read criticisms of it if it's something that doesn't matter? The other part of the paper is questioning whether it's answering exactly the question we want to be asking when we're doing model selection towards trying to achieve the best possible generalization.

• 1:32:21Speaker 03

  And so the question that the marginal likelihood answers is what is the probability that my prior-generated the training data? That's different than what is the probability that my posterior, after I've observed the data, is going to lead to reasonable predictions? And so we can construct examples that really illustrate this difference. Like, you could have a uniform prior over solutions that you expect to be easily identifiable from the data. And so the posterior will contract very significantly around something that will actually be quite reasonable and make reasonable predictions, but the marginal likelihood will be really bad. And so we construct all sorts of examples where there's actually a misalignment between the marginal likelihood and generalization for these reasons.

• 1:33:06Speaker 03

  You could also overfit, even though you have sort of some robustness to overfitting. Like, if you're considering arbitrarily many models, you could just get super unlucky and have a model which is like a point mass prior around something that can only generate that training data set but isn't going to do anything reasonable. So the marginal likelihood might prefer that, but it's not going to lead you to a model that's going to make good predictions. It is very useful as sort of a heuristic for model selection in many instances. It's got incredible practical value in learning things like hyperparameters that control complexity in Gaussian processes. And I think it often is the right tool for scientific hypothesis testing, which is subtly different. And so you could, this is actually a real historical example. So there was a dispute between statisticians. I think Columbia University thought that general relativity was not the explanation

• 1:33:51Speaker 03

  for Mercury's irregular orbit. It's called Mercury's perihelion, there must be some hidden planet or some orbital debris or something like that. And so Bayesian statisticians actually went ahead and computed the marginal likelihood associated with general relativity explaining Mercury's orbit versus these alternative hypotheses like some modification to Newtonian gravity, et cetera. And because general relativity was so falsifiable, like its predictions were so sharp and consistent with what we observed, it had orders of magnitude greater marginal likelihood than something like a modification to Newtonian physics where you have to have some distribution over the modification might sort of enable you to explain what we see, but it also is going to generate other data sets. And so its mass is going to be spread more thinly. And so I think that's actually a beautiful demonstration of how the marginal likelihood

• 1:34:36Speaker 03

  can be used for scientific hypothesis testing.

• 1:34:38Tim Scarfe?

  Yeah. Are there any other heuristics? I mean, we've spoken about marginal likelihood and model complexity and so on. And I'm just thinking of, take the game of life. You know, you have all of these simple rules and what we do there is we do this, you know, this sequence of computations and it's irreducible, as Wolfram would say. And we just look at the dynamics, right? I mean, is there something to that? My intuition is that just by looking at the thing in isolation doesn't tell you something. Like actually running it, you know, in the real world over several steps of computation, that's where the information is about whether the model is good or not. Is that a fair intuition?

• 1:35:16Speaker 03

  That's a great intuition and it's connected with the marginal likelihoods and sequential coding and ideas and information theory. So this is something we've been thinking about a lot, actually, in trying to go beyond Kolmogorov complexity and very much related to your earlier question, connected to benign overfitting, this observation that models first tend to fit structure and then they start to fit noise. And so if we can think of some sort of like compute limited Kolmogorov complexity, maybe we could start to get some idea of how to do model selection. And the marginal likelihood actually can be written as the, so the marginal likelihood is the probability of the data under your model, conditioned on your model. And so you can use the chain rule probability to write that as a product of P of di given d less than i essentially. And so you take the log of

• 1:36:01Speaker 03

  that and it looks sort of like the log under your sort of training curve as you're observing more and more data. And so these, these things are all connected together. And I think this is a very reasonable way of trying to understand how to do model selection properly, sort of thinking about compressibility, but also thinking about the dynamics of training, how like a model's representation evolves with increases in computation.

• 1:36:25Tim Scarfe?

  Yes. And David Krakow, I mean, when he talks about intelligence, he says it's, what was it, inference, adaptivity and representation. But when he was talking about emergence, he said it's about a fundamental reorganization in the micro substrate. So, you know, Navia Stokes, for example, it's a reorganization where this new high level description now does, you know, is a better description than at the molecular level. And surely there must be a thing in training dynamics as well, that, you know, when, if there is some kind of emergent behavior, there would be a fundamental reorganization. And then with some emergentist optimization, where you are looking at the macroscopic behavior, you would actually select that underlying model which generated it.

• 1:37:04Speaker 05

  Yeah, I mean, I think that's, that's, that type of reorganization is what's happening in, in double descent and probably Grokking too, right?

• 1:37:11Tim Scarfe?

  Oh, is it? That's an interesting.

• 1:37:12Speaker 05

  Yeah, yeah. Like, you know, it starts to reorganize. Like, it's, it hasn't, it's not that it's found lower, lower loss or anything. It's just because of the simplicity bias, all the hyperplanes have started to adjust into like a simpler phase, you know, almost. Of parameter space.

• 1:37:28Tim Scarfe?

  Yeah, we didn't we speak to Dan Daniel Roberts about that. He had that criticality thing during.

• 1:37:33Speaker 05

  Right, right.

• 1:37:33Tim Scarfe?

  It was a similar idea.

• 1:37:34Speaker 05

  The deep learning, there's a, we talked to these physics guys who, who, you know, were trying to start, if you will, a physics based perspective on theory of deep learning. And they they had some really interesting points about criticality and sort of, yeah. I agree. I think Grokking is similar, right? It's like you, you start forcing more and more data and essentially because it can't memorize anymore, it's forced to reorganize into these simpler, you know, more generalizable representations.

• 1:38:03Tim Scarfe?

  Yeah, more training.

• 1:38:05Speaker 05

  Well, more, more training and also it's related to the size of the model too, right? Like, is it, is it becomes, if it's too small to really memorize anymore, it has to reorganize.

• 1:38:15Tim Scarfe?

  Well, that's a good question, Drew, because you had this amazing paper out and we we were skimming it earlier and, oh, God, what was it? It's not, deep learning is not so mysterious after all, not so different after all.

• 1:38:23Speaker 03

  Not, not so mysterious or different.

• 1:38:25Tim Scarfe?

  Yes, and then so, so you're talking about this benign overfitting and, you know, double descent and over parameterization, but would you, would you lump potentially Grokking in with that as well?

• 1:38:36Speaker 03

  That's a great question. So in the intro, at the end of the intro to that paper, I say that I'm not talking about Grokking or scaling laws much in this paper because these phenomena are often not treated as particularly mysterious or distinct to neural nets. But I would say that they fit into this sort of suite of generalization phenomena that we can try to understand using some of the same tools. And so the generalization bounds that I present in that paper now are being used by us and others to try to understand the root of scaling laws. I mean, they seem like remarkable laws of nature almost, like if you increase computation by a certain amount, you can predictably improve generalization on a set of tasks. But it's sort of this empirical law, and so we want to understand why.

• 1:39:21Speaker 03

  And this relates to why, you know, larger models might have stronger simplicity biases and things like that. And so definitely the generalization frameworks that we present in that paper, that I present in that paper can be used to. Shed light on scaling law behavior. Grokking is not something I've thought about too specifically, but it does seem to be the case that by training for longer, the model is doing some reorganization that enables a more compressible solution. And so it would be very interesting, I'm sure some people have done this, like measure like the flatness of the solution and things like that as you proceed through grokking. And I think this is actually an older, like many things, older than people might realize.

• 1:40:03Speaker 03

  So like double descent is thought of as like a modern deep learning phenomenon, but it was actually first presented in the 1980s. And so this has been sort of around for a while. And I remember like Ilya Sutskever and others talking about behavior that was very similar to grokking, where like the training loss isn't really changing, but training for longer actually leads to better generalization. And this relates a little bit to a procedure we had called stochastic weight averaging. And so the idea there is you want to ramp up the learning rate to a relatively high constant learning rate and then maintain a running average of the parameters as you're sort of traversing this loss landscape with SGD or Adam or whatever else. And what happens when you do that is you're sort of spinning around the periphery of

• 1:40:48Speaker 03

  flat solutions. And by taking an equal average, you get to move inside that region and get a much sort of like flatter solution. And that reliably led to better generalization. And it's quite convenient because you can just load up a pre-trained model and then increase the learning rate, do this for, you know, a certain number of epochs and get sort of better generalization as a consequence. And so I think, you know, grokking might be related to that sort of behavior as well.

• 1:41:13Speaker 05

  I think, I think this ability to, to move in the parameter space during optimization, I personally think that's also part of the explanation for double descent, right? Because it's like as you provide More and more flexibility and it can shift around a bit. So I'll give you, I'll give you why I think that because you know, you're familiar with integer programming. So like where you're trying to solve some system of equations where you're looking for a solution only among integers. You know, so you have whatever thousands of integers, extremely difficult combinatorial optimization problem, you know, because what, what am I supposed to do? Try out like all the different integers? There's no smoothness, et cetera. So people figured out like really early on, you know what?

• 1:41:55Speaker 05

  Let's just expand the parameter space to go from integers to just floating point numbers. And then just do a floating point optimization. And when we get to the end, crystallize it to sort of the nearest, you know, integer solution that's like, you know, pretty, turns out being pretty good, right? And it's this ability to move like within this numerical space smoothly, you know, that allows it to find like reasonable integer, integer solutions. And I think that's sort of what happens with double descent too, right? You're preaching to the choir, so add flexibility.

• 1:42:27Speaker 03

  Yes, so we can have, yeah, flexibility and viruses.

• 1:42:30Speaker 05

  Right? So you can move around a bit more and then almost stumble into, or maybe it's, you know, by virtue of whatever the structure or the landscape or S G D, you know, whatever it is. But just the ability to kind of, and I think there's a paper about these wormholes, like sort of being able to wormhole to a better, a better solution, you know, because you have a more complex space and the higher and higher the dimensionality is, the more likely there's sort of a wormhole to a near, a nearby good solution. And maybe it's related to this connected modes.

• 1:43:00Speaker 03

  Exactly. It sounds a bit like mode connectivity. So actually after I heard that talk from Jorge Nosadall about flatness and its role in generalization and its connection to small batch optimization. And thought, well, I want to do Bayesian deep learning. I had done a lot of Bayesian M L and I had done some deep learning, but I hadn't thought about Bayesian deep learning until I saw that figure in his talk. I thought, well, before we get into Bayesian marginalization, let's try to understand the geometric properties of these objectives that we're minimizing first so that we can come up with like good posterior approximations. And so in some sense, this is a bit Bayesian agnostic. It should be useful even if you're just doing classical. Optimization. And then we had this, you know, the first thing that we encountered was this discovery we called mode connectivity, which shows that if you retrain your neural net, for example,

• 1:43:44Speaker 03

  with different initializations and find seemingly different modes, in fact, you can actually walk from one mode to the other in some subspace without increasing the training loss at all along the way. And so before that, it had been believed that the different solutions that we would find, for example, by retraining our model, were isolated from one another. So you walk in any direction and you increase the loss a lot along the way. And there were some results from Goodfellow and others to suggest that intuition. We showed that you can introduce very simple sort of parametric curves, like a polygonal chain with just a single turn or a quadratic Bezier curve. You can anchor the endpoints with whatever solutions you find in this procedure that finds your two parameter vectors, w1 and w2.

• 1:44:29Speaker 03

  And then as you vary the parameter of this curve, t, from zero to one, you walk from one to the other. And then the idea is, well, how do you sort of learn this curve? And you can discover these by minimizing your loss uniformly in expectation over the curve. So if you're doing classification, this looks sort of like a line integral of cross-entropy loss normalized by arc length. And it's actually a pretty simple objective to try to minimize because you could just sort of sample uniformly along the curve t and then take gradient steps with respect to the parameters of the curve theta that you're trying to learn. And you can always do this. And the larger you make the model, the less of an arc length, or kind of, sorry, less of a sort of bent turning that you need to do, the more it looks almost like a straight-line path between the two solutions.

• 1:45:11Speaker 03

  And so what this showed is that there were these regions within the loss landscape that were extraordinarily flat. So they all had sort of zero loss. And what was especially interesting about them was that the different parameters in these mode connecting curves actually led to models which made very different predictions on the test set. So of course, they're making the same predictions on the training set to have the same loss, but on the test set, they were different representations. And so this meant that you could ensemble them and get much better performance, for example, just uniformly sampling on the curves. And then this led to this stochastic weight averaging procedure where we were sort of thinking, well, how can we sort of spin around these like contiguous regions of flat solutions and find something that's centered within them? That ended up being fairly practical, yeah.

• 1:45:51Tim Scarfe?

  But even, I suppose, also related to the Grokking question is, where does the gradient come from? Because you just said, well, you know, it's the same on the training data, but on the test data, it's actually behaving differently. Where does the signal come from? Like, what happens when you continue to train a model which has ostensibly converged?

• 1:46:09Speaker 05

  Hmm.

• 1:46:10Tim Scarfe?

  It's behaving different on VAU.

• 1:46:13Speaker 05

  Well, I mean, I have a thought on that. Like, maybe this is helpful. But so I'm big fans of Balastrierro, you know, Randall Balastrierro is kind of, he helped me personally, at least, through the work on the spline theory to understand them better. And I think what's happening there is because if you agree with that and you kind of think of these spines as is essentially just this unbelievably hyperdimensional honeycomb, right? And all these shared kind of hyperplanes that are activating. I think what's happening there is they're really slowly just moving. You know, the slopes are just slightly changing. And then they happen to hit a phase change where it's like, wow, now we can combine all these hyperplanes into a much like less, lower complexity

• 1:46:56Speaker 05

  or, you know, simpler, more parsimonious kind of combination. And that's what it is. Like, I think that's what's happening. It's just slowly, slowly shifting and then snaps into place, right?

• 1:47:06Tim Scarfe?

  Well, yeah, and I don't know whether you saw his ICML paper from last year, but he had a spline interpretation of grokking. And he said exactly that, basically, you know, during grokking, the splines just suddenly

• 1:47:17Speaker 05

  Mm hmm. Slowly moving and then

• 1:47:18Tim Scarfe?

  Because there are so many of them overlapping. But what happens is it forms this honeycomb where they just kind of compress together. And I should have said this to David Krakauer, because he said there's a reorganization in the micro substrate. And what's this if it's not a reorganization where the, you know, those little hyperplanes, if you like, the honeycomb, it just does that during grokking.

• 1:47:41Speaker 03

  Yeah, it's fascinating. Grokking is not something I have thought about too specifically, but it seems to me that you probably are entering some region of the loss landscape that doesn't have different loss, but is providing a representation of very different properties. And so this does relate to procedures like stochastic weight averaging, where again, in the end, the solution doesn't really have a different loss than what you would have gotten training in a standard way, but it has properties that will lead to better generalization, better compressibility, et cetera.

• 1:48:08Speaker 05

  I still don't quite understand. Where the drive to more compressible slash simpler representations is coming from just mechanistically, like in the optimization process, you know? I mean, I don't know, maybe it's just, like you said, maybe it requires less floating point precision and so that the optimized, the jitter in the optimizer or something like that, you know? I just, I'm really curious about the actual mechanism, you know, that forces you there. Like if there is no change in loss, or maybe it's so tiny, we just don't really pay attention to it. I don't know, but I'm super curious, like what's really driving it downhill, if you will, to a simpler solution?

• 1:48:48Speaker 03

  You might sort of get to start to, I mean, this is really speculation and I haven't thought a lot specifically about Grokking, but it could be that you're just sort of edging your way inside because of gradient noise and so on towards a flatter solution as you continue to train.

• 1:49:01Speaker 05

  Yeah, yeah, just some type of, yeah, it's really fascinating.

• 1:49:04Speaker 03

  This sort of made me wonder, actually, after the mode connectivity discovery, like, what do these loss landscapes really look like? Like, the paths that we considered initially were just one dimensional. Eventually, we started thinking about multi-dimensional loss volumes, kind of. And so we had a paper which was creating a simplex. We were basically adding vertices to the simplex and sampling uniformly within the resulting simplex and trying to sort of add vertices such that when we did that, we would sort of have low loss and also maximize the volume of the simplex. And so that sort of enabled us to find these sort of like multi-dimensional loss surfaces. And then we had this that all had sort of low loss. And we had this picture, I guess, in the front of that paper where we started with

• 1:49:48Speaker 03

  this sort of original understanding that all the local optima are kind of isolated to wormhole-like tunnels between the different optima, to this idea that maybe everything is just sort of connected together in some manifold that's embedded in this really high dimensional space. And what seems like a sharp or a flat optima might actually just be like, how converged are you with inside that manifold? Because if you're on the edge, it'll look, it'll look sharp because you move in most directions and you increase the loss quite a bit. But if you move in specific directions, it's very flat. And so I think we still don't have a full understanding of what that looks like. And I think it has really fascinating implications for the generalization behavior.

• 1:50:24Speaker 05

  Really hard to think in a billion dimensions. Yeah, I mean, yeah, it really is.

• 1:50:31Tim Scarfe?

  Andrew, so let's sum up a little bit. Your philosophy is we should have maximally flexible models with soft regularization. How is that actionable to practitioners? I mean, what, because you also gave many practical empirical examples, you know, ensembling models together and you spoke about this residual pathway prior and so on. Help us understand.

• 1:50:57Speaker 03

  So my philosophy is that we should honestly represent our beliefs in the way that we do model construction. And our honest beliefs are usually that the real world is a nuanced place and we're going to want to have model expressiveness in order to represent that nuance. At the same time, it can't just be flexibility. It has to be flexibility combined with some sort of simplicity bias. It should have some kind of Occam's razor bias. It turns out, perhaps surprisingly, that making transformers and other types of neural net models larger often actually enhances rather than reduces a simplicity bias. And this is, in many instances, mostly what's been responsible for the better generalization

• 1:51:42Speaker 03

  behavior of larger models. So we had this example with double descent where in the second descent, all the models have basically zero training loss, but the larger models are generalizing better. Can't be because they're more flexible. Has to be because they have some other bias, some sort of simplicity bias. So how do we operationalize this? I think one way is, well, embrace expressiveness. So choose a model class that is going to be able to represent lots of different solutions. In terms of the simplicity bias, well, we're finding empirically that increasing model size can help with that. And so if you're able to build a really big model, you probably should, both for the expressiveness and the simplicity bias.

• 1:52:22Speaker 05

  Well, what if I'm a researcher who like, I just want, I want this simplicity bias to go to 11, but I can't afford more parameters. Is there something I can tweak in my objective function to just push me a little bit more towards simplicity without breaking things?

• 1:52:37Speaker 03

  Yes, so there are a lot of things you can do.

• 1:52:41Speaker 03

  I think, and we'll talk about them, but the question of how can you more elegantly encode this compression bias beyond just making the model bigger is really a fascinating open research question. So my contention, and I might be wrong, is that in many cases, a 7 billion parameter model is not doing better than a 1 billion parameter model primarily because it's more expressive. It's actually because of this simplicity bias. And perhaps things like knowledge distillation can help make this argument. If it's possible to really distill a large model into a much smaller model, then it

• 1:53:26Speaker 03

  means that the small model has some setting of its parameters that can provide a good approximation to the large model. It's just not able to find those parameters when trained directly on the data. It needs the help from the teacher model. And so the question is, can we build the 1 billion parameter model with some sort of like explicit bias that would have otherwise come about through scale in the 7 billion parameter model and find those solutions itself? I don't think we're anywhere close to being able to do that. I think it's sort of an open research program. However, there are little things we can do that will help. So like, there are ways of intervening in the training procedures, so like the stochastic weight averaging we discussed, that will help find a flatter, more compressible solution.

• 1:54:10Speaker 03

  There are, of course, all sorts of regularizers that can be useful in certain instances. Bayesian marginalization can be very helpful in sort of encoding an automatic simplicity bias in what we do. And so there are all sorts of interventions that will help us with this, but I think the dream is that maybe we can embrace flexibility in, you know, 15, 20 years from now by building these non-parametric models that like really do have an infinite number of parameters and are more expressive than any model we're using right now. But then they have this sort of like more explicit compression bias that is interpretable and is getting us what building huge models is inelegantly giving us right now. That's interesting.

• 1:54:52Tim Scarfe?

  I mean, the other kind of interpretation that some folks at home might have based on what you've said is going back to Rich Sutton's bitter lesson, right? So he said that designing things is bad. Don't put your symmetries in there. Don't, don't kind of create these multifaceted systems. Just scale and lots of computation is the way forward because it seems like one potential interpretation of what you're saying is that we should create hybrid systems that make models more flexible by combining different modalities. And what you're kind of saying is we should just have bigger models. So are you a Sutton guy or not?

• 1:55:26Speaker 03

  So I think the bitter lesson is widely misunderstood and incomplete. The bitter lesson describes how over short time scales, it's been appealing to try to encode our knowledge into our procedures in order to achieve good results. And that this is also a cognitive bias that people have. They like encoding expertise and things like this and being thoughtful and elegant about problem solving. But over longer time scales than a typical research project, computation becomes cheaper. And so quite often, more brute force seeming approaches based on search and learning end

• 1:56:08Speaker 03

  up working a lot better than elegantly encoding our priors and our constraints, et cetera. And there are a number of examples in the essay like Deep Blue for playing chess in the late 90s, speech recognition, where some researchers were trying to model things like physiology of the voice box, et cetera, to try to get any possible advantage. But then they were beat out by statistical procedures like hidden Markov models and of course AlphaGo and other procedures like this too. And so the takeaway seems to have been because of how computation is becoming cheaper over longer time scales, quite often it's going to be more practical to try to build procedures

• 1:56:53Speaker 03

  that are based on search plus learning as opposed to feature engineering.

• 1:56:59Speaker 03

  I think to a large extent, this is true. But what it doesn't say is that in order to learn, you need to make assumptions. And so, machine learning, as we discussed, is learning by example. And we can't do that without making assumptions. So if we go to like, maybe this coin toss example, where we're trying to estimate the bias of the coin, you have to make some assumption. Like, before I start doing that experiment, are we assuming a uniform bias? Are we thinking it's looking sort of more like, you know, centered around 0.5 or something like that? Like, being unbiased, but we have support for other things? These assumptions are going to influence how we do induction. And so, like, we just can't get away from making assumptions. And so, the question is just, what assumptions should we make?

• 1:57:44Speaker 03

  And to what extent can they be universal? And when it comes to things like scaling laws that describe how we can reliably improve performance with increases in computation, if we're able to make better assumptions, we can actually change the scaling exponents, which mean that we'll get exponential improvements in performance with increases in computation, which is just remarkable motivation for trying to do this. And it's not just a contention that this might be possible. We're actually starting to see some evidence of this. And so, in our own work, we've been very interested in how we can produce structured representations for linear layers in neural nets. And so, this might sound very abstract, but there's a very well-known example of doing this.

• 1:58:27Speaker 03

  So you can actually start with a fully connected layer, having every possible connection between the nodes in two layers. Remove a bunch of the connections and enforce parameter sharing, and you have a convolutional layer. And so, mathematically, what you've done is you've replaced a dense matrix multiply with a matrix that has sparsity and locality. And that's a topless matrix. And so, there's this question of like, could you actually systematize this process of creating structured layers towards better compute-optimal efficiency? And if you were to do that, what sorts of principles should we be embracing? Should we embrace things like parameter sharing? Because convolution, for example, has parameter sharing. Should we... Embrace sparsity, should we embrace other sort of features?

• 1:59:12Speaker 03

  And so we introduced this sort of einstein formulation over structured matrices that contains all sorts of different structures as special cases, as well as all sorts of different novel structures. And this taxonomy of interpretable hyperparameters that kind of control the properties of these structures. So how fast is it to do a matrix multiply? What's the rank of the matrix? What, how much parameter sharing there is? And we'd have the continuous values for these parameters that would control these things. And what we found is that actually, in general, towards compute optimal efficiency, assuming we have as much data as we ever need, parameter sharing was actually not a great principle. And that was a bit surprising. You also want to have full rank structures. And that moved from surprising to sort of disappointing. It's like, well, can we go beyond what we're doing with dense matrices then?

• 1:59:57Speaker 03

  Because that's what we're using now. And the answer is yes, if you can get faster multiplies. So we proposed a structure called a block tensor train, which is related to another structure called a tensor train and another structure called a monarch matrix. It's basically a sum of monarch matrices. And this sort of is full rank, doesn't have parameter sharing, but you can do a multiply faster than you can with a dense matrix. And this allows you to build wider layers for a given computational budget. And this did have sort of a meaningful effect on the scaling exponents. And so this is sort of at a proof of concept level. And a lot of work would need to go into parallelized implementations and so on of these structures. But it showed that it's possible. And we're not the only ones.

• 2:00:40Speaker 03

  So there are other groups that have been looking at neural tangent kernel inspired ideas to understand different regimes of learning, like easy versus hard feature learning and so on, and trying to think about, oh, what principles are actually going to modify these scaling exponents? And other things, like I think Albert Gu and others have been looking at like chunking and transformers and the inductive biases that are, you know, come with that. And it's like whether we can change these inductive biases towards better scaling exponents. And so I guess in short, learning requires assumptions. And so I don't think we can neatly compartmentalize like elegant ideas from having successful learning combined with computation.

• 2:01:22Speaker 03

  We really do need to, these things not only are not at odds with one another, they really strongly go together.

• 2:01:29Speaker 01

  Hey, listen! Power up your play with Nintendo Switch 2 and experience Link's legendary adventure, now reborn in vibrant detail on our fastest and most powerful system yet. And harness the power of the Ocarina with the built-in microphone.

• 2:01:51Speaker 01

  The Legend of Zelda: Ocarina of Time, a Nintendo Switch 2 exclusive. Game and system sold separately. Game rated everyone 10 and up.

• 2:01:58Speaker 02

  LinkedIn is pretty amazing at helping you grow your small business. We cannot make your email response time faster. We can help you sell, market, and hire in one place. We cannot help you find space for your three desk drinks. Why do you have three? And while we can't help you find the perfect volume for your presentation video, LinkedIn can help you find the perfect audience for your business. Grow your small business on LinkedIn. Learn more at LinkedIn.com/smallbusiness.

• 2:02:30Tim Scarfe?

  Yeah. When we spoke, I mean, by the way, out of all of those assumptions, sparsity seems like a really good one. There's always one thing where you think, "That seems like a really good one." Daniel Roberts was saying, you know, in many effective theories in physics, that's one of the earliest assumptions is sparsity. But then you kind of get to other factors to consider, like computational complexity and training tractability and so on. Because in an ideal world, we would make these things sparse. What's stopping us?

• 2:02:55Speaker 03

  It's a great question. So I'll say that I think convolutions are a great idea. And if you're in this setting that I described earlier, where you have a fixed dataset and you want to do something reasonable in order to achieve the best performance, this will often work really well. And even if we're moving towards soft inductive biases from hard constraints, I would normally advocate for having some sort of convolutional inductive bias, even if it's not a hard constraint anymore. And that's what we were doing with residual pathway priors. And there's been some interesting work on convolutional VITs and things like this to try to have these sorts of soft biases for more efficient learning. The reason that we found in this instance that parameter sharing didn't seem to be a good principle towards better compute-optimal scaling is because we were in the setting where we could have as much data

• 2:03:40Speaker 03

  as we ever needed. And there wasn't much of a generalization gap. And so we basically just need to fit the data as efficiently as possible. And so this basically led to the principle of having as many possible parameters per flop. And...

• 2:03:55Speaker 03

  You might sort of wonder then, well, can you go beyond one parameter per flop? And I think things like sparse mixtures of experts actually allow you to do this because you have some sort of gating function which might be acting over E different MLPs. And you only have some subset of them that are active. And so you kind of divide your computation by K over E, where K is the number of active efforts, experts relative to a model with the same number of parameters. And so we also tried to push that principle a bit further in this work, where rather than having the mixture of expert gating function operate over whole MLPs, it was actually operating over individual linear layers within the MLPs and the attention-projection matrices. And so you would represent these layers as like sums of structured matrices, and then

• 2:04:40Speaker 03

  you would have this gating function acting over the rank index of this sum. And this would allow like much finer-grain routing decisions across the experts and did lead to sort of better efficiency for a given amount of computation. But the short reason, I think, is we found parameter sharing to not be helpful because you basically just want to reduce your loss as efficiently as possible, and you have sort of as much data as you could ever need, and there isn't much of a generalization gap.

• 2:05:04Tim Scarfe?

  Also reminiscent of VIT surprised everyone that it did so well compared to CNN's. But final question, we were talking about this in the car on the way over, Andrew. The elephant in the room is that GPT-5, you know, we have all these huge over-parametrized models, and on the surface, they seem to be doing very well. They're bench-maxing, and they're brilliant to use, but it feels that there's something missing. I mean, I think they're not intelligent. I believe you would agree with that statement. What's missing and what's next?

• 2:05:33Speaker 03

  So one of the things I'm most excited about is developing AI systems that can discover new scientific theories at the level of general relativity or quantum mechanics. And we haven't even really scratched the surface in being able to do this. It's not even clear how data-driven that process would be, how much it would look like symbolic logic if we were to think about what Einstein did when he proposed relativity, and how we might want to write that down as an algorithm and automate it on a computer. And so I would love to see progress in this direction.

• 2:06:14Speaker 03

  I think that some of the ideas that we've discussed around compressibility could play an important role in how we think about

• 2:06:23Speaker 03

  Selecting for scientific hypotheses, also ideas around universality, like what sorts of assumptions might be more universal than others and at what level of abstraction. But it's something that we haven't really made progress on at all, despite a number of very exciting research projects in AI for science, where neural nets, for example, are being used as black box function approximators in some sort of pipeline targeted at a very specific application. And I think this is extraordinary work and it's really a way in which machine learning is clearly doing a lot of good in the world. But I think it's time to try to go beyond that paradigm

• 2:07:05Speaker 03

  towards really giving us new scientific insights into the data that we didn't have before. And in fact, that's something that I'm really more excited about than anything else in terms of how technology might develop in the future. Like if I were to go 1,000 years in the future, one of the first questions I would have is, well, do we understand something about physics that we didn't before? Do we understand how the brain works? And this is sort of a conventional sort of like approach to science where the theory is really the quantity of primary interest. And the applications, of course, are important, but they're not primarily why, like no individual application is primarily why we care about the theory. Like GPS would break within minutes if we weren't accounting for gravitational time dilation

• 2:07:50Speaker 03

  and general relativity, but Einstein wasn't thinking about GPS when he proposed relativity. And if you have the theory, then you can sort of suggest all sorts of applications that otherwise wouldn't have been on the horizon. And like, we probably could train a neural network to correct for gravitational time dilation without understanding what's going on, but that wouldn't be nearly as exciting or useful as having the theory of relativity.

• 2:08:14Tim Scarfe?

  Yeah. Professor Wilson, thank you so much for joining us today. It's been amazing.

• 2:08:18Speaker 03

  Thanks so much. It's been a real pleasure.

• 2:08:20Tim Scarfe?

  If you want to learn more about this work from Andrew, please check out the paper "Deep Learning is Not So Mysterious", also their paper "Bayesian Deep Learning and a Probabilistic Perspective of Generalization", which also discusses Bayesian principles in deep learning. And also their recent paper "Compute Optimal LLMs Provenably Generalize Better with Scale", that has further analysis on the source of the simplicity bias arising from scale. Cheers.

• 2:08:49Speaker 00

  Hey, listen!

• 2:08:51Speaker 01

  Power up your play with Nintendo Switch 2 and experience Link's legendary adventure, now reborn in vibrant detail on our fastest and most powerful system yet.

• 2:09:03Speaker 01

  And harness the power of the Ocarina with the built-in microphone.

• 2:09:11Speaker 01

  The Legend of Zelda: Ocarina of Time, a Nintendo Switch 2 exclusive. Game and system sold separately. Game rated Everyone 10 and up.

• 2:09:19Speaker 02

  LinkedIn is pretty amazing at helping you grow your small business. We cannot stop your new clients from emailing you at 3 a.m.

• 2:09:28Speaker 02

  We can help you sell, market, and hire in one place. We cannot help you be in three places at once. And while we can't help you organize your calendar, LinkedIn can help you land more clients so you have a calendar to organize. Grow your small business on LinkedIn. Learn more at LinkedIn.com/smallbusiness.