OpenAI Researcher Dan Roberts on What Physics Can Teach Us About AI

In recent years there’s been an influx of theoretical physicists into the leading AI labs. Do they have unique capabilities suited to studying large models or is it just herd behavior? To find out, we talked to our former AI Fellow (and now OpenAI researcher) Dan Roberts. Roberts, co-author of The Principles of Deep Learning Theory, is at the forefront of research that applies the tools of theoretical physics to another type of large complex system, deep neural networks. Dan believes that DLLs, and eventually LLMs, are interpretable in the same way a large collection of atoms is—at the system level. He also thinks that emphasis on scaling laws will balance with new ideas and architectures over time as scaling asymptotes economically. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital Mentioned in this episode: The Principles of Deep Learning Theory: An Effective Theory Approach to Understanding Neural Networks, by Daniel A. Roberts, Sho Yaida, Boris Hanin Black Holes and the Intelligence Explosion: Extreme scenarios of AI focus on what is logically possible rather than what is physically possible. What does physics have to say about AI risk? Yang-Mills & The Mass Gap: An unsolved Millennium Prize problem AI Math Olympiad: Dan is on the prize committee

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  In the 40s, the physicists went to the Manhattan Project, even if they were doing other things, that was the place to be. And so now, AI is the same thing and basically said OpenAI is that place. So maybe we don't need a public-sector-organized Manhattan Project, but it can be OpenAI.

• 0:37Speaker 01

  Joining us for this episode is Dan Roberts, a former Sequoia AI fellow who recently joined OpenAI as a researcher. This episode was recorded on Dan's second-to-last day at Sequoia, before he knew that he would go on to become a core contributor to O1, also known as Strawberry. Dan is a quantum physicist who did undergraduate research on invisibility cloaks before getting a PhD at MIT and doing his postdoc at the legendary Princeton IAS. Dan has researched and written extensively about the intersection of physics and AI. There are two main things we hope to learn from Dan in this episode. First, what can physics teach us about AI? What are the physical limits on an intelligence explosion and the limits of scaling laws? And how can we ever hope to understand neural nets?

• 1:20Speaker 01

  And second, what can AI teach us about physics and math and how the world works?

• 1:27Speaker 01

  Thank you so much for joining us today, Dan.

• 1:29Speaker 00

  Thanks. Delighted to be here on my probably second-to-last day at Sequoia, depending on when this airs and how you're going to talk about it. You will always be part of the Sequoia family, Dan. Thanks. I appreciate it.

• 1:42Speaker 01

  Maybe just to get started, tell us a little bit about who is Dan. You have a fascinating backstory. I think you worked on invisibility cloaks back in college. What led you to become a theoretical physicist in the first place?

• 1:56Speaker 00

  Yeah, I think, and this is, my stock answered this point, but I think it's true. I was just an annoying three-year-old who never grew up. I asked why all the time, curious, how does everything work? I have a 19-month-at-home right now, and I can see the way he followed the washing machine repairman around and had to look inside the washing machine. So I think I just kept that going. And When you're more quantitatively oriented than not, rather than going to philosophy, I think you sort of veer into physics and that was sort of what interested me. How does the world work? What is all this other stuff that's out there? The question that you didn't ask, but maybe I'll just answer it ahead of time.

• 2:39Speaker 00

  The sort of inward facing stuff felt less quantitative and more in the realm of the humanities. So what is all this stuff? That's pretty physicsy. What am I? Who am I? What does it mean to be me? That felt not very sciencey at all. But with AI, it sort of seems like we can think about both what does it mean to be intelligent? And also what is all this other stuff in sort of some of the same frameworks? And so that's been very exciting for me.

• 3:05Speaker 02

  So we should be trying to recruit your 19 month old right now, is what you're saying?

• 3:08Speaker 00

  Oh, yeah, absolutely. He's got his, he grew out of one of his Sequoia onesie, but he fits into a Sequoia toddler t-shirt now. And so he definitely is ready to be a future founder.

• 3:19Speaker 02

  I guess, at what point did you know that you wanted to think about AI? At what point did that switch start to flip?

• 3:26Speaker 00

  Yeah, so I think like many people, when I discovered computers, I wanted to understand how they worked and how to program them. In undergrad, I took an AI class and it was very much good old fashioned AI. A lot of the ideas in that class actually are coming back to be relevant. But at the time, it was, it seemed not very practical. It was a lot of, you know, if this happens, do that. And there was also some game playing there that was sort of interesting. It was very algorithmic, but it didn't seem related to what it means to be intelligent. Can we just pick up on that real quick? Like, what do you think it means to be intelligent? That's a great question. I can see, I can see wheels turning.

• 4:08Speaker 00

  Yeah, well, this is, this is one of those questions where I don't have a stock answer, but I think it's important to not say nonsense. One of the things that's exciting to me about AI is the ability to have systems that do what humans do and to be able to understand them from how, you know, what are the lines of Python that caused that system to do something, to, you know, to trace that through and understand what are the outputs and, you know, how the system can like see and classify what it mean, you know, what is a cat, what is not a cat, or can write poetry and, you know, it's just a few lines of code. Whereas if you're trying to study humans and ask what does it mean for humans to be

• 4:53Speaker 00

  intelligent, you have to go from biology through neuroscience, through psychology, up to other higher level versions of ways of approaching this question. And so I think maybe a nice answer for intelligence, at least the things that are interesting to me, is the things that humans do. And then now if you pull back the answer that I just said a second ago, that's how I connect it to AI. AI is taking pieces of what humans do and we're understanding at least an example that's kind of simple and easy to study that we might use to understand better what it is that humans do.

• 5:35Speaker 02

  So Dan, you mentioned when you were, you know, studying AI in college, there's a lot of what sounds like kind of hardcoded logic and brute force type approaches.

• 5:43Speaker 00

  Yeah.

• 5:44Speaker 02

  Was there a moment that kind of clicked for you of like, oh, okay, this is different? Was there a key result or a moment where it was like, okay, we're going bigger places than kind of the if this then that logic of the past?

• 5:56Speaker 00

  Yeah, actually it didn't click. I sort of wrote it off and it would be great, be great if there was a separation between like 10 years between then and the next thing that I'm going to say, but actually the writing off maybe lasted a year or two because then I went to, I went to the UK for the first part of grad school. I spent Very long time in grad school and I discovered I discovered machine learning and and and and a more statistical approach to artificial intelligence where you have a bunch of examples of large amounts of data or at the time, maybe we wouldn't have said large amounts of data, but we would have said, you know, you have you have data examples of a task that you want to perform and I mean,

• 6:38Speaker 00

  there's different ways that machine learning works, but you write down a very flexible algorithm that can adapt to to to these examples and start to perform in a in a way similar to to to the examples that you have. And that approach borrows a lot from from physics. It also, at the time I started, so graduated college in 2009, discovered machine learning in 2010, 2011. 2012 is the big year for deep learning. And so, you know, there's not a big separation here between write-off and and rediscovery. But I think and and machine learning clearly existed in 2009. It just wasn't related to the class class I took. But this approach made a lot of sense to me. And it started to have, you know, I got lucky and then it started to have real progress and and seemed to fit

• 7:23Speaker 00

  in a framework that I understood scientifically. And I got very excited about it.

• 7:27Speaker 02

  Why do you think there are so many people who come from a similar background or similar path to you? Like, a lot of ex-physicists and working on AI, like, is that a coincidence? Is that is that herd behavior? Or do you think there's something that's particularly that makes physicists particularly well-suited to understanding AI?

• 7:44Speaker 00

  I think the answer is yes to all the ways you

• 7:47Speaker 02

  All the above?

• 7:48Speaker 00

  You know, physicists infiltrate lots of different subjects and we get parodied about the way that we go about trying to use use our hammers to to tackle all these things that may or may not be nails. Throughout history, there are a lot of times that physicists have contributed to things that look like machine learning. I think in the near term, the path that physicists used to take when they don't go didn't remain in academia often was going into quantitative finance, then data science. And I think

• 8:17Speaker 00

  Machine learning was the and and its realization industry was was exciting because it again, it's it's something that feels a lot like actual physics and is working towards a problem that's that's very interesting and exciting for for a lot of people. You know, you're you're doing this podcast because you're excited about AI. Everyone's excited about AI and it it in many ways, it's it's a research problem that feels a lot like the physics that people were doing. But I think the the methods of physics actually are different from the methods of traditional computer science and and very well suited for studying large scale machine learning that that we use to to to work on AI. There there's, you know, traditionally physics involves a lot of interplay between theory

• 9:02Speaker 00

  and experiment. You come up with some sort of model that you have some sort of theoretical intuition about. Then you go do a bunch of experiments and you validate that model or not. And then, you know, there's this tight feedback loop between collecting data, coming up with theories, coming up with toy models, understanding moves forward by that. And, you know, we get these really nice explanatory theories. And I think the the way that big deep learning systems work, you have this tight feedback loop where you can do a number of experiments. The sort of math that we use is very well suited to the math that a lot of physicists are familiar with. And so I think I think it's very natural for a lot of physicists to to to work on this. And and those tools, a number of them differ from sort of the traditional, at least theoretical, computer science and theoretical

• 9:46Speaker 00

  machine learning tools for studying the theoretical side of machine learning and maybe also differentiation between just being an awesome engineer and also being a scientist. And there's tools from from doing science that are helpful in studying these systems.

• 9:59Pat Grady?

  Dan, you you wrote this, what I thought was a beautiful article, Black Holes and the Intelligence Explosion. And in there, you talk about this concept of sort of the microscopic point of view and then the system level point of view and how physics really equips people to think about the system level point of view. And that has a sort of complementary benefit to the understanding these systems. Can you can you just take a minute and kind of explain sort of microscopic versus system level and how the physics influence helps to understand the system level?

• 10:29Speaker 00

  Sure, so can, let me start with an analogy that I think is like very, you know, goes even further than analogy, but going back, what year is it, maybe like 200 years or so, there was around the time of the Industrial Revolution, there was steam engines and steam power and a lot of technology that resulted from this and ultimately powered industrialization. And in the beginning, there is a lot of engineering of these steam engines, you know, and there was this high-level theory of how this worked called thermodynamics where, you know, and I imagine everyone's seen this in high school perhaps where, you know, there's the ideal gas law that tells you that there's some relationship between pressure at volume and

• 11:14Speaker 00

  temperature and, you know, these are very macro-level things, like you can buy a thermometer, you can also like measure the volume of your room, and you can buy a barometer as well, maybe people don't, or look up on the weather report, but you know, these are things that, these are like measurements that we use and we talk about. But then, underlying this, and it took us a bit later to like validate this and understand it, there's the notion of atoms and molecules, the air molecules bouncing around, and somehow we now understand that those air molecules give rise to things like temperature and pressure and volume, I guess is easier to understand that the gases, the molecules are confined to a room and, but there's a precise way in which you can start with

• 11:56Speaker 00

  the statistical understanding of those molecules and derive thermodynamics, like derive the ideal gas law from it. And you can go further than that, derive, you know, that it's ideal because it's wrong, it's just a toy model, but you can, there are corrections to it and you can sort of understand, you know, from the microscopic perspective, which is the molecules, which we don't, we don't really interact, you know, we don't see them, we don't interact with them on a day-to-day basis, but their statistical aggregate properties give rise to sort of this, this behavior that we do see at the macro scale. And part of, to get to your, to get to your question, you know, I think there's a similar thing going on with deep learning systems, and I wrote a book with Shoyeda and Boris

• 12:39Speaker 00

  Hanan on, on how to apply these sorts of ideas to, to, to deep learning and at least in an initial, a framework that, that allows you to start doing this in an initial way. And to answer your question,

• 12:52Speaker 00

  We, the sort of micro perspective is you have neurons and weights and biases. And we can talk about in detail how that works. But you know, when people think of the architecture, there's, there's some very specific, some people say circuits, there's specific ways in which these, these things, you know, there's an input signal, which might be an image or text. And then there's many parameters and, and you know, there's a very, it's very simple to write down. It's not that many lines of, of code, even taking into account the machine learning libraries, but it's, you know, it's like a very simple set of equations, but there's a lot of weights. There's a lot of numbers that you have to know in order to get it to do something. And that's sort of the micro, that's like the molecules perspective. And then there's, there's like the, the macro perspective, which is, well, what did it do?

• 13:37Speaker 00

  Did it, did it produce a, did it produce a poem? Did it produce, did it solve a math problem? How does, you know, how do we go from that, those weights and biases to, to that macro perspective? And so for, for, for statistical physics to thermodynamics, we, we understand that completely. And you know, you could imagine trying to do the same sort of thing, literally applying the same sorts of methods in, to, to, to understand the, how do this, how does the underlying micro statistical behavior of the, of these models lead to, to the sort of macro, or as you said, system level perspective?

• 14:10Speaker 02

  Dan, maybe, maybe speaking of, of scaling laws and I think you were at our event, AI Ascend, Andre Karpathy mentioned that current AI systems are like five or six orders of magnitude off in efficiency compared to biological neural nets.

• 14:27Speaker 00

  Yep.

• 14:27Speaker 02

  How do you think about that? Do you think scaling laws get us there? Just combination of scaling laws plus hardware getting more efficient or like, do you think that there's kind of big step function leaps that need to, need to come in research?

• 14:40Speaker 00

  There's maybe two things that, that could be meant here. One is that The way humans seem to work at a similar scale to AI systems is much more efficient. You know, the amount of, we don't need to see trillions of tokens before we, we, we speak. We see a much, you know, my, my toddler has, is already starting to, to speak in sentences and he's been exposed to far less tokens than, than a typical large language model. And so there's some sort of disconnect between human efficiency at learning and, and what large language models do. Of course, they're very different systems, they're designed, you know, the, the way in which they learn is, is right now very different. And so in some sense, that's to be expected. So there's, there's this gap here that you could imagine bridging. There's another thing that, that I think is, is not what you meant, but, but I think

• 15:25Speaker 00

  is, is sort of the thing to answer about with respect to scaling laws, which is, and I talked about this a bit in the article, but lots of people seem to talk about this, which is, what is the final GPT? You know, there's GPT four right now, and it could be other companies as well, but since I'm going to join OpenAI, let me represent my, my, my new company, right? So is it going to be six? Is it going to be seven? At some point, right, if, assuming we have to scale things up, there are things that are going to break, whether they're economic, we're going to run out of, you know, we're going to try to train a model that's larger than the world's GDP or, or GWP, whatever the, however the D works for, for, for the world. And, or we're going to run out of, you know, we, we're not going to be able to produce

• 16:08Speaker 00

  enough GPUs, or we're not going to be able to put, you know, it's going to cover the surface of the, you know, a lot of these things are going to break down at some point. And so probably the economic one happens first. So how many, you know, how many more iterations do we get before we run out of actually being able to scale practically? And where does that get us? And then I think, I think to tie those two perspectives together, there's, so scaling on its own, and it, and of course, it's impossible to just tangle this because people are making things more efficient, but you know, there's like, you could imagine there's the, take literally what GPT two was, which was the initial big model, and keep scaling it up. Is that going to get us to some, you know, super different, exciting economically power,

• 16:49Speaker 00

  or however you want to define what the end state of AI research and, and AI startups and, you know, AI in industry is? So. Or do we need lots of new exciting ideas? And again, of course, you can't really just disentangle these. But I think the general scaling hypothesis is that it's just the scaling and it's not the ideas that matter. Whereas the how do we get to efficient like humans, I think, requires like non-trivial ideas. And to answer your question, the reason I'm excited about joining OpenAI is that I think there is high leverage to be had in the ideas, you know, in going beyond scaling and that we will need that in order to get to the next steps.

• 17:30Speaker 00

  And I'm, I have no idea what I'll be working on, but when this airs, I guess I will know what I'm working on. But that, you know, that, that's what's really exciting to me.

• 17:39Pat Grady?

  Dan, is there, is there almost like a pendulum that swings back and forth between scale and ideas in terms of how people apply their efforts in the world of AI? Like, Transformers came out, great idea. Since then, we've largely been in this race to scale. It feels like things are starting to asymptote for a bunch of practical reasons that you mentioned. Is the pendulum swinging back toward ideas as the currency? You know, it's less now about who can, you know, have the biggest GPU cluster and more about finding new architectural breakthroughs, whether that's, you know, reasoning or something else?

• 18:11Speaker 00

  Yeah, that's a, that's a, that's a really great question. I think there's this article by Richard Sutton called The Bitter Lesson, not The Bitter Pill. And, and it basically gives the argument that, that ideas aren't important, that scale is, is what you need and that, you know, that, that all the, all the ideas are always trumped by, by scaling, by scaling things up. And, and that says a bunch of things, but maybe that's, that's a high level takeaway. And, you know, there is, there's a sense of this where there are, there are a lot of, you know, interesting ideas that came out in the 80s and 90s that People didn't really have scale to explore.

• 18:53Speaker 00

  And then I remember when, after AlphaGo and DeepMind was writing a lot of papers, people were rediscovering those papers and re-implementing them in deep learning systems. But this was sort of still before people realized, no, the thing that you need to do is scale up. And even now with transformers, people are exploring other architectures or even simpler architectures that we knew before that seem to be able to. You know, there's a notion, you know, maybe scaling laws don't come from, as long as the architecture isn't sick in some way, they come from sort of the underlying data process and having large amounts of data rather than from having a special idea. I think the real answer is that there's a balance between the two, that scale is hugely important and maybe it was just not understood how important and we also didn't have the

• 19:38Speaker 00

  resources to scale things up at various times. You know, the things that have to go into producing these GPU clusters that are producing these models are, you know, you guys know this as well, but like there's a lot of parts along the supply chain or along the product chain, whatever you actually call it, in order to make those things happen and to deploy them. And even, you know, the way GPUs were originally, they've now co-evolved to be well-suited for these models. And the reason, in some sense, you can think of transformers was a good idea was because it was designed to be well-suited to train on the systems that we had at the time. And so, sure, these other architectures could do it at an ideal scientific level, but at a practical level, there's, it was important to get something that was able to

• 20:23Speaker 00

  reach that scale. So I think, you know, if you brought in ideas to be that sort of thing, that's married with scale in some way, then, I mean, I still think ultimately, like, you know, someone came up with the idea of deep learning, that was an important idea. You know, there's, Pitts and McCullough came up with the original idea for the neuron and then there's lots of, Rosenblatt came up with the original perceptron. And there's like a lot of people from going back like 80 or so years of people making important discoveries that were ideas that contribute. So I think, I think it's both, but it's easy to see how, you know, if you get to apply If you're bottlenecked and then all of a sudden you get to apply, if you're bottlenecked, people think about ideas.

• 21:07Speaker 00

  And then if you unlock a new capacity of scale somehow, then you just see a huge set of results. And it seems like scale is super important. And I really think it's more of a synergy between the two.

• 21:16Speaker 02

  Maybe on the topic of the race to scale, Dan, you mentioned kind of the, just the economic constraints and realities, which I guess are more like practically a ceiling in the private sector. You also mentioned the Manhattan Project earlier in terms of things that physicists have been involved with. Like, do you think we need a Manhattan Project style thing for AI? Like at the nation state or at the international level?

• 21:41Speaker 00

  Well, one thing I can say is that part of the process that led me to OpenAI is I was talking with your partner, Sean McGuire, who brought me to Sequoia in the first place, and trying to figure out, is there a startup that makes sense for me to, for me to work on that has the right mix of sort of scientific questions, research questions, and also is the business? And I think it was Sean that said, and I don't mean the analogy in terms of the impact of, you know, in terms of like negative impact of what the, you might think of the Manhattan Project, but it just in terms of the scale and the organization. He said, you know, in the 40s, the physicists went to the Manhattan Project, even if they were doing other things. That was, that was the place to be. And so now AI is the same thing.

• 22:25Speaker 00

  And, you know, basically said OpenAI is that place. So maybe, maybe we don't need a public sector organized Manhattan Project, but it, you know, it can be OpenAI.

• 22:37Speaker 02

  OpenAI is the Manhattan Project. I love that.

• 22:39Speaker 00

  Well, maybe, maybe that's not a direct quote that we want to be taken out of context. But I think in terms of-

• 22:44Speaker 02

  Is the metaphorical Manhattan Project.

• 22:46Speaker 00

  Yeah, in terms of scale and ambition, in terms of, I think, I mean, I think a lot of physicists would love to work at OpenAI for a lot of the same reasons that they probably were excited to. Well, okay, there's a number of different reasons. Maybe, maybe we'll just have to leave it as a nuanced thing rather than making broad claims.

• 23:03Speaker 02

  Can we talk a little bit about this? Like, can we ever understand AI, especially as we go to these deep neural nets? Or do you think it's? A hopeless black box, so to speak.

• 23:15Speaker 00

  Yeah, I think within the, this is, this is my answer to the, what are you a contrarian about, although maybe, you know, on the internet, everyone takes every side of every position, so it's hard to, to, to say you're, you're have a contrarian position, but I think within AI communities, you know, I think of my contrarian position is that we can really understand these systems. And in the, you know, physics systems are extremely complicated, and, and we have made a huge amount of progress in understanding them. I think these systems sit in the, in the same framework. And, you know, another principle of that, that Sean and I talk about it in our book, and that, that's a principle of physics, is that there's this often extreme simplicity at very large scales. Basically, due to, due to statistical

• 23:57Speaker 00

  averaging, or more technically, the central limit theorem, things, things can simplify. And I'm not saying this is what happens exactly in large language models, of course not, but I, but I do think that we can apply sort of the methods that, that we have, and also, you know, maybe hopefully have AI that can, that can help us do this in the, in the future. And by AI, I mean tools, not like individual intelligences just going, running on their own and solving these problems. But, but I, I guess I, I feel at the extreme end that, that, that this is not going to be an art, that it, that, that the science will, will catch up, and that it will, will be able to make extreme leaps in really understanding how these systems work and behave.

• 24:35Pat Grady?

  So Dan, we've talked a bunch about what physics can teach us about AI. Can we talk a bit about what AI can teach us about physics? Are you, are you optimistic about domains like physics and math, and how these emerging models can, you know, probe further into those domains?

• 24:53Speaker 00

  Yes, I'm definitely optimistic. I guess my, my perspective is that math will be easier than physics, which maybe betrays the fact that, that, that I'm, I'm a physicist, not a, not a mathematician.

• 25:09Speaker 00

  And I'll say, I can say, I can give, explain why I think that in a second. But, you know, I still have a lot of friends that work in physics and they, you know, there's a, there's like a growing sense and maybe even approaching a dread that, and maybe this is actually the answer to why do physicists work on AI, because, you know, if what you care about is the answer to your physics question and you want to make it happen as soon as possible, what is the highest leverage thing you can do? Maybe it's not work on the physics question you care about, but it's work on AI to make the, you know, to, because you think that the AI might end up solving those questions very rapidly anyway.

• 25:51Speaker 00

  And I don't know the extent that anyone really takes it seriously, but I think within the theoretical physics community that I come from, that this is sort of a thing that someone gets thrown around and discussed. I think, maybe to give a more object level answer, I think what's exciting about math and maybe when you have Noam Brown on, if you have him on, he'll talk about this, but this is something that he's talked about for a while before he joined OpenAI. I think that we have, you know, we made a lot of progress in terms of solving games by doing more than just looking up what is the strategy that we should use to play the

• 26:34Speaker 00

  games, but also being able to simulate forward and, you know, the way, the way that if I'm in a very hard position in a particular game, rather than just playing with intuition, I might sit and think about what I should do. Yeah. Sometimes this goes under the name inference time compute, rather than training, training time compute or pre-training. And, you know, there's, there's a sense in which what it means to do reasoning is very related to this, this ability to sit and think. So we know how to do it for games because there's a very clear witting and loss signal, so you can simulate ahead and sort of figure out what it means to do good or not. And I think math in some, some parts of math, again, I'm not a mathematician and well, you know, always scared about talking about math publicly and saying something wrong to

• 27:17Speaker 00

  all upset mathematicians, but it seems like certain types of math problems are not as constrained as games, but are still constrained enough where there's a notion of, you know, like finding a proof, right? You know, there's, there's definite problems in terms of search, in terms of how, how do you figure out what is the next, you know, move in the proof? But the fact that we might call it a move suggests that the, there's things in math that feel a lot like games. And so we might think that the fact that we can do well at games maybe means that we can do well at certain types of, of mathematical discovery.

• 27:47Pat Grady?

  Well, I was gonna say, since you mentioned Noam, he, he likes to use the example with test time compute of whether it could help to prove the, the Reiman hypothesis.

• 27:56Speaker 00

  Right.

• 27:56Pat Grady?

  Is there a similar, is there a similar problem or hypothesis in the world of physics that you are optimistic AI can help to solve sometime in our lifetimes?

• 28:06Speaker 00

  Yeah. So, I mean, there's a millennium problem relating to physics. And if I try to remember exactly what it is, I'm sure I will butcher it and then no one will believe that I'm actually a physicist, but it's related to the, you know, it's a mathematical physics question related to the Yang Mills mass gap. And, but, but I, I think what I wanted to say is that I think some of the flavor of what physicists care about in doing physics feels a little different. This is where I might get in trouble. It feels a little different than some of like the mathematical proof type things. Physicists are known to be more informal and, you know, hand wavy, but also on the other hand, connected to, in some sense, connected to the interplay between experiment and, and, and, and the

• 28:50Speaker 00

  sort of models that physicists study is maybe what saves them is that, you know, they, they have, things are informal and hand wavy, but very explanatory. And then the mathematicians, it's like, we were saying earlier, the, the engineers discovered all the exciting industrial machines and then the physicists maybe cleaned up a bunch of the, the theory about how that works. And then the mathematicians come later and clean up, like formalize everything and clean it up even, even more. And so there's a mathematicians or mathematical physicists that clean up a lot of, you know, make, make proper and, and try and, you know, understand in informal ways, some of the stuff that physicists do. But.

• 29:25Speaker 00

  Rather than talking about, I mean, I think the key point there is that the sort of questions that are interesting to physicists maybe don't look like proofs, but maybe they look like, and maybe it's not about how do we, given a particular model, how do we actually solve it? Like, once, once things are set up correctly, like, it's often, you know, senior or, you know, people that are trained in the field are able to, to sort of figure out how to analyze the systems. It's more the other stuff, like, what is the right model to study? Does it capture the right sort of problems? Does this relate to the thing that you care about? What are the insights you should draw from it? And, and so, for AI to help there, I think it would look different than the way we're sort of trying to build AI systems for math. So rather than, here's, you know, here's the word problem, go and,

• 30:09Speaker 00

  you know, solve, solve, solve this high school level problem, or, you know, prove the Riemann hypothesis. It's like, you know, the questions in physics are like, what is quantum gravity? What happens when something goes into a black hole? And that's not like, you know, start generating tokens on that, like, what is, what does that even look like? And, and, you know, if you go to a physics department, you know, people hang out at the Black Por, they chat about things, like, maybe they sketch mathematical things, but, but the, you know, there, there's a lot of other things that go, go into this. And so maybe that sort of data that you need to collect looks more like that, or maybe it looks like the random emails and conversations on Slack and the, and the scratch work. And, and so, I mean, there are definitely tools that we can use, like, help me understand this new paper so

• 30:54Speaker 00

  I don't spend two weeks trying to study it and understand it, you know, maybe let me ask questions about it. I think there are problems with the way that's currently implemented, but, you know, I think there are a lot of tools that will help accelerate physicists, just like Mathematica, which is a software packages that does integrals and, and, you know, it does a lot more than that. Sorry, Stephen Wolf, but, you know, I use it to do integrals. And, and, you know, sometimes it doesn't know integrals, and you can look them up in like, these integral tables and, and, anyway.

• 31:28Speaker 00

  You know, I think, you know, there's, and this applies to other branches of science, too. Like, I think there are the ways in which the questions are asked and what it means to do science in different fields, maybe looks, can look further and further from games, let's just say. And so to the extent that that's true, I think we'll need to, and not even clear that we'll need lots of ideas, or, I mean, we, sorry, we will need lots of ideas, but it's more just like, we'll have to, I think we'll just have to approach them all differently and maybe not, like, maybe eventually we'll have a universal thing that knows how to do all of it, but initially, like, at least to me, the, a lot of these things feel a little different from each other.

• 32:03Speaker 02

  You'll have a front row seat to it, in part because you're also on the prize committee for the AI Math Olympiad, which is something I'm personally super interested in. Maybe, to your last point, I'm kind of like, maybe eventually this stuff generalizes. Like, why do you think people are so focused on solving the hardest problems today, like, physics, math, those were the subjects that everyone was terrified of in school, right? Where it feels like there's a lot more other domains that are also unsolved for now, like, do you think, do you think going for the hardest domains first kind of lets you get towards a generalized intelligence? Like, how does, how does solving these different domains kind of fit together in the, in the grander puzzle?

• 32:40Speaker 00

  Yeah. The first thing that comes to mind when you said that is to just push back and say, well, it's not hard, these are the easy domains. I mean, I'm, I'm bad at biology, but I can't, can't, it doesn't make any sense to me at all. My, my, my girlfriend actually is bioengineering and in biotech, and so I, like, what she does just makes no sense to me, can't understand any of it, where physics makes complete sense to me. I, I think, I think maybe a better answer, or a less global answer, is that, like I was trying to say about math, there are, there are constraints, and, you know, and in particular with math, a lot of it is unembodied. You don't have to go and do experiments in the real world, you know, they're, they're sort of self-sufficient, and that's, that's close to like what generating text, like,

• 33:25Speaker 00

  the, the way language models work, or even the way some reinforcement learning systems Work for for games. And so I think the further that you go from that, the messier things become, the harder it probably probably is. And also, the harder it is to get the right kind of data to to train these systems. If you want to build a AI, and people are trying to do this, but it seems difficult. If you want to build a AI system that solves biology, I guess you need to also make sure robotics works, you know, so that it can do those sorts of experiments and like, it has to understand that sort of data. Or maybe it has humans do it. But you know, there's a lot, you know, for a self sustaining AI biologist, it seems like there's a lot of things that are going to go into it. I mean, on the way, we'll have things like AlphaFold 3, which, you know, just came

• 34:08Speaker 00

  out and which I didn't get a chance to read the details of. But, you know, I saw that they were trying to use it for drug discovery. And so, you know, I think each of these fields will have things developed along the way. But it but I think the the less constraints there are, and the sort of messier and more embodied it is, the harder it will be to to accomplish.

• 34:26Speaker 02

  That makes sense. So like, hard for a human is not the same thing, doesn't correlate to hard for a machine.

• 34:31Speaker 00

  Yeah, plus also, maybe humans disagree about what.

• 34:34Speaker 02

  Yeah, some of us think more like machines, I guess. And then I guess the second question was like, do you think it all coalesces into like, one big model that understands everything? Because right now, it seems like there's a lot of domain specific problem solving that's happening.

• 34:46Speaker 00

  Yeah. I mean, the way things are going, it seems like the answer should be yes. It's really dangerous to speculate in this field because everything you say is wrong. Usually much sooner.

• 34:56Speaker 02

  Good thing we're on record. We'll hold you to it.

• 34:58Speaker 00

  Yeah, exactly. But also, like, what does it mean to be differences, you know, like, there's a trivial way to make both things make the question meaningless by like, you know, you say the model is the union of all those other models. But there's also sense in which mixture of experts is not was originally meant to be that. It's not that in practice at all. But, you know, there is, there's a sliding scale here. But, you know, it does seem like people, at least the big labs are going for the one big model and have a belief that that's, you know, well, I don't know, but maybe I will in the future, understand what the philosophy is there. Yeah.

• 35:35Pat Grady?

  Dan, we have a handful of more general questions to kind of close things out here. So I'll start with a high level one. If we think kind of short term, medium term, long term, and call it, you know, five months, five years, five decades. What are you most excited or optimistic about in the world of AI?

• 35:54Speaker 00

  Five years, five years ago was about, it was after the transformer model came out, but it was around maybe when GPT-2 came out. So it seems like, you know, for the last five years, we've been doing scaling. I imagine within the next five years, we'll see that scaling will terminate. And maybe it will terminate in, you know, a utopia of some kind, you know, that the people are excited about, where we're all post economic and so forth. And we'll have to, you'll have to shut down all your funds and, you know, return monopoly money because money won't matter. Or, you know, or we'll see that we need lots of ideas. Maybe there will be another AI winter. I imagine that, and again, scary to really speculate, but I imagine like something interestingly

• 36:37Speaker 00

  will be, something will be interestingly different within five years about AI. And it might just be that AI is over and we're on to the, you know, we're on to the next exciting investment opportunity and, you know, everyone else will shift elsewhere. And, you know, I'm not saying that, that's not what's motivating me about, about AI. But, you know, so maybe five years is enough time to, to see that. And I think in, in one year, I mean, or there's a five, I messed this up, whatever. Maybe it was five months. I don't remember.

• 37:08Pat Grady?

  It's five months. It's okay. It's okay. These are approximations. I know you said physicists are very hand wavy. Venture capitalists are very hand wavy. These are approximations.

• 37:17Speaker 00

  Yeah, in physics, there's, I like to joke that there's like three numbers. There's zero, one, and infinity. And those are the only numbers that matter. You know, things are either arbitrarily small, arbitrarily large, or about order one. So, okay. Good, thanks for reminding me. But, but yeah, for, for five months, I mean, I'm excited to, well, to, to learn what's exciting at the forefront of a, of a huge research lab like, like OpenAI. And I think one thing that'll be interesting will be the delta between. The next generation of models, right? Because there's ways in which things are scaling up in terms of, and it's not really public, I guess, aside from meta, but in terms of size of data, size of models, and

• 38:00Speaker 00

  we see scaling laws, and scaling laws, they'll relate to, you know, something like the loss, and, you know, it's hard to translate that into actual capabilities. And so what will it feel like to talk to the next generation model? What will it look like? Will it have a huge economic impact or not? And I think, you know, in terms of like estimating velocity, right? You need a few points, you can't just have one point. And we sort of have, you know, we're starting to have that with GPT-3 to GPT-4. But, you know, I feel like with the next delta, we'll get to really see what the velocity looks like and what it feels like going from model to model to model. And maybe I'll be able to make a better prediction in five months from now, but then I guess I probably won't be able to tell it to you guys.

• 38:45Speaker 02

  Thanks, Dan. One thing that stood out to me is just like, your writing is so accessible and light and funny, and that's not what I'm used to when I read super technical stuff. Like, do you think all technical writing should be informal and funny? Like, is that deliberate?

• 39:03Speaker 00

  It's definitely deliberate. It goes into, I think in some sense, it's inherited, I mean, I definitely am a not-serious person, but I also think it's inherited sort of from the style of the field that I came from. But I'll tell you a story. I was at lunch, I was a postdoc at the Institute for Advanced Study in Princeton, and I was having lunch and joking around with this professor, Nadie Seiberg, who's a professor at the institute. And we got into, I think, we're talking about, someone asked a question about like, what is a good title? And I was like, oh, the title has to be a joke. And he was on board with that. And then I was explaining that, for me, the reason to write a paper is for the jokes.

• 39:45Speaker 00

  Like, you have a bunch of jokes in mind, and then you want people to read those jokes. And so you have to package it into the science product, and people want to read the science product, and they're forced to suffer through the jokes. And Nadie is this Israeli professor here. It's like, I don't get it. Why can't you just do the science? Why do you need the... Like, the jokes are great too, but you should write for the science, not for the jokes. And I was adamant that I write for the jokes. But I think it's what you said, that at some point, you learn about the scientific method and the formal ways of doing things, and you learn all these rules, and then you grow up a bit. Or maybe, I had a roommate who was a linguist, he's now a professor of linguistics at UT Austin, and he emphasized that you can...

• 40:28Speaker 00

  He would tell me which rules that I could break, or where the rules come from and why they're important or not. And you sort of realize that you can break these rules, and the ultimate goal should be, is the reader going to read it and understand it and enjoy it? So you don't want to do things that compromise their ability to read and understand, but you don't want to obscure things. You want to make it, if it's more enjoyable, people are more likely to read it and take the point. It's also more fun if you're writing it. So I think that's where that comes from.

• 40:54Speaker 02

  Dan, thank you so much for joining us today. We learned a lot, we enjoyed your jokes, and I hope you have a wonderful second-to-last day at Sequoia. Thank you for spending part of it with us, we really appreciate it.

• 41:06Speaker 00

  Thanks, I was absolutely delighted to be here chatting with you guys, it was wonderful.