Daniel Litt: The Mathematician's Guide to AI

a16z’s Lisha Li sits down with Daniel Litt, Assistant Professor of Mathematics at the University of Toronto, to unpack AI's rapid progress in mathematics, what today's frontier models can actually do, and what they're still missing about the way mathematicians think. Daniel explains why some recent AI-generated results are genuinely impressive, including an autonomous solution to the Erdős unit distance problem, but argues that solving problems is only one part of mathematics. Today's models can grind through calculations, combine known techniques, and search enormous spaces, but still struggle with intuition, theory building, identifying the right questions, and developing the kind of big-picture understanding that drives much of mathematical progress. Lisha and Daniel also explore how AI is already changing mathematical research, why an explosion of AI-generated papers could distort academic incentives, and what happens if researchers outsource the work of thinking rather than use AI to deepen it. Ultimately, they ask a question that extends far beyond mathematics: as AI gets better at intellectual work, how do we make sure humans keep getting better at thinking too?

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  The goal of mathematics is not to produce mathematical papers. It's to produce some kind of understanding. Maybe some of that understanding resides in model weights. To me, that's like pretty unsatisfying.

• 0:09Speaker 01

  Comparing Anthropic with OpenAI. Do you detect any differences in how that is similar to human reasoning?

• 0:16Speaker 00

  They definitely are not good at it autonomously. But with some hints, you can kind of get them to do something interesting. A lot of progress in mathematics comes from like letting, you know, a thousand different flowers bloom and people pursue their own curiosity. And then, you know, the boundaries of knowledge expand in some kind of fairly uniform way.

• 0:32Speaker 01

  What has been the most impressive result so far?

• 0:36Speaker 00

  My favorite fully autonomous result by an AI so far is the solution to the Erdgesch unit distance problem. There was some lema I wanted to prove. None of the frontier models could do it. So I like worked out a ton of examples on my own and I realized, oh, well, maybe like here is some reason why it could be true. Like, once I had that statement, the models were able to very quickly prove that sort of better statement.

• 0:57Speaker 01

  How should like the mathematics community kind of best adapt and benefit from this?

• 1:04Erik Torenberg?

  AI can increasingly solve math problems that would challenge professional mathematicians. But solving a problem isn't necessarily the same thing as understanding it. In this episode, A16Z infra partner Li Shili sits down with University of Toronto mathematician Daniel Litt to separate the headlines about AI and mathematics from what the models can actually do today. Daniel explains why recent results have changed his views of AI, where frontier models already resemble human mathematicians, and where they still fall short, particularly when it comes to intuition, developing new theories, and even figuring out which questions are worth asking. They also explore what happens to mathematics when generating a proof becomes cheap, why academic incentives may need to change,

• 1:49Erik Torenberg?

  and how mathematicians can use AI without outsourcing the understanding that makes the work valuable in the first place. And more broadly, they ask what mathematics can teach us about working with AI as increasingly capable models move into every knowledge profession.

• 2:06Speaker 01

  I am so excited to have you on, Daniel. And so Daniel Litt is a professor of mathematics at the University of Toronto. Toronto is my hometown, so also very exciting. But the thing that is most special here is Daniel's an actual practicing mathematician. And in addition, he's been incredibly vocal about his evolving views of AI in math. And so I feel like every, if I just don't check in with you, you know, like in two weeks, something, you know, different has been revealed. And then you're very kind of like, what do you call it?

• 2:35Speaker 00

  I have a lot of opinions.

• 2:36Speaker 01

  You have a lot of opinions, exactly. So I want to get into that. So, I mean, one of the things that I'm most interested in is not just like a discussion of how the capabilities of advance. I feel like in math, that's definitely the headline, et cetera. But also, you've been very thoughtful about how practicing mathematicians should respond. And so that kind of gives us a chance, an opportunity to talk about, actually, what is special about math? It's not just like, hey, AI has been really making progress here, but delve into what actually mathematicians do. And so maybe like with that arc and line, we can start with, what has been the most impressive result so far? Given all the recent progress for you? And then maybe, yeah, we'll kind of take it from there.

• 3:14Speaker 00

  Yeah, so, okay, so there've now been a lot of results. Some of them produced autonomously, some produced like semi-autonomously, some who's like where the AI contribution is just not at all clear. There are a lot of different areas. So anything I say is kind of, you know, I can only really comment on things that I have some expertise on. So it's quite possible that if you talk to a different mathematician, you'll get like different answers here. So my favorite like fully autonomous result by an AI so far is the solution to the Erdgesch unit distance problem, which I think was announced in mid-May. So, at least what I liked about that is it seemed to me that like, it was in some ways a little bit creative. So I think some of the results we've seen have had kind of the flavor. If you kind of take some known techniques and apply them in maybe a clever way,

• 3:56Speaker 00

  or you, I don't know, they have kind of been some kind of results I would characterize as like last mile. Like where some recent work was done, like deep work done by a group of human mathematicians. And then the AI took the final step. But yeah, with the Erdgesch distance problem, I think it was something where the result was like a little unexpected. So first of all, my sense was like that people working in the area thought it was true, and then there was a counter example found. But then also it like brought in some techniques from another area. I think those techniques were like not especially like deeper news. They were sort of classified ideas from the 60s, but they were new to this area of studying, you know, point configurations in the plane. And so that was pretty cool.

• 4:39Speaker 00

  And then afterwards, we got to see like it was kind of fruitful. So a bunch of mathematicians took those ideas and used them to find counter examples to a bunch of other interesting open questions. So for example, like the sum product conjecture for the real numbers. So that's why at least what way I like to think about how cool the result is. Like you look at it post hoc and you see like, oh, well, were whatever new ideas that were introduced, if any, like kind of useful to do other things, did they improve our understanding of something? And I think that's maybe so far the main example I know of a result of that form.

• 5:09Speaker 01

  Yeah, I think it's really meaningful that you're commenting on this because that result came out, to your point, in May. And there's been so many headlines so far. And it's kind of probably hard for somebody who's not a practicing mathematician to appreciate the differences in these headlines. And so you kind of already started laying out sort of a taxonomy of like, what is different in that proof? And so it'd be kind of interesting maybe to use that as kind of both an excuse to talk about where you sense the model differences are and what like mathematicians actually do. So in this case, I mean, the most kind of maybe naive understanding of what mathematicians do is that we're pushing around symbols in a logical manner. And this is why RL is so successful at this, because you can kind of both verify it somewhat cheaply compared to other domains. And then also because the rules are quite illegible.

• 5:52Speaker 01

  And so you can kind of like, if you're superhuman at that, you might be good at math. But I think that of course betrays most of actually what is interesting about mathematics, which is perhaps I think you said this as well, but I think anybody who's tried to do math is sort of like, it's about the understanding and getting at truth, remaining confused and developing intuitions. And I mean, the tool to do that stuff is of course in having really strong abilities to push out logical implications. But maybe if you can kind of speak to like, when you say it's most impressive and creative, like decoupling just the inhumane maybe feats of just like logical implication from like, where is it being creative?

• 6:34Speaker 01

  What is it helping engender in terms of like mathematical activity as well?

• 6:38Speaker 00

  Yeah, okay. So first of all, I mean, you kind of characterized it as like inhuman in some way. I actually think the argument was very human.

• 6:44Speaker 01

  Ah, oh, I'd love to go into that.

• 6:45Speaker 00

  OpenAI released some chain of thought.

• 6:47Speaker 01

  Yeah.

• 6:48Speaker 00

  It was very recognizable. It was like, if I tried to imagine like my chain of thought and trying to solve a problem, like it might look kind of like that.

• 6:55Speaker 01

  Yeah.

• 6:56Speaker 00

  We haven't seen the raw chain of thought, so maybe that would.

• 6:59Speaker 01

  Maybe they cleansed it a little bit, yeah.

• 7:01Speaker 00

  Yeah, or maybe the model like to swear along in the middle of the chain of thought and they cleaned that up or something, but no. But at least the summary seems pretty reasonable. And I would say that's actually like kind of typical of most of the results that I've studied. Like they don't seem inhuman at all. They seem absolutely like something a human mathematician could produce. And they're like typically understandable, if not so well-written. If you just look at raw model output, it's not like there's some move 37 or whatever. It's like a human mathematician doing math. It's like a human mathematician doing certain types of math. So like, they're definitely, the models are still like, not very good at some mathematical activities. And here I don't mean like by field, but just like certain things you do when you try to solve a problem, the models don't seem to be doing. But certain things they're very good at.

• 7:43Speaker 00

  So the ways they might be a little bit inhuman is like they don't get tired. They know a lot. But if you actually just read the final output, it doesn't seem kind of inhuman.

• 7:53Speaker 01

  It's interesting because, I mean, obviously, we can't get too much information from the labs who are producing these models. Why of what the training recipes are or how they're kind of advancing in reasoning? But at least one of the things we do know, and I think kind of OpenAI spearheaded this, is this like reasoning in natural language is actually what they happen to scale up. And it's not actually pushing a lot of lean verified proofs as like the training corpus. And that's kind of amazing. On another side, what's kind of interesting is it's not clear that a lot of the mathematical training data, if they use that to a large extent at all, is reflective of how mathematicians think. If that's fair, because a lot of it is not like legible as traces, right? Like most of the papers are crisp and like polished.

• 8:35Speaker 01

  The textbooks certainly just show very little motivation of how something is developed, which is why it's like usually easier to kind of follow a research direction by actually talking to the researchers and how they're thinking about it. So I'm kind of curious, before maybe even going to the taxonomy as an excuse, like when you're examining these models and their results, comparing Anthropic with OpenAI, do you detect any differences in how that is similar to human reasoning? And then also if you have any comments on like insights on perhaps why natural language scales so well that way, even though it's.

• 9:07Speaker 00

  Okay, so first of all, I really like your point, by the way, that they're mostly doing natural language reasoning rather than lean. Like I think that that suggests to me that like these, you hear a lot of people say like math is a verifiable domain, like that explains the progress or whatever. Like my sense is that because they're primarily scaling informal reasoning, like probably the techniques are going to generalize to other domains pretty well. That's just my guess. Okay, you asked a little bit about Claude versus ChatGPT. My sense is that they're pretty similar in terms of capabilities. I've played around a lot more with ChatGPT than Claude Fable. But it seems like there's a lot of cases where OpenAI will drop a solution to some problem and Anthropic will say, oh, you know.

• 9:44Speaker 01

  We also, exactly.

• 9:45Speaker 00

  It actually seems like they're solving a very similar collection of problems. And it's like kind of a relatively small portion of what human mathematicians do. So yeah, one thing that is interesting is like we see the solutions have a certain flavor, right? Like there'll be things that, this isn't surprising, like there'll be things that rely on the model's strengths, like their ability to round out a long computation or pull together kind of technical ideas from many areas, or like many, maybe many papers that, you know, a human mathematician might not have read. But they're not, like they seem like weaker in things like intuition or like having some big picture point of view. Like a lot of what I do as a mathematician is like I have some kind of philosophy that's like very non-rigorous. Like maybe I think this thing is kind of analogous to this thing. And then like a lot of what I'm working out is like figuring out how to make that precise

• 10:29Speaker 00

  and like, you know, trying to measure the extent to which I've succeeded in understanding that by like, can I solve a problem or whatever? Or like, can I find an interesting phenomenon which I don't understand and then now I can understand it? And like so far.

• 10:42Speaker 00

  I haven't, you know, you can see even the best results the models are producing, you don't see that much of this kind of reasoning. It's more like they're very, very good at applying some known techniques. Which is, to be clear, that's like a very powerful thing to do, to be very good at applying like all known techniques. There are mathematicians who have had great careers, doing very high quality work of that flavor. And I think a lot of what the models are producing is high quality in that way. But it's like some kind of fairly narrow band of what mathematicians care about. So far, I do think there's like signs of both, you know, all the frontier models starting to be able to do more fuzzy things.

• 11:22Speaker 00

  So like I've, I've tried to get both, both Fable and ChatGPT 5.6 Sol, I guess, to do some kind of theory building and it's like, they're not, they definitely are not good at it autonomously, at least with like whatever scaffolding I've set up. But with some hints, you can kind of get them to do something interesting. You know, when you do, when you give the models hints, it's always a little hard to tell like what part is from the model and what part is from you. But my experience is that like, if they can do it with like a hundred bits of hints or whatever in six months, maybe they can do it without hints.

• 11:55Speaker 01

  Yeah.

• 11:55Speaker 00

  You know, I do think there's signs that they're also kind of somehow picking up some of this like implicit and unwritten mathematical knowledge.

• 12:02Speaker 01

  Okay, I would love to, so go into the, the intuition part and where it sucks at. To put it in a very basic way, but you actually mentioned a small detail, which is that, you know, from the public, you've gleaned that Anthropic and OpenAI are probably neck and neck. But you personally are using a lot more ChatGPT, like why is that? Or 5.6.

• 12:25Speaker 00

  Well, why, I don't know, I mean, I just, I think it's just in our show.

• 12:29Speaker 01

  Creature of habit, yeah.

• 12:31Speaker 00

  It got, one thing is that ChatGPT got better at math earlier. So like for a long time, the Claude models were just like not useful for research math. And I think maybe around Opus 4.5 or Opus 4.6, they're like more or less caught up.

• 12:47Speaker 01

  Yeah.

• 12:47Speaker 00

  But you know, some experimentation suggests to me they're pretty neck and neck. And so for my own work, you know, except when I'm just experimenting, I mostly just stick with one.

• 12:56Speaker 01

  Yeah. As people outside of labs like us, it's really interesting just to compare how they differ on the frontier. And I mean, you know, to your point, it might be a little bit of momentum. I do think that, at least from my anecdotal experience, 5.6 has been like a lot more clear in exposition. And it just like, it's, there's a little bit, and this might not be true. I mean, obviously the, the models are just incredibly jagged at the frontier. But like, it's in the explanations of results to me, I always find that 5.6 is giving a more accurate theory of mind of what it assumes I know and don't know. Whereas Fable might be explaining something very trivial, but then just like jump at like, well, you know, obviously you should know these things and.

• 13:40Speaker 00

  Yeah, I'm not, I find they're both pretty bad at theory.

• 13:43Speaker 01

  Okay, great. So then seeing from your eyes, you're probably asking much deeper questions. Okay, so the thread that I really wanted to pull on was when you're talking about the models maybe starting to get better intuitions or even theory building. So maybe before we even dive into that, it would be useful to kind of talk through like, what is your primary, you know, activity as a mathematician, especially in your area of like algebraic geometry. Probably has a very different flavor than a combinatorialist or, you know, some other areas. So if you can give a little, maybe brief lay of the land and then kind of explain what your, what your mathematical activity was pre-AI and then maybe how it's kind of like changing with AI.

• 14:23Speaker 00

  Yeah, so, so I mean, I think there are a lot of different kinds of mathematicians. There's like a lot of different, you know, spectra on which one can put a mathematician. So definitely a lot of mathematicians like solving open problems. And then I think, I'm one of those, like I like to solve an open problem. That's, I think of myself as a problem solver as opposed to like one, you know, one other taxonomy you could, you could have is a problem solver versus a theory builder.

• 14:52Speaker 00

  At least for me, the point of an open problem is it's like supposed to measure your failure to understand something. So it's kind of like a benchmark.

• 15:01Speaker 00

  Right? Like, you know, one problem I really like is the Grothendieck's speech curvature curvature, whatever that is. It's, it measures something about our, our failure to understand differential equations. So it's like, there's some very basic object we would like to understand. If we can't answer this conjecture, we know we don't understand it. Okay. So in practice, like how do you get a, a problem which is supposed to be measuring something you don't understand? Well, like, of course you try to understand the thing better. And, and in practice, what that means is like, well, you try to find the smallest situation where you can't understand something and, and fiddle with it. And then you stare at like what, once you win, you stare at like what you developed to win and try to turn that into some theory.

• 15:43Speaker 00

  So that's one thing you might do, you might try to like solve a problem and in so doing, develop some kind of new understanding of the situation. You might also just like have some feeling that like this thing is related to this other thing. And you know, you might start building a table like, oh, this property A is related to property A prime. Property B is related to property B prime and so on and so forth. So for example, in my work, a lot of it is motivated by some analogy between comology of algebraic varieties and representations of fundamental groups. So that's some, some fancy stuff. But it's just that this analogy is like very, very fruitful. And like really any phenomenon that appears on one side, you can find an analog on the other side.

• 16:24Speaker 00

  And so, you know, trying to, to realize that dream has led to a lot of beautiful mathematics for the last like 30 or 40 years by people like Kyle Simpson and Jichiro Matsuzuki and others.

• 16:40Speaker 00

  And so here it's really just like someone noticed like, here's an analogy. And then that analogy has like led to a huge amount of developments. This aren't really like an open problem at the end, although of course like as you develop this, you come up with lots of open problems. There's just like a philosophy that you're trying to, you know, you're trying to realize. And that philosophy is like super not rigorous, actually. It's not like symbol pushing at all.

• 17:02Speaker 00

  Yeah. So that's another kind of activity that I like. Yeah. Beyond that, you know, a lot of

• 17:12Speaker 00

  what you do when you try to,

• 17:16Speaker 00

  you know, a lot of the activities, you're like, you're actually trying to figure out what the right question is even. Like here, you're, you have some object, you feel like you don't understand it. And like figuring out what you don't know is like actually a very challenging thing to do. So, so, you know, there's like a list of, you know, you can go online and list open conjectures or whatever. And this like doesn't really capture in a lot of ways what we don't know. Like often finding the conjecture is like really, really hard. So I don't know, a good example of this is like the Burchett-Swinerton-Dyer conjecture, which is one of the millennium problems. Is this like beautiful relationship between L functions of elliptic curves and the set of solutions to the corresponding equations of the rank of the, of the group of solutions, whatever that means.

• 18:00Speaker 00

  It was discovered by like, it was like the first big data conjecture. So Burchett-Swinerton-Dyer had like found all these statistics on elliptic curves in the 60s. Like this was one of the first ever computer aided bits of mathematics. And they like graphed these statistics and they noticed that, you know, some, the slope of some line on a graph was related to some other algebraic invariant they knew, and that was the source of this conjecture. So a lot of the time you're just like working out examples and like trying to, like you're doing kind of science. Like you run an experiment and you try to figure out an explanation for that experiment. And so, and I think going back to AI, I think like, these are things where AI seems so far to, you know, help a lot more in some things than other things. So like, you know, I think the more vague

• 18:45Speaker 00

  of phenomenon is, or like the, the less you have a precise question in mind, the less useful it happens to be. And so you were asking like how I use it in my daily life. Like actually what I've found is that the projects that I have that kind of predate AI, like the projects I've been thinking about for three or four or five years. It's just not that useful. Like it's primarily kind of a substitute for Google or something. Like I might use it to like learn about some related topic or like something where I would have earlier Googled something and then like read a paper. Like maybe I'll discuss it with AI instead. So it saves some time for sure. It's like not really doing deep intellectual work for me. But then, you know, because I, you know, I'm now like, well, I suck at coding. And so now I have, you know, my good friend who's really good at coding.

• 19:26Speaker 00

  And now I have all these coding projects. It's like suddenly, well, if I had a question where coding would have been really useful, I would have procrastinated on it for six months until I.

• 19:35Speaker 01

  Tireless PhD student.

• 19:38Speaker 00

  Exactly. Yeah. So, so yeah, now I've picked up all these projects where yeah, coding is really useful. Like the models are like very good for kind of massively parallel things. Like if you want to find an example of something, you can just ask it to find, you know, work through a thousand examples in parallel. It would be 10 examples at a time, you know, 10 different subagents. And like, that's really useful. But these are like kind of different activities, which are like, I don't know, on top of what I was doing before.

• 20:02Speaker 01

  I'd love to dig in.

• 20:04Speaker 00

  Yeah, go on.

• 20:04Speaker 01

  Yeah. Sorry. Cause you were mentioning, you know, there's projects that you've been working on for like three, five, you know, three, four, five years. And I don't know if it's correct to say like, those are more of the theory building aspect of it, because you did characterize yourself as like an open problem solver. Or like, what is the thing that is, because like the deep thinking part, maybe, maybe like, you know, kind of for this audience, it might also be useful to kind of say, or explain, you know, most of pure math is, it's not like it's being motivated by, and you know, nothing on applied math, but applied math is like, at least there's some external motivation for why a certain formal structure might be interesting to study. Whereas this one, it's like purely, it seems almost sociological. And to some extent, I think like Thurston made some comment or a point about this in the seventies that it is a sociological phenomenon.

• 20:47Speaker 01

  If more and more mathematicians start examining something, then you'll maybe, you know, converge on some interesting structures. But it's just like, it's not, it, there's some. Reason why people would prefer to study something where they think it's beautiful? Like, what is it that drives maybe you in particular? And then maybe you can make a more general comment about the profession?

• 21:07Speaker 00

  Yeah, I mean, so definitely some people are motivated by like beauty or aesthetic considerations. I'd try not to be motivated by that.

• 21:15Speaker 01

  Oh, that's more commercial.

• 21:16Speaker 00

  Yeah, I think that's a bad way. Like, well, one thing, like, that sort of limits you, right? Like, one thing, kind of a failure mode I see among young mathematicians sometimes is like, you have something and you kind of think you know how to prove it. And then, like, the proof feels really ugly and you decide. But like, okay, I mean, what if you're wrong and it's not ugly? Like, why limit yourself?

• 21:37Speaker 01

  For this audience, like, what is ugly? Because I have an intuition of what's ugly, but like, what is that for? Kind of spell it out.

• 21:43Speaker 00

  Well, I don't really know. I don't really have aesthetic feelings, but people, people sometimes feel this way. Like, maybe it involves a lot of grinding, some calculation that's not illuminating or something like that. But you should, like, win by any means necessary, in my opinion. Like, I like to think of what I'm doing as like doing kind of physics, except with concepts. So, you know, instead of beauty, I try to think about maybe like what is kind of fundamental? What's going to open up further understanding most?

• 22:10Speaker 01

  Yeah.

• 22:10Speaker 00

  Well, you know, am I introducing a new idea that will be like broadly useful to understand this object?

• 22:16Speaker 01

  Yeah.

• 22:17Speaker 00

  I mean, I guess, you know, in some sense, there's, there's like some aesthetic consideration there, but it's, I try to, I think the orientation of like trying to do good science rather than trying to do art is what I prefer. But there's a huge, I mean, there's a huge variety of, of opinions here. And like lots of mathematicians think of themselves as being like closer to poets or something.

• 22:37Speaker 00

  Yeah, what I, what I do think is broadly true is that like progress comes from people, like, it seems to come in mathematics from people sort of pursuing their personal curiosity. And that it's sort of been crucial historically that there's like a lot of different people with different views of what's interesting. And then the frontier of knowledge expands and expands. And then suddenly, you know, you have these opportunistic situations where a new idea has been introduced and you can kind of now find, you now can suddenly cascade through a bunch of other things that we didn't understand before.

• 23:07Speaker 01

  Yeah. I mean, so then going back to the three, four, five year problems and where are the models sort of not useful? Kind of ask it another way. Like, when you're doing the deep thinking, like, is it just that it's just not clear that you formulated as a problem? And it's more that you're thinking about these, like, what are, you know, what are the fundamental kind of physics of, you know?

• 23:32Speaker 00

  Yeah, so in some cases, I mean, the, some cases, there is like a well-stated problem here. You know, I said, I've been thinking about things for, you know, maybe 10 years now at this point for certain problems. Sometimes there is just like a conjecture that I would like to prove that, that is, is there.

• 23:49Speaker 00

  I think one reason the models might not be useful for some of these things is like the conjectures are true. So like, I think that certain, you know, for, for example, you know, this unit distance conjecture problem, like, the, the general belief in the community was that it was true and then it turned out to be false. And so what that means is that there's like a specific construction you can do to, to refute it. On the other hand, I think a lot of the things I, I think about, like, I don't know, okay, maybe I'm about to, you know, this, someone will come up with a counter example to the peak curvature conjecture or the three month hypothesis tomorrow. And like, I'll look like a fool. But in general, like, these conjectures fit into some very broad theoretical framework, which means that, look, we actually have a lot of evidence that they're true.

• 24:36Speaker 00

  And so, you know, yeah, so that's part of it. Like, there's not like a construction you can do to refute it. You need to somehow kind of, you know, this, we have this giant framework where certain, certain pieces of it are only conjectured. Well, and you probably need to resolve some of those conjectures to win. We also have like a pretty good sense, I think, that like very serious new ideas are needed to resolve those conjectures. So like, of course, you can't be sure. Like, maybe there's some very clever construction that will let you, I don't know, avoid having a big new idea. I don't know. Maybe, you know, it's quite possible we'll find this out. But my sense is that for at least a lot of the things I've been thinking about,

• 25:20Speaker 00

  there are, they're just not accessible to, you know, applying known techniques very, very technically, in a very technically strong way. So you need to develop a new technique. I'm, just to be clear, like, I'm not saying the models won't be able to do this. It's just like so far they seem not to.

• 25:36Speaker 01

  Actually, that's exactly the point that I wanted to delve into because it's, um, you know, to your point, it's like, okay, we can just try, try to calibrate and forecast like why they would get better at this. But it is true, like, you know, that just providing construction for a counter example, they seem to be strong on. If you have to start developing either new theory or, to your point, techniques, machinery to solve, to, to prove why conjecture is true, it struggles more. And it's probably because a lot of what it's drawing on is also just like techniques that have happened in other areas and they're porting it over. And to your point, that's why maybe the unit distance problem was such a more creative result because it was maybe doing more of that on its own.

• 26:14Speaker 00

  From another, at least it was like drawing in something from a non-expected area.

• 26:17Speaker 01

  Exactly, exactly. And so like, maybe then to kind of ask the question, it's not that, because now we're like, okay, fine, AI is getting so good, so fast. We can't count it out. But like, why, what do you think it has to?

• 26:29Speaker 01

  Yeah, I guess it's, you're spelling out what it has to get better at, but maybe some more kind of feelings on like, why it's kind of particularly hard to then develop that new theory and technique?

• 26:39Speaker 00

  Yeah, that's a good point. And I think you just need a different, like, my, my guess actually is it's probably totally doable. It just hasn't been done yet. Like, maybe you just need a different RL environment. I don't know. Yeah, so I mean, at this point, my expectation is just like that the trajectory will continue upwards. I'm not a skeptic of continued capabilities growth.

• 26:59Speaker 00

  But yeah, you know, I think what is definitely true is that like the skill of like developing a theory or you're like building your understanding of some poorly understood object is like a fuzzier one. So it might be harder, you know, I guess you can try, you can tell it, you know, develop your understanding of Zeta functions. Then once it proves through your hypothesis, you give it a reward. But it's like kind of harder, I think, to come up with some like intermediate things that you can reward. Yeah, I mean, that said, you know, I do think mathematics as a whole provides a lot of conjectures of varying levels of difficulty. So, you know, maybe, maybe this explains why there seems to be a little bit of progress in these areas.

• 27:42Speaker 00

  Like presumably, they are trying to, you know, get it to solve lots of problems. And some of those problems develop at least some of the skills of theory building. And humans are able to develop these skills. You know, I guess sometimes they get rewards from their PhD advisors and their advisor says, oh, that's a good idea or something based on some element of human taste or whatever. And that, that might be something one can do too. But yeah, my, my, my expectation is that just like as part of continued capabilities growth, we'll see growth in these areas too.

• 28:09Speaker 01

  Yeah, yeah. And I like the framing where you're sort of casting these increasingly difficult conjectures as a, as a, you know, a form of curricula for both humans and of course, AI. And it, you know, and maybe kind of getting a little bit too philosophical for some people's tastes. It's like, it, it gets at the question of like, why are we particularly good? Or maybe particularly bad at math too? Because it's, you know, what is it that we're either struggling to do or some people are particularly good at when you develop new theory? Because it's, it's not, again, it's like not really, or maybe it's related to like, why can we formulate good structures for physics as well?

• 28:51Speaker 01

  It's just like, it's not obvious all parts of the world are kind of understandable and legible that way. But some parts are, and therefore we try to do it because there's maybe compressive pressures on our, our, you know, our minds because we need to, we can't understand anything beyond, you know, compressing stuff more finely.

• 29:11Speaker 01

  I don't know if that's also like the correct interpretation of that.

• 29:13Speaker 00

  I think that's, that's, I'm not, I'm a little bit skeptical as like compression as a metric of interest, but it's definitely like as an anthropological reason for like why we do a certain theory building. I just think it's true. And in fact, I think it's like kind of like our, our inability to just grind is kind of important to our ability to make discoveries. So, and just like as an example, so I've, I have one paper out so far where the models were kind of useful. So they like, like proved a couple lemmas in the paper. Um, so this is some, some situation where like we'd kind of proven the main result. And then there were like some lemmas I was unhappy with, like they seem not to be optimal. And so I like worked with, actually, this was with, um, with Gemma Deep Think.

• 29:54Speaker 01

  Oh.

• 29:54Speaker 00

  Which at the time was also on the frontier, no, no longer.

• 29:59Speaker 00

  I, I, where I kind of worked with it to improve the lemmas. And so what happened here was there was some lema, um, I wanted to prove, uh, and the models couldn't do it. So the, none of the frontier models could do it. And so I like worked out a ton of examples on my own and I realized, oh, well, maybe like here's some reason why it could be true. Like, so I like a kind of better statement of the lema. And then, okay, once I had that statement, like, okay, probably I could have done it pretty fast. But also the models were able to very quickly prove this, prove, prove that sort of better statement. So like our inability to prove it like led to an improvement in the result. Okay, now you can take the original lema I had that the models weren't able to prove, put it into ChatGPT five point six pro and it will output like the worst proof you've ever seen.

• 30:42Speaker 00

  Like 10 pages of like just brutal calculation with no insight whatsoever. And so, okay, this would have been a perfectly fine proof, but it would not have led to this discovery of like, I think a kind of beautiful conceptual explanation for why this thing we discovered was true. So like, we found a better proof because we couldn't do, I mean, okay, I, I say we couldn't do the question, like what actually happened. And here I, I understand that I'm being hypocritical about my complaints about ugly proofs before. It's like, I realized like this horrible grind proof would work.

• 31:10Speaker 01

  Yeah.

• 31:10Speaker 00

  And I like could not bring myself to do it. Uh, and so I look for another argument. Uh, and so, I mean, it's not, yeah, it's, but you know, now the models can do this for very long. Technical calculations pretty reliably.

• 31:23Speaker 01

  Yeah. Well, I mean, to your credit.

• 31:24Speaker 00

  I don't know.

• 31:25Speaker 01

  I think I understand why you're saying it's like, don't, you know, just shy away from trying to prove something just because it seems ugly, because it's like you have to take the first step, right? And eventually you all try to work towards insight. And so, you know, you, you might be shy about admitting it, but it is still maybe driven by whether it's like aesthetics or just like pure, you know, it's like, I want to understand. And if understanding just means it's a little bit simpler or, you know, more compressed than, than I've gotten understanding. I mean, it's just like a deep philosophical question too, is like, what it is.

• 31:52Speaker 00

  Right. Like, I mean, of course, any, you know, there's a reason we do informal mathematics rather than like writing out long formal strings of symbols of CFC or whatever.

• 32:00Speaker 01

  Yeah, yeah, yeah.

• 32:01Speaker 00

  Even though, you know, besides length, it's just like somehow we're trying to put it in a way where we are actually getting some non rigorous understanding out.

• 32:07Speaker 01

  Yeah. I mean, and.

• 32:08Speaker 00

  I think somehow.

• 32:09Speaker 01

  Yeah. And even how that even relates to like, why it helps. And again, it's like, is it, is it because of our inability to grind? Or is it because like, if we, if we are pushing towards something that's more compressive, it also hopefully also ascends on the, does it help understand other fields as well? And it's, it's just, it is somewhat magical that when we try to optimize for two, the, the both, that it, it tends to coincide. I don't know, actually, you know, if that's a fair, it's, it's actually a quantitative statement. But like, why, you know, that is, is, is kind of magical.

• 32:39Speaker 00

  It's at least sometimes true. Yeah.

• 32:40Speaker 01

  Yeah, exactly. Or, or, you know, we, we certainly biased towards the cases in which it is. That's why we call that a good theory to build. But it's, it's, uh, yeah, it's very much, I think, you know, um, it's like, this is what we can study and understand. And therefore it's also very convenient that it was rich, um, in, in kind of mathematical, uh, results there.

• 33:04Speaker 01

  Yeah. I mean, okay, so I think there's two things that we can like, you know, uh, pull on, which is like, what, um, given you've seeded that, or not seeded, but you've believe that mathematical ability of AI is going to continue advancing. It can do some of the things that you're, you know, doing nowadays. How should, how should like the mathematics community kind of best, um, adapt and benefit from this? I mean, as a, just kind of saying, as somebody who's, you know, doesn't have the time to kind of practice mathematics anymore. This is kind of very, very great because I can maybe dabble more. There's a lot of results that can come out, but I can also see where, you know, you've made, uh,

• 33:46Speaker 01

  the more precise point of we can not motivate the right kind of behavior of understanding and development. And so I would love to kind of hear more about, um, your views there.

• 33:56Speaker 00

  Yeah. So, okay. So first of all, like, it is clearly really exciting, like that there are increasingly capable models that are like producing high quality results.

• 34:05Speaker 00

  At least, you know, some high quality results, also, also a lot of slop. Uh, yeah, some, some good stuff. And so, you know, as the models get really capable, you know, my hope is that they will answer a lot of the questions that I've been like, you know, kept up at night thinking about. I'm like, I'll, I'll get to learn the answers. It's like, that's really exciting. And, you know, a lot of people got into math, uh, like largely because they enjoyed learning math.

• 34:28Speaker 01

  Yeah.

• 34:29Speaker 00

  Right. Like the first thing you do as a, a math student is you like learn stuff that other people didn't. You do that for like 20 years, I think, before you, before you start doing, oh, maybe not quite 20 years, but 15 years.

• 34:40Speaker 01

  If you're lucky, 20 years, yeah. If you started early, yeah.

• 34:43Speaker 00

  Yeah. Um, yeah. So, um, that said, like the goal of mathematics is not to produce mathematics papers. Like it's to produce some kind of understanding. So maybe, okay, maybe that, some of that understanding resides in model weights or something. To me, that's like pretty unsatisfying, like, uh, my own, you know, my motivation for doing mathematics is like, I would like to satisfy my own personal curiosity. I think people should be, uh, have the capability to do that. And like, that requires a pretty substantial apparatus. Like, it's like simply not the case that you can like study the questions I think are fundamental, unless you've invested a huge amount of time and effort, um, kind of getting to the point where you can meaningfully do so.

• 35:27Speaker 00

  And then moreover, like that, you know, even the people, like, you know, you have the small group of people doing like fancy research math on the frontier or whatever, um, that relies on like a huge apparatus of like, you know, thousands and millions or billions of people who are trying to learn to think mathematically. Like you need a, an entire mathematical community to support a small group of people who are on the frontier. Like you just, without the pipeline, then the pipeline doesn't exist. So if you think that's important, like development of human capital that can like meaningfully engage with frontier mathematics, I think that, uh, you know, you still have to incentivize those people to actually invest their time and effort, getting to the point where they can engage and then do so in a, in a high quality, meaningful way.

• 36:09Speaker 00

  So right now, I think like the existing incentive structures for math research do not, uh, do not encourage people to do that. So, you know, right now, if you're like a postdoc on the market, you want to get a job, maybe for the next couple of years before the community adapts.

• 36:26Speaker 00

  The best way to do that is like, you want to produce a lot of papers, which maybe prove, you know, old conjectures or whatever, and you can do that by playing the slot machine until, um, the model produces a hopefully correct proof of such a result.

• 36:39Speaker 00

  So you don't even have to pick the, the, the theorem in advance. So like, here's an experiment you can do, you can take codex, you can say, go online, find five recent conjectures and algebraic geometry and prove them. And, uh, okay, I've run this experiment and with some back and forth, I was able to, you know, in an hour, get like three, you know, quite bad papers, but correct papers, um, which, okay, now sitting on my hard drive, waiting for me to email the relevant people, but this is not a good use of my time to invest in results.

• 37:08Speaker 01

  Yeah.

• 37:08Speaker 00

  But yeah, you, you definitely see people doing this. So there's, you know, been a huge uptick in post archive, mostly not very interesting. Some of it is interesting.

• 37:19Speaker 00

  But I'll, you know, a lot of it is kind of clearly low quality, even if insofar as the, like, even if the result is something that would have been like kind of highly rewarded a year ago, just like the, there's sort of no evidence that a human being is actually engaged with it. Like there's no development of human capital or understanding. Like sometimes, you know, we've seen examples where like three or four or five papers with the exact same proof of the exact same theorem out in, within a couple of days of each other, which is clearly, you know, some situation where someone's playing the slot machine.

• 37:48Speaker 01

  Yeah.

• 37:48Speaker 00

  ChatGPT is kind of consistently finding the.

• 37:50Speaker 01

  Well, that's also interesting. It's like not, it's kind of mode collapsed on like certain paths of reasoning. Yeah.

• 37:56Speaker 00

  Yeah. And I think it's like not obvious that this problem goes away as the models get better. Like maybe it does, but maybe it doesn't. Like, I think a lot of what I was saying earlier is that a lot of progress in mathematics comes from like letting, you know, a thousand different flowers bloom and people.

• 38:09Speaker 01

  Yep.

• 38:10Speaker 00

  Pursue their own curiosity and then, you know, the boundaries of knowledge expand in some kind of fairly, hopefully fairly uniform way. And like, it's really high dimensional space of mathematics. And then like opportunistically, you, you suddenly get some applications or like answers to old questions we found. And it's like not clear to me that if, if, you know, we kind of subordinate mathematical exploration to what the model want to pursue, like if what you're getting is like one mathematician duplicated a thousand times or like actually, you know, a million different mathematicians doing a million different things.

• 38:41Speaker 01

  You know, I think actually that's pretty, I mean, you know, aside from like, okay, math has to adapt, um, by changing incentive structures, I think this is actually a pretty, um, um, maybe dangerous or something that the labs have to pay attention to. Because, you know, and, and on one side, what's been successful for them is that this emergent reasoning capability is obviously incredibly powerful. And we've been, I mean, it's just created a lot of like great PR headlines. But, you know, to your point, and also like where my interests tend to is that like human mathematicians, they are coming from all sorts of weird intuitions that like, you know, and that is why you end up developing, you know, to your point, the frontier that is so diverse that you actually can draw from and then be able to connect and produce a lot more.

• 39:26Speaker 01

  And so it, if it's true that most of these like proofs that are being pushed out by the labs are converging on very similar things, because they are technically drawing on the same body of literature. And that's where they're strong at now. It's, it's not clear that, I mean, one, you know, where are they developing their intuitions? It's from practicing mathematicians. It's not clear where that other emergent, you know, stronger, diverse intuition might be coming from. It might come, but I don't have a good theory for where it comes from. And it's also just not clear that, you know, just with like test time compute and various post-training scaling that you can even, I mean, it's a huge debate whether you could introduce new capabilities. And it's like precisely, I think you should be studied in the math context, because like where it's good at and where it fails.

• 40:11Speaker 01

  And where human mathematicians are good is exactly a very kind of like. Precise question to study it on. So this is like a long way of saying that I it isn't clear to me that will maybe produce. If we don't incentivize enough people to interact with it, maybe unlike other domains, it might not actually continue producing a superior result without a human aid.

• 40:38Speaker 00

  Yeah, let me actually push push a bit further even. So like, let's suppose the models become like really robustly superhuman, like even like we're not even adding like meaningful cognitive diversity.

• 40:47Speaker 01

  Okay.

• 40:47Speaker 00

  Like I claim like still actually we still won.

• 40:50Speaker 01

  Okay, great.

• 40:51Speaker 00

  Human mathematicians.

• 40:52Speaker 01

  Yeah.

• 40:52Speaker 00

  So right, so we buy, so like it's because the, you know, there's like a, it's like, there's a question on how we're designing society, right? Like, maybe the optimal situation is like you have the models doing all sorts of math research and like that leads to very, you know, in this like highly, you know, whatever non-uniform diverse way, like maybe you don't need humans to add that kind of diversity. So like, maybe that's the optimal thing to do. And like, in the end, that leads to lots of applications and lots of improved understanding and so on. It's like despite something being optimal doesn't mean you do it, right? Like, there's no reason to think that, you know, if we hand over control of whatever math research to the models and just let them do their own thing, that it will do the optimal thing.

• 41:36Speaker 00

  And in fact, like if you, you know, if we are, if we kind of instrumentalize what we want them to do, like we want to say like, oh, you know, make our life better or whatever, it might not be the case like that, that like what they decide to do that is like, is pursue a wide variety of interesting research, right? Like, they may just try to take the direct path. We don't know what's gonna happen. So I don't know, if you believe that there is any value in sort of this broad-based like fundamental research, which I do, like, I think that's one of the most valuable things humans or whatever, or the models can be doing. Like, I think the easiest way to guarantee it happens is like to keep a community with like broad interests who are like pushing the models to do and like helping us to design a society. Like, that's what we're pushing for.

• 42:19Speaker 00

  Like, I think at least, you know, my hopeful vision for the future is that like humans are not like totally disempowered, like we have some control over where we're going. And like, if that's the case, like what we end up doing is gonna be driven by human interests. And so you wanna have people who have lots of different interests and also the capabilities to actually pursue them. Like, you want people who are like smart and engaged and like, you know, well, can do not just like mathematical thinking, but all sorts of thinking.

• 42:42Speaker 01

  Yeah, yeah. I mean, yeah, a great fear is like as AI advances, we don't develop the right ergonomic kind of interfaces to actually encourage us to also be good, continue to be good thinkers. And it's so easy to kind of relinquish that because you just off. And the models aren't even good at that level of like, you know, a thinking where it's like the top level structure, but despite that, it's so easy to. And so especially for math, I mean, just like, you know, another selfish reason is if you kind of math max, you know, I think it's actually a great pedagogical excuse to actually get just really rigorous at thinking about various things. I mean, this is not why mathematicians do it, but as just somebody like.

• 43:21Speaker 00

  It's part of what we do.

• 43:22Speaker 01

  Okay, great. Because it was kind of, you know, I thought it just really helped give me a very good framework to think about many things, not just mathematics. And as somebody who's also, you know, now a parent of a two-year-old, I kind of think about this a lot, too. You know, it's not about kind of grinding, even though, whatever, it's still good, but like not to shit on grinding too much. But, you know, just hearing, I used to collaborate a bunch with some Hungarian mathematicians, Volasz, Zegedy among them, and I just heard that in Budapest, they would just teach group theory when you're in primary school. And I'm like, well, we should definitely do that. We should continue doing that now that AI is so good at, you know, somewhat good at

• 44:04Speaker 01

  explaining, but it's far more accessible, we should actually, you know, probably proliferate that even more. And so maybe that helps with bringing more people to the frontier rather than, you know, just.

• 44:15Speaker 00

  Yeah, I mean, this is something I'm concerned about, right? I mean, of course, I mostly talk about math because that's like where I live. But like, I think, you know, one nice thing about thinking about this is that we're one of the first professions to kind of, you know, have a significant impact of high-quality models, although I think maybe we're one of the first professions for it to happen so publicly. But, you know, my sense is that there are plenty of third professions that are.

• 44:40Speaker 01

  Coding, 100%.

• 44:41Speaker 00

  Coding, but also just like, I mean, I think that, you know, anything you do at a computer, like probably a huge amount is being done by the models at this point. And then like, there's not even a public reckoning about it. But, you know, because math capabilities are useful for the company, for the labs to talk about, I think, a bit more publicly than everyone else. But yeah, and I think, you know, one reason to try to maintain like human capital in this area is just like, it's a model for all professions. Like presumably, we still want people who are like meaningfully engaging with the world and like experts and, you know, have like talents and trained skills and so on. And like, yeah, so I was, yeah, it's actually very convenient that the math profession is so inclined with education here because I think we're also seeing like, you know,

• 45:25Speaker 00

  some amount of challenges, you know, among college and like secondary education coming from AI, too. So as you said, I mean, it's also an amazing tool to learn, so.

• 45:38Speaker 01

  Yeah.

• 45:38Speaker 00

  You know, people have, I've heard people start talking about a bimodal distribution in their classes where there's some people who are really like figuring out how to take advantage of neutrals and other people who are just like letting them do their homework and then bombing everything else.

• 45:51Speaker 01

  Unfortunately, I don't think that adapts fast enough, but it's like, we definitely wanna be living in a world where we're producing better thinkers. I think it's just, you know, even if we talk about just the pure kind of optimization game, I think that's better for us. And so, but as, you know, just like a human being, I'm like, that would be pretty inconvenient if we became worse thinkers just as AI ascends. And it's too easy to let that happen. So we should kind of be thinking hard on how to actually take advantage of this and harness it for our own improvement as well.

• 46:21Speaker 00

  Yeah, I think it's like, it's sort of interesting to observe, like, the models at their current level capabilities, let you do a lot more, they let you do a lot of things you wouldn't have done more cheaply than, you know, cheaply enough to do them now. But it's not clear to me that like in many cases, they're actually improving the quality of outputs.

• 46:39Speaker 01

  Yeah.

• 46:40Speaker 00

  And I think this is common, like you have a new technology that's doing something a little bit worse than was previously done, but much cheaper. And so you get a lot of, suddenly, a lot of like low-quality outputs that are displacing previous high-quality outputs. But I think it's possible to use the tools in a way that actually like improves, you know, the quality among, along every dimension. It just refers, requires some thoughtfulness and some redesign of institutions to actually incentivize that.

• 47:03Speaker 01

  Yeah. Well, hopefully capitalism works there. I do feel like the most high-value things do require people to use it effectively. And right now, the models are not good enough without like the human experts to actually participate. But to your point, there's a vast majority of maybe like more junior and entry level. And if, you know, it's harder for for those roles to adapt as well. And so the thing that would be a mistake is to use the AI models in a way that doesn't, basically, you need to be ascending and using the models to deepen your understanding. And it's so easy for human nature just to be lazy. And you have to resist that because that is the moment that you will kind of, kind of lose, basically.

• 47:47Speaker 01

  And so you kind of, you know, kind of forgive the very competitive language, but it really is that like, it's just so easy to to kind of relinquish thinking to the models. The models can't really think. And so as things are ascending so fast, it's like critical that you continue developing those facilities and actually leverage it to improve those facilities rather than relinquish. Yeah.

• 48:09Speaker 00

  Yeah. Yeah, I mean, one thing I think has been nice about, you know, sort of this vast increase in like semi-expert attention or like model attention on math problems is like now there's been, you know, okay, I've been complaining about slot papers or whatever, like, you know, people who are not, you know, producing really high-quality stuff. Often, that's actually coming from professionals. Like, it's not, I'm not saying like, you know, there are people who, you know, like within academic mathematics, there are incentives to produce like a lot of.

• 48:37Speaker 01

  Gases.

• 48:38Speaker 00

  A lot of stuff. And that's, you know, that's one place that slot comes from. So there's definitely also like slot coming from non-experts, but that I kind of actually don't see as a net negative. Like, okay, there's a lot of, now there's a lot of like documents on the internet one might have to come through to figure out if a problem has been solved or not. But to me, it seems like just the fact that there's lots of people excited about math is like kind of a positive. So that's like a nice thing.

• 49:01Speaker 01

  I totally agree. I know, I get to talk to you.

• 49:03Speaker 00

  And not even kind of a positive, obviously.

• 49:05Speaker 01

  Yeah, yeah, no, exactly. It's like suddenly there's a spotlight on it and I can nerd out about math more. I was actually kind of curious, you've had any comments on the, I guess this is another constructive result, but the elliptic curve of rank thirty, that just came out yesterday. So if that.

• 49:22Speaker 00

  That's right.

• 49:23Speaker 01

  Yeah, yeah.

• 49:25Speaker 00

  We don't have any details about it yet.

• 49:27Speaker 01

  I know, there's rumors.

• 49:28Speaker 00

  Yeah, we don't know how it, I mean, so it's due to, I guess Claude Fable, prompted by Leventhal Poje and a collaborator whose name I unfortunately forget, maybe you can.

• 49:39Speaker 01

  So Levon.

• 49:39Speaker 00

  Add it in post.

• 49:40Speaker 01

  Yeah, I just know the Twitter handle. Yeah, we can.

• 49:44Speaker 00

  So, yeah, I mean, so a lot of these nice recent results have come from Levent with unclear amounts of autonomy. So my sense is that some of them are semi-autonomous rather than fully autonomous.

• 49:56Speaker 00

  Yeah, with this, we don't know anything about the methods. So, yeah, this is a fun construction. Without knowing about the methods, it's very hard to say how significant it is.

• 50:05Speaker 01

  Yeah.

• 50:05Speaker 00

  What I would say is that if you want to understand, kind of historically, how such results have been proved, or sorry, have been understood by the community, they're like, cool, but I wouldn't say they're like a big deal. So like the typical place a result like this might go is like someone's website of records.

• 50:25Speaker 01

  It's on annals level results.

• 50:27Speaker 00

  Yeah, it's not an annals, but that said, it's cool and it's like, you know, it definitely, there were a few very, very, or are a few very, very talented mathematicians who like these kinds of questions. So No Malkeys being maybe the most famous example. So Alkeys and Clagspurin are the ones who kind of have been pushing this record for a while. And they recently found a rank twenty-nine.

• 50:44Speaker 01

  Yeah.

• 50:45Speaker 00

  Example.

• 50:45Speaker 01

  One of.

• 50:46Speaker 00

  Which was, you know, the previous record.

• 50:48Speaker 01

  Yeah.

• 50:48Speaker 00

  You know, so those are mathematicians. And I think, you know, people like it and it's cool now that the models can do this sort of thing. But, yeah, it's very, you know, one thing I always say about a model result is like you cannot evaluate it except in retrospect. And like, this is also true of human mathematics. Like, sometimes a problem we thought was really important or would require really neat, deep new ideas does not. And sometimes it does, and you can't really know in advance. So it's always exciting when a problem like that gets solved. But then like to figure out how significant it is, you still have to look and like, well, Levent and Claude, and I think Ava Howell, maybe is the collaborator.

• 51:22Speaker 01

  Okay.

• 51:22Speaker 00

  Have not yet told us how they did it. So.

• 51:25Speaker 01

  Have they been mostly more secretive? I think they've released some traces for stuff, but.

• 51:30Speaker 00

  Yeah. So for this one, I think they haven't yet, unless I missed it. Yeah, they, they, you know, Levent likes to tweet out his, his results. But, yeah, he has been, you know, slowly releasing some kind of PDF write-ups too. I think it's, you know, he's just having fun on, on, on the internet.

• 51:45Speaker 01

  Yeah, yeah.

• 51:48Speaker 01

  It's, it reminds me of when you're saying, you know, you have to evaluate how the results came. So I had Mark Zelkey and Metab Swani on from OpenAI recently. And they were saying how, you know, it's kind of been the most charming or delightful is just that the proofs have been relatively short. They're not like two hundred page and, you know, maybe corresponding to your grinding point as well. But maybe this is kind of optimized for, in retrospect, it was picked that it was short? Or maybe, do you find that on average, stuff that you throw at GPT, you know, Saul or Fable

• 52:28Speaker 01

  tends to be shorter and more kind of legible to the human versus it might just go haywire and just grind it out?

• 52:35Speaker 00

  Yeah. So, so, I mean, I, I think it is nice to sometimes produce short clever proofs.

• 52:40Speaker 01

  Yeah.

• 52:41Speaker 00

  So that's, of course, everyone likes a short clever proof. I think my sense is that the reason they're not producing long, complicated proofs is that they cannot. Just like that the ability to check correctness is not yet there.

• 52:58Speaker 00

  So, you know, I mean, even actually, you know, if you ask the models to produce a short proof, you can then often ask them to, you also just ask, is that correct? And they will often say no. So like, they're much more reliable than they were six months ago, for sure. But like, it's still, you know, they will still sometimes produce things that are just wrong and that they know they're.

• 53:19Speaker 01

  Yes.

• 53:20Speaker 00

  Yeah. I think the problem with producing a very long thing is they might not know they're wrong. And so, but I wonder if, you know, presumably internally, OpenAI and Anthropic have probably solved a lot more problems than they've released. And I imagine quite a few of them are, they're just not sure if they're true. So for example, you know, with these, this recent list of ten problems released by OpenAI, those were all formalized in Lean, which is of course a very good, good evidence that they're true.

• 53:47Speaker 01

  Yeah.

• 53:48Speaker 00

  I have no doubt that there were a lot more that they could not formalize in Lean. Like because the prerequisite options have not been put into MathLip yet, for example.

• 53:57Speaker 00

  There were probably more that were longer, which also makes it challenging to check.

• 54:00Speaker 01

  Yeah.

• 54:00Speaker 00

  So, yeah, this is my guess. And we do actually see very long generated proofs on the archive. So for example, someone recently posted a claimed proof of resolution of singularities in positive characteristic, which was eight hundred AI-generated pages. It's definitely, I mean, I'm sorry, I haven't read it, I had an error, but there's no way it's correct. Like this would be a major result. It's just not within the capacities of the kind of models that you're reasonably well calibrated.

• 54:27Speaker 01

  Yeah.

• 54:28Speaker 00

  And there's definitely no human has read it. Definitely the models are not able to check this kind of thing. Yeah.

• 54:34Speaker 00

  So they, I mean, yeah, so this is my, my expectation is that the reason it's producing short clever things is just like, that's what we can check.

• 54:42Speaker 01

  Yeah.

• 54:43Speaker 00

  And you can get it to produce long things that are grindy or hard to check, but then.

• 54:47Speaker 01

  Yeah, yeah. We'll have to get there. This is actually more what it says about capabilities, frontier capabilities, rather than, in, in a perhaps more negative way, rather than, hey, it's just, you know, so good at these clever. No, I, I think that's totally fair. I mean, especially if you look at an adjacent domain like code, right? Remember reading something that Cursor put out about testing their long horizon, you know, harness? In this case, they're trying to reproduce Sequel Light in Rust. And it was just so telling, like how far we are from, you know, and that, that seems like a very, you know, comparable task of like, it just, it's very long. You have to make sure there is something to verify that it's a correct implementation by testing a suite of, you know, cases.

• 55:27Speaker 01

  But, you know, you actually need a harness there. In this case, it's not just like the raw models and it takes a while. And you can actually compare with different frontier versus, you know, non-frontier models, who's the planner, et cetera. Like differences in their capabilities as well.

• 55:46Speaker 00

  Yeah. I always think in practice, like the, to elicit a long proof, you kind of need a harness.

• 55:51Speaker 01

  Yeah.

• 55:52Speaker 00

  And when you make a harness whose goal is to elicit a proof, I think it often decreases reliability because you're just trying to produce output.

• 56:01Speaker 00

  So like ChatGPT five point six pro is like very, it really tries not to say wrong stuff, for example. And although, you know, it happens and then you'll ask it like, oh, was that correct? And it'll say no.

• 56:12Speaker 00

  But, you know, when you're trying to get it to ideate, like you try to get it to be creative, you try to get it out of this very like, you know, rigorous rut in order to like, you know, actually get it somewhere. And I think if your interest is in, I mean, there are a lot of people who are, you know, trying to arbitrage the prestige mechanics of academic mathematics and trying to elicit a lot of proofs, which are not necessarily being checked. And in order to do that, I think you just decrease the reliability in order to get a lot of stuff. So, you know, I think any harness that can elicit a two hundred and fifty page paper is probably not being very careful about what it's producing.

• 56:47Speaker 01

  And, and you're saying that it's decreasing or it's not reliable just because it's, the capability isn't there yet. And so it's just kind of forcing a long, longer horizon task on it. Yeah.

• 56:56Speaker 00

  Yeah. I mean, in practice, like how does, like a human checking it, like can't check a two hundred and fifty page paper either. Like, like you, you, you know, you can't reliably check it line by line. What you would try to do to understand it is you try to like understand the overall global structure of the argument. And you like stress test it in various ways, like, like would this argument imply something else that I know to be false? Like, oh, like, does it work in this special case, blah, blah, blah? And also the models seem not to be able to do that kind of like more, I don't know, fuzzy, like unit testing a proof very well yet. So actually, like one of my favorite tests for the models, which they haven't, haven't succeeded at yet, is there's a paper, I won't name it, that came out a couple of years ago that was wrong.

• 57:36Speaker 00

  And it was like very hard to find the, like in close enough area to myself that I like immediately, like was claiming some big result. Okay, immediately downloaded and like started reading it. And it was like very hard to find the actual specific error. But it was also very clear from the structure of the argument that it couldn't work. So it was like, you know, me and a bunch of other, well, he was proving something that was too strong to be true. He's going to have to go a little bit further.

• 57:59Speaker 01

  Yeah.

• 58:00Speaker 00

  So me and a bunch of other experts like immediately like realized it was wrong. And like, you know, we emailed the author and like we kind of went back and forth until someone figured out what the precise specific error was.

• 58:12Speaker 01

  Okay.

• 58:13Speaker 00

  And so far, the model seemed not to have been able to do this. Like the specific error is quite subtle. And they're also not able to do this kind of overall, kind of big picture task checking, which is kind of like how people in practice check papers.

• 58:27Speaker 01

  Yeah, yeah. Which is kind of, it mirrors, you know, why in code, it's like, it's so clear, it's good at the syntax, but you know, higher level architectural stuff, still very weak, maybe it'll ascend there, but probably need some harness help. Who knows? I mean, people have, you know, evolving opinions on how much the harness and the model have to co-evolve and which one is necessary. The next model requires less. So I feel like in, in math, it'd be very interesting to see how you, if you do any experiments with the harness there and how, how that improves, because it is a marker of general reasoning. Yeah.

• 58:59Speaker 00

  Yeah, I mean, I do, you know, I have my own sort of bad little harnax in Codex.

• 59:03Speaker 01

  Oh, yeah.

• 59:03Speaker 00

  In Codex. But, yeah, it's, you know, I, personally, I do not enjoy autonomous mathematics very much. So I mostly do not use the harness. I mostly try to use it to help me understand the stuff.

• 59:15Speaker 01

  Okay. No, it's totally fair. Yeah, exactly. You don't want to automate your job away because that does involve you being in the loop to understand it, which is, you know, necessary to participate. Maybe to, to finish off, I'd love to, this could be just something you haven't thought about or actually thought a lot about. I think you also have a toddler, right?

• 59:34Speaker 00

  Yeah.

• 59:34Speaker 01

  Yeah, yeah. So how?

• 59:35Speaker 00

  A three-year-old.

• 59:36Speaker 01

  Yeah, a three-year-old. Great, great. So you're one year more advanced and probably, you know, have more thoughts on this. Like, how are you thinking about, is it her education or his education?

• 59:46Speaker 00

  Her, yeah.

• 59:46Speaker 01

  Or how are you thinking about her education in, in math and? You know, not to grind, but you know, really just like how, you know, pass on the love of it and how to how to react to AI.

• 59:57Speaker 00

  Yeah. So she's she's three. She's she's never used AI. She is starting to add. That's that's about as far as.

• 1:00:04Speaker 01

  That's that's more.

• 1:00:06Speaker 00

  She can add single, single digit numbers like, you know, by counting on her fingers and

• 1:00:10Speaker 01

  Nice.

• 1:00:10Speaker 00

  count up to maybe thirty reliably and fifty semi reliably. So I'm very proud of.

• 1:00:15Speaker 01

  That's good.

• 1:00:16Speaker 00

  Yeah, I definitely encourage that. We talk about shapes and stuff.

• 1:00:21Speaker 00

  Actually, a couple of days ago, I woke her up and she was like hiding under the blankets. And I was like, what are you up to under there, Sophia? And she was like, oh, I'm doing some math.

• 1:00:30Speaker 01

  Oh, I saw that. I think you tweeted about that. That was.

• 1:00:33Speaker 00

  It was great. So, you know, I think she has like some sense that I like math and she's into it because of that. Yeah, I mean, I would say I don't know, you know, I think the world is probably going to look pretty different in, you know, 20 years or whenever she's kind of

• 1:00:49Speaker 00

  fully adult and and doing her own thing. But, you know, I think a lot of what we educate people for is like pretty robust changes in the nature of the world like. I think the reason to learn math has always been like to think clearly and like better understand the world. And like presumably that's something you want to do, even if there are sort of extremely capable AIs. And this is also true, you know, I personally like math a lot, but also the reason to like read a lot of books and do humanities and so on. So I mean, I think like the actual values of the math profession and like education more broadly are like things we definitely try to want to try to preserve that I I hope to instill in in my daughter.

• 1:01:32Speaker 00

  You know, how much institutions have to change to make sure that happens is maybe an open question, I think, a lot. But yeah, I mean, at least at a personal level, I'm definitely trying to, you know, convince my my three year old that math is super cool.

• 1:01:46Speaker 01

  Oh, yeah, 100%.

• 1:01:47Speaker 00

  One of her first words was like, I see it.

• 1:01:48Speaker 01

  So. What was what?

• 1:01:50Speaker 00

  She has she has a she my my parents gave her a little Icosahedron toy. When she was one. She, yeah, not literally a first word, but she learned the platonic solids quite quickly.

• 1:02:01Speaker 01

  Oh, very good, very good. Next up, group theory. I mean, it's like very natural.

• 1:02:05Speaker 00

  That's right. Yeah.

• 1:02:05Speaker 01

  I mean, it's.

• 1:02:07Speaker 00

  Yeah, I'm actually when I when I teach her about addition and subtraction, for example, we'll do it in the context of a general group.

• 1:02:13Speaker 01

  Oh, very good.

• 1:02:16Speaker 01

  Well, at least you give the motivation. I think a lot of probably skip that part. Yeah, yeah. And, you know, maybe this is how math grad students can also, you know, focus on like training the next much younger generation to use AI in service of actually getting better at math rather than just I think, understanding. Wonderful. Okay. Well, thank you so much, Daniel.

• 1:02:38Speaker 00

  Thank you. It was a lot of fun.

• 1:02:39Speaker 01

  Yeah, yeah, it was a lot of fun. And, yeah, I mean, I think there's going to be a lot more progress very soon. I'd love to maybe catch up and chat again.

• 1:02:46Speaker 00

  Sounds great.

• 1:02:49Erik Torenberg?

  Thanks for listening to this episode of the A16Z Podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or a review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at A16Z.substack.com. Thanks again for listening, and I'll see you in the next episode.

• 1:03:15Erik Torenberg?

  As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com/disclosures.