OpenAI Researchers on the Future of Mathematical Reasoning

a16z Infra Partner Lisha Li sits down with OpenAI mathematicians Mehtaab Sawhney and Mark Sellke to discuss how quickly AI’s mathematical capabilities are advancing, what recent results reveal about model reasoning, and what happens when AI begins making progress on problems mathematicians have struggled with for decades. Mehtaab and Mark unpack several recent results from OpenAI’s models, including advances in sphere packing and the construction of a non-sofic group. They explain why the surprising part isn’t simply that models can search more possibilities or work longer than humans: in many cases, the reasoning traces look remarkably similar to the work of an expert mathematician, including choosing promising approaches, backtracking when they fail, and combining ideas from across the literature. They also explore what this means for mathematics itself: how the role of human taste and judgment may change, whether AI could produce far more mathematics than humans can absorb, and why models that accelerate discovery may also make sophisticated results easier to understand.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Often as a practicing mathematician, you have an idea and then you kind of think it might work. Then you try for a few hours, a few weeks, and at some point you give up.

• 0:09Speaker 01

  Whereas for GPT, like, okay, a human told me to do this. Let's just do this. And so that's why we're sort of in this renaissance of reachable results.

• 0:17Speaker 00

  This is the best part about this problem, which is really nobody had any idea. It's the model just guessing in some insane way.

• 0:23Speaker 01

  It doesn't seem like there's a limit so far, but it doesn't have that context yet.

• 0:28Speaker 00

  It'd be nice for the world if it applied math that much faster. The ceiling for difficulty of a math problem is pretty high. Even if AI continues getting exponentially better at math, plausible will never solve something like P versus N.

• 0:39Speaker 01

  What's the ideal way that this is being taken up by the math community? Probably most at this point are like, okay, AI is obviously doing some non-trivial stuff.

• 0:48Erik Torenberg?

  So AI isn't just getting better at math benchmarks. It's beginning to make progress on mathematical problems that have resisted humans for decades. In this episode, A16Z Infra Partner Li Shih Lee sits down with OpenAI mathematicians Mehtaab Swani and Mark Selke to understand what's actually changing. They walk through recent results in sphere packing, coding theory and group theory, and explain why these advances can't be reduced to brute force. The models try different approaches, abandon dead ends, connect ideas across fields, and in some cases, produce reasoning that reads surprisingly like the notes of a human mathematician. They also tackled the bigger question. What happens to mathematics when proving a result becomes less of a bottleneck?

• 1:32Erik Torenberg?

  From mathematical taste and human judgment to understanding an explosion of new results, Li Shih, Mehtaab and Mark explore how AI could change not just what problems get solved, but what it means to practice mathematics.

• 1:47Speaker 01

  Well, thank you guys for coming. This is really exciting because I think math has been moving so fast with AI. I just love to get to both practicing mathematicians and who work at OpenAI to chat on some of these results. So we have with us Mark Selke and Mehtaab Swani. We're connected actually because Yufei was actually your advisor. And so both of you guys have worked much more deeply in math since I have quit many, many, like over a decade ago. So this is very exciting to kind of hear a download of your thoughts on how OpenAI has been sort of approaching this and also just like where you think math is going with the incredibly rapid advance of how AI has been helping. We can start off with some very basic questions.

• 2:32Speaker 01

  What do you do to the extent that you can, of course, share? And how did you come from being a practicing mathematician to working at OpenAI?

• 2:41Speaker 00

  Yeah, I mean, I guess we both broadly got excited last year when the models started to really take off in math. So I joined a little bit before Mehtaab. I saw the IMO gold medal last summer, basically. I thought, this is amazing. I want to see what the heck they did. Let me go see. And then, yeah, I guess in the fall, Mark gave me a GPT-5 account and then I started playing with the models and very quickly became convinced that, yeah, it was extremely exciting to play with them. And yeah.

• 3:09Speaker 01

  You two were collaborating before that.

• 3:10Speaker 00

  Yeah, we've known each other for a while. We have one paper we actually you wrote jointly.

• 3:16Speaker 01

  So GPT-5 was your conversion?

• 3:19Speaker 00

  Yeah.

• 3:19Speaker 01

  What was the magic that sort of, what question do you throw at it? What process?

• 3:23Speaker 00

  Yeah, so I think, actually, yeah, so I think how this started was, at least for me, the starting moment was something like, there's a collection of problems called, so Paul Erdos is a very famous mathematician. He posed a bunch of problems and so they've now all been collected on this site. And so I specifically worked in combinatorics and a lot of these questions are among the most important. So it's always fun to flick through the slide. But one thing that often happened to me that was extremely frustrating was I would look at a question, see that it's marked as open, and then not actually know if it's correct, not actually know if it was still unsolved because the literature is often quite hard to search. And one instance, I just plugged it into GPT-5 and five minutes later it found a reference. And this was a case where a few of my friends actually started thinking about the problem on the site.

• 4:08Speaker 00

  I was talking with them and, I mean, we had spent a few hours. It wasn't clear if the problem was within reach. And it was just very nice, okay, to be told, yes, this is in reach. Here's how you do it. And yeah, GPT-5 told me this and then I told Mark about this. And this, yeah, this is sort of, yeah, this for me was quite a surprising moment. Yeah. And then we looked more into it and we found 10 more cases sort of like this.

• 4:30Speaker 01

  At the time, I feel like being better at maybe making connections between, as you're saying, like the search for whether there's been a result or a related thing earlier is just kind of humanely hard, but maybe better for a machine. But I imagine as the progress has happened in the last year, what has been impressive has kind of reached beyond that. And maybe through talking about it more abstractly or if it's more natural to talk about it through one of the problems that has been recently announced through, you know, Astra, can kind of enlighten me as to like how the recent progress has been a lot more than just searching through more areas, making these connections between the field and perhaps just actually deeper, more mathematical reasoning that's similar to a working mathematician.

• 5:13Speaker 00

  Yeah, I mean, I think this like search point of being familiar with everything is still definitely like a relative strength that maybe informs like the types of problems that AI is solving now. I think there are some other relative strengths and weaknesses. Another relative strength that's pretty noticeable is just like, it's very good at executing on some like idea once it has it. Whenever you have an idea, there's usually some amount of getting everything lined up. Is epsilon like smaller than delta? This kind of thing, you have to get everything correct. And like for a human, it's easy to get lost in these kinds of details and the AIs just kind of always nail these kinds of arguments, I find.

• 5:55Speaker 01

  Yeah, I feel like you guys will know more detail on this, but like for the unit distance problem, it was just like the approach, there was definitely contributions from OpenAI, but like the approach perhaps was suggested even originally by Erdos. And then it's just that the actual reasoning was a very, very like momentous feat. And so for a human, you're like, well, I only have a limited amount of time. And if after so many steps, it is still not clear. I mean, maybe you're like Andrew Wiles and you actually spent 10 years alone and do something, but like it's not clear, then the risk reward is not good enough. Whereas for GPT, okay, I'll like a human told me to do this. Let's just do this.

• 6:37Speaker 01

  And so that's why we're sort of in this renaissance of like reachable results. Does that track? And did you feel like with the Astra results, is that sort of like where the strengths have been primarily? Or there's an extra ingredient or magic here?

• 6:51Speaker 00

  I feel, I think the unit distance example is about, it's quite telling in the sense of maybe the exact construction. You can make it look very similar to what people had tried before. But I think, I mean, often as a practicing mathematician, you have an idea and then you kind of think it might work. Then you try for a few hours, a few days, a few weeks. And at some point you give up. And then a not so uncommon experience is that you find out a year or two later that somebody else got the idea to work that you thought that didn't work. So somehow getting an idea to work can even be a large portion of the battle. And I think, especially in the case of the unit distance conjecture, there's just a lot of extraordinarily finicky details. And very often when you're doing mathematics, you're kind of gambling against the problem. You're like, maybe I should try this approach,

• 7:36Speaker 00

  but it seems really unlikely and just not worth my time. And the model, I think, in several of these cases, both by combining what it knew and sort of having good taste, kind of made the correct math. And you can kind of see this in the summarized chain of thought we released. You can sort of look at it. It's reasoning like a mathematician and because it knows a few very correct bits, it makes the right decisions and eventually able to prune the search tree. It's not really trying everything. It tries a lot of different things. It's extremely dogged. I mean, it can't try every idea. It has to try a limited set of ideas and it's able to kind of use its knowledge plus good mathematical judgment and find the right path to go along. So, I mean, that for me was like, because this was a problem which a lot of people had thought about. And I mean, the fact that the idea is not so foreign probably indicates that a lot of people had tried it

• 8:20Speaker 00

  or at least a few very serious mathematicians have tried. And I think that's what made it really interesting to see. I think something else that I feel when I see these proofs is like, if I have an idea and I'm trying to execute, it might be that I have some wrong plan for like how to get things to work. And as a human, if you have some wrong path, you go down for a while. It can be hard to like rewire your brain to start over and try a different path. The initial idea is kind of linked in your brain with these other things that ended up not working. It's sort of like your context window is like a little polluted and you can't just make another clone of yourself from last week and say, don't do this, try something else, build your intuition in another direction. But it's very easy to do this with an AI. So I think this is another reason that it's like,

• 9:05Speaker 00

  getting the details right, once you have some good general direction is much less of a barrier all of a sudden.

• 9:10Speaker 01

  And when you say it's much easier to do with AI, it's like, it's not actually being directed with human interference too. It's as you were saying, the reasoning traces, it's like making these choices. Maybe it backtracks, but then it's able to not be distracted by maybe the context in which it's thinking about the problem, like via these machinery. And it's like, do you see it kind of go back as well? Or is it just making good choices? Like, is it a lucky sample or is it actually reasoning like a mathematician? Or, okay, it doesn't do well on this path, but it goes back, but then it doesn't let that pollute?

• 9:38Speaker 00

  No, I mean, it definitely makes mistakes and then it goes back and thinks about it. I think it's somehow very calculating, very correct. I mean, as human mathematicians, you're not always perfect in making these decisions. And the first time something doesn't work, you automatically kind of downgrade how likely this approach is to work and you keep doing this a few times. The model somehow is much better able to like, it seems, for several of the solutions we've seen, it's somehow seem, it seems much better able to update like how likely the path is to work, like versus rejecting a path versus a human doing it. But I think even if it weren't, the fact that you could just start another model session over means that like, you know, it's always going to be the case that.

• 10:15Speaker 01

  Got it, got it. So in some sense, it is like still leveraging the fact that you could like run kind of parallel, you know, agents on the problem.

• 10:22Speaker 00

  Yeah.

• 10:23Speaker 01

  But if it were kind of backtracking, then does make it seem much more like a human, you know, mathematician. And perhaps it is kind of doing some of that stuff, too, because like, obviously, like, we have to like make mistakes in in in order to like even gain intuition for like, why that solution space is like not, you know, not in the the the set of paths that it could be in.

• 10:41Speaker 00

  I mean, I think this kind of thing happens with humans, too, where like, if you get stuck on some approach, you might tell another human your kind of general idea, and then they'll come back and like figure out how to get it to work. And, you know, it's just,

• 10:51Speaker 01

  Yeah.

• 10:52Speaker 00

  It takes more time to do this, like with humans.

• 10:56Speaker 01

  I wonder, I mean, you know, maybe this gets to extend that you can actually talk about sort of like, obviously, don't talk about the training recipes or whatever, but like, it's, it's interesting that if you're just studying, for instance, for math papers, it's like a very poor training set, like a priority for math, because, I mean, maybe math textbooks are even a pure example of this. It's like, really bad at actually reconstructing the motivation for why things were, you know, it's like, don't, I mean, maybe some people like it, but don't learn real analysis from Rudin. It's just like, it's very clean, ready, and crisp. And I think that that's bad, because it doesn't show the struggle that made us formulate definitions in a certain way. Like, why do we even need to

• 11:37Speaker 01

  have real numbers be defined in this like super abstract way, etc. And so, you know, I think paper is also, I mean, unless you're, most people don't write papers with the the context of I need to educate somebody to do to be a mathematician. And so like, the actual maybe curriculum of like, learning math is not inherent in like, a lot of our artifacts as mathematicians. So maybe another way of asking this question is if the reasoning traces are actually producing things that's like, okay, this is actually more close to mathematical thought, like, how does that arise?

• 12:10Speaker 00

  Um, I mean, I, yeah, I guess OpenAI has been like the pioneer of reasoning models and,

• 12:18Speaker 01

  Yeah.

• 12:18Speaker 00

  teaching AI to reason in this way. So, you know, we're doing a lot of work at kind of, kind of all possible directions on, on, you know, teaching models to reason better and for longer, and and, you know, in all kinds of different domains.

• 12:33Speaker 00

  Yeah, I mean, I think I think we're training general purpose reasoning models and it, you know, kind of one, a lot of these behaviors that we're describing mathematically, like backtracking or kind of starting again, I mean, these are, these are not really specific to math. I mean, we're seeing them specifically in mathematics in these examples, but kind of they're general purpose tools for reasoning. And I think if you work hard at reasoning, you should see these patterns eventually.

• 12:58Speaker 01

  So it's just emergent, because it's, I mean, I do think that's why the OpenAI approach was so, I mean, it's like, it doesn't rely on, you know, doing auto formalization in order to like, guide the reasoning. I think that's like, obviously more like us. But it's just like, so not obvious that if you're just like, training on, say, a corpus of like, math proofs, maybe auto formalized and lean, that you get,

• 13:25Speaker 01

  get the sort of like, projection of like, how to think well. Like, put it another way, like, with, maybe, maybe if we think about it with code, like, code is such a good corpus to train on, because it's one of the few data sets that has such large context. You just like, I mean, maybe you see this kind of with books, but they're less structurally interconnected. There's just like, less structure there, I think. It's safe to say, like, on average, in a, in a book compared to like, a piece of code. And so, like, with math papers,

• 13:57Speaker 01

  I feel like, but maybe what, what we're still bad at with coding models is stuff that that data set doesn't contain, which is like, kind of the semantics, like, the syntax is there, but there's a little bit of like, the higher level semantics of what produce, like, why do I have to write it this way? It's not, I'm kind of getting too much of the philosophical. But it is just like, really interesting how it's still emergent, that it's doing good mathematics. And we'll probably get into this in more detail if you guys, you know, wanted to talk in more detail about some of the problems, which is just like, it's, it's not just doing like, the expected, like, we'll push the brute force thing, like, you clearly are impressed with some of the reasoning traces, and it's just not obvious that's gleaned from, you know, what we would imagine would be the easy training set here.

• 14:39Speaker 00

  Yeah, absolutely. I mean, I think this kind of thing is one reason we decided it was important to release like, these summarized chains of thought for these kinds of results, because if you, if you've never seen these, and you just see all these proofs coming out, you're kind of, you're not sure what it means, like, like, is the model just guessing in some insane way? Like, is it thinking in some totally foreign, like, what what's going on? But but actually, it's, it's reasoning kind of shockingly like a, an expert human would.

• 15:09Speaker 01

  Yeah, yeah.

• 15:11Speaker 00

  Yeah, it's, it's very much like reading a colleague's like, notes. I mean, it's a little more disorganized in some way, that kind of, like, especially if you work close enough with the collaborators, sometimes you'll just see them like spill out their thoughts in an email to you. And it kind of, it, it feels like reading a lot of those chained together. So it's, yeah, it's, it's very, it's quite surprising the first few times.

• 15:32Speaker 01

  Were you two sort of very involved in choosing the problems to, to release in this, like, last 10 problem set that Astra has applied to?

• 15:42Speaker 01

  Which was your favorite?

• 15:45Speaker 00

  Yeah, we're definitely involved. Do you want to start? Yeah, I mean, yeah, I guess, yeah, so I guess my personal favorite among these problems is the following. It's, it's extremely simple question, which is just like, it's just about how efficiently can you put a bunch, my circles are not very good, and they're not all the same size, but,

• 16:03Speaker 01

  But we're assuming they are.

• 16:04Speaker 00

  Yeah. So the question is just like, how dense can you place a bunch of, so you have a bunch of spheres, you have a bunch of spheres of radius one and d dimensions?

• 16:16Speaker 00

  So the question is, how densely can they pack? And so, yeah, so, so in two dimensions, it's kind of like, the, so, so d equals one, this is not an interesting question, kind of, it's just a real line, and yeah, you can cut it up, and a sphere in dimension one is just a unit segment, so, okay, you can cover everything. So in d equals two, it's kind of the picture that you know, that everybody loves. It's just like,

• 16:45Speaker 00

  it's just a bunch of spheres, which sort of form like a hexagonal lattice. Hopefully, I've drawn it well enough, but I can draw the hexagon.

• 16:52Speaker 01

  Kind of betraying my naivete on this problem. Is that like, obvious? Is like, a very elegant proof?

• 16:57Speaker 00

  Uh, it's not so, yeah, it's not so obvious that this should work. It was only proven in the 60s, I think.

• 17:04Speaker 00

  There's a short argument, but it's not, it's not so easy. Yeah.

• 17:08Speaker 01

  Where's the intuition? Like, where's kind of like, the machinery of the argument?

• 17:12Speaker 00

  I mean, it kind of like,

• 17:13Speaker 01

  I mean, it definitely looks like it should work. That's why I'm like, oh.

• 17:16Speaker 00

  Yeah, I think,

• 17:16Speaker 01

  But so did.

• 17:17Speaker 00

  Yeah, I think this is the best part about this problem, which is really, nobody has any idea.

• 17:22Speaker 01

  Yeah.

• 17:23Speaker 00

  So, yeah, so, I mean, honestly, the best intuition I have for this is that, like, bees do this, and if there wasn't more efficient way, then probably bees would pack honeycombs some other way.

• 17:32Speaker 01

  Evolution is efficient.

• 17:33Speaker 00

  Yeah. I think beyond that, like, I don't have a great argument. I mean, and I think how little we know is demonstrated by the fact, so, okay, d equals three, the, the answer is just like,

• 17:46Speaker 00

  it's what, it's how you pack like oranges in a grocery store. And this was only, this was proved by Hales sometime in the 2000s, and like, and we don't have a short proof of this, like, I think the shortest proof is like, a few hundred pages.

• 17:59Speaker 01

  What area does it like, draw from in order to?

• 18:01Speaker 00

  So it's a lot of linear programming arguments, and it's very delicate, like, geometry. It's, it's quite ugly, actually.

• 18:09Speaker 01

  Yeah, it's like,

• 18:10Speaker 00

  This is like a famously ugly argument.

• 18:12Speaker 01

  Oh, no.

• 18:13Speaker 00

  And then the two most famous results are d equals eight and 24.

• 18:19Speaker 01

  Eight and 24. There must be some like, weird subspace.

• 18:23Speaker 00

  Yeah, so, yeah, exactly. So this was done in some, Yes, this was done in 2017.

• 18:28Speaker 01

  Sounds slightly prettier, though.

• 18:30Speaker 00

  Yeah, so, so it, so the reason it works out in these two very special dimensions is that, so this is called a lattice packing, so it's like, kind of very regular. And it turns out in these two dimensions, there are two very special lattices. They're called the E8 and Leach lattice. And they're very nice, and they're like unusually dense, like, kind of, they're just very, very pretty structures coming from other areas of math, and it turns out that they're the optimal structures.

• 18:56Speaker 01

  But they're still like, regular.

• 18:58Speaker 00

  Yeah, they're very regular. But, I mean, beyond this, so we don't know any more exact dimensions. We know these five dimensions, and we kind of don't know anything else. And, I mean, like, to give an indication of how little we know, so there are two very surprising things about this. So you can define like, delta d to be like, the densest sphere packing in d, d dimensions. So there's kind of an easy lower bound of like, 2 to the minus d.

• 19:26Speaker 00

  Basic, yeah, this is, this is not so hard to show. Basically, any packing where you can't put in another sphere has to have this density. So, okay, it's not ridiculously small, and we know that it has to decay exponentially. So it has to decay, like, it grows like, 1 minus c for some, at least for some context. So, in large dimensions, you can only cover, like, a vanishingly small portion. But we know, like, basically, nothing else. And so,

• 19:54Speaker 01

  That's just because of like, the high dimensional sphere, like, thing where it occupies, it's just like, yeah, the volume behavior is weird.

• 20:02Speaker 00

  Yeah, so basically, I mean, basically, they don't want to touch next to each other. I don't know, I don't think there's a particularly short way to see that, that it's like exponentially small, but it's known to be exponentially small. And for a long, long time, the best bound was something like, this funny number, like, 2 to the minus point 599 d. And this was proved by two mathematicians in the 70s.

• 20:27Speaker 00

  Kupchansky.

• 20:29Speaker 01

  Okay, that's a weird number, where's that?

• 20:32Speaker 03

  Spit out of a

• 20:32Speaker 00

  It's not

• 20:33Speaker 01

  It's like combinatorial or?

• 20:35Speaker 00

  It's it's the answer to some extremely ugly optimization problem. There's like, there's like a nice underlying strategy.

• 20:41Speaker 01

  Okay.

• 20:44Speaker 00

  Yeah, yeah, I'll say one last thing about this, yeah. So, yeah, I mean, these were these two Russian mathematicians in the seventies. Yeah, it's actually very hard to find their paper. It's like one page, it's like two pages long. Yeah, they don't write very many details because paper was small, but.

• 21:00Speaker 00

  Yeah.

• 21:00Speaker 01

  And the two negative D is just like the square lattice, like the dumb one or?

• 21:04Speaker 00

  So, so, yeah, it's actually not so easy to. So the argument for this is as follows. Basically, imagine that you have a set of spheres and I tell. You can construct a set of spheres so that like you can't put down another sphere. Because if you could put down an extra sphere, you just keep putting it down. So you have a set of spheres so that there's no other sphere which you can put down.

• 21:25Speaker 01

  That's your, it's like almost like a

• 21:27Speaker 00

  Just, yeah, just take any such packing.

• 21:29Speaker 01

  Okay, yeah, yeah.

• 21:30Speaker 00

  And I claim that this has to cover at least two to the minus D fraction. The reason is that like if you blew up each of these spheres by a factor of two, then like they have to cover every point in space.

• 21:42Speaker 00

  And the reason is otherwise you could put down. If there was any empty space, you could put down a sphere there.

• 21:47Speaker 03

  So I guess if you take the usual lattice, there are actually like more places you can put things kind of diagonally.

• 21:53Speaker 01

  Yeah. Okay, yeah, yeah. So that's actually not good. It's like a worse bound.

• 21:56Speaker 00

  Yeah, you can just keep plopping things in. Yeah, this is

• 21:58Speaker 01

  So this is like more, okay.

• 22:00Speaker 00

  Yeah, this is related to this like really funny fact where you like put a sphere on every point on. You can take a cube in high dimensions, you put a sphere on every point.

• 22:07Speaker 01

  It's like vanishingly small.

• 22:09Speaker 00

  It's vanishingly small. It's so small that you can put another sphere in the middle.

• 22:12Speaker 01

  Yeah.

• 22:12Speaker 00

  And it fits.

• 22:14Speaker 01

  High dimensional sphere behavior.

• 22:15Speaker 00

  Yeah, it's very weird. And so.

• 22:20Speaker 00

  So the great part is that.

• 22:23Speaker 00

  So the model shows the following, so I'll write two things, so this is Astra, I guess. Probably the right way to refer to this. So, okay, I'm gonna write something called the LP bound. I'll explain this in a second, and it shows that it's smaller than this very nice number.

• 22:42Speaker 03

  Do you want to say equals?

• 22:44Speaker 00

  Yeah, it's equals, actually.

• 22:48Speaker 00

  D to the two pi plus little o one to the D.

• 22:53Speaker 03

  And if you you can work out what this number is, it's like it's like roughly something like two to the minus point six.

• 23:02Speaker 01

  Close about.

• 23:03Speaker 00

  Yeah, it's surprising.

• 23:04Speaker 01

  But this one, you know, you're like, oh, maybe there's some like nicer kind of like structure there that that fell out.

• 23:10Speaker 00

  And it's M O's. This is like roughly something like two to the minus point six zero one, dot, dot, dot, D. That's the numerics?

• 23:19Speaker 03

  I thought it was six or four.

• 23:21Speaker 00

  Great. Yeah, this shows mine. Yeah. So, okay, so there are a couple of things. So first, what is this LP of D? So Viasoska's work actually builds on some earlier work. It turns out that there's a way to attack sphere packing via what's called a linear programming bound. So LP just stands for linear programming. So Konan Elke's

• 23:43Speaker 00

  gave an approach for sphere packing based on linear program.

• 23:47Speaker 03

  So it's like a, it's like a linear optimization problem over a convex set, but it's all kind of infinite dimensional here.

• 23:57Speaker 00

  And basically what this reduces down to is you try to understand the follow. So what you try to show is you basically you construct a function F. So this is in D dimensions and it's mapping to R. And it has the following properties. So first, F of X. So this is a function in D dimensions, so it's always less than zero if like the size of X is bigger than one. And you second have that the Fourier transform of X, this is always non-negative. So this is just, this is a linear program because the Fourier transform is a linear operator. And

• 24:36Speaker 00

  night race.

• 24:38Speaker 01

  So you're taking just some arbitrary F that satisfies this property?

• 24:41Speaker 00

  Yeah, so you can take any F that satisfies these properties. And what they prove is that delta D is bounded by

• 24:49Speaker 00

  the ratio of the Fourier transform of zero to its to the the Fourier transform of zero. An F of zero to the, over the Fourier transform of zero times the volume of the ball of radius one half in D dimensions.

• 25:07Speaker 00

  And so, okay, this, this proof is not so short for experienced mathematicians. It's like, it's a half a paragraph to prove it, but it's a little bit tricky. And the point is, so it turns out, so this is a relaxation problem. There's no guarantee that taking the optimal F will give you a good bound on delta D. But so what Viasoska did, and this was sort of the key. I mean, a large part of the reason she won a Fields Medal and in 2020 was that, or in 2022, was that she constructed a function in eight and 24 dimensions such that this upper bound matches exactly these two very special lattices. And these are kind of miracles of nature that both you can construct this function and that it gives you the optimal bound.

• 25:53Speaker 00

  But you can just, this is a very, very natural problem. It's a function with a very, two very simple properties and you just want to understand how this behaves in for large dimensions D. And that was a big mystery. We, there was a numerics paper by Kone and several others which conjectured that, just based on doing numerics, that this was the answer. But they had no idea why this would be the answer. And what the model shows is that actually the linear programming bound in large dimensions has this extremely nice asymptonic behavior. And the proof kind of explains where this is coming from. And because you understand this LP bound perfectly, this actually just gives a better bound on delta D.

• 26:37Speaker 00

  It turns out that this old bound can be kind of reinterpreted in this framework. And what the model does is it shows you the best possible bound you can get by this framework.

• 26:45Speaker 01

  So the model sort of made the connection and

• 26:49Speaker 00

  Yeah.

• 26:49Speaker 01

  What is this sort of like, what do we, yeah, what did?

• 26:51Speaker 00

  So I, I think, so the model gives a function F, which, so first, it constructs a function F, which gives you this bound. And then it shows that there's no function F which does any better. So it's an equality, which is quite strong. So in like sort of, we now understand this, this problem in high dimensions very well. And that's pretty remarkable.

• 27:14Speaker 03

  And the model was just kind of told like, analyze this linear program in high dimensions, you know, go have fun.

• 27:19Speaker 01

  Got it.

• 27:20Speaker 00

  And, and, and to give an indication of like how it was known, I think this conjecture was based basically only by doing numerics. Extremely clever numerics, but numeric. And so yeah, you have to kind of figure out why this is the right thing to aim for, and it does. And that was pretty remarkable. Yeah, I mean, I, I had actually thought about this problem for about six months at some point when I was a graduate student. And then, yeah, I just, I remember making like absolutely zero progress on it. So it was very nice to be like, explained why it was, yeah, why it was true. So this was, that was a pleasant experience. I think also in general, it was, it was one of these solutions which I, I knew several people had tried the problem. It's pretty remarkable because like the, the model solution, especially for this being like the LP can't do better than this was like quite short.

• 28:05Speaker 00

  It's a few pages of like complex analysis, but it's kind of exactly the right approach. Like, once you see it, it's kind of, it's like unbelievable, like why, why hasn't somebody done this before? It was, it was like, there are many types of good mathematics, but I think one of them is just like, you see it and you're like, oh, man, why didn't I think of this? And it was really fun and I, I mean, I sort of knew why I didn't think of it, but it was quite nice to see it and it was fun to see. That's why I like this a lot.

• 28:30Speaker 03

  Yeah, so, so this is the first of the 10 problems that Astra solved. But the second is actually closely related. So this was sphere packing. The second one is spherical and binary codes. So what's like

• 28:45Speaker 00

  You should draw a code.

• 28:46Speaker 03

  You drew a packing. It's, it's gonna be the same picture. Okay, sure.

• 28:52Speaker 01

  Otherwise, we're gonna have his picture, the hexagonal packing in our minds.

• 28:56Speaker 03

  Yeah, a spherical code is literally just a sphere packing, but on another sphere.

• 29:08Speaker 03

  Yeah, so I mean, a spherical code is basically just a sphere packing on the surface of another sphere. So.

• 29:18Speaker 01

  Yeah, it looks like a sphere.

• 29:22Speaker 03

  Okay, so same same picture as before, except you're kind of on a curved surface.

• 29:26Speaker 01

  Okay.

• 29:27Speaker 03

  Okay, so why is it called a code? Well,

• 29:32Speaker 03

  you can.

• 29:34Speaker 03

  I guess the the reason is because of binary codes, which is again the same sort of thing, but now it's on a cube. Okay, yeah, fine. Let me let me draw a picture of a

• 29:46Speaker 03

  cube and some like simplest possible code on it. So like when you're when you're like sending a so this is really like about error correcting codes.

• 30:01Speaker 03

  So so what are error correcting codes? So, you know, it's like I send you some string of bits, right?

• 30:12Speaker 03

  And maybe I'm like worried that some of the bits I send you get corrupted, right? So so maybe like just because of some errors in my system like this one gets gets changed. And we want some communication protocol so that like you can decode this like small amount of error and like recover what I was trying to tell you.

• 30:35Speaker 03

  And, you know, like, normal English language kind of has this sort of property, right? If I make a few typos, you know, you're going to be able to understand what I'm saying. But if I if we have some like really brittle communication scheme, it's not going to work. So

• 30:48Speaker 03

  codes are kind of the the way you you solve this. And mathematically, it just means like, you know, what's a binary string like this of a fixed length? It's like a point on some hypercube. And we want a dictionary of allowable code words that are like separated from each other. So like, in this case, if I don't want any two to be adjacent, I would kind of take these four vertices, kind of the like even ones, if you sum up the the digits, right? And like, okay, I guess, okay, in this case, I guess, if I if I have an error, you can't tell which one it's from. But at least you can tell it's like, not, at least you can tell there was an error.

• 31:27Speaker 01

  Oh, I see. I see. Yeah, because it's like kind of sparse in the yeah, it's like it's not too adjacent. So that like, this is like the one the hemming distance.

• 31:35Speaker 03

  Yeah, yeah, right, right. You want. Yeah. So you want like a large hemming distance between any distinct in your dictionary. And yeah, I guess if you take two opposite corners, then if I have like a single bit error, I can always like recover which point it was coming from.

• 31:50Speaker 01

  One that it's definitely closest to.

• 31:52Speaker 03

  Yeah. So there's there's kind of the, you know, same question in both of these cases, like in a very high dimensional setting, what kind of rate can you get? And like for binary codes, it's really like, you know, an extremely practical question. It's sort of like, if I send you like an end bit string, and there's like, you know, 1% error rate, like, how much longer does my message have to become to tolerate that amount of errors? Right? This is like some fundamental fundamental information theoretic limit of like, like, you know, send like communication.

• 32:25Speaker 03

  And, you know, but you can see, like, like, certainly, this this spherical case is like, it looks very much like sphere packing. For example, if you like, if you make all these little spheres really small, then like, the curvature of the big sphere is kind of not going to matter so much. And it looks like just packing spheres in full space. And in fact, yeah, like, these these problems turned out to be very related. So so for for these problems, there was like, there were similar bounds coming from the KL authors and like, there's like, something for the sphere and something for the cube, but it's all kind of the same stuff. And our models found

• 33:07Speaker 03

  better bounds for for these cases as well. And like, it, you, I mean, the the techniques look pretty different, actually, if you if you like rate them out. So so this this full space analysis of this linear programming was using like, just complex analysis. But if you like, the the method for for these cases, we're using representation theory. Like, like, both the sphere and the cube have a lot of symmetry. And basically, the the idea of the proof was to really leverage this symmetry. Like, there's some amount of this in the the previous, like, existing method. And and really, the improvement is to like lean into the representation theory, like, really hard

• 33:52Speaker 03

  and kind of make the the algebraic symmetry like enter in a more sophisticated way. And then it like, turns out that from the representation theory formulas, if you kind of take this like, like, small sphere limit in the spherical code case, you you recover like, part of this result, and you recover this this value. So like, this result isn't a special case, you kind of only went one direction of the bound from from looking at it from the code's point of view. But like, there's like a very close connection.

• 34:20Speaker 01

  Okay.

• 34:21Speaker 03

  Yeah.

• 34:21Speaker 01

  Yeah. So you guys let this like run in parallel. So it's like kind of discovering or because you're not sort of like feeding it.

• 34:28Speaker 03

  So so actually, this was the one case where there was some interactivity.

• 34:32Speaker 01

  Oh, interesting.

• 34:33Speaker 03

  So for for all of so except for this pair, it was just, you know, we had some problems, we fed them in, and we, you know, the model, the model came back with some solutions. What happened here is actually pretty interesting. So so we first asked it to improve the bounds for the codes. And it came back with an improvement that like, used some amount of representation theory. And then we kind of asked it, hey, can you like, push this further? Like, you know, what happens? And then it came back with some like, much more sophisticated representation theory. And I like, it turned out that you got this conjectured value for full space

• 35:15Speaker 03

  sphere packing, like, out of that method by pushing it as far as it can go. So then we we kind of asked to directly analyze this guy and try to complete the picture.

• 35:24Speaker 01

  Okay.

• 35:25Speaker 03

  Yeah. So the relationship like isn't a coincidence.

• 35:28Speaker 01

  Yeah, yeah. It's like interesting when you're saying the first prompt, which is, you know, maybe so basic, or just like, can you push this further? It does require some judgment from mathematicians. And but like, eventually, you would imagine by scaling the models, you don't need to do that. Or there's another view that the harness actually does matter. And this is kind of part of the harness apparatus. Do you guys have any views on that with your working with Astra, especially generations of models? And how how much you have to kind of input, or how much the harness matters versus not?

• 35:59Speaker 03

  I mean, I, yeah, I guess there have been some like, funny quirks like this that just come from like, exactly what you asked the model to do, basically. Like, in this case, what the model was asked to do originally for codes was to improve the bounds by like some exponential factor. So it really like, shows up in this like leading constant up here. And, you know, it improved the bounds, and it didn't try to push things too much further. Like, sometimes you see it do, but sometimes it just doesn't bother. But yeah, you know, you just ask it again, and it goes further. So it wasn't like a capabilities issue. It just didn't feel like it at the time.

• 36:35Speaker 01

  Do you call that judgment? Or like, what is the because there is a yeah, what do you call that?

• 36:41Speaker 03

  Models tend to be pretty task oriented. If you tell it to do a task, it accomplishes it's pretty happy.

• 36:48Speaker 01

  So yeah, the task orientedness, it's like, but do we expect that level to kind of ascend up to? It's not that they will be less good at being task oriented. It's like, they'll ascend to the level of like, okay, no, let's let's go in this direction. You'll have the judgment too. Because you guys had the judgment, too. Like, okay, this is pretty promising. Looks like you're using a lot of representation theory. It doesn't seem like there's a limit so far. But it doesn't have that context yet. But like, I guess what I'm trying to say is like, this one is hard to maybe harder to extrapolate. But from like, previous generations, when you had to give it more, maybe prompting more of that harness work, but eventually probably had to give it less. So it probably gives you some confidence that there's this like really fast ascension. And do you see? Yeah, like.

• 37:34Speaker 03

  Somehow solving a harder math problem is like, you have to solve many smaller, like, somewhat less hard math problems. And the fact that the math problems are getting harder is kind of an indication that you're, the model is able to take on more and more work in like a single continuous unit. And I think that's, that's the thing that looks very promising somehow. Like, any of these solutions, it's not like one idea, then you're kind of home free. You need several, you need several pieces to kind of interact and talk to each other. The models, I mean, the model doesn't come up with all the ideas at once, right? It doesn't pull everything out in in an instance of kind of the fact that it needs to sort of see how this piece interacts with another piece. That's kind of like solving a problem in itself, or piecing together many problems in itself.

• 38:17Speaker 01

  It could just be that, okay, when you're telling it, okay, push this even further, that was of the same order of like, magnitude as like, all the smaller things it's solving as well in between. And so you don't think that's kind of like a privileged direction. It's just sort of like, hey, let's give it like one one more help. Or you actually think that there's, I guess what I'm trying to get at a bigger question is like, is there a good sense of like,

• 38:39Speaker 01

  you know, taste? Because like, when people talk about, for instance, how well the models are getting at like, doing research, for instance, that's what we know a little bit of RSI. And and sort of like, there's surprising things about how that improves. And then there's like, oh, you know, maybe right now, it's a level of still like a junior researcher. It's like, not really asking like, the right problems. And so I'm just trying to get like a maybe a sense of like, where you're seeing that progress through the model advancements each generation. I mean, what is taste even?

• 39:15Speaker 00

  Yeah, I think I tend to be pretty utilitarian in my view of taste. And like, if you're able to solve problems faster by making better judgments, like, I think that's like, the best like, general proxy I have for taste. And somehow, the fact that solving harder problems means it has, kind of by definition, means it has better taste. I think there, there are these new, yeah, I think occasion, occasionally because they are task oriented, you do occasionally get these, these symptoms of like, oh, it clearly has made a breakthrough. It kind of understands it's made a breakthrough and then it doesn't kind of push all the way to the limit because that's not what you asked. But that seems, yeah, that seems, seems rather minor compared to like, the set, the state of progress we've seen so far.

• 39:56Speaker 01

  Mm hmm, mm hmm. Okay. Yeah, no, I think that's a pretty clear.

• 39:59Speaker 03

  Like, I think it's like, maybe you're liable to get confused if you're trying to like, do a concrete, long horizon task and show taste kind of at the same time. But like, you know, if you, if you have like, one model that's responsible for taste and one model that's responsible for going out and like, you know, working for a long time at solving a hard problem, kind of as the, as the like,

• 40:26Speaker 03

  you know, underling of the, of the supervising AI, I, I feel like that's kind of going to be fine currently.

• 40:35Speaker 01

  Oh, interesting, because that is like saying that they, these two things are somewhat sep-, if not separate, at least they shouldn't kind of pollute each other's context, which is

• 40:45Speaker 01

  a little bit, I mean, it could be potentially like a strong, stronger statement than, I guess, you know, it's just kind of interesting because it's, it, it might just be like, to your point, it's, you know, let's take the utilitarian answer, it's solving harder and harder problems. It's doing a lot more than just like, you know, brute forcing something, it's making choices. It's like pruning, you know, a vastly large space of possible paths into something that's like, really, you know, it's both tractable, but then ends up being like, it's a diminishingly small path within that space. But like, having, like, why would, would be like a separate model, a separate generation or something that's a different version of the model that would contribute to taste?

• 41:27Speaker 01

  Or maybe that's totally, like, it's too abstract, doesn't make any sense, you know? We should just let the actual, this might, like, the related question be like, you know, what is, what is the, the thing that gets us to a better version of intelligence, the harness and the model, is it just the model? And it's like, we, we see this in, you know, at least in applied AI or, you know, startups where it's like, it's a continual battle of like, you need the harness, but then the harness adapts very poorly to a new model. Because sometimes like, a very, very minimal harness is still the best way to expose to the raw power of the model. But then now we also have these like, training regimes where we require the harness to be, you know, trained with, I mean, part, part of this is to keep things more proprietary and harder to, harder for other people to use it.

• 42:10Speaker 01

  But I think partially it's, it's maybe actually that it helps have more control on like, the reasoning traces you care about. It's a long, rambling way of saying, it's like, yeah, I don't actually, like, I, I, this is so interesting to see how the models have gotten better at math. And maybe something that's like, very abstract and hard to describe, like taste, is a way to tease out like, what is actually necessary here.

• 42:34Speaker 00

  I think my only like, non-trivial thought here is that, like, when you're working, I mean, just when you're doing any task, occasionally you get pigeonholed and you like, work really hard. And just having a friend look over your shoulder and be like, what are you doing? And then just like, just having that one bit of like, step back for 10 seconds, like, this is often very useful.

• 42:51Speaker 01

  Yeah, yeah.

• 42:51Speaker 00

  I mean, I, I see no reason why humans would be so different than models somehow.

• 42:55Speaker 01

  Yep, yep.

• 42:56Speaker 00

  Having, or models would be so different than humans. Having, having a few humans working together is often more powerful than just having one.

• 43:03Speaker 01

  Yeah, it's like, in this kind of collaborative thing, you actually, you kind of, yeah, artificially created it, but it's very similar and dynamic.

• 43:10Speaker 03

  But I, I think a lot of taste is also like, having a sense of what problems you or like, some method you have in mind are going to be good at solving.

• 43:18Speaker 01

  Mm hmm.

• 43:19Speaker 03

  Like, it's, I mean, certainly there's some amount of like, absolute aesthetic point, right? But there, there's also just like, you know, having a nose for what, what you might want to pursue because you'll be able to make progress. And, you know, I, I think for that, like, there's, you know, you, you would expect that as a side product of being good at completing tasks, you would, you would get there, sort of, right?

• 43:45Speaker 01

  Let me know if we still want to do like, a section on Soffic groups? Because I think, you know, up to you guys, it's definitely a super interesting.

• 43:51Speaker 03

  So maybe the, the first question is, what is a group? Let's remind ourselves. So, a group is

• 44:03Speaker 03

  a set of elements with some multiplication operation.

• 44:11Speaker 03

  And basically, this is how mathematicians think about symmetry.

• 44:22Speaker 03

  So, so you're like, basically like, if G and H are elements of your group, then GH

• 44:33Speaker 03

  has some, is some other well-defined

• 44:37Speaker 03

  element of your group, and you have like, associativity.

• 44:46Speaker 03

  And you have an inverse. So for every G, there's some inverse.

• 44:54Speaker 03

  And there's some like, specific element in the group that is, is kind of the identity. Okay, so, you know, it's, it's some like abstraction of like, composing operations. So these could be like, numbers, they could be like, multiplying matrices, they could be like, like, rotating something, which is, you know, a special case of multiplying matrices.

• 45:16Speaker 03

  And a group is Soffic.

• 45:23Speaker 03

  If, well, there's some, you know, precise definition, but, you know, roughly, it means it, so, so I should say, like, groups that can be finite or infinite. So, like, you know,

• 45:43Speaker 03

  if, if you have like, a square, like, all the rotations of it form a group with like, four elements. If you have like, a circle, then the rotations form a group with like, uncountably many, many elements. And so Soffic groups are either finite or countable. You should think of them as being countably infinite, so there's like, the same number of elements as like, integers. And if it's Soffic, if in some sense, it can be approximated by finite groups. So we didn't know if there was a non-Soffic group. So, so the, the, the result that Astra proved is simply that there exists a non-Soffic.

• 46:25Speaker 01

  Yeah, and without like, I mean, we can, you know, before going to, to that proof, it is like, you know, it's like, I feel like a lot of the programs in math is like, okay, we are such finite creatures. Let's see how well our finite approximations are, you know, do. And in this case, especially for the countable case, it's like, maybe you'll, you'll be relating it to like, the Aldous Leons thing. It's just like, it helps kind of anchor the picture of like, it seems like such a, I mean, it's a nice result if it were true, but it's not. And it seems almost like reasonable. And so, yeah, I, I actually didn't, didn't go, I would love to hear the explanation of like, how it found a counter example.

• 47:05Speaker 03

  Yeah, I mean, I, I would say that like, you know, the, the hope that there was no non-Soffic group, so every group has this kind of approximation? Like, maybe this is sort of like, people hoping that there's a miracle. Because it turns out that groups like this have a lot of nice properties. Because you can run certain proofs for finite groups and then, you know, kind of approximate them in whatever way the definition of being Soffic lets you approximate them and get the result. So, so like, there's this notion of being a surjunctive group. So there's, there's some fact that any group which is Soffic is also surjunctive.

• 47:52Speaker 03

  Surjunctive is some property of like, dynamical systems on the group. And I guess the original question was whether every group is surjunctive. This is some question of Gottschalk from the 70s. And this, this fact that follows this like, pattern of prove it for finite groups and then do this approximation is, is what motivated the question about if there's a non-Soffic group.

• 48:20Speaker 03

  Yeah, maybe I'll say a little bit about this Aldous Lyons conjecture and.

• 48:24Speaker 01

  Yeah, sure. Yeah, yeah.

• 48:25Speaker 03

  So I, I guess I had like, heard of this a little bit beforehand because there's a related stronger conjecture in probability that was made popular by Aldous and Lyons. This, this conjecture, roughly what it says is like, any

• 48:44Speaker 03

  like, infinite graph with some nice property called, you know, modularity.

• 48:58Speaker 03

  A modular random graph

• 49:05Speaker 03

  can be approximated by large finite graphs.

• 49:14Speaker 03

  So maybe the way to like, explain what these kinds of things are trying to say without getting into technical weeds is to say what they mean about the integers. So, so like, how would I draw the integers as a graph?

• 49:31Speaker 03

  So this is called the Keilig graph. You're just gonna connect nearest neighbors. Okay, so there's some kind of canonical way in which this is like the graph that represents the integers. Okay, and

• 49:45Speaker 03

  there is some sense in which you can approximate this by finite graphs. Why? Well, if you look at integers mod n, then you kind of get the same picture, but

• 50:02Speaker 03

  like, you have like a big circle instead of infinite line.

• 50:07Speaker 03

  And the point is, if you like, look at any point here and any point here, like, nearby, things look the same. You have to go like very far away to kind of see this global geometric structure that you have a circle and not a line. And in fact, the integers and integers mod n are both groups, just by like adding numbers or adding numbers mod n. So these integers mod n are like Soffic approximations for the itful integers. So like this approximation is kind of why the integers are a Soffic group. So

• 50:43Speaker 03

  so the sophistication, the statement that every group is Soffic is sort of a generalization of the fact that you can do this approximation with groups. And this Aldous Sline's conjecture is kind of a broader conjecture that like any network, you can do this.

• 51:01Speaker 03

  And like, you don't require as much algebraic structure roughly. So it's kind of a broader conjecture. So, so this conjecture was disproved earlier, like two years ago. And it was kind of a really toe-to-force work. Like, it was like 250 pages, building on another 200 pages. It uses like, like quantum complexity theory. So it really builds this like, you know, very complicated bridge and like, you know, I, I think not very many people could understand this. Right, so since this is a stronger conjecture, the disprove like is weaker than disproving this statement that all groups are Soffic. But it turns out that the direct proof that all, that there's a non-Soffic group was like

• 51:44Speaker 03

  much shorter and easier than this, than this really amazing disprove of Aldous Sline's conjecture. It's like, like 15 pages, maybe. And it doesn't have any of this like, very complicated connection with quantum complexity. It just kind of stays in group theory land. I mean, it uses some important, like, you know, existing results by other mathematicians like Kuhn and Kuhn and Tom, but it's like, it's like, it's like a very reasonable, normal kind of proof.

• 52:12Speaker 01

  Yeah, and it's kind of spelling maybe out the obvious, but like, the connection between the Soffic group statement is just you take the Keilig graph and that's the one that is like what they use or for Aldous Sline?

• 52:25Speaker 03

  Yeah.

• 52:26Speaker 01

  Yeah, and so that's why it's like a, you know, a subset of.

• 52:29Speaker 03

  Right, so yeah, basically what happens is, yeah, so for a group, you can take exactly a Keilig graph. So you, you take some like, elements that like generate the group and you kind of connect elements that are adjacent. So in this case, like, this is a Keilig graph of the integers. So, so right, so when you do that, from a group, you get like a deterministic graph. Right, you just get like a single graph, you fix some set of generators. So this conjecture is stronger, basically because it allows a broader set of graphs that aren't deterministic. It allows them to be random, but have some extra, you know, you know, modularity property that constrains exactly how it can be random. But yeah, basically, that's, that's the difference.

• 53:12Speaker 03

  Like, here, you kind of have to give a deterministic network instead of a random one.

• 53:18Speaker 01

  Yeah, anything kind of interesting, surprising about the results? I mean, you mentioned some things, which is like, it's stayed within group theory, the techniques.

• 53:30Speaker 03

  I mean, I, I think maybe it's like a, a nice example of this general pattern that

• 53:37Speaker 03

  theorems produced by AI have generally been like, like the proofs are pretty short generally. They're like,

• 53:45Speaker 01

  Like with a counter example so far?

• 53:47Speaker 03

  Yeah, but this one, it's like, okay, it's sort of a counter example, but like, there's some, you know, there's some like, stuff you have to do to analyze.

• 53:56Speaker 00

  The difficult part here is that like, the property of being a Soffic group is not so easy to get your hands on. So you have to find like, a concrete way of saying like, like producing a, producing a way of saying, this group cannot be Soffic. And like, and the proof is actually, it's very short. It's like, it's almost, it's a combinatorics argument, but it's like a very, very delicate combinatorics. The model, like, somehow you need to both have the right statement and know what piece in the literature and then execute it correctly. And that's very nice. Like, the difficulty in this problem is that like, it's just a really, really hard, it's like, very hard to get your hands on like, being approximated by any possible finite group.

• 54:33Speaker 01

  It's like, what is happening at that countable infinity that's like resisting this approximation? Like, do you guys kind of, did it give a sense of, like, when you do postmortem and you're like, okay, Astra, explain to me, like, what is the?

• 54:44Speaker 03

  Oh, yeah, we did that.

• 54:46Speaker 01

  What was a good explanation you got out of it?

• 54:49Speaker 00

  I think there's like some concrete like combinatorial obstruction. Basically, it's like, it's hard to explain, but there's like some concrete combinatorics obstruction, which if you read the previous papers, you realize that that's what they couldn't rule out. And Astra found a way to kind of say, okay, no, no, if you add this one extra algebraic fact, this, this like weird conspiracy can't happen. It's like very clearly trying to rule out a conspiracy in the sort of previous authors had implicitly written about.

• 55:17Speaker 01

  And those were the actual suspects. It turned out, so they were sort of on the right track and then this did the last mile of, well, whatever, however you quantify that.

• 55:29Speaker 03

  But I think it's like, like a year ago, I would have been very surprised to learn that like, all of these AI proofs are like very short and elegant. Like, they're, you know, you, you're kind of like afraid that they're going to like, generate all these thousand-page things.

• 55:46Speaker 03

  Understand it, but it's been kind of the opposite.

• 55:48Speaker 01

  Yeah.

• 55:48Speaker 03

  Like, only humans can generate like 200-page proofs right now.

• 55:52Speaker 01

  Yeah. Well, and also I was like asking, like, if you do that postmortem, it ends up usually engendering more mathematics. Because when you do that with humans, like, that's what, you know, breeds new mathematics. So maybe, maybe if you kind of alter the prompt a little bit and be like, how would you, you know, generalize this or something like? Yeah, I don't know if that's been a technique for you guys to like have it explore and exploit what it has already developed.

• 56:17Speaker 03

  Well, there has been some, there has been follow-up on this already, actually, by, by Kuhn and Tom, who this result was building on. So they like.

• 56:24Speaker 01

  The math community is coming on.

• 56:25Speaker 03

  Yeah, yeah, yeah, which is kind of what, what we're hoping. You know, we don't want to be, you know, writing lots of follow-up papers ourselves, but if, if there's some interesting follow-up that, you know, it's like, we're very, very happy that, that there's some, there's some follow-up, building out these ideas more and giving like more examples of non-Soffic groups in this case.

• 56:43Speaker 01

  Yeah, well, actually, maybe that's a great segue into like, how, you know, what's the ideal way that this is being taken out by the math community? Because I feel like there's a spectrum of answers from working mathematicians, sense of like, some, you know, probably most at this point are like, okay, AI is obviously doing some non-trivial stuff. It would be a disadvantage not to admit that in my workflow.

• 57:06Speaker 01

  I've definitely heard some stories where people are, you know,

• 57:11Speaker 01

  would find it hard to either take AI as a co-author or like, how do you even do kind of attribution this way? But I don't know, like, what, maybe to paint the, the more optimistic picture, so you're saying you want the mathematicians to be building on these results? It definitely generates a lot more results to be verified, so it, you know, it puts pressure on, on the community and the profession. Like, how do you, how do you kind of expect the evolution of kind of uptake and, and collaboration with mathematicians?

• 57:39Speaker 00

  I mean, the, given that the fact that the models can produce sophisticated mathematics means that they can help you understand like sophisticated mathematics. I mean, like, I don't know, occasionally, like, I enjoy looking at the archive and I want to understand some proof. And like, I could read the introduction, but in practice, it's just much faster, take the PDF, put it into, put it into my favorite model and then like, get a, get an output of like, what is the rough proof strategy? And somehow this like, along, I mean, of course, models are going to help us produce exponentially more mathematics, but they'll also make it much easier to absorb it. And right now, okay, it's still a bit of a challenge back and forth, but I, I think it's, for me at least, a much, much faster understanding.

• 58:21Speaker 00

  It's much, much faster to understand a piece of mathematics with a model than without it. So it's helping solve the problem it creates anyways.

• 58:29Speaker 01

  Yeah, I feel like that at least, and it's, you know, I, I don't, I don't view it as like creating much more harm. But again, like, I don't have such, you know, high stakes in like, okay, I'm gonna get, I'm not gonna get tenure, etc. So like, I, I agree, like making it more accessible, like, if I'm not spending so much time absorbing an area, I can like, put it into chat GPT and then expect to, I mean, you guys have an even more powerful model, hopefully releasing for other people to enjoy as well, but like, it's. I think like the positive version of that is actually more people can participate in mathematics. It's like people might be coming with other intuitions and they could actually maybe generate good mathematics. Is that sort of like closer to the vision of what you're hoping this is, you know, pushing towards? Or like, what things do

• 59:14Speaker 01

  you think mathematicians should be wary of to kind of adapt fast enough to take advantage of AI?

• 59:20Speaker 03

  Yeah, I mean, I think certainly there will be a lot of changes, right? Like, I guess in math, like, there are, there are a lot of things that are kind of important for, for like a given result, right? You need someone to come up with it, but you also need people to understand and absorb it and like, you know, internalize it enough to do more with it and like figure out where it fits into like humanity's understanding, right? And like, a couple of years ago, like, the proving the result was like, so hard that kind of the other stuff was just kind of coming along for the ride, right? You know, like, if you, if you manage to like prove this thing yourself, you're automatically

• 1:00:04Speaker 03

  going to understand it quite well. You're kind of responsible for like maintaining it and in some sense and like, you know, explaining it to other people. And yeah, now this kind of, what was the main bottleneck before is kind of much less of a bottleneck. And, you know, these other kind of constraints come into play. So it, yeah, the like optimal structuring for, you know, organizing the knowledge could, could look rather different.

• 1:00:37Speaker 01

  Yeah. How does that look? I mean, does this make the field a lot more kind of empirical? Will people do sort of the hard, like the first thing that was scarce, which is like all the reasoning and then more? I mean, not that it's like a bad thing to make it empirical, but it's almost like it functions as a very different discipline. Like, a lot of the fun stuff is understanding, you know? And so, going to understand and communicating, maybe assembling, having still the human taste, does that sort of remain rarefied? And, and that's how, you know, current mathematicians need to adapt and reward, you know, contributions? Or, or is this too much of a caricature? It's like something else?

• 1:01:16Speaker 03

  I think certainly understanding how to put, as we get more and more mathematics, put it in like a proper framework and sort of how, sort of like being able to explain it to other humans so that they can also appreciate it. I mean, so implicitly we valued this, but it was usually because you were the person proving the result that gave everybody else the understanding. But I think increasingly it would be a function of like, you're sort of helping, you're the human who can sort of give this understanding to other people and sort of help them with it. I think that more, that communal understanding will, I think, become, it was much more implicit in how we viewed mathematics, but I think it will be an increasingly more explicit and valuable part of the subject. I mean, a nice thing about math is that the, the ceiling for difficulty of a math

• 1:01:59Speaker 03

  problem is pretty high. So even if, you know, kind of, even if AI get, you know, continues getting like exponentially better at math, like it might, you know, plausible will never solve something like P versus NP. And it could be that, like, the, the field kind of becomes more, you know, attached to like, like these big mysteries and less to like smaller mysteries that are more like routine now.

• 1:02:32Speaker 01

  Yeah, yeah. I think that's a positive vision of that.

• 1:02:35Speaker 03

  I mean, also, like, I don't know, there are things I spent like months or years of my life wondering about, not getting to know, and hopefully I'll get to-

• 1:02:41Speaker 01

  And now we get that. Yeah, that's like, what a joy.

• 1:02:44Speaker 03

  Yeah, some portion of them I'll get to know the answer to. I'm pretty happy about that.

• 1:02:46Speaker 01

  No, exactly. No, I'm, I'm excited about this, like renaissance of results and understanding. And I feel like, I mean, this is such an infinite, you know, field, like, okay, no pun intended, but like, it's just like, it's, it's, it's just, there's so much that you can actually create here. So, I mean, especially for somebody like me who's not going to have the time to actually like practice mathematics, now there's like a lot more that you can actually do in the activity of math. So, yeah.

• 1:03:13Speaker 03

  Yeah, I think the, the like, the ability of someone who's not working on math is like their literal job all the time. So like, understand what's going on and like, you know, learn about some of the mysteries they might have wondered about will, will go up quite a lot. Also, you know, if you're, if you're like, if you're working on something that requires some math, you know, suddenly you, you don't need to like find a world expert on this topic to, to be able to, you know, use it in your own work.

• 1:03:40Speaker 01

  Sorry mathematicians. Yeah, no, it's true. I mean, I think there was just like a dearth of actual, like, people who could do that. And so I think this is helpful. Maybe it's helpful for theoretical physics, but we'll see. But a lot of other applied areas as well.

• 1:03:54Speaker 03

  It'd be nice for the world if applied mathematics went a lot faster.

• 1:03:57Speaker 01

  Yes. I mean, I'm of that opinion. Well, thank you guys for joining. This was a lot of fun. And I'm, you know, just so excited for how much the models are advancing. So maybe we'll have you guys back soon.

• 1:04:10Speaker 03

  Yeah, thanks so much for having us. Yeah, thanks for having us.

• 1:04:14Erik Torenberg?

  Thanks for listening to this episode of the A16Z Podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at A16Z.substack.com. Thanks again for listening, and I'll see you in the next episode.

• 1:04:40Erik Torenberg?

  As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com/disclosures.