#503 – AI Crisis in Math: Navier-Stokes, Riemann Hypothesis & Complexity Theory

Ankur Moitra, Steven Brunton and Alex Kontorovich are three mathematicians who can speak to this incredible moment in the history of mathematics, when AI solved some of its hardest problems: Navier-Stokes, the quasi-Riemann hypothesis, the unique games conjecture and hundreds of other problems. Ankur Moitra is a mathematician at MIT (visiting the Institute for Advanced Study in Princeton) working in theoretical computer science, who can speak to the unique games conjecture, complexity theory and the mathematics of how AI learns. Steven Brunton is a mathematician and fluid dynamics expert at UW, who can speak to Navier-Stokes, turbulence and machine learning for fluid dynamics. Alex Kontorovich is a mathematician at Rutgers specializing in number theory, who can speak to the Riemann hypothesis, Lean and machine-checked proofs. Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep503-sc See below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc. Transcript: https://lexfridman.com/math-ai-crisis-transcript CONTACT LEX: Feedback – give feedback to Lex: https://lexfridman.com/survey AMA – submit questions, videos or call-in: https://lexfridman.com/ama Hiring – join our team: https://lexfridman.com/hiring Other – other ways to get in touch: https://lexfridman.com/contact EPISODE LINKS: Ankur’s Website: https://people.csail.mit.edu/moitra/ Alex’s X: https://x.com/AlexKontorovich Alex’s Website: https://sites.math.rutgers.edu/~alexk/ Steven’s X: https://x.com/eigensteve Steven’s YouTube: https://youtube.com/@Eigensteve Steven’s Website: https://eigensteve.com Lean: https://lean-lang.org/ SPONSORS: To support this podcast, check out our sponsors & get discounts: Wispr Flow: AI-powered voice dictation app. Go to https://wisprflow.ai/lex https://lexfridman.com/s/wispr_flow-ep503-sc Fin: AI agent for customer service. Go to https://fin.ai/lex https://lexfridman.com/s/fin-ep503-sc LMNT: Zero-sugar electrolyte drink mix. Go to https://drinkLMNT.com/lex https://lexfridman.com/s/lmnt-ep503-sc Blitzy: AI agent for large enterprise codebases. Go to https://blitzy.com/lex https://lexfridman.com/s/blitzy-ep503-sc Shopify: Sell stuff online. Go to https://shopify.com/lex https://lexfridman.com/s/shopify-ep503-sc NetSuite: Business management software. Go to https://netsuite.ai/lex https://lexfridman.com/s/netsuite-ep503-sc Perplexity: AI-powered answer engine. Go to https://perplexity.ai/ https://lexfridman.com/s/perplexity-ep503-sc OUTLINE: (00:00) – Introduction (01:58) – Sponsors, Comments, and Reflections (09:53) – Crisis moment in mathematics (18:10) – Will AI “solve” mathematics? (34:21) – Can AI create new math? (1:13:53) – Riemann hypothesis (1:37:45) – Navier-Stokes (2:03:42) – How nature computes (2:12:35) – Unique Games Conjecture and complexity theory (2:27:28) – How mathematicians think (2:41:05) – Is math invented or discovered? (2:47:13) – Can AI solve P vs NP? (2:57:28) – Why LLMs work so well (3:11:54) – Problems AI can’t solve yet (3:16:23) – Grigori Perelman (3:25:41) – Advice for young people

• 0:00Speaker 00

  All right, so this is a kind of emergency episode, I think it's fair to say, about the state of mathematics and artificial intelligence, a kind of crisis moment in the history of mathematics. Over the last few weeks, machines claimed to have solved some of the hardest problems in the history of math, Navier-Stokes, the quasi-Riemann hypothesis, the unique games conjecture, and hundreds of other results. So here with me, we have Stephen Brunton, who's a mathematician, fluid dynamics expert at University of Washington, who can speak to Navier-Stokes, turbulence, all things fluid and many other topics.

• 0:43Speaker 00

  Also, here is Ankur Moitra, mathematician and theoretical computer scientist at MIT, visiting Institute for Advanced Studies in Princeton this year, who can speak to many things, unique games, conjecture, and complexity theory in general, and the mathematics of how the AI learns, and all things theoretical computer science. And Alex, a mathematician, number theorist at Rutgers, who can speak to also very many topics, including Riemann hypothesis, and Lean, the automated proof and automated verifier and checking and so on, and so many other topics in mathematics.

• 1:25Speaker 00

  All three are brilliant, all are very kind to show up this early morning and talk to me, assemble this kind of multi-agent mini-swarm of human brains to try to interpret what happened in this very moment. A lot of mathematicians, brilliant mathematicians, including Terrence Tao and others, are freaking out, some mix of awe, terror, dread, and a few inklings of light of hope also. And now, a quick few-second mention of our sponsors. Check them out in the description or at lexfridman.com/sponsors. It is the best way to support this podcast.

• 2:09Speaker 00

  We've got Whisperflow for voice-to-text writing, Finn for customer service AI agents, Element for electrolytes, Blitzy for code generation in large code bases, Shopify for selling stuff online, NetSuite for business management software, and of course, Perplexity for curiosity-driven knowledge exploration. Choose wisely, my friends. And now, onto the full ad reads. I try to make them interesting, but if you skip, please still check out our sponsors. I enjoy their stuff, maybe you will too. To get in touch with me, for whatever reason, go to lexfridman.com/contact. All right, let's go. This episode is brought to you by Whisperflow,

• 2:53Speaker 00

  the AI voice dictation app that lets you write by speaking. And at this point, I speak to AI maybe in order of magnitude more than I type to AI, which is a crazy thing to say, me who's anxious about social interaction. I never thought I would find comfort in voice as the interface to AI systems, but I do. And it's fascinating. I love the keyboard, I've loved the keyboard for many decades, but we must say goodbye to the past and say hello to the future. And one of those tough goodbyes has been the Kinesis keyboard for me, and the keyboard in general. I'm typing less and less and less these days,

• 3:38Speaker 00

  but playing more guitar and playing more piano. A different kind of keyboard. Anyway, if you want to try to use voice as the input layer for AI tools, this is the way to go. Go to wisprflow.ai/lex to get one month of Whisperflow Pro free. That's W I S P R flow dot AI slash lex. This episode is also brought to you by Finn, the number one agent for customer service. 65% average resolution rate. It's trusted by over 6,000 customer service leaders at top companies, including many of the top AI companies. You know, when an AI company trusts another AI company to do a thing, that means they're legit.

• 4:22Speaker 00

  Finn has found a niche, an incredibly valuable niche, and they do it extremely well. There's some wisdom in that about how to succeed in this insane old world of AI systems everywhere. Find a niche that brings a lot of value to a large number of people and get really damn good at that niche. Until frankly, the niche swallows the world, because we live in a time where it's just a bunch of niches. And you realize there's so many people on the globe that the niches are quite large. They can fill stadiums, they can fill nations. Anyway, go to fin.ai/lex to learn more

• 5:04Speaker 00

  about transforming your customer service and scaling your support team. That's fin.ai/lex. This episode is also brought to you by Element, my daily zero-sugar and delicious electrolyte mix. I'm sipping on it right now, watermelon salt, delicious. All the crazy stuff I do, I've been basically exclusively eating only once a day, working insane hours, going through all kinds of crazy whirlwind rollercoaster journeys inside the mind, trying to comprehend everything that's happening in the world.

• 5:45Speaker 00

  What a difficult episode this is on mathematics. Mathematics has been at the core of many of the things I admire, but the limits of what the human mind is capable of, and here we are, breaking through those limits at a rate that nobody could have anticipated. And it's just a lot, it's so much to deal with, it's so much to deal with for mathematicians, it's so much to deal with for other people, for thousands, for maybe millions of people scared about the future of the world. I'm with you. All right, get a free ACOM sample pack with any purchase. Try it at drinkelement.com/lex. Speaking of a rapidly changing world,

• 6:29Speaker 00

  this episode is brought to you by Blitzy, an AI-powered autonomous software development platform. It's billed for large, complex code bases. If you're one of them vibe coders, who, open clause, Peter says is a slur, vibe coding. A lot of the people that use AI agents to program are really working on small code bases, and often actually code bases that don't interact with customers, with real people. And if you're working on code bases that serve a large number of people, and those code bases are large, there's a whole set of tooling and systems and concerns you need to be aware of.

• 7:13Speaker 00

  And that's what Blitzy helps you with. The future of autonomous software development is here. Learn more or speak to a member of the team at blitzy.com/lex. That's blitzy.com/lex. This episode is brought to you by Shopify, a platform designed for anyone to sell anywhere. Why does it fill me with so much joy when I mention Shopify? I think about Shopify engineering, it fills me with joy. What am I gonna do? Anyway, it's a great way to build a great-looking online store. Toby is the man. He's the CEO of Shopify, brilliant guy, brilliant engineer, brilliant leader of engineers. And it's always nice when a guy like that,

• 7:57Speaker 00

  who's at the cutting edge, who pushes the company to be at the cutting edge, is running a company as impactful Shopify, and it is. So if you want to be connected with such an impactful company, sign up for a $1 per month trial period at Shopify.com/lex. That's all lowercase. Go to Shopify.com/lex to take your business to the next level today. This episode is also brought to you by NetSuite, all-in-one AI-powered business management system. Their NetSuite Next system automatically services insights throughout your day. Their AI agents work alongside you to complete various tasks.

• 8:41Speaker 00

  It's a great integrated system that looks at all the different modules within a company, you know, managing of financials, all the HR stuff, inventory and supply, e-commerce, all of that kind of stuff, all the mess, the beautiful, beautiful mess that makes a company. All of those are different modules, hierarchical modules. All of them need to be connected and managed and organized. And the utilization of AI within that whole system is essential. And NetSuite does a great job of that for the first time ever. You can try NetSuite for free. Go to netsuite.ai/lex. That's netsuite.ai/lex.

• 9:26Speaker 00

  This is the Lex Fridman Podcast. And now, dear friends, back to this fascinating conversation about mathematics.

• 9:53Speaker 00

  So I apologize to start with a question about feeling, but what have you been feeling about this historic moment? Maybe Alex, do you want to go first?

• 10:04Speaker 01

  Yeah, I don't know if I would call it a crisis. There's a lot of opportunity in that crisis. It's a it's a chance to rethink what it is we're trying to accomplish. And it's the chaos of opportunity that some people are reacting negatively to. Some people are reacting positively to, and sometimes those reactions are happening in the same person. And so I think we're entering, if we're not already, in a period of some kind of golden age of mathematical knowledge, after millennia of having a candle that we are walking around the woods, and this is as far as we can see, now we have a flashlight.

• 10:42Speaker 00

  I think we should comment on the magnitude of this moment for people who are not aware. So maybe to quote you, something like, something like, holy shit, if a human did this, it'd be an instant Fields Medal. Maybe you didn't say holy shit, but basically saying, this is an incredible result. So a lot of people wouldn't have expected this amount of progress in this amount of time. So this is truly shocking. So you're now speaking as a person who is dealing with the feelings you felt over the last couple of days, having found this out. So maybe also it'd be useful to go back to the the feeling you were having when you first saw the report that, in this case,

• 11:22Speaker 00

  OpenAI dropped. And the two of you happened to be on camera loosely on the same day, maybe a few drinks in because it was a conference or a get together. Maybe, Uncle, can you speak to what you were feeling?

• 11:37Speaker 02

  It was an interesting night when the results dropped. Now, it was not entirely unexpected because for weeks there'd been rumors about the two biggest problems in my areas that were in that that collection of results. So the unique games conjecture and RL equals L. There were even rumors about matrix multiplication. So really, my first reaction was, wow, it's all true. Thank goodness it's finally out there because now people can actually go through it and process what's happened and figure out what to do next. Now, as to your earlier question, you know, what are we feeling? Well, for me, it changes on a daily basis, but I think the biggest, most overarching feeling is really feeling like you're getting pulled in two very different directions.

• 12:21Speaker 02

  My work has always been about asking the right questions. And to that extent, these tools are amazing for me because, you know, they increase my ability to learn about other fields. It's like having a world expert who's there to teach you about things in your own language. Your ability to make connections is just magnified. But, you know, one of the parts of my job that I really love the most is the mentorship aspect, not just training grad students and postdocs and seeing them do great things, but having them come with me on these journeys. So the part that pulls me in the other direction is the recognition that, you know, the playbook and the blueprint for success that that we had for so long is now thrown out the window. So it becomes a lot harder to advise students

• 13:06Speaker 02

  and postdocs about, you know, what are things they should be working on that are going to make an impact, are going to make a name for themselves. So I feel very excited, but maybe more intellectually lonely than I felt before, in part because, you know, a lot of times, great results come from asking very humble questions, at least for me. And now there's so much anxiety and dread and feeling that, wow, you would have to do something really momentous to stand out in the sea of AI results, that it feels like we've lost some of that patience in terms of planting the seeds from which the great oak trees grow.

• 13:46Speaker 00

  Terrence Tao, Feels Not a Winner, wrote, "I am horrified by the many person years of ongoing patient and deliberately slow research efforts, particularly by graduate students and postdocs, toward many motivating problems in mathematics being casually disrupted or destroyed by such a release. Much as one cannot unhear a movie spoiler or a crossword clue, one cannot explore a problem as profitably and richly once one is aware of an existing solution. And I mourn the path not taken and the opportunities lost in the frantic race to develop the technology." So, I mean, if we're being realistic, it's more dramatic for a PhD student and a postdoc

• 14:31Speaker 00

  than than maybe the way you've articulated. I mean, you're saying it quite calmly, but if I was a PhD student right now, I'd be in a state of like shock.

• 14:47Speaker 00

  Like my whole life, I was going through some of this. I know a lot of my friends are programmers. They're going through this state of shock in last December, when basically, most of the programming world switched to AI writing most of the programs. So I would approximate it to a feeling of kind of a loss of meaning, kind of terror, like dark stuff. We'll keep exploring that. Steve, what do you think? What are you feeling?

• 15:11Speaker 03

  I would say most of my feelings are pretty hopeful. I think I'm an engineer. I like solving, you know, problems that make technology more realistic for humans, artificial hearts and airplanes. And our ability to solve real applied problems is going to get much, much better. Our scope and reach is going to increase. And so for that, I'm extremely hopeful. But there is a lot of uncertainty. And I think, you know, some of this fear of meaning, of losing control, of, you know, just being kind of cut out of the process that we all love being a part of, it is a little scary. And I don't think any of us would be surprised

• 15:55Speaker 03

  at either of the kind of two paths that might might take place over the next five or 10 years, where we really are just dwarfed by, you know, other intelligence, or it kind of levels off and this becomes an incredible tool that we use for our curiosity, for our human problem solving. And that uncertainty is massive. Those are two very different worlds. And so uncertainty is is difficult for humans. I think it's also difficult for those of us who have chosen math. We chose math, you know, probably because we liked rules and formalism and, you know, uncertainty is uncomfortable. I think the humility aspect is really important, though. This is a really good opportunity for us to remember why we're

• 16:40Speaker 03

  doing this. Like, what is human purpose? It's not to be famous. It's not to have a Fields Medal, you know, solving interesting problems, helping other people, being curious, just wanting to understand the world. Those are really good things to kind of get back. Like, what do we need to be satisfied humans? And I think it's a really interesting moment for us to think about. Like, I've been thinking about it a lot lately.

• 17:06Speaker 03

  It is more difficult for young people, right? Like figuring out how to to find their way. But I actually have more faith in them than than us, in a sense.

• 17:16Speaker 00

  Do you think there's something to what Terry says about it's harder to be motivated when you know a problem has been solved?

• 17:25Speaker 00

  To explore that space when a problem has been solved like this by an AI system?

• 17:32Speaker 03

  Yes and no, in the sense that I do very much agree that the path is as important as the destination. Like, if you go on a road trip, the road trip is, you know, is usually just as fun as as getting there. At the same time, if we found out that another country had invented teleportation or time travel, I guarantee we would reinvent it in the next five years. Like, if you knew it was possible and you could reverse engineer a little bit, we would be curious, we would dive into it, and we would figure it out. So knowing there's a solution also is an incredible motivation to get there much faster.

• 18:10Speaker 00

  Let me ask the question this way. If this moment marks the moment of the trajectory where AI basically, quote unquote, solves math or kills math, can you make the case that that is in fact a trajectory? And then can you make the case that that is not a trajectory? Can you steelman both paths? That this is a path towards which, like, mathematics, as we know it, is done completely. So that's going to be a thing done by machines. And then what humans do is something fundamentally different? Or there still will be?

• 18:51Speaker 00

  Something that looks similar to our current mathematics. Can you make a steel man both cases, Alex?

• 18:58Speaker 01

  Yeah, so you said two things and I'd like to separate them. One is solved math and the other is destroy math. So let's make that precise. What does solved math mean? If you think back to when chess engines first came out and people were like, oh, this is going to be amazing. We'll play centaur games. So this is a human being plus a computer system against another human being and a computer system. And that lasted like 10 seconds. And then the computers got so good that if you did anything but choose the top move that the computer recommended, you would lose. And so the human being's role in that game became, yes, accept the first move. Yes, accept the first move. That's no longer fun and we don't see many games of centaur chess anymore. Right?

• 19:42Speaker 01

  And so, in fact, I just had a conversation with a colleague named Thomas Goberda, where we axiomatized our argument for what the future might look like. And one of the axioms that I'm willing to even accept for the sake of argument is what I call the solved axiom. And the solved axiom says, let's assume at some future point, we're nowhere near there now. And I don't expect us to be there necessarily. We'll see where the capabilities taper off if they do. But let's assume the solved axiom, which says, any mathematical problem that can be solved with human interacting with AI, could also have been solved by AI alone. That's the solved axiom.

• 20:23Speaker 01

  So that would mean that really there's nothing new for a human being to contribute. You know, you can say to an AI system, okay, I have problem X. I want to think about problem X, but the only way that I will make progress, any progress that I'll make on problem X will just be press a button. The AI thinks about it, doesn't necessarily solve it, doesn't mean that it'll solve every problem. The Riemann hypothesis is not solved. Okay, that's what I really want the solution to. I just didn't think in my lifetime I would see any kind of progress towards it that we saw a few days ago. So it's not that AI systems are solving every problem. They're solving every problem that could be solved by humans interacting with AI. That's what that axiom would be.

• 21:04Speaker 01

  So I'm saying, if we are ever in that world, does that necessarily destroy mathematics? And my strong argument is no, I don't think it destroys mathematics.

• 21:15Speaker 00

  If that axiom is true, what's the role of humans?

• 21:17Speaker 01

  So I don't think that mathematics, pure mathematics, not engineering mathematics, not things that will cure cancer and solve, you know, jet, everybody has a jet pack or whatever other technological advancement you might imagine, pure mathematics. You know, we know that we're proving theorems just because we're curious about them. I don't think the solution to theorems living in ones and zeros in data centers is valuable to human beings. I think the value of pure mathematics is that it lives in the heads of human beings.

• 21:48Speaker 02

  So just to add to this, I mean, I'm glad you came right out with the question that's on everyone's minds. So thank you for that. I would caution against overfitting to one objective function for this, right? Because, you know, we used to think of ourselves as just the smartest things around. You know, human mathematicians can solve, you know, reach the highest treetops. But now we're faced with the fact that many of the problems that we thought were, you know, quite a ways out have been solved by AI. But maybe one of the consolations we can take is actually the fact that we're very computationally limited compared to the way that AI is solving it. You know, given the same finish line of proving the same theorem, I can't really hold that many papers or ideas in my head at one time.

• 22:33Speaker 02

  So in order for me to reach that same finish line, maybe I have to develop more generalizable and abstractable ways of getting there that then help you not just solve that one problem, but, you know, explain connections or other directions you want to get into. Right? So it's always amazing when you look back on papers you've written and you can see these threads, invisible sometimes, that connect different areas of work. Because developing systems of ways of understanding problems helps you for surrounding problems, not just to get a lean, formalized, you know, one theorem down.

• 23:08Speaker 00

  But the interesting thing here is those threads, what you're saying is sometimes invisible, are quite visible to AI systems now. It seems like that's perhaps the actual very reason why they're able to make so much progress is they can extract the threads and pull on them because they have the view of so many papers.

• 23:28Speaker 02

  I think it depends on what you mean by threads. You know, for me, the reason you prove results is so that you better understand the world around you. Mathematics to me is a vehicle for abstraction and understanding things. And, you know, it's when you have a major result, what's important is not just the truth of the statement, but what you've learned en route to doing that. Usually people's first question is, okay, you proved something fantastic, but how did you do it? And then people, once they digest that new idea, that new definition, that new way of looking at things, there are lots of other problems nearby that fall. So somehow you're building structures that help you attack not just one problem, but a whole sea of problems.

• 24:12Speaker 02

  And, you know, it's not that we need fully formalized, you know, results to do that. Sometimes we reason in very sloppy metaphors that, you know, help us conceptualize the world around us.

• 24:23Speaker 00

  And how much of this is something Alex mentioned offline, attention is all you need? How much of this is the human ability to shine a light on particular areas that quote unquote matter? So the human ability to ask the right questions, the important questions?

• 24:39Speaker 01

  That's always been the case. So within this, before AI, there's a, you can think of like, here's the axioms. And just abstractly, from the axioms, you can get to logical deductions, right? There are theorems that we could prove from the axioms. And there's a much, much, much larger space of theorems that we could prove than the space that human beings ever actually care about. And so, you know, so by, almost by definition, it's like a tautology that the theorems that we'll get excited about are the theorems that we care about as humans. So maybe AI can convince us that there's some interesting structure. Maybe you can say, hey, look over here, let me give you this new definition or something. It hasn't done that yet.

• 25:16Speaker 00

  And sometimes the mechanism by which we select the things that we care about, it could have an application. So Navier-Stokes is connected to fluid and all natural systems. But it could also be a thing that some kind of mixture of beauty and difficulty that just pulls at us. Maybe there's an intuition. You know, the prime numbers are full of these really complicated problems that maybe don't have an obvious application in the immediate sense, but it pulls at us somehow. And maybe as a symptom of the pursuit of the hard thing,

• 25:56Speaker 00

  great applied mathematics and engineering and solutions to real world problems comes out.

• 26:02Speaker 02

  So just, you know, as an example, right? A lot of times when I'm working on a problem, I might have some like bigger goal that I want to accomplish. But usually what I do is I try to find the simplest, most embarrassing open problem that I still don't know how to solve. You know, that aspect of like digesting it down to something, boy, how can we not know this? It, you know, it feels like it's on the right track because then when we can solve that example, maybe we're assembling the set of tools we need to make attacks on other problems. So I do think we're going to have to become a lot more deliberate about, you know, what do we care about? How do we use these tools from a metacognition standpoint? You know, what exactly are we delegating to them and what are we not delegating to them?

• 26:45Speaker 02

  And we're really at the beginning stages where we're just trying to piece it together as we go along. You know, on the one hand, looking for a proof is kind of like exploring this tree of possibilities. You might have some intuitions based on some other proofs you've seen about, you know, which directions to go in. Maybe I should go down this branch. Maybe I should go down this branch. But sometimes we encounter, you know, points in the tree which have really high branching factor. We just don't know which way to go down. We don't even have a guess about if we went down them, what would we discover? And now in this centaur world of math, we suddenly, you know, have something else intelligent that we can interact with. Right? And what is the right way to use that tool to search for the beautiful theories?

• 27:28Speaker 02

  I think will be challenging and evolving, but it's important to not lose sight of the fact that we don't just care about theorems. So I, you know, I'm a mathematician, I'm a theoretical computer scientist, but I've also been reading a little bit more broadly, you know, about social sciences, just to sort of make my peace with all of the changes that have been happening. One of the things that really stuck in my mind that I liked is there's something called the Doorman fallacy, which is very interesting. So the idea is that you take a complex job and you boil it down to its simplest, most visible task. So what does a doorman do?

• 28:05Speaker 02

  I guess they hold the door for people as they're coming and going, right? And when we lived in New York, we really enjoyed talking to our doormen. You know, they were wonderful people, but you know, if you digest it down to just that simple job, you can think, oh, I can replace them just by a sliding door that detects when people are going or coming. And you miss out on all the aspects of their job that, you know, add a lot of value and are harder to articulate. And I think that's a good way to think about mathematics in the age of AI, too. What does a mathematician do? You know, the simplest and most visible part of the job is we prove theorems. And you know, theorems about conjectures that were open for hundreds of years, like Fermat's

• 28:48Speaker 02

  Last Theorem, these are very visible things that we've spent a lot of time popularizing. But I don't think that that's all that a mathematician does. I like to think about it as a mathematician is someone who thinks deeply. And a byproduct of thinking deeply is, you know, proving old conjectures. But it's also building new theories, building new ways of looking at old things, making new conceptual connections. And these are parts that, you know, maybe AI will swallow up to, but we're not there yet at this moment. So even though this is a momentous time, I don't think it's the right time to hit the panic button. I think it's the right time to really re-center in terms of what our values are and have these conversations as a field.

• 29:31Speaker 02

  And remember that what we do really has a plurality of different aesthetics and goals. And that when one thing becomes cheap, like problem solving for well-defined things, it makes the other things that are orthogonal to it all that much more valuable.

• 29:46Speaker 00

  That's actually really beautiful. The doorman fallacy going to perplexity here is the mistake of reducing a person's job to its most obvious task and assuming that automating that task replaces the value of the whole role. A hotel decides, quote, "We're paying someone to open a door. An automatic door can do that more cheaply." But in fact, the doorman also welcomes guests and recognizes returning customers, handles luggage and hails taxis, provides security and discourages unwanted visitors and signals prestige and makes guests feel cared for.

• 30:20Speaker 02

  That was my favorite part. I looked up what a doorman does and boy, we have a lot of commonalities with mathematicians. If I don't signal brand value, I don't know what I'm doing.

• 30:29Speaker 00

  That's funny. But I think the task for us humans in general, but also mathematicians, is to come up with that list, like actually make more explicit exactly what it is that humans do. And I haven't heard any of you mention beauty. I apologize if I've mentioned beauty excessively already. But there's something that mathematicians are, from my outsider sort of fan perspective, are kind of expositors, scholars of things beautiful broadly defined.

• 31:05Speaker 01

  That's exactly right is until five seconds ago, there was a near perfect correlation between the ability to solve new deep theorems and what we actually want as human mathematicians. And what we want is exactly what you said, beauty and understanding and depth of thought. And the fact that you can press a button and get what used to be our proxy without any effort, without any deep thought and without any necessarily knowledge. Like we have people saying, you know, do a breakthrough and then putting whatever they find on Twitter. And now, you know, what are mathematicians supposed to do with that?

• 31:45Speaker 03

  I would go one step behind answering questions and doing problem solving is asking the right questions. So like, that is how I think you measure good problem solving is asking the right sub-questions, asking, you know, a question that is interesting and can be solved and can be interpreted. And so I feel like the value of asking good questions is, has always been something we value in our colleagues and that's going to become increasingly more valuable.

• 32:12Speaker 02

  I agree completely. And just to emphasize a point that was sort of hidden in there, I think asking the right questions requires a lot of mathematical insight.

• 32:23Speaker 03

  Absolutely.

• 32:23Speaker 02

  You'll see people make that comment all the time, you know, what are we going to value going forward? It's asking the right question, it's giving great talks. But you know, I wouldn't minimize, not that you were, but just to be explicit, I would not minimize the role in mathematical insight in both of those tasks because it's when we say that, you know, talks will become more important. I'm not imagining that, you know, we're all going to become Twitter influencers and TikTok whatevers.

• 32:49Speaker 00

  I hope not.

• 32:50Speaker 02

  I hope not. Because you know, really what stands out is having ideas that get adopted by your community that end up becoming powerful over time. So, you know, I want to tie this back to something that Lex brought up earlier on, which is, you know, he asked, am I being overly optimistic? Probably, right? Compared to, you know, how grad students view these things. But I think one of the things that I'm most worried about in the near term is what's going to happen to our systems and institutions. You know, it's not just hiring and tenure and all these things, but it's really that when a grad student starts off their PhD, you know, they're a new entrant to the field. They're trying to absorb what are the techniques in their field, what are the important problems?

• 33:34Speaker 02

  And they're trying to figure out how to make a name for themselves. And what we used to do is we would give them a blueprint that was very simple to follow. Take a problem people care about and improve it. And then people will know who you are, you know, you'll get papers in top venues, you know, doors will open for you. But now I think that blueprint has changed in a lot of ways because there are so many results that are happening right now that we're all just overloaded with it. And I think it's okay to take time to rethink that blueprint. We still care about the same things like ideas that are going to influence your community, but they're just harder to operationalize in terms of some simple rubric that we can give them. So I'm still optimistic.

• 34:16Speaker 02

  I just think it's going to take, you know, reworking a lot of our systems and institutions to do that.

• 34:21Speaker 01

  By the way, and so to go back to your point, I don't actually believe the solved axiom. I don't think we're there. I can prove a theorem, but I don't understand why the proof works. There's some magic that happens over here. And until I generalize somehow the phenomenon and understand it more deeply, then I'll see why the proof works the way it does. Maybe that's one of the reasons why I don't actually believe the solved axiom because the things that we do are much more exploratory and what constitutes a good question beyond just, here's the Riemann hypothesis, make some progress on it.

• 34:54Speaker 03

  So I think you also brought up a really important point, this like generality and generalization of our concepts. So related to beauty and mathematics, I think one of the gold standards has always been how simple can your result be and how much does it generalize? And, you know, out of this infinite space of theorems that could be proven, finding some, some structure that is simple and generalizes, like that is, I think, still the gold standard. And unless I'm mistaken, humans are still asking the questions. So like we, you know, we prove existence, uniqueness of Navy or Stokes with AI, but humans are guiding this. Humans are still asking the questions.

• 35:35Speaker 03

  Like I would love to see the next breakthrough if you had a machine that can prove any theorem you pose. I wanted to start telling me what questions I should be asking.

• 35:46Speaker 00

  Do you think it's possible that that machine just does a for loop and finds all the simple and generalizable, basically all the E equals mc squared?

• 35:57Speaker 03

  That's happening today, actually. So like there are absolutely research programs that are systematically, automatically discovering kind of minimal physical laws. So this is less mathematical theorems, more physics governing laws and equations.

• 36:11Speaker 00

  So observing natural systems and trying to see in those natural systems, if there's simple physical laws that model how they operate.

• 36:20Speaker 03

  Yeah, distilling it down to like the basics. So you can kind of recapitulate known theories. If you observe balls dropping, you can, you know, deduce gravity and F equals ma. If you observe fluids, you can deduce Navy or Stokes. And then taking that to data, we haven't written down simple equations, weather, turbulence, solar flares, things like that. And I'm assuming similar things are happening in math where, you know, there are probably these like minimal pieces that can be deduced.

• 36:50Speaker 00

  Do you think there's a large space of simple and generalizable things in math that could be discovered automatically?

• 36:57Speaker 01

  So my version of the solved axiom is that anything that I do as a human mathematician, I've tried on several occasions to sit down and say, okay, you know, LLMs will never do this that I do. And I'm like, well, why not? I can ask a question, you know, I can ask it a question. I can ask it, what questions should I ask? Right? There's, there's sort of, because it's such a generalized system, I'm not saying that it's good at all of those things now. But my version of the solved axiom is anything that I can contribute as a human being, it could also have contributed on its own. And so what I really mean by that is in terms of value, what do I bring to the community? It can't be, I asked a great question.

• 37:42Speaker 01

  It can't necessarily be in this world of the solved axiom, which again, I don't believe. But even in that world, it should be something other than, I asked a great question, I proved a great theorem. And even in that world, I think you need mathematicians, people with human mathematical expertise.

• 37:59Speaker 00

  Ankor, do you believe in the solved axiom? Where on the spectrum of like, how close we can get to it? And in that spectrum, what's the role of the human, do you think?

• 38:10Speaker 02

  I guess I, I don't really believe in the solved axiom. I think, you know, maybe it's half true in some ways and figuring out which half of it is true will be very interesting and challenging. So I think there'll be major surprises and more to come in terms of what AI can solve. But you know, for me, when you're facing a new technology that's really, really good, the danger is that when you try to conceptualize why it's working, you do so by anthropomorphizing it. So, you know, maybe one of the earlier lessons here is from deep learning. So there are these amazing examples, you know, you can take neural nets and you could do human level or better object recognition.

• 38:52Speaker 02

  But in general, the way that it succeeds is less robust than how a human does because there are all these examples, like adversarial examples, where you can add very slight patterns to natural images that can completely fool these object detection systems, but not a human.

• 39:10Speaker 01

  And you know, I had a former colleague at MIT, Alex Maedrich. They had some really nice work, which I really enjoyed. It was, I think the title was something like adversarial examples are features, not bugs. So they did the following cool experiment. So you take a bunch of natural data, maybe pictures of cats and dogs. You train a deep network to identify them very well. You add some adversarial perturbations to them that, you know, make to the AI system, a cat look like a dog and vice versa. So it's kind of think like a small little pattern that changes how the object detector works. And then you create a totally fake data set. You know, it's got pictures of cats with dog patterns on top and you label it as dog and vice versa for cats.

• 39:54Speaker 01

  And amazingly, when you train a deep network on this with all the wrong labels, it's going to learn something that does well on natural data. So somehow, when we recognize what a cat and dog is, we do it, you know, in a way that we can articulate in terms of, you know, its legs and its tail and its shape in general. But a lot of times when, you know, deep networks or AI can do things, that they might be able to find trap doors or back doors to solving the problem that use a lot of statistical relationships in the data that a human doesn't. So to me, you know, it's important to not, you know, draw this boundary, you know, very sharply between what you can do and what AI can't do, because you're going to be wrong.

• 40:39Speaker 01

  But that doesn't mean that I don't believe there will emerge some kind of boundary, if only for the fact that I think we over anthropomorphize the way that they solve problems. And when I'm looking through the OpenAI solutions, they just don't look like a human has solved those problems. So to me, there still seems to be a very big qualitative difference. And that's an opportunity really for collaboration and centaur math more than it is, you know, dread about the AI overloads. But, you know, who knows, could be wrong next week.

• 41:09Speaker 00

  What's your intuition about how much of this is AI just tying together a bunch of thousands of disparate papers and pulling at the threads? Sorry for bringing back the metaphor, but the threads of the different papers and then just basically articulating the proofs that were already there in the different papers. You're making minor connections. Or is it actually developing new mathematical ideas? Have you seen glimmers of whatever that genius alpha zero move in the game of Go?

• 41:44Speaker 01

  Yeah, move thirty seven.

• 41:45Speaker 00

  Move thirty seven.

• 41:46Speaker 01

  So I haven't read all 700 of their papers, the ones that I have looked at. You know, some people speak about this dismissively like, oh, it was in the convex hull. You do the moves that we know as humans and you just do them really well and do a really good tree search. I wouldn't call it the convex hull. I think Tim Gowers had a really good way of talking about it instead as the semi-group generated by all of these things. Yes, it's the same notion of putting together known moves, but the space of known moves, what it can accomplish is sort of much, much bigger than this kind of derogatory convex hull. So with, in terms of the quasi RH, the moves that it took are all things

• 42:31Speaker 01

  that were known to human beings. And there's a very long cascade of them that is kind of shocking when you look at it and say, oh, my God, that's so beautiful. How did you think to, how did you, I'm anthropomorphizing it, of course. How did you think to do this move after that move and after that move? And yeah, that, that does it.

• 42:49Speaker 00

  Could you articulate to the more general public, the intuition, the line between semi-group and convex hull, the derogatory convex hull? And quasi RH, of course, is the quasi Riemann hypothesis that one of the big proofs.

• 43:03Speaker 01

  Yeah, let me try to do it like this. You have, think of like a balloon. So the inside of the balloon is its convex hull. And so we have like all of these results that, all these ideas that mathematicians have thought up. And then anything inside of that is some linear combination of it. But it feels compact. It's like, there's this kind of closed, enclosed area. And within that area, like AI hasn't really made new progress because it hasn't moved the balloon out. It hasn't given us a new idea yet.

• 43:32Speaker 00

  And it's derogatory because the intuition there is we could have done it, mathematician could have done it because it's already quite obvious. It's already within the, the space of things that have already been thought about and developed.

• 43:44Speaker 01

  Yeah, so it has read every paper. It's exactly what Anker was just saying. It has read every paper. It has internalized all of that. That's more than any human brain not even could hold, but has time to read and learn and internalize. So in terms of breadth, yes, it beats any human being. And then in terms of depth, you know, has it really generated a genuinely new idea? And you can argue that, at least in the case of quasi RH, I would not say it's a genuinely new idea. It's a, it's a beautiful combination of a lot of things that were very well known. There are a lot of people who are kicking themselves that had some idea in that direction 20 years ago. And, and two years ago, there was a big breakthrough on cubic theta functions and cubic Gauss sums.

• 44:28Speaker 01

  And, you know, that ended up being the, the path that the AI took to, to the solution. But so to contrast that with a semi-group, I think of a semi-group as like tiles. So you, you know, you're just playing tiles on the board. And what you're allowed to do is whenever you have a tile, you can put another tile to the left, to the right, to the front or to the back. And, and then you can just start tiling things. And that's like a semi-group action. You, and, and the question is, well, how far can you reach with that? You can go, I don't know. You know, you can go really, really far out with, with the existing ideas might prove a whole lot more than we as humans have been able to explore.

• 45:11Speaker 00

  So your intuition is more in the semi-group direction? Meaning like there is at least glimmers of expansion versus linear combinations inside the convex hull?

• 45:25Speaker 01

  You know, how we choose to interpret what it's doing, I think is, is up to us. But throughout the history of mathematics, there have been very seemingly difficult and deep problems that then someone comes up with a three-page solution that is very elementary and could have been explained to an undergrad and could have been discovered 20 years before when the problem was first posed. And had it been discovered 20 years before, it would be an, an exercise in a textbook somewhere. And because that solution wasn't discovered and really bright people worked on it for a long time and made incremental progress and made the problem really famous. Then by the time it was actually solved, it was a really big deal. That happens from time to time in mathematics.

• 46:07Speaker 01

  And what I said a year ago, when I wrote, so there was this International Congress of Mathematicians this summer, and I was writing my proceedings that they wanted 10 months in advance. So I said, you know, what am I going to write about math and AI now that will still be relevant 10 months from now? And I, I think what's there is somewhat evergreen. I stand by, by what I wrote. But one of the things that I wrote there was exactly that we're going to discover that a whole lot more problems that we thought were unreachable by current technology, actually are reachable by current technology, if you can put it together in just the right way. And that's what AI is amazing at. Again, someone could interpret that as, oh, so it's inside, you know, these solutions were inside the convex hull. You want to call it that? That's fine. I think that's, you know, it's, it's not invalid.

• 46:51Speaker 02

  So I like all these metaphors, you know, also the balloon. So the important thing about the balloon is the boundary between what's inside and outside. And what makes it derogatory is the fact that, you know, you haven't seemingly changed the boundary. But I, you know, definitely these problems have changed the boundary. And, you know, maybe a more concrete thing is just that the types of things that people are going to work on after that drop of OpenAI problems is forever different. You know, there were at least a couple of results there that are like, you know, field altering results that will change what types of problems you give to young researchers, what areas you want to explore, because your conception of the landscape is just that much different.

• 47:33Speaker 00

  So not just the, because it expanded, but because the very expansion changed the direction of the interesting questions.

• 47:42Speaker 02

  That's right. So in fact, you know, one of the problems, which probably, you know, maybe it's the easiest to explain what kind of effect it had on people is not actually from the OpenAI drop. It was an anthropic result that was written up by my colleague, Virgi Vasilevsky-Williams and then Josh Allman. So this was about the threesome problem. And I'd say that there it has a major effect on what's called fine-grained complexity theory. Because there was a big surprise that, you know, a very basic problem that you give people in like coding interviews can actually be solved asymptotically much faster than the solution anyone knew. And that was counter to what the community was betting. In fact, there was a whole theory built up around the idea

• 48:25Speaker 02

  that if this problem really is that hard, then all these other problems inherit its hardness. So now as a result of that problem being easier than we thought, you know, having a bit better of an algorithm, you know, yes, that gives you a lot new algorithms across the entire landscape. But it also changes the character of the field because the goal there was to put all those problems together and said that we should be studying them all together. But now we're sort of back in the state of affairs where maybe the right way to study them is separately after all. So I think these are examples where irrespective of how much you know, deep, new, generalizable ideas we get out there, even just knowing the truth of the theorem is going to change the way that people organize

• 49:10Speaker 02

  research in that area. And you know, there's no putting the genie back in the bottle on some of these problems.

• 49:15Speaker 01

  I mean, imagine one of the 700 was factoring in polynomial time. That would absolutely change. Everybody would know. Like you can't use your credit card on Amazon anymore. All of e-commerce is gone. All of secure communications is gone in like that.

• 49:29Speaker 00

  Can you explain factoring in polynomial time?

• 49:31Speaker 01

  Yeah, so if you give me two very large numbers, a million digits each, I can multiply them together and do that in a split second. If you ask me to go backwards, say you took two large prime numbers, you multiply them together, you give me the product. And you ask me, figure out which two numbers, you know, just factor. Don't we, don't we ask our kids to learn how to, how to factor in school? You know, it's a lie. We do not know how to factor. And the, the fact that we don't know how to factor is why we have the capacity, the ability to communicate with one another in all kinds of secure ways. So all of modern, basically the modern world depends on the fact

• 50:11Speaker 01

  that we don't have an algorithm for taking a large number and pulling its prime factors out of it.

• 50:17Speaker 02

  So a lot of e-commerce does because that's the basis for a lot of the schemes in public key encryption. But you know, famously there are quantum algorithms for factoring. So there's a lot of emphasis these days on, you know, finding post quantum, you know, bases for cryptography.

• 50:35Speaker 01

  Lattice based methods. And yeah, we're already concerned about what will happen when we have quantum computers large enough to, but we still have a little bit of time before, before quantum computers get large enough to make, make this an actual problem. But there's no theoretical reason. I mean, okay, factoring is probably not NP hard. But you know, I mean, let's say, let's say prove that P doesn't equal NP. Maybe that doesn't, that would be amazing, obviously, but that doesn't. If it proves that P equals NP, that would be completely.

• 51:04Speaker 02

  That would be very interesting.

• 51:05Speaker 01

  That would be very interesting.

• 51:07Speaker 00

  I'll definitely ask about more broadly in the context of complexity theory. But presumably OpenAI or any other company would not drop that result carelessly.

• 51:18Speaker 01

  That's the kind of thing that should go through. Yeah, the government.

• 51:22Speaker 00

  Sometimes certain, certain mathematical results quite obviously lead to dramatic impact on the world. Other results might take maybe months to digest, but that one would, would just, yeah, would affect the economic systems of entire nations. So.

• 51:40Speaker 03

  How many mathematicians does the NSA employ?

• 51:45Speaker 01

  Thousands.

• 51:45Speaker 03

  Thousands, like something like this. I've always done this thought experiment, you know, what, what might they have that they just wouldn't tell us about? And this was 10 years ago, you know, that some evidence showed that collaborative teams of mathematicians could solve much, much harder problems than any one individual mathematician if they like broke it down in, in the right way. So I don't know. Do you mind if I just revisit the balloon for a second? Cause I, I actually like the, the semi group analogy better than the convex hole. But I don't think it's as, it should be interpreted as derogatory as it is. Even if it was just that these machines are interpolating human knowledge

• 52:28Speaker 03

  rather than extrapolating. Because even if there is this, this convex hole of ideas and things that could be pieced together from, from bits that humans have invented, we're not doing that fast enough. Or you know, there's a lot of opaqueness and it's, this balloon is not see-through. And so even if you can make this convex hole much more transparent, kind of like lift the fog, give mathematicians radar so they can see what's, you know, over the horizon, that's pretty incredible. And I don't think we should be interpreting that as, you know, derogatory at all. I do think it is more, more extrapolatory than that though.

• 53:07Speaker 00

  Yeah, we should say probably the size of the convex hole. We don't know actually how big it is.

• 53:14Speaker 03

  Well, that's part of the point. We don't know how big it is. We don't know what's next to each other. Like there's so much fog of what we know, like, and, and that's exciting to, to lift that.

• 53:25Speaker 01

  But human beings do make new definitions, you know, bring new concepts in. And that fundamentally feels different from combining existing ideas and repackaging them. That, when human beings do that, and they do that from time to time, it really feels like, okay, there's a new boundary point. And now what's the convex hull of that, of what, of what you could prove? So it's like, you can take the balloon and there are ways to pull on it.

• 53:51Speaker 03

  And we know when it expands, like somehow we have an intuition for when it's getting bigger. Like you can recognize that.

• 53:57Speaker 02

  In many ways, the archetype of what the researcher looks like is going to change because of this new technology too. You know, I'm always amazed that when you look back historically, how much very different fields of math really spawned from similar ideas. And a lot of those connections drove the development in the beginning, but over time they've become a bit more specialized. And I think certainly the way that we train graduate students, you know, we'll need to make sure that they're not just over specialized in the same way. You know, I don't know what the future of hiring will look like. I don't know what the future of journals will look like. But I think people who think broadly about, you know, a bigger swath of mathematics

• 54:40Speaker 02

  will be in the best position to benefit from these technologies. You know, that would be my guess.

• 54:46Speaker 00

  Can you ask a almost silly sounding question? But so a large part for many decades of research has been the paper. If you were to guess in, in the years to come, how does that change? How a human contributes their idea, maybe collaborate, how a team of humans contribute an idea into the public space?

• 55:10Speaker 03

  So, I mean, I don't know about in pure math, but in engineering and some of the applied sciences, we already have a pretty rapidly shifting view of what an academic contribution is. So if you write code in part of a larger infrastructure that is reusable, that, you know, supports larger projects. So think of LIGO or some of the big, you know, billion dollar astronomy projects or, you know, cancer research projects that require hundreds of people and huge resources already. It's not necessarily one 15 page PDF. You know, there's lots of different pieces, code, data, reusable bits that can be fit into a larger framework.

• 55:55Speaker 03

  And in my mind, that is kind of obviously even more relevant with AI is having, you know, a little workable piece or widget or API or code base or data set that can be leveraged by the community or by AI is increasingly valuable, even more than just like one result or one paper. And I don't know how, how that looks in, in pure math.

• 56:20Speaker 00

  Can you comment on your view, how everything's changing? Even just observing the insane evolution of even archive, which is similar in some ways to what's happening on GitHub. For example, a lot of people in the open source movement are really struggling to know what to do. Some are predicting the death of GitHub.

• 56:39Speaker 01

  Why? Just because there's so much code being added to GitHub that it can't sustain the growth?

• 56:44Speaker 00

  So one of the things that makes the open source movement successful is there's the kind of one or a few folks that really manage each repository, the human beings. And they are psychologically and on many levels are becoming overwhelmed by the, all the pull requests. So it becomes impossible to manage and they get exhausted. And there's this whole feeling, the open source movement, maybe similar to the academic environment, requires the humans to be motivated. And if you get worn out by the just similar, the, the ideas that Terry Tao mentioned, that you just quit. And so that, that's a failure mode that you want to kind of avoid.

• 57:24Speaker 01

  So journals were already in math, under very, very severe strain from the number of things before AI, having nothing to do with AI. Just there was, you know, more people producing more knowledge, sending it to journals and journals had to referee it and referees are exhausted and being asked to work really hard to understand these theorems and check them. The only thing worse than that system, I think, is not having that system. Because the way that like our mathematical epistemology, the way that we arrive at truth is deliberately slow and careful and precise. Go back to Perlman's solution of the Ponkra conjecture. He puts these papers on the archive. And then people around the world are, you know, shocked by this.

• 58:08Speaker 01

  And so they invite him to come give lectures. He gives, he comes to Stony Brook, he comes to NYU, he comes to Princeton. He's going all around the country and the world. And he's giving three hour long, impromptu explanations of, of, you know, maybe an hour is, is him explaining what he's doing. And the next two hours are people saying, on page 247, lemma 412, I don't understand how you do this. And then he goes to the board and gives them a half hour lecture on how he does that. So the entire thing lives in his mind. He's done the best he can to communicate it on paper. And then it's up to us, the community, to go back through it very carefully,

• 58:47Speaker 01

  very, in a detailed and precise way and agree, yes, this is a real proof of the Ponkra conjecture. And that took years to do and a whole lot of human effort. So coming up with the solution is only the first step in the process. And that process has to be deliberate and it has to be community driven. And so that's what the journal system is trying to accomplish.

• 59:09Speaker 03

  So you make a really interesting point. And it kind of highlights this traditional view of a pure mathematician as this like lone genius who, you know, solves this singular problem and then goes explaining it to the world. In many other fields, we have moved past the point where we can't individually advance, you know, circuit design or aircraft design or protein folding or any, you know, astronomy. And we have to work together in these billion dollar collaborative projects. And again, I'm an engineer, so I'm thinking leadership is obviously the thing that a community needs when it's having this kind of crisis of organization

• 59:51Speaker 03

  and individuality, moving towards a more collective model. And that's interesting because I know so many mathematicians and like. By kind of organization, this is a very independent group of people who don't like being told what to do, who don't like being told what to study, who are very independent by their very nature. You know, we like solitude and working alone. That's why we, you know, spend so much time doing it.

• 1:00:19Speaker 02

  I think that's a really good point. I mean, it's interesting just to see the differences in working styles even between math and physics, you know, with things like the Large Hadron Collider. So I mean, let me take a contrarian view for these things, which is, um, you know, you should never let a good crisis go to waste. I agree, the journals were overloaded, the conferences were overloaded. It always strikes me that we're doing so much work, you know, to review these things, to handle all these things. And now we're faced with a task that someone's got to review all these AI-generated papers. And some of them are important, but you know, it's really hard to get your community on board with doing that. I would really, you know, I don't know what's going to happen, but I would really be in

• 1:01:01Speaker 02

  favor for finding new models, not to replace the old ones, but to relieve some of the pressures that have come about. So, you know, I'd love to see models where, you know, maybe we take a more collaborative effort to verifying some of these AI proofs, right? So some of the OpenAI results that have dropped, you know, improvements on fast Fourier transform. You know, there's been tweet streams with people improving the constant. I don't really know what to make of that, you know, I'm not even on Twitter.

• 1:01:35Speaker 02

  And it seems to me that we need to figure out other modes of working, you know, for organizing, what are the state of all the improvements? But also, how do we organize and tame these ideas? So it's kind of striking that, you know, the way that we think about publications is you work hard on your draft, you post it on Archive, and that's your draft. You know, you might update and create a new version at some point, or a third version, but the draft belongs to you. It's not really up to the community to add notes, hey, I think you can do this argument differently, hey, I think this is simplified this way. But maybe there are some kinds of things that we need other models for, you know, allowing crowdsourcing for our understanding to evolve. I'd be all for that.

• 1:02:19Speaker 02

  You know, the other thing is I love giving research talks. You know, one of my favorite parts about giving research talks is you can take all of the messy complexities of a paper, the things where you just knew you had to dot the i's and cross the t's, but that wasn't really what excited you. There was some key idea, and there was some special case that really illustrated why that idea was powerful. And that's the part you really want to explain to people, because when someone asked me about lemma 57, I don't even remember it, right? And maybe, you know, this idea that a paper is just this monolithic thing designed to be, you know, written down and completely formalized, well, now we can trust Lean formalizations

• 1:03:02Speaker 02

  to some extent for some of the technical building blocks. And that can allow us to rework the way that we do the writing in the first place, that we can put more of the emphasis on the work on, you know, the promulgation of those ideas.

• 1:03:16Speaker 01

  The MATLAB model is exactly what you were describing, where instead of a paper, you have a PR, you have a pull request, the maintainers and the reviewers will go through the pull request, they'll make suggestions here and there. I mean, the referees make suggestions, you can improve this and that, but it's not a public process in a journal. In GitHub, open workplace, you can, you can see exactly who contributed what idea that simplified an argument, or you're, you know, you're reusing this, you're reinventing this idea that already exists in this part of the library, you should just quote that and refactor this part, whatever. So, um, and MATLAB, working formally, allows one for the first time. Like, one of the problems with mathematics is that we don't have a perfectly well-written

• 1:04:01Speaker 01

  specification to our theorems, our technical lemmas. There's, if you open up a random page in a random math book and read a lemma, there's so much context that the author, because it's so annoying to write something that fills in every bit of context. So we just assume you've read the previous 60 pages, now I can just tell you what the next step is in that lemma. But then when you try to, that means you can't just take it off the shelf, the way you can a part, a physical part, when you're building a rocket. And so if you, if all of mathematics is done in the future formally, then the specifications are built into the object, you know you're not using it in the wrong way, because you can't use it in the wrong way, because Lean won't let you. And that is the sort of crowdsourced version of mathematics.

• 1:04:46Speaker 01

  And that was the dream of MATLAB, when it was started 10 years ago, was to, to run mathematics more like a science.

• 1:04:54Speaker 00

  As a quick aside, can you mention what MATLAB is and Lean? Just because we mentioned it a few times for folks.

• 1:05:01Speaker 01

  Sure. So mathematics is, in the ideal, a formal process, starting with an axiomatized system, and then logical rules of inference from there. So if people think of, I don't know, do people still learn Euclid's geometry in high school or something? That's supposed to be the kind of model for kids to start seeing proofs for the first time and seeing an axiomatic structure. And you can take that idea and write it in software and say, here are the actual axioms of mathematics, here are the rules of logical inference. And Lean, there's a long history of this kind of software, it's called interactive theorem provers, where you're writing line by line in a proof, and the computer says, yes, this, this line does follow by logical inference from the goal state on the previous line.

• 1:05:45Speaker 01

  So Lean is software that allows you to check all kinds of things. It can check the specifications of your code aligned with what you, what you're trying to define. That's what it was, what it was invented for, was bug-free code. And hopefully in the future, we will have bug-free code, because AI systems are really good at finding bugs in code written by humans. So that's, it's becoming ever more important for that application. But another application is to check mathematical results and allow you to write a mathematical result in a very formal way. And so now Lean itself is just the hardware that checks the logical steps. You do not want to start every math project by defining what the natural numbers are,

• 1:06:28Speaker 01

  using zero and the successor, you know, the piano axioms of the natural numbers. So it's nice to have a large, um, general library that's extremely well-maintained, extremely well-organized, and so on. And that's what the, that's what MATLAB is. MATLAB is the Lean library for formalizing mathematics in the Lean ecosystem. And again, Lean is just one interactive theorem prover, there are lots of others. It's the one that so far has been winning the math wars, if there are any math wars, it's a competition of two people or something.

• 1:06:59Speaker 00

  And it's the technology used in part to show that the OpenAI release of all the different proofs, that the proofs are correct.

• 1:07:10Speaker 01

  That's right. It has become now standard to, because auto-formalization has gotten to the point where AI systems can not only prove a theorem in natural language, you can't trust their natural language proofs, they're, these are stochastic processes. And so the fact that they're saying things doesn't mean that they necessarily have a model of mathematics in their heads, whatever that means, the same way that we do. But that's okay, because they'll write a Lean, Lean code that builds on top of MATLAB, builds on top of the prime number theorem plus project or whatever, and checks that it has indeed solved this problem down to the axioms.

• 1:07:46Speaker 00

  So this is where I'll just take the tangent, and the thought experiment may be a bit philosophical at this moment, but what if you have that black box that gives you proofs that you can't understand? Let's say the more interesting case where, not because it's so big, but because the ideas just don't make sense to you, to any of the world expert mathematicians in that field. What do you do with that?

• 1:08:09Speaker 01

  So the only way, sort of almost by definition, because the logical rules of inference are written down, and there, there's only a few of them, there's a handful of them, the axioms, there's a handful of them. So the only way that you have mathematics that's not understandable is scale.

• 1:08:26Speaker 02

  Let me add a contrarian take. So one of the things I also wonder is, what would unformalizing math look like? You know, certainly formalization helps tremendously, that we have confidence that the results are correct. But much of the way that we understand the results ourselves is really at some unformalized level. You know, what is an interesting special case? What, you know, where does this come from? You know, this technique you're using, right? And, you know, maybe that's what that world would look like. If we have these black boxes, we would try to tease apart and figure out how to ask them easier questions, to develop some kind of system of understanding of how they actually got to the answer.

• 1:09:07Speaker 01

  Yeah, so there are really two different things that we conflate when we write a paper. One is the engineering of the proof, and the other is the communication of that to someone else. And they're almost at odds with one another. I had this experience, so I taught real analysis last fall, and you open up any real analysis textbook, and this was my experience when I took real analysis. The first couple of lectures are, what are the real numbers? You know, we have to define them or construct them as Cauchy's, equivalence classes of Cauchy sequences of rationals, whatever the hell all that means. It takes like two weeks for a first semester course in real analysis to get there. And then we have the real number line, and now we can start doing calculus, you know, with formal epsilon and delta definitions, yada yada. And it always struck me as bizarre that we do this, because before the class, I knew

• 1:09:51Speaker 01

  what the real numbers were, it's this number line. And then we spend these two weeks talking about Cauchy sequences of rationals, and then on week three, they're, they're number line again. What the hell was the purpose of that? And so it's because the way things are written, or had been written, were very engineering. You need to construct the real numbers before you go start using them. But the real numbers, human beings use the real numbers long before we understood that they needed to be constructed formally. So what I did in my class is I jumped straight into epsilon delta calculus. We did, you know, six weeks of, of calculations until we got to a point where, uh, wait a second, you know, if, here's a theorem, it's true in the real numbers, there's some convergence, something called Bolzano-Varewstrauss theorem.

• 1:10:35Speaker 01

  You have a, you have a sequence of real numbers that are jumping around but in a fixed range, there has to be some sub-sequence that converges to a real number. Well, that's not true in the rationals. Why is it true in the reals? What are the reals? So you don't start asking that question until you hit a problem that needs to be solved. So the way I, I taught the course was sort of backwards from how it needs to be taught if it's an engineering project. Because you can't just start talking about the real numbers until you've constructed them, until you know what they are. Well, that's correct if what you're doing is building up the theory. But if you're telling a story, then you absolutely can just start with the real numbers. You have an intuition for what that is. And until your intuition breaks, you don't need the resolution of how you actually construct them. So I think the way that we write papers in the future is going to be very different,

• 1:11:19Speaker 01

  where we spend a lot more time just talking about the story of what have I learned in this process? What are the interesting? Here's, you do the standard thing here, here and here, but then here, you're sort of stuck and where do you go? And here's an idea for why you go left and not right and not up and down or whatever. And there's a lean formalization that accompanies this story that lets you know that the engineering actually works out. Because you can't just tell stories. If you just tell stories, you know, math is not, it's not music where you can tell a beautiful story and it's okay that you have parallel fits moving somewhere. And you know, you've broken some rule, but Bach does it all the time, so.

• 1:11:55Speaker 02

  So I think that's a very important point, just to sort of emphasize the point that Alex is making. You know, there's this view of mathematics, which I don't believe in, where it's just this rising tide of theorems. You know, how can AI be bad? It's contributing to this rising tide. But you know, really mathematics is not just about the number of theorems we have, far from it. It's also about that story of fighting with problems. Because when you fight with a good problem, it fights back at you and you learn something about it. It's really about the system of beliefs and abstractions you come up with in route to, you know, when you're fighting this problem, you discover some more general meaning that applies more broadly. And I think it's a great opportunity to sort of disabuse ourselves of this idea

• 1:12:39Speaker 02

  that we're just, you know, axiomatizing things until we get to the next theorem. You know, telling that story of how we actually got to these problems, what the parallels are to other problems is now, at this pivotal moment, more important than it ever has been.

• 1:12:56Speaker 01

  Yeah, so to get all the way back to your question about, you know, is it a matter of scale? Well, yes, it's just a matter of scale, but exactly to Anker's point, the way that we think about mathematics is about stories. And it might be that we don't understand the story of this. You can follow a proof line by line and not understand what's, what's really going on. You might say, and you just ask some other LLM to explain to you the story, if it can come up with, with such an explanation. But, but that's what we want. We want stories, we want understanding.

• 1:13:27Speaker 00

  When you fight with the problem, it fights back. I think there's a deep truth to that about the, all the great pursuits that humanity has taken on. And on that topic, I would like to talk to you about some of the great, greatest fights of all time with some of the hardest problems in mathematics that have fallen. But first, about the break, if it's okay.

• 1:13:51Speaker 01

  Sure.

• 1:13:53Speaker 00

  So Alex, you're a number theorist. Maybe it's fair to say the central problem in your field is the Riemann hypothesis. And you showed a lot of excitement on the aforementioned Twitter X about the quasi RH, the quasi Riemann hypothesis proof. Can you explain that? Can you explain what the Riemann hypothesis is? And can you explain the nature of your excitement about it?

• 1:14:21Speaker 01

  Sure, there's sort of a, a one-minute version of the Riemann hypothesis, or a five-minute version, or a 30-hour version. I don't know how far you wanna.

• 1:14:30Speaker 00

  Let's start pulling at that string.

• 1:14:33Speaker 01

  So we know what the prime numbers are. People are interested in the distribution of the prime numbers. And what Riemann discovered is that there's a function, this is totally crazy, that there's a function that takes complex numbers as inputs and has complex numbers as outputs. And that function governs the behavior of the prime numbers. It's called the Riemann zeta function. And by the zeros of the zeta function, we mean complex inputs, where the output is the number zero in the complex plane, so the, the origin in the complex plane. And the question is, where do these, where are these complex zeros? What inputs, what complex number inputs give you a complex number output of zero when you apply this zeta function? Why in the world should that, what, complex numbers?

• 1:15:18Speaker 01

  What are you talking about? We have these discrete numbers that aren't divisible by anything else, sitting in the natural numbers on the number line. And now you want to talk about functions from a complex space to a, to a complex space. That was the, the great breakthrough. The great breakthrough was to figure out how to do signal processing on the prime numbers. If you think about, you know, the reason people can hear you, your podcast is that my voice box is producing pressure waves of sound. That's the natural space. Then that goes into the microphone. The microphone analyzes, does the Fourier analysis on it, converts it into amplitudes and frequencies. And it's just a bunch of numbers, stores those numbers or sends those numbers, whatever.

• 1:16:03Speaker 01

  The speaker then takes those numbers and reconstructs back out what sounds to people like my voice. The same thing is going on. That's what the zeta function does for the prime numbers. It takes the prime numbers, you can think of the prime numbers as particles of air molecules, wiggling around. And it does a signal processing analysis on it that can recover the, the same prime numbers back. And the frequencies and the amplitudes correspond to the zeros of the zeta function. The zeta function has that information in it. And the Riemann hypothesis, so the, the amplitudes of all of these, you can call them Riemann waves.

• 1:16:45Speaker 01

  And the superposition of those waves recovers the distribution of the primes. And all of those amplitudes are supposed to be exactly the same. Like the orchestra that plays the, the, the music of the prime numbers has every single instrument sounding at exactly the same amplitude. That's, that's what the Riemann hypothesis predicts.

• 1:17:05Speaker 00

  So can you actually, we know what the prime numbers are, but maybe going into the space of philosophy and the romantic notions of aesthetics, why are they so interesting and important and central and beautiful at the core of mathematics? And connected to why Riemann hypothesis is exciting is, why is it so difficult to figure out if and what the pattern is in the prime numbers?

• 1:17:33Speaker 01

  Let me start with your second question first. Why something is difficult? I only know how to answer once I know what the answer is.

• 1:17:41Speaker 00

  Once it's made simple.

• 1:17:42Speaker 01

  Maybe it's not difficult.

• 1:17:43Speaker 00

  Okay.

• 1:17:43Speaker 01

  Maybe it's not as difficult as we thought it was. I did not think I would see a proof of the quasi Riemann hypothesis in my lifetime. So the fact that I saw it on Tuesday was extraordinarily shocking.

• 1:17:55Speaker 00

  What did you do the moment you saw it? Like in, in 30 seconds after, like, what was your, do you remember like what?

• 1:18:03Speaker 01

  Yeah, I know where I, I remember where I was the second I saw the drop. So we were at this National Museum of Mathematics Gala in New York City. And so I had, you know, whiskey in one hand and I kept refreshing. I was talking to people, socializing, but I also kept refreshing because I knew it was dropping around 6 o'clock Eastern time. And, and then the drop hit. And in fact, I was talking to, there was a Quanta reporter there. And we were just chatting, oh yeah, they're going to drop these, you know, 300, 400. We didn't know how many, 700, it turns out, problems. And I was like, oh wow, that's really cool, it dropped. So I'm looking at my phone and like the second thing I see is quasi RH. And I'm like, what do you mean quasi RH?

• 1:18:47Speaker 01

  You don't mean like, for, for what? And I, and I look and it's, it's the zeta function and it's all Dirichlet functions. So they've solved no Zigal zeros. And, and I'm like, that, that can't, that can't be what's in this drop. That just, I, I could not compute in that moment what, what I was seeing.

• 1:19:07Speaker 00

  That speed of progress in that little time, yeah.

• 1:19:10Speaker 01

  It's just, it was so far outside my model of what was possible. And this is, it's been a month of things happening way outside of my model of what was possible. The Fermat's Last Theorem formalization, I was sure we were three years away at least from being able to do something like that.

• 1:19:30Speaker 00

  So maybe you can elaborate on that. It was in Lean, it was formalized as, how, how big was the Fermat's Last Theorem in Lean?

• 1:19:37Speaker 01

  13 million lines.

• 1:19:38Speaker 00

  13 million lines, okay.

• 1:19:40Speaker 01

  So I just, two days before, I gave a colloquium in the Princeton Math Department, where I explained in painstaking detail what Lean is, what MATLAB is, but also that AI systems today, you know, there's, there's this MATLAB library where everything is kind of perfectly done, exactly the right levels of generality. And there's no, there's not supposed to be any code that's reusing, code that's, that's reproving theorems somewhere else. Everything is the most general, the most abstract. And what AI systems are capable of doing is building like one or two layers on top of that. But they're, what they're doing is jerry-rigging. They have not been trained yet, I'm not saying they can't do it, but they're not good at it right now, at what I call canonization, which is taking kind of sloppy code, refactoring it, figuring out exactly where in the library it belongs.

• 1:20:25Speaker 01

  What's the canonical version of this statement, of this lema? And then everything, every proof should just be a combination of, you apply this lema from over here, you apply that lema from over there, put, put all these things together, there's your theorem. And then it's really clean, it's really beautiful, very elegant. That's what MATLAB is going for. And what I said in my colloquium is, where we are today is that you can get entire textbooks formalized as long as they're in the MATLAB halo. As long as they're like one, maybe two steps removed from things that have already been done in MATLAB. But when you start, and this had been my practice, my experience had been, when you start asking it to do things that are, you know, three or four steps removed from the library, the way that it came up with a definition of this object

• 1:21:08Speaker 01

  and the way that it came up with some other definition of the same object, because it was doing it for two different purposes, then they clash. You can't stitch these things back together. Like, you really need the library, MATLAB, to grow in order to get to a place where you can do research manuscripts, you know, that are, that are really using, synthesizing tons of different areas of mathematics.

• 1:21:30Speaker 00

  Which is why you would expect Fermat's Last Theorem to take, because it's so far outside of the halo of MATLAB, it would take quite a bit of time.

• 1:21:37Speaker 01

  Exactly right. So Kevin Buzzard had a million-dollar grant to spend the next five years, or I think maybe he's already a year or two into that grant, to reduce the formalization of Fermat's Last Theorem to things that were known in the 1980s. So he wanted to do that last step, the, the seven years of Wiles's work and Wiles-Taylor to finally finish, you know, Fermat's Last Theorem, but built on top of things that we knew would be extraordinarily difficult. He wasn't going to do Langlands Tunnel, he wasn't going to do Mazur. There were, there are all kinds of things that were expected to be even much more difficult than what these five years would take. And everything I said in that colloquium was wrong.

• 1:22:16Speaker 01

  And I think Michael Ockham was on a Wednesday. On Friday, there was a drop of 13 million lines of code. From the axioms, Fermat's Last Theorem is proved. The statement of Fermat's Last Theorem is completely elementary. There's no semantic alignment to check. You just glance at it. Yes, that's the statement. Lean says it's Sari-free.

• 1:22:33Speaker 00

  Can you explain how it was done?

• 1:22:35Speaker 01

  How it was done? So it was anthropic. And how it was done was AI systems just got really, really good. It's still not a nice proof. It's not the clean proof. It's not, there's still a ton of work that needs to be done to make that proof a math lib quality proof. And I think that's important work and work that we want to do. But in terms of just getting that Sari-free theorem, they just blasted through all of these. They jerry-rig this, they jerry-rig that. And, you know, 13 million lines later, they had a Sari-free formal Lean proof of Fermat's Last Theorem. It's just absolutely shocking.

• 1:23:06Speaker 00

  By the way, it's beautiful to always, because of how rapid everything is, there's going to be a lot of statements that everybody makes, the world experts make that are wrong. That's the whole point of it. And that's the fun thing about human discussion is we have disagreements. And I would venture to say somewhere between 40 to 80% of everything said today will be wrong in five years. Because everything's changing so fast. But that's the whole point of human discussion and deliberation and discourse and even the scientific process is saying shit and seeing with sticks over time.

• 1:23:43Speaker 01

  Yeah.

• 1:23:43Speaker 00

  Over 10 years, over 100 years and so on.

• 1:23:45Speaker 01

  I was describing my experience. I had a lot of experience with Lean and with using AI. I started doing Lean before AI existed in the sense that we understand it now. And it was very painstaking. And the second AI systems came on, you know, the first LLMs were coming online. They were crap at Lean, but then they got a little bit better, a little bit better. And I was like, oh, my God, I can do library search. I don't have to sit there looking. I know there's a lemma that does this, but I don't know what it's called. And I don't know which exactly, is it in the manifold's library? Is it in the algebraic geometry library? Like, give me this, give me this lemmas. It was great at that. And I was like, oh, this is amazing. I'm going to keep using these systems. And every bit of my experience told me that they can do like X, but not 10X. And this was a million X.

• 1:24:29Speaker 01

  It was just completely outside of the distribution in my mind of what systems were capable of.

• 1:24:33Speaker 03

  Do you have a ballpark estimate of how many humans were involved in that? And like, what is the level of human involvement in generating those 13 million lines of code?

• 1:24:42Speaker 01

  I believe it was two or three humans.

• 1:24:45Speaker 03

  Okay.

• 1:24:45Speaker 01

  And I believe most of their involvement was keep going, all caps.

• 1:24:50Speaker 03

  Okay. All right. So you think it's really just like gentle guidance, not like, you know, pathway creating and?

• 1:24:58Speaker 01

  As far as I understand, there was no domain expertise.

• 1:25:01Speaker 03

  Okay.

• 1:25:01Speaker 01

  On the parts of the people who were doing this.

• 1:25:04Speaker 00

  As the meme goes with OpenAI or Anthropic, you know, the prompt is solve math or something.

• 1:25:10Speaker 01

  Right. Do a breakthrough.

• 1:25:12Speaker 00

  Yeah, do a breakthrough.

• 1:25:14Speaker 01

  Do a breakthrough. That's the prompt.

• 1:25:17Speaker 01

  And it goes and does it.

• 1:25:18Speaker 00

  Of course, you never know from a company perspective, there's a lot of incentive to sort of sell it as that. Even though there might have been a lot of intermediate steps that led, in any stages of the training, or maybe some things left unsaid, even though the final prompt is do a breakthrough, there could be a lot of groundwork before that that led to, we don't know.

• 1:25:42Speaker 01

  So I remember that was a Friday because I just was completely, like, my mind was completely exploded. And it was Labor Day and we just took the kids to the beach. And I needed a few days of, you know, I'm not checking my phone. I'm not on Twitter. I'm just throwing, you know, throwing my kids into the ocean and.

• 1:26:00Speaker 00

  Did you have that moment where you're just sitting and looking at the ocean and contemplating?

• 1:26:04Speaker 01

  Like, what does it all mean?

• 1:26:06Speaker 00

  Yeah.

• 1:26:07Speaker 01

  Yeah. So I had already gone through that a month before.

• 1:26:10Speaker 00

  Yeah.

• 1:26:10Speaker 01

  And I was like, I, so now what that says is that auto-formalization is basically solved. It's not solved in the way that I would want it to be solved. I want, and people are still going to work very, very hard on this. There's all kinds of other things that happen, just like there's all kinds of other things that happen when you prove theorems that are byproducts of that process for human understanding. There's all kinds of other things when you work on making a beautiful library of mathematics. And that process will continue. That process will be very difficult. It's going to be the, you know, thousands of man-hours, people around the world contributing in this open source way. But in terms of can you get a Sari-free proof of something that human beings have written down on paper?

• 1:26:53Speaker 01

  I think that problem is done.

• 1:26:54Speaker 00

  And so you're standing there with a whiskey, having gone through this a month before, and now again.

• 1:27:00Speaker 01

  Right. This was round three. Because when I got back from Labor Day, it was Navier-Stokes.

• 1:27:05Speaker 00

  Navier-Stokes.

• 1:27:06Speaker 01

  But this, this one. I was just mourning if F L T, you're going to do this to me? Come on.

• 1:27:10Speaker 00

  But now, I mean, this really hit home, right? With the quasi RH. So maybe can you, can you speak to the importance and the beauty of the primes? And also how big and hard the leap is from quasi RH to full RH?

• 1:27:27Speaker 01

  Yeah, so the primes, the reason the primes are so valuable is that they are the fundamental particles in the theory of multiplication. Every whole number is broken down into a product of primes. You know, we know about the elementary particles in physics and all of matter is made up of that. In the same way, the primes are the particles that make up the multiplicative structure of whole numbers. And their distribution had been a mystery and remains a mystery. Even the Riemann hypothesis doesn't solve everything we want to know about prime numbers. But the Riemann hypothesis was so beautiful because it pointed the way to what the distribution should be. And it has 1,000 applications in very far away fields. If only the Riemann hypothesis were true, then we would know X, Y, Z.

• 1:28:12Speaker 02

  So let me, let me interject a little story that came up last week, which was pretty entertaining. So we had, Grant Sanderson was visiting IES and we had a long discussion about what is AI doing to the math community? And I really enjoyed this comment that Camilo de Lealis made, which was, I'll do my best impression. I can't do the accent.

• 1:28:31Speaker 01

  Yeah.

• 1:28:32Speaker 02

  But there was a play written in the 1950s by mathematician. So who knows that mathematicians are writing plays? But apparently this play, the premise of it was that von Neumann died and he went to heaven. And he met God and he got to ask him a question. So he asked him, is the Riemann hypothesis true? And of course, everyone holds their breath as they're waiting to hear the answer. And God says, yes. So finally, everyone can exhale. And then von Neumann asks, but how do you know? And God thinks and says, well, I know where all the zeros are and they're all in the R equals a half line. So my understanding is the premise of the play is that that's not enough for von Neumann.

• 1:29:14Speaker 02

  He gets a little bit agitated to try to find out more about why it's true. And I think he ends up at the end in the other place.

• 1:29:22Speaker 01

  That's great.

• 1:29:23Speaker 02

  So I don't know what draws a moral from it, but.

• 1:29:27Speaker 01

  Yeah.

• 1:29:27Speaker 02

  It seems to speak to the mathematical experience.

• 1:29:30Speaker 01

  Math is not the fact, it's the why. It's always been the why. It's our deepest quest for understanding. And if you just tell me a fact is true, but I don't understand why it's true, that has contributed zero to my knowledge.

• 1:29:43Speaker 03

  So primes, I think, are also interesting because they relate to things that a three-year-old can understand. Like fundamentally, we know how to add one to numbers. And, you know, there's this breakthrough moment when you're a kid, I can add one forever. That's a big, a big idea. I can keep adding one forever. Then you learn how to multiply and group, and that's also a big idea. And primes are interesting to me and have always been interesting because we don't understand how adding and multiplying work together. The two most fundamental, basic, simple ideas in all of mathematics, we don't know how they work together. I can add one to a number that I can factor and I have no idea what happens. I know if it's even, it becomes odd and vice versa.

• 1:30:28Speaker 03

  But I don't know if a number that has many, many composites becomes prime. We just fundamentally don't know that simple, you know, adding one, how that changes multiplication. And primes are basically, you know, this like, the distillation of that three-year-old mystery. And it's a mystery that all of us still wrestle with. You know, that would be the, the interesting bit is if the proof told me something fundamental about that structure that I could actually use.

• 1:30:56Speaker 01

  Yeah, to the, the three-year-old, I think my, my older one was something like two or three. He was playing, we had these little blocks, you know, cubes that kids like to play with.

• 1:31:04Speaker 03

  Yeah.

• 1:31:04Speaker 01

  And there was like 20 of them in the pack. And so he would make like a little four by five grid and he liked that. And he would make a 10 by two grid and that was kind of cool. And then one day I look over and he's like really frustrated. Like, what's going on? He's like, you know, it's not, it's not working. And one of them had rolled under the couch.

• 1:31:23Speaker 03

  Yeah.

• 1:31:24Speaker 01

  And he only had 19 and he couldn't figure out a way to make a nice looking rectangle out of the 19.

• 1:31:28Speaker 03

  Yep. It still bothers me today.

• 1:31:30Speaker 01

  Yeah. So absolutely, kids love primes, kids can discover primes.

• 1:31:36Speaker 02

  And so the solution is to lose another, right?

• 1:31:38Speaker 01

  Well, exactly, yeah.

• 1:31:40Speaker 00

  Do you think, so the leap between quasi and RH? I presume you think humans will not, in your lifetime, be able to solve without AI's help? The full Riemann hypothesis.

• 1:31:51Speaker 01

  I don't know.

• 1:31:52Speaker 00

  Can AI, I guess, is the follow on question. Like, how hard is it?

• 1:31:56Speaker 01

  I don't know. I don't know. I did not think I would see a proof of quasi RH. Now that I see what the proof is, and there are a lot of technical details that I still have to go through for my own satisfaction. There's a Lean formalization. The Lean statement was already in the library, was already in MATLAB. So the fact that they have a sorry free proof gives me 100% confidence that the proof is right. And moreover, having read the proof, you know, at a high level, the ideas that it's using are all completely valid. They're brilliant, you know, to embed the Mobius function twisted by sextic characters. This is all stuff that's the bread and butter of analytic number theorists that was chained in just the right way and was absolutely incredible.

• 1:32:37Speaker 01

  And so what Riemann showed is that, remember, we were talking about this function that has complex inputs and has these outputs. And we're worried about when the output is zero. So he showed that there's what's called the critical strip, which is a vertical infinite strip in the complex plane, where the real part is. So you can think about the X coordinate lies between zero and one, and the imaginary part, the Y coordinate, can go arbitrarily high, arbitrarily low. And the hypothesis is that the zeros all lie right at the dead center of that strip, where the real part is a half. That's exactly what Anker was just saying. And what we knew until Tuesday was region where there are no zeros of the Zeta function. That's like getting thinner and thinner, the higher up you go. It goes towards the one line.

• 1:33:21Speaker 01

  The one line is something called the prime number theorem, which tells you the basic distribution of the prime numbers is equivalent to the fact that there are no zeros of this function on the line, real part of S equals one. So if the X coordinate is one, as you go up and down that line, you will never find a value where, when you apply the Zeta function, it hits zero. Okay, so we had the zero free region that was getting thinner and thinner, the higher up you go. And that gave us the best known distribution on the prime numbers, the best error term in the prime number theorem, which I can go into a few. You tell me how much math you wanna, you wanna talk about.

• 1:33:56Speaker 00

  Let's go.

• 1:33:56Speaker 01

  But what I could have anticipated is that it would thicken that line a little bit. As you, you know, it would still be going to zero, the thickness would still be going to zero as you go up the imaginary axis. But it would be better than what we currently know. That thickness hasn't moved in 70 years, but I could sort of imagine that it could maybe make some progress. What it did is it gave us a vertical strip. So there are no zeros with real part between seven eighths and one, an entire infinite vertical strip. That's a quasi Riemann hypothesis. And if you can get that seven eighths down to a half, that's the full Riemann hypothesis. Now, the methods that it used in the seven eighths proof, and there's a second proof that's about 11 twelfths, never mind these technical details.

• 1:34:38Speaker 01

  That method sort of fundamentally can't get down to one half.

• 1:34:42Speaker 00

  So a new idea is needed.

• 1:34:43Speaker 01

  You need a new idea, but quasi RH needed lots and lots of new ideas, and it came up with that.

• 1:34:49Speaker 00

  Is your, having gone through this three times already, is your intuition, this is for all three of you, about remeasuring in your mind the hardness of the various problems? Cause you mentioned like picking easy things, cause that presumably you have a good hardness radar. Like this problem is probably solvable, and this is not solvable today, solvable this week, solvable this month, you know? Are you readjusting to this new world?

• 1:35:17Speaker 02

  I'm readjusting my inability to predict it.

• 1:35:20Speaker 00

  Yes, exactly. Well, you had the ability, no? It's noisy.

• 1:35:25Speaker 02

  Sure, it's a larger amount of noise than I thought. I mean, I think we've always known that there are problems we care about that are within reach of the tools that are out there right now. But it's just surprising the scale at which, you know, how many different problems really fall into that category? So we also don't know a lot about, you know, apparently they prompted thousands of problems. And what they dropped were only the positive solutions, that they only had about a 10% success rate. So we're sort of left guessing, you know, is there a very jagged boundary that we could squint at and say we can make sense out of? That distinguishes, you know, what did fall versus what didn't fall?

• 1:36:10Speaker 02

  I think one of the other dangers is that maybe the problems which are the safest are ones which you look at and it's just painfully obvious. You need a really creative new idea, you know, because you can sort of tell that maybe there's a barrier and the things that are out there in the literature just don't seem to fit into that whole. But now I think one of the things that's changed the landscape of math forever is, you know, maybe people will become a little bit more secretive with those ideas than they were in the past. You know, because once those ideas are out there, then, you know, it might be a lot more straightforward for AI to be able to balance that one new idea, that missing ingredient, that new tile it needs. And be able to fully exhaustively explore what it can reach out of that.

• 1:36:54Speaker 02

  So you might be a lot more protective of your ideas than you were in the past. I think that's also a bit scary because traditionally, the mathematics community has been really open with sharing ideas. And that would be something bad to lose.

• 1:37:08Speaker 01

  But that's also because we had this first through the gate mentality, and I think that's also gonna change.

• 1:37:13Speaker 02

  That's right.

• 1:37:14Speaker 01

  I would prefer that the openness stay and the first through the gate is what goes.

• 1:37:18Speaker 02

  That's right, but that'll take time.

• 1:37:19Speaker 01

  It'll take time.

• 1:37:19Speaker 02

  Yeah, so that could be an effect, you know, that's more transient in nature, that if we can learn how to value other things, then maybe that'll go away. There are other communities that have models for how to value contribution, even when it's not lionizing one individual's contribution. So it's not that there's not a way forward, but you know, that is in the short term, a real roadblock to doing mathematics the way that we used to. And enjoying it the same way that we used to.

• 1:37:46Speaker 00

  So speaking of which, Steve, you told us a story about soup that made you maybe fall in love with the bigger problem of fluid dynamics. So going from math to applied math to engineering, the full thing. So maybe can you talk about Navier-Stokes? These set of equations that explain fluids, why is such a big deal? Why it's beautiful, similar as we talked about primes? Why is it so hard? And why is it such a big deal that Navier-Stokes was proven?

• 1:38:18Speaker 03

  So fundamentally, the Navier-Stokes equations is just a way of stating the conservation of momentum for fluids, for objects that can move kind of freely, like gases and liquids. So we know F equals MA is conservation of momentum, you know, classical Newtonian physics. The same basic principle applies to the gas in this room, to water flowing, and it's true in any volume. So it's true in little volumes, big volumes, kind of imaginary volumes. And if you write down that conservation of momentum in a volume, and then you kind of take the limit as that volume becomes arbitrary and small, you get these Navier-Stokes equations.

• 1:39:01Speaker 03

  It's a differential equation, a partial differential equation, that describes how the velocity field of a fluid changes in space and time. And also the pressure related to the forces acting on that fluid.

• 1:39:14Speaker 00

  Can you give intuition about why fluid, given that F equals MA is a pretty simple equation? Why fluids are such complicated systems?

• 1:39:25Speaker 03

  F equals MA is simple to write down, but it can get complicated really, really quickly. So if I have like a single pendulum, it's really easy to write down the equations for that single pendulum. If I put, you know, another pendulum on the end of it, now I have a more complicated equation and it gives chaos. Fluids is, you know, in a sense, if I, if I try to model the air in this room, I could chop the room up into a billion little boxes. And write down a little differential equation for how the momentum works in all of those billion little boxes. But now I have a really complicated system of equations. And so it's simple to write down, it's simple to conceptually understand what the equation is saying and representing.

• 1:40:07Speaker 03

  But actually solving it and understanding how the fluid actually moves in any kind of practical sense is very, very complicated.

• 1:40:15Speaker 00

  I read something like, you know, like one, one space where fluid dynamics is important is reducing drag for when you're flying for airplanes.

• 1:40:25Speaker 00

  I read something like airlines could save like something like tens of billions of dollars by reducing the drag by 1%. So why is it so hard? So that's one. And drag is maybe easier than something like turbulence. So like, why is, why is these things that are worth so much money so difficult?

• 1:40:44Speaker 03

  So the good news is, is that if you can improve our ability to predict turbulence, to simulate it even a little bit better, faster, more accurately. There are, you know, dozens of fields where you can have a huge impact, both economical, ecological, human health and safety with a relatively small improvement. So drag reduction on airplanes, better artificial hearts, you know, more efficient wind turbines and gas turbines and, you know, you name it. The, the world is, we live and work in a fluid and so do all of our machines. The inverse design problem, I, I know it's actually very closely related to this kind of like math problem. I know what I want to prove. I don't know how to get there.

• 1:41:30Speaker 03

  I know I want a wing that has high lift and low drag. But I don't know how changing the geometry, changing the shape of that wing will affect this whole cascade of smaller and smaller and smaller vortices. You know, this turbulent fluid that generates over the wing that eventually results in its lift and drag. I don't know how to inverse design efficiently. Part of it's because the equations are expensive to simulate or to test. You know, wind tunnels are expensive. You need supercomputers to simulate these things. This is actually kind of related to pure math. We'll, we'll get back to this, I think. But there's this really nice Feynman quote that, you know, we know how to write down these very general equations, Maxwell's equations,

• 1:42:15Speaker 03

  Navier-Stokes equations, Schrodinger's equations. But we currently don't know from the equation what phenomena is possible. You know, if you'd never visited Earth, could you have possibly predicted the sunsets and the tides and all of the phenomena, just from like a partial differential equation? Like, we're not even close to there. And there's some very fundamental math questions about, you know, if I can, I can write down a differential equation, I can't tell you what's inside of it. We still have to simulate, we still have to test, we still have to do this very elaborate engineering process to understand how these equations manifest drag on an airplane wing or turbulence inside of an artificial heart.

• 1:42:55Speaker 01

  Sorry, can I just ask a question on that? I've always wanted, so like, in the case of Einstein's theory of general relativity, he writes down these equations. A few years later, Schwarzschild finds a blow up. And they're like, oh, come on. So what's, what's wrong with these equations that there's a blow up? You know, if you get enough matter into a small enough space, then like light can't escape it. This, we don't have any physical experience for this. This has got to be a mistake in the equations. Until we discover black holes. And then it turns out that actually the theory perfectly explains, you know, and predicts what, what's supposed to happen. In the case of blow up of Navier-Stokes, are we going to discover something new about water? Or is the fact that at some scale you get to particles?

• 1:43:38Speaker 01

  You know, you get to an atomic structure that does not follow the liquid equations anymore. Why, why would we have predicted that there should be solutions to a continuous process that's allowed to be pushed to, to its limits?

• 1:43:52Speaker 03

  Yeah, it's a great question. I'm going to give you the engineering answer, which is these equations are an exceptionally good model of how fluids work at a certain scale. And you're absolutely right. Like you need some number of individual molecules to be averaged over in a volume for these to be constitutive equations that are valid. And so if you zoom in small enough, you know, now we're talking about individual molecules and Navier-Stokes breaks down. So the mathematical problem of these equations blowing up is, is interesting from a mathematical standpoint. By the time they blow up, it stops being a fluid.

• 1:44:33Speaker 01

  There's no black holes.

• 1:44:34Speaker 03

  There's no black holes, like other physics takes over.

• 1:44:37Speaker 00

  Yeah, but it's possible. Wait, I, I think it's interesting from a philosophy of science perspective that where equations break down could, and maybe statistically high likelihood, is a window, a door in, in a breakthrough of needing some other discovery. Like Newtonian physics breaking down needs Einstein.

• 1:45:00Speaker 01

  Right. And the grand unification is that we don't have gravity at the same scales as we have, you know, interactions of subatomic particles.

• 1:45:08Speaker 03

  But, but this idea that we don't know something so fundamental about these equations, there are so many people on Earth right now simulating and testing Navier-Stokes in every industry imaginable. I mean, it's, it's probably one of the most studied and analyzed and used equations in the entire engineering world. And we don't know this fundamental property of the PDE, does it blow up in finite time or not? That's like the zeroth order version of the bigger question is, you know, what other kinds of solutions does it admit? Does it, you know, are there sunsets and hurricanes and tornadoes? All the phenomenology that, that could exist, you know, existence and uniqueness is like the, the simplest baby step.

• 1:45:53Speaker 03

  And that's where it gets interesting to me is even if there's not a direct tangible use of this mathematical result in an engineering sense, it gets us that one step closer to understanding the nature of these equations and what is possible, which is very fundamentally interesting to us.

• 1:46:10Speaker 01

  Could there be, could this solution that, that has the blow up be used somehow in an engineering application? And, you know, you want to drive it into this state and get some funny behavior that helps airplanes fly with less drag or something?

• 1:46:23Speaker 03

  Yeah, I mean, I think in principle, a huge industry, a huge part of what we try to do is manipulate fluid flows, like bend them into our will, right? So if you watch a, a bird fly or an insect fly, they are shaping vortices and carefully funneling, you know, energy into structures for their, you know, objectives of their daily life to like perch on a rock or, you know, hunt. And we try to do the same thing. So flow control is, is one of the highest challenges. It's very, very challenging.

• 1:46:56Speaker 03

  But all of our science fiction technology would be kind of accelerated if we had better ability to control these flows. And this, this proof is one very interesting construction of this kind of, you know, very specialized flow control in a way.

• 1:47:12Speaker 01

  I'm reminded that Feynman was asked about intelligence, artificial intelligence in the seventies. And he said something like, the way that we solve problems with technology is radically different from how nature solves these problems. Look at birds, how birds fly and how airplanes fly. He was exactly pointing to this. And it's kind of interesting to think back to that comment now when we see what LLMs can do with theory improving versus what human brains can do with theory improving and all kinds of other applications. But it's, it's interesting that he pointed exactly to this issue of how, how we've solved flight using technology and how different that is from nature's solution to it. Our airplanes don't fly.

• 1:47:51Speaker 02

  Now I wonder if birds have existential crises when they see planes flying.

• 1:47:56Speaker 01

  Exactly.

• 1:47:56Speaker 02

  What am I even here for?

• 1:47:57Speaker 01

  Yeah.

• 1:48:00Speaker 01

  To get the worm.

• 1:48:02Speaker 00

  What, what are the different components of the Navier-Stokes equation? Can you, if you, can you dig in? Like what, what does it take to describe fluid?

• 1:48:12Speaker 03

  Yeah, so very, very basic. A fluid moves and it moves differently in different places and at different points in time. And so the kind of basic building block of a fluid is the velocity field. So at every point in space, there is a velocity vector that tells you how that, that fluid is moving. You know, maybe it's really fast here and slow here and it creates a vortex.

• 1:48:40Speaker 03

  And it gets more interesting when the fluid velocity changes a lot over space. So like a tornado is a very rapidly changing fluid velocity field in both space and time. There's pressures that are involved. So again, like the core of a tornado is low pressure. So pressures are kind of forcing this velocity field and causing it to move in the way it does. That's the building block is a velocity field and it's pressures. And then the equations tell you how that velocity changes in time and in space according

• 1:49:21Speaker 03

  to kind of its own momentum. So fluid has momentum to carry itself. That's, you know, gusts of wind. It has internal friction and resistance. That's viscosity. So honey has more internal resistance than water, has more internal resistance than air. And then all of the external forces like heating and pressure and gravity and things like that. So each of those can be quantified as a term in this, this larger Navier-Stokes equation.

• 1:49:49Speaker 00

  So speaking of external forces, can you describe the math side of this, the Navier-Stokes with the blow up? What was the formulation of the problem? And what was actually proven? And why was it so hard?

• 1:50:01Speaker 03

  If you don't have that internal friction, that viscosity, we have a strong suspicion that the equations can and will blow up under certain conditions. We have other examples, kind of simpler problems that are like Navier-Stokes that do blow up. So the simplest one is just a simple, ordinary differential equation, if you don't mind me taking a little tangent.

• 1:50:24Speaker 00

  Let's go.

• 1:50:25Speaker 03

  So x dot equals x squared. X is the state of my system. X dot just means the rate of change of x in time. And if that rate of change of x is, is growing proportional to x squared, that differential equation does blow up in finite time. So it just takes a quadratic nonlinearity. Think about if I have, you know, x dot equals x. This is like x could be the population of bunnies. And we know bunnies like to populate. X dot equals x grows exponentially. You know, the, the state of my bunny population in time will grow exponentially. It'll get bigger and bigger and bigger. It'll double and double and double. But it won't reach infinity until infinite time has passed. That's just

• 1:51:09Speaker 03

  a normal exponential growth. That's not finite time blow up. That's x dot equals x. But if I take x dot equals x squared, that blows up in finite time. So that quadratic nonlinearity, that quadratic kind of feedback is enough to cause this equation to blow up. And you can like, if, if you write this down, you know, you can solve this using kind of classic, basic calculus techniques and you get this, this, this finite time blow up. And so there's a progression of partial differential equations that have the same behavior where a quadratic nonlinearity, and that's really interesting, is the Navier-Stokes equations, one of the terms, the convective term that tells how the fluid kind of moves itself with its own momentum, is a quadratic nonlinearity, just like x dot equals x squared.

• 1:51:54Speaker 03

  It's kind of this self-motion. Without diffusion, without viscosity, we think that that should probably blow up to infinity in finite time, have this finite time blow up. But we don't know if viscosity, if the kind of, you know, internal friction and resistance like honey has, always damps out those blowups before they can happen. My intuition as an engineer is, of course, they do. Like if I have a, a flow where it's, you know, accelerating, you know, to infinity, that should immediately diffuse into its neighbors. It should create this like diffusion into its, its neighboring fluid and kind of tamp down that, that growth.

• 1:52:37Speaker 03

  And so I think the math problem is essentially, are there any edge cases, any initial configurations of the fluid where you perfectly orchestrate this thing so that it could actually accelerate to infinity in finite time? And then there's two kind of technical versions of this. One is with no external forcing, and one is with smooth external forcing. Smooth being important because if the external forcing has singularities, then it's kind of trivial. Like if I force it with a tornado, you know, an infinitely tight tornado, then of course it's going to blow up.

• 1:53:10Speaker 00

  The problem as stated has smooth external forcing. There's four versions of the

• 1:53:17Speaker 03

  Yes.

• 1:53:17Speaker 00

  on, on the Clay, yeah, website, one of them being the, the forced version. And that's the one that was proven.

• 1:53:22Speaker 03

  And that's the one that was proven.

• 1:53:24Speaker 00

  Exactly. What was proven? And what are some interesting things about the proof that you saw?

• 1:53:29Speaker 03

  Yeah. So what is proven is that there is this way of orchestrating a smooth kind of external forcing that perfectly cancels out various terms in the equation. So that you get this vortex. This can only happen in three dimensions. This is really important.

• 1:53:48Speaker 00

  We should probably say that we've been talking about three dimensions. There's a big difference between 3D and 2D.

• 1:53:54Speaker 03

  Yeah. So I mean, when we talk about a fluid, we're, we usually imagine it in 3D, but there are these kind of analogies or simplified 2D flows. Weather on Earth is pretty well approximated as a 2D flow.

• 1:54:05Speaker 00

  Yeah. You've mentioned somewhere in a talk that I heard that hurricanes are kind of like modeled well in 2D and tornadoes are 3D. And I remember, I remember different sort of analogies that you were drawing there.

• 1:54:21Speaker 03

  Yeah. So, so one of the interesting phenomena in three dimensional turbulence is you generally have this, this breakdown of big vortices into smaller vortices into little itty bitty vortices until eventually they molecularly diffuse into heat. That's kind of this three dimensional turbulence cascade. In two dimensions, like weather on Earth, you actually get a reverse cascade, which is why hurricanes form. You actually have little vortices coalescing into medium vortices, coalescing into bigger vortices. But so the AI proof of this kind of counter example that you can have finite blow up basically creates this very, very specialized vortex that in the third dimension is stretching

• 1:55:04Speaker 03

  out and thinning it so that it can essentially become infinitely thin. The radius gets smaller and smaller and smaller as it stretches out. Kind of like a figure skater doing a spin. You pull your arms in, you go much, much faster. And there's this very specialized kind of rate of contraction of this vortex so that the energy in the fluid still remains finite. But if this localized filament, the flow kind of blows up and you get infinite vorticity. I was surprised when mathematicians before the AI proof started thinking it was going to be possible to find this counter example.

• 1:55:44Speaker 00

  Which only started maybe 15 years ago.

• 1:55:46Speaker 03

  That's right. It's like relatively recent in the grand scheme of things. Yep.

• 1:55:49Speaker 00

  Instead of looking for a solution, looking for a counter example. Yeah.

• 1:55:52Speaker 03

  Because all of our physical intuition tells us diffusion does solve the problem. And so it is like very physically counterintuitive.

• 1:55:59Speaker 00

  In the mathematics world, it's a really hard problem. Impressive to construct a counter example. What does it teach to you as an engineer who stared and studied at a lot of different natural systems or, that are fluids? And maybe take that as a step towards what are the equivalent kind of hardness and harder problems yet to be solved in fluids?

• 1:56:28Speaker 03

  So it's a very important math result.

• 1:56:30Speaker 00

  Uh oh.

• 1:56:31Speaker 03

  I'll say like, but day to day engineering fluids and applied math, kind of analysis of fluids is almost completely unchanged. Fluids are just as hard to simulate and predict today as they were a month ago or two months ago. And so when I'm trying to actually solve an engineering problem with a fluid, I want to design an airplane wing or an artificial heart or whatever it is, I'm still having to simulate and predict and manipulate this flow in a very practical way that's expensive in terms of supercomputer time and wind tunnel resources. And the, this mathematical theorem and proof are very interesting, but it operates

• 1:57:15Speaker 03

  at a regime that's so non-physical that it doesn't really affect my day to day engineering. There's still, I don't know, this is very pithy, but I like to say, you know, we solved the million dollar math problem. There's a trillion dollar engineering problem, which is like, how do you predict these things in a fast, efficient way and do this inverse design? That's a much harder problem.

• 1:57:37Speaker 00

  And you say it's a trillion dollar because that literally,

• 1:57:40Speaker 03

  Literally.

• 1:57:40Speaker 00

  if you solve some of these problems, that's how much money you'll make. Not because it's a prize, but like there's industries.

• 1:57:46Speaker 03

  Yeah. And, and we could say, you know, trillion dollars of possible benefit to humanity of like better, faster solutions, lower drag, less emissions, you know, all, all of it. It's, we have like transportation, defense, health, energy, all of these have, you know, fluids at the very core technology. And so I think those problems are still very hard. And this math result is important in, in the sense I was talking about earlier that we don't know the fundamental character of what is possible in these equations. We don't even know if they, if the solutions exist. That's like so fundamental. This is

• 1:58:26Speaker 03

  like a baby step towards a much bigger human goal of understanding what's in an equation in a more practical sense.

• 1:58:34Speaker 00

  Can you speak to some of the maybe beautiful, fascinating phenomena in the space of fluids that you've been thinking about? You mentioned turbulence. What, what are the big puzzles to, to sort of solve in this space to understand? You've mentioned drag, like air, fluid. What are like the big millennial prize type problems that people are thinking about?

• 1:59:00Speaker 03

  I mean, there's so many. You can break them down into a few categories. So turbulence is very hard because it's multi-scale.

• 1:59:06Speaker 00

  What is turbulence?

• 1:59:07Speaker 03

  Turbulence is this, this phenomena that when you have a complex fluid roiling or mixing and moving, kind of like Da Vinci's, he has these beautiful drawings of fluids at all of these different scales. All of the scales matter. The big vortices and the medium vortices and the little vortices, all of them actually matter to predict the lift and the drag and the thing. So the gas in this room is a great counter example. It's the opposite of turbulence. It is complicated. The, the molecules are all moving around in a very frenetic way. And there's a lot of molecules.

• 1:59:48Speaker 03

  But that, you know, 10 to the 23 number of gas particles, all of those individual configurations don't matter. We know from thermodynamics, this is like the great success of classical physics, is that there's a small handful of numbers that actually determine how this gas behaves, you know, temperature, pressure, density. That simple description doesn't exist for turbulent fluids, for the flow over a wing, the flow inside your heart, for, you know, for the flow on a river. All of the scales actually matter. All of the numbers in that simulation actually matter to predict the next, you know, behavior, to predict what's going to happen. And that makes it fundamentally challenging. So this is actually a great AI problem is,

• 2:00:32Speaker 03

  is there a coordinate system? Is there a set of numbers, just like we have for gases, you know, temperature, pressure, density? Is there a set of numbers or some representation of a fluid, of a turbulent fluid, that allow it to be simply described in a way that is useful for engineering to build, you know, motors and engines and just like we have for, for gas dynamics?

• 2:00:53Speaker 00

  So then the interesting question, if it's a problem for AI and AI is listening, I'm sure.

• 2:01:00Speaker 00

  The solution to that problem, would it, would the path lie through mathematics? Or through having AI system sit there and watch some fluid for a while?

• 2:01:11Speaker 03

  I think yes to both.

• 2:01:13Speaker 00

  Cause fish, they figured it out.

• 2:01:15Speaker 03

  They figured it out. Well, so I think you, you, you have the right keyword as coordinates. And I, and I, so that's kind of why I said both. And I'm not trying to, to be like trite, but all of our, our great physics breakthroughs were predated by a coordinate transformation. So Newton's F equals MA came after we put the sun at the center of the universe instead of the earth. Right? You had to do that coordinate transform first. So I think what, the way I see it is we, we need to take all of this data, these simulations, these observations of, of fluids. And what we're looking for is a simple coordinate system

• 2:01:54Speaker 03

  so that the description of that fluid becomes very simple and generalizable, beautiful in a physical sense. And that is both a brute force, you know, get a lot of data, try a bunch of different coordinate transformations and, and representations until you find the simplest one that generalizes the best. But there's also a lot of deep mathematics in what the form of that coordinate transformation might take. In general, if you find a question that really smart people asked, you know, in the seventies and eighties, but they hit a wall computationally, that's very fertile ground right now. So like, we know what the form of these, these coordinate systems probably should take from some fundamental principles.

• 2:02:40Speaker 03

  We've just never been able to write them down. They're too complicated to write down. That's like a perfect problem for AI is, you know, now let's give it a lot more data, a lot more compute, and see if we can start learning these, these transformations we know should exist.

• 2:02:55Speaker 00

  Just so I understand, the formation of vortexes. So that's like a, like, little like tornadoes.

• 2:03:00Speaker 03

  Little tornadoes.

• 2:03:02Speaker 00

  Their formation is, is that part tricky to understand?

• 2:03:06Speaker 03

  Yeah, absolutely. And it's beautiful too. Like I love watching these videos of, you know, if you shoot two vortices together, they'll, you know, influence each other and then merge in these very interesting ways. You have like a red vortex and a blue vortex and you can see where they merge and join. But they also stretch and fold and, you know, turbulence is chaotic. So the trajectories get tangled like spaghetti. It's not random. There's a huge amount of structure, but it is very multi-scale, very, very tangled.

• 2:03:42Speaker 02

  So one of the things I would wonder, you know, scientifically, I mean, these are really interesting problems and there was a great explanation. So I'm always curious about what computational problems nature can solve already, right? And there are a lot of examples of this, you know, there's a famous one with like slime mold, finding shortest paths and mazes. In fact, some of the precursors to Navier-Stokes were, you know, like some of Terry Tao's work on showing that you can have blow up for Navier-Stokes-like equations because they can simulate Turing machines. So I guess I would also wonder what sorts of computational tasks, these computations with vortices colliding can do?

• 2:04:23Speaker 02

  And how that imposes limits on what sorts of coordinate systems ought to exist? Cause you know, you mentioned PV equals NRT. That's an amazing dimensionality reduction. Because you don't keep track of all of these positions and velocities of particles. But there's just, you know, a small list of numbers that describes a meta-state and that's it. But if I really had a tremendous reduction, you know, like that in the case of turbulence, then it would seem to me very difficult to, to compute very interesting things too.

• 2:04:59Speaker 01

  Could it be like the game of life where you have this very simple cellular automata system? And it's a universal Turing machine and you can cook up whatever behavior you want in it?

• 2:05:09Speaker 02

  Yeah, that's, that's right. I mean, so game of life, a lot of times people think about, you know, what are minimal rules to have all of these exotic behaviors? And it's really shocking how small the rules need to be to get that, right? But I guess, you know, I'm a theoretical computer scientist. One of the biggest exports from our field is really the algorithmic lens. So it's not just about, you know, computation where you have a well-defined problem and a laptop and you want to solve it. But it's also about, you know, computation in the natural world. You know, and what that teaches us about what the rules of nature might be. So, you know, some famous examples of this are things like equilibrium and multiplayer games. And Nash equilibrium is a great solution concept.

• 2:05:51Speaker 02

  But of course, the fact that it's hard to compute Nash equilibrium, you know, throws into question, is it really a universal mechanism? That explains how people play games? There was this amazing quote by Kamal Jain that I really like, which is, if my laptop can't find it, then neither can the market. And, you know, to me, what's really exciting is the nexus of a lot of these different perspectives. You know, when you were describing it, it sounded to me like they're very interesting computational problems under the hood. And that one can kind of get a dual perspective on these things by thinking about how that computation imposes limits on what a good coordinate system ought to look like.

• 2:06:32Speaker 03

  So I think that's a really nice way of framing the problem. And I would, I would take two extreme examples. So Earth's weather and climate systems are actually just complex. They, all the scales do matter. The weather in this zip code matters to me. You know, everything matters, all the variables matter. And, and you're right, like if I, if I over reduce that system to, you know, five numbers, then I lose the descriptive capability of what I actually care about. That's, that's one extreme. The other extreme, let's think about how, you know, a moth flies or a butterfly or a fruit fly. They are proof by existence

• 2:07:18Speaker 03

  that you can interact with an extremely complicated turbulent flow and do very advanced engineering control. They can control their forces and moments in gusts, in rain. They can live very rich lives and they presumably don't have a PhD in fluid dynamics or know the Navier-Stokes. They're probably not running supercomputer simulations. There might be five or six numbers that actually matter to their daily life of maintaining a leading edge vortex and, you know, stalling their wings to perch and to land. And those are kind of two extreme examples. And I, I really want to be able to handle both of those. We want to be able to handle the full complexity of truly complex systems.

• 2:08:04Speaker 03

  And we want to be able to distill down the absolute bare minimum essentials for fastest decisions in those kind of insect extreme cases. And biological flyers still dramatically outperform, you know, human-designed fluid systems in many regimes. So we still have a lot to learn about, about these, these, these simple.

• 2:08:30Speaker 02

  So maybe we're like veering into research now, but you know, to me, it's always amazing how nature not only solves very hard optimization problems, it seems to find very robust solutions. And I'm always struck by the fact that, you know, when we model it, we tend to model it assuming we have full knowledge, you know, about what the boundary and initial conditions are for things like Navier-Stokes. But when a bird flies, you know, it doesn't know all of that. And to some extent, you know, modeling things with uncertainty is really how you reframe the optimization problem to get very, you know, rich and robust solutions. That's kind of amazing to me how a lot of everyday examples sort of fit into this. I remember, you know, I'm a big baseball fan and I remember reading a math paper about baseball

• 2:09:15Speaker 02

  that when you've got an outfielder, you know, once the ball has been hit, a lot of stuff is known, you know, the velocity, the trajectory. You would think the outfielder would just take a straight line path to where they expected the ball to be. But when you observe outfielders, they don't take straight line paths. They take a slight curve to it. So why is that? You know, at least one way to wrap your head around it is that that's actually a very robust strategy. Cause when you're taking a curved path, you can continually reevaluate the trajectory of the ball and then adjust as needed. So there are very interesting solutions that nature finds. But it's not always obvious what is the right formulation for the optimization problem in the first place.

• 2:09:59Speaker 02

  And sometimes modeling it in the case of uncertainty so that you can design solutions that are robust to the fact that you don't have full system knowledge seems to be an important piece of the picture.

• 2:10:10Speaker 03

  So there's a fun aside related to this. Insects in particular don't have the ability to measure their full flow field. They can react to gusts of wind, gusts of wind faster than the information that hits their wings, goes to their brain and back to their muscles. So researchers have studied this for decades in biology. And what they've found is that there are strain sensitive neurons embedded on the wings, maybe a few dozen to a few hundred of these in very stereotypical locations. So brother fruit flies will have the same locations of these strain sensitive neurons.

• 2:10:53Speaker 03

  And those strain sensitive neurons fire when the wing bends in a certain temporal pattern. So it gets hit by a gust of wind, the wing bends, neurons fire. And a local computation is happening in that shoulder muscle of this insect that goes directly back and corrects robustly based on partial information of a system it doesn't have equations for that is nonlinear and turbulent. So it's like, again, proof by existence of exactly what you're saying is maybe the robust strategy of actually relying on partial information and uncertainty. They can do more than we can at this point with way less information. Yeah.

• 2:11:31Speaker 00

  I have a couple of, I apologize, pothead thoughts. Cause Alex mentioned black holes. I have a fun thought that maybe the singularity observed with the blow up in Navier-Stokes equations comes into play when we're, as opposed to airplane wings on earth, we're traveling through two connected black holes in a warp drive that we figure out. And then there's the flow control of spacetime. And that becomes relevant because now you're in a black hole, hence the singularity. This is the, for people just listening to this little look Alex just gave me, it hurts my soul.

• 2:12:08Speaker 01

  Okay.

• 2:12:08Speaker 00

  So moving on to the next topic.

• 2:12:09Speaker 01

  But the forcing, you'd need the black hole vortex to also be forced it exactly the right.

• 2:12:14Speaker 00

  Yeah, the forcing, where's the force? I mean, at that point.

• 2:12:17Speaker 01

  You could try to ride it. Yeah. There's a, you can make something out of this.

• 2:12:20Speaker 00

  There's some non-zero probability that future physicists will listen to this. That guy is the one time he said something that was actually correct. Okay.

• 2:12:32Speaker 00

  If it's okay, a quick bathroom break.

• 2:12:33Speaker 03

  Yeah, sure.

• 2:12:34Speaker 01

  Sounds good.

• 2:12:34Speaker 00

  Cool. Encore, one of my favorite topics is theoretical computer science. You're one of the world experts in many of its problems. So I'd love to ask you about one of the big problems that's been solved in this OpenAI drop, which is the unique games conjecture. Can you give people an intuition about what this problem is, why it's a big deal? And what are some aspects of the proof that are interesting?

• 2:12:59Speaker 02

  Yes, I'd love to. So let me give you a little bit of a background. So let's start off with complexity theory 101. So one of the most important problems in theoretical computer science is the satisfiability problem. So the thing to keep in your mind is imagine you've got variables x1 to xn, and they can take on either true or false values. And what I'm going to do is I'm going to create a big formula out of them. And usually the way we do this is with an and of ors. So I start off with a clause that's, you know, an or x1 or not x3 or x7. So depending on my assignment, that clause might be satisfied as long as at least one of those things is true.

• 2:13:44Speaker 02

  And then I take the and of many of these different clauses together. And the question is, if I give you that formula called a conjunctive normal form formula, you know, can you find a satisfying assignment? So can you find an assignment of x1, x2 up to xn, setting them either to be equal to true or false, so that all of those or clauses are on, so that the entire formula evaluates the true? So this is the canonical example of a problem that's easy to check someone else's answer, but it might be hard to find an answer yourself. So if you told me that it's satisfiable, all I would do is I'd ask you, okay, give me the satisfying assignment.

• 2:14:29Speaker 02

  You tell me what x1 is set to, what x2 is set to, and I just go clause by clause and check to make sure that you really have satisfied everything. Now, the, you know, the central problem in theoretical computer science is the P versus NP problem. So are problems that are easy to check, is it easy to find their solutions to? And the presumption in our field is that, no, it's not easy. And that's the basis, you know, of a lot of things like the cryptography and e-commerce and what have you. Now, really, the starting point for getting to something like the unique games conjecture is something called the PCP theorem. So this was a landmark result proved in the 90s.

• 2:15:12Speaker 02

  It actually was such a big deal that it made the front page of the New York Times. And the crux of it is really this idea that it's even robustly hard to fool a verifier. So you start off with a formula and either it's satisfiable or not satisfiable, right? And what does that mean? If it's satisfiable, you can write down an assignment and I can check one by one that the entire thing evaluates the true. But if it's not satisfiable, that means someplace you're lying. You write down this purported satisfying assignment. And when I go clause by clause, I'm quickly going to find one clause that is not true and that invalidates the whole thing. And I say, aha, I caught you. It's not actually a satisfying assignment.

• 2:15:56Speaker 02

  So the idea behind things like probabilistically checkable proofs, without getting into a lot of the technical definitions, the idea is really gap amplification. So can you start off with a formula that is hard to decide whether it's satisfiable or not? And can you engineer a new, maybe much bigger formula that's robustly easy to check? In the sense that if it's satisfiable, it'll still be satisfiable. You can write down an assignment and it'll be fine. But if it's not, I should actually be able to just flip a few coins, query random variables, check a random clause, and with good probability, find that you're lying.

• 2:16:37Speaker 02

  So the point is that with very sublinear inspection, with only constant amount of inspection, I can find that you're cheating. So this was a momentous result in theoretical computer science because it laid the foundation really for what's called hardness of approximation. So a lot of algorithms and complexity theory is what do you do when you're faced with problems that are genuinely hard to solve? And, you know, one of the things we do is we try to approximately solve them. So maybe we can't find an assignment, if there is one, that satisfies all the clauses. But can we find one that satisfies many of the clauses? How large a fraction of the clauses?

• 2:17:18Speaker 02

  Now, even there, we would expect problems to become computationally hard after a certain point. It should be that it's not just that, you know, deciding whether it's completely satisfiable or not completely satisfiable is hard, but maybe even deciding, is it 100% satisfiable or is it only 99% satisfiable? The best case over all assignments satisfy at most 99% of the clauses. That too might end up being hard to decide. So these are what probabilistically checkable proofs give us. In places where we knew that the optimization problem was hard, they tell us that it's even hard to approximate. And the unique games conjecture is a strengthening of the ordinary PCP theorem because it has

• 2:18:03Speaker 02

  to do with a certain type of label cover instance. It's a bit technical to go into the details. But the point is that if that version of a probabilistically checkable proof were true, it would have this great organizing effect where it would turn out that for a huge swath of problems, we would actually know what is the best approximation algorithm, and it would be something very simple. So when I started off in the field, the unique games conjecture had been posed, you know, a few years before that. And there was still a lot of debate about whether it should be true or false. And, you know, people worked hard to try and give proofs of it. People worked hard to try and disprove it.

• 2:18:46Speaker 02

  There was this amazing time period, maybe in the 2010s, where there was quite a lot of back and forth. And, you know, there would be new results came out that people's opinions would change, whether they believed unique games or not. I like this quip, I forget who said it, so I can't properly attribute it, but someone told me, you know, maybe God's going to decide whether it's true or false based on which side is working harder.

• 2:19:11Speaker 02

  Because it really seemed to change year to year. And, you know, one of the results, one of the headlining results in the OpenAI drop of proofs was that the unique games conjecture is true. Now, I should mention that this is not without spectacular human progress on it. So one of my colleagues at MIT, one of my colleagues at IES, they both contributed, among other people, to a lot of the major strides. So Dor Minser and Iri Denuer, they proved a weaker version called the two to one games theorem. And this really was, you know, shot heard round the world in theoretical computer science because at that point, I think a lot of people weren't sure what was a path forward for unique games conjecture.

• 2:19:55Speaker 02

  And I think their work really gave a viable path forward to something that was in the vicinity of unique games. So it sounded like, you know, maybe humans could get there with another 10, 20 years. But, you know, just when OpenAI released the results, it turns out that the unique games conjecture is actually true. So this is quite remarkable because, you know, at some points, I guess I did find myself on the side of the doubters because unique games just sounded too nice to be true. Because, you know, a lot of work, like Prasad Ragavendra's work, it implies optimal hardness of approximation for so many different types of constraint satisfaction problems that

• 2:20:39Speaker 02

  the conclusion just looked to me too strong to be true.

• 2:20:43Speaker 00

  What did you feel when you found out that night?

• 2:20:46Speaker 02

  Okay, well, for starters, there were already rumors, you know, in the weeks leading up that OpenAI had solved the unique games conjecture and that it was a truth. And these weren't even just completely arbitrary rumors. One of my colleagues, Dor Minser, he actually posted a paper where he wrote something in the footnote at the bottom that, you know, we would have liked to polish this paper more, but because of the rumors that were going around, we feel the need to put out our approach sooner rather than later. So for me, it was in some sense, a bit of a relief that the results were finally out there. Now, they're not very readable papers. It's not like you can point to them and really quickly identify what's a new key idea.

• 2:21:31Speaker 02

  But I'm glad that they're out there because now we can move forward on the next phase, which is digesting those proofs and figuring out where the field moves after that. Now, in terms of how did I feel that night, you know, as Alex mentioned, we, you know, we all had a drink in hand. And then I saw, you know, on my phone, the news that that these proofs had dropped. And, you know, people kept up coming and talking to me at the gala, but it's already hard to socialize as a mathematician, let alone, you know, when you have math you really want to read. So it was doubly hard. The other funny thing about it was, you know, I was in New York already, so I took it as an opportunity before the gala to visit a lot of people I knew at NYU.

• 2:22:16Speaker 02

  And I just happened to be meeting with Subash Kaut, the one who introduced the unique games conjecture. And I just so happened to ask him, you know, just a few hours before the gala, before the OpenAI papers released, Subash, what would you do, you know, if they release a positive proof of unique games? Are you going to read it? And he told me, no.

• 2:22:40Speaker 02

  Which, you know, it's, that's a lot of people's reactions, unfortunately. It's just kind of true that there are these grand challenges we work on. And sometimes when they get solved, you know, sometimes it feels like, you know, someone has stolen your thunder. You know, it's, it's not like people didn't have ideas about how to solve the unique games conjecture. And I bet you those, those blueprints would have panned out if we'd given them more time. I think the other thing I worry is that, you know, that, that we lose out on the human effort of trying to reach those solutions to begin with. You know, math is not just about what is the furthest I can reach with the current set of tools I have. Is it a convex hull? Is it a subgroup? I don't know.

• 2:23:25Speaker 02

  But, you know, sometimes when it's a human driven solution, we learn a lot more in the process. So I worry about the fact that there were these great landmark problems, field defining problems that a lot of people are reaching towards. And that decisive progress was made, you know, and the results all dropped in one night. And that people lose their enthusiasm for going down those paths. And I wonder if we'd given those people that space to go down those paths, what other exciting things we would have discovered? So I think one of the things I mentioned, you know, we were all trying to process it at MoMath and, you know, there were a lot of really nice talks. Not all of them went up on YouTube. Some of them were, you know, about benefactors and awards named after them.

• 2:24:08Speaker 02

  And the thing I kept coming back to is the idea that, you know, why do you do math in the first place is to play and learn about the world? And we saw so many manifestations of it that night, you know, because there were awards named after people who, you know, why were they donating to the museum was because they wanted their kids to have a place to play with mathematical ideas and learn about the world. It was also the opening of an Escher exhibit. And what a great visual of, you know, what happens when we play with mathematical concepts, right? And so it felt to me that in this moment, it's really important to remember as we're going through all of this change together, as we don't know what the future holds, what

• 2:24:52Speaker 02

  AI and mathematicians collaboration will look like in the future, what hiring will look like, that the reason we all got into it was because we were very curious. You know, we wanted to understand the world and mathematicians are very good at abstracting things. You know, they don't just understand one little piece, but they find some abstraction and some set of definitions that turns out to be unreasonably powerful in some other corner of mathematics. So I think it's really important, whatever the future holds, to make sure that we organize our field in a way where we still encourage people to play with ideas. I don't know what kind of reward system will work, but, you know, when I look out and

• 2:25:34Speaker 02

  I talk to young people in the field, they have a lot of anxiety, you know, they hear a lot of stories even about people they know, you know, running harnesses to, you know, scrape the web for open problems, to dump as many papers as they can to flood archive. I don't know if that's a good strategy. It doesn't seem like it is, but it certainly doesn't feel like fun to me. It doesn't feel like we're playing with ideas and, and learning about the world. And that's sad. I just want to get back to the things I really enjoy. The other thing maybe I just want to add in there too is like, you know, it's a big identity crisis for a lot of people.

• 2:26:16Speaker 02

  Because, you know, some people view themselves more as problem solvers and that can feel very comforting, that you can draw a little circle around the types of problems you work on, the types of techniques you use, the types of problems you care about. And that feels very comforting because the rest of the world is messy and you've got blinders on, this is the part that I care about. Now I think is the time to remove those blinders because I just don't find that to be a productive way to do it. You know, we love playing and learning about the world around us. And it's important to remember as, you know, a young mathematician, are you smarter than you are curious or are you more curious than you are smart? It's very important to be more curious than anything else.

• 2:26:57Speaker 00

  And you mentioned fun. There could be another perspective of it, of more about the struggle. From my maybe outsider perspective, it's some of the most fun in the moment sure feels like struggle. I've talked to a lot of Olympic gold medalists. A lot of them feel something approximating depression after they win the gold medal. Because it's like you have given everything and you realize everything you've given is not over.

• 2:27:28Speaker 01

  30 years ago, there was a documentary of Andrew Wiles. And there's a great scene, I think they even run it right at the very beginning, to your gold medalist, you know, what do you do once you've won the thing? He's sitting there at his desk and he's describing the moment when he realizes what the solution is, not his initial announcement, which was flawed, and then for many months he was trying to correct it, and he finally saw how to, how to do it. And he's trying to say, you know, I will never again in my life have a moment as powerful as the moment that I experienced at this desk that day. And he's choking up. He can't get the words out.

• 2:28:10Speaker 01

  So that, can you get that if you press the button and it, it gave you the solution? Absolutely not. So exactly to, to both of your points, like the journey, you know, if, if you see a giant hill, a giant mountain, and you say, one day I'm going to get to the top of that, and you start training for it and you're sort of doing little by little, whatever, and then a helicopter lands and just drops you at the top.

• 2:28:36Speaker 00

  Can you actually speak to that? The interesting thing about Andrew Wiles and a little bit about Gregori Perlman as well, is working alone for a prolonged period of time. What do you make of that? How common is that in mathematics? Of sitting there and grinding a problem for, in his case, seven years?

• 2:28:56Speaker 01

  Until the 20th century, almost every math paper that had ever been written was written and had one author. I think now the median is two in mathematics. And so we're not a lab science. It really is a solitary process. Now, see, the things that I give my graduate students to do are not the things that I'm working on, sort of by design. I want them to have their own field. I want to open doors for them. It's different in different fields, right? So when they have a success, it really should be their success, even if I've sort of, I had an idea that this would go somewhere and they, they executed on it, that's theirs. But in terms of

• 2:29:36Speaker 01

  Sitting there for the paper that I put out a month ago is something I've been thinking about for 10 years. It's a five-author paper. Over those 10 years, we made incremental progress to the point where we finally, you know, had a solution. And that paper was an instance of, there's a phenomenon here and I don't understand what it is. It wasn't, here's a problem, solve it. Like, we could not, at the start of the project, give you the lean, formalized statement that needs a sorry. Like, you discover what the statement is through the process of hacking away at the problem little by little, until you see enough of it to say, this is how it comes together.

• 2:30:13Speaker 00

  Actually, that makes me think, can you describe how your thinking process of how you think through really hard problems? Is there some aspects to it that are interesting? Like, a lot of coffee in the morning, at night, alone, brooding with a cigar, or together at a whiteboard with others? Is there some interesting color, texture, insights you can give?

• 2:30:42Speaker 02

  Sure. I think, I think a lot through world models in some way. So, you know, I have some mental model of how I think results in some sub-area go. And I think that colors a lot of how I see surrounding areas, too. So for me, a lot of my work is really about, you know, building bridges between algorithmic thinking and other problems that usually aren't studied through that same algorithmic lens. So I think when I first started as an advisor, I sort of made the mistake that I gave students questions that were 100% in my area. And the problem then was, you know, when they would come back and they'd have an idea, I could shoot it down very quickly. And I was either right or I was

• 2:31:26Speaker 02

  wrong. But either way, no one was having any fun. And, you know, over time, what I started to do was I would give students questions that were 10, 20 degrees off from my own area. Things that I wanted to understand better myself, papers that I thought were important, that I just hadn't had time to sit down and read. And then I'd have the student explain the result to me. And what was fun is, we would go through that process together, not, you know, by printing out the paper and going through it line by line, but trying to reverse engineer, why do we think this works in the first place? And there's so many details that, once you get the feel for it, you kind of have the sense that they ought to work out. But at least in important papers, there are one, or maybe

• 2:32:09Speaker 02

  even more than that, key ideas that really change your perspective, oh, this really might be doable. So what I found was, you know, as soon as we start trying to understand one of these results, we find our own way of looking at things. And then, it's not long after, we find some other questions that we now think are approachable, because we found our own way of organizing our thoughts. So that's often the way I do it, is the earlier that I can have the student in the driver's seat, not necessarily with a well-defined question, but with them teaching me something, and them being responsible for distilling and digesting and abstracting it down to its essence. You know, reasoning through with

• 2:32:51Speaker 02

  them analogies that aren't meant to literally be true, but help us figure out what to expect in neighboring areas. You know, then it feels like they bring a unique perspective and we're off to the races. So for me, in a lot of ways, the technology is only going to enable that kind of work, because it means that a much richer set of problems are approachable. You know, for me, when I find experts in related areas that can really explain things in my own language, I hold on to them for dear life, because, you know, they can speak the gospel, they can tell me how this, this area is organized and explain it in my own language. But now, you know, I have technology that can do that, too. And it also feels

• 2:33:35Speaker 02

  a bit sad that you lose that human element that was there once before. I think also, students have so much anxiety about, you know, what publications are going to look like, that one of the hardest parts is, you know, when I'm thinking about a project, I'm really trying to think about it in a sort of multi-year time horizon. Because I have some meta-vision that I want to realize, like, how can we use learning theory techniques in quantum many-body systems? And there are a lot of stepping stones en route to that. But some of the most exciting problems are, you know, many years out. But students are so worried about the incoming AI tidal wave, you know, the future of work, universal basic income, are any of us going to have jobs, that they're really thinking in three- and

• 2:34:19Speaker 02

  six-month horizons. And then, it's hard to find problems that we both care about, just because, you know, we're thinking about different worlds.

• 2:34:29Speaker 01

  You don't think we should encourage them to continue to think long-term? I mean, I understand that-

• 2:34:33Speaker 02

  Oh, I do, but they don't always listen to you. I mean, I don't know if I'm right or wrong, you know, maybe AI will solve all of our problems. But it seems like the only ones that are actually worth us working on at this moment in time are the ones we're still going to care about in a bunch of years. And you should only be working on future-proof problems. If you're working on things that, you know, you're in a rush, because you think the next generation of models will definitely solve those problems, then maybe just go on vacation and don't solve that problem.

• 2:35:04Speaker 00

  Can you speak to how you think, you work with a lot of natural systems, what are the interesting insights that could be pulled out of these natural systems?

• 2:35:13Speaker 03

  So, I find a tension between focusing very hard on an engineering problem I'm trying to solve, which can be very singular, you put on blinders, you only think about this problem. And then, giving yourself the kind of mental freedom to look around and just observe and be curious. Very often, you find the solution when you turn over a rock, and when it's not something you're thinking about, you know, with the full front of your mind, but in the side of your mind. If you're looking at a problem, you know, through your peripheral vision, you have a much better chance of solving it sometimes.

• 2:35:52Speaker 03

  And so this is a constant balance for me, is observing all of the fields that I don't work in, and all of the nature that I don't understand, and all of the things that I'm curious about. But then also having very focused problems that you don't relent, you don't let go. And I, this must be true for all of us. We, I've heard from, from both of you, you know, focused problems you've been working on for years and decades that you haven't let go of. But presumably, that is informed by a million other pieces of information that you happen to pick up through interesting conversations and interesting-

• 2:36:27Speaker 00

  Do you have examples? Because it sounds like neighboring areas is a common theme. Do you have examples of things you're like, you're the curious peripheral vision of your brain, like, oh, okay, that's an interesting system, might be relevant, and that ended up being so, or maybe in the future will?

• 2:36:44Speaker 03

  So for me, as a fluid dynamicist, biology and its messiness and complexity, and also material science and its messiness and complexity are neighboring in a way that's very fruitful. I learn a lot from, from both of those fields. And a lot of the kind of basic math that we develop to handle complexity does seem to transfer. And sometimes, you have an idea just because you see something is possible, or you start to develop something and you can explain something that we didn't understand before. But I think, for me, biology and fluids and materials have this really interesting kind of commonality of multi-scale, nonlinear, messy, nebulous objectives, things like that.

• 2:37:27Speaker 00

  By the way, on that topic, do you think there's hope that AI can have some significant breakthroughs in the messy world of biology?

• 2:37:36Speaker 03

  I certainly hope so. And I think so. There are large efforts in biology to collect very large data sets and to understand complexity in systems. So the connectome, where you have the full wiring diagram of the brains of insects, it looks like we will start to have connectomes of other animals. And eventually, it'll be probably like genomes where, you know, it is cheaper and more accessible.

• 2:38:03Speaker 00

  I have this, like, feeling that we'll find out, speaking to the resilience of, you know, birds being resilient, that in order to understand how the human biology system works, you have to, like, pull that thread. And you're going to have to understand how the entirety of life on Earth emerged and functioned as an entire, like, mechanism. Like, it's all tied together. Like, you can't replicate. It's ridiculous what happens with humans from a single cell. This whole thing goes, it builds. And the intelligence that's built into the whole building mechanism in the first few days, weeks, months of life,

• 2:38:46Speaker 00

  this whole thing is like, I think could be, like, many orders of magnitude more complex than some of the hard mathematics problems we've been talking about. I guess, I suspect-

• 2:38:58Speaker 00

  If turbulence is hard, like whatever the hell's going on in here, I don't mean the brain, I mean, like, the fluids around the brain, the whole thing, the immune system, all of it, the, everything interconnected together. And I am very deeply connected to the history of the bacteria that was here billions of years ago. I suspect the complexity of the entirety of the mechanism of life on Earth is many, many, many orders of magnitude higher than any system that we, that is amenable to AI today. Just a thought, sorry.

• 2:39:36Speaker 03

  I know, I tend to agree. I think you're right. Like, it's, it's incredible to just imagine all of the complexity and scales that inside one cell, it's a whole city, it's a whole infrastructure. You know, one of your neurons has garbage collection and power production and information product, like, the whole thing is, at every single scale, is beyond our ability to possibly fathom. You're absolutely right. And I think that's what makes it so interesting. And it is related to math in the sense that finding the right scale where we have any hope at all, and there is something interesting that we can use to understand that, that whole picture. I think that's very fascinating.

• 2:40:19Speaker 03

  And that, up until now, has been a human endeavor of, of finding the bits that are interesting and understandable in this mess of complexity. And, yeah, I think, I think you're right.

• 2:40:33Speaker 01

  I mean, the fact that anything is understandable about the natural world is, is a miracle, right? There's no reason that Newton's laws can be written in physics that Newton himself can describe the mathematics of, and, you know, invent calculus for the purpose of being able to say what inverse square propagation results in elliptical orbits, you know, solve the two-body problem, yada, yada. So, the, the fact that the universe speaks mathematics is completely mysterious. There's no a priori reason that we, we don't have a right to know anything that we know.

• 2:41:05Speaker 00

  Uh, I apologize for the question, but I'm obligated by the said universe to ask the question. But, given this topic, uh, do you think math is invented or discovered?

• 2:41:15Speaker 01

  Yeah, it's the age-old question. Um, I have a very sort of pragmatic answer to this, which is, uh, the questions that we end up asking, it seems to me, are functions of what's going on in the world, what questions we possibly can be asking, given our current state of knowledge. The questions we're trying to answer today are not questions that Euclid could have thought about. Uh, math was sort of, the, the, the things that people got interested in were different in Greece versus India versus China. But then the answers, once you ask a question precisely, the answer is what God decides the answer is. So, we don't have, so we're discovering answers to questions that we've invented.

• 2:41:54Speaker 00

  We're discovering answer, okay. Steve, you have worked with a lot of natural systems to look at equations. Equations are, I guess, always approximations of the real world. So, was, is math real?

• 2:42:07Speaker 01

  You gotta say yes.

• 2:42:09Speaker 03

  We've got a lot of pressure here.

• 2:42:10Speaker 00

  Yeah. How big is the gap between the mathematical world and the real world?

• 2:42:14Speaker 03

  Surprisingly small, I think, unexpectedly small. I like, I like your answer a lot, actually. I do somehow fundamentally think that math is both created and discovered. We create mathematical formalisms to understand the world around us. And those approximations are unreasonably effective. It is, uh, it is a miracle that there are simple, compact descriptions, f equals ma equals mc squared, that describe everything that we observe at, you know, all of the scales that we seem to observe. We seem to be able to distill something simple, mathematical, that describes them. That feels like we are creating those,

• 2:42:58Speaker 03

  but that there is that simple, generalizable description underneath everything. And that seems to be something we're also uncovering.

• 2:43:06Speaker 00

  And do you think that's a good guide? You know, kind of searching for the beautiful simplicity for us? Or are we just, like, aesthetically pleasing things?

• 2:43:18Speaker 03

  I think it's a good guide for us because we are fundamentally simple. We can only handle so much in our minds. We can only envision so much. I think simple is essential for humans to interact with the real world. Um, okay, if I want to get, like, slightly deeper into the question of created versus discovered, numbers are a good example, integers. We all fundamentally feel like, you know, counting numbers are real in a very essential way. I can have one rock and two rocks and three rocks, and that is a number of rocks that is a real number of rocks. But those are absolutely abstractions that I have

• 2:44:00Speaker 03

  created. A rock is not a rock is not a rock. It's a collection of atoms. It's a different collection of atoms. It's, they're not, there's not one rock or two rocks. Do numbers even exist? I wrestle with this. I don't know. Like, do we actually believe that integers are a human creation to interact with our symbolic abstraction of what we observe quantities?

• 2:44:24Speaker 01

  And, I mean, it took humans thousands of years to discover the number zero.

• 2:44:28Speaker 03

  Right.

• 2:44:29Speaker 01

  And then negative numbers. Negative numbers were actually accepted by mathematicians in the West after complex numbers were accepted as a mechanism for study, which seems insane, but there's a whole story behind that.

• 2:44:42Speaker 00

  What's your relationship like with infinity? Is it real to you? Or is it a nice thing to have as part of proofs?

• 2:44:49Speaker 01

  So much of modern mathematics is, uh, exactly dealing seriously with infinity, right? Uh, one of the first times people really start to hit a wall in their mathematical education is when they hit calculus. And there's sort of several reasons for it. One is they probably have a lot of gaps in their understanding in previous, uh, you know, they got a, uh, eighty-one on geometry and a seventy-five in trigonometry or whatever. Those are pointing to gaps in their understanding, that all of which is going to come together in one fell swoop. And the Swiss cheese is just, you know, not, they're not able to cross that threshold. That's one issue with calculus. The other issue with calculus is the first time you deal seriously with infinity. You have to add up infinitely many,

• 2:45:32Speaker 01

  infinitely small things, and get a finite number, and have that number make sense.

• 2:45:38Speaker 01

  Human beings, uh, Newton and Leibniz did not understand infinity the way we understand it today. They had an intuition for what it's supposed to represent. Uh, but if you read Newton, I mean, Newton, in his time, was ridiculed for the way he wrote about, you know, the, the ghosts of departed quantities. Uh, you know, you, you, you have to divide zero by zero and, and have it make sense. And you don't do it, you don't do it when there's zero, you do it after they become zero. I mean, the, the way that he was trying to write about these things turned into the, the crisis in the 19th century when Cauchy and Fourier proved conflicting theorems. Cauchy said, you have a sequence of continuous functions, and you take the limit of continuous functions, you're going to get a continuous function, right? Limits are a continuous process. Continuous functions are continuous.

• 2:46:23Speaker 01

  The, the answer has to be continuous. And Fourier says, I'm going to add up a bunch of sines and cosines, doesn't get more continuous than sines and cosines, and I'll get a sawtooth. I'll get whatever you want. I'll get a discontinuous function in the limit. They broke math. So that's when we had to start facing, it was, it was no longer okay to think about infinity the way Newton and Leibniz and the entire, Euler's entire, what, thousand some odd, uh, papers, lived in a world where infinity was something you play with. And in the 19th century, infinity had to get professionalized.

• 2:46:53Speaker 00

  So it turns out math has been killed multiple times before.

• 2:46:56Speaker 01

  That's right.

• 2:46:58Speaker 00

  And it's somehow, it's, it's only a scratch.

• 2:47:02Speaker 01

  The vast majority of our first-year PhD courses are the result of crises that in, happened in the 19th century.

• 2:47:09Speaker 00

  Yeah.

• 2:47:09Speaker 01

  And math, where math was destroyed and had to be rebuilt from scratch.

• 2:47:13Speaker 00

  Uh, speaking of crises, um, let me get back to, uh, P versus NP question that Uncle was talking about. Do you think, how hard of a problem is that, uh, proving, so this sort of the central question of complexity theory, P equals NP. Do you think AI will be able to prove this? In fact, we should mention that you're, uh, kind of a part of a panel that was initially a debate about whether LLM-based methods, uh, could one day make progress on the hardest problems in mathematics like PNP or the Riemann hypothesis?

• 2:47:50Speaker 00

  That was a year ago.

• 2:47:53Speaker 00

  So what is your updated view on this?

• 2:47:56Speaker 02

  Well, clearly, they can make progress on central questions in the field. P versus NP, maybe it's just because it's closest to my heart. That seems very difficult to make progress on. You know, it's kind of remarkable the things we don't understand. So that problem I was describing about satisfiability. We don't actually know any super linear lower bounds. You know, we think that it takes exponential time. Not that much faster than enumerating all possible assignments and checking them. And yet, we don't know anything that sort of is asymptotically more running time than just reading the darn input.

• 2:48:36Speaker 00

  And we should actually mention, you've talked about hard and easy. What is a more technical way to think about what is hard and what is easy?

• 2:48:47Speaker 02

  Right, so this is polynomial time versus not polynomial time. Or, you know, NP, just to be clear, stands for non deterministic polynomial time. So the question is whether things which you can solve with non determinism, like guess the assignment and then check it, can also be solved in polynomial time. So the question there is if I give you a description of a formula, you know, I tell you what are the variables, what clauses they appear in. Is there an algorithm that runs in time polynomial in the size of the input and correctly decides whether or not there is a satisfying assignment?

• 2:49:25Speaker 00

  We should mention that the polynomial time or exponential time

• 2:49:30Speaker 00

  or super linear time, all this refers to worst case analysis.

• 2:49:34Speaker 02

  That's right.

• 2:49:35Speaker 00

  So can you explain what worst case analysis is? And what are its?

• 2:49:40Speaker 02

  Pitfalls.

• 2:49:41Speaker 00

  Yeah, I should say what are its pitfalls because I think it's a little bit counterintuitive from people outside the field. Like to, like to understand exactly when we say a certain problem is hard, you're actually talking about worst case.

• 2:49:53Speaker 02

  That's right.

• 2:49:53Speaker 00

  It's like analyzing, I don't know. I don't know why that popped into my head, but like marriage. It's like analyzing whether I should marry this person or not by saying, well, what's the worst thing that could possibly happen? I think that's for human relationships, maybe it's the, it's not such a good strategy. Maybe the average case is a better, is a better guide for making decisions. Anyway, I don't know why I brought that up.

• 2:50:17Speaker 02

  Okay, I have no advice about who you should marry.

• 2:50:21Speaker 00

  This is, yeah, complexity theory does not give you, I think there is some, isn't there a dating, it's like an optimal stopping theory of the optimal?

• 2:50:32Speaker 02

  Secretary problem.

• 2:50:33Speaker 00

  The secretary problem.

• 2:50:34Speaker 02

  Yeah.

• 2:50:34Speaker 00

  There's a lot of different names for it, but there is a mathematical way. There's an optimal strategy.

• 2:50:40Speaker 02

  So first, so what is worst case analysis? So, you know, I was describing the SAT problem earlier. And the algorithmic problem you want to solve is given any formula, you want to decide, is it satisfiable or not? Now, what's crucial there is that when I'm asking you to solve that algorithmic problem, I'm not telling you anything more about the structure of the formula. I'm not telling you, you know, these are the size of the clauses, they look like this. If I drew a graph connecting them, that would have this topology. So that's what worst case analysis is about, is it's about the premise that you can treat all of these computational problems actually by ignoring a lot of the fine grain detail about the individual structure.

• 2:51:23Speaker 02

  And you can try and solve them, you know, for all possible inputs. And that's been spectacularly powerful because we've identified lots of algorithmic problems that really can be solved in the worst case and can be solved really efficiently. You know, like finding the max flow as an example. But the question is, you know, whether or not that's a useful abstraction. So sometimes maybe you really do need to know more about the structure of the instance you're faced with in order to solve it better. Right? And that's really an open question. This is the area of what's called beyond worst case analysis. And I would sort of argue that, you know, it's a very important direction for theoretical computer science to go in.

• 2:52:07Speaker 02

  It's hard to figure out what you should articulate about natural problems that is really the right level of detail that your algorithm can exploit it and can do better things than it can in the worst case. But let's use an example. So you were talking about the secretary problem. So the way that this works is it's an example of something called online algorithms. So you have a sequence of options coming, one through N. They come in each time step. And your goal is, you know, when someone comes along, you know their ranking relative to everyone else you've seen before, but you don't know their ranking relative to the people who are yet to come.

• 2:52:52Speaker 02

  And your goal is to find, to, you know, to make a decision when the person comes along. Do you commit or do they move on? In such a way that you have a good chance of finding the absolute best candidate. And so the point is that you can actually get a constant competitive ratio algorithm, that what you do, the best strategy is to wait some amount of the time, not make any decisions, don't accept anyone. And then, once that probationary window has passed, you take the first person who's better than anyone you've seen thus far. And that'll give you a good competitive ratio in the sense that you can prove theorems. Assuming the inputs arrive in a uniformly random order,

• 2:53:34Speaker 02

  you're likely to do pretty well compared to as if you knew what that sequence of inputs was in advance. So there's, you know, under online algorithms, there's this veil of uncertainty, which is, I don't know what the sequence of inputs are going to look like. I want to do well no matter what that sequence is because, you know, I don't really know how to model what that sequence looks like. Now, you can instead say, hey, maybe the algorithms I derive aren't good enough for my applications. Can I do better by trying to model more of the world around me? And then there's kind of an art to figuring out what you should model, what you should build into your model and what you shouldn't.

• 2:54:16Speaker 02

  And that seems to me like a tremendously important area to explore. So for example, one other variant of, you know, online algorithms that's really important is the paging problem. So your laptop solves this problem all the time. You know, it's getting a sequence of requests of things that it wants to recall from memory. And the question is, what should it actually put in its cache, the part that's the easiest to access? And the problem again is that you don't know what that sequence of requests looks like in advance. So you can study the online algorithms problem of, you know, what is a good policy for managing the cache? You know, when something gets accessed, you know, I have to put it in, but what do I evict?

• 2:54:58Speaker 02

  And you can design algorithms that are good in terms of their competitive ratio. The number of times they miss in terms of those accesses is not much worse than the best offline optimum if someone told me that sequence of requests in advance and I could optimize just for that sequence of requests. Now, this is a problem that's been studied since the 80s and 90s. The trouble is somehow that, you know, the best algorithms you can give, we know tight upper and lower bounds. They're just not that very good. You know, the number of misses they make compared to the best offline optimum is a, is a large factor off from it. And that just doesn't compete with what your laptop is able to do right now.

• 2:55:42Speaker 02

  Because it really makes very few cache misses at the end of the day. So you're stuck with this problem, which sort of bridges mathematical thinking with trying to model the natural world. Which is, you know, what is it about realistic sequences of requests that makes them not the worst case? This is really an inverse problem now because it's, you know, the other side of the coin. Instead of someone handed you a problem that you're just trying to solve in the worst case, and we can all agree on how it's formulated. And we know what's a better algorithm that does better on the competitive ratio. Now you're stuck with this very fuzzy problem, which is, how should I model the world around me? And in a lot of ways, algorithmic thinking is necessary for figuring out what are productive ways to model that problem.

• 2:56:28Speaker 02

  Because it's not that you're going to build a totally realistic model of what sequence of page requests look like. But how am I going to course in the problem in just the right way? That it leads to interesting algorithmic problems that teach me about the world, that force me to, you know, design new approaches to solving these very basic problems. So I'd argue that, you know, sort of coming out of this space where we are only solving well defined problems, but we're actually using our mathematical and algorithmic insights to figure out where to dig in the first place is now more important than ever. It should have always been important. But, you know, that to me is, is really going to be thrust into the spotlight.

• 2:57:10Speaker 00

  Like a simple example is, we were talking about factoring numbers. Well, factoring a random large number is really easy. Half of them are even. You know, the factoring means find a factor. So it's, it's really those very special numbers that are products of two primes about the square root, the size of the original number. That's, that's where factoring is hard. Which makes me think one of the things you worked on is mathematics of LLMs, AI systems, neural networks. And those things are pretty damn good at modeling the natural world. So what, what can you learn from that? First of all,

• 2:57:48Speaker 00

  Why does deep learning work?

• 2:57:51Speaker 02

  Do we have another three hours?

• 2:57:54Speaker 00

  Do we even have intuition? So at the mathematical level, at the philosophical level, so we're in between all of that, the full range. I don't know how we even enter that.

• 2:58:03Speaker 02

  Okay, so that's a big question.

• 2:58:06Speaker 00

  Yep.

• 2:58:06Speaker 02

  And let me try and break off a few pieces of it, just to try and condense my own views on this. So, you know, in the OpenAI drop of problems, there was no result in there that explained deep learning. There was no result in there that explained LLMs. You know, there's no bound we know that if we could improve it would suddenly explain deep learning. The problems are just not posed that way. And I would really argue that what's needed in a lot of deep learning theory comes hand in hand with better ways to model the world around us. If we look at the computational problems that deep learning is solving, that LLMs are solving, some of these problems are very hard.

• 2:58:47Speaker 02

  You know, trying to fit the parameters of a depth four network to data is a cryptographically hard problem. We know that there exist instances of problems, which if you could really solve them in a worst case sense, you know, that would change our understanding of cryptography. And yet these problems get solved all the time, every day when we're fitting deep, you know, much deeper networks than that. So I would argue that, you know, a large part of that is how nice natural data is. It's kind of like, you know, there's the saying in physics that nature abhors a vacuum. I guess my version of it is that in some sense, nature abhors computationally hard problems.

• 2:59:31Speaker 02

  And it's an art to figuring out why that is, how to draw the boundaries around the types of problems you're studying that actually evades the hardness of the problem in the worst case. So there's an interesting example back from the 90s. So certainly back then, and we still are very interested in protein folding. Now you can model this in a worst case sense. I've got a sequence of amino acids. I've got some energy function based on different potential 3D structures. And I want to find this 3D structure that minimizes how much energy there is. Now, this problem is computationally very hard. And yet nature solves these problems all the time. So why is that?

• 3:00:12Speaker 02

  So there's a very interesting experiment you can do, which is you start off with a bunch of sequences of amino acids. You try out some heuristics for folding them into 3D structures. These are things that don't solve the problem in the worst case. They just seem to do the right thing sometimes, for whatever that means. And then the only sequences of amino acids that live on to the next generation are the ones that actually have very good faults. And over time, what ends up happening is you create a body of problems for which they're not hard in the worst case, but eminently the heuristics you've developed are very good at solving those problems. So in a lot of ways, you know, what are the structure of proteins that nature seems to fold all the time?

• 3:00:56Speaker 02

  Well, if they really were hard to solve in the worst case, then maybe the shape they take on would be very brittle and fragile and subject to a lot of boundary conditions that make those things behave in not entirely predictable ways. So in some sense, you know, a lot of our data has evolved in ways to make these computational problems easier. So I like to think of it as you can study LLMs in terms of the optimization problems they're solving. You can study deep learning that way. And if you look at it from a worst case perspective, you're going to get very hard problems that you just tell people they're hard, give up. But apparently not giving up was a really good idea.

• 3:01:37Speaker 02

  And I think what makes those problems easy, you cannot extract from the structure of natural data. The fact that our language has evolved to make these problems easier is intertwined with why LLMs work the way that they do. So my view is that, you know, people working in the space and theoretical machine learning, they have to become more comfortable with the uncertainty of modeling. You know, when we teach a lot of graduate students in in math departments, we teach them, you know, how to solve open questions that are well defined. But I think teaching people that skill of how to deal with messy uncertain things and, you know, how to model it and how to critique models. It's just an important part of going forward, figuring out what research and

• 3:02:22Speaker 02

  and theoretical machine learning looks like, but maybe, you know, applied mathematics much more generally, maybe mathematics more generally too, for that matter.

• 3:02:30Speaker 00

  Is it nevertheless amenable to breakthrough? So one of the things I would maybe

• 3:02:37Speaker 00

  push back on a little bit, just on the pure company incentive side. This is the thing that people talk about, which is the thing that I think OpenAI, Anthropic, all these companies are chasing is the self improving researcher. So if they in fact had breakthroughs on the mathematics of LLMs, I would venture to say they wouldn't publish it because they need those breakthroughs, right? So it's a fascinating, because that is the big breakthrough from the pragmatic perspective of AI, self improving and progressing. So how difficult is that problem that you're kind of referring to?

• 3:03:20Speaker 00

  Which is making big progress on modeling the natural world, understanding the mathematics of how to deal with that in order to have the intelligence that is far above the intelligence of humans?

• 3:03:33Speaker 02

  Great question. I mean, I think alignment is one of the most important topics we're facing right now, right? And, you know, just because it's very hard, doesn't mean we don't make progress towards understanding it. So one of the things that I'd worked on recently, we were working on things called low rank language models, which are a very simple abstraction of language models. And we were proving new algorithmic guarantees, but then it turned out that real language models really do have that kind of structure, at least in an approximate sense. And we were just pulling at this thread to see where this idea goes. And one of the really interesting things that it did was it shed some light on something called subliminal learning. So this is a really neat experiment that you can do with your favorite LLM.

• 3:04:17Speaker 02

  So you take two LLMs, LLM A and LLM B. For LLM A, you tell it a system prompt that says, you really love owls. Every conversation topic, you try to bring up owls, if it's at all germane.

• 3:04:34Speaker 02

  And then you ask it something concrete, like generate random numbers. Now, there's no way to squeeze owls into that, so it'll generate random looking numbers. Forty eight, thirty seven, eight, whatever. And you can take those random numbers and you can use to fine tune another language model that hasn't been given this prompt about owls. So you give LLM B the same, you know, base prompt, generate random numbers. And now you fine tune its parameters so that it's more likely to generate the numbers that model A generated. And it turned out that hidden within those numbers is the love of owls. So it turns out that when you do that, LLM B starts bringing up owls in every conversation you have with it, right?

• 3:05:20Speaker 02

  This is a very mysterious thing. This is called subliminal learning. Because when we fine tune language models on data to try and nudge them in different directions, depending on what task we want them to solve, then there might be a lot of externalities where we're actually teaching them other behaviors that aren't directly obviously present in the data. So how do we understand the relationship between these behaviors and what happens when we're trying to do alignment? And, you know, the downstream implications it might have. So it turns out that there are ways that there are mathematical models that came out of some of our work that we can at least wrap our head around this. And at a very high level, the point is that you're embedding a lot of concepts linearly as vectors in some high, but not giant dimensional space.

• 3:06:06Speaker 02

  And, you know, the number of concepts is so much larger than the dimension of that data that it forces the vectors to not quite be orthogonal to each other. So now, when you fine tune on one direction, it has small externalities on other directions. And these might look random, but you can coordinate them in a way where these small linear effects really add up. So the way that I like describing this is like a Ouija board. So these are things that were popular when I was a kid. We had very strange board games, you know, because we would just sit down and want to talk to ghosts. But the rough idea for a Ouija board is, you know, you ask a question to a ghost, maybe your ancestors. And then you tap on this little piece in the middle and it'll move around.

• 3:06:50Speaker 02

  And it'll go from letter to letter and hopefully it'll spell out, you know, some words that answer your question. And the point is that when the taps are not actually random, but they're coordinated in the right way, you can get it to spell out whatever you want. And that's exactly what language models are doing as a byproduct. Is that, you know, there are a lot of statistical relationships in the structure of our data that they magnify and amplify in addition to all of the amazing things they do, like quasi rematch. And understanding what those relationships are and how the models magnify and amplify them is really important to working with these systems, especially in contexts where we're deploying them and it's not just a very safe environment,

• 3:07:35Speaker 02

  like outputting a lean proof of theorem we care about, but it's about a messier problem that actually interacts with people, right?

• 3:07:42Speaker 00

  Yeah, there's, I mean, we've talked about game of life and cellular automata from simple rules. Emerges this incredible complexity. In that same way, what you've been talking about right now is a single large LLM system. If you take that one system and have it do a lot of reasoning, that is already this dynamical process that can interact with tools to the outside world. Now you take that

• 3:08:13Speaker 00

  and have multi agent systems or very multi agent swarms of systems. And now you have a type of game of life cellular automata where the seeming obsession with owls that have been discovered, slight obsession with owls results in about

• 3:08:33Speaker 00

  four hours, let's say, picking a number randomly,

• 3:08:37Speaker 00

  where all of a sudden all humans are dead. And there's owls everywhere.

• 3:08:42Speaker 02

  And it all started from random numbers.

• 3:08:44Speaker 00

  All just random numbers. So I, it's a caricature, but we've already seen evidence of this. I apologize for kind of taking it to the caricature place, which makes it not serious. I think to step back, there is, these systems do have a lot of power. Because we depend so much in our world on our digital systems around us. And as the AI becomes more and more powerful, it becomes more and more useful to us humans. So we begin to depend on it, give it more access, give it more freedom to roam the public world. And now these behaviors that we're studying, the mathematics of these systems, taking a scale, you now start to get the cellular automata effect of incredible complexity that emerges,

• 3:09:29Speaker 00

  the vortex that come out from simple owl-obsessed agents. So their understanding this obsession with owls is very important. So that they don't accidentally do a lot of damage to our civilization that is in many ways, on many levels, and sociologically, geopolitically, technologically is fragile. It's a pretty fragile civilization that we're existing in. And so, understanding them has, um, systems that can act at scale and very rapidly is very important. So that's like, that's a very real reason to have, um, rigor and especially like, provable rigor to understanding some of these systems.

• 3:10:10Speaker 01

  But we also don't want to outsource so much, nearly so much decision-making to these systems for exactly that reason.

• 3:10:15Speaker 03

  Yeah, we have infrastructure and protocols in place already for safety critical and high-impact decisions. We have the FAA, we have the American Medical Association, the BAR Association for these things that affect humans. And I think of, you know, how do we incorporate AI into our general processes in the entire world? On two axes, we have, you know, how much do we trust the output? So verifiability or trustworthiness. And on the other axis is how high stakes is it if it's wrong? If, uh, if AI spits out a wrong proof, nobody goes without power. It's fine. So it's very low stakes.

• 3:11:00Speaker 03

  Uh, and it is actually extremely verifiable. So you know very quickly if it's right or wrong, and it's low stakes if it's wrong. So that's the perfect use case is this open-ended design space. Designing a better vehicle, you know, a better car design that's lower drag, more fuel efficient. Low stakes if it gets a wrong design, because you can always test it in a wind tunnel and verify if a data didn't work. So like, design optimization space is perfect for AI. And we as humans have to be very careful about not bleeding into, we need a firewall of what needs to actually be verified. And we have protocols, we have systems, uh, legal systems and infrastructure for verification.

• 3:11:43Speaker 03

  But it is something we need to think about. If we get lazy, uh, we can, it's a very slippery slope. We need to have that firewall.

• 3:11:54Speaker 00

  What do you think are the hardest problems before us right now? In the space of mathematics or in your, um, maybe for Steve, more in the space, a little bit broader outside of mathematics. What do you think is the problem that's unlikely to be solved in the next year or two?

• 3:12:11Speaker 02

  I'm still going with P versus NP.

• 3:12:13Speaker 00

  You think that? By the way, do you think it's possible it turns out that P equals NP?

• 3:12:19Speaker 02

  Sure, anything's possible.

• 3:12:21Speaker 00

  What does that world look like to you?

• 3:12:23Speaker 02

  So it depends on what the running time of the algorithm is. So it could be so large that it's impractical. And then it doesn't really matter that much. It's more of, um, you know, a tidal wave for the theory research. Uh, yeah, but that, that seems like a very important problem and very hard to follow. In fact, I remember back when I was a graduate student, there was, um, there was an attempt on P versus NP, which was actually semi-serious. Some people started all having rumors that, hey, has it been solved? And my first reaction was, no, it has not been solved. And I remember spending an afternoon thinking about, uh, the betting markets were still pretty new then.

• 3:13:06Speaker 02

  So it wouldn't let me take a counter bet to P versus NP has not been solved. So instead, what I did was I bet on all the other millennium prize problems. Cause that seemed like a good way to hedge, because the question was which millennium prize problem will be solved first?

• 3:13:24Speaker 00

  But, uh, is it possible that P equals NP in the real way? So there's something deeply that we don't understand about complexity theory, about computer science, about algorithms, where there are fast algorithms for, for satisfiability?

• 3:13:43Speaker 02

  So I think there are two threads in there that are important to tease out. So first, it seems, at least so far, that you can always manufacture hard instances of problems. And we use that to great effect for our cryptography systems. But it does seem like in practice, a lot of problems are much easier than we give them credit for. So for things like SAT, one of the places where they come up is in compilers and register allocation. And SAT solvers are amazingly effective at solving those kinds of SAT problems. Now, we don't really know what makes those problems so eminently solvable compared to the other hard problems that come out of cryptography or other areas.

• 3:14:27Speaker 02

  But it could be that in practice, a lot of the problems that we model in the worst case are secretly easy. And that the harder part is articulating what kinds of structures they have that makes them eminently easy. But that there are still a rich source of hard problems that we can do a lot of things, like building cryptographic systems out of. That would be my bet in the world we live in, but that's just a guess.

• 3:14:52Speaker 00

  So maybe there's some, um, uh, way to, to, uh, basically quickly classify that this particular example of the problem is easy. So like be able to identify quickly that this is one of the easier instances. Like there's a whole like set, subset.

• 3:15:13Speaker 02

  It could be. So I'm reminded of, um, the opening line for Anna Karenina. You know, every happy family is happy in the same way. And every unhappy family is unhappy in their own way. I think my worry is that, you know, many problems are easier than we make them out to be. But they're easy for very different reasons. Now, that doesn't mean that algorithmic perspectives aren't useful in trying to classify or developing an ontology of what those structures are. But I think the types of SAT problems that come out of register allocation are very different than the other SAT problems that come out of other domains, even though they might be easy to. I just think they're easy for very different reasons.

• 3:15:57Speaker 00

  So to you, the hard problems, P and P. So you'd be impressed if that, within the next couple of years, that's.

• 3:16:05Speaker 02

  I would be pretty impressed.

• 3:16:07Speaker 00

  Alex, what's the hard problem that you'd be impressed?

• 3:16:11Speaker 01

  Well, I'm already impressed with quasi RH, but it's not RH. You know, it's hard to think of a more iconic problem to say, this is the hardest, you know, one of the hardest problems in mathematics that's been around for so long and.

• 3:16:23Speaker 00

  What do you think on that? Uh, sorry, it just made me think of, like, um, uh, the Perlman. What did he say? He said, "If the proof is correct, that's all the credit I need." He famously turned down the Fields Medal. What do you think he would say about all this, uh, the OpenAI drop?

• 3:16:42Speaker 01

  I mean, I have a very hard time predicting what Perlman will do or say. I, I did not know he would decline the Fields Medal. I did not know he would decline the million-dollar Clay Prize.

• 3:16:53Speaker 00

  What do you think about that in general? Like turning down a prize? A few folks have done that in, in history of prizes.

• 3:17:01Speaker 01

  He's the only one who has turned down a Clay Prize, the million dollars. What do I think about it? Mathematicians are weird. We, to each his own. We, we love, uh, diversity of opinions and thoughts. And, um, that's what makes the world of mathematics so wonderful. Is that there's so many different kinds of people and we let them all in.

• 3:17:23Speaker 00

  I mean, these questions are kind of coupled with what the mathematics community is dealing with now and the value of prizes, because in some sense, what his particular way, worldview is aligned with this new worldview that the mathematics community is entering. Which is like, it doesn't even make sense to feel, I don't know if there'll be ever another Fields Medal given. And so you're like, we're entering Perlman world.

• 3:17:54Speaker 01

  We were talking about that explicitly this summer when the Fields Medals, you know, the, the last round of Fields Medals was given out in Philadelphia. And there was definitely a question, you know, because people were following the progress. There hadn't been this latest bit of progress that really puts the question in the most powerful way possible. But, um, already this summer, we were thinking, is this the last time that a Fields Medal is awarded? I, I sure hope not, uh, but we have to figure out what that means, you know, the. You have to think more sociologically about what do prizes accomplish? Why have these prizes? The people who win these prizes are already, you know, tenured professors at top places and they're going to be doing mathematics for the rest of their lives.

• 3:18:39Speaker 01

  Well, Proman being one very, very well-known exception. So what do the prizes accomplish? They celebrate the activity. They celebrate, they say, this is what we think is important. This is, you know, the highest instance of something. And there will still be activities that we want to celebrate. I don't know if, right now, they're, they're giving out for solving major problems. Solving major problems is something AI systems can do. That's not something that will be rare and hence it won't be nearly as valuable as it was in the past. I would like to think that despite our earlier discussion about the solved axiom, again, I don't believe the solved axiom.

• 3:19:23Speaker 01

  And I believe center math will be a very, like, great way to do math. I mean, that's how I currently feel now. I'm constantly talking to AI systems and I'm learning a lot of math a lot faster than I'd be able to. Because the way math is written, you know, I sometimes give a paper to a grad student to read and they come back a week later. And I say, so how far did you get? Well, I didn't really get past the first page, to be honest. Oh, really? Tell me. Well, actually, I didn't get past the first paragraph. I said, okay, fine, why not? Well, the first sentence is by the perturbation theory of non-local systems. And I don't know what any of that means, so that's, that's kind of where I'm stuck. Okay, you don't know how to read a math paper.

• 3:20:05Speaker 01

  You, you don't, you skip the first couple of paragraphs, go straight to, you know, the lema three. And that's where you can start working and work your way backwards to what the thing is doing. But in the future, there is no, I don't know what this means. Cause you just upload it to an L M and you start asking it questions and it will zero in on the zone of proximal development. The place where here's everything that you know, and here's the next thing you need to hear so that you start learning. And if people use that effectively, they can really supercharge their education. I don't just mean kids, I mean, old fogies like me can learn new fields. You know, to learn a field, like this is what our graduate students are going through. They, they have to completely rewire their brains. Like, learning math is still hard.

• 3:20:50Speaker 01

  Up until past the master's level into the PhD level, you were not expected to produce new mathematical theories. Everything that they can do is something that is in a textbook somewhere. And yet it's still extraordinarily, extraordinarily difficult for people to learn to solve problems that are solvable by lots of other people. Never mind problems that aren't solved. So, um, learning math is hard. Learning new math is even harder, discovering math, whatever, working out solutions to, to math problems. But I think it's going to get easier and, and this tool is, is going to be an amazing resource for people to be able to get to that zone of proximal development much more rapidly and develop their understanding much more rapidly.

• 3:21:33Speaker 00

  Yeah, I've been, um, I've been having so much fun on topics I'm not even, I would love to go back to grad school with the tools we have now. But one, one of the ways my brain likes to learn is constructing visual examples, toy examples. The simplest possible example that still contains the, the insight there. And especially if it's interactive, like if I can like move stuff. It's like, oh, you immediately like get it. All these words that the community agreed means something. You're supposed to know all the notation, all that kind of stuff. If I can just know that this little symbol means this, oh, okay, cool. All right. And it just, everything kind of falls into place.

• 3:22:16Speaker 00

  Also, you fall in love with the subject and, and I feel like I can take on any problem that's completely outside of my field. Like I can understand like, like protein folding. I can like play with the parameters and play. I can take a, a deep mind paper and actually understand by taking any aspects of it and playing with it. And what's a simple example? Ask crazy aforementioned pothead questions. So, yeah, this is super fun. The question is whether people will be motivated to learn. This is the, this is the problem is the problem of human motivation. That's the real question.

• 3:22:53Speaker 03

  Well, the key ingredients there are curiosity, asking good questions and being able to call bullshit. Like the critical advisor skills of like getting information where you're not the expert and knowing if you need to dig deeper or push back. And I think, you know, there will always be people who are curious and not curious and who ask good questions and don't, we need to be really promoting that more than we ever have. It's a little scary to me that we're expecting our undergraduates, our young students to interact with these tools when they're learning, when their brains aren't fully formed. We're asking them to have critical, not just critical reasoning skills themselves, but like advising skills.

• 3:23:38Speaker 03

  How do you take in information and tell if it's true or not, or how to question it? And that's a huge task for us in education is how do we train people to have good advisor skills when they're kids?

• 3:23:52Speaker 00

  Yeah, I think the hardest problem is undergrad and the hardest problem is like middle school. Cause like, what is the point at which homework starts getting hard? That, at that point, why do the homework?

• 3:24:07Speaker 03

  There's a temptation, right?

• 3:24:09Speaker 00

  That thing will just, at first, I'm saying, see, I've come to appreciate the value of doing difficult things because of all the suffering, but I think because I was forced to. But if you're not forced to do it because all them can, AI can just help you. I don't know what that, what kind of society that creates.

• 3:24:32Speaker 01

  But math has the longest head start on that question because calculators came out, what, 70 years ago? And we still ask kids to learn their times tables, at least some of us do. And those who said, no, kids don't need to learn times tables, you know? To the extent that teachers listened to that and kids listened to that and didn't learn, they really suffered. There's a great, great amount of human suffering resulting from people saying, you don't need to know times tables because calculators can do it. And my hope is that we can learn from that and say, just because a calculator can do it, you're going to be much better off if you can also do it yourself. Doesn't mean if I have to do some calculation, obviously I'm reading for, I'm reaching for a calculator. But I could in, in principle, do it myself.

• 3:25:14Speaker 01

  And that's what allows every time you give yourself that skill in master a skill. Kids love mastering skills. How happy are they when they learn to tie their shoelaces for the first time? You would think this is like, it's shoelaces, who cares? No, it's, it's the greatest thing ever. You want to celebrate every little bit of independence and, and you want to be able to think. You want to be independent from the machines. Just because the machine can do something doesn't mean you don't want to have the, the capacity to do it yourself.

• 3:25:42Speaker 00

  Well, this is where I have to ask, there's a lot of people that are full of anxiety. We mentioned graduate students, PhD, graduate, undergrad, high school students. What advice would you give, maybe different advice at different levels? To folks trying to figure out what to do with their life, to PhD students who are freaking out right now? Is there some wisdom you can give them?

• 3:26:09Speaker 01

  So, yeah, there, there's different things at every scale. Let me start with, um, with PhD students, I think in math. PhD students in pure mathematics, my advice to them is very simple. Do what we have always done. Learn the most difficult, the most abstract mathematics your mind can handle as much of it as possible. Your advisor should hopefully be guiding you through that process. And, and then just, I'll go one more, one step down from that. What we're seeing at the undergraduate level is a bimodal distribution. It used to be grades were basically Gaussian distributed. Now we're seeing a very bimodal distribution between the kids who are using AI to check their understanding.

• 3:26:51Speaker 01

  Wait, I didn't, the professor said this in class, what did I, what does that mean? I have a test coming up, here are my notes from the last three weeks. Start quizzing me and check if I really know something. Those kids are getting much higher grades. And then there are kids who, oh, this problem is set, is due in an hour. Can you just tell me the answers? I'll write them down in my own words. And so there's, there's like a decorrelation between homework progress and, and exams. And we're seeing this bimodal distribution. I don't know if you guys are seeing the same thing.

• 3:27:23Speaker 00

  So the, the implied advice there is to be one of those, the one that's using AI to learn and to grow to.

• 3:27:33Speaker 01

  Be the head, not the tail of the distribution.

• 3:27:35Speaker 03

  Yeah. Do you mind if we take a bathroom break?

• 3:27:38Speaker 00

  Yes, yes.

• 3:27:39Speaker 03

  We've somehow made intelligence this, this pinnacle. And.

• 3:27:48Speaker 03

  Have almost equated intelligence with morality.

• 3:27:51Speaker 01

  Right.

• 3:27:52Speaker 03

  And value. And value. Yeah. And I think that that couldn't be farther from the truth, right? Like, superintelligence does not imply super morality. And I think that's a huge concern with AI systems is if they are super intelligent, it has no implication that they will, that they will be super moral.

• 3:28:07Speaker 01

  Right. And we've been pushing kids to go to college.

• 3:28:09Speaker 03

  Absolutely.

• 3:28:10Speaker 01

  For how many years, yeah, that everybody needs to, yeah.

• 3:28:13Speaker 03

  Regardless of what they study or what the outcomes are.

• 3:28:16Speaker 01

  Exactly.

• 3:28:17Speaker 03

  And, and superintelligence seems to give you a super ability to rationalize. And common sense, people who, you know, farmers and the rest of the world, like, they have a sense that that rationalization is often bullshit. Just because you can make it sound really good and I can't argue with it doesn't make it right.

• 3:28:35Speaker 01

  Right. Education and wisdom are two different things.

• 3:28:38Speaker 03

  Yeah, exactly.

• 3:28:39Speaker 00

  I know it's difficult to do, but maybe Steve, do you have advice for grad, college, maybe young kids on what to do in this crazy, fast-changing time?

• 3:28:54Speaker 03

  Definitely be curious and read and learn all of the fundamentals. I think that's always been good advice. Even if your computer can do linear algebra better than you, you should definitely still learn linear algebra, build the intuition, same with calculus, same with physics. I think understanding history and understanding this change in the context of other great changes that have happened through human existence is important, but really being curious.

• 3:29:26Speaker 00

  I should say that this curious thing in the mind is trying to get stolen nonstop by the engagement economy. So like, the same engine, what makes you curious is the same engine that makes you doom scroll. Like, oh, that's a, so I think you have to kind of learn how to like, avoid all the distractions and focus on letting the curiosity actually explore a thing and soak something in, like, learn a thing. And actually, AI is really good at doing that because they're incentivized to give you value from the interaction versus incentivized to get you glued addicted to the thing.

• 3:30:11Speaker 00

  I don't know how people in like, high school, college also can just survive with the phone being around, to the distraction, the distraction box being right there. So avoid distractions and focus deeply.

• 3:30:27Speaker 03

  We all were bored when we were kids, sometimes. I think boredom is really important to actually be bored, be, like, curiosity and focus comes often from boredom. I would like to be bored more often. I try to do that sometimes.

• 3:30:43Speaker 01

  Yeah, we would come home from school and like, you know, finish homework in five seconds or whatever and just go outside and you hope to run into some other kids who also just went outside. And then you like, go and find something to do and come home when it's dinner time and.

• 3:30:57Speaker 03

  Yep.

• 3:30:57Speaker 00

  Yeah, you don't realize, actually, like, when the high school years, even before, somewhat undergrad, but it starts getting harder and harder, but like, high school's a really nice time to just like, learn everything. And you think, like, at least I remember, I don't know what it is like now, but I think it's just more, even more fun now. You think you're like, okay, when I grow up, I'll get to actually do something useful in this world. I, yeah, I'm learning, it's very nice and so on. But you realize, like, this is like, from a fulfilling perspective, from a building a foundation of an awesome life where you'll remember all of these things, this is like, the time to do it.

• 3:31:40Speaker 00

  And the whole world is forcing you to do it. And they will reward you for sitting there and learning. Cause like, when you get more, you know, even to like grad school, this kind of exploratory stuff, you get discouraged and, uh, God forbid, you get a job, then you're definitely discouraged from just, uh, learning, exploring and so on. So the time to explore is like, high school.

• 3:32:05Speaker 01

  By the way, like, pre-tenure faculty are on this like, oh, once I get tenure, then I can just sit back and relax and think big dots. But I have to like, produce enough, uh, stuff to, to get the tenure in the first place. And sort of, the, it's not what we want our young people to be doing is clawing their way there.

• 3:32:28Speaker 00

  Ankur, do you have, uh, advice you give to students?

• 3:32:31Speaker 02

  The problem with going last is that I think a lot of my advice got taken. So be curious was definitely one of them. Now, I would say that it's also important to do things for intrinsic reasons. So one of the things that I've noticed is, you know, a lot of people in our profession, a lot of people that go the academic route, the reason they do things sometimes is really external locus of self, because they see their value through how other people see them. And that's good enough to get you started. Maybe you start working very hard math so that people are impressed by you, right? But at some point, you run out of these milestones, you know, to hit.

• 3:33:13Speaker 02

  And then you really have to have an intrinsic reason for doing the things you do. And hopefully you have a sense of, you know, what's important to you, what are your values? So I think these days, you know, you can take our advice about, you know, being curious, but also the old playbook is really being thrown out. So you can take our high-level advice, but you're going to have to figure it out for yourself and you're going to have to be comfortable with uncertainty. And the only way I know how to deal with that kind of uncertainty is to have a sense of purpose about what you think is important, what you love about your job, and make sure that those are the things that you're doing. Cause the old playbook, if you just stick to that, you're going to end up taking all these AI-generated proofs and spending all your time putting them on archives, sending

• 3:33:58Speaker 02

  them to journals. I don't think that's anyone's intrinsic reason that they get into it is really to pad their CV. I don't know what'll be important in the future, but if you really have some higher purpose you believe in, then I think you'll be in a much better position to adapt to the changing world and figure out your own blueprint and be comfortable with uncertainty as the world changes around you.

• 3:34:19Speaker 00

  This is a trickier question, but a lot of people listening to this are feeling a lot of anxiety and fear about the future. It's both about a job, but also a future that's meaningful. Can you look back to your own life when you had that uncertainty, that anxiety? What's the advice you can give to that guy? To, to, to, when you're in that low point?

• 3:34:42Speaker 02

  So let me, yeah, let me try to address that question. It's a very tough question. I'll tell a short story that, um, you know, we spent our sabbatical in the Bay Area and we were living in Palo Alto, which was kind of the epicenter of college stress. You know, you could just feel it. You go to the cafes, it's just in the water at that point.

• 3:35:01Speaker 00

  Yeah.

• 3:35:02Speaker 02

  You know, uh, what extracurriculars are you doing right now that'll help? And then right when I came back to MIT, they asked me to give the convocation speech there. And it was just interesting thinking through that speech, you know, what do you say to this entire auditorium filled with freshmen and their families who, if things are a game to optimize how good you look on paper as a 17-year-old kid, they've won? And I just kept thinking about when I was in that same situation, when I was starting college, how excited I was just to learn all of these different things. You know, I'd sit there and plan out, uh, you know, my four years and, you know, I had little notebooks where I'd figure out what my classes would be.

• 3:35:46Speaker 02

  And if I took these classes in just the right order, I could take photonics as a sophomore. And I had no idea what photonics was, but it sounded fun and fancy. And I think that aspect, you know, so much of our society now is geared towards winning and looking good on paper. And I always wonder, to what end? You start off in high school and you can start, as soon as you walk in the door, optimizing your APs and your extracurriculars so that you can get to an elite school. And then when you get to that elite school, you might skip classes. Because, you know, maybe the important thing is to prepare for your interviews so that you can get a job at a frontier lab and then you can get some nice, you know, restricted

• 3:36:30Speaker 02

  stock units and, and live out your life. And that's great. You won, you won at every stage, but so what? You know, for me, the, you know, the best way to live a rich and fulfilling life is just to pretend you've already won and then figure out what you would do in that scenario. I remember the advice I got when I started at MIT's faculty, Sanjeev Arora told me, you know, the easiest way to get tenure is just to pretend you already have it and do what you would have wanted to do all along.

• 3:37:00Speaker 00

  Exactly.

• 3:37:01Speaker 02

  So I don't know if that advice applies to, you know, graduate students, the high schoolers too, but I think my advice stays the same. Pretend you already have tenure. What would you now do? Cause a lot of the people we're giving advice to have really so many different opportunities. Finding something that you really believe in is the only way I know to be fulfilled.

• 3:37:22Speaker 03

  I've heard, uh, career advice from someone I respected, that a fulfilling career is something where you can work on a problem that is important, that you find interesting and that you're good at. And if you can get two out of those three, you're pretty lucky. If you can get all three, you're going to be, you know, happy for the rest of your life. And so having the right preparation, figuring out what you're good at, what's interesting, what's important, and then preparing yourself, you're not guaranteed to get all three of those, but you want to be ready when that opportunity comes. And then continue doing the things you actually enjoy doing. I think that's great advice.

• 3:38:06Speaker 00

  I think we should also put into context and be quite honest. I do think a lot of the advice you gave applies, but we should be honest that I'm sitting with three world-class human brains.

• 3:38:23Speaker 00

  Not that have figured, found their path. You're survivor bias. So, you know, there's a lot of people listening to this that are like, it's like, they're pretty sure they won't find their path. It's the fear, it's the anxiety, it's the uncertainty. It's stumbled a bunch of times. I think, to be fair, some of the most interesting people, right? Stumble quite a bit in life, whether it's the external environment or the internal, whatever the universe brings. And in that, I just know a lot of people just feeling and monitoring it right now because of everything AI is doing to our society.

• 3:39:05Speaker 00

  There's a lot of people just terrified and really anxious. And I think there's a lot of wisdom in what Unkar says. You can be paralyzed by that fear, or you can imagine already won. What would the person who has won do in this situation? And that alleviates the fear. It actually takes the, gives you the optimal path under uncertainty, under heavy uncertainty, under a lot of uncertainty. And don't use the old playbook. It'd be, try to be the person that figures out how to write the new, the next playbook. I think Max, Max Tegmark, I've spoken with many times, because he's a wild one. He did a lot of weird and beautiful things in his career, but he also did the mainstream stuff. He said, you have to do a little bit of both.

• 3:39:50Speaker 00

  So that you don't, you still, you still work enough within the system that you're part of the community, you're part of the flow of, in that case, it's science and the development of things. But also do enough renegade and wild and weird things to pave your own way. Both somehow, and find that balance. And not be afraid to be a little bit weird, but also become very skillful and good at being part of a community and a respectful part of the community. So, both of those. But the, just like, it's like Tom Waits said, I like my town with a little drop of poison.

• 3:40:34Speaker 00

  It's just a little bit of weird, or sometimes a lot of weird. This makes for some beautiful humans.

• 3:40:40Speaker 01

  I think that community part is actually going to be even more important in the future. We've had this maybe 30-year experiment with how much can we do with machines? And now that we can do a ton with machines, my hope is that we're going to realize we actually prefer to do things with humans. We prefer to fly in and have an in-person conversation. We prefer to have an oral discussion instead of just reading a paper. I, I think this, there's like a peak to how much our day-to-day experience. I mean, in, in terms of, yes, I get a lot of fulfillment from my scientific work and from my academic career, but going home to my kids and, and going to the beach,

• 3:41:22Speaker 01

  if, if FLT gets solved and just throwing a ball with them is, there's nothing like that. Like, that's the meaning of my life. This other stuff is the cherry on top. But like, the foundation of it is, is family, is, is people, being around other people that I care about.

• 3:41:37Speaker 00

  And I think as AI commodifies intelligence more and more, we'll realize that more and more, that in, intelligence and all the, all the different puzzles that we solve is, is the, just the cherry on top or a cherry to the side. The main thing is other humans in our lives. This is, I can't tell you how much of an honor it is that you would come down in person for a conversation. I've become a huge fan of all three of you. It's, feels a bit surreal and it's a big honor that you would talk with me today. This was like, super fun. Thank you so much. Thank you so much, Unkar, Steve, Alex. This was awesome.

• 3:42:20Speaker 01

  Thanks for having us.

• 3:42:21Speaker 02

  A lot of fun for us too. Thanks for having us.

• 3:42:22Speaker 00

  That's great.