AI is changing mathematics -- but not how you think

Math has created some of humanity's oldest intellectual challenges. And AI is completely changing the game. This year AI solved one of math’s most difficult problems. Since then, OpenAI published hundreds of new math results. So is this the end for the people who’ve dedicated their careers to pushing forward math? We get into it with NPR science correspondent Geoff Brumfel, who weighs the risks and rewards of AI’s math takeover. Interested in more technology and AI news? Email us your question at [email protected] mailto:[email protected]. Support public media with NPR+ and enjoy perks for over 25 podcasts like this one. This show’s perks include sponsor-free listening. Learn more at plus.npr.org http://plus.npr.org. See pcm.adswizz.com https://pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences. NPR Privacy Policy https://www.npr.org/about-npr/179878450/privacy-policy

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  This message comes from the Southern Environmental Law Center. No one voted for dirty air and water. That's why S E L C is fighting back to defend the healthy environment everyone depends on. Because it's not political, it's fundamental. Learn more at S E L C dot org.

• 0:18Regina Barber?

  You're listening to Short Wave from NPR. By now, you may have heard that AI is revolutionizing the field of mathematics.

• 0:27Speaker 02

  What is going on with AI and math?

• 0:30Speaker 03

  AI is all over mathematics.

• 0:32Regina Barber?

  One of the hardest problems in mathematics may have just been solved, not by a human, but by artificial intelligence. AI appears to be solving some really complex problems. And that's leading mathematicians to question what the future of the field might look like.

• 0:47Speaker 02

  Or if the field has a future at all.

• 0:49Regina Barber?

  That's why you're here, Jeff, to help us sort it all out.

• 0:52Speaker 02

  I will say, things are not all what they seem. I've been looking into what's going on in mathematics over the past month or so. And I've learned there's a lot more than you might realize. It's a revolution, to be sure. But the way the media has been covering it isn't always really the way things look on the ground.

• 1:13Regina Barber?

  Today on the show, science correspondent Jeff Brumfield guides us through how AI is breaking the field of math.

• 1:20Speaker 02

  Don't worry, there won't be that much math.

• 1:21Regina Barber?

  There's going to be a ton of math. We'll discuss the latest breakthroughs, how they upended the field.

• 1:27Speaker 02

  And whether this really does spell the end for mathematics as a profession.

• 1:32Regina Barber?

  You're listening to Short Wave, the science podcast from NPR.

• 1:41Speaker 00

  Stay tuned for the following announcements and messages from our sponsors. This message comes from ProLon. Want to look and feel your best this fall? ProLon's five-day fasting-mimicking diet is a clinically-developed nutrition program with the goal of promoting fat loss while protecting lean body mass. Developed at USC's Longevity Institute, it aims to assist the body in entering a fasting-like state that helps reset metabolism, target visceral fat, and support healthy metabolic markers. Get 15% off plus a bonus gift when you subscribe at prolonlife.com/NPR. This message comes from Betterment. Dan Egan, VP of Behavioral Finance and Investing, explains why Betterment was designed to take the time-consuming work out of smart investing.

• 2:26Speaker 02

  I was doing all of these things that were pretty straightforward to implement. I just needed to spend my time doing them. Betterment automates those same practices so that I know I'm doing portfolio management and goal-based planning without me having to spend hours of my life doing it.

• 2:40Speaker 00

  Learn more at betterment.com. Investing involves risk. Performance not guaranteed. This message comes from Dell Technologies. Interruptions happen at work. But with the Dell Pro laptop powered by Intel Core Ultra with V Pro, built with optimized

• 2:55Regina Barber?

  battery and built-in intelligence, your tech won't slow you down. Dell.com/Dell-Pro. Built for you.

• 3:05Regina Barber?

  All right, so Jeff, we're talking about AI revolutionizing math. Where do you want to begin?

• 3:10Speaker 02

  I know I promised there wouldn't be that much math, but let's start with the math problem. You ready?

• 3:15Regina Barber?

  I'm ready.

• 3:16Speaker 02

  OK, it's 3 times 4 minus 5 plus 7.

• 3:21Regina Barber?

  So, yeah, so that's 12. And then the other chunk was plus 2, so that's 14.

• 3:27Speaker 02

  You are correct.

• 3:28Regina Barber?

  Yes.

• 3:29Speaker 02

  You want to hear what ChatGPT told me almost exactly three years ago?

• 3:33Regina Barber?

  Was it not 14?

• 3:34Speaker 02

  It was 20.

• 3:36Regina Barber?

  Oh, my.

• 3:37Speaker 02

  Exactly. And, you know, in those early phases, it kind of made sense because, like, AI is all about predicting the next word you want to see, but it doesn't in this kind of semi-random way. So something really structured, like mathematics.

• 3:51Regina Barber?

  With rules.

• 3:52Speaker 02

  With rules. Sometimes it'll give the right answer, sometimes it'll give the wrong answer. It might say 2 plus 2 is 5 every now and then, or 3 every now and then. I literally did this.

• 4:02Regina Barber?

  It's a theoretical mathematician. It's like, it could be.

• 4:05Speaker 02

  It could be. So that was back then. But now, when AI generates math, you know, it appears to be checking the math to see if the answer is right. And if not, it sort of tries again. And it's this, this feedback loop that's really started to supercharge math.

• 4:24Regina Barber?

  Which is cool, but researchers don't really know what's going on internally with those models and how they're getting that right answer, right?

• 4:32Speaker 02

  The truth is, we don't know about how these big companies are building their models to do math. And so there is a bit of a black box problem here. But sort of, the one thing I've heard is having an answer to check makes it easy to correct these mathematical hallucinations. Where an answer was 14, but now it's 20. So, you know, everyday math got good. And then things really started to accelerate last summer.

• 5:00Regina Barber?

  So what happened last summer?

• 5:01Speaker 02

  Google claimed its large language model, and just a reminder, large language models like ChatGPT or Claude, we are all using them now.

• 5:09Regina Barber?

  OK.

• 5:10Speaker 02

  But its particular large language model could solve math at the level of like a world-class student. It got a gold medal at the Math Olympiad.

• 5:19Regina Barber?

  So it's not only taking our jobs, it's taking medals away from our children?

• 5:24Speaker 02

  Something like that.

• 5:24Regina Barber?

  OK.

• 5:25Speaker 02

  Yeah.

• 5:25Regina Barber?

  I knew it.

• 5:26Speaker 02

  I don't know if they actually gave it a medal.

• 5:27Regina Barber?

  It better not have.

• 5:29Speaker 02

  Right after that, mathematicians started seeing big changes. So I talked to Javier Gomez-Serrano, who's a mathematician at Brown University. He said about a year ago after that Math Olympiad finding, L L Ms really started to get powerful.

• 5:47Regina Barber?

  And lately, over the last nine to 10 months, I've been using frontier L L Ms to, to help me to do mathematics in various capacities. OK, so so far, Jeff, this sounds to me like it's a story of L L Ms being used everywhere by people. But lately, there's been this narrative that AI is now replacing mathematicians. Can you tell me why all of a sudden we're talking about this mathpocalypse or mathapocalypse, which I don't actually believe?

• 6:16Speaker 02

  Yeah, well, you know, this mathpocalypse narrative has really come down to one company, and that is OpenAI. They've just released over 370 new results in mathematics. They put them all on GitHub.

• 6:32Regina Barber?

  Wow.

• 6:33Speaker 02

  They say it took them an average of three hours of compute to come up with each one. You know, one such problem was something a mathematician named Dana Mashkovitz at UT Austin had been working on for over a decade. She took a look at the solution and concluded it was probably correct, which kind of begs the question, like, what's she going to do now? I mean, this was a life, a life plan for me, right? And now it's, it's gone and I need to, to reevaluate. Dana is just one of hundreds, thousands of mathematicians around the world who's looking at all these papers. And frankly, there's a certain amount of despair in the mathematical community.

• 7:16Speaker 02

  But, you know, I got to say, the more I've looked into this, I've had more and more questions about whether not everything is exactly the way it looks.

• 7:26Regina Barber?

  I knew it. OK, so what do you think is going on here?

• 7:29Speaker 02

  OK, so to explain how much more complicated things actually are, let me talk about this one big solution that maybe you've already heard about. This is something called the Navier-Stokes problem.

• 7:40Regina Barber?

  Right, yes. So that was one of these so-called millennium prize problems, one of the biggest problems of our time, right?

• 7:47Speaker 02

  Right, right, right. And just to refresh your memory, the Navier-Stokes equations describe how fluids flow. So these are equations that are used every single day in physics and engineering for airplane wings, manufacturing, even like modeling how blood flows through the heart. The thing is, though, you know, even though these equations are everywhere, as you know, nobody really understands why they work so well. I mean, they're over, well over 100 years old at this point. And we just know they work. So the Navier-Stokes problem is described, it's this millennium prize problem, is to find a case where these equations, quote unquote, break down or stop working properly.

• 8:29Speaker 02

  And the thinking is, if you can find a place where these equations no longer can describe what's going on, right? That will tell you something both about how they work and, you know, how to build something better.

• 8:43Regina Barber?

  Yeah, so in September, OpenAI said that it found a solution, right? Like, that's when I think all of this really broke into the mainstream.

• 8:50Speaker 02

  Right, right. But the more I looked into it, the more the sort of solution OpenAI claims to have found didn't seem quite as solid as it was being reported. So first of all, let's talk about the written proof that was produced by AI. This thing is 166 pages long.

• 9:08Regina Barber?

  OK.

• 9:09Speaker 02

  And it is ugly. Javier Gomez-Serrano actually studies Navier-Stokes, and here's what he said.

• 9:16Regina Barber?

  It's not written for humans. And the paper does-

• 9:19Speaker 04

  And explain what parts are important.

• 9:22Speaker 02

  So it's just like this mathematical AI stream of consciousness.

• 9:27Regina Barber?

  Oh, gosh.

• 9:28Speaker 02

  That's like spewed out on the page. Yeah, I want to make clear because I think some people think, oh, the AI is operating at a level humans can't understand. No, no, no. This is like, we've all read AI Slop. I won't go so far as to call this AI Slop, but this is AI-generated mathematics. And it comes with some of those same problems. So the reason people have still had a lot of faith in this solution. One, it does look like a solution that humans were already headed towards. But the other is this thing called Lean. It's a programming language that I honestly had never heard of before. Have you ever heard of Lean?

• 10:05Regina Barber?

  So I've not heard of the coding language, but I have heard of the drug, yes.

• 10:09Speaker 02

  I haven't heard of the drug, but I hadn't heard of the coding language either. Anyway, Lean is this coding language that's used in mathematics to check people's proofs. And it works like this. You take a written proof, you convert it into Lean computer code, and then you feed that code into Lean to see if it's right. Basically, if the code compiles, the proof is correct. So when this AI wrote its 166-page proof, it also turned that Navier-Stokes proof into Lean code. And then a non-AI computer checked it and said, yep, that looks right. And for a lot of mathematicians, that's strong evidence this is correct.

• 10:49Speaker 02

  But then, when somebody, researchers went back and they looked, they found that the AI made unexpected changes to the work when it translated the proof from the written paper into Lean. So Anders Hansen is a professor of mathematics at the University of Cambridge. And he showed me this equation in, deep in the proof, where a four was switched to a five.

• 11:13Speaker 04

  Four, five, who cares? Well, here's the problem. This statement here, which is what is in the natural language proof, is stronger than what Lean proves.

• 11:23Speaker 02

  In other words, you know, the Lean proof had a weaker statement than what was in the natural language proof. Now, in this case, that mismatch, I don't think means the larger solution is wrong.

• 11:33Regina Barber?

  Okay.

• 11:34Speaker 02

  But Hansen says that's not the only case in this paper, and it's certainly not going to be the only case in these 370 papers. And so all of this stuff needs to be graded by humans.

• 11:45Speaker 04

  I would suggest that we should check them seriously.

• 11:49Regina Barber?

  Okay, Jeff, so what I'm hearing is that AI knows how another machine will check its work, so the proof in itself could be wrong.

• 11:56Speaker 02

  Yeah, and actually, in one case, there was a proof that was wrong in these 370 papers. There was a sign error, you know? And that actually led OpenAI to retract three of its results.

• 12:08Regina Barber?

  Okay, so, you know, when we're talking about sign errors, we mean like a positive number should have been a negative number or vice versa.

• 12:14Speaker 02

  Yeah, exactly.

• 12:15Regina Barber?

  So does this mean that all these proofs, these like 300 and something papers, some of them could be wrong?

• 12:20Speaker 02

  Some of them could be wrong. But I have to say, probably most of them are right, according to the mathematicians I spoke to.

• 12:28Regina Barber?

  Okay.

• 12:28Speaker 02

  Because they do feel strongly that, you know, the AI is doing really interesting work. But even a problem that's mostly right can lead to other problems. So I spoke to this one mathematician named Max Lieblich. He's chair of the department at the University of Washington in Seattle. And he said, a lot of little errors can add up.

• 12:52Speaker 04

  If in each layer you have enough imprecision that you make a few mistakes, then over time, you've built something that just collapses.

• 12:59Speaker 02

  And this actually happened in Italy about 100 years ago in his field of algebraic geometry. Mathematicians got a little loosey-goosey with their calculations. And slowly, over the time, the field had so many inconsistencies, it actually eventually collapsed under its weight of accumulated errors. And they had to start over, basically, in the 1930s. Now, this might not happen, you know, to all of mathematics with AI, but it is something that means these mathematicians have to keep a really close eye on what the AI is doing and check the results. And so even though we're getting this flood of papers, it could take years for actual progress to happen in the field as a result of all this checking.

• 13:41Regina Barber?

  See, Jeff, I'm telling you, mathematics is not over because you at least need to check all this stuff that AI is producing.

• 13:47Speaker 02

  That's right, that's right. So, you know, I still think some mathematicians would call that a mathpocalypse because if your job used to be to think big questions about math and the universe, and now you're checking a computer's work, that doesn't feel very appealing to a lot of them.

• 14:03Regina Barber?

  That's true. Okay, let's talk about why this story has resonated with people outside of math. Like, it really does raise the question of whether AI is about to replace humans. Like, you know, they're going to take our jobs. Does your reporting tell us anything about that?

• 14:18Speaker 02

  I think it does, actually. You know, it's really clear that AI is changing mathematics forever, probably in a way that's similar to the, how computers changed math in the last century.

• 14:29Regina Barber?

  Yeah.

• 14:30Speaker 02

  But it's also clear to me, as I get into the nuance of all this, that these tools, as powerful as they are, are not going to replace mathematicians. As you said, Max sees this as a really exciting time, actually.

• 14:42Speaker 04

  In my lifetime, I'd have a chance to participate in one of these moments where we get to decide as a community what it means to do mathematics and be a mathematician.

• 14:51Regina Barber?

  Yeah, things are going to change. You're not using slide rulers anymore, you know?

• 14:54Speaker 02

  No, it's true. It's true. I mean, and, you know, math has actually been, in some ways, very open to change over the years because of the way computers can help. But I think with this latest change, as unsatisfying as it is, it's really hard to say what the future of mathematics looks like in this moment.

• 15:17Regina Barber?

  This episode was produced by Juan Diego Ramirez and edited by our showrunner, Rebecca Ramirez. Jeff and Tyler Jones checked the facts. Jimmy Keely was the audio engineer. I'm Regina Barber. Thank you for listening to Short Wave from NPR.

• 15:34Speaker 00

  Stay tuned for the following announcements and messages from our sponsors.

• 15:38Speaker 05

  This message comes from the Sierra Club, fighting to ensure America's public lands are protected and accessible for everyone. The Sierra Club protects places like national parks and forests through legal action, advocacy, and nationwide grassroots organizing, because the future of public lands is worth fighting for. Join the fight to keep public lands protected and accessible for everyone. Donate at sierraclub.org/NPR. This message comes from Angie. Angie has been helping homeowners find skilled pros to get their home projects done well for over 30 years. For everything from minor repairs to major remodels, find reviews, pricing, and booking at angie.com or on the Angie app.