Terence Tao on how AI math solutions miss the point

Terence Tao says mathematics is about understanding, not just right answers, and quick AI solutions are forcing soul-searching in the field.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  WNYC Studios is supported by Wise. Wise, the smart way to manage the currencies you need around the world. With a Wise account, you can send, spend and receive in up to 40 currencies with only a few simple taps. Be smart, get wise. Download the Wise app today. Terms and conditions apply.

• 0:17Ira Flatow?

  Science Friday is supported by MathWorks, creator of MATLAB and Simulink software for technical computing and model-based design. MathWorks, accelerating the pace of discovery in engineering and science. Learn more at mathworks.com.

• 0:33Speaker 00

  WNYC is supported by Bear. Science is about asking questions, testing, trying again. It's not easy, but it's relentless. It evolves, and that's why science delivers. Learn more at sciencedelivers.com.

• 0:50Flora Lichtman?

  Hey, it's Flora, and you're listening to Science Friday. Mathematics has been top news in a way it rarely is, almost entirely because of AI. Last month, OpenAI claimed that one of their models solved the Navier-Stokes Millennium Prize problem. Mathematicians have been working on this problem for decades, and according to OpenAI, their agents solved it in 88 hours. Then, a few weeks later, OpenAI announced results for over 350 previously unsolved math problems. Now, there's debate about how well these problems have been solved and the model's ability to explain their work, but regardless, the news has prompted a kind of existential conversation among mathematicians.

• 1:32Flora Lichtman?

  In an age of AI, what's the role of the human mathematician? Dr. Terence Tao is considered one of the greatest mathematicians of our time. He was taking college-level math courses by age nine, was a full mathematics professor at 24, and won the Fields Medal, aka the Nobel Prize of math, at 31. He's also the author of the upcoming book, Six Math Essentials, which tells the story of math and how it shapes our world. Terry, welcome to Science Friday.

• 2:04Speaker 03

  Oh, pleasure to be here.

• 2:06Flora Lichtman?

  You told The New York Times recently that in mathematics, this moment feels like soul-searching at gunpoint. That's a arresting statement. Walk me through it.

• 2:19Speaker 03

  Yeah, so mathematics has been very stable and quiet, and it hasn't made the headlines, as you said, for over a century, pretty much. You know, we often focus inwards, trying to solve our own problems, mostly of interest to other mathematicians, although every so often, the things we do are of interest to scientists and engineers. So we largely do our own thing, and you know, we haven't had external actors come and sort of try to play with our problems so much. But suddenly, we have these sort of 800-pound gorillas in our profession, sort of feasting on our open problems, which, on the one hand, is great. We're getting all these new answers to problems that we wanted solutions for. But it is disrupting a lot of things that we are traditionally interested in doing.

• 3:05Speaker 03

  For instance, when we solve problems, often these problems are not interesting for their own sake. You know, the Navier-Stokes regularity problem doesn't help us advance weather prediction or any sort of practical impact right away. But often, the journey to solve these problems, the story, the collaborations along the way, the partial progress we write up, that is really what's valuable, and it helps mathematics become more useful in the real world. And that's been kind of lost in this big scramble to use these machines to grab solutions for these problems without getting any of the understanding that normally comes with it.

• 3:39Flora Lichtman?

  You said 800-pound gorillas. 800-pound gorillas feasting, those are the AI models.

• 3:44Speaker 03

  Yes, yes. So they've become very, very good at mathematics. So mathematics has been traditionally considered a very hard topic for humans, but it has many of the features that make it particularly good for AI. In mathematics, the problems we work on can be objectively verified. You can check on a computer whether the answer is correct or incorrect, and that turns out to be part of what makes AIs very effective at these problems. So, yeah, we had this sort of monopoly, if you wish. Professional mathematicians were almost the only people who could work on these problems. And now we've bought machines that can give us these solutions, but the solutions weren't actually what we wanted. They were just kind of nominal goals, you know, the flags on top of a mountain to sort of move towards.

• 4:28Speaker 03

  One of my friends called them lighthouses, you know, not destinations to go to for their own sake, but they illuminate the landscape around them. And so now we've kind of conquered these lighthouses without getting the understanding. So we now have this backlog of hundreds of proofs that are interesting, and slowly, mathematicians are going through them. But we don't have any help from, say, OpenAI to help us. Normally, when a human mathematician solves a major problem, they can come do talks, we can call them up, they will answer questions, and we can have really productive conversations with them. There's nobody at OpenAI I can call to help me understand this dump of 300 papers.

• 5:10Speaker 03

  We have to do it ourselves, and we will, but it's going to take a lot longer than normal. And in the meantime, there were many, many junior people who had research projects which have just kind of been made obsolete by this dump. So a lot of junior people are scrambling. Some people are suddenly now very, very secretive about what they're doing. They're now terrified that if they mention their research directions, which is normally what you do in our subject, someone will just put them into an AI and solve their research project for them before they can actually produce some valuable understanding of that problem. So priority used to be very important in a profession, but now priority is degenerating

• 5:52Speaker 03

  into who can feed their problem into an AI the fastest, which is a completely useless competition.

• 5:58Flora Lichtman?

  So what do you see the purpose of human mathematicians? Like, what is the purpose of math, and has that changed or not?

• 6:05Speaker 03

  Right, so mathematics fundamentally is about understanding. I mean, ostensibly, it looks like it's about answers. And if you take a math class in high school, it seems like that's all you're doing. You're given homework where you have to say, find x, or how many apples does Jane have, or whatever, and you hand back answers. But that's only part of what mathematics is. There's a lot of real-world phenomena which are governed, at least in part, by mathematical models, you know? So if you want to study climate, or traffic, or the energy grid, or can you build a more efficient solar energy cell, whatever. And so you do want answers to these questions, but in order to get the right answers, you need to understand what these models are and what their limitations are.

• 6:50Speaker 03

  Sometimes the model you have, on paper, it works great, but it actually doesn't describe the real world. In 2008, we had a massive financial crisis, in part because investment bankers were using a certain mathematical model to model mortgage defaults, and they were basing all these fancy derivatives and financial instruments on this model. And it turned out this model was only an imperfect representation of mortgage defaults, and it didn't take into account, in particular, systemic shocks to the entire mortgage market. And basically, people were using models that they didn't understand. And the rest is history, actually. So I think one of the purposes of a human mathematical community is to provide understanding

• 7:32Speaker 03

  of the mathematical models that are now increasingly used everywhere, including AI. AI is built on lots and lots of quite advanced mathematics now. And there's now this fear that we are deploying AI far faster than we are understanding its strengths and weaknesses. So, yeah, we need the understanding to catch up, and this is all that mathematicians will increasingly play in the future.

• 7:54Flora Lichtman?

  Why do you think this is so destabilizing for mathematicians?

• 7:58Speaker 03

  Right, so historically, we have organized our profession around the assumption that solving problems was difficult, and in particular, proving that the solution's result was true or false was hard and scarce and valuable. And so we've organized our entire culture, our institutions, our prizes around this scarcity. So if a young postdoc solves a major unsolved problem, they will get papers in prestigious journals, they've been prizes, they'll get job offers, this type of thing. And this is how we have organized our field. Proofs were hard and scarce, and so that was the valuable thing. But it was always a proxy.

• 8:40Speaker 03

  Most of the problems in pure mathematics are not useful for their own sake. You know, similar to how a high school math homework question, you know, what is the side length of this triangle? You're unlikely to have to use that particular problem in real life. It was always a proxy for just how good you were at problem solving, or just dealing with abstract situations that required some quantitative thinking. So we maybe have over-indexed on ability to solve problems, but now we have these machines that can do that for us. So there's now this historic decoupling between the traditional task of solving problems, which is what has motivated our field, and now we're seeing that this objective is being

• 9:23Speaker 03

  decoupled from the more fundamental objectives of mathematics, understanding what is going on, how to apply things to the real world, knowing the limitations of our mathematical models as well as our successes, and training the next generation of mathematicians to continue these traditions.

• 9:40Speaker 03

  And so the soul searching is how do we keep the long term health of our field now that our main guiding proxy for progress has been optimized to death?

• 9:53Flora Lichtman?

  You've talked about bringing humanity to math. What does that mean to you? And what does that mean to you now in this age of AI?

• 10:02Speaker 03

  Right, so math is half science, half humanities. But somehow for marketing purposes and particularly for applying for funding, we've kind of emphasized the science half and we have downplayed the human half. But math is a very social activity.

• 10:18Speaker 03

  Part of what gives a mathematical value is not just its application to the real world, but to what it excites other mathematicians or other humans. I think mathematics not only is a technical subject that other mathematicians appreciate learning more about, but knowing a bit of mathematics just as a regular person helps make the world less scary. You know, we are surrounded nowadays by technology we don't understand. That we have these magic devices to communicate across the planet. Now we have these AIs that can talk to us almost at the level of humans, or sometimes they can sound superhuman.

• 10:52Speaker 03

  And sometimes they fail and now it becomes scary. But knowing mathematics helps you understand technology and demystifies it and the world just becomes less scary. So I think mathematics has value in just sort of helping people be a piece of the world around them.

• 11:11Flora Lichtman?

  That's such a profound thought and it also reminds me a little bit of these proofs that are coming out, right? Because that must be also why it's so destabilizing. If you read these proofs and you don't have someone to ask about them, what does that even mean for understanding?

• 11:29Speaker 03

  Right, it's like this artifact, like this, you know, it has all the outward markings of an achievement, but it has none of the substance. People can sort of say, wow, this problem got solved, but I can't explain it and you can't explain it. We can just sort of gulp at it. And that's no way to do science. And now these problems, like these solutions are kind of being dumped by actors who just don't care about cleaning it up and they say, okay, someone else can explain this. We're just, we're done, we're going to move on to our next benchmark or something.

• 12:06Speaker 03

  Eventually, we will dig into these and understand it, but it will take quite a bit of time and a lot of volunteer labor. So yeah, it is, at the very least, very inconsiderate. Yeah, and it's forcing us to reevaluate, why did we want these solutions in the first place? It wasn't just to get the flag or the trophy. You know, the process was actually much more valuable.

• 12:30Flora Lichtman?

  I can hear the hurt in your voice.

• 12:33Speaker 03

  Yeah, we operate to a large extent on cultural norms and unspoken rules of etiquette and sort of a gift economy. For example, we refer each other's papers and this is volunteer work. So we are realizing that somehow our informal gift economy is not compatible with sort of this 21st century capitalist, you know, optimizing everything, 800 pound gorilla, which is now in our community.

• 13:00Flora Lichtman?

  I have to take a break, but when I come back, I want to ask you about your book. Is that okay?

• 13:10Flora Lichtman?

  Science Friday is supported by Planet Visionaries, the podcast created in partnership with the Rolex Perpetual Planet Initiative. Stay tuned for a trailer and subscribe wherever you get your podcasts.

• 13:22Speaker 04

  Do you want to hear about visionaries creating a better world? People working in conservation, exploration, science and tech alongside storytellers imagining and creating a better future. I'm Alex Honnold, professional rock climber and founder of the Honnold Foundation, bringing you season six of Planet Visionaries. This season, I speak to guests like National Geographic photographer Tom Peszek and Anna Rathman, executive director of the Jane Goodall Institute, USA, in partnership with the Rolex Perpetual Planet Initiative. Watch Planet Visionaries on YouTube or wherever you get your podcasts.

• 13:52Speaker 05

  And from the Alfred P. Sloan Foundation, working to enhance public understanding of science, technology and economics in the modern world.

• 14:05Flora Lichtman?

  Tell me about your book, Six Math Essentials. Why did you want to write it?

• 14:11Speaker 03

  Okay, so I have long felt that mathematicians do a lousy job of public outreach, to be honest. I think we spent, we indexed a bit too much time on solving problems of interest to our colleagues and sometimes not explaining what we do to outsiders. And maybe that's part of why we are now being hit so hard by external developments. But, you know, I see when I talk to people at parties or whatever, it's very hard for them to relate to what it is to be a mathematician, because I think many people, they liked mathematics maybe when they were in primary school. They may just ask questions about numbers and shapes, you know, just out of pure child like curiosity.

• 14:54Speaker 03

  But at some point, many, many people hit some sort of wall in mathematics. There's a point we have to start focusing on drill and getting answers correct and teaching people very precise mathematical techniques. You've got to solve for x using these very abstract looking rules. And suddenly math is becoming this very technical subject, almost like casting magic spells. And you know, you're losing this understanding of what's actually going on. Now, if you keep going and if you persevere in mathematics and you go to graduate school, that's when the understanding comes back. But this is something which most people don't get to. So I think what I wanted to convey in my book was that the type of things that research

• 15:36Speaker 03

  mathematicians do these days, either in pure math or in applications to the real world, it is still tied to the basic concepts that you already are exposed to as a child, you know, numbers, shapes, time, change, big and small. You know, we get the fancy words, you know, so the mathematics of big and small numbers we call analysis, the mathematics of change, we call dynamics. But it all comes from very basic concepts that you've seen in primary school even, but they've evolved and matured and we've just massively expanded in a concept like a number, for example. The concept of a number has been revised and revised half a dozen times over history. It used to be counting numbers, one, two, three, four, you know, how many sheep do you have? How many, how many rocks do you have?

• 16:22Speaker 03

  But then we have negative numbers, you kind of have negative three sheep, but negative three is a number, you have fractions and real numbers, complex numbers. We've enlarged what numbers mean, what geometry means, what big and small mean. But it's still buried underneath all this fancy mathematics is very common sense intuitive notions and I wanted to sort of convey this continuity of mathematics to the real world.

• 16:49Flora Lichtman?

  Your book makes math a narrative, which is very helpful for someone like me. I think I learn that way. Does it help you in any way understand what you do?

• 16:59Speaker 03

  Yeah, so I operate very much in terms of narratives and analogies now. I mean, when I was a student, I had a fairly traditional education, but as I transitioned into research, I found that the research that I really enjoyed and I was productive at was the one where I could see a narrative where, you know, like you see an equation and there's x's, y's and z's and like you could view it as a game of pushing symbols around and that's one way to view mathematics. But, but you can often give math problems personality, like, you know, this term is a good term, you want to keep it, this term is a bad term, you want to somehow hide it. And, you know, and then you've got all these very strategies to sort of help the good guy win and make the bad guy lose.

• 17:40Speaker 03

  And you can in some sense, humanize mathematics or draw analogies with the real world or some physical situation. You know, you may have some abstract problem with 100 dimensions, but there's some problem in one or two dimensions that you can just see and understand with your human senses and it gives you some idea how to deal with this very abstract problem. And it's really fun, especially when you're working with a colleague and you explain your way of thinking and they explain their way of thinking and suddenly you meet in the middle and like you both see things in a different way. It's actually a great feeling.

• 18:14Flora Lichtman?

  In an interview, you said that you wished you'd paid more attention to humanities. And I was so curious about that. Do you still feel that way? And this is from a long time ago. This is deep in the archive.

• 18:27Speaker 03

  Yeah, I did not understand the humanities much. Actually, I think the humanities, the humanities I enjoyed the most were the languages. So I learned some French and German because I liked the rules of grammar and like declining verbs and things that looked, sounded, I felt a little bit like mathematics, actually.

• 18:45Speaker 03

  So I was fairly good at sort of the grammatical side of the language, but you know, asked to write some, you know, poem in French or something like, I was terrible at these things.

• 18:58Speaker 03

  Yeah, so I didn't really get the humanities for a long time until I sort of started seeing them emerge in my profession. I said narrative is important and I find it helped me teach. Like you can make a subject become more alive when you have a little anecdote about some long dead mathematicians trying to do something very silly and accidentally discover a new type of mathematics. I'm still, you know, English and the arts, I'm still, I still, I'm not great at. I have very artistically mannered friends who are moved to tears by a painting and I can see, okay, I can see that technically the brushstrokes are really impressive, but why, why were you so moved?

• 19:41Speaker 03

  Sometimes I still struggle with that. Hopefully I'll understand that too.

• 19:47Speaker 03

  But over time, my whole life experience has been, you know, the more you respect and appreciate other points of view, whether it's in science or humanities or whatever, the richer your own experience is. So I try to be open to other ways of experiencing the world.

• 20:02Flora Lichtman?

  Well, yeah, I wondered in this moment if you feel like mathematicians, or maybe this is just a question for you personally, like, do you have the tools you need to adapt to the 800 pound gorilla?

• 20:16Speaker 03

  Yeah, so, well, there's different types of tools. So for example, at a technological level, one of the complaints we have is that right now the technology to solve these fancy open problems, there's only a few tech companies who have the tools to do this and often it's only internal models. A regular practicing mathematician may only be able to have access to a much cheaper model, which is much less capable. So there's some technological tools that we need.

• 20:44Speaker 03

  I think our reward system for how we assign prizes and promotions and things will have to change. We used to place a lot of importance on being the first to solve an open problem. There was kind of a winner takes all mentality in mathematics, which was fine as long as it was so rare to solve problems that we didn't have these very frantic races where people are racing to be one hour ahead of their rivals to get their paper out. This is now happening now that these AI tools are so fast. So I think we need to change to more collaborative models and deprioritize this whole obsession of being first. So there's some social infrastructure that needs to change.

• 21:29Speaker 03

  I think there's some emotional tools that we need. You know, mathematicians, we haven't thought much about philosophical questions like what is the purpose of mathematics? What is a proof? What does it mean to understand something, to explain something? It just kind of happened, you know, so naturally when you work on a problem, you naturally become good at understanding it, you naturally become good at explaining it, at least to other mathematicians, maybe to the general public, we were still pretty bad at doing this. So we haven't really done much introspection in the past about sort of how should we sell what we do to other people? You know, should we go on more podcasts and talk about our profession? I hope we do. I mean, we're beginning to, it's not just me.

• 22:14Flora Lichtman?

  Yeah, and I, you know, I think going on a podcast, like, it might sound to some people as trite, but, I mean, I'm so biased, but I feel like there is value in being able to talk about your work so that other people can connect to it.

• 22:35Speaker 03

  And in an age where, thanks to technology, so many things can be automated, human-to-human interaction is maybe the one thing that AIs cannot take from us. And people like stories, you know. We've had some mathematicians who are very good at solving problems and not so great at public communication, because that wasn't our number one priority. So one thing I hope, a cultural shift in our profession, is that we actually show behind the curtain, you know, how we talk to each other, show our blackboards, some behind the scenes work, which we normally conceal, we don't think it's actually all that interesting maybe, or we worry that it will make us look not so smart or whatever. But if you are producing things that no human understands, what is the point?

• 23:18Flora Lichtman?

  Dr. Terrence Towe is a professor of mathematics at UCLA and author of the upcoming book, Six Math Essentials. Thank you so much for joining me today.

• 23:28Speaker 03

  Yeah, it's a pleasure.

• 23:38Flora Lichtman?

  Before we go, spooky season is here and Sci-Fry's nightmarishly nerdy Halloween costume contest is open for submissions. We are looking for your best science costume, and we're interpreting that very loosely. Last year, a delightfully accurate Tartar Grave is a finalist, as were the heavy metal rockers, a band that included Led Zeppelin, spelled L-E-A-D. So to enter, submit a photo of you in your science-inspired costume at sciencefriday.com/halloween. It's open now through November 1. We'll peer review them, and winners get swag and a shout-out on the show. Sciencefriday.com/halloween to enter. This episode was produced by Rasha Aridi, and thanks to Sebastian Hernandez for making this sound so great.

• 24:22Flora Lichtman?

  Until next time, I'm Flora Lichtman.