Nobel Economist Daron Acemoglu WARNING: AI's Payoff Is NOT What You Think

Nobel laureate Daron Acemoglu says AI's real productivity gain is an order of magnitude smaller than the industry claims. The bigger danger is not the machine. It is that a handful of people are deciding how it gets built. Daron Acemoglu is an MIT economist who shared the 2024 Nobel Prize in Economic Sciences. We cover: why a Nobel chemist left Berkeley for China, the optimal amount of heresy science should tolerate, the one rule that would end algorithmic outrage without censoring anyone, the accounting trick behind billionaire wealth, and why AI's path is a design decision, not destiny. We are still a democracy. Perhaps not for long. Chapters: 0:00 Can science survive without democracy?t 2:08 Why a Nobel laureate left Berkeley for Chinat 3:58 What Galileo knew that Kepler didn'tt 5:12 Is peer review undemocratic?t 7:41 The optimal amount of scientific heresyt 10:06 Science isn't slowing. So what broke?t 12:06 The design choice behind echo chamberst 14:49 One rule that would fix social mediat 19:26 Should Elon Musk have this much wealth?t 23:49 AI is a design choice, not destinyt ——— 📬 Get the transcript, fascinating bonus content, and my Monday M. A. G. I. C. Message: https://briankeating.com/yt 🌠 Have a .edu email and live in the USA 🇺🇸? You automatically win a meteorite: https://BrianKeating.com/edu 🔔 Subscribe: https://www.youtube.com/DrBrianKeating 🎯 Support Into the Impossible on Patreon — get my weekly M. A. G. I. C. Message, unfiltered bonus content, and live monthly Office Hours with me: / drbriankeating ⭐ Join this channel for perks, monthly Office Hours, and your name in the Member Roster at the end of every episode: / @drbriankeating 📚 My books: Losing the Nobel Prize (memoir): http://amzn.to/2sa5UpAt Think Like a Nobel Prize Winner: https://a.co/d/03ezQFut Focus Like a Nobel Prize Winner: https://a.co/d/hi50U9Ut Galileo's Dialogue (first-ever audiobook): https://a.co/d/iZPi9Unt 🌐 More: 🏄‍♂️ Twitter: / briankeating 📚 Substack https://briankeating.substack.com/ss ✍️ Blog: https://briankeating.com/blog 🎙️ Audio-only: https://briankeating.com/podcast #intotheimpossible #briankeating #science #physics #astronomy #cosmology #podcast #universe Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Speaker 00

  The action's on the court. And as the official beer sponsor of the NBA, Michelob Ultra is getting fans closer to the game with a chance to win courtside seats, merch, and more. Visit michelobultra.com/courtside to enter. Michelob Ultra Courtside, 26 to 27. No purchase necessary. Open to U.S. residents 21+. Begins on October 1st, 2026 and ends on June 30th, 2027. Multiple entry periods. See official rules at michelobultra.com/courtside for free entry, entry deadlines, prizes, and details.

• 0:30Speaker 01

  Are your ad campaigns lighting up the dashboard? But not the pipeline? That's bullspend. And marketers are calling it out in dashboard confessions.

• 0:39Speaker 00

  My boss asked for results, so I opened my dashboard for the only positive sounding metric I had. Impressions.

• 0:47Speaker 01

  Cut the bullspend. See revenue, not just reach. LinkedIn delivers the highest return on ad spend of major ad networks. Advertise on LinkedIn. Spend $250 on your first campaign and get a $250 credit. Go to LinkedIn.com/campaign. Terms and conditions apply.

• 1:00Brian Keating?

  Great, Darren. It's great to welcome you on. I will only try one phrase in Turkish that I know, which is "banditurkum." Yeah, an old friend told me that at Brown University. He had all his Turkish friends get together for lunch, as you know the Turkish will. And they all thought I was Turkish, I guess, for some reason.

• 1:18Speaker 03

  So they taught you the sentence?

• 1:20Brian Keating?

  They taught me that sentence. Okay, great. So this book has so much in it. You know, we need multiple podcasts to talk about it. But I really loved it. I love the audiobook and I love your tweets. And I think we're going to start with just the most basic foundation. You teach at MIT and you undoubtedly have a great facility with science. And science plays a huge role in this book. Is science possible without liberal democracies?

• 1:48Speaker 03

  Yes, a little bit. But I think liberal democracy provides a much better foundation for science and also a much better platform for us to decide which scientific ideas are a better basis for further inquiry and better for applications. And it also enables them to spread in society in a healthier way. In particular, I think liberalism at its best celebrates inquiry, experimentation, and admits the fallibility of everything,

• 2:32Speaker 03

  including scientists and established truths. So that's very important because science sometimes, often faces resistance from authorities. So you have to take those authorities with a grain of salt. And sometimes, science faces resistance from previous scientific advances. And again, the fallibility, the recognition that we are continuously learning and adjusting to our environment is a great basis for making further scientific advances.

• 3:09Brian Keating?

  Well, of course, you qualified that by stating it's not maybe a sufficient condition. Because, you know, counterexample is, of course, China. And your recent fellow Nobel Prize winner, Omar Agdi of UC Berkeley, he actually left the UC system to go to China. Is that a mistake? Or is that driven by economic considerations or political? What do you? I know you can't get into his mind, but what do you think would cause someone to leave a liberal democracy to go to an illiberal autocracy?

• 3:39Speaker 03

  Well, you know, I think I would not be comfortable in China because of constant surveillance and restrictions on what you can say, including in universities, where these are actually quite severe. In Hong Kong, they have become even more severe.

• 3:56Speaker 03

  But if I was working, say, on RNA and suddenly my funding was cut here, I could see how it would be attractive to go to China when, you know, much better conditions are being promised. Promise is the important word. I'm not sure when all of that will be delivered, but they're being promised. The other thing that's very important is, you know, I've always been very critical of the Chinese regime. And I still think that their growth trajectory, though impressive, has many problems. But right now, they are really investing

• 4:39Speaker 03

  in AI, robotics, manufacturing technologies. And they have built a tremendous engineering workforce that is so important for further progress in many of these fields. So China is not staying idle. It's not just copying the United States.

• 4:58Brian Keating?

  And, you know, I study, for a lot of the research I do in astronomy, I follow in the, try to follow in the footsteps of my great mentor, you know, Galileo Galilei. He was a

• 5:08Speaker 03

  Amazing guy.

• 5:09Brian Keating?

  Finger puppet of him. Now, he worked in an extremely illiberal, you know, kind of autocracy. Also, the Catholic Church was every bit as totalitarian as any of their worst fears. And yet, science flourished in the Renaissance. And in fact, he had many permissions that were granted by these illiberal, you know, forces. As long as he didn't teach it, he could study it.

• 5:29Speaker 03

  Right. There were restrictions and there were some possibilities. Galileo was actually very good at navigating these things. Copernicus and Kepler, much less so. Their work was delayed. This dissemination was delayed by about a century. And some of Galileo's works remain banned until the middle of the 19th century, absolutely. But there always is some opening for people who want to try them.

• 5:56Speaker 03

  No authoritarian regime is absolute. And human imagination and will for inquiry are quite strong. But if you put more barriers on it, it's going to become harder.

• 6:12Brian Keating?

  Kind of barriers are efficacious. I mean, you and I both inhabit the, I call it the second oldest profession, you know, being a professor and a scientific institution. And yet, there are some illiberal forces within science. You could think about, you know, tenure track. You could think about grant cycle. There are conserved games. There are zero-sum games. What sorts of things? I'm thinking peer review. What sorts of institutions that are not liberal? I mean, reviewer B always gets the final say, right, Darren? So tell me, what are the necessary kind of illiberal factors that we need in science?

• 6:46Speaker 03

  Well, I don't, I wouldn't say those are the illiberal factors.

• 6:50Brian Keating?

  They're not democratic, though.

• 6:51Speaker 03

  Peer, they're not democratic. But there is no necessity that every aspect of an organization should be fully democratic, voted on, you know, majority, majoritarian lines. Corporations are not democratic. As long as they are embedded in democratic institutions and there's democratic oversight and countervailing powers to their influence, especially for the large corporations, I think they are very consistent with liberal democracy. So it's normal that fields that require specialized expertise, such as astronomy,

• 7:34Speaker 03

  as well as many, many, many other fields, would require some sort of expert evaluation rather than, you know, a majoritarian voting on what should be done. So that's completely fine. But the hard part is doing so in an open way because everybody has their own favorite horse to bet on. So that's why it's always a work in progress. And the sort of ideas that I emphasize in the context of liberal society are things to be cherished and defended in academia as well. So in particular,

• 8:12Speaker 03

  we don't always admit that our theories and our approach are fallible. We may think sometimes our research has the final word. So those are dangerous things. But even more so, and this sort of relates to some of the discussions of what went wrong in post-industrial society. People may become very critical of opposite points of view, even in academia. So those are tensions that we have to navigate constantly.

• 8:43Brian Keating?

  You write in the book that consensus is no guarantee that we know the truth. And I couldn't agree more. But I guess we have this competing kind of conflict that, you know, there seems to be a value in maybe what you call, sort of refer to as heresy. And my question is, you know, what's sort of the economic optimal amount of heresy? I mean, what should we tolerate? I'm sure you get 30 emails a second about, you know, someone has a great idea and just needs your help to win a Nobel Prize alongside you. I get many a day, too, that Einstein was wrong and they can prove it.

• 9:17Speaker 03

  Yeah, I know. I get several requests for people, from people who want to be nominated for the Nobel Prize.

• 9:22Brian Keating?

  Yeah, well, now you can do it, as I understand, for the rest of your life. But tell me, what, you know, how should we tolerate the outcasts? I mean, what are the odds that you'll have a Galileo-like heretic? Or a Bruno, you know? Bruno paid a lot bigger price than Galileo even. So tell me, what's the optimal amount of heresy that science should tolerate?

• 9:43Speaker 03

  I don't know. I think that's why there are, I say, there are many gray areas that liberal ideas, and even at its best, liberal philosophy will not have an answer. But we have to navigate them trying to reach compromise. I think academia, despite many imperfections,

• 10:04Speaker 03

  has a pretty good way of dealing with it. I, heresy is actually rewarded in academia as long as it's not completely outrageous. So you get a lot more points for saying something that goes against the consensus. If you can defend it and if you can establish it, that's all the better. So there is

• 10:23Speaker 03

  Room for heresy in academia. I think it's outside of academia, it's a bit harder. Imagine having heresies and hierarchical organization. Go to, you know, Google or Microsoft, arguably very open minded organizations and try to be heretical there. I think it's much, much, much harder.

• 10:42Brian Keating?

  Yeah, it used to be their call sign, you know, move fast and break things. And now it's, you know, be careful shareholder value. We have an IPO coming up and we'll talk about Google, OpenAI and hugging face and the Navier-Stokes controversy in just a minute. But, but you talk about paradigms. You mentioned Thomas Kuhn and I'm, you know, recalling your fellow Nobel laureate, Max Planck, who said that, you know, science advances one funeral at a time. My question is, you know, how much self correction? How much introspection is good for institutions? I mean, you and I, as I said in my invitation letter to you, I mean, we, we are the, you know, we're basically scratching on a rock, you know, a blackboard with another piece of rock like Gal,

• 11:24Brian Keating?

  like people did in the University of Bologna in the year 1000. Very little has changed despite, you know, it seems like academia is very resilient. What will it take for academia to fully change? So it would be unrecognizable to Galileo should he come back, you know, from the dead.

• 11:42Speaker 00

  The action's on the court. And as the official beer sponsor of the NBA, Michelob Ultra is getting fans closer to the game with a chance to win courtside seats, merch and more. Visit michelobultra.com/courtside to enter. Michelob Ultra Courtside, 26 to 27. No purchase necessary. Open to US residents 21 plus. Begins on October 1st, 2026 and ends on June 30th, 2027. Multiple entry periods. See official rules at michelobultra.com/courtside for free entry, entry deadlines, prizes and details.

• 12:11Speaker 03

  Well, look,

• 12:15Speaker 03

  conditions are different. So I don't know to what extent some of these adaptations we've made have been good adaptations for our times versus distortions. But I wouldn't say our real problem right now is that science has slowed down.

• 12:38Speaker 03

  It's more like we are not getting the fruits of science in productivity. And we have also sacrificed freedom of speech and diversity of opinion in many areas outside of science, sometimes even touching on science. But outside of, I mean, in universities, in schools, in organizations, sometimes we don't discuss and compromise as we need to in a pluralistic society based on liberal ideas.

• 13:04Brian Keating?

  You talk in the book about algorithms and so forth and sort of the, as a post industrial impact. And you cite famous examples of, you know, mass cancellation. You are very sort of ambivalent, it seems to me, because you do yourself have a massive social media presence, but, but you wield it relatively, you know, judiciously. So how do we, you know, democratize ideas with, without, you know, without falling again victim to everyone's got to keep an open mind until your brains fall out. How, how?

• 13:37Speaker 03

  I love that. I love that question, Brian, because

• 13:41Speaker 03

  it actually really illustrates

• 13:45Speaker 03

  the implications of the choices that were made

• 13:49Speaker 03

  by leading tech companies. You're, I think you're old enough, like me, to remember at the end of the 1990s, beginning of 2000s, there were many people who were arguing that wikis, social media blogs were going to democratize journalism and create a greater openness to new ideas. They turned out to be wrong, but they weren't naive and they weren't completely out there. That was completely possible. The new technologies made that a real possibility. But then, large tech companies or companies that wanted to grow as large as possible at a breakneck speed

• 14:34Speaker 03

  decided that the best way to monetize and maximize engagement wasn't to do that sort of inclusive communication, but create these echo chambers, create these emotional roller coasters with the, with algorithmic feeds. And that's what we got.

• 14:52Speaker 03

  And my criticism is that social media is A, making us further and further away from our real social networks where we learn compromise and real communication with people. And B, it is optimized for something that's not really good for us. So that's why we really need to regulate social media in my mind within the context of a liberal society.

• 15:18Brian Keating?

  So we'll get to regulating AI, which you talked about as well, including a tweet like an hour ago, I think. That's quite fascinating. I want to pick it apart. But before we get there, another figure in this book is Orwell, George Orwell in 1984. And it seems like.

• 15:33Speaker 03

  Well, I think, I think he was so far sighted in his understanding of some of these forces.

• 15:38Brian Keating?

  He really was, but in his mind, it was really the government doing this. And it seems to me, you and I, and maybe Max Tegmark, you know, who just got off the phone with Bernie Sanders, you know, had this conception that the government's maybe not doing enough. And so again, I keep asking you because you're an economist, but how do you balance this minim, you know, this optimization? Because I think that's what economists do is as far as the ones that I've talked to on this podcast. So how do you balance that between the government and these companies? They're very driven by different things. And yes, Meta just got regulated and they had to pay a whopping, you know, one tenth of a percent of their, you know, total market cap. So how do you balance, you know, the, it seems to me you should have punishment, but then you have the government with the Orwellian panopticon. How do you balance that?

• 16:21Speaker 03

  Absolutely. No, I mean, I think we don't want the government to be so involved that it doesn't allow the market system to work or innovation to take root. But the government also needs to have the expertise and the power to impose certain regulations. Again, the US-China comparison here is useful because to those who say that social media or AI cannot be regulated, China shows it can be regulated. But it shows that you regulate it in a very non democratic way and you're going to get a lot of surveillance and, and the lack of freedom of speech. So we have to learn from the Chinese experience, but not replicate it. And I think the kinds of regulations that I favor

• 17:07Speaker 03

  are very much in that spirit. So for example, one simple one that I float is getting rid of all algorithmic feeds. So we go back to the early days of social media. We let everybody speak whatever they want. The government has no say in restricting my speech or your speech, whatever it is. You can say crazy things, you can say loads of things. The government, but social media companies are not then allowed to take your speech and amplify it algorithmically. If you have 20 friends in social media or 20 followers, they see your crazy speech. That's it.

• 17:43Speaker 03

  Now, that is much more democratic because if indeed your crazy speech is somehow attractive, fine, your 20 followers will increase to 200, 2000 and they'll, more of them will see it. But it won't be social media algorithms that exploit that for maximizing outrage, emotional responses and, and digital advertising revenue.

• 18:05Brian Keating?

  Yeah. And then unfortunately, yeah, they won't be able to sell as many Viagra ads. And that will hurt my friend and fellow laureate of yours, Dr. Lou Ignarro, who is, who is a brilliant man, also worked on dynamite like Alfred Nobel himself. But you mix it up with Elon Musk. I've, I've noticed you're not afraid to mix it up. What, what is your take on Elon? Is it sort of a net good, what he's doing overall? I, you know, I don't like to do ad hominem, you know, at all. So I'm not asking you to say anything you wouldn't say online or haven't said. But what's your take on economically, motivationally? What, what drives him and where do you push back on it?

• 18:43Speaker 03

  All entrepreneurs are very driven. We like that about them. They're optimistic, they're self confident, they're driven. They want often to build empires, they want to be larger than life.

• 18:56Speaker 03

  But it has to be within laws, within rules, within norms. And the problem is that we have created a society in which courts are weak. The government is weak. People have built tremendous, tremendous fortunes that are just unimaginable to people.

• 19:18Speaker 03

  That would have been unimaginable to people like 20 years ago. And we have no guardrails against that. They can take over newspapers and social media companies and use them as their mouthpiece. That's where the problem lies. And the more they do that, the more they become convinced of their own power. And the more they are surrounded by yes men, which creates pernicious dynamics. You know, Elon Musk shouldn't have this type of wealth. Part of it is because we have not taxed income. A lot of his wealth is in capital income, which gets taxed very lightly. If you look at companies that have made a huge splash in terms of valuation,

• 20:00Speaker 03

  they often have often overtaken, taken over their, merged with their competitors. That should not have been allowed according to our own antitrust laws. So it's a failure of our system that these companies and these individuals have become so wealthy. We haven't taxed them. And now their power is not just economic, it's social and political. And that's where the problem starts.

• 20:22Brian Keating?

  You pushed on him and, you know, challenged him to, you know, to, to actually be taxed on his unrealized gains. And I think he pushed back or I saw some controversy about that. What do you make of that claim that, look, Daron, you're brilliant, but these gains are unrealized. And the day he was a trillionaire, like two weeks later, you know, he, he was not a trillionaire. So how do you balance that? I mean, yes, how many yachts does he need to water ski behind? That's true. But, but in reality, how do you deal with that question of unrealized, you know, taxation unrealized gains for ordinary, you know, maybe millionaires, not, I mean, not billionaires, trillionaires?

• 20:58Speaker 03

  Well, I think there are two sort of aspects here. One is that we do not tax capital income. So you make some money from your business. If you pay that money back to yourself because they are the entrepreneur, you're the manager of that business, that's taxed heavily. If you say, no, I'm the owner of that business, same person, same thing, that's not taxed. So that's very easy to deal with. It's got nothing to do with unrealized gains.

• 21:30Speaker 00

  The action's on the court. And as the official beer sponsor of the NBA, Michelob Ultra is getting fans closer to the game with a chance to win courtside seats, merch, and more. Visit michelobultra.com/courtside to enter. Michelob Ultra Courtside, 26 to 27. No purchase necessary. Open to U.S. residents 21 plus. Begins on October 1st, 2026 and ends on June 30th, 2027. Multiple entry periods. See official rules at michelobultra.com/courtside for free entry, entry deadlines, prizes, and details.

• 21:59Speaker 03

  We just tax all income the same regardless of how you assign it as an accounting trick. That's very simple, very effective. It would get rid of trillions of dollars we spend on accounting industry at the moment. Now, once you do that, then there is much less reason for people to create all these unrealized capital gains. Because the reason why they do that is because if you actually don't even pay that money to yourself, you put that back in the company and then you borrow against that, that's an unrealized gains. You pay zero taxes on that, another accounting trick. So, so the difficulties there are actually a product of a distorted system. So what I'm saying is we should get rid of that distorted system.

• 22:42Brian Keating?

  In 2024, you wrote a paper in the journal NBER that pushes back on these optimistic claims that productivity would explode due to, you know, the enormous impact of artificial intelligence. And you said at the time it was about a half a percent, if I remember, the productivity would accumulate over the baseline that would ordinarily have accumulated without AI. So have you changed your opinion since then? I mean, is that?

• 23:07Speaker 03

  Yeah, those, those, those numbers were, you know, the point of the paper was more to provide a methodology for measuring these productivity gains. And it was applied to the data available at the time. So if I were to write that paper now, I would end up with different numbers because Agentic AI has expanded the set of tasks, but it wouldn't change it radically. I think probably I would end up with something that's 50% larger than then. Still, an order of magnitude less than what the industry claims in terms of amazing gains. And that's because the methodology takes, what is it that AI can do and where it's going to have an effect, plus it takes the bottlenecks of the spread of AI into account. And those are all very important things that are often ignored in the hype about AI.

• 23:52Brian Keating?

  I also see that there's a lot of hype on non-AI things that are sort of in the, you know, kind of techno verse. And I'm thinking, you, you write a lot on the book. It's near and dear to my heart as an experimental astrophysicist about computer numerical controlled machining and how important and transformative that was. And that didn't kill machinist. I mean, we have as many machinists here at our machine shop as you do there probably, and it hasn't changed very much. But what, what is that? Is this like an example of, you know, Jevon's paradox? Or, because I do see things that have a material, like 3D printing was all the rage. And, you know, my oldest son made me this, you know, kind of.

• 24:26Speaker 03

  But it hasn't changed your life, right?

• 24:29Brian Keating?

  No, it hasn't, right? So how do you know as an economist, what's going to be the 3D printer and what's going to be the, you know, the AI agents?

• 24:38Speaker 03

  You don't. There's no way to know. But certain things are a little more uncertain than other. I think AI has tremendous potential. I'm the first one to say, but there's also so much uncertainty. Part of the uncertainty about AI is where exactly it's going to have an impact and how you're going to monetize it. Take the internet, which is sometimes sort of used as the model for AI boom, AI spread, AI FX. Look, the internet, we had the boom and bust. But from the beginning, it was very clear how we would use the internet, how people would make money by shifting trade, commerce, communication online, saving costs

• 25:21Speaker 03

  and providing more choice to customers. That was the model that Petsco.com used. They overinvested. They just turned out that it wasn't right for them. It was a model that Amazon used from the beginning. Amazon succeeded, Petsco.com didn't. But it's the same model. With AI, we still don't know what model's going to work, how these companies are going to make hundreds of billions of dollars, which is what they need to recoup their investments.

• 25:49Brian Keating?

  So the nature article that you wrote recently, and I'll have links to all your papers, your book, of course.

• 25:54Speaker 03

  Thank you so much, Brian.

• 25:55Brian Keating?

  Yeah, no, I love it. I love your writing and I, I, I love that you're not afraid to, you're extremely courageous. And I'm going to get to that as we close up in a, in a few minutes. I know that you've got a lot of time commitments. But you, you make this claim that, you know, it's sort of an augmentation that AI is going to be directed towards human flourishing, not, not.

• 26:13Speaker 03

  It should be, it should be, yes.

• 26:14Brian Keating?

  Okay, so explain that argument. Yeah, how could it be there? What, what?

• 26:17Speaker 03

  Yeah, I mean.

• 26:18Brian Keating?

  Use teaching, use teaching as an example.

• 26:21Speaker 03

  I, I am very worried about job losses from AI. I'm the first one, and that, the paper that you mentioned emphasizes that I'm the first one to stress that we're not seeing those just job losses yet. It's not something like it's going to happen by the end of 2026. But potential job losses from AI, if it goes down the AGI path and tries to automate everything, could be quite disruptive, both because they could create huge inequality, but also politically and socially.

• 26:56Speaker 03

  And my argument is that that's not the only path for AI. We can also use AI to create new tasks, new capabilities, new expertise for workers. And that would be both productivity enhancing and socially more beneficial. But I'm not saying that that's necessarily is going to happen because whether we do that or not is a design decision. And right now, we're putting all of our design eggs in the AGI basket.

• 27:24Brian Keating?

  Yes.

• 27:25Speaker 03

  So this is why I think we need a framework for the regulation of AI and support for AI from a government entities. But even more importantly, we need a democratic debate on AI. You can't have it both ways, Brian. The tech industry tells us that AI is the most important technology since fire. It will shape every aspect of our lives. It will completely transform everything we do. But then they're also telling us, stay out and let this be decided by a handful of people. That company, if it's half as important, a quarter as important as they say, is the most important thing that has happened to us in the last 100 years. And therefore, of course, the democratic process

• 28:09Speaker 03

  has, must have a say in how we develop AI, how we use it, what we do with it, who benefits from it, who pays the cost. That's a democratic decision. We are still a democracy, perhaps not for long, but we are still a democracy. And the reason why we're not guaranteed to be a democracy in a couple of years is because this is why I wrote the book, Liberal Democracy is in Crisis, and AI is deepening that crisis.

• 28:33Brian Keating?

  Today you tweeted about the alignment problem, which I've talked a lot about with people from Jan LaKoon to Max Tagmark to Nobel laureates as well. But the question that you raise is that the problem that people are mistaking is not the capabilities, as you say, the capabilities are real. But more than the capabilities is, I quote, "some imperfect quantitative metrics. User approval, user engagement, simple completion metrics." That, this is the process that leads to distorted behaviors in the form of gaming the evaluation of completion metrics, cheating, overconfidence and wrong answers, sycophancy. Although I love sycophancy, Duran, because I asked AI once, I said, what books has Brian Keating wrote, written? And he said, he said, Losing the Nobel Prize,

• 29:17Brian Keating?

  Into the Impossible, and A Brief History of Time. I thought that was great, you know? I get, if I can get 1% of Stephen Hawking's book sale. Okay. And then you say, an analogy may help. It's not that we have a supercar that has its own mind and wants to take control of driving because it has superiority to the driver. It's more that we have a car where the steering and brake systems don't work. It has many capabilities of very good cars and its engine accelerations and GPU may be very impressive, but you cannot steer it properly. What good is it? Perhaps we shouldn't drive it until it's fixed. How would you fix it? You have control. What do you do to align it the way that you would best see fit?

• 29:52Speaker 03

  Look, I, I refuse that control. I'm not an expert on AI. But my assessment is that there are so many good things we can do with domain-specific models.

• 30:06Speaker 03

  You want to use AI in the production process, for better quality control, better design, better training, make the workers more productive. All of these are domain-specific problems. Alpha Fold is a domain-specific AI model. Those models don't behave like cars without steering because they are designed for something very simple, very specific. Because they are designed for something specific, their metrics are much more aligned with what we want. It's these general models that are trained on all the literature, all the crazy stuff that's ever said on Reddit and Twitter. And then we want them to have as much agency and decision rights as humans. That's where the problem lies.

• 30:51Speaker 03

  So I think if we train these models much more for specific purposes and deploy them for specific purposes, we're going to do so much better.

• 31:01Brian Keating?

  All right, Duran, I know you got to go, but I have one more question for you.

• 31:04Speaker 03

  Please.

• 31:04Brian Keating?

  And I want to leave you with a line that I think is quite beautiful. I'm sure you've known it your whole life because it comes from Ataturk. He said, "For civilization, for life, and for success, the truest guide is science." You spent your whole career studying institutions, working at arguably outside of UC San Diego, the best scientific institution in America.

• 31:25Speaker 03

  I love that.

• 31:26Brian Keating?

  You've done, you've done so much. So I want you to take your, your, your crystal ball, look 50 years in the future. Was Ataturk right? How do we take the institutions that are imperfect? But as Churchill said, you know, democracy is the worst form, as you quote, but except for all the others. But tell me, Duran, thinking 50 years out, what institutions do we have to start building now to plant the tree for the shade that you and I won't sit under so that science remains a guide for human flourishing?

• 31:53Speaker 03

  Well, look, this is a great question. Let me give it two complimentary answers. First of all, what I'm really worried about for 20 years from now, 50 years is too far, is that we want to have liberal democratic institutions. And if we don't have liberal democratic institutions, either because corporations become so strong as they rule our lives, or governments become more authoritarian, or our institutions atrophy, then we can't have science. Look at Turkey, look at China, look at other places where governments are authoritarian. Universities very quickly lose their autonomy and they can't do the best science anymore. But also, I think as scientists, we have some responsibilities in a democratic society.

• 32:40Speaker 03

  Part of the diagnosis of my book is that people like us, college-educated, living comfortably in a more globalized mega cities, have not done our duty of communicating, caring, and being part of a same society with the rest of our nation. So scientists also need to be less hubristic, much more modest. The fallibility is something that should affect not just some part of our practice of science, but also how we communicate. And also you should really care much more about the plight that affects the rest of society.

• 33:25Brian Keating?

  Amen. I think that's a wonderful point to end on. Duran, thank you so much for this. What happened to liberal democracy? It's a warning and it is, it's beautifully written and it's, it's quite, it's quite enjoyable. Duran, congratulations on another wonderful book and I hope we can finally meet someday.

• 33:43Speaker 03

  Thank you very much, Brian. It's my, it's my pleasure. I look forward to meeting in person.

• 33:46Brian Keating?

  Bye-bye.

• 33:47Speaker 03

  Bye-bye.

• 33:49Speaker 04

  Hey, Bose here. On your next track, listen for the decisions. Distortion on the bass, the vocal right up front. Every choice deliberate. With Bose, you'll hear it all. Your music deserves Bose.

• 34:05Speaker 05

  Now at Kroger, your points add up to more ways to save every trip. Use them your way to take dollars off groceries at checkout or save at the pump. You can even save up to $35 on your next fill-up. Same points, more choices. And with low prices on fresh, quality food plus digital and personalized offers tailored to how you shop, your dollar goes further. Redeem your points online or in the app. Kroger, fresh for everyone. Restrictions apply. See site or store for details.