How to Think About AI Before It’s Too Late

Cory Doctorow has a refrain: “The most important thing about a gadget isn’t what it does; it’s who it does it for and what it does it to.” In this episode of “Galaxy Brain,” he sits down with Charlie Warzel to talk about the AI boom, making the case that the hype, vision, and dreams of endless growth are unsustainable. Doctorow expands on his viral “enshittification” thesis: a critique of AI based around power and whether we are using AI tools or being used by them. Get more from your favorite Atlantic voices when you subscribe. You’ll enjoy unlimited access to Pulitzer-winning journalism, from clear-eyed analysis and insight on breaking news to fascinating explorations of our world. Atlantic subscribers also get access to exclusive subscriber audio in Apple Podcasts. Subscribe today at TheAtlantic.com/Listener. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Tired of rage scrolling through AI slop and clickbait just to get a handle on today's news? Meet Inpress, the algorithm-free news app that makes it fun and rewarding to stay informed. Track your media literacy and rediscover reality. Download today at inpress.app. That's I, N as in news nerd, press.

• 0:19Speaker 01

  When was the last time you indulged your desires, felt true Range Rover refinement? The last time you felt total serenity, total confidence, no matter the terrain? Limitless, effortless, peerless? How far can Range Rover take you? Range Rover, designed for distinction.

• 0:39Speaker 02

  Bosses are infinitely horny for firing workers and replacing them with machines. And they have been since forever, right? That's the story of the industrial revolution. They're just, there's, there's something about this, and it's not just cutting costs. I think that if you're the boss, you are haunted by the knowledge that if you don't show up for work, the business just ticks over as per normal. But if all the workers don't show up, that's, you're out of business. And so maybe you tell yourself you're driving a car, but secretly you worry that you're in the backseat with a Fisher Price steering wheel. And one of the things about AI is it, is it dangles the possibility of wiring the toy steering wheel into the car's drivetrain.

• 1:22Charlie Warzel?

  I'm Charlie Warzel, and this is Galaxy Brain, a show where today we are going to examine the case against AI. Well, sort of. What follows isn't a case against the technology, machine learning, generative AI, coding agents. It is instead a case against this particular ideology behind AI, how it's built, how it's implemented. Cory Doctorow is a little bit of everything. He's a science fiction writer, he's a technologist himself, he's a prolific blogger and a journalist. He's also an activist who has been working with the Electronic Frontier Foundation on digital rights management, among other issues. And it's this combination of all of these jobs that make Doctorow this particularly shrewd critic of technology. He's somebody who has both the skill set to see how technologies are being implemented

• 2:07Charlie Warzel?

  in ways that work against human flourishing, while also being able to imagine all the ways that the future might be different. And most recently, Dr. O gained notoriety for coining the term "enshitification." This has become the shorthand for the way that companies and their platforms start out with this promise of empowerment, but then, once they've captured the market share, they begin to degrade their services and extract more from their users. Now, enshitification put a name to this pervasive feeling about technology, and especially the tools of the Web 2 era. But a lot of the dynamics of Dr. O's work are extremely relevant to the AI boom, which is dominated by many of the same personalities of the platform era. Dr. O has a new book. It's called "The Reverse Centaur's Guide to Life After AI," and it builds on a lot of

• 2:51Charlie Warzel?

  this past work. We'll get into what a reverse centaur is in just a moment, I promise, but it's the book's subtitle that got my attention: "How to Think About Artificial Intelligence Before It's Too Late." The AI conversation is, as we've explored before, intensely polarized, and you have some critics who just seem unable to engage with the usefulness of this technology. What makes Dr. O's perspective different is that his critique is not purely about the technology itself. It's about the power dynamics that surround it. Again and again in his book, he comes back to the same refrain, quote, "The most important thing about a gadget isn't what it does, it's who it does it for and what it does it to." Dr. O's animating questions needle at something bigger, this question that I think is shared

• 3:35Charlie Warzel?

  by so many right now. Why, in this era of intense technological progress, do so many things feel like they're getting worse and more exhausting? Who really benefits from these tools? Which groups of workers are actually using them and which groups are being used by them? Is there an AI bubble? What does it look like if it pops? What does meaningful resistance to AI look like in an era of growth at all costs? Corey Dr. O joins me now to talk about it all.

• 4:06Charlie Warzel?

  Corey, welcome to Galaxy Brain.

• 4:09Speaker 02

  Thank you, Charlie. It's a pleasure to be on.

• 4:11Charlie Warzel?

  So in the intro to your book, you're talking about different ways that people use artificial intelligence and you mention that you use it in certain instances. What is your relationship to artificial intelligence in terms of tools you use?

• 4:30Speaker 02

  The most common use I make of AI, and it's daily, and I just did it again, is I publish a blog, pluralistic.net. And because I am keenly aware of how the platforms trap us, I control that blog. I publish it on my own self-hosted WordPress site on a server on a shelf that I own. So my prompt for Ollama, one of the Ollama models that I have running on my laptop, is find typos.

• 4:58Speaker 02

  And that is remarkably good. So not only does it catch a bunch of stuff that my text editor or word processor would normally catch, things like you type a word that is a correct word, but not the word you meant. So you type shocker instead of shocked. Double punctuation, typing the word and twice in a row, kind of stuff we do all the time. But also, this is where it's like faintly magic, is the other day I referred to something in, as I was writing, and I pasted in a link to the Wikipedia article on it. And then I referred to it again later by a slightly different name. So it was something like saturation theory, and I called it the saturation hypothesis. And the chatbot said,

• 5:40Speaker 02

  "In the URL, it's called saturation theory. Did you mean saturation theory here?" Which was amazing.

• 5:47Charlie Warzel?

  Yeah, genuinely helpful.

• 5:49Speaker 02

  Yeah, super helpful. Look, you know, I don't know where your computer origin stories are, but my first word processor was a program I bought in a magazine that I typed into an Apple II+ My word processors have gotten new features since then on the reg. And an enormous number of them are things I have no use for at all. Many of them are things that I think other people use very badly, and some of them are things that I find Dispensable and AI is a plugin to my word processor as far as I'm concerned. And I have never used a plugin and said, right, well, that's it. Now we need to gather up all the writers and put them in a wood chipper. We should probably spend, I don't know, 1.4 trillion dollars on this. See how good we can get this word processor plugin to run.

• 6:31Charlie Warzel?

  Well, this brings me to your book. It brings me to the central operating principle of centaurs, reverse centaurs. I want to do some defining for people in the audience who aren't going to understand it. What is a centaur? Not the mythological animal, but the automation theory that you've based the book around. And, and what is a reverse centaur?

• 6:49Speaker 02

  So in automation theory, centaur is someone assisted by a machine. So, you know, you're wearing glasses, you're riding a bicycle, you're using a spell checker. You're playing a harmonica instead of making mouth noises. You're a centaur. And a reverse centaur is someone who is conscripted to help the machine. So there's a process a machine can almost do, but there's a part of it that it can't do. And someone has inveigled you to do that other part. And because the capital that went into that machine is depreciating, right? It's the machine only lasts so long. You want to sweat that asset as hard as you can. And because machines don't get tired and they typically work faster than we do. You, the reverse centaur, are the bottleneck in the machine.

• 7:31Speaker 02

  You are Lucy and Ethel trying to get chocolates off the assembly line and get them into the chocolate box. And the boss is running the assembly line at the maximum speed that the humans can conceivably do it at. And they are expecting you to work at two or slightly beyond the point at which you collapse from exhaustion. So someone who is a reverse centaur isn't just used by a machine. They are used up by the machine.

• 7:53Charlie Warzel?

  Let's think of that in the real world. Would you argue an Amazon delivery driver is the example right now of somebody trapped in, in reverse centarism? Or?

• 8:04Speaker 02

  Yeah.

• 8:05Charlie Warzel?

  What have you?

• 8:06Speaker 02

  Yeah, it's it's hard to overstate how much automation in that van is about seeing if you can get the person to work so hard that there is just no slack left for them. So we all know about the drivers not peeing, right? Or rather, peeing in bottles. That's not just because they have a hard schedule. The van is determining what route you're going to take and how long it's going to take. And then you have to make the prediction real irrespective of traffic conditions and so on. You are covered by many cameras and many sensors when you drive an Amazon van. And they are doing things like penalizing you for looking in the wrong direction. So you might be looking at something that you anticipate is a danger, but the van disagrees.

• 8:50Speaker 02

  And so they are dinging you on that. And then there are the more sort of mundane complaints, like the machine thinks that you can deliver these three parcels after parking your van because they're all in the same housing complex. And what the machine doesn't know is that one of those drop offs is a mile away. And so either you drive to the other half of the housing complex and get fined by the system for driving when you were supposed to be walking, or you walk or run to the other place and you get fined by the system for missing your your time quota on that delivery.

• 9:23Charlie Warzel?

  Well, and then there's this story that 404 Media reported recently. The one about the Rivian vans that Amazon uses that have been wired to turn off the air conditioner after between like 10 minutes or 30 seconds, depending if the driver is out of the seat. And so you have these drivers who are like trying to hack them in order to just have safe conditions when they're driving in the middle of the summer. And I think it speaks to this technology using us, right? Part of the thesis of this book is that these tools are created for the express purpose of creating these reverse centers, right? Something that none of us out here in the world want to be. So why are they engineered this way? Why are they engineered to make us reverse centers?

• 10:07Speaker 02

  You know, you and I and the people listening to this, we've been through a lot of tech bubbles lately. You know, whether that's like web three or crypto or what have you, we get bubble after bubble after bubble. And this bubble is a bubble like those, but it's much bigger. And one of the things this book wants to interrogate is, first of all, why do we have bubbles? And second of all, why is this bubble so much bigger? And it is so much bigger. I mean, when I wrote the book, it was a $700 billion capex expenditure worldwide. Now it's a $1.4 trillion capex worldwide and it's going up and up.

• 10:36Charlie Warzel?

  Yeah, and that 1.4 trillion is Goldman Sachs's prediction for spending on AI infrastructure in 2027.

• 10:43Speaker 02

  So this is a shocking amount of money. We've never set this much money on fire before for anything. It makes tulip bulbs look like rational conduct, you know, the South Seas look like an achievable objective, right? So where's this all coming from? Well, first of all, I think we have bubbles because firms that have saturated their market

• 11:02Speaker 02

  go very quickly from being treated by investors as growing firms to being mature firms. And when that happens, their shares are grossly overvalued because if a share in a company is a claim on its future earnings, if those future earnings are going to be larger than they are this year, then it makes sense that the share will be worth a lot because those earnings are going to grow and grow and grow. But if we know how much the company brings in, and if it brings in about the same every year, plus or minus a few points, then it has a much lower, what's called a price to earnings ratio, right? It trades at a lower multiple of its annual earnings. And from the perspective of management at the firm, this is bad. Because once you start being a mature company and your share price drops precipitously,

• 11:46Speaker 02

  a thing that just keeps happening whenever anything threatens these companies. The market's really telling us that they think these companies' growth is going to tap out soon. That's very bad. It's bad because, you know, if you're an executive who's been compensated in shares, well, that's your net worth, right? When the share price goes off the, falls off the cliff. But also, it's really bad because Shares in growth firms are very liquid. They're like money. And so, yeah, you can hire executives with shares. You can also buy companies with shares. Both of those are really good ways to grow. Once you've hit the limit on growth, once you are Google with a 90% search market share, and the market starts to treat you as mature again, then you have to use money to buy things that you used to use shares to buy.

• 12:28Speaker 02

  So you have to go like find a creditor or like a customer or an investor to grow by money. Whereas you can you can just grow by shares by deciding you're going to issue more stock. And so this is very dangerous. And so we see the string of bubbles. At first, the bubbles were quite grounded. You know, you had Google saying, oh, we're about to become Facebook because we have this thing called Google Plus. And the problem with that is that while we can all agree on how much Facebook is worth because it's there on their balance sheet, Facebook vociferously disagrees with Google that they're about to become Google Plus. And they make that point very forcefully in public, and it makes it hard to sell that narrative. So we switch to like imaginary things that we're going to conquer, Web 3, crypto, blah, blah, blah, right, metaverse.

• 13:11Speaker 02

  And then we get to this one, and this one is much bigger. And so why is this one so much bigger? Why is the market willing to spend so much more money? Well, some of it is, again, it's just material. This is like more real than crypto or metaverse or whatever.

• 13:26Charlie Warzel?

  Yeah.

• 13:27Speaker 02

  It's like, it's a super interesting computer science breakthrough. And not only that, because we have crypto, we have computer science breakthroughs all the time. Usually, when you find a way to get a computer to do something cool, if you do it harder, it doesn't get cooler, right? The returns to scale are pretty limited. And AI had these incredible returns to scale, which are now plateauing. But there was a period where we could just spend more and do the same thing and get more out the other end.

• 13:52Charlie Warzel?

  I think it's a really great point about growth companies that there is almost a danger when you're in that phase where, oh, no, we're going to grow very soon, right? Just you wait. That's a really good place for them to be. The period of actual growth, like we are booming right now, essentially can only mean to the market, soon we will stop. And that is that is the scary point. I take that totally. Tell me how you connect that to the idea of We are not only working on this computer science experiment, but we are working on it in such a way that it is going to turn us into tools of the

• 14:35Charlie Warzel?

  development. Why do these things push these companies to make the products that turn us into these reverse centers?

• 14:41Speaker 02

  So yeah, that's the ideology question, right? What is it that makes capital allocators so excited about AI versus all the other things they've been excited about? And I think a big part of this is that bosses are infinitely horny for firing workers and replacing them with machines. And they have been since forever, right? That's the story of the industrial revolution. They just, there's, there's something about this, and it's not just cutting costs. I think that if you're the boss, you are haunted by the knowledge that if you don't show up for work, the business just ticks over as per normal. But if all the workers don't show up, that's, you're out of business. And so maybe you tell yourself you're driving the car, but secretly you worry that you're in the backseat with a Fisher Price steering wheel.

• 15:26Speaker 02

  And one of the things about AI is it, is it dangles the possibility of wiring the toy steering wheel into the car's drivetrain. So the boss has an amazing, cool idea, and then that gets pooped out by the bot. And there are no ego destroying confrontations with people who know how to do things, who tell you that your idea is dumb. You know, when I was on the picket line with the screenwriters, because I live in Burbank, and I'm in an affiliated union, I'm in the Animators Guild Writers Unit. So when I was on the picket line with the writers near my house, one of them said to me, you know, you give notes to a writer's room the same way you prompt a bot, right? Like, give me ET, but it's about a dog with a love interest and a car chase. And the difference is that you say that to writers, and they're like, go make a spreadsheet.

• 16:10Speaker 02

  The grownups are making a movie. Whereas you say it to a bot, and it just gives you a script.

• 16:15Charlie Warzel?

  Well, it's also interesting, too, how

• 16:19Charlie Warzel?

  A lot of really, really rich influential people, even higher than the level of normal CEO, we're talking like in the billionaire class, have a lot of people, I've spent time around these people. A lot of people who say like, "Oh yeah, sir, that's an awesome idea. Yes, let's, let's draw that up. We'll get our people on it. We'll do that." Right? It's interesting that the chatbots also behave that same way, right? The chatbot is, "Sir, we'll get right on it. Absolutely."

• 16:44Speaker 02

  And if you are familiar with the psyche of bosses, you might say, "Oh yeah, offering them a way to like get stuff done without having to argue with workers is going to be very popular." But also, those upper-tier capital allocators, those billionaires and oligarchs, they themselves already live in a world that's pretty solipsistic. You know, if you become a billionaire, I think it's axiomatic that you have to hurt a lot of people. And to hurt a lot of people, you have to, in some sense, believe that what they feel isn't as real as what you feel. That they're not real people the way you're a real person. You know, Elon Musk calls everyone he disagrees with an NPC. I don't think it's a coincidence that these are the same people who got really excited about effective altruism,

• 17:27Speaker 02

  and particularly the strain of it that said, "You can hurt as many living people as you want, provided that you've spent a lot of time thinking about bringing a small amount of joy to the lives of 10,000, or rather, 10 to the 53 artificial people who will come into being in 10,000 years." Right? It's a very solipsistic way of viewing the world. You know, Mark Zuckerberg wants to build social media without people, right? Like in social media without socializing is like, you know, your friends are the reason you can't leave Facebook, because you love them, but they're a pain in the ass and you can't agree on when it's time to leave. And so that's great for me, but I've been trying to get your friends to see reason and understand that the best way to have a friendship is to maximize your engagement with a platform.

• 18:10Speaker 02

  And for some reason, they just keep having a friendship the way normal people do. So I think we're going to get rid of your friends and replace them with chatbots, right? You know, social media without socializing. It's very similar to the idea of a workplace without workers and a screenplay without screenwriters and a movie without actors.

• 18:29Charlie Warzel?

  But it can only go so far, it seems, right? One of the defenses right now of the idea, potentially, that these guys are going to fire everyone, right? That this tool is going to allow bosses to, you know, AI wash their companies. I think what we've seen already is the fact that it's difficult. You have companies who have laid people off, hiring people back in. There's problems. Do you think, it seems like you have a very low opinion of a lot of bosses in this sense, do you think they're that dumb that they're just going to continue to do this even in the face of if there aren't these same productivity gains?

• 19:11Speaker 02

  I mean, I think unless you spent 2008 in a cave, you have to at least entertain that possibility. Like, the analysts who are telling us that AI is whatever, a $17 trillion industry, are the ones who told us that 95% of us would be using the metaverse by 2025. There is a degree of credulity there, where if you think that you can find exit liquidity for an investment, it doesn't really matter whether the investment pays off, it just matters whether it fails after you've given the bag to someone else.

• 19:44Charlie Warzel?

  But first, a quick break.

• 19:54Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait and AI noise. Thanks, social media. That's where Inpress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with Inpress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.

• 20:27Speaker 01

  When was the last time you indulged your desires, felt true Range Rover refinement? When was the last time you drove in total serenity? The outside world reduced to a hush? The last time you were confident on uncertain terrain? The last journey that made you feel at home, conveniently connected at every step? In a Range Rover, pioneering innovation meets leading-edge luxury to make every drive an occasion. Limitless, effortless, pureless. How far will you let it take you? Range Rover, designed for distinction.

• 21:02Charlie Warzel?

  Understanding all of that, understanding that we are living in a bit of a Wild West here. We are also living in a time of a lot of people operating with impunity, sort of saying the quiet part out loud. I want to talk about the AI bubble. You predicate a lot of this on there being that AI is a bubble. I think that that case was the absolute majority case in the fall. And I think now we have some of these new coding agents. You see Anthropic turning what looks like it's going to be a pretty record profit here. Give me your case for the AI bubble as you see it here in the summer of 2026.

• 21:43Speaker 02

  Well, if these companies are going to be profitable, it will have to generate revenues in excess of its depreciating assets, including capital expenditure on new product development and in excess of its operating costs. It will have to make more money when it adds more customers. And when those customers use its product, it will have to increase its profit, not decrease its profit. So it will have to find a pricing and market strategy that has good unit economics and where depreciation and amortization are commensurate with the rate at which they are able to replenish their capital. And that's just not the case.

• 22:23Charlie Warzel?

  What do you say to people who are looking at the demand right now? It is a narrative that is coming from inside that industry. There is also like adoption that is real. How do you parse that? How do you see that playing into that?

• 22:38Speaker 02

  So giving away hundred dollar bills or selling hundred dollar bills at a dollar apiece will have a high demand. One of the things that we've seen with with market leaders in the last couple of months is that when they try to charge five dollars for those hundred dollar bills, people are like, oh, my God, we can't afford five dollars for these hundred dollar bills. So I'm not surprised to to hear that there is um demand for it. So the question isn't if you prompt an L L M often enough and harness enough compute, can you make software that is useful? That's very clearly true. And and I should say, the best programmers I know, many of them are love working with L L Ms. I don't question it for a minute. We do know that in this sort of reverse center or center configuration,

• 23:20Speaker 02

  that you hear lots of people who are in charge of how they write software who say, I write software the way I like, and I use these tools when it makes sense, and I am feeling really good about it. And then you hear from a lot of people who are like, they fired nine tenths of my colleagues, and my job is to mark the bots homework, and we're shipping the worst code I've ever seen. And, you know, God help you if you ever use one of our products. And again, this resolves that that conundrum, right? About how it is that two groups of people who are reliable narrators of their own experience can use the same tools and conclude completely opposite things about it. It's because the important thing about a technology is only secondarily what it does, and it's primarily who it does it for and who it does it to.

• 24:01Speaker 02

  And people who get to use the tools the way they want, they make better decisions than when you dictate to them how you do it. This is why Taylorism was a failure.

• 24:08Charlie Warzel?

  I want to get a little bit into like the harms that you show of of this type of dynamic, especially with this type of A I as it is foisted on people. You have a story about Air Canada call centers in there. But tell me what an accountability sink is and how something like Chat G P T might play into that.

• 24:26Speaker 02

  Yeah, this this comes from Dan Davies, who's a British writer, investment analyst, and and basically like a cybernetics and systems theory guy. And this term accountability sink is basically when you have a a system or a person or um something that you can blame for otherwise foreseeable mistakes.

• 24:47Speaker 02

  So that when people get hurt, it's not your fault. It's the accountability sink's fault. And an example of this is, as you say, Air Canada, I have a dog in this fight. Like all the best Americans, I am Canadian. Air Canada's customer service has always been pretty bad. So you could see why if the only reason to have customer service is to have someone explain over and over again that the company isn't going to solve your problem, why you wouldn't just ask an AI to do that. And that's what Air Canada did. So there was a man who was going to his grandmother's funeral and he got on the Air Canada customer service portal. And he said, how does a bereavement fare work?

• 25:28Speaker 02

  And the chatbot said, you buy a full fare ticket today, go to your grandmother's funeral, come back, get a copy of the death certificate, send it in, and we will refund you the difference, which he did. And when he did, the human being who was in charge of issuing refunds was like, I don't know what you're talking about. That's not how we do it. You had to do this before you flew. You don't get any money back. And he appealed it all the way through every level of customer service at Air Canada and eventually got a judgment against Air Canada for on the order of seven or $800 Canadian. So this was like a multi-month process. And you could see that from Air Canada's perspective, it doesn't matter if it's wrong.

• 26:09Speaker 02

  It just matters if it doesn't produce more of a loss than a gain. And there is a certain perverse incentive to give people bad advice that costs them money. If there's no way for them to get the money back. Air Canada is the nation's flagship carrier, the majority of routes. They have one or fewer competitors. You're going to keep using Air Canada anyway. So why would they, why would they care? You know, if you, if you remember on Saturday Night Live, Lily Tomlin used to this bit. She started on Laugh-In, Ernestine, the AT&T phone operator. She'd do these ads for AT&T and she would turn to the camera at the end of each one. And she would say, we don't care. We don't have to. We're the phone company.

• 26:51Speaker 02

  Right? So if you enjoy that kind of market power and you know that people have few, if any avenues of recourse, you can use AI and blame the AI for the mistake. When really this was the foreseeable and foreseen outcome of putting an AI in place of the human being who is already not there to do a good job.

• 27:14Charlie Warzel?

  Tell me about automation blindness. How humans become this dull node in the loop reviewing AI output. It's sort of similar to, it's a little like the corollary to, you know, chatbot takes over the call center.

• 27:27Speaker 02

  Yeah, well, there's a funny thing that happens at TSA checkpoints. TSA people train all day, every day to find water bottles. And they are the world's water bottle finding as people like that we have ever created. They are the Louis Pasteur of finding water bottles. Because they get more practice than anyone else. But the number of people who deliberately bring a gun or a bomb onto an airplane. It is indistinguishable from zero. Now, obviously, at the very furthest margin, there are people who do it, which is why we have airport security. But there is every reason to think that without

• 28:07Speaker 02

  a regular supply of people walking through deliberately carrying guns and bombs through that have been disguised. That airport security people just cannot remain vigilant for it because they detrain to find one thing when they are training to find the other one, which is water bottles, which is the thing that we all bring through.

• 28:24Charlie Warzel?

  And so you're suggesting that in this paradigm where all these jobs are, if they're not being fully automated away, if you have a human in the loop. You're suggesting that this type of automation blindness is going to increase.

• 28:38Speaker 02

  Yeah, especially when you think about this in combination with this reverse center paradigm of work speed up, of sweating the asset. So if your job is to have a new AI judgment or result put in front of you at the maximum speed that you could theoretically evaluate it. And if in the majority of times, it's good enough. So your job, the majority of times is to click OK. And you have to do that relentlessly at this incredible pace. It's going to require some very like atypical kind of neurological makeup to spot that. I won't say there's no one ever born who couldn't do that job. But most of us are not going to be that person.

• 29:22Speaker 02

  And there aren't enough of those people to be the human in the loop. So let me give you an example of how you might configure automation in two different contexts. So I have an extremely treatable form of cancer. So I spent a lot of time following the news about cancer treatment and AI. And it looks like AI can sometimes spot solid mass tumors that humans miss on X-rays. And so if there was a sales call between an AI salesperson and the hospital administrator at the Kaiser Hospital where I get my care, and they were saying, right, here's the pitch. Right now, you spend a million dollars a year on radiologists. They're each assessing 100 X-rays a day.

• 30:07Speaker 02

  I want you to give me half a million dollars a year for my robo-radiologist, over and above the salary of those million dollars that you're spending now. And now your radiologists are going to do fewer X-rays because one or two times a day, the AI is going to say, have another look at that one. And they're going to stop and they're going to look at it again. No risk of automation blindness, right? That is an arrangement as someone with cancer that I would be very happy to see. That's not the pitch because there aren't enough hospitals that want to spend an extra half million dollars a year over and above their payroll. The pitch goes like this, fire nine tenths of your radiologists. Put the remaining 10% in charge of rubber stamping the output from the X-ray, but they're supposed to look at it closely enough to actually make an assessment about it.

• 30:51Speaker 02

  But you're going to work them as fast as you possibly can to sweat this asset as much as you can, realize as much of a savings as you can, fire as many workers as you can. In that circumstance, when the AI is usually right about the chest X-ray, but sometimes wrong, and the radiologist is just clicking, okay, okay, okay, okay, okay, five times a minute, that radiologist is going to kill people and we're not going to blame the AI. We're going to blame them.

• 31:17Charlie Warzel?

  And that's the accountability sink.

• 31:19Speaker 02

  That is the accountability sink.

• 31:21Charlie Warzel?

  So it seems to me, what I appreciate about your work is that it tries to consider these tools in the context of the types of people who are in control of deploying them, control of purchasing them, control of all that, right? And you write that the workers, the non-oligarchs, bosses of the world, need to puncture the AI bubble as soon as possible, right? Before it gets even further out of control. Wouldn't that be quite bad for it to pop? Now, I know that's a naive sounding question, but just as part of it, like, wouldn't it be really bad?

• 31:55Speaker 02

  Yeah, it would be really bad for a $1.4 trillion bubble to pop. It would be even worse for a $2 trillion bubble to pop. And so, you know, I, I, I do believe that history is ours to make, that, that the future has not been written. But I, when I say that, I don't mean that there's a way that AI becomes profitable. I mean that there's a way that we deal with the fact that it is a bubble and what we do when the bubble pops. If, if we know that it's coming, if we understand who created it, if we regulate in response to it, you know, one of the reasons we have the string of bubbles we've had in this century is because the repeal of Glass Steagall. Glass Steagall was a law put in place after the, the great crash and the

• 32:39Speaker 02

  great depression. And it was, it was intended and, and worked very well to structurally separate investment banks from, from commercial banks, from consumer banks. And we got rid of that law. Right? And, you know, this is not all that different from other things that we deregulated in that era. You know, we, we, we stopped enforcing antitrust law around the same time. And then we got a lot of monopolies. Those two facts are actually really closely related. And, you know, if I, if I get grumpy or cranky, it's, it's really because it sometimes feels like we have so little object permanence that we couldn't win a game of peek-a-boo. And we just forget that in living memory, we had

• 33:22Speaker 02

  a thing we did that stopped a bad outcome. And then we stopped doing that thing. And then the bad outcome happened. And we're like, this is a mystery. What an unfortunate situation for us. But, you know, as Margaret Thatcher told us, there is no alternative. So I guess we just have to live with it. And, and I sometimes feel like, you know, the people who advocated for the removal of structural separation in finance and the end of antitrust law, those are the same people who are inflating this bubble and who are also saying, how dare you blame me for all the bad outcomes of all the other bubbles and all the economic problems we have with monopolies. So. You know, we can choose how we react to this bubble. When we had a bubble in the 30s that destroyed society,

• 34:05Speaker 02

  we responded by changing the structure of markets to dampen the likelihood of bubbles. And we installed policing mechanisms that looked for exceptions that escaped from the market structuring efforts and that either addressed them legislatively or through enforcement. These are not the lost arts of a fallen civilization. This is not building a pyramid without power tools. It's just prudent regulation. We could choose that when this bubble bursts. Or we could choose to do austerity again. And when we do austerity, we drive people into the arms of fascists. And so we could have a collapse followed by an authoritarian surge, or we could have a collapse followed by a prudent regulation and a reckoning with the mistakes that we made

• 34:50Speaker 02

  that brought us to this juncture. And maybe with the people who claim that these mistakes wouldn't happen.

• 35:06Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait and AI noise. Thanks, social media. That's where Inpress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with Inpress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.

• 35:39Speaker 01

  When was the last time you indulged your desires? Felt true Range Rover refinement? When was the last time you drove in total serenity? The outside world reduced to a hush? The last time you were confident on uncertain terrain? The last journey that made you feel at home, conveniently connected at every step? In a Range Rover, pioneering innovation meets leading-edge luxury to make every drive and occasion limitless, effortless, pureless. How far will you let it take you? Range Rover, designed for distinction.

• 36:13Charlie Warzel?

  What does opposition look like then? There's clearly a moment to seize on something. If you are an activist or someone who doesn't want this or a politician, what is your version of real opposition to this in the moment? This type of artificial intelligence and these companies, what does it look like?

• 36:31Speaker 02

  I think that a lot of the times when people criticize AI, they stipulate to a bunch of its claims without interrogating them. So when we say this AI data center is going to be polluting, it's going to be noisy, it was undertaken under undemocratic conditions.

• 36:49Speaker 02

  We have better uses for that land. It's going to be hard on our water supply and our grid. We should also add, and by the way, this company might be bankrupt before they turn it on, and not stipulate to the idea that this thing that can't go on forever will never stop.

• 37:05Speaker 02

  I think when we criticize AI more broadly, we should remember that as offensive as it is for the AI companies to say to creative workers, our goal is to make you bankrupt and our tactic for doing that is training our models on your work, that we shouldn't stipulate to the idea that the work is as good as ours, right? That they've done anything good. And we should remind investors who have seen these splashy demos, because I think that's what AI art and AI text gen and AI video and so on. We should remind them that the workers that this stands to displace

• 37:50Speaker 02

  have a wage bill globally that sums up to less than the kombucha expenses for one training run of Mid Journey. And that even if they fire every illustrator in the world and replace them with slop, that it won't turn a buck, right? That this is a money-losing proposition. That if you stop selling $100 bills for a buck a piece, it's always going to be cheaper to hire illustrators. Illustrators are among the most immiserated, low-wage workers in the creative industries.

• 38:19Charlie Warzel?

  Do you see? A possible future in which it does just keep going like this, like the irrational nature of this in the, you know, in the way that we have closed the Strait of Hormuz for a very long time. And a lot of people are like, you're going to feel the pain very soon and the market will soon reflect all of, you know, this shortage. And then Donald Trump posts something on Truth Social and then the market goes, yeah, you know, we'll believe that. And like, you know, every Friday we'll do, we'll just do this dance that the bubbles, so to speak, and the, the cultural part of the bubble too, right? The political, the cultural, the economic part of the bubble, that we've become so used to that, that we

• 39:01Charlie Warzel?

  just do the irrational, this is just how it's going to be. We're going to operate in this irrational dystopia of yes, the bottom is going to fall out. No, it's not.

• 39:14Speaker 02

  So the question is, can this thing that can't go on forever never stop? And, and the answer lies in material things. So to keep the AI bubble going, you need to keep sending money to guys in bunny suits in Taiwan who sit in clean rooms for eight hour shifts while like, you know, tin is evacuated into, into, or vaporized into an evacuated chamber and hit with a laser and then hit with a different laser to create a vapor that creates the visible wavelength of light that etches a four nanometer scale feature on a chip. So someone needs to send real things to Taiwan so that those people can eat. They won't keep doing it if they can't eat. Right? So they need, so, so there needs to be real

• 39:56Speaker 02

  things, not just things on balance sheets, but actual things. There needs to be energy, right? That's a, like it's a non-negotiable. And there's only so much mismanagement that you can do and prop up through money creation and stimulus without actually doing industrial policy that produces real things people need. Uh, you know, healthcare and so on that you can do before things that it starts to that it starts to tell. Right. You start to see a failure. So this is not investment advice. I'm not short or long on AI. I'm, you know, putting all my investment in laser tag because I think we've got lots of laser tag arenas coming online soon.

• 40:41Speaker 02

  But, but I don't, I don't believe in, I, I don't know when the AI bubble is going to pop. But the other thing that I goes around my head all day long is Stein's law, which is that anything that can't go on forever eventually stops. And there are hard material limits on the ability to cook the planet and devote an ever larger share of our national project, our global project to building GPUs and putting them online. And those hard material limits are not remediable through belief. They require a material response.

• 41:24Charlie Warzel?

  A lot of this book and a lot of your work sense is obviously in conversation with your last book. The topic of incentivization, she has taken off in, in ways that are kind of unbelievable to see.

• 41:41Charlie Warzel?

  The idea that these platforms and companies are slowly getting worse, exerting more power over people, getting locked into these ecosystems. I don't know if we're in fully in the incentivized era of AI yet, but you can see the contours of that with token use and things like that starting to go. But incentivization has also become, as you know so well, the shorthand for a general feeling of anger and frustration around the degradation of just services and experience of everyday life. And recently you wrote this, this article that cited the national customer rage survey, which has been surveying a panel of a thousand representative consumers every three years for a decade. It's been going on since the seventies. And essentially part of the takeaway here is that people are furious now, like historically pissed off

• 42:26Charlie Warzel?

  in this customer rage sense. AI is a big part of this, and I think a lot of people in so many ways are feeling helpless.

• 42:39Charlie Warzel?

  Does this cycle break? Because it just feels like it's everywhere. Like, we can talk about it in the sense of AI, but you can also feel it at the grocery store, at the wherever, you know, like, and I, I think it has this real curdling effect, especially when you're having so much trouble and you see the Elon Musk trillionaire news and there's people all over the place be like, congratulate this innovator on having, you know, more money than anyone could ever have imagined. How does this cycle break?

• 43:14Speaker 02

  Well, about 15 years ago, I guess, there was a book by Thomas Piketty, the French economist, called Capital in the 21st Century. And Piketty did this monumental research project, going through every ledger of capital flows going back 300 years to trace capital flows and capital concentration over centuries. And their conclusion was that, all of the things being equal, capital tends to accumulate, that if you have money, you tend to make money. They summarize this as R is greater than G. The rate of return of on capital is always greater than the rate of growth. And so Piketty's point was, if you don't do something about that, rich people get richer and richer and richer and richer and richer and richer. Doesn't matter how big the pie is getting,

• 43:58Speaker 02

  the share of the pie owned by the rich gets bigger, faster than the pie gets bigger, always, as an iron law. And that eventually, this becomes untenable. Eventually, the rich, because they are unchecked, there is no democratic accountability, they are subject to folly as the rest of us, they look to one another with great jealousy and use us as pawns in bids to take one another's fortunes. Once our fortunes have been stripped bare, that this creates massive instability and that those instabilities tip. And he sees these points over and over again, where the share of income in the top decile,

• 44:41Speaker 02

  the top 10% reaches, or wealth, rather, not income, reaches a certain point. And then you get the French Revolution and then World War I and World War II, and so on and so on. And

• 44:55Speaker 02

  His message was that unless we do something about oligarchy, the end state of oligarchy is not stability. The end state of oligarchy is chaos. It's collapse. And that is my worry. And you mentioned before, you know, that there are these billionaires who love the fantasy of AI and love the fantasy of a world without people and that they live in a world without people now, where everyone around them is a non-player character who just glazes them with AI-like nonsense. But there's another group of people who love AI, which is politicians, who want to exert their will without having to go through a permanent civil service. Just like there's the dream of a workplace without workers. There's a dream of a government without bureaucrats. Right?

• 45:40Speaker 02

  This is what Trump wants to do with Doge. This was the whole nature of the Doge project, was you take away the deep state, which is to say everyone who knows how the government works, and you replace them with chatbots. And then Trump has an idea and it happens. You don't have these collisions with people like that's not, just not how the government works. We can't do it that way. There are people who won't be able to fill in that form you want them to fill in. If you take away this support, this other thing will collapse. It doesn't, doesn't work that way. You just tell the chatbot to do it and the chatbot just does it. You can see how this would be really unstable. I mean, we're living through it. We're living through it with just screw flies. No amount of belief is going to help you when beef is $30 a pound. Like, that's politically destabilizing.

• 46:22Speaker 02

  And so I think that unless we can confront oligarchy by democratic means, that we will find ourselves in the midst of a circumstance where it is being confronted by non-democratic means, whether or not that's what we want or what we aim for. And I really worry about it. I think oligarchy, it is the force that stops us from addressing the polycrisis of genocide and climate degradation and rising authoritarianism. And until we meet that oligarchy challenge, I think we are going to struggle to do more than fight a holding action on everything else. And when we get rid of the oligarchs, it's still not going to be easy.

• 47:04Charlie Warzel?

  I think that's a tough place to leave it, but that, that chaos, I think, is very much where, where we are today. Corey Doctorow, thank you so much for coming on Galaxy Brain for talking about all this.

• 47:14Speaker 02

  Thank you, Charlie. It's been a real pleasure.

• 47:20Charlie Warzel?

  That's it for us here. Thank you again to my guest, Cory Doctorow. This episode of Galaxy Brain was produced by Renee Klar and engineered by Miguel Carrascal. Our theme is by Rob Smersiak. Claudine Abaid is the executive producer of Atlantic Audio, and Andrea Valdez is our managing editor. If you liked what you saw here, new episodes of Galaxy Brain drop every Friday. You can subscribe on YouTube, or Spotify, or wherever you get your podcasts. And if you want to support this work, and the work of my fellow colleagues, you can subscribe to the publication at theatlantic.com/listener. That's theatlantic.com/listener. Thanks so much, and I'll see you on the internet.

• 48:14Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait, and AI noise. Thanks, social media. That's where Inpress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with Inpress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.