AI’s Drawbacks: Environmental Damage, Bad Benchmarks, Outsourcing Thinking — With Emily M. Bender and Alex Hanna

Emily  Bender is a computational linguistics professor at the University of Washington. Alex Hanna is the Director of Research at the Distributed AI Research Institute. Bender and Hanna join Big Technology to discuss what their new book, “The AI‑Con," which they describe as the layered ways today’s language‑model boom obscures environmental costs, labor harms, and shaky science. Tune in to hear a lively back‑and‑forth on whether chatbots are useful tools or polished parlor tricks. We also cover benchmark gaming, data‑center water use, doomerism, and more. Hit play for a candid debate that will leave you smarter about where generative AI really stands — and what comes next. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. Want a discount for Big Technology on Substack? Here’s 25% off for the first year, which includes membership to our subscriber Discord: https://www.bigtechnology.com/subscribe?coupon=0843016b Questions? Feedback? Write to: [email protected] Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay?

• 0:43Alex Kantrowitz?

  Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 1:03Alex Kantrowitz?

  Two of AI's most vociferous critics join us for a discussion of the technology's weaknesses and liabilities, and a debate on the finer points of their arguments. We'll talk about it all after this. Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. We're joined today by the authors of The AI Con. Professor Emily M. Bender is here. She's a professor of linguistics at the University of Washington. Emily, welcome.

• 1:30Speaker 01

  Glad to be here. Thank you for having us on your show.

• 1:33Alex Kantrowitz?

  My pleasure. And we're also joined by Alex Hanna, the director of research at the Distributed AI Research Institute. Alex, welcome. Thanks for having us, Alex. Always good to have another Alex on the show. So we try to get the full story on AI here. And so today, we're going to bring in, I think, two of the most vocal critics on the technology. They're going to state their case, and you at home can decide whether you agree or not. But it's great to have you both here. So let's start with the premise of the book, What is the AI Con? Emily, do you want to begin?

• 2:03Speaker 01

  Sure. So the AI con is actually a nesting doll situation of cons. Right down at the bottom, you've got the fact that especially large language models are a technology that is, it's a parlor trick. It plays on our ability to make sense of language and makes it very easy to believe there's a thinking entity inside of there. This parlor trick is enhanced by various UI decisions. There's absolutely no reason that a chatbot should be using I, me pronouns because there's no I inside of it, but they're set up to do that. So you've got that sort of base level con. But then on top of that, you've got lots of people selling technology built on chatbots to, you know, be a legal assistant, to be a diagnostic system in a medical situation,

• 2:46Speaker 01

  to be a personalized tutor and to displace workers, but also put a Band-Aid over large holes in our social safety net and social services. So it's, it's cons from the bottom to the top.

• 2:59Alex Kantrowitz?

  Okay. I definitely have things that I disagree with you in places on, and we will definitely get into that in the second half, especially about the usefulness of these bots and whether they should be using I or me pronouns and the whole consciousness debate. We're going to get into that. I don't think, I don't think any of us think we're, think that these things are conscious. I just think we have a disagreement on how much the industry has played that up. But let's start with what we agree on. And I think that from the very beginning, Emily, you were the lead author on this very famous paper about calling the large language models stochastic parrots. And at the very beginning of that paper, there is concern about the environmental

• 3:44Alex Kantrowitz?

  safety and the environmental issues that large language models might bring about. So on this show, we talk all the time about the size of the data centers, size of the models, and of course, there is an associated energy cost that must be paid to use these things. And so I'm curious if you, Emily, or you, Alex, Alex, you worked at Google, right? So you probably have a good sense of this. Can you both share, like quantify, how much energy is being used to run these models?

• 4:15Speaker 02

  So part of the problem is that even, you know, even if you're working at Google, you've you are directly working on this. They're not very public estimates of how much cost there is. I mean, the costs vary quite widely. And the only cost that I think that we know was an estimate being made by folks at Hugging Face. Um, that worked on the Blooms model because they were able to actually have some kind of insight into the energy consumption of these models. So part of the problem is the transparency of companies on this. You know, as a response at Google, after after the stochastic parents paper was published,

• 4:55Speaker 02

  one of the complaints from people like Jeff Dean, the SVP of Research at Google, and David Patterson, who's the lead author of Google's kind of rebuttal to that, was that, you know, well, you didn't factor in X, Y, Z, and factor in renewables that only we talk about at this one data center in Iowa. We didn't, you didn't factor into off-peak training. And so it's part of the problem. I mean, we could try to put numbers on it, but there's so much guardedness about what's actually happening here. We can't quantify it. We don't know. When it comes to model training, I mean, we might have something like we know the number of parameters that are in a new model or in an open weights model like Llama.

• 5:41Speaker 02

  But um, we don't know how many kind of fits the stars they were with stopping training and restarting or experimenting. So, you know, we could speculate, but we know it's a lot because there are real effects in the world right now. What are those effects?

• 5:57Speaker 01

  What are those effects? Uh, so um, you see communities losing access to water sources. You see communities, you see electrical grids becoming less stable. Um, and this is starting to be, I think, very well documented. There's a lot of journalists who are on the beat, doing a lot of good work. And I also want to shout out the work of Dr. Sasha Lucioni, who's been looking at this from an academic perspective. And one of the points that she brings in is that it's not just the training of the models, but of course also the use. And especially if you're looking at the use of chatbots in search, instead of getting back a set of links which may well have been cached, if you're getting back an AI overview, which happens non-consensually when you try Google searches these days, right? Um, each of those tokens has to be calculated individually.

• 6:41Speaker 01

  And so it's coming out one word at a time, and that is far more expensive. I think her number is somewhere between thirty and sixty times more expensive, just in terms of the compute, which then scales up for electricity, carbon, and water than an old fashioned search.

• 6:57Speaker 02

  I would also say that speaking about existing effects, there's also a lot of reporting coming out of Memphis right now, especially around the methane generators. That XAI has been using to power a particular supercomputer there called Colossus there, specifically around emissions there. Affecting Southwest Memphis, traditionally a black and impoverished community. There's also reporting on, well, actually in research from UC Irvine, in which looking at backup generators and

• 7:38Speaker 02

  emissions from diesel that are support that are connected to the grid, but just because the SLAs on data centers are, you know, incredibly high. You effectively need some kind of a backup to kick in at some time, and that's going to contribute to air pollution.

• 7:56Alex Kantrowitz?

  And which communities have been affected by the loss of water due to AI?

• 8:02Speaker 02

  So in, I think the best reported one is the Dulles in Oregon. That means I think that's the one that is the best known. That is kind of pre-AI, in which we're focusing on the development of Google's hyperscaling. And it wasn't until the Oregonians sued the city that we knew that half of the water consumption in the city was going to Google's data center.

• 8:25Alex Kantrowitz?

  That was before generative AI.

• 8:27Speaker 02

  That was before generative AI. I mean, we have to imagine the problem is probably exacerbated right now.

• 8:33Alex Kantrowitz?

  But do we know that? I mean, you both wrote the book on this.

• 8:38Speaker 01

  So we have, we certainly point to environmental impacts as a really important factor. It is not the main focus of the book. I would refer people to reporting of people like Paris Marks over at Tech Won't Save Us, did a wonderful series called Data Vampires, looking at, I think there was stories in Spain and in Chile. And yeah, so this is, you know, we are looking at the overall con and the environmental impacts come in because it is something we should always be thinking about and also because it is very hidden, right? When you access these technologies, you're probably sitting, you know, looking at them through your mobile device or through your computer and the compute and its environmental footprint and the noise and everything else is

• 9:23Speaker 01

  hidden from you in the immateriality of the cloud.

• 9:26Speaker 02

  I would also say that, I mean, the reporting on Memphis, I want to give a shout out to the reporting in Prism by Ray Levy Yuyeda, I don't know if I'm pronouncing their surname correctly, but they have an extensive amount about the kind of water consumption of this, saying that this would take about, I think, a million gallons. I'm checking it, but I'm looking, I'm looking at the, the, the reporting on it. I think, I'm seeing the exact number on this. I'm going to look at it. Yeah. So they're focusing, yeah, a million gallons of water a day to cool computers. They don't,

• 10:08Speaker 02

  they're saying that they need to build a gray water facility to do it. I mean, this is not anything that any, that these facilities don't exist yet, so they'd have to be built. But I mean, this thing is already being constructed and is, is, is, is using water. So I mean, I don't think it's a far cry to say that this is happening in an era that was in the hyperscaling era, in pre the generative AI era. I mean, it's The unfortunate fact about it is that a lot of these community groups are fighting this on a very local level and a lot of these things are getting under-reported on just because but from what we know from the fights in the Dulles and London County and

• 10:50Speaker 02

  and parts of rural Texas, I mean, we'd be surprised if this similar kinds of battles weren't being fought.

• 10:58Alex Kantrowitz?

  Right. I agree with the under-reporting and that's why we're leading with it here and we're going to go through a list of some of the things that might be wrong with generative AI. I think it is an issue. I think Emily, you basically hit on it, right? Where you're producing all these tokens when you're going to generate an AI overview that I checked and it is, you cannot opt out of it. You're correct.

• 11:17Speaker 01

  Well, you can if you add minus AI to the query.

• 11:20Alex Kantrowitz?

  Okay.

• 11:21Speaker 01

  But you have to do that each time. You can't like put a setting somewhere.

• 11:24Alex Kantrowitz?

  That's interesting. I didn't know about that. Okay, so you can opt out minus AI. But these things do take more computing than traditional Google search. I guess the argument from these companies would be that they're just going to make their models more efficient. I mean, we see the increasing amounts of efficiency over time and, you know, there might be a big upfront energy cost to train, but inference might end up being not that energy intensive. What would you say to that?

• 11:52Speaker 01

  I would say that we've got Brad Smith at Microsoft giving up on the plans to become a net zero carbon since the beginning of Microsoft and he said this ridiculous thing about we had a moonshot to get there and turns out with generative AI, the moon is five times further away. Which is just an absurd abuse of that metaphor. But yeah, and you see just, you know, Google similarly also backing off of their environment goals. And so if there really was all these efficiencies to be had, I think they wouldn't be doing that backing off.

• 12:20Speaker 02

  And I want to also add, I mean, I think this argument about the large amount of training and carbon use on the front end and then it tapering off with the inference. I mean, this is an argument that straight came from Google. This is, this was again in the same paper by Deepid Patterson. I think the title of the paper, I'm not going to get it exactly right, was, you know, the cost of, the cost of training or the cost of generative AI will, probably not generative AI, I think it was the cost of language models will plateau and then decrease or the training costs. And effectively, the argument being that you have this large investment that we can offset with renewables and then it's going to decrease.

• 13:04Speaker 02

  But you have to also consider that. Given that the economics surrounding it, it's not one company training these, right? I mean, if it's multiple different companies training these and then multiple different companies providing inference. And so as long as there there's some kind of incentive to keep on putting this in products, then they're going to proliferate. So if it was just Google, sure, maybe there might be a case in which there was some kind of planning and there was some kind of way to measure and focus on that and then it actually tapering down. But you have Google, Anthropic, XAI, of course, OpenAI, Microsoft, Amazon, everyone trying to get a piece, doing both

• 13:47Speaker 02

  training and doing inference. So I think that's again, you know, like, it's hard to put numbers on it. But what we see in this is just a massive investment in this. And that gives a good signal to say that the carbon costs have to be incredibly high.

• 14:03Alex Kantrowitz?

  Right. Look, I I think it's it's important for us to again to lead here. It's clear that there's some real environmental impacts. And I mean, we have Jensen Huang, the CEO of Nvidia, saying inference is going to take 100 times more compute than than traditional L M inference. And every C every well, every top executive from these firms that I've asked, well, is inference going to take more compute? It's not exactly as much as Jensen is saying, but there is a spectrum. So these things are going to be more energy intensive. If everybody listening out there, I do think, you know, this is important context to take in that when we talk about AI,

• 14:44Alex Kantrowitz?

  there's an environmental cost out there. It's not fully clear what that is, although there is one. And I agree with the authors here that more transparency makes a lot of sense. Now, let's talk about another issue that you bring up in the book, which is benchmark gaming. It's been a hot topic in our. Big technology discord over the past couple of weeks that we see these research labs keep telling us that they have reached a new benchmark or beat a certain level on a new test. And we're all trying to figure out what that means because it does seem like a lot of them are training to the test. And you have some pointed criticism in the book about the gaming of benchmarks and what that's meant to tell us. So just lay it out for

• 15:29Alex Kantrowitz?

  us. What's going on with benchmarks and tell us about the gaming, Emily?

• 15:33Speaker 01

  So yeah, so when you say the gaming of benchmarks, that makes it sound like the benchmarks are reasonable and they're being misused. But I think actually, most of the benchmarks that are out there are not reasonable. They lack what's called construct validity. And construct validity is this two-part test of the thing that we are trying to measure is a real thing, and this measurement correlates with it interestingly. But nobody actually establishes what these things are meant to measure as a real thing, let alone that second part. And so they are useful sales figures, right, to say, hey, we now have state of the art soda on whatever. But that is not interestingly related to what it's named as measuring, let alone what the systems are actually meant to be for.

• 16:16Speaker 02

  Yeah, and I would just add that, I mean, there's a lot of work. And I mean, we, prior to the book, Emily and I, I've spent a lot of time writing on benchmark data sets. And so this has been, you know, like, I'm personally obsessed with the ImageNet dataset. I'm thinking of another book on the IMSD dataset, just from what entails. But I mean, you know, the benchmarks, what they purport, there's a, there's a lot of different problems in the benchmarks, right? So the construct validity is probably first and foremost. And when we get something where you have something like MedPalm two or MedPalm one and two being measured on the US medical licensing exam, that's not really a test that determines whether

• 16:58Speaker 02

  one is sufficient, you know, is prepared to be a medical practitioner. There's so much more involved with being a medical practitioner, and above and beyond taking the US medical licensing exam. You can't take the bar and say you're ready to be a lawyer, right? I mean, there's so much more that has to do with. With relationships and training and other types of professionalization. There's huge literature in sociology and sociology of occupations on what professionalization looks like and what it entails and what kind of social skills involved and what that means and how to be adept at being in the discipline. But then the

• 17:42Speaker 02

  the kind of different benchmarks are, there's so many different problems just in terms of the way that companies are doing science themselves, they're releasing these benchmarks and often these are benchmarks that they themselves have created and released. So it may be the fact that they are quote unquote, teaching to the exam, but they're also, they have no kind of external validity in terms of what they're trying to do. So OpenAI is saying, we had a model that did so well, we had to create a new benchmark for it. Well, who's validating that, right? I mean, even the old benchmarking culture, you had external benchmarks and multiple people would be going to it and saying, oh, we've done better in this benchmark.

• 18:27Speaker 02

  Now OpenAI is saying, we have our own benchmarks because we did it so well. Not like the old system was any better, but this new system is that where's the independent validation of this that says it can do this thing that is purported to say?

• 18:40Alex Kantrowitz?

  What do you think about the Arc AGI test?

• 18:43Speaker 02

  Yeah, well, I mean, we spent some time focusing on the Arc AGI test, right? The Arc AGI test.

• 18:50Alex Kantrowitz?

  It is independent, at least that.

• 18:52Speaker 02

  It is, it is ostensibly independent. I mean, is this, is that the the French That's for show legs, yeah.

• 19:00Alex Kantrowitz?

  And by the way, for everybody who's listening, it basically asks, let me see if I get this right, it asks the models to be able to generalize its ability to understand patterns and putting shapes together.

• 19:13Alex Kantrowitz?

  I think that's the best way to explain it.

• 19:14Speaker 01

  Yeah, so it's a it's a it's a bunch of visual puzzles where it's I think they're all in 2D grids. And in order to make this something that a large language model can handle, those 2D colorful things are turned into just sequences of letters. And the idea is that you have, I think it's sort of a few-shot learning setup where you have a few exemplars and then an input and the thing is can you find an output like that? And it is when we want to talk about how the names of the benchmarks are already misleading, the fact that that's called ARC AGI, right? That suggests that it's testing for AGI. It's not. It's one specific thing. And I think Shulay's point is that this is something that is a very different kind

• 19:57Speaker 01

  of task than what people are usually using language models for. And so the sort of gesture is towards generalization, that that if you can do this, even though you weren't trained for it, then that's evidence of something. But if you look at the OpenAI paper-shaped object about this, they used a bunch of them as training data in order to like tune the system to be able to do the thing. So, all right, fine, supervised machine learning kind of works.

• 20:24Alex Kantrowitz?

  Right. And the next the next test, there was ARC AGI 2 that came out with a whole bunch of new problems and instantly all the models started doing poorly on those. So let me let me just ask this. Is there a measure that would allow the two of you to assess whether these AI models are useful? Or have you just written off their ability to be useful completely?

• 20:46Speaker 01

  So useful for what?

• 20:49Alex Kantrowitz?

  I mean, you you tell me.

• 20:50Speaker 01

  Well, that's that's sort of my point is that I think it's perfectly fine to use machine learning to do specific tasks and then you set up a measurement that has to do with the task in context. I'm a computational linguist, so things like automatic transcription are very much in my area. If I were going to evaluate an automatic transcription system, I would say, OK, who am I using it for? What kind of speech varieties? I'm going to collect some data, people speaking, have someone transcribe it for me, a person, and then evaluate how well the various models work on doing that transcription. And if they work well enough and it is to within the tolerances of the use case for me, then great, that's good.

• 21:26Alex Kantrowitz?

  Do you believe in the ability to be general?

• 21:30Speaker 01

  So the ability to be general, and here I'm thinking of the work of Dr. Tanit Gebrew, is not an engineering practice. That's an unscoped system. So what Dr. Gebrew says is, the first step in engineering is your specifications. What is it that you're building? If what you're building is general, you're off on the wrong path. That's not something that you can test for, and it is not well scoped technology.

• 21:53Speaker 02

  Yeah, I mean, this notion of generality has always had some specificity in AI, too. I mean, we mentioned in the book this idea, this, uh, this is a word I struggle with, and I have, I've taken so many time, but I'm just gonna say fruit flies, right? So, right, Josephila, the, the kind of fruit fly model of genomics, this idea that you have some kind of sequencing that's very common to this one very specific species, right? And there is, in the past, what that's become in AI is the game of chess. It's been game playing, right? I mean, these are very specific tasks, and those aren't, those don't

• 22:33Speaker 02

  generalize to something called general intelligence, as if something like that actually exists. I mean, one of the problems in AI research is that the notion of intelligence is very, very poorly defined, and the notion of generality is very poorly defined or is scoped to what the actual benchmark or what the task is that it is being attempted to achieve. So, I mean, that's, I mean, so this notion of generality is very poorly understood, and it is deployed in a way that is, that makes sense, sound like there is a notion of kind

• 23:14Speaker 02

  of general intelligence. And it seems to be the fact, I mean, and there's, you know, one of the One of the, um, a great paper that we, we, we point to in the footnotes of the book is this paper by Nathan Ensmerger, um, which, um, Ensmerger, um, that is talking about how chess became the, the dress field of the, um, the, uh, the AI research age and the prior AI hype cycle in the sixties and seventies. And it just happened to be, you had a lot of guys that liked chess and they wanted to compete with the Soviets who had chess dominance, right? And so those tasks become kind of these tasks about like, well, these are the

• 23:57Speaker 02

  things we kind of like. And, and we're actually seeing some of that. Again, it's like, well, we, these are tasks that we think are suitable. These are tasks that are scoped in a way that think we think are the most worthwhile problems, but they're not tasked to think about, well, what exists in the world that is going to be helpful and useful in scope to specific execution, right? This, this notion of an everything system is, is wildly unscoped.

• 24:23Alex Kantrowitz?

  But, okay, so it is unscoped, but I think everybody listening or watching right now would probably say, well, just my basic use of chat GPT, it can tell me about history, it can write a poem, it can create a game. I, okay, I see Emily reacting already. Uh, it can search the web and give me plans, can do all these different things in these different disciplines. So there is a, a, I think for people listening, there would be a sense that there is a, an ability to go into various different disciplines and perform. And whether you say it's a magic trick or not, it's, it's clear that it can. And so what I'm, I guess I'm trying to get at is, do you, I mean, is there a way to

• 25:05Alex Kantrowitz?

  measure that? Or do you think that that is in itself a wrong assertion?

• 25:11Speaker 01

  So yes, I think it's a wrong assertion. What ChatGPT can do is it can mimic human language use across many different domains. And so it can produce the form of a poem. It can produce the form of a travel itinerary. It can produce the form of a Wikipedia page on the history of some event. It is an extremely bad idea to use it if you actually have an information need, setting aside the environmental impacts of using ChatGPT and setting aside the terrible labor practices behind it and the awful exploitation of data workers who have to look at the terrible outputs so that the consumer sees fewer of them. And by terrible

• 25:54Speaker 01

  outputs, I mean violence and racism and all kinds of sort of psychologically harmful stuff.

• 26:00Alex Kantrowitz?

  Yes.

• 26:01Speaker 01

  What's that?

• 26:01Alex Kantrowitz?

  No, we've had one of the people who've been rating this content on the show. Folks who are interested, I'll link it in the show notes. Richard Matanga was here to talk about what that experience was like. Sorry, go ahead.

• 26:14Speaker 01

  So setting aside all of that, if you have an information need, so something you genuinely don't know, then taking the output of the synthetic text extruding machine doesn't set you up to actually learn more on a few levels, right? If you don't already know, you can't necessarily quickly check, except maybe doing an additional search without ChatGPT, at which point, why not just do that search? But also, it is poor information practices to assume that the world is set up so that if I have a question, there is a machine that can give me the answer. When I'm doing information access, instead what I'm doing is understanding the sources that that kinds of information comes from, how they're situated with respect to each other, how they land in the world. And this is some work I've done with Shrag

• 26:58Speaker 01

  Shah on information behavior and why chatbots, even if they were extremely accurate, would actually be a bad way to do these practices. So just to, you know, back to your point, yes, this system is set up to output plausible looking text on a wide variety of topics, and that's, therein lies the danger. Because it seems like we are almost there to the robo doctor, the robo lawyer, the robo tutor. And in fact, not only is that not true, not only is it environmentally runous, etc., but that is not a good world to live in.

• 27:29Alex Kantrowitz?

  And thinking about, thinking about. I just want to hit on this point. Uh, this is, I agree, I disagree with you on this one. I, I do think that some of the points that you're making are, are well founded. We don't want these things to be lawyers right away. Uh, but let me at least point, point you to one use that I've, I've had recently and you could tell me where I'm, where I'm going wrong, if you think I am. I mean, I'm in Paris now, uh, little work, little vacation at the same time. And what I've done is I've taken two documents that I've had, uh, friends, uh, who they have, they've been here often. They put together documents that they send to friends when they go here. I've uploaded that into chat GPT and then I have chat GPT like search the web and give me ideas of what to do. I tell it where I am.

• 28:10Alex Kantrowitz?

  I tell it where I'm going and it searches through like, for instance, like all the museums, the art galleries, the festivals, the concerts, and it brings it into one place. And that's been extremely useful to me to, eh, to find new cultural events, uh, concerts. Uh, there's even a bread festival going on here that I had no idea about. Uh, and now I'm going to go because it's found it for me. So there's a link when it, when it comes to this stuff, there's, there's a link uh, that you can go out and double check the work. But as far as finding information on the web, um, the, the fact that it's able to go and comb the internet for these events and then take into context, some of the, um, the context that I've given it with these documents, I think is very impressive.

• 28:51Alex Kantrowitz?

  And that's just one use case. So I was, I'm not asking it to be a lawyer. I'm kind of asking it to be, like what you said, an itinerary planner. What's wrong with that?

• 28:59Speaker 01

  Uh, so, I mean, first of all, you, you have these lovely documents from your friends and I guess what you're saying is missing is whatever current events are. So they've given you some sort of like, these are general things to look for, but they haven't looked into what's going on right now. Um, what's wrong with that? You know, on several levels, um, what, what would we do in a prior age, like even pre-internet? Right? The local newspapers would list current events. Here's what's going on. If you landed in a city, you would go find the local, probably local indie newspaper and, and look up the events page. And that system was based on a series of relationships within the community between the people putting on festivals and the newspaper writers, and it helps support probably the, the local news, uh, information ecosystem, which was a good thing.

• 29:43Speaker 01

  Um, but on top of that, uh, If something wasn't listed, you could think about why is this not listed? What's the relationship that's missing? Your chat GPT output is going to give you some nonsense. And you're right, this is a use case where you can verify whether this is real or not. It is also likely gonna miss some things. And the things that are not surfaced for you are not surfaced because of the complex set of biases that got rolled into the system, plus whatever the role of I was this time. And anytime someone says, well, I need chat GPT for this, usually one of two things is going on. Usually it's either there's another way of doing that that is giving you more opportunities

• 30:28Speaker 01

  to be in community with people to make connections. Or there is some serious unmet need, which doesn't sound like it's this case. And if we sort of pull the frame back a little bit, we can say, why is it that someone felt like the only option was a synthetic text extruding machine? And here I think you've fallen into the the former of these, which is what are you missing out on by doing it this way? What are the connections you could be making to the people around you? If you're staying in an Airbnb, maybe the Airbnb host, if you're in a hotel, the concierge, to to get answers to these questions when you're looking to the machine instead?

• 31:00Speaker 02

  I would also say, I would also say this is, I just want to add, you know, I mean, I would also say that this is a pretty like low stakes scenario, right? You can go out, you can verify these things. You can go to existing resources of, you know, event calendars that people also spend a lot of time curating online. I mean, there's a lot of stuff that's already curated online. And I mean, it's not like this this exists in prior incidents of technology. I mean, you know, one of the people that we cite in the book and talk a lot about it is Dr. Sophia Noble's work on Google and in the kind of way that Google results, you know, present very violent content with regards to racial minorities. One of the parts of the book that I I like to reference and that a lot of people don't reference.

• 31:47Speaker 02

  Uh, initially is this, this kind of part that she talks about, she talks about Yelp. And she talks about Yelp and like, um, specifically and what it's referring in terms of a, um, black hairdresser and the way that like Yelp effectively was like shutting this person out of business, um, because there was a specific need that she had for, for, um, uh, black residents of the city that she was studying and braiding hair and doing other black hairstyles, right? And so this is, this is kind of a function of all kind of information retrieval systems, right? You're thinking about what they, what they're including, what they're excluding, right? So this is not very consequential here, but in any kind of, any kind of area of say summarization

• 32:29Speaker 02

  or any, any kind of retrieval, you do need to have some kind of expertise where you can verify that and ensure that what's getting in there is not missing something huge and you're, it's going to basically then take this set of information access resources or systems, in this case crawling the web and, and knowing that that's going to miss something and then it's going to exacerbate that because then you cannot situate those sources in context.

• 32:54Alex Kantrowitz?

  Okay, let me just give my counter argument and then we can move on from this. Uh, my counter argument would be a couple of things. Uh, first of all, I don't speak French, so the local newspaper would kind of be lost on me. Uh, I am speaking, I, okay, so I am staying at a residence place. We swapped apartments, so she's in my New York apartment, I'm here. Uh, so maybe we, she and I could have gone over that newspaper together. That's, that's fair. But the newspaper, speaking of things that leave stuff out, the newspaper leaves stuff out all the time. It exercises editorial judgment. So it is bot editorial judgment for newspaper editorial judgment, but the bot can be in some ways more comprehensive because it's searching the entire web. Uh, and I'll just say one, one last thing about this.

• 33:37Alex Kantrowitz?

  I never felt, I didn't feel the need to use it. Um, I didn't say I need to use it to figure out what's going on. Like again, I had these documents. What's useful about it is, speaking of making connections with the local community, if I'm able to, here's the word, be efficient, uh, in my research through using it, I could spend much more time, uh, out in the community versus searching the web or reading the newspaper. So what's your thought on that, on, on those arguments?

• 34:06Speaker 01

  Sorry, so I've getting distracted by Alex's cat walking around.

• 34:09Alex Kantrowitz?

  So listeners, Alex's cat is here. Alex, what's your cat's name?

• 34:12Speaker 02

  This is, this is Clara, and I lift her up, but my, I have a shoulder injury.

• 34:17Alex Kantrowitz?

  Okay.

• 34:18Speaker 02

  But she is, she's knocking the mic around, so I'm going to not, I'm just trying to keep her off the mic.

• 34:25Speaker 01

  Yeah, yeah, thank you. So, you know, the, the efficiency argument. So this is efficiency argument in the context of leisure activities as opposed to in the context of work. You mentioned along the way that it is searching the whole web for you. You don't know that, actually.

• 34:41Alex Kantrowitz?

  That's right, yeah.

• 34:42Speaker 01

  And also, the whole web includes a lot of stuff that you don't actually want, like lots and lots and lots of the web is just garbage SEO, you know, stuff. And maybe you're seeing more of that in your chat GPT output than you would in with a search engine, which, as Alex mentioned, also has issues. And then finally, I'm going to take issue with you.

• 35:00Alex Kantrowitz?

  SEO garbage is made for the search engine, so.

• 35:03Speaker 01

  It is, but the search engine is also in order to stay in business have to be fighting back against the SEO garbage. It's a constant battle.

• 35:08Alex Kantrowitz?

  Probably the chatbots as well.

• 35:10Speaker 01

  So you mentioned newspaper editorial judgment versus bot editorial judgment, and I'm going to take issue there because a bot is not the kind of thing that can have judgment, nor is it the kind of thing that can have accountability for exercising judgment. And so I think that, yes, as Alex is saying, this is low stakes, but if you're using it as sort of a motivation for these things being useful in the world, then you have to deal with the fact that the useful in the world is going to entail many more higher stakes things, and then we really have to worry about accountability.

• 35:41Speaker 02

  I would also want to say too, I mean, there's a lot of, I think this argument from like quote unquote capabilities, which I don't know really what that term means. And that's another poorly defined term, I think, especially when it comes to AGI. But I mean, this argument from kind of like, well, I find it useful. I don't find terribly. Convincing, right? I mean, it's sort of like, well, okay, you you have found it useful in in either a situation in which there is a way to have some kind of verification of sources that you know about and have some kind of ground truth about, or you found it useful

• 36:25Speaker 02

  kind of from a variety of this of these different situations. But if I'm asking a chatbot about something about an area that I know quite a lot about, say sociology or social movements literature, I then have that knowledge to verify that just from my social skill in that area. And this is a term I'm kind of borrowing from a sociologist, Neil Flingstein, and my knowledge of how to navigate those areas and my professionalization as a sociologist. Okay, but then I'm but then, once it gets into those areas in which verify ability just escapes me, which is most areas, because we're not professionals in most areas. And

• 37:08Speaker 02

  although a lot of us want to be jacks of all trade, jacks and jills of all trades, then we lose that ability and we don't have the we don't have the social skill or depth of knowledge to to verify that in the same way. And so I'm really not convinced by those, well, these are useful for me in these pretty low state contexts, because that slippage then means that we're going to miss some pretty big things in some really dire contexts.

• 37:34Alex Kantrowitz?

  Okay, well, let's turn it up a notch when we come back, because we're going to talk about AI at work and AI in the medical context. And maybe we can even touch a little bit on doomerism, which you write about in the book. And and there's plenty else on the agenda. So we'll be back right after this. And we're back here on Big Technology Podcast with Professor Emily M. Bender and Alex Hanna. They are the authors of the AI Con: How to Fight Big Tech's Hype and Create the Future We Want. Here it is. So let's go to usefulness.

• 38:07Alex Kantrowitz?

  And we'll start with AI, generative AI in the medical context because why don't we just go straight for the example that we'll probably have the biggest disagreement on here. And I'm not saying that I think generative AI should play the role of a doctor. In fact, when I wrote my list of things I agree with you both on, I don't think that AI should be a therapist, at least not yet. And we know now that AI is the number one use, according to a recent study, is companionship and therapy. And the therapy side really scares me. And I think the companionship isn't the best thing in the world either. But in medicine,

• 38:46Alex Kantrowitz?

  I do find that there is some use for it. Medicine is a field overrun by paperwork and insurance requirements that I think keep have ruined the healthcare system because they keep doctors effectively tied to their computers, writing notes as opposed to seeing patients or living their lives. And Alex, before the break, you mentioned that one of the areas that this stuff is useful is when it starts to operate in your area of expertise because you're able to verify that. So, I mean, we're going to go with one use that I find to be pretty good here and would sort of, to me, doesn't make AI generative

• 39:30Alex Kantrowitz?

  AI feel like a con, is when a doctor is seeing a patient and AI, they can put a transcription, take a transcription of the conversation that they have with the patient, and then have AI synthesize what they talked about and summarize it and put it into the systems that they have for electric medical records and then verify that. So they don't have to spend the time writing those summaries up and can actually go and spend some more time with patients. So what's the problem with that?

• 40:02Speaker 01

  There are so many problems with that. And the first thing I want to say is that you named the underlying problem when you talked about insurance requiring so much paperwork. So this is one of those situations where there's a real problem here. It's not that doctors shouldn't be writing clinical notes. That is actually part of the care, but there is a lot of additional paperwork that is required because of the way insurance systems, and especially the one in the United States, are set up. And so we could work on solving that problem. And this is a case where the turn towards large language models, so-called generative AI, as an approach to this is showing us the existence of an issue. But that doesn't mean it is a good solution. So, many problems. One is, writing the clinical note is actually part of the process of care.

• 40:47Speaker 01

  It is the doctor reflecting on what came out of that conversation with the patient, and thinking it through, writing it down, plans for next treatment. That is not something that I want doctors to get out of the habit of doing as part of the care. Now, they might feel like they don't have time for it. That's also a systemic issue. Secondly, these things are set up as like ambient listeners, which is a huge privacy issue. As soon as you've collected that data, it becomes sort of this radioactive pile of danger. Thirdly, you've got the fact that automatic transcription systems, which are the first step in this, do not work equally well for different language varieties. So think about somebody who's speaking a second language. Think about somebody who's got dysarthria, so an older person whose speech isn't very clear.

• 41:29Speaker 01

  Think about a doctor who was an immigrant to the community that they're working in, who's got extra work to do now because their words are not well transcribed, and so the clinical notes thing doesn't work well for them. But the system is set up where there's these expectations that they can see more patients because the "AI" in quotes is taking care of all of this for them. And there's a beautiful essay that came out recently, I think in STAT News, and I was looking for the name of the author, didn't find it real quick. Really reflecting on how important it is to her that the doctor do that part of the care of actually pulling out from the conversation, this is what matters. And it's not just simple summarization. It is actually part of the medical work to go from the back and forth

• 42:13Speaker 01

  had with the patient, all of the doctor's expertise to what goes into that note.

• 42:17Speaker 02

  Yeah, so I want to add on, Emily has said so much of what I want to get at, which I think is, but I have, I think, three or four separate points addition to that. So first off is the technical point. So there's so tools that are that are purported to be summarization. There's a some great reporting by Gerensberg and Hilda Shelman and the AP from last October that was looking at Whisper specifically. So that's OpenAI's ASR system, automated speech recognition system, that said that medical transcription had basically was making up a lot of shit. And then we knew that this they had quote unquote hallucinations. Again,

• 42:59Speaker 02

  this is not a term that we use in the book. We, we, we, we say that it's, I say it's making shit up, but that is maybe even granting too much anthropomorphizing of the system for me. And, and so, but there is a lot of these things. Some from that quoting from that text, some of that invented text includes racial commentary, violent rhetoric, and even imagined medical treatments. So that's one major problem. The second problem is that medical transcription has been this area, which has been an area in which medicine has been forcing kind of this casualization of work for years. Right. And so medical note taking that exists in hospitals now, many, much of that is done remotely. So it's gone, gone and take this,

• 43:44Speaker 02

  taken this work that has been seen as kind of like this busy work or this, this thing that like, I don't want to write up my medical notes to be this type of work that needs to be foisted on someone else. So prior to this kind of ASR element of it is, is we've had these, oh, thanks for linking that, Emily, and I'll link the, I'll link the AP article that I'm, that I'm looking at too. Part of that work has actually been offshored a lot into this kind of movement of, of outsourcing. So a lot of that is done remotely as, as a part of this casualization. And this seemed to be, I think, to be a lot of.

• 44:25Speaker 02

  A lot of I want to point out the gender nation notion of this. This is like a very kind of like women's based work and that reflects a lot of the ways in which so much of uh quote unquote AI technology wants to basically take the work that has been traditionally the domain of women and is saying, well, we can automate that or we can casualize that in different ways. And that's important because it sees this work as not actually part of quote unquote the work. It is seen as um work that should be that ought to be casualized and offshore. And so, and I appreciate the essay that Emily shared because um that essay say like, no, this is actually part of the element of doctoring.

• 45:09Speaker 02

  And then I want to also just kind of couch all of this stuff in the kind of political economy of the medical industry, thinking about what does it mean to rush and put and have more and more remote medicine and having more and more doctors see more patients and these efficiency gains from doctors isn't going to like make their jobs necessarily easier. It's going to put more of a pressure on them. Now that you're in a position where you don't have to take medical notes, you're going to be running from position to from appointment to appointment to appointment. And my sister is a nurse. She's a nurse practitioner and she's basically seeing this in her job right now at her clinic. She's well, like now we have these things where I have to see patients now, you know, and

• 45:52Speaker 02

  if you know, it's it's not that I'm gonna go and be on the beach anywhere. It means that I'm gonna have, you know, I'm gonna have nine to 10, 15 minute appointments a day. I'm not gonna have enough proper treatment, uh proper time to spend with patients. So if these things could be, you know, like I I would say to the coda to all of this is that if AI boosters could really offshore all of doctoring to chatbots, they would. And this is one case in which Bill Gates has said, you know, in 10 years, we're not going to have teachers and doctors. What a nightmare scenario to have non-teachers and not doctors. And Greg Carrata really gives it away. We cite him in a book where he says, and a med palm two, you know, this thing is really efficient. We're going to increase tenfold our medical ability.

• 46:36Speaker 02

  But I wouldn't want this to be part of my family's medical uh medical journey.

• 46:41Alex Kantrowitz?

  Okay, but you're again here, you're you're picking out what is like some of the most extreme statements and I I started my question saying

• 46:48Speaker 02

  It's Bill Gates.

• 46:49Alex Kantrowitz?

  And he Bill Gates can make extreme questions, extreme statements.

• 46:52Speaker 02

  He's the guy.

• 46:54Alex Kantrowitz?

  I don't think he's the guy and I think that that that doesn't reflect the broad consensus here and definitely not the question that I asked, which again was about using using this to take some of the time that the doctors are using, you know, in paperwork and give that back to either the doctors or to have them be able to see more patients.

• 47:15Speaker 01

  So we very much address that point. First of all, I want to name the author of that essay. Her name is Alia Barakat and it's a beautiful essay. She's a mathematician and also a patient with a chronic condition. Wonderful essay. But yeah, you said, give that time back to the doctors or have them see more patients, right? It is not going to be going back to the doctors. That's not how our healthcare system works and it's also going to therefore decrease quality of patient care. It is lose lose, except for the hospitals maybe getting more money and certainly the tech companies that are selling this to the hospitals.

• 47:46Alex Kantrowitz?

  Okay.

• 47:47Speaker 02

  I'm also curious in terms of thinking about it. I mean, yes, what is that? I I'm curious in thinking about the more nuanced position and like, who are the reference here that you're thinking of, Alex? What's the consensus on this? Because I I'm I, you know, like, we see the egregious, you know, elements of this and I'm wondering what the medical consensus is, you know, like, who's what's an example, you know, just to put poison now I'm interviewing you, but like, who's who's someone that you think is doing this very well on?

• 48:15Alex Kantrowitz?

  Well, I mean, someone doing this well, like, again, I don't think that this stuff is well developed yet, but I've definitely seen enough doctors just buried in paperwork. And we we started this whole segment talking about how this is, I guess it's an insurance driven thing. And so it's interesting that, I guess, do you both not like the way that the insurance companies are guiding the system, but also think that it's good practice to have doctors write those notes for them or?

• 48:44Speaker 01

  Hold on, there's two use cases for doctors' notes, right? There is actually documenting for the patient and for the rest of the care team what has happened in the session. And that I think is a super important part of the work of doctoring. I believe you that there's a lot of additional paperwork that has to do with getting the insurance companies to pay back. And no, I don't like that system at all. It does not, the insurance companies are not providing any value. They are just vampires on our healthcare system in the US.

• 49:10Alex Kantrowitz?

  Okay, I think we can, we can agree on that front.

• 49:16Alex Kantrowitz?

  And anyway, but I do, I do think that as this stuff gets better, I understand a patient wants this to happen. Do I think a doctor would be giving them worse care? If they allowed the AI to summarize the notes or to pick out the more important parts? If this stuff was working well, not necessarily.

• 49:33Speaker 02

  So that's a, that's a big if, you know, what does that mean when this stuff is getting better? And this stuff working well, do you mean kind of like the absence of making shit up?

• 49:43Alex Kantrowitz?

  Right, definitely. I mean, but we all, we both, we all, we all agree that the doctor will have to verify and check this information after.

• 49:50Speaker 02

  Well, I guess the problem is there then like, then why are we having the doctor double check that to begin with, right? In an area where the doctor has 15 minutes to see every patient? And there is an AI quote unquote scribe doing, or quote unquote AI, I don't want to call it AI scribe. There's an automatic speech recognition tool. Doing automatic speech recognition on these things. In what, in what space or with what time does the doctor have to verify those in an area? I mean, this is, I mean, like.

• 50:22Alex Kantrowitz?

  Well, the time that they would be spending writing those notes in the first place.

• 50:26Speaker 02

  Is verification an easier task than, than transcription? I guess that's my question. I would proffer no. I mean, just from my experience using these systems and I mean, I'm not a doctor. Thank God, although, although I've thought about it. Not that kind of doctor to the sugar in my parents. But then I guess the question is, is of that is I, in my experience, I've used these tools for interviews, specifically and kind of qualitative interviews with data workers. And have spent time with these tools and have just had such an awful time trying to think about this, especially with regards to,

• 51:08Speaker 02

  you know, this isn't, you know, this isn't medical terminology, but it's terminology around doing data work. We're talking about training AI systems and it just, it is such a terrible job. And at one point I, I threw it all out and I said, okay, I just am sending this to somebody to actually transcribe, because this is not helpful for me and it's taking me more time. Starting with the transcript and then from doing it from scratch and I've transcribed. You know, I'm not a primarily qualitative interviewer, but I've spent time, you know, transcribing dozens of interviews um in my research career and have found it just very difficult. So I mean, I guess the question is, is that verification, you know, is that

• 51:51Speaker 02

  taking the time that could be just be used for the doctoring and working with patients? And I mean, like, you know, the holding, you know, everything about the insurance industry, you know, stable, you know, like, is that kind of notion of thinking about, you know, different patient, um, how the patient presents, how the patient is describing their, their, their, how they're, you know, how they're presenting is that, you know, that is often the work of doing it, you know, and the medical training I do have is that I am at one, I was at one point a licensed EMT, uh, and writing up PCRs is not like, you know, no

• 52:31Speaker 02

  one wants to write up the PCRs at the same time you're spending time taking note of how a patient is presenting. The patient is, you know, Alex rhythmic, just bringing back to the Alexes, um, the, you know, the patient is cyanotic around their lips. Like, these are things that, you know, a med, a healthcare professional would be paying attention to is making notes, maybe because they're like writing it later. So I'm thinking about this process of writing and what it does to our own practice of viewing and aiding and administering medical care.

• 53:03Alex Kantrowitz?

  Okay. I mean, we'll agree to disagree on this front. Uh, but again, I, I think we are all on the same page that insurance companies requiring additional writing just because they hope you don't ever get to the claim. If you don't file it, that's probably bad. Uh, and we don't think that there should be, uh, that, that there should be AI doctors, at least yet. I, that's what I say. I think you guys probably say never. So, all right. I want to end on this, which is, or maybe, maybe we can do two more topics. Um, I guess like, here's my question for you. A lot of the discussion of AI's usefulness in jobs uh, in the book discusses this, these tools being imposed top down. Uh, but what if they

• 53:47Alex Kantrowitz?

  come bottom up? Like, what if a worker can find use for them and actually make their job easier by getting good at using something like a chat GPT or a Claude? Or if, you know, we, again, we like kind of talked through the medical use case. If a doctor does find that this is useful for them. Are you opposed to that?

• 54:06Speaker 01

  So, yes, and I think that actually Cadbury, of all people, put it best. There's this hilarious commercial that was for the Indian market, sort of showing how the supposed efficiencies that you're getting out of this just ramps up the speed of things and doesn't leave you time to really get into the work that you're doing and be there. I think that the most credible use cases I've heard for these things are, first of all, as coding assistance, so that's sort of a machine translation problem between natural language and some programming language. And there I really worry about technical debt, where you have, you know, output code that was not written by a person that's not well documented, that becomes someone else's problem to debug down the line.

• 54:51Speaker 01

  But also in writing emails. People hate writing emails and people hate reading emails. So you get these scenarios where somebody writes bullet points, uses ChatGPT to turn it into an email, and the person on the other end might use ChatGPT to summarize it. And it's like, okay, so what are we doing here? Like, and again, taking a step back and saying, what are the systems that are requiring all this writing that everyone finds a nuisance to write and to read? Can we rethink those systems? And also, I just have to say that whenever I'm on the receiving end of synthetic text, I am hugely offended. And one of the things that we put in the book is-

• 55:23Alex Kantrowitz?

  I definitely got one of those emails yesterday. And I was like, you use ChatGPT for this. I know you did.

• 55:28Speaker 01

  Yeah, if you couldn't be bothered to write it, why should I bother to read it?

• 55:31Alex Kantrowitz?

  Right.

• 55:31Speaker 02

  Yeah. And I mean, I mean, it's very interesting putting this in, like, thinking about cases in which workers are using this kind of organically. And I kind of like, in the case where it's, you know, like,

• 55:46Speaker 02

  this is the case where, like, first off, I've, I've heard a very little of that personally.

• 55:51Alex Kantrowitz?

  Really? I mean, I think there's plenty of workers that are finding a lot of use of this, but-

• 55:56Speaker 02

  I would say the analog that I find to be where it's not top down is in education. And to that degree, I think that's kind of a failure in thinking about like what education is, right? I mean, in that case.

• 56:07Alex Kantrowitz?

  For students to be using this to get through their classes.

• 56:10Speaker 02

  Yeah, right.

• 56:10Alex Kantrowitz?

  Are you talking about teachers putting stuff together?

• 56:12Speaker 02

  Well, both, but I'm thinking about, I'm thinking about the students, right? And I'm just thinking about areas in which, but I'm using that as sort of an analog. And then thinking about what are the conditions that are forcing students to use this, right? If there's kind of cases in which this seems to be sort of useful, okay? What are the cases in which, what does that say about the job? What does it say about like how the work is oriented, right? In that case, then maybe there might be, needs to be kind of different efficiencies or thinking about how the job is operating, right? I then worry then that these things become mandated in work environments. And you're saying, well, people are using this and so everybody's using this.

• 56:50Speaker 02

  And then where does that leave the people who are resistors or thinking about, well, I'm, I know this can't do a good job. So where's that putting me? And I think we've already seen such a justification for this as being a place where

• 57:06Speaker 02

  employers have been reducing positions by the scores because there's a notion that these tools can do these jobs suitably and to a certain kind of degree of proficiency, which is just not the case. That has me worried about down the line in these areas that Emily's mentioned, the kind of technical debt area. The kind of, how do we know? And there's kind of an overestimation of capabilities of these tools in that case.

• 57:35Alex Kantrowitz?

  Okay, I know we're at time or close to time. Can I ask you one question about doomers before we get out of here?

• 57:40Speaker 01

  Sure, let's end by dogging on doomers.

• 57:43Alex Kantrowitz?

  Okay, so I definitely, I saw that there was a chapter about doomers here and I was excited to read it because my position has been largely that those who are worried that large language models are going to turn us into paper clips or either marketing what they're selling or just very into, I don't know. They like the smell of their own body odor. It, I mean, I guess it's not a terrible thing to be worried about, but there's so much more. And it seems so unlikely that this is gonna hurt us. So I definitely wanted to get your take on why you are, um, why you're, you're down on doomerism? And I, and let me just give my one caveat here.

• 58:24Alex Kantrowitz?

  There's a line in your book that says, uh, the, um, that AI safety is just doomerism. And it's only about these long-term problems. But I've definitely heard some of the AI safety folks like Dan Hendricks from the Center of AI Safety, talking about like really important near-term issues like whether this technology could help virologists with bad intent. Uh, so I wouldn't, I wouldn't malign the entire AI safety field, but the doomerous stuff, I, I hear your point. All right. So, so attack that and then we'll get out of here.

• 58:57Speaker 01

  So I just want to put in a shout out for, um, a new book by Adam Becker called More Everything Forever, um, which really goes deep into the connections between the sort of doomerous thought and these more palatable looking sides of what's called effective altruism. And also in that context, there's a wonderful paper by, um, Tameet Ghibru and Emil Torres on what they call the Tescrill bundle of ideologies.

• 59:23Alex Kantrowitz?

  Yes.

• 59:23Speaker 01

  And, you know, I think that if you're, um, if your concern about the systems is not rooted in real people and real communities and things that are actually happening, like even this like, oh, but bad actors could use it to, you know, more quickly designed, you know, uh, viruses and stuff like that. That's still speculative, right? So anytime we are taking the focus away, it's, it's like, has that happened? Right? This is, this is still people writing science fiction, fanfiction for themselves. And, you know, it's not, it's based on these jumped up ideas about what the technology can do and taking the focus away from the actual harms that are happening now, including the environmental stuff we started with.

• 1:00:04Alex Kantrowitz?

  Right. But you don't want to wait until it synthesizes a virus, right? You want to get, you want to get ahead of that, right? Like we had it with social media, there were some issues with social media, but some of these, there was not a focus on some of like the potential long-term issues. And that only came up later on, at least in the beginning.

• 1:00:24Alex Kantrowitz?

  You don't agree. Say. Wait.

• 1:00:26Speaker 01

  There are problems with social media, for sure.

• 1:00:28Alex Kantrowitz?

  Yeah.

• 1:00:29Speaker 01

  And some of those problems were documented and explained early on and people were not paying attention, but they were real problems that were being documented as they were happening as opposed to imaginaries about, well, someone's going to use this and Dr. Evil up a bad virus.

• 1:00:45Alex Kantrowitz?

  Okay. Yeah. Go ahead, Alex. For the sake of time, I think that's fine. I don't have much to add. All right, well, look, the book is called The AI Con: How to Fight Big Tech's Hype and Create the Future We Want. The authors are Emily M. Bender and Alex Hanna. Emily and Alex, I've been reading your work for a long time and it's great to have a chance to speak with you. Like I said at the top, you know, for those who are listening or watching and you may not agree with everything, either everything I said or everything our guest said, hey, at least now you know these arguments and you know the arguments for and against. And we trust you to make up your own opinion and do further research. And we've definitely

• 1:01:26Alex Kantrowitz?

  had plenty of good stuff to keep digging into, shout it out over the course of this conversation. So Emily and Alex, great to see you. Thank you so much for joining the show.

• 1:01:35Speaker 01

  Thank you for this conversation and enjoy Paris.

• 1:01:37Alex Kantrowitz?

  Thanks, Alex. Have a great time. Thank you both. Thank you, everybody, for listening. We'll see you on Friday for our news recap. Until then, we'll see you next time on Big Technology Podcast.

• 1:01:59Speaker 03

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPact.com. Not authorized by any candidate or candidate's committee.