Can you spot AI-written stories? Listen and find out

Can you tell the difference between a human-written short story and an AI-generated one? Most people can’t. In a series of experiments https://www.cambridge.org/core/journals/judgment-and-decision-making/article/bot-or-not-can-people-tell-the-difference-between-stories-written-by-a-human-or-by-an-ai-system/45E6DC0BB90AA648654D5AE243F6C667 led by a team at Villanova University, researchers found two things: First, most people said they preferred human written stories, but couldn’t tell the difference between AI and human writing; and second, when presented with two similar short stories, one written by a human and one generated by AI, readers tended to prefer the AI-generated piece – especially if they had been told the AI story was written by a human. We excerpted two of the short stories https://npr.org/2026/08/12/nx-s1-5913976/ai-human-detector-writing-check on our website for you to guess which is AI-generated and which is human-generated. Check out the full set of stories here https://osf.io/d8f6m/overview. Interested in more science and technology news? Email us your question at [email protected] mailto:[email protected]. Support public media with NPR+ and enjoy perks for over 25 podcasts like this one. This show’s perks include sponsor-free listening. Learn more at plus.npr.org http://plus.npr.org. See pcm.adswizz.com https://pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences. NPR Privacy Policy https://www.npr.org/about-npr/179878450/privacy-policy

• 0:00Speaker 00

  This message comes from Planet Visionaries. Host Alex Honnold sits down this season with extraordinary doers creating real climate solutions. Listen or watch wherever you get your podcasts. You're listening to Short Wave from NPR.

• 0:18Speaker 01

  Hey, Short Wavers. Regina Barber here with producer Burley McCoy.

• 0:21Speaker 02

  Hey, Gina.

• 0:22Speaker 01

  We are back with our biweekly Science News roundup with friend of the pod, host of All Things Considered, super awesome lady, Mary Louise Kelly.

• 0:30Speaker 03

  Hey, happy to be here. Hi.

• 0:33Speaker 01

  Hey. So our first topic is AI. And Mary Louise, we're going to read two different short story excerpts to you, one written by a human, one written by AI.

• 0:43Speaker 03

  Oh, I can't wait. I'm gonna nail it. Let's do it. Let's see who goes first. Regina, go first.

• 0:49Speaker 02

  OK, OK. In the stillness of the early morning, I find myself at the edge of a koi pond, a relic from my childhood, watching the fish swirl in patterns only they understand. Their vibrant colors, a mosaic of oranges, whites, and blacks, mirror the autumn leaves that dance on the water's surface. Each koi, with its graceful, unhurried movement, seems a silent sentinel guarding the mysteries of life and death. The pond nestled in the heart of my mother's garden has been a constant in my life. As a child, I would spend hours by its side, my mother sitting quietly beside me, her eyes always carrying a soft, distant gaze. She would tell me stories of her youth, of

• 1:32Speaker 02

  her dreams and fears, her voice weaving tales that made the past seem like a vivid tapestry full of color and life. I feel like that's a hard act to follow, Gina. Like, I've got to put on a voice, too.

• 1:45Speaker 03

  You really, you really put yourself into it, Gina. Burley.

• 1:49Speaker 02

  OK, story number two. Something is wrong with the fish in the koi pond outside my mother's floor-to-ceiling bedroom window. There are three of them: small, golden orange, medium marmalade, and large vermilion with black markings. Last time I visited for the weekend, I named them, in reverse order of their descriptions and with no assignment of gender intended, Kurt Russell, Goldie Hawn, and Kate Hudson. My mother's husband says he thinks one of them is about to give birth. Then, he says the big ones usually eat the young, but that they're pretty good at hiding among the hydrangeas growing hydroponically in one end of the pond. I haven't seen a new fish born here in some 10 years, but perhaps that's why. Maybe they're not so good at hiding after all.

• 2:37Speaker 01

  What do you think?

• 2:38Speaker 03

  OK, so we're viewing the assignment. We've just had two koi pond readings. One is AI, one is written by a human, and I'm guessing which. I'm torn because I liked the second one better. I've just got like an image in my head. I see Goldie Hawn swimming around.

• 2:56Speaker 01

  Yeah.

• 2:57Speaker 03

  So I want to go with that. I'm wondering a little bit, the no assignment of gender intended, it feels like something AI would have been programmed to be politically correct about. So I wonder if I'm wrong, but I'm going with number two, Burley, the one that you read.

• 3:10Speaker 01

  Is AI or is?

• 3:12Speaker 03

  Is my favorite and is human-written.

• 3:15Speaker 02

  OK. All right. Well, you will find out after the break.

• 3:18Speaker 03

  Wait, I don't get to know? I was such a tease.

• 3:23Speaker 01

  Well, just to tease you a little more, today on the show, you'll also hear about a newly discovered wandering supermassive black hole, and how pumas could make roads safer for humans. You're listening to Short Wave, the science podcast from NPR.

• 3:37Speaker 00

  Stay tuned for the following announcements and messages from our sponsors. This message comes from Schwab. Investing with Schwab is like a hike with endless trails. Go solo with self-directed investing, take stops along the way with trading, or choose a guide with full-service wealth management. Whatever trails you take, you can invest your way with Schwab. This message comes from Edward Jones, where they believe rich isn't about having life all figured out. It's opening yourself to all the possibilities. That's why your dedicated financial advisor provides long-term planning built around you, meeting you where you are and helping you get closer to where you want to be. So no matter where you're starting

• 4:20Speaker 00

  from, or wherever life takes you, you can move forward with confidence. The key to being rich is knowing what counts. Let's find your rich together. Edward Jones, member SIPC. This message comes from Planet Visionaries, the podcast celebrating the people creating practical, hopeful solutions for the planet's urgent conservation challenges. Hosted by legendary climber and advocate Alex Honnold, each episode goes beyond the headlines to show it's actually working and who's making it happen. In partnership with the Rolex Perpetual Planet Initiative, this is Planet Visionaries. Listen or watch wherever you get your podcasts.

• 5:00Speaker 01

  OK, Mary Louise, earlier, we read you two stories, and you guessed which was written by AI. You guessed the second one was the human and the first one was AI.

• 5:09Speaker 03

  Correct.

• 5:09Speaker 01

  And now, I'm going to reveal to you. This is like a reality show.

• 5:14Speaker 03

  Yeah. I feel like I'm about to get dumped into a shark tank.

• 5:17Speaker 01

  You were right. You should have seen her face. She's so happy.

• 5:22Speaker 03

  I am so happy. I have a question, though, if I'm allowed to ask.

• 5:26Speaker 01

  Of course.

• 5:26Speaker 03

  Those were excerpts.

• 5:28Speaker 01

  Yeah.

• 5:28Speaker 03

  Just like, I don't know, a couple paragraphs she all just read me, like, not a whole story. What was the filler content that participants read?

• 5:35Speaker 01

  So we're talking short stories. So this paper that was recently published in the journal Judgment and Decision Making included a few experiments. In the first one, participants read one of several short stories, some written by people, some of them AI-generated. Then they rated them. But here's the catch. Participants were either correctly or incorrectly told that their story was written by a human or AI.

• 5:56Speaker 02

  In the other experiments that these researchers did, participants got a pair of stories. One was human-written, and one was AI-generated, and they were asked to identify which was which. And these researchers found some key things. First, most participants couldn't tell which stories were AI-generated and which were human-written. And second, they tended to prefer AI stories, especially if they thought those AI stories were written by humans.

• 6:22Speaker 03

  Ah, OK. So what people said they preferred, human writing, and what they actually preferred was often different?

• 6:29Speaker 01

  Yeah, yeah. So this fits with a growing body of research suggesting that people's beliefs about AI involvement in a message influences how they feel about that message.

• 6:39Speaker 03

  Expectation bias.

• 6:40Speaker 01

  Exactly. So Dina Weisberg says that it's a sign that large language models have gotten a lot better over the past few years.

• 6:47Speaker 04

  I personally feel confident in saying the time has passed when it is simple to just be able to tell what a human has written and what an AI has written.

• 6:57Speaker 01

  Dina's an associate professor of psychological and brain sciences at Villanova University and the senior author of this study.

• 7:03Speaker 02

  And now she did caveat that these results might be because of the length of these stories. So they were all around 1,000 words each. And when we talked to some other experts in the field who have done similar research, they emphasized that length is often a key factor in our being able to detect AI-generated content. We also talked to Mohit Iyer, an assistant computer science professor at the University of Maryland. And he wasn't involved in the study, but he told us that because AI-generated content tends to be pretty formulaic, the longer they are, the worse their quality is.

• 7:37Speaker 03

  AI got nothing on Tolstoy.

• 7:39Speaker 02

  No, but.

• 7:41Speaker 03

  Go back to the study, because I'm trying to figure out why. Like, why, why do people prefer AI-generated content?

• 7:48Speaker 02

  Yeah, Dina had this hypothesis.

• 7:51Speaker 04

  The AI-written stories were very obvious, and I think more predictable in their flow, which isn't surprising, given how AI generates text. And the human-written stories were more difficult to read. They were more subtle. They didn't spoon-feed you the themes in quite the same way.

• 8:08Speaker 01

  In a way, they're easier stories to read. Dina and the rest of her research team have made those stories available, so you can read them yourself, the human-written ones and the AI-generated ones. We'll link them in the show notes.

• 8:20Speaker 03

  OK, so, Burley and Gina, you said topic two is wandering, supermassive black holes. What even is that?

• 8:28Speaker 01

  Yeah, so, supermassive black holes are these objects in space that are millions of times more massive than our sun. And all the ones, except for two, that have been found, are in the center of galaxies, hence wandering. This is one of those two.

• 8:42Speaker 02

  OK, but don't worry, Mary Louise, this wandering black hole is 750 million light years away from us.

• 8:49Speaker 03

  Okay, so we got a little bit of time. But if it's that far away, how did they find it?

• 8:55Speaker 01

  It ate a star.

• 8:56Speaker 03

  It ate a star?

• 8:58Speaker 01

  Yeah.

• 8:58Speaker 03

  Poor star.

• 8:59Speaker 01

  Yeah.

• 8:59Speaker 03

  And now I'm nervous again.

• 9:01Speaker 01

  No, no, no, don't be. It's fascinating, really, because it's most likely not really wandering, like you say. It's not going to come towards us. It's really probably orbiting a very, very faraway galaxy. Usually, black holes are invisible unless they're feeding on something. So the way scientists detected this one was by observing a flash of light created when the black hole ripped apart and ate a star.

• 9:23Speaker 02

  It's like this little black hole holds up a flashlight and is like, here I am. Look at me. So that's Gillian Rist-Dinisad, an author on the paper that came out in the Astrophysical Journal Letters about this black hole. And her colleague, Robert Stein at the University of Maryland, trained an algorithm to recognize the pattern of these special light signatures that indicate that a supermassive black hole ate a star.

• 9:47Speaker 05

  They are very rare, but we started to find them in the centers of galaxies. And what we did was we trained the AI algorithm to recognize this very characteristic pattern of light.

• 9:56Speaker 03

  Regina, stay with that point he just made, that these are very rare. Why have scientists only found two of these black holes wandering? And I'm using air quotes, wandering outside galaxies?

• 10:06Speaker 01

  Yeah, so that flash of light is rare, but on top of that, we were only looking for those flashes of light in the center of galaxies. And it kind of blew my mind that astrophysicists, we never thought to look at anywhere else until Yale theoretical astrophysicist Pramavada Natarajan and her team proposed the possibility of wandering supermassive black holes and how to find them with this special light signature. And she says this discovery is a huge leap forward to learning more about black holes, which are still pretty mysterious.

• 10:35Speaker 06

  But it looks super exciting, right? So you could find dozens and dozens per year. That's our prediction.

• 10:44Speaker 03

  Let me drag us back to Earth.

• 10:45Speaker 01

  Yeah.

• 10:46Speaker 03

  Topic three, which is how pumas could make roads safer for human drivers? And we should say a puma, that's what? It's a mountain lion, right?

• 10:56Speaker 02

  Right, or they're called cougar sometimes, too. So here, the researchers looked at puma movement in the Olympic Peninsula in Washington State. And they found where these big cats were most active, the number of vehicle-deer collisions were 76% lower than in areas where pumas were least active.

• 11:15Speaker 03

  76% lower, that's huge. Okay, vehicle-deer collision, 76% lower. Is that because, why, because pumas are eating the deer? There's fewer to hit?

• 11:26Speaker 01

  So I thought that, too, but it's not that. So researchers controlled for deer numbers and still found this huge difference in collisions. And the team says it's because the pumas are likely changing deer behavior, both by prompting deer to be more active during the daytime, which makes them more visible to people driving versus in the nighttime, and by pushing them to more remote places away from roads.

• 11:47Speaker 03

  That's interesting. Why are pumas pushing them away from roads?

• 11:51Speaker 02

  So the researchers think it's because roads break up the landscape and make for really effective hunting grounds for pumas. So think about like a road by a forest. The road is essentially a place the deer would be out in the open. Meanwhile, a puma could be hiding in the forest, waiting to ambush them. So the deer avoid it, opting for more secluded areas. And this study was recently published in the journal Current Biology.

• 12:15Speaker 03

  I'm thinking about applications. I'm guessing you all are not about to tell me we should just release a bunch of mountain lions to make our roads safer?

• 12:23Speaker 01

  Yeah, not quite yet, or maybe ever. But lead author Justin Sirachi from Conservation Science Partners said it could add to the conversation some people are having in places where pumas are starting to come back naturally.

• 12:36Speaker 07

  And, you know, one response to that could be, it's a predator, get it out of here. But another response could be to allow that natural recolonization to occur. And I think this at least provides an argument in favor of that.

• 12:48Speaker 02

  Otherwise, pumas coming back could be a hard sell to folks who may be worried about safety or hunting competitors.

• 12:55Speaker 03

  Well, with that, I will say thanks to you both so much for having me, Gina and Burley.

• 12:58Speaker 01

  Thank you. Congratulations on being right.

• 13:03Speaker 03

  It is my honor. I wear my crown with humility.

• 13:06Speaker 01

  You'll never forget this. You're going to hold on.

• 13:07Speaker 03

  Never.

• 13:10Speaker 01

  You can hear more of Mary Louise Kelly on Consider This, NPR's afternoon podcast about what the news means for you.

• 13:16Speaker 02

  And for more stories just like this one, follow Short Wave on whatever app you're listening to.

• 13:21Speaker 01

  This episode was produced by Hannah Chin, Juan Diego Ramirez, and Jordan Marie Smith. Hannah also contributed reporting. It was edited by Rebecca Ramirez and Patrick Jaren-Watananen.

• 13:32Speaker 02

  Tyler Jones checked the facts. Ted Mebane and Quacey Lee were the audio engineers. I'm Burley McCoy.

• 13:38Speaker 01

  And I'm Regina Barber. Thank you for listening to Short Wave, the science podcast from NPR.

• 13:50Speaker 08

  Stay tuned for the following announcements and messages from our sponsors. This message comes from Google Workspace with Gemini. Ever notice how every new AI tool asks you to explain your business to it all over again? Google Workspace with Gemini skips that. You get Google's AI built right into the premium versions of Gmail, Google Docs, Sheets, and more. No new tools, no extra logins, just one subscription that gives you personalized AI right inside your actual workflow. Try Google Workspace with Gemini at g o o dot g l e slash Workspace with Gemini.

• 14:26Speaker 00

  This message comes from EasyCater, the workplace food platform. EasyCater helps organizations order food from favorite restaurants, meet dietary needs, and stay on budget. With employee meal programs, flexible payment options, and 24/7 customer support, all on one platform. Learn more at easycater.com.