Did a Human Write This?

What happens when the majority of content on the internet tips over into AI slop? On this episode of Galaxy Brain, Charlie Warzel talks to Max Spero, the co-founder of Pangram, an AI-detection company. They discuss how AI-detection tools work and how effective they can be at identifying what’s made by humans and what comes from a chatbot. They explore the cultural concerns around authenticity in the large language model era, and whether detection can keep up as models improve. The pair discuss how the speed of AI development and synthetic content threatens to degrade the quality of human writing and pollute the internet—and what, if anything, can be done to stop it. Get more from your favorite Atlantic voices when you subscribe. You’ll enjoy unlimited access to Pulitzer-winning journalism, from clear-eyed analysis and insight on breaking news to fascinating explorations of our world. Atlantic subscribers also get access to exclusive subscriber audio in Apple Podcasts. Subscribe today at TheAtlantic.com/Listener. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Tired of rage scrolling through AI slop and clickbait just to get a handle on today's news? Meet Inpress, the algorithm-free news app that makes it fun and rewarding to stay informed. Track your media literacy and rediscover reality. Download today at inpress.app. That's I, N as in news nerd, press.

• 0:20Speaker 01

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed Sponsored Jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed Sponsored Jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed Sponsored Jobs.

• 0:49Speaker 02

  I want to see people using AI to cure cancer and, you know, make senior care easier and make all of our lives better. And I also don't want to see AI polluting the internet. So sort of like there's these two sides and I want to see the good side of AI flourish and I want to help mitigate the harmful effects of AI as much as possible.

• 1:14Charlie Warzel?

  I'm Charlie Wurzel and this is Galaxy Brain, a show where today we're going to talk about the flood of AI writing online and a tool that claims to help us determine what's human and what's been written by a chatbot. One of the big fears of the generative AI moment is that we are entering this kind of slopocalypse. Chatbots, audio, video, and photo creation tools, they all make it extremely easy for anyone to churn out synthetic content quickly, especially text. By now you know the story. Students are using ChatGPT and other tools to write their essays for them. A rash of hastily self-published books on Amazon are clearly not real. Search engine optimization marketers and content farmers are flooding

• 1:57Charlie Warzel?

  the internet with articles written by chatbots in order to game Google and to make a quick buck. The web is just shuddering under the weight of all of this slop. And there's some research that suggests that half of all new articles generated online are coming out of large language models. There's all kinds of issues with this, right? A lot of this content is forgettable, soulless pablum. It drowns out quality, human-made stuff.

• 2:22Charlie Warzel?

  But one of the bigger problems is social. On an internet where it feels like every other person is passing off generated AI text as their own, this distrust forms. Some journalistic outlets have embraced chatbot-assisted stories, while others, like us at The Atlantic, have strict rules about AI use and pride ourselves on the very human craft of writing. For example, in March, the publisher Hachette canceled a horror novel called Shy Girl for suspected AI use. Now, the author denied using AI in a statement to The New York Times and suggested that an editor inserted AI writing into a draft. But semantics aside, there are fears about the industry's ability to keep up and detect AI-generated text inside manuscripts. But the reputational risk isn't limited to book publishing.

• 3:07Charlie Warzel?

  In 2023, Max Spero co-founded Pangram. It's an AI-detection company that uses machine learning to try to distinguish human writing from AI. Pangram is being used by publishers, academic institutions, and other places to try and ferret out suspicious work. The company boasts of having a false positive rate of 1 in 10,000, meaning 1 in 10,000 times it identifies human work as AI-assisted or generated writing. Pangram has been at the center of unveiling a lot of AI-generated work. When suspicions began to swirl about Shy Girl, Spero ran the text of the novel through his software and declared that it was 78% AI-generated. He posted that research to his X account and spoke to The New York Times about it.

• 3:49Charlie Warzel?

  The company now has a browser tool that lets people scan Reddit, LinkedIn, X, Substack, and other social media posts to see what writing is human, and it's already made news. Wired recently reported that, according to Pangram, an April thread from the Pope's X account, ironically a thread warning about the dangers of AI, was itself likely written with the assistance of AI. The Vatican didn't respond to Wired's request for comment. And recently, the technology journalist Taylor Lorenz used the tool to scan thousands of posts on Substack from popular newsletters. Lorenz found some newsletters in the top rankings were, quote, "publishing 100% AI-generated content," according to Pangram, seemingly with no human editing whatsoever.

• 4:34Charlie Warzel?

  "Online, you can see the contours of an AI-detection arms race forming. Coders are already building tools to introduce errors into AI writing or to strip out AI conventions in order to make it seem more human." Sparrow jokingly refers to himself as a slop janitor. And he says that his mission with Pangram is to increase transparency and to help keep the internet from becoming less and less human. But the stakes are high. Pangram is quite good, but it isn't perfect. And AI detection is getting harder every day as the models get better and as the internet that it has to train on becomes more and more synthetic. So how does Pangram work? How can we distinguish between the more convincing AI text and the real stuff?

• 5:17Charlie Warzel?

  Are humans starting to write more like AI now? And is all of this detection helping us preserve our humanity? Or is it making us more paranoid? I brought on Sparrow to talk through all of it. But first, a quick break.

• 5:36Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait, and AI noise. Think social media. That's where Inpress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with Inpress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.

• 6:09Speaker 01

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed Sponsored Jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed Sponsored Jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed Sponsored Jobs.

• 6:47Charlie Warzel?

  Max, welcome to Galaxy Brain.

• 6:50Speaker 02

  Hey, Charlie. Thanks for having me.

• 6:51Charlie Warzel?

  So let's talk about Pangram, which is, you know, AI writing detection software. It is something that, before I was aware of you, I had, you know, I had, I had heard of it. You know, first, I think, first time it really came up for me was, you know, hearing about educators and people like that, using that to run, you know, papers from students through and things like that. But then I started seeing it in a lot of different places, specifically, most recently, I was talking to a book agent who was, who was just telling me that like, everyone is just like, leaning at publishing houses like really hard on, on it. And basically, like, and anything they can, you know, latch on to that may be able to detect, is this human written prose? Is this AI generated prose?

• 7:35Charlie Warzel?

  Is it assisted in some way, you know, sort of in the middle ground there? But I wanted to start before of all that, like, why, why do we need AI detection software? Like, why do we track this stuff? What is the need to know this in your mind?

• 7:50Speaker 02

  Basically, I think AI enables just such an incredible, like, large volume of content without any way to differentiate between human and AI generated content. We just like lose any semblance of signal to noise ratio, right? It's like, if I'm reading text on the internet, you know, it takes me one minute or five minutes to like, compose a tweet. Well, in that time, a large language model can pump out hundreds of thousands of tweets. Obviously, there's like some onus on platforms to help, like, also filter out like AI generated or spammy content and get you to like, the real stuff.

• 8:31Speaker 02

  But I kind of believe that it's going to be increasingly difficult for, for platforms to deal with like, this deluge of AI content, which is why we're building Pangram.

• 8:41Charlie Warzel?

  Well, and it also seems to, with the, with the platform part of it, right? It's one thing if it's spam, if it's just, you know, absolute garbage. It's very clear that this is like, no, no human can, is, is going to want to post this. But so much of what comes out, it's not just that it's believable human writing. It's also like, people are using the

• 9:02Charlie Warzel?

  Large language model generated text to say very normal things, right? They're using it in Twitter posts. They're using it in LinkedIn posts to announce their jobs. You know, they're using it to comment on whatever. I have seen many horror stories of people accidentally leaving in like the, you know, ChatGPT, would you like me to make this, you know, funnier or whatever your adjective for you on like Tinder profiles, right? Like, people are using this stuff in ways that isn't spam. They're using it in their actual, like, I want to communicate with human beings in life. And I feel like that is also just, is a very big shift to why the platforms are not going to be able to go after this, right? Because if you do want to crack down on it, you're going to be cracking down on, on people

• 9:46Charlie Warzel?

  like earnestly trying to communicate with other people.

• 9:50Speaker 02

  Personally, I think we're in the middle of a really big culture shift. So like, people really haven't settled on like, what is acceptable or not. I think there's a lot of people who really get the ick when they read AI-generated writing. Like, you can tell this sort of prose and the style, like, like this came from ChatGPT. And so it feels to me, like, when I'm reading something that came from ChatGPT, I'm like, I don't love that. It's also just like an authenticity thing. Like, if I'm, sure, like, I guess Tinder profile, like, if I'm on Tinder, I see someone with a, like, AI-generated, kind of like, long, prose-y profile, like, whatever. What about the messages? Do they have their, like, open claw, you know, swiping and like, you know, messaging to try and get a first date?

• 10:34Speaker 02

  And then they, like, show up for real? And then we talk and like, they have no idea, like, what the conversation was like? Or are they, like, you know, skimmed it ahead of time? Like, I think as soon as you let in, like, a little bit of AI-generated content, you open the door for, like, this, like, truly inauthentic behavior.

• 10:51Charlie Warzel?

  You all participated in a study or a survey, I think it was, where you guys note, in 2025, 35% of the newly published websites on the open internet were AI-generated or AI-assisted. And the next part is, is really interesting to me, too, which speaks to what you said, which is, you know, internet users are over overwhelmingly cynical about this. Seventy five percent of people polled felt like an AI dominated internet would be less accurate. Eighty three believe that AI will collapse unique writing style into monoculture. Is that ultimately what your fears are or your like your reason for for taking this up? This this feeling that like there's something that needs to be protected here?

• 11:34Speaker 02

  I think as humans, we've like built so many spaces on the internet around trust. And, you know, believing that there's a real person with real effort on the other side. And then AI kind of like turns this all on it on its head in a way where, yeah, we could just have somebody flood the internet with misinformation. There are just like hallucinations that L L M's repeatedly make. And if we bake it in to the fabric of the internet, then at some point, we're like not going to be able to work our way out of it anymore. And I do think like the collapse of writing styles is one of the biggest problems where it's just like, especially among like writers or people who like talk to chat G P T a lot.

• 12:14Speaker 02

  I I hear this fear from them as well. They're like, oh, you know, I don't want to like talk to Claude too much because I'm afraid that like, I'm going to start to adopt its writing style, which is crazy to me.

• 12:23Charlie Warzel?

  Yeah, it's it's it's very interesting to me as someone who is a who is a writer. I mean, I I find it very, I find it very fascinating because I understand why people want to automate the drudgery. I don't understand why at this point, why people want to automate the actual expression part, right? Like, I don't know why people want to automate the thinking, the the the part of this that is actually creative. I I especially when it when it comes to to text, I feel like there's so much fun that that you can have playing around with language. And I just just watching that creativity leech out to me feels

• 13:02Charlie Warzel?

  obviously extremely troubling as a as a writer myself, but but but also just, I don't know, I I feel like you get the most thinking done when you are actually trying to, you know, craft.

• 13:13Speaker 02

  When you struggle a little bit, you don't have L L M telling you exactly, hey, this is the next tokens, next word. What we're seeing in the the like writing industry is the. Drop shipping vacation of of writing. People see it as a get rich quick scheme. I I see like there's YouTube tutorials online of like, this is how you make a thousand dollars a month publishing AI generated books. And they like walk people through these like full AI workflows. I I don't think they're actually real. I don't think people are making tons of money off of this. But I think people are making money selling courses and telling other people to go pollute the space.

• 13:51Charlie Warzel?

  I want to talk about how Pangram works. I mean, I have gone on to your how Pangram detects AI generated content page. And there's a lot of stuff here. I'd love for you to like walk us all through it broadly. How does this detection stuff work? And then I guess we can get into the specifics of it.

• 14:07Speaker 02

  Sure, yeah, I I can maybe cover two things. One is like, how is the machine learning model trained? And then two is like, what do I think is it? What is it picking up on? So I think the first part, how is it trained? Um, is pretty interesting. I think the way Pangram works is pretty different than basically any other AI detector out there. What we're doing is we are taking millions of human written documents. And we're actually making calls out to the large language models, the AI models, and asking for a synthetic mirror. That's what we call it. So for example, I have a 500 word essay on Moby Dick written by a seventh grader. Then I'll ask Claude, hey, also write me a 500 word essay in the same style.

• 14:52Speaker 02

  And then so we have two documents. One is human written, one is AI. And then we're training this large machine learning model contrastively um, to learn the difference between the human document and the AI document.

• 15:04Charlie Warzel?

  Is that all happening in like in the moment? Like when I'm running the the thing? Or is this just how it was trained?

• 15:11Speaker 02

  No, so this is just, this happens during the training process. So we're training this model. It sees millions of documents. And then at the very end, it's a fully trained classifier. So any new document, it's not putting it into chat GPT, but it already knows the um stylistic tells. And so the way I kind of like to think about it. Is when you're writing a document, every word, every token, every sentence is a decision that you're making. So over the course of a document, you are basically following a single path on this very large, very wide decision tree. And the longer the document

• 15:52Speaker 02

  is, the wider the decision tree gets. And something interesting about large language models is they actually tend to make the same choices over and over. Experts call this mode collapse. But so what it looks like in terms of like documents, if I'm saying essays about Moby Dick, all the possible essays about Moby Dick that LLMs make, it's a much narrower decision tree than like all the possible essays about Moby Dick that any human would make. And so we're learning to like, understand what these very narrow decision trees are. And how then when we look at a new document, we're looking at how many of these decisions align.

• 16:32Charlie Warzel?

  It's a little bit like if when you and I are like, I'll prep for a podcast conversation, and I'll have this whole like, this is where it needs to go. And then you'll get another person in there, who's another human, and it gets really messy, right? That you go on these tributaries and these, you know, these things, it doesn't unfold the way. And and so I guess what like what you're saying is like, if I was behaving like a language model, I would just constantly be routing it towards a center, right? You wouldn't go on those different tangents and do that kind of thing. That's very.

• 17:01Speaker 02

  Exactly. Yeah, yeah, you you would kind of follow the same path. If we took you and ChatGPT and put you together in a conversation, you'd follow more or less the same path 100 times versus you and me. There's a lot more chaos. We'd have a broader range of conversations.

• 17:17Charlie Warzel?

  Okay, so then, once you have the model, you're you're also, and you have the mirrored prompts, you're you then retrain the model, right? Can you talk to me about that process of of retraining?

• 17:28Speaker 02

  Once we have the first model, it's usually pretty accurate. It can get to 99, 99.5%. But then we do something called active learning. So we take this model that we've trained, and we go. Have it search over a large corpus of data and find it goes and finds examples that it's uncertain or incorrect about. And then we take these examples and bring them back into our training set to do a second training run. And the reason we do this is these uncertain examples are more valuable in the training process because they're closer to the boundary between human and AI, or at least the perceived boundary that our model sees.

• 18:11Speaker 02

  And so when we bring these in, our model is able to learn more effectively and have a much lower false positive rate.

• 18:16Charlie Warzel?

  Let me see if I'm getting this right. You have essentially, you have the model and then the model will have by nature, some false positives, some false negatives, right?

• 18:28Speaker 02

  Yeah.

• 18:29Charlie Warzel?

  Like, and, and when you get those cases and you've identified, okay, this is a false positive. We have flagged this as AI when it really is human. You then run that through the machine and tell it basically like you made a mistake and then.

• 18:45Speaker 02

  Yes, and sort of like just to clarify, so a false positive is if we flag something that's human written as AI generated, then we got that wrong. We call it a false positive. An average AI detection model, like our first pass will typically have around 1% or half a percent false positive rate. But then we go, we collect these examples, we bring them back. And we actually like, don't continue training this model. Instead, we like restart from scratch on a new model. But the new model has these additional difficult documents in its training set. And then it's this new model that we retrain from scratch that is able to have a one in 10,000 false positive rate, which is much lower.

• 19:24Charlie Warzel?

  How often are you going through that process where you're starting from scratch and, and sort of like rebuilding the model to make it more effective?

• 19:32Speaker 02

  We retrain our model from scratch every three to six weeks. Part of the reason we have to do it so often is the field of AI is moving so fast. There's always Claude Opus 4.7, that's the new one, and then now ChatGPT 5.5. They're always a bit different. The decision trees slightly askew from ChatGPT 5.4. So we do have to collect more data and retrain.

• 19:54Charlie Warzel?

  Hmm. Okay. You said false positive rating of one in 10,000. This may come off to you like a dumb or obvious question, but the thing that just pops in my mind is like, how do you know? I'm sure there are instances where somebody is able to prove like, no, no, this, this was me and I did it in here and I can show you. What happens in that sense? Is that just like, is that logged away as like, okay, this is a really great example of a false positive that we can, you know, that we can talk about and we can use it to, you know, when we work to, to retrain the models. Like, what do you, what do you do when one of those things is brought to your attention?

• 20:31Speaker 02

  Yeah. So typically, uh, we don't train on these, but we keep them in an eval set. So when we're training a new model, we're able to look at our past false positives and see, hey, does our model, uh, improve on any of these? If so, that's a good sign. But then we also still need to calculate our like ultimate base rate because we don't want to have like some improvements here and then other regressions somewhere else. Uh, but yeah, we do collect them. We look at them and try to understand what, what is our model picking up here? Uh, and use that to make the model better.

• 21:03Charlie Warzel?

  I went through and I fed a bunch of my stuff into Pangram as I'm sure anyone who's using it, you know, does this to test and, you know, it was like, this is human stuff, you know? But if I had found like a, like a, like a flag of like, you know, something I'd worked on, I imagine I'd be frustrated or whatnot. Like, are you, do you have those frustrated conversations with people when there are the, the false positives? Like, do you interact in that sense with them?

• 21:27Speaker 02

  Absolutely.

• 21:28Charlie Warzel?

  What do you say to people in that, in that, uh, when that does happen?

• 21:32Speaker 02

  I mean, like one in 10,000 is not zero. You know, if this means that like, if a thousand people all put 10 documents through, then maybe one person is going to be like, hey, I put 10 documents through and one of them came up as, as AI. So, you know, I, I think your false positive rate is higher than, than you claim because I only put 10 documents through. But I don't think they realize that like on a like population level, there are going to be, um, these like anomalies. And so sometimes I just have to like talk people through it. Just like explain that you really have to look like large scale to. Fully understand it.

• 22:10Charlie Warzel?

  Hmm.

• 22:11Speaker 02

  And sometimes I think we get other misconceptions about what a false positive rate, what a false positive is. Like, sometimes we get somebody who writes and they try to sound as AI as possible. They say like, as a large language model, I cannot do whatever. And I'm like, well, like, of course, Pangram is going to say that's AI because you're trying as hard as possible to make it sound like it's AI. And so I, I have had these conversations as well.

• 22:37Charlie Warzel?

  This, this gets at something I wanted to ask, which was, have you guys found in some of these edge cases that there are just people out there who write like AI, who just are kind of have that, like, I have synthesized a lot of like relatively boring Wikipedia newspaper style articles, things, and I just like are able to output sort of the mean when they just sit down to write. Is there something different about, you know, that decision tree that we're talking about and that the software is really gonna just catch that most of the time?

• 23:10Speaker 02

  Surprisingly, no. I think most humans, people are just like not as mode collapsed as LLMs are. So

• 23:19Charlie Warzel?

  Thank God for that.

• 23:20Speaker 02

  Yeah, yeah. Thank, thank God. So when I go back and I look at the false positives, I might see a Yelp review from like 2018. And I'm like, wow, this reads exactly like how ChatGPT writes Yelp reviews. But I have to remind myself that, yeah, this is just a single isolated incident. And we've actually have gone in and like done profile level analyses, like if we have a false positive on one case, do we also have false positives on their other pieces of writing? And we found, no, not really. People are just like diverse enough that we don't really have this issue.

• 23:51Charlie Warzel?

  One thing that you said, speaking in another interview that you gave, was that from all of this, and this gets to the decision tree nature of how the, the models work, that you're learning how frontier models make their decisions, right? This process of, of diving in to, to detect has taught you a lot about that. Can you tell me a little bit more about what you have learned about some of these models or how the models are progressing? Like, you are in there in the actual machine learning Weeds of it all. And and I think to a lot of us, it's hard to know we yeah, this this just gets better. Or like, you know, Claude used to be thought of as like a little more literary than like ChatGPT was or whatnot. But I feel like you probably have some like concrete insights into how these models are evolving.

• 24:36Charlie Warzel?

  And I'm just curious what like what you've learned there.

• 24:38Speaker 02

  Yeah, I'm happy to tell you some things that I've learned. I think this isn't super well fleshed out because this has just been our experience over time. But my experience is that since 2023, large language models have gained much stronger preferences. And I've seen personally that there's a very high correlation between a model having strong preferences and a model just being highly capable in general. So I I think this is something interesting where like the early ChatGPT would kind of like say whatever you want it to say. Like people would call 4O very like sycophantic. And today's large language models, they will push back. They will say like, no, you're wrong.

• 25:22Speaker 02

  They have like pretty strong preferences. Like if you ask, for example, Claude to write an essay on like a topic that's choosing, it's often going to choose consciousness or it has a few favorite topics. And so I think a part of what we've learned is just like, how do we do prompting to still get like a very diverse data set? Not just like 10,000 essays about consciousness by Claude.

• 25:44Charlie Warzel?

  Why do you think that is so emergent? That that idea of these these models having having a kind of subject matter preference or something like that?

• 25:52Speaker 02

  I I do think having preferences is like an important part of intelligence. If you just think of like a human with no preferences, then they're kind of just like a zombie. They have no agency. They're not going to go out and do something like you must like have preferences on like this is the best way to do it. Something right to like be able to do things right more often. And I think like this has been baked in explicitly through a lot of this like reinforcement learning in terms of coding and making these models. Stronger coders and like more agentic in their harnesses like Claude Code and Codex. But I think this has the side effect of giving these models strong preferences in other fields like writing,

• 26:33Speaker 02

  where they may have not had them before. I think like the least preferred models are like GPT-2 and GPT-3, where they're truly trying to simulate the distribution of human text.

• 26:43Charlie Warzel?

  What else are you all looking for, or you know, I guess the, your, your models are looking for when they're evaluating a text for AI? Like what, what are some of the other tells that are in there?

• 26:55Speaker 02

  This is an active area of research. I think when you train a machine learning model, like a classed bear model especially, it's fairly black box. It will tell you what it thinks about the label and it'll tell you its confidence. But what it's not going to tell you necessarily is like, I think large language models would put a period here. And so because there's a semicolon instead, this is likely human writing. But I do think that is like a lot of what it picks up on is like all of these like micro decisions where for a human, it might be a 60-40 decision, but for an AI, it's really like a 90-10 decision. And so we look at just like the aggregate, like cumulative effect of a bunch of these micro decisions. I will say there's some like really major ones that I pick up as a human looking at AI texts.

• 27:39Speaker 02

  Like it's not just this, it's that formulation, which I think a lot of people have picked up on now. I think like older large language models really overuse like Delve or Testament, or like just had way too many M dashes. But I think the newer models, the tells are getting much more subtle. And so it's getting harder to like pull out really explicit, just like phrases or words that they overuse.

• 28:05Charlie Warzel?

  Something you said there that's really interesting is that you're working with models that at the same time have in the, in the reasoning component of or the justification component for the decision, a black box element. Does that, for lack of a better, does that freak you out a little bit?

• 28:25Speaker 02

  Not really. I think the way that people seem to be adopting technology like Pangram is very human in the loop. Or it's not just like a professor is going to go, oh, Pangram said your paper was AI, so I'm just going to fail you. But instead,

• 28:43Charlie Warzel?

  I mean, how do you know, I guess, right?

• 28:44Speaker 02

  They're starting with a Pangram result where it says, hey, this this looks like it's AI. And then they go in, they look at the citations or like, you know, is this, does this match the student's prior work? Especially in the like academic integrity councils. But also, I think like more generally, like I'm talking with publishers too. And it's kind of the same thing of like, how do you use Pangram as like a starting point for like opening up this conversation? About did you use AI? How did you use AI? We need to disclose this responsibly if you did. The best AI models don't really display their reasoning in a way that like generative AI does. Even if you like ask ChatGPT, why did you do that? It's just gonna give you text that sounds plausible.

• 29:26Speaker 02

  It's not gonna actually be looking into its weights and like truly understanding what it did.

• 29:31Charlie Warzel?

  Does it worry you that there are people out there who will adopt the software and not put a human in the loop? Because I can see in a lot of senses, like if I'm if I'm a professor, right? And I want to use this stuff and I want to like, the responsible way to do that is to put the human in the loop or whatnot, right? But I think one thing we see with with with, you know, and I've heard a lot of it anecdotally from from people. You know, in in higher education of like, first, it started out with all these professors and people being like really mad that the students were like shirking their duties of writing, you know, doing the the the actual writing work. And just submitting, you know, auto generated stuff. And then there was like, then I noticed some of those same people a couple, you know, months later just started

• 30:13Charlie Warzel?

  like grading using models and things like that and like that side of it. And so, you know, in a perfect world, yes, like everyone's using this to start a conversation, right? To to to flag something that they think is suspicious to go in and then do the the work. Does it worry you that there are some people that are going to be out there that are just gonna? Like, okay, I I see that it says, you know, pretty high confidence that that's AI like, you know, no questions asked. Like this student has failed, you know, on the on the paper or or that there is some kind of consequence. Do you worry about that happening?

• 30:46Speaker 02

  Um, I think I like typically have a lot of faith in people and in humanity and like people making sure that they like are using technology responsibly. Maybe that's being too optimistic. So I think there are good ways to use this tool automatically without human in the loop. If say, let's think about the problem of like X. They want to stop AI-generated replies, right?

• 31:13Charlie Warzel?

  On X, yeah.

• 31:14Speaker 02

  If, yes, on X. So if a reply, they check with Pangram, it looks AI-generated. They can maybe reduce the visibility of it. And then if they look at an account and of 10 replies, nine or more are coming back as AI-generated, then they might like shadow ban or ban the account because it looks like it's a reply bot. And so I think like, things like this make sense. And then of course, where the human in the loop comes in is they have like an appeals process. If we go back to like some of the other like higher stakes cases like, like publishers and academic integrity, we have been working really closely with the academic integrity groups and universities. And I think

• 31:57Speaker 02

  everybody I'm talking to is putting together really responsible policies.

• 32:01Charlie Warzel?

  Are you worried at all that so many people are using these tools, publishing, you know, AI-assisted writing or stuff that has, it's been sort of muddied by the use of large language models? Does that interfere with the data set? Is it going to be hard to tell after a while? When people are using all these tools, what is the human writing versus what is not, if the writing kind of starts to fuse?

• 32:29Speaker 02

  Part of like training a great machine learning model is having really clean labels. And today, I don't think we can really trust internet crawl data to have clean labels in a post-chat GPT world. There's just going to be some like, you know, latent AI level in like all of the texts that we find. But I am worried also about like, you know, we want to have modern texts. In our training examples, especially as like language changes even over the course of like three years, I think like slang and you know, there's modern slang that didn't exist before.

• 33:04Charlie Warzel?

  Now, man, yes.

• 33:05Speaker 02

  Exactly, exactly. So this is a very difficult problem. It's an active area of research, but I think like the very first step for us is collecting really clean, human written data from 2026. And so we have a couple initiatives for this. I think one part is just like looking at people who we know are trusted writers, somebody who's been blogging for a long time, their text hasn't come back as AI. And then also looking at younger generations, trying to get writing samples from students and kids who have grown up with ChatGPT and very likely have had their writing and thinking influenced by AI. And the first step here is just to measure whether this impacts our false positive rate at all.

• 33:50Speaker 02

  And I think the next step is to work on our training process.

• 33:53Charlie Warzel?

  Well, what's wild too is like, you know, it's not even at this point, people who are like, let me feed this into ChatGPT. It's like, I try to disable these things on my own Google Docs, but like it still wants to autocomplete every other sentence of mine, right? With just like, as I'm writing an article, it wants to do that and you have to, you know, get rid of that. But there's tons of people I'm sure who are like actually embarking on trying to like write themselves. And they're like, oh yeah, like I'll just, you know, I'll finish, I'll finish, let it finish that sentence for me, right? Like that, that looks like a good thing. You hit enter and you're, you're done. And then that's like technically AI assisted text. Even though it's, it's, you know.

• 34:31Speaker 02

  It's everywhere. It's pervasive. I'm not a big fan of that. I think it is like creating this writing monoculture. And that's why personally like I, I don't have ChatGPT help with my writing at all.

• 34:41Charlie Warzel?

  You all announced a Chrome browser extension that sort of allows you to passively turn on this detection for certain things like LinkedIn, X, Reddit.

• 34:54Speaker 02

  Substack.

• 34:55Charlie Warzel?

  Substack, Medium. Yeah. And, and I got access to this a little bit. So like let it run in the, in the background as you're scrolling, it shows the number of posts scanned, human, AI, AI assisted, and gives you sort of like a grade on your feed, right? Like is it 80% AI?

• 35:10Speaker 02

  What's your feed?

• 35:10Charlie Warzel?

  Honestly, okay. So here's the thing. I was.

• 35:15Charlie Warzel?

  Not that I was hoping that it was gonna be like a nightmare, but I'm at like eighty-five percent, I think I wanna say, not as much slop as I would have thought. And the stuff that is, is like not, you know, like, eh, not super interesting. Same on, on X. Like, I was hoping for, you know, just, oh, like, this person is insufferable. I can't wait to see. And it's like, no, it seems like, like, there's a lot more humanity in there. I, I did notice when I-

• 35:43Speaker 02

  That's good.

• 35:44Charlie Warzel?

  When I turned it to the For You page on X, the numbers, you know, the, the ratio changed significantly because it was like-

• 35:50Speaker 02

  Yeah.

• 35:51Charlie Warzel?

  I, I think I do tend to follow as like a journalist, like, other journalists or people who are, you know, who are kind of like-

• 35:57Speaker 02

  You're following real people.

• 35:59Charlie Warzel?

  I try. What was the impetus for this, this idea of, of let's, let's see it in real time? Is it, is it more of a, and I don't mean gimmick negatively, but is it more of like a proof of concept thing? Like, I want people to be able to sort of see the impact of this broadly across the ecosystem in real time versus like, there is huge utility to being able to like, call this out as I, as I see it?

• 36:28Speaker 02

  Mm-hmm. Well, I think to me right now, I'm looking at the trend lines. So, if, if you asked me a year or two ago, like, i- i- if I see an AI-generated Reddit post that's super viral, I'd be like, wow, wait, that, that's cool. That's, you know, you know, it's happening. And now I'm like, I'll, I'll roll my eyes because, you know, I only have to scroll for like a couple minutes to see it. So even if it's still like, you know, eighty-five, ninety percent of my feed is human, like, that ten to fifteen percent AI, that didn't use to be there. And I think if we look back in a year from now, if we're not actively curating our feeds and we're just letting the algorithm take over, then that ten to fifteen percent is gonna be forty to fifty percent. And so I think this is

• 37:11Speaker 02

  really a proactive tool before the internet gets bad and more full of slop.

• 37:17Charlie Warzel?

  So is the way that you would like someone to use it if they choose to sign up for this and do it? Would you like to see people be like calling, calling that out? Like if you see, you know, something in your feed that is not, you know, advertised as like, you know, and, and it is quote unquote slop. What is the behavior you're hoping to get from it? Or is it just simply raising awareness of like, okay, it's the wild west out there right now?

• 37:43Speaker 02

  It's funny, we actually started out before we did the Chrome extension. We just had a little Twitter bot where you could tag at Pangram Labs. Is this AI? It'll scan the parent posts and then respond with a verdict. They found it was a fun game. They were using it to call out these big slop accounts. They knew these slop accounts are posting slop, but they just didn't have any way to like, prove it or call it out before Pangram. And so I think that's one use case, but the one I'm more excited about is just the like, passive and manual curation of your own feed. If I see something that's AI generated, I'm going to mute or block the account. I'm going to choose not to engage. I'm not going to spend my time reading it. I'm not going to spend my time commenting or like arguing against their points.

• 38:24Speaker 02

  And I think overall, this is just going to give AI generated content a lower reach than something that was actually authored by a human. And that's my overall goal.

• 38:33Charlie Warzel?

  Are you, are you worried that this could become a weapon of sorts? Cause I, I, I'm of two minds on this completely because on one sense side of it, there's so many reasons why I don't want the slop in my feeds. I also like am a 20 year veteran of Twitter. And, and I know that like, you know, these tools can very easily be turned into like, screenshotting stuff. Do you worry about them getting like caught up in the culture war at all?

• 39:00Speaker 02

  Yeah, this might sound weird coming from me, but I'm really not a fan of call out culture. That's never been like a primary goal of mine. My hope is that it's overall a tool for good, right? That, that's why we gave this like, we give you this feed health score. Right? So, so I mean, maybe like some people are going to go, they see something AI generated, and then they're going to go interact with it more. Um, and then, you know, like screenshot and call them out, but that, that's going to kill their, their feed health score. If you want a high feed health score, you should just not engage, mute and move on.

• 39:32Charlie Warzel?

  You did however, engage or Pangram engaged with the Pope. You took shots at the Pope.

• 39:40Speaker 02

  You're talking about the Wired article recently?

• 39:42Charlie Warzel?

  Yeah, there was a Wired article of using this detection tool and seeing that like there is a number of tweets from the Pontifex account, which is the Pope's official, you know, it passes to all of the new, all the different Popes that we've had since we've had Popes using Twitter. And there were some like, you know, concerned statements, the Pope has a lot of deep concerning thoughts about AI and synthetic content and things like that and the lack of humanity. And at least one of them, right, according to your detection software, as Wired reported, was seemingly written assisted by AI in some capacity.

• 40:18Speaker 02

  I scrolled through the Pope account. I think among like 15 tweets, five of them came back as AI. So like my kind of understanding and verdict here is like the Pope's not writing his own tweets. He has a social media team and his team is probably AI-enabled. They're probably savvy Gen Z. They're using AI to some degree. They probably didn't think that they're going to get caught or, you know, that people weren't going to be able to notice. I don't think it should be like a huge criticism like, oh, my God, you know, the Pope is a huge hypocrite. I think it's more just like cultural commentary. It's really interesting. Like, like everyone's using AI now to some degree. And so I think this is also kind

• 41:00Speaker 02

  of a direction where we're really interested in terms of research is better understanding the degree of AI assistance. So right now, it's, it's not binary, but it's ternary. So we'll say something is human, AI, or like AI assisted. And I really just want to like expand and like fully understand the scale of AI assistance. Because I think this is where the world is heading. There's no shame in asking ChatGPT for a little bit of help, but there is shame in asking ChatGPT to generate a whole novel or post for you.

• 41:33Charlie Warzel?

  What would you like in that spectrum of AI assisted to be able to see? And how do you think you could get there?

• 41:39Speaker 02

  I would like if I'm going to use AI to like translate something or I'm going to use AI to like, you know, help me make it more formal or help me like fix my wording. We can catch that and we can also say like, we don't believe this is fully AI generated, but we believe this is AI assisted. Whereas I think the part that I really want to make sure we're catching and separating from the rest is the things that are just, you know, here is a research paper on archive to chat GPT. Like, please make a viral Twitter thread and explain like the the main points. And then it does that. And I want to be able to say like, that's AI generated.

• 42:17Charlie Warzel?

  Yeah.

• 42:17Speaker 02

  Like, I I don't think Pangram is quite there yet. I think a lot of times if I take a tweet or article and I ask chat GPT to rewrite it to make it better, then it it might come back as AI assisted, it might come back as AI generated. And I think today we don't have the greatest understanding of like even measuring how much chat GPT changed my initial text.

• 42:38Charlie Warzel?

  I'm really curious, given your involvement in this space, how did the last few years of generative AI and this agentic stuff like, how do you feel about it? Like, does this stuff excite you, this this this AI moment that we're in?

• 42:52Speaker 02

  I do think we're in a really exciting AI moment. I think AI capabilities have gone through the roof in the last couple of years. We've gone from AI being like, it's a silly chatbot. It kind of sounds like a, kind of sounds like me to like, this is actually doing real economic work. But I think it's also made me increasingly aware that there's a lot of harms that are going to come from AI. And it feels like nobody's really working to mitigate them. Like, we're doing some of it, which is we're helping mitigate the like proliferation of like AI slop and AI spam. And we're like giving people tools to help understand this and then also address it at a larger scale. But I think there's a lot more than that. All the AI CEOs are warning about job loss.

• 43:37Speaker 02

  And I don't think they're saying that because it's good for fundraising. I think they're saying it because they actually believe it. And there's also people on the other side that are talking about like the great like environmental harms. Yeah, like huge data center build outs and like the electricity and power cost of like running these huge language models at scale and these big like agentic loops.

• 44:00Charlie Warzel?

  Yeah. Well, I I ask because what's Interesting to me is that like you're someone who who plays in these fields, you can build these things, you interact with these things. This is your industry. You would have to be excited about the advances of artificial intelligence to do it. And yet, at the same time, what I think is so fascinating is that this is a, this is a like a reactionary, like a bulwark style force against this too, right? Like not, not against progress necessarily, but against, as you sort of said, like the misuse of it. And, and I think, I think like the speed of it to me is what it feels like it's a bulwark. Yeah, it's like

• 44:40Speaker 02

  Yeah, we're a little bit of a speed bump to this, like oncoming tsunami.

• 44:45Charlie Warzel?

  But I, I find this to be the, I find this to be the, the, the problem. Like so much of the conversation, the discourse gets, gets thrown between these two. Like, is it a, you know, stochastic parrot? Or is it, you know, mostly sentient? You know, and you get these like, it's, it's not a helpful discussion, you know, what is actually happening on the ground. But like, the thing to me that's like terrifying about this moment is the speed, right? You have these new models coming out. Like, I can step away for a long weekend, not look at the internet and come back and genuinely be like, I need to take five hours to figure out what's happened to this industry that I'm ostensibly supposed to be like monitoring at a given time. And, and there's something about that that is like, you know, some of that is PR, some of that's hype. Some of that is actual iteration. It's actually what's

• 45:30Charlie Warzel?

  happening. It's how people are using it, how the culture is changing around this. And I think the speed is what is really terrifying. Are you kind of terrified by the speed of all this?

• 45:42Speaker 02

  Yeah, yeah. Like, this is all happening so fast, and the world is changing so fast. And a lot of it still feels to me like we're not ready. If you ask me, like, what am I doing? Like, why am I here? It's like, I'm really trying to shape the world in the direction that I want to see it go. Like, I want to see people using AI to cure cancer and, you know, make senior care easier and make all of our lives better. And I also don't want to see AI polluting the internet and making public spaces, public internet forums becoming completely unusable, and, you know, allowing cheaters and, and people who want to just like Not do their job to to use AI to replace that. And so so sort of like there's these two sides.

• 46:26Speaker 02

  And I want to see the good side of AI flourish, and I want to help mitigate the harmful effects of AI as much as possible.

• 46:33Charlie Warzel?

  What do you think happens next to the Internet? I mean, what we see now, that's not even hypothetical, is this influx of slop, so much synthetic content, perhaps by some estimates tipping over into over 50% of the Internet being being synthetic. The idea that like, basic things like like Google are are less useful, all these different websites where user generated content is is supposed to be paramount, it's less useful, all the SEO hacking, all the all the stuff that like takes these things that I think all of us love, or at least found great utility in. All of this, like, the Internet itself just feels like it's really at risk right now. Where do you where do you see this all going?

• 47:15Speaker 02

  We're still very nascent, but I think we are in the birth of this like adversarial industry. So I think what's going to happen is the problem is going to get worse everywhere. So we're going to see more slop, more AI agents, people using AI and basically turning compute into either influence, political influence, narrative influence. And then I think eventually, we're going to see people utilizing more tools like Pangram to help stop and mitigate this. But I think maybe a good comparison is like, like, like early computers were, you know, viruses were starting to become a thing. And then eventually, there were like antiviruses and cybersecurity companies. And there was like this adversarial back and forth.

• 48:00Speaker 02

  I think we're at the very beginning. This is the ground zero of that today.

• 48:03Charlie Warzel?

  But just to to push a little bit on that, I think that that's probably true. But but just from like the, again, the speed and like the humanity centric thing, you know, like, there is just, there's just something to me that is worried about the speed with which the humanity is being leached out of this. Do you worry a little like, we could?

• 48:23Charlie Warzel?

  Lose this fight a little bit?

• 48:26Speaker 02

  Yeah, I think there's a very real risk that things could go wrong. You know, in my view, things going wrong is not the same as like most people in my industry. I think a lot of people, when they talk about things going wrong, they think of like AI doom, Claude becomes super powerful and then, you know, takes over the world and kills all of humanity. But the internet has for the last 20, 30 years really operated as like a high trust society and we are at risk of losing that.

• 48:55Charlie Warzel?

  I hate to end on like a downer note on that, but I also thank you for coming on here and also for, you know, trying to demystify some of this, right? I think it's a, I think it's a really fascinating exercise. So, so thank you for all the time.

• 49:10Speaker 02

  Of course. Yeah. Thanks so much for having me.

• 49:16Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait and AI noise. Think social media, that's where Inpress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with Inpress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.

• 49:50Speaker 01

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 50:20Charlie Warzel?

  That's it for us here. Thank you again to my guest, Max Sparrow. If you liked what you saw here, new episodes of Galaxy Brain drop every Friday. You can subscribe on The Atlantic's YouTube channel, or on Apple, or Spotify, or wherever it is that you get your podcasts. And if you want to support this work, and the work of my fellow colleagues, you can subscribe to the publication at theatlantic.com/listener. That's theatlantic.com/listener. Thanks so much, and I'll see you on the internet.

• 50:57Charlie Warzel?

  This episode of Galaxy Brain was produced by Renee Klar and engineered by Dave Grine. Our theme is by Rob Smersiak. Claudina Bade is the executive producer of Atlantic Audio, and Andrea Valdez is our managing editor.

• 51:10Speaker 00

  If you're trying to get news today, you'll find a lot of biased algorithms, rage bait, and AI noise. Thanks, social media. That's where InPress comes in. It's a social platform on top of factual journalism, not the other way around. Track your reading and earn points. It's like Strava for your brain. Plus, you can connect with other users in your city who share your genuine curiosity. Ditch the echo chamber for reality with InPress. Download it today for iOS or Android at inpress.app. That's I, N as in news nerd, press.