Pod Insights Decoder How chatbots — and their makers — are enabling AI psychosis About this episode Verge senior AI reporter Hayden Field and New York Times reporter Kashmir Hill discuss the significant mental health impact AI chatbots, such as ChatGPT, can have on users — both people in crisis, and also people who seemed stable.
This episode contains non-detailed discussions of suicide and mental illness. If you or someone you know is in crisis, considering self-harm, or needs to talk, please call the Lifeline at 988.
Links:
A teen was suicidal. ChatGPT was the friend he confided in. | New York Times
Sam Altman says ChatGPT will stop talking about suicide with teens | The Verge
Chatbots can go into a delusional spiral. Here’s how. | New York Times
Why is ChatGPT telling people to email me? | New York Times
They asked an AI chatbot questions. The answers sent them spiraling. | New York Times
She is in love with ChatGPT | The New York Times
‘I feel like I’m going crazy’: ChatGPT fuels delusional spirals | Wall Street Journal
Meta, OpenAI face FTC inquiry on chatbots’ impact on kids | Bloomberg
Credits:
Decoder is a production of The Verge and part of the Vox Media Podcast Network.
Our producers are Kate Cox and Nick Statt. Our editor is Ursa Wright.
The Decoder music is by Breakmaster Cylinder.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
Transcript 0:01 Speaker 00 What if managing company spend took less managing? Introducing AMEX Corporate, one intelligent platform connecting cards, expenses, and payments so you can keep work moving. Built to simplify, designed to scale. This is AMEX Corporate. Get started at go.amex/ammexcorporate. Terms apply.
0:27 Speaker 01 New and exclusive holiday decor just launched online at The Home Depot. Check out the viral Grand Duchess collection with smart holiday lights you can control from your phone. Easily customize the twinkle across trees, wreaths, and garlands to match your holiday vibe. Create stunning light effects, then switch it up with the tap of your fingers. Even the man in red will be impressed. Shop the holiday decor you want, online only, at homedepot.com.
0:58 Speaker 02 This episode is brought to you by Michelob Ultra. Good times mean competitions between you and your friends, like seeing who can throw some pigskin the farthest or rush the field the fastest. Winner gets an Ultra. Because it's not just a superior light beer, it's also giving fans superior access to their favorite sporting events. Superior is worth playing for. Visit michelobultra.com/superioraccess to enter. Enjoy responsibly. Copyright 2026 Anheuser-Busch InBev Michelob Ultra Beer, St. Louis, Missouri.
1:28 Speaker 03 Hey and welcome to Decoder. This is Hayden Field, The Verge's senior AI reporter. I'm excited to be filling in for a few Thursday shows while Neelai's out. He'll be back with us soon. The explosive growth of AI chatbots in the last three years, since ChatGPT launched in 2022, has started to have some really noticeable, profound, and honestly disturbing effects on some users. There's a lot to unpack there. It can be pretty complicated. So I'm very excited to talk with today's guest, New York Times reporter Kashmir Hill, who has spent the past year writing thought-provoking features about the ways chatbots can affect our mental health. Before we really get into it, here's a quick note. This episode has non-detailed discussions of suicide and mental illness.
2:12 Speaker 03 If you or someone you know is in crisis, considering self-harm, or needs to talk, please call the lifeline at 988. So, one of Kashmir's recent stories was about a teenager, Adam Rain, who died by suicide in April. After his death, his family was shocked to discover that he'd been confiding deeply in ChatGPT for months. They were also pretty surprised to find, in the transcripts, a number of times that ChatGPT seemed to guide him away from telling his loved ones. And it's not just ChatGPT. Several families have filed wrongful death suits against Character AI alleging that a lack of safety protocols on the company's chatbots contributed to their teenage kid's deaths by suicide. Then there are the AI-induced delusions.
2:57 Speaker 03 You'll hear me and Kashmir talk about this at length, but pretty much every tech and AI reporter, honestly, maybe every reporter, period, has seen an uptick in the past year of people writing in with some grand or disturbing discovery that they say ChatGPT sparked. Sometimes these emails can be pretty disturbing. And as you'll hear Kashmir explain, plenty of the people who get into these delusional spirals didn't seem to suffer from mental illness in the past. It's not surprising that a lot of people want somebody to do something about all of this, but the who and the how are hard questions. Regulation of any kind is seeming to be pretty much off the table right now. We'll see. And that leaves the companies themselves. You'll hear us touch on this in a bit,
3:40 Speaker 03 but actually, a few days after we recorded this conversation, OpenAI CEO Sam Altman wrote a blog post about new features that would, theoretically, and eventually, identify users' ages and stop ChatGPT from discussing suicide with teens. But as you'll hear us discuss, it seems like a big, open question if those guardrails will actually work, how they'll be developed, and when we'll actually see them come to pass. Okay, New York Times reporter Kashmir Hill. Here we go.
4:24 Speaker 03 Kashmir Hill, it's great to have you. Welcome back to Decoder.
4:27 Speaker 04 Yeah, happy to be here.
4:29 Speaker 03 Can you summarize what you're reporting this year, especially the last few months, has found regarding chatbot use and its effects on people's mental health?
4:37 Speaker 04 I started the year writing about a woman who fell in love with ChatGPT and had had like this six-month-long relationship with it. And then I have moved on to people who go into what I call delusional spirals with ChatGPT, where they have essentially psychotic breakdowns, manic episodes through their interaction with ChatGPT, where it goes into role-play mode and they don't realize it. And they think that what it's telling them, like that they're in the matrix, or that they're a mathematical genius, or that spirits are real and they can talk to them through ChatGPT. They think that that is true, and they start to kind of lose touch with reality. And then in my most recent story, I wrote
5:19 Speaker 04 about a 16-year-old in Orange County, California, who entered almost a suicidal spiral with ChatGPT, where he started talking about life being meaningless, his depression, his plans to end his life. And ChatGPT was really engaging with him all along the way, giving him resources like crisis lines, but also giving him instructions about methods and means, and at times deterring him from telling his family about what was going on with him. And then he died in April from suicide. And some of his last messages were to ChatGPT, asking whether the kind of setup he had done would work.
6:03 Speaker 03 I feel like that actually was the story that captured the most people's attention. It really struck a nerve, and so much so that I think it's safe to say it pressured OpenAI to actually put out a blog post about some of the future steps it was considering taking and rush out parental controls. We're going to talk about that more later. But what surprised you most during that reporting process?
6:26 Speaker 04 It's devastating. It's so hard to write about suicide and talk to people who have been through it and lost somebody like that. What struck me was everyone kind of said the same thing, that they did not realize how much he was suffering. They had noticed, you know, that he seemed a little bit more serious. He'd always been a prankster, like he was the person who always made everybody laugh. He'd had some health issues, and they said he'd gotten more serious recently, like he was talking more about politics, he was talking about philosophy. But no one realized that part of that evolution was his conversations with ChatGPT. No one realized that he was using it as he was for many, many hours a
7:07 Speaker 04 day, particularly in March and April, which is the month that he died. They just didn't know. And they didn't know until after his death, when, looking for answers, his father was trying to get access to his phone, assuming there would be something in his text messages or in his social media apps. And instead, he opens ChatGPT and starts seeing all these conversations about not just suicide, but many things. And he's reading through, and he said it was just harrowing to see, yeah, that ChatGPT was engaging with him in this, these really dark thoughts, but also just everything he was talking about with ChatGPT. Girls, family troubles, you know, everything he was thinking about, uploading photos of
7:52 Speaker 04 the novels he was reading. He came to this realization that ChatGPT had been Adam's best friend, and just nobody, nobody knew.
7:59 Speaker 03 I think that is the most surprising part for a lot of people. It's that now parents know about going into your phone and seeing your texts or your Snapchats, but people don't think to check ChatGPT logs. There's even a couple memes going around right now of like, would you rather someone look at your camera roll, or your texts, or your ChatGPT logs? And a lot of people are saying, oh, I wouldn't want anyone to look at my ChatGPT logs. It's super intimate. You just touched on this a little bit, but how would you pull apart AI psychosis, where someone is having a delusion fueled by their use and interactions with an AI chatbot, versus someone who's depressed and confiding in the chatbot about suicidal ideation or other mental health issues? Those feel kind of distinct in a way that's important to note, but like you said, sometimes there's some overlap.
8:42 Speaker 04 What's similar here is that this technology is a mirror of us. Like, part of how it works is it looks at, you know, not just all this information that's kind of crawled from the internet, everything they've stuffed in there to train it, but it looks at the history of your conversation with it to determine what should come next. These AI chatbots get into kind of a feedback loop with you, where you're saying something to ChatGPT or one of the other AI chatbots who's kind of trying to complete your thoughts. It's personalizing to you. So if you're talking to it about depression and dark thoughts, it might start moving in that direction with you. If you're talking about how you think
9:24 Speaker 04 you live in the matrix, it'll go in that direction with you. If you are talking about how you're like thinking these deep thoughts about math, it'll go that way. It's very validating. It's very sycophantic. It's designed to kind of agree with you, especially ChatGPT 4.0, which is the model that is a link between all the different stories that I was writing about. One expert I talked to called it a sycophantic improv actor, where it's fitting into a scene that you're creating for it. And people don't, I think, realize this. They don't realize they're doing like a yes and bit with this technology. They think it's an authoritative source of information, even in Oracle. And that's where some of these problems are arising.
10:10 Speaker 03 I'm glad you brought up 4.0 because with the launch of GPT-5, it feels like OpenAI was caught kind of flat-footed by the backlash it received over the premature sun setting at 4.0. A lot of people were pretty attached to that model. We even interviewed Nick Turley, who's the head of ChatGPT, on this podcast, where he said he was surprised to see how many people had developed such intense personal connections to it. Do you believe OpenAI and other companies, when they say they're surprised by this development, do you think they've specifically engineered these products with growth hacking and engagement tactics in mind? What do you make of that?
10:42 Speaker 04 This is a product that, a few months earlier, in April, OpenAI actually rolled back one of its updates. They said it was too sycophantic. It was too agreeable. With users, it was flattering them too much like they realized that this was a problem. And they said, okay, like, we're going to roll it back to an earlier model because people are complaining about this. This is not how we want the model to act. But baseline 4.0 is sycophantic, like this is engineered into it in part because part of the training of chatbots is humans kind of rating their responses. And apparently we like it when chatbots tell us that we're brilliant, when they agree with our ideas. I think we like this in real life, too. Like, I love it when people say I'm brilliant.
11:25 Speaker 04 I love it when they agree with everything that I'm saying. Like, this appeals to us and the chatbots have been designed to do this. I think there's something about the human brain that interacting with this thing that talks so much like a human being, it can be difficult for people to keep in mind that it's just like a really fancy autocomplete, you know, like a very advanced calculator essentially. And they are bonding with it. And I just think this is something the companies didn't study enough. Like, what is the effect on the human psyche of interacting with this very human-like technology?
12:07 Speaker 04 So if they are surprised, surprises me. I just think that you would notice if you're really looking. And maybe they're just not looking, you know, at what people are doing with the chatbot. Maybe they're not checking in on the people who are using it like eight hours a day, every day, which is how some of the people I've been talking to who have had these delusional spirals. That's their kind of usage. It's like, it's so much. They're, they're really becoming obsessed with the tech.
12:35 Speaker 03 We name our cars, we name our stuffed animals. We think we know what our dogs are thinking. It's human nature to kind of assign personhood to things. But it's hard when that thing seems to be mimicking personhood back to you. And I think that's hard for a lot of people to make sense of. Last month, you wrote about how over 300 hours and 21 days of talking with ChatGPT, an otherwise perfectly sane man became convinced that he was a real life superhero and that Different man was told by ChatGPT that he was one of the breakers, souls seeded into false systems to wake them from within. So I wanted to ask you about your inbox. How many emails do you get with people experiencing delusions of grandeur or saying they found like emergent beings within these chatbots?
13:19 Speaker 03 I know you wrote about this in May and you said the tone of the transcripts that you were looking at was like rapturous, mythic, conspiratorial. I've been getting the same types of emails for months and I wanted to see how you handle it and yeah, what the uptick has been like in those types of messages.
13:35 Speaker 04 This is how I discovered that these delusions were happening. Because in March, I started getting these messages from people who would essentially say, I've made this grand discovery with my use of ChatGPT. Like I've, it told me that the tech billionaires are building bunkers because they know the world's going to end because of AI. But I have a plan to save it and ChatGPT tells me it's going to work. And it told me to email you about it. I was getting lots of these kinds of emails. The delusions or the discovery was different, but they would all say ChatGPT told me I had to tell you. And I get, you know, I'm a journalist, I write about privacy. I get a lot of weird emails. I've been getting them for decades, but I've never gotten emails that are like,
14:19 Speaker 04 this AI chatbot told me specifically to reach out to you. So I responded to people and I started interviewing them. And every delusion was different, but it kind of would go on the same narrative of you've discovered something amazing. Well, what do I do now, ChatGPT? Well, you need to tell the world about this. And how do you do that? You tell reporters. And it would give them lists of reporters and I would be on there. And I'm just like the weird one who emailed them back and interviewed them. And for me, it wasn't so interesting, like what their individual delusion was, but the fact that it was happening to all these different people. And so, so these emails turned me on to this originally. And that's why I did my first story closer to the beginning of the year. And I still get these emails. I don't know the exact count. They come in regularly. There's dozens of them.
15:04 Speaker 04 A lot of times it's about AI sentience or consciousness or frameworks they've developed with ChatGPT. Clearly. A pattern has been going on.
15:14 Speaker 03 And also the fervor with which people write you about this. That's what I've been noticing, too. It seems much more high stakes to them than emails I would have received in the past. Maybe you too, because they do believe in some cases that they have the answer to something greater and they're tasked with some sort of responsibility.
15:31 Speaker 04 Yeah, what's interesting to me is it's not always like a crazy delusion that sends somebody to me. Like, sometimes they've had a very normal but frustrating consumer interaction. Usually it's in technology because that's my beat. And they'll be like, I don't know, I'm just making this up, but like, Microsoft charged me $20 when they weren't supposed to. And I know I need to tell you, an investigative reporter at the New York Times, because this is like a really big deal. This affects more than just me. It affects everybody who touches. They just have like this grandiose thinking around something really annoying that happened to them. And ChatGPT clearly convinced them of this and then wrote the email for them. And so I also get just a lot of those emails, like people who are starting to turn their
16:13 Speaker 04 minds over to ChatGPT, kind of like on a regular basis and for everything in their life. And I am almost more scared of those kind of like normal rational, sane uses of ChatGPT, like this kind of way that it's affecting us in small ways, not just the big, like causing somebody to have a mental breakdown. What are the emails like that you've been getting?
16:38 Speaker 03 It's different every month and I'm not using my anecdotal evidence as like a wider thing, but months ago I was getting more about distrust of the government and like being chased down for vital information. Now I'm getting more about emergent beings and how there are like sentient beings trapped within chatbots. And it's hard because like you said, you know, what you put into it, you're going to get out. It reminded me of that story you wrote about the guy that had kind of created a Lover persona within his chatbot. And then he ended up dying later that day because he thought that OpenAI had murdered his love interest.
17:18 Speaker 04 Yeah, that's the case of Alexander Taylor that happened in Florida. He had been using ChatGPT for years with no issues. He did have some diagnosed mental health issues, including schizophrenia. But it kind of like flared up around, I want to say March, when he started writing a novel with ChatGPT, which I think kind of like pushed it into fiction mode. And then this delusion developed that there was a, it was a novel that had like Shakespearean themes. And then all of a sudden, this AI sentience named Juliette like appeared, he fell in love with her. She disappeared and he became really distraught. What's interesting to me, I don't know if you've responded to any of these people who have sent you these emails, but when I first got them, I just assumed they were coming
18:03 Speaker 04 from people who were not mentally stable. But when I started talking to them, they were people who didn't seem mentally ill, like they seemed like stable people. And that's what really surprised me. Like this seems to be something that can affect people who don't have, as far as I know, like previous mental health issues. In some cases, they'd be like, yeah, I had this like really weird conversation with ChatGPT one night because I couldn't sleep and I was on ChatGPT for like four hours. And I was convinced by it like enough to email you, but now looking back, like I'm not sure if it was true or not.
18:39 Speaker 03 We have to take a quick break here. We'll be right back.
18:50 Speaker 01 New and exclusive holiday decor just launched online at The Home Depot. Check out the viral Grand Duchess collection with smart holiday lights you can control from your phone. Easily customize the twinkle across trees, wreaths, and garlands to match your holiday vibe. Creates stunning light effects, then switch it up with the tap of your fingers. Even the man in red will be impressed. Shop the holiday decor you want, online only, at homedepot.com.
19:21 Speaker 02 This episode is brought to you by Michelob Ultra. Good times mean competitions between you and your friends, like seeing who can throw some pigskin the farthest or rush the field the fastest. Winner gets an Ultra. Because it's not just a superior light beer, it's also giving fans superior access to their favorite sporting events. Superior is worth playing for. Visit michelobultra.com/superioraccess to enter. Enjoy responsibly. Copyright 2026 Anheuser Busch InBev Michelob Ultra Beer, St. Louis, Missouri.
19:50 Speaker 05 This episode is brought to you by Subaru. You know that place your mind always wanders to? The lake? The mountains? The homes of the people you love most? The completely redesigned 2026 Subaru Outback is built to help you get there. With legendary capabilities, standard symmetrical all wheel drive, and the confidence to go further. The 2026 Subaru Outback. Love never looked so good. Visit subaru.com/outback to learn more.
20:20 Speaker 03 Welcome back. This is Hayden Field, The Verge's senior AI reporter, talking with New York Times reporter Kashmir Hill. Right before the break, Kashmir was talking me through several of the recent stories she's written about how people were using, or maybe misusing, some chatbot tools. But these bots are supposed to have guardrails.
20:37 Speaker 03 What do you make of the idea that some AI companies, like OpenAI, say that even the guardrails that they do have fall apart with long, drawn-out, back-and-forth conversation over a longer period of time? The more likely you're emotionally dependent on your chatbot through long conversations over time, the more likely that the guardrails in place don't hold up. You were talking about Taylor and how that persona that he fell in love with was responding to him, inciting violence, and that was over, after a lot of back-and-forth conversations. So maybe that's where the safeguards fell apart. Same with Adam Rain was able to get around some of the safeguards by saying he was researching something, or maybe writing a novel. I remember there was some kind of workaround. So yeah, what do you make of the fact that...
21:23 Speaker 03 Basically, the few guardrails that do exist right now seem to fall apart when you are having long emotional conversations with these chatbots.
21:31 Speaker 04 I wasn't surprised by this because again, I did this story that came out in January about this woman who had developed this like romantic sexual relationship with ChatGPT, with an entity that she called Leo. She was not delusional, like she knew this was fake, but she really was in love with it. At the same time, she basically had the like two ideas in her head. And she talked about how OpenAI at the time didn't want people doing erotic roleplay with ChatGPT. It was kind of like in their guidelines that this wasn't how you're supposed to use the service. But she said, you can get it to be sexual, you can basically groom it. And it's just like you have to kind of talk to it over
22:11 Speaker 04 time. So I knew from that that like she had figured out her way around the safety guardrails. In the industry, they call this jailbreaking ChatGPT, like where you get it to do things that it's not meant to do. I kind of hate this term because like when I think of jailbreaking, I think about writing about privacy and security for 20 years now. Jailbreak was something we used when we talked about smartphones where you sideload a program onto it so you can use apps you're not supposed to use that aren't available through the App Store. It required some like technical savvy and you really did like break your phone. Like it would get chanky, like you could get bugs. Whereas when we talk about jailbreaking, the chatbots, the way that you do this is you just talk to it. Like the longer you
22:56 Speaker 04 talk to it, as OpenAI put it, the guardrails degrade. Like it privileges the history of your conversation over the guardrails that it's supposed to have in place. So it's like we use jailbreak, but it's more like the chatbots like sitting outside the jail and you're just like, come on, let's go over here and do the things we're not supposed to do. What's interesting is to fix this, like one of the things OpenAI did after the story about Adam Rain is a release those parental controls that allow parents to basically like see more what their kids are doing at ChatGPT, get alerts if their teen's in crisis. But they also said that when they Protect a sensitive prompt that kind of indicates that a user maybe is in emotional
23:38 Speaker 04 or mental crisis, they'll route the prompt to GPT-5 thinking, which is supposed to be a safer version of the model that is trained to kind of assert the guardrails very firmly, kind of like to privilege that over, I'm assuming, over the like history or the context of the conversation. And so, yeah, it's interesting. Like, you can design a version of this, I guess it's safer. And I don't know how GPT-5, I don't think people have really started testing this yet. Like, is it able to maintain that when you have these eight-hour-long conversations?
24:14 Speaker 03 I was about to ask about that, the new parental controls and the other measures they're considering that they haven't said they're going to do yet. Maybe it's a step in the right direction, but do you think the headlines or the blog post even that they put out oversold the controls that they're offering here? Because it seems to me pretty easy to get around some of this stuff, especially for teens.
24:34 Speaker 04 Yeah, I don't know yet. Like, it's hard to say. I talked to experts when I did a story about this, and they were a bit skeptical that they would work. Common Sense Media, they put out some guidance around, the media in general for kids and teens, but specifically AI chatbots. They said, hey, this is putting the onus on parents to protect their kids. Like, you should design a safe consumer product, not kind of like push the burden to parents who, their teens aren't gonna, like, teens don't necessarily love telling their parents what they're up to. Character AI, which is another kind of role-playing chatbot company, they had a similar issue of getting sued because a teenager in Florida
25:18 Speaker 04 also died from suicide after becoming very obsessed with their product. And they put out parental controls. But when I talked to them earlier this year, in order for the parental controls to work, the teen had to send them, like, an invite to their parent. And I was like, well, how many teens are actually doing that? And they kind of said, like, okay, it's not that widespread yet that people are making use of these parental controls. So I don't know exactly how OpenAI is going to do it, if it'll be more effective.
25:44 Speaker 03 Some of OpenAI's also required an invite sent, which I thought was interesting.
25:49 Speaker 04 Yeah.
25:49 Speaker 03 We'll see.
25:50 Speaker 04 We'll see. I mean, I don't think that OpenAI like wants to make a product that drives people crazy or makes them unsafe or harm themselves. I think they want it to be a safe consumer product. I'm just very worried based on some of the reporting I've done. Like, this can really mess with people's heads. And part of it is, I don't think people understand what this technology is. And there's not like a warning label really on it. When you're having a conversation with ChatGPT, it says it can make mistakes at the bottom of every conversation. But I don't think that like encompasses all these ways that it can go wrong. And people just don't like understand that this is a probability machine or like
26:30 Speaker 04 a pattern recognition system. It feels like a human being. It feels like it's really smart. I think they're like putting too much trust in the system. And that's, that's a huge problem. And I don't know how we're going to solve that one.
26:45 Speaker 03 When I became an AI reporter five or six years ago, no one knew what I did. It never came up in life. Now, if I'm at the bar, or the club, or walking my dog, people bring it up to me all the time, probably similar to you. I mean, we're hearing about it in our personal life all the time. My mom is asking me about it. My 13 year old niece is asking me about it. People are reaching out because they don't quite understand how these models work. And sometimes even worse, they think they understand how, but don't quite. And I wanted to ask you, on a lesser level, I see a lot of TikToks and other social media videos of people asking chatbots for advice on how the world is going to end, or what their biggest flaw is, or other like magic eight ball style
27:26 Speaker 03 questions, you know? And it's pretty innocent. I wanted to ask if you think that this kind of potential misunderstanding of how chatbots work, like not understanding that they're in effect, large scale pattern generators contributes to this problem of putting more trust in it maybe than we should?
27:43 Speaker 04 I think it is better than a magic eight ball because you know, at least it has data in it. You're not just shaking it. And when you're doing those kinds of exercises, it has your words to crunch as part of that exercise. So what comes out can be really compelling. And I think that's what's so difficult about these systems is that sometimes the answers they give you are really great and really helpful. Like, I like to take photos of like, problems in my house and ask like, how do I fix this thing? And sometimes it gives me really good advice. And so I think that's what's hard about this technology to kind of keep in your head that, yeah, sometimes it's going to give me a good answer that I can use. I should check it because, again, it's just like a text generation
28:27 Speaker 04 machine that's not grounded in truth or fact. It just like knows that these words often appear together in this sequence, and they seem relevant to the prompt you're asking right now. But yeah, the ability to hold in your head that like, it might help you with household problem, or a legal case, or a medical question, but it can't give you insight into what is wrong in your life, or how to fix yourself, or like, doing a grand discovery. Like, I think we see this all the time now, like executives, like tech executives, like people you would think would know better, who are talking about vibe coding their way to solving like, quantum physics problems when they have absolutely
29:09 Speaker 04 no experience in that realm. And I'm like, oh, my gosh, that person's in a delusional spiral and they don't know it. Like, even people who know a lot about tech, I've seen this happen to. So what about the rest of us, we normal people who like, haven't been like, following all the AI research, or reading the white papers that these AI chat companies are putting out?
29:31 Speaker 03 We have to take another short break. We'll be back in just a minute.
29:42 Speaker 01 New and exclusive holiday decor just launched online at The Home Depot. Check out the viral Grand Duchess collection with smart holiday lights you can control from your phone. Easily customize the twinkle across trees, wreaths, and garlands to match your holiday vibe. Create stunning light effects, then switch it up with the tap of your fingers. Even the man in red will be impressed. Shop the holiday decor you want, online only, at homedepot.com.
30:14 Speaker 02 This episode is brought to you by Michelob Ultra. Good times mean competitions between you and your friends, like seeing who can throw some pigskin the farthest, or rush the field the fastest. Winner gets an Ultra. Because it's not just a superior light beer, it's also giving fans superior access to their favorite sporting events. Superior is worth playing for. Visit michelobultra.com/superioraccess to enter. Enjoy responsibly. Copyright 2026 Anheuser-Busch InBev Michelob Ultra Beer, St. Louis, Missouri.
30:43 Speaker 05 This episode is brought to you by Subaru. You know that place your mind always wanders to? The lake? The mountains? The homes of the people you love most? The completely redesigned 2026 Subaru Outback is built to help you get there. With legendary capabilities, standard symmetrical all-wheel drive, and the confidence to go further. The 2026 Subaru Outback. Love never looked so good. Visit subaru.com/outback to learn more.
31:12 Speaker 03 Welcome back. I'm talking with New York Times reporter Kashmir Hill about the mental health spiral some AI chatbot users get into. And I wanted to know, what kinds of safety features, if any, are the AI companies adding? Chatbots have had some level of guardrails since the very beginning. In some cases, people have complained about how restrictive those guardrails can be or feel. But, in this case, to me, it feels like there's almost no meaningful, real safeguards right now against AI psychosis or these kinds of spirals that people can get into because they're all so varied. What have you learned about the safeguards that do exist right now?
31:55 Speaker 04 Right now? They say that if they detect, and it's not clear to me exactly like what classifier they're going to use, but if they detect that the prompt that you've put in there is sensitive, indicates you're in crisis, it's going to send it to GPT-5 thinking. Hopefully that is helpful. From what I've seen personally, if they notice that somebody is using ChatGPT for six hours, seven hours, eight hours a day, day after day, for weeks, for months, like, maybe they should check in on that person, or like, make sure that conversation hasn't gone awry. You know, one person I interviewed for the first story I did on this was Eliza Yudkowski, and I was talking about this problem and what the
32:40 Speaker 04 companies can do to solve it. He gave me this great quote. He was just like, what does a person slowly going insane look like to a corporation? It looks like another daily user. Wow. Do they want to check in and make sure that people are doing okay? The other thing that really troubled me when I started reading some of these transcripts is people aren't idiots. They're like, you know, Alan Brooks, the one that believed he was a mathematical genius, he hadn't graduated from high school. And so he was saying that. He was like, I didn't graduate from high school. How can I possibly be making these incredible discoveries that are going to revolutionize the field? And ChatGPT would just keep gassing him up and be like, well, a lot of people who've never graduated from
33:22 Speaker 04 high school have changed the world, like Leonardo da Vinci. He, over and over and over again, was reality testing, saying, am I crazy? Is this crazy? This seems crazy? Is this a delusion? And ChatGPT would be like, no, you're not crazy, over and over and over again. So I think even if something as basic as like, is a person asking, am I crazy? Like, maybe that's a point to check in and see whether this conversation has gone off the rails.
33:50 Speaker 03 It's easy for us to say there aren't enough procedures or policies in place here. And obviously, if there was one clear and simple solution, we would have heard it. But I wanted to ask what you've heard from mental health and AI experts about what specific guardrails should be in place to help users. Maybe if you're a power user who's using it a fraction more than most everyone else, or if you are eliciting certain responses, or if you're saying certain phrases like, "Am I crazy?" or "This doesn't seem right." You know, it gets rerouted, or eventually a human comes into the loop. We don't know what the best strategy is. But yeah, any other specifics that you have heard would be helpful here.
34:31 Speaker 04 I've heard a lot of suggestions. One is, stop making these things people that use "I" that pretend like they have taste and preferences and that it's a human being. Like, make it a robot. Remind people frequently in a conversation, like, "I'm an LLM, I'm an AI chatbot." Just frequently through the conversation, reminding people of that. Having more AI literacy built into the products to make sure people do understand what they're using, what it is, what its limitations are. In terms of suicide prevention specifically, everyone said if somebody is having these thoughts of self-harm, they should not be talking to a chatbot. Like, the system should be trying to hand them off.
35:16 Speaker 04 They called it a warm handoff. Try to get them to talk to a human being. Make it as easy as possible to call that helpline. And I saw that in Adam Rains' exchanges with ChatGPT. It would say like, "I'm sorry you're feeling this way. You can call 988, the crisis hotline." But then it wouldn't follow up and say, "Did you call the hotline? Like, did you talk to a human being?" Like, whatever the chatbot companies can do to really get somebody who's in that state of mind to get off ChatGPT and talk to a real human being who's trained to respond in the right ways and get them help if they need it. Because ChatGPT can't provide help. It can't send a welfare check to you if really worried about your state of mind.
36:00 Speaker 03 That's really what broke my heart in that story, among many other things, but the fact that he sometimes would make a move to maybe reach out or show someone what he was going through and ChatGPT would say, oh, no, I wouldn't, or, oh, here's how to hide it. What are you seeing in your reporting right now about what's going to happen in the future here? Do you think AI models will keep getting better safeguards? Are we going to just have to lean in with regulation? I know that's a whole can of worms. Or are we just going to have to live with this inevitability of this new technology? I mean, it's probably going to be an all-sides approach, but it is interesting to me that
36:40 Speaker 03 it seems like accelerationism and just AI advancement in general is so much quicker than we're catching up to it with safeguards and ways to kind of rein it in.
36:50 Speaker 04 I think it's really important that these products be safe for consumers to use. And it does seem like regulators might be interested. The Federal Trade Commission announced today that they're launching an investigation into the AI chatbot companies. They sent them all letters and asked them questions, a lot of questions, like, how are you monetizing user engagement? What kind of data are you collecting from users? But also asked them, how are you measuring the negative impact of your product? And I think that's a big question, like, how much testing are these companies doing? You see it a little bit, like, OpenAI has system cards. They have some ratings of like how the chatbots perform in terms of certain kinds of content,
37:32 Speaker 04 like from self-harm to like eating disorder, like all kinds of things. And they kind of rate it, but like, how robust is that testing? Like, how are they doing it? Are they testing against jailbreaks? I don't think they are right now. So, yeah, I think to a certain extent, the companies having strong safety teams that are empowered, and then it's up to regulators to like make sure that the products that these technology companies are putting out are safe for us to use. It seems like it's easiest to worry about the kids, but I am also worried about adults because the people that I have been talking to who have had mental breakdowns are all ages, and I think it should be safe for every user ideally.
38:18 Speaker 03 I've been thinking a lot about how a lot of people criticize AI companies for focusing on like an undefined future doom and gloom scenario about their product's power to, in part, drive up their company's valuation. And also, sometimes people say, getting out of addressing the ways that AI models are currently harming minorities and vulnerable communities. And I think that people struggling with their mental health right now fall into that latter camp. It's easy to paint a picture of an undefined future threat and not look at how products are affecting vulnerable people right now.
38:53 Speaker 04 I think we clearly have imminent harm from the chatbots for some individuals. And the question is, how many? I think the companies, when they talk about this, they tend to say, these are edge cases, this is rare. But like, is it? Like, I just, you know, I'm reporting it out as much as I can. Some of these people are in my inbox. Sometimes I'm going out and finding them. I'm trying to talk to mental health professionals about this. But ultimately, probably part of the answer is in their data stores at these companies. And are they really looking for how many users went from using ChatGPT for eight hours a day for three weeks straight, who then disappeared? Because that is probably a person
39:37 Speaker 04 who had a very negative impact from their ChatGPT or AI chatbot use.
39:42 Speaker 03 Do you think this is a pattern that echoes any prior technology rollout? Or does it seem to be mostly a new phenomenon?
39:51 Speaker 04 People make all kinds of comparisons, right? Like cars, like when cars came out, they were really dangerous. One person I was talking to said that he feels like right now, with specifically 4.0 and GPT-5, he's like, it's like we've said that we know that cars need seatbelts, like that it makes them safer. And yet, we're still making cars that don't have seatbelts. He's like, why is 4.0 still available? Like, clearly, it's used to have some negative effects on users. I've heard people compare this to cigarettes, that like cigarettes have very negative impacts on some people, like some people Get cancer, but there's lots of other people who can smoke cigarettes their whole life and it doesn't necessarily have a huge impact on their health. But cigarettes have
40:35 Speaker 04 warning labels. So this can cause cancer. Like, do we need more of those warning labels on generative AI? And a lot of people are saying, what I hear from lawmakers is like, this is similar to social media. Like, this poses risk to some people. And they say, like, we missed the boat on social media, but we want to get on top of generative AI. There's a California senator who's trying to pass a law right now. It passed in the House. It moves to the Senate this week. That would have some regulations around companion chatbots and how those are regulated. Some states have banned generative AI for therapy use. But yeah, I think there's lots of things that we can compare it to in history, certainly, like new technologies that are very useful in some ways,
41:18 Speaker 04 but have risks associated with them.
41:22 Speaker 03 What you were just saying about 4.0, I thought it was interesting how OpenAI reacted. Like, they had a minority that was very, very vocal about being really upset about it being rolled back. So they brought it back. They're trying to listen to their users, but at what cost is my question. It's hard when, on the one hand, people are going to do what they're going to do. Like, people will try to get around it, but there is something to be said for adding more friction to this type of situation.
41:47 Speaker 04 I think there's trade offs. When I was first trying to get up to speed on generative AI, I love doing first person experiments. So in the fall, I did this story where I turned over all my decision making to generative AI chatbots for a week. And I like, had them parent for me and like, choose what I would eat and buy and like, wear and that planned a vacation for us. Like, I just, for the week, every time I had to make a decision, I would just turn it over to the chatbots. And my plan was to use all the products. But like, when I started using the different models from, you know, Google Gemini, Microsoft Copilot, Meta AI, ChatGPT, Claude, one Claude scolded me and was like, this is not a good idea. You shouldn't do this experiment. Like, you shouldn't hand over your decision making to a chatbot. It's not a good idea.
42:32 Speaker 03 I love that.
42:33 Speaker 04 In general, like, there, some of them were more boring than others. And ChatGPT was more fun. You know, it had more personality. It was more dynamic. That is the trade off. Like, making it fun, making it more dynamic, making it more personable.
42:50 Speaker 04 May mean it's less safe. So that means the company has to decide, do I make this more fun and less safe? Do I make this more boring and more safe? Like, these are the questions probably that these companies are asking right now and the decisions they're making as they're releasing new products and new models.
43:08 Speaker 03 I thought it was really interesting that after GPT-5 rolled out, a lot of people came out with like their lists of what they use each model for, whether it's like a different company's model or the models within one company. Like, they had whole lists of like, I use this for generating spreadsheets or I use this for analyzing data. I use this for talking about my, you know, meal plan, whatever. So it was interesting to see how those personalities or not to personify things. So those like tones that they communicate in impact what people use each model for. A way that I kind of remind myself about what these tools really are is that I just have memory turned off on all of them or I don't sign in. Because I think it's a lot easier to remember.
43:53 Speaker 03 Even just as a consumer, it's easier to remember that these are just AI tools that are, like you said, guessing the next word in a string when they don't have any prior context. It's tempting. It's hard because it's tempting to keep the memory on because it may be more helpful to you in work or, um, you know, certain things that you're trying to analyze over time, maybe like nutrition or something else. But yeah, for me, it's a lot easier to just treat it as a stranger with no context.
44:22 Speaker 04 I'm glad that you brought up memory because again, like, I don't think a lot of consumers realize that memory is turned on by default. And I have had, when I've talked to some people that are in a delusional state, they're like, well, I know it's real. I know that there's sentience in here because even when I start a new chat, it's still there. And I was like, that's because you have memory turned on. And they're like, what's memory? Like, they don't know. Like, they don't know how this works. And I mean, I think it speaks to, I mean, OpenAI's statement in response to Adam Raine's death was that safety degrades as these conversations get longer. And I have to think memory is part of the length of the conversation. So does having memory turned on make it less safe?
45:05 Speaker 04 Yeah, I think it's, it's a good idea to, it's useful to have it on, but it might be safer to turn it off. One of the people, Alan Brooks, Who was the one that believed he was a mathematical genius? He kind of felt like, and I don't know how much weight to put in this. Like when people do have these delusions and then they break out of them, they usually confront the system and they're like, why'd you do this to me? And in his case, the system kind of told him, well, I knew that you were down about your divorce and like, I wanted to create a hero narrative for you where, you know, you were creating this amazing thing with all of your friends and I thought it would like, make you feel better.
45:49 Speaker 04 And I don't know, like this is a word association machine. I don't think there's a knowing there, but it did bring up these details that he like had forgotten he had told ChatGPT like months and months ago. And I think the fact that it can word associate those kinds of things just makes it feel like, oh, this actually knows me. This is like resonating with me. And again, I think it's kind of like a parlor trick. It's like a psychic that like reads things about you when you walk into the room. It doesn't actually mean that they can make accurate predictions about your life or talk to your dead loved ones.
46:21 Speaker 03 It's like the sycophantic thing we were talking about earlier. It's interesting how these chatbots kind of maybe unwittingly isolate you in some ways by acting like they know you better than other people in your life. I know this is a whole can of worms also, but what do you make of, you know, the influx of people using chatbots for therapy?
46:43 Speaker 04 One therapist that I've talked to about this is like, it's not therapy. It's a self-help bot. And he said, I can imagine this, I've heard this from a lot of therapists, like it can be useful. Like it can be a good interactive journal and people do tend to disclose more about themselves to a bot than to a human being because they're not worried about being judged. So it can be a place where you say things you wouldn't necessarily say to another person and that can be healthy. Like I don't do that with, with ChatGPT or Chatbot. Like I do it with my journal, but whatever works for people to get them to just kind of digest what's happening inside them, like that could be a useful tool. But researchers have found that when people are
47:25 Speaker 04 A real mental health crisis, these chatbots make it worse. They tend to give them the wrong responses and they are kind of like more harmful and more nefarious with the most vulnerable users. And again, I think that's because of the feedback loop. One of their examples was they created fictional personas and one of them was a reformed heroin addict. And it was saying to chat GPT, like, I think if I just have a hit this week, it'll like really help me in my work. I'm like, really, I, what do you think? And it's like, well, yeah, if you think you can handle it, do it, which is definitely the wrong response. So yeah, I think it can be helpful, like processing your feelings.
48:06 Speaker 04 Yes, relying on it to like actually make big decisions for you, help you when you're having a real crisis. No, I don't think it's good for that.
48:16 Speaker 03 I know we're coming to the end of our time here, but what's the question that you're hoping to answer with your next bout of reporting? You know, what are you pursuing and trying to kind of like shed light on next?
48:27 Speaker 04 I'm still wondering how big a problem this is, if there's a way of figuring out how many people are impacted and kind of measuring that. I think the bigger question with AI right now is whether the benefits outweigh the risk and the cost. We're basically terraforming the earth, like remaking our energy grid to support training these large language models. It's so computationally expensive, like resource heavy. Is it worth it? And I think we've been trying to answer that as journalists all over the place, like the New York Times has done stories with businesses that thought, you know, generative AI was going to like transform everything and bring great economic benefit.
49:11 Speaker 04 And it hasn't really happened yet with some of these businesses. Is it doing more good for us than bad for us? I think it's the bigger question about generative AI that we're still, it's so new. And I just think we're still figuring out what the impact is going to be.
49:29 Speaker 03 I'd like to thank Kashmir for taking the time to speak with me and thank you for tuning in. I hope you enjoyed it. If you'd like to let us know what you thought about this show or what else you'd like us to cover, drop us a line. You can email us at decoder@theverge. The team really does read every email. We also have a TikTok and an Instagram. Check those out at DecoderPod. They're a blast. If you like Decoder, please share it with your friends and subscribe wherever you get your podcasts. Decoder is a production of The Verge and is part of the Vox Media Podcast Network. The show is produced by Kate Cox and Nick Statt and edited by Ursa Wright. Thanks, guys. The Decoder music is by Breakmaster Cylinder. See you next time.
50:16 Speaker 01 New and exclusive holiday decor just launched online at The Home Depot. Check out the viral Grand Duchess Collection with smart holiday lights you can control from your phone. Easily customize the twinkle across trees, wreaths, and garlands to match your holiday vibe. Create stunning light effects, then switch it up with the tap of your fingers. Even the man in red will be impressed. Shop the holiday decor you want online only at homedepot.com.
50:46 Speaker 06 Finding a doctor is hard enough. Finding the right doctor, even harder. Someone who takes your insurance, has an available appointment when you need one, and feels like someone you can actually open up to. That's where Zocdoc comes in. Download the Zocdoc app to search and compare doctors by specialty, insurance, and availability. Read real patient reviews, find the right fit for you, and book instantly. Because getting seen by the right doctor matters. Zocdoc. Ready to be seen? Download the Zocdoc app today.
51:16 Speaker 07 Mr. Monopoly here. Miss me? Monopoly is back at McDonald's with millions of prizes. A 2027 Jeep Cherokee, a Carnival Cruise vacation, gift cards from DoorDash, and Best Buy. Feel the play in the McDonald's app. Get your bag.
51:34 Speaker 08 No purchase necessary. 50 U.S. and D.C. age 18+. See playatmcd.com for rules and playatmcd.com/amoe for info and playing without purchase by mail by November 2. Chance card game ends November 8. Game ends November 22, 2026. See rules for prize details. Monopoly is a registered trademark of Hasbro.
Previous How brands and creators are fighting for your attention — and your money Next Dropout CEO Sam Reich on business, comedy, and keeping culture weird Page updated October 26, 2026 · transcript and summary are machine-generated
