Inside the Hunt for the Discord Leaker + Twitter Chaos Updates

Aric Toler untangles the web of teens, gamers and memes at the heart of the latest intelligence scandal. Then, an update on Twitter — where things have gone from bad to worse. Plus: How A. I. is bringing us closer to “Westworld.”

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 0:34Casey Newton?

  I'll tell you something that went on for me this week. For a long time, I've used the Google Docs product. Have you used this product?

• 0:40Kevin Roose?

  The Google Docs?

• 0:41Casey Newton?

  Yeah. You've seen this one? Okay.

• 0:43Kevin Roose?

  So. Obscure, buried, you know.

• 0:45Casey Newton?

  We like to highlight up and coming apps. So I'm in the Google Docs product yesterday, and I'm working with some folks for a project I'm excited about. And these people are Google Docs masters. Okay? I didn't even know that Google Docs could be mastered because it looks so basic. But they keep saying things like, "We'll just go down to where I am," or "go up to where I am." And I was like, "Bro, I don't know where you are on the doc, okay? Like, I don't have some sort of like doc ESP."

• 1:07Kevin Roose?

  You gotta click the circle.

• 1:08Casey Newton?

  You can click the circle. Did you knew you could click the circle?

• 1:12Kevin Roose?

  Yes.

• 1:12Casey Newton?

  Okay, I have to tell people this. If you are in a Google Doc with other people and you're editing collaboratively, you can just click the little circle at the top of the page with their little initial or their picture, and it will take you to wherever they are in the document.

• 1:26Kevin Roose?

  Wow, I can't believe you're just learning about this in the year 2023.

• 1:28Casey Newton?

  How long would you say you've known this?

• 1:30Kevin Roose?

  At least five years.

• 1:31Casey Newton?

  There's no way.

• 1:32Kevin Roose?

  Yes.

• 1:32Casey Newton?

  Five years? Why has this never come up before? This is insane to me. You know, you know why it is? Because it's in the New York Times. There's like five bylines in every story and six editors. And so you actually have to know, like, I work for normal media organizations where like, you're lucky if you could get an editor to like, come into the newsroom to help you out. And so I, maybe there just haven't been that many people in my Google Docs.

• 1:54Kevin Roose?

  Wow. Casey, I'm gonna blow your mind.

• 1:56Casey Newton?

  Okay.

• 1:57Kevin Roose?

  Pro tip? You can print a document on Google Docs.

• 1:59Casey Newton?

  Now we're just saying things that no millennial would ever do. But I think it's great that Google has added these features. You know, generally speaking, you don't want other people in your Google Doc, and I'll tell you why, because they're gonna have ID. Ideas and those ideas are going to make more work for you. But if you find yourself in this sort of nightmare situation of working with other people, you can click the circle.

• 2:17Kevin Roose?

  Wow, I love the idea of a podcast where every week we just teach you how to use one extremely basic office product.

• 2:23Casey Newton?

  You're laughing, but I think that that could actually be quite useful to our readers.

• 2:27Kevin Roose?

  Next week on Hard Fork, Casey learns Excel.

• 2:30Casey Newton?

  We're going to need more than one episode for that.

• 2:38Kevin Roose?

  All right, I'm Kevin Roos, tech columnist at the New York Times.

• 2:40Casey Newton?

  I'm Casey Newton from Platformer.

• 2:42Kevin Roose?

  And you're listening to Hard Fork.

• 2:43Casey Newton?

  This week, how a private Discord server ended up at the center of a national security crisis. Then more chaos and clownery at Twitter. And finally, what happened when researchers gave AI memories? I'm guessing it was nothing good. We'll find out.

• 3:14Casey Newton?

  All right, Kevin, can we talk about national security for a minute?

• 3:16Kevin Roose?

  Absolutely.

• 3:17Casey Newton?

  All right, is this a secure line?

• 3:20Kevin Roose?

  Yes, I am in the sensitive room inside my house where I handle classified information. So we are good to talk about this stuff.

• 3:28Casey Newton?

  Let me tell you, every room in my house is a sensitive room, depending on the day. But what I want to talk about is the national security aspects of it all.

• 3:34Kevin Roose?

  Yeah, so this is a big military intelligence story that is currently like making front page headlines in the New York Times and other publications. So on Thursday, federal investigators arrested a man named Jack Tashara, who's a 21 year old Air National Guardsman. And they believe that he is linked to this leak of classified military intelligence documents. These documents have been circulating on the internet for a while now, and it wasn't clear who had leaked them or why. But these documents really did cause a stir. They've compromised some of our relationships with other countries. They've exposed weaknesses in the Ukrainian military. This is a big deal, and it seems to have stemmed from a

• 4:20Kevin Roose?

  Very strange, very online source. So before we talk about the online piece of this, let's just talk about what these documents were and why it's important that they're out there.

• 4:31Casey Newton?

  Yeah, so these documents were, one, just very politically sensitive. Two, this was also supposed to be top secret. And there is stuff in there that the United States was trying to hide, both from its adversaries and allies alike. It included things like details about secret American and NATO plans for building up the Ukrainian military, for example. And then, third, these are just timely documents, right? Sometimes when we hear about these leaks, they're years old. Somebody finds them in a file cabinet and blows a whistle. The information in these leaked documents is, in some cases, weeks old. And it seems like stuff has been leaking for even longer than that. So that's just a really big deal.

• 5:10Kevin Roose?

  Yeah, and I would say like, this leak of classified information has really shaken the national security apparatus. So, you know, military intelligence officials have been looking into it furiously, trying to figure out where it came from and who did it. The State Department and other agencies are trying to handle some of the geopolitical fallout. And Merrick Garland, the United States Attorney General, made a statement on Thursday about the case.

• 5:35Casey Newton?

  That's right. And what's really interesting to us is how these documents have been circulated. These documents were found on Discord servers, and not just any Discord servers, but video game Discord servers. And at least at first, it seemed like people were posting these documents not to blow the whistle, but to win arguments online. And so, Kevin, when you and I heard that, we said, well, we need to get to the bottom of this.

• 5:59Kevin Roose?

  Totally. And before we get into the details of these documents and why they're being shared inside these peculiar Discord servers, let's just explain for people who maybe don't know what Discord is.

• 6:10Casey Newton?

  Yeah, so Discord is a communications app. It's free. And while it started as something that gamers primarily used, now, basically, everyone's using it. And there are Discords devoted to posting memes. There's Discords devoted to making generative AI. I have a Discord for my newsletter, where people who read my newsletter can just kind of hop in and talk about today's issue and other things that are on their mind. So it's just kind of a very fast-growing, very popular, next-generation social network, I would say, that is used by all sorts of folks.

• 6:41Kevin Roose?

  Right, so I would compare it to something like Slack or Microsoft Teams, but it's not work focused. It started, as you said, as a gaming thing. And there are different kinds of Discord servers, as they're called, right? So anyone can start a Discord server. And some Discord servers just have a handful of people in them. But I've been in ones where there are also millions of people. So it can range from very small group chats to very large message board type things.

• 7:07Casey Newton?

  Yeah, and an interesting thing about Discord is that you typically belong to both public and private Discords at once. So you might have a very small, essentially a group chat with your friends. And then you also belong to a bunch of larger servers. And this winds up playing a key role in the plot here. Because what it seems like happened was that some materials that were designed to be shared only with a group chat, pretty quickly migrated to some much bigger servers.

• 7:34Kevin Roose?

  Yeah, so we are still piecing together exactly what happened with this leak of classified information. But what we do know is that the leak appears to have started on a private Discord server that was populated by a group of friends who played video games together. The server at one point was called Thug Shaker Central, which is, do you get that reference? I don't.

• 7:56Casey Newton?

  Yes, I looked it up. And I would say it is a meme that is not great.

• 8:04Kevin Roose?

  Like, how not great?

• 8:05Casey Newton?

  Like, I would say that this meme originated in the world of pornography. And I would say pornography with a somewhat racist bent.

• 8:14Kevin Roose?

  Oh, okay. So we won't talk about the origination of the name Thug Shaker Central. But that was the name of this Discord server. A bunch of friends are in there talking about video games, presumably talking about lots of other things. And at some point, someone in this Discord server, who we now believe is the person who was arrested, the airman, Jack Tishara.

• 8:38Kevin Roose?

  Shares these classified documents in this essentially group chat. And this might have just remained in this group chat. But because this is the internet, because this is Discord, because as you mentioned, lots of people belong to multiple Discord servers, things just got really weird and out of control.

• 8:58Casey Newton?

  Yeah, and look, I'm sure you're in many group chats, Kevin. I know our listeners are too. And if there is one lesson you can take away from this story, it is that stuff is actually going to get outside of the group chat.

• 9:07Kevin Roose?

  Yeah, what happens in the group chat doesn't stay in the group chat.

• 9:10Casey Newton?

  That's right.

• 9:10Kevin Roose?

  And we don't know exactly what happened in this case, but it appears that some of this classified material was eventually taken from this Discord server, Thugshaker Central, and shared to a public Discord server that was devoted to a niche YouTuber called WowMao. Have you heard of WowMao?

• 9:31Casey Newton?

  I have now heard of WowMao. I've heard of WowMao now, but before now, had not heard of WowMao.

• 9:38Kevin Roose?

  How now, WowMao? And this is a thing, like lots of YouTubers have their own associated Discord servers where their fans will go and talk about their new videos or just talk about whatever. It's sort of this fan community. So these documents, these classified military intelligence documents, end up in this niche YouTube Discord server for WowMao. And then from there, it seems like somebody in that server saw those documents, saved them, and then re-uploaded them to another Discord server. This one, a big one devoted to Minecraft.

• 10:13Casey Newton?

  And this really is the funniest part of the story because what appears happened is that in the midst of a minor spat, one of the people who had access to these documents just threw them into the chat by saying, "Here, have some leaked documents." Which I think we can now agree is just a funny thing that we will now start saying to each other all the time.

• 10:34Kevin Roose?

  It is funny, but it's also like, this is a serious leak, right? Yeah. This is leading every major news organization right now. The State Department and other agencies are scrambling to contain the geopolitical fallout from this. So this is a major event. And the fact that it has such a weird and sort of hyper online origin story just made it totally fascinating to talk about. And so I thought we should bring in one of the reporters who has been really leading the charge on this story. And that person is Eric Toller. Eric is a reporter for Bellingcat, which is a group that specializes in what they call open source research, or just basically using publicly available internet data to kind of solve mysteries and do reporting.

• 11:18Kevin Roose?

  He teamed up with reporters on the visual investigations team at the New York Times. And he is an incredibly good internet sleuth. And he was actually one of the reporters who broke this story in the New York Times. So I thought we should get him in and talk to him about where this leak came from and what he can tell us about how this strange online discord culture has birthed a true national security scandal.

• 12:04Kevin Roose?

  Eric Toller, welcome to Hard Fork.

• 12:05Speaker 03

  Hello.

• 12:06Kevin Roose?

  So you are part of a team of reporters that have been digging into this discord leak and just published a story in which the name of the alleged leaker, or the leader of the discord server where these groups were originally leaked, is this 21-year-old from the Massachusetts Air National Guard named Jack TeShara. So my first question is, how did you and your colleagues track this person down?

• 12:36Speaker 03

  Yeah, it's a little convoluted. And there's a lot of different platforms that most people probably know about, maybe not everyone. So this whole thing came to a head and everyone realized what was going on. And the Times published a story about how the Pentagon was looking into this leak. And at the time, it mentioned that there were some posts on the messaging app Telegram. And so I was looking back and like, you know, there's no way this is the original source. They had to come somewhere. And I looked and looked and I found... Three photos and two other ones that weren't in the Telegram cache on 4chan. But four other ones on Telegram weren't on 4chan, so clearly they got it from the same place. I just don't know where that place is. So I looked and looked and looked, couldn't find anything. And then somebody DMed me of like, hey, I think I saw this on Discord.

• 13:17Speaker 03

  And they pointed me to a Minecraft map server. It's where people make maps like of Earth, like they try to make like real life Earth on Minecraft. And this guy had posted ten on there. And of the seven that were posted on 4chan slash Telegram, those seven were from this cache of ten. And there were three additional ones in here that no one had seen before, as far as I know. So I talked to that Minecraft guy, very nice Minecraft guy. And he's like, no, it's not me, because he was terrified, for good reason. He's like, no, no, he's also a kid. He's ever, by the way, this is gonna be a recurring theme. They're all kids, a bunch of seventeen year olds. And this kid was like, no, I'm not the leaker. Everyone, no, please, everyone leave me alone. I'm innocent. I got this from a different server, a different guy.

• 13:56Speaker 03

  And he got them from another seventeen year old, another kid, who posted them on another server, a precious YouTuber guy.

• 14:03Kevin Roose?

  Yeah, so you're of course talking about the WowMao server.

• 14:06Speaker 03

  Yes. So I, from there, I talked to a few other people, four people in total in the server. And eventually I figured out, kind of from these people I was chatting to, that it was a different server that this kid was a part of, that he pulled from and then posted on the WowMao. So there were hundreds and hundreds on this other server called, it had a lot of different names, but the name at the time of the leak was Thug Shaker Central, which is like this meme from like a couple of years ago. And this is where hundreds and hundreds of these were posted on here. The seventeen year old kid took a hundred and seven of these and cross-posted them onto the WowMao server. And they've been being posted on this older server, this Thug Shaker server, since like at the latest, um, October, it could have been earlier, but at the latest, around October of last year.

• 14:50Kevin Roose?

  So you followed those sort of trail of breadcrumbs back to this server, Thug Shaker Central.

• 14:55Speaker 03

  Yeah.

• 14:56Kevin Roose?

  And then from there.

• 14:57Speaker 03

  I think you'd say that out loud before.

• 14:59Kevin Roose?

  And then how did you figure out who in that server had probably leaked these documents?

• 15:06Speaker 03

  Yeah, and from there, I was like, okay, well, there were only about 20 or so active members on this server. So the pool is pretty small. And I know, because I talked to the kid who also was 17. And he told me about, like, yeah, so he called him OG. So he, I, that's not what anyone else called him, by the way. He just, just because I was like, the original leaker. And he's like, well, let's just call him OG, like, the original guy, or the original gangster, whatever.

• 15:29Kevin Roose?

  So this 17-year-old, this literal 17-year-old who's in this Discord server called Thugshaker Central, tells you that the person who originally posted this in that server, the leaker, we're assuming, is someone that he was calling OG?

• 15:44Speaker 03

  He was like, yeah, there's this guy, we call him the OG. He, you know, he has money and he's well-armed. He's like this really cool dude. And we played lots of games together on Steam. So Steam is a gaming platform. It's kind of like equivalent of like the PlayStation Store or like the Xbox Game Store. Like, we played Arma 3 together. We played Project Zomboid. And we played Halo. And he mentioned all these games they were playing. And that's like how like they're, they're buddies and they just play game together, right? So it's a bunch of teenagers, not just Americans, they're from all over, who are on this little Discord server, about 20 active users or so. And then comes this guy who we now know is Jack. He came in and was sharing all these leaked documents, like, hey, look, look what's going on, guy, right? That sort of thing. So I knew a handful of people who were involved. So I knew that these guys all played Steam together. I knew some of the games they played and I, you know, all that.

• 16:29Speaker 03

  So I used a few different tools to find, because all these, obviously, these people almost all made their accounts private or deleted their account, and it's basically everywhere after the story broke. So there's a couple of sites where, like steamid.uk, for example, that has historic scrapes of Steam data. And like, not just who their current friends are, but like who their deleted friends are. Either people deleted their account when private or removed them from their friends list. And I looked at some of these people's accounts and I kind of like, okay, here's 400 friends, here's 250 friends, here's 300 friends. And like, who are the intersections? Like, who is, who's the, who's common friends with all of them? And basically, all the people who were common friends with all the players that I knew about were people who were also in the server. So like, okay, well, which of these is the most likely guy? And there's one guy in particular who kind of stood out.

• 17:11Speaker 03

  I looked into a little bit more into this guy and I found, uh, well, actually one of my colleagues over at Bellcat was looking into his ID and they found that he was on this like arm, like this gun kit selling website, like he used, one of his Steam usernames he also used for selling some kit, like scopes and body armor, that sort of stuff. So, um, we looked more into this guy's Steam list and we saw, I guess, a few things that matched up, like all the games that the, this guy who's on the server told me they played together, Armor 3, Project Zonboy, Counter-Strike, blah, blah, blah. He played all those games, like there were screenshots he had posted, like there's activity, he played those games. And so then I went through the usernames that he had for Steam, because Steam, people, you can change your username over and over and over and over. And the very first username he had was something that's like very different than the other. It was like Jack something, something Tex.

• 17:57Speaker 03

  And I was like, that's kind of, that kind of sticks out because the other names are like the Excalibur effect, like, you know, very like dramatic names. And so I Googled this name and there's two results, one Steam profile and two was this Flickr album showing a guy and his kid. And this is from 2007. And it was Jack and Jackie was the name of the album, for example. So, um, I was like, well, Jack is the name on the Steam ID. And, uh, but you know, 2007 is pretty old. So I, at first we thought that, that this dad was, was the leaker. But then we looked more and more and we realized that, uh, we looked into the mom and we found, and there we found some other family members. And we found photos of a new Air Force Reserve person whose name was Jack,

• 18:42Speaker 03

  who had just recently entered into the Air Force National Guard. And he had like some kind of like shitposting adjacent stuff on some of his, like his Instagram and his Facebook. And I was like, oh, this makes way more sense. And then we looked a little bit more and we saw that one of the usernames for his Instagram and his Facebook was also something linked back to his Steam account too. So we realized that he just hijacked his dad's Steam account. So the Steam username and the Flickr username were the same. That's because, imagine you're like 12 or 13 and you play your dad's Steam account, but you don't want to make a new one because you already bought games on the account, right? You don't want to make a new account where you lose access to all your, you know, Counter-Strike and UG and all that stuff, right? So he just, he just basically took over his dad's account, renamed it and started using it. But the sites retained the historic name info.

• 19:28Speaker 03

  So the dad's Flickr account and the Steam account were the same, which led to the family, which led to photos of him. And it linked back to the username he used for his Instagram and his Facebook were the same as for his kind of revised hijacked Steam account. And then from there, realized, oh, he's actually with the intelligence wing of this Air National Guard. And then everything kind of fell together.

• 19:49Kevin Roose?

  Wow. And besides deleting the server, the Thugshaker Central server off of Discord, did the leaker try to cover his tracks in any other ways?

• 19:59Speaker 03

  Uh, I mean, he made his Steam profile private, but there are services out there, Steam ID dot UK, big shout out to those guys who run that server. I'm, I'm a nine dollar a month Patreon subscriber, so I got access to their premium stuff to help with this story.

• 20:10Kevin Roose?

  So Eric, this is a very impressive bit of internet sleuthing. And I know that you have spent a lot of time tracking down not just this leaker, but many other people and incidents on the internet. That's kind of your, your specialty. But I think the details of this are surprising to a lot of people, just because it seems like such a mismatch. You know, usually when you have a leak of military intelligence, it's coming from someone with an axe to grind or a point to prove, a whistleblower who has, you know, some conscious driven reason that they want to get this information out there. And it just seems so strange that it emerged in this Discord chat, Thugshaker Central, where these gamers were, you know, many of whom were, were literally teenagers.

• 20:54Kevin Roose?

  Were just sort of hanging out, trading insults and memes and whatever else. So help us understand for people who have not spent as much time in these kinds of Discord servers as you have, like, what is the culture of a server like Thugshaker Central that would make someone like this alleged leaker, Jack Tashara, want to post classified military documents?

• 21:17Speaker 03

  So, I mean, it, you know, again, they were just, you know, like 20 or so kids and they just kind of, it's very intimate environment. Cause like, it's like voice chat too, right? You're doing a lot of voice chat, you're playing games and all that stuff. So it's not like, it's like this very like, I don't know, like sanitized neutral, like distant, like a lot of times like Twitter or like, you know, whatever is a little bit more distant, where you see tweets and you interact with them very publicly. This is a lot more intimate, you have voice chat, things like that. So because of that, people probably felt like they were very close to one another. I mean, like this, the kid who gave the interview to Washington Post said like, he was my best friend. You know, he felt like he was very tight, very close to him. I'm not sure if it was reciprocated, but I mean, you can kind of get the idea that like, there was this sense of like they were tight in a community. So think of it as like a, you know, a more intimate group chat, I guess you could say,

• 22:02Speaker 03

  rather than like, you know, a Discord server, which makes it sound like it's a lot more official than it really was.

• 22:06Kevin Roose?

  Right.

• 22:07Speaker 03

  Which kind of builds the trust, I guess.

• 22:09Kevin Roose?

  I guess what I'm wondering is like, you know, I've been in some gaming group chats.

• 22:15Kevin Roose?

  I have friends that I know through playing video games. I have never once thought to leak them secret information from my job.

• 22:22Speaker 03

  You're too old. You're too old to think that, yeah.

• 22:25Kevin Roose?

  But sort of zooming out, as a person who has been studying extremism and online mischief for a long time, who has sort of specialized in tracking down hard to find people and things on the internet, was it surprising at all for you that this new leak that is really turning the United States government upside down, was it surprising to you that it came from such a seemingly strange source?

• 22:50Speaker 03

  Not really. I mean, if you think about like, how is this leak gonna happen? Because I'm sure this happens. I'm sure that there are people who have classified documents who share them with their buds, like Podgossip, right? But most people don't have a 17-year-old friend who will then post them onto a different Discord. So maybe the people who are doing this right now just have better friends, or at least maybe friends with better filters or a sense of judgment.

• 23:11Kevin Roose?

  Hmm. I mean, yeah, one lesson I'm taking from this is that people are just not as good at doing crimes as they think they are, and that especially when the internet is involved, they just like do not understand how to, you know, share things with other people in a secure way.

• 23:26Speaker 03

  Yeah, it's insane. Yeah.

• 23:28Kevin Roose?

  Eric, thank you for coming on Hard Work.

• 23:30Speaker 03

  Sure. Thanks for having me.

• 24:04Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner. Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 24:38Speaker 04

  It's time to innovate at the speed of AI without compromising control. That's Outsystems. Take a look at what happened with one global retailer during a period of massive expansion. They used Outsystems to build AI agents to automate invoicing and translation processes, shifting 95% of their daily store operations to a new digital app, saving thousands of hours of manual work. Outsystems gives you the flexibility to deploy how you want and the governance to do it safely. Own your agentic future. Learn more at outsystems.com/NYT. That's outsystems.com/NYT.

• 25:13Speaker 05

  AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 25:46Kevin Roose?

  Casey, it has been several weeks since we have talked about Twitter on this show.

• 25:52Casey Newton?

  Yeah, we had to set it aside for a minute, sort of let it cool down.

• 25:55Kevin Roose?

  Yeah. Right, it went into timeout.

• 25:58Casey Newton?

  Yep.

• 25:58Kevin Roose?

  But now it's back.

• 25:59Casey Newton?

  Yep.

• 26:00Kevin Roose?

  And a lot has been happening. So I have to confess that a few months ago, I was kind of doubting whether all the predictions of Twitter's doom were actually going to come true. Like, people kept saying, "The app is dying, people are leaving, it's gonna break, they're laying off all these people." It still ran and seemed pretty normal. But now it looks like things are seriously starting to decline.

• 26:21Casey Newton?

  Yeah.

• 26:21Kevin Roose?

  And we don't have time to run through all of the disasters. So I just picked sort of three categories of Twitter chaos that I want to ask you about. Checkmarks, circles, and clown shit.

• 26:34Casey Newton?

  The three Cs.

• 26:36Kevin Roose?

  So let's start with checkmarks.

• 26:37Casey Newton?

  Okay.

• 26:38Kevin Roose?

  So we're talking about the checkmarks, which are officially called verified badges. And Elon Musk, you know, he said that on April 1st, every person who does not pay for Twitter Blue, who has one of these checkmarks, you know, members of the media, or celebrities, or politicians, would lose their checkmarks. That date came and went.

• 26:56Casey Newton?

  Yeah.

• 26:57Kevin Roose?

  And as of right now, I'm pulling up my Twitter profile, and I still have a checkmark.

• 27:03Casey Newton?

  Oh, look, I do too.

• 27:04Kevin Roose?

  So what is the deal? What is going on with Elon Musk and these checkmarks?

• 27:09Casey Newton?

  Well, look, on one level, checkmarks did play an important function on the old Twitter, because they would tell you that somebody was who they said they were. And that helped with impersonation. And then Elon decided that the best path forward for Twitter as a business was to pivot it away from advertising and towards subscriptions. And the main kind of carrot that he wanted to offer to people in order to get them to subscribe was a verification badge of their own. As you well know, the first round of this went very poorly. They let anyone buy a badge for $8. They didn't check to see who anyone was. People impersonated, you know, the pharmaceutical company Eli Lilly and many other institutions at some

• 27:49Casey Newton?

  points, causing wild swings in their stock price. And so it kind of got put on a backburner. And Elon said, "Let us think this through a little bit more." And so, yeah, as you say, the new deal was gonna be, "Okay, everybody, April 1st, if you're not paying, you're gonna lose your badge." And then that got delayed. And so now he's saying everyone's gonna lose their badges on April 20th. That's 4/20 for all you stoners out there.

• 28:15Casey Newton?

  And we'll see. I mean, you know, Elon has a very chat GPT like relationship with the truth, you know, like this is somebody who is not drawing on a store of facts all the time, I would say. And so when I hear that he has set a new deadline, no part of me believes that it's real. I think it could happen, but like, I don't, I don't have any expectation that it will.

• 28:34Kevin Roose?

  Right. What do you think is actually going to happen on 420, the new day when all the checkmarks are supposed to disappear from these legacy verified people and organizations?

• 28:44Casey Newton?

  Well, here's what we know. So far, only a very small percentage of Twitter users are paying for Twitter Blue, which includes this badge. He keeps extending the deadline, I think, because he assumes people are going to fall in line and they're going to say, oh, I've gotten my extension. I guess I'll get that eight bucks together and I'll finally pay. And he's just banking on the fact that these badges are going to become more precious to people. I don't think that's going to happen. So at some point, he's going to have to put his money where his mouth is. He's just going to have to start taking away people's badges and then see if that has the desired effect. I don't think it's going to. But I can also see why he would want to delay it because after he does that, he doesn't have a lot of moves left, right? I mean, you could imagine moves, like he could start

• 29:27Casey Newton?

  charging people to tweet, right? Something like that. But if we get to that point, it's going to be in a very desperate place. So I could see him delaying this again and again and again.

• 29:36Kevin Roose?

  Yeah. My bigger question is like, we know that Elon Musk wanted to shift Twitter from an ad-based model to a subscription model, right? That is the business problem that he is trying to solve by instituting this $8 a month, like Twitter Blue verification scheme. Does the fact that this program has been so unpopular mean that people just aren't willing to pay for social media at all?

• 29:59Casey Newton?

  I think people are willing to pay for all sorts of things on social media. It tends not to be these features that were previously offered for free, but like TikTok was the highest grossing app last year. And the reason is it because you have to pay for your badge. So there's a lot of money out there for the social media ecosystem, but you have to bring real product into Design chops to it, you have to offer a stable functioning product. And Elon just hasn't been able to deliver on either of those.

• 30:32Kevin Roose?

  Yeah, I mean, to me, it's it shows the power of anchoring, right? If you start a new social app and you say, you know, this is going to cost $8 a month. And here are all the things you get, or, you know, you can give tips to your favorite creators. Like, that seems like a proposition that people might actually go for. But if you take something that used to be free and then start charging for it without any real increase in value, like, people are anchored to that free price. And it's going to seem not just a stretch for them, but it's going to seem offensive to them. They're like, I used to get this for free. I'm giving you my tweets for free. You're using them to make money. Like, I'm not going to pay you for the privilege of filling up your platform with my content.

• 31:10Casey Newton?

  Totally. And to me, the exciting thing here is this is a move that is finally making people start to say, you know what? Maybe I should stop giving this guy my tweets for free.

• 31:18Kevin Roose?

  Hmm. All right. Next C on the list, Circles. Twitter Circles are a product that I have never used. What are they?

• 31:27Casey Newton?

  So Twitter Circles is very similar to the close friends feature on Instagram. So if you use Instagram, you can create a list of your very close friends. And then if you post to your story, these sort of ephemeral posts that only last for 24 hours, those stories will only be seen by people in your close friends. Twitter Circles launched with the same basic idea that instead of broadcasting your tweet to the entire world, you could broadcast to a subset of the people that you follow. Now, I never used this feature very much myself because most of the people that I would include in my close friends are media people. And if I said something stupid on Twitter to my close friend circle, they would just screenshot it and get me fired.

• 32:10Kevin Roose?

  This is a huge relief to me, by the way, because I was paranoid that you did have a close friend circle on Twitter and that I wasn't in it, so.

• 32:17Casey Newton?

  I was posting detailed reviews of your podcast performance every week to a close group of friends. No, I wasn't using it. But I'll tell you, I have some gay friends in particular who were using Circles to post, let's say, some spicier, more adult oriented content. And so that made. What happened on Twitter circles this week, something of a catastrophe.

• 32:38Kevin Roose?

  Okay, what happened?

• 32:39Casey Newton?

  So what happened was, as I said, these posterior circles are supposed to be limited to a group of people that you've put on an allow list, okay? But then some porcals browsing the Twitter feed started to see posts on the For You page. The For You page is the algorithmically ranked feed of posts that Twitter might think you would enjoy for any number of reasons, right? It's ranked, but it could potentially be seen by anyone. And so essentially, you had private posts that were being not just made public, but recommended to the public. So the best analog I can offer here is imagine that you posted something really heartfelt or embarrassing or vulnerable on your close friend's Instagram story,

• 33:24Casey Newton?

  and then a stranger opened up the explorer page on Instagram and saw it promoted at the top of their feed. That's what just happened on Twitter circles. And in a way, it's like funny to me that this hasn't gotten more discussion, because this is a truly catastrophic privacy breach.

• 33:38Kevin Roose?

  Right, I mean, it's sort of like if your DMs were just showing up

• 33:42Casey Newton?

  Yes.

• 33:42Kevin Roose?

  on people's Twitter timelines. It's like that level of a privacy scandal.

• 33:48Casey Newton?

  Yeah, absolutely. You know, Elon did an interview this week where he boasted that a lot of folks said Twitter would be dead by now, but it's not dead, because like his standard for his Twitter Live, I think, is like, can you still log into the website? But this is a great example of how when you get rid of a huge number of people that worked on your technical infrastructure, weird stuff is just going to start breaking in ways that can have quite a terrible effect on your user base.

• 34:12Kevin Roose?

  Yeah, I mean, I've been wondering whether there will be some kind of mass DM leak that stems from all of these engineers getting laid off. That hasn't happened, but this Twitter circles thing does seem to be pretty close. Like, these are supposed to be private, but all of a sudden started showing up on people's For You feeds.

• 34:31Casey Newton?

  Oh, yeah, and when the layoffs started, Twitter engineers started messaging me saying, like, don't use DMs anymore. Like, they're just not a safe product. So I don't use Twitter DMs anymore for really anything remotely sensitive. But, um, yeah, watch out, man.

• 34:43Kevin Roose?

  Wow, so do you? Have any idea of like what went on behind the scenes to make these Twitter circles posts start showing up for people they weren't supposed to be seen by?

• 34:50Casey Newton?

  To my knowledge, they have said nothing about it. Which does not exactly inspire confidence.

• 34:55Kevin Roose?

  So put your spicy tweets somewhere else because they are not safe in Twitter circles.

• 35:00Casey Newton?

  Yeah, OnlyFans is now accepting spicy, private, subscription-only posts.

• 35:05Kevin Roose?

  When are you starting your OnlyFans?

• 35:07Casey Newton?

  Well, we'll see how Substack goes.

• 35:09Kevin Roose?

  Okay, that brings me to my third kind of catch-all category for what I am calling clown shit. This would include things that Elon Musk has done in the past couple of weeks that just seem like he's just kind of playing for laughs or, you know, not like behaving in a serious way. He briefly changed the Twitter logo to the Doge image, the Shiba Inu dog that is also the face of Doge Coin. That lasted for a few days. It was not on April Fool's Day.

• 35:40Casey Newton?

  No, I believe it launched on April 3rd and everyone was joking like Twitter tried to do an April Fool's joke, but it took like an extra two days to launch.

• 35:52Casey Newton?

  Which seems plausible to me.

• 35:53Kevin Roose?

  Yeah, totally. He also painted the sign on the Twitter building in downtown San Francisco, or he had it painted so that it just now reads "Titter."

• 36:03Casey Newton?

  I truly don't believe that even an eighth grader would find this stuff funny. I can't imagine anyone is observing any of this and thinking, "This is so funny and cool."

• 36:13Kevin Roose?

  Right. This is the man that I want to entrust my company's advertising budget to. He also changed his display name on Twitter for a day or two recently to Harry Balls. Harry spelled H-A-R-R-Y and Balls spelled B-O-L-Z.

• 36:29Casey Newton?

  This man is not well. Like, can we just say, "The man is not well"?

• 36:33Kevin Roose?

  Yeah, what is going on?

• 36:35Casey Newton?

  I, look, I think that driving a social network can actually drive you crazy. It is, it's very difficult. You're running a platform that is hosting this huge portion of the world's speech. You're getting criticism from absolutely every corner, and that's hard enough. Even if you are a very good CEO with kind of a good plan for where you're going. I think if you come into that with no background in the subject matter and you've committed $44 billion and you are losing money every month and none of your ideas are working. Oh, and also you're working on a handful of other giant companies that have their own problems. That is just not a recipe for success, right?

• 37:19Casey Newton?

  And so many people when Elon took over said, well, look, this guy's going to be different. Okay, he's special. He builds rockets. A social network? Yeah, I think he can handle that. And then you fast forward to April 2023. Guess what, honey? It didn't work.

• 37:30Kevin Roose?

  He's like, give me the rockets back.

• 37:31Casey Newton?

  That's right.

• 37:33Kevin Roose?

  Right, this isn't rocket science. It's actually harder.

• 37:35Casey Newton?

  It is hard, but like, is there one person who wouldn't be relieved if Elon Musk says, I am stepping away from Twitter and I'm just going to work on rockets full time? Like, everyone would be overjoyed.

• 37:43Kevin Roose?

  I think even his fans would be overjoyed.

• 37:45Casey Newton?

  Completely.

• 37:46Kevin Roose?

  Because, you know, he's been getting blowback from some people who previously have supported him. People like Paul Graham, who is a big Silicon Valley investor. He's a co-founder of Y Combinator. And he's been criticizing Elon recently for basically just taking his eye off the ball. You know, like wasting his time running Twitter instead of working on SpaceX or Tesla. So even the people who are sort of predisposed to support him are not fans of what he's doing at Twitter. I want to ask you about one more thing that I would put in the clown shit category, which is what is going on between Twitter and Substack?

• 38:20Casey Newton?

  Well, so, you know, ethics disclosure. I published my newsletter platformer on Substack and they've helped me in various ways over the years. And I have a note on my website about that. But Substack decided in the midst of all this chaos to...

• 38:35Kevin Roose?

  And Substack, for the people who may not know what it is, it is a newsletter platform. You can subscribe to newsletters, either free or paid. Substack is just kind of the infrastructure that powers that and allows people to pay you to read what you have to write.

• 38:50Casey Newton?

  Yeah, that's right. And they saw everything going on at Twitter and they said, well, gee, it seems like this thing is having some trouble. Maybe we could build something similar of our own. And so they built this product called Notes. It launched this week. Lets you post text and links and pictures and you can reshare those. They call that restacking. And when Elon got word of this, he really overreacted. And for a time, he blocked users on Twitter from engaging with tweets that had links to Substacks in them. If you clicked the link, it would say this website is spammy or dangerous.

• 39:33Kevin Roose?

  Which is true in the instance of Platformer.

• 39:35Casey Newton?

  Yeah, Platformer is a spammy and dangerous newsletter.

• 39:38Kevin Roose?

  But for other Substack newsletters.

• 39:40Casey Newton?

  It's better.

• 39:40Kevin Roose?

  Unfair.

• 39:41Casey Newton?

  Yeah, it's unfair. So they marked Substack links as quote, potentially spammy or unsafe. They also blocked searches for Substack. So if you search the word Substack, it would just show you results for the word newsletter. So it's like, oh, you're looking for a Substack. Well, we're not going to show you that, but we will show you newsletters. So when he got asked about this, Musk said that Substack was quote, trying to download a massive portion of the Twitter database to bootstrap their Twitter clone. And I went over to Hacker News to see what they were saying about this comment, which I found completely inscrutable. And the Hacker News people, this is a community of engineers and some of the most delightful nerds you'll ever meet. And the Hacker News people were just laughing their heads off because they were like, what does this even mean?

• 40:25Casey Newton?

  It's like, you can't download, like, it just didn't make any sense whatsoever. And so, you know, as usual with him, there's not a relationship with the truth there. It's just kind of like, you know, spewing words and it was all nonsense.

• 40:37Kevin Roose?

  I mean, what I found most interesting about this beef between Elon Musk and Substack is that one of the things that animated him when he took over Twitter was that he thought that the previous management of Twitter was sort of throttling or shadow banning sources of news that they didn't like.

• 40:52Casey Newton?

  That's right.

• 40:53Kevin Roose?

  He was very upset about what happened with Hunter Biden and that story during the 2020 election where Twitter, you know, blocked people from sharing that New York Post story about Hunter Biden.

• 41:04Casey Newton?

  Right.

• 41:05Kevin Roose?

  And then. He Hunter Bidened Substack. He made it so that you could not share Substack links or comment on them or like them or search for them. He disappeared Substack from the platform. And Substack is where a lot of the people that he is ideologically allied with, these sort of Twitter files journalists and other opponents and skeptics of the mainstream media are going. He was basically cutting off his friends and allies from one of their main sources of distribution.

• 41:32Casey Newton?

  It truly is the funniest possible outcome to this story. And I just think the more serious point is, this is not a principled man. This is someone who talked about nothing except his speech principles and his desire for bipartisan dialogue and let's move to the center. And he spoke about it in the most grandiose terms. And the minute he felt the slightest bit of discomfort, all those principles went out the window and he said, I am going to punish my enemies. And look, some people are very upset about this. I can't get upset about it because every day people learn what I truly have been saying all along, which is like, the emperor has no clothes here.

• 42:16Casey Newton?

  And you can just go on Twitter every day now and see it for yourself.

• 42:19Kevin Roose?

  But one of the other things that you have said is that this sort of chaos at Twitter is going to result in people moving.

• 42:25Casey Newton?

  Yeah.

• 42:26Kevin Roose?

  So far, that hasn't really happened. There has been no mass exodus of Twitter users to another platform. You're very active on Mastodon now. I think it's safe to say like, that has not really threatened Twitter in a massive way. Now, people on Twitter are talking about Blue Sky, which is this sort of decentralized alternative to Twitter that was started by Jack Dorsey. But none of these alternatives to Twitter have really gained momentum. So what is it going to take for something to pop up and replace Twitter? Or is it just kind of like, the entire concept of Twitter in our media ecosystem sort of dies off and is replaced by, I don't know, group chats and Instagram accounts and whatever else?

• 43:06Casey Newton?

  So there's two answers to that. There's the product answer and there's the NPR answer. I want to talk about the product answer first, right? So you just named some products that compete with Twitter. In various ways, and you're right, none of them has supplanted Twitter in the popular imagination. But again, let's remember Elon has only owned this website since October. We are six months out from when he took over. We first started talking about this on Hard Fork. The reason that these products are now being built is because there is an appetite for them. They are growing and frankly, there are just now so many of them that even if one of them hasn't like entered that period of hockey stick growth quite yet. I do think it is probably going to happen for someone.

• 43:51Casey Newton?

  So you're going to have to give it some time. Someone is probably going to have to invent some fun, new little mechanic. There's going to have to be some weird, silly new toy that just kind of captures our imagination. But I do think it's going to happen. So first answer is these products are still getting built, but they are getting built. And that was not true before Elon Musk took over, right? Nobody smelled blood in the water until the past few months. At the same time, Kevin, some important people actually are starting to leave. And I want to deliver one of our signature Hard Fork tributes this week to a little media organization called National Public Radio. You know these guys?

• 44:27Kevin Roose?

  I've heard of them.

• 44:28Casey Newton?

  NPR, they were just minding their own business. They were just tweeting links to their stories. And then one day they wake up and they've been slapped with this label, state affiliated media. And state affiliated media was a label that the old Twitter created to identify essentially propaganda.

• 44:46Kevin Roose?

  Right, it was like a little label that went under the profile of like essentially foreign propaganda outlets just to let you know that the tweets that you were seeing from this account have state affiliation.

• 44:58Casey Newton?

  Yeah, it's just trying to give you a little bit more information so you know where the account is coming from. They allowed them to post, but they wanted you to know that. Well, all of a sudden NPR, which does receive some minor amount of government funding, but is completely editorially independent of the United States government. They now have this label as well. So it effectively said, this is propaganda. That was the message that I believe the people who added this label to NPR's account wanted. To receive and messages are exchanged. And eventually, the state affiliated media label gets downgraded to the government funded media label, even though it gets less than 1% of its funding from the corporation for public broadcasting.

• 45:43Casey Newton?

  So the entire point of this was not to serve users of Twitter in any way. It was just to slag NPR and to try to make it look less credible.

• 45:54Kevin Roose?

  Right, it was a dunk.

• 45:54Casey Newton?

  It was a dunk. And on Wednesday, Steve Inskeep tweeted a picture of John Lancing, who is the CEO of NPR, at a Morning Edition meeting, Morning Edition, one of the flagship shows of NPR, and said that NPR will de-emphasize Twitter. He said that aside from the misleading label, Twitter isn't used by most Americans, drives little traffic to NPR, and no longer has the public service relevance that it once had. And to that, I just want to say, slay. Well said. This is what I have been saying, and pretty soon, a lot more people are going to get it too.

• 46:32Kevin Roose?

  You think media organizations are going to leave Twitter?

• 46:34Casey Newton?

  This is what I have been waiting for, is for somebody to wake up and say, I do not want to come to a platform every single day and get punched in the face. You know, we know what I've been thinking about lately, Kevin. Do you remember during the Trump campaigns, when there would be these rallies? And in the center of the rally, there would be a pit for the media? And a signature moment of every rally would be Trump pointing to the people in the media so that everyone at the rally could say, boo, we don't like the press. That is what Twitter has become. It is the press pit where a bunch of people are standing around you in a circle, jeering. Adding this state-funded media badge was one of those steps. But I'm barely joking when I say that I think eventually, every reporter who is still on the service will have a clown badge next to their username.

• 47:18Casey Newton?

  And you just have to decide if you still want to be there when that happens.

• 47:22Kevin Roose?

  So you are still on Twitter?

• 47:24Casey Newton?

  Well, in the way that I started with. So in December, I was like, I gotta change the way that I'm using this website. And so I basically started using it as an RSS feed. Whenever I have a sort of Twitter-like thought, I put it on Mastodon. And when I have some new piece of journalism to share, a newsletter, a podcast, I put it on Mastodon and then I repost it to Twitter. I'm not in there having fun. I'm really not replying to people. I'm reading it because it feels important to know what is being said on Twitter and it's important to know how Twitter is falling apart. Social networks are my beat. I have to pay close attention to it. But I am not really doing free labor for Elon Musk anymore.

• 47:58Kevin Roose?

  Hmm, fascinating.

• 47:59Casey Newton?

  But you still are. And let's talk about that.

• 48:02Kevin Roose?

  Well, I want the clown badge. That would be fun for me.

• 48:07Kevin Roose?

  Okay, so Twitter is in clown town right now.

• 48:11Casey Newton?

  Yeah.

• 48:11Kevin Roose?

  But next, I want to talk about a different town. One that is populated entirely by AI.

• 48:19Casey Newton?

  Uh-oh.

• 48:20Kevin Roose?

  We'll be right back.

• 49:14Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner. Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 49:48Speaker 04

  Building and scaling AI today can feel pretty chaotic, mostly because teams are juggling too many fragmented tools at once. That's where Outsystems comes in. Outsystems gives you one unified agentic systems platform to manage your entire portfolio of agents, apps, and workflows. You get to innovate at the speed of AI, but with the security, governance, and control your enterprise actually needs. And the results are already there. A major manufacturer used Outsystems to build a multi-agent system that analyzes blueprints. And a global bank cut their development time by 60%. Own your agentic future today at outsystems.com/nyt.

• 50:24Speaker 05

  AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 50:57Kevin Roose?

  Casey, did you watch Westworld, the HBO show?

• 51:00Casey Newton?

  Yeah, I watched the first few seasons, but I gave up as it became increasingly incomprehensible.

• 51:04Kevin Roose?

  OK, so I also watched like one season and I think it fell off because I just didn't get it anymore.

• 51:10Casey Newton?

  Yeah.

• 51:10Kevin Roose?

  Which I attributed to me being dumb, but I think maybe the show just also got worse.

• 51:14Casey Newton?

  Well, no, here's how it was. It became one of those shows where it was like, at first, it was very exciting to learn that that person you thought was a human was actually a robot. But then that was just the entire show. It was like, well, guess what other human is a robot? Every member of the cast.

• 51:25Kevin Roose?

  Right, so for people who haven't watched Westworld, the premise is there's this like theme park where there are humanoid robots that you can go and interact with and they'll talk to you.

• 51:36Casey Newton?

  They're perfectly lifelike.

• 51:38Kevin Roose?

  Perfectly lifelike. But when the guests leave, the robot's memories all reset. And so the next time you go, you can just kind of have like brand-new interactions with them.

• 51:48Casey Newton?

  Except what Westworld asks is, what if they didn't actually forget?

• 51:51Kevin Roose?

  Exactly. So I want to talk about this next story because I think it is actually moving us in quite an interesting direction that is a lot like Westworld. So a paper came out last week by researchers from Stanford and Google, and the title of the paper is called Generative Agents: Interactive Simulacra of Human Behavior.

• 52:12Casey Newton?

  Another one of these catchy clickbait titles from the academics.

• 52:16Kevin Roose?

  And I want to talk about this paper because it's really fascinating, and I think it actually points to the next phase of what's going to happen in AI. You know, we've been talking a lot on the show about these large language models, these chatbots, things like ChatGPT, and how you can actually like use them as a kind of persona. Like you can say, you are my tutor, or you know, you are my speechwriter, or you are my editor, or you're my friend. Like, talk to me as if you are that thing. And what these researchers wondered is, could you create a city, like a virtual simulated environment, and fill it with AI-generated characters?

• 52:55Casey Newton?

  A simulated city, or SimCity, if you will?

• 52:59Kevin Roose?

  Right. Basically, could you make The Sims, but The Sims are coming up with things on the fly? So The Sims is this, you know, very popular computer game, but in The Sims, like, all of the interactions between characters are scripted.

• 53:14Casey Newton?

  Right.

• 53:14Kevin Roose?

  And programmed. So what these researchers were wondering is, could you do The Sims with no scripting? If you just took a set of AI-generated personas, which they're calling generative agents, and placed them into a virtual environment, like a city, and just pressed play, what would they do?

• 53:36Casey Newton?

  I have to say, very interesting question.

• 53:37Kevin Roose?

  So the way this works, they created 25 characters. They put them in a virtual environment that they called Smallville, which is kind of like a kind of Animal Crossing style video game. And they gave all 25 of these characters short personality descriptions. So one example was a Character named John Lin and the thing that the researchers told the AI. And in this case, they did this using the chat GPT API. So all this is happening with the help of chat GPT. So the thing that they put in as the sort of description of this character was John Lin is a pharmacy shopkeeper at the Willow Market and Pharmacy who loves to help people.

• 54:18Kevin Roose?

  He is always looking for ways to make the process of getting medication easier for his customers. John Lin is living with his wife May Lin, who is a college professor, and son Eddie Lin, who is a student studying music theory. John Lin loves his family very much.

• 54:33Casey Newton?

  Sounds very sweet.

• 54:34Kevin Roose?

  So they build these sort of like pre-programmed personalities for these characters. And then they set them loose. They just say like, you know, we're going to observe you for two days. And we're just going to see what these characters will do.

• 54:49Casey Newton?

  And within the context of this game, like, are there objectives? Are these characters moving throughout the world? Are they just sort of bumping into each other on the street and striking up conversations? Or what is happening?

• 55:01Kevin Roose?

  Yeah, so the way it works, the researchers who are controlling this experiment can sort of prompt the characters. They can say, you are going to the store to get some food. Then the language model sort of fills in what happens next.

• 55:14Casey Newton?

  Okay.

• 55:15Kevin Roose?

  Or you're a character, you're bumping into another character in the park, and you're going to have a conversation. What does that conversation say? And then the language model sort of fills in the rest of the conversation and any subsequent actions.

• 55:28Casey Newton?

  Got it.

• 55:29Kevin Roose?

  So I can actually show you what this looked like.

• 55:32Casey Newton?

  Let's see this.

• 55:33Kevin Roose?

  I'll share my screen here.

• 55:34Casey Newton?

  All right.

• 55:35Kevin Roose?

  So this is a sort of replay of the Smallville. So you can see it's sort of like a blocky, like pixelated map.

• 55:42Casey Newton?

  Yep.

• 55:42Kevin Roose?

  With some various rooms. And then you can scroll down here. And here is a list of characters, 25 characters.

• 55:50Casey Newton?

  I mean, they're these sort of like very cute little sprites.

• 55:53Kevin Roose?

  Right. And so you can click on any of the characters and see a log of all of their actions. So here's a character named Ryan Park. And Ryan Park's current action is sleeping. And he is in his apartment in the bedroom.

• 56:09Casey Newton?

  And that's goals. Cheers to you, Ryan Park.

• 56:12Kevin Roose?

  And here is another character named Sam Moore. Sam is taking a walk around Johnson Park. He's stopping to take pictures. Oh, and he's having a conversation with another character named Adam Smith. And all of this is being generated by the language model. And along with these characters, the researchers actually developed something really interesting that is sort of like a memory, like a kind of virtual memory for these characters. Because one of the problems with language models, one of the limitations, is that they don't remember things. So you can have a conversation with ChatGPT, you can tell it some things, and then if you come back a different day and start a new thread, like, it's not going to remember what you talked about.

• 56:55Kevin Roose?

  You have to kind of re-instruct it to be a therapist, or a teacher, or a friend, or a negotiating counterparty, or whatever. So they sort of get around this with something that they call the memory stream, which is basically a natural language list, like a text list, of what a given character has been doing and experiencing. Any conversation they've had gets sort of stored in this memory file. And then periodically, these memories get kind of synthesized into inferences and reflections sort of summarized. So you basically have a system where each of these 25 characters can refer back to a file that contains sort of a synthesis and a summary of everything that has happened

• 57:39Kevin Roose?

  to them in this simulation. And they can refer to that in different contexts. So, you know, if one character is walking through the park and sees two other characters talking, they can like infer that those people must be friends and must know and like each other, and then they can use that information later on in the simulation. So it's sort of a neat, technical trick that simulates a kind of memory. Okay, so these researchers, they let this experiment run for two days, and they just observe what these characters do prompted by this large language model. And what they found was really interesting. So they did a sort of control group experiment where, after this two-day period was up, the researchers asked the generative agents a series of questions to kind of...

• 58:22Kevin Roose?

  Assess their self-knowledge, their ability to make plans and remember things. They also gave that test to a group of humans, 25 humans, who had access to all of the same information about these characters. And they basically gave them a replay of what the characters had been doing for these two days and all of their memories. And so the humans answered those questions, and then the researchers had these other humans, these evaluators, go in and rate how believable they thought those answers were. And the generative agents were evaluated as being more believable than the humans.

• 58:56Casey Newton?

  So that's bad news for humans, so go on.

• 58:58Kevin Roose?

  So that was one finding. The other very strange finding that they had was that there were these emergent behaviors that they did not expect and did not intend to happen between these generative agents.

• 59:10Casey Newton?

  OK, if you tell me that one of these things started asking if it was part of a simulation, I'm going to need a drink.

• 59:15Kevin Roose?

  No, they started leaking military documents. So one of the emergent behaviors was that these generative agents gossip.

• 59:24Casey Newton?

  What?

• 59:26Kevin Roose?

  They shared information with each other without prompting.

• 59:30Casey Newton?

  You're telling me that these Sims are spilling the tea on each other?

• 59:33Kevin Roose?

  These Sims are spilling the tea. So in one interaction, the researchers told one character, this character named Sam Moore, that he was going to run for mayor of Smallville. Sam Moore then shares this information with the rest of the town and starts a mayoral campaign. And the other agents start talking about this without interacting with Sam Moore. They're just like, they have learned this information.

• 1:00:00Casey Newton?

  I mean, this is admittedly big news in a town where, for the most part, people are just sleeping and talking on a park bench. Like, I could see why this would sort of be the buzz in Smallville.

• 1:00:09Kevin Roose?

  So, but not only are they talking about the fact that Sam Moore is running for mayor, but they're sharing opinions of him. So one agent at one point says, "I'm still weighing my options, but I've been discussing the election with Sam Moore. What are your thoughts on him?" And another agent says, "To be honest, I don't like Sam Moore. I think he's out of touch with the community and doesn't have our best interests at heart."

• 1:00:29Casey Newton?

  Well, go off, King.

• 1:00:31Kevin Roose?

  And not only are they sort of gossiping about each other, but they're also coordinating and, and planning together. So in one case, the researchers instructed an agent named Isabella to throw a Valentine's Day party. That was the instruction that they gave this character. And this character, Isabella, starts spreading the word. And by the end of the simulation, 12 characters know about this party. And how many of you do you think showed up?

• 1:01:03Casey Newton?

  I mean, in the real world, a lot of people would flake, but I'm hoping that these AI agents are, are more respectful. So I want to hope that all 12 showed up.

• 1:01:12Kevin Roose?

  Well, you were wrong because these are believable entities. And much like humans, many of them flaked. Seven characters flaked on the party. Three, three of them said they had other plans and the other four just didn't show.

• 1:01:26Casey Newton?

  It's like setting up a grider day. This is horrible.

• 1:01:29Kevin Roose?

  But five characters actually did show up at this party. And this was not something that the researchers had predicted. So why are we talking about this? You may be asking.

• 1:01:40Casey Newton?

  Well, I assume that each week you show up and try to terrify me with some new glimpse at our coming dystopia. Is there another reason?

• 1:01:47Kevin Roose?

  So I, I do think this has dystopian implications, but I think the most obvious application for this is something like gaming.

• 1:01:54Casey Newton?

  Absolutely, yes.

• 1:01:55Kevin Roose?

  So in games, a lot of them, especially role playing games or open world games, there are these things called NPCs or non-playable characters. And they're the kind of characters that you, you know, come across and they tell you some line or give you some hint. Or like, how would you describe an NPC?

• 1:02:10Casey Newton?

  Yeah, and NPC is essentially an object in the game that exists to give you an objective or to advance you along a quest. But they are scripted to make you feel like you are interacting with something sentient. And of course, you know, it's not. But the question that this poses is, well, what if word got around about what you were doing in this virtual world? What if that character could ask you about that? Or maybe change their behavior? This sort of Sort of thing does already happen, but all of those interactions are scripted. Right. What you're sort of suggesting here is that we might not have to script these things in the future. They can just happen.

• 1:02:51Kevin Roose?

  Right. They could just act as the language model tells them to act. So one example that was sort of a joke, but I think someone said in response to this paper that you could be playing a game now and you could kill an NPC in the game and you could return back to that same spot later and all of the other NPCs could be holding a funeral. And like, that's not something that the game designers would have to necessarily script. I did talk to the lead author of this paper, who's a Stanford researcher named Jun Sung Park. And I sort of said like, well, beyond games, like, how could this be useful? And he said that one of the ways that this could be used was to simulate behaviors in online groups.

• 1:03:32Kevin Roose?

  So for example, if you were going to start a new subreddit devoted to like celebrity gossip and you wanted to see like, okay, how would a set of rules given to users of this subreddit, you know, encourage them to react? Would they become rebellious? Would the thing sort of devolve into a toxic cesspool? Like, how could you basically simulate that without actually having to start the subreddit?

• 1:03:56Casey Newton?

  Yeah.

• 1:03:57Kevin Roose?

  And this experiment suggests that you could actually just start a simulated subreddit and make some simulated users and see what happens.

• 1:04:05Casey Newton?

  Totally. You know, another thought that comes to mind is I think we both believe that AI companions are going to be a really big deal. There's going to be some kind of virtual agent that you're talking to, whether it's on your phone or your desktop. And in order for that to work, they need to have a memory. They need to remember what kind of day you were having yesterday. They need to know the names of your friends and family and other people who are important to you. What I think is interesting about this is it suggests, what if you actually have multiple different companions for different purposes, right? You're going to have your personal trainer and your therapist. And of course, at first, none of these are going to be as good as humans, but they probably all get better over time. And what if they talk to each other and share information? What if you have a virtual doctor who remembers your entire health history

• 1:04:49Casey Newton?

  and doesn't make you write down your birthday every time you go? To the office, right? So there, I think there is a huge amount of potential here. And that is a very fun glimpse into somebody already running the experiment.

• 1:04:59Kevin Roose?

  Well, so, yeah, so this is the experiment that got a lot of attention this week. But there's also this sort of related thing that is happening in the AI world right now, which is that a lot of people are trying to make these large language models into agents. So turning them from chatbots into things that can act autonomously without human intervention.

• 1:05:19Casey Newton?

  We've talked a fair bit about like the need for AI vocabulary. And I'm telling you, agent is just one of those words that you are going to start hearing absolutely all the time.

• 1:05:26Kevin Roose?

  Yes, so in the last couple of weeks, there have been a number of projects that have tried to basically cut out the human. So right now, if you go to use ChatGPT, like you prompt it, you know, you type something in, it types something back. You type something in, it types something back. It's like a kind of ping pong. And what these experiments with AI agents are asking is like, what if the AI model could essentially talk to itself and come up with plans and execute those plans and then generate AI agents to carry out different parts of this task?

• 1:05:56Casey Newton?

  We are really just determined to wipe out humanity, aren't we?

• 1:05:59Kevin Roose?

  Well, some people literally are. So right now, there's a lot of research and energy in the AI world around turning these large language models into what are called agents. One of them is called AutoGPT, and AutoGPT is sort of a tool that someone hacked together that allows you to, instead of just like typing to a chatbot and it types back, you actually give it a task. And the AI can then sort of talk to itself and kind of recursively execute that task or try to execute that task. So you could tell theoretically this AutoGPT, like start a dropshipping business. And you could then sort of talk to itself and say like, okay, for that, I'm going to need someone who's an expert in shipping.

• 1:06:44Kevin Roose?

  I'm going to need someone who can talk to the big e-commerce companies. I can, I'm going to need someone who can set up, you know, maybe a Stripe account. I'm going to need someone to code up a website. And also, I'm going to need someone who can go on Twitter and advertise this business. Now, just to be clear, there's no evidence yet that AutoGPT can actually do this stuff well. It's still not very good, and there are still some pretty big technical hurdles standing in the way of these agents being able to actually do things autonomously. But that's the direction that some people are trying to push these AI systems.

• 1:07:17Casey Newton?

  Yeah, it's like when you're fighting a boss in a video game and they're like, and you think you're finally whittling them down and then they just summon a bunch of other demons to eat your face. That's what we're building.

• 1:07:26Kevin Roose?

  Right. And the examples of this AutoGPT stuff that have been floating around are mostly just kind of people trying to do these business tasks. But there is one project called Chaos GPT.

• 1:07:37Casey Newton?

  Oh, no.

• 1:07:37Kevin Roose?

  Where someone basically used AutoGPT and gave it the explicit instruction to try to destroy humanity.

• 1:07:46Casey Newton?

  Well, the world is still intact as we record. So what happened?

• 1:07:49Kevin Roose?

  So, yeah, this has not worked, as you can see by the fact that we still exist.

• 1:07:53Casey Newton?

  I mean, my world was destroyed when Taylor Swift and Joe Alwyn broke up, but other than that.

• 1:07:58Kevin Roose?

  But there is a mode on AutoGPT called Continuous Mode, which basically allows it to, in theory, run forever, just doing these kind of like recursive task loops. And when you turn on Continuous Mode, it does actually pop up a little warning saying like, you might not want to do this because it is, quote, "potentially dangerous and may cause your AI to run forever or carry out actions you would not usually authorize." But the mode exists. People are using it. It's out there already. And I've talked to some AI safety researchers in the last few days who are very concerned about this shift from thinking of these AI language models as tools to using them to create autonomous agents.

• 1:08:39Kevin Roose?

  And what they'll say is like, look, these experiments, these Smallville, you know, these generative agents that are just going to parks and talking to each other and planning Valentine's Day parties, like, these are not dangerous because, you know, they're contained in this virtual environment. And they're, you know, pretty safe in the things that they are talking about. But you actually don't want AI agents operating without human oversight. You don't want them carrying out their own goals. You don't want them, you know, doing long-term planning and coordinating with each other. That's just like not a capability that you want to build, right?

• 1:09:11Casey Newton?

  Yes. And I, while I'm glad that some of this research is being done in the academic realm, the fact that these kind of experiments are already being run does make me hope that our government is paying attention and is pretty soon going to have, start to have some ideas about whether and how all this ought to be regulated.

• 1:09:26Kevin Roose?

  Yeah, I mean, I will say like, I'm pretty skeptical of regulation in this space, at least the proposals that I've seen so far. But when I saw Chaos GPT, which stated that its explicit goal was to destroy the world, I was like, yeah, that should probably be illegal.

• 1:09:40Casey Newton?

  Yeah, I mean, there's, yeah.

• 1:09:44Kevin Roose?

  Like, and this is the thing, like, all the scenarios of AI doom out there, like, imply that the robots are gonna have to, like, become power-seeking and, you know, greedy for some resource that it will kill everyone to get. But like, actually, it may be that, like, some troll on the internet is just gonna, like, give it an instruction to go destroy humanity.

• 1:10:03Casey Newton?

  And as soon as that's done, you know what the first thing they'll do is? They'll go on Discord and say, "Hey, look at this. Guess I won."

• 1:10:09Kevin Roose?

  Right. So, the last concern that I heard from AI safety researchers about these AI generative agents is that these things that start off as being very benign and not all that capable tend to get more powerful over time.

• 1:10:26Casey Newton?

  Like the Hartford Podcast.

• 1:10:27Kevin Roose?

  Exactly. And so, you can imagine hooking some of these generative agents up to physical robots. Like, that is a thing that can be done.

• 1:10:36Casey Newton?

  Like Megan?

• 1:10:37Kevin Roose?

  Yes. You could take one of these generative AI personas and tell it to control a hardware robot. And like, it could do that.

• 1:10:45Casey Newton?

  Oh, you know what? My dream of a sassy Roomba has never been closer to reality.

• 1:10:49Kevin Roose?

  So, I really just think this is an area to watch, and that I hope we'll come back to on this show, because this use of AI to create these autonomous agents, I think, poses some really interesting opportunities. I do think there are applications for this beyond gaming. And I think it's an area where there is the potential for a lot of harm and mischief. So, stay tuned. We will be giving you more updates on the AI agents.

• 1:11:17Casey Newton?

  But in the meantime, I think I wanna go check out Smallville, actually. I wanna see what the buzz is. Are they talking about us? Are they listening to the show?

• 1:11:23Kevin Roose?

  I don't think so. But, you know, I'm gonna, I'm gonna check out another character. Maybe we'll find something here. Latoya Williams is currently taking a yoga class, doing standing poses.

• 1:11:34Casey Newton?

  Everyone in this town is having a more relaxing day than I have, and I'm jealous.

• 1:11:37Kevin Roose?

  Let's go to Smallville.

• 1:11:39Casey Newton?

  All right, let's do it.

• 1:11:40Kevin Roose?

  All right.

• 1:12:33Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner. Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 1:13:07Speaker 04

  Companies are pouring money into AI, but few leaders can explain how those AI investments work with the systems you already have. You know, the ones that actually run your business? Without Systems, you don't have to start over with AI. You can choose the projects where AI will have the most impact. And OutSystems gives you the platform to make it agentic on your terms with complete control every step of the way. So you can trust AI with the mission critical systems that run your business. Own your agentic future. Learn more at outsystems.com/NYT. That's outsystems.com/NYT.

• 1:13:42Speaker 05

  AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with realtime guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 1:14:17Kevin Roose?

  Hard Fork is produced by Davis Land and Rachel Cohn. We're edited by Jen Poyant. This episode was fact-checked by Caitlin Love. Today's show was engineered by Alyssa Moxley. Original music by Dan Powell, Alicia Baetoup, Marian Lozano, Rowan Nimmesto, and Sophia Landman. Special thanks to Paula Schuman, Nell Galogly, Kate LaPreste, Quiwing Tam, and Jeffrey Miranda. You can email us, as always, at [email protected]. But please, no top-secret documents. Yeah, save those for your Discord server.

• 1:15:31Speaker 04

  It's time to innovate at the speed of AI without compromising control. That's OutSystems. Take a look at what happened with one global retailer during a period of massive expansion. They used OutSystems to build AI agents to automate invoicing and translation processes, shifting 95% of their daily store operations to a new digital app, saving thousands of hours of manual work. OutSystems gives you the flexibility to deploy how you want and the governance to do it safely. Own your agentic future. Learn more at outsystems.com/NYT. That's outsystems.com/NYT.