OpenAI's Fog of War + Betting on Iran + Hard Fork Review of Slop

“The Pentagon and OpenAI are saying to the public, You’re just going to have to trust us. And the public is saying, Well, we don’t.”

• 0:00Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 0:34Speaker 01

  Casey, something horrible has happened.

• 0:36Speaker 02

  What's that?

• 0:36Speaker 01

  My wife has fallen in love with an AI.

• 0:38Speaker 02

  Ah, which AI?

• 0:41Speaker 01

  So for background, you know, my wife is not like a very techie person. She's not an early adopter. She works in city government.

• 0:49Speaker 02

  She's like a regular adopter. Like, when it gets good, she's interested. But don't talk to her before then.

• 0:54Speaker 01

  Exactly. And it just got good enough for her to take an interest. And so yesterday, she discovered Claude Code and started using it to do some stuff at work. And now it's all she wants to talk about. She were like out at a party last night, and she's like, I just can't stop thinking about my coding. She had a dream about vibe coding last night.

• 1:13Speaker 02

  Really?

• 1:14Speaker 01

  And maybe all of this is sort of some like karmic, you know, revenge for the Bing Sydney episode, but I do feel like I'm getting a taste of my own medicine here.

• 1:24Speaker 02

  I mean, to me, this seems like your dream come true. Like, what does any man want more than his wife taking an interest in his hobby? This is like, could be one of the best things that's happened in a long time.

• 1:34Speaker 01

  It's true. I think we've always benefited in our marriage from the fact that we are-

• 1:39Speaker 02

  Are you speaking about our marriage or your marriage to your wife?

• 1:41Speaker 01

  My marriage to my wife. That we're sort of like interested in different things and we can kind of cross pollinate. And so I guess I want to ask you as someone who's in a relationship with someone who works in AI, like, how do you stop talking about it? Do you have like set hours where you're like, we're not going to talk about AI for this next hour?

• 1:59Speaker 02

  Here's what I'll say. If I ever figure that out, you'll be the first to know.

• 2:02Speaker 01

  Okay, thank you.

• 2:05Speaker 02

  In my house, the few respite's we get from AI during the week, I would say would involve Friday night episodes of RuPaul's Drag Race. That's a good solid one hour of not talking about AI.

• 2:16Speaker 01

  Okay.

• 2:17Speaker 02

  Um, and yeah, outside of that, we're, we're really monitoring the situation, Kevin. We are fully locked in.

• 2:29Speaker 01

  I'm Kevin Roos, a tech columnist at The New York Times.

• 2:32Speaker 02

  I'm Casey Noon from Platformer.

• 2:33Speaker 01

  And this is Hard Fork.

• 2:34Speaker 02

  This week, OpenAI scrambles to contain the fallout from its deal with the Pentagon. Then, how prediction markets have become one of the most controversial parts of the US attack on Iran. And finally, it's a Hard Fork review of Slop for Children. You'll never guess what Peppa Pig is doing now.

• 3:00Speaker 01

  Well, Casey, we are now in week two of this incredible, high-stakes drama that's been playing out between the Pentagon and America's leading AI companies. There's been a lot going on. We now have more clarity on why the deal between Anthropic and the Pentagon fell apart. We also know how this Anthropic's supply chain risk designation is actually going into effect and impacting the way that government agencies are responding. And we have been learning this week about how OpenAI's deal with the Pentagon is shaping up. So lots to discuss here, but first, we should make our disclosures. I work at The New York Times, suing OpenAI and Microsoft in perplexity over alleged copyright violations.

• 3:37Speaker 02

  And my fiance works at Anthropic.

• 3:40Speaker 01

  Okay, let's start with OpenAI, because they are sort of the late arrival into this story, but in some ways, the most dramatic. Since Sam Altman announced last Friday that OpenAI had arrived at an agreement with the Pentagon, we have learned a little bit more about that agreement. As a reminder, according to Sam Altman, this agreement did include some prohibitions on domestic mass surveillance and autonomous weapons systems, basically the same two red lines that Anthropic had set out that were causing them so much trouble with the Pentagon. And I think it's fair to say, like, this provoked one of the biggest backlashes in that company's history.

• 4:17Speaker 02

  It really did. We've seen it across social media. Many sort of top-up voted posts on OpenAI-related subreddits have been condemning this move. OpenAI has been scrambling to try to rebuild trust. But at the end of the day, Kevin, I think both the Pentagon and the OpenAI Are saying to the public, you're just going to have to trust us. And the public is saying, well, we don't.

• 4:43Speaker 01

  Right. So there's been a lot of people cancelling their chat GPT subscriptions and switching over to Claude as a result of all of this, people who don't agree with the Trump administration or the stance that the Pentagon has taken here. And presumably because they're seeing some, you know, some, some pain in the cancellations department, as well as just a general feeling that this narrative is not going well for them. Sam Altman has been doing some damage control. So on Saturday, he hopped on X to talk about this and answer questions about the Pentagon deal, along with two other employees. And these questions were sort of the kinds of things you'd expect, you know, people asking, what did you guys agree to that Anthropic didn't?

• 5:25Speaker 01

  Where are your red lines? Who's going to be making the kinds of hard decisions during something like a war? About how these models can and can't be used? What about this domestic mass surveillance thing? So I think he answered some of these questions, but really the thing that they did was also to release the language of this contract that had been in dispute, that had been the subject of so much speculation.

• 5:46Speaker 02

  Well, they released what they called the relevant portion of the contract. But then we would see later commentary from experts in government procurement that said, essentially, look, until we see the entire contract, it's just very difficult for us to take at face value the idea that this is the only relevant language here.

• 6:03Speaker 01

  Right. So they did not release the whole contract, but they did release some relevant language from this contract with the Pentagon in a blog post. Then on Monday, Sam admitted that he made a mistake. He said, we shouldn't have rushed to get this out on Friday. He also added that it looked opportunistic and sloppy.

• 6:19Speaker 02

  Or sloppertunistic, to point a phrase.

• 6:22Speaker 01

  Yes, it was sloppertunistic. And he announced that OpenAI was going to amend its deal with the Pentagon to explicitly rule out the use of OpenAI's tools for domestic surveillance of US persons and nationals. And it includes the language, quote, "The department understands this limitation to prohibit deliberate tracking, surveillance, or monitoring of US persons or nationals, including through the..." Procurement or use of commercially acquired, personal or identifiable information. I found this all slightly confusing. Casey, do you understand what OpenAI has said and the various evolutions of its position on this?

• 6:58Speaker 02

  Well, I think the key takeaways here is that they are saying that they have put in some amended language that will prohibit certain uses of their systems by the government. So for example, they're going to prevent the government from using commercial data that they sort of acquire legally and sort of running that through GPT models for domestic surveillance. I just want to say, though, that there is always a high risk here for what I would call Jedi mind tricks, Kevin, and for the government, because we have seen Democratic and Republican presidents do this, right, of sort of going to the absolute limit of what the law will allow when it comes to surveillance of Americans. And a way that they'll get around that is by saying, well, we're

• 7:43Speaker 02

  not doing surveillance, Kevin, we're doing some intelligence gathering, right? And so as annoying as it is to fixate on the semantics here, I'm telling you that whether or not you personally are surveilled will come down to semantics, right? And so that's why we're digging in the way we are.

• 8:00Speaker 01

  Okay, so still lots of questions about some of the details here. And I think there is a lot of doubt and concern among some employees of OpenAI that this actually did end up in a place that they're comfortable with.

• 8:11Speaker 02

  Boy, is there. Some of that employee discontent spilled over onto X, where you had some employees saying essentially that they didn't trust their leadership either. An employee named Leo Gao called the contract language window dressing and pointed out that it still seems to give the Pentagon control over when to deploy autonomous weapons and just doesn't do much to address some of the other loopholes. And then maybe more dramatically, Kevin, on Tuesday, Max Schwarzer, who was the post-training lead for OpenAI, a vice president of research at the company, announced that he was leaving. And in his X post, while he was pretty vague, suggested that this was an important time and that he had come to really respect Anthropic's values. And so he said he's going over to work there.

• 8:56Speaker 01

  Yeah. So what's your take on how the damage control is going for OpenAI? Do you think they have warded off the the most heated criticism? Or are people still really mad?

• 9:05Speaker 02

  I I do not think that they have stemmed the tide. I think they they made they put a lot of effort into changing the narrative here. When I saw that they were doing that X AMA, that they had put up a blog post, that they were quoting at least some of the contract language. I thought, these guys are really going for it. That also told me that they were really scared. But here's the thing to remember, Kevin. Most Americans just don't like AI very much. They did it in the first place. They didn't like it for all the normal reasons of, well, my social media feed is filling up with slop, but my manager is telling me I have to use it every day or I'm going to get fired. When you add into that mix, it's potentially also going to be used by your own government to spy against you or maybe kill you with a murder bot. Of course, Americans are going to say, well, this fricking sucks, right?

• 9:48Speaker 02

  So I think this was kind of the strategic miscalculation that Sam Altman made, was that, at least according to him, he thought he was going to get into this this dispute and sort of be able to deescalate it and sort of come in as the white knight and save the AI industry from the overreach of the US government. And what he found out instead is they're still, they're kind of holding the bag of all of the discontent that the Pentagon whipped up with this forced policy change.

• 10:12Speaker 01

  Yeah, it's really interesting to me because I think my assumption had been that we were sort of over the era of like worker empowerment in Silicon Valley, right? Like years ago, sort of pre-COVID, we had all these like Google walkouts and all these employee protests over these military contracts. And I think a lot of CEOs and leaders at these companies sort of said, we're not doing that again. Like, we're not going to give our employees veto power over the deals that we make or the contracts we sign. And it suggests to me, what is going on at OpenAI right now, that at least for them, in their specific case, where you do have, you know, this staff of elite technical talent that are not easily replaceable. There aren't that many people who know how to like build and train these models. You actually do need to keep them happy.

• 10:54Speaker 01

  And so those people, maybe only those people have significant leverage still.

• 10:58Speaker 02

  Yeah, let me make a sort of sweeping generalization, right? Like, I think there's sort of like two major camps at at OpenAI. One are the camp That have sort of been there for, you know, let's say, three plus years that are the the real experts that you just mentioned that have this kind of critical knowledge for how to build next generation frontier systems that almost nobody else in the world has. And those people tend to just really care a lot about how the technology is used. These are people who joined OpenAI in part because it was a nonprofit, right? And like, there is like a solid core of those folks who are still working there. And then there's a group at OpenAI that I'm just going to call the Meta people.

• 11:37Speaker 02

  The people that came over from Meta a little bit more recently that are, you know, maybe a little bit more flexible in what they're willing to see their company do. And I don't think that they're going to raise a big stick about this. The problem if you're OpenAI leadership is you actually need that original core, right? If you're going to build a GPT six and seven that is going to blow everybody's minds, those are the people you're going to need. And so, yes, almost everything that we have seen over the past few days that they've tried to do damage control is aimed at those people.

• 12:03Speaker 01

  Okay, so that's a little bit of the drama going on at OpenAI. What is happening at Anthropic?

• 12:08Speaker 02

  Printing money in two words, I would say. Well, I mean, you know, I wrote this in my newsletter this week, Kevin, but has an American technology company ever had such a good week and such a bad week at the same time?

• 12:20Speaker 01

  Explain.

• 12:21Speaker 02

  Well, so on the bad side, obviously, they're in a very heated fight with the Pentagon that continues, by the way. It seems like there is still some risk that perhaps the president will try to invoke the Defense Production Act to try to compel Anthropic to make the version of Claw that it does not want to make that would sort of do its bidding. And it seems also that this supply chain designation risk is now official. We learned on Thursday that the Pentagon sent a formal letter to Anthropic. So if nothing else, this is going to result in a long and costly legal battle as Anthropic tries to ensure that American companies can still use it for non-military purposes, right? So there is actually an existential threat to the company that is buried somewhere inside there, and it is by no means over.

• 13:05Speaker 01

  Right, but on the good side.

• 13:07Speaker 02

  On the good side, Bloomberg reported this week that Anthropic is on track to hit $20 billion in annualized revenue. At the start of 2025, Kevin, they were on pace to earn about $1 billion in annualized revenue. So this company has 20 Xed over the past year. They were on pace to make about $9 billion by the end of 2025. So it has doubled in barely over two months, which speaks to the rise of Claude Code, right? And the overwhelming adoption of Claude in the enterprise. So in that respect, this really has become maybe the fastest growing American technology company of all time.

• 13:44Speaker 01

  Yeah, and like what's so strange about this sort of dual quantum state right now of Anthropic is like, at the same time that they are printing money and people are signing up for Claude and they're switching from ChatGPT and like, things appear to be going well for them. At the same time, they are also being pulled out of the federal government, right? Forcibly, there was some reporting this week by Reuters that the US State Department has sort of started to comply with this order from President Trump to sort of stop using Anthropic's models. They have switched the model powering their sort of in-house State Department chatbot from Anthropic's models to OpenAI, according to this memo seen by Reuters. And furthermore, this Reuters report said that the State Department is going back to GPT 4.1.

• 14:29Speaker 01

  Now, if you have not been tracking all of the model names and numbers as closely as we have, that is several generations ago. That's like, that's like a 20, an early 2025 model.

• 14:40Speaker 02

  Yeah.

• 14:40Speaker 01

  And basically, what that means is that the average college freshman with a ChatGPT subscription now has access to substantially better AI tools than the Department of State.

• 14:50Speaker 02

  It's not great for a lot of reasons, Kevin. And one of them, as the blog Lawfare covered this week, is that there appears to be no statutory authority for the president to do what he did. There is not a statute that lets the president just sort of declare that, that federal agencies cannot use individual software. But because this is just the way the Trump administration works, everyone has just decided to defer to the president.

• 15:11Speaker 01

  Yeah, I want to ask you about this other sort of interesting piece of OpenAI's response over the last week, which is that Sam Altman has said multiple times that he wants the Pentagon to extend the same deal to Anthropic that it extended to OpenAI. Do you think that is sincere? What is going on here? Why is Sam Altman saying, hey, if you're making these terms available to us, you should also make them available to other AI companies?

• 15:35Speaker 02

  I think that that is the the part of Sam that appears to be sincere in saying that he wants to deescalate this conflict. He does not want the United States government to come in and nationalize the AI companies, at least not right now, right? And so maybe if if OpenAI could reach some sort of agreement that would provide at least some protections for Americans and other AI companies would sign on to it, that would just sort of release the pressure on the industry overall. Now, of course, at the same time, it would buy him a lot of cover. And all of a sudden, people wouldn't be mounting these quit ChatGPT campaigns because Sam could be on X saying, well, you know, Claude is doing the same thing.

• 16:11Speaker 01

  Do you think that's real? Like, how big a deal do you think this consumer opposition is? I mean, I, you know, I am somewhat jaded on this point because I can't count the number of times that people have said, you know, oh, we're we're all going to cancel our subscriptions to this thing, or we're going to delete Uber, or we're going to, you know, quit Facebook in protest. And like, it never really seems to have much of an impact. But like, do you think in this case that enough people are mad about this at the consumer level that it could actually impact their business?

• 16:40Speaker 02

  Not really. I think you're exactly right. I think that usually these things just tend to blow over in a few days. And I'm sure that OpenAI is counting on that. At the same time, though, Kevin, I think back to the lesson that Meta learned, which is that as it had its own series of controversies, by and large, people did not quit Facebook. They did not quit Instagram. But you know what they did do? Just kind of start to hate Meta as a company and develop really low trust in that company. And that winds up hurting Meta in all sorts of ways. And the particular way, by the way, that I think this is going to hurt OpenAI is they're gearing up to go out and build a lot of data centers around this country. And there's already enormous backlash and that we are seeing, right? We're starting to see it creep into our politics.

• 17:23Speaker 02

  And so if they are not able to sort of reverse the narrative and convince people that AI is going to have like hugely positive outcomes in their lives, I think you're going to see the data center opposition ramp up as a proxy for people's just kind of distrust of that company in general.

• 17:37Speaker 01

  Right, it's the it's the visible, physical symbol of all of this. And for most people, the only one that is like anywhere near them. And so I think you're right, it could turn into a political problem for them, even if people aren't cancelling their ChatGPT subscriptions en masse.

• 17:50Speaker 02

  Yeah.

• 17:51Speaker 01

  I want to ask you about something else that I've been thinking a lot about this week, which is this idea that you mentioned of nationalization. There's been a debate happening on social media about this idea that if we are headed to a world with very powerful AI systems in it, as Dario Amadei calls it, a country of geniuses in a data center, that eventually that will just not be allowed to happen inside a private corporation, that the US government, whether a year or two years or five years from now, at some point, will step in and say, hey, you guys built this really cool thing that's really useful and has all these like important geopolitical and national security implications, we're gonna just take that now and you work for us now.

• 18:34Speaker 01

  And I'm curious what you make of that as a possibility, because some people who I consider quite serious and credible have been talking about this threat of nationalization for several years now.

• 18:47Speaker 02

  Yeah, if you go to the sort of nerdy AI conferences that Kevin and I do, this comes up a lot at the tabletop role-playing games that people do during lunch, right? Is that at some point, a government of one or more countries kind of steps in and takes over the AI lab. I understand in this moment that that feels like a kind of sci-fi scenario, right? Like, most of the time when you're using ChatGPT, you probably don't think this is a dangerous superweapon and we need to ensure that, you know, this is being controlled by the president. At the same time, we are now at war with Iran. We know that these systems are embedded in the like command and control operations of the military, and so to some extent, they are already becoming weapons, right?

• 19:29Speaker 02

  So if you say to me, do I think that once these systems become three, four, five, 10 times more powerful, the government will want to take an interest in them and potentially oversee their development and deployment? I absolutely believe that will happen. I see no reason why that would happen. And unfortunately, how that goes, I think, depends a lot on the quality of the government that is overseeing that AI, right? And like, what do they want to do it? Do they want to use it to create opportunity and safety and democracy for all? Or do they want to, you know, mount an authoritarian takeover of the globe?

• 20:06Speaker 01

  So if you are a leader at one of these companies and you know that, you know, at least until 2028, we are likely to have sort of the same administration in power. If you believe that the technology is rapidly accelerating such that a year or two years or three years from now, we might have something like a superhuman country of geniuses in a data center, what does that mean you should do? I mean, one thing that I've been thinking about is like, should these companies be doing deals with the government at all, right? If the lesson of the past couple of weeks is that the federal government is not a trustworthy counterparty in these negotiations and is going to insist on total control and obedience, or else they're going to try to nuke your company? Like, I think a very rational response from these AI companies will be like, well, we're just not going to make any more deals with you.

• 20:51Speaker 01

  You're going to have to use some open source models for your State Department and your military and your treasury, because it's just too risky for us as a business risk, and you can't be trusted with it.

• 21:02Speaker 02

  I could see why that may seem somewhat rational to them, but like, I don't think that that is the tack that they're going to take. I mean, even this week, after everything that has happened with Anthropic, Dario Amade is still out there saying, "We were very close to an agreement with the Pentagon. We liked working with the military. We want to work with the military again." Right? So I think that's very important to note. Like, Dario did not like throw up his middle fingers like on his way out the door. He is still trying to reach some sort of agreement, and I think in part that likely is to avoid the exact sort of scenario that you are describing, right? It's you kind of want to like, keep the tigers at bay for just a little while longer at least, while you maybe like, think through the rest of that scenario, which is admittedly a very difficult one.

• 21:43Speaker 01

  Yeah. I've been rereading The Making of the Atomic Bomb this week, which is Dario Amade's favorite book, and he used to give it to all Anthropic employees, and there's still like a bunch of copies at their headquarters. It's sort of the company book, as far as their mission goes, and they see a lot of parallels between what they're building and the Manhattan Project. And so I went back and I've been rereading it, and the piece that struck me from that experience was just right before the bombs were dropped in 1945, there was this point where the scientists got really worried about how Their creation was going to be used and a number of them from the Manhattan Project sort of created these petitions and reports and tried to get them to the government and say like,

• 22:25Speaker 01

  "Hey, could you guys like not use this against a city, at least as like a first line, you know, act of war?" And the military and the government sort of like pretended to hear them out and then they just went ahead and bombed Japan anyway. And there was sort of this moment where it was like, "We hear you. You're the scientists. You're the geniuses who made this all work, but now you're playing in our turf and so we're going to control the technology from here and like, thank you for your input." And like, I think the comparison between the Manhattan Project and the AI industry is somewhat overstated and I think it breaks down in some key ways, one of which is like, "That was a government project. You know, the Manhattan Project was paid for by the government. These were government employees. What we're talking about now are private companies that have been developing this thing outside the public sector."

• 23:09Speaker 01

  So I think there's some important differences, but I do worry that we are headed toward a moment where this stuff just gets so useful to governments and militaries and confers such a decisive advantage to the countries that control it, that the US government, no matter kind of who is in power, is just going to say like, "This thing is too important to be left to the private sector."

• 23:27Speaker 02

  Well, I mean, keep in mind that one of the original ideas for OpenAI was that it should be a government-funded project, but Sam Altman and his co-founders just came to the conclusion, correctly by the way, that no government would give them the amount of money they needed to build this technology, right? And, uh, you know, they just sort of quickly came to the conclusion that it was just going to have to be a private enterprise. But, you know, going back to the earliest days, there was thinking, among the people that created this technology, that the government was going to take an interest in it eventually. Another reason though, Kevin, why I find the current situation so vexing is that you and I both covered President Biden's executive order on AI, which I personally felt like was a pretty gentle way of attempting to regulate the industry.

• 24:06Speaker 01

  Very gentle.

• 24:07Speaker 02

  It was sort of like, you know, "Inform us about your safety testing, please, when you test these new models" and sort of, you know, told federal agencies to get ready for this technology. And the howls of protest on the right that said, "How dare, you know, this administration come in and try to put these fetters on capitalism? We are going to lose to China." Because of this sort of nanny state behavior. And then to see those same people come to power and now say, "We are going to tell you exactly how you are going to build your models, what they're going to do for the military, or else we will destroy you." Is ju- like, the whiplash is insane.

• 24:45Speaker 01

  Yeah, we didn't like that government trying to control the tech industry. But this government tried to control the tech industry. That's just business as usual. That's fine.

• 24:51Speaker 02

  Right.

• 24:52Speaker 01

  So I guess my worry, zooming out from all of the stuff that's been going on for the past two weeks, is that we are sort of living through like an early dress rehearsal for what something like nationalization of the AI companies could look and feel like. I don't think it's going to be as sort of cut and dry as like it was during World War II, where like the government showed up to a bunch of like steel plants and was like, "Hey, we run these now." I think it's going to be kind of this soft nationalization like we've been seeing over the past week, where it's like a little pressure to build your models differently. "Oh, maybe could you remove some of those safeguards? Oh, maybe this is actually so strategically important that we need to be the people putting the clauses in the Constitution of Claude

• 25:34Speaker 01

  or whatever that dictate how it will behave in these high stakes situations." And I think that is a more likely direction, but I would not take full sort of like brute force nationalization off the table entirely. I think there's a decent chance that something like that happens.

• 25:48Speaker 02

  Hmm. Well, maybe we should set up a prediction market for it.

• 25:52Speaker 01

  Speaking of prediction markets, when we come back, we'll talk about how prediction markets have made it to war.

• 25:58Speaker 02

  So predictable.

• 26:23Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data. Data, governance, infrastructure and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner. Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 26:57Speaker 03

  Modern enterprise is made up of a lot of moving parts. And Comcast Business helps you orchestrate it all. With SD-WAN working at scale to keep 150 hospital locations connected and working as one. Plus, SASE and Zero Trust Security protecting financial data across a bank's 2,000 branches. And AI-powered networking that optimizes traffic across five continents. No one does business like Comcast Business. AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails by team and model, helping ensure spend goes to the highest ROI areas.

• 27:43Speaker 03

  Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 27:59Speaker 01

  Okay, Casey, so the other big news from the past week is that the United States is now at war in Iran. And one angle that really has been sticking out to me about this is the role that prediction markets are playing in this conflict, because I think that is something that we truly have not seen before.

• 28:17Speaker 02

  Yeah, it seems like every new war brings along some grim new technology. And I would say that prediction markets are maybe grim technology number one for this conflict in Iran.

• 28:29Speaker 01

  Yes, it's a grim technology already, even absent the war. And now, just with the war, it has become even grimmer. And we've talked about prediction markets on the show. We talked about them way back in 2023, when they were sort of this new thing that was like kind of in this legal gray area that wasn't really. Being done at any scale yet. It was sort of an interesting idea. Now, of course, you cannot walk down a street in a major American city without seeing one and probably multiple ads for prediction markets like Kalshi and Polymarket.

• 28:58Speaker 02

  Yeah, the sort of gambling mania that has taken over all media and advertising, you know, from DraftKings to FanDuel, has now extended even further into these prediction markets.

• 29:10Speaker 01

  So both Polymarket and Kalshi, the two leading prediction markets platforms, took a lot of heat this week on bets they were allowing their users to make on questions related to Iran. So Kalshi, which is kind of the more regulated US-based prediction markets company, does not allow bets on war or assassination, but it did allow the question Ali Khomeini out as supreme leader, basically sort of as a kind of careful proxy for betting on the outcome of a war or a strike on Iran.

• 29:42Speaker 02

  Yeah, and out, I suppose, could have, you know, many meanings, you know? Perhaps there would be a sort of gentle democratic revolution in Iran, but I'm going to assume that most of the people who were wagering on that one assumed that he was going to be killed in war.

• 29:57Speaker 01

  Yeah, so people got really mad at Kalshi for allowing these bets on the fate of the Iranian leader. They also got mad when Kalshi sort of voided this market and said that it was going to reimburse anyone who may have lost money on this, basically make sure everyone ends up in the black. But people who were supposed to make a bunch of money because they correctly predicted the death of Khomeini were mad that they didn't get paid out their expected winnings. So just a big cluster all around.

• 30:25Speaker 02

  And I just want to say, if you were one of the traders who did not get your expected winnings from the death of the Ayatollah, I just want to say, I don't care, and it doesn't matter.

• 30:36Speaker 01

  So Polymarket, the other sort of less regulated offshore crypto-based prediction market, was even more permissive. They allowed people to bet on the dates of strikes on Iran and other details related to the war in Iran.

• 30:50Speaker 02

  Their policy was really like, imagine the worst thing you could do on our platform. You can do that.

• 30:56Speaker 01

  Actually, they did draw a line when it came down to markets that allowed users to bet on the likelihood of nuclear detonations by specific dates. So, sorry to anyone who is trying to cash in on nuclear war.

• 31:09Speaker 02

  These woke liberals that won't let me bet on nuclear explosions need to go, Kevin.

• 31:14Speaker 01

  Okay, so no one was happy about this. Senator Chris Murphy posted that, quote, "It's insane this is legal. People around Trump are profiting off war and death," and also said that he was introducing legislation to ban this. And there are also a bunch of people looking into whether any of this has been done via insider trading. Basically, do you have people in the military or close to the decision makers in this conflict placing bets once they have this sort of non-public information about what is going to be happening?

• 31:45Speaker 02

  Yeah, and I think it speaks to why allowing prediction markets to take bets at least around sort of like, you know, war and death is so corrosive and bad, Kevin, because not only is it just kind of like grim and like, how do we live in this society where, you know, gambling on war and death has become a sort of form of entertainment, but also you're just creating incentives for like, the worst things in the world to happen, which doesn't seem logical to me.

• 32:15Speaker 01

  Well, and it's not even a theoretical harm here. Recently, Israel arrested a number of people who were accused of using classified information to bet on military operations on polymarket. So this is already starting to happen, and I think this is why people like Senator Chris Murphy are so alarmed about this, not just because it's sort of like gross and aesthetically offensive to have people betting on wars.

• 32:40Speaker 02

  Although it is.

• 32:41Speaker 01

  But, although it is, but also because it could create direct incentives if you're a member of the military and your commander gives you an order to go do an airstrike on an Iranian compound, to log onto your phone and head over to one of the prediction markets platforms and say, you know what, I could make a couple grand off this.

• 32:59Speaker 02

  Yeah, that's your little call sheet bonus. You know, this is not theoretical at all, Kevin. In fact, your colleague, Amy Phan, at The Times, wrote that it is relatively uncommon for someone to bet a significant sum of money that a US strike will happen within the next day, but last Friday, more than 150 accounts placed hundreds of bets of at least $1,000 correctly predicting that there would be an American airstrike on Iran by Saturday.

• 33:23Speaker 01

  Yeah. So I think one of the interesting things here is like, I am not like a blanket opponent of prediction markets, right? I sort of bought some of the kind of theoretical arguments for why something like a prediction market could, for example, outperform political polls because it would incentivize people to like, come up with really good polling data and like, use that to trade on and you could end up with kind of a better picture of a given election.

• 33:49Speaker 02

  Or people will like, say what they really think because their money is at stake and they're not just trying to like, impress a pollster.

• 33:55Speaker 01

  Yes. And you've actually had some of the people who are in charge of these prediction markets sort of talking about the fact that insider trading can be good because it can get the best information to the markets as quickly as possible and kind of like, give people an unfiltered understanding of what the real insiders are thinking. Now, of course, officially, you are not supposed to be able to insider trade on these platforms, right? They all have policies against it. Kalshi says, you know, they've investigated people, that it is actually illegal per the CFTC, which is their main regulator, to place bets using inside information. But there are a couple of problems with this. One is, the CFTC is a tiny agency. It doesn't have a huge team of enforcers going out to investigate

• 34:39Speaker 01

  what I assume must be hundreds or thousands of trades using inside information on their platform every day. It's also not clear what is public information and what is private information. You know, there are certain types of information in the stock market that are considered material, non-public information that it is illegal to trade on. But it is also legal to, you know, fly a drone over an oil facility to see how their production is going, or to park outside a store and see the foot traffic going in and out and use that to sort of calculate how well their sales must be going.

• 35:12Speaker 02

  I find it suspicious how much you know about the insider trading rules. I have to say, I didn't know you had this much facility with the law here.

• 35:19Speaker 01

  I'm calling my lawyer. But of course, this is part of the appeal of prediction markets in general, is that they incentivize people with good information to trade on that information.

• 35:28Speaker 02

  Yes. And if you allow people to wager on almost anything, how are you ever possibly going to? Police the entire platform to understand who is insider trading and who isn't.

• 35:37Speaker 01

  Yes. So I think in this specific case of war, I think it's very dangerous for some of the reasons that we've talked about. Not only do you have military officers and service people disclosing classified information in some cases to to sort of make a little extra for themselves, but you also have just this incredibly strange war profiteering innovation where you like, you can just go on one of these platforms and try to make a bunch of money from something that involves a lot of devastation and destruction.

• 36:09Speaker 02

  You know, the other thing that comes to mind for me, Kevin, is that, you know, as you say, the prediction market backers, their argument is like, this just helps us understand the world better, right? This is a new kind of information that helps us see more clearly. And yet, as I look across all of the trades that you just described, I don't understand really what I was supposed to see more clearly, right? Like, maybe you get a, you know, a brief heads up about something horrible that is about to happen. Maybe that's, you know, useful in at least some circumstances. But for the most part, I just don't feel like we actually have a much better understanding of the world because all of these bets are happening.

• 36:45Speaker 01

  Yeah. And I think in this specific case, that's especially true because if you actually look at the markets that were being traded before this strike on Iran, the conventional wisdom of the crowd was that this was not going to happen. It was a very low probability. I think something like 17% probability on one of these platforms an hour before the strikes. So these markets aren't actually distributing the best possible information at all times. They're just kind of like aggregating vibes until like someone with inside information shows up and like makes a fortune.

• 37:19Speaker 02

  Well, I think that's exactly it. It isn't as if these have been adopted by the mainstream and everybody's placing these sort of casual bets. And now we have this like beautiful, perfect understanding of the world. What we have, as you say, is a bunch of vibes plus some insider trading. And it just doesn't actually seem that useful to me in practice for most things.

• 37:36Speaker 01

  Yeah, I want to try to like sort of steel man the defense of prediction markets here and see what you make of it. So I think someone who Who believes that these prediction markets are good in the aggregate might say something like the following: People have been betting on war forever. They bet on the stock prices of defense companies. They bet on things like oil prices. That is all legal. We consider that sort of part of the normal markets. Those things all fluctuate when you have a war breakout. How is this any different? Your response?

• 38:08Speaker 02

  Well, I think that it is actually really meaningful that these are indirect ways of betting on war, right? It seems very unlikely to me that if I like, you know, buy oil stocks, assuming that they are going to go up, that I'm creating an incentive for somebody to assassinate the supreme leader of Iran.

• 38:28Speaker 01

  But wasn't this the whole conspiracy theory about the war in Iraq? Was that it was just motivated by like Dick Cheney owning a bunch of stock in Halliburton?

• 38:34Speaker 02

  Well, I mean, yes, that was like the conspiracy theory. You know, I don't know that that was what was actually driving. And I think that, you know, as with most wars, at least at that time, there were sort of like a number of interrelated factors that were going on. And, you know, maybe oil was one of them. But my point here is just that when you have the betting at some sort of meaningful remove from the action, it just like feels better for me. It doesn't create the same horribly grim incentives that this particular approach does.

• 39:03Speaker 01

  Right. I think the difference for me is the directness that you mentioned. And, you know, one thing that came up over and over again, when I was talking to people about prediction markets a couple of years ago for this story is the assassination markets get really dark. Because if you have something like, you know, will this world leader, you know, be removed from power in air quotes?

• 39:23Speaker 02

  Wink, wink.

• 39:23Speaker 01

  Before a certain date, that could actually create a bounty on that person where someone might go out and say, hey, if I want to make money on this, I need to like kill this person before this date.

• 39:34Speaker 02

  And you know what is going to be the first thing that actually takes action on that, Kevin? Open claw. Mark my words, one of these bots plugged into a Mac mini is gonna see a prediction market for the assassination of a world leader and it's gonna say, well, I have some ideas about that.

• 39:49Speaker 01

  So I think everyone, most people agree that like the assassination prediction market is sort of, you know, out of bounds and is a bad idea for lots of reasons. But I think there is still a lot of gray area around these questions about conflict and war and politics. And I think it is, the, the risk here is that these prediction markets have gotten so popular so quickly with so little regulatory oversight that it is just kind of legal to do a bunch of stuff on them that it's not legal to do in the regular stock market.

• 40:19Speaker 02

  Yeah. Well, so you mentioned that some lawmakers have talked about introducing legislation. My experience is that that kind of legislation typically doesn't go anywhere. What, if anything, do we know about what is going to happen as this war continues to unfold in Iran when it comes to these prediction markets?

• 40:36Speaker 01

  I mean, I think the Trump administration is very unlikely to do anything to sort of stop the growth of prediction markets. We've already seen them signal via these sort of regulatory actions that they've dropped against polymarket that they are not going to take a firm line against these prediction markets. We've also seen them adding members of the Trump family to their advisory boards. So I think all of these prediction markets are sort of becoming entangled with the administration in ways that are going to make it very hard for them to do anything. But I certainly expect like Democratic lawmakers to stand up and say like, what the hell are we enabling here? Why are we allowing people to bet on the assassination of world leaders or the outcomes of a war in Iran? This just feels all incredibly fraught to me.

• 41:20Speaker 02

  Hmm. My, my fear is that we're in a sort of time race where like if Democrats were able to like somehow advance some legislation, maybe they win some seats in the midterms, maybe they retake the presidency, maybe sometime within the next few years, they could meaningfully rein these prediction markets in. I think though, if they continue to grow, my fear is that they will become a massive entrenched interest group like the crypto world and they will then lobby to ensure that Democrats and Republicans both feel like they have a vested And interest in these things sticking around. So, you know, I, my fear is that if we're to do anything about some of these excesses we've been talking about today, it needs to happen soon. Or otherwise, platforms like Call She and Polymarket might just have too much money for that to happen.

• 42:05Speaker 01

  Yeah, I have a proposed rule for these prediction markets.

• 42:07Speaker 02

  What's that?

• 42:08Speaker 01

  Which is that you should have to go to a physical place like you do for a casino. I think that putting this stuff on people's phones, making it super easy for them to do it, like, if you want to go bet on the war in Iran, you should have to, like, go to a seedy, like, OTB betting place to do it. Like, you should have to, like, put in some effort. It should not be as easy as whipping out your phone.

• 42:30Speaker 02

  All right. Well, it's very interesting, Kevin. I predict we are not going to try that.

• 42:34Speaker 01

  No, I also predict we're not going to try that. But it's a good idea. People should listen to me.

• 42:43Speaker 02

  When we come back, slop, collaborate, and listen. YouTube is back with a disturbing new invention.

• 43:15Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner. Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 43:49Speaker 03

  Modern enterprise is made up of a lot of moving parts, and Comcast Business helps you orchestrate it all. With SD-WAN working at scale to keep 150 hospital locations connected and working as one, plus SASE and Zero Trust Security protecting financial data across a bank's 2,000 branches, and AI-powered networking that optimizes traffic across five continents. No one does business like Comcast Business. AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails by team and model, helping ensure spend goes to the highest ROI areas.

• 44:35Speaker 03

  Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 44:51Speaker 01

  All right, Casey, well, it's time to look at some kids' slop.

• 44:53Speaker 02

  Yeah, Kev, we have recently been alerted to the fact that YouTube has been beset by a bunch of AI-generated slop for toddlers, and it's time to take a look and see what we're dealing with.

• 45:03Speaker 01

  My colleague at The New York Times, Arietta Laka, has a new story about this called "How AI-Generated Videos Are Distorting Your Child's YouTube Feed." And I read the story, I loved it. I thought, I've got to see some of this slop for myself. So today, let's take a look at this emerging new genre of AI slop directed at kids with a new installment of the Hard Fork Review of Slop.

• 45:39Speaker 01

  All right, what do we have up first? First is a video about the alphabet. Do you know the alphabet?

• 45:45Speaker 02

  You know, I keep meaning to learn it.

• 45:46Speaker 01

  Okay, let's see how the AI-generated alphabet videos are doing.

• 45:50Speaker 02

  Yeah.

• 45:52Speaker 04

  Kids TV. ABC Farm Animals.

• 46:00Speaker 01

  So far, so good.

• 46:06Speaker 01

  Okay, okay.

• 46:07Speaker 02

  Why aren't they being squirted out of a little paint bottle?

• 46:16Speaker 01

  Oh, I'm deeply uncomfortable with this.

• 46:22Speaker 01

  I'm gonna have to answer so many questions from my three-year-old about why ducks come out of toothpaste tubes.

• 46:26Speaker 02

  Yeah, this is, if you're, listen, if you're three years old, you need to know this. That is not how a duck is made. So, this video depicts the alphabet showing a series of animals whose names begin with a letter. And then, a hand holding, I guess, a tube of paint, sort of squirts out a little dollop of gross goo that then transforms surreally into an animal, while sort of demented slop children sing the name of the animal in the background.

• 46:57Speaker 01

  Yes, like if you've ever seen the TV show Alex Mack from like the 90s on Nickelodeon where she like sort of assembles herself out of goo on the floor, it's sort of like that but for animals.

• 47:06Speaker 02

  That one was a little before my time, I'm afraid, but I'll research that one back in the archives. In any case, okay, yeah, very strange one. Let's take a look at this next video, Kevin.

• 47:17Speaker 01

  What do we have now?

• 47:18Speaker 02

  I believe we're gonna see some animals appearing out of colorful clouds.

• 47:43Speaker 01

  Rolling the Rs on his roars. Why are they, why is the other animal a doctor? Why are they running toward the camera? Why is the koodoo pink?

• 47:53Speaker 02

  And what is a koodoo? Now, I have to say, I'm, I'm a little older than the, the target age for these videos, but I'm learning things myself.

• 48:01Speaker 01

  This is for you.

• 48:02Speaker 02

  I'm learning. So, this video shows another series of animals, each one connected to a letter of the alphabet. But this one uses a trope of having this doctor figure inject these animals with color.

• 48:17Speaker 01

  And the doctor is also an animal.

• 48:18Speaker 02

  And the doctor is also an animal.

• 48:22Speaker 01

  Is this like anti-vax propaganda?

• 48:25Speaker 02

  Well, the thing is, like, whoever is making the slop knows that needles are scary to children. So this is effectively just an engagement hack, right? You know, the kid is going to watch the injection because the kid is afraid of needles. So this is just, again, this is just one of these like little miniature insidious ways that these like slop makers grab the attention of kids is by like showing them something scary in order to hypnotize them into continuing to watch the slop.

• 48:49Speaker 01

  It does teach the valuable lesson that giving injections to children does result in them turning into pink kudos and running toward the camera. So all children need to learn that lesson eventually.

• 48:59Speaker 02

  All right, this next one, Kevin, shows animals turning into armored vehicles, trucks and planes, going to show once again that a key theme in slop for kids in 2026 is the transformation of animals.

• 49:16Speaker 01

  Oh, wow. We got sort of Mecha Quail.

• 49:23Speaker 01

  Okay.

• 49:27Speaker 01

  Okay, I wonder if that's a licensed use of the Thomas the Tank Engine IP?

• 49:37Speaker 01

  Okay, unfortunately, that one slapped for me. My kid would be super into that, and I must never show him that.

• 49:43Speaker 02

  This one reminds me of like cartoons that I watched as a kid where there was a lot of, you know, Transformers transforming. And so I could understand why, you know, a lot of little kids might like that one. But, you know, this is making me realize that another reason why slop makers love the alphabet is because they can just stitch together a bunch of very short clips. And of course, most of the AI video generators that we have in this moment can only generate clips of up to a few seconds. So the alphabet just becomes a perfect way to stitch together all of that into one piece that still at least has some sort of coherence.

• 50:18Speaker 01

  Yeah. I mean, I guess I can see a world in which like this stuff is not actually that harmful. Like, maybe it is gonna teach some kids the alphabet, but it is just so weird. And it strikes me as just like, what, like, we already had a- A lot of videos teaching kids the alphabet. Why are people doing this?

• 50:35Speaker 02

  But also, you know, listen, I'm no child neurologist, but I wonder about the consequences of essentially just creating images that are designed to overstimulate a child. You know what I mean? It's like, at least when you're watching like a normal cartoon, there could be moments of relative calm or like a story might unfold over a few minutes. When you're just showing like raw visual stimuli and bombarding a kid with it, it just doesn't seem like it's probably that good for them.

• 51:04Speaker 01

  This is how I know you do not have a child

• 51:06Speaker 02

  Yeah.

• 51:06Speaker 01

  in the year of our Lord 2026. Because if you go onto like Cocohellen or any of the other like extremely popular children's programs, like they are essentially just this. It's like a series of very short clips. Maybe they do one song and then they do another. It cuts away to a different song, cuts away to a different song. It is like these same sort of like hyper stimulating environments. So like, I'm not saying that any of this is good. I'm just saying that like, we have crossed the Rubicon a while ago and we are now in the land of the hyper stimulating children's entertainment. And like, the big difference to my mind is now that it's just easier and cheaper to create this stuff.

• 51:40Speaker 02

  Listen, this is my takeaway. When I have my child, which I do hope to do, the only visual stimulation that I'm going to allow them is a pile of sticks.

• 51:50Speaker 01

  And etch a sketch?

• 51:51Speaker 02

  They can, they can make their own fun, you know, with the pile of sticks, but we are not going down this road, Kevin. All right. Why don't we wrap up with a lullaby about animated children tucking themselves into beds made out of fruit?

• 52:04Speaker 01

  Hmm. Love a fruit bed.

• 52:08Speaker 02

  What'd you call me?

• 52:13Speaker 01

  Okay, we've got a girl with a crown going into a bed made of strawberries.

• 52:22Speaker 01

  Banana bed.

• 52:25Speaker 01

  Kiwi bed.

• 52:35Speaker 01

  Casey, what do you make of the fruit beds?

• 52:37Speaker 02

  You know, this is just another one that feels a little more surreal than I am comfortable with. You know, it's like, what is the child supposed to conclude from any of it? Like, it just sort of seems designed to confuse children more than to educate or even really entertain them.

• 52:54Speaker 01

  Yeah, and there have been, I will say, like, a few sort of videos in here where I'm like, oh, my kid would like that. But I don't like that he would like that. You know, it's like, I don't want him watching stuff that's just like a bunch of animals like driving buses or lying in, you know, fruit beds.

• 53:10Speaker 02

  Also, kids like lots of things that are bad for them. That's why we don't let them use cocaine.

• 53:16Speaker 01

  Yes. Well, now that we've seen the videos, I feel like we are armed and ready to have the conversation about what they all mean. So let's bring in my colleague Ariadaleika.

• 53:24Speaker 02

  I don't know. I feel like I need to cleanse first. Maybe go read a play by Shakespeare or something.

• 53:29Speaker 01

  Just go watch an episode of Bluey. Very heartwarming.

• 53:37Speaker 01

  Ariadaleika, welcome to Hard Fork.

• 53:39Speaker 05

  Thank you so much for having me.

• 53:41Speaker 01

  I wanted to just ask, like, first of all, how did you get interested in this? How did you learn about this? Did you get a tip from a toddler submitting things through the New York Times Confidential Tip Process?

• 53:51Speaker 01

  How did this come onto your radar?

• 53:52Speaker 05

  Yeah, so no toddlers were involved in the making of this story. We had some ethical concerns there about showing them these videos repeatedly. But I started by getting interested in this because there is so much AI slop on all of our social media feeds, regardless of the platform. And I was interested in learning more about how this content is being presented to children and how it's being moderated. And so where do many children watch media? It's YouTube. So I started by looking at channels that parents approve of, channels like Miss Rachel and Bluey that are considered more high quality and thoughtful, and other popular channels like Cocoon Melon, and said, well, if I click on a Cocoon Melon video...

• 54:37Speaker 05

  And I am a toddler and I look at the recommended videos right next to that video. What sort of videos would I be recommended? And we focused on YouTube Shorts because a lot of these AI tools default to this vertical format. And so we just started scrolling through the feed and making a note of the different videos we were seeing. One of my colleagues actually coded a tool where I wasn't really interacting with the screen. So wouldn't influence the algorithm and I was just like scrolling down and it was making a note of all the different links I was seeing. And then we later analyzed these videos frame by frame, determined which videos were AI generated.

• 55:14Speaker 02

  And we're using regular YouTube or YouTube Kids?

• 55:17Speaker 05

  So we started looking at regular YouTube in a private browser because a lot of parents tend to put on their own YouTube accounts for their children. But then we also look through YouTube Kids, you know, because it is a more controlled environment with content that's supposed to be approved for for children.

• 55:35Speaker 02

  And when you started this process, what were your sort of expectations? And then how much AI generated stuff did you wind up seeing as like kind of a proportion of the total?

• 55:44Speaker 05

  So I think especially when I was watching channels like Miss Rachel and Bluey, I was expecting to see content that that would be, you know, more in the lines of of those programs, right? More Bluey shorts and I wasn't really seeing that. I was surprised just about how much AI generated content was being recommended. Like in one session alone, in one 15 minute session that we were scrolling, more than 40% of the videos were AI generated.

• 56:11Speaker 02

  Wow.

• 56:11Speaker 01

  And how are you determining that? Is there like a detection system that you're plugging these things into?

• 56:16Speaker 02

  Yeah, the detection system is called looking at the videos.

• 56:19Speaker 01

  Well, no, I'm saying like if you have a video of like a, you know, an alpaca getting squeezed out of a paint tube like that's very clearly not photorealistic, but it could be made using some like CGI software or something like that.

• 56:30Speaker 02

  I suppose, yes.

• 56:30Speaker 01

  How did you determine that this was AI?

• 56:32Speaker 05

  Right, so I think some of these videos were a little bit more obvious, but we looked at them frame by frame and tried to look for inconsistencies. Like in some cases, the objects were disappearing in the background. There was text in the background that was distorted. If there was an animal in the background, it would morph into something else. Some of the videos were actually labeled as synthetic media, even though they were animated. Um, and then we would also look at the YouTube channel and get a sense of like, okay, what sort of content have they been posting in the last few years? And even some of the channels that had been around, you know, before the last two years where this technology really improved. They were making very simple, low-quality animation that looked nothing like what they had been posting the last few months.

• 57:13Speaker 01

  Now, Arietta, I have done a lot of reporting on YouTube, but it's been a few years. What are their actual policies like today on this kind of content?

• 57:23Speaker 05

  So I sent them a few channels as examples. And I asked them what their policy is around flagging AI content that's being made for children, and that includes animated content. And they told me that content creators are required to label content that is realistic looking. In some cases, we saw that creators weren't labeling the content, and I'm referring to videos of animals, you know, displaying behaviors that those animals don't do, like elephants doing gymnastic maneuvers on a tight rope. Those videos, for example, are not labeled as synthetic media. Um, some creators were labeling the animated videos, but it's not part of YouTube's policy.

• 58:02Speaker 05

  So it's really, the burden is falling on parents, especially for parents who may not want their child to watch AI-generated media to determine, okay, what sort of video is this? And not only that, but you have to click through the video and then you see the synthetic media label. And when it comes to kids' content on YouTube, YouTube doesn't allow comments. So sometimes when there's an AI-generated video, you'll have people in the comments say like, this is AI. And in this case, you know, parents can't really talk amongst themselves as to what the video is.

• 58:33Speaker 02

  Has there been any backlash yet that we've seen from parents against these kinds of videos? Like, are some parents upset by what they're seeing?

• 58:42Speaker 05

  Yeah, we came across several Reddit forums where people were asking how to get around this AI slop for kids, whether there was a filter on YouTube. Some parents recommended making a playlist, other parents were like, "Get off YouTube altogether." And I think a lot of it is also going to fall on parents to closely supervise what kids are watching, since the algorithm is pushing so much of this content.

• 59:06Speaker 02

  Right, although, of course, the whole point of YouTube is that it's what you show your kids so that you don't have to spend time with them, right? You know, and not in a mean way, but like, maybe you have to cook dinner.

• 59:13Speaker 05

  Right, parents are busy.

• 59:15Speaker 02

  Yeah, like, if parents were, you know, sitting down to like, watch YouTube with their children, like, we would not be in this situation.

• 59:20Speaker 01

  Yeah, and I'll say, like, as the parent of an almost four-year-old, like, the couple times that I've, like, found him watching something like this, because he, like, you know, went on to the YouTube Kids app when he wasn't supposed to or something, my feeling about it is not like, "This is damaging my child." It's just like, "This is so bad."

• 59:38Speaker 02

  Yeah.

• 59:38Speaker 01

  Like, the quality is just so bad. I wonder if you think, Arietta, like, there's any sort of evidence? What do we know about the actual effects of this on kids? Is the worry that it's actually harming them? Or is the worry that there's just better stuff they could be watching?

• 59:52Speaker 05

  So there's a lot of factors to consider. So these videos contain extraneous effects. And we know that when videos contain all these bedazzling elements, kids can learn as well. They're also devoid of a narrative arc, which is really beneficial for kids to watch content with the beginning, middle, and end, with characters they can relate to, that they're familiar with, watching content that uses short phrases that they can understand and doesn't feature these abstract concepts. You know, worst case, a lot of these videos are fantastical, and that could be cognitively overloading to the child. And when it comes to short form content, experts say for children under five, we know their attention systems are still developing. So it's hard for them to follow rapid changes.

• 1:00:37Speaker 05

  And it really puts a heavy burden on children to process that information, especially when we we see this more realistic fantastical content of animals, you know, showing bizarre behaviors.

• 1:00:48Speaker 02

  You know, when I was growing up, Kevin, in the 1900s, I would watch He-Man. Remember He-Man?

• 1:00:53Speaker 01

  Yes.

• 1:00:53Speaker 02

  It was a cartoon about a gay guy who had a sword and rode a tiger around. And as a young-

• 1:00:58Speaker 01

  Was he gay canonically?

• 1:00:59Speaker 02

  Well, I think not canonically, but read between the lines. Okay, read between the lines. And, you know, as a young gay kid, I could see that and relate to that. And it, you know, and yes, the stories were fantastical. You know, but to Arietta's point, there was a beginning and a middle and an end. And I, you know, it didn't exactly like teach me in the ways of the world. But like, it maybe told me something about like storytelling and narrative. And it wasn't just sort of like raw flashing shapes and colors.

• 1:01:22Speaker 01

  Yes. I mean, I remember before I had a kid, having this argument with people at YouTube about YouTube Kids, because there was, I think there was, you know, discussion and concern even back in like 2017 or 2018 about these weird videos. They weren't AI generated at the time, but they were like weird sort of cocoa melon rip offs. And I remember saying to these YouTube people, like, you guys gotta change the way that you do this. Like, instead of just default allowing channels to serve content to very young children, you have to like kind of set up some whitelist where like, if you prove that like your stories have a beginning, middle and end, and like there's some thought being put into them, you are allowed to be on YouTube Kids. And if you're not, if you're just some like content farm that's like churning out cocoa melon rip offs,

• 1:02:06Speaker 01

  like you're not going to be allowed on. So do you get the sense, Arietta, that there's any like concern or consideration of this at YouTube? Or are they basically just sort of like, you know, we got bigger fish to fry?

• 1:02:18Speaker 05

  I mean, these videos are recommended to me and these channels multiple times. And I I've been looking at these videos since November, so.

• 1:02:26Speaker 01

  And what is it under your brain?

• 1:02:28Speaker 05

  And

• 1:02:28Speaker 01

  I bet you know your alphabet.

• 1:02:30Speaker 05

  I do, yeah, my, the alphabet and you know, those songs really get stuck in your head, right?

• 1:02:36Speaker 01

  Oh, trust me, I know.

• 1:02:38Speaker 05

  And then it's just so, it's so fascinating to me how much more of these videos are recommended to me than more thoughtful content like PBS Kids, for example, because PBS Kids also puts out shorts, but I was seeing more of this than PBS Kids.

• 1:02:50Speaker 02

  I mean, just to sort of like, you know, echo Kevin's question, this has gotten me thinking about Elsa Gate, right? This 2017 controversy where parents see a lot of really surreal content in YouTube and they get very upset. It seems like we have not seen that kind of backlash yet. And I'm wondering, is it because we just feel like the content is actually better here in some ways? Or are most parents just not actually aware of just like the volume of slop that might be being served to their kids?

• 1:03:18Speaker 05

  Because so many of these videos are animated, it may not be obvious to some parents that these are AI-generated. But we were working in a more contained environment. So like else were on YouTube. There are so many videos featuring popular characters that children like in very violent scenarios. There's this show called Masha and the Bear, and I remember I typed in Masha and I was coming across videos of Masha's stomach being cut open. And, and that channel was putting up these videos and YouTube didn't take them down until I linked out to one of them in my story. And I don't know if any of you know children who are obsessed with that K-pop Demon Hunters film.

• 1:04:00Speaker 05

  But there are so many horrible videos featuring those characters.

• 1:04:06Speaker 05

  And again, it's nonsensical. Sometimes the characters are pregnant and those are all over YouTube as well. If you type in those characters' names.

• 1:04:14Speaker 02

  And by the way, that was the exact scandal of Elsa Gate, was that people were seeing Elsa from Frozen pregnant all over YouTube. So I mean, like this truly is just a sequel to Elsa Gate. And I think it's, you know, interesting that it has flown under the radar until your story, Arietta.

• 1:04:28Speaker 05

  Right. And before, I mean, you had to have some sort of animation skills to do that. But like now anyone can do this in a few minutes. So, you know, we're gonna, we're gonna be seeing more of this content.

• 1:04:39Speaker 02

  And again, what is so insidious here is that the people who are making these videos are counting on the fact that regular kids are gonna go searching for K-pop Demon Hunters and Masha the Bear, and they are unwittingly just going to be served this slop of like, you know, the poor bear getting cut open and the pregnant demon hunter, just through the recommendation algorithm. And like, this is where YouTube's responsibility lies, right? Like, this is where YouTube should be saying, "We're actually not going to show this incredibly disturbing stuff to kids."

• 1:05:08Speaker 01

  Yeah, I would say like, among the parents of young children that I know, like, very few of them let their kids watch YouTube, even YouTube Kids, for this exact reason. That it's like, you might start with Masha the Bear, you might start with, you know, a K-pop Demon Hunters video, you might start with Bluey or Cocohellen, but like, you know, you go away for 10 minutes to cook dinner, and you come back and like, they're on their third hop from the recommendation, and they're watching some like, you know, weird AI-generated thing with like Spider-Man, you know, on the surgery table or something like that. It's like, what are we doing here?

• 1:05:41Speaker 02

  Yeah, because it turns out that all of these recommendation algorithms work exactly the same way. They find the video that is the closest to going over the line, and they find that it gets more engagement than any other. And so that's what shows up in the feed, right? So, how, how many times are we going to have to revisit this story? It really depresses me, honestly.

• 1:05:58Speaker 01

  Ariadne, until the platforms like YouTube take more action on this, what can parents do in the short term? I mean, if you're going to let your kid watch YouTube or YouTube Kids, is there anything you can do to like, toggle off the AI slop, or do you just kind of have to make peace with it?

• 1:06:13Speaker 05

  In terms of filtering it out, there isn't an option to do that right now. I mean, in January, YouTube announced they were gonna add some controls where parents can set time limits to YouTube Shorts. We're seeing a lot of this AI content in YouTube Shorts, so that might be an option.

• 1:06:27Speaker 02

  They could call it the slop-watch. You know, it's like a stopwatch, but for slop.

• 1:06:32Speaker 01

  I just wonder if like, all of this is a losing battle. I feel like-

• 1:06:37Speaker 02

  Kevin Roos, do not bring your nihilism into the recording studio today.

• 1:06:40Speaker 01

  Well, just like, look at what is happening to older children with like, Tung Tung Tung Sahur and like, all the AI slop from TikTok. And I just feel like, you know, we're seeing old people falling for AI slop on Facebook and middle-aged people fall for it on X and Instagram. And like, it just feels like we are kind of creating this like lifelong pipeline of just like, whatever slop is going to be most engaging to you.

• 1:07:07Speaker 02

  Well, I'm glad you brought that up because that is the thought that I'm having as we wrap up, is that while, you know, right now, we're talking about the concerns we have for kids, I think that, you know, older, older teens and adults wind up having the same issue, which is, they open up TikTok or Instagram reels and they see something that is designed to hypnotize them as well. And it might not be colors and shapes and the alphabet, but it does seem like the basic idea is just sort of turn your brain off and experience the raw visual stimuli for as long as, you know, we can get you to.

• 1:07:37Speaker 01

  Yes. Well, Arietta, fascinating story. Thank you so much for doing this digging and watching all of these children's videos so that I don't have to and so I can protect my child from them.

• 1:07:47Speaker 05

  Thank you so much for having me.

• 1:07:48Speaker 01

  Way to take one for the team.

• 1:07:50Speaker 02

  Yes, Arietta, you really did take one for the team on this one, I have to say.

• 1:07:54Speaker 01

  All right, thanks so much.

• 1:07:55Speaker 05

  Thanks.

• 1:08:20Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner. Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 1:08:54Speaker 03

  Modern enterprise is made up of a lot of moving parts, and Comcast Business helps you orchestrate it all. With SD-WAN working at scale to keep 150 hospital locations connected and working as one, plus SASE and Zero Trust Security protecting financial data across a bank's 2,000 branches, and AI-powered networking that optimizes traffic across five continents. No one does business like Comcast Business. AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails

• 1:09:35Speaker 03

  by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 1:09:59Speaker 01

  Hard Fork is produced by Rachel Cohn and Whitney Jones. We're edited by Viren Pavich. We're fact-checked by Caitlin Love. Today's show was engineered by Chris Wood. Our executive producer is Jen Poyant. Original music by Alicia Beetoo, Rowan Nemestowe, and Dan Powell. Video production by Sawyer Roque, Pat Gunther, Jake Nickel, and Chris Schott. You can watch this full episode on YouTube at youtube.com/hardfork. Special thanks to Paula Schuman, Quiwing Tam, and Dahlia Hadad. You can email us, as always, at [email protected]. Send us your toddler slop.

• 1:10:58Speaker 03

  You can't undo everything in life, but you can undo prediabetes. More than 2 in 5 adults in the U.S. have prediabetes. Most don't know they have it. But with early diagnosis and action, you can delay or even prevent type 2 diabetes. Visit doihaveprediabetes.org to take the 1-minute risk test and start undoing the things you can. Brought to you by the Ad Council and the Centers for Disease Control and Prevention.