Does YouTube Radicalize? A Debate Between NYT's Kevin Roose and Software Engineer Mark Ledwich

In June 2019, New York Times reporter Kevin Roose wrote The Making of a YouTube Radical, a story about how a 26-year-old man, Caleb Cain, was radicalized through YouTube. For the story, Roose examined Cain’s entire YouTube history, and plotted the path he took toward radicalization. Software engineer and researcher Mark Ledwich took issue with the story, citing his own research and claiming the notion that YouTube could radicalize was a myth. Instead of yelling at — and past — each other, Ledwich and Roose came together for a moderated debate on the Big Technology Podcast, where both stated their points of view, got a chance to respond to each other’s points, and ask each other questions. Further reading: Kevin's story: https://www.nytimes.com/interactive/2019/06/08/technology/youtube-radical.html Rabbit Hole podcast: https://www.nytimes.com/2020/04/22/podcasts/rabbit-hole-prologue.html Ledwich's story: https://mark-ledwich.medium.com/youtube-radicalization-an-authoritative-saucy-story-28f73953ed17 Ledwich's research visualization: https://recfluence.net/ Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay?

• 0:43Alex Kantrowitz?

  Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 1:03Alex Kantrowitz?

  Hello and welcome to the Big Technology Podcast, a show for cool-headed nuanced conversation of the tech world and beyond. And today, we're trying something new. We're going to have a debate that's going to discuss whether YouTube is indeed radicalizing its users. Joining us to discuss it on the con side is Mark Ledwich, who's a software engineer, who's done some data engineering about this stuff. And on the pro, I feel like it's fun to put you in this category, is Kevin Roos of The New York Times, who's written and widely shared a very interesting story about someone who did indeed get radicalized on YouTube and then produced a very popular podcast about it called Rabbit Hole.

• 1:57Alex Kantrowitz?

  So just quick introductions, Mark, you are Australia-based, you're a software engineer.

• 2:03Speaker 01

  That's right. Yeah, I um, my background's in software and I um, I've heard a lot of the reporting and research in this area. And I felt like there was a, there was a gap missing for someone just to monitor exactly what YouTube was doing. So I took a sabbatical.

• 2:21Alex Kantrowitz?

  You went ahead and did it.

• 2:22Speaker 01

  I went and did it, yeah. And now I'm just trying to disseminate the information that I'm finding and bring attention to it.

• 2:29Alex Kantrowitz?

  Okay, great. Well, I appreciate you joining the show and, you know, being willing to talk about it here. Not only that, but being able to discuss it with someone who's going to be critical of your work, which, which I appreciate. And so Kevin, you're at the New York Times?

• 2:42Speaker 02

  Correct.

• 2:43Alex Kantrowitz?

  And I think a good way for us to begin with, begin this is to start with a bit of a recap of the story that started it all. And Kevin, I'd love it if you could tell us a little bit of a recap of what you talked about at Rabbit Hole. And by the way, folks, if you haven't listened to Rabbit Hole, I highly recommend it. And maybe this discussion will inspire you to do it with the added context. But Kevin, can you introduce us to the guy that you met, Caleb, and how he became radicalized through YouTube?

• 3:16Speaker 02

  Sure. So I've been looking into online extremism and radicalization for a number of years. And after the 2009 shooting in Christchurch, New Zealand, that was the one where it was, you know, streamed on Facebook and posted about on 8chan. And, you know, the shooter had this very like online manifesto. It was awful. And it was, you know, he referred to PewDiePie and, and sort of that sort of catalyzed for me what became the next sort of year of reporting, where I was really trying to answer the question like, like,

• 3:55Speaker 02

  how does this happen? How do people encounter extremist views online? What role do platforms play in introducing them to new and compelling extremist voices? And sort of what are the forces that power that process? So I was really looking for A case study, someone who's sort of, who would sort of let me, talk to me about their sort of their, their journey, their process. And so I started looking around and eventually found this guy Caleb Kane. He's, he was 26, he's probably now 28. He was from Virginia

• 4:35Speaker 02

  or West Virginia and he had a really interesting story. He basically was, you know, a Obama supporting liberal, dropped out of college, was having some real troubles in his, in his personal life. Not a lot was sort of going right for him. Started looking on YouTube for self-help videos, things that might help him feel better and stumbled onto this network of creators, including people like Stefan Molyneux, who really helped him feel better. They sort of, you know, had videos about self-confidence and, you know, meeting people and getting a job and just sort of, almost like life coaching. And so he started watching

• 5:17Speaker 02

  these videos and started watching other videos that were recommended from those videos. And he ultimately became pretty far right. I mean, he was, he considers himself alt-light, not fully alt-right. That's what he's, how he sort of characterizes it now, but at, at the sort of base level, he, he said that he was radicalized into the far right through his YouTube recommendations and eventually started agreeing with many of the sentiments of people like the Christchurch shooter. And then actually came out of that was sort of de-radicalized from watching videos by this other cohort of YouTubers known as BreadTube,

• 6:00Speaker 02

  who are sort of the left-wing, they do a lot of sort of counter-programming of the kind of alt-right section of YouTube. And so he eventually sort of got out of it and started making videos and talking about how he had gotten out of it. And I was just fascinated. I was, I was really interested in meeting him. So I went to West Virginia, talked to him for days. And then he provided me with his entire YouTube history.

• 6:25Speaker 02

  So 12,000 videos spanning four years and I could see it was just big giant HTML document.

• 6:33Alex Kantrowitz?

  So you were able to download the whole thing?

• 6:35Speaker 02

  Exactly. Yeah, there's the, there's this Google takeout function where you can download all your data. And so he just downloaded his whole YouTube history and sent it to me. And so we were able to kind of go through that and kind of retrace his, his journey and all the people that he met along the way. And so that became the story making of YouTube radical that Caleb is featured in. And it's, it's about him and his journey, but it's also about what was going on at YouTube at the time. And the way that those things interacted and the way that the, the changes that YouTube was making around its algorithm, you know, reprogramming it around deep neural networks, changing some of the key metrics that it was optimizing for,

• 7:18Speaker 02

  changing from clicks to watch time, how all of those factors sort of helped create the environment in which Caleb was radicalized. So that's the story.

• 7:32Alex Kantrowitz?

  Right. And in your reporting, and we'll get to Mark in a second, but in your reporting, did you ever put a finger as to how many Caleb's there were and whether this was a one person problem? Or you were tracing the route that the Christ church shooter could have taken? Or was this something that was a larger issue happening, not just to one or two people, but far more than that?

• 7:52Speaker 02

  Well, I should say at the time I started reporting it, I had already heard a bunch of stories from people. I was doing things like looking through Unicorn Riot. Unicorn Riot is this, is this group that is sort of a a left wing sort of counter extremism group that, you know, has leaked, has sort of obtained and leaked a bunch of Discord chats and other communications from far right groups. And a lot of the times, if you read through those things, it would be people who are, you know, currently pretty far to the right. Maybe they're white nationalists and they would say like, I got into this through YouTube. So you would see these testimonials of how people encountered these ideas. And since the story in the podcast came out, I mean, I've heard from thousands of people

• 8:37Speaker 02

  who have similar stories to Caleb's or whose family members or friends do. So it's impossible to quantify, but like, this is not a single person problem. And this is. You know, this is, this is a, this is a problem that was big enough that YouTube, you know, felt it, it had to address it through sort of changing its policies on white nationalism, hate speech, borderline content, and also changing its recommendation algorithm.

• 9:04Alex Kantrowitz?

  Great. And one more follow up, and this is probably an unfair question, but here we go. When was the last time you've heard from someone that's come to you with an issue similar to Caleb? So is this a common issue currently, or is this something that was in the past?

• 9:18Speaker 02

  Oh, you mean like, who is saying this happened to me last week? Or?

• 9:21Alex Kantrowitz?

  Yeah. Have you had it some, have you had like someone, you said thousands of people have reached out to you. So how, how recently has that happened? Or has it tailed off recently?

• 9:30Speaker 02

  It happens almost every day. And I would say it, many of the stories that I, I hear are about

• 9:42Speaker 02

  people, there, there seems to have been this sort of golden era of right wing YouTube from about 2014 until about 2019, when they started sort of making changes to tamp down the influence of some of these channels. And so a lot of it took place in kind of that five year window. But there's been other rashes of it this year. I mean, we've seen a lot of people becoming radicalized during the pandemic. QAnon has been, you know, has, has grown like crazy. And some of the adjacent sort of communities to that have become quite large. And so it's not just happening on YouTube. It's happening all over the internet. But I, I think that, that period, the sort of 2014 to 2019 period is the one that I was really interested in.

• 10:25Alex Kantrowitz?

  Okay, great. Yeah, I really wanted to establish Kevin's argument, but now, Mark, I'm going to turn it over to you. What's the counter argument? You seem to believe that people aren't being radicalized or on YouTube, or aren't being radicalized to the same extent that Kevin might be telling us. So where's your proof?

• 10:42Speaker 01

  So we did a, we did a study where I've been collecting data about YouTube's recommendations since late 2018. And we saw those changes. So some of them at least. So we weren't, we weren't collecting at the time what was really promoting clickbait kind of material. Um. When we started, it was fairly neutral in terms of the amount of views that videos got. Um. Recommendations were pretty much proportional to that. And then early 2019, YouTube really clamped down on borderline content and conspiracy content. So I think, I think we're aligned with the way we understand what's happened there. But I think more broadly,

• 11:23Speaker 01

  my, um, I think, uh, the influence of recommendations in terms of someone radicalizing is just a very small part of a larger process. So with the Christchurch shooter, I read that report, and there were so many other factors. And as they say in the report, it, um,

• 11:44Speaker 01

  it's often a once-off, highly personalized journey. It's, it's, it's no one model can explain the different ways people were at a class to extremism. So I feel like my analogy for recommendations would be that there are, there are like a gentle breeze that's able to be controlled by YouTube. But there's larger factors at play in the environment that's more like a storm. That's, you know, pushing all sorts of directions at different times and for different people. And but the recommendations are much more gentle than that, but something that that's in their control.

• 12:21Alex Kantrowitz?

  Okay, so so unpack that a little bit. So basically, what you're saying is that people do get recommended videos and they can go down these rabbit holes, but uh, you have to look at that as just one factor of many. And, and yeah, why don't you, why don't you go ahead on that one?

• 12:37Speaker 01

  Yeah, like it's one factor of many, and I wouldn't describe the recommendations people get as a rabbit hole. We found that, um, um, on average, recommendations go towards more mainstream content, at least since, um, early 2019.

• 12:54Speaker 01

  So it may have been back then, but we don't have data on that. And I haven't seen anyone else with good data on that either. So we're speculating as to whether the recommendations were pushing that direction or not.

• 13:06Alex Kantrowitz?

  Okay, so Kevin, there's two arguments here. And Mark, correct me if I'm not summarizing them right. But one is, we shouldn't put too much emphasis on YouTube, given that, you know, people are living not only in YouTube, but in the world, and there's other factors in the world. And two is that, you know, these recommendations don't always take people, you know, down this, down this rabbit hole. And in fact, the term might be wrong in and of itself. So what's your response on that front?

• 13:34Speaker 02

  Well, to the first argument, I mean, I agree with Mark that there are other factors. I mean, in the story about Caleb Kane, you know, I talk about the sort of larger forces at work here. I mean, it's, he was living in an economically depressed area. He didn't have a lot of career prospects. He had kind of a shitty family life. And it was like, it was not, he did not arrive at YouTube as a blank slate. I mean, he was coming in with a number of different, you know, personal traits that made him especially vulnerable to this. But I just, I don't think that we can discount YouTube's influence, especially

• 14:19Speaker 02

  now, when all of us are just experiencing the world through our screens. YouTube hasn't released data in a while, but the sort of last thing we know, as of a couple of years ago, people were watching a billion hours a day of YouTube. It is something like 15% of all internet traffic is through YouTube. So it's, this is not a small part of people's media diets, and especially for the group of people that I'm, you know, that I started studying, which was kind of this, these like, people who are maybe a few years younger than than I am, people who are in their, you know, teens and twenties, who really, for them, like YouTube is media. It is culture, it is politics, like YouTube is a sort of...

• 15:07Speaker 02

  All encompassing frame of reference for everything. And so for those people, it's not just, you know, it's more than just one in a system of of inputs. It is like the primary input by a long way. And on the second point, the sort of rabbit hole pipeline argument. I mean, I'm interested to hear more about this from you, Mark, because I've read your study and I've read some other studies about this. And there have been some studies that have found strong evidence for for sort of the migration of viewers from more sort of centrist and kind of alt

• 15:47Speaker 02

  light videos to alt right and sort of hard right content over time. Whether that's through recommendations or through other forces, you know, is a question that probably can't be answered, except if you work at YouTube. But I'm curious, you know, there have been some studies sort of suggesting that this doesn't happen. Like, do you think it is possible with the data that we have that is currently available to us as non YouTube employees to account for and and quantitatively study the effects of radicalization on YouTube? Like, do you think that's even like a an achievable goal?

• 16:28Speaker 01

  I think you can do much better than we're doing now. But no, I think right now, like, there's still much, there's still room for a lot of other, there's a lot of room for opinion. That the data that we have right now isn't definitive for the more holistic question. Like, is YouTube influence influencing things or things considered? People towards extremism? Like, are they making that worse? We don't know that yet. But

• 17:00Speaker 01

  There's definitely studies coming and that have just come out that are quite good. So there's one recently that uses web panel data from a representative group of Americans. They have real traffic and they they're looking at actual click throughs on recommendations and looking at which direction they're going. They have issues with their classifications, but they're fixing those. So I think that will be really good information. The study you referenced, I think that's the auditing radicalization pathway studies by Riberio and others, in that they looked at people commenting on videos and seeing over

• 17:40Speaker 01

  time whether they moved towards the, they classify channels as IDW hotline and not right, and whether there was a direction towards that. I thought that was quite clever, but they only looked at that one direction. So they didn't look at people moving like from something like Stefan Mullen you to Jordan Peterson to something more centrist. They didn't look at that direction. So I felt like that was an example of research that is technically quite good, but the bias really affected the way that they they ran it.

• 18:15Alex Kantrowitz?

  Mark, your your research itself found that a lot of the recommendations pointed people to more centrist content?

• 18:23Speaker 01

  Um, yes, so towards mainstream content. So it's got an authoritative bias.

• 18:29Alex Kantrowitz?

  Yeah, and that's after 2019?

• 18:31Speaker 01

  And that's that's since like April 2019. So that's the recommendations. But if you're thinking radicalization, you do have to look at it more holistically. That's just one input.

• 18:43Speaker 01

  So it could be just the existence of certain content you could blame YouTube for, or that video itself is engaging enough that it's more likely to radicalize than like a book.

• 18:56Speaker 01

  I don't believe those things, but yeah, that that's in the question when we're thinking about this more holistically.

• 19:03Alex Kantrowitz?

  Yeah. So you would say that since these changes in 2019, would you say before the changes in 2019, they were radicalizing people? Or you would say that would be an exaggeration? And then after the changes in 2019, it's even further an exaggeration?

• 19:17Speaker 01

  Yeah, I think it's an exaggeration. Like, one thing I'd point to is that we would expect places where people watch YouTube more for it to have an influence. And I can see, like, stories are like what Kevin wrote about Caleb Kane. They're like, I'm not against that type of looking at this problem. In fact, I like to do that whenever I do data analysis to look at the stories. So you've got a real, concrete version of what you're trying to analyze. But I feel like that kind of look, you need to look at all the different types. So you need, you need another person, which was, who has said they've been de-radicalized by Jordan Peterson, for example.

• 20:00Speaker 01

  He says he gets a lot of people telling him that they pulled him out of the old ride as well.

• 20:08Alex Kantrowitz?

  I guess like one thing I think we should talk about is, is scale. And I think, Kevin, you might have mentioned this on, in your article or on the podcast, talking about how if there's only a, you know, small, you know, let's say 99% of YouTube folks are watching it and not getting radicalized, that if 1% are, then that's an issue. So I'd like to be able to explain this, like, because, Mark, even if what you're saying is right, that the vast majority goes to, goes to centrist or authoritative stuff versus some of the radical stuff, then you still have that, that error at the end, that, you know, people end up going down, going down that path and becoming radicalized.

• 20:53Alex Kantrowitz?

  So why don't we do this? Kevin, can, you know, can you riff off of that and sort of give your take on it? And I'd like to hear Mark's response also.

• 21:02Speaker 02

  Yeah, I mean, the scale question is, is the scale, there's a scale question, there's a prevalence question. So the prevalence argument is one that YouTube loves to make. Facebook also loves to make this, where they say, you know, only, you know, 6% of the content on our platform is political, or, you know, it's, it's, you know, borderline content is less than 1% of all content on YouTube. And I have no reason to doubt that that's true. I mean, we can't audit that. They don't make, you know, data like that available to the public. But, I mean, just if you think about all the many things that people use YouTube for, like figuring out how to fix, you know, their broken toilet, or, you know, tie a bow tie, or, you know, listen to music, or, I mean, there are people

• 21:43Speaker 02

  who only use it for, for those things. And so that's, that's totally plausible to me that this might be a small percentage of, of, you know, overall usage of YouTube. But the denominator is so big. I mean, if you have people watching billions, you know, at least a billion hours a day, probably, I would, I would guess that it's increased substantially since that figure came out. 1% of a, of a number that big is still a fairly big number. And especially when you're, you know, when you're looking at the kind of possible outcomes here. I mean, it's not just, it's not just that people are getting radicalized and then, you know, posting a bunch of memes on 4chan. It's that they're,

• 22:27Speaker 02

  in some cases, going out and conducting mass shootings. They're, they're, they're becoming violent. They're, you know, engaging in sort of coordinated harassment. Like, this is not a sort of, if it were just, you know, people getting sucked into Bigfoot conspiracy theories or whatever, like that, that's, that to me doesn't really register as a, as a grave harm. But there's a real danger here. And I think that, you know, I'd be, I'd be curious to hear Mark's take on the sort of scale question. Because I think that even if it's true that, you know, a small number of people, relative to overall YouTube consumption, are experiencing this pipeline, I do think the, the existence of

• 23:11Speaker 02

  the pipeline is something that, you know, we need to study and we need to get more transparency from YouTube about.

• 23:17Alex Kantrowitz?

  Yeah, I'd love to hear Mark's answer on that also.

• 23:20Speaker 01

  I agree with YouTube that it's a small percent, but yeah, it's a huge platform. And it skews young, so I'm expecting it to grow like quite a much, much, much larger than it is now. I did compare, I don't have the numbers on me, I compared Fox News on cable versus YouTube. And cable is still bigger, just the amount of views they supposedly get. But I think in the next five years, we'll see that flip.

• 23:52Alex Kantrowitz?

  So, but Mark, then just like talk about the scale issue though. So if you're writing this paper pushing back on the fact that journalists are saying, journalists like Kevin are saying that YouTube is radicalizing and you, you say that there is a big scale and maybe this is happening to a percent of folks. Yeah, I'd like to hear like, you know, isn't that something that we need to consider in this discussion?

• 24:15Speaker 01

  I guess only if you're worried about it, so I'm not, I just don't think it's a problem in terms of YouTube being an influence for extreme radicalization. Because I see it more, there's a part of it where I'm doubting that that's a big factor. But I'm also doubting that it's a YouTube specific problem because people are coming and watching video content on the internet, no matter what. So let's say YouTube got rid of all right wing content and the extreme right everything. That would just be another platform that people watch. And we saw, we saw a big migration to Rumble when the QAnon crackdown happened. So they're getting millions of views on Rumble now.

• 25:00Speaker 01

  So it's not as if this pressure to remove the content or really change recommendations will have a massive effect.

• 25:12Alex Kantrowitz?

  Okay, but let's focus on YouTube though, because that's sort of the topic of discussion. You know, it's one thing that people are going to go elsewhere. But I want to, I mean, you know, we only have anecdotal data mostly on terms of the idea that, you know, recommendations are, or YouTube, the platform is, you know, driving people to radicalization. So, so far from you, I've heard arguments that have said, you know, they're, they're recommending authoritative content, but not always. That people would be radicalized, people would be radicalized no matter what because they'd be watching videos somewhere else.

• 25:55Alex Kantrowitz?

  And that there are other factors on you, outside of YouTube, that are, that are causing this issue. But like, what, can you make your most convincing argument that, you know, YouTube itself is, is not doing what Kevin has argued?

• 26:07Speaker 01

  Apart from those points, I'd say that the way I look at YouTube, the content is reflecting what people want to watch. So I think there's an intuition that is more radicalizing for someone that just watches mainstream. When they come onto YouTube, it's definitely more right wing and there's more edgy stuff. And I think it's intuitive to think that's a rabbit hole because it's reflecting the population's demand for content more directly than mainstream news. Although this mainstream music is starting to become more like YouTube in that way.

• 26:47Speaker 01

  So I can see the intuition there, but I just, I'd say I'm, the type of information that I'm looking for doesn't exist to show that there's a radicalizing effect. And also the over concern about it is like, when we talked about, when the discussion of radical, Islamic radicalization comes up, a lot of the same arguments apply to this, which is, you know, we're talking about a small amount of numbers, more people die, farther means, you know, alcoholism is more important, things like that. And I think of those things when this comes up as well.

• 27:21Alex Kantrowitz?

  And so you did research, so you're, you're, you know, you're coming to this conclusion that there's no real issue here. What in your research led you to believe that? Can you talk us through that a little bit?

• 27:33Speaker 01

  So my research just focused on recommendations. So what we saw in recommendations was a mainstream influence overall. And we're doing more study to look at what effect personalization has on that. So that's the area of research that I've done. So we're kind of talking about a wider question where I'm just speculating, much like everyone else is about what causes radicalization.

• 28:01Alex Kantrowitz?

  But when you looked at recommendations, did you find any radicalizing influence? Like, you know, I know you talk about authoritative. You said that YouTube steered people to authoritative sources, but did you find any evidence of even, you know, a smaller group of folks being steered towards more radicalizing content?

• 28:20Speaker 01

  Um, we found, um, no, I didn't, I can't, not my, I can't think of a channel. Actually, right now there's one channel, NTD, which is doing well with recommendations.

• 28:35Speaker 01

  And I think that's, that's, I can't remember what it stands for. They're part of the

• 28:40Speaker 02

  It's New Tong Dynasty. It's a part of the the Falun Gong media empire.

• 28:46Speaker 01

  They've been paying for a lot of ads on YouTube as well. And perhaps I'll get to that soon. And the effect of the demand for content on the election fraud conspiracy content. I think that has a large amount to do with that.

• 29:03Speaker 01

  But I lost my train of thought, sorry.

• 29:06Speaker 02

  I'm curious, Mark, if I can jump in with a question. I'm just curious. So, Mark, I remember when your original study came out, I got a lot of people emailing it to me, you know, saying, what do you think about this? It was sort of interpreted as kind of a, you know, a response to my story, although I know you you were looking at this before.

• 29:32Speaker 02

  You know, you seem pretty angry about the narrative of the YouTube rabbit hole. And I think, you know, you were saying things like, you know, this is a, you know, conspiracy that the mainstream media is using to sort of repress the influence of, you know, social media and, you know, this, this is, you know, these narratives aren't trustworthy and this is a self-serving thing. So I guess I'm just curious, like, and I don't want to, you know, attribute any arguments to you that you, that you, you know, aren't comfortable making, but I'm curious, like, what you think is behind, like, the, the sort of meta narrative here? Like, like, why you think, if this radicalization pipeline doesn't exist, and in fact, if

• 30:17Speaker 02

  it's pulling people, you know, in a more mainstream direction and has been, you know, for, for as long as we've had the data, like, why would people be coming out with these stories? Why would people be saying, I was radicalized by YouTube? Why is this such a persistent thing that we keep hearing over and over again? Like, what, what is, what is, what are the incentives of the people who are doing it if it's not actually true?

• 30:42Speaker 01

  Um, I think there's a, there's a elite culture which looks down upon the popular culture. And I think this is part of it, and it's especially true of the New York Times, which I find like a very small subculture of that.

• 30:57Speaker 01

  I was listening to Ezra Klein talk to another reporter from the New York Times who said he felt he was incentivized to write articles like this, reflexively anti-tech platform articles, and that benefited him at the New York Times. So that's definitely my background into what I'm thinking the incentives are inside the place that you work. And I definitely think the rabbit hole meme took on legs in a lot of places and I just wasn't seeing.

• 31:30Speaker 01

  Anything more than stories? And I think you get the selection effects at play. So when you tell those stories, you'll get more of them. And it's the same on the other side. So if you're, if you're more right wing or an anti woke YouTube channel, you also get lots of stories of people feeling like the places that they work at stifling free expression and things like that. So I think when you're in the public arguing about these things, you naturally attract a certain side that you keep hearing from. And it's, it's hard to maintain perspective.

• 32:05Alex Kantrowitz?

  Okay, this is actually a great cliffhanger to take a quick break on. And then we can have Kevin respond when we come back from the break. Cause got to get our sponsor and make sure that we can pay to get this edited. All right, we'll be back right after this. This episode is brought to you by App Point. Everyone's racing to roll out AI right now. Co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data and most teams have no single way to see it, secure it and prove it's under control. That's exactly what App Point does. For 25 years, they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate.

• 32:47Alex Kantrowitz?

  Your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. App Point, the unifying trust layer for AI. Head to apppoint.com to see how enterprises deploy AI with confidence. Learn more at avpt.co/bigtechnologypodcast. That's avpt.co/bigtechnologypodcast. Violent threats against executives are growing at an alarming rate. When a company experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door.

• 33:32Alex Kantrowitz?

  The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a white glove enterprise security service, they protect your people with continuous personal data removal, proactive prevention tools, and human led emergency support. So when someone goes looking for your executives, Ironwell ensures they hit a dead end. So here's what to do. Go to ironwall.com/bigtechnology, fill in the quick form and request your free risk assessments. Their team will show you just how exposed your executives are and how to lock it down before a threat reaches their front door. That's ironwall.com/bigtechnology. Learn how to stop online threats before they become real world attacks. The link's

• 34:14Alex Kantrowitz?

  in the show notes. Okay, we're back here on the second half of the Big Technology Podcast, a debate. Our first debate between Kevin Ruze, opinion writer at the New York Times, Mark Ledwych, software engineer who studied YouTube, looked at the algorithm and said, you know, there's no evidence of radicalization, at least, you know, on behalf of YouTube. So Kevin, before the break, Mark was making some arguments about the New York Times and sort of, I guess, how the media is incentivized to write these anti-tech stories and that there's no, it seems like there's no room for the other side is what he's saying. There was a lot

• 34:58Alex Kantrowitz?

  of stuff that was said. The floor is yours to respond.

• 35:01Speaker 02

  I mean, I think the, there are two issues here. One is like, what are the incentives of reporters generally? Right? And I think that often people who aren't, you know, close to the media, who don't work in the media, who don't have a lot of experience in the media, tend to think that it's just, it's clicks, right? It's like, you know, traffic, it's attention. I would say that's, that's true of some outlets and less true of others. Like, I certainly don't feel like I'm motivated by traffic. And then there's the sort of,

• 35:41Speaker 02

  there's the argument that journalists are motivated by sort of prestige and that, you know, stories that win prizes. Like, you don't win a Pulitzer for the story that investigates the, you know, the Wall Street bank and finds no evidence of wrongdoing. Like, I think I would grant that our incentives skew toward holding institutions to account and finding...

• 36:11Speaker 02

  Instances in which, you know, the public is being manipulated or taken advantage of. I often tell people like, you know, we don't write about the Boeing planes that land on time and safely. But that's always been part of the news business and I don't think that's changed meaningfully. There was this other argument that I think Mark and some of his, you know, people who agree with him make, which is that the New York Times and other media institutions are sort of being mean to YouTube because they want YouTube to promote their channels more and to like, you know, boost authoritative mainstream sources and to sort of disappear independent creators.

• 36:56Speaker 02

  And I've heard this from, I heard this after I profiled PewDiePie, I've heard this for years, that there's a kind of like institutional incentive for media organizations specifically to be mean to YouTube because they want their own content to be favored in YouTube, in the algorithm, on the homepage, in trending, wherever. And so I guess I would just love to hear Mark like talk about what he thinks that that incentive is like, why? Why do you think YouTube gets sort of criticized by mainstream institutions? Do you think it's more like, I don't know, why, why do you think that happens?

• 37:38Speaker 01

  I think it's largely political and cultural. So it's like YouTube represents a more right wing version and more, more scrappy, sort of low, low quality information type of platform. And that, maybe not everyone, but definitely I think maybe Tristan Harris and yourself want to be, have like a narrative about themselves that they're a large player in saving people from these problems. So from that, that influences the way you describe or think about the systems.

• 38:22Speaker 01

  So that you can say, oh, here's this one problem that if I can just shine a light on, we can fix. And that, that's where the bias comes in.

• 38:31Speaker 02

  And do you think like YouTube is better for these changes that it has made since, you know, people started paying attention to issues like radicalization like they they say that their algorithm changes have resulted in 70% less content or 70% fewer views of sort of borderline content conspiracy theory, stuff like that. Like, do you do you think YouTube is is a better platform today than it was, say, you know, two or three years ago?

• 39:00Speaker 01

  Yeah, I think I think it's better. I definitely like, and this is where I definitely credit to you and others pointing this out early, like Zainab as well. It's good that they're not recommending

• 39:15Speaker 01

  videos that are promoting things like conspiracies or, you know, fireite ideology. I think that's a definitely a good thing. But I think the changes, there's like, you know, five steps forward, two steps backwards. So I think they've curtailed some really good independent YouTubers. People like Dave Pakman or Aisy Green, they don't get recommended anywhere near as much as they used to because of these changes. And I feel like they have very high quality content, like better than a lot of mainstream content. But because of this blanket heuristic, they're being disadvantaged.

• 39:53Speaker 02

  That's interesting. I mean, I often wonder, I think YouTube would would love nothing more than for there to be a universe of YouTube native creators who do straight news, for lack of a better term. You know, people, you know, like Phil DeFranco, people who, I mean, he's doing some opinion too, but people, you know, who's who could just do what TV people on the news do, but like, do it in a YouTube way. Like, I think that would make them very happy because they they love promoting their own creators. And I don't think they, you

• 40:34Speaker 02

  know, my sense is that they kind of, they promote mainstream news sources because they they don't have to like, stay up at night wondering if, you know, NBC is going to publish some crazy conspiracy theory.

• 40:50Speaker 02

  It's sort of a proxy for how much they can be sort of trusted to at least report as close to the truth as they as they can. But I I do I do think they would like there to be a universe of YouTube creators who do that kind of thing. I just don't think YouTube creators are incentivized to do that kind of thing. Because, you know, it's not it's not good for views.

• 41:16Speaker 01

  Yeah, I I agree. It's that the demand for content is much more high for opinion than it is for straight news. I think you guys find that between your departments as well. Do you worry about the the criticism from media has made YouTube defensive? So they wouldn't take risks in terms of what content they promote?

• 41:42Speaker 02

  I I think that they are very sensitive to elite opinion and media is part of that. I think they're sensitive to politicians. I think they're sensitive to, you know, their peers in the tech world. They wanna, you know, they wanna be seen as the good guys. And so I think they want to sort of be you know, small C conservative with respect to, you know, if they if they, you know, take it as a given that their algorithm is gonna throw, you know, X billion views to a set of channels. They want those to they want to make sure that those those channels are not gonna, you

• 42:25Speaker 02

  know, be the next Alex Alex Joneses. And so I think, you know, a shortcut to do it's sort of like how, you know, now if you search for nine 11 conspiracy theory videos or moon landing videos, like they'll give you a little thing from like Wikipedia on the video. And so they're sort of outsourcing sort of trust to Wikipedia because they don't want to write their own little blurbs. And I think that that's I think of what they're doing with sort of this authoritative news push as a version of that where it's like they they don't.

• 42:59Speaker 02

  They don't want to be recommending conspiracy theory videos, but they know that they have this algorithm that is, you know, needs to recommend something. And so they feel more comfortable, you know, recommending, like creating sort of a safe bucket of things that they know are not going to, you know, be extremist or contain hate speech and recommending those. And I, and I do think there's a danger of sort of making that bucket too small and, and sort of too, you know, not, not inclusive enough. But I also think that that problem is more easily solved than, you know, the question of radicalizing people. Like, I think

• 43:42Speaker 02

  that it's, I think that it's really, I think what they don't want to do is create a situation like they've had for the past couple of years where they end up, you know, by sort of negligence, recommending these awful videos to, you know, hundreds of millions or billions of people. And they don't even really know that they're doing it.

• 44:03Speaker 01

  Yeah. Another, another aspect to that is there, they are, they do have competitors. And in terms of the engagement of the platform, how entertaining those recommendations are, matters in terms of whether they're going to lose to TikTok. And I feel like TikTok did a better job with their recommendation algorithm, which, you know, caused much more of the views than on YouTube. And they have to think about that. So if they, if they make their recommendations extremely bland, then that's opportunity for competitors that aren't doing that.

• 44:40Speaker 02

  Right. Alex, I'm cognizant of the fact that you were, you promised your listeners a debate. So I feel like we should like.

• 44:46Alex Kantrowitz?

  No, this is good. I'm letting this breathe because this is, oh, well, if you want to amp it up, by all means, but I think people are going to find this fascinating.

• 44:54Speaker 02

  I, I agree with Mark about, you know, the, the fact that this is a broader phenomenon than just the YouTube algorithm. And in fact, one of the things I'm sort of looking at now is like, what, what is this demand side, you know, part of the equation? Like what, you know, what we see.

• 45:10Alex Kantrowitz?

  You mean there are people involved in these decisions too?

• 45:14Speaker 02

  Yeah, obviously, like people, you know, people make choices. The thing that's that I sort of push back on is this idea that people have total control over their information environments, even when things are being recommended to them. There's been a lot of academic research about the power of recommendations, the sort of psychological power. There have been some interesting studies around things like music recommendations that people actually like a song more when they, you know, know that it's gotten five stars from Spotify for, you know, or ended up on some personalized playlist for them. They trust the things that are fed

• 45:55Speaker 02

  to them by an algorithm. In the case of YouTube, like what YouTube's recommendation algorithm does is it really, it not only recommends videos, but it dramatically constrains the universe of possible videos. There are billions of videos on YouTube, but you are only going to see a couple dozen of them on any given day because those are the ones that are going to sort of appear in your sidebar and on your homepage. And so I think people have sort of free, free will and free choice in some aspects. But I really, I really think that, you know, part of the mistake people make in the opposite direction is assuming that, you know, people are in total control of their choices because we know, I mean,

• 46:36Speaker 02

  these platforms make billions of dollars a year by, by sort of trying to change people's minds in the form of targeted advertising. They know how influential their recommendations are. And that's a huge part of what's made platforms like YouTube so successful.

• 46:52Speaker 01

  Yeah, so I'm not arguing that recommendations like totally controlling people. Like in your article with Club Can, you called it like steering people. And I feel like that's too strong a word in terms of the influence that it's having. Like, that's why I used the gentle breeze and a storm analogy. And I think what happened with the election fraud, which you looked at quite closely. Shows this in that YouTube was curtailing recommendations to videos. On average, you'll find exceptions, but on average, videos that were promoting the election for narrative were recommended a lot less.

• 47:32Speaker 01

  But despite that, the content that was promoting it did really well. So because Fox News, who are promoted by the algorithm, were disputing mostly the election for narrative. They lost views at the same times at places like Newsmax and NTD and One American News Network. They gained a lot of recommendations. And so gained a lot of views despite not being recommended as much. NTD was an exception, but the others, that's true.

• 48:05Speaker 02

  Right. I mean, I think that really if we're talking about the YouTube algorithm, we have to sort of separate it into two, like algorithm 1.0 or depending on how you count, you know, the sort of pre 2019 algorithm. And then what's happened since, because I do think that, you know, just anecdotally and also from some studies that have come out, like it does appear that YouTube is recommending, you know, much more, it's much more likely to be recommending things for like big, you know, professional media organizations than it used to. And also a lot of the sort of quote unquote worst people have been deplatformed. You know, Stefan Molyneux is no longer on YouTube.

• 48:47Speaker 02

  You know, Richard Spencer is no longer on YouTube. These people who were, I mean, Stefan Molyneux was not a marginal figure. He had, you know, hundreds of thousands of subscribers. He got hundreds of millions of lifetime views. Like, these were some of the most popular political voices on the platform. And so, you know, not only is the algorithm different now, but the pool of available videos it's picking from is is different in some meaningful ways.

• 49:13Alex Kantrowitz?

  Okay, I do have a few pickups here. Mark, hold the question, we'll let you ask it, but I have some follow-ups here. So first of all, Kevin, you know, Mark asked you about, you know, do you worry that your stories will get YouTube to remove, you know, too much content, something like that. And you answered, looking at it from YouTube's perspective, but I'm interested from your perspective. Do you ever think about like, the fact that you're reporting on the stuff that you're reporting on, what might lead to a crackdown for YouTube that gets folks who shouldn't be demonetized, demonetized, and independent creators never get a real chance to get off the ground? Like, how do you personally feel about that?

• 49:52Speaker 02

  Well, first of all, I don't think that, you know, the goal of my reporting is not to get YouTube to take down stuff. That is not, you know, a metric that I am sort of, you know, aiming toward. My, my goal is to report on what's happening on YouTube. And if that leads them to want to take down stuff and to feel pressure to take down stuff, then that's that. But that's not my goal. I would, I would say I-

• 50:15Alex Kantrowitz?

  Right. But you have to know that that, that's a, you know, very clear potential outcome when you write a story like this.

• 50:22Speaker 02

  Of course. And I think, you know, that I'm not naive about that. I, I guess I worry about the, I worry about the false positives less than the, than the false negatives, to be sure. Like, I, you know, something like this happened with, with QAnon and the crackdown over QAnon. And, you know, one, a podcast that I really love is this podcast, QAnon Anonymous, which is a sort of anti-QAnon podcast. But because it had the name QAnon in the title, like, it got swept in the crackdown, which sucked, which, you know, I was like, I don't, I don't want to miss this podcast. This is a great podcast. So, but they, you

• 51:05Speaker 02

  know, they made a stink about it and got it restored. And, and, you know, there are avenues for sort of redress of grievances in the case of a false positive. And so I guess I worry less about- The sort of overbroad application of these rules because I mean, really, we're still in a phase where these companies are being flooded with sort of misinformation. Like it's, we are nowhere near the point of having a totally clean house as far as, as you know, information integrity goes. And so I just think it's a little premature to worry about, you know, whether we're sweeping up too much stuff, you know, when, whether YouTube is, is sweeping up too much stuff when it goes after white nationalism and neo Nazis.

• 51:49Alex Kantrowitz?

  Right. And then how about, how about the Streisand effect, right? Do you think that the fact that all these folks are getting banned will actually make their message resonate? Maybe with a smaller group of people, but they'll use it as proof to say, look, we're right. And then, you know, big media and big tech don't want you to know because that's certainly the argument they make.

• 52:08Speaker 02

  Well, let's just look at like, what actually happens to people after they get deplatformed. And when was the last time you heard from Alex Jones?

• 52:15Alex Kantrowitz?

  When he shows up on other people's podcasts. Like he did just show up on-

• 52:19Speaker 02

  He just went on Joe Rogan.

• 52:20Alex Kantrowitz?

  Right. Who has a, not a, is a smaller audience than this show, but not by much.

• 52:26Speaker 02

  Right. So like, but Stefan Molyneux is a great example. Like he, you know, he still puts up his videos on Bitshoot and other, you know, minor video platforms, but he's struggling. He's not happy that he was deplatformed from YouTube. And, you know, I think that the people who have been banned by these platforms generally don't, it's really hard to rebuild that audience on a, on a smaller platform. And I guess as far as if I was ranking harms, like I definitely worry about building these sort of concentrated spaces, you know, where there is like, you know, like Parler or, you know, places where there is like Gab, you know, a concentrated amount of sort of extremist activity. But I actually

• 53:07Speaker 02

  worry about that less than that sort of contagion effect where you have people who go onto YouTube, go onto Facebook, they're looking for self-help or parenting advice or, you know, health information or, you know, they're, they're watching, you know, unboxing videos or whatever. And then they're sort of coming across this universe of extremist content that sort of pulls them in. That is what to me is, is more worrisome than the kind of the small clusters of extremist activity.

• 53:38Alex Kantrowitz?

  Yeah. Okay. And then I got one more, one more pickup for you and then could go back to Mark, but.

• 53:44Alex Kantrowitz?

  You know, Mark mentioned competition and TikTok as a competitor. It's kind of ridiculous because if all this bad stuff is powering YouTube,

• 53:55Alex Kantrowitz?

  and then maybe TikTok uses the dark arts of social recommendation engines and grows. You know, we might end up with a company based in China that's crushing YouTube. And yeah, and sort of less responsive to legitimate concerns and maybe doing stuff in the background. So how much do you think about that in terms of, and I know it's kind of a crazy situation because it's almost asking, how could YouTube weaken itself by removing all this radicalization? But what do you think about that, though?

• 54:31Speaker 02

  Well, I think TikTok and YouTube are different products. I mean, they're both video platforms that are driven by recommendation engines. But TikTok is very short form videos. YouTube, I don't know what the median length of a YouTube video is, but it's probably significantly longer than a TikTok video. And I think TikTok is an interesting case of moderating in the other direction. So YouTube started off as like total free for all, and then has sort of gradually winnowed down. Like, we don't want clickbait, we don't want nudity, we don't want neo Nazis. They've shrunk the pool over time. Whereas TikTok started out very constrained.

• 55:13Speaker 02

  Like, there was basically no politics on TikTok at the beginning. And that was a conscious choice because ByteDance wanted it to be a fun, lighthearted place. They didn't want people talking about oppression and injustice. And they wanted to keep it light. They wanted teenagers renegating in their bathrooms or whatever. So they have sort of expanded what is considered acceptable over time, where they started letting on political videos. And then they started broadening out. I think the diversity of content is probably higher on TikTok now than it was a year ago. So their approach, I think they're probably going to end up in a pretty similar place,

• 55:56Speaker 02

  but they're approaching it from different directions, which is a really interesting thing to me.

• 56:01Alex Kantrowitz?

  Yeah, yeah, we'll see. We'll see what happens there because I mean, just having the experience of spending more time on TikTok lately, that thing is addicting. And I mean, I have a pretty good YouTube addiction right now as it is. And it'll be interesting because it's to me, it's a zero sum game. Time on TikTok is time, not on YouTube. So that competition will be interesting to watch, especially as they make decisions based off of safety. Okay, we are coming close to being out of time. And I'm sorry, Mark, I kind of took the floor from you. So do you have any last questions or thoughts that you would like to ask Kevin?

• 56:39Speaker 01

  Yeah, I just want to downplay the influence of recommendations a little bit in that the best estimate for political videos I've seen is about 40% of the recommendations, sorry, 40% of the views coming to political videos coming from recommendations. So there's a large amount where it's coming from links and search and other means.

• 57:00Alex Kantrowitz?

  And direct, right? People just, yeah, well, that's search. Right?

• 57:04Speaker 01

  Yeah. So people are often in social media, they're in private chats with each other, or they're on Facebook or Twitter, and they're clicking on links through that amongst other means. So there's lots of ways for the content that's in demand for people to get at. And if, and I think if you, and that will change depending on what you do with the recommendations. So if you make your recommendations particularly bland, I think that would actually change that makeup. It would go down to less than even 40%.

• 57:30Alex Kantrowitz?

  And you could also get recommend, like if you, the way that I, I think rabbit hole's a good term because it's how I kind of fall into these things. And generally, like what happens is I'll just watch like one genre of videos so much. And then, you know, YouTube will recommend something new and then I'll just start going direct to it. And just for the listeners, most of my YouTube videos are just like sharks and whales in the ocean, not a, not any political leaning. And I think that's probably mirrors most YouTube users. But, but yeah, it's the recommendation plays one part and then you start going direct. So it's a process. It's a process. Kevin, do you have any last thoughts?

• 58:16Speaker 02

  I'm really appreciative for all the research being done by Mark and others on this. I think it's super important for the academy, for engineers and data scientists and for journalists all to be looking at these problems simultaneously. So even though you disagree with my reporting, and I'm really glad that you're looking at this and doing the work.

• 58:43Speaker 01

  It'd be great. My next study, if you could write it up in the New York Times.

• 58:48Alex Kantrowitz?

  So do you guys feel like you, I think there was a bit of a standoff? Do you guys feel like you understand each other's points a little bit more broad after this conversation?

• 58:59Speaker 01

  Yeah, I think we got much deeper into the dynamics of this problem than you would in an article. Yeah, it's good.

• 59:06Speaker 02

  You're a peacemaker, Alex.

• 59:08Alex Kantrowitz?

  Okay, well, I did want to say that I'm a believer in this type of dialogue. I think that this is good, benefits everyone. So if there's folks listening who have bone to pick with other reporters, or reporters who don't like what's going on in industry, we can see if we can set more of these up. But I have to say, I'm thrilled to have the opportunity to host this discussion. There was a point like here in the second half where such a great conversation that it was time to just let it breathe. And I'm grateful for you guys coming on and running with it and being able to address each other and bring up these questions. And I agree, it's something that we're going to be talking about for a long time. More dialogue, more research, more reporting, all this is good.

• 59:49Speaker 02

  I will say, if you want this to go viral on YouTube, you're going to have to give it a title like, you know, "Programmer Owens Journalist."

• 1:00:00Speaker 02

  You're not going to get any views with the boring title.

• 1:00:03Alex Kantrowitz?

  Yeah, it's "New York Times Admitter," "New York Times Reporter Admits the Censorship Campaign Is Well on Its Way."

• 1:00:12Alex Kantrowitz?

  Okay, great. Well, we'll find a creative title. I mean, I don't know, I'd like to go with the bland ones. It's probably why I'm losing out to TikTok. But thanks, everybody for listening and thank you, Kevin and Mark, for joining us here on the Big Technology Podcast. If you're a new listener, please subscribe. We release a new episode every Wednesday with tech insiders, outside agitators, and maybe we'll do some more debates like this. If you're a longtime listener, rating goes a long way. So if you could rate us, doesn't matter how many stars, but a rating would be great on your app of choice. We would appreciate that. Thanks again. We will be back next week with a new edition of Big Technology Podcast. Until then, take care of yourself and we will see you then.

• 1:01:03Speaker 03

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPact.com, not authorized by any candidate or candidate's committee.

• 1:01:32Speaker 04

  In this life, you can't win.

• 1:01:34Speaker 05

  When the world gets you down.

• 1:01:35Speaker 04

  No!

• 1:01:36Speaker 05

  Escape to Pluto TV.

• 1:01:37Speaker 04

  This is objectively fantastic.

• 1:01:39Speaker 05

  Relax into the movies and shows you already know and love, all for free.

• 1:01:43Speaker 04

  Schwing!

• 1:01:44Speaker 05

  Even our news is uplifting.

• 1:01:45Speaker 02

  You stay classy, San Diego.

• 1:01:47Speaker 05

  Watch School of Rock, Crazy Ex Girlfriend, Anchorman: The Legend of Ron Burgundy, The X-Files, Wayne's World, and so much more.

• 1:01:53Alex Kantrowitz?

  We're not worthy!

• 1:01:54Speaker 05

  The best things on TV really are free.

• 1:01:57Speaker 04

  You just have to know where to look.

• 1:01:58Speaker 05

  Escape to Pluto TV. Stream now, pay never.