Pod Insights The Vergecast Why big companies will never get content management right, with UCLA’s Sarah T Roberts About this episode Behind the screen: content moderation in the shadows of social media author Sarah T Roberts joins Verge editor-in-chief Nilay Patel to discuss the business and dark side of content moderation while pondering future solutions.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Speaker 00 Soft, cushy, plush, cozy. Soft, smooth, buttery, marshmallow-y, warm. Soft. Socks, underwear, tees, and more. Bombas. Everyday comfort worth obsessing over. Visit bombas.com/audio and use code AUDIO for 20% off your first purchase. That's B-O-M-B-A-S dot com slash AUDIO. One purchase equals one donated, always.
0:31 Speaker 01 Real innovation is powered by real people. This is Amex Corporate. Trusted experts, smarter technology, 24/7 dedicated support from day one to answer questions, help set up policies and approvals, and consult on spend capacity as your business needs evolve. Plus, on-demand resources help level up operations at every stage. That's the trusted service you've come to expect. Built to simplify, designed to scale. This is Amex Corporate.
1:02 Nilay Patel? Hey everybody, it's Sinead from The Vergecast. On this week's interview episode, we have Professor Sarah T. Roberts. She's a professor of information studies at UCLA. She's just written a book called Behind the Screen, which is all about content moderation. As you might know, we're very interested in content moderation on the show and on the site. Casey Newton has been writing a bunch of pieces about what it's like to actually moderate Facebook in America. Sarah has been working on this book for a year. It's a deep dive into what she calls "commercial content moderation," the business of content moderation, how it's structured, where it came from, where it's going, how it works. She's talked to content moderators in America, in the Philippines, at companies big and small. She says content moderation, the way we think about it, is not really a speech problem for these companies. It is fundamentally a brand management problem for companies
1:46 Nilay Patel? that sell advertising, which is what Google and Facebook and all the rest do. Really interesting conversation. The book is out now. We've been waiting for it. It's out now. You can go get it. You should read it. It's a fun read. And check this out. Professor Sarah T. Roberts from UCLA.
2:02 Nilay Patel? All right, I'm here with Dr. Sarah T. Roberts. She's an assistant professor at Information Studies at UCLA. Welcome, Sarah.
2:08 Speaker 03 Thank you so much for having me.
2:09 Nilay Patel? So you have just published a book that I think you know we at The Verge are very interested in. It's called Behind the Screen. It's all about content moderation. You've been working on this book for a very long time.
2:18 Speaker 03 Yeah, I actually, I've been working on the book for about a year, but I've been working in this area of research for going on a decade.
2:26 Nilay Patel? So in this book, you talk a lot about the nature of content moderation. You try to put a framework around what it means for the world, for the companies that engage in it. And you actually talked to a bunch of moderators, not only in the United States, but in the Philippines and other places like that. How did you decide to try to put that framework around content moderation?
2:46 Speaker 03 Yeah, when I came to this topic in the summer of 2010, it wasn't clear to me that this phenomenon of content moderation, or what I call commercial content moderation, which is specifically the organized, for-pay, industrial-scale practice of content moderation. It wasn't clear to me yet that it was a globalized and deeply stratified industry practice. At the time, I read an article in The New York Times in the summer of 2010 that talked about a group of workers in central Iowa, in what had been an agricultural area of Iowa. At the time, I was doing my PhD and I was sitting in a cornfield in central Illinois.
3:31 Speaker 03 So that really struck me. So my first kind of entree to this topic was to think about this group of workers working in this rural part of Iowa in what essentially, for all intents and purposes, were call centers, who were screening content on behalf of mostly unnamed social media platforms and other companies that had mechanisms whereby users could upload content. And these folks were third-party workers. They were contractors. They weren't directly employed, for example, by social media firms. But they were adjudicating and making decisions on behalf of those firms. And when I stopped to really consider it and think about it, I realized that this
4:15 Speaker 03 must in fact be a massive problem for social media platforms, how to deal with, contend with, and maintain some semblance of control over the user-generated content that they, of course, solicited.
4:30 Nilay Patel? So you have visited many of these places. You've talked to some of these folks. Obviously, you know Casey Newton, our senior reporter. He's been reporting on a bunch of sites in the United States that have horrible working conditions. But there's a little bit of a distance between what you're doing on the academic side and Casey's sort of, I'm on the ground, these people are telling me what their lives are like, true other reporting side. I'm very interested in, as an academic, when you look at the state of commercial content moderation, it's a business that has had to develop, it's gotten really big, right? The platforms have scaled to massive size. Facebook just put out a report saying it's going to, you know, build a Supreme Court of content moderation.
5:12 Nilay Patel? They're saying words that the government usually says. As you've watched that from sort of your perspective as an academic, are we getting it right? Is this the right way to go about it? Or are we just sort of barreling down the road that we set upon 20 years ago, when someone realized if you have a report button, you need to have someone look at it?
5:30 Speaker 03 Yeah. I mean, I think you're putting your finger on a little bit of how we've gotten to where we are today in 2019, which is the fact that commercial content moderation as a phenomenon, first of all, and then secondly, as an industry, has come to exist as, at best, an afterthought for these firms. So when the social media firms got into their their business, which it's worth reminding all of us, their business is advertising, and their business is using our user-generated content as a means to hook advertisers into their platforms and connect users with products, or monetize data about our behavior and so on. When they went to do that, the idea of
6:16 Speaker 03 Sort of exercising editorial control, which effectively is what commercial content moderation is, over this vast amount, even 10 years ago, this vast amount of content, it was not front of mind. So as the, as these social media companies and platforms ramped up exponentially, which meant exponential influx in user-generated content that was coming onto the platforms, the need to be able to gatekeep around that material, if for no other reason, for brand protection of the firms themselves and their platforms and their relationships to advertisers, grew. And so suddenly, the firms in 2019 are finding themselves squarely
6:59 Speaker 03 in the business of dealing with the adjudication of content, the thing that they were diametrically opposed to doing in the first place, right? Which is now a major source of hiring, of expense, of development in terms of engineering and computational tools that can assist or take over some of the duties. And of course, responding to regulation and other, other kinds of very high-level questions.
7:23 Nilay Patel? So in the book, you lay out several kinds of firms that have to deal with content moderation. You have the world-scale players, like your Googles and your Facebooks. You have some boutique firms. You've got the firms that are just for hire, brand management. Should we have the same set of rules for all these players? I mean, look, The Verge employs a moderator. We have comments and forums. We have a YouTube channel. We are very strict about how to behave in our community. I think it strikes me that us falling under the same regulation as Facebook or the same rules as Facebook, it seems to me wholly unfair. Do you think in each of these categories that you lay out that we should treat them all the same? Or should we pull them apart and make it more obvious that there are different kinds of things going on?
8:08 Speaker 03 I mean, it depends on what you mean. Should everybody follow the same moderation rules? I don't think that's likely to happen for the simple fact that this isn't the public square. This is actually, these are private companies setting up rules of engagement for their own space. Just like you said, at the verge in the comments section, there are things that can be done and not done. You kind of set those parameters. That's the rules of engagement and you moderate accordingly. But in terms of working conditions, for example, I think it's important to recognize that one of the things I found that surprised me, in fact, early on in this research, was that some of these major Silicon Valley firms, for simple questions of volume and also the financial cost of the labor they needed,
8:52 Speaker 03 they were using all of these services, all of these types of work sites to get the work done. So they might have had, you know, for example, the case I give in the book of Megatech, which is a pseudonym for a major Silicon Valley firm. Megatech had a crew of content moderators, commercial content moderators onsite at its headquarters. Of course, those people were contractors. They weren't directly employed by Megatech, but they came to work every day at Megatech. Then they had a call center team that they would outsource to in India, which effectively was the same kind of setup in a way as the contractors who were coming in and working at Megatech, although they were working in a differently branded call center in another part of the world. And it also came to light
9:37 Speaker 03 through a variety of conversations I've had with people over the years that Megatech was also outsourcing some of its needs to platforms like Amazon Mechanical Turk. So even though that's supposed to be anonymized and the workers aren't supposed to really know who they're doing the work for, sophisticated workers on that platform are able to parse that out and be pretty, pretty accurate about the work that they're doing. So it turned out that firms like Megatech and others, because of their need, were doing this sort of like patchwork mechanism to get enough labor to respond to the content moderation that they already had in the queue. And of course, it's important to note that what actually gets reviewed by humans is really frankly, a drop in the bucket compared to
10:22 Speaker 03 the amount of content that's uploaded. So that's kind of the first thing. To the second point, Part of the benefit, imagine air quotes right now, the benefit to firms to doing this outsourced, globalized kind of piecemeal labor process is that there are different norms for labor in other parts of the world. So some places have higher protections with higher base wages, where certain kinds of benefits are required. Other places don't have those things. So, you know, there are these other kinds of reasons that companies might look for those differentials in terms of standards.
11:05 Nilay Patel? What's interesting about that, though, is fundamentally, this is speech regulation, right? And exporting speech regulation to a country that may not speak the language that they're confronted with is very difficult, which is why you see Facebook in particular trying to build sites that are more local, more regional. Has that changed over time as you've been watching the space? The push to make this a little bit more local, or is it still kind of globalizing?
11:30 Speaker 03 There is certainly pressure on all the firms to respond appropriately to local norms, linguistic, cultural, but also certainly, you know, in terms of like what political conflicts are going on in a particular region, what, you know, what are the different ethnic groups and what are their relationships to each other? So it doesn't suffice, for example, to necessarily have a call center in Europe where there are Spanish speakers on site adjudicating content for, say, the northern part of Mexico, right? Like, that still might not make sense, even though the linguistic capability is there. So one of the things I've seen over the years as I've been watching this industry, one of the things I like to
12:13 Speaker 03 do to keep tabs on it is to look at job solicitations. And it was interesting to me over the past few years that I would see solicitations for work in the Philippines that was very, very localized and very site-specific. So they would be seeking workers to come to the Philippines to work in these call centers doing content moderation, but it would say things like, "Must be fluent speakers of Gulf Arabic," right? It would be that specific, or "speakers of Thai." And of course, we know that baked into the notion of being able to be a native speaker
12:50 Speaker 03 of Gulf Arabic, you are likely also acculturated to norms and politics of said region. So I'm seeing more and more of that. And of course, then there's also the side of, you know, legal mandates that are pushing on firms to do some of this. So in the case of Germany, where we've seen the Nets DG law go into effect, let's say over the past year that that's been in effect, we've seen a law that says any social media platform that has more than 2 million users, which frankly is quite a few of the platforms, that endeavors to be in the German marketplace must, must be able to respond to German anti-hate speech laws. Now, Germany
13:34 Speaker 03 has particular reason to have some pretty stringent hate speech laws, particularly as it pertains to Nazi glorification and antisemitism. And so Facebook, for one, has a giant call center through a third-party company outside of Berlin. And you better believe that the folks in that call center are absolutely steeped in the German context in every way imaginable so that they can respond to that German law, because the flip side is massive fines if they don't take that material down within the window mandated.
14:07 Nilay Patel? So we've talked a lot about how it works, who these folks are, where they're situated. I want to come back to something you said briefly a few minutes ago, which is that this is for the platform companies, this is fundamentally a brand management problem. Depending on who you talk to, it's, this problem is all sorts of things. This is a, you know, the conservatives say it's a censorship problem. I think this is like a hate speech and harassment problem. But you're saying for the platform companies, this is a brand management problem. Explain that a little bit.
14:39 Speaker 03 Sure. I think you're right. I think this is a shapeshifter. And so it's important to point out that depending on where you sit in the world, you're going to perceive it in a certain way. And here's, I'll call out a study that a fellow academic did, Sarah Myers-West. That was so interesting. She looked at about 500 users who'd had a takedown of some sort and found that in every case, each one of those people felt that they were being targeted for their political affiliation. And you can imagine that those people ran the gamut of the political spectrum. So what I mean by brand management is that, you know, this, this kind of takes us back in history a little bit to the ways in which these companies came to be.
15:22 Speaker 03 And of course, there was a social internet before Facebook. There was a social internet before YouTube. I mean, there were, not even that long ago, there was something called MySpace, which most of us have forgotten about. But, you know, but prior to that, there were all kinds of social interactions going on online and which by any other name could be a self-expression action or a speech action. And I'm thinking about Usenet and BBSs and the kinds of places that I cut my teeth. One of the primary allures of participating in those spaces was the sense that there was an unfettered capacity for self-expression that, and that was sort of coupled with a sense of anonymity.
16:06 Speaker 03 Whether or not that anonymity ever existed, it certainly is much harder to achieve in 2019. But these social media companies that exist today, they come out of that history. They are, they are steeped in Silicon Valley ethos that puts primacy on the circulation of information over almost everything else. The circulation of free expression, maybe at the expense of someone else's comfort, for example. And so that's kind of how those platforms were sold to the public. I mean, upload anything you want, share anything you want. We're not going to get in the way. Broadcast yourself was YouTube's slogan on and off again for many years. And yet.
16:51 Speaker 03 When there are vast commercial interests at play, and when the business model of a platform is to attract advertisers, and to connect advertisers to users, or in some cases, to place ads next to certain content videos or other kinds of content, there is a pretty obvious need for those companies providing that mechanism and that platform to have some control over how those relationships get created. Because what could happen when an advertiser finds that her video is placed next to a distasteful, disturbing, abhorrent piece of content?
17:36 Speaker 03 You're going to have an angry advertiser, right? So despite the fact that all of us as users were sort of brought into the platform on this idea of expressing ourselves in any way we kind of saw fit and at any time, that discretion of the platforms to kind of make decisions about potentially rescinding or removing content, or even just the fact that they've always had some sort of community guidelines that we can point at and gesture at, however nebulous they may have been, all of that is, you know, is to the end of controlling their product. And I think that when I lay it out this way, it's pretty obvious. You know, it's a different sort of argument
18:21 Speaker 03 to say, "Come broadcast yourself", versus, "Put some content up", and there might be a legion of people in the Philippines who are going to look at that and think about whether or not it's okay, and if they decide, and dot, dot, dot, right? But I think if we can kind of get back to this notion of what it was meant to do in the first place, and what it still does in so many ways on behalf of the platform, Then we can have a different kind of dialogue about how we feel about that.
18:50 Nilay Patel? That seems like you kind of want to make it smaller, which I think is a good thing. I mean, one thing that I'm constantly saying on the show is the United States has been trying to stumble towards a free speech policy for over 200 years. And we're not very good at it. And so to expect Facebook to get it in 20 years seems perhaps misguided at best. So it seems like one of your prescriptions here is like, let's make this conversation smaller. Let's make this about what it really is.
19:18 Speaker 03 That's correct. That's certainly one of the ingredients I would put in the mix. And I think the second one that really matters to me, too, is let's talk about what we're really talking about. If we're talking about brand management for platforms because platforms are advertising companies or they solicit advertisers as clients. Now we're talking about a horse of a different color from the public square where everyone can go and emote and have free speech and information wants to be free and all of this other kind of rhetoric that just isn't realistic. Now, if you're a person who wants those spaces, which many people do want, I mean, I'm thinking of our colleagues at EFF, for example, or other places.
20:01 Speaker 03 That's fine to advocate for that, but I'm telling you that Facebook, YouTube, Snap, Instagram, that ain't it. So if that's what you want, you're going to have to build something different and we can have a collective conversation about that. But I think understanding what these platforms are and are not and can and cannot be is it's got to be a more realistic conversation, as you're saying.
20:22 Nilay Patel? What do you make of the sense that these companies are, they're gigantic in the case of Google and Facebook, they're huge. Twitter is not so big, but it has an outsized cultural weight, obviously.
20:33 Speaker 03 It does.
20:34 Nilay Patel? What do you make of the sense that, well, maybe we should just call them public squares. Right now, you're saying, look, it's not going to happen. We have to be, we have to be realistic about it. But the pushback on that is real. It is very large. It's very passionate. It's saying, these are our public squares. We should treat them like that. How do you respond to that?
20:51 Speaker 03 I mean, I think that's absolutely a legitimate argument. But I think if we are to imagine them as being public squares and enacting that place, and we might also You know, the case might also be made that they have a responsibility to the public to act more like that since they've really gained our favor by tacitly or implicitly and sometimes overtly suggesting it over the past decade and a half. If that is the case, we have to get them to do better around that. And I've argued this many times, even to folks in industry, when I've had the opportunity to speak with them. You know, the mushy middle isn't going to work anymore. It doesn't satisfy anyone and it gets the
21:33 Speaker 03 firms in hot water. Their own lack of clarity, their own kind of existential crisis about what they are and what they're willing to be will put them in the regulatory crosshairs. So either they can make, kind of make the decisions about what they're willing to do and not do, and then have utter clarity about it to the public, or regulators will do that for them.
21:54 Nilay Patel? Does that clarity start to look to you like they're building shadow governments? Because from my perspective, every time I hear Mark Zuckerberg or Facebook talk about their Supreme Court of moderation decisions, or we're going to have radical transparency in how these decisions are made in appeals process, it just, it just sounds to me like they're building a judicial branch inside of their giant sort of monopolies. Is that the right outcome? Can that function be pulled out of those companies? Is there another way forward?
22:25 Speaker 03 I think there has to be probably a constellation of solutions here. And I will welcome, I welcome any attempt by Facebook and other firms that I imagine are soon to follow suit to bring in more voices, to open up decision-making processes, to receive more input around those processes and so on. However, it is still this strange phenomenon, for lack of a better word, of building a system of governance. You know, does it matter if, if it's a real government or a corporate one? If we spend 24 by 7 of our political,
23:07 Speaker 03 interpersonal, social, community lives within these spaces, it starts very much to look like a government. And then we might ask questions about, you know, well, what about representation and other things like that? So I think one solution that I do welcome is for these firms to push on themselves. To open up. And I think we also have to think about it. Those moves that they're making are in response to or kind of trying to get out ahead of regulation. That's the first thing. You know, I could see Facebook having its body. I could see Article 19 or other advocacy organizations sort of calling for and creating these independent bodies. Of course, then there's the question of, is it going to sit, you know, globally or will there be regional bodies? And how will that play out? And who, who will be called on to sit on them?
23:50 Speaker 03 And, uh, and, and what real powers will they have? And how will we, how will we assess that impact? So I guess from my perspective as a researcher, I have a long career ahead of me.
24:00 Nilay Patel? Yeah. Yeah, I know. You've, you've set yourself up very well.
24:03 Speaker 03 Yeah, I guess so.
24:05 Nilay Patel? All right. We're going to take a quick break for an ad.
24:12 Speaker 04 Your heart can tell you a lot about your health. Apple Watch Series 12 measures your heart rate every five seconds with the most accurate heart rate sensing in a wearable. So your vitals app, now with heart rate variability, can tell you when something is off. And your readiness score can let you know when to rest and when to push. Hear the story in every heartbeat with Apple Watch Series 12. The features described are for wellness purposes only and not for medical use. iPhone 11 or later required. Based on Apple conducted study of heart rate accuracy, August 2026. Visit apple.com slash Apple Watch Series 12.
24:42 Speaker 05 Purchase new carpet from The Home Depot and get it installed for free right now. Shop the stylish choices and simple pricing of carpet from Home Decorator's Collection, Traffic Master, or Life Proof with Pet Proof Technology. And see for yourself why nobody beats our prices. Enjoy new carpet with free installation right now at The Home Depot. Offer valid October 15th through November 1st, 2026. Exclusions apply. For licenses, see home depot.com slash license numbers.
25:12 Nilay Patel? Okay, we're back with Professor Sarah T. Roberts from UCLA.
25:17 Nilay Patel? Are there any inflection points along the way where things might have turned out different that you've encountered, that you've seen where there was a big decision and we've landed here and maybe we would have gone in a different direction had we picked something else?
25:33 Speaker 03 Well, you know, I mentioned in the book, early in the book, that I was a terrible predictor of the future of technology and social media. And part of the reason for that was because I was very embedded in my own experience of the early social internet. I got online in 1993, at which time I considered myself to have been such a late adopter as to obviously have missed the golden era of how awesome the internet was prior to that. But that said, I mean, that's quite a while ago now. And I think in the mid 90s point where we had a kind of a confluence of a few things happening, we had the development, you know, the World Wide Web had already been created, but in terms of it having any purchase, that
26:16 Speaker 03 sort of came about when we saw graphical web browsers developed. And that also came in conjunction with the development of high-speed internet service to the home. And that also came alongside the adoption of the Communications Decency Act of 1996 with Section 230 that we referenced.
26:36 Nilay Patel? Our favorite.
26:36 Speaker 03 Right. So there was this particular confluence of events that was very significant, in my opinion, that came together, not to mention the increased computational power and storage power of personal computers at that time. All of those things came together to sort of conspire to see the commercial internet take off. It went a certain way. It went into this commercial space. It certainly changed the world. As an academic and as someone interested in the history of computing, it's important for me to always remember that the sort of winner takes all narrative isn't always it. So there's a lot of important work, for example, on the case of the French Mintel system and how that looked in a country
27:17 Speaker 03 like France and how that in many ways presaged the rise of the social internet in the United States. So, I mean, for me, I guess I go back to some of that ancient history of the mid 90s where these particular factors conspired to make today possible.
27:34 Speaker 03 But I also, as a person who's a professor in a department that trains, among other people, folks who intend to go into librarianship and in the information professions in the public sphere, I also remark that, you know, the rise and the prevalence of these media that we're talking about, social media, for many people that they serve as their primary news source, as we know, I kind of make the case that there's been a rise in prevalence of that as we've seen a decline in things like funding for other kinds of truly public media.
28:10 Nilay Patel? Mm-hmm.
28:11 Speaker 03 PBS, for example, in the United States, or the rescinding of budgets for places like public libraries. We've seen school libraries shuttered across the country, so students don't even have access anymore to a library in their school. Why? Because you can just Google it, right? Now, we know that that, you know, one-stop-shop from a commercial information source isn't healthy in terms of having an information-rich, well-informed population, and yet we've sort of gone over to that. So I see these things sort of in a longer view and in concert, and we can't talk about the rise of the social internet and people's desire to participate in it without kind
28:52 Speaker 03 of discussing the foreclosure of other spaces and opportunities where they might also participate.
28:57 Nilay Patel? So if you had to go back, this is like the dream question. Knowing what you know, if you were to architect a moderation system from scratch for this moment, what would be your first move?
29:09 Speaker 03 Oh, boy. Well, I think there's a couple of things that would have to go together. The first thing I would do is make that moderation activity knowable and known and understood. So as I describe in the book, one of the hallmarks of "good moderation" in the contemporary social media environment is to leave no trace of said moderation.
29:31 Nilay Patel? They have not achieved that goal.
29:33 Speaker 03 First of all, they haven't really successfully achieved it. But also, the lack of sort of highlighting of that activity has led to all sorts of confusion, problems, a sense that things are uneven, and not knowing who's on the other side of the screen, all of these issues. You know, it was a business decision to decide that content moderators who work in these commercial capacities should be invisible, they should be low wage and low status, their work should kind of go unheralded. And yet, the moderators themselves in the book will tell me, "You don't want to be on an internet without me. You can't handle that internet." Right? So it could have gone another way.
30:18 Speaker 03 It could have been a platform saying, "You should come to our platform because we have the very best moderators who are system guides and tastemakers and who will give you a curated experience," right? Rather than thinking of it as, you know, takedowns and deletions and interruptions of your speech activity. It's a whole different kind of framing. And in part, that came about because Silicon Valley itself was reluctant to acknowledge its own need for human cognition to do these tasks. They're still very much aspirationally oriented towards this notion that at some point computation will take over.
30:59 Speaker 03 And so to acknowledge a whole bunch of human beings whose humanity was relied upon and central to the work that they did was anathema to the politics of the culture in Silicon Valley. So I would probably flip the switch on that to start.
31:15 Nilay Patel? Yeah, you know, Casey says to me, "If you live in a world where your dream is to replace human beings with math, of course you're going to treat the human beings poorly." And that's, as he talks to moderators in his pieces, that's what he comes back to over and over again, is they're designed to be replaced, that's why they're contractors. Have you encountered the sort of AI vision of content moderation? Have you seen their attempts to build it? Do you think it works?
31:38 Speaker 03 Well, I think, first of all, you know, it's a fundamental cultural and political orientation to work. So Silicon Valley puts primacy on any kind of mechanized, routinized computational mechanism, right? There's an inherent belief that those systems are somehow less biased, they can scale better, and they're just somehow preferable. I would argue that there's a lot that goes unsaid in such an attitude, right? Like, here's something that algorithms don't do. They don't form a union. You know what I mean? Yeah, they don't agitate for better working conditions. They don't leak to journalists and academics.
32:22 Speaker 03 So we have to be very critical about that notion. But yes, absolutely, since 2010, as I looked at the work-life and the behavior of moderators on the job, what they were being asked to do, it was very clear to me that the processes that they undertook were decision tree, binary decision tree, if then, if this is present, then do this. If this is not present in an adequate amount, then leave. Go to line 20, like in BASIC, right? And that's an algorithmic kind of thinking that, of course, not only is sort of endemic to the culture, but also would go very easily toward building a system-based tool, a computational
33:07 Speaker 03 system tool, that could replicate. So one of the things that I've been seeing as a trend more recently is the fact that there's this whole, you know, there are whole new kind of pockets of what I consider to be commercial content moderation work that have opened up that may go by another name. And so now what we see are a bunch of humans whose full job it is, rather than to deal with like kind of live content on a system, is to train data sets for machine learning tools so that their decisions are taken up, you know, their decisions on a particular piece of content or on a set of, you know, pre-screened images or something like that.
33:49 Speaker 03 Those decisions are then captured and put back into a computational system with the hope of using that to replicate and then ultimately replace the humans. That said, if you talk to actual industry insiders who Will speak candidly, who are actually working directly in this area, not sort of the C-suite people who are somewhat removed, but people who are a little closer to it. They will tell you that there is no time that they can envision taking humans entirely out of this loop. And I believe that to be true, if for no other reason than what I just described. We need human intelligence to train the machines.
34:27 Nilay Patel? Right. And people are always going to try to defeat the algorithm.
34:30 Speaker 03 They're going to, they're going to try to defeat it. They're going to try to game it. There will be new, we can't possibly imagine all the scenarios that will come online. And of course, those decisions need to be vetted at various points along the decision-making chain. So, you know, at best, what we'll have and what we'll continue to have is a hybrid. But over the past few years, all I've seen is an increase in hiring, not a decrease.
34:56 Nilay Patel? So, we seem to be at a pretty intense moment for the discussion around content moderation. For example, you have just published a well-researched book about it. There's a lot of reporting happening. You know, the United States Congress is holding hearings of minimal value, but they're doing it. It's true. Right. We're paying a lot of attention to content moderation lately. What's the next turn?
35:20 Speaker 03 The next trend? I actually think that it's already happening, and I think it's coming from outside the United States. So, as, as you sort of gestured, our own government has been fairly ineffectual on these issues. I mean, across the board in terms of tech regulation, whether it's because they're woefully ill-informed, or because they are too close to the firms, it's hard to say. I know that when I was watching the Senate hearings last summer, I, I was sitting in my car at a grocery store watching it, and I actually had to turn it off because I couldn't take it, and I went and did my grocery shopping. Came back to it later. But, that said, you know, the kind of reporting you're talking about, which is absolutely in the spotlight because of Casey Newton, but also because of the kind of reporting
36:03 Speaker 03 that, you know, Olivia Solon of The Guardian, Julia Angwin of ProPublica, all of these people have been pushing for years, a lot of that has gotten uptake elsewhere. So, The Guardian, obviously, being from the UK, right? And so, some of the governments outside the United States have taken it a bit more seriously, for whatever reasons. I think in part because the regimes of social media firms.
36:30 Speaker 03 Feels like an exportation of American political values. And I think that other countries, I think the EU in particular and its member states, the UK and other places, Canada, are acutely sensitive to that fact. Right? So if we have, as you were indicating before, an entire governance system being bundled up and exported through the operationalizing of it via content moderation and other kinds of platform rules of engagement and so on, that look eerily similar to things that, I guess coincidentally, support US domestic and foreign policy, other countries are, they're not feeling it.
37:14 Speaker 03 And so they are, they're on guard for that. And it's not the first time that, that countries have felt that way. Again, I look to history to think about the ways in which, you know, IBM owning the computing industry throughout the mid 20th century was seen as a major threat to other places in the globe who rush to build their own state computing infrastructures to avoid that. That's what I think is happening probably most significantly in this moment.
37:43 Nilay Patel? Professor Roberts, thank you so much for coming on. You've given us more time than we asked for, so I really appreciate it. Your book is Behind the Screen: Content Moderation in the Shadows of Social Media. It is out now, which is very exciting. I'm assuming you're going to be doing a lot of promotion around this book. Where can people find you next?
37:58 Speaker 03 Well, they can find me on various radio programs. I'm hoping to do some tours and visits to various campuses. They can follow me on Twitter at Ubiquity75 if they dare. And yeah, I look forward to engaging with everyone who's interested in this issue. And I thank you very much for your time.
38:16 Nilay Patel? It's been a real pleasure, and we'll have you back soon. Thanks so much. All right. My thanks to Professor Sarah T. Roberts from UCLA. You can check out her book, Behind the Screen: Content Moderations in the Shadows of Social Media. It's out now. If you are interested in this subject, highly recommended. We're taking the rest of the week off. It's the 4th of July, so no chat show this week, but we are going to run an episode of Why'd You Push That Button in the feed. Ashley and Caitlin investigated the curious case of Gibbo, which was a robot that people loved, that died. It's a robot that died. It's a real thing that happened. They turned off the cloud service and now people are mourning Gibbo. It's one of my favorite episodes of Button ever made, so we're going to run that this week as your Independence Day gift. Sure. Lights and fireworks, listen to Why'd You Push That Button. We'll be back next week
39:00 Nilay Patel? with the Vergecast interview episode, chat show on Friday. We want to hear from you. You can tweet at me, I'm at Reckless. Let me know who you want me to talk to. I love your suggestions. I love your feedback. Happy 4th of July. We'll see you in a week.
39:18 Speaker 06 New and exclusive holiday decor just launched online at The Home Depot. Check out the viral Grand Duchess collection with smart holiday lights you can control from your phone. Easily customize the twinkle across trees, wreaths, and garlands to match your holiday vibe. Create stunning light effects, then switch it up with the tap of your fingers. Even the man in red will be impressed. Shop the holiday decor you want online only at homedepot.com.
39:48 Speaker 07 Finding a doctor is hard enough. Finding the right doctor, even harder. Someone who takes your insurance, has an available appointment when you need one, and feels like someone you can actually open up to. That's where Zocdoc comes in. Download the Zocdoc app to search and compare doctors by specialty, insurance, and availability. Read real patient reviews, find the right fit for you, and book instantly. Because getting seen by the right doctor matters. Zocdoc. Ready to be seen? Download the Zocdoc app today.
Previous Jony Ive leaving Apple, iPadOS public beta, and Bill Gates’ greatest mistake Next Mourning a robot with Ashley Carman and Kaitlyn Tiffany of 'Why'd You Push That Button?' Page updated October 17, 2026 · transcript and summary are machine-generated
