Former chief security officer of Facebook Alex Stamos

Nilay Patel interviews Alex Stamos, director of Stanford's Internet Observatory and former chief security officer for Facebook. Nilay and Alex talk about how Cambridge Analytica changed Facebook, the tradeoffs big tech companies make with working with law enforcement and keeping users secure, and how prepared Facebook is for the next presidential election. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Soft, cushy, plush, cozy. Soft, smooth, buttery, marshmallow-y, warm. Soft. Socks, underwear, tees, and more. Bombas. Everyday comfort worth obsessing over. Visit bombas.com/audio and use code AUDIO for 20% off your first purchase. That's B-O-M-B-A-S dot com slash AUDIO. One purchase equals one donated, always.

• 0:31Speaker 01

  Real innovation is powered by real people. This is Amex Corporate. Trusted experts, smarter technology, 24/7 dedicated support from day one to answer questions, help set up policies and approvals, and consult on spend capacity as your business needs evolve. Plus, on-demand resources help level up operations at every stage. That's the trusted service you've come to expect. Built to simplify, designed to scale. This is Amex Corporate.

• 1:02Nilay Patel?

  Hey, everybody. It's Eli from the Vergecast. On this week's interview episode, we have Alex Stamos. He's currently the Director of Stanford's Internet Observatory. He explains to us exactly what that means. But you might know him as the former Chief Security Officer for Facebook. Alex and I talked a lot about what is going on with Facebook. He was there during the Cambridge Analytica scandal. How that coverage has proceeded, how it's changed Facebook, the trade-offs big platforms have to make between things like end-to-end encryption, working with law enforcement, keeping users secure from bad actors, what the threats for the platforms are, and honestly, how the media doesn't really understand the trade-offs every platform constantly has to make and navigate as they operate at these scales. A super interesting conversation, extremely thought-provoking.

• 1:47Nilay Patel?

  Alex is a very smart guy. I don't know if I agree with everything. We disagreed a little bit, but I think I learned a lot. Check it out. It's Alex Stamos, Director of the Stanford Internet Observatory.

• 1:59Nilay Patel?

  Alex Stamos, welcome to the Vergecast.

• 2:01Speaker 03

  Hey, thanks, Neelai.

• 2:02Nilay Patel?

  You are the Director of the Stanford Internet Observatory. What is the Stanford Internet Observatory?

• 2:07Speaker 03

  So first off, the word "observatory" shows up nowhere in the Stanford academic handbook. So that's why it is named what it is, and that it's great to be in that liminal space that nobody knows how to handle you. We are a research and teaching program under what is a new cyberpolicy center here at Stanford. Our goal is to build technology and tools and training to help all of the folks outside of computer science to deal with the impact of the internet on their fields. So for example, if you're a political scientist these days and you want to understand how Twitter, or Facebook, or some other kind of technology is impacting your field, the activation energy for the amount of work you got to do is really high, right? You got to get a grad student, have them write some Python, get access to the Twitter API.

• 2:50Speaker 03

  We're trying to build all of these tools and techniques one time and then to offer them to a variety of different people from across the social and political sciences to understand what's going on. So we see ourselves as both a group that will do its own research, but also really a support and engineering team to try to help academia deal with the incredible impacts the internet is having on society.

• 3:12Nilay Patel?

  And it's a theme I think we're going to come back to a number of times. There's an element of the internet right now where just to look at it correctly and deal with what it's become, you have to either reinvent the wheel at a very high scale over and over again, or you have to find a service provider that can do it for you. It sounds like you're kind of like, "We're going to help you look at the internet and measure it correctly, so you don't have to do it yourself."

• 3:33Speaker 03

  Yeah, exactly. The term "observatory" is actually not an accident, right? Astronomers have figured this out. If you're an astronomer, you don't raise $5 billion and build your own Hubble Space Telescope, right? You have a theory of what you want to study, you go rent time on the Hubble, or the Aracibo, or the very large array or something. So we want to do the same thing, like build a set of capabilities that then are going to get used in ways that we have no idea how they'll be used. So I mean, it's a theory, right? Our thesis is if we build it, they will come. But so far, we've had a lot of interest from a variety of different academic groups who have deep expertise in different areas of society, or area studies, for example, but don't have the technical wherewithal or the access to data to really do this kind of work.

• 4:14Nilay Patel?

  Okay, so that's now. Before that, before this, you were the chief security officer at Facebook. I mean, before that, you were the chief security officer at Yahoo.

• 4:22Speaker 03

  Yeah, I like boring jobs. I like to go from...

• 4:24Nilay Patel?

  Well, those were not boring moments for either company. Obviously, I want to talk to you about Facebook a little bit, but we just had Michael Bennett, a senator from Colorado, on the show, and he wrote a book about election security. That's obviously, when you think about Facebook, it was the, it was the center of election interference, basically like posting memes from Russians on Facebook. I'm curious, do you think, you think we're, we're ready for 2020 right now? Because Bennett really did not think that we were ready.

• 4:51Speaker 03

  Yeah, so I, I've met with Senator Bennett, and we've talked a lot about this kind of stuff. You know, so we just put out a report from our group at Stanford. You can go to electionreport.stanford.edu if you want to see it, but we have around 40 recommendations for how Congress, how tech companies, how individuals can prepare for 2020. If we look at 2016, there's actually three or four different kinds of interference by the Russians, right? So you have what you referred to, which is the, the online meme wars, which was mostly on Twitter and Facebook. You have the GRU hack and leak campaign. So that was the campaign of breaking into Podesta's email, breaking into the DNC, and then leaking out information in a way that changed the overall information environment to the detriment of Hillary Clinton.

• 5:35Speaker 03

  There's the overt propaganda campaign, so there's Russia Today and Sputnik and such. And then there was the direct attacks against election infrastructure. So I think our response as a society has been different for those four different lanes. So on the, the kind of meme lord stuff, I, I think that's actually where we've been best prepared in that the responsibility there kind of cleanly falls to the tech platforms. And they have done things. I mean, the big difference between now and 2016 is organized government propaganda was not anybody's job at the tech companies in 2016, right? So I kind of inherited this as an issue that I, you know, got to, to lead the team that, that worked on this. Because we had a intelligence team whose job it was to look for governments doing

• 6:18Speaker 03

  bad things online, that was based upon a very traditional idea of what is government interference online, malware, account takeovers, suppression of dissidents. It did not include hot takes and edgelording by people pretending to be Black Lives Matter activists who are actually sitting in St. Petersburg. And so, a lot has changed in that that is now an entire field of, kind of subfield of trust and safety, is being invented right now at places like Google and Twitter and Facebook, and that there are people whose entire job is to do that. And then, the government has reacted as well, in that there are people inside the government side whose job it is to work on these issues. I think if you did a kind of a big look at 2016, as a society,

• 7:03Speaker 03

  we had this big blind spot, because it was really nobody's job to be tracking the Internet Research Agency and the other kinds of online propaganda outlets, because it wasn't considered a traditional part of cybersecurity. And so now, inside of NSA and Cyber Command, there's people working on this, there's a Foreign Influence Task Force in the FBI, there's people working on this at DHS, and they're working with the folks in the company. So, whether the precautions are great or not, at least, this is now a field that people are focusing on, and that was not true at all this time, three years ago.

• 7:31Nilay Patel?

  I mean, too, not to credit the Russians, but it was a pretty innovative technique. I don't think anybody saw it coming. They came up with it, and they employed it at mass scale, to what appears to be the effect that they desired.

• 7:44Speaker 03

  You've got to be careful ever saying, "nobody saw it coming." The truth is, for any bad thing that's happened on the internet, somebody called it. It's really easy to predict that something bad's going to happen on the internet. The truth is, is when you're in a defensive role, you always have a finite amount of resources, and there is an infinite space of possible badness. And so, you do have to have some kind of evidence that something deserves you to put some finite resources on it. And it's true that, while the Russians, this is not the first time somebody invented this, this is by far the deepest use of this kind of propaganda in an international context. If we had been paying more attention, we would have seen this as in the Ukraine.

• 8:21Speaker 03

  Everything they did in the United States in 2016, they've done in Ukraine for a lot longer period of time. But I think even people who were paying attention saw that as something special, right? That the Russians would treat a former Soviet state, their neighbors, and that they would never use the same kind of techniques against the United States. And that just turned out to be an incorrect assumption that a lot of people made.

• 8:41Nilay Patel?

  So as you're thinking about looking at the internet now with the tools you're building in the regulatory, as you're thinking about trust and safety into 2020, the push from Facebook to become a more private platform, do you think it's gonna get harder or do you think it's gonna get easier?

• 8:58Speaker 03

  Yeah, that's a great question. The move to, for Facebook towards a more private platform, I think, this really raises fascinating questions about what do we want out of social networks? So if Facebook and other social networks move towards small group messaging to less amplification, it will reduce the amount of spread you can get of propaganda and hate speech and such with a small number of people, right? So, you know, when you look at kind of the Russian model, they were looking for lots of amplification. You might have one or two people in St. Petersburg running a persona. They would get hundreds of thousands of people to follow that persona over a multi-year period. And then if they had some truly hot spicy thing that they wanted to post, their goal was to get millions

• 9:42Speaker 03

  of people to see it versus reshares, reshares, reshares. And so that kind of amplification becomes much less likely in a future where everything is much more like Snapchat or an Instagram story. The flip side is, things become invisible to those who are watching for it, both on the outside, like us, but also to the people inside the companies. And I think this is where this actually starts to become a hard problem. And kind of one of my bugaboos about kind of the media, especially the way they talk about tech companies, is there's a lot of kind of wanting it both ways without dealing with the fact that there's hard trade-offs here. And for a couple of years, everybody's been talking about privacy, right? Like, the worst possible thing that's ever happened in the history of mankind is

• 10:24Speaker 03

  the Cambridge Analytica scandal. And so if that's true, then you should love the fact that Facebook is encrypting everything and making everything private. But- The truth is, is when you give privacy to people, you also give privacy to bad guys. And so the turnabout here about giving people privacy is it will greatly reduce the ability for the companies to police their own networks for bad activity and for people on the outside to spot it. So in the long run, that's good for the social networks because they don't have to be responsible for things that people don't know are happening, but the actual harm will continue to happen. And disinformation campaigns work great on end-to-end encryption. Like if you look at what's just happened in the Indian election, is that it turns out that both the BJP and the Congress party were running their own troll farms. But instead of getting one message seen by a million people, what they do is they get

• 11:08Speaker 03

  tens of thousands of people to forward the message to all of their friends and family members. And so you get just as much amplification, it just takes a little bit more work. And since it's all encrypted, it's very hard to study it. So I think for the most part, the move to privacy, there are good things. I'm pro-privacy, I'm pro-end-to-end decryption, but let's not hide ourselves from the fact that this will be better for certain kinds of abuse.

• 11:31Nilay Patel?

  Yeah, it's funny, just as we're talking today, this morning, last night into this morning, there was like the Instagram copyright hoax that just like flew over that network, right? And obviously stakes are very low, like, doesn't matter. But you can't actually stop the people from posting that shit, right?

• 11:50Speaker 03

  Oh, Facebook absolutely could stop that from happening. They don't because Facebook never uses its power to defend itself from things that criticize itself, right? Like Instagram completely has the ability to take that copy pasta image, to generate a perceptual hash and to ban it from Instagram and Facebook, but the company will not use that kind of power for something that is seen as criticizing them.

• 12:11Nilay Patel?

  Really?

• 12:11Speaker 03

  So it is an interesting thing that if that was a screenshot of the Christchurch manifesto, it would not be spreading on Instagram like this, right? This is just an interesting issue in that the companies are always very careful to utilize power in a way that can be seen as doing so for proprietary purposes. If actually, these companies were as evil as people thought, there'd be a lot more subtle manipulation of people's anti-tech verge articles, for example, would be down ranked and stuff. But there's never any evidence of that happening because they actually never want to utilize the power and therefore, you know, attract all of the negative attention that would create.

• 12:44Nilay Patel?

  There are no anti-tech Verge articles. We're just, we're just

• 12:47Speaker 03

  I'm sorry.

• 12:47Nilay Patel?

  We're just here to make it better, man. We criticize out of love.

• 12:51Speaker 03

  No, and I shouldn't use The Verge as the example. Let me, we'll use The New York Times as, as the organization that allows their distaste for the tech industry to override their critical thinking at some point.

• 13:02Nilay Patel?

  Uh, this is a very rich vein of conversation. We're gonna, we're gonna come back to that. But I want to just stick on the, that tech companies have a lot of power that is sort of undisclosed. So this, this to me is like the core of it. Can Facebook go into WhatsApp and say, we want you to stop sharing these memes that are obvious troll farms from the Congress party, the BJP?

• 13:20Speaker 03

  So without breaking end-to-encryption, there are some options. So this is actually an area that I've been working on a bit. We're, we're holding a event here at Stanford where we're gonna have the major tech companies. So we've got, uh, Facebook, Microsoft, Google. Unfortunately, the, the fruit company hasn't been interested yet, but I'm gonna keep on trying. Uh, we're gonna have people from ACLU, EFF. We're gonna have Nick McThorne, so there's a child safety advocacy organizations. We have representatives from GCHQ and FBI, so we have government folks, and we have a lot of academics. Uh, and the whole point of the conversation is that the entire end-to-end encryption conversation gets stuck on the backdoor conversation, right? That governments want it backdoor, companies don't want to give it to them. We don't have to have that argument. I'm anti-backdoor. We'll just, I'm just gonna say that.

• 14:04Speaker 03

  But like that, you could spend hours and hours going around without any progress. And as a result, we don't talk about all of the other options. And it is totally, there are possible ways that you can make some privacy trade-offs to push certain kinds of intelligence into the client that allow end-to-end encryption to happen, uh, but then prevent people from doing certain things. So in theory, uh, WhatsApp could regularly download a database of, of effectively a number of different kinds of, you know, compressed data structures that represent thousands of different known, uh, fake stories or rumors, uh, could do a matching on every single message that comes through, and then without violating the privacy of the user, pop up some kind of counter messaging, saying, hey, this looks like this is disputed fact, or it's not true

• 14:48Speaker 03

  that, uh, these people, uh, harmed these girls in the next village. Uh, please don't forward this. And then do things like Forwarding that message, that would be a decision to reduce the freedom of the user. Right? Some people would call that breaking end to end. I think we've got to be careful in our language here. There are situations in which you can reduce that freedom, but so far the companies have have been very careful not to use utilize their power to do that. I think that's a discussion we have to have because I do think there are situations in which we should be providing people with privacy, but we should also blunt the damage that they can do through the incredible, the fact that now they have the ability to reach hundreds of millions, perhaps billions of people in an extremely private manner. There are downsides to that, and there are situations in which we might want to reduce the choice that people have of how they're going to use that.

• 15:33Nilay Patel?

  So I mean, that's a, that's a huge conversation, a huge debate, but you're, you're saying it should be focused on before you ever get to send the bits off your phone, the client locally should be doing some screening of, hey, we know this is fake news. Like, we know this is, we know the Pope didn't endorse Donald Trump. Like, why are you, why are you sending this off?

• 15:52Speaker 03

  Yeah, so I'm not, I'm not endorsing that exact solution. What I'm saying is that there are options, both for the people receiving information and people sending information, to have intelligence in the client. Right? An easier example is there's a bunch of forms of abuse that for which the victim of the abuse is part of the conversation. Right? So let's say, you know, unwanted pictures of male genitalia is something that dominates every messaging product. Like, if you were a woman and people can reach out to you in an anonymous fashion, you will be sent these images. There's no reason why you can't push the same AI Facebook currently uses on its servers to look at an image and say, this looks like male genitalia, based upon this machine learning model. There's no reason why you can't push that into a client and have, you know, effectively Clippy pop up

• 16:36Speaker 03

  and say, hey, somebody's sending you images that are sensitive. I've blurred them out. Do you know this person? If you don't want to receive these images, you can report it now, and then you could break the security of that encryption by them turning over the encryption key for that specific conversation, and then allow the company to take care of it. Right? So I think there are a bunch of different forms of abuse where you can reduce the impact via, you know, more intelligence in the client, through smoothed out reporting structures, through interesting crypto stuff that allows you to report specific conversations, but not all of them. But in the long run, it means the company's asking, acting in a more paternal way than they traditionally have. And I think this is where the companies have got themselves in. A corner here because they don't talk about all the things they do right now.

• 17:20Speaker 03

  Right. And the truth is, is that as bad as the internet is right now, as bad as it is for women and for children and for people who don't like getting abused by Nazis online and don't like getting hate speech and don't like getting images they didn't ask for, the companies are doing a huge amount of what is semi creepy stuff with machine learning, with image recognition and such to police their networks right now. And most of that goes away in an end encrypted future. And so the fact that the companies don't talk about it means that the baseline of the conversation is actually pretty bizarrely twisted. And that makes it hard to have kind of an intelligent conversation about what should happen next.

• 17:54Nilay Patel?

  I mean, do you think Facebook is moving to the end to end encrypted privacy, we're going to merge the back ends of the three services purely cynically? Or is that a good business decision? Or is it just we're going to evade the Department of Justice antitrust division?

• 18:08Speaker 03

  First off, Facebook never makes a product decision without hard data backing it up. Right? Nobody is sitting on more good data about what people want to do online than Mark Zuckerberg. Right? He has instrumentation from Facebook, from Instagram, from WhatsApp. And so clearly there are things showing that the features of Facebook's existing social networks that that protect privacy, that are small group, that are ephemeral, are more popular. So I am sure that is absolutely the number one goal is to, you know, Zuck has always been really, really good at seeing where the puck is going and skating there. Right? Just as he did with the purchase of Instagram, just he did with the Instagram stories. And so I am sure that that is the biggest one. But I do think that there is

• 18:48Speaker 03

  a probably a cynical component to the encryption in that effectively the companies are being held to two standards that they can't live to. Right? One is, we want you to provide privacy and not know anything about these users. And the other is, we want to keep people, you to keep people completely safe. You can't have both those things. Right? Like in the same week, the New York Times will both criticize Facebook for not finding every bad guy, and then criticize Facebook for having too much data on people. When you, when you give people privacy, again, you give privacy to bad guys, too. And so historically, what Facebook has done is tried to like, have this kind of, if you're on a one to 10 scale, been in like the four to six range of like, let's be in the intermediate on some of this stuff. And that's not working because that opens you up to be able to criticize you from the one or the 10. And so Zuckerberg is just choosing to slam all the dials over to one

• 19:33Speaker 03

  side. We're going to encrypt everything. We're going to throw away all this data. Everything's ephemeral. And so he is maximizing privacy with, but also then reducing the ability to do safety and especially content moderation. I think the other reason is that Facebook has demonstrated that there is a trap you can never get out of, which is, once you start moderating people's content, there is no logical end to what people will ask you, what speech by other people, you will be asked to control. Right? And so if you're on this slope and you're trying to deal with 97 different legal regimes for speech, then probably the only way out is to put yourself in a situation where you can do the minimal amount of content moderation. And I think that that's part of it, too. The whole DOJ thing, I find that kind of silly. Like, if that is Facebook's goal, that

• 20:16Speaker 03

  tying these things together stops antitrust action, that's really stupid. Like, the Department of Justice broke up AT&T, right? Like, the most complicated thing ever built by man at the time was the phone network. And the Department of Justice broke up AT&T, probably actually caused all kinds of problems and reduced the amount of innovation that you could see out of the baby bells, but whatever.

• 20:37Nilay Patel?

  Oh, I totally disagree with you. On that point, specifically, I totally disagree with you. Yeah.

• 20:42Speaker 03

  Okay. Well, I've heard both things on the AT&T breakup, right? That like Bell Labs loosened, like that we lost a lot out of having it. And then also, you know, the phone network still uses like SS7, which is the signaling system that was built by AT&T pre-breakup. Like, it's the fact that everything has to go through like a standards body now in the GSM form and all that kind of stuff, maybe reduces the speed at which those companies moved. But, you know, the benefit of that is that kept, that opened the door for the internet companies. So, I mean, I'm sure you can see it both ways. Whatever happened to AT&T? It's like, obviously DOJ didn't look at AT&T and be like, it's too hard to break this up. So like, I think that's a silly calculation for Facebook to make.

• 21:19Nilay Patel?

  Well, I don't want to get too lost in the AT&T thing, because I love thinking about it, which is dangerous for a podcast host to be like, We're gonna talk about this one lucky thing from 100 years ago that I love, but I think that DOJ was obviously, at that moment, much more motivated to do the thing than our DOJ is, even though it seems like the drums are rising in volume than they are to go after Facebook, right? And I think you see that over and over again. Like, they try and they bounce off, and they try again and they bounce off. And I think Zuckerberg, as he pulls the services together, creates more of a bulwark against it.

• 21:52Speaker 03

  Maybe. Well, so you're assuming that DOJ actually wants to do an action. People in the media are, all of a sudden, much less cynical about the Trump administration when it comes to the Trump administration's investigations of tech companies, right? So, a truly cynical read is that the President of the United States controls the Department of Justice, controls the FTC, controls the SEC, right? Control has a huge amount of power to make your life hell if you're the executive of a Fortune 500 company. And so, if you are the Trump administration and you do not want Facebook and Twitter and other people enforcing their content moderation policies, if that means shutting down your supporters, then a great thing to be doing in 2019 is to be flexing your muscles and to be threatening as many investigations as possible to demonstrate that you have

• 22:33Speaker 03

  the ability to make, you know, Facebook and Twitter's life hell in a way that the quote-unquote liberal media will totally applaud. And so, I think there's an interesting question here of how much actual antitrust interest there is and how much there is of, you know, the Josh Hawley kind of, "I'm gonna make you a little bit afraid so that every time you make a difficult decision that might have a disparate impact on somebody on the right wing, you think twice about it."

• 22:58Nilay Patel?

  Yeah, and that Josh Hawley proposal is, like, the FTC will become the speech police. I mean, that's an outlandish proposal in terms of its scope.

• 23:06Speaker 03

  It's the most ridiculous thing I've seen. And it's like, they should take away his Yale JD for even proposing something like that.

• 23:11Nilay Patel?

  Yeah, I mean, Ted Cruz, Harvard lawyer, like, maybe should read the statute, right? Like, it seems very confusing, and I agree with you, it's designed to chill content moderation decisions. So I think putting it together, you just described in a way that I don't think I've heard crystallized before. We obviously cover content moderation a lot. Casey Newton covers, like, the lives of moderators. I've suggested to him that one of the defining questions of tech in this moment is what goes on the internet, like, what belongs on the internet. He suggested to me that there's like a sub-question of like many other things, so it's a little discreet, but I think it's one of the most important questions of like our time in tech. And I don't think I've ever heard anybody crystallize it as, hey, I can turn all the

• 23:53Nilay Patel?

  dials to 10 and actually take this responsibility away from myself. I can devolve this responsibility if I just say everything's private. I can, I no longer even have the opportunity to moderate. Do you just see that at Facebook? Do you see that elsewhere?

• 24:06Speaker 03

  I think you'll see it elsewhere. I mean, Facebook is on the cutting edge of dealing with these issues. There's, you know, about 80% of the, the things that people write about Facebook, they're really writing about the entire sector, but Facebook happens to be the biggest, uh, and so therefore is the first one dealing with it, right? But if it works for Facebook, then you're going to see other people follow for sure. This is really about like, I actually, if, if this is what Casey's idea is, I do see the content moderation as a sub-issue. Like, the big picture issue is, as a society, we have not decided about, A, how safe should people be kept online and how much we want to control their choices to keep them safe, for what definition of safe you have? And B, who does that? Right? Like, one of my colleagues here at Stanford, Daphne Keller,

• 24:50Speaker 03

  likes to talk about how a lot of people have an irrational exuberance for speech control that they never had in the pre-tech era, right? That we have hundreds of years of people trying to figure out what is legitimate political speech, what is hate speech, what are you allowed to say? And that what the internet and private regulation has done has allowed people to advocate for speech controls that would never have been considered acceptable for the last couple hundred years. And so, all of this built up interest in controlling the speech of others is coming out all at once, right? But that is a sub-issue of like, how safe are we going to make the internet, right? Like, is it going to be a rubber room that you can't hurt yourself in? And then if so, are the rules going to be made in a democratic manner, or are they going to be made by private actors?

• 25:29Nilay Patel?

  And so what's interesting to me about that is, the government really can't make any of those rules. Right? I mean, the First Amendment exists-

• 25:35Speaker 03

  The American government can't.

• 25:36Nilay Patel?

  Right.

• 25:36Speaker 03

  Right? The American government can't. And that's why most of the action on speech regulation is happening elsewhere. Effectively, every non-US Anglophone country is currently considering some kind of ridiculous online speech control, right? The Australians move first, uh, as is their want. But there's this kind of nutty paper, uh, by DCMS in the UK- Which only talks about the downsides of online speech and never talks about the upsides or the civil liberties issues. But clearly, it's pointing in a direction that other countries are going to move more aggressively. And so, when you think about any decision Facebook's making, you actually can't think about the United States, right? Something like, I think it's less than 5% of Facebook users are Americans.

• 26:18Speaker 03

  You've got to consider the international ramifications first. And I expect that the giving people privacy and the flip side being less speech moderation is specifically targeted at a number of countries. Because if you think globally, what is passing? Is it laws that say that more content moderation should happen? For the most part, no. What's happening is we see pro-privacy laws like GDPR passing. So if you put all, there is a trade-off here between privacy and safety. And if you put all of the legal weight on one side of this equity, then the companies are eventually going to respond. And I think that's part of what's going on here, too.

• 26:52Nilay Patel?

  So, we're having a conversation that is complicated, right? There's all this set of trade-offs. There is multiple legal regimes around the world that are potentially in conflict around what's allowed on the platforms. There is building this stuff, which is hard. Then there's like the threat models, which are increasingly sophisticated. You tweeted a couple days ago, "One of the reasons we're putting together a trust and safety engineering course for undergrads at Stanford is the fact that every product with social user-generated content feature now needs a full-fledged trust and safety team. That's a lot of startups need specialized talent." So, I'm curious, like, I want to hear about that course. Like, what does it look like? And two, should this just get pulled out of the companies?

• 27:34Nilay Patel?

  Like, can you, if I'm a person and I want to start a photo-sharing startup to compete with Instagram, because I think the market needs more competition, and that's what I'm going to do, is there a world in which Instagram just spins out its trust and safety team and that becomes a resource like AWS is a resource? Is that, do you see that as a potential solution here? Because otherwise, it seems like no startup stands a chance.

• 27:57Speaker 03

  Yeah, so, to work on the definitional issues a bit, so, effectively, what I, something I learned at Yahoo is that If if my goal was to maximize my positive impact on people, then I had wasted most of my career in that the vast majority of bad things that happen to people online have nothing to do with all the sexy security issues that people like me love to study and work on, right? It's not about ODA, it's not even about malware or password resets. The most harm that happens online is what we term abuse, which is the technically correct use of these products to cause harm, right? The people who deal with abuse at most companies is generally called like a trust and safety team. At Facebook, they now call it integrity, but nevertheless, it's a a different field than traditional information security.

• 28:39Speaker 03

  And one of the things we're trying to address here at Stanford is you can't learn anything about this entire field of software engineering except by going to one of these big companies and becoming an apprentice on a team. So if you want to become the world's best expert in online fraud, you go and you work at PayPal and you go work on their trust and safety team for several years. But you can't, none of that makes it back in academia. So we're we're building a course to do that. I have 18 people from a number of companies. So we have Google, Microsoft, Facebook, people from Nickmick and Thorn. I have some law enforcement people. I have a bunch of folks who are helping create this content. And then the goal is to open source it so a number of universities can teach it. But we're, we're going to talk about content moderation. We're talking about hate speech.

• 29:24Speaker 03

  We're talking about bullying and abuse, non-consensual intimate imagery, which is effectively revenge porn and dick pics. We're going to talk about child sexual exploitation. We're talking about human trafficking. We're talking about suicide and self-harm. So it's a, it's a real uplifting class. I'm looking forward to people, you know, like, rate my professors as a little pepper for the hot professor. I'm expecting to get the little sad face. We'll be like, this is a super depressing class. But the truth is, is like, you know, we've got to graduate students who understand that when you, when you allow people to communicate online, when you create situations where images can be uploaded or any kind of interaction can happen, there are risks that are created by that. And it's not acceptable in 2019 to start a company and then to be shocked that this kind of stuff happens. Now, to your second question, I think you're totally right.

• 30:08Speaker 03

  Like, one of my overall theses is that on trust and safety issues, where were we were on information security in the late 90s, early 2000s, which is In the late 90s, nobody knew how to write secure software. Nobody knew how to build security into a software development cycle. And Microsoft got all this criticism, not as the only people who built bad software, but as the biggest. And they reacted by building out what has become the ways that people do product and application security now across the entire industry. We effectively need the same revolution in trust and safety. And coming with that is not just the companies changing. It means academia reacting. It means having kind of a mindshare change of what people study and learn. And then also the creation of an industry.

• 30:53Speaker 03

  So in the late 90s, if you wanted to build secure software, there's effectively nobody you could hire to help you with that. If you have a Nazi problem today, like say you start a company and all of a sudden the Nazis take over. So a great example of this is Discord. You know, Discord starts as a chat for gamers and very quickly ends up with this white supremacist, radical underbelly that they were not expecting to have. If you have that kind of problem, there are very few people you can call to come help you with your Nazi problem, or with your child safety problem, or your suicide problem. And so I think you're right that this is actually going to become an entire industry, including managed services, including cloud services, that you can't expect that every company builds all these things from scratch.

• 31:35Speaker 03

  And maybe that becomes actually a business line for a Facebook or an Instagram, is that, you know, just like Amazon turned their experience scaling infrastructure into AWS, you could see Facebook turning their experience scaling trust and safety into an actual service they provide to all of the smaller companies who can't afford to hire thousands of people themselves.

• 31:54Nilay Patel?

  So that leads to a really interesting question about size and scale, right? If Facebook is the biggest and it's hard to compete with them, who would they sell to? Like, there's like a trade-off here of, are we just going to have a few giant incumbents and we push them and they sort of compete on their own trust and safety metrics? Or is there going to be a wide ecosystem of companies and there are like service layers that they need to become their own economies and their own ecosystems? Like, I would love, we run a commenting platform, we run Chorus, right? We have our own social channels across different platforms. I would love to hire a company to come in and make sure our communities across our platforms, our owned and operated and out in the world, are healthy.

• 32:38Nilay Patel?

  But we are not a big enough market to support, you know, two or three independent companies doing that. How do you see that market developing?

• 32:45Speaker 03

  Well, I think it is a lot like the cloud computing market in that, you know, 15 years ago, if you said that even decent-sized companies, it was not economical for them to build and run their own data centers, you would have said that's crazy, right? But that's where we are right now. That unless you are humongous, it makes very little sense for you to have your own data center operations and perhaps even kind of your server operations. There is an interesting question here, and this actually comes back to the anti-derection question, right? Because there's a number of kind of arguments around, if you break up Facebook, then the safety issues become impossible to solve. I think that's a poor argument. I mean, it's true in the short term, the Instagram trust and safety team, like the Instagram anti-abuse team, is the Facebook team, right? So, yes, if you broke up a Facebook over a three-month period,

• 33:30Speaker 03

  you would have lots of problems and that you couldn't backstaff. But a company of Instagram's size, standing alone, should have the ability to run its own trust and safety. And kind of where we need to work on is the cutoff here, I think, is Twitter, right? Like, all of the companies that have some kind of user-generated content, Twitter is the smallest one that is at least holding their own. They don't do everything great. But a lot of that is actually intentional or product decisions. But, you know, they do have people that work on things like Russian disinformation, right? Below Twitter, it is a wasteland, right? Like, of the smaller organizations than that, like the reddits and such, it's not their fault, but it is very difficult for them to have a capability that is scaled enough and that can touch all of the different areas of abuse well enough to run a full-fledged user-generated content site.

• 34:14Speaker 03

  And so I think there are, I mean, we don't think about it, but as much as people talk about Facebook and Twitter, there's this huge long tail of user-generated content, right? Of all of these different sites, all of these different discussion forums, most of which are doing nothing in this area. And so we don't think about the possible impact. I think that's actually one thing we're interested in trying to study in our new project at Stanford, is the impact of that long tail on disinformation operations. Because especially when you look outside the United States, you'll find that lots of countries have one or two products that we've never heard of in the United States that are actually quite important within that area. And so as a result, almost by definition, they don't have people who are prepared for these kinds of issues. And so I think the, you know, in some of these specific areas, it really is a big deal.

• 34:59Speaker 03

  But anyway, I do think it's going to develop. I think as the expectations rise about You know, I think what Facebook is trying to do is they're trying to turn this into a competitive advantage, right? Of like, they're gonna do all of this work on different kinds of abuse and then try to get the legal bar to be set right below wherever they are at that moment. And then hopefully for them, that bar is higher than what anybody else can do. So if we end up with laws that actually, you know, dilute CDA 230, or in other countries create some kinds of liability for carrying content, then I think you could see a growth industry in this. I think one of the places to look is where this has first started is Germany, that, you know, Google and Facebook were able to adopt to NetsDG pretty quickly, and there's a number of other companies that have had more struggles.

• 35:41Nilay Patel?

  That's the law that prevents hate speech online, right?

• 35:43Speaker 03

  Yeah, exactly. So that's like the, basically a German law saying that you have to enforce German hate speech, anti-hate speech law, or you are liable, and that has been a real challenge for smaller companies. And my understanding is that there have been a couple of very specialized companies that have popped up to specifically help with enforcement in Germany. The question is if you have enough laws globally, then maybe this becomes a global enterprise that you could run.

• 36:08Speaker 04

  Your heart can tell you a lot about your health. Apple Watch Series 12 measures your heart rate every five seconds with the most accurate heart rate sensing in a wearable. So your vitals app, now with heart rate variability, can tell you when something is off. And your readiness score can let you know when to rest and when to push. Hear the story in every heartbeat with Apple Watch Series 12. The features described are for wellness purposes only and not for medical use. iPhone 11 or later required. Based on Apple conducted study of heart rate accuracy, August 2026. Visit apple.com slash apple watch series 12.

• 36:38Speaker 05

  New and exclusive holiday decor just launched online at the Home Depot. Check out the viral Grand Duchess collection with smart holiday lights you can control from your phone. Easily customize the twinkle across trees, wreaths, and garlands to match your holiday vibe. Create stunning light effects, then switch it up with the tap of your fingers. Even the man in red will be impressed. Shop the holiday decor you want, online only, at homedepot.com.

• 37:09Nilay Patel?

  So I think we have a lot of listeners who are product managers who build things, who start companies. Where should trust and safety live in their product development cycle? Because I also know a lot of trust and safety people who, as you've said, they can see the problems coming a mile away. They just can't get in there early enough to stymie them. So should every product manager think of themselves as a trust and safety person? Should it be built into the composition of every team? Should it be more diffuse than a single department?

• 37:35Speaker 03

  Yeah, I think where I think we are going to end up with both kind of core cybersecurity teams and then the broader trust and safety is that you will continue to have centralized teams, but that those centralized teams will be consultants, service providers, and auditors, and will not be the owners of risk. So I think at a company, a smaller company that only has one product, then trust and safety is probably part of the product organization with an offsuit over whatever customer operations team that you have to handle kind of operational tasks, right? Running a team that has engineers, investigators, with people who work the midnight shift, and actually organizationally, it's difficult to work those kinds of things in the same org chart. So often what you have is trust and safety teams are actually split, where you have

• 38:19Speaker 03

  engineering and product investigations as one part, and then the day-to-day operations as part of a larger operational team who's being managed to SLAs of effectively like a call center, right? Of the stuff that Casey writes about.

• 38:30Nilay Patel?

  Yeah.

• 38:31Speaker 03

  So I think that's reasonable for a company that has one product. If you have multiple products, then I think what we'll end up having is what we probably should have in security, which is you have a central security team whose job it is to help all of the product owners judge and manage risk, but not to make the risk management decisions themselves. Like, one of kind of the problems that we've had in Silicon Valley up to this point is that the SVPs and VPs who own lines of product are measured on growth, they're measured on money, they're measured on the success of the product, they are never measured on whether or not they generated a huge amount of risk for the organization. And that's got to change. Because, and the place we can look for this changing is the banks, right? Like, post-2008 crisis, the financial services industry has really reorganized its risk management.

• 39:15Speaker 03

  They have chief risk officers, but they also have all kinds of controls in place to push risk management out to the owners of different lines of business. So you can't go to Jamie Dimon and say, "My division of JPMC just made $10 billion, but it's all fraudulent accounting, but I still want my bonus," right? Like, that kind of thing is not acceptable when it comes to financial risk, and we need to get to the same place on kind of the general tech risk of cybersecurity plus trust and safety risk.

• 39:42Nilay Patel?

  And so that works for the big company, but if you're starting a new company, what's a proposal you have? Like, in your class around trust and safety, how do you teach a student or a startup founder to think about that risk from the jump?

• 39:54Speaker 03

  So, part of the goal of the class is, is most companies, just like they don't have a CISO, most startups, they're not going to have a dedicated trust and safety resource, and that's probably fine. What you really need is, you need the people in product and engineering to understand the classes of issues that have been seen in the past, so that they themselves are responsible for considering it, right? So I'm not going to argue that if you're an eight-person startup, that you should then, your ninth person should be a TNS engineer. What would be great is if the engineers you have already have a bit of a background, so when they're like, somebody proposes, "Okay, I have an idea. Let's take anonymous photos and then encrypt them and send them to women without them knowing where it came from," and, you know, that they have in their mind, "I know the kinds of things that can go wrong in that kind of scheme." In the long run, when it's time to hire the first person, I think they absolutely have

• 40:39Speaker 03

  to be part of product, right? Because when it comes, this is true for security overall, but especially when it comes to trust and safety, what you end up happening is that big product decisions about how the product works is what accumulates 99% of the risk in that initial huge decision, right? So you're effectively filling up a orange Home Depot bucket full of risk, and then you give your trust and safety people a little spoon to go try to spoon out that risk before something bad happens, right? And that doesn't work. And so like, when the product decision is made for hypergrowth or some kind of growth hacking, or because people want it, then the part of that initial decision needs to be, "Okay, well, there's a potential downside here, and we need to prepare for that downside." Doesn't mean you don't take the decision, but it means if you're looking for the downside,

• 41:23Speaker 03

  then you're much more quick to respond. That's the other thing about trust and safety, is it's a lot harder thing to predict than traditional cybersecurity, right? Like, we're moving towards... A steady state where if somebody is building a mobile app that does this and that and it's up in AWS, you can hire a consultant to list out for you the top 12 potential things that are going to go wrong from a security perspective, but the trust and safety issues are incredibly specific to exactly how the product works and then also specific to who's using it, what society they're in, cultural issues, language issues, legal issues. And so it is a harder thing to predict. So as a result, more of your effort is going to be on detection and response than it is on prevention. That's an okay thing, but you then have to, just because you can't predict that a risk is going to happen, doesn't mean that you should not

• 42:05Speaker 03

  be staffed up to be ready for if something bad happens. I think that's part of the thing for startups as well, is they get caught behind the ball of the first time something bad happens, they're not finding it themselves, they're reading about it on The Verge or in the New York Times or in the Wall Street Journal. And if you're reading about it in the papers, the first time you knew you have a trust and safety problem, then you're already in deep, deep doo-doo.

• 42:26Nilay Patel?

  (laughing)

• 42:27Speaker 03

  Sorry. I don't know. Does this have an explicit label?

• 42:29Nilay Patel?

  It does not, no.

• 42:30Speaker 03

  Uh, okay.

• 42:31Nilay Patel?

  We are still protected by CDA 230. We can do whatever we want on the platforms.

• 42:34Speaker 03

  Excellent.

• 42:35Nilay Patel?

  For now, until Josh Hawley comes around. So those are little companies. I want to talk about big companies. You mentioned Twitter, they're like struggling along. How, I just want you to evaluate. You have a position now outside, you're working with all these companies. A, I want to know for a list of companies, how are they doing now? Like, where do they stand? And how prepared are they for the upcoming elections? So let's start with Twitter. How are they doing? How prepared do you think they are for 2020?

• 42:59Speaker 03

  So Twitter has a significant issue in that a early product design decision was to allow pseudo anonymous accounts and allow people to have multiple pseudo anonymous accounts, right? So the policy framework under which Facebook enforces a lot of things that are related to elections is around authenticity. What does it mean to be a fake account on Twitter is really confusing. I actually gave a talk at Twitter. Right, I basically said like, I'm an expert in this area, this is all I do. I have no idea what the Twitter blue checkmark means, right? So Sarah John, who I know both of us know, right? Has a blue checkmark, will change her name to a literal Psyduck, will have a picture of a Psyduck, keeps the blue checkmark. What does that mean then? The checkmark mean in a situation where, once you're blessed,

• 43:42Speaker 03

  you have the ability to pretend to be whoever you want to be? And in what situations do they pull that back, right? So I think Twitter has some basic product design decisions that they still have not dealt with of what does authenticity mean? And how are we gonna signal this to people? I think they have people who are working really hard on this. My experience at Facebook is that they had a team that was very open. I have to give Twitter credit. Out of the three major companies, they are the only one who is dumping out the stuff they find, right? So this last week, both Facebook and Twitter, I think this last week is very indicative, right? Facebook and Twitter announced simultaneously that they are taking down a bunch of Chinese propaganda accounts. They both gave examples and talked about it. Twitter also released zip files of all of the content, right?

• 44:27Speaker 03

  Facebook did not. And I think the core of this is Facebook over-legalizes this decision. It is technically very hard around privacy laws, around the FTC content decree, GDPR, ECBA, SCA. These things make it hard to dump out content that has been deleted. But I think Facebook over-legalizes it and Twitter's decision is like, well, maybe it's legal, maybe it's not. If the Chinese want to sue us, they can go ahead, right? And so they dump it out and they make the decision, which is the right decision, which is to share this content for external research. Facebook, you know, did the announcement, did the right thing, did not release the data. Google said nothing and has done nothing, right? And the big difference between those companies is Twitter and Facebook have no future in China. They have both effectively given up on the Chinese market.

• 45:10Speaker 03

  Google is trying very, very hard to make a lot of money in China, right? And so, and they have done nothing to blunt the propaganda campaign and they haven't said nothing about PRC activity on YouTube. And so I think that's a microcosm of where the companies are, right? Facebook has the most resources behind this. They are significantly larger than Twitter. I think Facebook has added more content moderators in the last year than all of Twitter's employees and contractors. So. Just from an amount of money spend, Facebook continues to do that. I think Facebook also just has the biggest challenge of having the biggest network and also having these three different products that have very different kind of propaganda models, right? WhatsApp is very different than Instagram is very different than than the big blue app, Facebook.com. I think Google's probably done the least. And it's unfortunate because

• 45:53Speaker 03

  I think Google had this opportunity to see what was happening to Facebook, starting in 2017, and to cut the corner on a bunch of these problems on YouTube. And instead, they kind of just kept their heads down. So they have always been the least honest and open about what's going on. They will quietly clean up problems without coming out and publicly discussing them. And that has generally worked, right? Like, the amount of criticism they've gone on these issues is much less than Facebook. But I think what that has created is a backswell of of criticism, and they the deluge will come because they have not been honest about all the things they've been doing, and they haven't been as aggressive.

• 46:29Nilay Patel?

  Well, I mean, it seems like the the the deluge is starting to build already, right? I mean, it doesn't seem

• 46:34Speaker 03

  Yeah.

• 46:35Nilay Patel?

  That was the next time you were going to ask about. So Twitter, it seems like they're they're doing what they can. They're doing well. Specifically, how do you think they're doing for 2020?

• 46:43Speaker 03

  I think they're doing OK. I mean, again, the actions of these companies only affects one of those four different kinds of Russian interference.

• 46:50Nilay Patel?

  Right.

• 46:51Speaker 03

  So even if you completely eliminated all online trolling activity, we should not consider 2020 a done deal, right? So I just want to to say that. I think Twitter's doing OK. Like, they've cleaned things up a lot. They've they've gone a lot of fake accounts, but they have a fundamental product weakness that will always leave them open that doesn't exist at Facebook, right? And that is a problem. And they just have staffing issues as well. But I have to give them credit for being the most open company on data. And like, I think they've done the right thing there. They've made it easiest for people to find and spot bad stuff on Twitter. And so that helps offset the fact that they have the least resources.

• 47:24Nilay Patel?

  So let's go to Google. Google constantly in the news. President is attracting Google for basically a bad study.

• 47:31Speaker 03

  Yeah.

• 47:32Nilay Patel?

  Right? And they have they have the two different service areas. They have, obviously, the search results page, and they have YouTube. Let's specifically talk about YouTube. How is YouTube doing?

• 47:41Speaker 03

  It is very unclear what YouTube is doing around disinformation, right? They are acting the way Facebook acted a couple of years ago, right? So when I was at Facebook, one of the things that really frustrated me is the company would make content moderation decisions completely knee jerk based upon the immediate press feedback. And that reduces the pain in the short term and greatly increases the pain in the long term because they are doing what Facebook used to do, which is signal that if you yell at us enough, we will make decisions in your favor, right? And Facebook is trying to grow outside of that through like more documentation of these policies of why these decisions are being made, of creating kind of an external process for adjudication of the really hard decisions.

• 48:23Speaker 03

  Uh, where YouTube, you know, uh, if you look back just a couple of months ago, took down a video, untook down it, uh, you know, demonetized a guy, all of this within 48 hours without any actual intellectual scaffolding. It was clear like just based upon how much feedback they're getting internally and externally, they made snap decisions. And so I think YouTube is probably the worst placed because they, they have done the least kind of intellectual work in what it is that they want to do, um, and what kind of information environment they want to provide. The other challenge YouTube has is unlike Facebook or Twitter, that the number one determinant, while both of those companies have algorithmic ranking of news feeds, the number one determinant of what shows up in your news feed of Twitter and Facebook is who your friends are, right?

• 49:04Speaker 03

  The number one determinant of what you see on YouTube is the algorithm itself, right? Like YouTube, while a social network is not really powered by who you're following, your friends are, right? It's based upon the recommendation engine. And so that makes them uniquely more responsible for what people are doing, I think more than, than a Twitter or Facebook or any other social network that is powered by who your social graph is. Uh, and they have not responded that great to that, as, as we've all seen, right? Like it is still possible to start with Jordan Peterson videos and then to end up with semi white supremacist videos that sends you to gab where you can get more highly radicalized. That is still a progression that you can do on YouTube.

• 49:41Nilay Patel?

  One thing I think is important to disclose, you described a video that was like up and down and demonetized, that was all around a guy who works at Vox Media, Carlos Spata. I feel like it's important for me to disclose that was our company. Uh, we were right in the middle of it, but I agree with you. Like it seemed like YouTube was making a lot of snap decisions on Twitter, which is a weird place to make big policy swings. I guess the question I have about that algorithm that you pointed out very directly is, when you describe YouTube as an information environment, that's not how any consumer sees YouTube, right? Is there a level of this where the consumer needs to get smarter about how the information environments are being presented to them or being created? Because that is very opaque for anybody who opens up one of these apps right now. It's opaque for Netflix creators, right?

• 50:25Nilay Patel?

  Like Netflix showrunners are mad that Netflix didn't promote their show in the algorithm right now. And they're presumably at a much sort of higher level of interaction and discourse with their service provider, Netflix. Can we change that so that the consumers get more power back in these moments?

• 50:41Speaker 03

  Yeah, so I think the underlying thing here is that the solution to these problems is going to be with individuals, no matter what, right? Like, no matter what the companies do, we have gone through a revolution in the structure of the information we consume that has undone the centralization that we saw in the 20th century, right? So we have, for the most of human history, the only people you can speak to are the people you are physically speaking to, right? And then you have the written word, something that, you know, for much of European history is controlled by one organization, the Catholic Church, which is the keeper of the history of the things that get recopied. And a classics professor that talked about everything we know about the ancient Greeks and Romans, we should see as being filtered through the Middle Ages and what was burned

• 51:24Speaker 03

  or left to rot versus what was copied. And so we have this humongous information filter. And then you have the printing press, which greatly democratizes not to everybody, but a lot of people having a lot more voice and being able to amplify their voice out to thousands. And then in the 20th century, you have radio and television, which give you huge amplification, but also massive consolidation, right? Like, only a television station was something that you could only do if you're a massive corporation. And we've just gone very, very quickly, much more quickly than those other shifts, from mass centralization to mass decentralization, that you can have just as much amplification as a television station had 30 years ago, except that can belong to one person. Our society survived all those shifts.

• 52:06Speaker 03

  Every single one of them had people saying, you know, kids these days with their new fangled whatever are destroying society, right? Like, we've gone through these moral panics during every information shift, so it is natural for us to again have a moral panic about the use of the internet for communication. And this doesn't mean the company shouldn't do anything. I think they have a responsibility to help get us through this shift. But in the end, it's going to be up to individuals, right? Like, we're not going to actually end up with an internet that is a rubber room, that all the information that you're fed is safe and trustworthy and is appropriate for you. We are always going to have, we're never going back to a world where Walter Cronkite controls the conversation again. We're never going back to a world where 30 middle-aged

• 52:50Speaker 03

  white men decide what is newsworthy in the United States. And it's because of the breaking of that oligopoly that you have Black Lives Matter, that you have the Me Too movement, that you have a lot of positive things, because it is not a small set of non-diverse people who are deciding what is newsworthy. But the flip side is, you also have Gab, Nate Chan, and white supremacists, and other people who can utilize it. And I think, no matter what, as a society, individuals are going to have to adapt to that news environment. The good news in the United States is that there is some academic research that demonstrates that, especially kind of the fake news problem, is mostly a baby boomer problem, right? It is a problem of people who grew up in the Walter Cronkite era, who are having difficulty

• 53:31Speaker 03

  adjusting to the Fox News slash Facebook era. And so there is quantitative data showing that. A guy named Brendan Nyhan has done some work on this at the University of Michigan. That is not true globally, though. There are some other places where young people are just as vulnerable to spreading this kind of stuff. So it is not a societal shift that will happen evenly. But the fact that young people are less likely to share fake news, are less likely to spread rumors, are less likely to have Instagram posts saying, "I declare by the Treaty of Rome that all my information is copyrighted." Like, our Secretary of Energy, the man in charge of our nuclear stockpile, posted that. The fact that young people don't do that as much, I think, is a good sign that that societal shift that needs to happen, hopefully, will happen. The question is, is like, how do

• 54:15Speaker 03

  we shepherd the next 20 years or so to get through this safely?

• 54:19Nilay Patel?

  I think this all comes back to one of the first things we talked about, which is how much, how much of a conversation are we having that's open and honest about the responsibilities of these companies to make the internet a safe place? And in particular in the United States, the government just can't do it. Like, it is prevented from regulating speech on the internet in this way. So it has to fall, I mean, I look at every proposal to modify 230 is like a backdoor around the First Amendment, right? Like, we're going to hold this gun to your head. Your company will go out of business if we take this law away unless you do some shit we like. And, fine, I mean, that's a way to do things. It's maybe the only way to do things. But it seems like the companies have to make a big set of decisions around

• 55:01Nilay Patel?

  how much of a rubber room they want to be. And they haven't really been transparent about those decisions yet.

• 55:06Speaker 03

  They absolutely have not been transparent. This is one of the big problems, right? Is that Facebook, Google, Twitter, these companies are quasi governments. They have powers that we generally reserve for democratically accountable elected positions. But they don't act with the accountability, they don't act with the transparency. So if they're going to make these decisions, which, like you said, they're pretty much forced to in the United States, right? You know, Facebook cannot punt to the US government, "Please make a decision on what is appropriate speech in the political realm." Then, if they're going to make these decisions, they have to do so in a much more transparent manner. And they have to be predictable. That's one of the crazy things. Like, when I say that YouTube is going back and forth on their decisions, one of the real problems that creates for them, and then creates for their users, is that any individual YouTube creator cannot predict if what they do is going to be bad or not, right?

• 55:51Speaker 03

  Like, if you run a stop sign, if you get caught, you know that that is not going to be considered an okay thing, right? Like, yo, no, seriously. Like, if you kill somebody, it is understood that that is a crime. Like, there are obviously gray areas of all kinds of crimes. But for the most part, we live in a rule-based society where any individual can predict whether or not the action they're taking is legal or not. But that is completely and totally impossible on any of these platforms. And so, they have to be transparent enough in their enforcement decisions so that any individual can make a video, can write a post, can do something, and then know to themselves whether or not they're pushing the line or not. Because if the line continuously moves, then it means nobody has respect for the companies, and that there will be no respect for the process. And it's respect for the process that allows a legal system to operate, right?

• 56:36Speaker 03

  If you believe that every once in a while... The courts will go your way, but sometimes they'll go for you and against you, then you're willing to be part of the process. If you believe that every single decision is completely arbitrary, then you're going to do everything you can to argue against those decisions, to put pressure on the organizations. And it's just like a bad way to operate. The other thing I'd love to see from the companies, I'd love to see a way for all of these content moderation decisions to be reviewable externally, right? So something like a database of here's all the tweets that have been taken down from Twitter. Now, it'd have to be accessible under NDA. There's all kinds of, again, GDPR, FTC, there's all kinds of privacy law issues. But again, without the ability for external groups to see what decisions are being made,

• 57:18Speaker 03

  it's impossible to have any kind of pushback and to have any real scholarship. And so that's something that I'd love to see the companies move towards, probably in a unified manner, right? If there's like one database of here are all the content moderation decisions made by these folks over the last 30 days, available under NDA, you can write about the decisions, but not de-anonymize people, maybe do some differential privacy stuff. I think that would be an incredible step towards establishing some trust that the companies are, if they're going to operate like governments, that they have some of the trappings of a real legal system.

• 57:50Nilay Patel?

  How does that mesh with Facebook's Supreme Court of Content Moderation proposal, which to me seems, when you talk about these companies being governments, like that's how you know that they think they're governments, right? They're setting up a legal system. Do you think that's compatible with that or incompatible with that?

• 58:07Speaker 03

  It is. I mean, just like the US Supreme Court does not sit on traffic court, like the Supreme Court decides very difficult decisions that bubble all the way up. That is how this Facebook Supreme Court, which they never use that term, as you can tell, I'm no longer at Facebook and that I'm willing to accept your framing, but it's the right framing, right? The Facebook Supreme Court is going to be for the really tough ones, right? So like, the truth is, Facebook makes more content moderation decisions in one hour than the entire federal court system makes in a year, right? So you clearly can't have real due process for every single content moderation decision. What you can have is an ability for the tough corner cases of deciding, huh, does Alex Jones violate these three rules? That that kind of decision that will end up being

• 58:52Speaker 03

  A precedent that will have massive impact on the rest of the network, that that kind of thing can go to that Supreme Court and be decided in a much more open fashion. And then the precedent that's created will be, continue to be enforced mostly by robots. And then with the help of the kind of content moderators that Verge has written a lot about.

• 59:11Nilay Patel?

  So this is my last question, I'm asking how the companies are doing, and then I want, I have a real spicy one from

• 59:17Speaker 03

  Okay, great.

• 59:17Nilay Patel?

  From Casey. He gave it to me because he said

• 59:20Speaker 03

  He doesn't get to ask if I not, if he's not gonna show up, he doesn't get to send a question.

• 59:23Nilay Patel?

  I will say, the note on this question is, and I quote, "He will go off." So I gotta do it. It's true, it's what it says. How is Facebook doing? How prepared are they for the election?

• 59:33Speaker 03

  So I think Facebook's pretty well prepared for what happened in 2016, right? It has the broadest set of advertising and transportation, unlike Google, Facebook considers issue ads to be political ads. I think that's a really important step because, under our assessment at Facebook during our investigation, something like 80-some percent of the Russian ads that they ran were not illegal under US law because they're not electioneering, right? And so Facebook actually takes a much broader definition of what is a disallowable political ad than Google does. I think Facebook has the largest team. I think the hardest thing for Facebook is gonna be to try to predict how the non-Facebook products are gonna be used, right? So Instagram has some of the same problems Twitter has in that you can have a pseudo-anonymous identity on Instagram. The fact that Instagram is mostly

• 1:00:18Speaker 03

  images gives some benefit, but not a ton. As you know, the Russians did lots of, the Russian troll factories have professional meme farms. Like, they have graphic designers using Illustrator all day to create memes. So, is Instagram ready is actually a big question. I'm guessing Instagram is well behind what has happened on Facebook.com. And then the use of WhatsApp, WhatsApp number one source of disinformation in Southeast Asia, will WhatsApp with its end-to-end encryption be used in the same way in the United States? It seems unlikely in 2020, but after 2020, as people move to those platforms, I think that'll become an issue.

• 1:00:53Nilay Patel?

  Do you see that problem on, like, iMessage, right? Like WhatsApp isn't used as probably in the United States. Apple does have an end-to-end, very popular messaging service on iMessage. Do they have a disinformation problem in iMessage? Is it even possible to know?

• 1:01:04Speaker 03

  It's impossible to know. It's pretty well-known that iMessage has a child safety problem, right? Which is something that does not require kind of the international reach that WhatsApp has to have that as a significant problem. I see no evidence of a disinformation problem on iMessage.

• 1:01:21Nilay Patel?

  All right, here's my last question. I'm actually very excited to ask you this question. I'm very excited.

• 1:01:25Speaker 03

  Oh, God. Okay, I'm getting myself ready.

• 1:01:27Nilay Patel?

  What are we getting wrong? What is the media getting wrong in this conversation?

• 1:01:31Speaker 03

  Oh, that's not that spicy.

• 1:01:34Nilay Patel?

  That's how I know. See, that's you. You get that reaction. You're like, oh, it's beginning. But like, what are we getting? What do we consistently get wrong?

• 1:01:40Speaker 03

  There's a couple of different issues going on here. First, there are three groups who really screwed up in 2016. Right? There's Mark Zuckerberg in a hoodie is standing next to James Comey in a suit is standing next to a New York Times editor in a press hat. And Mark Zuckerberg is saying, man, we really screwed up in 2016. And the government guy and the media guy are saying, yeah, you really messed up. Right? Like, everybody agrees that tech made mistakes in 2016. The government made serious, serious mistakes. There are the obvious, like, specific operational mistakes in the Obama administration. But then just also, just like in tech, we missed propaganda as a legitimate area of cybersecurity. The government did not have anybody working on it. And that has changed a lot. So that has been great. At least the government's dealt with it.

• 1:02:23Speaker 03

  The media has had almost no self-reflection on its role in the Russian disinformation campaign in 2016. The most if you there's a number of books about this. One of the great books is by a lady named Kathleen Hall Jamison. She's done quantitative studies of this. Of all the things that the Russians did in 2016. Her work demonstrates that she believes it is the hack and leak campaign, the GRU, getting John Podesta's emails, getting the DNC emails, creating scandals around Debbie Wasserman Schultz and the like, that had the most impact. And that had the most impact, not because of Facebook or Twitter, it had the most impact because of the media. Right? That the amplification for the GRU campaign was the New York Times, it was the Wall Street Journal. It was the Washington Post. It was MSNBC. It was obviously Fox. Right? But it was the legitimate media did the work of the Russians.

• 1:03:09Speaker 03

  And while the tech companies have dealt with that fact, that that we were used, that I made mistakes, I screwed up personally. I've never heard somebody in the media say that. And so that is one of the things I think that gets wrong is that while you will read 3,000 words in the New York Times about all the screw ups at Facebook, you will not read 3,000 words in the New York Times about the screw ups at the New York Times. Right? Maybe don't write about yourself. You can write. The one about the Washington Post and the Washington Post writes about the Times, right? But like, that is a serious problem. And it's a serious problem because if today, yo, somebody on Tulsi Gabbard's team all of a sudden got access to a ton of emails sent inside the Joe Biden campaign, would the media treat it any differently? Right? Or would the amplification of the message that the Russians want to get out happen just as it did in 2016?

• 1:03:53Speaker 03

  And the answer is probably very little would change because there hasn't been any kind of self-reflection. I think the, my bigger picture is the thing that we've been talking about, which is all of these issues are trade-offs. And in other areas of public policy, those trade-offs are obvious, right? Like, generally, you will not see the Wall Street Journal say, "We both believe that the government should make more revenue and cut taxes." Right? And that people in the media who cover public policy understand the trade-off between, you know, taking resources away from cops and the potential to, to fight crime, cutting taxes and revenue. What they don't understand is the trade-off between privacy and safety. They don't understand the trade-off between data protection and antitrust. Right?

• 1:04:33Speaker 03

  And so you end up with the kind of the position of the media being, "We believe the tech companies are too powerful and we want that power to be used to squash our enemies." Yeah. Which is a completely inconsistent position. Or, you see this a lot from the New York Times especially. The Times wants there to be a dynamic, uh, social media environment where lots of people can compete against Facebook. They also never want Facebook to ever share information with anybody under any legal agreement ever. Right? And that you have this series of New York Times articles that try to create scandals out of things like Netflix being able to send messages that like, they imply that all of a sudden Netflix can read your private Facebook messages? That is completely and totally ridiculous. It is a complete technical misreading of the leaked documents these guys had.

• 1:05:13Speaker 03

  But it also strikes to the core point, which is if you believe that the companies need to be data hoarders, you are also telling them to be monopolists. Uh, and so I think that's like my biggest problem is that- A lot of folks in the media don't believe they need to have a position on what the world is that they want, right? That they'll complain no matter what. And so when you're in tech, the fact that they have to never, they never have to take a position that could be criticized, that all they, all they're gonna do is criticize you, whatever difficult decision you make, really gets frustrating. I, I, I don't, I think actually The Verge has been great on this. I think you guys are much more opinionated. This is more of like, an issue for, the other kind of big issue is that, at the big media outlets like the New York Times, everybody's a tech reporter now. So you have like these really skilled people who have been in the Bay Area, who have good sources, who've been doing this work for years, right? So you have

• 1:05:58Speaker 03

  Bob McMillan at the Journal, you've got Shira Frankel at the Times, you know, there's a, you know, a bunch of Joe Man at Reuters. These are like, people who really know what they're talking about, and so they have sources to help them out, but everybody else in their organization now considers themselves a tech reporter, even if they're working out of DC or New York. And so that kind of reduces the overall quality of the reporting, because you, you end up with a, a large number of people that have access to grind and don't understand the countervailing equities of any decision.

• 1:06:24Nilay Patel?

  You know, it's funny, you said we're opinionated. I, I do think The Verge is opinionated, like, we're very forward about our values, right? Like, I don't know that we, that would survive at a bigger place, right? Like, we're very clear about the future. Like, we're, we're like a hopeful organization. We think the tech companies should do better and work harder, and, but at the end of the day, we're also like, there's a new phone, it's fucking cool, right? And like, that helps us kind of center, okay, we have to make progress. I just don't know that any other big media organization will ever really figure that out. I think about that a lot. Like, it is very complicated, the trade-offs are very nuanced. If you are telling people about Cambridge Analytica, you have to do 2,000 words of backstory, just to get to what happened so you can understand it. How do you think somebody listening to

• 1:07:09Nilay Patel?

  this podcast can evaluate what they see from the rest of the media? Right? Because people who listen to this podcast are obviously very into it. If they've gotten an hour and seven minutes into you and me, they're obviously invested in the subject. How can they better evaluate what they see from the big media organizations?

• 1:07:25Speaker 03

  I mean, I think, this is true for not just tech reporting. I think you, when you read a, a position from what is supposed to be a non-opinionated source, but clearly they have their own opinions, you have to separate the, the things that they are saying are factual, and that they have checked with multiple sources, with their interpretation, which, what's going on right now is that there's a number of media outlets like the Times, that will, that will apply the worst possible Interpretation to any possible decision that was made. So like, the example I like to use, Facebook Live has, when it launched and continued to have a serious safety problem, right, is that Facebook Live is used to do a bunch of bad things, right? Obviously, the Christchurch shooting is an extreme example. Most of the safety problem on Facebook Live is suicide.

• 1:08:07Speaker 03

  Is that you have teenagers who, in a cry for help, will livestream themselves. And unfortunately, if they attract the wrong people, instead of people saying, "Don't do it, somebody loves you," and reporting it to Facebook, they will egg them on, right? And it creates kind of a real toxic and perhaps actually leads to them committing suicide. The New York Times has rightly written stories about the safety issues of suicide in Facebook Live. They also wrote a story about how creepy it is that Facebook will call law enforcement when somebody's about to commit suicide on Facebook Live, right? Like, if you're, you can read, either one of those positions is okay. It's okay to say, "Facebook needs to police this product." It is okay to say, "This is not good."

• 1:08:47Speaker 03

  And then, in that story about policing suicides, they explicitly kind of infer, implied that the only reason that any of this happens is because, you know, Facebook wants to look good in the media or look good to politicians. Which, I know the people who work on that team, there's a number of people who work on that team who have had family members who have committed suicide. I was disgusted to see the New York Times imply that these people who have worked very, very hard on a very difficult safety problem are only doing it for the bottom line of the company, right? And so, that's one thing, like, if you're reading the media and they're implying that because somebody works at a tech company, that they're just kind of innately an evil person, that's something you could disregard, right? And look to see whether or not that same organization has had any opinions on that exact same topic.

• 1:09:31Speaker 03

  Now, I talked to somebody at the Times about that, and they basically pointed out, the Times has over a thousand reporters, it's quite possible that the person who wrote that story had never read the other Times story on Facebook Live, right? So, you're right, like, that's a problem of an organization that that's large. As they have all these different opinions, but I would like to see the larger media organizations start to understand that they should have a vision of the world they want to move to, and they should incentivize people to move that way, instead of just complaining about whatever decision people make at that moment, right? Like, that is not helpful, and it's just getting kind of old. Like, it's easy to write a story saying the world's a shitty place.

• 1:10:06Nilay Patel?

  It is. Right now, especially, it's tremendously easy to write that story.

• 1:10:09Speaker 03

  It's incredibly easy. It's much harder to make a proposal of an idea of how to address it that doesn't have massive side effects, or accumulate a lot of power in people's hands where you don't want it to be, right? Like, that's a lot tougher. And so, I'd like to see folks who write the "the world's a shitty place," and that's reflected on the internet, also take a position about what should happen.

• 1:10:28Nilay Patel?

  So, here's my last question.

• 1:10:30Speaker 03

  Sure.

• 1:10:31Nilay Patel?

  What are the, if you had to say, here are the three big trade-offs, and you mentioned some, but I want to crystallize them for the listener. What are the three big trade-offs for the platforms, as they think about security, dynamism, making money? What are the three trade-offs people should be looking at whenever they see decisions? Are there three, or are there five? Like, what are the big ones?

• 1:10:49Speaker 03

  Yeah, I mean, there's a bunch, obviously. I mean, I think, clearly, you know, in a targeted advertising world, knowing information about people is directly beneficial to the bottom line, the ability to do targeted advertising. And so, there is a legitimate privacy revenue trade-off, that I think the companies have over-optimized on the revenue side, and are starting to give some of that up. I think what they're also finding is that they thought they needed all this data, and they don't actually, right? And so, my hope is that we'll get a little bit of a free trade-off here, in that as pressure's put on the companies to gather less data, they'll find that the lazy assumptions they made turns out not to be true anymore. I think one of the bigger ones that we've talked about a lot is the privacy safety trade-off, right?

• 1:11:33Speaker 03

  So, you know, a great example of this is that Google had a flaw in Google Plus that allowed people to gather up a bunch of data they're not supposed to, very much equivalent to a flaw that Facebook had, except Facebook had the logs. And so, Facebook came out and said roughly 30 million people have been affected by this flaw. Google said, "We're throwing away our logs, so we don't know if anybody was affected." That doesn't mean it wasn't 30 million people. It just means they don't know. The Facebook thing was a multi-week news cycle. The Google thing died over a day. Right? And like, that is a great example of when things GDPR pushes for is for impression logs and other kinds of logging to be thrown away, which is good, but it also means that forensically you can't look back and figure out what kinds of bad things happened.

• 1:12:17Speaker 03

  And so, having data to be able to look backwards, and then understanding whether or not something was actually exploited, or whether there's some kind of bad behavior in a platform, that's a hard trade-off. I think the encryption versus safety trade-off of whether you want people to do moderation is gonna be a big one. I think the big one is how much power they have, right? So, the more you ask the companies to do, the more power they accrue. And so, that's the big trade-off, is if you say, "I don't want these huge companies to have this much power," then you need to ask them to do less, not more. That's kind of, you know, from a political perspective, there's actually real incoherence on the Democratic/Progressive side. At least conservatives are pretty consistent, which is, "We want companies to do less. We want them to be smaller. We want them to do less. We want them to have less power."

• 1:12:58Speaker 03

  The Elizabeth Warren, "These guys are platform monopolists," is not compatible with also saying, "But we want them to control the speech that we don't like," right? And so, I think that's like, the biggest trade-off, is if you ask them to make decisions about stuff you don't like, they will accumulate power to make decisions about the things you do like, right? And so, don't assume that the powers that these companies are taking on for themselves are only gonna be used in ways that you find acceptable.

• 1:13:24Nilay Patel?

  Yeah. I mean, that to me, the Warren plan, to me, is at least the most honest, right? Like, she's not trying to backdoor a speech regulation through 230. She's just saying, "Your company should be smaller," because that's what the government can do.

• 1:13:37Speaker 03

  Yes. And that's our criticism of Warren. I think, actually, the Warren position is actually quite consistent. The problem is, is that a bunch of Warren supporters also want there to be heavy regulation of speech of people they don't like. And so, that is incompatible with what Elizabeth Warren wants, right? So, like, you just, there's a reason why Ted Cruz endorsed the Elizabeth Warren plan, right? And it's not because Ted Cruz thinks that white supremacy needs to be wiped out from Twitter and Facebook, right?

• 1:14:03Nilay Patel?

  Yeah. I mean, the sort of, the one-to-one overlap between what is defined as conservative speech and hate speech seems like an own goal, like a constant own goal, but it also seems to be very effective at this moment. All right, Alex Thamos, I've taken up so much of your time. I really appreciate you coming on. We're gonna have to have you back on, particularly as election cycles heat up, and we'll try to get Casey on as well sometime.

• 1:14:26Speaker 03

  Yeah, I'd love to be in the studio with Casey. No more smuggling questions through you.

• 1:14:30Nilay Patel?

  It's been entertaining. I have to say, my Slack over here has been very entertaining this entire time.

• 1:14:34Speaker 03

  Excellent.

• 1:14:35Nilay Patel?

  I appreciate it. We'll talk to you soon.

• 1:14:36Speaker 03

  Yeah, thanks, man. Thanks for having me on.

• 1:14:37Nilay Patel?

  All right, my thank you to Alex Thamos. He's the director of the Stanford Internet Observatory. He is very active on Twitter. You can find him. He's at Alex Thamos. You can also tweet at me. I'm at Reckless. Love to hear your thoughts on the show, who you want me to talk to, what you want me to dive into. It's always very interesting, and I really appreciate the feedback. We'll be back later this week with the chat show, then another interview show. Next week's interview show, I think, is gonna be pretty good. You're gonna like it. But we'll see you on Friday with the chat show.

• 1:15:05Speaker 05

  Fall savings are happening now at The Home Depot. Save up to $500 on select appliances. Plus, enjoy free delivery when you spend $998 or more. From prepping to cooking to cleaning up, Frigidaire has you covered. The custom Flex Temp drawer switches from fridge to freezer. With more than 15 cooking modes, including stone baked pizza mode, you can enjoy restaurant quality pizza in as little as two minutes. Then clean up with the Frigidaire dishwasher featuring the industry's fastest normal wash cycle. Shop fall savings now at The Home Depot. Offer valid through October 21st. U.S. only. See store online for details.

• 1:15:35Speaker 06

  At Zocdoc, we know you care about your health. You wouldn't do that early morning cardio if you didn't. So the fact that your annual physical is overdue just shows how hard it can be to get seen by a doctor. Sorry, that doctor's not in network.

• 1:15:48Speaker 07

  Press 1 for more options. You've reached us outside normal business hours.

• 1:15:52Speaker 08

  The next availability is in three months.

• 1:15:54Speaker 06

  But Zocdoc makes finding the right doctor easy. Download the app and search by specialty, insurance, and availability. Read real patient reviews, then book instantly. Zocdoc. Ready to be seen.