Most Replayed Moment: How AI Companies Are Gaslighting You - Karen Hao

What if the biggest risk from AI isn’t the technology itself - but who controls it? Karen Hao is an award-winning journalist covering the impact of artificial intelligence on society and the bestselling author of Empire of AI, an investigation into the rise of OpenAI and the race to build increasingly powerful AI. In this moment, Karen Hao examines the assumptions driving the race towards AGI and asks why building technology that can replicate human capabilities became the goal in the first place. She explores the enormous resources behind that ambition, and the relationship between AI companies and the researchers and journalists scrutinising them. Listen to the full episode here! Spotify: https://g2ul0.app.link/c6dLG2e595b Apple: https://g2ul0.app.link/5tusFIE595b Watch the Episodes On YouTube: https://www.youtube.com/c/%20TheDiaryOfACEO/videos Karen Hao: https://karendhao.com/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Steven Bartlett?

  If you know me well, you know how much I care about data. I see data as a way to be able to act intentionally, but really to turn the lights on in any facet of your life in business. That is why we have data scientists in our company. The problem is in business, you can have reams and reams of data, but if you're having to sift through your systems every time you need an answer, it becomes incredibly difficult. And that's exactly why NetSuite, our sponsor, has rolled out NetSuite Next, an update which turns your business data into an intelligent system that you can interact with. You simply ask it a question and because it understands the context behind your business, it surfaces genuinely useful personalized insights and actions in return that can give you the information you need to make better decisions. And so no wonder it's trusted by over forty four thousand companies.

• 0:43Steven Bartlett?

  If your revenues are at least in the seven figures, go to netsuite.com/bartlett. Get your free guide at netsuite.com/bartlett. Netsuite.com/bartlett.

• 0:58Steven Bartlett?

  What I found really interesting is to look at how people's quotes evolve over time with their incentives. So I was looking at all of the, all of the things they've said on the record, on podcasts, in their blog posts to see how it's evolved over time. And Dario, who was the former VP of Research at OpenAI and has now moved on to Anthropic, who are taking a slightly different approach to developing AI, said back in 2017, while he was still at OpenAI, that, this is a quote, "I think at the extreme end is the Nick Bostrom style of fear that an AGI could destroy humanity. I can't see any reason in principle why that couldn't happen. My chance that something goes really quite catastrophically wrong on the scale of human civilization might be somewhere between 10% and 25%."

• 1:42Steven Bartlett?

  And also you mentioned Ilya, who was a co-founder of OpenAI and then left. I guess the first question I'd ask is, why did Ilya leave?

• 1:51Speaker 01

  That's a great question. So he was instrumental in trying to get Sam Altman fired and he's another one of the people who, over time, began to feel like he was being manipulated by Altman towards contributing something that he didn't believe in.

• 2:08Steven Bartlett?

  How did you know?

• 2:09Speaker 01

  Because I interviewed a lot of people. Ilya, in particular, had two pillars that he cared about deeply. One is making sure we get to so-called AGI and the other is making sure that we get to it safely. And he felt that Altman was actively undermining both things. He felt that Altman was creating a very chaotic environment within the company where he was pitting teams against each other, where he was telling different things to different people.

• 2:40Steven Bartlett?

  Have you ever spoken to him?

• 2:42Speaker 01

  I have. So I interviewed him in 2019 for a profile that I did of OpenAI for MIT Technology Review.

• 2:50Steven Bartlett?

  And back in 2019, he has a quote where he says, "The future's going to be good for AIs regardless. It would be nice if it was also good for humans as well. It's not that it's going to actively hate humans or want to harm them, but it's just going to be so powerful." And I think a good analogy would be the way that humans treat animals. It's not that we hate animals. I think humans love animals and I have a lot of affection for them. But when the time comes to build a highway between two cities, we are not asking the animals for permission. We just do it because it's important to us. And I think by default, that's the kind of relationship that's going to be between us and AI, which are truly autonomous and operating on their own behalf. And that was in 2019, the year that you interviewed him.

• 3:30Speaker 01

  One of the things that I, I feel like we should take a step back to examine is going back to this idea of what even is artificial intelligence and what do we mean by intelligence? And a huge part of the views of the different people and the quotes that you're reading derives from a specific belief that they each have in this question of what is intelligence? What constitutes intelligence? For Ilya, he has throughout his research career felt that ultimately our brains are giant statistical models. This is not something that, you know, we actually know, but this is his own hypothesis, also the hypothesis of his mentor, Geoffrey Hinton, who also was on this podcast.

• 4:15Speaker 01

  This is why they have such a strong conviction in the idea of building AI systems that are statistical models and that this particular approach is going to lead to intelligent systems as we are intelligent. It's a hypothesis that they have. It's not one that has been proven by science. And some people vehemently disagree with them on this particular thing. But if you step into their shoes and take on that hypothesis and assume that it's true, that our brains are in fact statistical engines and that these systems that they're building are also statistical engines that they're making bigger and bigger and bigger until they become the size of the human brain.

• 4:57Speaker 01

  That's why they say that making this comparison where the system will become equal to human intelligence and then maybe exceed human intelligence is relevant in their framework. And Ilya gave a talk at one point at this really prominent AI research conference that happens every year called Neural Information Processing Systems. It's a mouthful. But he gave this keynote where he shows this chart of the size of brains and the intelligence of a species. And it's roughly linear, the bigger the size of the brain, the more intelligent the species.

• 5:36Speaker 01

  And so for him, he thinks he's building a digital brain because he, he thinks brains are just statistical engines. So from that logic, it's like, okay, if we then build a bigger statistical engine than the human brain, then based on this chart, it will be more intelligent and then we will be subjected to the same treatment that we've subjected animals. But it's really important to understand that these are scientific hypotheses of specific individuals within the AI research community. And there's a lot, a lot of debate about whether this is in fact the case. And some of the biggest critics say it's very reductive to think of our brains as simply just statistical engines.

• 6:19Steven Bartlett?

  Why, why does it matter to know the mechanism? Is it not just important to know the outcome, which is that it's going to be able to do, make a video for me or agents are going to be able to do the work that I do? Does it, does it really, really matter for us to know the mechanism behind it?

• 6:38Speaker 01

  Yes and no. So it matters because these companies, they are driving their future actions based on this hypothesis. So they have decided, we think that this hypothesis is true, like we should just continue building larger and larger statistical models in the pursuit of artificial general intelligence. And that's then having global consequences, like in order to continue doing that, they're hoovering up more and more data, they're building more and more data centers, they are having, they're, you know, exploiting more and more labor in order to continue on this path. Here's a question that I think is important to ask is, why are we trying to build AI

• 7:22Speaker 01

  systems that are duplicative of humans? We're kind of having this conversation right now where we've just taken the premise of this industry as a good thing. Like, they said that we should be building AGI, so we say that we should be building AGI. But I would like to ask, like, why are we doing that? Why is it that we are building a technology that is ultimately designed to replace and automate people away? That is not the enterprise of technology. Like, we should be building technology and the purpose of technology throughout history has been to improve human flourishing, not to replace people.

• 8:02Speaker 01

  And so this is like a, a critical part of my critique of these companies and, and these scientists that have just adopted this goal and have relentlessly pursued it and have had enormous capital and enormous resources to pursue it, is, is this the right goal? Like, why are we doing this? Why can't we just build AI systems that do things like accelerate drug discovery and improve people's healthcare outcomes, which are systems that have nothing to do with the statistical engines that they're trying to build to duplicate the human brain?

• 8:35Steven Bartlett?

  So why are they doing it? I mean, you've interviewed all these people. I think it's, what, 300 people in total? Eighty or 90 of them from OpenAI, the maker of ChatGPT. Why do you think they're doing it?

• 8:46Speaker 01

  I think it's because they're driven by an imperial agenda. And that is why I call these companies empires of AI.

• 8:51Steven Bartlett?

  What do you mean by an imperial agenda? What does that term mean?

• 8:55Speaker 01

  Empire is the only metaphor that I've ever found to fully encapsulate all of the dimensions of what these companies do and the scale that they operate and what motivates them to do what they do. And there are many parallels that you see between what I call the empires of AI and the empires of old. They lay claim to resources that are not their own in the pursuit of training these models. That's the data of individuals, the intellectual property of artists, writers and creators. They're land grabbing in order to build these supercomputer facilities for training the next generation models. Second, they exploit an extraordinary amount of labor. They contract hundreds of thousands of workers all around the world,

• 9:37Speaker 01

  including in the US, to ultimately make these technologies. We can talk about that more. And they also design their tools to be labor automating so that when the technologies are deployed, it also affects labor rights because it erodes away labor rights. And this is a political choice that they have. Third, they monopolize knowledge production. So they project this idea that they're the only ones that really understand how the technology works. And so if the public doesn't like it, it's because they don't actually know enough about this technology. They do this to the public, they do this to policymakers, and they've also captured the majority of the scientists that are working on understanding the limitations and capabilities of AI.

• 10:20Steven Bartlett?

  You think they're gaslighting the public in a way?

• 10:22Speaker 01

  They are, yeah. So if most of the climate scientists in the world were bankrolled by fossil fuel companies, do you think we would get an accurate picture of the climate crisis?

• 10:33Steven Bartlett?

  No.

• 10:34Speaker 01

  And in the same way, they employ and bankroll, the AI industry employs and bankrolls most of the AI researchers in the world. So they set the agenda on AI research in soft ways, simply by funneling money to their priorities so that only certain types of AI research are produced. But they also will censor researchers when they do not like what the researcher has found. And so I talk about the case of Dr. Timnit Gebre in my book, who was the ethical AI team co-lead at Google. When she was literally hired to critique the types of AI systems that Google was building, she then co-wrote a critical research paper that was showing how large language models specifically

• 11:19Speaker 01

  were leading to certain types of harmful outcomes. And in an attempt to try and stop this research from being published, Google ended up firing Gebre and then fired her other co-lead, Margaret Mitchell. And so they control and quash the research that is inconvenient to the empire's agenda.

• 11:44Steven Bartlett?

  Did you have an example where this is happening to journalists as well that are asking questions of their team members? I think I was watching a video of yours where there was a young man that was saying he had someone show up at his door, knocked on his door and asked for information, emails, text messages. And this person was from one of the big AI companies.

• 12:03Speaker 01

  This was opening, I started subpoenaing some of its critics, yeah. As, as part of a

• 12:11Speaker 01

  what's, what appears to be a campaign of intimidation, but also what appeared to be a campaign of fishing for more information to figure out, to map out the network of critics further. But this was a man who runs a small watchdog nonprofit. And they had been doing a lot of work during that time to try and ask questions about OpenAI's attempt to convert from a nonprofit to a for-profit. Ultimately, OpenAI was successful in that conversion. But during the period where it was sort of existential for OpenAI to complete this conversion, there were a lot of civil society groups and watchdog groups like Midas who were trying to prevent the process from happening in the dead of night.

• 12:56Speaker 01

  They were trying to get more transparency. They were trying to have more public debate about this because it's unprecedented. And it was then that there was a knock on his door and he was served papers.

• 13:10Steven Bartlett?

  What did the papers say?

• 13:11Speaker 01

  The papers asked him to reproduce every single piece of communication that he had had that might have involved Musk. So this was like the strange paranoia that OpenAI had that Musk was somehow funding these people to block the conversion. None of them were actually funded by Musk. So in this particular case, their request, he simply was just answered, you know, I don't have any documents because this doesn't exist.

• 13:35Steven Bartlett?

  So going back to this point of empires, you were saying that one of the factors of an empire is a land grab. And then the next one was?

• 13:42Speaker 01

  Was labor exploitation.

• 13:43Steven Bartlett?

  Labor exploitation.

• 13:45Speaker 01

  The third one, controlling knowledge production. And one of the other ones that's really important to understand about the AI empires in particular is empires always have this narrative that they, they say to the public, like, we're the good empire and we need to be an empire in the first place because there are also bad empires in the world. And if you allow us to take all the resources and use all of the labor, then we promise we will bring you progress and modernity for everyone. We will bring you to this utopic state akin to an AI heaven. But if the evil empire does it first, we will descend into a hell.

• 14:29Steven Bartlett?

  And the evil empire being in this case?

• 14:31Speaker 01

  In this case, most often it's China. But actually, in the early days, OpenAI evoked Google as the evil empire. So all of their decisions were about, we need to do it first because otherwise, Google, this evil corporation that's driven by profit, us as a benevolent nonprofit, like this is a, this is a critical contest of who wins.

• 14:56Steven Bartlett?

  Do you think the people building these AI companies believe that the outcome is going to be all good now? Do you think they think that it's going to be, it's going to serve everyone? It's going to be the age of abundance? Everything's going to go well? What do you think they believe? What do you think Sam believes?

• 15:12Speaker 01

  So this is so funny is such a core part of the mythology that they create around the AI industry includes the belief that it could go very badly. It goes hand in hand. Like, they need that part of the myth in order to then say, and that's why we need to be in control of the technology because that's the only way that it's going to go really, really well. And Altman has said publicly, you know, the worst case, lights out for everyone. But best case, we cure cancer, we solve climate change, and there's abundance. And Dario Amade, same kind of rhetoric. It's like, worst case catastrophic or existential harm for humanity. Best case, mass human flourishing.

• 15:57Speaker 01

  So this is like two sides of the same coin. Like, they have to use both of these narratives in order to continue justifying an extremely anti-democratic approach to AI development where there should not be broad participation in developing this technology. They must be the ones controlling it at every step of the way.

• 16:18Steven Bartlett?

  Sam Altman did a tweet saying, there are some books coming out about OpenAI and me. We only participated in two of them. One by Kesha Hagey?

• 16:27Speaker 01

  Keech Hagey.

• 16:28Steven Bartlett?

  Keech Hagey focused on me and one by Ashley Vance on OpenAI.

• 16:34Steven Bartlett?

  He went on to say, no book will get everything right, especially when some people are so intent on twisting things. But these two authors are trying to.

• 16:43Steven Bartlett?

  You quote retweeted that tweet from Sam Altman and you said, the unnamed book Empire of AI is mine.

• 16:53Steven Bartlett?

  Do you believe that tweet from Sam Altman was in reference to your book?

• 16:56Speaker 01

  A hundred percent because there's only three books coming out about him.

• 17:00Steven Bartlett?

  And he'd caught wind that your book was coming out and?

• 17:02Speaker 01

  He knew my book was coming out because I had contacted OpenAI from the very beginning of my process and said, I'm working on a book now. Will you participate in it? And actually, initially, they said yes, even though, so my history with OpenAI, I profiled the company for MIT Technology Review. I embedded within the office for three days in 2019. My profile comes out in 2020. The leadership are very unhappy. And in my book, I actually quote an email that I received that Sam Altman sent to the company about my profile saying, yeah, this is not great. And from then on, the company's stance to me was

• 17:45Speaker 01

  we are not going to participate in anything that you do. We are not going to respond to anything, any questions that you receive. And this was, you know, this was things that they explicitly articulated. It wasn't like me inferring. So I had a colleague at MIT Technology Review that also covered AI. And at one point, OpenAI sent him this press release being like, we would love for you to cover this story. And he was like, I'm really busy. Will you send it to Karen? And they were like, oh, no, we have a history, you understand. And so, so for three years, they, they refused to talk to me. But then I ended up at the Wall Street Journal where if they felt a bit compelled, because it was the Journal,

• 18:29Speaker 01

  to reopen the lines of communication. And so I, I started having, you know, more dialogue with them. Every time I wrote a piece, I would always send them, here's my request for comment. I would always ask them, like, will you sit for interviews? And we did get to a more productive relationship. And then I embarked on the book. So I, I left the Journal to focus on the book full time. And I told them right away, I'm working on this book. I want to continue this productive conversation where I make sure I reflect OpenAI's perspective in the book. And so they were like, we can arrange interviews for you. You can come back to the office. We'll set up some conversations.

• 19:12Speaker 01

  And then as we were going back and forth on this, the board fires Sam Altman.

• 19:19Speaker 01

  And that's when things started going kind of south because the company started becoming very sensitive to scrutiny. And so then they started pushing, kicking the can down the road, down the road, down the road. And I kept saying, hey, when are we rescheduling this? What's going on? And then I get an email saying, we are not going to participate at all. You are not coming to the office. You're not doing interviews. And I had actually already booked my tickets. So I was already going to fly to San Francisco to have the the interviews. And so then I told them, I was like, that's fine. I will still engage in the process. Well, I'll give you extensive requests for comment. I'll ask through my reporting, I'll keep you updated on all the things that I'm finding so that you can

• 20:02Speaker 01

  choose to still comment. I gave them 40 pages of requests for comment. And I gave them over a month to respond to all of that. So this was when the tweet came out was we were doing all this back and forth, trying to. And that's when Altman tweeted this.

• 20:23Speaker 01

  And they never responded to a single one of the, one of the 40 pages.

• 20:27Steven Bartlett?

  Sam Altman does a lot of interviews.

• 20:29Speaker 01

  Yeah.

• 20:30Steven Bartlett?

  You know, he's doing a lot of interviews all the time. He's done every podcast. I've seen him on everything from Tucker Carlson to I think he's done Theo Von, Joe Rogan. Podcasts all over the world. I wonder why he won't do mine.

• 20:47Speaker 01

  Well, maybe.

• 20:48Steven Bartlett?

  I don't know why. I don't know. I think I'm fair with everyone. I just ask, I just ask questions I genuinely care about. I don't come in with huge preconceptions, or at least meet people for the first time. But I've heard through the grapevine that he doesn't want to do mine.

• 21:02Speaker 01

  I mean, going back to what you were saying earlier, that with this, the way that OpenAI and these companies control research, you asked, do they also do this with journalists? I mean, yes, the answer is yes. And apparently they, they also do it with anyone who has, you know, a broad mass communications platform. It's not just about the conversation that you're going to have with them. It's about who you also choose to platform. And there's this huge problem in technology journalism where companies know that a really big carrot that they can give to technology journalists is access.

• 21:41Steven Bartlett?

  Yeah, yeah, yeah.

• 21:41Speaker 01

  And they will withhold that access at the drop of a hat if they catch wind that you're speaking to someone that they didn't want you to speak to.

• 21:51Steven Bartlett?

  This is so true. And I don't think the average person really truly understands this.

• 21:55Speaker 01

  Yeah.

• 21:56Steven Bartlett?

  So this kind of sounds like theory as you say it, but I'm not going to name names here because I don't think it's important. But there is a particular person in AI who, whose team have basically dangled the carrot of them coming here for like 18 months. And I'm like, you don't, you don't have to dangle the carrot. I'm going to speak to whoever I want to regardless of the carrot or not.

• 22:17Speaker 01

  Yeah.

• 22:17Steven Bartlett?

  And when this person comes, if they want to come, I'll, I'll give them a fair shot. I'll ask them all genuinely curious questions about what they're doing, their incentives. I won't gotcha them. I don't have a history of ever gotchering anybody, even if I dislike, like, even if I have a different of opinion, I'll ask the question.

• 22:31Speaker 01

  Yeah.

• 22:31Steven Bartlett?

  But they dangle carrots and they say, well, if you know, he, he's thinking about it, let's think about a date. And what the, what the strategy is, and I don't think they, they think most people don't understand is if we just dangle it for long enough, then they will perform in the way that we want them to do. And they'll be, they'll be pleasant about us. They won't be critical. They won't give a, give a.

• 22:53Speaker 01

  They won't platform our critics.

• 22:54Steven Bartlett?

  Our critics. And I think all of their game is just dangle the carrot forever.

• 22:58Speaker 01

  Yes. Yeah.

• 22:59Steven Bartlett?

  That's like the optimal outcome is if we just dangle it, if we just tell them, yeah, no, we're just trying to look at the schedule. It just doesn't work. I think in the modern world, you just have to go there and give your opinion and allow the clash of ideas in the public forum. Let the viewers decide for themselves.

• 23:12Speaker 01

  Yeah.

• 23:13Steven Bartlett?

  What they think.

• 23:14Speaker 01

  Yeah. But this is a, yeah, this is such a huge part of their machinery is the way that they use these tactics to massage the public image of these companies and make sure that information that they don't want out, and even opinions that they don't want out there, go out there. And so this is, this is, you know, I feel very lucky now that OpenAI shut the door early on me. At the time, I didn't feel lucky. I felt like I had screwed myself over. I was like, should I have been nicer to them in the profile so that I could maintain access, which is a horrible question to ask as a journalist, right? Like, you're supposed to report the truth and you're always supposed to report in the interest of the public. Like, that is the point of journalism. And in that moment,

• 23:59Speaker 01

  it, I was like relatively junior in my career. I was like, did I misunderstand what journalism about is about? Like, should I have actually been playing the access game? But it was too late. I had the door shut to me. And so I had to build my career, understanding that the door, the front door was never going to be open.

• 24:20Steven Bartlett?

  Yeah.

• 24:20Speaker 01

  And that actually really strengthened my own ability to just tell it like it is.

• 24:28Steven Bartlett?

  Be objective.

• 24:28Speaker 01

  Yeah. And just report what I see are the facts being presented to me irrespective of whether the company likes it or not. And most often, the company really does not like it. But I can continue to do the work. They don't need to open the front door for me. I was still able to do more than 300 interviews.

• 24:48Steven Bartlett?

  What you just listened to was a most replayed moment from a previous episode. If you want to listen to that full episode, I've linked it down below. Check the description. Thank you.

• 25:02Steven Bartlett?

  If you know me well, you know how much I care about data. I see data as a way to be able to act intentionally, but really to turn the lights on in any facet of your life and business. That is why we have data scientists in our company. The problem is in business, you can have reams and reams of data. But if you're having to sift through your systems every time you need an answer, it becomes incredibly difficult. And that's exactly why NetSuite, our sponsor, has rolled out NetSuite Next, an update which turns your business data into an intelligent system that you can interact with. You simply ask it a question. And because it understands the context behind your business, it surfaces genuinely useful personalized insights and actions in return that can give you the information you need to make better decisions. And so no wonder it's trusted by over 44,000 companies.

• 25:45Steven Bartlett?

  If your revenues are at least in the seven figures, go to netsuite.com/bartlett. Get your free guide at netsuite.com/bartlett. netsuite.com/bartlett.