Empire of AI: Inside OpenAI's Race to Conquer the Future | Karen Hao

In Episode 418 https://hiddenforces.io/podcasts/empire-of-ai-openais-race-to-conquer-the-future-karen-hao/ of Hidden Forces, Demetri Kofinas sits down with award-winning journalist Karen Hao to discuss Empire of AI — her inside account of how OpenAI evolved from an idealistic, safety-first nonprofit into one of the world's most valuable private companies in its race to conquer the future. This conversation takes you inside that transformation—from the heady idealism of OpenAI's founding, through the billion-dollar Microsoft deal and the 2023 boardroom coup, to the unresolved questions that hang over Silicon Valley and Washington alike about the private concentration of power in the age of artificial intelligence and the nature of the world we are building. Whether you're an investor, a policymaker, or simply a concerned citizen trying to make sense of today's headlines, this episode will equip you with the context you need to understand what's really at stake in the race for AGI—and the levers we still have to steer it. Subscribe to our premium content—including our premium feed, episode transcripts, and Intelligence Reports—by visiting HiddenForces.io/subscribe https://hiddenforces.io/subscribe. If you'd like to join the conversation and become a member of the Hidden Forces Genius community—with benefits like Q&A calls with guests, exclusive research and analysis, in-person events, and dinners—you can also sign up on our subscriber page at HiddenForces.io/subscribe https://hiddenforces.io/subscribe. If you enjoyed today's episode of Hidden Forces, please support the show by: Subscribing on Apple Podcasts https://podcasts.apple.com/us/podcast/hidden-forces/id1205359334, YouTube https://www.youtube.com/channel/UC8URhgYos5fjHqFSO4RSIEg, Spotify https://open.spotify.com/show/0TK6u251zGSlMgJKH6cOaP, Stitcher https://www.stitcher.com/podcast/demetri-kofinas/hidden-forces, SoundCloud https://soundcloud.com/hiddenforces, CastBox https://castbox.fm/channel/2629539, or via our RSS Feed http://hiddenforces.libsyn.com/rss Writing us a review on Apple Podcasts https://podcasts.apple.com/us/podcast/hidden-forces/id1205359334 & Spotify https://open.spotify.com/show/0TK6u251zGSlMgJKH6cOaP Joining our mailing list at https://hiddenforces.io/newsletter/ Producer & Host: Demetri Kofinas Editor & Engineer: Stylianos Nicolaou Subscribe and support the podcast at https://hiddenforces.io https://hiddenforces.io/. Join the conversation on Facebook https://www.facebook.com/hiddenforcespod/, Instagram https://www.instagram.com/hiddenforcespod/, and Twitter https://twitter.com/hiddenforcespod at @hiddenforcespod Follow Demetri on Twitter at @Kofinas https://twitter.com/kofinas Episode Recorded on 05/12/2025

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Demetri Kofinas?

  What's up everybody? My name is Demetri Kofinas and you're listening to Hidden Forces, a podcast that inspires investors, entrepreneurs, and everyday citizens to challenge consensus narratives and learn how to think critically about the systems of power shaping our world. My guest in this episode is Karen Howe, an award-winning journalist and the author of Empire of AI, an important new book that tells the eye-opening account of arguably the most fateful technological arms race in history. While the book tells the inside story of OpenAI, it is at its heart a meditation on power that reveals how an idealistic nonprofit devoted to the safe development

• 0:43Demetri Kofinas?

  of artificial intelligence morphed into one of the most valuable private companies in the world. Today's conversation takes you inside that transformation from the heady idealism of OpenAI's founding through the multi-billion dollar Microsoft deal and the 2023 boardroom coup, to the unresolved questions that hang over Silicon Valley and Washington alike about the private accumulation of immense sources of power around the deployment of this technology and the nature of the world we are building. Whether you're an investor, policymaker, or simply a concerned citizen trying to make sense of the headlines, this episode will equip you with the context you need to understand what's really at stake in the race to build AGI

• 1:27Demetri Kofinas?

  and what levers we still have to steer it. If you want access to all of our premium content or you want to learn more about the Hidden Forces genius community and how to attend one of our many virtual Q and As, in-person events, and dinners, you can do that at hiddenforces.io/subscribe. And if you still have questions, feel free to send an email to [email protected] and I or someone from our team will get right back to you. And with that, please enjoy this incredibly important and timely conversation with my guest, Karen Howe.

• 2:08Demetri Kofinas?

  Karen Hao, welcome to Hidden Forces.

• 2:12Speaker 01

  Thank you so much for having me, Dmitry.

• 2:14Demetri Kofinas?

  I'm so excited to have you on, Karen. I told you that this episode, I think, follows nicely on our previous episode with Patrick McGee. Even though that episode was focused on China and Apple, you know, there's a tech focus here, obviously, but there is also a China angle as well, which hopefully we can get into with respect to DeepSeek and your observations of the book about the scaling hypothesis and what is the best way, an optimal way to scale these systems that we're going to talk about today. But before we do that, well, first of all, congratulations.

• 2:41Speaker 01

  Thank you.

• 2:41Demetri Kofinas?

  On the launch of your new book. Yeah. How does it feel?

• 2:45Speaker 01

  Both stressful and relieving. I don't know. It's like an interesting tension. I'm excited to get it out in the world and also it's, yeah, then the reaction of it goes out and like it's like watching your baby fly out of the nest. So.

• 3:00Demetri Kofinas?

  Yeah, I'm envious. You know, I interview so many authors and I feel like, well, I'm not envious of the process. I feel like the process would be draining and difficult. And I'm always amazed that people can have a full-time job and also write a book. Maybe it's easier for journalists because you've got a network of editors and the experience of writing it, but then you've also got all these crazy deadlines. So anyway, it's quite a feat. I'm going to mention the name of the book a number of times, but it's Empire of AI. Again, we'll get into that. Before we do, tell me a little bit about you. How did you get your start in journalism? Did you start with an interest in journalism? Did you come in through the technology side and you ended up just writing about the stuff that you were already interested in? What's your story?

• 3:38Speaker 01

  Yeah. So I never thought that I was going to be a journalist. I studied engineering and specifically mechanical engineering at MIT for undergrad. And when I graduated, I went straight into the tech industry, moved to San Francisco, started working on a startup, and I thought that that was going to be my career. That's what I had studied. That's what I was set out to do. But I very quickly, within my first year at this startup, realized how Silicon Valley works. And what I mean by that is the startup very quickly imploded because it didn't have a business plan, even though it had this incredible mission and it was a very mission-driven group of people. And it was focused on problems that I really cared about, which was building technology to

• 4:23Speaker 01

  foster sustainability. And it suddenly dawned on me that a lot of the problems that I was most interested in and the kinds of technology I was most interested in building most likely would not have a business plan and therefore could not survive in Silicon Valley. And so I then started thinking, what should I do? Like I've been, I studied engineering for four years in order to do this specific thing. And now I don't really see myself having a long-term future in this industry. And kind of on a whim, honestly, I decided to go into journalism in part because I also really love writing. And I didn't know anything about journalism, other than that it was

• 5:04Speaker 01

  an ability to use writing as a tool for social change. And originally I'd gone into technology to use technology as a tool for social change. And so that was kind of my naive analysis that led me to go into the industry. And then kind of through getting my feet wet in journalism, I very quickly initially thought I was getting shoehorned into technology reporting because I was actually not that interested in reporting on technology. I wanted to report on the environment. My cause was very much sustainability. But over time, I realized that reporting on the technology industry was just this really great way to dig into kind of what I saw as the misaligned incentives within

• 5:46Speaker 01

  the most dominant technology-making power there is today. And really use that as a way to explore what's sort of gone wrong with why we can no longer seem to make technologies in the public interest. And try and figure out how we can actually get back to a place like that.

• 6:05Demetri Kofinas?

  You know, this is so clarifying. I did not know this about you. I knew that you studied mechanical engineering, but I didn't know all these other things, including your interest in sustainability. I mean, obviously, it's obvious that you have an interest in sustainability from reading your book, but I didn't realize that you had one coming into it. This makes so much sense. This is very clarifying because this is a sort of also in some sense, I mean, it isn't a philosophical exploration of the questions that you just raised, but they're kind of undertones of them. And certainly, the public interest and what is in the public interest and what are we building and how are the tools we're building aligning with that larger interest kind of runs through the book. So I think I can imagine in your future, a book where you're actually maybe delve into some of these things more concretely. So tell me a little bit about how you got interested in OpenAI, because that's what

• 6:48Demetri Kofinas?

  the book is about. Again, you do explore these larger themes within it. But what was the origin of your relationship to the company and to Sam Altman, its CEO?

• 6:56Speaker 01

  Yeah, so I started covering OpenAI in 2019. And at the time, I was an AI reporter at MIT Technology Review. And MIT Technology Review is a publication that is very focused on emerging technologies that are extremely cutting edge, not really like we used to say, by the time the technology has some kind of commercial application, it's too late for us. And so I was covering the bleeding edge of research that was coming out of academia, coming out of corporate research labs. And in that context, OpenAI was a really interesting organization at the time, because it had been founded specifically as a research nonprofit

• 7:37Speaker 01

  by big names, Elon Musk and Sam Altman. But they were focused not on commercialization. They were focused on those kind of early stage ideas of how do you develop AI to be more advanced than the kind of techniques that we are already seeing. And so I developed an interest initially because of that, because they were kind of squarely within what I was focused on and what MIT Technology Review was focused on. But I also came to realize that they were very rapidly building influence as an organization, not just within the AI research world, but also within the tech industry and within the Policy world. The organization from very early on kind of was already quite savvy in

• 8:21Speaker 01

  understanding that it needed to position itself and it needed to cultivate relationships with people in power across different, both DC and Silicon Valley. And so I saw that they were sort of beginning to influence not only the direction of research that was happening within the AI world, because every time they made a research release, they would kind of do this big splash with a lot of marketing and a lot of communications. And you could see other researchers, it would sort of grab the attention of other researchers and it was slowly starting to shift the way that other researchers were doing AI research. But it also had an outsized influence on the way that policymakers

• 9:03Speaker 01

  and therefore the public was starting to talk about AI because it was doing that kind of legwork to shape the public narrative early. And so I decided to profile the company under the suggestion of an editor of mine at MIT Tech Review where I was like, this organization is really interesting. He was like, why don't you just go amass them for interviews, see if you can profile them. And so I ended up embedding within the company in the summer of 2019 for three days, just trying to understand what is this organization about? When they say that they are just doing nonprofit research for the good of humanity, like, how are they doing that? How do they see themselves? Why are they spreading out their tentacles in all of these different directions? And very quickly,

• 9:48Speaker 01

  over the course of those three days and also many other interviews afterwards, I realized that everything that they were saying seemed to be a facade. And that the organization, it had already transitioned into having a for-profit arm within the nonprofit. It seemed to be on the cusp of starting to engage in commercial endeavors. It said that it was a highly transparent organization, but I found that it was incredibly secretive. And it said it was a very collaborative organization and I found that it was incredibly competitive. And so there was this Joinedness that I was starting to discover and I ended up writing a profile kind of just calling that out.

• 10:30Speaker 01

  Like, it seems like this organization that has kind of accumulated a significant amount of goodwill and also a significant amount of capital based on these altruistic notions that is going to be a bastion of ethical AI research is all just that. It's just storytelling. It's really effective storytelling that is not actually reflective of what's happening beneath the surface. My relationship with the company after I published that profile immediately soured. They were extremely unhappy with my reporting. And ever since then, it's been a very tenuous, a very tenuous relationship where over time I have continued to report on the company.

• 11:13Speaker 01

  I've continued to be critical of the company because I think a lot of the things that I said in my initial profile ended up being exactly right. And they have continued to be really frustrated by my work.

• 11:26Demetri Kofinas?

  So I feel like there are a competing set of incentives within the company or sort of objectives and cultural threads as well. Like, obviously, there's the sort of like accelerationist narrative that grows over time. There's also the influence of EA, which isn't necessarily incompatible, effective altruism. And there's also like Sam Altman and his personality. And so much of what you're describing here seems to really be a description or it seems to align very closely with multiple descriptions, not just from your book, but other descriptions I've also gotten of Sam. How much is what you're describing and the ethos of OpenAI and where it is today really a reflection of his own political goals and ambitions?

• 12:07Speaker 01

  Yeah, it's a great question because I think there's kind of two ways to answer it. One is, yes, we are talking about Sam and OpenAI is very much an embodiment of him, which is something that I sort of came to the conclusion after doing reporting for my book is there's so much about that company and that organization that really is just a direct reflection of kind of the way that he operates in the world, which is very much like he is just a phenomenal storyteller. He is like a once in a generation. Storyteller. This is why he's also a once in a generation fundraising talent. He is able to create these sweeping visions of the future that you really want to be a part of and that you want to give lots of money to. And he also figures out how to frame things in a way

• 12:50Speaker 01

  that if you are a part of it, you will be enriched greatly. And I think the other thing that I've sort of come to the conclusion about over time is that he also has a very loose relationship with the truth, and that's part of his superpower when it comes to storytelling is if you can kind of divorce yourself a little bit from the truth to tell whatever story you need to tell, you're going to be incredibly effective. Like, that's a phenomenal weapon. The other way to answer this question is that Sam Altman is also the perfect product of Silicon Valley. And so, like, is OpenAI just a manifestation of him, the man? Or is it also just a manifestation of the system?

• 13:31Speaker 01

  And he is sort of like the ultimate product of that system. And I think it's a little bit of both in that, like, the tech industry, Silicon Valley, has been on this trajectory over the last two decades where it really does index on storytelling. And it really does index on using narrative as a way to accumulate wealth and power. And so, he is one of the most effective at doing that. But I don't think he's uniquely, he's not like an island. He exists in his context, and he is a product of that context. And therefore, OpenAI is an extension of it as well.

• 14:13Demetri Kofinas?

  So what is the, maybe this is a good time to talk about origin stories? Because the book begins, actually, with the 2015 meeting that happens with Sam Altman, Elon Musk, and some other folks that are part of the original team. How did the company get started? Like, and how was that relevant to the telling of this book?

• 14:32Speaker 01

  Yeah, so OpenAI was originally started as an idea that Sam Altman had, and he approached Elon Musk to essentially get a big name attached to the organization. And the way that he approached Musk was by kind of appealing to specific things that Musk had become deeply obsessed with during that particular time in 2015. So Musk was really, he was starting to think a lot about AI as a technology for power and control. And he was particularly concerned about the fact that, one, he was not the one that was accumulating that power and control, but, two, that it was happening

• 15:12Speaker 01

  under a for-profit entity, because Google and DeepMind at the time had essentially really effectively created a monopoly on the best AI research talent in the world. And so he was worried that therefore, if they monopolize talent, they can monopolize the technology and the technology is going to be developing and evolving under this for-profit framework. And so Sam Altman tells Musk, like, I totally agree with you that this is a really deep concern and we shouldn't allow Google to do this. And it seems to me that the best way to counter Google would be to just create another lab that competes with them, but on totally opposite ideals. We're going to be a nonprofit. We're going to be transparent.

• 15:56Speaker 01

  We're going to be completely open and share all of our research to the world. And so Musk really loves this idea. And Sam Altman then organizes this dinner with Elon Musk there with a bunch of AI researchers that ultimately become kind of the leaders of OpenAI. And the book opens with that dinner and the kind of discussions that they were having at the time, where they were really, for the time, they were talking in these very outlandish and very like highfalutin ways about AI, that this was going to, like, they were going to shape the future of humanity through shaping the future of this technology. And part of the reason why a lot of the researchers that were

• 16:38Speaker 01

  actually even came to that dinner was because they heard Elon Musk was coming. So this kind of also gets to Sam Altman's very strategic ability to kind of bring the right people into the room such that he's able to get them focused on like a particular goal.

• 16:54Demetri Kofinas?

  And how important was just Musk's brand name?

• 16:58Demetri Kofinas?

  In terms of, let's say, Sam's vision for fundraising and building talent at the company, like, how much was he focused on actually bringing in someone like that in order to get the flywheel going at the company?

• 17:12Speaker 01

  I think it was a key part of the strategy because Altman kind of identified early on that in order for this to work, he would need to get some really top talent AI researchers. And he would need to compete with the fact that they were being paid millions of dollars at Google. And he was not necessarily going to have that kind of budget. And so he was cognizant of like, if I can't compete on money, I need to compete on something else, like some kind of reputation. I need some kind of draw. And he had identified specifically Ilya Sutskever, who ended up becoming the chief scientist at OpenAI, as the main person that he wanted to sway to get to OpenAI. Because Ilya had a lot of cachet within the AI research world. So if he could get Ilya over, that would then be the reputational

• 17:57Speaker 01

  draw for other AI researchers to come in. But in order to hook in Ilya, Musk was like the key to that. And Ilya, at the time, specifically went to the dinner because he heard Elon Musk was attending. And so it really was like a linchpin of the strategy to kind of bring Musk in, to then gather all these people, and then to be able to fundraise off of all of their names and reputations.

• 18:20Demetri Kofinas?

  Just another quick question, not to take us too far afield. But back then, so this is 2015, right, was the dinner?

• 18:24Speaker 01

  Yeah.

• 18:25Demetri Kofinas?

  That was the same year, if I'm not mistaken, that Nick Bostrom published Superintelligence?

• 18:29Speaker 01

  I think he published it in 2014. But yeah, it was around that time.

• 18:33Demetri Kofinas?

  You might be right. I read it in 2015. So you might be right.

• 18:35Speaker 01

  Yeah.

• 18:36Demetri Kofinas?

  And this is basically around the time that I was introduced to the subject, beyond like, you know, Terminator 2.

• 18:40Speaker 01

  Yeah.

• 18:40Demetri Kofinas?

  And the existential risk thread was sort of the dominant thread through which I was introduced to it. And it seemed like this was what a lot of the conversations were about. How important was that also, the fact that Musk was talking about it, Musk was interested in it, he was getting influenced by these ideas? Ilya was also sort of focused on the risk component. How big was that here?

• 19:00Speaker 01

  Yeah.

• 19:00Demetri Kofinas?

  The founding of OpenAI in terms of where the focus was?

• 19:02Speaker 01

  Yeah, it was a really core part of OpenAI's founding DNA was, as you said, Musk was really obsessed with this idea that AI could somehow kill humanity. It could destroy all of humanity. And in this kind of brand of existential risk, the fear that people with this belief have is not that it'll, is specifically that it will destroy humanity beyond its regeneration, that that is like the end, end, end, end.

• 19:28Speaker 01

  And he had read Nick Bostrom's book. He really loved the ideas in it. He was actually so obsessed with this idea that he thought about writing his own book about existential risk. And there were about half, maybe more than half of the researchers that ended up coming to the dinner were all kind of similarly minded in that they were all deeply concerned and unsettled by the notion that AI could become super powerful and fall into the wrong hands. And so that was like an impetus for, it's not just Google that we're worried about. It's just any evil corporation or government we're worried about. And the only way to solve this problem is for us, the goodies, to actually do it first.

• 20:15Speaker 01

  And so that was ultimately what led to the formation of this organization that then became a company.

• 20:23Demetri Kofinas?

  Yeah, again, I want to bring us back to the origin story and how things progressed from that initial meeting with my follow-up question, but I just want to make the observation that this is really, this is important because so much of the early messaging coming out of OpenAI was focused on existential risk and this sort of high-minded, in fact, isn't it right there in the, in the sort of, not the tagline, but the mission statement of the company about, we want to build AI that benefits all of humanity? And yet, what we've, what we have seen in the last few years has been a move away from that and this sort of broader push for commercialization and also a framing within the paradigm of the US-China strategic competition that we need to build AI in the Western world before China gets it, which is really a complete flip of the narrative that we were living with for the last however many years.

• 21:09Speaker 01

  Yeah, it is completely a flip of the narrative. But what's interesting is, I mean, they have not moved away from their mission. The mission stays the same, to ensure AGI, artificial general intelligence, benefits all of humanity. And one of the things that I talk about throughout the book is how this particular mission statement is so vague that it's effectively, it's just like whoever wants to interpret it however they want, it gets to do so. And so that's why you're able to see this organization just do a 180, but continue to argue in the public sphere that they are continuing to uphold their mission in exactly the way that it they're supposed to. Because there's no good definition of what ensure means in this phrase, there's no good

• 21:53Speaker 01

  definition of what AGI means, there's no good definition of what benefits humanity means, like, and so they can just interpret it, reinterpret it, redirect it however they want, to do whatever ultimately is the most effective way to continue, to continue accumulating power. Because when you actually just look at what they're doing at face value, that is really ultimately the single, like, clearest thing that they have accomplished.

• 22:19Demetri Kofinas?

  All right, so we'll get in more into that as well, but let's bring us back to the dinner and to the early founding. So how did things progress from here? I mean, eventually Elon left the company, I think you started to delve into that. And of course, OpenAI moved from a nonprofit into a quasi-for-profit. I mean, I forget the particular, and they had a unique name that they used for that.

• 22:37Speaker 01

  Capped profit.

• 22:37Demetri Kofinas?

  For-profit, limited partnership, yeah, capped profit, which was first I ever heard of something like that. I don't know if they innovated.

• 22:43Speaker 01

  Yeah, they invented it.

• 22:44Demetri Kofinas?

  They invented that. So, so walk me through that process, from that 2015 dinner to maybe up to the Microsoft deal and through that.

• 22:52Speaker 01

  Yeah, so when they were founded as a nonprofit, basically a bunch of different backers pledged to give a billion dollars to the organization. And by pledge, they just signed like a statement saying that they would be willing to contribute that much. And it was Musk's idea at the time that he wanted the billion-dollar figure because he wanted the announcement to be splashy and he wanted to quote unquote, not sound hopeless against Google and DeepMind and the budget that they would have. And so he said, like, everyone else chip in and whatever, we fall short, I will then fill in the rest. And then in 2017, 2018, the organization at that point had, I mean,

• 23:33Speaker 01

  Early days, they were very aimless, you could say. Like, they were an extremely, like, talent-dense group of people that had an extraordinary amount of runway, or so they were told. But they didn't really have any articulated vision of what they were actually doing. And so they were kind of just throwing spaghetti at the wall. And there was, like, not really that much interesting stuff coming out of the organization. And so in 2017, in part because Musk was starting to get quite frustrated about Google and DeepMind making rapid progress, and AlphaGo coming out and making all of these international headlines, and just clearly bringing lots of love and adulation to DeepMind, Musk started putting a lot of pressure on the organization, like, we need to figure out an actual plan.

• 24:17Speaker 01

  And so in 2017, 2018, Greg Brockman and Ilya Sutskever, the CTO and chief scientist, sat down and started figuring out what would it actually take to try and make rapid AI progress, such that OpenAI becomes the number-one leader? And around the same time, there were other researchers within the organization that were kind of asking a similar question, but from a different angle, which was, what has been the pace of AI progress thus far? And how much computational resources have been required to keep up that pace? And essentially, through the strategic conversations and the research that was happening,

• 25:00Speaker 01

  they came to this decision that the way to accelerate their progress was simply to try and build the biggest supercomputers possible. Because if they could own the largest supercomputers in the world, then they could train the largest AI models in the world, and therefore, kind of brute-force their way to being number-one quickly. And that is when they realized... A billion dollars is not even enough. We need way more money to build the largest supercomputers in the world.

• 25:34Demetri Kofinas?

  So, quick interjection for clarification. At what point did the view emerge that computational power and training ever more data and brute forcing it, as you said, was the way to actually scale the fastest? And to sort of achieve, quote, AGI, which again is a sort of fuzzy benchmark. We can get into that as well. But when did that sort of view emerge?

• 25:57Speaker 01

  So that was a view that had kind of been floating around in the AI field as one extreme in a range of different opinions about how to make AI progress. So the other end of the extreme was this idea that whatever is going to get us to more AI progress doesn't exist yet. Like, we need new research, fundamental research. We need new breakthroughs, new ideas to actually reach that progress. And then the compute computational resource view was, we actually already have all the ideas and it's just a matter of throwing an insane amount of computational horsepower behind it. And it so happens that Ilya Sutskever was of that extreme camp. And so when he became chief scientist of OpenAI, he brought his particular belief

• 26:41Speaker 01

  into the organization and started orchestrating its research around that belief. And so when, when all of the researchers and Greg Brockman and he sat down to think, how do we actually get to number one as quickly as possible? He came already with this philosophy of, well, the best way to do it is through computational resources and therefore let's just go, go, go and try and buy the biggest possible computer we can get our hands on.

• 27:11Demetri Kofinas?

  You know, like, one of the things that emerges in the book is that obviously this was convenient for certain folks because it allowed them to build, in their mind at least, a sort of moat around the company and around the project. It doesn't seem like that would have necessarily been Ilya's thinking. Do you have any sense of sort of what convinced him so early on that this was the way forward?

• 27:32Speaker 01

  I think throughout OpenAI's history, what's interesting is different people have totally different objectives that kind of then end up coalescing around like certain directions because they end up aligning in that way. So Ilya, he's very scientifically motivated. Like, he is just really fascinated by what would happen if, like, from a research perspective. And in his own career, at that point, he was already considered an AI luminary. He had already been part of a number of breakthroughs, but the most important one had happened in 2012, which actually kicked off the entire deep learning revolution. And that was when he was a grad student studying under Jeff Hinton, who recently won a Nobel Prize in

• 28:16Speaker 01

  part for this work, where they used deep learning to do image recognition at a academic contest called ImageNet. And they showed in 2012 that their deep learning system was able to dramatically outperform any of the other techniques that were not deep learning. And it was like the first time in that contest where there had just been a step change in progress in recognizing what's in an image. And that specific breakthrough and other breakthroughs that Ilya was part of had in part been aided by scale. Scale was not the only thing that they were aided by. There were also new techniques that were being deployed. There

• 29:01Speaker 01

  was more data curation methods, things like that. But for Ilya, I think he just became really convinced that scale was the most important factor in all the breakthroughs that he had been a part of. And he's talked before about like, there's sort of this, one thing that's really interesting when listening to Ilya talk about deep learning and his particular philosophy is he does describe it as a belief. He doesn't describe it as any kind of like observational science. It's like he has a kind of religious belief in the idea that scaling deep learning will create magical things. And he puts up this chart sometimes where

• 29:41Speaker 01

  He'll talk about like the size of human brains being the largest brains of all animals. And if you generally map out like animals in the animal kingdom along a trajectory of both intelligence and brain size, you see that there's sort of a linear relationship with brain size and intelligence. And he also has this belief that neural networks, the software that underpins modern AI systems today, are an effective approximation of brains. Not everyone believes this or agrees with this, but like he just has like, that is his religion. Like, he believes that neural networks have effectively figured out how to replicate brains. And therefore, if you put those two ideas together, you just have

• 30:22Speaker 01

  to scale. You just need the biggest brain possible to get the most intelligence. So that's essentially like why OpenAI set down the path from his, like it got his sign off. But to your point, like, there were many other reasons that other executives signed off for, like, they weren't necessarily interested in the science, they were interested in the competitive moat that it would bring OpenAI. If you can get the biggest supercomputer, then that means other people are not going to be building this technology that you think is going to give you accrue you a lot of power. And so it creates like a really effective way to solidify your advantage.

• 31:03Demetri Kofinas?

  Yeah, I remember coming across the work of someone named Giulio Tenoni, who proposed this emergent theory of consciousness back in the early 2000s called integrated information theory. It was one of a number of theories being bandied about in various transhumanist and Kurtzweilian thought circles. And the idea was essentially that the more connections you have, the more integrated the information is, and ergo, the more consciousness, whatever thing you are examining has. And it feels Very much like this was the same sort of connectionism idea being proposed here. And I love how you point out that this scaling law isn't some kind of testable hypothesis,

• 31:46Demetri Kofinas?

  that it's more of a religious belief. And that this field in general, AI, the commercialization of AI, even that term, "commercialization," doesn't quite fit the rhetoric used to describe both the opportunity and the responsibility to broader humanity associated with the development of this technology. And I feel like many of the people within this field are also grappling with some very unfamiliar questions, not engineering questions per se, but teleological questions like, why are we building this technology? And who are we building it for? That fall outside the scope of traditional discussions about product market fit and audience targeting.

• 32:29Demetri Kofinas?

  And maybe what's missing in this equation is that the public isn't represented as a stakeholder in this process, the way he or she was during the development of the atom bomb or the Gemini and Apollo missions. Does that resonate with you?

• 32:48Speaker 01

  Yeah, I think that's exactly right. And I think there's kind of two reasons that I feel like this has happened. One is that, is what you're describing, that there isn't really like a clearly articulated idea of what we should be doing the technology, building this technology for, or who we should be building this technology for. And I think that ties a lot into some of the ideas that you've talked about in the past of just like the kind of decline in, the moral decline in, like, people are doing things purely for the sake of the game now. Like, they're doing things because it is like, yeah, there isn't like a strong conviction around, like, why should I be doing anything that I'm doing? It's just they're kind of chasing numbers and chasing economic value.

• 33:33Speaker 01

  But I think the other challenge with AI is the original field of AI, which is a very old field, it was founded in 1956. It was based on this idea that, like, we're going to create this field that tries to do everything to recreate human intelligence. And to this day, there is no scientific agreement on what human intelligence is. And that's very different from any other technology in that there's basically every other technology, you have a very clear definition of what it is in the first place. And so that's part of the reason why this AI is so ripe for ideological interpretation, religious projection, you know, like it is, it is just so ripe for,

• 34:17Speaker 01

  it's sort of just a mirror that people can put up to themselves and they see whatever they want to see. And there isn't really a collective consensus around what it is, why it is, like who it should be for, and all of these things.

• 34:31Demetri Kofinas?

  All right. So I, I took us on a little bit of an interlude there, but we were talking, I think we were back in the 2017, 2018 period where folks at the company decided that scaling and throwing more compute at this was the way to achieve AGI or achieve a competitive advantage over other companies. How did things progress from there? And how did that lead us to the partnership with Microsoft?

• 34:51Speaker 01

  So once they realized that they needed a lot of money in order to pull off this particular path for getting to AGI first, they actually then immediately lost their main backer, Elon Musk, because they started talking about, we need to convert into a for-profit because there's no way that we're going to be able to raise the amount of money that we foresee ourselves meeting under a nonprofit. And once there were discussions of a for-profit, Musk started getting really pissed off about that. And there was sort of a tussle between him and Altman over, okay, well, if there's a for-profit, who would be in control of it? And they both wanted to become CEO. There wasn't any agreement over it. And then eventually Musk pulled out.

• 35:36Speaker 01

  He pulled out of the project because he was like, I don't like the fact that there might be a for-profit and I might not be CEO. So I wanted to stay a nonprofit. If you're not going to stay a nonprofit, I'm out. And so when he left, it sort of sent the organization into this bit of a crisis period where they had just realized they need extraordinary amount of money. And now they have also in the same moment, lost the capital that they originally thought they had. And so Altman did what he does best. He started going out and fundraising. And essentially, he ended up going to the Sun Valley Conference, which is this kind of gathering of a lot of billionaires, I think with the intent to kind of just rub shoulders with people that might be able to give money to this effort. And he ends up running into Satya Nadella,

• 36:21Speaker 01

  Microsoft's CEO, on the stairwell, a stairwell at the conference, and kind of quickly pitches him this idea of OpenAI. And it piqued Nadella's interest just enough that they started having more conversations about it. And then it kind of very rapidly then turned into Microsoft deciding to plug the hole and joining up with OpenAI in a partnership where Microsoft would give a billion dollars. And specifically, part of the reason why Altman was looking for a backer like Microsoft was because Microsoft also had the ability to give them computational resources. And for Microsoft, they were very intrigued by OpenAI because they felt that they had also fallen behind Google.

• 37:05Speaker 01

  And Google had, because of its ambitions around AI and because of its acquisition of DeepMind, had done a lot more to innovate on hardware than Microsoft had. And Google had surpassed Microsoft's, well surpassed Microsoft's ability to build powerful supercomputers. So Microsoft wanted to kind of give itself a kick in the butt by backing OpenAI and using that to add urgency to its own development, our own advancement of hardware, so that it could also kind of learn by doing and start making really powerful supercomputers.

• 37:42Demetri Kofinas?

  So now you have this really important commercial relationship, and you have this kind of quasi nonprofit commercial entity, which I think eventually sort of leads us to that infamous boardroom coup, which was in 2023.

• 37:56Speaker 01

  November.

• 37:57Demetri Kofinas?

  November of 2023. Walk me through what happened there. And what do we know about the the internal discussions and rifts within the company that emerged during this period?

• 38:08Speaker 01

  So because OpenAI had this Frankenstein corporate structure where it was a nonprofit, but then it added this capped profit arm within it to raise money and do the partnership with Microsoft, it ended up being plagued from 2018 all the way through 2023. It was just plagued with very different factions within the company that had very different understandings of what even OpenAI was. And this is, it's like, they had very different understanding of what OpenAI was, also very different understandings of what the mission was, because as I mentioned, like, the mission is just so vague that anyone can kind of put the mission up to themselves as a mirror,

• 38:51Speaker 01

  and it's just a projection of whatever they want it to be. And so there were two main factions that kind of emerged. One that kind of had these, still had this very existential risk mindset around why they were building this technology. They wanted to be the ones that build it and keep it in the hands of good people, not bad people. And they wanted to make sure that it was done with utmost caution so that it wouldn't lead to the demise of humanity. And they were basically overlapped with the group of people who also then believed that OpenAI, at the end of the day, even though it has this weird capped profit arm, is spiritually a nonprofit, and it should operate like a nonprofit, where even if it's

• 39:35Speaker 01

  commercializing things on the side, ultimately, the decisions that the organization should make at a high level should never prioritize capital over safety. The other faction was people that were being brought into the organization to run the capped profit and to actually figure out how to build technologies that could commercialize and therefore give a return on investment to investors. And so those were the people that were much more philosophically similar to just a standard Silicon Valley person. They're trying to figure out, how do we build really exciting user interfaces? How do we build a sustainable business model? How do we try and get like rapid user growth?

• 40:17Speaker 01

  So that we can demonstrate hockey stick growth to people, then convince them to give us money. And those were the people that saw OpenAI as, maybe it was a nonprofit at one point, but now it's a for-profit and we need to operate like it's a for-profit. And also, we need to operate just within the reality of the world today, which is like if we want to achieve our mission to get to AGI first, we have to have capital, we have to play that game. And that group also increasingly also felt that as OpenAI started deploying AI technologies more and more into the world, that the world wasn't ending. And actually, in fact, they were seeing lots of evidence of getting this technology into people's hands, bringing benefit.

• 41:00Speaker 01

  And so their interpretation of the mission became, actually, we need to deploy this technology as fast as possible to ensure AGI benefits all of humanity. And so obviously, you can see that these are totally diametrically opposed views, both on what benefit means and on what OpenAI even is. And it was the board of the nonprofit. The board had for a while been sort of fifty fifty split, or I should say, it was fifty fifty split among the board members that were not also employees of the company. So they were the independent board members. And over the course of end of twenty twenty two and early twenty twenty three, or maybe just early twenty twenty three,

• 41:42Speaker 01

  the three board members that were most inclined to think about the company as a company, left for various reasons, in part actually because of ChatGPT's success. And two of them ended up, it wrapped up in conflicts of interest where they started being involved themselves in OpenAI competitors. And so they couldn't serve on the board anymore. And so what happened with the board crisis was basically there were three board members, independent board members left, that were most inclined to think about OpenAI still as spiritually a nonprofit and therefore something that needs to be always prioritizing safety over profits. And then three employees, one of whom, Ilya Sutskever,

• 42:27Speaker 01

  then increasingly also became just disenchanted with Altman, both his personality and his vision for how he was taking the company. And so he then ended up allying with the other three board members. And that resulted in just the most spectacular. Drama of in probably the history of the tech industry.

• 42:48Demetri Kofinas?

  So let's walk through, I mean, again, I, you know, I don't think the most interesting thing here is the drama, but people remember this episode and I think that was when I first reached out to you, actually. Cause we had been in touch before the publication of your book and it might've been around this time. Because there was a lot of confusion about what was going on and also Ilya was, you know, he signed off on the quote coup. But then he reversed himself and then he ended up leaving the company. So did his reversal just reflect the power that Sam Altman actually had and that it just wasn't practical, also given the relationship with Microsoft and the fact that Sam could leave and take a bunch of engineers with him? It just wasn't practical to actually try to unseat him?

• 43:26Speaker 01

  Yeah, I think Sam and Ilya are opposites, very much so, in that Sam is a political animal and Ilya is not. Ilya is someone that is quite politically naive. And honestly, he just did not understand, he didn't even think to think through what would happen once they fired Sam. I think that's honestly what happened is he didn't, it never occurred to him that other people would not celebrate this and that they didn't see Sam the way that he saw Sam. And he never thought about the fact that they would be up in arms if he didn't give them a good answer.

• 44:06Speaker 01

  Not just him, but the board didn't give employees a good answer or investors a good answer or Microsoft a good answer about what had happened. And so basically he did this because he wanted to strengthen OpenAI. Because he was gravely concerned about how OpenAI might turn out under Sam's leadership and how AGI, his conception of AGI might turn out under Altman's leadership. But the moment that it became kind of clear to him that it was not at all strengthening OpenAI, but in fact might lead to its dissolution, that's when he flipped.

• 44:46Speaker 01

  So it wasn't that he didn't have high conviction in his decisions, it's that he was outmaneuvered, like he did not.

• 44:52Demetri Kofinas?

  He's naive.

• 44:53Speaker 01

  Yeah, he just didn't consider that fact that there would be consequences.

• 44:58Demetri Kofinas?

  So has the industry just abandoned the alignment problem as a primary focal point of concern for developers in the space? I mean, are we just like full blown now, just focused on commercialization of AI?

• 45:13Speaker 01

  I think maybe to take a step back, one of the things that I am kind of critical of in general is I think a lot of the framing around the AI debate today ends up being framed as like safety versus profits, but I'm actually critical of both. Like, I think, I don't think it's one or the other. To me, they're actually a little bit two sides of the same coin in that the safety people and the profit people, they both have this argument that they should be the ones that control this technology because they're the ones that are going to do it best. And part of through the history of the organization, you see how like each of these ideologies, even though they hate one another, actually end up just pushing to accelerate this technology

• 45:57Speaker 01

  faster and faster and faster. Like, they both bear that responsibility for doing that. And so, has the industry started moving more and more, indexing more and more towards commercialization and less towards like this doomer safety ideology? I think so, but does that really mean much?

• 46:18Demetri Kofinas?

  Well, one of the things that comes across in your book is both of those orientations are sort of ends justify the means.

• 46:25Speaker 01

  Yes.

• 46:25Demetri Kofinas?

  Approach, right? So is that sort of also what you're getting at? That in, in the service of these larger ideals, we're sort of losing touch with the larger externalities that this race is, is creating. Because you go through that in the book, not just the ecological effects of building these data centers, but also like fascinating looks into people responsible for cleansing some of the data, Venezuela and Kenya, or some of the reinforcement learning through human feedback that is sort of outsourced to these farms of people working for one to two dollars an hour. Like, how much of that story is actually not, not known to most people, do you think? Because it wasn't really, I'll be honest, I didn't really know much about it.

• 47:03Speaker 01

  Yeah, I don't think a lot of people know about it, and I think it's by design because like, I guess the way that I would talk about this particular part of the book is I've always been really fascinated by the AI supply chain because people don't really think of digital technologies as having a physical supply chain, the way that, you know, fashion or coffee or gold has a physical supply chain, but it does. And this is by design that Silicon Valley has always tried to make themselves appear magical by hiding the physical supply chain. And, you know, a lot of industries hide their supply chains by putting them far and away from where consumers can see them. But I think it's, there's even more layers of obfuscation because the average consumer doesn't think when they're using ChatGPT

• 47:48Speaker 01

  or they're using some other AI tool, that there is actually an incredibly large physical footprint somewhere out there in the world. And so the thing, I guess to tie back to like the, what you were saying about like ultimately, what is my criticism of these two different groups is they are having conversations completely detached from the reality of how AI is impacting people today, in part because they are completely detached from the reality of how people in the vast majority of the world live today. And when I talk with people within the AI world who are either optimistic or pessimistic,

• 48:29Speaker 01

  they have no recognition of the fact that their bubble in Silicon Valley is not representative of, I mean, obviously they realize that they live differently potentially from other people, but they can't even conceive of how differently other people live. And when I probe them, I remember I was talking with one researcher who was very much in the utopia camp and was like, you know, we get to AGI and that's it, like euphoria for everyone, utopia for everyone. And I was like, can you help me understand, like, can you walk me step by step through, like, how you see it helping the bottom of the rung, like the people that can't put food on the table for their kids? And he went,

• 49:15Speaker 01

  Oh, I wasn't talking about like the bottom. Like I was talking and I was like, wait, so what do you mean when you say all of humanity? Because it seems to me that only some people deserve humanity here. And there's just other people that are written off as, oh, and the other people. Yeah, so that's my biggest criticism is like, I don't really see either side, the profit or the safety side, actually be cognizant about the fact that the majority of the world lives in precarity.

• 49:51Demetri Kofinas?

  Yeah, I mean, actually, you know, not to mix up metaphors here, but like the alignment problem is also an alignment problem of cultures and Silicon Valley has its own culture. And their concerns have grown ever more sort of disconnected from those of, I feel like, the mass of humanity. You know, there's been this long obsession about, I never quite understood it, about becoming an intergalactic species, we need to go to, and also we need to go to Mars. And that's the first step because, you know, humanity is not going to survive long term on this planet. And we need to seed other planets. I never really understood that, quite frankly.

• 50:22Speaker 01

  Yeah.

• 50:23Demetri Kofinas?

  And I don't feel like most people really care about that. I mean, I don't know how popular that idea really is. It's popular on social media. And, you know, this religious thing keeps coming up. I had done this episode with Megan O'Giblin and actually even Nick Bostrum, you know, who had been on the podcast, you know, I was introduced to his work through superintelligence. And it was super interesting, all these philosophical ideas. And then he published Deep Utopia and it was almost like kind of complete reversal. So he went from being sort of like Duma focused to being sort of utopian focused. And I was saying, I did this episode with Megan O'Giblin, who wrote a book about her experience, both with religion and then also delving into a lot of these communities. And there's such strong religious overtones that it almost feels like, you know, a lot of what's also happening is that there is a kind of religious cult that's developed within Silicon Valley. And the conduit for expressing these views is AI development.

• 51:07Speaker 01

  100%. I think that's exactly the right way to think about it is, yeah, there are, I usually call them quasi-religious, but they are just fully religious when you think about. What they talk about, which is that like, some of them even adopt, like they say, I, we're on a quest to build an AI god, you know, like they will adopt religious language themselves. But they are based in this, these belief systems of we're going to create a thing that will either bring us to heaven or to hell. But the difference between previous religions was that, like, there was a superior, higher power that was determining whether you go to heaven or hell. And in this religion, you are the one that determines whether the human race goes to heaven or to hell.

• 51:50Speaker 01

  And so there is this profound religious undertone to the whole enterprise of AI development that you can't really understand. Like, some people say that the story of the AI industry is just a story of capital. And I actually think that is not true at all, that you would be unable to understand all of the strange phenomenon and the kind of clashing and the like decisions that people and organizations make just under the lens of capital. It is also 100% a story of ideology.

• 52:22Demetri Kofinas?

  But in that sense, also, it reminds me of 100 years ago during the Gilded Age and the rise of social Darwinism and eugenicism as sort of popular theories among elites in society. And it feels like this is something similar. But I guess a key distinction is rather than us breeding supermen and superwomen, we were getting to a place where we created a sort of god around which we can transform ourselves. And the people that are in a position to have the capital to be able to do it will be able to sort of escape. And there's also this strong, that's the other thing that's really interesting about the time we live in today, which wasn't true 100 years ago. There was no talk 100 years ago about escaping from the earth.

• 52:58Speaker 01

  Yeah.

• 52:58Demetri Kofinas?

  Escaping from this sort of like dystopian future, you know, and getting out. Today, there's this strong cult of escapism within the AI community. I actually saw a recent interview of Paul Tudor Jones, where he was talking about how he was with sort of in this small group of sort of AI luminaries. And someone who was very high up in the field was saying that his sort of strategy is he bought like, I forget how many acres in the Midwest. And he's got provisions and everything else. He's preparing for a sort of doomsday where 50 to 100 million people die because AI developed some kind of new virus that kills all sorts of people.

• 53:30Speaker 01

  Yeah, all of the big wigs in Silicon Valley are doomsday preppers. I mean, Sam Altman's also a doomsday prepper, even before he developed, he started working on OpenAI. And I think this ties into. What you talk about with nihilism is like the average person is checking out, but also the elites are checking out. Like, they no longer want to deal with all of the problems that are present today. Like, they no longer feel conviction around salvaging our current planet and our current country and our current, you know, like preserving just the things that work well. They don't feel this kind of attachment to it anymore. They just want to throw it all behind and move to something else. And so I think that's

• 54:13Speaker 01

  part of the reason why we just have all of this capital and all of these environmental externalities that are coming out of developing a technology. Like, I think in part it's because they don't really care how much cost it takes on the current planet and the current paradigm of our social order, because they're done with it. Like, they're just trying to, while like this earth still exists, like pump all of that, the resources that they can.

• 54:40Demetri Kofinas?

  Rape as much of it as possible.

• 54:40Speaker 01

  Yeah, exactly.

• 54:41Demetri Kofinas?

  Exactly.

• 54:42Speaker 01

  Exactly. To try and like, get that rocket ship to lift off, and then they're done.

• 54:47Demetri Kofinas?

  I think that, again, that speaks to the alignment problem that we talked about, not the traditional alignment problem in AI, but the larger alignment between the haves and have-nots in society and this area of real just concentrated power. So like, what is the conversation that you feel, Karen, should be happening today that isn't happening? And how do you feel like your book, or how do you hope your book can help sort of get that conversation going or bring it more to the fore?

• 55:11Speaker 01

  So I think the thing that I'm most disappointed by in AI discourse in general is that people still deeply buy, not everyone, but there's still so many people that deeply buy into this idea that there is such a thing as artificial general intelligence and that we shouldn't trust Silicon Valley to build it and that it will somehow bring us to utopia. And I think all of these things are leading us instead to just continuously cede more and more control to people, a group of people that have already demonstrated that they have no interest in protecting. Like good things in this world.

• 55:53Speaker 01

  And we're leading to a place where ultimately we are undermining like the liberal world's order. We're undermining democracy. We're reversing all of the progress that we've made and returning back to an age of empire. That's why my book ultimately is called Empire of AI. I I make this argument that we are rapidly recreating a time where there are small group of people that get to make all of the decisions and they act completely in self-interest. And they extract and exploit and do whatever they need to do to continue to enrich themselves and fortify their empire. And everyone else kind of lives with the ramifications of their whims.

• 56:38Speaker 01

  This has already been something that I think within Silicon Valley has already been brewing for a long time. And, you know, people have colloquially called tech companies empires for a long time. But I think we need to take AI companies much more seriously as literal empires. Because the amount of capital and the amount of resources and the amount of extraction, the amount of exploitation has all upped one to two orders of magnitude from previous social media and search companies. Like the amount of environmental devastation that is happening because of the need to build, to cover our planet with data centers and supercomputers is unparalleled. We have never seen this level of build out at this speed before.

• 57:21Speaker 01

  Just one figure, like President Trump announced earlier this year, the Stargate Initiative, which is meant to be a $500 billion private investment into building data centers and supercomputers, which OpenAI says it's all for itself over the course of four years, $500 billion over the course of four years. The Apollo program to send the first man to the moon was around $300 billion, $2025 over 13 years.

• 57:47Speaker 01

  So 500 billion for a single company to just create a technology that it's still not

• 57:54Demetri Kofinas?

  And it's proprietary.

• 57:55Speaker 01

  And it's proprietary and it's not entirely clear what it is and who it's going to benefit. That is insane. And yeah, that's ultimately why I want to write the book is like to just point out like, what are we doing here? You know, like when this is happening, there are still people buying into the idea that we should just wait and see and hope that on the other end we reach euphoria. Like it's, there's not going to be an end that we can see because at that point it's too late. Like the empires have already constructed themselves. The earth has already been laid to waste and democracy is no longer tenable. So like, we can't, yeah, this is, we don't get another chance.

• 58:37Demetri Kofinas?

  I think that's a really insightful framing. Didn't Shoshana Zuboff endorse her book?

• 58:41Speaker 01

  Yes, yeah.

• 58:42Demetri Kofinas?

  So that's quite fitting. You know, she had been on the show back when she published her book and I, that book was incredible.

• 58:47Speaker 01

  Yeah.

• 58:48Demetri Kofinas?

  You know, it was such an important book in terms of giving people a framework for thinking about the nature of the problem. And both of you have actually touched on this analogy of colonialization. And I think she actually used the example of the Spanish conquistadores coming to the Americas.

• 59:03Speaker 01

  Yeah.

• 59:04Demetri Kofinas?

  And the sort of natives welcoming them as sort of gods. And there does seem to be something similar happening here in sort of how you're describing this.

• 59:14Speaker 01

  Yeah, yeah, exactly. And Professor Zuboff was, I mean, her book and her ideas in general were so hugely influential to my work and this book. But yeah, it's exactly that. Like she opens her book with this idea that when you are confronted with something so new, it sort of short changes your brain circuitry to really even recognize what's happening. And she talks about her own personal experience, like where a fire started in her house and she thought that she would have more time. So she was like running around the house, trying to grab things. And then it wasn't until a firefighter like threw her out of the house and then the house burst into flames that she realized that she had been completely anchored on historical examples that were, that were irrelevant.

• 59:58Speaker 01

  And I think that is what happens with emerging technologies a lot is that they are so different that people don't recognize anymore that they're actually the same.

• 1:00:13Speaker 01

  And like the fact that Silicon Valley has somehow been able to reinvent itself from a social media era where everyone, everyone has sort of in society started agreeing that social media is bad for society, you know, like we're no longer having that debate. And yet somehow the same cast of characters has reinvented themselves now under this new branding of AI. And it's short circuited everyone into thinking, oh, wait, this time it's going to be different. This time it's going to be better. It's going to work out. They are doing it with our best interests at heart. And it's just not, that's just not what's happening.

• 1:00:50Demetri Kofinas?

  The same is true for Elon. And, you know, we live in this era of cult personalities and populism. And Elon has always reminded me of Trump. You know, the two of them are so similar in terms of how they appeal to people. And it's disheartening that the current administration has really aligned itself with these same power structures. The other thing is that, you know, like, I love that you brought up that example that Trishana brings up with the fire in her home, because she also made the point in that same chapter about how social media has resulted in the loss of sanctuary and the loss of the spaces in which we sort of come together as communities. And so how do you think, and this is sort of brings back to my question about the conversation that you hope people can have, like, how, and I don't know that you have an answer

• 1:01:34Demetri Kofinas?

  to this, but like, it's something that I've tried to explore on the show, like, how do we get to a place where we can sort of build a kind of resistance to what we're talking about here, which is the accumulation of private power in the hands of ever smaller groups of people, so that we can actually realign our economies and our political systems toward the public interest?

• 1:01:52Speaker 01

  Yeah. Going back to the AI supply chain idea, I've always thought of, okay, so it, modern day AI systems, these colossal AI models that companies are building, they need a lot of data, they need a lot of computational resources, they need a lot of land, energy, water. These are all, to me, sites of resistance. Like, we have somehow entered a headspace now where we just feel like there's nothing we can do and they're just going to take up all this stuff. But actually, if we can contain the company's ability to actually access just endless quantities

• 1:02:34Speaker 01

  of these resources, like, they can't do what they're doing. So to me, like, When thinking about like how to contain their access to data, like the US still doesn't have a federal data privacy law, we should have one. We should not be eroding away our copyright laws such that they get even more data to train their models on without any more intellectual property protections on like people's life's work. There should be like consortiums of people that actually debate publicly owned data and whether or not it should be allowed to go to these companies and be used to train these models. And there should be like public debates about the content moderation of the data sets. Because in the course of my book, like I talk about many instances in which OpenAI actually

• 1:03:19Speaker 01

  just, like there were people within the company that just decided themselves, like, should we include pornographic content in our data set or not? Actually, yes, why not? Because it is representative of the human experience. And like, that was like not at all done in any kind of like democratic way. It was just completely like a snap of judgment decision that now has like huge ramifications about like the way that these models work and the risks that they have to exposing, you know, kids to potentially illicit content. We should also be like building coalitions of people that are informed about and can ask questions about and demand changes to data centers being developed within their communities.

• 1:04:01Speaker 01

  Right now, companies actually, a lot of them enter communities under shell companies, shell entities, where they won't even say that it is them that's building a data center. Like Meta has a shell company called Greater Kudu LLC that they use to do some of their data center stuff. And communities are like, what is this random company that, and like, what project are they doing? And then, once the first brick is laid, it's like, ta-da, it's Meta and it's a data center and you can do nothing about it anymore. And we need like more, you know, regulation around how utility prices are affected. Because when data centers move in, it raises the energy costs for regular, common, for families.

• 1:04:43Speaker 01

  And there needs to be like a protection of basic human resources being affordable. So I think that's kind of how we need to contain the empire. That's the way that I think about it, is like, all of these different inputs that the empire needs to continue its accumulation are places that there can be resistance, there can be coalition building to kind of. Start pulling the control of those resources back to the people instead of to the empire.

• 1:05:12Demetri Kofinas?

  So Karen, I think this is a very important book that you've written in large part because it helps with that specific framing in terms of thinking about what are we actually facing here? What is the challenge that we're that we're facing? And I think also, to be quite honest, it was very courageous that you did write this book. It was very courageous that you've taken such a contentious stand on this issue with this company, which is not easy. I mean, this so much power has accumulated in Silicon Valley and it's not easy to begin a career or to sort of grow your career by necessarily sort of facing that down. It certainly isn't the easy way of doing it. You have to really carve out your own identity. Besides the fact that people can find this book on Amazon, which is, you know, everyone knows how to use Amazon. How do people follow you? How do they follow your work? What would you recommend people do?

• 1:05:53Speaker 01

  I am primarily these days posting on LinkedIn and Blue Sky, so people can follow me there. I have not created a newsletter yet.

• 1:06:01Demetri Kofinas?

  A Substack, yeah.

• 1:06:03Speaker 01

  Yeah, but yeah, I usually post my work on those profiles. And I just hope that people can also feel free to reach out to me. Like I'm, I try to be as responsive as possible through like my website. You can just contact me there and I try to email people back with the different requests that they have. But I also hope that like people can take the book and turn it kind of into their own and use it as a platform to do the work that they're doing. So ultimately, if I can help in any way in doing that, whether through my ideas or and words, or through literal reaching out and having a conversation, like, please let me know.

• 1:06:41Demetri Kofinas?

  Well, you're, you're an important voice in the space, Karen. So I thank you so much for writing the book and for coming on the podcast.

• 1:06:46Speaker 01

  Thank you so much for having me.

• 1:06:49Demetri Kofinas?

  If you want to listen in on the rest of today's conversation, head over to hiddenforces.io/subscribe and join our premium feed. If you want to join in on the conversation and become a member of the Hidden Forces Genius Community, you can also do that through our subscriber page. Today's episode was produced by me and edited by Stylianos Nicolaou. For more episodes, you can check out our website at hiddenforces.io. You can follow me on Twitter at Kofinas and you can email me at [email protected]. As always, thanks for listening. We'll see you next time.