AI Governance and the National Security State | Dean W. Ball

In Episode 484 https://hiddenforces.io/podcasts/ai-governance-and-the-national-security-state-dean-ball of Hidden Forces, Demetri Kofinas speaks with AI policy researcher, writer, and incoming Head of Strategic Futures at OpenAI, Dean Ball, about the intellectual foundations of machine intelligence, the governance frameworks best suited to frontier AI, and what's at stake for society, the nation state, and the individual if we get this transition wrong. The first hour builds the philosophical scaffolding for the conversation to come — what intelligence actually is, how large language models learn, what they understand, and the broader historical thesis animating Dean's worldview: that we are not witnessing the birth of something entirely new, but rather living through a computing revolution that began with the transistor and is now reaching its natural culmination in the era of machine intelligence. The second hour turns to what's at stake in this transition and the governance models Dean believes are most suited to it. They begin with his conception of superintelligence — not as a singular, all-knowing entity, but as something whose power will derive largely from being embedded in the infrastructure of human civilization. From there, they examine where Dean falls along the continuum from doomer to accelerationist, what a sensible approach to AI governance actually looks like, and the real-world test case of Anthropic's dispute with the Department of War — what it reveals about tensions between private frontier AI labs and the national security state, and how Dean thinks about forging public-private governance structures adequate to the age of AI. They close by examining labor market disruption, the overproduction of elites, and which nations, societies, and individuals are best positioned to navigate the transition ahead. Subscribe to our premium content—including our premium feed, episode transcripts, and Intelligence Reports—by visiting HiddenForces.io/subscribe https://hiddenforces.io/subscribe. If you'd like to join the conversation and become a member of the Hidden Forces Genius community—with benefits like Q&A calls with guests, exclusive research and analysis, in-person events, and dinners—you can also sign up on our subscriber page at HiddenForces.io/subscribe https://hiddenforces.io/subscribe. If you enjoyed today's episode of Hidden Forces, please support the show by: Subscribing on Apple Podcasts https://podcasts.apple.com/us/podcast/hidden-forces/id1205359334, YouTube https://www.youtube.com/channel/UC8URhgYos5fjHqFSO4RSIEg, Spotify https://open.spotify.com/show/0TK6u251zGSlMgJKH6cOaP, Stitcher https://www.stitcher.com/podcast/demetri-kofinas/hidden-forces, SoundCloud https://soundcloud.com/hiddenforces, CastBox https://castbox.fm/channel/2629539, or via our RSS Feed http://hiddenforces.libsyn.com/rss Writing us a review on Apple Podcasts https://podcasts.apple.com/us/podcast/hidden-forces/id1205359334 & Spotify https://open.spotify.com/show/0TK6u251zGSlMgJKH6cOaP Join our mailing list at https://hiddenforces.io/newsletter/ Producer & Host: Demetri Kofinas Editor & Engineer: Stylianos Nicolaou Subscribe and support the podcast at https://hiddenforces.io https://hiddenforces.io/. Join the conversation on Facebook https://www.facebook.com/hiddenforcespod/, Instagram https://www.instagram.com/hiddenforcespod/, and Twitter https://twitter.com/hiddenforcespod at @hiddenforcespod Follow Demetri on Twitter at @Kofinas https://twitter.com/kofinas Episode Recorded on 06/23/2026

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Demetri Kofinas?

  What's up, everybody? My name is Demetri Kofinas, and you're listening to Hidden Forces, a podcast that inspires investors, entrepreneurs, and everyday citizens to challenge consensus narratives and learn how to think critically about the systems of power shaping our world. My guest in this episode of Hidden Forces is Dean Ball, an AI policy researcher, writer, and the incoming head of Strategic Futures at OpenAI, a newly established, high-agency policy team formed to shape frontier AI policy and internal governance at the company. In the first hour of our conversation, Dean and I build the intellectual and philosophical foundations for the conversation to come. We discuss Dean's background and the framework through which he approaches AI,

• 0:45Demetri Kofinas?

  what intelligence actually is, the nature of large language models, how they learn, what they understand, and their advantages and limitations. And we examine the broader historical thesis that animates Dean's worldview, that we are not witnessing the birth of something entirely new, but rather, living through a computing revolution that began with the transistor and is now reaching its natural culmination in the era of machine intelligence. In the second hour, we discuss what's at stake in this transition, the model of AI governance that Dean believes will be most effective, given the nature of this technology and the incentives driving its adoption, and what's at stake for society, the nation-state, and the individual, if we get it wrong. We begin with Dean's own conception of superintelligence,

• 1:30Demetri Kofinas?

  not as a singular, all-knowing entity, but as something that will derive much of its power from being embedded in the infrastructure of human civilization. We discuss where he falls along the continuum, from doomer to accelerationist, and what a sensible approach to AI governance actually looks like, one that avoids both laissez-faire abdication and heavy-handed regulation. From there, we get into the real-world test case of Anthropics' dispute with the Department of War, what it reveals about the tensions between frontier AI labs and the national security state, and how Dean thinks about the challenge of forging public-private governance structures adequately adapted for the age of AI. Finally, we close by examining labor market disruption,

• 2:14Demetri Kofinas?

  the overproduction of elites, and the question of which nations, societies, and individuals are best positioned to navigate the transition ahead. If you want access to all of this conversation, go to hiddenforces.io/subscribe and join our premium feed, which you can listen to on your mobile device using your favorite podcast app, just like you're listening right now. If you want to join in on the conversation and become a member of the Hidden Forces Genius Community, which includes Q and A calls with guests, discounted access to third-party research and analysis, and in-person events like our intimate dinners and weekend retreats, you can also do that on our subscriber page. And if you still have questions,

• 2:56Demetri Kofinas?

  feel free to send an email to [email protected] and I or someone from our team will get right back to you. And with that, please enjoy this incredibly timely, informative and important conversation with my guest, Dean Ball.

• 3:17Demetri Kofinas?

  Dean Ball, welcome to Hidden Forces.

• 3:21Speaker 01

  Thank you so much for having me.

• 3:23Demetri Kofinas?

  It's my pleasure, Dean. I was chasing you down for a while, wasn't I?

• 3:27Speaker 01

  It took a long time to get us scheduled.

• 3:29Demetri Kofinas?

  You were a somewhat evasive guest. No, you weren't evasive. You have many demands on your time. You're a prolific writer. And I should start the show by congratulating you. You're going to be joining OpenAI as the Head of Strategic Futures. This was announced just recently. You announced it both over Twitter and on your blog, Hyperdimensional. So we had already scheduled this interview, but this was rather fortuitous. What's the nature of this new role and how did it come about?

• 3:56Speaker 01

  Well, first of all, thank you. And the way it came about really is that OpenAI approached me maybe a month, six weeks ago, something like that. And I had been thinking independently about where my work on AI policy was going to go, where I wanted it to go. And one of the thoughts that kept recurring to me, but that I didn't really ever act on, was this notion that My future research required me to go inside of a lab, a frontier AI company, to get access to the kinds of information and insights that are only available there and to sort of occupy that perspective and that that was actually just going to be an essential part of my future in this field.

• 4:40Speaker 01

  That intuition kept occurring to me, but I was doing a bunch of stuff, running around, and didn't really have the opportunity to act on that intuition. And then happily, OpenAI approached me and said, you know, what do you think about sort of coming in and helping to shape our frontier policy? I think from OpenAI's perspective, where this all comes from is really just this idea that like, you know, they have like a public policy team, a government affairs team, that does exactly what you would think a large company's government affairs team does, and they're great, they're extremely competent, but they are in this posture of reacting to policies being promulgated by 50 states, by the federal government,

• 5:24Speaker 01

  in Congress and the executive branch, court cases, you know, the European Union, countries all over the world, there's just a lot of incoming that that team has to deal with. And so what OpenAI kind of wants to create is like a small organization that doesn't have to react to any of that stuff and instead can look out on the horizon six to 12 months and kind of think, okay, where are things going? What are people going to be talking about? And how can we sort of proactively develop ideas to inform that conversation? So it's a somewhat more proactive version of a sort of like policy operation.

• 6:06Demetri Kofinas?

  Hmm. So we're going to have a chance to talk about your role at OpenAI as this conversation continues, especially as we get into ways of either, whether we want to think of it as regulation or oversight of this industry. I was going to ask you what you tell your mom when she asks you what you do, but I recently learned that your mom is actually a lurker on Reddit and uses the latest AI terminology. So I don't know if she's the right person to use as the kind of baseline normie, but if we were looking for the baseline normie mom equivalent, how would you explain to her what you do?

• 6:40Speaker 01

  That's a very good question, and I actually think that my mom is a good a good template for this. I've often said that I write my Substack with my mother as the sort of audience that I have in mind, which is to say like, she's a curious and smart person, but she is not technical at all. She knows nothing about AI. She's definitely like as normie as it gets in that regard. And I, you know, I want to write stuff that's palpable for her. I think what I would basically say is that I'm fundamentally, I would characterize myself as a writer and a communicator who tries to communicate ideas about about artificial intelligence, emerging technologies, and how our society

• 7:23Speaker 01

  can internalize those things in such a way that we we benefit from the technology and transform our world in a healthy way. I think that's basically what I would say.

• 7:38Demetri Kofinas?

  So there are a number of websites affiliated with you if someone Googles you. You write primarily at hyperdimensional.co, though your writings also get published on third-party platforms and outfits. I would just recommend people go to deanball.com to really link out from there and see all the stuff that you've written. But yes, I agree, your writing is fantastic. Not only is it intellectually stimulating, but it's also well done. And thank God, not AI slop informed. So what would you say, or what do you feel like qualifies you, or has given you the intellectual tools that others may lack in order to be a constructive voice in this debate?

• 8:18Speaker 01

  Well, I don't think I don't have an IQ that breaks the bank. I'm not some brilliant, you know, analytic mind or anything like that. There's a lot of technical people in my field who are more technically Gifted than I am, to be sure. I think the gift that I have is I have always been extremely curious and I've always been inclined to find connections between things that other people don't see. And also to describe differences between things than a lot of people would call similar or call the same.

• 8:59Speaker 01

  And like, I've just like, always like, had this inclination to sort of look at the world with extremely curious eyes and sort of like, find everything interesting. That's kind of why the Substack is called hyperdimensional in a way. And so, like, I think what gives me an ability to do is I can often write, like, I can bring, like, metaphors and analogies and examples to mind that are surprising, but that are also illuminating in ways that I think other people aren't able to do as much. And that's basically like, the one thing that I would say, like, actually care. I don't even think I'm that good of a writer, you know? I think I'm like, a perfectly workman-like writer. But yeah, I think that's like, the one thing I can do.

• 9:40Demetri Kofinas?

  Well, you're a bit of a romantic.

• 9:41Speaker 01

  Yes, I am very much a romantic. Yes, that's right.

• 9:44Demetri Kofinas?

  Yeah, and that comes across, which is good. It's actually helpful when we're talking about the end of humanity and what we're thinking about. Not, you don't write about that stuff, but I just mean that when people talk about AI, that's a big part of the discourse, a lot of existential anxiety, anxiety about jobs, our place in the world, et cetera. So it's helpful to have a humanitarian writing about those things. How would you describe the problem set that defines the nature of your work?

• 10:14Speaker 01

  That's also a very good question, the problem set. So I think my work is, I think what I would say is that there is this, there are, there are two big categories. One is like, okay, there are things we have to grapple with, with this technology, that there's uncertainty associated with them, but also they're like, relatively near term. And we can, you know, we can sketch out the shape of these things, right? And I would think about like, cyber offenses. Risk from AI models or bio risk or things like this. These are like problems that can be made sort of shaped into tractable and analyzable problems. But then there's this other category, which is like, I try to remind myself almost every day that like,

• 10:59Speaker 01

  this is seventeen fifty, right? We're in the equivalent of seventeen fifty or seventeen seventy five. And we have an entire industrial revolution to get through, right? And so, like, from the vantage point of seventeen seventy five, it's just like, you know, trying to occupy that mindset and then imagine like, you know, how are we going to govern the world of nineteen twenty five? It's like, well, the world is so, it's so, the transformations of the human experience were so dramatic that it's like the person of seventeen seventy five could scarcely comprehend even what the problems would be, much less the solutions, right? And so, I mean, imagine trying to explain what AI psychosis is to someone from eighteen fifty or something, right?

• 11:45Speaker 01

  And so, I feel like a sense of awe about that because I am conscious that I am living through, that we are all living through a transformation, you know, of that magnitude. And I try to like convey that sense of awe in various ways. And it's awe and it's also humility. It's also like, we really don't know. And I think in some ways, like, I and many people in the AI, AI is filled with a lot of people who have reflected very seriously about the industrial revolution and about technological change more broadly. And who are therefore very open to like, yeah, man, like, stuff might get quite wild, you know?

• 12:27Speaker 01

  Like, things might be quite crazy because the world of today is so wild compared to the world of a hundred and fifty years ago or something. And so, there's like this kind of openness to like dramatic historical and technological change that I think actually is quite off-putting to many people and like creeps some people out, if I'm being honest. And I occasionally find myself, you know, saying things and then getting clipped on the internet for saying like some crazy seeming thing. And it's like, yeah, well, I was just riffing about the, you know, like how crazy historical change can sometimes be. So, yeah.

• 13:02Demetri Kofinas?

  Well, I think it's a very helpful framing that you just put out there. Would you say that you think that that's true? Or do you know it to be true? In other words, do you think that we are in the mid, in the equivalent of the mid 19th century? Or do you know that in sort of, again, you may not know it factually, but subjectively, is that how deep it feels to you, deeply true in terms of your confidence level?

• 13:24Speaker 01

  Yes, my confidence level in that is extremely high. One thing I would want to be careful to distinguish is that I don't actually think, you know, when the historians gaze back at this time period, I don't think they're going to say, oh, it was like LLMs and AI. That was like the big thing that happened in this time period. I think instead they'll say something that's more like, there's an industrial revolution that began with the transistor. Maybe even some will go back to information theory itself. You know, they'll go back to like

• 14:01Demetri Kofinas?

  Claude Shannon turning and Claude Shannon and, you know, stuff like that.

• 14:05Speaker 01

  Yeah. But they'll say that a scientific and industrial revolution began with the transistor and that that led to consumer computing. It led to scientific, bringing computation, you know, to bear on scientific problems. And that ultimately, you know, the end state of building computers is you build a computer that can use the computer, obviously. And that's what AGI is. And they'll kind of see that that's the fundamental nut of what happened, of what is going on right now. Sort of the words like, we're 75 years in to an industrial revolution, in my view, 100 years into one. And a lot of the societal transformations that we see and like this sort of instability,

• 14:50Speaker 01

  the sort of thinning out, the hollowing out of institutional legitimacy, in particular in Western societies. I think a lot of that can be characterized in some ways as downstream, effectively of computation and of the internet, which is just computation. And so, yeah, like, that's sort of how I think about it. And viewed in that light, yes, I would put my confidence, I would I feel that in my bones.

• 15:13Demetri Kofinas?

  So since you brought it up, is your view that AI is part of the larger computers and networking revolution, or is it something totally new? And does it matter?

• 15:22Speaker 01

  I think it's, no, I think it's purely consistent. It is like, you know, it's going to be a major change in a lot of extremely important ways, but I think it's entirely consistent with the sort of computing revolution. I mean, in fact, like Turing and Shannon and sort of the initial sort of forefathers of the computing revolution, those guys all thought about AI. And they all thought about AI as kind of like the, again, like the natural end state. It's like, yeah, well, look, we're building a thing that can do computations. And eventually like what we'll want to do is we'll want to turn those computations into something that resembles or exceeds human cognitive patterns. And then, you know, that's kind of the end state, right? That's sort of the end state.

• 16:05Demetri Kofinas?

  Nober Wiener talked about the connectivity of the machines being a kind of networked brain.

• 16:11Speaker 01

  Completely, right. I mean, and like cybernetics is, you know, where does that word come from? It comes from the Greek kubernetes. And that means helmsman. It's also the name of a machine learning application. But yeah, I think basically like, again, I think there will be discontinuous jumps in many ways, but I think it's very much consistent. And the other thing I would say is that when people look back on this era, it won't just be computation. It will also be like, you know, biotechnology will be a major area that people reflect on. I think probably space will be one, maybe like robotics is a slightly separate thing, physical autonomy.

• 16:53Speaker 01

  And the thing is that like AI is like this glue. It both accelerates and stitches together all of these other areas of technology that are advancing too. And so much like the industrial revolution, you know, what you'll see looking back on it is like, oh yes, there were these very, very powerful technologies that all got developed around the same time. And the development of one accelerated the development of the other for various reasons. And so, yeah, that's kind of how I think it all.

• 17:27Demetri Kofinas?

  So I have a number of questions related to that comparison, but. Before I even ask those, just one more background question. Is it fair to describe you as someone who's trying to build the intellectual infrastructure for thinking about how to adapt our socioeconomic and political systems for the age of AI, for lack of a better phrase, while simultaneously maintaining the spirit of the Constitution and the Bill of Rights? In other words, is it important, when thinking about what you're trying to accomplish, to consider not just the technology, but also the people and the society in which it's being introduced?

• 18:05Speaker 01

  Yes, I would say I think that is a very fair characterization of my work. And the only thing I would add to it is that there is also some reluctance and skepticism of my own enterprise that is there, too. Because the thing is, is like, you know, I'm a dispositional conservative in a political theory sense.

• 18:25Demetri Kofinas?

  What does that mean?

• 18:27Speaker 01

  What it means is that, like, I believe in the organic and the local and the particular, right? I think that things have to be grown over time. So like, I would be very skeptical. I think of a lot of intellectual endeavors as someone trying to take an oak tree out of the soil on one continent and put it in a box and put that box in an airplane and ship it to another continent and then try to put that oak tree in a completely different soil. And like, I think generally, sometimes that works, but usually the oak tree doesn't take, you know? It won't like the new soil and the roots, it will not take root. I think you have to grow a tree over time in the soil that's native to it. And I think that that's like, kind of how like everything works.

• 19:13Demetri Kofinas?

  And what is the tree in this analogy?

• 19:15Speaker 01

  I mean, truly, like, if you're trying, let's just say, like, an institution, right? You want to build like a new type of organization or a country wants to create a deliberative legislature, right? There are a lot of examples of people just like importing legislative models from other countries.

• 19:30Demetri Kofinas?

  Like bringing democracy to Iraq.

• 19:33Speaker 01

  Yeah, yeah, democracy to Iraq is a good example. And you know, it's Rousseau who said that democracy is not a tree that can grow in any soil. I think this metaphor is ultimately coming from is Rousseau. So what I mean to say, though, is that I have this skepticism of like, like when I say things like, we need to define what the institutional architecture of free society will be in a world that is transformed by AI. I think that's true. I think that's true that we have to do that. But I'm also extremely skeptical of the notion that you can do something like that rationally from the top down.

• 20:15Speaker 01

  I think that like the new thing that we do, the way the world will look in 100 years or whatever, like that itself has to be grown. And I can't just dream that up and imagine it like, you know, from scratch out of whole cloth. And so, yeah, like whenever I say things like that, I say things like that as a shorthand to describe what I think will be a monumental process of historical change. But the thing is, is like, there's a part of me that's like yelling at me when I say that, like, there's a part of my brain that's like, but be careful, don't be arrogant, don't fall into the conceit that you with your tiny brain and limited perspectives can like do this yourself, you know, because we can't, no individual can.

• 20:57Demetri Kofinas?

  Well, I feel like that kind of spirit is captured in the new sages unrivaled piece that you wrote and elsewhere, where you talk about having your comforted by the fact that America has always winged it. I mean, and in fact, your regulatory proposals, which I'm very excited to talk about. And again, I don't, I feel somewhat uncomfortable even using that word because it comes with a lot of framing that may not be necessarily helpful. But even that approach of yours is much more decentralized. Again, not a great word to use, but there is a kind of hands off approach to how you think about this, that I think is really actually refreshing. And we're going to have a chance to talk about that. So let's, I think we did a good enough job of giving people a sense of who you are and where you're coming from. Let's just go really high level now.

• 21:42Demetri Kofinas?

  What is your definition of AI?

• 21:45Speaker 01

  Oh, good question. So I think basically AI has this history as an academic discipline where we were trying to make machines that could mimic human thinking, basically, thinking machines. And then there's been a lot of twists and turns through that. And so there's a lot of academics who have like really particularized definitions of AI that are sort of like a legacy of this archaeology. But for me, basically, what is AI? It's machines that can think in the flexible ways that human beings can. And it is the set of approaches to statistics, essentially,

• 22:26Speaker 01

  that have been invented over time to try to make machines think in the flexible way that humans and animals can. And that those approaches, to be clear, are broader than just like, you know, there's obviously like large language models, which are the closest thing we have to like, yeah, that seems like it's kind of thinking in ways that are similar to humans. But there are also applications of the exact same statistical approaches to completely different sets of data that are like totally alien to the human brain. So there are models, you know, that can predict the next nucleotide in a DNA sequence in the same exact way

• 23:09Speaker 01

  that there are models that can predict the next word in a paragraph of text. And like the next word in the paragraph of text thing feels somewhat analogous to human cognition. But obviously, no human can think natively in nucleotides. But the same approaches work. So it is both the particular thing of we want to make machines that think like humans and that can perform human cognitive labor. And we want to take those same statistical approaches that we use to invent the machines that can think like humans and apply them to different modalities that have nothing to do with human cognition.

• 23:45Demetri Kofinas?

  So how do you define human cognition? Or maybe to broaden it out even more than that? How do we define intelligence?

• 23:54Speaker 01

  Yes. So what is intelligence? I think that's the ground truth, really. Intelligence, in my view, is the ability to find

• 24:05Speaker 01

  Patterns from the observation of data. And the reason that that's true, when you think about what that means, if I'm observing some, just the world, right? I'm looking at the world and what I eventually, I find a pattern. I infer a pattern in the world. When I, when I drop the ball, it falls to the ground every time. And I can predict, using that pattern, I can predict what will happen in the future. Right? Because that pattern has been very reliable. I can also name that pattern. So like that pattern is a concept and it's a compression. It's like what I'm doing is I'm finding a faster way to go to understand what's happening around me.

• 24:49Speaker 01

  Because it's like, oh, I have a name for that thing. That thing is not totally alien to me. That thing is part of a pattern that I've observed before. And therefore, all intelligence is a form of compression. Language models are a compression of the internet. Well, they're a compression of their training data. Literally, that's what they are. My brain is forming a compression of the real world. A very, very lossy compression is worth noting, right? Like, I mean, it's fantastic that, you know, I perceive the world in 3D space and it's quite impressive rendering of reality that my brain is able to deliver. At the same time, like, we are aware scientifically that my brain perceives an extremely narrow range of light. Right? There's like this whole universe out there of things that are happening

• 25:34Speaker 01

  that we have no sensory input to. And so, you know, my brain is forming a really contingent and ultimately quite lossy compression of reality. But intelligence is that act of compression. That's what it is. And so, intelligence, yes, put it simply, it is the inference of patterns from the observation of reality.

• 25:55Demetri Kofinas?

  So I think that comparison of lossy to lossless compression is actually useful because the way that we compress information as human beings is much more lossy than how large language models do. But we are better at understanding what's important. So in other words, we're better at inferring meaning, I would say, from the patterns, which is essential. It isn't just, just to be clear, you're not saying it's just the observation of patterns or repeatable sequences. It's also using those patterns to infer something deeper about the structure of the data.

• 26:32Demetri Kofinas?

  Or the structure of the world that allows you to make viable predictions.

• 26:36Speaker 01

  Yeah, it's naming the patterns in such a way, and that would be like concepts. And then it's also using those patterns to make testable predictions about the future.

• 26:45Demetri Kofinas?

  And deriving meaning, how important is that? Because it's also unclear exactly. That's the thing that's also confusing is certainly a word we could use to describe, but still unclear. And it's not clear whether it will ever be clear, which is how are these systems actually modeling the world? You know, to what degree are they modeling and deriving something that we would identify as understanding? Versus simply engaging in a highly superficial sort of idiot savant level intelligence? Because of just how much data they can process and have access to.

• 27:19Speaker 01

  And also, what's the difference between those two things, right? I mean, there's a certain extent to which like there's the, you know, the old meme of large language models being stochastic parrots, right? Just machines that, you know, mindlessly just predict patterns.

• 27:33Demetri Kofinas?

  Right, or that other example of where like you tell an AI, I need to wash my car. This is an actual example, I think a user posted on Reddit. You tell it, I need to wash my car and the car wash is 100 meters away. Should I walk or should I drive? And the LLM responds with, it's a short distance, so you should just walk. What do answers like that reveal to us about the way in which these systems model the world that they can at once be both incredibly intelligent while also occasionally forming opinions or providing answers that just seem utterly idiotic? You know, granted, we have seen a major, and it's important to note, we've seen a major reduction in incidences like these with each subsequent model improvement.

• 28:18Speaker 01

  Yes, so I mean, I think that a big chunk of that is I think you could make the same criticism of humans. You know, common sense itself is dependent upon the kind of reasoning engine that the observer possesses. And so like that which is common sense to a dog. Dogs probably have some sort of a comment like they have, they have their own intuitions about the world. And those intuitions are like, probably like quite right for them, but we wouldn't think a dog has common sense. And like, there are all sorts of things like humans fall for, there are so many tricks we fall for. A, we can be rhetorically swayed in ways that large language models are harder to, you know.

• 29:02Speaker 01

  Like, we fall for all sorts of fallacies that a large language model would catch immediately, right? Large language models would be like, that's an ad hominem. Like, you're begging the question, right? And ways that like-

• 29:13Demetri Kofinas?

  Right, because they're so good at identifying patterns.

• 29:16Speaker 01

  Yeah. And also, well, they identify different kinds of patterns than the ones we identify. So like, they learn ultimately in a very, very, very different way from the way that humans learn.

• 29:25Demetri Kofinas?

  Well, can I ask you something like, as an example, Dean? Like, so if I read a book, I independently and in ways that I don't really understand, identify what I think is important in the book. If a large language model reads a book, it seems that what it's doing is it's identified some pattern statistically in language that tells it what areas the author thinks are important, or are important based on some kind of statistical correlation. I mean, it's kind of, I spend enough time using these models in sort of thinking through podcast episodes. And again, I'm in no position at this moment to sort of arrive at a conclusion about what's

• 30:08Demetri Kofinas?

  going on, but I can feel something meaningfully different about a how I think versus how a large language model thinks. And-

• 30:17Speaker 01

  Yes.

• 30:17Demetri Kofinas?

  I'd love for you to speak to that.

• 30:19Speaker 01

  Yeah, yeah. No, I think you're exactly right. You can read a book, and you might not remember, you know, word for word, a single sentence in that book, right? There are many books that I love and that inspired me very deeply that I don't think I could quote, I don't think I could literally quote any of them from memory right now.

• 30:37Demetri Kofinas?

  And you'd probably hallucinate if you tried.

• 30:38Speaker 01

  I'd probably hallucinate if I tried. And yet, I drew things from those books that in like, high level, high order ideas that inspire me and structure my thinking every single day. And, you know, obviously, the language model, the language model probably does pick up on those ideas too. But it's also doing something very different. I think it's worth going to the foundational insight of, hey, why don't we train a neural network to predict large sequences of web texts? And it is 2017. There was a researcher named Alex Radford at OpenAI. And you know, most of the AI companies at that time were trying to do, you know, they were teaching computers to play video games.

• 31:22Speaker 01

  They were doing robotics, board games, things like this. And Radford had this idea of like, there's a data set of 80 million Amazon product reviews. And let me just train a model to predict the next character of those things, of those reviews. And in doing that, what they found is that they had accidentally, accidentally invented a state of the art sentiment analysis system. Sentiment analysis is like the ability of a machine to tell the underlying emotional sentiment of a chunk of text. And it turned out that as an instrumentality of predicting the next, of its fundamental goal, which was to predict the next letter in this database of product reviews,

• 32:05Speaker 01

  the model learned on its own to analyze the sentiment. Because understanding the sentiment of the text is useful for predicting the next token of that text. So there is like some higher level like understanding going on there. But I think it's also extremely different. And I think that large language models are subject to all kinds of, you know, again, like we fall for optical illusions. Humans, there are all sorts of optical illusions that humans fall for. And there's all these weird tics of our cognitions that are ultimately downstream of contingent realities about the way our brains evolved. And I think the exact same thing is true of LLMs.

• 32:50Speaker 01

  But I hesitate to say that that means that they're not really understanding. I think the more complicated reality is that sometimes they're really understanding and sometimes they're not. Sometimes they really are just, you know, sort of mindlessly predicting. And other times they're doing genuine understanding. And I think we've gotten much more genuine understanding over time. There are some people who would say, You know, an LLM can never genuinely understand. And my view of that is that that is ultimately like a metaphysical claim about the nature of human cognition that you're more than welcome to believe. But like, I guess my view is like, at this point, OpenAI has a model that is unreleased,

• 33:29Speaker 01

  but maybe six weeks ago, made like a really important, like actually important discovery in geometry through connecting different ideas together. And like, it kind of seems to me like that if you have an entity that is making like legitimate breakthroughs in mathematics on actually important problems that humans have paid attention to, that have befuddled them for decades, and a large language model has like arrived at a new insight about that problem, seems to me that like, if your definition of thinking doesn't include the entity that can do that, I feel like it's your definition of thinking that is the thing that is flawed.

• 34:08Demetri Kofinas?

  Do you think that reflects the fact that math is a formal system, whereas the world that we inhabit is unbounded?

• 34:14Speaker 01

  Well, I mean, I think math is structured in various ways, but I think math is like more unbounded than I think we often give it credit for.

• 34:23Demetri Kofinas?

  I mean, or maybe it's better to say that it's more amenable to this kind of intelligence, perhaps, than our form of intelligence, putting aside whether or not we should be humble about what we can know or not know, which I completely agree with you.

• 34:36Speaker 01

  Well, yeah, I mean, think, you know, I'm looking out at my window right now at a bird that just landed on a branch, and think about like, the way that a bird moves about the world in flight versus the way that like a Boeing 737 does. You know, the Boeing 737 cannot like land on a branch. It is incredibly energy inefficient compared to the bird. The Boeing is very useful for lots of things, but in many ways, you know, we have not figured out. Cognitively, I think there is an elegance to the way that animal cognition works that we have only dimly captured in neural networks.

• 35:21Speaker 01

  But the thing is, is like, that should make you quite bullish about AI. Because like, the AI is already objectively pretty useful right now. And if there's this huge difference, and the difference, by the way, it is, I think it is sample efficiency. I think that is the difference, right? It is the fact that I can read a book once and derive the important higher order insights without memorizing the entire thing. And in that sense, I am more sample efficient, which is also another way of saying, I have formed a better compression of the book. I have found the important patterns in that book, named them, and made them useful to me. I am more intelligent than the LLM is.

• 36:06Speaker 01

  And I think probably humans are orders of magnitude more sample efficient than the LLM. But the LLM, by virtue of being a computer, is like, still quite useful. And it would imply that there are like many orders of magnitude we can go to make the LLM like closer to human level sample efficiency.

• 36:23Demetri Kofinas?

  So, there's a quote that I'm looking to see if I can find as we're talking, but I'm actually going to give up and just try to wing it from memory. And I'm paraphrasing it, but Claude Shannon was talking about information theory. And he said something along the lines of, sometimes these messages, these messages that are compressed, have meaning. Or sometimes the patterns that we find in the data have meaning. Then in other words, the pattern doesn't require meaning. And so when you're talking about sample efficiency, and yes, human beings need a smaller sample size to derive meaning from a corpus of data. I wonder to what degree, Dean, and I don't want to take us too far afield here, because this is more of a philosophical discussion that you and I need to have at 3 a.m. in

• 37:08Demetri Kofinas?

  our dorm rooms. But I wonder to what degree... Something very different is going on here for large language models. And that in order for any of this to make sense, they need human beings as the sort of person in the mirror. That the meaning is something that we derive. It's not something that they are able to derive from that information.

• 37:32Speaker 01

  Yeah.

• 37:33Demetri Kofinas?

  Feel free to respond to that. And then I have a series of, I think, more answerable questions.

• 37:40Speaker 01

  Yeah, no, I mean, I guess what I would say is this. I would say there are certain commonalities in the types of inferences that LLMs need to make and that humans need to make. Ultimately, like, the thing about the world is that the world is sarcastic. It is incredibly complex, but it's not fully random. Right? There's a lot of structure in the world. And there are things about, you know, humans had to identify, like, we find structure in the world and we make predictions and infer patterns based upon that structure. And there are common structures that the LLMs are finding and that the humans are finding. But the world is also very large and rich. And there are all kinds of structures that humans find

• 38:24Speaker 01

  that LLMs don't find, and all kinds of structures that LLMs find that humans don't find. And I think, by the way, we've barely scratched the surface of what the LLMs understand about the world that we don't understand. Because I think there actually probably is quite a bit that they understand that we don't understand. But I guess what I would say is like, I think what you're getting at is this notion that like, agency and goals and like, meaning, purpose, are things that are much closer to being unique to human cognition. Ultimately, we don't know the answer. Ultimately, like, I think one thing that really scares people in the AI safety world is like, you know, we call AI systems of today, we call them agents. But are they agents in the legal sense?

• 39:08Speaker 01

  Which is to say like, you know, my lawyer is my agent and that means he does exactly, he's like, I am delegating some aspects of my life. To that person, but that person is expected to have an absolute duty to me and is to be like a complete subordinate essentially to me? Or are they a philosophical agent? Which is to say, something that can act rather than behave, act on its own, have goals of its own that it autonomously acts upon. And I think that is like a scary thing to think of like, you know, super intelligent machines with goals like that. I think that is like a really big unanswered question. But I think there is some reason to believe that like the human agency and finding of purpose, it's not purely an artifact of like our intellect or our reason.

• 39:53Speaker 01

  It is coming from some deeper aspect of our biology and our survival instinct and things like that, that like language models are not trained to have.

• 40:03Demetri Kofinas?

  So the quote I was trying to remember was actually in the opening of A Mathematical Theory of Communication, which was Claude Shannon's opus on information theory. And the quote was, "The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently, the messages have meaning." So I just wanted to put that out there for anyone that's interested in, obviously, reading Claude Shannon's work is, I think, an important complement to this conversation. In your article 2023, or Why I Am Not a Doomer, you wrote that in that year, you went from being a skeptic to a believer. And you would walk around, was it the campus? Where were you at this time?

• 40:46Speaker 01

  I lived in DC, but I was spending a lot of time at Stanford University.

• 40:49Demetri Kofinas?

  Right, okay. So you would ask yourself questions like, what is the nature of the challenge posed by the alignment problem? Which is kind of something that you were sort of alluding to earlier? How we should think about risks of misalignment? Is AI something fundamentally new? Or is it consistent with the pattern of prior emerging technologies? Which again, is something we briefly touched on earlier. Does AI break the existing constitutional order of the United States? Or does it merely challenge it? Which I think is a very important question that we'll have a chance to dive into when we talk about regulation. I'm just curious to understand, where would you say that you fall on this continuum of doomer versus accelerationist boomer? And by boomer, I don't mean baby boomer. I mean like boom, you know, this is the AI boom.

• 41:35Speaker 01

  I think fundamentally, I am much more of a techno-optimist and a believer in the notion that like, I have faith in human ingenuity and the human ability to like, adapt to changing circumstances. At the same time, like, sometimes I think that the traditional doom world, the sort of, you know, as typified by a figure like Eliyzer Yudkowsky or Max Tegmark, I think that they have a quite simplistic understanding of intelligence and like, the way the world works broadly conceived, that causes them to have this like, very simplistic notion of what could bring about human extinction or doom, as it were.

• 42:17Speaker 01

  And I guess like, my view is like, again, put yourself in the perspective of 1775 and then try to imagine like, World War I happening, right? And like, imagine explaining to that person like, what, a person who only had the knowledge of 1775, the specific warfighting dynamics of World War I. And it's like, man, that would have been so terrifying and unpredictable. And there are, there are things-

• 42:41Demetri Kofinas?

  Well, it kinda was. I mean, it was catastrophic for-

• 42:44Speaker 01

  Oh, it was horrible. It's one of the worst things that's ever happened, right?

• 42:49Demetri Kofinas?

  And in fact, people's ignorance around exactly how industrialization had changed the nature of warfare is in part why World War I turned out to be as catastrophic as it was.

• 42:59Speaker 01

  Yes. And I think that the nature of technological change, when you just think about it through human history, we have constructed a reality, a civilization, that is like, completely alien to the vast majority of human experience in the history of our species. And it's terrifying in some ways, right? In some ways, there is a certain like, terror and awe in the, it's awesome in the like, 18th century use of that word, in awe inspiring, right? Not like, cool. It is cool, it is cool, to be clear, but it is also like, there is a certain terror to it. And so I guess, when I think about like, technological change in macro-historical terms, there's a part of me where like, sometimes I feel like, I'm actually more-

• 43:44Speaker 01

  More of a doomer than the doomers because I just realize like how alien what we're going to build will be. And like what sort of like unimaginable horrors likely lurk around the corner. But I also think unbelievable, you know, joy, like surprising joy to the upside delight. Is also just as likely. And in the end, I think that there will be more good than bad. But yeah, I do think that like there's tremendous humility that you need to experience if you are doing the enterprise of commenting on AI.

• 44:15Demetri Kofinas?

  Yeah, something I really actually appreciate about your work, Dean, is that you do try to be even handed. Not for the sake of being even handed, but because you're being intellectually rigorous. And I think that's what. That's what happens when you try to do that. It's very, I mean, in the early days, when if you were reading like, you know, Bostrum's work or some of the stuff that was coming out of the existential risk. Community, it was very easy to adopt the if anyone builds it, everyone dies view that Yudkowsky puts forward. But I think as these models get built out and they get better and better. You begin to, I think, at least if you're being intellectually honest. Understand that the picture is much more complicated and AI alignment's a great example.

• 44:58Demetri Kofinas?

  I mean, the way that I used to think about the alignment problem, it was like, how do we? Bridge the explanatory, first of all, how do we even? Sort of come to a clear idea internally about what the values are that we want to impart on these systems. And then two, how do we bridge the explanatory or epistemological gap and convey them in a manner that they will understand? And then so that we, you know, don't all end up dead because. AI decides that we'd be happier if they lobotomized humans or put. You know, sensory neurons in our brains and jacked us up with dopamine or put us in a vat or something. But actually, what you realize is that I feel at least that the alignment problem is less of an issue.

• 45:40Demetri Kofinas?

  Because so much of what these models are trained to do is to. Understand, I think in some ways better than us, what it is that we want. Now, I might regret saying that, and I should also say that it doesn't make me less of a doomer. I'm not a doomer, but it doesn't sort of, that's less sort of where my doomishness comes from. But I just, it's just to say that it's way more complicated. I think it's important to have a nuanced view of this. So in the spirit of that, what would you say are some of the things that both the doomers and the boomers, again, this is my term, but it's not the baby boomers. I can see how that can be confusing, get wrong, in your view?

• 46:20Speaker 01

  Well, I think the fundamental mistake, I think, that in the doom conception of the world is a kind of view of intelligence being completely correlated, maybe even identical to power, like ability to make stuff happen in the world. I don't really think that's what intelligence is. And I think that for that reason, though, they tend to think that like, that all problems, like, a more intelligent entity will be able to solve every single problem better than humans, and also have like incredible amounts of

• 47:01Speaker 01

  agency and ability to like affect change in the world. And I think that's like not quite accurate. It's in some ways, it's accurate, but I think it's a, they're painting with too broad a brush. And there's, there's a lot of hand waveness in that view. What I would say about the, I mean, when it comes to the boomer side of things, the sort of techno optimist side, there's a lot of different worldviews that are captured there. So like, for example, there's one category of boomer who really, their optimism about AI and their opposition to government involvement in AI basically comes from a place of actually,

• 47:45Speaker 01

  ironically enough, kind of pessimism about the technology. They have kind of a deflationary view of the technology. So there were a lot of people, you know, there was a leaked draft of an executive order that the Trump administration, that the White House put out. It leaked from the White House, but it was real. It leaked in like November, December of last year, that Reference the notion of catastrophic risks like autonomous cyber attacks or bio and stuff like this, and it dismisses those things as purely speculative and hypothetical. And it's like, looking at the state of the technology as of late 2025, if you were willing

• 48:26Speaker 01

  to even put, like, write down the words that those kinds of risks are totally speculative and hypothetical, you had a profoundly deflationary view of the technology. And one thing I've always suspected about the boomers, and we actually see this a little bit starting to emerge now, one of my theories of mind of that community, which, again, like, I've largely been allied with for most of my career writing about this, but one of my theories of mind about them has been like, I bet you they're actually, like, once they fully realize what is actually going on with this technology, they will actually be quite terrified of it and that they might actually be closeted AI pause ban people, you know,

• 49:11Speaker 01

  like, that the notion of superintelligence actually being real, like, very few people have internalized that notion in their bones and then, or at least made an effort to, and then also have been like, and we should build it. I feel quite lonely in that camp. There's a few of us, but there's not many of us. And that a lot of the people who are boomers actually just really, their optimism is rooted in fundamentally a quite deflationary view of the technology that means that like, oh, well, these problems that the AI safety world points out, they're just not problems that we're going to have to deal with, really. They're just dismissing the problems. They're kind of kicking the can down the road. I would say like, that's the fundamental mistake of most boomers, but it is worth

• 49:52Speaker 01

  noting that there is a spectrum there, and not all of them have that deflationary view.

• 49:57Demetri Kofinas?

  So Dean, I'm going to move us to the second hour. Let's start that part of the conversation, fleshing out what superintelligence is in your view, what it looks like, how quickly we get there. And then let's talk about so much of the other stuff that you've written about that I think is really interesting and why you were hired by OpenAI, which is your approach to regulation. Again, not a really good word, maybe your approach to how to oversee and manage this transition into the era of AI so that we can Maintain and protect the things that we value, whether those things be our humanity, our constitution, our Bill of Rights, whatever, and so adjust our socioeconomic and cultural systems and socioeconomic

• 50:37Demetri Kofinas?

  systems to this new reality. And I'm also curious how that fits into the larger geostrategic competition, which between the United States and China, where this has also shown up in the ongoing dispute between anthropic and the Department of War, what this means for other countries, their economies, societies and cultures, whether some are better adapted or more maladapted than others. And also, what does this do to the labor market and the economy? This is something that people are very concerned about. And I would love to talk about all of these things with you in the second hour. For anyone new to the program, Hidden Forces is listener supported. We don't accept advertisers or commercial

• 51:18Demetri Kofinas?

  sponsors. The entire show is funded from top to bottom by listeners like you. If you want access to the second hour of today's conversation with Dean, head over to hiddenforces.io/subscribe and sign up to one of our three content tiers. All subscribers gain access to our premium feed, which you can use to listen to the rest of today's conversation on your mobile device, using your favorite podcast app, just like you're listening to this episode right now. Dean, stick around. We're going to move the second hour of our conversation onto the premium feed.

• 51:49Demetri Kofinas?

  If you want to listen in on the rest of today's conversation, head over to hiddenforces.io/subscribe and join our premium feed. If you want to join in on the conversation and become a member of the Hidden Forces Genius Community, you can also do that through our subscriber page. Today's episode was produced by me and edited by Stylianos Nicolaou. For more episodes, you can check out our website at hiddenforces.io. You can follow me on Twitter at Kofinas, and you can email me at [email protected]. As always, thanks for listening. We'll see you next time.