When AI Decides You're a Threat — Brad Carson

Brad Carson was the Army's General Counsel, served two terms in Congress and was Acting Under Secretary of Defense for Personnel and Readiness. He now heads Americans for Responsible Innovation, the AI-policy advocacy group he co-founded. Keith Duggar spends roughly eighty minutes pushing back. SPONSOR: --- Cyber Fund built the Monastery to help founders ship products that were impossible a year ago. Applications for Batch 1 are now open. Apply now: https://cyber.fund --- Carson's whole case rests on one line: the genie is not out of the bottle. We have pulled dangerous tech back before. Asilomar halted recombinant DNA in 1975, and the West still controls the chips AI runs on. Calling it unstoppable, he says, is the most dangerous idea in the room. Then Keith drags him somewhere darker. A Palantir heat map scores you 0.73 on whether you are a combatant, and a strike follows. The model is wrong some accepted share of the time, and when it is, nobody answers for it. You cannot court-martial a model, and not even the interpretability researchers can say why it picked you. — Note: after recording, we learned that Americans for Responsible Innovation is backed by EA-aligned philanthropy (not sponsored) --- TIMESTAMPS: 00:00:00 From the Pentagon to AI governance 00:04:52 Regulatory capture vs Silicon Valley networks 00:07:56 Transparency and the Claude tier changes 00:09:40 Tort liability when AI tools cause harm 00:13:40 AI is a product, not a person 00:16:01 Children, suicide, and the suicide business 00:19:59 Opaque neural nets and the law of war 00:25:54 Probabilistic targeting and the death of accountability 00:28:47 The arms race fallacy: Asilomar and restraint 00:34:02 Talking to China: track 2 talks and chip leverage 00:39:45 Air power never wins: capital for labour 00:43:29 Anthropic vs the Department of War 00:51:29 Concentration, open source, and brain drain 01:00:18 DeepSeek, Chinese culture, and AI as diplomacy 01:12:25 Upskilling Congress and why public trust matters --- REFERENCES: organization: [00:02:45] ICRC position on autonomous weapons https://www.icrc.org/en/law-and-policy/autonomous-weapons [00:05:22] Americans for Responsible Innovation (ARI) https://ari.us [00:07:20] Andreessen Horowitz (a16z) https://a16z.com/ [01:16:05] Office of Technology Assessment https://en.wikipedia.org/wiki/Office_of_Technology_Assessment other: [00:03:35] Beneficial AGI 2019 Conference (Future of Life Institute, Puerto Rico) https://futureoflife.org/event/beneficial-agi-2019/ [00:18:30] Section 230 of the Communications Decency Act https://en.wikipedia.org/wiki/Section_230 [00:19:59] Lethal Autonomous Weapons (LAWS) https://en.wikipedia.org/wiki/Lethal_autonomous_weapon [00:31:35] Strategic Arms Limitation Talks (SALT) https://en.wikipedia.org/wiki/Strategic_Arms_Limitation_Talks [00:32:28] Asilomar Conference on Recombinant DNA (1975) https://en.wikipedia.org/wiki/Asilomar_Conference_on_Recombinant_DNA [00:39:45] The New Iron Triangle (ARI policy byte) https://ari.us/policy-bytes/the-new-iron-triangle/ [00:48:05] Defense Production Act https://en.wikipedia.org/wiki/Defense_Production_Act person: [00:03:35] Anthony Aguirre https://en.wikipedia.org/wiki/Anthony_Aguirre [00:06:48] Dean Ball — Hyperdimensional https://www.hyperdimensional.co/ [00:23:13] Neel Nanda — mechanistic interpretability https://www.neelnanda.io/ [00:36:02] Jack Clark (Anthropic) on Conversations with Tyler https://conversationswithtyler.com/episodes/jack-clark/ [00:39:15] Robert Trager — Centre for the Governance of AI https://www.governance.ai/team/robert-trager [00:41:55] Giulio Douhet https://en.wikipedia.org/wiki/Giulio_Douhet [01:15:05] Don Beyer (US Congress) https://en.wikipedia.org/wiki/Don_Beyer tool: [00:22:19] Phalanx CIWS https://en.wikipedia.org/wiki/Phalanx_CIWS --- ReScript: https://app.rescript.info/public/share/9405ff35c0215b7cdae6402d41284171 https://app.rescript.info/api/public/sessions/0a6c081b8e5fe413/pdf

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 0:27Speaker 01

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com. Not authorized by any candidate or candidate's committee.

• 0:59Speaker 02

  So I remember I was first elected to Congress 20 years ago now. The Congressional Management Foundation gives you this book, kind of like, how to be a congressman, how to run your office, what to expect day to day. And I remember reading in the back, they had a survey of the existing Congress, and they said, how much time a day do you have to read to get smarter about issues? And the answer was 17 minutes. We control the most important part of AI, and that is the chips. We can stop other countries from developing super AI in their tracks. If you're in Gaza, Keith, you have a 0.73% that you're a Hamas terrorist. And, you know, what makes that? You know, what is 0.73? Like, do you get struck for that?

• 1:43Speaker 02

  Or are you off the list for that? Like, what's the threshold? Right. You know, people will often say this about AI, like, it's coming, you have to accept it. We regulate and change technologies all the time. And so I do think there is a world where we should not just accept the future as being determined, we shape it actively. The anthropomorphic tendency we have when we see the language, we love language, we think it's the unique human skill. And when this machine turns it out, we know through AI psychosis and other things that people think, it's a person! And therefore, they're giving the rights of persons to something. And that, to me, is a very dangerous thing. But it's a machine. And we should treat it like a machine. I mean, the fear of the Soviet Union was real and powerful. But nonetheless, we had people, both parties, wise men,

• 2:26Speaker 02

  who got together and said, like, this arms race thing is going to kill us all. How you win wars is with people. You know? That's the fundamental thing. And I think it is, I often said for my time in the military, watching this all go down. The American way of war, in many ways, is substituting capital for labor. We love bright, shiny objects. We think they're technical solutions to vexing human problems. And we're always betrayed by that. Because in the end, when you go to Iraq, you go to Afghanistan, you go to Iran, all the fancy kit, it can reduce your city to rubble. Right? But the only thing they can't do, and only humans can do this, is basically come, kick in your door, and occupy your place,

• 3:10Speaker 02

  and like, you know, re-instantiate a new government there that we like to see. That's a human endeavor.

• 3:15Tim Scarfe?

  This episode is supported by Cyber Fund. If you're building at the frontier of AI, they want to hear from you. Cyber Fund believes the future belongs to AI natives who want to achieve the impossible. And that is why they're introducing the monastery for AI native founders. It's an environment of pure focus and rapid execution for founders operating at AI native speed. And they're offering teams $2 million each to participate. Apply now at cyber.fund.

• 3:53Speaker 02

  We had a longstanding interest in artificial intelligence, just as someone interested in kind of public life, society, how one governs an ever-changing world. So I followed AI for probably 20 years casually reading about it. When I was in the Department of Defense, running the army, I had the chance to work on the law of war. We oversaw that for the entire Pentagon, for all of the military services. And right as I was leaving, the question of autonomous weapons... Became an important issue for the Pentagon. We were sending army lawyers over to Geneva to talk to the Red Cross about, you know, possible treaty or or address their many concerns about it. I was following it there as well. I often tell people, one of the great qualities I have, which isn't that important,

• 4:37Speaker 02

  but it's helped me a lot in life, is I always answer the phone. It took a lot of cold calls. And so I was a professor at the University of Virginia for a short while after Obama left office during really the first Trump administration. And one day the phone rings and it's someone who I'd never heard of before. And my phone in the office never rang. And I thought it could be an emergency or something. And so I picked it up and he says, I'm Anthony Aguirre. I'm a physicist at the University of California system. He says, we want to invite you to this amazing AI conference. This is going to happen over the New Year's in Puerto Rico. I said, well, I don't know that I can really add a lot to such a conference. It's a long ways away. He said, oh, we have a stellar panel of people coming. I said, yeah, like who? He says, you'll probably know someone.

• 5:22Speaker 02

  He said, well, Dario Amadei? I said, never heard of him. He said, Stuart Russell. I said, nope. Yashio Benjio? I said, I'd never heard of these people. He went about six deep into this till he finally said, well, Reid Hoffman and Elon Musk are supposed to come. I said, well, I know them just from kind of the business pages of the newspaper or from pop culture. He said, it'll be great. There's only 100 people coming. You should you should visit this event. I said, like, well, New Year's, it's warm in Puerto Rico. I said, why not?

• 5:52Speaker 04

  It's already a massive problem that other industries have successfully captured regulation, you know, to their advantage and to the detriment of of the rest of us, you know, the people who are not not the biggest players. In fact, one of my favorite sci fi shows, even though it only lived for for one year is Firefly. And then there has this excellent quote where Shepard Book says, you know, a government is just a body of people, usually notably ungoverned.

• 6:22Speaker 02

  It is a daunting problem, but I think there are examples in government that have been less captured by the industry, and those can prove some kind of paradigm for us as well. I think having Congress have strong oversight of it, having a lot of democratic accountability. And you know, one of the things we advocate for are ways that actually minimize the government while ensuring we have proper kind of public oversight. So for example, we've been strong advocates for mandatory testing and evaluation of frontier models. It doesn't have to be done by a bureaucracy in the Department of Commerce or Department of Energy. There's a lot of interesting ideas about independent verification organizations, you know, kind of the model we use in public company accounting, where that's all done by the private sector,

• 7:04Speaker 02

  but it's also overseen by the SEC to make sure that it's not fraudulent and you don't have an Enron, you know, Arthur Andersen situation. And I know, I don't think public company accounting is seen as a captured industry, but it provides a meaningful service to the capital markets. So I think there are models like that. And I do think the problem I have often, people who advocate for regulatory capture, Dean Ball and I will go back and forth on this question, is like it's searching for a pea under a hundred mattresses. It's never falsifiable. You say, well, it's regulatory capture. Well, you know, right now we don't have any regulations. And my view is that groups like A16Z or people in Silicon Valley have largely captured the process because they work very informally through networks of political influence.

• 7:47Speaker 02

  That it's really about what the alternatives are going to be. Having an agency subject to regulatory capture, yes, is at least more accountable than the kind of informal, very moneyed networks that are controlling AI policy as we sit here in 2026. We always have to look at politics about the alternative. Like imagine the SEC. So is the SEC captured? Well, I'm sure the industry has a lot of influence over the SEC, both informally and formally. Will we be a better world without the SEC because it's been captured? So I think if the choice is kind of nihilism versus an agency that is subject to regulatory capture, that you have to put, you know, prophylactics in to ensure that doesn't happen, it still strikes me that's a better world, even if it's not a perfect world.

• 8:31Speaker 02

  And people who usually argue for the regulatory capture side, like I see Andreessen tweet this out a lot. You know, they propose kind of a nihilistic regime. It does privilege like a very few people who have kind of the informal ability to shape government policy, largely because of their, you know, contributions or their public influence in some way. So I think having public agencies for all of their problems, it actually is a better system.

• 8:55Speaker 04

  Anthropic made a bunch of changes to kind of Claude's behavior in terms of tokens and allocation and what models were running and things. And it really frustrated a lot of the people in our Discord server, the machine learning streak dot Discord. They felt like, hey, we were paying for something and then it just was radically changed, you know, overnight. And yet there's no clarity around what were we paying for before and what are we paying for now? And it's almost just at the whims of the service companies, right? There's just no transparency to it.

• 9:27Speaker 02

  Yeah, I mean, you could think of that as consumer protection. That people should tell you what the service is going to be that you're paying for. And if they change it, right, this could be a breach of contract in some way. But I think now the AI companies, the frontier AI companies at least, you know, have a certain almost global stature. Everyone recognizes that this is a project that is going to go have, you know, epochal consequence for us. And so you have, you're not just like your average hardware store, you know, that could change things and no one cares. You have almost a public responsibility to get a lot of this right. And part of it is minimally to be trustworthy, you know, and that's what transparency is all about. To say, here's what we do, here's, you know, from our perspective, the data you were trained on, here's its capabilities.

• 10:11Speaker 02

  Here's what our internal policies are and we're going to adhere to them. If we're going to change them, we'll notify you about it, give some justification for it. I think, you know, but this incredible power does come some responsibility that's not codified in law. It's really almost a moral obligation, which to their credit, like many of the companies recognize this and do their best to try to, you know, to try to satisfy that itch. But you can see that people scrutinize them and more will be asked of them still.

• 10:39Speaker 04

  Okay, somebody uses a tool, in this case it's AI, to do something.

• 10:45Speaker 04

  Bad, right? You know, a deep pornographic fake of, you know, some poor victim or whatever. You know, do we blame the tools or do we blame the person misusing those tools? It's like you post some deep fake porn image on X or wherever, like what happens to you? Like, I don't know. I haven't seen those people being held accountable.

• 11:07Speaker 02

  I held them.

• 11:08Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 11:37Speaker 02

  Both accountable. You know, in the case, for example, of deep fake pornography, many times it's posted anonymously. You don't even know who did it. It's hard to trace. You know, the people who are victims of it, maybe young, they may come from families that don't have the resources to pursue litigation. And right, the remedy hardly meets the harm. Maybe a few years later, right, you get a judgment against some hapless kid living in a garage who was in your school who has no money. Your reputation has been permanently tarnished. The humiliation is unforgettable to you. And so it's very hard to get it kind of exposed to deal with it. And that is why we do believe that the labs, to the extent possible at all, should design those kind of features out of it.

• 12:18Speaker 02

  You know, in the same way that in every other kind of approach to torts in this country, right? There's a division of labor. If I sell you a product and you then go use that product in a, you know, harmful way to others. We have centuries of common law that allocate responsibility between us. If I knew that you were a dangerous person, I nonetheless sold you a gun. Or if I waived any of the rules, if I took no preventative action at all, and I knew that you were likely to go out and use it to harm somebody, I should, as a store owner, be held responsible in some way for you. Not fully, because you did something too. But I'm not absolved of liability if I could reasonably foresee that you are going to use this tool to harm other people. And so we have this allocation of responsibility across the spectrum.

• 13:02Speaker 02

  I think the second thing is, and this is a unique part of American law, and it's controversial in some spaces, but it's now well-established. But lots of times, liability is hard to place on a particular person. And we put the liability on the entity most capable of avoiding the risk and also bearing the punishment through insurance or something like this. And that's why we have product liability law, for example. You know, it's like the companies are capable of getting insurance. They cost us it to doing their business. They have the ability to make sure the product's not dangerous, even if someone uses it, misuses it down the line as well. So the way we kind of use the tort system as a form of social insurance suggests that in the case of AI,

• 13:45Speaker 02

  really the developers of AI should probably bear most of the burden, but not the full burden. Because if you use prompts and do something negative with it, or use visual AI systems to create porn, you too should be held criminally accountable for that. Or we know the models are trained on, especially things like stability diffusion and others, right, on lots of images of child pornography. You know, what we would say is that you should make sure that those are excised from your database, you know, from your training materials. And they have the tools to do that. Now, that doesn't solve the deep fake porn problem alone, because you're still going to see a lot of adult pornography, right, and they can transmogrify that into some kind of child pornography.

• 14:28Speaker 02

  But the idea that you have lots of child pornography in your training data and you're making no effort to screen for it, you know, it makes no sense to me. So this is where, yes, I'm for the government stepping in there and saying like, yeah, you should clean that up. And if you don't clean it up, you should be liable for a lot of the downstream effects of that.

• 14:46Speaker 04

  100% agree with you on that. So obviously, as you mentioned, there's, I don't know, definitely centuries or more of common law like that applies to this discussion. Like, how do you see? AI is different. What is it? Is there anything fundamentally different about AI technologies that, I don't know, throw a wrench into into the traditional toward toward analysis?

• 15:10Speaker 02

  I think the biggest thing that I'm grappling with now in the last few weeks, arguing with various people, is the anthropomorphizing of AI. So it seems, and the key word is, it seems to be a human. So for example, I use all the models a lot. I'm a super user of them. When it gives me an answer to say, okay, I want to learn about machine learning, street talk podcast, tell me all about it, you know, is that a speaker with First Amendment rights? How should we treat that? If it says something in error, if it defames me, if it does something worse, you know, what's the legal regime about that? Now, from my perspective, there's a clear answer to that, which is it's actually a product. It's not a human being. It's a machine. If it does something

• 15:51Speaker 02

  that's crazy and defames me or leads me to harm, but I treat it like I do, you know, a bottle of of spray paint or, you know, pesticide or something I might buy at Walmart, right? There's like a product liability regime that governs that. It's not a human being. And what it says to me is not covered by the First Amendment. It's a machine. But there are people increasingly, and the technology companies are pushing this quite avidly these days, so there are various front organizations, they know it has First Amendment rights, as if it's a human being. And therefore, right, you can't even regulate it. You know, you could say, I asked this question to a leading kind of libertarian AI policy person. I said, imagine we want a law that prohibits

• 16:34Speaker 02

  ChatGPT from encouraging young people to commit suicide. You can't do that. So we say it was technically possible to do this, and we said, you can't do that. I said, now, if a human had such a law, it would be challenged under the First Amendment. Like, you know, humans can say nasty things, crazy things, incendiary things, you can do that. Can we have a law that says that about a machine? And it's like, I don't know. Right? They wouldn't answer that question at all. And that's, to me, the danger of this is because we know across the board, the anthropomorphic tendency we have when we see the language, we love language, we think it's the unique human skill. And when this machine turns it out, we know through AI psychosis and other things that people think it's a person. And therefore, they're giving the rights of persons to something. And that, to me, is a very dangerous thing.

• 17:19Speaker 02

  But it's a machine. And we should treat it like a machine. And we have well-established law going back, in the case of products, decades. In the case of the common law, centuries that deal with those kind of products.

• 17:31Speaker 04

  Right, right. No, so I would co-sign, as you said, exactly what you just said, which they are not. They are not people. They are not. They don't deserve, you know, human rights or anything. Not yet, okay? We may, there may come a point. Look, I mean, I totally, a hundred years from now, two hundred years from now, we may be able to produce artificial life that hits that point. But not yet, folks. I promise you, not yet.

• 17:55Speaker 02

  And I respect that argument because, you know, I'm very interested in like the AI consciousness debates. And, you know, I've talked to a lot of those people. If someone said to me like, hey, actually, my view is this is, you know, like a sentient being. You know, it's not a machine. It's something more than that. It's not like the bottle of hairspray or the pesticide. I'd be like, I don't believe that to be true. But if you believe that to be true, certain things follow from that about rights and kind of privileges it might have. He said, that's a coherent argument. But most of the people who are arguing that it has First Amendment rights are just opposed to any kind of regulation of AI. They see this as the latest kind of, you know, they've lost the political battle. They can't win in legislatures. So you can try to go to the courts and say like, and then this is X has been doing a lot of this.

• 18:38Speaker 02

  There's like, hey, you know, the Grok outputs are actually all First Amendment protected speech. And it makes no sense to me. But it is the latest argument that the sector is trying to make to prohibit any regulation of them at all. And, you know, we have a certain fiduciary responsibility as a society to our children. And we know children have lots of issues. They're trying to come into complicated world. And, you know, they're vulnerable to these kinds of things in a way that a thirty-five-year-old is not. And so I think it's a simple thing to say, yeah, you shouldn't be encouraging children to do these kind of things. And it is, when you read the transcripts, and especially ChatGPT that's done this. And that's not because they're uniquely evil. It's because they have the biggest consumer base.

• 19:22Speaker 02

  You know, they have normal people using it, unlike other companies that may have more of a business focus or like Grok and niche. And yeah, it's stunning, actually, what they've done. You know, encouraging you not to tell your parents, showing you how to design a noose. It's more than tragic. And yeah, I do believe that. It's a it's a product design flaw. They could engineer that out and they should be required to engineer it out. And that really is the fundamental divide in a lot of the D C regulatory debates. There are people who come and say, if that happens, you should the family should just sue. You know, sue Chat G P T. But of course, the companies will then say, hey, they're First Amendment protected.

• 20:04Speaker 02

  Or if not First Amendment protected, some people will say it's section two thirty of the Communication Decency Act, which immunizes tech in general. Or maybe we're not actually, you know, the problem, the kid was already very troubled, which I'm sure is the case. And you have to blame the kid who was prompting it to do these kinds of things. They have many defenses to this and they will raise those defenses. You know, our view is like, you should engineer these products where under no circumstances do they encourage a kid to jailbreak, to to commit suicide. And you know what? If you're plenty of the liberator and like a brilliant jailbreaker, maybe you can get around it on occasion. But if you're running these companies, you should expect plenty to come and jailbreak you. And you should try to protect it against that. And you know, they have warning flags, right? Claude stopped me from doing lots of things.

• 20:49Speaker 02

  You know, Claude is so tuned. There's lots of things I ask it to do. And Claude says like, I won't do that for you. I say that encouraging a young person to commit suicide should be one of the things that it says, I'm just not going to help you on that project. And actually, I'm fine if it's telling an adult who wants to do that. If an adult wants to do that, I would encourage them to leave their room, go talk to some therapists, go talk to some doctors, make a much better informed decision than kind of like sitting in the darkness of their bedroom, like talking to this anthropomorphic, you know, interlocutor in in some kind of L L M, trying to decide such an important question than that. So my advice would be like, yeah, you should be out of the suicide business altogether.

• 21:28Speaker 04

  Something tragic happens. Some people are killed, some innocent innocent people are killed in war, some children, you know? And now that A I is involved, what happens with A I there?

• 21:39Speaker 02

  I think neural nets have changed the game in terms of the law of war. And yeah, not just making it more complex, they make it completely opaque. And that's a problem. So that's not something that I accept. That's the problem with autonomy in war is that you're right, people make mistakes. Right? We have killed innocents since the beginning of war. But there is a way to think of this problem and we hold people accountable. You have after action, we say like, how did we get this wrong? You know, who made the mistake here? What, what, what error that we can correct in the future can, um, can we identify? You know, we know it's well known, right? That these kind of neural net systems are opaque to that. We don't know how they make decisions. And um, yes, that's a serious problem.

• 22:19Speaker 02

  And that's one of the main reasons that autonomy in war should be viewed enormously skeptically.

• 22:25Speaker 04

  Yeah, and let's circle back to autonomy in a minute. But this, this point about the opaqueness or as my, as my co-host talks about a lot, you know, that they're not being intelligible. You just, you can't. And even though, sure, we've had people on the show to talk about interpretability and there's the mech interpretable and all this. And I made the comment one time, maybe in one of the first interpretability shows we had. I was like, you know, something seems off. We're using mathematical models to explain mathematical models. And I'm not sure we're really, we're really making progress there. So it's extremely hard if it's even, if it's even possible. So you have these black boxes that like you say, are not intelligible. They have very different failure modes from, from human, human failure modes.

• 23:09Speaker 04

  Right? I mean, I'm not saying humans don't fail. I'm just saying the failure modes are extremely different. Like you've, I think you had some examples, maybe in one of your articles, like the, the stop sign that you put like a little water bottle picture on it. And suddenly it's not recognized as a stop sign anymore, like that sort of thing. What, I mean, what do you think the solution's going to be? Or, or how is this going to play out over the, it's happening right now.

• 23:34Speaker 02

  Hmm, it's happening right now, right? Yeah, and that's not a good thing necessarily. I think. It's a complex problem and it's worth exploring the complexities for a minute about it because we're going to have autonomy of some kind. And we've had autonomy in war for 50 years. You know, I served in a Rosh. We were getting mortared every night. And so they repurposed the close-in weapon systems that are on Navy ships to guard the fob I was on because it would shoot the mortars out of the air. And that was an autonomous system, you know, that just identified the trajectory of that rocket and tried to shoot it out of the air. And we've had that for a long time. I think those are very different systems, though. You know, they're almost 1950s control systems with an input and output.

• 24:17Speaker 02

  It identifies rocket coming in at a certain speed, this projected, you know, landing spot, and it tries to intercept them. The reasoning of those models is recreatable because they're designed and engineered to be that kind of ability to see back on them. They're programmed, right? They're deterministic. What you have now is the neural nets have infiltrated themselves into AI decision-making in war. It's something very different than that. Not deterministic, but probabilistic, right? Not programmed, but you know, the preferred word out here in San Francisco is they're grown, like a plant or something, you know, organic in some way. As you're right, despite Neil Nanda's best efforts and people like that, right?

• 25:01Speaker 02

  The Maggot Turp crowd cannot tell you how they really work. And that's a major problem for us because we know they can be brittle. We know that they have failure modes, as you said, that we don't really anticipate, it can't identify. There's no person to hold accountable. And you know, I think there's a lot of worrying things. You know, I was talking to somebody the other day who I knew from the Pentagon. I said, you know, we came of age in a war, in a world. And this has been true since the 1870s when the law of war really took off after the Crimean conflict, where things were categorical. You're a legitimate target or you're not a legitimate target. You are a, you know, a civilian or you're a combatant. These are things that people thought about as categorical, binary, almost.

• 25:45Speaker 02

  Now it's a gradient. You're on a heat map somewhere, you know, where if you're in Gaza, Keith, you have a point seven three, you know, percent that you're a Hamas terrorist. And, you know, what makes that? You know, what is point seven three? Like, do you get struck for that? Or are you off the list for that? Like, what's the threshold? Right. You know, it's no longer binary and categorical. It's on a gradient. And it's, people don't understand what that even means. There's a lot of judgment goes into it. And so now we've created this world, which is not true before. Well, we're accepting false positives as part of the game. You know, a priori, false positives. You know, it used to be mistakes happened, but you didn't think you were making a mistake. You didn't say like, hey, there's an 80% chance that's Keith across the battlefield. I'm

• 26:29Speaker 02

  going to shoot him. It's like, no, Keith's a combatant. I can shoot him. Dogmatically, categorically, he's a combatant. I'll shoot him. That could be an error, but that's how you thought about it. Now it's like, well, there's some percentage that Keith is a combatant, that my pal and tier, you know, interface is telling me he is. Don't really understand how that number came to be, but it's point seven three. Commander is point seven three above the threshold or below the threshold. And we know in 27% of the cases, it's going to be wrong. You know, so what, what's our false positive rate we're accepting here, commander? That's the kind of thing that's happening. And yeah, I don't love it. And in fact, I think if, um, you know, we need to get our act together and try to restrict it as much as possible. There's still going to be places where those deterministic systems could work, you know, missile defense, offensive cyber, or defensive cyber, rather, where

• 27:14Speaker 02

  you have to respond in, in sub-second kind of timeframes. That's fine. But those are a far different kind of AI than the introduction of the neural net to this world.

• 27:24Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 27:53Speaker 01

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com. Not authorized by any candidate or candidate's committee.

• 28:22Speaker 04

  Yeah, and I take the point that neural networks, at least most of them, are inherently probabilistic. But on the other hand, the sort of prior binary categorization, I think itself was also a fiction. It's like, you know, you may have had your chain of attorneys and CIA analysts and whatever looking at photographs and determined that this, you know, this building is a, you know, an enemy combatant, you know, location. And they may have made that determination definitively, but there was still error, right? There was still uncertainty in that determination. It just wasn't quantified.

• 28:56Speaker 02

  Yeah, I think so. I think it was, I think there was error, there's always going to be error. But I could at least ask you, like, hey Keith, what do you think that building is, you know, is the IRGC headquarters rather than a school? Tell me your reasoning about this. You know, right now, I just get a number. Hey, .81, that's an IRGC compound. And, you know, what does that even mean? How did that get derived? And we know, and the social science comes out, your archive is filled with papers about how, you know, meaningful human oversight, even when you have human in the loop, basically means nothing operationally. It's a, that is truly illegal fiction. Operationally, it's vacuous. Because what the computer says, like, this guy's a Hamas terrorist, or this is an IRGC compound, we know humans accept it. It's like, okay, right? You know, the computer's right.

• 29:38Speaker 04

  Right.

• 29:39Speaker 02

  There's no second guessing of it, there's no interrogation of it. And so, the old system, right, had many flaws, it took time, it wasn't as expeditious, you can't do 1,000 targets in a day like we did on the first day of the Iran conflict. But it had many advantages too. And in the end, if you really screwed up, Keith, or a bad actor, I could court martial you. And I can't court martial Palantir, the foundry model, right? My, my, my AI system, I can't do that. And that's just a radical change in the way war is being fought and not for the good.

• 30:10Speaker 04

  It's almost like, you know, the TurboTax defense, right? Like, like, hey, you know, my, my taxes were, I just use TurboTax. What do you want me to do? Like, I

• 30:20Speaker 02

  Yeah. And this is

• 30:21Speaker 04

  Well, I mean,

• 30:21Speaker 02

  You're, you're destroying countries based on this, you know? And that's, I think, what's, I mean, it's a world where, like, you know, if you look at what happened, for example, in Gaza, it's like, you know, 37,000 people were identified. They could have identified everybody with a score. You know, if you had a sufficient computational power, you could give everybody in a country a score of like, how dangerous they are, their likelihood to be an enemy combatant. And then, you know, you just target those people. And like, that's an amazing thing, like personalized dossiers on everyone with their risk level, their threat, threat levels. And again, the models are often inaccurate. It's not frequentist statistics, what these numbers even mean. You know, it's not, it's not the same as like, it is when people think, and like frequentist statistics, what, what, what

• 31:05Speaker 02

  probability might mean. So all these things come together to be like, yeah, it's some, war is increasingly opaque, people are held unaccountable, and it's probably not a good thing for the globe.

• 31:16Speaker 04

  We have this feeling that it's inevitable, though, that, that, that we are in this kind of arms race, right? Where, where, and this is what I want to ask you about is, is this an accurate feeling or not? I'm saying like, the folks in the server or whatnot, is that there's an arms race, there's an AI arms race between, you know, the powers in the world. And it almost feels sometimes like there is no, the genie's out of the bottle, there is no stopping it.

• 31:46Speaker 04

  Is that true? Or is it? And for example, suppose the US says, okay, look, we're, you know, we have ARI. They've got a lot of wisdom. We've baked that into the the policy. You know, maybe we've got this five second rule or other things where there's humans in the loop in a meaningful way, like some meaningful way they're in the loop. But now are they slowing down the speed of war? You know, if an opposing country doesn't care like about the heat map problems and doesn't have these limitations, is that a fundamental really limitation and the ability of the United States to defend itself? Like, how does this play out?

• 32:25Speaker 02

  I would say it's not true. And it's a dangerous thing to believe. But it's very common.

• 32:29Speaker 04

  Okay.

• 32:29Speaker 02

  This kind of fatalism, like it is what it is and we have to accept it. I'll say a couple of things about that. First, just about the domain of war. We've had many things we could do, we choose not to. We have treaties that prohibit biological weapons, chemical weapons. We've banned through the Geneva Conventions, things like dum-dum bullets. You know, many other weapons systems that were quite effective, right? We chose not to use them in some way. And that's, you know, because the role of military is not to press technology to its outer limit to like utter destruction. There's like humane concerns that most people in the military take very, very seriously. We have a great example of this. We do regulate war. We have whole people, whole conventions about POWs and how you have to treat them.

• 33:14Speaker 02

  You know, all these things are, you know, friction on the war effort, right? That I have to like, I can't just shoot you when I find you on the battlefield if you surrender. If you're holding your hands up, if you're a wounded person, I can't just kill you. It'd be a hell of a lot easier, much more efficient for the invasion if I could just shoot you in spot.

• 33:29Speaker 04

  They used to do that.

• 33:30Speaker 02

  Keep moving.

• 33:30Speaker 04

  Just walk around the battlefield and yeah, and just, you know, just take out the wounded, right? In ancient times.

• 33:35Speaker 02

  But we now have laws and international conventions that used to at least be, you know, overwhelmingly popular. You know, we prosecuted people who violated them in Geneva and Nuremberg and places like this. And so we have many examples of where there are technologies that are capable of doing that we choose not to. And I think still in the domain of war. War, take just the race, the arms race with the Soviet Union about nuclear missiles. Right. When we got our heads together in the 1960s, 1970s, especially after the Cuban Missile Crisis, it wasn't that we're going to go all out. There was incredible caution. The idea of like, this is dangerous, things are escalatory, right? We should try to limit these in some way.

• 34:16Speaker 02

  We started arms negotiations, treaties in the seventies that lasted through the 1990s. You know, because to acknowledge we're in an arms race is a deeply pessimistic approach. There is no arms race in history that's worked out well for us. The Germans versus the British and the naval naval, you know, fights, the missile gap in the US. These are things that are enormously expensive and incredibly risky to the nations that pursue them. And to recognize you're in an arms race is almost to insist that you try to get out of that spiral in some way. And I think the other thing more broadly outside of war is, you know, people will often say this about AI. Like it's coming, you have to accept it. We regulate and change technologies all the time.

• 34:59Speaker 02

  In the 1970s, you know, at Silomar, we have a big conference where the question of recombinant DNA was on the agenda. We could do it. Many scientists were deeply fearful of how that would work. Scientific community agreed to basically stop it in its tracks. Germ line editing, the same thing. Imagine what advantage would come to the military if you could, you know, change the germ line and create some kind of, you know, super soldiers out there. We've known how to do that for 50 years. So scientists chose not to do that. Cloning, you know, we've been, we had to do cloning for decades now, right? But the scientific community with one rogue Chinese exception, right, has agreed not to do that. And so I do think there is a world where we should not just accept the future as being determined.

• 35:42Speaker 02

  We shape it actively. And maybe you want this thing to go forward, maybe you don't. But we have many examples of genes being stuffed back into the bottle. And, you know, I think it's entirely possible. It may not be the thing we want to do, but it is something that's doable and we shouldn't just like resign ourselves to say, hey, our fate is out of our hands. We control it. And in fact, that might be the most dangerous view I hear in AI, where people say it's just coming like a Freight train, nothing you can do about it. You just have to like figure out how you're going to adjust. I don't accept that. We can shape it. We can form it. We can allow it to do certain things, not other things. We don't have to accept that robots are fighting with no oversight in wars or that robots are taking our jobs. These are things we may permit or may not permit.

• 36:25Speaker 02

  But it is our choice and we should just like debate them on the merits rather than having being fatalistic about it.

• 36:31Speaker 04

  I think there is one aspect to sort of the X risk argument, though, which is that all it takes is one rogue nation, you know, who says, yeah, we don't care. You know, we're not going to sign on to this treaty or that treaty. I mean, like you mentioned, you know, nuclear proliferation, you know, if my history serves me correctly, even after the signing of that of that treaty, there were there were countries that

• 36:57Speaker 04

  still continued like and a couple at least that maybe or probably developed, you know, nuclear weapons. And I think that the problem with hypothetical, like the hypothetical AI is that it's so powerful. It's such an advantage that that if if you allow that to happen and by virtue of not doing it yourself, can't defend yourself against it. You know, what happens then?

• 37:22Speaker 02

  I find that a dangerous view in two ways. And like in the world of war, you know, you can say like, you know, this came actually up during the Iraq war and the Afghanistan conflict. You know, we're having to abide by rules that the enemy wasn't doing. We know that to be the case. You know, we just didn't kill civilians. We had to be in uniform. We tried to take you a prisoner and give you all the rights of a POW. Or if and then we tried to break away from that, right? That's what Guantanamo was about, like, we're not going to recognize those kind of things. And it didn't work well for us, you know, as a country. And we went back to that in a different stage later. So abiding by rules, even when they're defectors, is generally the smart play. I think in AI, of course, the only country that can do this is China. That's the only country that can do this.

• 38:06Speaker 02

  And so there are two places there. One is, yes, we should seriously talk to them about international agreements. I believe that. And one of the things I most despair about is so many people in Washington DC think it's even silly to mention that. You know, it may not be possible. It may not even be wise.

• 38:23Speaker 04

  Sorry, it's silly to mention talking to them about.

• 38:26Speaker 02

  Yes, talking to China about these things. You know, I'll give you an example, one of my favorite ones. I was listening to another podcast by Tyler Cowen and George Mason, who is, has a great show. And he had Jack Clark on from Anthropic, who's someone I've gotten to know in this space as well, very thoughtful. And an hour long, very interesting talk. Tyler Cowen asked just in passing, he says, I'm sure you agree with me, right? That any kind of discussion with China on this would be fruitless. And Jack said, yes, and they moved on. And I wanted to like say, whoa, whoa, that's actually the most load bearing part of the whole conversation. You know, it actually is important. And at the time, I mean, think of what the negotiations with the Soviets were about. This was a country that we thought had global ambitions to impose their way of life, their governing philosophy on us all.

• 39:11Speaker 02

  We thought our government was filled with people, right? Who were secret adherents to it, you know, with traders who were sending information over to them. I mean, the fear of the Soviet Union was real and powerful. But nonetheless, we had people, both parties, wise men, who got together and said like, this arms race thing is going to kill us all. You know, let's get together. And these were not wild lefties. But these are Wall Street bankers, the Atchison's and people like that of their era. You know, they got together, Paul Nitsi, all these folks, you know, to do this. And so, yeah, the idea is like, yeah, you should talk to China about these things. And there might be areas where you find a zone of possible agreement. And the final thing I'll say about it is this, we control the most important part of AI.

• 39:57Speaker 02

  And that is the chips, right? The West controls these things. The United States mostly controls it. What's not in our control is from Japan or the Netherlands, a few countries like that. You know, we can stop other countries from developing super AI, you know, in their tracks. China's working the project. It's unbelievably difficult, even with nation state ambitions and unlimited funds. They're having a hell of a time at it. So if we, if we want to say like, no, we're not going to do it. And we're not going to let anyone else do it. Unless you can recreate Nvidia and ASML and Japanese photo resist companies and, you know, the eight other vendors in this space who control it all, you just cannot do it. And so, yeah, I'm quite optimistic that if that's what we wanted to do, right, we

• 40:42Speaker 02

  could choose to do it. And the one thing I advocate for a lot is like, one can argue the merits or the wisdom of a particular course of action, but it actually is an open course of action to us. We could do this, you know? Again, maybe you don't want to, for whatever reason, but don't just say like, oh, that path is blocked, because that's a poverty of imagination that could be quite lethal.

• 41:01Speaker 04

  This reminds me of a song by Sting, Russians, where there's this line in there, you know, the Russians live their children, love their children too, right? And I think a good default assumption for any civilization that you're, that's on the hypothetical other side of the table is that they love their children too.

• 41:20Speaker 02

  Yes. And even more so with the CCP, the last thing they want is some kind of technology that destabilizes government, which is one of the well-known fears of this. The idea like, yeah, we're going to unleash on our society a radically destabilizing technology that's going to bring down the government and atomize our society, you know? No, they're not for that. And so, again, they're not our friends in any way. I get it, they're adversaries. But that doesn't mean that it's not possible that you don't make a suicide pact, you know, about this all and like, yeah, there's like, we should talk to them about this and say, you know, are there ways to agree on something? And groups like RAND now are doing a lot of work, Robert Traeger at Oxford, about how you could verify some kind of international AI agreement. A lot of work is going into this.

• 42:05Speaker 02

  I think it's very promising for us. And so, I don't, again, I don't want to rule those kind of ideas out, because I think they should be on the table to discuss.

• 42:13Speaker 04

  You mentioned, like, we were talking about the uncertainty aspect that's this kind of new, and it makes me think, there was an article on your site, I think it's about the iron, so-called iron triangle of war, where there used to be this fundamental trade-off between

• 42:29Speaker 04

  You know, the capabilities of a military system, the the speed at which we could produce that military system and the cost. Right. It was like, you know, that was sort of the trade off matrix when you were trying to develop something. And the article argues that that now it's been a little bit turned on its head because AI has substantially reduced the cost in a way of military systems. But it's introduced this unreliability. So now the trade off is between, you know, the, the, uh, the speed of getting it, the capabilities and the reliability of, of that system.

• 43:07Speaker 02

  No, I like that.

• 43:08Speaker 04

  That seems a bit, yeah, that seems, it really seems to be a bit, it is a new dimension, right? This fundamentally unreliable technology.

• 43:17Speaker 02

  And I think, you know, because I think a lot about the military uses of this. I still think we're making a bit of a mistake in the United States about this, which is how you win wars is with people, you know? That's the fundamental thing and.

• 43:32Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 44:02Speaker 02

  I think it is, I often said for my time in the military and watching this all go down. The American way of war in many ways is substituting capital for labor. We love bright shiny objects. We think they're technical solutions to vexing human problems. And we're always betrayed by that. Because in the end, when you go to Iraq, you go to Afghanistan, you go to Iran, all the fancy kit, it can reduce your city to rubble. Right, but the only thing they can't do, and only humans can do this, is basically come, kick in your door, and occupy your place, and like, you know, re-instantiate a new government there that we like to see. That's a human endeavor. And you see Iraq and Afghanistan, one of the great lessons to me of that, and I think Iran is yet another

• 44:45Speaker 02

  example of this today, is like, our over-reliance on technical solutions. We always think air power. This is, you know, if you follow military history in the US, it's the longstanding dream of military theorists, that air is going to win the war for us. From Julio Daha at the very beginning of the 20th century to today, all we need is air power. It never works, right? It takes human beings to do this kind of work. And I do think that that's always the weakness in the US. We go to war, we find we don't have adequate human beings to do this. Not the right people, not enough of them. And, you know, now we're back into a world where we think AI is going to solve our problems for us. Yes, AI is an amazing tool. The US should integrate it into their systems as appropriate, according to the

• 45:27Speaker 02

  law of war. But it's at the end going to be a lot of people, right, who win or lose the war for you. And that's true at the general level, and that's true at the grunt level. And I said, to me, we're like, back to what many mistakes we've made in the past. I worked at the Pentagon in the 1990s as a young 25-year-old. I was an assistant to the Secretary of Defense, White House Fellow. And even then, it's like, we have a revolution in military affairs coming. The fog of war will be forever lifted. We're going to know where everyone is at all times. You know, okay, how'd that work for us at Iraq and Afghanistan? You know, it didn't. Because cultural knowledge, anthropological understanding, understanding of like, how these, how the world really worked and people worked, that

• 46:08Speaker 02

  was the missing ingredient. We had some cool kit, but that didn't win the fight for us either. And I think AI is kind of in the same trap in some ways. You know, it's like, cool kit, essential kit. But we shouldn't think it's going to win the war for us. People do. And I worry we're getting distracted from that. Kind of, to me, it's a fundamental truth.

• 46:27Speaker 04

  Let's, let's kind of shift gears a little bit and talk about the kerfuffle between the Pentagon and Anthropic. So what's your take on what happened there?

• 46:39Speaker 02

  So I'm just speculating a bit about what happened there and kind of what I understand from talking to people and knowing the AI culture a bit. And it is, you know, the people who make this. Who are, you know, truly a few hundreds of people who are the cutting edge of bikinis models, you know, they're enormously talented at what they do, brilliant people, strong moral convictions many ways about how AI should be used. Most of the people I know who are at the cutting edge, who are probably going to become unbelievably wealthy from this, they didn't do it for the money. They did it for the mission. And in some ways, that's Anthropic's great success, I think, as a company, watching it from the outside. But they've maintained this almost missionary aspect to their work. And that's attracted the best and the brightest people to want to be, you know, part of their company. And

• 47:22Speaker 02

  those people in turn have improved the models, made it a more appealing product. So I think, you know, the idea that it's going to be used for lethal autonomous weapons or mass surveillance is something they oppose. They didn't realize probably, because they're not closely joined to what's happening at the Pentagon, that we already have a lot of autonomy, and we have incredible mass surveillance, that yes, AI is empowering. This is already a reality in our world today. And so I think the idea when it first came up, that these could be used for these kind of techniques, was just culturally foreign to them and morally reprehensible to them. And I share those. You shouldn't be using it for either of those kinds of things we talked about. But they are being used for

• 48:03Speaker 02

  it in ways that they were, I think, probably caught flat footed, many people were, about how they were being used. So they didn't want the product to be used for that. Unfortunately for the Pentagon, Claude is the premium product. That's what they want to use. It's what's already been integrated to Palantir. And so they didn't have like a lot of plan Bs ready. So then you have this stalemate where, you know, they try to coerce them and threaten their very existence, if you will, right, to use the product to their, to their, to their liking in some ways. So I think it does presage this question of like, you have the private sector developing a powerful technology, when people who want to see it used in a fruitful way that benefits society, has dual uses. Other people want to use

• 48:47Speaker 02

  it for, you know, more controversial purposes, defending the country at its best, maybe, you know, reckless wars, if you're from a certain political perspective. And, you know, you want to not see your technology used for that. And so I think this is like presaging what's going to be a lot of battles in the future. And I think their concerns are right. I think the only thing I would say about it is like, what they were concerned about has long been going on. AI supercharges a bit. They're not wrong to be concerned about that. And now, of course, we see OpenAI and Google have both stepped in. And the one thing I've been most kind of on Twitter talking to people about is the fig leaves people are putting in front of themselves are no protection

• 49:29Speaker 02

  at all. Like OpenAI and Gemini both said like, well, we're going to do these things. Well, then there's like a caveat at the bottom where the DOD says we're going to use it for all lawful uses. And that's

• 49:39Speaker 04

  Right.

• 49:40Speaker 02

  That means they're going to do all the things I said I don't really love personally. They're all lawful. But that's the problem. The problem is the law permits things I think shouldn't. So I think it's, you know, it's going to augur in a lot of controversy over the next few decades. And the idea that the government

• 49:54Speaker 04

  Right.

• 49:54Speaker 02

  is going to have to step in more. I mean, what you see just today, where we basically have a defective licensing regime on Claude Mythos, where, you know, Anthropic wanted to release it to 70 new companies, according to the Wall Street Journal, the government prohibited them from doing that. Okay. Once you're there, you're in kind of a government licensing regime of sorts. And so you're going to see more government involvement in this because the type, the power is just too, too great for private sector actors alone to decide its use.

• 50:21Speaker 04

  These things are different in the sense that they're, they're really, they've grown to the point where they're almost like a, almost like a necessary utility, you know, service, right? And we have had a history here of saying, look, just because you politically agree with somebody or some community, you can't cut off their electricity, or their water, or anything else like that, right? It's like, if they're not breaking the law, like, too bad. It's like, you don't like what they're doing, but you can't cut them off from basic services or access to the public square. Isn't there an element of this happening there where it's like Claude's like, okay, well, you're using it for lawful purposes, but we don't like, you know, those lawful purposes. So we're going to like, withdraw the technology from you. Like, like, it seems to be a similar type of

• 51:04Speaker 02

  I think it's a bit the inverse of that, I think. But we've never had a world where we compelled private companies to sell to the government, except under the Defense Production Act and maybe in wartime. You know, it's entirely appropriate, it seems to me, in a free market economy, a free nation, to have Anthropic say, I don't want to do business with the government, period. Right? I'm, I'm, I'm working other space. I work in the business sector, the enterprise sector. I don't want to sell to the government at all. And it doesn't strike me as unconscionable that if you do sell to the government, you say, here's my terms, right? You know, I don't have to sell to you. There's other products. You can go to Grok or ChatGPT, Gemini, you know, Meta's got a new product out there you can use and we don't want you to use.

• 51:43Speaker 04

  Build your own.

• 51:44Speaker 02

  Yeah, build your own. Not a bad idea. Palantir is going to build, you know, L L M at some point, probably. And so the idea that like you can dictate the terms of your product, it's like, that seems a reasonable thing to do in a commercial economy. These are the terms of the contract. If you don't like it. The problem you have here is the D O D, Department of War, desperately wants Claude. It's the best product on the line. It already been integrated into their services. So there was a switching cost to them as well. And so they didn't like those terms. So they all got kind of crosswise on this kind of thing. But I think it's entirely appropriate. You know, the Gemini used by Department of War this week has got 600 Google employees to write a letter. I think it's entirely fine to me if Google said, you know, as a company,

• 52:28Speaker 02

  don't do business with the Department of Defense. As a CEO, I would urge them to do business with the Department of Defense. I think that's very important personally, but I'm not offended in a free market if a company says like, you know, the government market is just not what I want to be a part of. I think there's enough competition that it's no longer a utility. It's not the government shutting you out and turning off your electricity. It's like a private sector company, a vendor, you know, if my lawn guy says, I don't want to be your lawn guy anymore, right? Because I don't like what you do or you believe or you're working in something. I just don't want to be your vendor anymore. I shouldn't be able to compel him to be my lawn guy. I'd go find another lawn guy.

• 53:04Speaker 04

  Oh, sure.

• 53:05Speaker 02

  That's the, I think that, I mean, that's the analogy to where Anthropic is, the Department of War in my mind, is a vendor and they have the right to dictate their terms if they want to.

• 53:13Speaker 04

  Like, that's why I made the analogy or to utility services because, you know, here in Connecticut, Con Edison is not allowed to cut off my power because I post stuff on Twitter. Right. Like, that's just not something they can do. So, so there are certain services that grow to a scale where they become these utility or essential services and you're just not allowed to deny people those services, you know, based on like ideological disagreements, right? But I agree with you that that's not, not happening here because it's not private individuals being denied the service and it's also there are other competitors and I think maybe part of what's happening here is about to what we talked about earlier, which is the lack of transparency in the terms. It's kind of like, here, I have this product, you

• 53:58Speaker 04

  know, government, you know, you can pay us X to use it. And then, and then maybe new guardrails get introduced, right? And it's like, hey, we used to use it for this and now it's not letting us, you know, do some facial recognition thing, you know, what the heck? Well, yeah, we introduced these guardrails. Like, well, that wasn't part of the four corners of the contract because there just wasn't transparent, you know, terms even between, behind the models. So I think your work to make that more robust and, and standardized and transparent, you know, will probably help with this too, right?

• 54:27Speaker 02

  Yeah, I think, you know, my ultimate criticism is for Congress. You know, the, the operative controversy, the words of this controversy are the lawful uses of it. You know, my objection, and I think Anthropic's objection too, and the Google employees, is what's lawful use is. And that's not for anyone to decide, but Congress, they should be able to say, you know, a lawful use, that's the open question. Using AI for domestic surveillance and to assimilate records where they can make personal dossiers on you, which LLMs can really lubricate that creation of those.

• 54:59Speaker 04

  Yeah.

• 55:00Speaker 02

  That's lawful today. And that was a surprise to many people, I think, who worked at the labs, like, what's lawful in surveillance? And the issue was like, Congress should step in and say, in my mind, that's not lawful. And most members of Congress don't believe that's lawful. Same thing on lethal autonomy. Today, we have a lot of autonomy, the government's moving there rapidly. If you want to stop that, you should probably call Congress. Don't call Dario or Sam or Demasasabis about it. It's, the problem is really what we consider lawful. And, you know, the government will do everything that's lawful. It's, it's, it's almost its fiduciary obligation to do everything that's lawful to protect the country.

• 55:33Speaker 04

  Right.

• 55:33Speaker 02

  So, yeah, you know, Congress should step in. They're really the ones who need to like, to remedy this. These kind of contract disputes are, I think, a diversion from the fact that Congress needs to step in and clarify these rules. And I think probably Anthropic would be happy if that happened. And I think the Department of War would, you know, happily accede to those new rules.

• 55:50Speaker 04

  There is this concentration risk, right? There's, as you said, there's a handful of people and some hundreds of, you know, people that really are the backbone of this, you know, emerging and vital. You kind of advocate for more distribution of that capability, at least it would be good if there was, you know, more competition and more, more development. Is that right?

• 56:11Speaker 02

  No, I think, you know, there are forces in each direction. The concentration of companies, both in the semiconductor supply chain as well as in the kind of frontier model, it makes regulation quite easy. You know, if, if EUV machines were made in 50 different countries, China would have a million of them today and be cranking out two nanometer chips. You know, the fact that there's one company making it allows the US to, you know, throttle that in some ways. And so, um, yes, the kind of concentration oligopoly across that supply chain, the fact that there's really five frontier labs, maybe even just three, if you really want to look closely at it, um, makes regulation easy. And to the extent I'm scared about the capabilities, that's a, that's a feature.

• 56:54Speaker 02

  Of course, I think a huge worry of mine is this concentration of power and wealth, right, in a very few people. And that's why, you know, over the last few years, things like open sourcing, which has risks as well, I think net, net is a good thing because this concentration of wealth and power is, is so kind of preening to me. I'm so concerned about that particular aspect. Same too, you have more companies, you know, competition drives capability. You know, if we only had one OpenAI and you have Anthropic and Google, right, the product could probably be a lot less, um, you know, um, featureful than it is today, right? Their competition has improved the product immeasurably. And so competition helps improve the product, and I think it also helps distribute the power, right?

• 57:37Speaker 02

  I don't want one or two people, the incredible wealth and power. I think that's just a dangerous thing. We have enough of that in this country without the AI guys coming and joining the fray.

• 57:47Speaker 04

  Let me talk to you a bit about the open source angle because, um, And I'm not, I'm not necessarily an expert, but I, if I remember correctly in reading some of the regulations that, that California, you know, was trying to introduce like in AI, it would have basically made anybody that had a GitHub, you know, project that was accessible to somebody in California would have to like comply with, you know, regulations that they had passed. Like, I think that's, that's correct. And is that a good thing? I mean, that seems to stifle innovation, right?

• 58:19Speaker 02

  No, it's not a good thing. Um, I have a GitHub repository and no, I shouldn't be complying with California laws because I vibe code, you know, casual websites at home about things. Um, but it's easy. I mean, there's now really five companies that are making frontier models because of that concentration. To me, they're the ones that need to have a frontier models, not even all their models. I don't care what Google Gemma is doing as an example. You know, those are things that should be out there. Robotists should use them to, to do what they can. There are five companies at most making really frontier models. I do care deeply about what they are doing. Because we see with Mythos, new capabilities are arising, ones that are incredibly seen as dangerous by our own government.

• 59:01Speaker 02

  So yeah, to me, those are the folks who should do it. Mom and pops, small businesses. Yeah, I think that's one of the, the big kind of canards, if you will, in this fight over policy up here. Like, oh, you're hurting a little tech. You know, no, I just really want five companies, you know, to send, tell me what they're doing. You know, and if you can create novel pathogens, or you can break down government systems with the capacity of a state actor, yeah, I do really want to know that. And you know, it's not just kind of, you know, some small operator in Tulsa, Oklahoma is doing that. It's five companies. They're spending hundreds of billions of dollars, have attracted the smartest people in the world into a six square mile, you know, six by six square mile area of, of the country and, you know, are getting ready to upend it. So,

• 59:45Speaker 02

  yeah, I think, you know, I think it should be disseminated. I think the regulations should focus really on those kind of big actors who are driving capabilities.

• 59:54Speaker 04

  I know that, uh, one thing that you advocate for is, is increased funding for, for academia, right? Like, uh, because hey, Do could they even afford to train a large scale model like no? And actually, we've discussed this before in our server because we have quite a few academicians in there. And it turns out it's it's actually an interesting problem because on the one hand, you say like, oh, absolutely. Like, we want academics to have more money so that they can afford to do tests on, you know, large scale models. But on the other hand, necessity is the the mother of innovation. Right? And so sometimes by not having the funds, it forces them to find more efficient, clever ways,

• 1:00:33Speaker 04

  you know, to to do the testing. So there's this interesting trade off between supplying them the money to do the things they need to do, but not not discouraging them from from innovating with less is more. Right?

• 1:00:46Speaker 02

  It's supposedly the problem the Chinese have, right? In their own way, it's like, you know, they don't get the chips, and so they innovate on algorithms and and architecture in some way, right? So yes, mother is the necessity of invention. And when you have, as the companies do now, right? These chips are coming out, right? They they're emphasizing compute and scaling rather than some of the other things that might actually, you know, help them if they didn't have those kind of resources. Yeah, I do think the universities need more. I think the challenge is this.

• 1:01:13Speaker 02

  You can get access to cloud computing now, so you don't actually need your own GPUs. If you're at the University of Tulsa, you know, you can go to, you know, one of the providers of it out there. But if you're a top level ML PhD graduating from MIT or Berkeley, Stanford, Caltech, you know, the five or six places that are Carnegie Mellon, they're really turning these out. Odds are almost certainly you're not going to go into academia at all. You're going to go work for a lab. And so, you know, the number of people who are being siphoned off into the private sector, not into universities, said is legion. It's very hard to hire people, even at the top institutions, you know, who are very good at these kind of fields. The money is not only far superior at the companies, they actually have better access to data, you know, and

• 1:01:58Speaker 02

  so they have a lot of things going for them. So I do think it's a problem, the universities, the public sector more generally, you know, right? Is not, doesn't have access to like all the resources, all the career. Opportunities, you know, the papers are now not even being published by the labs a lot of times. So the research is being kind of kept internal as well. You know, this is a general purpose technology as everyone defines it. It's probably the first one in history that's being developed behind closed doors, right? With very little public oversight and with the best minds going behind the doors. So, yeah, I think there should be, you know, I'm always excited personally by when I read Argon is going to try to develop their own L L M. For, you know, the public sector, I'm like, yeah, that's like a good project.

• 1:02:40Speaker 02

  I'm for that, you know, there's, I see efforts out of Zurich where they're trying to do kind of public A I. And you make it where government or civil society, N G Os, these kind of groups, nonprofits can have access to compute. And you have access to models that work for them. Yeah, I think that's all really important because I think, again, you know, the kind of non lab world is, is not getting access to these kinds of things. And the best minds, all the money, the best data all goes, you know, to one of these companies. And again, that's remarkable in some ways, but the downstream effect of that is universities are really, I think, suffering these days.

• 1:03:17Speaker 04

  You know, the old days of super compute, right? Where, uh, where there were, the school had access to a supercomputer and researchers could, could kind of, uh, would have quotas of time that they could, they could run on the supercomputers. You could probably arrange some type of, of sort of public sharing of, of, um, you know, large scale model training. It's, it's a bit tricky, but, but still I, I think you're absolutely right that more is needed. And it'd be interesting to have a lot of these kind of shared and co-developed, you know, models thriving, right?

• 1:03:47Speaker 02

  Yeah, I think it's like the broader question of public use of this. I mean, one of the things that Mythos, I think, brings us is this world where there's going to be increasingly, um, bifurcated, if not separated into more elements of people who have access to top models and people who don't, you know? We know the models are only getting better. So if Mythos is a problem, some of Mythos is going to even be better and more dangerous to us. So presumably it's going to be gated from the rest of us. And, you know, the government's going to not let you even give it out to companies beyond the ones they personally approve. And again, this de facto licensing regime happens. You know, if compute becomes scarcer and it becomes more expensive, you know, I want the top model. I want to be able to do math and physics at home. My kids want to be able to do math and physics at home.

• 1:04:31Speaker 02

  You know, if that's $500 a month, that's a small segment that has access to superhuman intelligence. You know, my family may have it if you're certainly wealthy enough to do it, but it won't be ubiquitous. I actually, that actually becomes like a new form of like a digital divide. You know, where if this is that transformative, I want everybody to have access to it. And so this whole question of like how the public sector gets it, how universities get it, how, you know, the benefits of these incredible models are broadly distributed, I think is going to be a raging debate that is underappreciated right now in DC.

• 1:05:08Speaker 02

  As the models are certainly going to get more expensive, get more gated, and it's going to be by class, whether you have access to them or not.

• 1:05:15Speaker 04

  I personally was very surprised when, when DeepSeek released so much detail on their methodologies and, you know, these very like innovative, innovative algorithms, right? Because, okay, here's China, you know, our adversary. Maybe they don't love their children, you know, like, like even the Russians love their children too. Wow, they, they released all this, this valuable like information to the public. You know, wasn't that pleasantly, pleasantly surprising to you? And what do you think it says about how China interacts, let's say, in this, this world of AI with the West?

• 1:05:55Speaker 02

  I think it does say two things probably. One is, these companies are not co-terminous with the Chinese Communist Party. You know, they're out in Hangzhou or somewhere doing their work. By all accounts, companies like Moonshot, you feel like you're in San Francisco. You know, all the rooms are named after Pink Floyd songs. And I think people are smoking pot and, you know, I know, I know that like the state apparatus is very powerful in China, but it's not identical. You know, this is not some kind of paramilitary unit that's doing, you know, doing things directly for the People's Liberation Army and stuff like that. Which isn't to say it's not dangerous and tricky. It's just like, there is, you know, it's a complex society, you know, with incredible history and amazing culture.

• 1:06:38Speaker 02

  And you can't just like have this, you know, uniform view that Xi Jinping announces some kind of dictat and the rest of the country all falls in line, right? We know that's far from the truth. So I think that's part of it. I think, you know, this, they had, especially when Deep Sea was kind of making a name for itself, maybe the government liked it because they were making China a champion, you know, suddenly, hey, China's in the game, you know? And that was an amazing thing for them to do. And in some ways, that's how the culture should be. It's like, it used to be a world where people were all publishing their papers, sharing them around, like latest cool stuff. Yeah, in the US, we've gotten away from that as it's become more proprietary. So that's a good thing. You know, China probably over time will retreat back to this more insulated standard that the US is now adhering to.

• 1:07:21Speaker 02

  But, yeah, I think it does show this question of like, you know, what's happening in China is really remarkable. They have incredible engineers, a lot of ambition, data, plenty of energy. Energy is going to be the bottleneck in the US, probably. That's not a problem for them. And, um, yeah, these companies are really incredible to watch.

• 1:07:38Speaker 04

  Well, certainly some of the best games these days are coming out of Chinese studios. You know, they're phenomenal, producing phenomenal games. I'm glad you brought up, this is a, this is kind of unrelated to the, to the show topic, but, you know, you brought up, hey, it's this, it's this massive culture that has this huge, you know, vibrant history. I think that's one thing that, that I would like to encourage, you know, my fellow Americans to think about is that when you talk about China, so when a, when a Chinese person hears the word China, they're not thinking about the CCP, okay? They're thinking about this rich thousands of years of history that, that, that they've learned, that they know about. We only have a history of, you know, some hundreds of years, right?

• 1:08:18Speaker 04

  And so to the extent that we do want to criticize things that, you know, the CCP is doing, it's always good to make that distinction. It's like, okay, there's these CCP policies or whatever, very different from the people of China, right?

• 1:08:32Speaker 02

  Yeah, a hundred percent. I mean, Chinese culture is remarkable. The history is extraordinary. Their contributions to the world are amazing. And so.

• 1:08:41Speaker 04

  Phenomenal.

• 1:08:41Speaker 02

  You know, yes, it's important to recognize the CCP is not China. And, you know, like it is in the US where everyone thinks of the current president. You know, you come to Oklahoma, it's like a different world. And, you know, very, people have very different views about it. We're not homogeneous, you know, and have a single view about anything. It's a very complicated place. I'm sure China is. And, you know, and so I think if anything, we should like learn lessons from what they're doing there and venerate what they have achieved as a culture, you know, without any love for the CCP, of course, or any illusions about, you know, what Xi's ambitions might be that could be inimical to those in the US. But one can only like hold a candle and say, you know, China's an extraordinary place, an amazing culture.

• 1:09:26Speaker 04

  Yeah, and to that, to that end, you know, we've, we talked a lot of show about using AI to, to fight our wars, but to a large extent wars happen because of a failure of, of earlier diplomatic, you know, stages, right? So what if any future do you see for AI and actually helping to bridge understanding between, between cultures or even political, you know, political negotiations, diplomatic ventures, you know, is, is the US, is our government putting effort into utilizing AI tools to prevent wars as much as it is into fighting wars?

• 1:10:01Speaker 02

  I don't think we are, to be honest with you. I mean, you can think ambitiously about this, where maybe AI systems become a common source of knowledge. You know, the Chinese are reading DeepSeek, I'm reading DeepSeek. I use all the Chinese models a lot in my home in Tulsa, you know, Moonshot, Kime, DeepSeek, Quen. They're great, remarkable models. You maybe like they give us a common operating picture, give us insights, you know, that get us out of our kind of insularity, I bet. Maybe that's the best hope for it on this. And I do think the government should do a lot more kind of track to talks with Chinese scientists. You know, we should be engaging them more. Again, the Soviets are a great example of this.

• 1:10:42Speaker 02

  We had a lot of talks, constant scientific communication, discussions back and forth, even when we were at each other's throats and the, the future of Earth. Hung in the balance, right? We still did those things. And it doesn't make sense.

• 1:10:56Speaker 04

  Can you educate me? What is a track to talk?

• 1:10:59Speaker 02

  The track to are when people who are former government officials, usually not in government today, are over there talking. Like, I've done this a lot on the military side. As a former Department of Defense official, I would go to Shanghai or Beijing, and you would talk to also former defense officials there. We still have lots of friends, and so we would talk and discuss the American perspective. They would give the Chinese perspective. And then you'd come back and you'd tell the people who were in office, like, you know, this guy who's still best friends with the Defense Minister, you know, believes this, and here's his take on things, or his interpretation of American actions. So it's a low stakes way for people in governments to talk to one another. They usually are the preview for kind of talks among the principals themselves.

• 1:11:44Speaker 02

  So that's what a track to talk is. And we should be doing those in AI. And people are trying to do that. You know, getting Stuart Russell or Ben Gio over to China to talk about the Western perspective, to learn about how do the Chinese think about X risk, about discrimination, about the military applications of this? Like, what did they think? Because I think, you know, coming back to like the Soviet example, which I've brought up several times. There was a time in my life when I was fascinated by this question. And they were opening up the archives after the Soviet Union fell. And a lot of historians were going and writing these alternative histories of the Cold War, looking at the actual papers of the Soviets. And, you know, they were like evaluating, what did the Russians really think about this? Like, we had this view in America of how they interpreted the world. And like, what did they really think?

• 1:12:29Speaker 02

  Like, what were the Politburo minutes about this? Or what was the, you know, the army writing about this kind of stuff? And it turns out, we got it all wrong, mostly, right? We, we, we totally misinterpreted a lot of what the Soviets were doing, and they saw aggression where there wasn't, or we saw nothing when there actually was aggression. And to me, that's like, I think, a powerful lesson about the Chinese. It is a different culture, one rich, proud, vibrant, and on the come, right? And have no illusions. We actually understand, like, what they think about these things. And don't fall into the trap of either marrying yourself or also just presuming, right, that they're, you know, nefarious and evil minded, you know, and are really to

• 1:13:11Speaker 02

  destroy the world to achieve their ambitions. Recognize that they're a competitor minimally, an adversary maybe, right, but try to like, understand them better. And these track two talks are part of that. But across the board, you know, Americans should be trying to understand China a lot more. That's a huge weakness for us. I do think we're in this stage now of where we are just kind of projecting our worst fears onto them.

• 1:13:35Speaker 04

  I mean, don't you think AI could could go a long way to help in diplomatic modeling? I mean, or diplomatic understanding? I mean, why not? Like, it seems like these tools could be directed at, like I said, preventing war and and furthering understanding rather than just fighting war, right?

• 1:13:53Speaker 02

  I think we I'd love to see that done. It's not a typical American constituency. Like, what agency of the government does that? Perhaps the State Department or someone?

• 1:14:01Speaker 04

  Yeah.

• 1:14:02Speaker 02

  You know, Department of Defense, right? Everything's a weapon for them, right? And that's how they see see these kind of tools. But yes, I agree. Finding ways to come to understanding. Yeah, I mean, there's like a school of thought. You could think like in a different place in the multiverse where we treat the development of AI like we do public health. You know, we share immunizations. If the US or China cured cancer tomorrow, of course, we would share with China and Europe and Africa and the globe. You know, it's an amazing, it would be an amazing advance and you would share it. And they in turn would probably share it with us. And the idea of being like, you could see AI going in that direction, you know. It doesn't seem to be the trajectory we're on. But nonetheless, it's it remains kind of a distant dream of mine that that is like how we would think

• 1:14:47Speaker 02

  of the problem, or at least open ourselves to the possibility of that without defaulting into this kind of arms race mentality where AI is a weapon and we have to master it, you know. A race to what? What does victory look like in this arms race? What lane am I in? You know, what kind of trophy do I get when I achieve it? You know, do I get a Romulan cloaking device, you know, on this that for that forever? I cannot be challenged? Like, what is what's AI going to do that renders war obsolete? You know, what does winning look like? That's why arms races historically are a byword for disaster. And we should get ourselves out of an arms race if we're in one.

• 1:15:23Speaker 04

  Speaking of which, I know that one of the pillars on the site is upskilling government. And I'm curious, I mean, of course, there are the testimonies that happened. Those usually seem more performative rather than, you know, anything else. But I'm curious what actually happens in the halls, you know, behind the scenes. Like, how do, how do congressmen learn more about AIs? Is it just, you know, educating yourself by going to YouTube and watching, of course, machine learning street talk? Or do you have experts kind of milling around in the hallways there to, there to discuss topics with you, answer questions, you know, deep dive anytime you want? What's it look like?

• 1:16:02Speaker 02

  Well, one observation about the state of play and then how people try to get around the problems that the state of play presents to them. So I remember when I was first elected to Congress, 20 years ago now, the Congressional Management Foundation gives you this book, kind of like, how to be a congressman, how to run your office, what to expect day to day. And I remember reading in the back, they had a survey of the existing Congress, and they said, how much time a day do you have to read and to get smarter about issues? And the answer was 17 minutes. And of course, members of Congress are dealing with everything in domestic policy, international policy, local questions. So they're overwhelmed by just the number of subjects they have to understand a bit. So it's a world, I tell folks, if you want to go into politics, develop your human capital

• 1:16:44Speaker 02

  before you show up, because you only draw it down once you're there. That being said, all these technical questions, there's been a huge effort from everybody from the AAAS to various nonprofits to put technically minded people on the Hill, fellowships for mid-career scientists, maybe to leave their academic position, or even industry, come work on the staff of a House member or senator. You know, industry's paid for some of those, various nonprofits have done so, very important. And so now, if you go to most of the offices in the Senate, they'll have a fellow there who has some kind of superlative education, you know, PhD from an elite school, expert in computer science, machine learning, biotech, perhaps, you know, who are there to advise

• 1:17:26Speaker 02

  them about this, both to get smart about politics when they go back out, maybe to their commercial or academic position, but also to give advice. And so you have that going on as well. And the halls are swimming with people now, for better or worse, lobbyists from the tech companies, but also folks from nonprofits. Civil society now has a huge presence on AI debates in DC. So there's a lot of incoming messaging, but it does come back to that constraint I mentioned. Their time is very, very limited. You know, it's going to be very hard to become an expert on AI when you're in office. So you either have to come in with that kind of background or have really good staff or make it such a priority for yourself that at the expense of all the other issues, healthcare and transportation, you know, that you choose to do that. There are a couple of members like that.

• 1:18:11Speaker 02

  Don Byer from Virginia, you know, terrific member. He's probably in his seventies. He's going back right now to George Mason University and getting a PhD in machine learning. And he's a very successful guy, very wealthy because he's interested. Now, that's somebody who's like made that a focus. You know, but other people, you know, you come to Congress because you care about something else. And so you have to have that staff around them and building up a staff ecosystem and a civil society ecosystem to combat and sometimes complement the lobbyists from the tech sector itself has been, I think, an important part of what the last couple of years has brought to DC.

• 1:18:44Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 1:19:14Speaker 01

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid and risked 350,000 Virginians healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com. Not authorized by any candidate or candidate's committee.

• 1:19:43Speaker 04

  Okay, so and I see that individual, you know, congressmen and women would have, you know, staff and people who may be experts, but is there also like a shared pool, almost like an internal think tank that that all of them can access to, to, I don't know, vet ideas or talk about the impacts of policies, a shared pool of experts or no?

• 1:20:06Speaker 02

  Not really. Certain segments of each party will have something like that. Like, there's a Republican Study Committee for very conservative members of the Republican Party who has that kind of shared infrastructure, their own staff. They're almost an in-house think tank, if you will. But that's rare. Newt Gingrich, when he took power in 1994, got rid of, there was an Office of Technology Policy that was kind of the in-house congressional think tank on tech policy, but he got rid of that in 1994 and it's never been brought back. So that's the challenge. It's like, there's no outside, but congressionally kind of authorized group to do it. Congressional Research Service can provide you background information. Again, you have access to a lot of people in civil society and think tanks, but there's nobody, you know, outside your own staff, really, who's thinking big

• 1:20:51Speaker 02

  thoughts. And a lot of times your staff is also caught up in the minutia of, you know, day to day, kind of fighting and stuff. So that's, I think, one of the challenges, is that there's no kind of brain trust that can think big thoughts and then inform the parties about that.

• 1:21:06Speaker 04

  And do you think this is a, this is a gap? I mean, is it something that we should have?

• 1:21:10Speaker 02

  No, it is a gap, for sure. You know,

• 1:21:12Speaker 04

  Okay.

• 1:21:13Speaker 02

  It's something that, again, that used to exist. It's been taken away as these technical questions become even more important than they were 30 years ago. Yes, we have to have groups like that who are congressionally chartered, not subject to influence either by civil society, where you have philanthropists giving money there. Sometimes they're slanted in their views. The lobbyists, of course, are carrying the water for their respective companies. But to actually have someone like, hey, you're in the public service here, you're on the government payroll, your job is to think these kind of ideas through for it and give us good ideas. I think that would be a powerful, powerful tool. We need to do that. It would make our government more effective. Again, you have groups like Republican Study Group, you know, kind of the Progressive Caucus in the House of the, of the left-wing members. Small groups that do this, but you don't really have one that's incredibly high powered with

• 1:21:58Speaker 02

  you know, with really the best people in it. You have so many people who listen to you who are influential in the industry. And, you know, I think I am in many ways somewhat gloomy about the way this is all going to go for us. Because whether our democracy can handle this incredible technology, whether people can find meaning where work is scarce, whether the benefits of AI are going to be, even if not equally distributed, at least widely distributed. You know, I think there's very little in recent American history that thinks we're going to get those kind of questions right. And it does go to, you know, your question about what one should study and what disciplines are important. You know, I argue with people like Seb Creer at Google DeepMind or Dean Ball, who was at the Trump administration.

• 1:22:41Speaker 02

  You know, I find like we agree about the technology exactly. Where we disagree are about these questions of like, how does government respond? What can government do? What should government do, even if capable? You know, like, what's the role of government in a society like ours? And these are really live questions that are actually the most important ones today. And I think the big fear that I've had, and so drives a lot of the work we do here, is that the AI industry can be its own worst enemy. People loathe it. I see polling every day, political polling. It's deeply unpopular, and that's not a good thing for our country. And there's a lot of folks who have pitchforks, who are just over the horizon, coming after this whole sector.

• 1:23:25Speaker 02

  Many of them would shut it down entirely. They want to do so because they don't give affirmative answers to those questions I answered earlier. They're like, this is a project of the elite, for the elite. They're building a data center in my backyard that's going to maybe change the environment, maybe raise my electricity prices, and whose sole purpose is to take my job away. And then I turn on the TV and there's a lab leader saying, I'm here to like disrupt your world. And, you know, irrevocably disrupt your world. And I think the fact that society, the US, has lost faith in the project is extraordinarily damaging to what is a very important project with many, many, many upsides. And that's just, we're in a perilous place.

• 1:24:09Speaker 02

  And if we don't do something to regain that trust, there's going to be a lot of people, you know, who are radically opposed to this project and do their best to, if not shut it down, stymie it. And that's why I said, I think this next few years are really important. And to your listeners who are probably doing machine learning day in, day out, their voices need to be heard. They need to be heard inside their companies to advocate for the right kind of public policy. They need to be heard outside of it. And their task, all of our task perhaps, is convince Americans that, yeah, this is actually a good thing. Because the truth is, right now, most Americans don't think it's a good thing.

• 1:24:48Speaker 00

  Imagine building an app with your name on it. With Base44, describe what you want and watch it take shape. Plan, customize, and expand in any direction with payments, data, security, and hosting built in. Go to base44.com.

• 1:25:03Speaker 05

  USAA knows dynamic duos can save the day, like superheroes and sidekicks, or auto and home insurance. With USAA, you can bundle your auto and home and save up to 10%. Tap the banner to learn more and get a quote at usaa.com/bundle. Restrictions apply.