Anja Manuel on AI, China, and national security

What role can AI play in international diplomacy and national security? In this episode, Reid and Aria sit down with Anja Manuel, a former diplomat, author, and one of the leading voices in AI and foreign policy. She has the unique ability to bridge the gaps between technology, policy and diplomacy. They discuss international AI governance, China, potential threats of bio-terrorism, and the strategic decisions that the US and its allies must make to navigate this complex landscape of innovation with regulation. For more info on the podcast and transcripts of all the episodes, visit https://www.possible.fm/podcast/ For the referenced Possible episode featuring Kim Stanley Robinson: https://link.chtbl.com/8YMsBlzg Topics: 1:21 - Episode introduction 3:44 - Hellos and intros 4:38 - Where the U. S. and China rank in the Great AI Race 6:09 - Deepseek explainer 7:27 - Which technologies the U. S. should prioritize 9:05 - Why it’s important for the U. S. to stay in the lead 10:41 - How to maintain that lead 16:27 - Current examples of AI in national security work 18:30 - Impediments to further integrating AI 20:13 - How to connect national security to cutting edge technology 24:35 - The potential risks to incorporating AI 28:50 - Midroll ad 28:55 - How to stay safe against the risks 32:05 - The helpfulness of AI Safety Institutes 33:25 - International treaties for AI 36:11 - The current U. S.-China relationship 38:50 - How to return to a more positive relationship 41:28 - The economic race 43:25 - Anja’s advice for the new administration 45:01 - Key players and swing states 50:24 - How to make a cyber treaty work 52:57 - Rapid-fire questions Possible is an award-winning podcast that sketches out the brightest version of the future—and what it will take to get there. Most of all, it asks: what if, in the future, everything breaks humanity's way? Tune in for grounded and speculative takes on how technology—and, in particular, AI—is inspiring change and transforming the future. Hosted by Reid Hoffman and Aria Finger, each episode features an interview with an ambitious builder or deep thinker on a topic, from art to geopolitics and from healthcare to education. These conversations also showcase another kind of guest: AI. Whether it's Inflection’s Pi, OpenAI’s ChatGPT or other AI tools, each episode will use AI to enhance and advance our discussion about what humanity could possibly get right if we leverage technology—and our collective effort—effectively.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Speaker 00

  If everything breaks our way, the US and China will find a way to get along as two great powers. And think about what would happen. Here's an alternative future. Both countries are comfortable enough with each other that we can pull back our military stances. We have to spend less money on our militaries. That frees up resources for pensions, for healthcare, for collaborating again on innovation in science and technology, all of the things that are going to help all of humanity. Goods flow more freely between the US and China and the rest of the world, and we get on a pretty positive path. It feels even almost naïve to say that,

• 0:44Speaker 00

  but I think unless you put a big positive goal out there, you'll never achieve it.

• 0:56Reid Hoffman?

  Hi, I'm Reid Hoffman.

• 0:57Aria Finger?

  And I'm Arya Fingur.

• 0:59Reid Hoffman?

  We want to know how together we can use technology like AI to help us shape the best possible future.

• 1:06Aria Finger?

  We asked technologists, ambitious builders, and deep thinkers to help us sketch out the brightest version of the future, and we learned what it'll take to get there.

• 1:16Reid Hoffman?

  This is possible.

• 1:21Aria Finger?

  The year is just getting started, but already some uncertainties are present in the conversation: political division, economic security, and geopolitics. And people seem concerned. In late December of last year, Gallup reported that the majority of Americans anticipate more conflict, not less, in these three areas in 2025.

• 1:42Reid Hoffman?

  And at the same time as AI continues to advance, it's clear that this technology will deeply intersect with those three areas: international security, political division, and economic stability. All influence the advancement of AI, both inside and outside the US. That is to say, if you're concerned about those things, AI should also be on your mind. So where do we stand? And what should we be looking out for on the foreign policy and international stages?

• 2:16Aria Finger?

  Few understand these topics better than Anja Manuel, a former diplomat, author, and one of the leading voices in AI and foreign policy. Anya has had a remarkable career thus far. She's the co-founder of a strategic consulting firm alongside leaders like Condoleezza Rice and Robert Gates, and she currently leads both Aspen Strategy Group and Aspen Security Forum. That's two of the most influential spaces driving the conversations on AI and its impact on national security.

• 2:47Reid Hoffman?

  Anya brings a wealth of knowledge on how AI is reshaping the geopolitical landscape, whether it's the global race for semiconductor supremacy or the potential threat of autonomous weapons. She has the unique ability to bridge the gaps between technology, policy, and diplomacy. Her critically acclaimed book, This Brave New World: India, China, and the United States, explores these global dynamics on a grander scale.

• 3:18Aria Finger?

  In today's episode, we'll explore her insights on international AI governance, China, potential threats of bioterrorism, and their strategic decisions that the US and its allies must make to navigate this complex landscape of innovation with regulation.

• 3:35Reid Hoffman?

  Here's our conversation with Anja Manuel.

• 3:44Aria Finger?

  Anya, let's start with a little bit about your own background because it is pretty unique. Like, you grew up in a diplomatic family. You spent a better part of your childhood in Pakistan, near the border of China. Do you have a story from your upbringing that you think helped shape your view of the world?

• 4:00Speaker 00

  That is such a great way to start. Yeah, I did grow up, I'm German, but I grew up partly in Pakistan, actually near the border with Afghanistan. And we had an Afghan refugee camp in our backyard because it was during the Soviet conflict. And I also grew up with a German grandfather and an American grandfather who fought on opposite sides of World War II. And so that is what got me interested in diplomacy and foreign policy and thinking that war is the dumbest possible way for humans to resolve their conflicts. And it's, and it's that simple. That's why I got into this, into this world.

• 4:38Reid Hoffman?

  So let's dig in. You've said on AI development that countries with the technological edge tend to lead the world. Where would you put the US in the great AI race? Where does China stand? And how do you see the jockeying for AI leadership between these two countries playing out?

• 4:58Speaker 00

  Well, the race is on. Oh, my gosh. And actually, when I write about technology and countries leading, I mean it even broader than that. So if you start with Roman roads, Chinese gunpowder, British steamships, if you look at what countries end up being the most powerful in the world, they tend to be the ones that are technologically ahead. And as the United States, we, with our friends and allies, have been the undisputed leader for the past three, four decades. And now the Chinese are giving us a run for our money. And maybe, let's start on AI, but I'm happy to talk about some of the other technologies, too. So on AI, I know you all follow this really closely.

• 5:39Speaker 00

  I think everyone was a little bit shocked when the latest DeepSeek model came out. Apparently, and all of our friends who are testing it say it performs extraordinarily well, on par with some of our own best foundation models in the US. And it was trained using much less compute, using the chips that were just under the border and not export controlled, and much less expensively than what we're doing with our training runs here. So never count out the Chinese.

• 6:08Aria Finger?

  And can you say a little bit more for folks who haven't been totally paying attention? Tell us a little bit more about the DeepSeek model and how were they able to do this more cheaply? Is it just because they're ahead of us? What is the state of that game?

• 6:22Speaker 00

  Yeah, I think people are still figuring it out. And of course, DeepSeek isn't the only, it's a brand new company. I think it was created in 2023. They have only about 100 employees. And people were really quite surprised. There's also other fabulous Chinese AI companies. Jipeo, all of the big Chinese tech firms also are racing in the large model, foundation model world. But I just want to take a step back because the- Podcast we're doing today is around AI and national security, technology and national security. Everyone focuses on the foundation models and the race that's on there, and that's super important. But in my view, when you look at how AI intersects with other technologies, in all

• 7:04Speaker 00

  of those key areas that could be dual use or even military use, the Chinese are doing great. They're amazing at computer vision, they're unbelievable at surveillance, they're unbelievable at sensing, they're unbelievable at autonomy and drones and advanced manufacturing. So even taking aside what's happening on the foundation model side, they're doing pretty well.

• 7:27Reid Hoffman?

  And what do you think are some of the things that it's really important that we do as the US, as America? Obviously, we have some pretty strong assets in Silicon Valley, large-scale tech companies and some other things, but on the full stack, as you're going through, like, you know, the kind of call it the modern world of technology broadly, we have a lot of deficiencies. What do you think is important for us to do?

• 7:59Speaker 00

  You know, AI is what everyone is focused on. It's front and center. It's amazing. It's so fun to live in Silicon Valley right now with this Cambrian explosion. But there are other technologies that matter. Different people that you ask would do the rankings slightly differently, but for me, here are the technologies where we, as the United States, with our friends and allies, need to remain world-class and/or in the lead. So obviously, semiconductors, because it's the building block for everything. Artificial intelligence, we've already talked about. General-purpose AI, it's going to drive all of the innovation of the future. 5G and 6G, because that is the rails on which everything is already running, the Internet

• 8:40Speaker 00

  of Things, self-driving cars, lots of back and forth with the AI models themselves. Bio doesn't always get put in that group, but I think it's very important, especially in how it can intersect with national security. And then finally, I would put on that list, partly because I work with some of these innovative fintech companies, financial technologies and the economics rails on which value gets sent.

• 9:06Aria Finger?

  Well, can I actually step back and ask, and maybe this is a naive question, but I can imagine some of our listeners are thinking to themselves, why does the US have to win? Like, when we talk about Silicon Valley and technology, we often talk about nonprofits like Endeavor or Kiva, who are building entrepreneurs around the globe, and we want so many places to be technologically advanced so that, you know, there's innovation happening everywhere. So, it might be a simple question, but why is it important that US stays in the lead for all these technologies?

• 9:37Speaker 00

  It's important, first and foremost, because if we are in the lead, we control the values. And that was really automatic. You know, when you had the beginnings of the internet in the 90s, you know, when Reid and I were in college in Silicon Valley, it was just automatic that the US was in charge, and it was this very optimistic view of, oh, the internet is going to open the world, it's going to create democracy. And even the early days of social media, people are like, well, look, we're helping the 2011 revolutions in the Middle East. Our technology is going to bring democracy to the world. Well, it turns out that all of these technologies in themselves are neutral, and they can be used for good and for bad.

• 10:21Speaker 00

  And so, the more we are, and I also, by the way, Aria, it's a really good question. I don't think we need to be in the lead in absolutely everything. That's why I tried to be a little bit narrow about the key ones. But I think on those that really intersect with national security, it's very important that we drive the values.

• 10:41Reid Hoffman?

  I completely agree with everything you just said. And it's, you know, the part of it is not just kind of military power, not just economic power, but also kind of the question of the values that are engaged in the technology that provides the kind of the infrastructure of life, right? Like, kind of like how you think about the world, how are people enabled? How are individuals enabled? You know, like, a classic kind of question is, you know, what's your view of human rights? Or the kind of the human condition for that? It's not just the pure, you know, kind of thing of being ahead in what I'm calling the cognitive industrial revolution, which I think is very important for the prosperity

• 11:22Reid Hoffman?

  of our country, you know, our, our, our children, you know, generations to come, but also the question about like, what kinds of societies flourish in the global world and how that kind of human condition evolves. And now to return to the question of, you know, well, what, what should be the kinds of things that we as thoughtful leaders in kind of, call it, Western democratic traditions should be doing, whether we're business people, technological innovators, government folks, you know, intelligent, you know, strategists like yourself, press people, like, what should we be doing to try to advance the ball?

• 12:04Speaker 00

  So we talked a little bit about why it's important for us to stay in the lead on some of these technologies, on the soft power way, right? The hard power way is we are really in an absolute race with the Chinese in particular, but to a lesser state with the Russians around technology being used for hard power. So we'll get back to that in a second. But so what should we do to win for both of these? Um, look, I give both the Trump and the Biden administration a lot of credit. I know it's hard to say that I'm being bipartisan. Um, the Trump administration really called out how that the race was on and that the race was on with China and that we needed to think about it. And they started

• 12:49Speaker 00

  the beginnings of trying to put a moat, if you will, around some of our technologies. You know, they were tough on SMIC with, which is a Chinese semiconductor manufacturer. They took the first steps to move one of the TSMC, which is another big semiconductor manufacturer, to Arizona. The Biden administration really doubled down and they did it in two ways. One,

• 13:15Speaker 00

  Hobbling the other guy so we can run faster. So that's all the export controls you've seen. You know, another huge swath came out just this fall. And it's not just on the most advanced chips, but it's on semiconductor manufacturing equipment, the components of that. They try to be tough. I think that approach is, was important, but it's reaching the limits of what it's going to be able to do. So to really answer your question, Reid, now it's about us running faster. And thank God, we have some of the most innovative people and companies in the world. I mean, the billions and billions of dollars that are being invested in R and D, that's AI specific, that's semiconductor specific.

• 13:59Speaker 00

  All of these things is amazing. However, we could do more to coordinate our efforts. We could do more in the government. I'll give you just some examples for government. Permitting for electricity. Nightmare. Takes years. Where are we ever going to get the electricity and the data centers and the GPUs to do all of these training runs? Now, people are already talking about they're going to be faster, cheaper. And as you saw with DeepSeek, clearly it can be done. But that is one place where I would start with a new administration of what are we going to run faster? A second piece is something that only the US government can do, really frankly, better than any other government on earth,

• 14:42Speaker 00

  is fund basic research and development. Basic science. In the Cold War, we were spending up to 2% of our GDP, on how you count on that. Now it is just a fraction of that. And Congress did an amazing thing in a bipartisan way. They passed the Chips and Science Act. All we ever heard about was the chips part because that got all the lobbying. But the R and D portion of that was supposed to put $170 billion to work over five years in basic R and D. And that has been chronically underfunded. So I hope that this next administration will go in and with, starting with those two important things, they could do a whole lot of good. And I'll give you one more. And that is

• 15:27Speaker 00

  If you think just about the labs creating the foundation models, it's an imperfect analogy, but I sometimes think it's as if multiple companies, mostly American, but some Chinese, some in the Middle East, one in France, all created nuclear energy at the same time. It can be used for good and it can be used for bad, and it's not really controlled or controllable by governments. But so one thing that our government should be doing is helping our companies make sure that their cybersecurity is up to snuff, that they can look for insider threats, people who are researching, who might be spying for foreign governments, we know it's happening.

• 16:09Speaker 00

  And then a super basic one that I'm amazed, there are laws that if one of our companies fires someone because they're an insider threat, they can't tell their competitors. So that guy just gets hired across the street two weeks later. So those are really pretty basic brushing and flossing things that we should be doing.

• 16:26Aria Finger?

  And so are there other ways? I think, you know, when people talk about the dangers of AI, a lot of people actually go to defense. They say, oh, my gosh, we're going to have these autonomous drones, AI is going to be doing all these things, and people get nervous. Yet at the same time, there's so many technological innovations or just actually getting rid of like red tape bureaucracy, waste that is created right now in our government and the Department of Defense. Are there examples you've seen of AI being integrated right now into national security work? And are they using these AI services in the Department of Defense? Or what would you like to see?

• 17:02Speaker 00

  We're starting, but it's a trickle, not a flood. You know, we could be doing a lot better. There are some really cool pilots happening within DOD. One that I thought was hilarious is they're using AI just to analyze the, you know, enormous stack of regulations that they have to follow for every single thing they do. And the AI says very quickly, oh, you can do this or you can't do this. Sort of ridiculous. But that's a good one. And as you know, there's the Replicator Program, which has been put in place under the leadership of Kath Hicks. That's going enormously well. We're from basically a standing start.

• 17:40Speaker 00

  We're now doing a lot to both manufacture and acquire inexpensive autonomous weapon systems and integrate them with everything else we're doing. So we're starting to do it both in the tooth, which is what we in the national security community call it, in the weapon systems, and sensing, multi-sensing, how do you react faster? How do you command and control better? All of that is starting to happen. And in the back office, like making production more efficient. Here's another one for you. Just using AI to understand when you need to do maintenance, so you don't do it too fast and you don't do it too slow. I mean, there are just so many processes. This is a huge lumbering organization.

• 18:24Speaker 00

  Over 3 million people, $850 billion budget. Oh, my God, could AI make it more efficient?

• 18:29Aria Finger?

  I love how like the things that need to be done, to your point, are like some of the boring stuff, like any bureaucracy can be made more efficient and AI will help us do that. What are the impediments to doing it? Is it legislation? Is it financial? Is it just stasis? Like, what do we need to be able to move faster in this regard?

• 18:47Speaker 00

  So, Ari, you asked about the impediments. There are too many to count and list. And it's everything from this is an organization that is extraordinary. The people in the US Pentagon are patriotic. They are kind. They are good human beings, and they have the welfare of the United States at heart. And they are not on the cutting edge of technology. And I know, Reid, you tried at the Defense Innovation Board. I've tried at the Defense Policy Board. There's just problem after problem. Very hard to get technologists in there, in the acquisition side, in the user side, very hard to get people trained up, very hard to get people allowed to experiment and

• 19:31Speaker 00

  use AI. And then it's impossible to get money to do it because the procurement processes are slow and painful, and there's a lot of red tape. And so it's just problem after problem. So when you look at some of the young Cool defense tech startups that are making inroads. What they tend to do, and this gets really national security nerdy, is they go to one of the combatant commands, like basically the special ops command, because they're more tech forward, and they get small contracts with them, and then they try to get into the mothership. But every secretary of defense that I've ever interacted with has tried to reform that process, and I know the Trump administration is going to try, and, oh, boy, it's not easy.

• 20:13Reid Hoffman?

  And, you know, part of this is this question, which, you know, we've talked about before, including at the Aspen Security Forum, which is, you know, how do we get kind of the tech-forward companies and industries that the US excels in better connected to the Department of Defense's mission and engagement? Because the procurement process is a terrible way of doing this. The prime contractors are so far behind in software, I don't know what decade they're in, and that, of course, goes to everything else. And so what do you think about, like, you know, it's the startup contenders, it's, you know, the scale-up contenders like, you know, Anduril and Palantir, it's the, you know,

• 20:57Reid Hoffman?

  the hyperscaler companies and other startups. What's some of the ways that if historical constraint weren't an issue, you know, you would be saying, here's kind of ways that we can be bold to try to leverage these American assets for the kind of American capability?

• 21:16Speaker 00

  Right. So Mike Bloomberg and a small team is just out with a report on exactly that. They have some really good recommendations. My out-of-the-box one that I always tell to DOD, and they sort of smile, and they're not going to do it, is why don't you hive off a super tiny part of the acquisition budget? The R and E, so the acquisitions, we spend $280 billion a year as the Pentagon buying things. About half of that is things you need for soldiers: housing, food, transportation. The other half is research and equipment, so that's the actual arms. Okay? Of that, let's say you take 1%. That would be $1.4 billion.

• 21:59Speaker 00

  And you treat that like you would a venture portfolio, and you tell the people who are in charge of that, make some investments.

• 22:09Speaker 00

  Buy some stuff. We're going to assume that nine out of 10 of these things are not going to succeed and that 1 in 100 is going to be a total game changer. And just get the muscle going for people to think a little bit differently. So, you know, we'll see. Maybe the new team coming in will push on that. I'm actually pretty optimistic that some of the people who are going into the Pentagon are people who've been in industry and been in tech. So hopefully they can drive a little bit of change.

• 22:37Reid Hoffman?

  You know, part of the question will be is how do we get the organization to rate itself based on some amazing successes, not in the average result? And what do you think the necessary thing to try to get that org culture evolution, even for the small part, into the right shape?

• 23:04Speaker 00

  My guess is, and you and I can riff on this all day long, you need to hire some new talent and put them in with the current talent. Like in a way that when you saw DIU version 1.0.

• 23:18Reid Hoffman?

  Pi, could you explain?

• 23:20Speaker 03

  In the context of American government, DIU stands for the Defense Innovation Unit. It's an organization within the Department of Defense that helps the military adopt new commercial technologies more quickly. It was founded in 2015 and has been called the Pentagon's Innovation Experiment.

• 23:39Speaker 00

  They brought out to Silicon Valley a bunch of really well-meaning, smart folks from the Pentagon who had no clue how the Valley worked. And then they moved to version 2.0 and they had people who are in the Valley and who are making inroads. And now we're kind of on version 3.0 and it's working pretty well. But I think it's small, like all cultural change in organizations. You've led lots of big companies. It just is small and incremental. And you get there by notching up a few wins and then driving leadership from the Secretary of Defense level. And I would tell you, Bob Gates, who's my business partner, we've been having lots of conversations about DOGE, you know, Department of Government inefficiency and the new things happening at DOD.

• 24:21Speaker 00

  And his view, which I think is very wise, is you can drive change, but it has to happen from the Secretary of Defense level. And you have to bring your organization along. You can't just shove it down their throats because otherwise you will just build too many antibodies.

• 24:35Reid Hoffman?

  So you know, obviously, part of the thing that we're navigating this technology is there's both huge opportunities, but also risks and things to navigate. And so what's your kind of, you know, kind of, we've been, we've been addressing some of the opportunities and some of the things we can do. What are some of the risks that you particularly think are important, especially within the national security arena and the, you know, kind of US position in the world? And what are the things that we need to do to kind of, you know, potentially start thinking about navigating them?

• 25:03Speaker 00

  Just how AI is going to intersect with a lot of other technologies to do an enormous amount of good in the world. It will intersect with other technologies in ways that are deeply worrisome from a national security perspective. And let me give you a couple of examples. Cyber, you're already seeing it happen. Soon, if not now, the best AI models will be able to write their own malicious code. They're already doing that. What you've seen in the last few months is an insane uptick in how good and tailored and specific phishing messages are. I mean, I feel like I get them every single day from fake DocuSign. Don't click on those. And so that's kind of the beginning.

• 25:47Speaker 00

  And you'll also then, of course, see the, the countervailing the defense side. I know some of our cyber companies, Palo Alto Networks and others, are already totally revamping their product suite so that the AI cyber defense, the cyber defense is AI-enabled, just how the offense and the bad guys are AI-enabled. So that's one. Two, how AI intersects with chemical and biological weapons? Whenever I say this, people say, well, don't you still need a web lab? And it's hard. Yes. But the difference here is it's not like someone tinkering and looking up a Google search of how do I make a bioweapon? It's already now like having a PhD student perched on your shoulder and allowing real

• 26:32Speaker 00

  novices, access and non-state actors, really unsavory folks, access to knowledge and technology that would have been unfathomable. Two years ago, frankly. So that's chem and bio. Then autonomous weapons. We started talking about it in the positive way, and I do think this is, in large part, really positive. You are seeing an unbelievable race. Like, I've never seen a scramble happen so quickly. From when you first started using, the poor Ukrainians were using essentially hobbyist drones. And then the Russians were using electronic warfare to get those out of the sky. And so they're using better ones. And like, the amount of innovation that has happened in the autonomous weapon space

• 27:16Speaker 00

  in the last year or 15 months has been just unbelievable. So that's all good. And the US, in particular, should run faster on that. It also has a really dangerous downside. Because if the AI is always going to be faster than a human, the incentive is always to escalate, in our terms. So if the AI fighter pilot is faster than our fighter pilot, just shoot them down. Don't wait for the human in the loop. So it's very escalatory. So that's three. Then, of course, we've all talked about, previously, about models jailbreaking their own safeguards. That's an issue, not just for national security, but just in general. And then, people always ask about nuclear.

• 28:01Speaker 00

  I have to tell you, I don't know why, I worry slightly less about this. AI being used in command and control of nuclear weapons. That's what people are terrified about. And deterrence breaking down because of these sensors and because you might be able to get at vulnerabilities and other people's nuclear weapon system. There are people who are very smart and who worry about this all day long in the bowels of the Pentagon. And there are people who are very smart and very worried about this in China and in Russia. And I think one of the great last things that President Biden did is he created an agreement with Xi Jinping just last November of 2024, saying, "Nuclear command and control, there will always be a human in the loop." So that's one important area where we're actually talking to the Chinese.

• 28:55Aria Finger?

  So I'm asking the obvious question. I hear about cyber. I hear about bioweapons and, you know, it makes your brain go to terrible places. What can we do? What are the ways that we can fight back or use AI to make us safer against all of these things that AI makes us unsafer about?

• 29:15Speaker 00

  Yeah, very good point. So A, of course, we're going to be working on good AI to defeat all of these negative things. I think we can do two really important things. One, narrow, national security-focused testing of the largest foundation models. This is what the UK AI Safety Institute did so effectively. The US now has a very nascent, fledgling little AI Safety Institute. I would hope that the new administration coming in keeps that in some form. Look, tear it down, rename it, call it the National Security AI Institute or the Anti-Woke AI Institute, but do super-basic

• 30:00Speaker 00

  national security-focused testing on these issues that I just ran through. Can it jailbreak? What does it do with chem and bioweapons? What does it do with respect to cyber? What does it do? Is it malicious? Is it tricking the humans? Right? Those types of things. And frankly, the large model companies have been really impressive. Almost all of them have voluntarily made their models available for testing. And the testing was happening quickly, days or weeks. So this isn't like an FDA process that's going to slow innovation for months and months. This is fast, and we should keep it going. Second point, we have got to talk to the Chinese. So that's really hard. And whenever I say this,

• 30:44Speaker 00

  people say, well, will they just be milking us for information? A little bit. But I'll give you an example. The US was first to the nuclear race. We dropped bombs on Hiroshima and Nagasaki in 1945. In 1946, the United States came out with what's called the Baruk Plan. It didn't go anywhere. But we basically said, we need to start having conversations about how to control and think carefully about the safety of this amazing new technology. I think that's kind of the era we're in now. Nuclear, it's a very imperfect analogy. But you already have the very beginnings of this.

• 31:28Speaker 00

  You have scientists speaking to each other. Stuart Russell and some others have what's called in our world a track two. So non-government, non-government scientists talking to each other. Reed, you and I are a part of a dialogue with the Chinese about AI. There was a government-to-government dialogue. So far, they're not getting anywhere. But that's OK. Because in the beginning, you just build trust. You make sure that the right people are at the table. You see that the other side has people of goodwill. And then gradually, you move to ideas about how you might make sure that neither of you destroys the world.

• 32:05Aria Finger?

  I think that's good. I think that's a good place to get neither side destroying the world. Love it. You talked a little bit about this, that the UK is sort of ahead of the US when it gets to sort of regulating and safety. Do you think the US needs to catch up to where the UK is? Is there a fear of overregulation in the face of, well, if the Chinese aren't regulating and we are, then we're actually falling further behind? How do you see those trade-offs?

• 32:30Speaker 00

  Yeah, so what I described with the UK AI Safety Institute is so narrow. This is not like the EU AI Act, which I think in some ways is good. In some ways, it probably goes too far. It's just focused on where AI intersects with other technologies in a way that could really harm people in the physical world. This has nothing to do with human rights. It has nothing to do with discrimination. All of those things are probably important. But I'm just saying, from my perspective, as a baseline, that's important. I think it's fine that the UK is first. They're more well-funded than we are. We should probably scale up our effort. There have been 10 or 12 little copycat AI safety institutes, including the Chinese have a little baby fledgling one.

• 33:15Speaker 00

  All those guys are now talking to each other. And that is all positive. We should just be doing more of that more rapidly funded better.

• 33:25Reid Hoffman?

  Well, I think part of what also Very fortunately for us and for the world, the UK AI Safety Institute deliberately kicked off was for global exchange. So, you know, the next summit after it was in Seoul, this one coming up in February in Paris. I know that they've had great collaborative relationships, the US and others. So I think it's the right thing. And they're kind of saying, hey, look, let's just try to make sure a very trusted environment to make sure that the safety side of these things at the high end is really well managed. I mean, one of the challenges in international treaties in this arena is you, it's really hard to validate.

• 34:10Reid Hoffman?

  It's really hard to actually know and monitor what each other are doing. And we instituted under nuclear, a whole bunch of different protocols to try to keep, you know, kind of countries aligned, which worked really well in nuclear, seemed to not work quite as well in bio for exactly this reason. So what's kind of the current edge of thinking about making treaties work here?

• 34:37Speaker 00

  Look, I'm an international lawyer by training. I'm a big believer in treaties, international agreements, executive agreements, even if they are sometimes abrogated. It's better to have a law and have some people violate it rather than having no law. However, I think we are so far from a treaty. We just are not there. You know, back to my imperfect nuclear analogy. If this is 1946, the first real limitation on nuclear weapons didn't happen until 1963. And we had to go through the Cuban Missile Crisis and really terrify ourselves before we got there. So what I'm proposing is super narrow. There are just the safety tests, very limited.

• 35:20Speaker 00

  There is talking to each other about safety and how you would monitor safety. How do you do ongoing monitoring of the models? And there's some different, really interesting technological ways that people are thinking about it. And then just having the scientists in particular, and some of the policymakers, get to know each other and trust each other. That sounds really fuzzy, but it's really important. Ash Carter, who's a former mentor of mine, always talked about when he was a young scientist, he would go to the Soviet Union to do these safety talks around nuclear weapons. And part of what happened is They just, the people started liking each other. And so they would call and say, hey, we're worried because this is going wrong. Can you help us with this? And that's what we're trying to establish here.

• 36:05Speaker 00

  It's small, it's piecemeal, but I think it's unfortunately where the world currently is.

• 36:10Reid Hoffman?

  I think one of the things that may be helpful for people is to kind of get some sense of why the soft connections can actually be so important to ultimately doing this. This is one of the reasons why, you know, you've led a number of track two dialogues, doing some now, you know, why I participate. Say a little bit about how, like, because it may, it may seem a little odd to your average person about the importance of track two dialogues and like, how do they function?

• 36:41Speaker 00

  Yeah. Let me actually, that's a really good question. Let me back off and, and talk about how the relationship is currently with China. Because they're only important because the relationship is so terrible.

• 36:55Speaker 00

  Because we went from knowing each other pretty well, you know, five, six years ago, all of us were in China constantly. We knew a lot of the senior leadership on the political side. We knew a lot of the senior leadership on the economics and tech side. And so you had fewer misunderstandings because people were just exchanging all the time. And then if you don't have misunderstandings, you're less likely to come to blows. Okay? During the height of the Cold War, the US and the Soviet Union always had really good military to military communications. So when something in, went wrong, as it inevitably does, everybody knew who in the theater commander calls whom, all the way up the chain to the Secretary of Defense and to the leaders of the countries.

• 37:40Speaker 00

  And that worked pretty well. We diffused a lot of problems. The Chinese are famously very hesitant to do that. And they'll say, when we bring it up, they'll say, well, if we do that, you will just take more risks. We just want you not operating near our shores. We want you out of the South China Sea, out of the East China Sea, out of the area around Taiwan. So stop doing that and then we don't need to communicate. Our view is, uh, this is freedom of navigation. We're going to do it anyways. We need to communicate. And that's one example on the military side. But it carries over into everything else. If we, on this AI track two, or the three or four that are going on in this,

• 38:21Speaker 00

  can start understanding where the Chinese science is going, what they're pushing towards, what they're worried about, can we help them understand that we're not trying to threaten them? And then can they help us understand how they're not trying to threaten us? Because right now, man, if you read the Chinese press, the vitriol against the United States is extreme. And here, it's not quite as extreme. But, boy, you never hear anything good written about China.

• 38:50Aria Finger?

  If five or six years ago, we were in a better place with that relationship, like, do you think it's possible to get back there, to get to that good relationship with some of our, you know, countries that aren't considered allies anymore?

• 39:02Speaker 00

  You know, in international relations, personalities matter. Especially when you have an authoritarian regime. And you have, in Xi Jinping, the most authoritarian leader of China since Mao Zedong. And I firmly believe that if China had continued to have leaders more in the mold of Hu Jintao, the previous ones, or some of the ones throughout the 1980s, who were, you know, they were authoritarians. Don't get me wrong. They did the crackdown on Tiananmen Square. But they were opening the Chinese economy, and they were allowing some rule of law to happen. And I'm going to be really careful here because the US foreign policy community

• 39:47Speaker 00

  has really beaten itself up over the last five years, saying, "Well, we just assumed the Chinese, if they were open economically, then they would declare democracy and they'd become just like us." I think a lot of people didn't actually think that. But if you're a country that's open economically, even if you're an authoritarian regime light, like, you know, Singapore, the United Arab Emirates, there's a lot of regimes like that that we deal with, we would have been fine. And we would have had a much more positive outcome.

• 40:16Reid Hoffman?

  But actually, one of the things also, in addition, like just on the optimistic side on AI, in addition to, you know, what are the capabilities? How does it make us stronger? I think a couple of things I might add in and see how you reflect it. One is AI, you know, part of what is actually very helpful for global stability is having good intelligence and good monitoring, like, you know? And so obviously, AI can add a bunch to that, even within the military sphere. I presume, you know, that would be something you'd go, yeah, that'll work.

• 40:47Speaker 00

  Yeah, there is about to be, or already is, an AI revolution in intelligence gathering. So both the collecting, because now sensors are so ubiquitous between satellites and civilian sensors and everything you have, and then analyzing what has been collected. So that's really great for our intelligence community. It is also really convenient for authoritarian regimes who want to do Orwellian kind of surveillance of their own people, right? So here we have again with the yin and the yang. And it's the same technology. It can be used for really positive things and kind of for darker.

• 41:27Reid Hoffman?

  Agreed. And it's part of the reason why being the technology leader is so important for kind of the course of humanity throughout the globe. Another one is I actually think it's very important for us to win the various parts of the economic race, because too often, and you know, kind of national security and defense, you know, especially within the hollowed walls that you and I have tread on occasion, that the Pentagon, you know, is kind of like, too often considered as like material, physical, you know, hard power, et cetera, and not as much kind of the global economic sphere. And I actually think that the fact that we're leading this revolution on the economic

• 42:08Reid Hoffman?

  side is actually, in fact, a very good thing from a national security point of view. And I'm curious also for your commentary, you know, kind of the pros and cons and the detail of that.

• 42:21Speaker 00

  It is 100% important that we're leading on the economic side. Look, Sometimes when you read our press, they paint a very dark picture of what's going in the United States. We're the fastest growing large economy on earth. Our innovation system is going gangbusters. It's not, not everything is perfect, but by and large, we're doing really pretty well. And if we can translate these huge gains that we're getting in innovation into labor productivity, you already see that there's a really good report out by, I think Accenture and Microsoft that talks about just how much more labor productivity they think they will get and how much it will add to GDP. I think they have something like $3 trillion over 10 years, like enormous.

• 43:06Speaker 00

  And all of that is great because frankly, you can't lead just with military power alone. You lead by being the beacon of the world in your values, in your innovation and your economy. And all of that, I think, is deeply positive.

• 43:25Aria Finger?

  Well, I think related to that, you know, as of this recording, there's a new administration who has come into office. And so what is the advice that you would give to them for how to sort of look forward and deal with this new intersection of AI and national security?

• 43:41Speaker 00

  We talked about it a little bit already, and I think this is, it's advice for the new administration, the past administration. Our most innovative companies could use some help running faster and doing that in a way that is actually safe and secure. So we talked about, one, electricity permitting. Two, making sure that the data centers and the GPUs are mostly either in the United States or in friendly countries. Bringing swing states along, let's come back to that. But I think the Biden administration made a good start on that. We could do probably more. And then on the flip side, helping our most innovative companies, they don't have to be shut down.

• 44:22Speaker 00

  I've heard someone write, you know, you should nationalize the biggest AI companies. That's a terrible idea. I'm more of a small government person. But helping them think through, giving them real advice on insider threats, on people spying, on information sharing without instantly having legal problems, on thinking carefully, frankly, about the antitrust strategy, rather than just having that be, oh, they're too big, therefore, we must break them up. Like, we just need to think about all of these things from a whole of society perspective, rather than in those narrow silos.

• 45:01Reid Hoffman?

  I completely agree on everything you were saying, because in part, it's like, how do we harness these positions of leadership to the benefit of, you know, America, the American, you know, global order, and a set of things. And I think that engaging with having these large, you know, kind of tech companies deploying in ways that are positive. One of the things I think that intersects to is something you gestured at, which I think is very important to also be winning the alliance, the political support, the hearts and souls of the swing states. And I think that's a key thing to do. And so, could you go a little bit more into that?

• 45:43Speaker 00

  So, we're doing really well in AI. The Europeans, much less than we would have hoped, frankly. I would hope that there's more innovation there. But who's doing well? The United Arab Emirates. The Saudis are beginning to. They're putting a huge amount of huge. I mean, if you look at the numbers, they're just staggering 100 billion in a new AI fund in Saudi Arabia. More than that in a new AI fund in the UAE. The UAE has a very good foundation model in Falcon. When I was just there a month ago, you know, all of their best and brightest want to work for G42, which is their big AI company, or want to work at the new AI ministry. And they're really impressive.

• 46:27Speaker 00

  And then there is India. India doesn't have as much money to throw at the problem, but boy, do they have some good engineers and they have some people in the government who are thinking very strategically about it. So for example, the Indians have put together, or they're in the process of putting together a national compute resource in the way that we've talked about here. So scientists can also have access to chips and GPUs. The Indians are actually doing it. So there are some good things happening in all those places. And you could imagine a situation where with our friends and allies, we say, this is going to be one interoperable ecosystem and we'll work with you. So let's say there's a huge amount of geothermal free electricity in Iceland.

• 47:12Speaker 00

  Can you put some of the data centers there? Can you protect them? Maybe. Do they all have to be in the US where permitting is terrible? You know, you could, you can imagine some coalitions that are really positive. You have to be a little careful on the flip side that the swing states are not playing China and us off against each other to get more goodies, which you see in some cases. And so let's cooperate. Let's just not be naive.

• 47:38Aria Finger?

  And just for our listeners who think of swing states as Michigan, Pennsylvania and Nevada, how would you define a swing state? This is, this is a country who has some innovation but has not yet sort of chosen a side? How would you define that?

• 47:53Speaker 00

  Oh, that is a very good point. I would define swing states in the national security context at ones that want to have close diplomatic and economic relations with both the US on the one side and Russia and China on the other side. And by the way, there are a lot of those. Whenever I hear, oh, this is a new Cold War in our press, you know, between the US and China, I cringe because this is actually very little like the Cold War with the Soviet Union. I think that is a very oversimplified analogy because the Soviet Union was self-isolating. They just traded with themselves into the other Warsaw Pact countries. And they had some good scientists, but they were all employed by the government.

• 48:36Speaker 00

  There wasn't really a natural innovation ecosystem. The Chinese are the opposite of that. They are the largest trading partner of something like 120, 130 countries around the world. They're much better at trading than we are. They are unbelievably innovative and impressive, as evidenced by the fact that for the past year, we've all been talking about how far behind they are in AI. And it turns out, they're not really behind. So I would never count them out. And that means, I'm so glad you talked about swing states. Swing states are people who haven't quite made up their mind who they're with, and frankly, in a lot of cases, don't want to. They'd like to be able to work with both of us.

• 49:15Aria Finger?

  And is it in our interest not to force them to? Because if we force their hand, they might not choose us. Like, are we OK with having partners, whether it's India or the UAE, that are sort of choosing both sides? Or do you think we have to make them make a decision?

• 49:29Speaker 00

  Not in all things, but in certain, very narrow things that impact our national security. I'm sorry, you're going to have to choose. If you want our most advanced chips and you want to build models there, then you should not be leaking those chips to China. And you should not have, you know, just like we were talking about, safety for, by safety, I mean, safety from spying, cyber, all these kinds of intrusions. For our companies, I think our allied and friendly state companies should also be taking the same precautions. I mean, I'll give you a perfect example. My son, who's in high school, spent last summer working at the Pasteur Institute in Paris. Like, every researcher was Chinese,

• 50:13Speaker 00

  which is great. On the one hand, that's wonderful. You want basic science interactions. But is there also a little bit of checking what's going back, what shouldn't go back? It's a hard problem.

• 50:24Reid Hoffman?

  Yeah, the, I mean, I do think that the question of, you know, the track two and building bridges, et cetera, is extremely important. It's actually even more important with the global swing states.

• 50:39Reid Hoffman?

  One, kind of, given the international lawyer question, I think I'll kind of refocus on the cybersecurity thing. Because to some degree, my optimism about an AI global treaty probably would be preceded by optimism about a global cyber treaty. So, you know, what would be, if we were going to try to do a cyber treaty, what would be key things to try to make that work? Because I think most people don't realize that essentially there's almost like a state of piracy and war in the cyber environment. And it's weird just because it's not visible.

• 51:14Speaker 00

  The cyber environment is very unusual because the incentives are all to use cyber weapons all the time at a low level. Because attribution is so difficult. You don't ever quite really know who's doing it. Often, the people who are targeted by it don't want to admit that they were targeted. We don't have very good information sharing. Or if we do, then we punish companies for saying that they've been hacked. So it's a particularly difficult one. You know, if I were running the world, what would be in a cyber treaty would be, everybody abides by the Geneva Conventions. So the Geneva Conventions, very famously, are these international law treaties

• 51:59Speaker 00

  put in place about 100 years ago, really after the end of World War I, to talk about things that are beyond the pale. So killing civilians is beyond the pale. Not returning POWs, shooting POWs is beyond the pale. Bombing civilian infrastructure when those people aren't combatants. It's things like that, that, by the way, in cyber war, everyone's violating all the time.

• 52:25Reid Hoffman?

  So if you were to be able to wave a wand, how would you try to make that work?

• 52:30Speaker 00

  I think we're just not there yet. It's just too, it's too difficult. And actually, now I'm going to sound really tough. I think we haven't really brought the fight to our adversaries who are attacking us all the time on the cyber side. I think we are much more hesitant to do what's called offensive cyber operations than they are to us.

• 52:54Speaker 00

  And maybe if they get thumped a few times, they'd be more willing to come to the table.

• 53:03Reid Hoffman?

  Now let's move to rapid fire. Is there a movie, song, or book that fills you with optimism for the future?

• 53:12Speaker 00

  Yes, and in fact, Kim Stanley Robinson wrote it, The Ministry for the Future. And I saw that he was recently on your podcast. That is an amazing book. It is all about how humanity does everything wrong with climate change, and then we get our act together. And using technology that's currently available, we actually solve it. And it's particularly lovely because, you know, using cryptocurrency and blockchain, they kind of save the world from the Salesforce tower. San Francisco plays a big role. So anyway, I love that book.

• 53:43Aria Finger?

  Rapid fire question number two. What is a question that you wish people would ask you more often?

• 53:49Speaker 00

  The one you all asked right up front, like, why are you, you know, what is it about my background or anyone's background that you're talking to that motivates them to do the things they do? Because there usually is something. And for me, it really is as simple as, I wasn't personally affected by war, but I saw war very up close as an observer, very early. And it really impacts the future of your life. So that's why I like to be involved in diplomacy.

• 54:18Reid Hoffman?

  Where do you see progress or momentum outside of your industry that inspires you?

• 54:24Speaker 00

  So I am not in the AI industry, and the progress and the momentum I see there is so unbelievable. And I would just give you two examples. So my husband works at Google X. What they are doing at the intersection of AI and clean tech, like, they have an amazing AI that looks at the grid, outdated, very large machines, our electricity grid, all around the world, and makes them way more efficient, way more resilient, much clearer when you need baseload, when you don't need baseload. So that's totally amazing. And then, I mean, I don't need to tell you all because you're steeped in this ecosystem,

• 55:04Speaker 00

  but how AI has democratized science? So my high school kid is literally working on identifying new...

• 55:15Speaker 00

  Proteins, and I don't even understand what he's talking about because it's not my world. But, you know, he's a high school kid who's playing around with AI models and building some of his own to think about the biologics of the future. And that's really fun. So if you're a young person going into science and technology today, what an amazing time. I think there's never been a better time.

• 55:37Aria Finger?

  Well, can you leave us with a final thought on what you think is possible to achieve? If everything breaks humanity's way in the next 15 years, and what's our first step in that direction?

• 55:47Speaker 00

  If everything breaks our way, the US and China will find a way to get along as two great powers. And think about what would happen if instead of this pretty dark path that we're currently traveling down with ever greater distrust, harming each other in the cyberspace, saber rattling and everything else, here's an alternative future, okay? Both countries are comfortable enough with each other, will never be friends, but we're comfortable enough with each other that we can pull back our military stances. We have to spend less money on our militaries. That frees up resources for pensions that need to be shored up in both countries, for

• 56:30Speaker 00

  healthcare, for collaborating again on innovation and science and technology, all of the things that are going to help all of humanity. It frees up an opportunity to really work on climate change together. Currently, those efforts are really pretty moribund. And it allows us in a way that protects our workers and thinks about how we can prosper, and all of us, not just those of us in Silicon Valley. But goods flow more freely between the US and China and the rest of the world, and we get on a pretty positive path. So that's what I'm working towards. It feels discouraging now, and it feels even almost naive to say that, but I think unless you put a big positive goal out there, you'll never achieve it.

• 57:20Reid Hoffman?

  Possible is produced by Wonder Media Network. It's hosted by Aria Finger and me, Reid Hoffman. Our showrunner is Sean Young. Possible is produced by Katie Sanders, Edie Allard, Sarah Shleid, Vanessa Handy, Aliyah Yates, and Paloma Moreno Jimenez. Jenny Kaplan is our executive producer and editor.

• 57:40Aria Finger?

  Special thanks to Surya Yalamanchili, Saída Sepieva, Venasi Dilos, Ian Alice, Greg Beato, Parth Batil, and Ben Relis. And a big thanks to Janet Ohm, Tiffany Frieza, Malia Agudelo, Clean Cuts, and Little Monster Media Company.