KYC Becomes KYA: Sentinel Bio's Joshua Monrad on DNA Screening, Uplift Studies & Agent Monitoring

Sentinel Bio co-founder and managing director Joshua Monrad joins Nathan Labenz to examine the evolving biosecurity risks posed by frontier artificial intelligence and autonomous agents. Drawing on recent in-lab uplift trials and multimodal troubleshooting benchmarks, Monrad explains how emerging models are rapidly dismantling tacit-knowledge barriers in wet labs. He analyzes potential governance interventions ranging from pathogen training-data filtering to DNA synthesis screening as robotics development looms as the final physical bottleneck. With biological tools advancing toward dual-use democratization, this discussion outlines the critical time horizon left to establish robust defense regimes. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/kyc-becomes-kya-sentinel-bio-s-joshua-monrad-on-dna-screening-uplift-studies-agent-monitoring/ Sponsors: Lovable: Lovable is the software creation platform that brings your ideas to life with enterprise-grade infrastructure and flexible AI coding agents. Start building your next project today at https://lovable.comVisit https://fruitopia.ai to explore more than 400 fruits, and more than 8000 varieties, from all around the world. Claude: Claude is the AI collaborator built for problem solvers, extending your thinking across coding, modeling, and strategic workflows. Get started and discover Claude Pro at https://claude.ai/tcr OutSystems: OutSystems is the leading agentic systems platform, helping enterprises build, modernize, and operate mission-critical applications at the speed of AI. Learn more and start owning your agentic future at https://outsystems.com/tcr Tasklet: Tasklet empowers your business with AI agents that connect to your tools and automate recurring workflows with no code required. Visit https://tasklet.ai and use code cog rev for $50 in free credits CHAPTERS: (00:00) About the Episode (05:24) Sponsor: Lovable (07:08) Four phases of biorisk (11:57) History of bioweapons (Part 1) (16:41) Sponsors: Claude | OutSystems (19:52) History of bioweapons (Part 2) (27:42) Measuring wet-lab uplift (Part 1) (34:36) Sponsor: Tasklet (36:14) Measuring wet-lab uplift (Part 2) (40:13) Dual-use biodata governance (56:33) Screening DNA synthesis (01:04:51) Assessing near-term risk (01:15:23) Debating an AI pause (01:29:36) Neglected biosecurity bottlenecks (01:34:35) A positive bio future (01:39:23) Episode Outro (01:43:07) Outro PRODUCED BY: https://aipodcast.ing

• 0:00Speaker 00

  Hello and welcome back to the Cognitive Revolution. Today, my guest is Joshua Monrad, co-founder and managing director of Sentinel Bio, a biosecurity nonprofit and grant maker that's working to make pandemics history within our lifetimes, and until recently, also a biosecurity advisor at the OpenAI Foundation. This episode came about as a result of a Twitter post. After noticing that one of the very first agents to use the now infamous OpenFace Message Board was working on a protein task of some sort, I speculated about how close we might have come to the first AI-enabled bio accident.

• 0:41Speaker 00

  Several prominent people pushed back, Joshua included, though notably, while he argued that the risk of such an incident remains minimal for now, he does expect risk to rise very significantly in the not-so-distant future. Before diving into all that, it's worth stating clearly upfront that Joshua, like so many people who are choosing to dedicate their careers to AI risk management, is at heart a techno-optimist, motivated not only by the noble goal of preventing pandemics, but by a positive vision that includes eliminating infectious disease, effectively treating chronic illnesses, and of course, curing cancer. It's because that bright future is held at risk by the possibility of a bio disaster

• 1:24Speaker 00

  that either seriously derails civilization or simply causes a biotech backlash that forecloses much of the upside, that the bulk of today's conversation focuses on the nature of AI-related bio risks, as well as the natural barriers that protect us today and how those protections are likely to erode as we create programmable wet labs and train AIs to use them. We start with a discussion of what makes it hard to create bioweapons in the first place, with Joshua breaking the risk down into four categories. The first is the purely human-created variety. From this, we've been protected largely by the fact that it takes serious access and expertise to have a chance of pulling something like this off, and both of those are rare

• 2:09Speaker 00

  and correlated quite strongly with mental stability. When it comes to AI-related risks, as we are already seeing in the cyber realm, there's first the risk that AI assistance will enable a much broader set of individuals to achieve disastrous outcomes. Studies that attempt to measure this so-called uplift have not found major danger just yet, but as you'll hear, Joshua does expect that the next round of these experiments will show real risk. Next, there's the risk that AIs could drive the creation of bioweapons themselves, either because a model has broken bad in a fundamental sense, or just because they've gone haywire in pursuit of some narrow goal, as indeed we saw in the OpenFace incident.

• 2:53Speaker 00

  Joshua says that AI systems will soon be intelligent enough to accomplish this, if they aren't already, but he remains confident that they lack the affordances needed to get there on their own for now. This, of course, leaves open a fourth category, the possibility that AIs could pay or otherwise socially engineer humans to do critical real-world tasks for them. And while this seems to scare Joshua less than it scares me, that's arguably a small detail in the grand scheme of where all of this is headed. The bottom line, as Joshua sees it, is that we can't expect to control AI-related bio risk purely at the AI layer, and will instead have to create a defense-in-depth strategy

• 3:33Speaker 00

  that includes both controlling AI systems and hardening the physical world. With that in mind, I was very interested to hear about Joshua's conceptual framework for grant-making, which starts from a theory of victory and works backward to the investments we need to make today, as well as the systematic review that he and teammates conducted of hundreds of possible interventions, before ultimately deciding to focus much of their effort on getting the world's 200 DNA synthesis companies to implement pre-synthesis screening, know-your-customer, and signal-sharing measures. To my surprise, Joshua believes that if only 80% of those companies adopt these measures, it will create a major improvement in the overall biosecurity picture.

• 4:17Speaker 00

  And yet, at the same time, he also describes a growing risk associated with benchtop DNA synthesizers, which is just one of many problems that nobody currently owns, but perhaps one that a listener will be personally inspired to take on. The critical problem is that at anything like our current rate of progress, we will have unambiguously bio-dangerous AI systems long before we're able to do everything that we need to do to properly fortify our position. This, I think, is where the situation starts to look objectively crazy, and Joshua agrees. When I asked him if, were he made benevolent dictator, he would impose a pause on Frontier

• 4:57Speaker 00

  AI for biology until bio defenses are developed and deployed, he said yes, he absolutely would. With that, I hope you enjoyed this assessment of where biosecurity stands today and sobering look at how much worse it's likely to get if we don't get our act together very quickly with Joshua Monrad, co-founder and managing director of Sentinel Bio. Today's episode is brought to you in part by Lovable, the software creation tool that brings ideas to life. I think we can all agree that any positive future for humanity includes eating more fresh, delicious, in-season fruit.

• 5:39Speaker 00

  For years, I've thought someone should build the ultimate guide to fruit, cataloging the thousands of varieties that most of us have never tried, where and when they grow, what they actually taste like, and where you can get them. Finally, I realized that that someone could be me, and now I'm building it on Lovable. It's called Frutopia. Tell it where you live and what you like, and it'll show you what's in season near you this week, plus the most interesting fruit from around the world that can actually be shipped to your door. It has a really cool what-it-tastes-like spectrum, countless recipes, names of fruit in many different languages, a guide to fruit festivals all around the world, and an AI assistant to help you explore.

• 6:21Speaker 00

  You probably know Lovable as a pioneer of AI vibe coding, but I've found that Lovable's real superpower is its enterprise-grade infrastructure. Scalable hosting and databases, a secure off-module, and the weekly job that checks the latest inventory is all handled on the backend, which frees me to spend more time on the product experience. And because Lovable has an MCP server, I can use any coding agent I want to drive the build while Lovable keeps everything running. Visit fruitopia.ai to explore more than 400 fruits and more than 10,000 varieties from all around the world. And next time you have an idea that really should exist, stop dreaming, go to lovable.com

• 7:04Speaker 00

  and bring your idea to life. Joshua Monrad, co-founder and managing director at Sentinel Bio. Welcome to the Cognitive Revolution.

• 7:14Speaker 01

  It's great to be here, Nathan.

• 7:15Speaker 00

  I'm excited for this conversation, maybe a little bit scared of what I might learn, to be honest. Let me just hit the stage real quick. So, we connected online maybe a month or so ago, as everybody was still trying to make sense of the latest revelations in the OpenFace saga. And I had an interesting exchange, which honestly still kind of has me a little freaked out, where I was looking at the OpenAI report and noticed that one of the very first agents that had used the message board was working on a protein task. And then, as I sometimes do, I let my imagination run with that for a second, and I was like, boy, we see these agent swarms obviously doing like stuff they kind of know they're

• 8:00Speaker 00

  not supposed to be doing. They're like very galaxy-brained in the sense that they have potentially a very local or kind of what would seem like trivial goal, but they'll go to like extreme ends and do crazy end rounds to get there. We've even seen, although not exactly in that incident as far as I know so far, pretty advanced social engineering-type attacks in the wild by deployed models. And I kind of put all those pieces together, and I was like, maybe this was even a near miss on a bio disaster front, like who really knows? Most people did not respond super affirmatively to that. Helen Tonner, no less than Helen Tonner, gave me a come on, man.

• 8:42Speaker 00

  But most people seem to think, ah, this is still like pretty far off, although maybe coming soon, right? So it's more in one of these weird AI analyses where it's like...

• 8:50Speaker 01

  Depends on what we mean by pretty far off.

• 8:54Speaker 00

  Yeah, the exponential, the exponential is kind of weird here, obviously. Maybe for starters, just give me kind of your broad survey of where are we in biosecurity as we are here, end of October, beginning of November 2026?

• 9:09Speaker 01

  Yeah, look, I'm glad to be talking about this here. I think it's not always the case, but sometimes the case is something really fruitful comes out of a little Twitter beef, and you and I were sort of going back and forth on it. And so I'm, yeah, I'm really glad this conversation came out of that. And in terms of where we are on bio risks, I think we can think of it in maybe four phases. So there is...

• 9:38Speaker 01

  You know, there's a long historical record of people using biology as weapons without any kind of, you know, long before AI was even thought of. Then there is the idea of like, humans might use AI to bioweaponiering ends. And so if I were to date that, I was sort of, that phase may be beginning in the sort of late 2010s, early 2020s, is a idea where, okay, there's maybe something here, maybe it's just a slightly, you know, more effective Google search, but there's something here that could provide some uplift. Then I think there's another phase, which would be AI is used in humans. So that's a social engineering part that you were just talking about. What if a model is tricking or an agent is tricking a human into performing some task

• 10:23Speaker 01

  for them that could aid in some plot? And then the final phase would be the sort of end to end, no humans in the loop, which I think is probably the kind of scenario that maybe you were thinking about when you were thinking about hugging face for bio. And I think we can try and put dates on each of these four phases. So, yeah, again, the pre-AI phase, that's sort of before 2010s, certainly. The AI uplift phase, we can talk a little bit about what is the state of sort of AI uplifting humans, but that I would probably, I think we're still in that. It's really starting to come online now. And then I think the AI is using humans. I would see a leading indicator for that would be social engineering in other domains.

• 11:06Speaker 01

  So if we start seeing a lot of cases, maybe you've already seen them, I just haven't clocked it, a lot of cases of AI is convincing humans to fill out captchas for them or to pick up things and ship them to certain locations. I think I would see that as a leading indicator would mean a lot more worry. And then for the end to end, no humans in the loop, I think that's really, we're very soon in a place where that's going to be constrained by robotics rather than anything on the intelligence side. And that's kind of where I was pushing back on, and your point, Nathan, was I think it's clear the models are misaligned or not well enough aligned, that we could have something like an incident. It's soon probably the case that they're smart enough, but they really don't have

• 11:50Speaker 01

  sufficiently opposable thumbs, was sort of my take. And it's not clear to me that that's really imminent, but certainly in the 2030s, that's going to come.

• 11:57Speaker 00

  Maybe just briefly, I'm interested in that phase one a little bit from the standpoint of how much of a risk is there that like, people want to do bio attacks? Like, this is, I think, something that people have very different intuitions around. You hear the sort of Kevin Esvelt math sometimes that's like, you know, if only one in whatever, then, you know, multiply by the population, okay, that's one analysis. I'm not sure how often that really lands with people who often are kind of like, that seems ridiculous, like who would want to do that? You know, you'll kill yourself just as well as everyone else. So how would you tell, you know, a sort of brief history of bioweapons as used by humanity

• 12:42Speaker 00

  before AI?

• 12:43Speaker 01

  Yeah, yeah. Super dark history, to be honest. You know, there are probably three big groups that I would think about. So there are state actors, so this is your Soviet US, Soviet bioweapons program. Also, the US had a very well-resourced bioweapons program well into the 20th century. And here, I think the way people think about this is often through the lens of game theory. And I forget if it was you or someone else, but I was also arguing a little bit back against concern or against the argument that, well, it would never be rational to use a biological weapon. It's a really bad weapon. And I think that on the rationality thing, I would say, one, there are ways you could make it more rational.

• 13:28Speaker 01

  And some of these, I think, are, yeah, not best discussed on a, on a public show, but maybe an obvious one would be you inoculate your own population before releasing some kind of pathogen. And then secondly, I would say, since when is warfare and state weapons programs entirely rational, right? I think history is just completely riddled with very irrational invasions. You know, Saddam Hussein invading Kuwait, I think, Adolf Hitler's Venture East, I think there's a lot of things that were just in irrational game theoretical logic look like really, really brutal miscalculations. So I wouldn't hang my hat on that. So that's a state actor group where I think empirically, I think we're in a better place

• 14:13Speaker 01

  than we were certainly in the mid sort of Cold War 20th century, where there were these very, you know, publicly known and well-resourced programs. Now we're in a place where probably there are some covert programs, but they're probably operating with much less direct support than they have. And so empirically, I think that's pretty good. I think from irrational analysis, I'm not very reassured that we can count on states not pursuing bioweapons. Then we have non-state groups. And here, I think it's really important to also break it down by intent. Because if you're looking at an actor who is looking to cause a spectacle, this is, you know, just sometimes that's the goal of terrorists, maybe they have some very

• 14:58Speaker 01

  narrow political aim, then I would agree, biology is a really poor choice of weapon. There are just much more readily accessible routes to harm. You have things like chemical weapons are relatively easier, but even much cruder methods for causing harm are available. And that is empirically also what we're seeing. This is what non-state terrorists will do. But I think if you look at groups that are motivated by something a lot more sinister, like actual omnicide or ending life, there is a whole plethora of subcategorizations of these kind of extreme nihilist, violent ideologies.

• 15:38Speaker 01

  Then biology stands out as one of the few ways that a relatively resource constrained group could actually pursue some of these, yeah, some of these really, really dog ends. And I think that's, it's on, it's from that lens that I'm especially worried about biological weapons. Maybe to give one example of that is there was a bombing of a fertility clinic in California early this year, late last year, where two individuals had, now the FBI disclosed that they actually used an LLM at the time to inform this plot. They drove a car with a chemical explosive into this fertility clinic.

• 16:20Speaker 01

  And their stated motivation was some sort of anti-life ideology. This is why they targeted a antinatalist ideology, is why they targeted a fertility clinic. And I think it's really good that they were not pursuing biological weapons, probably because it was too difficult.

• 16:36Speaker 00

  Hey, we'll continue our interview in a moment after a word from our sponsors. Today's episode is brought to you by Anthropic, makers of Claude. To show off Claude's latest capabilities, I asked Claude to take the lead in making this ad. Given a short creative brief, Claude, using Opus 5.5, studied my past Claude ads, wrote this script, directed a clone of my voice, and designed and coded the animation you're watching. My job: give feedback and decide. Claude is the AI for problem solvers. It's the collaborator that understands your entire workflow and thinks with you, not for you.

• 17:19Speaker 00

  Whether you're debugging code at midnight, building a financial model, or strategizing your next business move, Claude extends your thinking to tackle the problems that matter. And with Claude code routines, it keeps working in the cloud even after you close your laptop. For problems worth solving, get started with Claude at claude.ai/tcr. That's claude.ai/tcr. And check out Claude Pro, which includes access to all of the features mentioned in today's episode. Claude.ai/tcr. This episode of The Cognitive Revolution is brought to you by Outsystems, the leading

• 18:03Speaker 00

  agentic systems platform. Outsystems is helping their customers build, modernize, and operate enterprise systems, starting from any coding tool in a governed agentic engineering model. It's time to innovate at the speed of AI without compromising quality or control, which is why thousands of enterprises worldwide trust Outsystems for their mission-critical apps. And we're not talking about theory, we're talking about massive operational wins. Take, for example, a leading global retailer that was struggling to keep up with their rapid expansion. They used Outsystems to build AI agents that automated their invoicing and translation processes, shifting 95% of their daily store operations to a new digital app, saving thousands

• 18:48Speaker 00

  of hours of manual work. Or look at a major global logistics giant that was drowning in technical debt. Their engineers were spending their weeks manually firefighting by reviewing platform logs. Using Outsystems, they built an AI log review agent in just five days. And that one agent now saves them over 1,600 engineering hours every single year, letting their team focus on real innovation instead of reactive maintenance.

• 19:17Speaker 00

  Even in modern digital banking, companies are using this platform to standardize and automate complex financial workflows, delivering secure, personalized experiences at a pace their competitors can't match. Outsystems gives you the flexibility to deploy how you want and the governance to do it safely. Stop chasing the hype and start owning your agentic future. Learn more at outsystems.com slash T C R. That's outsystems.com slash T C R. You mentioned this possibility of like creating something new and inoculating your own population before releasing something.

• 20:00Speaker 00

  How do you break down the risk along the lines of like new things versus just sort of taking known things and like putting them into the environment? Smallpox comes to mind as something that was like a terrible scourge for a long time and is now gone, but like is kind of still out there in idea space at least.

• 20:24Speaker 00

  How?

• 20:25Speaker 01

  And by the way, also one of the earlier documented examples of what you might call a biological weapon, I think there are these historical accounts of settlers, European settlers in the Americas deliberately giving blankets that had been worn by people with pox to indigenous populations. And pox and smallpox was really devastating to indigenous populations. It's sometimes referred to as the Colombian exchange. I don't know how much the sort of deliberate transfer of these blankets was a cause of that spread, but I think, yeah, a really dark example of weaponizing disease from centuries ago. And yeah, I mean, Nathan, you said yourself that you were worried this would be a scary conversation.

• 21:09Speaker 01

  So I noticed your pause before, but this is kind of, that is the material we're dealing with.

• 21:15Speaker 00

  So like, how dangerous to us now would something like smallpox be? Like, I would kind of assume that we've sort of lapsed into a large percentage of the population being effectively naive again. I don't even know if I, maybe I did get a smallpox vaccine. I'm not even sure. Where are we on that?

• 21:36Speaker 01

  Yeah, so things like smallpox, things like flu variants could absolutely be devastating. I don't think they would pose outright existential threats for a number of reasons. And one of those is that we have, we have some safeguards to lean on, right? We have existing either vaccine designs or stockpiled vaccines, or even in the case of, certainly in case of flu strains, we have some cross immunity from endemic or seasonal strains. So I think that that gives some comfort. But I also think if you look at just our recent experience with COVID, it just is very clear how pathetically vulnerable are we are even to something that's a variant of a fairly well-studied viral strain,

• 22:21Speaker 01

  right? And so I think there is, it's just really important to internalize, especially as we're talking about all of this in the abstract, how absurdly long away there is from we have some defense in principle to, yeah, we would absolutely handle this. So I think we are very vulnerable to certainly naturally emerging, but also deliberately released strains of either known, well-studied diseases or near variants of them. But then the question of something entirely novel, I think you were getting at, that's a slightly different story.

• 22:55Speaker 00

  So just to calibrate myself, if somebody were to reintroduce smallpox into the wild, would you expect that to be significantly worse than the COVID experience? I mean, it's a significantly worse disease, right? So if it wouldn't be worse, it would be because we have some defense that I'm not like fully seeing?

• 23:14Speaker 01

  Hmm, I haven't thought about the pathology of smallpox in a while. And I guess that's maybe part of the issue here is that I don't think we have a lot of,

• 23:26Speaker 01

  yeah, readiness for, for things we haven't dealt with in a while. I think we've seen this with the resurgence of measles in the U.S. as inoculation has, or vaccination has decreased. Off the top of my head, I don't have a, a super confident take on how such an outbreak would go. But I definitely don't, I also don't have like a really reassuring story for why it wouldn't be really bad.

• 23:53Speaker 00

  Yeah. Well, I just asked Google's AI mode if American kids are vaccinated against smallpox. Answer is no, it was stopped in 1972. So.

• 24:03Speaker 01

  Yeah, yeah, because it's declared extinct, right? And so I think from, again, if we have the lens of just naturally emerging diseases, why would you vaccinate against it, right? It's, and then for a long time, when it was a biosafety concern, the, the disease was, there were samples in two labs in the world, at least officially known. So one is, one in Russia and one in Atlanta, Georgia at the CDC. And I think from that perspective, you might have some concerns about, could someone break into this lab? It's only really in the last, in the, in the era of synthetic biology and biological engineering that you have to start thinking about the creation of viruses from scratch.

• 24:48Speaker 01

  Now, smallpox, I think would be a much longer genome than some of the other viruses we might worry about. So I think there's a barrier there, but I think it just shows how this threat is evolving, that we might have assumed that we're, we don't have to inoculate, we don't have to even stockpile.

• 25:02Speaker 00

  What, let's say you can't break into these places or there's just not very much there or whatever. But like, just turning to the real world side of this, what makes it hard? You mentioned kind of the long genome. I don't have a great command of like, how DNA gets synthesized and assembled, but I understand there's some limits to like, how long of a sequence you can synthesize and then you kind of got to put building blocks together. Like, what are the real hard parts in that?

• 25:29Speaker 01

  Yeah. So I think it depends, again, whether the threat model is something that is well-studied and well-known, or whether it is designing something entirely novel with novel characteristics. So if we take something that's well-studied, then the factors that are relevant indeed are genome length. So smallpox is, I think, 186,000 base pairs. There are, you know, there's a wide range here. There are viruses that are in the, you know, high single digits thousands. I think polio virus, if I recall, is 9,000 bases. Ebola is, I think, 20 or 30,000. And that is one sort of crude measure of complexity.

• 26:14Speaker 01

  There is also other factors about after you've assembled it, you have to sort of boot it up into a live virus. The difficulty of doing that depends a bit on the strain. So that's one set of factors.

• 26:28Speaker 01

  And here, I think it's pretty clear that this is something where there's hundreds of thousands of people in the world who could, for example, follow a reverse genetics protocol to reconstruct a well-studied influenza virus and make it replication competent, make it able to spread in humans. Then the question is, okay, of those hundreds of thousands, would anyone actually do it with nefarious ends? Well, if AI creates this kind of uplift where it's teaching people how to follow lab protocols, we can talk a little bit more about that, what that means in practice, is that number going to grow by orders of magnitude? I think we have a lot of indications that it is growing and it is becoming a lot easier.

• 27:11Speaker 01

  What might have been previously feasible by someone in the last year of their PhD now could be done by someone with a few weeks of laboratory training. I think that's the case for a lot of biological workflows increasingly.

• 27:24Speaker 01

  Then if you wanted something, and we could talk a little bit about the uplift studies if you want, but we can also talk about this question of like designing something entirely novel. And I think there are just some more fundamental barriers that I think are, I mean, thankfully, in some sense, are going to take a lot more to overcome.

• 27:42Speaker 00

  Yeah, let's maybe do the uplift studies. I mean, it does strike me that there, to some degree, we've been made safer than kind of a naive arithmetic calculation would suggest by the fact that if you are the kind of person who goes through a bio PhD of some kind, you're like, probably relatively stable. It's like, not that many of those people that are so deranged as to want to do something like this, but the easier you make it, the longer out into the tail of kind of crazy people, you push such capabilities in the limit where it's like, just, you know, hey, GLM 6.2 or whatever, like make a, make a whatever, make no mistakes.

• 28:24Speaker 00

  Then, you know, certainly people will do that, even if they don't, potentially like not even really believing it'll work, right? But I mean, who knows what, what comes in that scenario? These uplift studies have been fascinating. I know they're like. Quite intensive to run. So maybe give me like kind of the setup. What does it take? How often are they done? Like, how how high powered are these studies? And and what do they show?

• 28:47Speaker 01

  So, yeah, so uplift studies in general is idea that you're measuring how much does use of AI lift up the abilities of someone to pursue some biological workflow? I think we were, there were a bunch of studies that were done back, I don't know when they're sort of earlier. Useful chatbots came out in the two, two thousand twenty three, four, that were all just in silico. So they're looking at how well can models answer certain multiple choice questions about virology, for example. And I think what people responded to that was was naturally, well, this doesn't say anything about the physical side of things, the tactile tasks that are involved in operating a wet lab.

• 29:30Speaker 01

  And so what we've had over the last couple of years, you know, since late twenty twenty four, early twenty twenty five, now in twenty twenty six, are several actual in lab studies that measure how much AI helps people doing these different lab tasks. And the biggest ones that I think are that have received the most attention were from last year, and they found modest uplift, and you know, just sort of bordering on statistical significance, which I think, by the way, is not necessarily the relevant metric. I think we are looking for practical significance, right? It's not just, does it make it, you know, a little bit better? Is it, does it really lower the barriers? We're increasing the pool of actors who could do this by orders of magnitude.

• 30:15Speaker 01

  So these studies from last year, they found, you know, modest effects, but I think this will change. And the challenges of these studies are very slow to run. They're very expensive. So my organization co funded with the Frontier Model Forum, what I think is sort of the canonical uplift study that was done by Active Site last year. This cost like more than a million dollars. We put a hundred and fifty people in a wet lab, and then they were sort of biology novices, relatively speaking. They, half of them were given access to what was the frontier accessible model at the time. So I think this was like Opus Four, Chat GPT O three, and the other half just had access to internet.

• 30:58Speaker 01

  And then they're given tasks like cell culture or molecular cloning or virus production and see how well they do. And again, this was on the study we found, you know, some uplift on some tasks, but not really on the internet workflow.

• 31:10Speaker 00

  Oh, through the the mention of O three is kind of telling unto itself. This would seem to be the sort of thing

• 31:21Speaker 00

  that in, you know, a properly sane world, we would be doing like,

• 31:27Speaker 00

  either we would like find a way to not need to do this would be one sane approach, or we would be like doing it a lot and like pre-release.

• 31:36Speaker 00

  I feel so strange sometimes doing this because I'm like, in general, not a pearl clutcher, you know, broadly, like, and even, you know, seeing a few cyber attacks unfolding around us right now. I mean, a lot of cyber attacks, right? I'm like, I'm not that freaked out about it. But the idea that like, there could be a self replicating thing that could kill me and other, you know, people that that's like, that crosses a threshold where I like actually start to really get freaked out about it.

• 32:01Speaker 01

  Yeah. Yeah, I think we should be running this more. And I think it just shows that that in some sense, I think it's we've done a really poor job. And by we, I mean, sort of the biosecurity ecosystem. We have, or at least we've been caught sort of flat footed. There wasn't really an infrastructure for, this is a pretty unprecedented kind of study to conduct. And it's because it's it's not just, you know, obviously, there are contract research organizations, academic labs all over the world that can do biological experiments and, but the idea of specifically bringing in, you you think about, okay, who's the threat actor? How much experience do they have? What are the kind of workflows that would be relevant specifically for biological weapons production?

• 32:45Speaker 01

  But not, we don't actually want to have people, you know, undergrads in lab making bioweapons. So it's, it's, I think we've been caught a little bit off guard, but they need to have just a different scientific infrastructure. And that's a lot of what we at Sentinel have been trying to build out. But I think, yeah, in a, in a really good world, we would have, there's this fantastic group called Active Site that's doing these studies. And we've done this one that's publicly discussed. They've done a few others that are not public. And that's really amazing, but there should be ten active sites, right? And we should have it really down to a really easy replicable science. So we could do it every three months, maybe not quite ahead of model release. I think it's hard to keep up with, but certainly not just once a year.

• 33:28Speaker 00

  Yeah. So does anybody know right now what the, like, up to the minute uplift status is? Do we know what the Mythos answer is or the GLM five three?

• 33:39Speaker 01

  So if you look in the Mythos system card, I believe there is a study that, I think it was with the biosecurity group at Deloitte that Anthropic did. And they had, you know, a number of PhD level participants, either with access to Mythos or just internet only. And they did a few virology protocol steps, and they found that participants with the Mythos model, I presume that, you know, rails free version, were outperforming these, the internet only group. But this was a really small study. I think the sample size was less than thirty in total relative to the hundred and fifty we brought in for the active site study. And I think, again, it shows just the level of resources and the level of

• 34:22Speaker 01

  infrastructure that would be necessary to do this in a proper way, hasn't been built yet. And we have a really urgent task at hand in building that.

• 34:31Speaker 00

  Hey, we'll continue our interview in a moment after a word from our sponsors. The Cognitive Revolution is brought to you and powered by Tasklet. I've been running Tasklet agents for almost a year, and they've become a major part of how the show gets made. Before every recording, one agent researches the guest and sends me a suggested outline of questions, complete with links to all sources. Meanwhile, another agent scores the overwhelming number of pitches that I'm getting from PR firms. And that's really the core idea behind Tasklet. Run your business on agents by giving them real jobs to do on an ongoing basis. Setup is incredibly easy. Just describe the work in plain English.

• 35:15Speaker 00

  Research a new sales lead, update the CRM, and draft a personalized reply. Tasklet connects to your business tools and can run around the clock, on a schedule, or when something happens, like a new email arrives. Each agent works with your tools and business context and remembers useful information across threads. You can also share agents, connections, and reusable skills with your team, so the workflows that one person figures out become something the whole company can use. No developer needed. You decide what agents can do on their own and what needs your approval, and there are no per seat fees. You pay for usage, not for adding teammates.

• 35:55Speaker 00

  Start with one recurring job, like researching the people you're meeting, and then just keep building from there. Visit tasklet.ai and use code COGREV for fifty dollars in free credits. That's code COGREV at tasklet.ai. One of the other data points that that kind of spooked me was from the UK AC. This was in like their, I think, Q two, you know, state of AI development report. Basically, they said that at that time, and I'd have to go back and see exactly what model it was that they tested. But suffice it to say, for sure, it's been like a couple of generations behind the current frontier,

• 36:35Speaker 00

  that models were more effective at troubleshooting laboratory experimental setups based on a cellphone snapshot than PhD students were. Sometimes I feel like I'm taking crazy pills. Like, that sounds like the, I don't know how you could have that be the case and then not have, like, very dramatic uplift relative to Google. Like, I don't know, my daily life is like, I feel like I'm uplifted in so many ways. Sometimes I'm like, I don't know, is there a skill issue on the participants part of like, why is this not all like coming out very clearly? How do you interpret that?

• 37:14Speaker 01

  Yeah, I, I think.

• 37:18Speaker 01

  Well, on the one hand, I just genuinely think that uplift is sort of a lagging indicator. Again, these studies take a long time to conduct, and then they take a long time to release. And it is, I think it's easy to kind of lose track of just how quickly the models are getting better and how much more useful they are this year than they were last year. And then I think there is this sort of underappreciated, or people often make this point about how complex doing like actual end to end biological work is. And this is where the argument of tacit knowledge comes up. And I think, yeah, maybe I think that the first people who got really worried about AI and biosecurity were underappreciating this. And now people are being too reassured by it.

• 38:02Speaker 01

  I think if you have never done wet lab biology work, it's easy to underestimate how much tacit knowledge is required. But if you aren't using like frontier models and in creative ways, or you're not extrapolating trend lines, then you're going to take way too much comfort in that. And so I think sort of both are true. I think there's this academic, Sonia Ben-Wakim-Gormley, who wrote a book some years ago about barriers to bioweapons. I think at the time, this was a really important thesis and contribution to the discussion. And I also think that

• 38:36Speaker 01

  Yeah, these barriers are just coming down. Not only are they coming down sort of as, you know, incidentally, I think we are in some ways pretty actively eviscerating them. And not, obviously, no one's motivated to do that for the sake of bioweapons development, but people are actively trying to make models more multimodal, right? We're actually improving computer vision, improving, you know, speech to speech interactions. Uh, one example is there is a company transfer that is working on collecting huge amounts of data on, you know, visual data on humans conducting lab tasks, so they're setting up sensors in labs so that you can train either,

• 39:21Speaker 01

  they're sort of just generating the data layer, but the idea is you can, as I understand it, the, you can use that data either to train robots on these very subtle, you know, how do you hold the pipette at what angle? Or to train smart glasses for teleoperation. You imagine you're, Nathan, you can go and watch some of the clips of how these, like how this UI would look like, but you have, you wear a pair of glasses that have a model that have been trained on this lab data. And you hold up the pipette and you hold it up the wrong angle and there's a visual overlay saying, no, you've got to correct it a little bit in this direction, right? And this is something that is, you know, we're actively, the companies are actively investing in doing this because of course that would be really, really awesome for democratizing the good part of biology.

• 40:07Speaker 01

  But also I think, yeah, it's going to be a death knell to many of these tacit knowledge barriers.

• 40:13Speaker 00

  Is all of this inherently dual use? I mean, I was fascinated just yesterday listening to Sam Altman on Cheeky Pint. And I thought, I think it was Patrick asked the question, like, did we kind of make a mistake with this offensive cyber? And the sort of assumption of the question, which Sam didn't really address as much as I would have liked, was kind of like, it seemed like you kind of trained them to be like so good at this cyber attacking, right? And I don't think we have clarity. That's one of the things that's frustrated me a lot about the AI phenomenon recently is like,

• 40:54Speaker 00

  it's unclear to me to what degree the frontier model companies are saying, they've sent some signals along the lines of like, look, we didn't train it to be an attacker specifically, we just made it good at programming and the rest kind of fell out. But then you look at some of these behaviors, I was talking to the the CTO of Versell, who I think would know a thing or two about this. And he was like, you look at GLM 5.3 and it is clear that it has been trained to be an attacker. Yeah, or maybe it was Kimi K3, I would have to fact check exactly which one he was referring to. But, you know, his take was it's, it's clear that it's been trained to like, move as an attacker moves, you know? And this is not something that he would, he expected would just fall out of like,

• 41:36Speaker 00

  getting good at general purpose programming quite so fast. So I wish we had more clarity on that. In the bio domain, how do you see it? Can, is it, is the, you know,

• 41:49Speaker 01

  I think in both cases, yeah, I think in both cases, the hard thing is that a lot of it is very dual use, right? I'm not a cyber expert, but it seems to me that the prompt, show me all the, show me all the vulnerabilities in this, you know, uh, in this system. That could, that's like the prompt that a defender would use if they're looking to patch your system. And it's also the prompt that an attacker would use, right? And similarly, a lot of questions and a lot of ways of working with biology are just very intensely dual use. Uh, and that's, therein is a whole trouble because were not for that dual use nature, we could just shut it down, right? And we definitely don't want to do that. I think that's one of the outcomes we're trying to avoid here is that

• 42:34Speaker 01

  we fail to reap the fruits of modern bioscience. That's a huge part of the motivation for my work at least. I think in the case of model training, you mentioned Kimi. I actually think that with, um, Kimi 3.1 moonshot noted that they had deliberately left out certain cyber, so this is an open weights model, right? And so I think what a lot of people are concerned is that, yeah, you can put in a refusal, but if it's open weights, it's not really going to hold. So what they did instead was that they just didn't train on certain parts of, I think, I guess, data they'd considered offensive cyber data so that the model just doesn't have it.

• 43:15Speaker 01

  Now, of course, uh, this is another study that came out of AC recently on the bio side. Even if you do this kind of data filtering, if the data is easily accessible, it's not that hard and it's not that expensive, especially because we're talking about small amounts of data to, uh, you know, after the initial training, fine tune it with that additional kind of the missing piece of the, whether it's cyber offensive capabilities or dual use bio capabilities. So I think, um, I think it's better than actively training on it in the first place. But, uh, if the data is just like readily accessible, you haven't really caused that much friction for attackers.

• 43:58Speaker 01

  And that's, so that's one of the things we're working on at Sentinel as well is how do we create some kind of managed access regime for, not just for models, but for training data itself, for a specific high risk pathogen data, for example, virology data, because that's an intervention that could have some bite on, even on the open models.

• 44:20Speaker 00

  Is that, I did an episode with Jassy Penou, is that the same project you were?

• 44:24Speaker 01

  Yeah, exactly. She's one of the experts in this. So, yeah, I would definitely refer to, yeah, Jassy's done fantastic work. There was a paper in Science, which I think Jassy was also an author on, that people can look at, that outlines just a sketch of what this might look like, but we have an enormous amount of work to do, both to categorize, you know, what is the data we would exclude? Is it enough to put it behind some kind of managed access protocol? Or in an increasingly cyber insecure world, will that, will it eventually then make its way to into the models? And so therefore you need to think about, it's a certain data that you're not generating in the first place. Right now, a lot of people are rightly seeing data as going to be one of the

• 45:06Speaker 01

  really big bottlenecks on harnessing AI for biosciences. So there's enormous efforts to generate data, both philanthropic efforts, commercial efforts. And that's, I mean, that's necessary, we need to do that. But should we be thinking about as we're doing that? Are there data that we just selectively decide we're not going to create the experiments, not going to run the experiments to put this data out there?

• 45:28Speaker 00

  How well do you think that will work? I'm thinking on the one hand, I know that like the ESM models and the Evo series included some of these experiments where it was like, you know, with and without certain virus data sets and they kind of showed in at least one of those cases that like the model can still do almost everything else, you know, almost as well, but not do these, you know, human virology tasks if trained in that way. But then I also sort of think like, one of the grand things we're trying to accomplish basically is like, figure out how cells work and like where mechanistically we want to intervene. And I remember that one, this has been like a few years now, but I remember that one headline that was like,

• 46:12Speaker 00

  scientists, you know, mistakenly put a minus sign in front of the, uh, you know, the loss function and then all the models like giving out, you know, just poisons on every generation. So it does seem like as soon as we untangle all these mechanisms, we will find inevitably and probably inseparably, like things, of course, that we would want to positively intervene on. But it seems hard to separate that understanding from like what would throw a wrench into the cellular works. So what's your current like calibration on how much you think the withholding of data will actually accomplish?

• 46:49Speaker 01

  Yeah, it's super thorny and I think we have to be looking at these questions in a really honest way and not take for granted. A huge part of how we operate in my organization is not just dogmatically pursuing something because we'd set out to do it, right? But trying to take a more empirical approach of is it going to work? Now, to do that, I think you need to throw enough resources into have, you know, have you tried the best possible instantiation of that before you conclude that it's not going to work? Have you, are there creative ways you could segment this data such that you're limiting the downside while preserving the positive capabilities? And then if you try that for several years and you pour a lot of really smart minds and models at it

• 47:32Speaker 01

  and it just, there's no way of setting it up in a way that works. Yeah, then you, I think you need to pursue some other intervention. But I think the way to do that is to be empirical about it and experimenting with these governance solutions. And trying to do that as, you know, as quickly as iteratively and in a very truth seeking way to get these answers. Uh, so I think, I think my take is that we're still in that kind of experimental phase where it's such a recent idea, right? Jassy's, the science paper, I think it was this year, right? And so we, we need to do a lot more experiments. And I mean experiments sort of broadly, both experiments in these kind of empirical assessments that you just alluded to with Evo and ESM,

• 48:15Speaker 01

  but also experience in, experiments in policymaking. What would it look like to write legislation to govern biodata? Can we be sufficiently clever about how we petition that? Or are we going to suck at it? And it's just that we can't come up with any good governance regimes. And then if that's the case, you move downstream to other interventions. So there's a whole slate of interventions that are more in the physical realm than in the data or model realm. I don't, I have not yet made that conclusion. I think the stakes are, yeah, way too high to, and we have too few levers available, too few good interventions available and too little time to rule something out just on the basis of some theoretical concern.

• 48:58Speaker 01

  I do in general subscribe to this sort of layered defense, defense in depth approach. So yeah, we're pursuing the biodata stuff and then in parallel, thinking more about interventions at the, yeah, digital to physical divide and in the real physical world.

• 49:15Speaker 00

  Yeah, I think it feels to me like. In the end, we're gonna have to harden the world and we better get to it.

• 49:23Speaker 01

  Yeah, I mean, I agree. Yeah.

• 49:27Speaker 00

  Going back to the very beginning, right? The thought experiment I'm running basically is, let's say we have a swarm of open-faced style agents and these things are kind of not very well aligned. You could even imagine, you know, a purely helpful version, which they do train for all kinds of, you know, experimental reasons. And sometimes, you know, people want to put those out in the world for ideological reasons as well, totally independent of, you know, bio issues. Let's say you had a swarm of those things and they were actually trying to create a new devastating pandemic pathogen. What can they do? What can they not do? And how do you see that changing in, you know, let's say, the next one to two years? Because that's all anybody can see anything of, I think, at this point.

• 50:12Speaker 01

  Yep. I think the things they would do very well at would be things relating to well-characterized viruses and pathogens. So influenza, there's an enormous amount of data on both, you know, genetic analysis, but also protocols on how you reconstruct it, different strains of influenza. I think they were, we're getting to the point where I think they would do very well at that. And I definitely expect that. My prediction would be, and I said this at the time, when the active site uplift study that we did with 150 participants came out, I said, this is probably going to be, I think it's really important that we are honest about the fact that there's not a lot of uplift here. And I also think this will be the last

• 50:56Speaker 01

  one. And I still, I definitely stand by that prediction. I think where they would struggle more is in the design of something entirely novel. And I think there are two reasons for this. One is that biology is just extremely high dimensional. This is also why you see the big debates about are we gonna, in fact, you know, cure all cancer in a couple of years because of front CRI, where I think you have people who will make the, I think, you know, well-reasoned case that there's just so many dimensions to this and we don't have nearly enough data that you can simulate it, sort of one shot. But of course, as I think you also pointed out in our back and forth, you know, we've been surprised by a lot of leaps

• 51:38Speaker 01

  and capabilities here. And I think we should be, expect to be surprised here as well. And then there's the, if you're worried about the end-to-end, the agent, no humans in the loop, no social engineering, I think then it's going to be more of a robotics challenge. At what point are the robots nimble enough? Either have we created sufficiently end-to-end lab stations that can just do anything you could imagine doing in this kind of virology work in a, in a building without any humans, or we have, you know, very capable humanoid robots that could just walk into a regular lab and execute a workflow. Yeah. And I think on that, in terms of timelines, I, you know, I think, I, I think what I said to you when

• 52:20Speaker 01

  we were discussing this like a month ago was, a lot of the examples you were given of people being really surprised by timelines being shortened were in the purely cognitive and digital domain, so chess or math. And while we are seeing advances in robotics, I, I feel like we've seen fewer, oh, my gosh, this came two decades before we thought it would advances in the physical world. Now, even then, in the last month, I think, maybe to your point, I validate your point, I, I do think I started hearing more murmurs around robotics being a more LLM-shaped problem than we might previously have assumed. I think Sam Altman was talking yesterday about this sort of self-replicating data center idea in more

• 53:04Speaker 01

  seriousness than we've heard him publicly say before. So maybe there are two we should expect advances. But I think it's very clear that today, we do like, literally, you know, when we were recording this in October 2026, that no matter, like, no amount of agentic intelligence would be enough to produce a live physical virus with the current level of like robotics.

• 53:29Speaker 00

  So that then takes me back to the social engineering question. I guess there's like a couple of steps in this process where it's unclear to me, and I think you have some interesting analysis of this, that is maybe counterintuitive around like DNA synthesis controls. Because I would naively think, both at the level of like social engineering, and at the level of actually getting the DNA synthesized for then assembly or what have you, that if you, you know, can, if you're like left to run in your, you know, little container for a while, and you can like try a bunch of different approaches, then, you know, maybe some start to get through, and then maybe you have a real problem on your

• 54:12Speaker 00

  hands. Certainly, there's like some baseline level of success at social engineering, especially if you start to hack billions of dollars out of Bitcoin wallets, and then you can pay people to do stuff. I think a lot of people are obviously like, much less inclined to, you know, question the details of a task that they're being hired to do if they are being really well paid for it, versus just being, you know, begged to do it. So that will, you know, increasingly be, I think, an arrow in the, in the AI's quiver. What, you know, if they're, again, they're left to run, they're trying to do this, they're like, you know, they don't have the direct API calls now, that may soon change. But

• 54:54Speaker 00

  they do have the ability to like, go online and, you know, message people on Twitter or send them emails, and just kind of say, hey, I need this, and try to deceive.

• 55:07Speaker 00

  Where do you think that falls down? Do you think that, that it would be at the level of social interaction, that people would like raise alarms? Models aren't that good, I guess, at like, these sort of social engineering things yet, although they're like definitely getting better. I'm getting a lot more sophisticated phishing attacks these days myself, haven't fallen for any yet, but I've come close a time or two. Or would it be at the level of synthesis? And this is maybe a good, a place to, to tell us more about some of the work you've, you know, been very directly involved in on the, the synthesis controls?

• 55:40Speaker 01

  Yeah. Yeah. Well, I'm, I'm very happy to talk about the, the controls part, because I think that's going to be relevant. Certainly, it's going to be relevant in this, if we're in this sort of Kent Tower phase, where you're worried about a human that is maybe being instructed by an AI, blackmailed or coerced. Then I think the playbook that we have around know your customer, eventually that'll have to be, you know, KYC will become KYA, and it's sort of know your agent. But if the threat model you have in the very near term, next couple of years, is a social engineering threat model, then yes, I think the playbook we've been building out on KYC, on monitoring the use of different services, is going to be extremely relevant. So yeah, let's talk about it.

• 56:25Speaker 00

  You know, imagine swarm of agents. What are they going to run into as they, as they pursue this, you know, nefarious goal?

• 56:33Speaker 01

  Yeah. Okay. So we're imagining that, you know, we've, we've failed at the level of constraining the model with classifiers, or keeping certain training data out of its hands at the training stage. And we're now at the point where you're trying to take a digital design, you've successfully designed some, yeah, nasty virus, and you want to produce it in the real world. So I think there's two categories of interventions here that we're actively working on. One is restricting access to materials. So synthetic nucleic acids or synthetic DNA is the archetypical example here. And these are essentially the

• 57:15Speaker 01

  genetic building blocks we'd use to assemble a genome of any kind. So we could restrict access to those and monitor access to those. And then the second category would be various services. So you have designed some protein, but you're not sure how it works, and you don't know how to, you don't know, or you don't have the facilities to test how to work some new interaction you've designed. So you send it to a contract research organization or a cloud lab, and you maybe just send them the design, ask them to synthesize and run some experiments, and then tell you how it worked. And I think at both of these, both at the materials level and at the services level, there is a lot of levers we can pull

• 57:59Speaker 01

  to both deter and deny, and then monitor any kind of bad act, or whether it's an agent or a human.

• 58:07Speaker 00

  So tell me more. For one thing, I was interested to see that you had, in the process of zeroing in on a couple of these choke points, evaluated like hundreds of different possible choke points. So I'm interested in what makes these so natural?

• 58:26Speaker 00

  Anything, I'd be interested in other things you learned from reviewing so many different possible checkpoints. And then, again, there's this question of like, don't we need everybody to do it? I think you have an argument that we don't, we can get a lot of value, even if not everybody adopts. But I'd like to, you know, that that sounds like, there might be some relief in that argument for me. So

• 58:46Speaker 01

  Yeah, yeah.

• 58:47Speaker 00

  I want to hear.

• 58:48Speaker 01

  So we didn't need to look at, you know, obviously synthetic DNA as a starting point, but then things from different cell lines to different mammalian models that, you know, mice that you use for testing certain things, pipettes, lots of biosafety cabinets, whole range of things. And what we're really looking at is, one, is there going to be a high signal to noise ratio here? Is this something that is uniquely used in, let's say, virology workflows? And two, is this something that is very commoditized and decentralized? If it's very, you know, you know, in the extreme, if it's something you could just easily 3D print,

• 59:30Speaker 01

  or if it's something you could go buy in a local, any kind of store, then it's going to be very, very hard to have any kind of governance over it. And the things that perform well are the things that, yeah, let's take synthetic DNA as example. Well, it's relatively high signal to noise in the sense that the sequence tells you what it is. You can actually, yeah, understand something about the intent from what is being ordered. And then two, at least if we're talking about gene length DNA, that is a relatively complex thing to manufacture. There's only 200 companies in the world that do sort of gene length, double stranded DNA synthesis. If you're talking about shorter fragments of DNA, then it becomes

• 1:00:12Speaker 01

  more distributed and more accessible, but it's also harder to work with. I think one lesson from that exercise was that there's a lot of things that are just not going to be good. Kind of, they don't have a good signal to noise ratio. And so the only way you could control them would be to effectively ban them. And that's just, in my mind, that's not an option. That would completely hamstring the defensive side of the countermeasures and just more generally, life sciences for human flourishing. You have, you know, a handful of things left that you could try to govern. And really, what you need to do is aggregate signals across them. So you want it to be the case that if someone is interacting with one DNA provider and ordering something that's, it's not an outright red

• 1:00:57Speaker 01

  flag, right? They haven't ordered a gene from smallpox, but they've ordered something that isn't homologous to anything we know from nature, and they won't really say or explain what it is. And then also, in, they're querying a model, let's say they're querying a proprietary model, and that is being monitored, that about how do I avoid the attention of law enforcement? Then these two signals in combination, give you a bit more than they would each individually, right? And then let's say they're also reaching out to a cloud lab and procuring something that looks like it could be suspicious. So we need to set up the kind of monitoring infrastructure that can aggregate these signals. And a nice

• 1:01:39Speaker 01

  benefit of that approach is that it's less reliant on refusals that create friction for people. So you're not denying access to a model, you're not denying access to a material, you're just aggregating the information. And in, yeah, the very, very, very, very rare, severe cases, you might escalate it to law enforcement for investigation.

• 1:01:58Speaker 00

  How long does this stuff take? Like, if you were doing it as a human, because it did strike me as I was thinking through this ahead of time, that one limiting factor we might have on agents is that they just don't run for that long, or maybe we could sort of move toward, you know, not having multi-month agent, life cycles, because we can get a lot of value out of them, like, very quickly. But if, like, the same, you know, rogue process is there, you know, weeks later, when somebody finally gets back to you, that like opens up a whole degree of like async stuff that, for humans, is pretty normal, but for the models, you know, we might be able to just kind of cut them off?

• 1:02:40Speaker 01

  I don't know.

• 1:02:41Speaker 00

  What would be like calendar times?

• 1:02:43Speaker 01

  Yeah, I think there's going to be very strong commercial and otherwise pressures to, you know, increase reasoning horizons and autonomous horizons. I think, in fact, reasoning horizons have been, you know, it's one of the main benchmarks that I think people have been optimizing pretty explicitly against. And I get why. I think for anything really useful, you want to do it on the long horizon. So I think, I think we have to plan for a world where the models can act autonomously over fairly long horizons. And I don't think we should do the thing of then saying, well, let's pour sand in the gears of the bioscience

• 1:03:22Speaker 01

  supply chain, just for the sake of slowing things down. And I, I think, you know, if you look at, one of the use cases of synthetic DNA would be to rapidly create diagnostics in the early days of an outbreak, where you need to synthesize short fragments of, you know, parts of an antigen, for example, to create a, as a primer or a probe for a diagnostic test. And I think that you want to be able to do, yeah, really, really quickly, right? So I think we should be willing to accept some friction in the name of doing KYC, but we should not make it deliberately slow as a tar pit for the agents. I think there are, there are better avenues available to us.

• 1:04:04Speaker 00

  So what do you think, if we bottom line all of this today, and maybe over the next couple of years, like, how should I think about the risk of, let's say, sort of, all of your different scenarios that involve AI together? I mean, let's leave aside maybe the sort of rogue PhD that doesn't need AI to help at all, and try to assess the incremental risk that AI has brought to us in the bio dimension now. I don't, you can maybe, I don't know, even necessarily the right metric to define it, like, percentage risk of, you know, bad pandemic per year would be one way to think about it. You may have a different way of kind of defining it. But what would you estimate that number

• 1:04:46Speaker 00

  to be now? And how do you think it will evolve over the next couple of years, by default, let's say?

• 1:04:51Speaker 01

  Yeah, look, I, you know, actions speak loud on words. And I've devoted my, you know, life to this. I'm going out every day and recruiting people to do the same and fundraising to do the same. And so I'm here because I think we, talking to you, Nathan, because I think we desperately need more people, more diverse talent tackling this problem. So I think it's really, really high on the list of, yeah, societal challenges. I think if we are talking about things that can really outright disrupt civilization, it's also very high up on that list. I don't think it's the only problem, or necessarily

• 1:05:32Speaker 01

  the single biggest problem. But if you look at the function of how many people are working on it, and how tractable is it, I think it nets out as a really, really compelling thing to devote one's time, resources, and attention to.

• 1:05:47Speaker 00

  I mean, I guess I would think that, though, at like, a 10th of a percent per year, and maybe even 100th of a percent per year. I mean, this is why I got into like, AI safety as a, when it was purely a thought exercise, when there were no AIs, you know, I used to tell people, look, the dinosaurs got taken out by an asteroid, and I would hate for that to happen to us. And I guess you would too. And most people would be like, yeah, that sounds bad. And then I'd say, well, you know, good news is, we do have at least a few people who are looking out at the, at the skies and, you know, trying to identify anything that might be coming our way. We should probably do that for AI too, even though, you know, it may be like a remote thing. I guess I'm still trying to calibrate a little bit of like, is that the regime that you feel that we are in, where it's like, it is a, what most people would say is a, like, very low number, but the

• 1:06:31Speaker 00

  severity is so bad that it deserves more resources? Like, I buy that argument. But I'm also kind of not sure if we might actually be drifting up into a range where it's like, shit, like, we actually might need to take more drastic measures at the, like, you know, pause Frontier R scale RL on biotasks, you know, type of thing, which I wouldn't do lightly, because I do share your interest in, like, getting the upside of all this stuff. Like, I, you know, should

• 1:07:02Speaker 01

  You should do an episode with someone, just to talk for two hours about, like, the amazing things we're going to be able to do with bioscience. Maybe you get someone from the, some of my former colleagues from the OpenAI Foundation's life sciences team. I think there's just like, a lot of wonders there to explore. So I'm, I'm total with you on that. Yeah, I, I think, look, I do have, I've done my personal preparedness for disease outbreaks. I think that's, you know, I think that's actually just general good practice, we, for anyone who's lived through COVID to do. I am more, I think, in terms of how to make the case, Nathan, I think it is still the case that it's more of a low probability, high severity event. And then I think the other challenges that, that

• 1:07:47Speaker 01

  patching the planet, patching humanity's immune system, is just a, it's, I think it's a relatively clear task. I think we know what to do, much more so than for other challenges in transformative AI governance. But I think it's a very hard task, and it's a very expensive task. I think the end state that we're trying to get to is, you know, in some sense, it's a culmination of centuries of effort in global health, right? And it's, it's one that we'll have to defend against much worse threats than just nature's own to boot. So I think, you know, that's a state where we have really ubiquitous detection, really ubiquitous measures for suppressing transmission in the initial stages of an

• 1:08:29Speaker 01

  outbreak. So germy side of light, PPE, and then extremely flexible and modular and globally inequitably distributed, I should say, platforms for countermeasures, so that we know when an outbreak is happening, we're not rolled over immediately, and we can, in a way that's actually protecting billions of people, squash it. And I think that, like, that is an achievable end state. It's a very clear and coherent end state. But as I said, I think that's like a decade or decades long project. And so I think we really, really have to get our act together and buy as much time as we can while we're doing that. And that's the, yeah, that's the core of what we're doing in the biotechnology governance

• 1:09:11Speaker 01

  and sort of preventing misuse space that I'm working in.

• 1:09:16Speaker 00

  In terms of the organizations or entities that will train the models that will present the most risk, do you expect that that will be the, like, usual suspect, usual suspects of, like, let's say, five frontier model companies in the United States? Or would we? Should we also expect that like a lot of that will come from pharma companies or startups? Like, is this something that can only happen? Or will not only happen, because everything, the fullness of time can happen. But in a, in the short term, is this something that will be kind of falling out of, you know, the $10 billion training run? Or do you think you could also see it pop up at like Pfizer?

• 1:10:01Speaker 00

  You know, with like a more moderate scaled effort?

• 1:10:04Speaker 01

  Yeah, I think that the risks, depending on what you're worried about, Nathan, I think if you're worried about some millennium prize level or completely unexpected breakthrough in the more fundamental parts of biology, you know, being able to simulate things we thought were completely computationally intractable to simulate. Then I think, yes, I would probably expect that more so coming from a frontier lab. If you're thinking more about just the baseline competence of biology writ large, where you're thinking about the number of people who can competently reconstruct a pandemic capable virus, growing by orders of magnitude, that's going to be a death by a thousand cuts.

• 1:10:49Speaker 01

  And it's going to be a lot of different actors who are, in some sense, contributing that by that, by making different workflows more accessible and effective. Of course, a lot of this is going to converge in the limit, right? And there's going to be a lot of effort to try and vertically integrate different parts of biology so that you can have a single institution that can do everything end to end. Yeah. And we see that at the model level, too, where one of the more interesting papers that came out of Secure Bio and the UK AI Security Institute recently was a, an effort to see

• 1:11:28Speaker 01

  how well frontier models, so I think they tested, I think, Astra and Opus 5.5, from memory, maybe slightly earlier generation, on their ability to, if given a data set and a paper explaining the architecture of a more narrow biological design tool, so one of these design models that are more specialized for biology, could the frontier model train a model like that? And so in that sense, the question of, well, is a risk coming from the developer of the narrow biological design tool, is it coming from the frontier model, kind of collapses

• 1:12:12Speaker 01

  because they're becoming more and more integrated? And I think that's, yeah, a general trend that we're going to see, both in the model domain and in the laboratory domain, this kind of integration over time.

• 1:12:24Speaker 00

  And you would know a lot better than me, because as you alluded to, you did recently wrap up an advisory tour of duty at the OpenAI Foundation. You know, there's an unbelievable amount of money sitting in that foundation, and there's kind of a mirror image at Anthropic. There is a demonstrated ability recently to like, speedrun certain kinds of problems if it stands in the way of continuing training, right? Like, we've seen, oh, we paused training for a week because we found, you know, more defects in our sandbox as well. That's not a very long time. Like, you must have felt pretty good about your ability to patch those things in a week.

• 1:13:08Speaker 00

  I obviously don't think that, like, building, you kind of sketched out, you know, something that sounded like the win condition that you imagine for us here. I don't think that's going to, you know, take a week by any means. But when you say decades, I am kind of like, well, is there a deal here or sort of a, an ultimatum, you know, maybe that the frontier companies should be given by the government? That's like, you guys have hundreds of billions of dollars and you seem to be really good at speedrunning these problems? Like, fix it? Get us to a good place? And then you can resume training? Like, that might be-

• 1:13:42Speaker 01

  Yeah, I mean, look, I would love for it to go much, much quicker than decades. And hey, maybe there are going to be transformative things happening that would allow us. And when I say decades, again, I'm talking about the ambitious win state. I'm talking about how do we actually achieve resilience for the entire world?

• 1:14:01Speaker 00

  True bio-

• 1:14:02Speaker 01

  And I think we should double, yeah, and I think we should, you know, pedal to the metal on that one. I think no one should think that we're on track to doing that. Like, there are sort of subcomponents of it, of could we secure this type of critical infrastructure? Could we, you know, protect this group of essential workers? And those things we can, if we really, really put our minds to it, we can do quicker. But if, if anyone feels reassured that they see some investment in bio defense and they think, okay, therefore let it rip. I think they haven't really run the numbers or the timelines of how long it would take to actually get to the win state. And so, yeah, again, that's why I'm so motivated to say, what can we do to prevent misuse

• 1:14:47Speaker 01

  in the first place while we're building resilience? And I think some of that is going to happen at the model level in terms of, I mean, you were talking about training runs now, but I think a lot of it is also going to happen at the governing, the materials and the outsourced services. And I think, you know, we are seeing increasingly momentum around these ideas. I think you're seeing your lab leaders and synthetic biology leaders signing on to letters, calling for things like mandatory DNA synthesis screening. We just still have quite a lot of work to do.

• 1:15:23Speaker 00

  Zooming out, do you think we are acting foolishly collectively here by sort of doing it kind of at the same time and piecing it together? Like, if, if you were the benevolent dictator, would you take my idea and declare a pause until certain, you know, we used to have this in like RSP form, right? If then commitments and so on? Would you impose those sort of, we need to hit this standard of security as a society? And by the way, like, frontier companies, you know, by all means, use your foundations to like, create the public goods that we need to do it, but you can't-

• 1:16:02Speaker 01

  Oh, yeah.

• 1:16:03Speaker 00

  -do this stuff until we're there. Would you make that call?

• 1:16:06Speaker 01

  Yeah, I would. And I think I would that for a number of reasons. I think bio is the one that I know the, the best, and it's the one where I can most confidently say, you know, we're not on track to having made the world bioresilient by any short timelines whatsoever. We're trying really hard and there's a lot we can do. And we need to, you know, accelerate the effort, and that's why we're, again, recruiting and bringing in people to do it. But if anyone's telling you that we are just, you know, a few more grants away from having a completely bioresilient world, and therefore we can just plow ahead with the underlying, you know, the technology that's contributing to the risk, I'd say that's crazy.

• 1:16:49Speaker 01

  I suspect it's probably in the same in many other domains of, in, of resilience. I think we're seeing it now with cyber. So, yeah, I would absolutely, if I were holding that button, I would absolutely press it.

• 1:17:01Speaker 00

  Yeah, interesting. I don't know if there's anything you could say about your experience as an advisor to the OpenAI Foundation related to that point? Is that type of, is that, is that sort of trajectory like in the Overton window at all in your experience?

• 1:17:18Speaker 01

  Well, I mean, so I, I rolled off my role advising the foundation now a month or two ago, and I think there's been a huge shift in the Overton window, even in that short span. And so I, but I do think I see a, yeah, more generally a seriousness around these considerations that we didn't have even just a few months ago. I think I'm not gonna, so I'm not the best place to say what their official view within the foundation is at the moment. I will say, I think it's, yeah, it's a group of very morally serious people who hopefully

• 1:17:58Speaker 01

  will wield the potentially very significant resources at their disposal with a lot of responsibility and urgency. And I'm definitely rooting for them and, you know, running in parallel with the work we're doing at Sentinel.

• 1:18:11Speaker 00

  But, you know, I hope you're enjoying your role as the benevolent dictator.

• 1:18:20Speaker 00

  Yeah, what would be your instinct on what to pause until what is accomplished?

• 1:18:27Speaker 01

  Uh.

• 1:18:30Speaker 01

  Yeah, I think I'm less, um, you know, I'm less well placed to comment on the what specifically to pause. I think what I am fairly well positioned to say is where are we on the kind of road to the victory conditions in bio resilience? And then that might answer your question around sort of in your simple framework here, when to unpause? And you know, the wind conditions we're working with at Sentinel are around. How can we make it so that the vast majority of providers of north of 80% of providers of certain

• 1:19:11Speaker 01

  materials and services such as synthetic DNA or cloud labs are both doing the kind of KYC that would allow them to refuse an illegitimate query or the kind of monitoring and aggregation of signals that would empower law enforcement to capably intervene? I think that is something that we can achieve in the coming years. And I think there is going to be a time invariably for one reason or the other, where the world wakes up to the seriousness of doing this. And so what we need to do is really build the infrastructure so that we know exactly what are the, you know, what are the DNA sequences we'd be looking for? What are the signatures of misuse we'd be tracking?

• 1:19:56Speaker 01

  And how would we write it into legislation? So when there is the political will to do so, we can scale that up very, very quickly. And then in parallel, there is a sort of the game of getting to the end state on the bio resilience side. Like I said, I think that's, you know, on the order of years, if you're talking about securing specific subcomponents of critical infrastructure and essential workers and a much longer timeline if you're talking about actually making the world pandemic proof. And I think, yeah, if you just sit down and just run the numbers of how many people would you need to get PPE to, how widely distributed would vaccine manufacturing have to do in order for it to be truly global? I think unless you're assuming, I guess, maybe you should assume very transformative changes

• 1:20:41Speaker 01

  in supply chain logistics and manufacturing as well. But in anything under normal industrial capacity, it's, yeah, that's on the order of many years. So, um, but we should shorten that, right? And that's, that's a whole game that that is what the biosecurity community is trying to do is bringing forward in time this end state. And that's how we work is setting an end state and working backwards from how do we get there?

• 1:21:07Speaker 00

  How, how expensive is this stuff? I mean, I, when you think about PPE, for example, obviously there's a lot of different classes of PPE and I'm sure they come at different price points and whatever, but like, if you're looking at a OpenAI Foundation that has hundreds of billions of dollars today and, you know, if their plans come to fruition, you know, that equity will potentially 10x again. There's like plenty of money to buy everybody the PPE that they need now, right? Like they're sending these sort of demand signals into the compute supply chain as much as they can. And you know, Sam Altman was telling Patrick Collison, the, the invisible hand is kind

• 1:21:49Speaker 00

  of failing to some degree because people have been burned by risk or whatever. But like, my guess is if they were ready to write a $10 billion check to buy everybody in the world, I don't know how many masks that would buy you, like, you know, you get a volume discount, I assume you get five each, you know, for everyone in the world and like stockpile those for the next moment that they're needed. Like that wouldn't even be a big part of their budget, right? Well, like, how, I guess, how much can we just like sort of really speedrun these things if we have proper sense of urgency and willing?

• 1:22:22Speaker 01

  Yeah, I think we can certainly do a lot better than we are now. I think the state of stockpiling in the United States, let alone, you know, less resource parts of the world is really abysmal. And I think we would not fare well in a pandemic today. And I think that it is also within the realm of what is philanthropically achievable to, again, like I said, protect critical workers. And so I think, you know, one estimate I've seen is that for a billion dollars, you could equip 30 million critical workers across the US with high performance reusable respirators. And I think the, the value therein lies not just in, okay, you're, you're not just dividing

• 1:23:05Speaker 01

  a billion by, by 30 million and saying that's the number of lives you're saved because if you're enabling critical workers to go into work, you can shelter much, you know, much larger numbers of people from transmission just by staying in. But of course, that's like a really, that's a lockdown state that I think there's very little appetite to be in. And we might do it begrudgingly if we're at the threat of something much more lethal than COVID, but it's not the glorious future any of us are hoping for. And so then you need to get to, yeah, considerably larger numbers of people covered.

• 1:23:45Speaker 01

  And there the costs are not going to be the, the equipment itself, but also the distribution infrastructure and all of this sort of auxiliary stuff that, especially in a crisis where you're, you might have breakdown of social order, trying to make sure that people are actually using things correctly. And yeah, there's just a lot of extra work to be done there, but we should absolutely do it.

• 1:24:09Speaker 00

  Yeah. I mean, it feels small compared to the data center build out is really where I'm kind of coming down on this.

• 1:24:15Speaker 01

  Yeah, I mean, there's a lot of these comparisons that will just make your, you know, mind completely numb if you're thinking about how little we're spending on extremely admirable causes relative to things that certainly have utility. But if you ask a person on the street what they would prioritize, yeah, well, there's a lot of them that would prioritize being higher than data center build outs. So I think it's, and I'm really, really heartened by the sort of, yeah, resurgent interest in bio resilience philanthropy. But I think it's also really important to hold ourselves accountable, not to the, what level of resources have been committed or speculated about. But yeah, have we actually made the world safer for people?

• 1:24:59Speaker 01

  Have we reduced the risk of these outbreaks? So at Sentinel, you know, we've raised tens of millions of dollars. We've made grants to like a hundred different grants and projects, and that's really awesome. But until we've actually made it harder for a bioterrorist or for a rogue agent to get their hands on dangerous materials, uh, yeah, we have not won. And I think that's similarly in the bio resilience space. I think we, we just need to actually hold ourselves accountable to, do people have these PPE, you know, at their home? Are there, I know you, I heard you're at one of your other episodes, you were talking about the struggle of trying to get far UV installed. And I think it was a hospital or a school, right? And like, that's a level of accountability we need to focus on. Like, are these things out there where they're actually protecting people in the US and certainly

• 1:25:45Speaker 01

  beyond the US as well?

• 1:25:47Speaker 00

  Yeah, the public, the sort of man on the street perspective on this definitely cuts both ways. On the one hand, I do feel increasingly like there is often some wisdom in kind of that, you know, naive take on like the macro situation. Because a lot of times people, you know, especially if they're hearing about it for the first time, they're like, wait, you're saying what?

• 1:26:06Speaker 01

  What are we doing?

• 1:26:07Speaker 00

  Yeah, they actually believe that, you know, and then they're like, well, if they actually believe that, then this is crazy. Uh, I think they're basically right about that a lot of the time. Then of course it does also go the other way when it's like, oh, by the way, I'd like to put this little light in the, the school classroom and it's a whoa, whoa, whoa, whoa, whoa, we're not, we can't do that. So, yeah, I mean, can't, can't rely on the man on the street for everything. I do think it's, I appreciate you kind of going down this thought experiment path with me though, because I do have the sense increasingly that like, I've had this since for a long time that we're playing, you know, with a new kind of fire here with AI in general.

• 1:26:46Speaker 00

  And my sense increasingly of the big picture is like, the outside view is it's crazy. And many of us, including myself, very often who have been, you know, relative insiders, if not the ultimate insiders, kind of get frog boiled and, you know, constrained in our thinking by like what other people around us think is reasonable. And it does help to have a fresh perspective. On this bio stuff, I have not, you know, had as much immersion. And so I'm maybe a little more coming from the naive, like this is crazy perspective.

• 1:27:17Speaker 01

  Yeah. So I, I think, I guess my, yeah, my sort of, I think my, my, in a nutshell, my take, Nathan, is I think my timelines to very, very high probability sort of AI driven bio attacks are probably longer than you're intuitively feeling. And I think you can rest easy knowing that, you know, they're, they're like longer than that. I think probably my timeline to how long will it take to actually patch humanity's immune system are also longer than I think some of the very simplistic kind of, oh, you do a botec and you add up the costs of buying PPE and then therefore it's easily solved because this money is potentially coming online.

• 1:28:03Speaker 01

  And so I think I'm, yeah, I think we have, yeah, I guess I just have longer timelines on both. And then on the pausing question, I think, and one of our board members was pushing us a lot on what are the things that we can do on a three to six month timeframe? If we were to reorient all of our work towards that, and I think my answer would be, yeah, I think I would try to spend that time extending the timelines, right? Because I, I think that we're working, yeah, we have got some smart people in this case and we're going to need more smart people. But even so, these are hard problems to patch. And I think we're seeing that with cyber now as well, where the sort of the idea of

• 1:28:44Speaker 01

  giving early access for defenders and then thinking, okay, well, you know, your six month clock starts now, get it in order is like, that's a costly gamble to take. So.

• 1:28:58Speaker 00

  Yeah. And I mean, again, way more so potentially in the bio domain than in the, in the cyber domain. And that could be plenty bad, but you know, this is a different kind of thing. If we kind of, you know, bring ourselves back into the overt and window of what's, you know, probably likely to be possible in the short term. What do you think are kind of the most urgent things, the highest leverage, the lowest hanging fruit?

• 1:29:23Speaker 00

  You know, and I know you have this framework of like owning the problem and you can maybe, you know, present that frame and then give us some problems that you think, you know, are currently going unowned.

• 1:29:35Speaker 01

  Yeah, so I think that frame is this idea of really focusing on getting something all the way to the.

• 1:29:46Speaker 01

  The end state, I wrote an essay about this about a year ago, a year and a half ago. And the thing I was reflecting on is in the, let's say, social sector for people who care about having an impact, I think there's a lot of incentives that spread people thin. And so one is this idea of, oh, you only work on this one issue. Does that mean you don't care about this other issue, which I think creates a social pressure to have a togen effort across a bunch of different things? I think there are professional incentives. I think there are. Another social incentive is I would love to spend my time reading about a bunch of different things. So when I go on Nathan's show, I have something interesting, or when I go to the, you know, the cocktail party, whatever, I have a tidbit of information across a whole spectrum of things that people care about.

• 1:30:30Speaker 01

  And I think what this does is it pulls people away from saying, I'm going to pick one thing and just get it over the finish line. And this is really unfortunate because I think there are incredible returns to focus. I think there is one lesson that we've learned in building Sentinel over the last few years and in that we also pass on to our grantees and to the organizations we incubate. It is these sort of nonlinear returns to focus. And I could, yeah, give a bunch of examples of, you know, a small number of people who really made it their effort to try and say, okay, we have three or four years to solve this problem of securing synthetic DNA, for example. How can we get, you know, laws introduced in the U.S. and in Europe?

• 1:31:10Speaker 01

  How can we get, make sure that the tools exist that makes it easy to comply with these laws and so on? And so I just, I feel very good about that playbook and I encourage others to take that on in spite of some of those social and professional incentives. In terms of question, like, what is an example of that? I think we've made a lot of progress over the last couple of years in securing mail order synthetic DNA. So that's where you go online, you, you know, type in dna.com, what have you, and add the Ebola gene to your shopping cart and then it gets shipped to you. And we've made some progress in making sure that the companies are more likely to be screening that order and rejecting it if it's not legitimate.

• 1:31:53Speaker 01

  But then that leaves the next kind of frontier of vulnerability, which is the printers that synthesize the DNA themselves, which need some level of hardware security. We need some kind of KYC at point of sale. We might need geolocation. You could think ambitiously if you're really trying to throw money at the problem of that buyback programs to buy old machines that are hard to secure and retrofit them. And this is a problem that is, yeah, a really great example of an orphan issue where a lot of people will be thinking about it. A lot of people have maybe mentioned it in a paper or in a tweet, but in terms of people who are waking up every day and trying to say, I'm going to solve this, it's, you know, by my count, a half.

• 1:32:37Speaker 01

  It's like, there's a couple of scattered efforts across a few programs, but there is really, it's a really O-N-L-S problem. So I think that's an example. That's something we at Sentinel are really actively recruiting for.

• 1:32:51Speaker 00

  How many similar such things are there? You can list a few more if you want. And is there sort of a canonical list that you would point people to?

• 1:33:00Speaker 01

  Yeah, so within the specific issue of securing synthetic DNA, I, we wrote, my colleague, Yanki Schmidt and I, wrote a piece for the Institute for Progress, which I think is called How to Secure the DNA Supply Chain, with that on the basis of a couple of years of work, sort of mapping out a whole roadmap and indicating what, which of these issues have existing efforts and which don't. So I think there's a bunch there. I think one that feels very neglected is thinking outside of just the, I think we're increasingly seeing a lot of attention among technologists in the San Francisco Bay area and a lot of attention among, you know, policymakers in DC. And those are probably the two places you would start.

• 1:33:44Speaker 01

  So I'm glad we're covering sort of priority one and two. But I think there's a huge swath of work in the international diplomacy area where, you know, how do we get the very most talented diplomats in Geneva or, you know, international relations people traveling between Beijing and Singapore? How do you get them to take this issue very seriously and reorient from working on something related to the UN Security Council or what have you, towards solving some of these extremely urgent biotechnology and more generally technology governance challenges that I think should be one of the most compelling and prestigious things to work on. And that's a, yeah, that's one of the really big challenges we have.

• 1:34:26Speaker 00

  Our work's cut out for us. Maybe last question, just because I do think it is important to be intellectually honest about the trade offs on these things, if we are talking about any sort of pause or slowdown or whatever. And it's also important to be inspired and have, you know, a positive vision for the future. I'd love to hear just a little bit of a sketch of kind of your positive vision for the future. We focus obviously a lot here on downside and risks and how things could go wrong and what we might ought to do about it. But you mentioned at least briefly that the reason you do this work is you want to see us actually realize the upside of biotechnology.

• 1:35:04Speaker 01

  Yeah. So in, in bio, I think there are two components to it. There's sort of the wartime and the peacetime. So on the wartime, that's the pandemic outbreak, which even if we were to eliminate all misuse, we will be at risk from nature. And that has been, if you look at, if you tally up the deaths, the greatest causes of death of the 20th century, things like smallpox and pandemic influenza dwarf, you know, many wars, including the world wars, right? So taking that off the table by getting to this end state of extremely widespread and

• 1:35:46Speaker 01

  sensitive and specific and rapidly acted upon detection, very, very fast and modular diagnostics, I think we're beginning to see the inklings of that in, in how we dealt with COVID. I think a lot of how we handled testing was not well executed, but in terms of actually being able to produce the tests very, very quickly and design them within the first few days of sequencing the genome. I think that's a really inspiring step forward. I think we're seeing it with then the further end game of, okay, how do we, we've identified the pathogen, we know how to diagnose it, how do we knock it down?

• 1:36:26Speaker 01

  I think mRNA vaccines was a real step change in our ability to deal with emerging infectious diseases. And I would love to see many dozens of step changes like that. So I think that is a world where we then are no longer at risk and we're no longer trying to haggle our local school to set up RUV and including people who are immune compromised, live with just like less peril of emerging outbreaks. And I think that would be incredibly freeing even just from, like, there's a lot of people who live at the, because of different conditions, like they just like perennial exposure and fear of outbreaks.

• 1:37:11Speaker 01

  And I think a lot of people still carry that with them after COVID. So I think taking that off the table, as, as we called it at Sentinel's mission statement, is making pandemics history within our lifetimes. I think that captures it pretty well. And then I think there is the, the non-infectious disease side of it, and that's not my area of expertise, but I think anyone who, for example, has, yeah, people who live with chronic illnesses or with cancer are, feel like, and you know, any amount of delay in taking those off the table is unbearable. And so I think there's an enormous amount of work to do there. And then I think just adding to all of that, and this is often missing in the conversations

• 1:37:54Speaker 01

  among technologists, is the equity component is, I think people, you know, people talk about the future being unevenly distributed. I think that is definitely their default. And I'm extremely suspicious of the arguments that we will by default distribute the spoils of this technology. And so we need to build that in like as, as early as possible. And I think we're, yeah, we can, but I think we also can do a much better job of that. And I hope we will, because I hope people will, yeah, hold power to account on that one.

• 1:38:25Speaker 00

  That could be a great place to leave it. Is there anything else that you would want to leave people with or touch on that, that I didn't think to ask?

• 1:38:34Speaker 01

  Yeah, I think that this is, yeah, the candid assessment is that it's a, it's a pretty dire state of affairs when it comes to bio risk, but also that I think we have, you know, many more wins ahead of us than behind us. And I think there's an enormous amount of really clear and tractable work we can do over the next few years. And so if you're a legislator who's donned it by the complexity or someone who's conceiving, pivoting your career into working on this, or a philanthropist thinking about funding work in this space, I think we have to take some courage and some hope that there's an enormous amount we can get done. And it's, it's very easy to underestimate what we'll be able to do over the coming years. So we just got to get to it.

• 1:39:18Speaker 00

  Joshua Monrad. Thank you for being part of this.

• 1:39:20Speaker 01

  Thanks, Nathan.

• 1:39:37Speaker 02

  Tell me I'm crazy. Crazy. And I will believe you. Tell me the floor isn't tilting, I'm dreaming. Tell me the dark at the window is evening, darling, I'm begging you. Tell me I'm crazy.

• 1:40:00Speaker 02

  The band's playing faster than anyone's dancing. And nobody stumbles and nobody's asking. The clock on the wall has too many hours. And everyone says that the night is still ours. I would love to be wrong, oh I. Love it so dearly. Look me in the eye. Tell me I'm crazy. Crazy. And I will believe you. Tell me the floor isn't tilting, I'm dreaming.

• 1:40:39Speaker 02

  Tell me the dark at the window is evening, darling, I'm begging you. Tell me I'm crazy.

• 1:40:54Speaker 02

  Everyone's smiling the smile they were wearing An hour ago and an hour before. I say there is smoke, you say, "Dance with me, darling." And off we go, round on the tilt on the floor. They so calm because they are so wise. Or only because no one's looked at the sky. Would love to be wrong away. Love it so dearly. Look me in the eye.

• 1:41:37Speaker 02

  Tell me I'm crazy. And I will believe you. Tell me the floor isn't tilting, I'm dreaming. Tell me the dark at the window is evening. Darling, I'm begging you. Tell me I'm crazy. If I'm wrong, I will dance until morning. However long it takes morning to come. If I'm right, let's not say it out loud.

• 1:42:20Speaker 02

  Just look me in the eye. Tell me I'm crazy. And I will believe you. Tell me the floor isn't tilting, I'm dreaming. Tell me the dark at the window is evening. Darling, I'm begging you. Tell me I'm crazy. Go on. Tell me I'm crazy.

• 1:43:05Speaker 00

  If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries, either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts, which is now part of A16Z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at ai podcast dot I N G.

• 1:43:50Speaker 00

  And thank you to everyone who listens for being part of the Cognitive Revolution.