🔬Bio-security is an AI Arms Race - Eric Nguyen (CEO, Radical Numerics)

The OpenAI → Hugging Face attack has people asking “what else do we need to worry about?” and Anthropic’s filters flag two things: cyber-security and biology. The natural question is: what about bio-security, then? Clem Delangue argues that cyber-warfare defensive capabilities need to be open and to keep pace with frontier models’ attack capabilities Radical Numerics co-founder Eric Nguyen sat down with us and explained why the same models that increase biological capability can also keep defense from falling behind. Building a virus from scratch While he was at Stanford, Eric couldn’t get traction on Genomic Language Models (GLMs) for a long time. Biologists didn’t believe it would work, didn’t think they could verify the output, and didn’t see important applications beyond what they could already do. He kept pushing, eventually helping lead the development of Evo and contributing to Evo 2 at Arc Institute. Those models were later used by a separate Arc/Stanford team to generate entire bacteriophage genomes that were synthesized into functional viruses https://www.biorxiv.org/content/10.1101/2025.09.12.675911v1! Long context unlocks biological intelligence Early ChatGPT spit out poems and email, and early DNA language models like Evo and Evo-2 could build a genome from scratch. DNA is different, however, from natural language in that it has a very small alphabet (4 characters ACTG) and that its sequences are very long: * 60K for an average human gene * long being up to 2.3M * the whole human genome around 3B. Innovation in long-context models https://github.com/togethercomputer/stripedhyena made this possible about 3 years ago (footnote: striped hyena), long before the frontier labs were building 1M+ context models. Now Eric and other AI x Bio luminaries have founded Radical Numerics to build and scale GLMs to tack a wide range of biological problems, extending well beyond generating DNA. Thinking in DNA Their GLMs already do pretty well with RNA and protein because there are clear markers in the DNA sequence for genes (RNA sequences the perform many functions) and specific genes that encode proteins. This means that the models already generalize to multiple “languages,” before even attempting to train in other modalities, such as 3d protein structure, epigenetics and natural language. If a model thinks in the DNA language, maybe it understands the imprint that environment left on different genomes as well? Perhaps the model has learned the functional relationship between different sequences, and could extrapolate to new sequences based on that? And so what we wanted to showcase was that if we show the model progressively better RNAs in a series of steps with its score, right? So you have like low scores first and then you gradually move up the chain. Can the model continue that trajectory on its own? And then in the final step, does it self optimize to a point where it's like the best score it can get? That was the experiment. Can we do that? And so we took a data set, a large data set of aptamers. We held out a portion of the best performing ones and we showed it only the lower ones, but then we ranked it, right? So we showcase lower scores with the RNA aptamers and then progressively got higher, and then ask the model to just like continue with that pattern. And it turns out it was able to recapitulate some of those higher scores that we had not shown it yet. So, voila: chain-of-thought, thinking in DNA! The arms race But much as long-context inference, chain-of-though and multi-modal perception unlocked sophisticated reasoning in natural language LLMs, these capabilities in GLMs are enabling increasingly sophisticated “biological intelligence,” and along with it, greater danger. According to Eric, defense is currently losing this battle, but Radical Numerics argues to push the frontier harder! I won’t spoil the details for you. In the episode we talk in detail about: * Biosecurity as an arms race — and how defense can keep up * The genome as the imprint of the environment on DNA * Going truly multi-modal * How chain-of-though works when you “think” in the language of DNA This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe https://www.latent.space/subscribe?utm_medium=podcast&utm_campaign=CTA_2

• 0:00Speaker 00

  The design side is going to get more capable. The defensive side needs to try as best to get ahead. So I think inherently there is this arms race style dynamic that the defensive side has been far, far lagging. And so what we want to do is bring the defensive side to par essentially. We felt it was important as a lab that a team that was both building the design capabilities is actually also best suited for building the defense capabilities because they're basically the same models. And a model that is good at generating turns out is also very good at discriminating or predicting if a sequence is pathogenic or not. For us, we as a company thought it was very important to have a dual mandate.

• 0:39Speaker 00

  It's this idea of essentially being cognizant and feeling responsible for the capabilities that we're enabling on the design side. So if we're going to create models that can design function into sequences, we believe and see a gap in companies being able to safeguard that technology.

• 1:00Speaker 01

  Welcome to LaneSpace. I'm Brandon. I build RNA therapeutics at Atomic AI. I'm joined by my co-host RJ Honnicky, CTO and co-founder of Mirromix. Today, it's a pleasure to have with us Eric Gwynne, CEO and co-founder of Radical Numerics. Eric started, got his PhD in Chris Ray's group. He spent a lot of time thinking about how to do long context genomic models before long context or genomic models were cool. He was the first author and I think basically visionary behind the Evo generative model, one of the first generative genomics platforms, developed Evo 2, which naturally led into Radical Numerics. Yeah. Thank you for being here.

• 1:45Speaker 01

  Did I miss anything?

• 1:46Speaker 00

  That sounds great.

• 1:47Speaker 01

  Cool. Welcome.

• 1:49Speaker 00

  Thank you.

• 1:50Speaker 02

  So Eric, let's talk about Omni and the blog post that you guys did about the benchmarking. But I want to hear first, okay, what is a genetic language model? Why do I care? What does it do? And then let's talk about the top line results from the blog post.

• 2:08Speaker 00

  So a genome language model or GLM is a large language model trained on DNA sequences. So very much like natural language and chatbots you see, but not trained on words or natural language, but on the raw fabric of life, which is these sequence of letters that make up DNA. And we ourselves, our company, our team is known for creating the first generative genomics models, which are models trained on DNA, not just to read, but also write, meaning able to generate new sequences of DNA. And we felt this was an area that was overlooked and that if AI could read and write DNA, it could change a lot.

• 2:53Speaker 00

  Science of discovery and understanding of human health and how to treat it. And so we felt there was a big opportunity to train AI on the genome.

• 3:02Speaker 02

  What kind of things can you potentially do with a model like this?

• 3:06Speaker 00

  Great to start for us. When we first started working on DNA models, we worked on this model called Hyena DNA, which is a large language model, but it used a convolution instead of a tension. So a little more technical details. DNA has this property that, well, it's very long, right? At the time, these large language models had limited constraints on context, right? Being able to fit long sequences. And so we were looking for a more efficient algorithm to be able to handle something like DNA. And so we came up with this, what we call the Hyena operator, use convolutions. Long story short, it let us process longer sequences, in this case up to a million, and

• 3:49Speaker 00

  at the time was the largest context for a language model. And what we did with it was essentially use it to read DNA, predict function. So given a sequence of DNA, string of characters, we predict its regulatory function, its effect on a genome. And this is interesting to scientists because a lot of the DNA in our bodies, perhaps people are less aware, but actually we don't know a lot that much about our genome. We know it obviously, it encodes the information for making us us and how all the different complexities and potential diseases.

• 4:30Speaker 00

  At the same time, the grammar, sort of grammar rules about how the combination of those letters are sort of formed, what they encode and how they encode function and traits is not fully understood. And so the hope was using these DNA models, language models, to be able to map some function from the raw DNA sequence. And so our first generation models was able to show that yes, we can train AI to be able to read and understand to some degree DNA sequences and especially what we called the longer range interactions, meaning over sequences. You know, if you use chatbots, for example, like if you've heard of the phrase context rot, you know, the longer the input you put into a language model, it starts to deteriorate.

• 5:14Speaker 00

  And so being able to pick up long range information and sort of patterns, motifs, grammar over long sequences was what we were trying to accomplish. And so we showcased in that first generation of Hyena DNA that that was possible over a million context. And then really what started the field now known as generative genomics was the model called Evo. And Evo, we tried to showcase there was this idea of not just reading DNA, but being able to generate it. And so we wanted to accelerate essentially how biologists and scientists had learned from biology and particularly genomics.

• 5:53Speaker 00

  And we felt like this whole field of generative AI being applied to language, great, accelerated the, obviously our understanding and ability to manipulate the natural language. But here's this other language, DNA, the genome, that we don't understand and it's barely being applied with AI in our, at the time, a few years ago. Models that we saw were really small, short contexts, so they could only pick up small patterns and limited context, and none of them generated DNA. So they all just would read. And we felt that the idea of generation was so powerful and transformative in natural language, what if we could bring that to biology and DNA in particular?

• 6:35Speaker 02

  What can you accomplish that you can't do in a lab? Right? So what does that unlock for you if you could do that very well?

• 6:42Speaker 00

  I think one of the first things that we showcased that got folks sort of intrigued by the potential of this was a CRISPR-Cas system. So it's an enzyme that's able to cut DNA itself. And I think what was particularly enabled by the Evo models was the ability to generate over not just one modality or one type of sequence, but spanning multiple modalities and spanning multiple scales. So CRISPR-Cas, it's a molecule made up of both RNA and proteins. And so at the time, you hadn't really seen models that can generate multiple modalities.

• 7:27Speaker 00

  You can, they had protein language models that can generate proteins. Sometimes you had RNA models that could generate RNA, but you didn't have a single system to sort of co-design. We showcased that a single DNA model, sort of the foundation of both of those, right? From DNA, you can get RNA and proteins, that you can design a single system to generate and also function in the real world. So we asked Evo, we showcased it a bunch of natural CRISPR-Cas systems and essentially asked if can you, can you make a new one? And we're able to sample from that model that we trained. And indeed, we showcased that Evo was able to discover a new CRISPR-Cas system.

• 8:10Speaker 00

  And you know, folks were intrigued by it and cover of Science Magazine and later get to give a TED Talk about the work, which is interesting because obviously the audience is very general and so trying to make, you know, what is CRISPR-Cas system? What is DNA? And how do you generate it? Why? Why would you generate it? All sorts of fun topics. And I think the more even intriguing, exciting thing that scientists eventually, just last year showcased what you can do with a generative DNA model was to generate the first genome from scratch using AI. So this is something not possible by humans, right? Humans usually, you can think of it like copy and paste parts of other genomes or other DNA, put it into something else. But they would just take out small motifs that, you know, they know the function and they understand the rules.

• 8:53Speaker 00

  But to build something from scratch, from the ground up, had not been done before at the genome level. And so Evo turns out was able to generate a functional genome. And in this case, it was, it's known as a bacteriophage, also known as a virus. And this, this was a key turning point for, I think, the scientific community and for us as a company at Radical Numerics, because we felt this was such an indicative manifestation of the potential, right, to create a whole organism.

• 9:24Speaker 00

  Not existing in nature, but also the potential harm that that means as well. If you can control, if you can manipulate the fabric of life itself, control its function, what kind of implications does that mean? What are you enabling into the world? And so we actually got a lot of feedback, a lot of comments, a lot of outreach from folks, both excited and concerned about this capability, this kind of capability and just the trajectory. This is the early stages, the first thing one can sort of project and imagine what this could lead to. And so we felt as a company, it was important to not only push on the biological design capabilities of these models, but also the ability to use them as defensive tools for

• 10:07Speaker 00

  the potential of misuse and biological risk that emerges. And I think indeed, a lot of companies, a lot of frontier labs are also being concerned about this emerging risk of AI models being capable of designing biological sequences. And at the same time, it's being mostly attacked from like a natural language standpoint, like safeguards and things like Claude. If you talk about viruses, it'll just like shut you down, which is great. I think to some degree, you need safeguards at the natural language level. But I think what you also need clearly is safeguards at the biological sequence level too. So you need models that not just can understand language and the trajectory of your chat,

• 10:51Speaker 00

  but to understand the substrate itself is the next step in ultimate limit. If you can have models can understand if the sequence is pathogenic or virus, that's the level of defensive capabilities that you want. And then being able to push that out into surveillance systems, national security of being able to monitor emerging sequences in environment. This is what that kind of capability makes possible. And then we think bringing this frontier technology to that community as well is also important, just as important as using this for human health, which is what we primarily focus on.

• 11:29Speaker 02

  There was this evolution that you mentioned, there's the hyena DNA, and then there's Evo, Evo two, and now Omni. Can you just talk about a little bit about Evo and Evo two struggled to beat specialized models across many tasks, whereas in the blog post, you talk about how across a wide range of tasks that Omni is actually able to outperform them now. So can we just talk a little bit about that?

• 11:57Speaker 00

  So yeah, Evo was intriguing to folks in many ways, showcased the potential for applying to multiple types of modalities. But it's still, it's still in many ways, underperformed sort of the specialist DNA models, especially on human genetics or genomics. And so although Evo was competitive, it still wasn't state of the art or kind of pushing the needle. And so some parts of the community thought like, why use a giant LLM when I can use these smaller, more specialized models? And so what we wanted to do with Omni was amongst many things, but one of the first things was to showcase this idea of mid-training and post-training or broadly alignment.

• 12:40Speaker 00

  So the way we think about language models and bio and genomics in particular, is that mostly we've only seen base models trained. So they're pre-trained, but they're basically unaligned. So in the analogous space for natural language, it's like you're doing all the pre-training, but to make it actually useful in the real world and answer questions that users actually want and that is in the form factor they find actually informative, there's a bunch of alignment and post-training and mid-training done to get the models to be production ready and actually useful. And so we felt Evo was just showcasing the potential of that pre-training, but Omni

• 13:22Speaker 00

  is a step of actually making it useful for folks like scientists. And so we spent a lot of time on alignment and mid and post-training, which is essentially showcasing tasks in the form that people generally would want to understand about genomics. Given a wild type and a mutation sequence, help me find the causal variant. These types of questions and form factors for how you might want to analyze genomics doesn't just emerge necessarily easily on its own from pre-training. Pre-training is this next token predictions task or infilling. And basically I think of it as like the raw pattern making ability that you're teaching it is in the pre-training.

• 14:06Speaker 00

  But then taking those learned embeddings or features and pointing at specific tasks or a bunch of tasks really, in aligning it, meaning have it show you the output in a way that is meaningful to you, takes a little bit of teasing and manipulating that so far the frontier labs are the ones that drive that research in the natural language community. And so we wanted to bring a lot of that research and more to genomics. And so I think Omni is really just a preview to showcase that potential. And I think once you do that even just a little bit, we were surprised that it did start being state of the art and pushing the boundaries, not just being affected of

• 14:51Speaker 00

  multiple tasks just broadly like Evo was, but actually pushing the frontier of each of those areas of variant effects prediction, causal mutations for disease. It could start actually being useful for human genomics. And so we're really excited to share that with folks. And it was just a preview in that sense, because we were still actively training and incorporating additional techniques into the model, like additional modalities. But I think we were basically too excited and we wanted to get this in the hands of folks faster. And some of the feedback we got in early interest, lots of hospital systems, nonprofits that have tons of genetic data.

• 15:32Speaker 00

  And for example, they know there's some kind of condition or symptom for the patient, but they can't figure out which parts of the DNA are causing it. So these got these VUSs or variants of unknown significance that we're extremely excited to apply these models to and actually help diagnose a lot of these patients is one example of a real use application.

• 15:54Speaker 01

  I'd like to talk more about the applications in a bit, but I am curious just from a technical standpoint, like what does it look like or what does it mean to align a genomics model? I can imagine with large language model, there's a sort of natural chain of thought. RL, there's like a clear paradigm here. I'm curious, what does it look like for genomic model, which is not maybe better described as something like fine tuning in a different language?

• 16:19Speaker 00

  Yeah, I mean, I think in many ways, one could describe it as fine tuning, but then introducing, I'd say sort of the key components are the right structure of the inputs. So feeding them in a certain sequence so that the model is aware that a certain task is being asked of it. So there's a mix of special tokens to basically, you can think of it as like, if you're going to do disease prediction for disease A, you expect this special token, right? Just kind of like a way to prompt it. If you expect it to do design, have another special token and then showcase the examples kind of like in a chain of thought manner, meaning showcase a sequence of desired outputs and the trajectory of it.

• 17:04Speaker 00

  This is a little vague sort of intentionally because it's part of our, part of our secret sauce that we're still developing. And I think over time, we want to showcase more and more of it. But in many ways, it does mimic a lot of the natural language community. It's a lot of it is fine tuning, but really it's also carving out specific data sets that you want it to focus on and then structuring the questions or tasks in specific ways as opposed to pre-training is really, pre-training is really just feeding it in and everything in really, and just doing next token prediction or mask infilling, if you're doing mass language modeling. And it has no sense of like this Q and As type structure where you have a question, a prompt, and then an output.

• 17:47Speaker 00

  And you could think of mid and post-training as starting to showcase, given this type of input, I expect this type of output, whether it's score, a prediction score or a design is largely the mid and post-training. And the post-training also includes things like reinforcement learning too. But I think the bigger steps are introducing structure of like question and answering.

• 18:09Speaker 02

  Do the models have multiple heads that are task specific? Or are you training one set of heads or whatever that can answer multiple questions at the same time? Just change the input tokens or whatever?

• 18:25Speaker 00

  Yeah, broadly, it's a little, you can, I think we're flexible on this. The idea, yeah, sometimes you can use different heads, but the idea for us is to unify more so. And so I think early experiments, we did have different heads, but in some cases, the different heads do better. In some cases, the, you know, a single head, a single model does better. And so I think we're flexible on that. But the, I think broadly, the direction that we are moving toward is a single. And the reason for a special motivation for that is that we're trying to unlock a lot of modality, a lot of generalization.

• 19:06Speaker 00

  And I think the more unifying we're able to make these models, I think that's when you see more emerging capabilities happen. And in large part, that's what motivated the DNA work. We felt like a lot of models were specialized into other modalities, a little more downstream from DNA, so like RNA or proteins or molecules. We largely felt DNA is the foundation and that from DNA, you can learn a good deal of other modalities, potentially all of them. And I think of other modalities as sort of like additional context that you're showing the model. That's how I kind of view it philosophically in my head.

• 19:48Speaker 00

  But yeah, the idea that single models unifying across modalities scales is what the lab builds toward.

• 19:58Speaker 02

  Can you walk through a few of the tasks that you talk about in the blog and just explain and remembering to? To narrate for the listener only audience, but talk about some of these top line results and we can maybe dig into them a bit.

• 20:18Speaker 00

  Sure. One of the areas that we thought was really interesting for showcasing in this particular release for Omni was on understanding variants and their effects, which are essentially in DNA, a change in a position, you know, change in the letter of one of the other three letters in your genome in DNA, sometimes can cause a disease and sometimes it doesn't do anything. Actually, many times it doesn't do anything. But there are specifics, specific areas in your genome that if you have a different variant or a different letter there, it can cause disease. And in many cases, because the combinations of these changes in your, in the genome, over three billion letters, right?

• 21:00Speaker 00

  It's so vast that for clinicians and scientists, we actually know only a very small portion of which variants are causal to disease or not. And so there's these benchmarks from folks who collect, you know, variants, different hospital systems and clinics, and some are known and some are still, you know, unknown. Folks have created some benchmarks from ClinVar or Trachym to basically the ideas, given a mutation in a DNA, can you tell if it's going to cause a disease or not? Right? And so this is a very good setup for DNA models because

• 21:40Speaker 00

  they're probabilistic. And so when you do make a change, it basically can modify its confidence or the probability of predicting the next letter. And in this case, we've sort of leveraged that predictability of these models. They've essentially seen and been trained on so much DNA, in particular, human DNA, they kind of understand what's common in cells and usually common or conserved across different other folks, usually typically means healthier. And so if it's less common, you can think of it this way, if it's less common, the model can pick it up and, and sort of predict that it's potentially pathogenic or disease causing. And so we've taken some of these

• 22:20Speaker 00

  benchmarks. And when you introduce a variant or a mutation, there's different types of mutations and variants. So sometimes you can delete a letter altogether, you can just flip it, you can remove big portions of the DNA, but these are generally single variants. And in this case, this is where previous DNA models really struggled, especially on humans. And so we showcased that not only is it, you know, competitive or capable for humans, but in many cases, it's state of the art and actually mostly these cases, it's state of the art. And I think the exciting part is that areas where other models that were, you know, currently on the frontier, they still were lagging

• 23:03Speaker 00

  behind quite a bit in terms of where in the genome. So in the genome, there's coding and non-coding regions, like protein areas, protein regions.

• 23:11Speaker 02

  Meaning areas that are actually the coding, the structure of a protein versus other areas that do other things like regulate what genes are expressed.

• 23:22Speaker 00

  Yeah. And so these non-coding regions are largely regulatory, they kind of control how much or when to use a certain gene or turn them on. And in many cases, these non-coding regions, these regulatory regions, variants there, mutations there are much harder to predict if they cause disease or not. And I think what's exciting thing about the new generation of models we're building with Omni is that that's where we shine especially. The models are able to pick up mutations and be able to distinguish if it's disease causing in these non-coding and especially long range areas. So I think this is particularly exciting to a lot of geneticists that have struggled to use traditional bioinformatic tools or statistical methods because they've largely

• 24:05Speaker 00

  focused on coding regions, which is only about, you know, one and a half, two percent of the genome. And turns out many, if not most of the diseases are in these non-coding regions. And so there's been a real desire to build models that can actually pick up these variants of, of disease causing variants in these non-coding regions.

• 24:25Speaker 01

  I have several questions.

• 24:26Speaker 00

  Yeah.

• 24:26Speaker 01

  First, just while we're here, you know, for the, for the listeners, there's this column on this benchmark chart called Borzoi, reference number four in the blog post. I think for maybe some historical context, could you talk about what this column represents? And, you know, maybe this also helps give context for, you know, the Omni column on the right.

• 24:49Speaker 00

  Yeah.

• 24:50Speaker 01

  Yeah.

• 24:50Speaker 00

  Yeah. Great point. So what we show in this benchmark here is really taking some of the representative models or the strongest models in the deep learning side and also in the traditional methods. So we have Evo two is the latest previous genomic model that our team had worked on. And then Borzo is, it's also a DNA model, but a very different kind. Essentially, it's a supervised model that predicts from DNA functional genomic tracks. So it, it too is inherently multimodal, but it's not, it's not a language model. So it doesn't predict like a next token prediction. It goes from a DNA sequence directly to a functional genomic track like chromatex, accessibility or gene expression.

• 25:29Speaker 02

  And these tracks have been annotated extensively by people writing their dissertations and whatever.

• 25:35Speaker 00

  Yeah, yeah. Right. So the big difference there is that it's a supervised task, right? So it requires labeled outputs. And in, in our case, these language models, they do not require labeled outputs, right? So you're, you're doing raw pre-training on unannotated sequences. So, you know, why is that desirable? Well, there's a lot more data, a lot more genomic data that's not annotated. Actually, most of it, pretty much in many ways, all of it, almost all of it is not annotated. And so being able to learn from an unsupervised manner, hugely desirable, right? For us, we wanted to showcase the benefit of pre-training on raw genomic sequences and, you know, comparing it to state of the art models in other, in other spaces in DNA.

• 26:18Speaker 01

  And so your prediction is, when you say it's unsupervised, how does the unsupervised property work? Like, like, how do you convert the output of whatever your model is to an actionable, like, ranker, scorer, whatever?

• 26:32Speaker 00

  What I talked about before is, you know, to train the Omni model is pre-trained, so it's unsupervised. But when you're pointing it at specific tasks, there is a, there is a supervised step. So it's just, you know, taking a smaller data set that is labeled, but essentially what we're doing is using the likelihood scores. So the raw outputs of the language model, which basically you could think of it like a probability for predicting what the next letter is, we can essentially showcase the probability score, the likelihood score for the mutation versus the wild type. So that's seen in the reference genome versus the mutation in this particular case. And then we'll have two scores. And then you can think of it as like using a, a ratio of

• 27:16Speaker 00

  the two to showcase basically how different are you from, how different is this mutation from, from a normal or baseline, basically. And then that's, you could think of it as like a surprise factor that the model is able to use and leverage. And then we can use that to essentially score an actual prediction for disease or not. Does that make sense?

• 27:38Speaker 01

  Yeah. So Omni auto-regressive is that, or is it diffusion or something else that you can't tell me?

• 27:45Speaker 00

  Yeah. We, yeah, this time we, we're not describing the exact makeup, but Evo was auto-aggressive. I think that was the first large scale auto-aggressive. And so I think for us, we don't tie ourselves down to a specific training objective. We use every tool in the box, toolbox, essentially, you know?

• 28:08Speaker 01

  Okay. But for the specific benchmark, you're going along and you're just using the, the likelihood distribution of, you know, the tokens and some tokens are, you know, the model thinks these are unlikely. And that is probably because some evolutionary constraint, like this doesn't show up often. And because it doesn't show often across genomes, it is probably going to cause problems and, you know, people will not survive, so on. So you think that is a, basically your, your ranking metric or something?

• 28:39Speaker 00

  Yeah, it's, it's one interpretation of, of what, you know, how the model's thinking about it. And, you know, very similar to in natural language, you can describe the same kind of paradigm. And there is additional case in mid and post-training to leverage more than that, I suppose, because we can teach it specific structure and benchmarks so that.

• 29:02Speaker 00

  It can build on top of what you just described, which is like what's common in nature, but also because it's a specific task for disease variant prediction, then the model has additional training introduced during mid-training to showcase and to add additional learning power essentially.

• 29:20Speaker 02

  What are some examples of that? Again, probably secret sauce to some extent, but can you give just a gist of what that looks like? What are the kinds of things you would throw in there?

• 29:29Speaker 00

  We would actually throw in the score to itself. So, you know, that like that I mentioned, a ratio, it's a ratio of wild type versus mutation. I would say that's more of a zero-shot method where you don't even have to do any mid-training. And that's what Evo2 is doing in particular in this column. So Evo2 is not fine-tuned essentially. There's a E V E column, which people are basically fine-tuning, you know, using them scores from Evo2. That's from Goodfire. And that's also, you know, folks that we greatly expect and they kind of showcase that these models are able to be state-of-the-art when you can, when you can fine-tune them as well, not just zero-shot. And then our model is introducing sort of a step about that, not just fine-tuning, but

• 30:12Speaker 00

  also introducing structure into, by structure, I mean the format of these benchmarks into the model itself, that Q and A style formatting during mid-training, which is what gives us an extra boost even.

• 30:25Speaker 02

  I see.

• 30:25Speaker 00

  An extra boost, but I think the other benefit too, that, you know, we didn't emphasize too much in the blog, but I think it's really convenient for practical use for scientists is that you don't, you're doing this without taking the embeddings and then tapping on a head and then, you know, doing some regression, which is the extra step. It's extra hurdle. Can you imagine ChatGPT if like every time you ask a question, you had to like fine-tune it for a certain domain? We've done it so that the model is flexible during mid-training to be trained on many tasks at once. And so the, that fine-tuning, that, that last step of training the embeddings doesn't have to be done. It's out of the box at that point. You just prompt in a certain format and it will, you know, that certain format tells it which task you're going to do.

• 31:09Speaker 00

  And then we'll output the answers in that, in the desired format, basically.

• 31:13Speaker 01

  How careful were you in designing this post-training scheme to avoid kind of data leakage with the ClinVar, trait gem, RNA gem and so on? Like these data sets? Like how, like how confident are you that there is no data leakage, either like accidental or something upstream in that, you know? Because I would not be surprised if a lot of these sequences showed up also in your training data, even in a, you know, unsupervised sort of way.

• 31:41Speaker 00

  Yeah. The short answer is we're extremely cognizant of the risk of data leakage and in extremely hard to not mislead or, you know, be careful. And so we, yeah, we have bioinformaticians that are able to basically comb through the data and curate, dedupe and align sequences to make sure that things that are similar potentially to what's in the benchmark are not there. And if they are there, we remove it. And so, yes, we actually have steps to, to QC the data quite extensively.

• 32:11Speaker 01

  Yeah, cool. Yeah, I guess maybe before we move on, I think it's really cool seeing that there are these supervised methods, which previously several of these numbers were, let's say, within the airbars, if not just straight up beating what came before them. And now you have significantly improved upon that.

• 32:28Speaker 00

  Yeah, we're super excited. And I think for the longest time, this is area of this other method called CAD, which has been state-of-the-art and state-of-the-art for a reason, which is what the, sort of by design, they'll take the best methods and kind of do an ensemble, right? So they'll take, they'll take up another, even if the best method is another previous model, they'll mix it with like an SVM and just like throw the kitchen sink at it. And so you can see why it would be the best, right? And so that was the bar for us. We're like, if they're going to throw the kitchen sink at it, like we're not going to, you know, cherry pick one model and say we're better than that. We need to be what's possible, humanly possible now, like across everything.

• 33:05Speaker 00

  And so, um, yeah, our researchers were, were setting their sights on that to, to see if they can actually improve performance across every method.

• 33:17Speaker 02

  I'd be interested to see there, there's a, some discussion of chain of thought. And that, that broke my brain a little bit when I was first, first hearing about that. I'd be really interested to hear about what that even means.

• 33:33Speaker 00

  Yeah.

• 33:34Speaker 02

  I had to pour through the blog post to really understand that.

• 33:38Speaker 00

  Yeah. Yeah. I think this is a really just a taste of where we think the design capabilities can move toward and way more usable for folks. So chain of thought stems from natural language community. I believe Jason Wei at OpenAI showcased the first examples. And really what the breakthrough there was showcasing that these language models perform better, we just show your work essentially. You showed the steps of how you came to a conclusion or an argument. And it turns out, even if they were like simple steps, but it just gave the model a chance, you know, maybe it's sort of like philosophically, who knows exactly why it

• 34:18Speaker 00

  works, but essentially feeding more tokens in and giving it more scratch space to think. And so people just think this is the, sort of the, the beginning of reasoning for these language models, this ability to kind of get to an answer by thinking to itself, by itself. And so it seemed quite successful in language, very successful. And that's why you have a lot of agents that just spent just tons of tokens, right? Just showing its work, right? In many ways, it stems from this chain of thought paradigm. And in biology, we saw very little of that. We saw, we started to see some of that in protein design a little, a little.

• 35:00Speaker 00

  And so we wanted to push that and showcase that, you know, well, at first explore it, is that possible in DNA? Like, what does that even mean in genomics? Because you don't really have words that describe, you know, its thinking. So how do, how do you take that pair, that same paradigm and introduce it to a DNA language model? And so what we did was a simpler version in so many ways. We had this data set of RNA aptomers. So just think of it as these desired sequences with some kind of fitness score associated with them. So we took this large data set that had RNA input and a fitness score. The fitness score go high, it's good, right? The simplest version. It's a big data set.

• 35:43Speaker 00

  And so what we wanted to showcase was that if we show the model progressively better RNAs in a, in a series of steps with its score, right? So you have like low scores first and then you gradually move up the chain. Can the model continue that trajectory on its own? And then, you know, in the final step, does it self-optimize to a point where it's like the best score it can get? That was the experiment. Can, can we do that? And so we took a data set, a large data set of aptomers. We held out a portion of like the best performing ones and we showed it only the, the lower ones, but then we ranked it, right? So we showcased lower scores with the RNA aptomers and then progressively got higher and then asked the model to just like continue with that pattern.

• 36:25Speaker 00

  And it turns out it was able to recapitulate some of those higher scores that we had not shown it yet. We were actually in the process of validating in the wet lab right now. So we didn't get to show it here, but we, we want to know, right? Actually, can it not just do this in Silco, which it can? It showcased that it was able to continue this trajectory and create, design plausible aptomers with higher fitness scores. And now we think this is a, you know, obviously if this works in the lab, we think this is a hugely, hugely valuable paradigm that can be, that can be pretty much applied to every other type of biological sequence. That's usually what you have. You have sequence, you have some kind of fitness score, desired output. And if we can get models to eventually learn that structure and, you know, basically just

• 37:09Speaker 00

  show a series of progressively stronger sequences, the model can then predict the rest. That's a very powerful paradigm.

• 37:17Speaker 01

  Honestly, a bit surprised about this, that specifically this task saw strong improvement. Maybe my personal bias is coming in here, but RNA is somewhat notorious for not having good co-evolution data in terms of like constraining structures, right? For, you know, viral genomes, oftentimes there is strong evolutionary pressure, but for mammalian, usually RNA does not have evolutionary pressure. And I think the community has seen that very, very clearly the predominant pressure is like RNA will code, you know, will, will carry information, coding information.

• 37:58Speaker 00

  Yeah.

• 37:58Speaker 01

  So I, I mean, I'm wondering like genomes carry lots of different information. They code for proteins, they have regulatory elements and, you know, different types of genomes have different types of structure. So I'm wondering, where do you think the, this capability might have emerged in this language model?

• 38:20Speaker 00

  Oof, that's a good question. And, and honestly, I'm not sure. Like we're surprised too. One, because the model is pre-trained on, on DNA and it's really just like mid-trained. On RNA, very, very, you know, in a small way. Yeah, I think what your intuition about the DNA having a lot of evolutionary effects or information is probably where. And so I think this is hinting at the idea of why we think it's so important to pre-train on DNA genomes and genomes first. And then sort of add additional modalities on top because you get a lot of transfer. And you want in the modality generalization is the thing that we're working toward.

• 39:02Speaker 00

  You know, what I didn't talk about as a company for the company as well is this idea of a building toward general biological intelligence where we are unifying a lot of the different so-called like languages or modalities of biology. At the end of the day, they stem from DNA. And I think people have not exploited that fact as much. It's usually really specialized domain specific or modality specific models and not leveraging a lot of inherent shared structure from other modalities. And so one example of that is like, you know, when people talk about virtual cells, for example, this is a little bit tangent, but you know, they tend to focus on just RNA and you know, transcripts and they want to generalize to describing an entire cell.

• 39:46Speaker 00

  But it's obviously a cell is a lot more things than that. In my mind, if you want to learn a system, you want to learn from all the signals or sensors of that world or that system. You know, if it's a cell and you want to fuse, you want to understand the DNA, you want to understand the metabolomics, the epigenomics, the proteomics. And that that's when you get closer to like, quote unquote, a virtual cell. And in our minds, we don't even want to stop at just the cell, but we wanted, we want to fuse all of these sensors across all of biology. Will it get us to a super intelligence that understands every component and minutia of biology? Who knows? But I am confident that this type of paradigm will get us a hell of a lot further than we are now. Like, that's like, that's my bars.

• 40:30Speaker 00

  Like, can you make something far more useful than than now? Right?

• 40:33Speaker 01

  So I'm curious, are you focusing on eukaryotic cells? Are you focusing on like human genomes? Do you have you gone so far as to do viral genomes? I mean, there's a lot of DNA viruses, but it seems like plausible that RNA, that there's a lot of RNA sequence, virus sequences out there. And I'm not sure fundamentally they would be much different in terms of training.

• 40:56Speaker 00

  Yeah.

• 40:57Speaker 01

  I'm, you know, I'm curious, like, what's the scope of that? If you can talk about it.

• 41:00Speaker 00

  Yeah, absolutely. We are interested in all domains of life. So here we focused on humans in particular because we thought this was an area that of previous models, Evo and Evo two, were not as strong and sort of got a lot of feedback from folks asking, you know, what are these models useful? Useful for? They can't understand human genomics because it's too complex of grammar and rules and DNA. It's too noisy. It's too, there's so many repeat characters and all that stuff. So we wanted to showcase, we think this is actually useful and it can be applied to humans. And it's sort of the most complex of the complex in some ways. But I think there's opportunity to apply these models generally to every form of of life. So we absolutely are interested in prokaryotes and viral.

• 41:46Speaker 00

  Viral in particular, we care about especially for bio defense and bio security especially. And I think there are also lots of therapeutic applications that we can learn from microbial life of, you know, maybe obviously for some folks. In particular, they, I mentioned that folks had used Evo to generate the first AI genome, a bacteriophage. Turns out you can use bacteriophage as potentially for AMR or antimicrobial resistance. You know, if you have a superbug bacteria infection, which in the world, there's about 2 million deaths from bacteria infections still. The idea of using viruses, designed viruses to target specific bacteria has been done

• 42:29Speaker 00

  for a long time, particularly in Eastern Europe. And there's a potential to make a new class of antimicrobials that are not like antibiotics, but very similar that can be used just like it. And so I think we're gravitating toward things that are high impact and the potential to save lives. And so we don't stop at just one type of genome. I think we're interested in anything that's beneficial to humans.

• 42:54Speaker 01

  Maybe going back to my question about solex and RNA predicting kind of a chain of thoughts of RNA evolution. I'm curious, was this model trained on RNA sequences or sequences which have some, might have evolutionary pressure on RNA structure?

• 43:12Speaker 00

  Only during mid-training.

• 43:13Speaker 01

  Only during mid-training.

• 43:14Speaker 00

  So pre-training is all just genomes and DNA. And so the only time we introduced RNA was for this specific task where, and only RNA from this data set.

• 43:25Speaker 01

  Okay.

• 43:26Speaker 00

  So not even outside.

• 43:27Speaker 01

  I see. It was, so this really was something along the, there is something encoding RNA structure in this model to some degree, maybe. Or either that or implicitly maybe.

• 43:40Speaker 00

  Yeah, implicitly, I would say implicitly. I mean, because

• 43:43Speaker 01

  That's cool.

• 43:44Speaker 00

  Yeah, because I'd say, you know, sequences, as we know from proteins, like implicitly should learn structure from just sequence. Yeah. And so we don't, we don't add in 2D or 3D information at this point, but we absolutely plan to. Yeah.

• 43:59Speaker 02

  So just so I understand, first of all, the chain of thought ideas, this is a demonstration of it, but the idea is anything that you can get sort of a training set that has a sequentially better measurement of some sort is maybe a candidate for this technique. And so can you just describe for this particular experiment, just so we can understand how we're mapping chain of thought to this data set and what the data sets have to look like? Can you just describe how is the, for this? I know this, this wasn't your data set, but how was the data collected in such a way

• 44:37Speaker 02

  that you could map accurately from sort of fitness or whatever to a particular phase or part of the data set?

• 44:48Speaker 00

  Yeah, so I'm less familiar with how the data was actually synthesized or generated from the experimental point viewpoint, but they are validated from a wet lab in the real world when it was collected. So it has some kind of fitness score, I believe through

• 45:03Speaker 01

  I can maybe provide a bit of context if you want. I mean, so the idea here is you just generate a bunch of random sequences and then you take those sequences and so you have like an aptamer structure, which is essentially like a switch with RNA, which sort of when something binds to it, it will do something like cleave off a sequence. And you can use an NGS, like big generation sequencing readout. I'm very high through, but so you create lots of these different sequences. You, I guess in this case, they were targeting a specific HIV

• 45:36Speaker 00

  Yeah.

• 45:36Speaker 01

  like protein or genome or something. And if it binds, you can, you basically get the signal of like, you get more reads of that. And so the more time, the more sequences which are floating around, kind of like the more fitness, the more likely it is to bind. And then you take those and then you mutate them again and you kind of iterate on this.

• 45:57Speaker 02

  Right. So you have this iterative experiment where you're progressively using the Petri dish to basically identify the most fit

• 46:05Speaker 01

  Yeah.

• 46:06Speaker 02

  thing. So and then what you're doing here is you're basically doing this same experiment in silica.

• 46:11Speaker 01

  And they're actually doing very high throughput, like I think there's like 10 to the 11 or something sequence, some like really high number of kind of sequences explored in parallel for this experiment.

• 46:21Speaker 02

  But the key is, is the biology of the experiment is actually doing the filtering, right?

• 46:26Speaker 01

  Yes, yes.

• 46:27Speaker 02

  Yeah. And so, and so you can imagine other types of experiments where you could apply the same kind of idea where you're, you're doing this like progressive refinement of something which is very common in biological lab work. And, and so if you're capturing those intermediate states and you can maybe feed them into the model and do that kind of thing.

• 46:46Speaker 00

  Absolutely. Yeah. So another example is for the antimicrobial resistance. I think that's something we're, we're very interested in as well. The ability to selectively target specific bacteria strains or kill bacteria strains. Yeah, that's measured basically a score zero to one of how effectively that is done. And so showing progressively more effective phage genomes and their associated scores for effectiveness fits that paradigm very well. There's another design task that we're working with a national lab to do this with as well in an area that is quite different for us, but it's on designing proteins to extract rare earth minerals. So

• 47:30Speaker 00

  and so it turns out that you don't just care about proteins that could bind to something, but you want it to be selective. You want it to be bind to one type of rare earth mineral. And so you have scores associated with how much affinity or binding affinity for each type of mineral. We want to essentially do the similar thought, this similar exercise with rare earth minerals and show it a series of progressively, you know, desirable scores, not just for binding to this, but like, you know, lower binding scores for other ones. You can selectively do it. So I think there's the creativity in which you can showcase sequence and desirable, desirable sequence with, you know, some kind of fitness or functional output is a relatively intuitive

• 48:13Speaker 00

  way for people to design, you know, just prompt by just by prompt engineering essentially, which I think is, is very exciting to. To to explore more.

• 48:22Speaker 01

  Following up on the rare earth mineral extraction, I find that is an interesting use case for this. In fact, maybe one where you probably wouldn't have a comparative advantage compared to some other techniques.

• 48:33Speaker 00

  Sure.

• 48:34Speaker 01

  Because it seems like your strengths are probably in larger scale design across like entire organisms, but focusing on individual proteins, that may be one much more of a like a structural task. And I'm curious, like going to talking about ClinVar, the argument here is that the model, and that was a really good statistical representation of what type of mutations are common or not common.

• 48:55Speaker 00

  Yeah.

• 48:55Speaker 01

  And I think in order to do design as a, if you want to design structures, I think you want to understand structure. If you want to understand disease, I think that is, I think, more natural, or for many diseases, is much more natural in terms of like a population genomic sort of way.

• 49:14Speaker 00

  Totally.

• 49:14Speaker 01

  So I'm curious, like, where do you think your strongest competitive advantage is? You know, in using this strategy? And do you think that your model understands structure in addition to function?

• 49:27Speaker 00

  Yeah. I'd say what drew us to this particular application rare earths and in our strengths in general, why we thought it might be suited for it is two things. One is, I think, in areas where context matters. So, yeah, you're right. So a protein design task where you're just designing structure, not maybe not naturally, where we see ourselves competitively advantaged. But in the rare earths case, our hypothesis is that context matters. And what matters here possibly is certain microorganisms with proteins that have the function that we desire.

• 50:07Speaker 00

  We can potentially prompt and provide us context for, this is the neighborhood in which, you know, tell Omni, this is the neighborhood of genomes or microorganisms that you should search for new proteins. So it's really a sort of like a mining exercise. And what we can showcase to our models that other folks can't, like protein structure models, is that we can feed in non-coding regions before that gene of interest or that protein of interest, and then ask for the model to provide variants, essentially. So like mutate this protein, but you know, know that you're in this microorganism, but show me different variants that you've seen in nature, or combine different things that you've seen in nature, given this context.

• 50:53Speaker 00

  And I think that is one reason why we can make very evolutionary diverse sequences and potentially phases. Folks have used Evo models to do this with toxin, anti-toxins, and a very similar technique. They've prompted on things upstream from the proteins of interest, and then ask the model to kind of generate a bunch of plausible other ones. And I think this case for rare earths, that's super exciting because then we can come up with plausible variants and then test them relatively simply for what they bind to and selectively bind to, which I think is really interesting for this partner who cares, for example, about securing the supply chain of rare earths for the US strategically. And so we thought that's absolutely worth something that is worth supporting.

• 51:37Speaker 01

  So your point is not just you're designing a protein, but you're designing an organism which generates a protein, and this protein has an action. But or is it's that that the way to design this is you need to understand how through the tree of life, interactions of proteins with rare earths have, or with certain minerals have occurred?

• 51:59Speaker 00

  Yeah, I'd say in this particular case for like the rare earths, we got really interested in the organism, like designing the whole organism. But I think the organism does tell us about what proteins are plausible, and their selectivity is not, or the way they evolved, not just around the protein itself, but also the regulatory elements around it. And it helps us narrow and provide additional context. I guess I think of the genome or DNA broadly as sort of the imprint of the physical world into DNA. And so there's parts of the DNA that we want, but the things around those parts that we want also tell us a little bit about the context of where it came from and how it came to be and its function. And so I just think of it, you know, like going back to that word, but context.

• 52:44Speaker 00

  I think context matters in many of these applications, or at least in some of them.

• 52:51Speaker 01

  Yeah, I think context matters a lot in biology. I think maybe one of our big bottlenecks is the lack of context and how we as humans mostly approach biology in terms of a very engineering, like let's isolate individual systems. And a systems biology approach is very hard to get any sort of meaningful quantitative predictive power. So maybe going off on a tangent, but I mean, I'm curious, you know, going to context and thinking about how context scales to an organism. You're talking about right now, two million length context, right?

• 53:25Speaker 00

  Yeah.

• 53:25Speaker 01

  You know, I think the human genome is roughly, you know, a thousand times longer. So, or but even like a lot of, you know, say bacterial genomes, if you're trying to engineer them, are quite a bit longer than that. So how do you leverage something which has a long, but still finite context compared, you know, to do synthetic biology across like large organisms? Maybe actually maybe for context, how did the Evo2 bacteriophage design work? Which was probably much more than two million for that as well.

• 54:00Speaker 00

  Yeah, so I could expect the Evo bacteriophage just a little bit because it's actually a separate group that worked on that. But that context was actually pretty short, actually. The reason why they started with phases is because it's amongst the shortest genomes. And so I believe it was something around 6,000 base pairs.

• 54:17Speaker 01

  Oh, that's really short.

• 54:19Speaker 00

  Yeah.

• 54:19Speaker 01

  Yeah.

• 54:20Speaker 00

  So extremely short.

• 54:21Speaker 01

  Viruses are insane. They're incredibly efficient.

• 54:23Speaker 00

  They packed a lot in there.

• 54:24Speaker 01

  Yeah.

• 54:25Speaker 00

  Yeah. But yeah, no, I think your question about how, how do you get longer context with something smaller? Is a very key question that we, I think broadly, the AI community is constantly trying to fix and be creative about. And so, I mean, I think that's a large part why we as a company are in AI research lab first, because we, we think the innovation needs to be constantly pushed. It's not a space where we can just grab open source models and, and expect that, you know, many of the tasks that we care about are just going to be solved. We want to continually push the envelope. And so context is one of the key research areas that we drive in.

• 55:05Speaker 00

  I'd say that's probably how we got our name because we worked on long context before it was a, it was a thing, I guess, like 2023, 2022.

• 55:14Speaker 01

  You and your team, I mean, your collaborators have a long history of these state, state, space models.

• 55:22Speaker 00

  Yeah.

• 55:22Speaker 01

  Doing, you know, pushing context, like what seemed insane at the time.

• 55:26Speaker 00

  Yeah.

• 55:27Speaker 01

  I mean, now I think routine of books, the big slabs.

• 55:29Speaker 00

  Yeah, exactly.

• 55:30Speaker 01

  But at the time was like orders of magnitude longer. I don't know if you want to talk about that a bit. And also, I think one, maybe one thing which I think is really fascinating is how, in for both this model and other ones, you really go on a first principle way, like diving into the architectures of how GPUs work and designing models which are, you know, exploiting the architecture of GPU in addition to, I mean, it's not just, oh, we're building longer. It's like, what can we do special, given the compute constraints we have to push the boundary?

• 56:03Speaker 00

  Yeah, absolutely. If you want to distract our researchers at Radical Numerics, this is, this is how you nerd snipe them. You talk about, you bring up long context and kernels, GPUs, then they're like, wait, did someone say kernels? And then they start trying to figure out, you know, how to make things fast. Yeah, long context was a special place in my heart because that's what I focused on in my PhD at Stanford. And that's how we started thinking about DNA. And, you know, going back a little bit for fun, we were looking, working on language models in general. And then we noticed our models were good at long context. And so that's the progression of like how we started working in the space was like, oh, these models seem to be really efficient on long context.

• 56:46Speaker 00

  And then with Michael Polly, my lab mate, he worked on the first design of a hyena, this convolutional architecture. And then we started thinking like, let's push this further. Let's see what new, new applications open up if we really lean into long context. And we asked, what's the longest sequence out there? And, and eventually, unsurprisingly, we landed on DNA. We were like, DNA has got to be the longest, three billion base pairs. And, you know, and we started thinking like, okay, what's being done there? Like, what kind of context lengths are people doing there? They're doing super short. They're doing like one or 2,000 base pairs or tokens at a time. It's like, you know, way smaller than what one would want for DNA.

• 57:29Speaker 01

  I mean, most human transcripts are like 3K or so. So that's not even like, you know, most, that's not even what you need to represent like a protein or most of the time.

• 57:39Speaker 00

  And so it's clearly a need there and overlooked. And we saw it as a way to initially, like, let's see if we can do something that we had no idea if it was going to work and no idea who would want it. And so we just started tinkering around. And it turns out, out of the box, relatively out of the box, it was, it was doing pretty well at reading DNA. And then it led to, okay, let's, people keep asking about like DNA. They didn't really care about our language stuff as much. And they would say like, can you, can you do longer? Can, you know, what can you do with it? They always ask, what can you do with it? We didn't know, really.

• 58:11Speaker 00

  And then for some reason, it latched on this idea of like, well, no one's writing DNA. Can we, can we get it to write DNA? And that was this really simple question, but. In hindsight, almost seems obvious that yeah, you would want to write DNA and design it. But when I was first pitching the idea of Evo to folks, I spent six months, which I guess in bio words, not that long, but I spent six months going around saying like, hey, if I generate DNA, like, would you find that useful? Like, what would you do with it? Would you back us? Like, would you want to be a part of this? And crazy enough, most, almost every scientist at Stanford I talked to, thought it was a stupid idea. It was, I was like, this would be so cool, like generating DNA. How much would we accelerate the field?

• 58:56Speaker 00

  And then people would say like, what would you do with it? I'm like, I don't know. And then I would get that comment or comments like, that's not possible. Like, we as humans don't understand the rules. How could you expect an AI to learn it? You can't even tell if it's right or wrong. Like, you can't tell the AI, yes, that's right or wrong. How can you expect it to learn it? Or there's too many repeat characters, or DNA is too noisy of a distribution. Like, there's no real rules and there's just a bunch of junk in there. I heard all the reasons and I was just so stubborn about it. There's gotta be a use case from, from being able to generate DNA. I just, it just feels right and I didn't know what it was. So it was, it really was an experiment of like, what happens?

• 59:35Speaker 00

  And so when we first trained Evo, I remember we had no idea if it was gonna work. We, we, we had no idea, we didn't know what thing was gonna emerge. We just thought, let's just train a big one, which is kind of ridiculous. But somehow, you know, folks at Arc, they were like, sure.

• 59:52Speaker 02

  Yeah, why not?

• 59:53Speaker 00

  Why not? Let's see what happens. Let's pay, let's pay some money for the GPUs and let these crazy kids train the model. And I remember when the first result came back and, and it, it kind of like gave us chills of like, oh, maybe there's something going on, which was, somebody took the model checkpoint, the first one, and threw it at Protein Gym, one of the protein benchmarks, and turned out to be competitive with protein-specific models. And we were like, okay, that is pretty surprising because we never told the model what's proteins versus not. And there's actually, it was, you know, there's also protein and DNA, right? So there was one piece.

• 1:00:35Speaker 00

  I think it was just surprising that it was actually competitive with protein-specific models. And this was like the first experience. And then they just kind of kept on coming one after another, like, oh, this is competitive here. Oh, it's state of the art on RNA and RNA. And we started seeing like, oh, it can learn across different modalities, like not just DNA. And then, then we felt like, okay, this, there's something there. And so it kind of went from there and then, you know, folks at Nvidia were like, let's, let's back Evo two, let's make this even bigger. And then Greg Brockman from OpenAI was like, I'll take a break from OpenAI and take us four months of a battle call and like help these crazy kids out. And, you know, then we're Slack messaging Greg Brockman at 3am, trying to debug our code, which is wild. Yeah, so it just, it went, the trajectory was very surprising in many ways.

• 1:01:20Speaker 00

  But at the same time, we still got a lot of feedback like, hmm, you know, what are these models good for? What are they, you know, what are they useful for in the, in the real world? And so that's what really motivated us to start a company. We thought, what we showed was just, really just a taste from like an academic flavor. In a similar way, the language models, when they first too, you know, if natural language came out, people asked similar questions, like, what are these things good for? Oh, cool, it can write some jokes for me. Is this gonna lead to like, an all, you know, all encompassing AGI that can automate everything? They did not think that, right? They thought it's like a toy, it's got emerging capabilities, and they would extrapolate the potential. And in many ways, I think what we're seeing here is even more exciting or reminiscent of that trajectory for, for DNA.

• 1:02:04Speaker 02

  But Evo two even was showing the bacteriophage and that, that wasn't a persuasive. I mean, that's almost a scary example, right? As you mentioned, so the people didn't see that, like, that isn't a lightbulb moment for people.

• 1:02:20Speaker 00

  Yeah, so I mean, and some people, right? I think either fair or, or, you know, rough critiques, you can even, you know, play devil's advocate and say, what it's generating is kind of, you know, pretty close to nature and you're kind of recapitulating, just kind of a small variance. And so there's, I think there's a lot of ways to critique and, and kind of, you know, minimize the, the potential, which can be fair argument. Like, so I think at this point, we think what is more important is not just pushing on the science and like, cool, this can be done and like, kind of leave it there as a sort of thought experiment. But like, how do we actually use this to improve human understanding of disease?

• 1:03:04Speaker 00

  Improve treatments, make better rare earth mineral extractors? How do we actually make this useful is what we care about now as a company, in addition to some of these more scientific questions.

• 1:03:16Speaker 02

  So maybe that's a good segue. There's, that brings up two things for me. One is the Meccan Tarp stuff that's in the end of the blog post and then also the biosafety stuff. Those are both, I think, important applications. So can we, let's do Meccan Tarp first. Can you talk a little bit about, and this is actually kind of building on the work that Evo did as well? I think, and I remember in the Evo paper, there was some Meccan Tarp work in which they were discussing using the, reversing the question and using the model to extract insights about biology.

• 1:03:55Speaker 02

  Right? And so, so, and this has actually become a common theme with, I think, maybe biology more than any other domain, is that people, because it's a scientific question and it's learning patterns about the world, you can actually say, okay, well, what patterns did you learn?

• 1:04:10Speaker 00

  Yeah, so I think Meccan Tarp is an emerging field in, in bio that we're extremely excited about. And admittedly, we are on the early side, I'd say. So we're building up that team, but so far what we've showcased and been excited about is looking, really analyzing the embeddings and some of the activations in the model. And so this idea of Meccan Tarp for, for bio is borrowing a lot from the natural language community currently. You could think of what the models are doing as compressing a bunch of information it's seen, right? And so in this compression, it's really basically distilling it down into the key components, key patterns that helps it understand the data or learns the distribution of the data.

• 1:04:51Speaker 00

  And so what we're trying to do is probe the models to see what did the model distill into its weights and its activations, or sort of like the outputs. And so for us, we started off with a lot of the outputs of the model, so the activations. And we, we wanted to see what kind of structure, what kind of visualizations can we see that help us understand some of the complexities of, of DNA, which is a ton, right? And so some of these complexities can, can range around GC content, certain motifs of transcription factors. There's a bunch of regulatory types of patterns and motifs that the model, we believe, has to pick up to be able to do its tasks, right? To understand whether a disease is caused by a variant.

• 1:05:35Speaker 00

  And generally, it's going to compress all these different motifs and distill them into the model weights. And so our job is to then find these and see if we can distill a pattern or structure so that can be generalized to other cases where we don't understand the patterns, right? So that's sort of largely the goal.

• 1:05:51Speaker 02

  So in this case, GC means the two nucleotides are, the fraction of those in a sequence?

• 1:05:58Speaker 00

  Exactly, yeah. So the fraction of the G's, G and C letters in a genome, which are, I guess, amongst the more simpler things. But also, you know, simple things as repeats, number repeats. I think transcription factor or TF motifs is another, one that is especially interesting for folks. Cause I, eventually, we're going to get to a case where we can design transcription factors, meaning we'll have certain transcription factor patterns via like ChIP-seq modalities.

• 1:06:23Speaker 02

  Transcription factors are, are, well, you go ahead.

• 1:06:27Speaker 00

  Transcription factors are molecules that combine to DNA and they would alter essentially the gene expression pattern. And so it has this regulatory effect that doesn't modify the DNA itself, but can modify sort of the effects of DNA in the products that DNA makes. And it can have a lot of implications on, well, pretty much everything in your body. So it can modify disease states, it can modify, I guess, it's a lot of aging-related researches around transcription factor design. And so we think being able to understand some of the motifs via DNA, but also additional modalities will eventually let us be able to design transcription factor patterns as well. And so I think this is very exciting.

• 1:07:11Speaker 00

  And in many ways, the combinatorial space of learning these transcription factors, like what binds and where they bind and what effects it causes, is just far too vast to be able to do this in a manual way. And so we want to take a data-driven approach to, to learn some of these motifs.

• 1:07:25Speaker 02

  The complexity here is partly because the transcription factors themselves are coding genes. And so that you can, right, those can regulate each other. And so that you have this, that's where that combinatorial effect.

• 1:07:38Speaker 00

  Yeah.

• 1:07:39Speaker 02

  So like, just narrating for the listeners only, we're kind of marching through these increasingly complex and higher level factors, all the way from GC content we started. Now we're looking at disease, which is maybe the most complex thing you could, or you were pointing towards in this analysis.

• 1:07:59Speaker 00

  Yeah, so I think what we, our first idea for previewing this was, and we're working toward this broadly, is this idea of mapping the manifold of disease, right? So manifold, there's like the sort of representation space of what disease looks like to a model in terms of the output scores or embeddings. We believe that there's a lot more structure that can be gleaned from understanding some of these outputs. And so, you know, mapping this manifold or the landscape of what the structure of disease, and obviously there's many diseases, and so I think would be a big, exciting area of research for us to, to actually drive motivation for Meccan Tarp in bio.

• 1:08:44Speaker 00

  I think, I think we're just really scratching the surface because if you can get this much of, glean this much of insight potentially from just DNA, which is in my mind, just one of the sensors that you want to ultimately fuse into, you know, modeling biology.

• 1:09:01Speaker 00

  Then being able to do something similar across all modalities is something like, by all modalities, I mean protein, RNA, epigenomics, you know, the attack, chromatin accessibility and methylation patterns. All these other different types of sort of molecular phenotypes around DNA just presents such a huge opportunity that is kind of laid in front of us. That is all green space, green, green field. No, I haven't seen anybody do this level of sophisticated techniques from, you know, machine learning, deep learning into what I think is going to be the most important impactful area of understanding and applications for AI. That's what we're excited about.

• 1:09:43Speaker 00

  And I think we're just showing a preview, a very simple preview from the DNA only models. But in our next generational models, which will be increasingly multimodal, we're talking dozens. It's a very exciting moment for us to showcase.

• 1:09:58Speaker 02

  Sidebar on that, is language, natural language, one of the moats?

• 1:10:02Speaker 00

  Not yet.

• 1:10:03Speaker 02

  Not yet.

• 1:10:03Speaker 00

  Yeah, but it will be. So I'd say there's a lot of questions about how to fuse that with bio. But I actually think it's pretty, will be relatively straightforward. I think the more tricky, the tricky part for us is actually how to fuse the biological signals more so. Fusing language, we see, there's a lot of examples with that with like, with image and video space. So we, we feel pretty good about that. And we've done some early experiments with language. And I think that will make it extra accessible for folks when you can connect it to language. But I think the part, that recipe that folks still are trying to figure out is how to do this across biological modalities.

• 1:10:43Speaker 02

  It just seems like a natural way to be able to do chain of thought, right?

• 1:10:47Speaker 00

  Exactly, yeah, absolutely. And I think, you know, especially when you start having chain of thoughts from orchestrating tools, things like Claude Science, I think the idea of incorporating language is already in a lot of people's minds.

• 1:11:02Speaker 02

  If you don't have any questions, let's talk about the bio.

• 1:11:07Speaker 01

  Biosecurity.

• 1:11:09Speaker 00

  For us, we as a company, thought it was very important to have a dual mandate, what we call a dual mandate. It's this idea of essentially being cognizant and feeling responsible or wanting to feel responsible for the capabilities that we're enabling on the design side. So if we're going to create models that can design function into sequences, we believe and see a gap in companies being able to safeguard that technology and make sure that it's used responsibly increasingly more. I mean, folks have, I was just at a panel last night, a panel on AI scientists, agents that can do scientific discovery. And one of the last questions was, what's, what are some of the biggest risks

• 1:11:53Speaker 00

  or doomsday scenarios with AI can learn, learning about science? And every one of them talked about biological weapons. And at the same time, I'm curious, I was curious because I was like, okay, so then what are any of these folks doing about that? And basically I didn't hear anything about that. And I, you know, these companies, I won't say their names, but, and I look at the companies, they don't have big efforts in those spaces. So anyways, we felt it was important as a lab, in a lab, that a team that was both building the design capabilities is actually best, also best suited for building the defense capabilities. Cause they're basically the same models. A model that is good at generating, turns out is also very good at discriminating or predicting with, you know, if a sequence is pathogenic or not.

• 1:12:37Speaker 00

  So we felt it was not just, you know, from a principle standpoint, necessary to work on both biosecurity and the design, but that it was, you know, strategically, it just made sense as well. And so we felt this resonated with our team, but also the broader community, folks in, in the US government and abroad even, that this, there was a clear gap and need for a type of entity to exist and to do this. And so, yeah, we felt it was necessary to build it into our mission. And so for us, what we have done and plan to do, we look at it, or I should say bio defense, biosecurity sort of has three or four different pillars of, you know, strategy for bio defense. The first one is around detection and surveillance, right?

• 1:13:22Speaker 00

  You know, broadly, can you detect from the environment? If a sequence has a pathogen in it, something that can cause disease, let's say from, you know, swabs at a airport, a nasal swab, or this sewage system, right, collecting samples. The next is attribution, which is, once you've detected danger, can you figure out where it came from? Is it natural? Is it from, you know, a rental country abroad? Is it engineered by a human from a specific lab in a country? And that helps you figure out, you know, what to do about it, right? So this next idea is around countermeasures. So once you've detected, figure out where it's from, what do you do about it? Can you make a counteragent? Can you make it antiviral or antimicrobial?

• 1:14:04Speaker 00

  And then fourth one's generally around deterrence, but that's more of like a government kind of level thing. But yeah, so we focused primarily on the first three. So we create tools that can, given a sequence, detect if it's pathogenic. But also what we felt was missing from the community was not just detect, you know, broadly if it's pathogenic, but characterize the heck out of it. Meaning what parts of the sequence are dangerous, what genes, for example, attributing where it came from, being able to not just look at its, you know, sequence and match it to a database, but be able to attribute its signatures. And then also a big component to make this effective in the first place.

• 1:14:44Speaker 00

  The community, the bio defense community broadly focuses on sequence matching. So they'll take a sequence and they'll basically align it to a known database and see, say, have I seen this before? Does it match, you know, this list of known pathogens? But I think what's emerging and a concern for a lot of labs is, well, one, new stuff, right? If it's not on your list. And two, things that were intentionally obfuscated to not be detected in sequence space, meaning the letters matching up exactly, but also function space.

• 1:15:19Speaker 02

  Yeah.

• 1:15:19Speaker 00

  Right? Because basically models that we're enabling now, they will be able to be function aware or structure aware. And so that means concretely, you can have a sequence that has the same function, like a pathogen, but actually look different in terms of the letters.

• 1:15:36Speaker 02

  And you can imagine based, actually what we have on the screen here still is the smegintur thing. And there's this sort of manifold that, that, that the model constructs internally that, that is kind of coding for these, among other things, functions. And so that you can imagine how it would be able to say, oh, well, that's, you know, maybe genetically quite different, or at least somewhat different, but it still has a similar function.

• 1:16:01Speaker 00

  Exactly, exactly. So what we talked about in our defense blog is this idea of things that can function similarly. They can start having separation in terms of what the sequence looks like, while maintaining the same functionality, right? So this is, can happen in nature, sort of naturally. But also what these AI models allow you to do is also intentionally do that as well. So have the same function, functional capability, but have diverse letters, basically, diverse spelling, but describe the same thing, basically. And so in this case, this work from Microsoft called paraphrases. So it's like, you know, kind of rewording things where they showcased that you can,

• 1:16:45Speaker 00

  for example, use protein language models to essentially keep the same structure, which structure implies similar function, but then change the spelling, right? And not just that, they wanted to test that if you have this capability and you send this through existing detection systems, would it break the system? Like, would it actually detect it or not? And I think one of the interesting facts that maybe the general public, but most biologists know, is that there's these DNA synthesis companies, right? Where you can basically send a design of sequences and get back a DNA, you know, DNA molecule like it's an Amazon package. Like you just send it off and they'll send you the physical DNA of the sign.

• 1:17:30Speaker 00

  They'll manufacture it for you. And this is, this runs the scientific community, right? It's the pipeline that allows people to do research and understand biology and make drugs and everything. So it's prevalent and it's public. And so I think one of the concerns for folks is, well, and one of the concerns for us when we first started working on this was, in a world, for example, where agents are prolific online, presumably billions and trillions of patients building and taking all sorts of actions on the internet. It's wild that they don't have any tools to basically tell it if it's making anything dangerous or not. And so that was, that was literally our first motivation of like, we should probably make something that can detect if something is dangerous or not, right?

• 1:18:14Speaker 02

  Do you guys know a filter in the, for these manufacturers that they could say, what am I making here? Is this dangerous or whatever?

• 1:18:21Speaker 00

  Yeah.

• 1:18:21Speaker 02

  And maybe you need, you could have an exception if you're like some licensed lab or something and I'm doing something dangerous, I know I'm doing it, please let me do it anyway.

• 1:18:29Speaker 00

  All sorts of cases. So that's, that's one scenario. To be fair, the, many of the DNA synthesis companies have detection tools, but I would, I would strongly hypothesize that they're not AI based and fairly, they're probably not robust. Yeah, they're all pattern matching mostly.

• 1:18:46Speaker 01

  Sorry, maybe this, I should ask this question later. I'm just curious about, you know, we've all, everyone who is working in bio has tried to use Fable and instantly got picked out on literally everything you type in.

• 1:18:59Speaker 02

  My website, for example.

• 1:19:00Speaker 01

  Yeah. You, you know, you can't do anything in Fable without. But in terms of people like scientists exploring synthetic biology and creating new sequences and designing new sequences, it seems like it'd be very hard to get, you know, to have a, an ROC curve, which you can, you can live on that doesn't like impede novel scientific research, which legitimate purposes. How do you avoid, you know, even with an F1 of point, you know, nine, nine, which you're not even close to right now, I think. You know, that, that still could easily, if there are billions of sequences, you know, sequence generated a day or at least like a year.

• 1:19:43Speaker 01

  I mean, I think you could really have a lot of, it seems like a very hard balance to.

• 1:19:48Speaker 00

  Yeah, it's a tough question, right? So I think broadly, the way we look at it is, you know, if we thought about like, how do we 100% stop the dangerous design? I think it's, it's a harder question to ask. I think the question we asked is. On the capabilities design side, there's plenty of folks pushing the frontier of that. When we look at the defense side, do we see frontier technology being applied there? And the answer to that was no, right? So we saw this huge gap and we wanted to sort of aid, come to its defense, I said, come to its aid to give it a boost. Right? So I think that perspective, it's an easy choice for us to say,

• 1:20:32Speaker 00

  let's push, let's bring that to a better, you know, head to head match against the design side. Is it going to solve everything? Well, I think that's what we're going to aspire to be. But realistically, there's always going to be cases where it can get around, right? And I think that's one, that's a big motivation for why we think, you know, safety for language models and chatbots is one, one layer, right? But also, you're right, there may be things that kind of just get out there, get past that anyways. And so what do you do about those cases? It's already passed, you know, the chatbots, right? It's already, it's already aided someone into making dangerous sequences. So it's out there. I think the cool thing about our tools is that we, what we're building is tools for the folks that care about things that,

• 1:21:17Speaker 00

  that's already out there, out there in the environment that's made it somewhere. And now there's this whole ecosystem that we want to build into that does the surveillance, that does the attribution, that does the countermeasures. We want to boost that community, right? And build stronger tools for that space. Does it have to be perfect to be useful there? I don't think so. I think we can, we can be helpful and move the biodefense community forward in terms of, you know, bringing AI technology and AI frontier technology to their aid without being perfect. And so that's kind of.

• 1:21:47Speaker 01

  Maybe another way to say, so threat model, it's not even clear what threat model you're actually trying to defend against at this point. But the capabilities don't currently exist at all. So, you know, you provide something and now this can be worked on with, in terms of a larger regulatory framework or government or, you know, nonprofit, whatever, larger framework. And it now provides you a tool that the community can build upon.

• 1:22:15Speaker 00

  Yeah.

• 1:22:16Speaker 01

  Even if it's not perfect, it's, it provides a starting point. And if you don't have a tool, then you can't do anything.

• 1:22:22Speaker 00

  Yeah, the bar is the improvement, right? So the bar is like, where are you at now? Can we move the needle? Can we move beyond sequence based matching, alignment matching? Absolutely. I think there's tons. And I think the interest is only growing. I think we're hearing a lot of chatter at the, at the regulatory side, you know, from different politicians and different folks in think tanks. It does seem like folks are mobilizing. So we're optimistic that this gets out and is more top of mind for folks to actually take action as opposed to just talking about it. Because like right now, it does feel like there's a lot of talk. And then one of the reasons why we felt like, let's build tools and like, put it out and give access to people. Because there's been a lot of talk about AI companies saying like, we should do biodefense. But then like, okay, what does that mean? Like, what are you going to do about it?

• 1:23:06Speaker 00

  And so that was our, our approach.

• 1:23:08Speaker 02

  So I can just steel man us for a minute though. Can you go to the other diagram where you have the? Yeah. And if you click on them, then they show other similar, similar compounds or similar genomes. So arguably, just steel manning the, the opposing viewpoint. If you are on the frontier, which it seems like you are, and you certainly strive to be, you're moving the frontier of the attack and the defense at the same time. So like, right now, that what you're doing clearly helps. But that, that maybe if you're on the frontier, then that doesn't really matter for, in the long term. So how do you think about that? I mean, I guess you just have to nerf what people are doing or something.

• 1:23:52Speaker 00

  Yeah. The mindset of what was hoping to get folks to start thinking about it as less of a, oh, let's make a biodefense tool and like call it day. It's basically an arms race, right? It's similar to the cybersecurity community. You're going to make better technology, especially with something like Fable, that can potentially attack. And that means you're going to have this back and forth. The design side is going to get more capable. The defensive side needs to try its best to get ahead, right? And then that, then that just motivates other folks to do past that too, right? On the design side. So I think inherently there is this arms race style dynamic that the way it appears to us is that the defensive side has been far, far lagging.

• 1:24:37Speaker 00

  And so what we want to do is bring the defensive side closer to, to par essentially. So that, that's the mindset I picture or the framing I think about it.

• 1:24:48Speaker 01

  The cybersecurity analogy is interesting, but I think it differs in some like key points. So first of all, I think that with cybersecurity, with a sufficiently strong model, you might actually be able to close all loopholes which are not sociological. There are certain ones which will always be hard, always be ways of getting around things. But you could in principle catch every single exploit. And I think that might be possible in the future. And then you can patch them. And as long as people say updated, you know, you're secure. We have fixed genomes, right? So you can't patch a human. I mean, so once something, so I think that in some sense, the, the, you know, attack surfaces or the, the way that you're defending it is that is much higher or much harder.

• 1:25:32Speaker 01

  But then maybe the, the converse is that the, it seems much less likely that someone would have the incentive to go on offense to the same degree. And also the, the like barrier to entry to success, even if you can print out arbitrary DNA and the, the process of going from that to making a successful, you know, virus, especially one which doesn't kill the person designing it, is actually quite large. So I guess I'm maybe, I'm curious, like, what is the single, from your opinion, what is like the single biggest threat that we actually have? What would the thing which keeps you asleep or keeps you awake at night? Is there something in particular?

• 1:26:13Speaker 01

  Or is this like, you just think this is something we need to build and let's build it?

• 1:26:18Speaker 00

  One, it's hard to get in the mindset of a person wanting to design a bioweapon. So we're, we're not trying to necessarily think of all the potential people or scenarios that, you know, the bad actors might work on. What I think broadly, what I worry about is we're lowering the bar for how much expertise is needed and the speed at which folks can engineer this kind of things. And that means the volume is going to just exponentially increase at some point. And so there's a mix of, yes, there's intentional worries because there's certainly state actors that have had biological war programs, very, very, very large ones. And one of our advisors on a company has, you know,

• 1:27:03Speaker 00

  physically seen these facilities and decommissioned them. And so we've heard a lot of stories about states actually being motivated to create such weapons. That is one concern. And I think during peacetime, it's less scary. During wartime, it's particularly scary. I think the unintentional ones are also things that, in my mind, potentially more likely in the near term, where folks do try to generate things and control function and inadvertently things that maybe, maybe they tried to make a certain thing to understand and to study, but it got out, right? Because these things can be hard to contaminate, to, to, to contain, for example, and they just have a leakage.

• 1:27:45Speaker 00

  So I think those scenarios potentially seem the most likely. Does it keep me up at night? Not necessarily. I'm a much more of an overall, an optimist. I think building this type of technology is ultimately a game that you weigh out the pros and cons and the costs and benefits. I think the benefits far outweigh the potential harm. And so that's why we, we work on it. Because ultimately, we do think it's going to be an engine for, for discovery and human health improvement. But at the same time, we just felt that the defensive side was sort of losing its, this, this arms race. And so we work on it as well and try to push the frontier. But I'd say overall, I think as a community, of researchers,

• 1:28:29Speaker 00

  but also folks in the, you know, the policy side, I think the, the community is resilient enough and has the ability to mobilize, to get ahead of it. And so I'm very optimistic about it.

• 1:28:40Speaker 02

  We have two questions that we, we like to ask every guest. So the first one is if you could by fiat, remove a bottleneck that is important to you, what would that be?

• 1:28:56Speaker 00

  Interesting. Could I get two answers on this one?

• 1:28:59Speaker 02

  Sure.

• 1:29:01Speaker 00

  The first one is kind of a cop out because every AI lab says this, like.

• 1:29:05Speaker 02

  GPUs.

• 1:29:07Speaker 01

  We've done that answer a few times.

• 1:29:08Speaker 00

  We can use GPUs. The other one, which I think is a little more philosophical, I think in this space, in what we're trying to do, aspire to do, is reinvent how scientists do their work in the space. And I think one hurdle we run into is folks who, and you see this in many domains, folks who are, the more expertise you have in something, the more pessimistic you become about that space. And I think you especially see this in bio where you know a disease area or modality so well. And then someone introduces something else new and you're like, oh, but what about this? And this and that. And they're very pessimistic and probably rightfully so.

• 1:29:50Speaker 00

  But also, I think what I've noticed at the company, what we strive to do and the folks that we try to bring in are domain experts that do know that field, but also are still dreamers, meaning they still do have that imagination and desire to change how things are done. I think that barrier, we see that a lot in the field. I think if we embrace that more, we can see a lot more progress and step change that I would love to, to see.

• 1:30:17Speaker 01

  Brings us to the last question, which is, yeah. Is there something that you want the audience to take away? A single message?

• 1:30:24Speaker 00

  Yeah. I think one message is, I think folks have had, especially into the AI research community, felt like there was a choice that they had to make sometimes to either work on the frontier of AI technology. And that was like consumer related apps or enterprise related apps. And like, they just had to work on chatbots. And that's, that's, that's the cutting edge technology. But also doing, you know, I think when people think about the true potential of what AI can do. And I think a lot of it is about improving human health, understanding our biology. But people have felt like they have had to choose. And like, if I work in that space, I can't work in the frontier of AI.

• 1:31:06Speaker 00

  And what I would like people to take away is that you don't have to choose. I think we can work on things you truly care about, that you think will push humanity and work on cutting edge technology. And that's what we're trying to build at Radical Americas.

• 1:31:19Speaker 02

  Yeah. And I mean, clearly, and I encourage people to read the blog posts, the architecture, the mecon turf. There's a lot of innovation that is going into building these models. And I firmly believe that biology is really on the forefront of AI.

• 1:31:35Speaker 00

  Awesome. Yeah. Glad you feel that way. Amazing.

• 1:31:38Speaker 01

  Thanks for chatting.

• 1:31:39Speaker 02

  Yeah. Thank you for making the long trip.

• 1:31:45Speaker 00

  Anytime.

• 1:31:47Speaker 00

  Appreciate the invite. I had a blast.

• 1:31:49Speaker 02

  Great. Thank you.