🔬 The Lab of the Future Should Feel Like a Data Center — Andy Beam & Rafa Gómez-Bombarelli, Lila Sciences

Imagine a dark warehouse. Racks and racks of devices with wires, tubes, and electronics sticking out. The next AI data center? No. This is Lila Sciences https://www.lila.ai/‘ dream for the future of science. A dark warehouse full of AI-guided robotics and lab equipment, cranking out new experiments 24/7, building toward a scientific superintelligence. Their automated lab is almost hypnotizing to watch. They have floating plates zipping around on Wall-E-esque tracks, used vision-language models to control Windows 95 boxes, and created the world’s largest collection of voided warranties. In the process they’ve built a massive library of scientific reasoning tokens. Over 10 trillion of them, all experimentally validated. No warranties were voided in the making of this video To say Lila is ambitious is an understatement. Their goal is a scientific superintelligence wired directly into the wet lab. They are all in on the bitter lesson https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf, and the thesis follows from it: a lab is an infinite token generator. Produce data at scale, and the synergies give you a general reasoner that can tackle any scientific problem. They are committing hard. Biology, chemistry, drug discovery, and materials science, all at the same time. Time will tell if it works, but it is an exciting hypothesis. In our latest episode we sat down with Lila’s very own Andy Beam https://www.lila.ai/team/andrew-beam (CTO) and Rafa Gómez-Bombarelli https://www.lila.ai/team/rafael-gomez-bombarelli (CSO, physical sciences) and went on a journey through the possibilities of AI-run science, almost as wide-ranging as Lila’s goals. Did we mention they do both materials science and biology? In the same AI science factory? Same time, same lab, same AI. Finally a guest who can settle a long-running debate we’ve had amongst ourselves: is biology or materials science harder? Watch to find out! We discuss: * The internet is spent, science is next. Why Lila thinks the scientific method is the last untapped internet-scale dataset, and why they treat RL as a data generation mechanism with nature as the verifier. * The lab as a data center. Instruments as nodes on a graph, a magnetically levitating “PCI bus” transport layer between them, orchestration as a slurm queue. Andy is not short on analogies. * Why Lila insists it is not an automation company. They optimize for flexibility and generalizability over raw throughput, which means humans stay below the API line wherever automating does not pay. * Your experiment has a runtime. We put Escalante Bio’s question https://blog.escalante.bio/your-experiment-has-a-runtime/ to Andy: if science is the token generator, what is the runtime of your data collection? His answer, in short, is that you cannot make the ribosome go faster. Why Lila bets on fast round-over-round iteration rather than big noisy multiplexed screens, and how Rafa’s team rebuilt a gas sorption measurement to run roughly 2,500x faster. * What is actually in 10 trillion scientific tokens. Not sequences. Experimentally verified reasoning traces, a kind of data that Andy argues exists on the internet in quantities that round to zero. * Breadth as a path to depth. Small molecule chemistry priors transferring to metal organic frameworks for carbon capture, and the claim that the general model beats domain-specific models sample for sample. * If you have the data, what do you need the model for? Sri Kosuri’s koan about the ML-for-drug-discovery business model, and Andy’s answer: the coding model got better because it also read Shakespeare and carnitas recipes. * The serendipity they want to automate. Emily Whitehead survived the first pediatric CAR-T cure only because the doctor treating her happened to know, from pediatric arthritis, which antibody would blunt her IL-6 response. Roll that dice again and you probably lose her. Breadth is how you stop depending on luck. * Move 37 for catalysts. Model suggestions for platinum-group-free electrocatalysts that went from boring, to what a 40-paper expert called stupid, to the best performers they have made. * Six months to in vivo CAR-T data in non-human primates, and the zero-FTE virtual startup commercial model that fell out of it. For context on why that number is startling, AbbVie paid $2.1B for Capstan https://www.biopharmadive.com/news/abbvie-capstan-acquisition-in-vivo-cell-therapy/751944/ on the strength of preclinical in vivo CAR-T data. * You cannot have scientific superintelligence if you are just a good test taker. Ken Stanley https://www.kenstanley.net/, who wrote Why Greatness Cannot Be Planned https://www.amazon.com/dp/3319155237, runs open-endedness at Lila. RL at scale gives you a ruthlessly Vulcan problem solver. Machine creativity is a different thing, and it is the part nobody has solved. * The chain of thought is an unreliable narrator. The model reasons in latent space and only emits tokens. Sometimes it skips the experiment entirely and is still right. So how much do you trust the reasoning versus the verifier? * Reward hacking when the rollout is physical. Chains of thought that collapse into repetition, and a model that got annoyed and swore at the scientist who kept asking it to redo a plate map. What happens when a pathological loop has a wet lab inside it? * The bittersweet lesson. Rafa’s inversion https://bidmap.berkeley.edu/seminars/rafael-gomez-bombarelli-bittersweet-lesson-scaling-ai-materials of the bitter lesson: in AI, scaling is a roadmap. In materials, scaling is a filter, because only the things that scale end up mattering. * Not your typical Flagship company. Why a famously single-asset biotech incubator spun out a platform bet, and Andy’s line that if Lila called itself a biopharma it would have a top-three GPU cluster. * Bottlenecks they would remove by fiat. Sim-to-real for physics-based simulation, and the fact that RL training runs at roughly 5% mean FLOP utilization. Watch on YouTube: This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe https://www.latent.space/subscribe?utm_medium=podcast&utm_campaign=CTA_2

• 0:00Speaker 00

  But not just tech bio, what do you do in terms of science?

• 0:03Speaker 01

  We are all in on the bitter lesson and scale. We think that methods that scale and that are general beat those that are not. You know, as Elia said at NERIP last year, we have but one internets. It's the fossil fuel. We fracked, we got every ounce of data that we could out of the internet, but it's gone. And so the question AI is like, where is the next internet scale dataset coming from?

• 0:24Speaker 00

  You know, people normally talk about different scaling axes. You have compute, you have data, and for science, data is not necessarily an infinite resource. And your point is that we now want to add a new scaling axis for data.

• 0:35Speaker 01

  We think that like the lab of the future should feel like a data center, rows of server racks, as densely packed as possible, and also as energy efficient as possible and things like that.

• 0:46Speaker 00

  Welcome to latent space science. I'm Brandon. I'm here with my co-host RJ. Today, we have Rafa Gomez Bombarelli and Andy Beem from Lila Science. We'll just start off and let you introduce yourself.

• 0:58Speaker 01

  Yeah, thanks for having us on the podcast. Like, you're a long time listener, first time caller. Excited to be here. I'm Andy. I'm the Chief Technology Officer at Lila. I've been an AI researcher now for something like 20 years, going back to the like pre-deep learning days, SVMs, random forest, things like that. I did a neural net PhD around 2010 to 2014, right as deep learning was taking off. It was clear neural nets were the thing to back, but like autograd libraries really hadn't been developed yet. So I did the backprop by hand, you know, back in my day, walking uphill both ways kind of thing. Got very interested in AI for healthcare and life sciences. My wife's a physician, so I watched her struggle through different things

• 1:40Speaker 01

  and thought that AI was obviously a natural solution for a lot of those problems. Did a postdoc at Harvard in the medical school, doing early work on medical AI. And was really, you know, I'm in it for the AI. I was really interested in what problems could AI solve. But I've also always been like startup curious. So I took a break from academia for a year and helped start a company called Generate Biomedicines, which was an early generative biology company. I was the founding head of machine learning there and got to do the fun kind of hybrid professor, startup founder thing for the next five or six years. So I had a lab at Harvard, again, sort of between the School of Public Health and the medical school, doing methods research, but also a lot of applied work. That was fun. Those are a great set of jobs. But I got a sense that the like AI moment was changing in like a very significant way. And I wanted to be a

• 2:25Speaker 01

  part of it. So I started to think about where could I work at the frontier of AI in on really, really exciting problems. And, you know, academia has a lot going for it. Access to scaled compute is not one of the things that it has going for it, or scaled resources. So I'd been an early advisor for for Lila and got very excited once the thesis crystallized. But basically, you know, science is as an infinite token generator to train models at scale. Why would I want to work on anything other than creating a new frontier model that can solve scientific problems? So I kind of joked that like I hung up the tweed jacket two years ago, left my position at academia, in academia, and joined Lila full time as the inaugural CTO.

• 3:07Speaker 02

  Yeah, I go by Rafa. I'm the Chief Scientific Officer for Physical Sciences at Lila and a co-founder. I was a computational chemist back in the day. We used a commodity resource that is compute. So it was clear that we could scale up compute to do molecular simulations. And that's sort of something that produced enough data that in the early 2010s, we realized we had a data problem. And so things switched gear for me right around then. I was, I worked with David Dubenau and Ryan Adams in sort of blending what I think first, like the felt like the first instances of deep learning for science. I was one of the first people to do generative AI for

• 3:50Speaker 02

  chemistry and with an autoencoder on tokenized molecules. And so I'm deeply in love with latent spaces. We actually have a very similar to your guys' logo, but for molecules. And that has taken its own life, that figure.

• 4:06Speaker 00

  This is the one that'll be on your tombstone.

• 4:07Speaker 02

  Exactly. My students have a Slack channel just to post it when it shows up in the wild. So not as much a story as Andy, but same. Converting the in the 2015, 2016 era, I spun out a company out of my postdoc at Harvard, a computational materials platform company, and then went to MIT, where I started my group in material science and engineering. And the group there was sort of working at the interface of molecular simulations and AI with things like generative models for material structure, autograd for sort of really cool gradients that we want to see in the molecular simulations. And by 2022, 23, sort of things were taking sort

• 4:52Speaker 02

  of the turn that Andy just mentioned, right? We had seen the bitter lesson come to computationally generated data. And that's the reason why, you know, Meta and DeepMind and Microsoft, they have teams doing AI for computational material science. But it was clear that we needed to bridge a gap and get this thing all the way out and make do AI for actual material science and not just the computational version. And that's sort of lined up with this opportunity to start spinning out something again. In, like I said, 22, 23, started sort of thinking about the idea. I'm very excited now to sort of being pushing this this integrated vision of scientific reasoning across all the modalities of science we can validate in the lab.

• 5:37Speaker 00

  All right. That brings me to, what is Lila's thesis? It seems like you have a very ambitious goal here.

• 5:43Speaker 01

  Yeah, it's a great question. I'll try and like, give you the TLDR, and then we can go a couple levels deeper. So like Rafa said, we are all in on the bitter lesson and scale. We think that methods that scale and that are general beat those that are not. That's actually, sounds straightforwardly true, but is actually counterintuitive and contra to much of the 70 year history of AI research. But the realization that we had is that what gave rise to large language models over the last, you know, four, five, six years, is the access of the combination of scaled compute and scaled data. That data came from the internet. It was human generated, and we have used it all. You know, as Elia said at NERIP last year, we have but one internet.

• 6:27Speaker 01

  It's the fossil fuel. We fracked, we got every ounce of data that we could out of the internet, but it's gone. And so the question AI is like, where is the next internet scale dataset coming from? Post the pre-training era, we moved into reinforcement learning with verifiable rewards. So people talk about RL a lot. But really what RL is, is a way for a model to generate its own data. And the reward signal reinforces good data and penalizes bad data. So that has been a very productive framework for problems in math encoding. But at Lila, what we believe is that actually, science, running the scientific method and using nature and experiments as verifier is like the ultimate version of that. And so what we're building, we'll talk about

• 7:10Speaker 01

  these things that we call AI science factories. They are scaled verifiers for science, so that we can do post-training at scale and push out the frontier of what reasoning models are capable of. So that's like the thesis in a nutshell.

• 7:24Speaker 00

  Your proposal is basically, you know, people normally talk about different scaling axes. You have compute, you have data, you know, you have parameters. And for science, data is not necessarily an infinite resource. And your point is that we now want to add a new scaling axis for data. So I want to quote some of my friends at the Escalante Bio have a blog post, really good blog post, I recommend you read it. It says, "Your experiment has a runtime." So what is the runtime of your data collection?

• 7:50Speaker 01

  I mean, so that is an awesome question. It's so in, it will obviously vary by experiments. So like, you can't make the ribosome go faster, at least to my knowledge. There is the biology sets a limit for how fast you can go. In material sciences and chemistry, there are smaller time scales, there are bigger link scales. What you're actually kind of asking is a technical question, though. So how do you train a model against feedback mechanisms that vary by orders of magnitude in terms of a feedback? So we think about all of Lila as being able to generate different kinds of data on different link scales. We can then synchronize how we train the model, once that data has been generated. Again, for some of the experience we do, the

• 8:34Speaker 01

  link scales are on the order of days or weeks. And the question is, can we multiplex? Can we get more data per unit time? But the infinite token generator is still there. We just have to solve the technical problem on the other side of that to be able to line all these pieces up and train it into the model.

• 8:49Speaker 00

  So when you say the infinite token generator is still there, what do you mean by that? Because there are many different scientific tokens, you can imagine, and some tokens provide much more information than others. And certain things you can collect maybe at scale, like people who love NGS, you can basically collect an infinite amount of NGS data.

• 9:07Speaker 01

  Yeah.

• 9:07Speaker 00

  And yet, there are certain cases where, you know, another human genome is probably going to be a, you know, incremental update versus, you know.

• 9:15Speaker 01

  Yeah, like my genome relative to a reference genome is like a couple kilobytes worth of information. There's not a lot of information there. So that's, you're exactly right. So we don't want to generate the same kind of data over and over again. And so the platform that we're building is qualitatively different than traditional automation frameworks. So actually, the experimental platform that we're building prioritizes generalizability and flexibility over raw throughput. We want the model to be able to design a new experimental protocol, run the protocol, and receive the feedback, even if that's not an experiment we have thought about doing ourselves. So it's, the next incremental token has to be something that is valuable to the model versus yet another NGS sample to teach us something where it's

• 10:00Speaker 01

  already hit diminishing returns.

• 10:02Speaker 03

  So when you say next experiment,

• 10:06Speaker 01

  Yeah.

• 10:06Speaker 03

  what I think of is, traditionally, you would go into the lab and reconfigure the lab in whatever way, and then run some experiments by hand, maybe over the course of weeks or whatever. How does the lab get reconfigured for the new experiment at Lila?

• 10:26Speaker 01

  The way to think about the lab is it's almost like a graph. And so each instrument is a node in this graph and an edge between the node indicates that there's a physical transport layer between those two instruments. And so we'll probably have a video that we'll show in a bit, but we have a physical transport layer that connects almost every instrument that we have bought at Lila to each other. These are currently planar motor systems where there's a 96-well plate that magnetically levitates over a track. You have sort of millimeter control over where that plate goes. And so you can, I think of it almost as like a PCI bus where each instrument, I'm not short on analogies for like the

• 11:05Speaker 03

  That's a good one.

• 11:06Speaker 00

  I think half of the audience might not know what a PCI bus is.

• 11:10Speaker 01

  Yeah, so it's a universal serial bus that on your motherboard allows you to connect a new device. So if you plug a new graphics card, if you plug a new hard drive in, there's a bus that allows that device to speak to the rest of your computer.

• 11:25Speaker 03

  And this works for like biosystems and material systems and et cetera?

• 11:30Speaker 01

  Increasingly, but not totally yet. So the other thing to keep in mind about automation is there's a very long tail

• 11:36Speaker 03

  Yes.

• 11:36Speaker 01

  of things that you have to solve to be able to automate. And to date, people have not been thinking about end-to-end automation in this like flexible kind of way. And so there are instruments that are not connected to this now. There's not a lot of high throughput automation in material sciences, for example, and we've been building custom instruments for that that then are brought onboard. But there's like an 80/20 rule at play here where things that are easy to onboard and automate are plugged directly into the PCI bus. And then things that are not, people still move, people will still move a sample to that. Or it turns out that removing a cap from a test tube is a very hard thing to automate. Like a lot of the lab assumes that you have opposable thumbs and you're good with them.

• 12:18Speaker 01

  And some of the things that we've seen discussed about Lila frames us as an automation company. And that's like kind of the wrong perspective to think about what we're doing. We're not automation maximalists. We are actually sort of like token generation maximalists and flexibility maximalists. So we will over time automate things that make sense to automate and then again, use solutions now where they make sense.

• 12:40Speaker 03

  So the, the system designs the experiments, it gives instructions, there's like, oh, people need to actually do this thing. So you get, you recruit some of the staff to go and do that.

• 12:51Speaker 01

  Yeah, everything's an API call. And so sometimes when you call an API, there's a robot arm, sometimes there's a, there's a human arm that does something.

• 12:58Speaker 03

  People are literally below the API line.

• 13:00Speaker 01

  Yeah, please. I think that like, again, we want to spend resources where it makes to spend resources and make rational decisions. And sometimes it just doesn't make sense to try and automate a step when a person can do it in a tenth of a second. But what matters is that the, the model has the ability to give instructions to test hypotheses and that all of that data is visible, transparent, stored, so that those tokens flow back into the model.

• 13:28Speaker 00

  Do you have your AI models doing entire like experimental designs, which are beyond just a pre-existing protocol where you tweak like relative ratios or sources from, or like what, you know, all it goes, go into a pipe or pipe fat or something?

• 13:43Speaker 01

  I mean, it depends on like your threshold for novelty here. Certainly for expression protocols, for some gene editing work that we've done, we have tested like the, the platform's ability to do that versus humans. Model gets like 80% of that zero shot, humans get 0% of that zero shot. Are we doing like fully open-ended, free-form experimentation now? I mean, no, not yet. That is the goal, but we're building towards that. That is the end state that we want to be in. But we have seen the ability to do what would be an enormous amount of human intellectual labor over a very, very short time horizon.

• 14:20Speaker 00

  So when you're giving these, you know, giving your AI models kind of free rein to start designing new experiments, like how do you make sure that these are things that should be measured or that you validate that this is, you know, a good strategy or that you didn't just waste so much of money?

• 14:36Speaker 02

  The first one is, there is, I think, maybe an underlying safety question there. And I think that we've been taking very seriously from the beginning, right? Both security and safety, security of the data and the safety of the model suggestions. We have a very strong team, it's growing and there's sort of very strong leadership. And that's the first layer, right? Like we have strong safety, AI safety protocols that look similar to the sort of uplift considerations that people have been looking into large language models, only it's absolutely, we have strong AI safety protocols that look similar to the sort of uplift considerations that people have been looking into large language models, only it's absolutely for real.

• 15:20Speaker 00

  In a lab automation setting where you're working on some, you know, biophysical material science type problem, what are actually the dangers you have to worry about? There's like, I generally think of malicious actors and/or, you know, situations where you have a sufficiently complicated system that it could genuinely output something dangerous. It seems like from the scope of Lila, as I understand it, which we haven't talked about yet, maybe it'll come in a minute, it doesn't seem like safety is actually going to be a major concern at this point.

• 15:52Speaker 02

  I mean, it's something we need to take seriously from the beginning, right? Something where we cannot afford to not get it right. I agree with you. Right now it's in the hands of Lila employees that are sort of whose interests are aligned and whose understanding of the platform is aligned with, with our mission. So I agree, we don't have to worry about malicious actors. We still need to worry to some degree about the model giving a suggestion. I think it's more some things that start touching into lab safety more than malicious. I don't think we're going to have emergent behavior where the model suggests an extremely toxic chemical. It's more about sort of pushing an instrument such that maybe it is sort of, it overflows, it combines chemicals it shouldn't have.

• 16:35Speaker 02

  So I think there's like a chemical EHS safety layer that needs to be there since the beginning because we're doing sort of open-ended.

• 16:44Speaker 01

  I mean, I do think Rafa is right in that safety is not something you can procrastinate on because capability curves tend to be sigmoid shaped and it can look like everything's fine and then all of a sudden there's something that you didn't anticipate the model being able to do. So we are definitely proactive on that, that side. We have an AI safety team like Rafa said, but I think you're also right in that like we can constrain the problem in meaningful ways in the way that like a broad-based AI system that interacts with the general public cannot. We can also lean on biosafety levels and things like that, you know, good old fashioned lab safety to help in the meantime.

• 17:20Speaker 02

  And of course the nodes that are exposed to a particular, we don't necessarily need to expose all the experimental capabilities to all the scientific questions, right? For an antibody design question, we probably don't even need to expose the model to the fact that we have gas canisters that contain gases, right? Because it's not going to need them. So we can still be creative with, with sort of questions that relate to one particular area of science.

• 17:44Speaker 01

  Yeah. So your question though is, sorry, is interesting, like how do you know if something is dangerous is like actually kind of hard to do or actually, how do you know if it's wasteful? Some of the work that we've been doing in like electrocatalysts, we have someone inside of Lila who's like published 40 papers on the topic and some of the suggestions from the model initially were boring, but then transitioned from boring to what he considered to be stupid. These are non-platinum group electrocatalysts for separation of hydrogen, oxygen from water to make hydrogen. And those turns out to be our best non-platinum group electrocatalysts that we've made. So the line between like obviously wrong and like quasi move 37, surprising even to a human expert is hard to know. And so we will do wasteful things because we kind of want to know the difference between,

• 18:29Speaker 01

  between the two.

• 18:30Speaker 03

  That brings up the question, so for like an experiment like what you're describing now, it is obvious whether it works or not, right? But you can imagine, and there was some controversy in previous, at the Berkeley Lab around measurements that were misinterpreted, right? How do you know that your measurements of effectiveness or whatever you're, you're optimizing are actually correct?

• 18:58Speaker 02

  Yeah, I'm very familiar with that part of, of the landscape. I would say we cannot relax our standards of scientific rigor because it's AI, right? It's not, you know, maybe five years ago, when, you know, we started doing generative models for X and Y, they were like, yeah, it's cute, it kind of works like you would do with a kid. It's like, but now we're past that and we need to hold the AI science to the same standard we hold regular human-led science. And I think that's, I think that 2023 paper was the switch over from the community, a part of, you know, the AI community, AI for science people were always excited to see incremental progress. And I think at that point, we started collectively touching upon the rest of the community's

• 19:43Speaker 02

  awareness and they were like, fantastic, but.

• 19:47Speaker 02

  You know, now we're going to talk about the way we do things to our highest standard. So I think we have lots of experimentalists and I want to go back to the API point. Yeah. I think we've had the fortune by starting from zero to build a company where people are sort of AI aware, AI excited. We have sort of across all the people and the networks that I collaborated with, we've managed to build a team of experimentalists and automation engineers that really believe in the mission and really want to make it happen. They're really taking this graciously, right? So whenever AI gives something that is sort of very, very wrong and they're there to just, okay, they push the red button, watch out, this is a bad idea.

• 20:29Speaker 02

  But they're also gracious in, for instance, trying false positives. False positives are terrible for human scientists, right? Because you go to try something, it doesn't work. For the model, it's fantastic. It reduces uncertainty a lot. For the operator, it's kind of a bummer, right? Because you thought you were going to get something cool. And I think we've managed, and going back to the point Andy made, for the, you know, I think about until three months ago, people would be sort of approving AI decisions. And I think about three months ago, we started seeing that the model's crazy ideas started being sort of surprising to people, but surprisingly good. It's like, I don't know, I guess we need to try. And we've seen the switchovers.

• 21:11Speaker 02

  So I think the ability of people, experimentalists, to sort of challenge the AI, but being gracious, like that interface of human and in computers has been very rewarding over the last few months.

• 21:22Speaker 01

  Also say like, giving the model control of the lab forces you to build infrastructure and to expose pieces of data that you wouldn't normally want to or care about. And, you know, maybe no experiment is wrong, but you want the ability to explain the outcome. So if you think about, if you have an experiment, you fit a statistical model, what you're trying to do is use variation inputs to explain variations and outputs. And so we have the ability to explain variation and outputs because we measure so many different things, because we have to expose that to the model. So we can say, okay, the humidity was off in the lab that day. Maybe that explains exactly the thing. And then we can also push button and rerun the experiment to verify. And so we do not believe, like Rafa said, in some sense, we have to be more skeptical of any outcome,

• 22:07Speaker 01

  but we can then quickly go and rerun that experiment because it's all software effectively.

• 22:12Speaker 03

  And you find that the team spends a lot of time on verification? Or like, how's the breakdown?

• 22:18Speaker 02

  Less and less so. I think at the beginning, it wasn't so much, I mean, there is an execution of, okay, we've got a hypothesis. We've got a set of instructions that's going to go off to the API. We vouch for it and then some parts of the API are people doing things. And I think that will stay, right? Like the label of it. But I think the double checking that the intuitions were right, I think we're starting to see this super intelligence, local spikes of places where we're kind of, you know, supporting this emergence of super intelligent behavior more than we are sort of gatekeeping and that the ideas are not just wasteful. So I think that that's happening for domains. And maybe to elaborate a little on the example that Andy mentioned,

• 23:04Speaker 02

  we care a lot about energy sustainability, right? Something that, and you know, we're not just a biotech. We really care about energy and sustainability and materials. We're trying to make green hydrogen. And, you know, in order to make green hydrogen, you need to use light to split the water molecule. A bunch of that energy you need to pay for because it's the energy that's stored in the chemical bond and that you get from sunlight and electricity. And then there is some overhead that you pay. It's called the over potential, which has to do with the fact that the world is not perfect and things are lossy. And the loss comes from something called the catalyst. And today, the catalysts that are out there are okay, but they're expensive and rare. They're made out of ruthenium and iridium. So we set up a model to explore what can we do to not use these two elements?

• 23:49Speaker 02

  And people do these papers and there is, you know, there was something a couple of weeks ago that said ruthenium, you know, low ruthenium alloys for X, Y, Z. I mean, yeah, it's like, sure, you can, you know, dope it down, right? You can water it down, but it's still the same fundamental problem. You're using 50% less. So we've set out the model rules on this type of problem and we have the ability to make the material, measure the properties, measure the catalysis, measure the stability. And then in the second, third generation of sequential learning, right, this interplay between sort of information and what the model knows, we started seeing suggestions that were like, I mean, the words were fine. It was using the concepts that we use. It's just that, you know, I wouldn't apply that idea to that element.

• 24:34Speaker 02

  I wouldn't have put them together in that way. And it turns out those have been our best performing chemicals so far.

• 24:41Speaker 00

  I do want to get to the, Lila is not a biotech. But before we do, one last question along this train, thought train. RL is famous for reward hacking. You just, I forget what you said. I don't know if you said you were using RL or, you know, learning iterations. I would be very concerned that, you know, you throw some rewards and that you can really hack the, you know, physical sciences in a way that you can't do with compute.

• 25:04Speaker 01

  Yeah, I'm not going to disagree with that. 100% agree with that.

• 25:07Speaker 03

  What's the funniest example?

• 25:09Speaker 00

  Fun source.

• 25:09Speaker 01

  We have lots of funny, like RL fails that are not explicitly reward hacking. Well, I mean, one is when we trained, one of the early things we did was like, can you just like make a plate map? Like, can you like lay the experimental conditions out on a plate? And it got annoyed when the person would ask. So you would ask the model to do a plate map and it would do it. And they'd be like, actually, could you change these reagents? And it would like swear. It would be like, it's a 96 well plate. Come on, man. Like it's not done before, like in the chain of thought. I don't know where that came from, but it would like.

• 25:43Speaker 03

  Somewhere on the internet.

• 25:43Speaker 01

  Yeah, somewhere. The internet's in there. I can't forget that. So we've seen lots of like funny, like personality quirks like that as a function of RL. There's like obvious RL, I wouldn't call them reward hacking, but pathologies like repetition. So like the chain of thought will collapse and it will just like repeat its final answer over and over and over again. For some reason, that reliability sometimes leads to higher rewards. Like, we're not sure exactly why pathological chain of thoughts or non-legible chain of thoughts in some cases lead to higher rewards.

• 26:15Speaker 03

  So, sorry, I want to interrupt. So we're talking about RL.

• 26:19Speaker 01

  Yeah.

• 26:19Speaker 03

  We're talking about the way, way you're talking about it. It sounds like just RL on chain of thought, just like everybody's doing. But your RL actually has a lab step.

• 26:31Speaker 01

  Yeah.

• 26:31Speaker 03

  If you're in a pathological loop, does that mean the lab is just like doing the same experiment over and over?

• 26:35Speaker 01

  Well, it's just not, so a chain of thought, maybe just to step back, is tokens that the model uses to solve a problem. So if you were solving a math problem, you would do theorem one, theorem two, corollary lima. You know, you decompose the problem. In science, the chain of thought, there's some of that too. So there's reasoning that happens. You know, I'm trying to make an antibody for this target. What do I know about this target? What are the known epitopes? Like, what's my plan of attack? In the chain of thought are also tool calls. So I'm, maybe I'm going to use a structured prediction model in this case to get some read of how the sequence folds in three dimensional space. So tool calls are part of the chain of thought. At Lila, the fun thing is that the lab instruments are also tool calls

• 27:15Speaker 01

  or a series of tool calls if for a workflow or a, but it's all human legible. It's all in English.

• 27:22Speaker 03

  Right.

• 27:22Speaker 01

  And so what, some of the pathologies we see is it just skips all the middle part, which we would think is important for solving a problem and just goes right to the answer. And says, I don't need to do an experiment in this case. I don't need to call a tool. And, you know, in some cases where we can judge something, because maybe we've already done the experiment or something like that. For some reason, it is actually not a bad strategy in some cases. And so there's some mystery there.

• 27:44Speaker 00

  It's a theorist.

• 27:44Speaker 01

  Yeah. Yeah, it's still the calculation. And this is probably too much of a tangent, but like, it actually thinks in latent space. It emits tokens. So like the chain of thought is often an unreliable narrator for what the model, the computations of the model is actually doing. And so one of the big things we're trying to think about is when we're moving into working on a problem, you know, like Rafa said, for electrocatalysts, that we actually don't know what right and wrong looks like. How much should we rely on the chain of thought versus just trusting the experiment, trusting the verifier, trusting the simulator as the ultimate ground truth?

• 28:18Speaker 00

  So, you know, Lila is not a biotech company. Lila is actually fairly unique, I think, in this way.

• 28:25Speaker 01

  You know, I've been involved with biotechs, I've helped start biotechs. Often the goal is to sprint to a clinical trial. So you want to develop an asset. You develop the platform in service of having optionality of what space you move into. But once you do the, you have the clinical asset. Put everything into a medically induced coma and you get through the clinical trial. And if it goes well, then other things get to. So we are taking that option off the table. The model itself is the thing of value at Lila. So in that sense, we're much more of like a NeoLab, trying to think of a new way to push forward capabilities of a core reasoning LLM-based model.

• 29:01Speaker 03

  Not even the lab platform?

• 29:04Speaker 01

  So the lab platform is the token generator.

• 29:06Speaker 03

  Okay.

• 29:07Speaker 01

  That is the data generation mechanism that ultimately is the mode for Lila is that once that continues to scale, the amount of data that we can generate, both per unit time, but per unit square foot will go up. And that feeds back into the model to make it smarter, that then suggests the next experiment to do. And so we really are focused on making this core model as performant and smart as possible. And we can talk about how that lends itself to different commercial strategies, but ultimately we're interested in creating this new type of AI model.

• 29:41Speaker 00

  So I want to quote Sri Kosuri from Octant Bio, who had a great tweet, I really loved, you know, a few weeks ago was, "What is the business model in ML for drug discovery? Because if you need the data to train the model, but if you have the data, what do you need the model for?"

• 29:54Speaker 01

  That is true when you are narrowly scoped. So that is true within a given vertical of science. The analogy that I would use is like, if you went back 10 years and you tried to create like a coding assistant model, you would just get coding data. You wouldn't also get Shakespeare poetry, Carnitas recipes. It just, it turns out that there is spillover as the model is able to train on a broader swath of data and a deeper cut of data. And so again, the core bet that we're making is that is true for science. That if the model is trained on an increasingly broad set of data, the amount of data that you need in a given domain, that data requirement is reduced. In some cases will be reduced to zero if it's adjacent to what the model has already seen before.

• 30:36Speaker 01

  And so there's a data efficiency argument that would suggest that, again, having a general platform that can create a broad swath of scientific data. I'll also just mention that like, obviously we are using things that are already commodities. So, so public data sets we use, simulators we use, and experimental platform is a complement to these existing commodity resources.

• 30:58Speaker 03

  This brings up a question in my mind about, there's a concept of applicability domain, where you have different scales, different, and they result in different types of, completely different types of information and relationships between entities, right? So you have the, the quantum realm, you have chemical realm, you have, you know, sort of different bio realms. One concern I would have with cross-cutting approaches, is there domain transfer between these at all? Whereas, you know, Carnitas recipes and, you know, chess problems have the commonality that they're written in language, whereas you almost have a completely separate, not even language, right?

• 31:42Speaker 03

  It's a completely separate model between these domains.

• 31:46Speaker 01

  Human scientists work on all of those domains.

• 31:49Speaker 03

  Correct.

• 31:49Speaker 01

  And they mostly communicate with each other in written language using, using tools. So I would say there's a, there's a common reasoning process that allows someone to solve problems in each one of those domains. And so I think that that logic carries over to a reasoning model that we're training that again, uses tools, can do math, can do code, but it's, it's having all of that knowledge stored in one place.

• 32:13Speaker 03

  One classic example for me of domain transfer is between complexity theory and quantum gravity, right? Where now, a lot of the quantum gravity theories are, are basically recognizing the, the, the identical math behind the two of them, right? Do you have examples of this kind of sort of, oh man, this domain actually applies to this domain?

• 32:39Speaker 01

  So we have assembled this reasoning dataset of 10 trillion scientific tokens, reasoning traces that are experimentally verified across life sciences, chemistry, and material sciences. And we have seen that this general model often beats the domain-specific models. And so it's, it's hard to point to what's in the model that is making it, that it, what connections it has realized, but clearly, having seen more data across all of science beats sort of in a sample for sample kind of way, domain-specific reasoning models.

• 33:10Speaker 03

  The future of, of science is language, right?

• 33:13Speaker 01

  Well, so, yeah, so I don't.

• 33:15Speaker 03

  Yeah.

• 33:16Speaker 01

  Future of chemistry is.

• 33:17Speaker 03

  Yeah, yeah.

• 33:18Speaker 02

  Maybe?

• 33:21Speaker 02

  I don't think it's necessary, I don't think that's a necessary condition for a scientific superintelligence. I mean, there was this quote from Demis Hassabis, what, last week?

• 33:29Speaker 01

  Yeah.

• 33:29Speaker 02

  That it might not be worth distilling all the ways that live in sort of, you know, protein, alpha form, right? Or, or there are data modalities that are so different from language. And, you know, and he always tells me, "Well, Rafa, English is Turing complete, so you could express everything."

• 33:47Speaker 01

  It had been several Turing complete languages, but yes, yes.

• 33:50Speaker 02

  So, and I agree with that. There might be places where it's more efficient because of the nature of the, I mean, you've done geometric deep learning, right? I think for, for geometry and maybe, you know, I call my, my colleague, Ted Smith, I think geometry is one of those places where people feel that there might be sort of, just the nature of the problem is more amenable to other architectures. So if we need to call a protein folding model, or we need to call an equivariant diffusion model to make crystal structures, that's fair game. So I would say, um, the future of the way science talks with us, for sure is through language. That the model needs to be thinking in English about chemistry all the time? Maybe yes, maybe not. Chemists don't think about chemistry in English.

• 34:31Speaker 03

  Okay, yeah. Fair, they talk about it in English.

• 34:33Speaker 01

  Yeah, yeah, exactly. And I, I agree with everything Rafa said, like, token-based reasoning with tool use is very powerful. And I think the claim that we're making is we have barely scratched the surface for that in science. We're not trying to distill, uh, domain-specific models into a reasoning model. It can use those tools productively. And so it's just the combination of reasoning, often in English, but also in Python and things like that, combined with tool use is very powerful. And we're very early in science and understanding how far we can push that forward.

• 35:04Speaker 03

  I see. So can you give some examples of campaigns that you are running that are, are representative?

• 35:14Speaker 00

  And actually, before you do that, I realize we still haven't explained that you don't just do bio. It's not just your tech bio. So I think this is a great, great lead into this.

• 35:23Speaker 01

  Yeah.

• 35:24Speaker 00

  So not just tech bio, what do you do in terms of science?

• 35:28Speaker 01

  Science? No.

• 35:31Speaker 01

  So maybe, yeah, so the, the way that we train the model is, is, I mean, it's across life sciences, so DNA RNA protein cells, small molecules, different kinds of chemistries, and different types of materials. So that is like where we're scoped now, which is admittedly a large.

• 35:48Speaker 00

  But materials itself is also not just, that's also as largely scoped as everything that's in the bio.

• 35:53Speaker 02

  Yeah, we can give some examples. So today we can make thin films, we can make powders, we can make quantum dots, we have a cute quantum dot. Are you folks familiar, quantum dots are the luminescent technology in some TVs? And you need to control to make them of exactly the same nanometer size. And the nanometer size you make them controls what color they're going to be. And you need the purest red and the purest blue and the purest green to make really sharp and, and rich color palette for your TV. And you need to make them as homogeneous, they all need to be the same, otherwise the color gets again, and blended. So we have a cute demo where our self-driving lab, we ask our visitors to pick a wavelength, what color you want the quantum dot to be when they come into the office?

• 36:35Speaker 02

  And then we fire off the machine, the model reasons, even sometimes we've been throwing new chemicals that the model had never seen, just to see what, how it moves. The machine is running and by the end of this sort of hour, hour and a half tour, the machine has made maybe one, maybe more generations of quantum dots that tend to hit, otherwise we wouldn't do it, right? Tend to hit the color that people suggested. So we have the ability to make lots of materials. We can formulate liquids and polymers and soft matter. We care about, about energy and sustainability a lot. So we have a good chunk of electrochemistry capabilities, about the interplay of chemical transformations and, and electricity as a renewable energy source. We care about traditional catalysis, and, and we care about mechanical properties of materials.

• 37:20Speaker 02

  And all this comes together in programs where, you know, we make catalysts, we make high-performance coatings for corrosion or, or aerospace, high-performance mechanical applications. And over the last few weeks, with a, with an external partner, we started multiple sprints of things we weren't doing before that touch, you know, from adhesives to cooling fluids. So we've, we've been able to sort of more and more spin up just exciting discoveries in sort of open-ended chemistry and material science spaces.

• 37:52Speaker 00

  Do you have any, you know, connection between quantum dots and let's say protein design?

• 37:58Speaker 01

  It's the same platform that does that. It's the same set of capabilities. And so there's a shared infrastructure that lets us do all of those things under the same roof. If there were no connective tissue, then our ability to do, we just would not have the ability to do all those things. Have we done like the Meck and Terp thing where we look inside the model and see, does this insight from electrocatalyst inform? We haven't done a deep dive on, on the Meck and Terp thing. We have seen that our ability to, to do these programs has gotten faster as the platform has become more mature.

• 38:29Speaker 00

  So is LMP just like a common thing along, like, is this something really common in your toolkit that because of this, this enables like a large fraction of these ideas that you just mentioned? It certainly makes sense on the bio side, but I, I know bio much more than materials. Is that, is that like a common theme amongst your, like your lab toolkit?

• 38:49Speaker 02

  The AI science factory, the more capabilities has, the faster we've been able to go after new target product profiles and about new exciting opportunities. Because, you know, the model is prepared to do more things, the lab can do more things, our scientists are more flexible and, and more, and faster in order to incorporate new capabilities. Adding new instruments has become faster, the more instruments we have. So maybe echoes with sort of what type of company we are, sort of the, there is echoes of hyperscaling here of this sort of. Scaling in software is backed by scaling in hardware. And the fact that we have sort of tens of thousands of square feet of lab coming online with sort of dozens to hundreds of instruments is giving us this breath to move fast.

• 39:34Speaker 02

  There are places, you folks had my colleague Heather Kulik here in the podcast recently. One of the areas you were seeing, desorption, so I think the audience will be familiar with these materials. I don't need to spend a lot of time introducing them. These materials are made of the interaction of a molecule with a metal. And it turns out that our models have been trained on small molecule drug discovery. And all of the chemistry that they had learned thinking about drug discovery carried over to start reasoning over this metal organic framework materials that we can use to take CO2 out of the air or to filter ammonia.

• 40:10Speaker 00

  I find that fascinating. Like when I think so many times I've seen people work on machine learning where they train some big data set and then they move to some new domain. And oftentimes, the amount of transfer you see is small.

• 40:25Speaker 03

  So are there like a group of, I don't know how to say this, primary colors that you have that you combine together that oftentimes result in your experiments?

• 40:36Speaker 01

  So on biology, they'd be the obvious candidates, nucleic acid competence, self reexpression, and then downstream assays are things that we care about. So there are core competencies that we can then, you know, sort of give rise to a factorial number of different things that you can do. And on the material side.

• 40:53Speaker 02

  I think formulation.

• 40:54Speaker 03

  Yeah.

• 40:54Speaker 02

  And it wasn't even, it's so, I don't know, I don't want to say, you know, mundane, but it's so common, it's so important that it wasn't one of the first sort of, you know, super intelligent places. We thought of flashier things back at the beginning. And it turns out, a lot of people, you know, in industry and in the rest of the world, care about formulation meaning mixing liquids and gooey things to make other gooey things. But that's lubricant, that's lipid nanoparticles, that's deodorant.

• 41:25Speaker 03

  Yeah.

• 41:25Speaker 02

  Like there's all these things in consumer products and industrial products and in medicine, you know, gels to skin grafts, all those things emerge from this sort of mixing gooey materials. And that's a muscle that we're building. That's, that's a very common platform that is showing up all the time, the more we talk with people.

• 41:45Speaker 03

  Interesting. I have a rule of thumb that I often use when I'm thinking about scaling, which is that every time you scale an order of magnitude in a system, that your set of problems completely changes. You guys pick the two hardest problems, right? Materials and, and bio, you do other stuff, but materials in bio are notoriously difficult to get to market, right? That it, you know, 10 year, 15 year time horizons. And the reasons are, especially for materials, scaling. So how are you thinking about that? Are you just saying we're discovery? Or are you saying that we'll get to it? Like what, how are you thinking about it?

• 42:24Speaker 02

  The last academic lecture I prepared before I stopped giving academic lectures called the bittersweet lesson of scaling in materials and chemistry. Because it's this, it turns out, you know, in AI, scaling is a good thing because it gives you a roadmap of what you need to do. And in chemistry and materials, scaling is a spooky thing because it turns out only the things that you can scale matter. So we're extremely cognizant, right? Our product team, our lab team, we all know. For instance, in the quantum dot example, we've been able to use the same recipe from a single digit milliliters to 100, almost a liter. So there are places where, you know, our capability today takes bites into scaling and into technology readiness level.

• 43:09Speaker 02

  Then we are making the system such that they can reason about what's going to matter later as they're doing the experiments now. And this will be our rare earth free or sort of platinum group free catalysts, right? Precisely the nature of the question is that we need to be able to scale this, right? So it's supply chain conscious. As we're firing off the first experiment, we've already read every paper, we've already have a techno economic analysis agent sitting on this, in the corner, ready to do the techno economics of anything we do. At the end of the day, we're not going to do clinical trials. We're not going to make pilot plans for one particular process that you would put in your refinery, right? At that point, these are places where we will work with our customers.

• 43:54Speaker 02

  Or, you know, if we find something so amazing that we don't even need any instruction and we just go sell it. But typically, we will hand off, just like we're going to support therapeutic discoveries for our customers, we're going to support materials innovations at the pain points that our customers have. And those have to do with scaling.

• 44:13Speaker 03

  How far have you gotten so far?

• 44:15Speaker 01

  So on the life sciences, and I think I agree with everything Rafa said there. The way to think about like how people would use the platform is, or like just kind of like what we're building is much more of like a Claude Codish kind of thing for science. So one of the things that has drawn early customers to us is, so we're not an in vivo CAR T company. There's lots of in vivo CAR T companies, super hot right now. We did see, you know, six months ago with the Capstan acquisition for like $2 plus billion. If folks aren't familiar with that, like in vivo CAR T is this very new heart therapeutic modality, previously for blood cancers, but now increasingly for autoimmune disease. We did have in the internal, like sort of triumvirate of capabilities that you would

• 44:58Speaker 01

  need to do in vivo CAR T. So binder design, obviously, we can do that, LMP formulation, and then mRNA design.

• 45:05Speaker 03

  Just so people know what CAR T is.

• 45:06Speaker 01

  Yes.

• 45:06Speaker 03

  It's really freaking.

• 45:07Speaker 01

  No, I love it. It's so cool. Can I talk about CAR T?

• 45:10Speaker 03

  Talk about CAR T.

• 45:11Speaker 01

  So CAR T has been worked on since like the late 80s or 90s. Really caught fire around 2010 or so for, for cancers. The way it used to work is you'd extract someone's T cells, you would engineer what's called a chimeric antigen receptor that goes on top of that, that tells the T cell to what go in, to go, what kind of cell to go and kill.

• 45:28Speaker 03

  So you're, you're basically modifying people's T cells. You take them out, you modify it so that it has this weird antigen receptor on its surface.

• 45:37Speaker 01

  To seek and destroy tag. Usually they use a protein called CD19, which is preferentially expressed on B cells. When B cells get malignant, they create blood cancers, they create autoimmune diseases. You wipe out someone's, almost their entire B cell repertoire and do this. There's a lot of collateral damage. But essentially, you're telling the T cell what to go and kill. So this, what really started to catch fire around 2015, it was expensive and slow. You have to extract someone's T cells, you have to engineer them, it's like $400,000 per infusion and just like, it's still a miracle cure for lots of different types of cancer. Too much of a tangent for this, but there's this child named Emily Whitehead, who was treated at the Children's Hospital of Pennsylvania CHOP. She was one of the first cures in pediatric cancer by CAR T.

• 46:22Speaker 01

  She was going to be referred to hospice care, got CAR T. Another is like side tangent, she almost died of a fever from this initial CAR T treatment. The only reason she survived is because the doctor who was treating her had a daughter with pediatric arthritis and knew that this specific antibody would blunt her IL6 response to CAR T. So like, there's a lot to unpack there in terms of AI for science, all the serendipity that had to happen in that specific case for all of that to go right. And in probably, if you roll that dice 1,000 more times, you probably don't get that. That doctor at that moment, who knew exactly what antibody to give her to make that treatment curative instead of lethal. So again, like those are the types of like serendipity things that we'd actually like to automate. So anyway, it's so inexpensive.

• 47:05Speaker 01

  People then realized that actually through just an infusion, if you take an mRNA that encodes for the chimeric antigen receptor, you put it in a ball of fat called lipid nanoparticle, you put a CD8 targeting MOID on the outside of this ball of fat, it will then, you give them the infusion, it will go bind to the T cell, get ingested, ball of fat dissolves, mRNA comes out, chimeric antigen receptor gets expressed and presents on the top of the T cell.

• 47:28Speaker 03

  So you're just telling it, you're reprogramming the T cells to express these weird antigens?

• 47:33Speaker 01

  Literally programming biology.

• 47:34Speaker 03

  Yeah.

• 47:34Speaker 01

  And then the T cell goes and does its thing and wipes out whatever has CD19 in this case. So malignant B cells are, explain a lot of blood cancer. They also explain a lot of autoimmune diseases. B cells often make antibodies in response to auto antigens and things like that. So recently, six months ago, as the result of about six years worth of work spun out of a Nobel Prize winner's lab and about $100 million worth of R and B, R and D, we saw.

• 48:01Speaker 03

  That's some good music, man.

• 48:06Speaker 01

  We saw like some of the most compelling preclinical data for in vivo CAR T treatment of autoimmune diseases. It was by a company called Capstan. They were bought by AbbVie for like $2.1 billion. So at Lila, we had been working on all three of those things in isolation. So about six months ago, a team of two or three people inside of Lila tried to see what we could do to, in the in vivo CAR T. And what we had been working on was mRNA design. So like, most RNA medicines, the biggest knob that you can turn is expression peak and expression durability. So how many proteins do you get per unit of mRNA when you give someone a vaccine or some other mRNA medicine?

• 48:47Speaker 01

  So we have developed some monster UTRs, untranslated regions, which flank the protein coding region, which dictate those expression properties, something like 10x, the references from Moderna and Pfizer. And over the course of six months, got to in vivo data in non-human primates, where B cell depletion was significantly better than what was shown in the Capstan data. And the sort of like durability of that was also, all the characteristics that we looked at were significantly better. Having more CAR expression is probably one of the most potent ways to improve. CAR T therapy, the number of receptors that get expressed dictates how likely that T cell is to bind to the bad cell

• 49:27Speaker 01

  once it finds it. And T cells are literally serial killers and that they will go, they'll kill a cell, then they'll go the next one, the next one. And so how long they can do that is dictated by the, how durable the expression of the, the car is. Again, we're not a CAR T company. We're science nerds. We like to do cool stuff. So we got to that proof point in about six months where again, all the way up to where you might think about filing an IND for a new clinical asset. We're not going to do that. We're not going to do a clinical trial. Again, that would be all encompassing. But some folks who had been around Lila for a long time saw that as a way to do essentially like a two to three person FTE startup, where there's a couple of scientists who have domain knowledge and a combination of the model plus platform can do five years worth of biotech work

• 50:11Speaker 01

  over a six month period for 10% of the total investment. And so a lot of the like commercial relationships we're thinking about now are essentially like the zero FTE startup model, where someone comes with an idea, they say, you know, if I, if there was a CAR T in the market that could bind to do things, if it was a biospecific or if it had these other properties, I know the hole in the market that that thing would plug into. And so a lot of our commercial engagements are effectively virtual startups running on Lila now, where someone comes with a very well specified problem. They don't know how to get there. There may be some things related to target identification and things like that, too. But they can effectively run that entire program over as much shorter amount of time with a fraction of the cost.

• 50:53Speaker 03

  And so those are, those are like a partner comes to you, says, I have this idea. I don't want to build it.

• 50:58Speaker 01

  I don't want to build a lab. I don't want to hire a team. I want to get there fast.

• 51:01Speaker 03

  I just want to get it done.

• 51:02Speaker 01

  Yep.

• 51:02Speaker 03

  So it could be some academic at a university that's like, I have this idea. I kind of did a little bit of validation. I think it'll work. Can I do it? Can I sit with you guys for six months and make it work?

• 51:12Speaker 01

  Yeah. I mean, that, that's the right way to think about it. Um, yeah. The way that like contractually it plays out is there's like a platform access fee. There's like, we have to pay for reagents and running the system and then some overhead and stuff like that. And then there's like some upside sharing. That is a scalable model where we can service as the platform gets better, instead of doing dozens of those, we can do hundreds and then thousands of simultaneous, you know, kind of virtual startups being developed on the platform where we, we have revenue that helps pay the bills in the near, near term. But then we also have this upside partnership with folks who decide to, to build with us.

• 51:47Speaker 03

  That's amazing because this is what we're seeing is that people are more and more pushing towards getting rid of all the extraneous infrastructure and using automation and focusing on the idea.

• 52:01Speaker 01

  I mean, the way that I think about it is like, most of us got into science because we're curious and want to answer questions. You know, I'm a computer scientist by training and like, I like to answer questions through software. However, if I had to program in binary, I would enjoy that significantly less. There, there are high level abstractions, increasingly high level abstractions, you know, used to just be Python and Java. Now it's like Claude code. They help me answer questions faster. You know, the analogy is that like scientists are still programming in binary. They have a question that they want to answer. They have to just compile that down to an experimental protocol. Then they have to go and do the manual labor and get arthritis by like moving liquids from one, that's the equivalent of scientific programming in binary.

• 52:41Speaker 01

  And so we're trying to help scientists move up the abstraction ladder where what's the like, you know, maybe your idea isn't going to work. Most clinical trials fail, but you can at least get to failing fast if you don't have to do both the physical labor and also some of the intellectual labor to, you know, get all the, the pieces in the right place.

• 52:59Speaker 03

  You know, most clinical trials fail, you know, somewhere between five and eight percent of clinical trials actually get from IND to approval. So the, the discovery is not actually the constraint. I was interested, you were talking about the sort of economic modeling agent. I can't remember what exactly you called it, but I mean, that seems like the problem to solve. How do you think about this?

• 53:26Speaker 01

  The pro, you mean the, the success rate of, of clinical trials?

• 53:30Speaker 03

  The economic model underlying what scaling in general?

• 53:33Speaker 00

  For both bio and materials, like oftentimes there's this huge, huge process, like once you have something which is you consider final, like a IND or D development candidate.

• 53:44Speaker 01

  Yeah.

• 53:45Speaker 00

  For, for material, like there's still usually like 10 years of clinical trials or, you know, qualification in the material science world to just get that into a product. And oftentimes the bottlenecks there are things about scale manufacturing, about regulatory, about safety and things that are oftentimes just very hard to answer upfront. So every time you do this, you just have to, you know, roll a die. And there is, the typical, if you will, deal with this is, you know, essentially a portfolio model.

• 54:14Speaker 01

  Yep.

• 54:15Speaker 00

  You know, financing wise, it's, it's very much a, you know, the only way you can make money is if you scale with some level of like, you know, risk calibration.

• 54:24Speaker 01

  Yep.

• 54:24Speaker 00

  You know, it's really exciting to hear that you can do these things specifically, but how does it feed into the larger thing where even if you solve these problems, immediately it's still only 10% of the problem.

• 54:37Speaker 01

  The reason why US biotech is losing to Chinese biotech is not because of an innovation problem. There's a regulatory framework too, that has to go to enabling like fast clinical trials. The FDA has made motions towards that recently, both for the preclinical data that you have to submit in some cases, but also how we will run and monitor trials. So it'd be crazy to think that like one company could come, or even like any company combined could, could change, could change that on their own. So it has to be done in tandem with the regulators. However, the minor moves in preclinical probability of success matter a lot. You know, from a portfolio theory perspective, it makes investment much more attractive.

• 55:20Speaker 01

  It means, you know, in expectation medicines get to patients faster if fewer of them fail. And so I would say like, you know, that is the area that we're focusing on now is that a medicine created by

• 55:32Speaker 01

  a system that has had the benefit of, in this case, a million unique mRNA designs to maximize things that are known to translate to therapeutic benefits will, will meaningfully move those pre, it's still, you know, I guess you would say it's better to throw a loaded die than it is a fair die. And so we're just trying to like make the die as loaded as possible.

• 55:51Speaker 03

  I guess my thinking, this is the thing that I think about constantly is how do you bring, you know, basically translation, right? And whatever the equivalent is in materials. So however you name that, that I really want to see a model that thinks about, these are the factors that in, in reasons about and is very good at saying, I'm filtering my designs to the ones that I think are going to make it through phase three.

• 56:18Speaker 02

  Great. My wife is a translational scientist in biotech. So she reminds me very often, like, you guys should be doing AI for translational science.

• 56:27Speaker 01

  Yeah.

• 56:27Speaker 02

  In a sense, I think that's sort of some of the eco, especially in the materials and chemistry, you know, our, our tools can call process engineering simulators and go figure out what pipe diameters and what heat exchangers you should be using in order to scale up the process for the economics to be worth it. So Steve, maybe there is, you know, I don't know if we're going to gang up a filter until we go measure it. But the ability to reason now about the things that will come downstream, which is sort of a little bit what the translational sort of AI would do, is reason now about sort of what's going to matter. Because I, when you said earlier, when you have the IND, you're locked in.

• 57:08Speaker 02

  And it's true on the chemistry side, the molecule, the, the sequence you've chosen, of course, which population you're going to give it to, and how you're going to measure success, those choices you make afterwards, right? And I would say to the, I don't work on the preclinical stuff, but on the chemistry and materials, that is precisely the type of behaviors we're trying to instill now for the, with the verifiers and the data sources that we can access, either because somebody has thought about them, either because the physics allows it, or because we can measure good enough proxies now that tells us what's going to happen later.

• 57:45Speaker 01

  And to be clear, like the, all those things are things that we talk about internally a lot. We're already on the verge of being pathologically overscoped.

• 57:56Speaker 01

  But absolutely like the, the belief that we have is that as models get smarter, as they ingest clinicaltrials.gov, as we partner with pharma companies and get access to, you know, that cookie jar, these, these probabilities will meaningfully change. On the, the biomanufacturing side, having access to, you know, by good manufacturing processes, scale up processes too. We think the models will be able to contribute there. We've just chose to focus a lot of our commercial and collaborative activity on the sort of like frontier of science that we think we can address now, but the goal is, is to push past, push past that. If I may summarize, it's kind of like, it's a tool call.

• 58:36Speaker 03

  Yeah.

• 58:37Speaker 01

  It's all tokens. It's all tool call.

• 58:38Speaker 03

  Yeah.

• 58:39Speaker 01

  Yeah. Tokens and tool calls are all you need.

• 58:40Speaker 02

  But also the reasoning mechanisms that maybe you mentioned for that doctor that treated, you know, the IL antibody, IL-6 antibody that you mentioned, well, that person had learned that from, you know, a combination of lived experience and reading the literature. And we get, since we believe our thesis that the breadth gives us depth, we will get better at those things by doing more.

• 59:03Speaker 01

  And in so many counterfactual worlds, that doctor was not the one treating Emily Whitehead in that case. And CAR T may have looked like, it may have been yet another gravestone in E room's law, you know, for, you know, yet another failed drug. So I do think that like, that went from like a 2% success probability to a 98%, just because that person happened to be in the room. And so if we could just operationalize that, again, like the, you're going to move a lot of probabilities.

• 59:29Speaker 03

  Yeah. So that's a, that's a good example of where just having really broad knowledge of scientific information.

• 59:36Speaker 01

  Yeah. So that's like, that's almost like Google.

• 59:38Speaker 03

  Cause that was only being used in pediatric arthritis, like another like very niche area of medicine.

• 59:43Speaker 01

  I see. Yeah.

• 59:44Speaker 00

  So you have Ken Stanley in your team, famously written, wrote the book, Why Greatness Cannot Be Planned.

• 59:50Speaker 01

  Yeah.

• 59:51Speaker 00

  And is very big on open-endedness and serendipity in ML research. So what is the role of open-endedness at Lila?

• 59:58Speaker 01

  Oh yeah, one like Ken is awesome. So for those of you who don't know, Ken pioneered an area of machine learning AI called open-endedness, which I think of as machine creativity. Like, how do we get models to do open-ended exploration? And also like have a sense of taste about what's interesting, what things we should go down. So you can't have scientific superintelligence if you're just a good test taker. So like, if you think about what reinforcement learning is doing, even at scale, it's answering questions in kind of like a ruthlessly Vulcan-esque, like, you know, Spock kind of way. But you probably only in like limited ways would think of that model as being supremely creative. And so Ken has created or built an open-endedness team at Lila

• 1:00:42Speaker 01

  to sort of take, take the outer loop or the meta part of that reasoning challenge on. So how can we get our models to not only be able to answer tough questions, but ask interesting questions in the first place? And so that's really Ken's mandate. He's been building like a world-class team over the last like several months and they're, you know, they're in the kitchen cooking now. And I think by the end of this year, we'll have some cool stuff from Ken's group to share. So we're going to hop into a video here of the lab. This is going to show a couple of different things. All right, so that's probably a peeler or sealer.

• 1:01:16Speaker 01

  So when you move plates from instrument to instrument, obviously there's liquid in it. Most of biology is wet. So you put these stickers on it. So that was a plate being sealed.

• 1:01:28Speaker 01

  All right, so here we go here.

• 1:01:30Speaker 03

  It's picking up a

• 1:01:31Speaker 01

  Yeah, so this is inside of a liquid handler.

• 1:01:35Speaker 01

  Let me go, we'll wait till it gets to a wider shot here. So that you can see the PCI bus and see some of the robotics.

• 1:01:42Speaker 03

  So the liquid handlers are off the shelf.

• 1:01:45Speaker 01

  Yeah, so this is the planar motor system here where the plate magnetically levitates. This is the PCI bus where the transport layer connects all the instruments. You can see benches there where all the instruments sit. A robot arm picks it up, is now going to transfer it to a different plate to go on to the next. There's a little bit of a traffic control thing that you have to do here. Like they actually will go and park for a while while congestion clears.

• 1:02:08Speaker 03

  Yeah.

• 1:02:09Speaker 01

  And here's a long shot of the PCI bus. And again, like, all of that's fully controlled, all of that's fully automatic. And this is a material science example.

• 1:02:16Speaker 02

  This is a physical science example, yeah, where it takes us back to the scaling point. Here it's making our hydrogen catalysts in a scaled up form factor by, that's an ink that contains nanoparticles of the material. That's a spin coder, as you can guess from the fact that it spins the plates. And then this robotic handler moving around little pieces of catalysts to test. And this nice looking purple 90s neon vibe. This is called a magnetron spattering machine where we make atoms fly from a source and deposit on the other side of the chamber in a very thin atomic film where we can make arbitrary mixers of elements based on what's on the three or four sources,

• 1:02:58Speaker 02

  which is vaporize them and make them fly over the chamber and make these nice thin films that are very material efficient. We can do this with very, very little material. And it's one of the workhorses for us to design, test, design, make, test fast in many applications, in catalysis, in corrosion, in mechanical properties. Many things you can test in this sort of very convenient form factor.

• 1:03:20Speaker 03

  The liquid handlers and some of those machines, those are kind of off the shelf mostly. And then, so the, and then you have this, you've come up with this sort of form factor that works for lots of those, those machines, both for material and for bio.

• 1:03:38Speaker 01

  Quantum dot is a good example of the combination of the two. It's actually a liquid handler that we've repurposed for quantum dot synthesis and design. And that, you know, I think that speaks to like, you can get very far with, you know, 20, 30, 40, 50 instruments. The ability, they just have to be on platforms that the model can use them. I think a big eye-opening thing for me coming into Lila, because I wasn't, I wasn't in lab automation, like in any meaningful way before coming to Lila, is it's not the automation that I was hoping for. Like, a lot of automation is point automation, where there is a tablet attached to the side of a liquid handler where you can enter a macro. That device is not meant and then sometimes purposely designed not to talk to other things.

• 1:04:21Speaker 01

  And so a lot of what we have done has been to, you know, I kind of joke that we have the world's largest collection of voided warranties in biology. Because we have written our own custom drivers, our own custom firmware to get sort of low-level granular control of a lot of these instruments and make them, make them talk to each other. So the video is cool because you see magnetically levitating plates. What you don't see is like the custom software wrapper that like stitches all that together. And a lot of this comes down to like really hard software, hardware, interface challenges. Some of the machines literally still run Windows 95. And so think about how you automate that. Like, we actually have a vision language model controlling a Windows 95 machine.

• 1:05:00Speaker 03

  Yeah.

• 1:05:00Speaker 01

  Because that's the only way to automate it.

• 1:05:01Speaker 03

  I was just going to joke about a mechanical finger pressing.

• 1:05:04Speaker 01

  Yeah, you joke, but no, we did that. We actually did use a robot to push the iPad on the side of the thing. I think the other thing to like, the other thing to call it here is this is still automation made for people. Like, the instruments sit on benches, which are approximately chest-high because there's the assumption that someone needs to reach in there to service it or to fill the reagents. So this is the like V0, V0.5, what we think lab automation will look like. And because we've just decided to vertically integrate and own the hardware-software stack, the V2 will look very different than this, where we'll be able to integrate things. This is happening already on material sciences because those capabilities just don't exist. And, you know, we often think about labs in terms of like their X, Y coordinates.

• 1:05:48Speaker 01

  As we integrate, like, we'll have like a Z component too, because we'll be able to stack things. Again, tokens per like unit volume is like what we'll be thinking about then. But we think that like the lab of the future should not be made for people to easily walk into it. It should feel like a data center where you go and you see the rows of server racks. There's room for like a crash cart behind it to service the nodes. But it should be as densely packed as possible and, you know, also as energy efficient as possible and things like that. So, yeah, so we're, to answer your question, we're using commodity things now because it makes sense to get started, but over time, almost surely the form factors of those will, will change quite a bit.

• 1:06:23Speaker 03

  I see, I'm, I'm just a little surprised that you can come up with this common size of tray that kind of matches your needs for a good percentage of your problems.

• 1:06:35Speaker 01

  Well, it's just working backwards. 96-well plates are the like atomic unit of experimentation in lab automation. And so we now do 96-well form factors for material sciences as a result. Not everything fits into that, that form factor. But again, the, the coverage that you get from adopting a 96-well or 384-well plate format.

• 1:06:52Speaker 03

  80/20.

• 1:06:53Speaker 01

  80/20, exactly, yeah.

• 1:06:54Speaker 02

  Yeah, I mean, you can see some of those words. The, the pieces of deposited material were bigger, so we still use the plate shape to carry them over. But then the number of samples, right, that you have them is smaller. I think some of them are like maybe 12, four times three.

• 1:07:09Speaker 03

  Yeah.

• 1:07:10Speaker 02

  And this takes me also to a point you folks asked earlier about scaling and sort of how, when you scale, your programs are different. And a problem I think we're looking forward to collectively at the company is the orchestration and the scheduling of a data center-sized AI science factory, right? When it's, of all the experiments you could run concurrently, how are you going to think about the, you know, the logistics and the orchestration of moving all these samples and interfacing all these instruments to create sort of the maximum value for our customers, the maximum information for our model? And that's the exciting part. That problem is going to look very different from some of the other problems we're thinking about now.

• 1:07:49Speaker 01

  What we think about, Rafa said, is like orchestration on top of that is like a Slurm queue or something like that, that lets you globally maximize throughput of the, of the system that you have. But again, using those same abstractions to think about throughput, scheduling, orchestration. And as the system gets complex or gets larger, the, you know, the complexity in maximizing that throughput. So if you're like a, you know, a CSP, a constraint satisfaction problem nerd, like, we have like one of the coolest ones to, to think about.

• 1:08:18Speaker 03

  Are you thinking about scaling as like that one cluster? And then you just cookie cutter that? Or is it like, I have all of my liquid handlers and all of my whatever, spin coders? And all over in different parts of the lab or something?

• 1:08:34Speaker 01

  I mean, currently what we have essentially is one big fully connected graph. And like, that won't scale indefinitely, just, some of the material stuff use, throw off hazardous fumes. And so that's isolated for, for safety reasons. I don't know exactly what the exact configuration layout of the science cluster of the future looks like. But I think that it will probably have fewer instruments on it than like you might guess you would need. You know, hundreds, maybe thousands. But we do think about scaling it in the same way that you would think about scaling a data center. In that it's a multi-level building, occupies millions of square feet, and it's like a lights-out facility, as they say. It's like running 24/7, generating data in real time. And you would want the same uptime that you would expect of a data center.

• 1:09:19Speaker 01

  Now, that's very hard to do. That's like an insanely hard thing to do. But that is the, that's the endpoint that we, that we're, we're trying to work backwards from. What problems do you need to solve on the way to that, like, that endpoint?

• 1:09:32Speaker 00

  So this goes back to my previous question though about like the runtime of your experiments too. Because scaling means different things. And one of them is experimental design, which intrinsically scales, but maybe at the cost of signal noise ratio or some other idea. But, you know, getting broad data quickly and efficiently at some cost. Or scaling is, you know, lower throughput, but just paralyzing, you know, wildly. So like, in general, I would approach those as two different sets of problems. I don't think that generally the same strategy really works for them in general. So like, what types of scaling is more important for you as a scientist?

• 1:10:09Speaker 01

  I would say like round over round iteration is more important than like a broad, hugely multiplexed, highly like noisy kind of thing.

• 1:10:17Speaker 00

  Iteration time is really the single thing.

• 1:10:19Speaker 01

  Exactly, yeah.

• 1:10:20Speaker 00

  Yeah. Okay. So does that limit the domains that you, you want to focus on? Like, you know, now do you think like? If we are going to try to tackle a new problem, do we ask, can we just solve this problem with fast iteration versus something where maybe the answer is, will you scale up by massively multiplexing something, but with like month-long turnaround?

• 1:10:42Speaker 02

  Parallelizing and multiplexing are somewhat different, right?

• 1:10:46Speaker 00

  Yeah, that's my point.

• 1:10:48Speaker 02

  I would say pooled, we love pooled.

• 1:10:49Speaker 00

  Yeah.

• 1:10:50Speaker 02

  Pooled, we love, because you get fast and broad.

• 1:10:53Speaker 01

  What is pooled made for?

• 1:10:54Speaker 02

  Pooled are things like DNA encoded libraries where you

• 1:10:59Speaker 01

  Each well has a bunch of crap in it, then you can sort out the crap after you do the experiment.

• 1:11:03Speaker 02

  Somehow, the form of the assay allows you to throw a thousand or a million or a billion experiments at the same time, and the way the assay is set up, the readout picks the winner. So you try a million things in one plate, and you get one readout of, or a thousand readouts of the thousand winners.

• 1:11:19Speaker 00

  There's a joke that all biotech is just mapping whatever readout you want to on NGS sequencing.

• 1:11:26Speaker 02

  Yes.

• 1:11:27Speaker 00

  Tag it, you know, multiplex, there you go. You can get lots of data.

• 1:11:33Speaker 02

  The other argument would be that if the standard for a field is a month, and it's going to take us four days, a four-day learning cycle is amazing, because it's really going to move the needle for that part of the field. This is where our automation engineers and our teams are thinking about other ways of measuring things. And you know, in coulants and in catalysis, there are places where we just made different instruments that measure a different property that turns out response a thousand times faster. For instance, in absorption, I can tell you folks a little bit, in gas absorption, people typically measure, they pressurize an amount of gas, well, for the muff and cough materials I was talking about, sucking CO2 out of the air. You know how much from the ideal gas law, if you remember high school, you know how

• 1:12:18Speaker 02

  much gas you put in the little box, and then you wait for the gas to be absorbed in the material, you check the pressure, and from the difference in pressure, you know how much went into the thing. Then you up the pressure again, and you see how much extra went. And if this sounds low, it's because it's very slow. It's called BET, this is like about day per sample. And it's very tough to parallelize, because it's another gas line, another canister. Or you can take other types of proxy measurements from other instruments that are parallelizable. And that's something we build in the lab now, where instead of measuring pressure, we're measuring another property we care about, that is a readout for what actually pressure would tell us, but we can do 96 well plates for 96 metal organic frameworks in like an hour.

• 1:13:00Speaker 02

  So it's like, maybe 2,500 times faster. So this is a place where there's a little bit of room for ingenuity, or just, you know, an hour is still slow compared to other readouts, right? Other things in electrochemistry, maybe we can do in a minute, but now we're sort of, you know, a thousand times faster than the way we were doing it.

• 1:13:18Speaker 01

  I think the answer to your question also too depends on how much we think the model is starting from a dead start versus a walk versus a jog. So if there's some area that we care about, some question, it's clear there's like zero knowledge in the way to the base, in the model that we're using, then we may prefer a big, slow thing to move it in. If we think that it's already relatively competent in that, then we would vastly prefer the RAID serial fast iteration cycle. So it will, we will do both. The bet is that the sort of like, as the model performance improves, sample efficiency goes up, and therefore, like, the compound interest that you get from round over round

• 1:13:58Speaker 01

  experimentation will outweigh that, that you would get from a big, noisy but broad dataset.

• 1:14:05Speaker 03

  So do you have any concern, this is, I'm just thinking out loud here, but do you have concern that you're going to quickly sort of saturate the problems that you can solve using?

• 1:14:16Speaker 01

  Concern or hope?

• 1:14:17Speaker 03

  Or either, okay, like, concern and hope, maybe? But maybe you have these systems that you're putting in place in right now, because they're new, then there's like a lot of green field, you can go and tackle all these problems that are amenable to high-throughput experimentation, you're going to do that for a couple of years, maybe, and then all of a sudden, now, like, everything is different, and you have to, like, completely retool your, like, gazillion-dollar

• 1:14:48Speaker 01

  I mean, I hope that that is true, to be fair. Like, so I hope that we don't have to measure a binding KD again in two years. Like, if we didn't have to do that, like, I'm very pumped about that, because the model has essentially mastered binding connections.

• 1:14:58Speaker 03

  So you think that eventually you get to the point where the model knows how to do that?

• 1:15:02Speaker 01

  Let's go back to the PCI bus again. So like, what we actually want to do is to reduce the amount it takes, the time it takes to bring a new instrument on platform. So you want that to feel a lot like a USB. I don't know how old you guys, but like, when I was old, I mean, when I was a kid, you got a new device, you got the drivers on a floppy disk, you had to, you know, beat your head against the wall to get the driver to install, and two days later, your printer only kind of works. So that's kind of what, like

• 1:15:29Speaker 03

  Yeah, exactly.

• 1:15:30Speaker 02

  And if you're a Linux hardcore person, you can still live that experience today.

• 1:15:34Speaker 01

  Your audio driver still doesn't work. So that is like what it's like to bring a new instrument on platform in biology and physical sciences now, is that we're in the like, driver on a floppy disk and the manual to try to get to work. So, again, one of the things that we hope a unified platform enables is instrument onboarding time eventually goes to zero, where you have the spec from the manufacturer, the model reads it, the right APIs get abstracted, we're working with some instrument vendors to make this process easier. But I think a lot of the way that we think about modulating a system is conditioned on how we do it now. And so, again, we're hoping that a unified platform makes onboarding instrument two years from now, you know, a 30-minute exercise versus a 30-day exercise.

• 1:16:19Speaker 01

  Again, it's a hard thing to do, could be wrong, we might not be able to do it, but like, that is the, that's the future that we're pointing to, where currently, we actually can swap out existing instruments very quickly. So if we need to replace a Hamilton, you know, with a different liquid handler, that swap actually happens very quickly already. And so we do have some reasonable belief that onboarding new instruments will get faster, better, more reliable over time. And again, like, the, we don't want to be doing 2026 science in 2036. And so we hope that some of these instruments get deprecated, or the way that we're measuring things changes. Otherwise, like, lots of assumptions we and everyone else made about the rate of progress in the next decade will have been wrong.

• 1:17:00Speaker 03

  They were wrong.

• 1:17:01Speaker 02

  And we're already benefiting from, you know, the instrument vendors, right? Like, I wish the problem we have is what you're describing, that we'd run out of science to do with the instrument we have, that would up the ante for the instrument vendors. The instruments we have now are as powerful as a beamline would have been 10 years ago. We're taking measurements today that 10 years ago would have requested you to ask the federal government for a time slot at 2:00 in the morning, somewhere out there, you know, to waste a couple of nights of sleep, taking measurements at a really bright neutron or X-ray source. And today, the vendors make instruments like those that we can put next to, you know, the quantum dot or next to the protein expression. So, yeah, I wish, you know, that's an end state that is, you know, desirable, but very, very unlikely.

• 1:17:46Speaker 02

  And I'm sure there's going to be new science to be asking of the instruments we have.

• 1:17:50Speaker 00

  We've had guests who have had both of these themes of, first of all, the, none of the devices you buy are set up to do high-throughput AI science, and also that there are new scientific devices which come up every day, which just like, open up something which was impossible like five, 10 years ago.

• 1:18:06Speaker 02

  Like inline NMR, there is lots of sort of characterization, miniaturization, and sort of more resolution, more bright sources, that are just transformational, and they marry really well with a kind of automated high-throughput science we're doing.

• 1:18:22Speaker 03

  So we're moving into this facility in Cambridge, in Aewife, Massachusetts, and it's just a 3D rendering, it's 100,000 square foot space, and we will move towards AMR's autonomous mobile robots as some of the transport, and so you can see some of that there.

• 1:18:38Speaker 01

  We'll put it in the show notes.

• 1:18:39Speaker 00

  Okay, so, kind of switching topics a little bit, so you were talking about your scientific pile of 10 trillion tokens. When I hear 10 trillion, my first thought is, man, that sounds like a lot. Things like, this is 30,000 human genomes, which would cost roughly 3 million to sequence. It is roughly 1/2,000th of the, several of these large foundation models like Evo and, you know, Nuclear High Transformer and so on. So in some sense, it is a lot of data, in other sense, it's not a lot of data. And there are certain, not all tokens are the same. So I'm curious, like, what went into creating this? What were your thought processes? And then, how much actual useful information is in 10,000 tokens?

• 1:19:17Speaker 01

  10 trillion tokens, yeah. So it's tokens in the same way that we think about counting post-trade tokens from the internet or from post-training runs. So these are, again, like, RL is, the best way to think about RL is a data generation mechanism. It's a way to steer the model towards more and more valuable tokens, like, better tokens. And so these are the result of running that process across many different scientific RL environments at Lila, where the tokens are a mix of English tool calls and experimental feedback. So they're, you know, quasi-English tokens, as we've been talking about, tokenized by the tokenizer. So that's where they came from.

• 1:19:58Speaker 03

  So you're not tokenizing, like

• 1:20:00Speaker 01

  Which, so we're not nucleotizing, so it's in general. Implicitly, because if the model's asked a question about DNA, like, there are DNA tokens in there. It's not like we downloaded dbGaP, or the PDB, or SwissPro, or something like that, and tokenized at the sequence level. These are reasoning tokens, model-generated, that are experimentally verified.

• 1:20:21Speaker 00

  On top of this, you also still have your AlphaFold, your nucleotide transformer. You have all your sequencing data, which goes into this. So 10,000 tokens is

• 1:20:29Speaker 02

  10 trillion.

• 1:20:30Speaker 00

  Say 10 trillion. I'm like looking at 10 T on my laptop.

• 1:20:33Speaker 01

  Yeah. The reason why we think that level of data is important. Pre-training corpuses are usually somewhere between 15 and 30 trillion tokens. And so that's a scale at which you see these emergent things happen. And so, once you're in the trillion token regime, we feel confident that that's enough for the model to start to master and see emergent capabilities.

• 1:20:56Speaker 03

  So are you starting from scratch with your model? Or you have some open source model?

• 1:21:02Speaker 01

  Again, in the interest of being ambitiously overscoped, but not pathologically so. We have not decided to take on pre-training as well, just because the black magic that you have to do is insane. And we've been gifted something like a billion dollars worth of compute in the form of open weight models.

• 1:21:19Speaker 03

  Yes.

• 1:21:19Speaker 01

  So we start with an open weight model that has been pre-trained. And the assumption that we are making is that the model has been pre-trained on the internet in a large fraction of the scientific literature. Therefore, it's a good scientific prior over what is known and therefore a good base camp to build upon.

• 1:21:37Speaker 03

  So it's 10 trillion on top of the

• 1:21:39Speaker 01

  Added into that.

• 1:21:40Speaker 03

  Yeah.

• 1:21:40Speaker 01

  Been Yeah. And we use Pneumatron quite a bit because we have a partnership with Nvidia. And I think there's like 30 trillion tokens that go into the pre and post-trading for that model.

• 1:21:50Speaker 00

  Have you all, so in the process of these reasoning tokens, you are also creating what are arguably probably just rather useful data sets themselves. Have you thought about independently releasing some of those data sets open source? And even in the absence of the reasoning model, which may still be quite valuable to the community, but doesn't actually deteriorate, break your mode at all?

• 1:22:09Speaker 01

  So one of the things that we've developed along the way is a test suite of something like 1,000 unique scientific RL environments, where you can drop in a frontier model, you can drop in your own model, we drop in our models. So almost surely, we're going to open source a subset of that. Some of it based on data that we've generated, some of it that we have curated for the community to use. So there will be some open source version of the benchmark that we've assembled as doing part of that. And there will be probably some data, training data that goes along with that.

• 1:22:42Speaker 03

  Cool. Do you have benchmarks internally that actually operate the lab?

• 1:22:47Speaker 01

  Like essentially like a benchmark for how well does

• 1:22:51Speaker 00

  Maybe in other ways, saying, do you have experimental, automated experimental controls?

• 1:22:55Speaker 02

  Yes, I mean, we have, I think for every of the, we've put together from the beginning of the company, sort of these multidisciplinary teams to work on a specific sort of closed ended problems. And the modus operandi has always been to benchmark, train something naively from zero, calling frontier, the frontier models that everybody would go sort of use right out of the box, and our own internal. So with everything we've done, we do have an internal benchmark. Now, the domains are very specific, right? They're not as general and all encompassing as the benchmark that Andy was describing, because they are the things we really care about and the products that we want to deliver and sort of the places where we want to make a difference.

• 1:23:37Speaker 02

  But in all those places, we've typically seen that the scientifically pre-trained model that Andy is describing with access to tool calling typically demolishes, of course, anything else that we

• 1:23:50Speaker 01

  I mean, it's worth like, thinking about like what we're trying to do, how that is additive with like LLMs. If you think about like, an experimentally verified reasoning trace, how many of those do you think exist on the internet or in the pre-training corpus?

• 1:24:06Speaker 00

  Order of zero.

• 1:24:07Speaker 01

  Order of zero. I think it certainly rounds down to zero versus the next order of magnitude. So like, we have just seen incredible lift from showing the model that even if we're at like a parameter disadvantage relative to the frontier models, just showing it an experimentally verified reasoning trace, you see just immediate lift when we do that.

• 1:24:27Speaker 00

  Lila is a flagship company. Flagship is like basically one of the biotech incubators in the world. You have had something like, what, I think 30 successful IPOs? Or not you, but your parent. You know, including you yourself, were just part of Generet Biomedicine. And just had a successful IPO very recently. So, you know, Lila is very good at biotech. I would say from history, it's very much single asset, you know, traditional

• 1:24:59Speaker 01

  Flagship is very good at biotech.

• 1:25:00Speaker 00

  Flagship. Yeah, that's what I just said.

• 1:25:02Speaker 01

  Lila.

• 1:25:02Speaker 00

  Lila. Well, yeah, Flagship is very good at biotech because historically been very focused on single assets. I guess in the last few years with Generet, with, I guess Expedition, Velo, there are some branching out into more platformy things. I'm curious about one, how does Lila fit into the broader flagship ecosystem? Was there a specific reason why Lila is now? Like, why this sort of pivot from single asset into scientific reasoning? And really, what is the broader interaction? Like, in particular, you mentioned that you had a drug, or you had a CAR-T drug, which was at the level of IND.

• 1:25:45Speaker 00

  So you clearly have the ecosystem to make that into something. So I'm curious, like, well, you know, maybe, like, where is this going?

• 1:25:53Speaker 01

  Yeah, great question. Let me do a little flagship framing and then I'll sort of talk about. So we all started the same pluripotent stem cell, but there's differentiation that we all take. The traditional path for a flagship company is there's, so the history of Generet is I was an early advisor to generate a consultant over 2018. There was this idea to use machine learning for protein engineering. Me and a couple other folks at Flagship and some other external folks who came in, Dartmouth professor named Gavorgian was part of this, got seed money from Flagship to then go and spin that out. We worked on building the technology. And then usually the deal is that Flagship is the sole investor during a Series A. And then the Series B is normally the first point at which external capital comes into that.

• 1:26:38Speaker 01

  To your point, they often end up being asset-based companies. Generet has a phase three trial for monoclonal antibody to treat asthma. You know, phase one behind that to treat COPD. I think the recognition from some folks at Flagship, especially our CEO Jeff Ben Maltzen, had created, been involved in creating a lot of these companies. And he saw, he's like hiring the same team over and over and over again. You need the ML team, you need the platform team. And so I think he saw shared DNA between all these companies. And like, let's have one company that can essentially support all these different things. Year one of Lila was essentially like when O1 dropped. And so we had all these pieces in place and it just became clear

• 1:27:22Speaker 01

  that we could create a platform to support a new kind of scientific model. In the early days, we didn't know, like, how do you monetize that? What's the commercial strategy? We've gotten a lot of clarity over that over the years. But sort of the core conviction that we had two years ago was the bitter lesson is correct. Science could be an infinite token generator. If, operationally, the way that we're different from an overall flagship is outside investment came in before the Series A. Again, the lead of the Series A was not Flagship. So we do have that lineage, we do come from Boston. We do have a lot of the shared learning that what a company that has created 110 startups that I think,

• 1:28:06Speaker 01

  so they normally, so Generet was FL5657. It was actually a merge. In the early days, Lila was 9697. And so the flagship has this, you know, long history of creating companies. So we have that network and we have the learning of leaders who have created that many companies. But we were such a weird creature that we essentially went down a very different path very, very early.

• 1:28:30Speaker 00

  So why is it that when I hear, you know, you have a very promising CAR-T therapy, you said you had a DC, like, why not just, you know, partner with that out? Partner that out? Or maybe this is on the horizon or something, but?

• 1:28:41Speaker 01

  The short answer is that we are engaging in commercial partnerships around CAR-T therapies.

• 1:28:46Speaker 00

  Okay.

• 1:28:46Speaker 01

  For sure. Some of them are further development to increase some of the, or change some of the properties. So like, you know, bispecifics and things like that, going after novel indications. But we've used that one CAR-T to essentially launch several partnership programs.

• 1:29:05Speaker 00

  Okay, so it's sort of like the proof of principle, but itself was not, you know, quite exactly where you, what a drug needed to be or something.

• 1:29:12Speaker 01

  Well, so just to, it's like, to be clear, like, we could go and try and license or partner that specific thing. We found that it was better to take that and secure several partnerships around further development.

• 1:29:25Speaker 00

  I see, okay.

• 1:29:25Speaker 01

  Yeah.

• 1:29:26Speaker 03

  You're getting a, basically, you're doing some sort of code development thing.

• 1:29:31Speaker 01

  Well, this is the virtual startup idea, where a company starts a virtual startup around one of these indications and they essentially pay us revenue to further development. And again, we have these milestones and things around it.

• 1:29:46Speaker 00

  Yeah. So like, long-term, since Flagship is specifically, you know, bio and is never really branching the materials, how does that sort of weight flagship or Lila's strategy? Does that play into it at all? Or is like, at this point, you've kind of launched and sort of

• 1:30:02Speaker 02

  It's kind of one of the reasons why it sort of differentiated into something different so quickly, right? I think part of, you know, the breadth of the mission clearly was beyond biotech from day one. And the people we needed to hire came from different networks. The instruments we had to buy came from different vendors than the flagship vendors would have usually been. So I think that was part of sort of, you know, the reasons why it feels somewhat different. But it's also core to the mission, right? We cannot get this to work on a narrow field. By definition, we want to be as broad as we can possibly be, because that's where the emergent behaviors are going to come from.

• 1:30:45Speaker 01

  And I think if you looked at the composition of people who work at Lila now, it would look like categorically different than what you would expect like a median biotech company to look like. So we hire out of, or compete for and sometimes when, against people who are considering frontier lab offers. We have a heavy software engineering and tech presence. The amount that we spend on GPUs would be atypical for a biotech, I will say. I think that, you know, if we called ourselves a biopharma, we probably would have a top three GPU cluster in the world. It's true that that's part of our DNA, but we've been intentional about trying to make decisions that put us on what we think is the most promising trajectory for us. So this isn't just like kids rebelling against their parents or something.

• 1:31:29Speaker 01

  We think that like, the thesis is right and it points towards a very look valuable, but also important company for not just biotech, but for materials and chemistry.

• 1:31:39Speaker 00

  Okay, that brings me to what I think is my last question. What's harder, materials or biology?

• 1:31:47Speaker 02

  They're actually very different. It's funny, right? They're they're

• 1:31:49Speaker 01

  I feel like we're about to do the Spiderman meme and like

• 1:31:54Speaker 02

  Well, I was around for the first merry-go-round of AI for a small molecule drug discovery. Well, I mean, the atom-wise, right, the generates, you know, the in C T R O S. So I think the hardest is the thing that is the small molecule has all the difficulties of chemistry of knowing reasoning over synthesis. And then it has all the difficulties of reasoning about biology and there's effects on

• 1:32:19Speaker 00

  The counterpoint being that we have so many tricks in our toolkit, which you can borrow from biology, right? So it's harder, but you also have

• 1:32:26Speaker 01

  Well, I think materials are harder. So they have the benefit of like great simulators like that we don't have in bio. Well, I think materials are harder. So they have the benefit of like great simulators like that we don't have in bio.

• 1:32:40Speaker 01

  Like in material science, you don't have the like mature, high throughput automation that you have in biology. For me, materials as a subject is interesting because there's not a unifying principle like the central dogma. Like materials means lots of different things. Like it means like, I actually still don't quite understand the unifying principle. When we say material science, like what exactly? That means. And then the commercial dynamics are completely different. Like, like again, with CAR T, we know if we wanted to like how to monetize that directly with material, there's this supply chain, there are devices. The testing that you do in the lab is only partially predictive of like the lifetime of how that material will be used. And the math is harder.

• 1:33:19Speaker 00

  I mean, like in terms of supply chains, still matter for both. You know, maybe you replace clinical trials with some, you know, product validation and verification, I guess qualification is the term. So like, there are direct analogies and there are hard parts for both of them.

• 1:33:35Speaker 02

  The economics are very different. Like if you pass a clinical trial,

• 1:33:39Speaker 01

  You make money.

• 1:33:40Speaker 02

  You're gonna like that thing is valuable. And kind of how much it costs to make it is very rarely the blocking element. It can be for

• 1:33:47Speaker 01

  It's much easier to underwrite an asset in biology than it is in immateriality.

• 1:33:52Speaker 02

  Do you guys know the name of a company that makes a superconductors? You know, this always come up. Are you guys doing superconductors? Yeah, we care about magnets. We care about superconductors. They're really cool science. Do you folks know the name of a company that makes superconductors? Nobody knows. Like this thing is super

• 1:34:06Speaker 01

  I know that they're using MRI.

• 1:34:08Speaker 02

  Exactly.

• 1:34:08Speaker 01

  That's the only commercial application.

• 1:34:09Speaker 02

  But it turns out, right, like these things, when you succeed, you kind of are, when you make a cool material that does something, you're kind of a nameless company that makes its thing and it's successful and has good cash flows. But you don't get to break sort of, you know, everybody knows a big pharma, but other than, you know, you've got your three ends, right? This sort of

• 1:34:30Speaker 01

  Yeah. And like, most of the big material companies are behind closed doors. Like, most of commercial engagements look like getting them to tell you what the important problem is. And there's like less of an open innovation ecosystem. There's a couple of things in materials that are obviously recognized to be valuable, but like, it's just, I think, very different than than life sciences.

• 1:34:48Speaker 02

  And maybe one of the last things that we haven't touched upon a lot, and I want to flag out, I think chemistry and especially materials, government-sponsored research is a big driver. So in the same way that, you know, the government doesn't feel they need to do drug discovery other than funding NIH for early-stage, open science, hypothesis-driven science.

• 1:35:09Speaker 02

  You know, the government and national security drive materials innovations in ways that are unique. And you see this in the way we engage with the British government. We have partnerships. We work with the US government. We have awards. We participate in sort of developing materials and technologies, which is a different part of the ecosystem that drives innovation. That's also different.

• 1:35:32Speaker 01

  Yeah, definitely.

• 1:35:33Speaker 03

  Are you guys working in mission Genesis?

• 1:35:37Speaker 01

  We were one of the named partners. We've had an ongoing relationship with a lot of the national labs, and so we have been working on that.

• 1:35:43Speaker 02

  We did send 25 Genesis lighthouse proposals last week. So yeah.

• 1:35:50Speaker 03

  He left academia thinking great writing was behind him, only to have to write

• 1:35:57Speaker 02

  They expect about 20,000.

• 1:35:59Speaker 03

  The question that we like to ask all of our guests is if you could remove a bottleneck in your domain by, and you can define domain, by fiat, what would that bottleneck be?

• 1:36:15Speaker 02

  To me, I'm gonna go to old timey Rafa that was doing physics-based simulation. I would say the Sim2Real. I mean, Sim2Real for the people that come from the physics-based world, I think the Sim2Real having like an What I mean, so these people have typically meant it in the context of robotics, where your virtual simulations in 3D spaces kind of allow you to train robots that will move in physical spaces. But there is a gap and they call it the Sim2Real gap. For us in physics-based simulations is that, you know, we do molecular simulations of gooey stuff. We do electronic structure simulations of hard stuff. And they're okay, but they're not predictive enough. So this is the reason why, you know,

• 1:36:57Speaker 02

  if I, if it wasn't for that, maybe we would have had to make a self-driving lab for materials because we would have been able to just predict. So I think we know there's physics, but it doesn't quite go the way to being predictive. Meaning that the models that we train on physics cannot possibly close the gap either because they're still missing this, right? They're training on approximations that are just not good enough. So I think the thing we've been chasing for a decade in AI for materials has been sort of, if we train on computational data, can we answer real-world experimental questions? And that would have been the place where if I get to go also go back in time, in addition to taking the bottleneck out, it would be the accuracy of the underlying simulation that we've been training on all the time.

• 1:37:42Speaker 00

  So this is sort of like Heather Kulik said, there is no alpha fold for materials.

• 1:37:47Speaker 02

  Well, the funny thing is Alpha Fold was trained on experiments. So it's a different, I mean, that's a funny. She and I would both come from doing physics-based simulations. And the fact that she called out something that had no simulations in it whatsoever, it's kind of admitting the same underlying issue, which is like all this, you know, Meta has produced tens of millions, hundreds of millions of training data points. But they're all virtual simulations that just don't carry enough water for the thing we actually want to do.

• 1:38:13Speaker 01

  This is gonna be like a boring and obvious one, but like there's a metric that you use to track how efficient your training runs are. It's called mean flop utilization or MFU. So the GPU comes with an advertised like peak flop throughput, which is under the best situation, doing a calculation that you don't actually care about, how many floating point operations can you do per unit time? MFU is always a very small fraction of peak theoretical flops. And for reinforcement learning, it's always somewhere like around five to like 6%. So said differently, that means that we're getting like 5% of the actual GPU computing power that we're paying for.

• 1:38:54Speaker 01

  So if I could buy Fiat, wave a wand and make our stack perform at like 100% mean flop utilization, I would do that because we would, one, get to the answer faster, but then also be able to buy fewer GPUs and redeploy that capital to the to the lab or something like that.

• 1:39:10Speaker 03

  That's interesting though, because your rollouts, aren't they constrained by the lab?

• 1:39:14Speaker 01

  They are, but when we train a big model, like all that data, so there's our old training pipelines are very complicated. So like, one, one way to think about how you would do this at scale is just to have the model doing rollouts left and right, waiting for enough trajectories to pile up and then back propagating that into the model. A different way to do that would be to factorize that, have a bunch of expert models that are trained in parallel that are either generating data or being trained themselves. And then you distill that back into the central model. And second way is the most efficient, the more efficient way to do it. So because all those things are happening at different time scale. And so it's that big. When you have the 10 trillion tokens and you want to push them through the model as efficiently as possible,

• 1:39:56Speaker 01

  you're still going to be doing some reinforcement learning on top of that. So like, if you, if we could get all the flops that we're paying for, I would buy Fiat, declare that.

• 1:40:07Speaker 00

  Cool. Well, yeah, before we end, is there anything you want to leave the audience with?

• 1:40:13Speaker 01

  Let me say like why we're here. So we have an office in San Francisco now. It's 181 Fremont Street in downtown San Francisco. There's currently 20, 20-ish, 30-ish people who sit there, but we are looking to expand that aggressively. We're looking to pull from sort of all areas of the stack. So both like post-training, obviously aggressively hiring for that. Folks have been working in like domain AI, like life sciences and material sciences. We're also hiring for that. No wet lab here currently, so it's all, it's all computational work. But if any of this stuff that people have heard about today sounds interesting, feel free to shoot either me or Rafa a message if that sounds interesting.

• 1:40:51Speaker 03

  Thank you for being here. It's been really, really fascinating conversation. Appreciate it.

• 1:40:55Speaker 02

  Thank you for having us.