Ep 10: Insitro CEO Daphne Koller on Using ML to Change Drug Discovery

Jacob sits down with Insitro CEO Daphne Koller to discuss founding Coursera, where and how ML can drive the most impact in drug development, and if foundation models can transform core drug discovery work and edtech.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Jacob Effron?

  I'm Jacob Effron, the host of Red Point's healthcare podcast Vital Signs and AI podcast Unsupervised Learning. And for our first cross-episode, we really have the perfect guest in Daphne Koller. Daphne was an early machine learning pioneer. She spent 18 years as a professor at Stanford before leaving to co-found the ed tech company Coursera. She then founded and runs a company in Citro that's raised over $600 million and lives at the intersection of machine learning and biology to create better medicines. Daphne was Time Magazine's 100 Most Influential People, a MacArthur Genius Fellow. She really is the perfect person to sit down and talk all things machine learning and drug discovery. We talked about where machine learning can drive the most impact in drug development, whether foundation

• 0:42Jacob Effron?

  models really do mark a change in drug discovery in ed tech. And we also hit on her fascinating story from Coursera to bio. I think you guys are really going to enjoy this episode. Daphne, thanks so much for coming on.

• 0:52Speaker 01

  Thank you for inviting me, Jacob.

• 0:53Jacob Effron?

  I mean, you have this really interesting background, I guess. You know, obviously, you were a Stanford professor, a MacArthur Fellow. You ended up leaving Stanford to kind of co-found Coursera. You had kind of done some biology work earlier in your career and then came back to it through Calico and Citro. I'd love to kind of just hear the story and the journey. You've had more, you know, probably four chapters of a career that are more interesting than most people's one.

• 1:16Speaker 01

  Thank you. Yes, it's been a bit of a convoluted career. I find it hard to explain to people exactly how it came about. But I think my early path was fairly straightforward. I came to Stanford as a PhD student in 1989. I was actually one of the first people to become interested in the more probabilistic side of machine learning, of artificial intelligence. At a time when artificial intelligence as a discipline was actually not very popular. It was one of those recurring AI winters where the field had made extravagant promises and hadn't delivered. And so I was working in a space where it was a little bit, shall

• 1:59Speaker 01

  we say, fringe. And then within that, the probabilistic modeling work that I was doing was even fringe within that. It wasn't really embraced by AI until much later. So I used to tell people. People that I didn't start out doing AI, AI became what I was doing. I graduated from Stanford in nineteen ninety-three, went and did my postdoc at Berkeley and came back to Stanford as the first machine learning hire when a lot of the people who were there at the time were much more traditional AI researchers, focused more on logical reasoning, rule-based systems, the things that had really built the field. And I was a bit of an outlier. And so there I was at Stanford, working on machine learning and the data sets that were available to machine learning researchers at the time,

• 2:44Speaker 01

  which is not very interesting. It was things like spam filtering on relatively small numbers of email, tagged email messages, or classifying articles into twenty news groups. And I felt like this just didn't inspire me. And so this was towards the late nineties, when the first larger biological data sets were starting to be collected. They're tiny by today's standards, but they were, but they had more interesting structure. Genes were doing things to other genes. And actually, the first project that I worked on was in a topic that is very timely, which is epidemiological transmission of disease. Specifically, at that point, it was tuberculosis. And now, of course, we can do that much better with much larger data sets.

• 3:29Speaker 01

  But again, it had a much more interesting structure. And so that led me to start working on biology, initially more in just, it was technically more interesting, but then I became interested in the field in its own right and ended up having a bifurcated existence at Stanford, where half my lab continued to do core machine learning research published in traditional computer science venues. And the other half ended up doing biology and medicine and such, published in Nature and Cell and Science. And what was interesting is that my computer science colleagues, many of them even today, didn't realize that I did biology. And my biology colleagues had no idea I was in the computer science department. And so it was kind of this really weird existence,

• 4:12Speaker 01

  which lasted for a number of years, until the late, until the early teens. And at that point, I developed an increasing sense of urgency to make a big difference in the world directly and had tried to take some of the work that we've done on, you know, cancer diagnostics using machine learning and histopathology data and such and get someone to use it to the benefit of patients and found it very difficult to make an impact from within academia. And so when the work that I'd been pursuing as more of a passion project unrelated to my, to my research agenda, which was always this deep commitment to education and teaching, when that erupted in late 2011 with the first three Stanford massive open online courses,

• 4:56Speaker 01

  it was like, whoa, this is a really big opportunity to make an impact. And sure, I can go back to the lab and write some more papers and hope someone does something with them, or I can embrace it and put my research on hold and actually try and bring this to fruition. And so I chose the latter and it was always intended at the beginning to be a two-year leave of absence. My, my vesting schedule was based on two years and a long tail of, you know, the usual day a week that Stanford professors are allowed to do. And so that was the intention. And then when I got started in my really first industry job ever, because I'd had a career long with brilliant academics, I was like, wow, we're really making a big

• 5:41Speaker 01

  difference here. And it wasn't done when my two years leave of absence came to an end. And so when Stanford said, okay, you got to make a decision, you either come back or you resign, I ended up doing an insane thing, which is resigning a, a now chair at Stanford. My mother thought I was nuts because who on earth resigns a faculty position at the world's top computer science department? But I did it anyway and they had Coursera to really build an amazing company. We touched the lives of so many people in such impactful ways. And I am just incredibly proud of what we were able to build there. And so that was kind of chapter number two. And then chapter number three came in two thousand and sixteen.

• 6:26Speaker 01

  And now, if you look at the timeline, I left Stanford in late two thousand and eleven. The machine learning revolution started in two thousand and twelve. Um, I missed that whole beginning. And I was just so deep in the weeds of, you know, signing up universities, putting content online, you know, trying to get learners to be more successful, that I kind of like barely paying attention. Um, and then in two thousand and sixteen, Coursera was on a good trajectory. And I picked my head up over the trenches and said, whoa, machine learning is really transforming the world, but it's really not having much of an impact in the life sciences. And part of the reason for that, I felt at the time, actually, there were two reasons I felt at the time.

• 7:07Speaker 01

  One is that the data sets that had been larger than machine learning data sets at the beginning were now considerably smaller than what was available via, say, ImageNet or other machine learning data sets. Um, but at the same time, I felt like there were opportunities to create data using technologies that hadn't existed before. And the other reason why I felt there wasn't as much of an impact is that there weren't very many people who spoke both languages. Um, and I was one of the few people who had had the opportunity to really learn both languages in a deeply meaningful way. And I felt like I could, if I could really build a bridge between those communities and the kind of bridge that chasm.

• 7:51Speaker 01

  And so that was when I left Coursera and decided that I could make an even bigger impact by really building an effort that bridged those two worlds. Um, and so at the beginning, I didn't quite know what I wanted to do. I'd been out of science for five years at that point. So I went and asked a number of people whose opinion I respected. What do you think a person like myself can contribute? And one of the people I spoke to was Art Levinson, who was a legendary, the legendary Genentech CEO who had brought the company to greatness, had been on the boards of both Google and Apple. So I felt who, who would be better positioned? To give me some advice on how to bring the worlds together.

• 8:32Speaker 01

  And he was very unabashedly straightforward and said, I think you should come to Calico. And the opportunity to work with someone of that of that stature, that, you know, level of leadership was one that I couldn't pass up. And so I didn't know very much about Calico, but there I went. And and I learned a ton about biology, about drug discovery, which I had known nothing about, and it was incredibly formative experience. But I realized relatively quickly that this really wasn't what I personally wanted to do. I wanted to build a platform company for doing drug discovery differently. And so that was when I made the decision in early 2018

• 9:15Speaker 01

  to kind of launch off on my own and build a completely different kind of company.

• 9:18Jacob Effron?

  You know, I feel like you could apply AI to so many different parts of, you know, what what pharma companies do, right? I mean, obviously, there's the biology discovery side. There's molecular design, there's trials. You know, I think in many ways, the biology discovery side is probably the hardest problem you could you could tackle in the space. I guess you've never shied away from from those types of problems. But how did you kind of think through where to where to start?

• 9:38Speaker 01

  Yeah, you know, it's a great question. And I will tell you that actually at Insitro, while we started out with a biology discovery problem, we are building an arc that goes all the way through. And we can touch on that later in the conversation. But I felt like if you look at the challenges that the industry has faced and the record and why things take so long and cost so much. It's because we make some wrong decisions, often at the beginning, that don't manifest until much, much later. So most drugs that fail, fail in phase two. Some of them fail in phase three, typically because people squinted at the data in phase two and pushed forward when they probably shouldn't have.

• 10:21Speaker 01

  And the reason why most of those drugs failed is because they're simply going after the wrong mechanism. It's not because the pharma industry can't make molecules. They're actually pretty good at making molecules. And certainly, one can accelerate that and make new kinds of molecules that people quite hadn't quite figured out how to make before. But that really wasn't the crux of the problem. The crux of the problem was we don't understand biology. We're kind of You know, feeling our way in the dark and latching on to things that just don't make sense biologically. And so I felt like if you were really going to make a difference, that was the place to make a difference.

• 10:58Jacob Effron?

  And I'd love if you could kind of contextualize for our listeners, like how the Nsitra approach compares to maybe the traditional pharma approach and then obviously, you know, other kind of AI drug discovery approaches like a recursion or something.

• 11:10Speaker 01

  Yeah, so these are actually quite different questions. And so let me start with what the Nsitra approach is, and then we can do a little bit of comparison to other entities in the space. So what we do is we try and take as data-driven and unbiased a perspective as possible. We feel like right now we're kind of feeling around with very, with very sort of hypothesis-driven, intuitive-driven direction, saying, oh, this looks like a good direction, let me explore here for a bit. Whereas what we ought to be doing is kind of rising up, collecting a whole slew of data, sort of almost from a much higher plane.

• 11:52Speaker 01

  And then having surveyed human biology, we can start to identify patterns and say, okay, this is a path that will take us in the right direction. So this is kind of a, if you will, an analogy. And what it translates to in practice is that as we identify certain areas of biology that we think are promising and where we feel like the technology stacks that we're building has a, has a meaningful value to contribute, we collect lots and lots and lots of data about that particular biology. And the data that we collect comes in two flavors. We print data in our internal automated wet lab capability where we generate cells

• 12:35Speaker 01

  that have different genetic backgrounds with different burdens of disease. We differentiate them into disease-relevant cell types. So if we're working in neuronal diseases, we don't use cancer lines. We use stem cells differentiated into neurons. If we're working in liver disease, we differentiate them into hepatocytes. And then we measure them in ways that don't come in with preconceptions about measuring this is what matters to disease. We try and take as broad-based a perspective as possible. On the other side, we also bring in data from humans because ultimately, while cells are a wonderful and very intervenable model system. What we care about is curing not cells but humans.

• 13:18Speaker 01

  And so we also have via partnerships, data sets that we can collect that also attempt to measure human pathophysiology in a very, very holistic way. Whether it's histopathology imaging, a modality that I worked with all the way back when I was at Stanford, MRI images, various types of omic biomarkers in blood or in tissue. And you take all these data and you feed it into the machine and you say, okay, what patterns do you see? What is the language of biology? So if you think about the large language models that are very much the talk of the day, we're building medium language models where the language is the language of cells and the language of tissues.

• 14:01Speaker 01

  It's not human language, it's biology language. And with that kind of way of thinking about the world, we can ask, okay, in this language, what does healthy look like? Where does sick look like? And then what is a set of interventions that might move us from one to the other? And for interventions, we look to genetics because genetics is one of the very few sources of ground truth that we have about causality in biology where causality is very hard to come by. So that's kind of the approach that we take. And what's important is that as we take this broad landscape survey, a bunch of hypotheses emerge. And then rather than kind of closing your eyes and picking one, we say, we design platforms,

• 14:46Speaker 01

  industrialized platforms that progress those from one step to the next. The ones that don't seem to be panning out fall by the wayside and the other ones proceed through in a wave. And that allows us to interrogate not one target at a time, but really think about what is the best way of intervening in a particular biology. So much more industrialized approach to discovery, an engineered approach, if you will, to discovery.

• 15:12Speaker 01

  Which is really not something that exists in very many other places. So that's kind of the vision. And I did mention to you earlier, because I think it's an important component also of the comparison, that biology discovery, although it's been the place where we've spent the most of our effort, is only the first part of our journey. We also have an in-house capability for turning targets into drugs, into small molecule drugs. That too uses industrialized processes having to do with, you know, hundreds of millions, if not billions of compounds that are all pooled together, attached to little DNA tags,

• 15:53Speaker 01

  so you could do massive chemistry experiments in a single pool, if you will, and come up with a ton of chemistry data as well. And then maybe even more importantly, you mentioned clinical development as being the third says, the work that we do with human clinical data really helps present the path forward to translating our findings into the clinic specifically by telling us, based on data that are acquired from patients, which are the patients that are most likely to respond to a particular therapeutic intervention. And that, I think, is one of the big, I would say, this is, if you will, in the industry

• 16:34Speaker 01

  today, that in most therapeutic areas, cancer being somewhat of an exception, although still could be pushed forward, but certainly in other areas as well, drugs are given to an all-comers patient population, and you have 20% that respond, 80% that don't. The ones that don't are subject to unnecessary toxicity, also not having an opportunity to take another drug that might be helpful to them, because we just don't know. We don't know who should be getting what drug. And I think that's, we made such a huge step forward in oncology when we stopped treating breast cancer as one disease, and we said, you know, there's BRCA breast cancer, and then there's HER2 positive breast cancer, and so on and so forth. And it created a menu of treatments that are really fit for purpose, and a much, much

• 17:19Speaker 01

  greater therapeutic index, as it's called, like the delta between the efficacy and the toxicity is much, much larger for these targeted treatments. And so we, as we do our early biology work, it comes together with a biomarker. Um, so that's kind of the spectrum of who we are. Now, when you ask yourself, okay, so how do we compare to other companies? Um, when I think about other smaller biotech companies, um, most of them end up being in one or the other of these, uh, of these three steps of biology discovery, um, you know, targeted drug and then, um, and then going in and doing clinical development.

• 18:03Speaker 01

  Uh, the vast majority are in that middle step, target to drug. There's a good reason for that. It's well-defined and easily measurable, so you know when you've succeeded. Um, there's a few that are kind of playing an assistive role in clinical development by doing better biomarkers from whatever wearables or, or imaging or such. Um, there are very few companies that span the gamut and we're one of them. Um, so I think that's, uh, an important, um, differentiator. I think the, a second important differentiator in, uh, that we bring together both kinds of data. There is companies that do discovery using primarily cell-based functional genomic approaches.

• 18:48Speaker 01

  There's companies that do discovery based, uh, primarily on human clinical data and looking mostly for genetic or genomic correlates. There are very few companies that really bring the two together and we think that the, they really synergize with each other in a way where the, uh, whole is considerably larger than the sum of the parts. And then I would say the third differentiator is just the scale of automation and data creation that we've, um, that we've put together, which I don't want to say is, is unique. I mean, certainly you mentioned recursion, it's probably the one that, uh, has most. Has made the biggest investment into the creation of data at scale. They started before us and they certainly have done an incredible job of putting in

• 19:33Speaker 01

  place that type of automated data creation. So we are in that same bucket, if you will, but they only do the cell-based stuff and focus on the, you know, early discovery. Now they're starting to move into molecular generation, whereas we're kind of, as I said, by bringing the human data component into this, we think it gives us the ability to really bridge the gap between cell biology and translational medicine.

• 20:03Jacob Effron?

  I think one thing you said that really resonated was, you know, a lot of the techniques you're doing kind of in the understanding biology side of this much more detailed segmentation and phenotyping of patients is obviously incredibly relevant to clinical care down the line. And, you know, to your point in oncology, there's some, you know, there's early like, you know, kind of segmentation of patients based on, you know, some basic genetic tests to kind of determine treatments. But I'm wondering how you think, you know, you guys obviously do such detailed levels of kind of phenotyping and segmentation. Do you think kind of similar methods, you know, ultimately are inserted at the point of care going forward to, you know, kind of find the right subpopulations for these treatments?

• 20:38Speaker 01

  You know, that's a great question. I think the answer to that is undoubtedly yes, and one could take that same set of methods and really deploy them in clinical care. I think that honestly, as we derive new treatments that have a biomarker attached to them, it will be a regulatory requirement to have that same biomarker deployed as part of the decision to provide that medicine to a patient because it was approved for that population. So I think eventually that will happen and certainly as we've talked about, again, the precision oncology as an example, the drug Herceptin, which is arguably the first precision

• 21:19Speaker 01

  oncology drug ever made and maybe the first precision drug ever made, was approved for patients that are HER2 positive and for good reason. We just had, we were very privileged, we had a company-wide get-together last week and Sue Desmond-Hellman from, who was the president of product at Genentech, who developed Herceptin, came and spoke to the company. It was a wonderful, truly inspiring experience. And she spoke about The challenges of developing that kind of therapeutic and the skepticism that existed at the time and also about how without having that very well specified patient population, the drug probably would never have been approved because it would have needed a clinical trial

• 22:04Speaker 01

  like 10,000 people to see statistical significance and maybe not even then because the adverse side effects may have outweighed the benefit to the her positive population. And so I think it's really important for us as a society to move towards precision therapeutics in other areas as well. Now, whether we ourselves are going to go into the pure play diagnostics business, I don't think that's on our trajectory right now. We have a lot of other things that we are really focused on and as a company, one needs to focus on a business model, but certainly we would be very much thinking of as we develop

• 22:48Speaker 01

  either proprietary or partnered programs, what is the right biomarker by which we can select the patients who are truly going to benefit the most.

• 22:56Jacob Effron?

  Yeah, no, it makes a ton of sense. And I mean, you know, I think to that point of where you guys have focused, I mean, it seems like obviously, you know, to start, there was a massive effort on, you know, owning your own data generation and that was obviously so core to the company. You know, I'm curious as you kind of put that together, you know, how you've thought about like, you know, obviously you're combining it with real world and animal testing data and you have, you know, some, you have some other data sets, how you've kind of thought about what you do need to generate yourself and then also, you know, you're obviously, you know, an ML guru, you know, how you've kind of architected those systems to kind of best feed the underlying models?

• 23:30Speaker 01

  Yeah, no, I think those are great questions and I think first it was clear to me from the very beginning that, that creating data that's fit for purpose for machine learning is going to be a critical differentiator for success. I think that having worked with a whole bunch of data sets in my Stanford days that were downloaded from the supplementary materials of this or that paper. And realize that the data were just not geared in a way that allows machine learning to be done well. There's just so much, you know, biases that are introduced via confounders that are not appropriately corrected for, sometimes not even appropriately documented in in the write-up about this,

• 24:15Speaker 01

  that it's very difficult sometimes to ever tell the signal from those confounders because they're just inextricably intertwined because of how the experiment was done. That was very clear to me that that was going to be a significant factor for success. What I don't think I maybe fully appreciated at the time, maybe because I'm not very financially minded, is that as the industry evolved and there would be more and more companies jumping into this space, having a proprietary data source also is a significant competitive mode that other people have a hard time replicating. So that's, I think, part of the answer to your question of how do you make data that's fit for purpose for machine learning. One thing I will say is that,

• 25:01Speaker 01

  boy, biological systems just continue to throw curveballs at you. I mean, you know, you think you've designed the experiment with all of the right controls and all the right ways to eliminate confounders and randomize around them. So at least they're not systematic and can skew your result. And then they throw yet another curveball at you. It's like, you didn't correct for this. And so we spent a lot of time on automation, a lot of a lot of effort on that, on QC, on data pipelines and data recording, as well as developing machine learning methods that when the inevitable confounders do come up because you haven't corrected for everything, because you could never correct for everything

• 25:43Speaker 01

  in advance, you can at least correct for it post hoc. And so that's been a very significant. Lift and I would say that's a two-edged sword, if you will, because while it's been incredible and sometimes very painful amount of work, it also is a very significant competitive barrier.

• 26:00Jacob Effron?

  No, totally. That makes a ton of sense. I imagine you've learned a ton in just the time you've been building in CTR around these data sets. I mean, I'm curious if you could go back to the beginning and tell yourself something, like are there any kind of mistakes that you made at the beginning that you wish you guys had done differently?

• 26:14Speaker 01

  Oh, so, so many. We got into the work on human data a little bit through the back door. I think we started out from the premise that there's a whole bunch of incredibly smart statistical geneticists out there that are working to de-convolute the genetic architecture of diseases. We could trust them to publish their results because they typically do. That's a norm in that community. And then we could take their findings and just engineer those into our cellular systems and use our cell-based systems to do the primary discovery work. When we did our partnership with Gilead and we were faced with a disease that had relatively poorly understood genetic architecture.

• 26:59Speaker 01

  It was a disease called NAFLD or NAS, non-alcoholic fatty liver disease. And there wasn't just a lot of genetic studies that had been done because it's a severely under-diagnosed disease. It requires a liver biopsy to be ascertained. And then that obviously is not something people are very excited about. And so there's not a lot of patients with ascertained NAFLD. And so people are working on non-invasive testing, by the way, but it wasn't available at the time. And so, and so we ended up having to de-convolute the genetic architecture of that disease. And we started out with this histopathology data that had been collected as part of the Gilead clinical trials.

• 27:40Speaker 01

  And we began with what we thought was a great set of annotations from their clinician, who had manually annotated every single slide, a single pathologist, so that it was wonderful, because there wasn't any inter-pathologist variability, which is huge, by the way. And we realized very quickly just how low of a resolution of a dataset that was, and how much information was getting left on the table. And so that really is what inspired us to look at high-content data from human patients, whether it's histopathology, which is a huge core focus for us right now, especially in oncology, where it's collected from every solid tumor patient, pretty much anywhere around the world.

• 28:21Speaker 01

  But also other high-content data modality, like MRI and omics and things like that, that are currently annotated by clinicians, who extract, I would say, a reasonable amount of signal from the data, but there is a lot of subjectivity, a lot of variability, and a tremendous amount of information that is just left sitting there, because people just don't know how to look for it and what to call it. And so I think that that was something that, you know, probably we could have realized in 2018, rather than in 2020. That would have given us another two years of head start on that.

• 28:58Speaker 01

  I think that that's one big lesson learned. I think the variability of biological systems, and being even 10 times more careful about things like randomizing place, which is, by the way, really hard to do. Liquid handling systems that are the commodity, liquid handling systems do not lend themselves well to sophisticated programming in many cases. It's only now that people are developing Python-based systems for some of the, Python-based interfaces, some of those systems. I would have placed an even bigger premium on that than I did at the beginning, because, again, these things just keep blindsiding you with

• 29:40Speaker 01

  You know, nasty surprises about how sensitive cells are to very specific things.

• 29:44Jacob Effron?

  No, it's interesting. And obviously, you know, I think it's still kind of difficult to, to, you know, correctly generate some of these data sets. And but I am struck by even hearing you speak to some of the real world data that was available. It does feel like, you know, especially relative to when you started, you know, you were starting earlier in your career looking at biology and saying, hey, the data sets aren't quite there yet. Or interesting enough, you know, it's just, it's unbelievable how, you know, it feels like each decade, there's just so much more data. There's new techniques available to gather that data. I'm curious, as you look, look forward and think about, you know, data that we don't have yet, but like, would get you excited, like in a decade from now, when, you know, the companies of the future are working on that and you guys are focused on it. Like, what gets you excited about kind of data sets on the come?

• 30:23Speaker 01

  So I'm going to give an example from each of the two sides of the data modalities that we work with. One is on the cell based side, and one is on the what we call cell ml and clinal. So on the cell ml side, right now, the focus that we have is primarily on phenotypes that you can collect in single cells. That so things that are largely what's called cell autonomous. Sometimes we put those cells in with other cells because turns out that neurons are much happier when they're with astrocytes. And so the phenotypes are still neuronal, but the astrocytes make them happy and grow faster. And so that's great. But when you look at some of the more complex interactions and also getting cells

• 31:07Speaker 01

  to a certain level of maturation, and some cases does require that they be in more structured environments. There are obviously a lot of efforts around organoids and, you know, printing of little microorgans and organs on chips and such. And I think those are great, but they are not yet at the scale and robustness of what we would be looking for for a discovery perspective. We'd use those for validation. But boy, wouldn't it be nice to use those for discovery because they were sufficiently scalable and robust.

• 31:40Speaker 01

  Um, the even bigger side of my wish list, and it's bigger because I think it's harder. Um, because that technology on the cellular side is being developed and it's gonna come and it's gonna be here in a few years and I don't, I mean, people are working on it. Um, the other side is larger cohorts with deep phenotyping that's done longitudinally. And the reason why that's, and I, I know people, and the reason why that's hard is because if you want longitudinal data, you either have to wait a really long time or you have to go backwards and hope someone collected it already. And some of those collected it already data sets exist here and there and you kind of

• 32:21Speaker 01

  have to collect them and curate them and make sure that there's no confounders, which are so prevalent in, in, um, clinical data. But what you really want is someone, you know, 70 years ago had picked a cohort, a large cohort, and deeply phenotyped them on an ongoing basis so that today we would have all those data, um, collected over time. That doesn't exist anywhere, I know. Uh, the closest that I know of is, um, the UK Biobank where they collected a cohort of 500,000 people, you know, at this point, close to 15 years ago, they did deep phenotyping at the start.

• 33:02Speaker 01

  They're now starting to lose some recall of the cohort to do a second round of deep phenotyping for at least a subset of them. That's the closest that I know of. It's, it's amazing. I, my, really truly hats off to the creators of the UK Biobank. They're some of the most impressive people I've ever met. Um, but, you know, it's smaller than I would like. It's not, the deep phenotyping doesn't occur as often as I would like. It's Eurocentric, so it's not very diverse from a, you know, population perspective. So it's not the perfect cohort by any stretch. So that would be my dream, uh, dataset.

• 33:36Jacob Effron?

  You know, it's a really interesting answer because obviously there's always gonna be, you know, when there was some big advance, you can kind of immediately implement that on, on, on studying cells and tissues and organs in the lab. But obviously for a longitudinal dataset, uh, you make the advance and then you wait the, however long you need to track patients for, uh, and, uh, I, I guess on like the, on the, on the cell side, I'm curious, you know, obviously today, you, you guys do, I assume, a fair amount of generalization to tissues and organs from your work with kind of single stem cells. Like, how do you think about solving that problem?

• 34:05Speaker 01

  So, as I said, we put in cells with other cells and those are called co-culture systems. And I think they've been quite successful, although they are still quite finicky. And that to us would be the next obvious direction. I do think that there are some organoid systems that are starting to show sufficient scalability that in a couple of years, we might feel they're mature enough for us to really onboard them in a serious way. I will say, however, and this is one of the reasons that I'm, you know, that I was excited to start the company, even recognizing as I did that organoids were not quite where I

• 34:45Speaker 01

  needed them, is that there are so many diseases where there is a very significant autonomous component. Maybe it's not the only thing that's going on, but there's certainly enough that's happening within the confines of a single cell. Maybe you need to stimulate it a little bit because it might be getting some signals from its environment, like for example, the work that we're doing in liver disease, you need to put in some fat into the medium so that the cells can be on the McDonald's diet. And so, that's fine, but they don't necessarily need to have everything else around them. And I think that will keep us busy for quite a while in terms of addressing those diseases.

• 35:27Speaker 01

  And then, over time, new technologies will unlock in the next tranche, if you will.

• 35:31Jacob Effron?

  You know, going back to one thing you said earlier, I'm always, I'm struck by just the breadth of, of kind of, you know, the different parts of the processes you're tackling in Citro.

• 35:39Speaker 01

  Yeah.

• 35:40Jacob Effron?

  You know, one thing I'm always curious about is, is how you think about, you know, partnering with larger pharmaceutical companies, right? You've publicly announced some, some pretty interesting partnerships. How do you think kind of where it makes sense for you to go at it yourself versus where it makes sense for you to work with others?

• 35:53Speaker 01

  We take a very agnostic view of this in that we don't pre-stipulate that this is going to be ours and this is going to be somebody else's. I think we're in a position that's a little different to that of most companies because most companies, because of the way in which... You know, people identify target hypotheses. In most cases, they have one or two or maybe three things that they're really excited about. And it's very hard for them to give up one of their babies to somebody else because it's not like, it's not clear what they would have if they gave that away. We have an engine for generating these hypotheses, and that engine continues to give more and more every day. And so we can feel a lot less precious

• 36:37Speaker 01

  about, you know, let's take this particular program and partner it with someone and share in the risk and share in the reward. And that's okay, because there's more to come from, uh, that, more to come from where this came. And so, uh, I mean, there's certainly things that I would say are probably out of reach for us to take on directly. There's certain therapeutic indications where just a safety trial, like if you were doing a broad-based cardiovascular drug, just a safety study would be enormous and unbelievably cost prohibitive for a small company. So that we probably wouldn't do. Um, as we get into more precision therapeutics, where you say, well, we're going after this segment and

• 37:20Speaker 01

  it's defined by this biomarker. And so, uh, we can run this trial managably because we expect that it, you will see a signal in a smaller patient population with a large effect size. And so the trial can hopefully be certainly more constrained and maybe even shorter. Um, those would be ones that we would be much more open to taking on ourselves.

• 37:43Jacob Effron?

  And one thing I'd be really curious to get your thoughts on, just given, have you been at the forefront of this field for so long? Um, obviously, I feel like these multi-billion parameter foundation models are all the rage these days. And, you know, there's folks that are applying diffusion models to, to, you know, uh, to pharma research now. And, and one thing I, you know, from the outside have been, yeah, I've been wondering is, you know, it, it's kind of unclear to what extent are, you know, is this progress outside of the drug discovery space relevant for drug discovery? Or is it like, you know, hey, the same approaches that, you know, made sense pre-GPT-3, pre kind of stable diffusion are still like the ones that make the most sense?

• 38:16Speaker 01

  You know, it's a great question. I don't think anyone knows the answer until we try it. There's certainly pieces of the large language models that are potentially hugely enabling, um, maybe in less interesting ways, but still really important ways to the work that's done in, uh, in biopharma. So for example, as you get Whatever, one or more target hypotheses that come off whatever discovery approach you're using, you need to go and read a whole bunch of literature about this target to figure out what's known, what's been tried, because, you know, your own experiment is never the 100% definitive, there's always some stuff that you might learn from reading the literature. Reading, summarizing, and kind of distilling that into something that's cogent is really

• 39:01Speaker 01

  very time consuming. And so that's, is that sexy? I don't know, but it's certainly very, very useful. Now, when you think about, and I could give you lots of other examples as well, of that type. I think the question is, does it transform core discovery work? I think that's very much an open question, while it can certainly hypothesize that, oh, we could use... And also depends on how broadly you define large language models. Like, are large, I mean, if you train a model to chemistry data, is that a large, is it inspired by large language model, or is it a large language model? I mean, I don't know, it depends how you draw the boundaries. But I think generative chemistry is going to be a big thing over time, especially as

• 39:45Speaker 01

  we collect more and more chemistry data. I think that what we're doing with the language of biology is really very much in the same vein. It doesn't sit on top of the large language models, but it certainly uses the same kinds of ideas of self-supervised learning to a proxy task and collecting very large amounts of data so you can actually learn a representation before you apply it towards any particular prediction problem. All of that shares that same kind of philosophical idea tree, if you will. Whether diffusion models are the right way to interrogate MRI data, I don't think anyone knows until they try it. Maybe.

• 40:26Jacob Effron?

  Yeah, no, it makes, it makes a lot of, I mean, there'll be a ton of experiments with this stuff, and, you know, I think it'll be interesting, obviously, it feels like on the, on the large language model side, it ended up, you know, there being breakthroughs that allowed you to just, you know, that then allowed massive scale and compute to really have an impact, right? So you could just throw a ton of unlabeled data, and I think it seems, at least from the outside, that on the biology side, the bottleneck still is the data, right? And that even having and throwing more parameters in these models and more compute is only so helpful with the, with the data sets we have.

• 40:51Speaker 01

  But I think philosophically, the idea that you don't need to have labels in order to learn a representation, that once you learn a representation... A relatively limited amount of label data can go a long way. That has been very much part of our vision from the very beginning.

• 41:08Jacob Effron?

  Super interesting. I mean, on the LLM vein, I would be remiss not to ask you about your old gig at Coursera. I just think it's so fascinating. Obviously, I think there's so many different ways that LLMs and these latest models can be used on the education side. I'm curious if you were still in that space, how you'd be thinking about these models and what you might be building.

• 41:30Speaker 01

  Well, so actually, I still kind of am in that space, although not at Coursera. In the midst of the pandemic, my husband and I, when our two daughters, who were both then in high school, were shoved into Zoom school and we saw what that experience looked like, we realized that the teaching and learning experience that one gets online is just not a very engaging one. And so that led us to actually found a company, which is called Engagelee, which is in the education space. It's very different from Coursera. It's not kind of like a large play content platform. It's really much more about the engagement in a teaching and learning environment and actually engagement more broadly, because

• 42:13Speaker 01

  you can also think of it as a collaboration platform. And it has human elements, such as, you know, interacting with other students in meaningful ways that, you know, these large online platforms just don't do, interacting with the instructor in meaningful ways, providing an instructor with data on student success metrics and student engagement so that they can targeted interventions on the students that need the help versus having just a blank sea of faces on Zoom or, for that matter, honestly, in a large auditorium is not any better. And like knowing where to pay attention and try and bring students

• 42:52Speaker 01

  back into a successful outcome. So that, I think, is the core vision behind that platform. And now with the large language models, you can start to think about an AI-enabled form of engagement. And each of those things that I mentioned lends itself to an AI enhancement, whether it's As students engage with the material, you could create sort of AI-generated tutors or AI-enabled tutors that can help answer questions and point them towards course material. You could help the instructor create better course materials, not having it be as much work to do course design.

• 43:36Speaker 01

  You could help an instructor create meaningful personalized interventions with students in a way that still has that instructor voice, but you don't have to write a full, you know, half-page email personalized to every student from scratch, which is just time-prohibitive in large classes. So I think there's lots of ways in which that could be woven into the education technology stack. And that is certainly the future of education, especially in the online education.

• 44:04Jacob Effron?

  Well, you know, we always like to end our interviews with kind of a standard closing set of questions in kind of a quick-fire round. And so I guess to kick that off, you know, would love to hear, you know, obviously, there's a ton happening in the broader AI world right now. You know, one thing you think's like overhyped and one thing you think's underhyped?

• 44:22Speaker 01

  I think what's overhyped is the risk to humanity by these large language models, that somehow people have taken what is the appearance of consciousness and ascribed to the actual consciousness. Now, I almost said the appearance of intelligence, and I stopped myself because I think it's not, I think we've gone beyond the appearance of intelligence. I think these things are, in fact, intelligent in the sense that the behavior that they exhibit is intelligent behavior, but I don't think they're sentient. And I think that they're going to take over the world, I think, is really highly overinflated and overhyped. I think underhyped, I would say,

• 45:07Speaker 01

  is this convergence of biology and technology or machine learning, because I think it's two disciplines that are very, very hard. Very few people understand both. And I think Only a few people truly appreciate the real magnitude of that opportunity.

• 45:24Jacob Effron?

  Yeah, no, but I think it'll be exciting. Obviously, there's kind of this first wave of companies like yourself or Fursion, and as alumnus of those companies go out and you're kind of seeding this whole ecosystem of folks that do know both spaces.

• 45:35Speaker 01

  I think the space could stand to have a lot of successful companies and we cheer on companies that are of our type if they are done well, because I'll say there are there is a range there. There's companies that I think do really solid, very thoughtful, scientifically rigorous work. And then there's ones that are much more about the bluster than about the science. And I wish there were more of the former and fewer of the latter.

• 46:00Jacob Effron?

  I guess, what's the most exciting application of AI in healthcare you've seen outside of what you're doing at Inseedro?

• 46:06Speaker 01

  Um, I think that there is a tremendous opportunity for, we alluded to this earlier, to better, more personalized diagnostics. And I think, you know, we've seen some of that in even in earlier days with the early detection of cancer and things like that. But I think that with the collection of very large amounts of data, we can start to diagnose diseases way earlier and track progression in a much more quantitative way, using data sources that are less invasive and more easily collected. And there's a whole slew of these, and I think they are going to be incredibly impactful.

• 46:44Jacob Effron?

  Any particular approach or thing you've seen that gets you particularly excited?

• 46:49Speaker 01

  I don't know. I mean, it's, there's a bunch of things that are in relatively early stages and, you know, just like two weeks ago, they found a non-invasive, sorry, it's not non-invasive, it is invasive, but it is much easier, more easily collected biomarker for Parkinson's progression. And I'm like, we've seen very clearly that if you have such a biomarker, that you can tell who's likely to get sick at a time that it's easier to treat them, that if you can measure whether a drug is working, it not only Improved patient care, it also accelerates drug development. One of the other things that Sue Desmond Hellman told us when she came to visit is

• 47:34Speaker 01

  how she'd been one of the earliest AIDS physicians and the incredible, and also then moved into AIDS-related drug development. And one of the things she said, that without viral load as a biomarker for when a drug is working, the process would have been so much slower and so much less successful. Because you could tell within a matter of a few months from the inception of a clinical trial whether a drug was working versus sadly waiting for patients to become really sick and die. And that endpoint, not only is it a very sad endpoint, it is also one that just is incredibly challenging from a drug development perspective because you just don't have the rapid turnaround cycle.

• 48:19Jacob Effron?

  Yeah, no, makes sense. Well, Dabby, this has been a fascinating conversation. I'm sure our listeners will want to pull on all sorts of threads from what we discussed. So where's kind of the best place for them to learn more about you and the work you're doing in Citro?

• 48:31Speaker 01

  So you should go to www.citro.com. I will say the website right now is still kind of old and janky. We haven't updated in a while, but stay tuned because that is coming very soon. So there's also some links there to videos and such of talks that we've given from, you know, within the company. And so I would urge people to watch those talks or listen to some of those podcasts because they provide a much better view of our science than just looking at the website.