E25: Revolutionizing Patient Care with Neal Khosla of Curai Health

Nathan and Erik sit down with Neal Khosla, founder of Curai Health, a venture-backed virtual care startup using AI to provide low-cost primary healthcare. Prior to his current role at Curai, Neal was a machine learning researcher at Google and Stanford. In this episode, they discuss the current state of AI in medicine, what the future patient experience may look like, and how developments in AI healthcare may interact with different regulatory and social forces across the globe. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics FEEDBACK We'd love to hear your feedback or answer any listener questions live on an upcoming episode. DM @labenz on Twitter or email us at [email protected] with "TCR" in the subject line. We'd appreciate it immensely if you left us a review on Apple Podcasts or a rating on Spotify! TIMESTAMPS (00:00) Preview (04:32) The future of AI in medicine (12:50) Patient experience in AI-driven medicine and the current state of AI in medicine (15:04) Sponsor: Omneky (20:34) Building the LLM architecture for medicine (25:25) Current state of the art for AI in medicine (28:54) Evaluating LLM performance in medicine (31:49) Benchmarking LLMs in medicine (39:17) Using the Socratic method in training LLMs (43:57) Multimodal systems and deploying multiple models in medicine (51:30) The future of LLM creation and usage (59:47) The interaction of AI medicine with social and regulatory forces (01:11:00) AI adoption in countries with centralized healthcare systems (01:13:37) How should we regulate the usage of AI in medicine? (01:16:57) AI-first systems in medicine (01:18:19) Is there an obligation to deploy AI in medicine in its current state? (01:20:48) Neal’s favorite AI products (01:22:21) Would Neal get a Neuralink implant? (01:24:54) AI hopes and fears TWITTER: @CogRev_Podcast @nealkhosla (Neal) @labenz (Nathan) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries. From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  If you imagine that for your particular condition, there's one doctor in the world who's like the world's expert on it, that person should be available to you around the clock. It's a little bit alarming to me that in 2023, as a patient, I am still living in a world where it's like a couple of smart people sat down and talked it out as opposed to we've had millions and millions of people who go through all these medical conditions. Why can't we understand what happened to them, what was done to them and really use that data to help improve our interventions? There was that survey data that came out that was like, who's more most optimistic? And it was basically like a direct correlation between like optimism on AI and the wealth level of the country and like basically poorer

• 0:42Speaker 00

  countries are just like abundantly optimistic about this stuff. Like, it's very clear to me that in the next 10, 15, 20 years, whenever it happens, we're reaching a point as society where like, there's going to be so much infrastructure that cognitively can do so much for us that you have to decide, understand what spiritual fulfillment means to you.

• 1:01Nathan Labenz?

  Hello and welcome to the Cognitive Revolution, where we interview visionary researchers, entrepreneurs and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas and together we'll build a picture of how AI technology will transform work, life and society in the coming years. I'm Nathan Labenz, joined by my co-host, Eric Torenberg. Hello and welcome back to the Cognitive Revolution. Today, we continue our exploration of AI in medicine with our guest, Neil Khosla, founder and CEO of Curai Health, online at C U R A I health.com. As you'd expect for an entrepreneur who's raised more than $50

• 1:41Nathan Labenz?

  million in venture capital, even before the current AI moment, Neil is a polished communicator. I thought he did an excellent job of describing Curai's vision for the future of medicine in just the first few minutes of our conversation. So I'll keep this introduction relatively brief. Of course, we do cover a variety of topics and angles, including the impact of GPT-4, Curai's recently published research, which uses multiple instances of GPT-4 to improve performance.

• 2:09Nathan Labenz?

  How Neil personally uses AI for medical advice, what's missing and still needs to be built in order to ensure consistent quality of AI medical advice, CureEye's go-to-market strategy, how the medical establishment is reacting to AI progress and potential, whether poor countries are likely to leapfrog rich countries when it comes to AI adoption, an inconvenient truth about today's LLM landscape, how medical use of language models should be regulated, and plenty more.

• 2:39Nathan Labenz?

  Before getting into it, though, I want to take just a quick moment to, again, thank everyone for listening and share a few quick updates. First, we've heard your feedback about sound quality and have recently begun offering to send guests an external microphone, should they need one, to ensure that you can hear them as clearly as possible. This change will come online over the next couple of weeks, and sound quality issues should be a thing of the past. Second, if you have any other feedback or questions for me, you can email us at [email protected] or feel free to DM me on Twitter, where I am @LeBenz. I really love doing these interviews, but we've also got great feedback on our Eric and Nathan discussion episodes, so we do plan to do more of those in the future as well.

• 3:24Nathan Labenz?

  Please, let us know what's on your mind. Third, I invite you to subscribe to our newsletter, which is online at cognitiverevolution.substack.com. We send out new episode updates and also cross-publish my AI mega threads there.

• 3:40Nathan Labenz?

  Finally, for now, if you're enjoying the show, I'd ask you to help pay it forward by writing a review on Apple Podcasts, Spotify, or the podcast platform of your choice. I've received a bunch of private messages of thanks and encouragement, which has been super rewarding, but I'd love to see more of this posted online as well, as I'm told that this is the single best way to help others discover the show. Now, without further ado, I hope you enjoy this conversation with CureEye Health founder and CEO Neil Koslowski.

• 4:10Nathan Labenz?

  Neil Khosla, welcome to the Cognitive Revolution.

• 4:13Speaker 00

  Yeah, thanks, guys. Thanks for having me.

• 4:15Nathan Labenz?

  Super excited for you to be here. We are, and I am increasingly obsessed with the role that AI is going to play in the future of medicine. So really excited to get your take on it from all sorts of angles. In your role as the founder and CEO of Curai, I'd love to just start off with giving you a chance to give us your vision for the future of medicine. Like, what is my experience as a patient going to be like as AI starts to have an impact, you know, maybe say two years from now, 2025, and then if you can see that far into the future, end of the decade, 2030?

• 4:50Speaker 00

  Well, there's a couple of things I say, like, if you actually take a step back and just like think about medicine, and like what it means to practice medicine, for as long as humanity has been around, it's basically been you go and spend 15 minutes with a doctor. And like, the only lever we have is like, how much time you spend with the doctor. Like, if you're wealthy, you get more time with the doctor. You go back to like, the Middle Ages and stuff, like, you know, the kings and queens would get like, a lot more attention, obviously, than like, anybody else. But, but now, it's still not that different. Like, people pay for concierge doctors. But it all comes down to doctor time. And really, the idea is that we have the doctors, they're basically like oracles and sages, they're

• 5:35Speaker 00

  supposed to be well read on like, all the biomedical knowledge that's most up to date that we have. And they spend 15 minutes with you, and they give you a recommendation. And if you think back over the last 60 years, like, the most amazing thing about the computing revolution is like, it fundamentally has not changed that at all. It's probably the only profession in the world where if you take a human being today and drop them into the same profession 60 years ago, they would function just fine. Like, the only thing that's changed is that there's not an MRI or an X-ray machine. Those things were invented in the 70s. Outside of that, like, you have some therapeutics, but otherwise, it's the same job, which is pretty mind-boggling to think about. And so, when you say like, what's

• 6:18Speaker 00

  your vision for the future of medicine, I think what we have to... We have to start with this like fundamentally like what we do in medicine today is a very old thing. It really has not changed at all. And so our, our, our notion from the beginning has to be, has been like, we should fundamentally reimagine the way that a physician practices with data and computing at, at their center and at its core. And I think there's some really obvious things that, um, that fit into that, that everybody sort of talks about, like, you should be able to pull out your phone and talk to your doctor, or, you know, the data from your wearables and

• 7:00Speaker 00

  your other devices should, should bake into your health. Um, but I think the main thing that we've really focused on is that we think that AI can be a great equalizer in terms of the ability to make healthcare broadly available to many, many more people. And, you know, the way I always explain it is like, if you imagine that for your particular condition, there's one doctor in the world who's like the world's expert on it, that person should be available to you around the clock. And so our conception is that in the future,

• 7:36Speaker 00

  every human being is going to be able to talk to that person, maybe not directly, but at least somebody who represents the same set of knowledge. And a lot of that starts with building like AI systems that can scale more like software does. That's kind of how we, how we think about it at a large scale. I know that's, you know, like a 10,000 foot view, but in the future, everybody should be able to pull out their phone and have basically best in class, super personalized knowledge about whatever issue they're going through or their particular health that's available to them basically at zero marginal cost. And I think if you build that future,

• 8:20Speaker 00

  not only is it going to be really meaningful here in the US, but then broadly across the world where you've got eight billion people who are never, ever going to get access to the kind of care that somebody going to Mayo Clinic, for example, um, gets here in the US. So I don't know if that maybe answers your question to start, but I, I'll, I'll pause there.

• 8:39Nathan Labenz?

  Yeah, that's great. I mean, there's a couple of themes there that we've been. Increasingly, you know, developing and kind of obsessed with, I've got this notion of zero marginal cost expertise in general. And you kind of, you know, have a obviously much more developed in particular version of that for medicine. And then your comments about the, the difference between the level of access and, you know, therefore kind of the level of impact that developments like this would have in the U.S. versus much of the rest of the world is something that I also kind of harp on every chance I get because I think, you know, that should not be lost. And there's a lot to worry about with AI in general and with, you know, turning over medical decision making to AI. People

• 9:24Nathan Labenz?

  are, you know, certainly understandably cautious and concerned. But boy, when you think about the impact globally, it certainly gets me extremely excited. So it's, it's cool to hear you kind of talk about that, you know, right off the bat in your first, you know, first run through of the vision.

• 9:41Speaker 00

  Yeah, I would say the one other thing that's, that's popping out, just like hearing you talk was prompting some of these thoughts. One of the things that's most amazing about medicine is like, it is not a particularly data-driven science. It's much more a judgment-based art. You know, some of the studies on this are actually pretty fascinating. So if you look at clinical guidelines, like basically how the medical establishment says that doctors should treat an issue. There's a review done a couple, now probably about five or six years ago, that basically looked at them and said, how many of them are based off of grade A clinical evidence? And the answer was about 11%. So that means 90% of the time when you are getting a clinical best practice guideline, you're getting something

• 10:26Speaker 00

  that's based off of really what comes down to opinion, expert opinion. And then when they look deeper, they say like, what percentage of time do doctors actually follow these guidelines? It's about 50% of the time. And so if you take a step back, you go as a human being, like 95% of the time, you're not actually getting a very data-driven recommendation. Now, there is some data that informs these things, and there's usually expert panels that sit down and kind of talk through it. But it's a little bit alarming to me that in 2023, as a patient, like I am still living in a world where it's like a couple of smart people sat down and talked it out as opposed to we've had millions and millions of people who go through all these medical conditions. Why can't we understand what happened to them, what was done to them and

• 11:11Speaker 00

  really use that data to help improve our interventions? And that is one thing that we've worked on at Curia as well, that I think is a much longer path. You know, that's not something that's going to have an effect in the next three years or probably even five. But if you look on a 20 year timeframe, I think the ability to create the infrastructure, to collect that kind of longitudinal data on what happens to patients, and then use it to surface decision support right at the point of the care, so that we can make a decision based on that data. It's never existed in humanity. It's kind of interesting if you go to like a state like Utah, where it's mostly been a homogenous population because

• 11:52Speaker 00

  the Mormon population, like many of them have lived there for multiple generations, and they were very progressive in getting into electronic health records and genealogy and some of these things. It's one of the few places where they tend to have some of these like more longitudinal and sort of genealogical databases on patients. And they're just starting to figure out what they can do with that. But the rest of the world doesn't have that. And it doesn't even have a glimpse of that. And I would argue what they have in Utah probably is insufficient to really getting insights. So that's one other thing is like, I do think medicine over the next 20 years really needs to start thinking about how we lay the foundation so that care can be made in a super data-driven way as opposed

• 12:37Speaker 00

  to where we are today, which is, I'd say, you know, it's still incredible. Modern medicine is a miracle in many ways, but it's It's not all the way there for what I would want as a patient.

• 12:50Nathan Labenz?

  So let's maybe spend another second just filling in a little more detail, giving a little more color on kind of this future of medicine in kind of an experiential, you know, from the patient's point of view. And then I want to talk about where the state of AI for medicine is today. A number of exciting things have been published recently, including work out of Curai, which I definitely want to dig into. And then we maybe take a step back and talk about, okay, now where are you today? And how are you going to get to that future? And, you know, some of the barriers that might arise, including like regulatory, et cetera, et cetera. So if I'm a patient and just kind of trying to, you know, imagine my experience, right? And this is just a few years from now, potentially.

• 13:34Nathan Labenz?

  The idea would be that I have a 24/7 availability. That when I begin an interaction with the system, that there is a seamless, you know, presumably, uh, you know, embedding backed, uh, database of all of my previous interactions that have, you know, kind of the semantic representation of issues I've had, conversations I've had, you know, notes that the doctor took in the past, right? Presumably all of that is kind of dumped in and can be used for retrieval. Presumably like the medical literature is also, you know, deeply baked into such a system. And then what do I do?

• 14:16Nathan Labenz?

  Am I like interacting with a doctor in the same way that I would today, except it's just like a language model? Like, how does that then play out into care?

• 14:25Speaker 00

  Yeah, so I think it's important to root in like, what do patients do today? And there's basically two ways that patients access medical expertise. One is they talk to their doctor or a doctor if they don't have their doctor. And the other is they basically go online and do self-research, right? And ultimately, I think these two things probably need to become one, right? Like, there's a lot of ways in which self-research is maybe suboptimal for the patient in terms of coming to the right conclusions, because patients are self-directing and they may not always know, and the information online is not always vetted or it's not always good. And on the other hand, a lot of those pieces of information also should inform what your doctor is doing.

• 15:04Speaker 02

  Hey, we'll continue our interview in a moment after a word from our sponsors. I want to tell you about my new interview show, Upstream. Upstream is where I go deeper with some of the world's most interesting thinkers to map the constellation of ideas that matter. On the first season of Upstream, you'll hear from Marc Andreessen, David Sacks, Balaji, Ezra Klein, Joe Lonsdale, and more. Make sure to subscribe and check out the first episode with A16Z's Marc Andreessen. The link is in the description.

• 15:32Speaker 00

  I'm sitting here, I take fish oils every morning. My doctor, most likely, if I'm an average American, has no idea that I'm taking fish oils or that I'm not taking fish oils and that I've considered taking fish oils. Because most Americans don't have the time to talk to their doctor about those things. Most people are going on Google, they're going on Reddit, they're going on these other places to figure out, what do I need to do for given whatever my goals are? At the end of the day, we basically think that the appropriate interface does look something like a text box, which is, I, as a patient, have a way to say, this is what I'm thinking about, this is what I'm wondering about, this is what I'm dealing with. And on the other end, there's a system that helps me figure out, like, what exactly does that mean I should be doing for my health?

• 16:17Speaker 00

  So if I'm having symptoms, or if I'm considering taking a new supplement, or starting a new diet, or exercise regime, or if it's something more medical, I'm managing my diabetes long term, or I'm trying to get pregnant, or anything like that. You know, it all starts with a text box. And on our end, it's important to build the magic, as you kind of alluded to, to be able to parse through that, understand what the patient's looking for. It almost always starts by asking more questions, and this is something that's completely not present in sort of any kind of online research, is the ability to have a dialogue with a system that says, okay, you're considering taking this supplement, like, why are you

• 17:01Speaker 00

  considering taking it? What other things have you tried? How long have you been thinking about this? What are your goals? I don't know. And let's dig into your biomarkers, right? So like, do you actually have? You know, are your triglycerides high? Is that why you're thinking about taking fish oils? And so for us, it always starts with like, what is the patient's goal or intent? And then how do we drill down and understand more? And so that's always an automated questionnaire system. I think, you know, to get into the technical side of things, it inherently does have to tap into a long, large amount of patient history and data. And depending on the patient, we have a very differing amount of data.

• 17:46Speaker 00

  We are finding increasingly that even with more simple patients, you're starting to extend beyond the context window of the patient pretty quickly. And so you need to find other ways to kind of index that prior data you have on them, as well as what you're collecting in conversation with them right now, and use that to intelligently feed that stuff into your context window so that you can help the language model make a decision. I'd say the other big thing that we think a lot about is like, these problems are not totally solvable just entirely with a language model. Like, you've got to build a lot on the guardrails and safety side. So especially for where we are today, just from a regulatory perspective,

• 18:28Speaker 00

  you can't have a language model giving a patient direct personalized advice, like, go take this medication. That has to be done by a doctor. That's a regulatory requirement. So you have to build requirements or sort of guardrails around that. You have to build guardrails around when patients have mental health issues, handling those things appropriately. I don't know if you folks saw, there was a news article that came out about a person the other day who killed themselves sadly. And their significant other mentioned that they felt it was because they were talking to a chatbot. And so these kinds of situations are real. And so for us right now, there's a lot we're building that's sort of like on the safety side. Yes, from an experience perspective, you like come in.

• 19:12Speaker 00

  You you talk about what you want and then it's our job to drill down with questions and then serve that kind of biomedical expertise for you. And a lot of it is happening on the language model side, but there's a lot that's scaffolding that's being built around these models as well as support and checking to be able to surface good recommendations. And the the thing that we always do before we give a patient a final recommendation, we're always connecting them to a physician. And and that's where we are today, where things are getting escalated to a physician for final review and and to give that advice. That also allows us to do things like have the physician give you a prescription or a medication, or send you to get lab work, and you know, then further personalize the

• 19:55Speaker 00

  information or the recommendation we're making. So it's a really simple thing and concept. At the end of the day, we're trying to get people to talk to their doctors about the whole range of things that they're dealing with. And we're just trying to make it scalable and possible for the doctors to actually respond to those things by using tools that maybe do eighty percent of every visit that they're having.

• 20:16Nathan Labenz?

  So a couple of things to about to me about your description there. One is just how similar the architecture is between what you're building for medicine and basically everything else we're hearing about. You know, it's like whether you are trying to build an AI assistant, you know, you have a lot of those same things, right? People are like, I want it to be a text box. It needs to know my preferences for, you know, which do I like the the window seat or the aisle seat or whatever? And there's kind of this, you know, retrieval component to it. It's striking the degree to which things are just kind of converging architecturally, both in terms of like, the nature of the models themselves, but then

• 20:59Nathan Labenz?

  also the surrounding tooling. You can complicate that view for me, certainly if you if you want to.

• 21:06Speaker 00

  I'd say the main thing, because quality and safety is so important in medicine, one thing that I haven't seen a lot of people working on that we are, is what I'll generally describe as kind of like unit testing, testing or regression testing. Completely hypothetical scenario, a patient comes in and is like dealing with some issue. They've got, they've got some sort of like bacterial infection, they need an antibiotic. Great. Like, we can have a language model interview them and then the doctor can review the summary of What what the language model gathered and then they can work together to come up with a prescription for this patient. We want to have visibility into how the language model is handling a broad range of these kinds of scenarios.

• 21:47Speaker 00

  And so instead of just like releasing this product, we're actually, we've been working on a sort of large set of test cases where you basically can run a like a set of regression tests that say like for a common range of clinical cases, like, how does the language model handle these things? And because these things are non deterministic, you want to be able to have like, it's much like building software where you want to know that it works repeatably. And unit testing is a great way to get insight into that. So we haven't seen a lot of people working on like robust unit testing suites, but at least in medicine, it feels like a must have to know that like there's predictability. It's even more so true as you use language models that update over time.

• 22:31Speaker 00

  Like if you're doing any kind of fine tuning or human feedback or any of these things, the model is changing and you wouldn't want a scenario where we do really well with antibiotics today, but then tomorrow, all of a sudden the thing goes haywire. And so that is one area where I'd say our stack maybe is starting to diverge from what I'm reading about on the internet is like, you really want to have a large set of test cases to understand how the model is is thinking through and reasoning as it changes over time.

• 23:00Nathan Labenz?

  Yeah, that's super important. A little glimmer of that type of thing has come out from like OpenAI recently with their evals library, but I'm sure you are going, you know, comfortably 10x deeper in the use cases of interest. So maybe let's talk for a second about just kind of what is the state of the art in terms of what AIs can do in medicine now? And, you know, devoted listeners of the show will have heard me talk a little bit about my experience with the model we now know as GPT-4. I was, just for your knowledge, I was an early red team member and it hit me hard immediately.

• 23:41Nathan Labenz?

  One of the very first things that I tried was setting up a dialogue between me and GPT-4 as my doctor. Knowing what you said is so true that evaluation in general on language models is hard, but it becomes particularly hard when you don't have the expertise in the domain of interest, right? So as a non-doctor, you know, no real training myself, very difficult to do any sort of evaluation of the model's performance as a doctor. What I did was just attempt to recreate episodes from my own life. And I'm fortunate to have been pretty healthy, so I haven't had like a ton, but you know, went in to see a doctor about this, you know, saw my dentist about this, whatever. And I kind of recreated those, those little episodes.

• 24:25Nathan Labenz?

  I found that for my like, probably pretty routine, you know, certainly not the most difficult cases, the performance that I got from early GPT-4 was almost indistinguishable from my actual real life conversation. And I was kind of immediately like, whoa, like this is gonna be a huge deal. It was really that, maybe more than anything else, that caused me to become like even more obsessed and, you know, just drop everything else I was doing and focus on that red teaming thing for a while. But that's just an anecdote, right? So like, can you kind of characterize for us? I think people are probably gonna be pretty surprised by what you're, what you'll say. But I'd love to just kind of hear the characterization broadly of like, what are the state of the art things? What have they achieved?

• 25:10Nathan Labenz?

  And then we can obviously get to your recent publication as well.

• 25:13Speaker 00

  Yeah, so there's the quantitative and there's the qualitative. So we've also had access to GPT-4 for quite some time. And I'd say the anecdotal experience of ours, as well as most of the doctors on our team, mirrors what you're saying. I, at this point, and I would not recommend this to anybody at home who doesn't understand the risks associated with this, but I primarily, like, most of my medical, I, I hurt myself the other day working out. And the first thing I did was go on ChatGPT, pull up GPT-4 and type it in. That is primarily how I am getting my medicine right now, at least my medical advice, with the understanding that it's not always right, but it's easier and faster than trying to get in front of a doctor and it's pretty damn good.

• 25:58Speaker 00

  We have a doctor who moonlights at Stanford and he, he really insists on taking these tools into clinic with him. He's found that's made him a better doctor and is really, he's, he's said repeatedly that it does an incredible job. So. I would say, yes, anecdotally, these things are really, really capable. In terms of the qualitative, there's been a number of folks working on testing these systems, large language models on the US medical licensing exam, USMLE.

• 26:32Speaker 00

  And I believe GPT-3, they said, you know, the paper came out in January, suggested it was basically on par with an average physician. And now, sorry, it was passing, and now it exceeds the average physician, if I believe, if I have my data correct. And we just published a paper just a week or two ago that exceeded the performance of state of the art, which we had tweeted out, but I think Microsoft was the one who had, like, formally reported at post-GPT-4 launch. And so what we're seeing is these systems are, by any objective measure, as capable as any physician. I think there's still a long ways to go in terms of investigating.

• 27:14Speaker 00

  You know, one of the challenges, as everybody in AI is seeing, is these benchmarks don't really work or make sense anymore. They start being based off of, like, these assumptions that maybe don't work for the world today. One of the interesting things we've seen, for example, is like, on USMLE questions, they're, they tend to be short and have short responses. And so they're little vignettes and cases that are, like, more easily answered. And so some of the cases where these models struggle more may be on longer form, more kind of drawn-out reasoning. And that's where these things don't work as well out of the box, and you've got to build a lot of scaffolding, you know, some of the things that came up, memory retrieval,

• 28:00Speaker 00

  reasoning systems that help them work over time especially. So, you know, the short answer is, what we're seeing already is that out of the box, they're pretty damn good, and they've pretty much exhausted the state-of-the-art current benchmarks. And qualitatively, you know, what we're seeing is, we believe these models can do as well as or better than the median doctor here in the US. But there's work to be done on kind of building the scaffolding and proving that out in a more robust and rigorous format before you make any kind of egregious claims at scale.

• 28:36Nathan Labenz?

  So can we maybe unpack the nature of these questions a little bit more? I find that, you know, these numbers get thrown around so much, right? And like, most of our listeners will have taken the SAT presumably at some point, but I would venture that most have not taken, and I have not taken, the USMLE. So as part of your recent paper, Dialogue-enabled Resolving Agents, which we can unpack more as well, like exactly what that works and how that works and why it works. But I noted that you said that you had exceeded the previous state of the art on these open-ended questions.

• 29:19Nathan Labenz?

  Could you just give us a little bit of a sense of like, what those open-ended questions kind of are like? Just to ground the audience and even me, in terms of like, what sort of questions are we evaluating language models on right now?

• 29:32Speaker 00

  So the type of question you get is like, I'm reading one right now. Like, "A 67-year-old man with transitional cell carcinoma of the bladder comes to a physician because of a two-day history of ringing sensation in his ear. He received his first course of neoadjuvant chemotherapy one week ago, blah, blah, blah, blah, blah. The expected beneficial effect of the drug that causes patient's symptoms is most likely due to which of the following actions? Inhibition of thymidine synthesis, inhibition of proteasome, hyperstabilization of microtubules, or generation of free radicals? And then the fifth option is cross-linking of DNA.

• 30:15Speaker 00

  That is the kind of question that is in this open-ended dataset.

• 30:21Nathan Labenz?

  So, safe to assume, nobody is lucking their way through the exam. Yeah, I think it's worth just taking an extra second here to reflect on the fact that, in some really important ways, current systems are superhuman. You know, it's a weird shape, right? I'm always kind of interested in...

• 30:45Nathan Labenz?

  The ways that reality kind of diverges from our like expectations or our shorthand. And I think one key way is that like we're seeing sort of superhuman things in some ways, but not in all ways, right? Like I have in all of my obsessive GPT four testing, I never saw anything that I was like, that is, you know, more brilliant than anything I've ever seen a human do. I never saw any single insight, you know, that was like superhuman insight. But then you look at breadth and you're like, man, this same thing can answer that question. And it can also do like comparably well in law and it can do comparably well in like, basically, you know, many of the professions, if not most of the professions. Like that is superhuman in and of itself. So that's, that's just kind of worth not glossing over in my mind.

• 31:31Speaker 00

  One of, one of the other things I'll say about that data set is the interesting thing, the interesting step here too, is we took the step of basically stripping out the multiple choice answers. And historically, when these things have been tested, it's like pick one of these five answers. We kind of said like, answer this question. And the model can actually answer it as an abstract concept, which makes the problem statement significantly harder. In terms of it doesn't just get to pick from a list. And, and then we introduced this metric, we actually introduced it a few papers ago, called like GPT recall or GPT precision, which is basically a way of saying like, you can actually get the model to like, ask the model if the, if the stated, if the open ended answer was

• 32:15Speaker 00

  the same as the reported answer. So, you know, if the model says like, high cholesterol and the answer is cholesterol over, I don't know, 200, like those should be noted as the same thing. So you actually have to do this like reverse mapping problem to figure out if the open ended answer was the same thing as, as the real, the spirit of the answer. And so this is one of the ways where like these previous benchmarks start to get a little bit broken by these models, which are just becoming so capable at answering these questions.

• 32:46Nathan Labenz?

  Yeah, there's a really, I'm glad you mentioned that. And it's something that's been on my mind quite a bit recently as well. Going back to the red teaming thing, there were at least two instances of papers that were published during that, my personal two month red teaming window, which was September and October of last year, where the conclusion published was basically language models still can't do X, you know, for whatever X was. And at the time, I was like, well, I'm pretty sure GPT-4 can do it. So I kind of spot checked, you know, and sure enough, like in the two that I checked, it was able to do the thing.

• 33:24Speaker 00

  I know exactly what you're talking about.

• 33:26Nathan Labenz?

  That sent me, and that's still an open question in the broader research community. That sent me down a little bit of a benchmarking rabbit hole in that I started to think, well, how good is that on all these different benchmarks? And I found exactly what you found also, which is that if you set up your benchmarking script with a slightly dated paradigm, like if you take a 2021 Big Bench script, for example, and you just run it, first of all, it's set up on like a few-shot basis. And second, the structure of that few-shot is such that the model is basically forced to give you a multiple choice answer straight away.

• 34:09Nathan Labenz?

  And as a result, its performance really suffers compared to what it would do, not even if you like get really creative and do like amazing prompt engineering, but literally if you just take away the, you know, the few-shot structure, take away the multiple choice and just present it with the question, you'll get like way better answers than you do by using the kind of established structure of the benchmarks. So as an aside, if there's any listener that wants to sign up for a little project, I think there are a number of papers recently published that I would like to go dig in and kind of retry some of the experiments that have been run with like honestly, just more naive prompting strategy. So reach out to us if you want to do that.

• 34:51Nathan Labenz?

  And I'm glad that you are on top of that and not falling victim to that pitfall.

• 34:57Speaker 00

  Well, I think it goes in both directions, right? So like on the one hand, it can hurt the performance of the model. It can also inflate the scores. Like it can reduce the problem. Like there's no scenario in real life in medicine where the doctor is given five options. If you want to compare these things to how a doctor would do. In real life, you have a patient in front of you and you have to, you have to guess one of five answers. But like, no, that's never the case. You have a patient in front of you and you have to come up with the answer from, you know, the depths of your imagination. I don't know where it comes from if, if you're a human practitioner, but what we've noticed and felt is like measuring these things on, on multiple choices is just like a bad benchmark because it makes the problem so restricted in domain.

• 35:43Speaker 00

  And especially for areas like biomedicine, biomedicine is a very open-ended thing. I mean, if you think about real patient cases, they're never like straightforward clinical things and, you know, we can talk more about the implications of this for language modeling, but like, you never have a scenario where like you get a list of symptoms from the patients that are incredibly straightforward. And then like you just map it to some, um, to some diagnosis. Like if you actually look at like the history of AI and biomedicine, that's how people started trying to do it. They would come up with these big lists of like, they, these basically these big graph structures that were like these symptoms mapped to these, you know, these they call clinical findings mapped to these diseases.

• 36:23Speaker 00

  And the problem is the expressivity of those models, right? Like you have somebody who's lactose intolerant. What is the finding there? The find, like if you want to diagnose the disease of lactose intolerance, you, the findings have to be like eat milk or like, um, you know, in some cases they have to be like eat cereal because they didn't even eat, like they never tell you they ate milk. They just tell you they eat cereal. And this is how like the human brain works is it can kind of generalize these concepts to make the diagnosis. In real life, like the world is really messy and open-ended. And one of the things that language models have really unlocked in medicine is the ability to understand this broader context and represent a lot of clinical findings, um, in,

• 37:05Speaker 00

  in very abstract conceptual terms and still be able to reason on them. So one of the things that we're a big fan of in these benchmarks is kind of removing this artificial structure because medicine never has artificial structure. Patients come in and really messy scenarios and doctors have to adjust and, and sort of treat them with very, very messy data.

• 37:25Nathan Labenz?

  So kind of upshot there is in addition to my complaint that the multiple choice benchmark, when presented the wrong way to the language model can lead to understating its performance, you're also highlighting, you know, equally important point in the opposite direction, which is that just giving it multiple choice answers is like a far cry from actual challenge in practice. And so that's why you've created this additional sort of elaboration where you remove the multiple choice and then you kind of do another language model mediated assessment to say like, did it sort of come up with the right answer on its own?

• 38:05Speaker 00

  Yes, I'll just leave it at that. Yes, I think, I think that's very, very much true. And I think the, the problem statements themselves, like this is where we're starting to hit the ceiling of the benchmarks is the problems are controlled, right? Like what I read was a very controlled vignette and patients are messy. One of the things we found really messy about like diagnosis and decision support in medicine is that it's an evolving thing. So like a patient comes in and you have one differential diagnosis. As you learn more about the patient, it changes. You're not just dealing with like a snapshot in time. You're, you have a moment and then you talk to the patient, you get more information and it changes. And then a week later, the diagnosis changes again and like so on. So these, any

• 38:47Speaker 00

  model you build has to be able to like dynamically reason and change over time. And, uh, and these, these benchmarks maybe don't do that, but anecdotally, the language models do really well.

• 38:59Nathan Labenz?

  Well, first let's go to your research because you guys just published this paper and this is the perfect time to talk about it. So you're digging in on this benchmark. You've got GPT four access, you know, anecdotally and even qualitatively, uh, we're finding that GPT four can do a lot straight out of the box. And now you've added this layer of dialogue enabled resolving agents, which kind of reminds me of like a couple of different things, like the Socratic models paper was maybe the first one that, that cracked my, um, Consciousness that has kind of a similar paradigm. But tell me how it works in this case, like you guys have brought multiple models together or multiple instantiations, maybe of the same model, and you're getting better results.

• 39:42Speaker 00

  So for this paper, we have two instantiations of the same model. And the concept is like, you give these cases to one of the models, we call it the decider. And then you have another model, basically poke holes. We call it the researcher that goes around and pokes holes in the conclusions that the decider is making. And it becomes this Socratic style dialogue. We actually originally, before we posed a patient paper, we called it student teacher. And then for reasons, we kind of moved away from that terminology. But it's a really clever way of getting two models to kind of work together to come to a better set of conclusions. And I think the paper basically shows across a variety of tasks that this works to lead to state of the art performance in medicine.

• 40:27Speaker 00

  I mean, candidly, I'd love for somebody to try this kind of model and go take it to other things other than medicine, because I suspect it will work. And it's a recurring theme that we're seeing right now in the AI world, which is GPT-4 is great. And it's great to like, create a prompt and give it a, you know, and like, give it a problem and see how the prompt does on a problem. But what seems to be even more powerful is setting up multiple instances of these, of different agents and have them interact in complex ways. And we're sort of unlocking the, the sort of, we're still unlocking where that can take us as a paradigm. And in medicine, I think it makes a lot of sense. As I mentioned, like the challenge is, I mean, for obvious reasons, so much biomedical

• 41:11Speaker 00

  knowledge is embedded in the latent knowledge base of a large language model. The question is sort of, how do you elicit it? And I think what we find in this paper is that kind of skeptical questioning of the deductions or the conclusions made by the model can, can kind of push you to a better resolution. And I just think this is a super, super exciting paradigm. And we're continuing to kind of explore this as an area of research.

• 41:40Nathan Labenz?

  You said it's the same model. Is it basically GPT-4 with different lenses on prompt engineering that you're then just bouncing back against one another? For this paper, yeah. That's amazing. You know, another thing that that really reminds me of is one of the authors of the diplomacy paper that came out of Meta. Cicero. Yes, Cicero, the model, yeah, that played diplomacy the game, talked about the very general strategy of trying to bring more compute to bear at runtime. And he was kind of talking about how, you know, if you went back to like Deep Blue in

• 42:20Nathan Labenz?

  the original chess days, you know, basically what you had a lot of was deep search and a ton of compute running at, you know, at runtime for each individual move, just crunching, crunching, crunching through, you know, the trees of possibility and, you know, some smart heuristics around like word truncate search and, you know, which trees are worth exploring and which not, but like it was just a ton of compute at runtime. And then he contrasts that to today where he's like, you know, by and large, the compute is all done in the, in the training. And then at runtime, you're just, you know, predicting one token at a time. And that's like, whatever, like a million times less or something, maybe even more than that.

• 43:02Nathan Labenz?

  And so his kind of paradigm and what they did with the Cicero paper was they tried to figure out ways to bring more compute into the picture at runtime. They had kind of multi-part approach that included like a constellation of models more than purely a language model.

• 43:24Nathan Labenz?

  And you're doing something similar here where it's two, you know, two kind of summonings of the same language model into different roles that you can then place into dialogue. But effectively, you're multiplying the compute with, you know, a certain flavor on it. I wonder if you would add anything to that? And then I also really wonder about like, what about other kinds of models that might be added into this system, right? Like I imagine, you know, people have been talking about forever, like, well, the radiologist will be the first to go because you're, you know, you. It should it should be easy for AI to read a scan. We haven't seen that, but we also haven't seen GPT four multimodal deployed widely at all either. So, yeah, I don't know.

• 44:07Nathan Labenz?

  Any thoughts on the kind of bringing computation to runtime and and kind of more different kinds of models working together?

• 44:12Speaker 00

  Yeah, I mean, I'd say there's a couple of things. One, like, yes, I think this is a really powerful paradigm. You know, in the old classical world, you'd you'd call this like ensembling, right? I think one of the really interesting intuitions and why like two instances, it doesn't make sense in a lot of ways to people that like two instances of GPT four can like. Like, in theory, they shouldn't have like independent failure modes. And so like, you know, the model, it shouldn't get better. But what I think a lot of people are finding in research is like this state space of the model is so large that what you're really trying to do is you're trying to figure out how to sort of elicit the right knowledge out of the model. And there was a paper that came out the other day.

• 44:57Speaker 00

  There was about like, why? I haven't actually read the whole thing, but I'm very excited about it. People are starting to research this. It was about chain of thought prompting. And it's like, basically, they argued that like reasoning is emerging like from, I think they called it the locality of experience. So it's like, sort of local clusters of variables in the in the model that influence each other. It's a super cool concept and and I'm curious to see where this takes us. But I suspect that we will go really far with even one, like multiple versions of one language model. As these models get bigger and bigger, just prompting and and obviously, you know, you're kind of seeing the auto GPT stuff.

• 45:42Speaker 00

  There's some differences there in that it's doing more coordination and orchestration, I would argue. But I I think it's a lot of the same phenomenons that like, hey, one version of the model that's coordinating and orchestrating can kind of prompt the model to do other things. And that is a very powerful paradigm. I think the other thing you're kind of getting at is other modalities and other kinds of models. To date, what I will tell you, we don't feel that like.

• 46:10Speaker 00

  There is a tremendous amount of value in taking a worse performing model over the better performing models, or I should say I don't. From my vantage point, these models are so large that you like you haven't hit the performance limit of using, you know, X versions of the same model in conjunction. That seems to be better than like use one GPT-4 and one GPT-3 and one Bard and what have you, which is very counter to the intuition. I think of a lot of machine learning scientists. It's counter to what my intuition was, but that's not anecdotally what I'm seeing. And then I think on the multimodal stuff, I think it's it remains to be seen how

• 46:55Speaker 00

  powerful this can be. But my suspicion is it's a really powerful paradigm. I actually was looking at a patient conversation this morning where the language model asked the patient to upload a picture. I was kind of sitting there thinking about this. My suspicion is that like GPT-4 multimodal is which I have not touched is going to be able to synthesize information across these modalities, which will lead to like a net improvement in the performance of these things. Like if you think about like having a picture of a rash and the patient describes it as like itchy and flaky, as well as you can see that it's red, like that is a lot of information that if it can be synthesized and combined is much more powerful than like independently a model that knows

• 47:38Speaker 00

  like can distinguish from the image that it's read and then has the description in the language that it's itchy and flaky. This is pure speculation because we haven't played around with these models. But I'd say what we see is like the more diverse data and the larger data you give any of these models, the more powerful they seem to generalize. And we see we don't seem to have hit the ceiling there. So I'm very excited for throwing for throwing imaging data for throwing whatever kinds of other multi modal data we can get. I mean, a human being, you know, theoretically listens to sounds to do diagnosis. One of the interesting things we found, though, that you might find surprising, Nathan, is that these models already generalize to other kinds of data really well.

• 48:23Speaker 00

  One of the things that really surprised us with GPT-4 is that can interpret continuous glucose monitor data out of the box. So you take a glucose monitor and basically you can think of it as a graph of glucose versus time. And I suspect that the model never saw like actual time series data of glucose, but it probably saw time series data and it probably read things about glucose. And so it can generalize to say like, at 1pm, the patient's glucose spiked to 170, which was a sign that they ate a, you know, a carb heavy meal. And you sit there and you go, holy shit. It's never seen this kind of data. And all of a sudden, it can generalize to it and do a pretty, pretty damn good job. And we

• 49:08Speaker 00

  didn't do super, super rigorous evaluations, but anecdotally, what we saw was it did pretty well. And I think this is like a scary proposition for a lot of people who are talking about their data advantage. Like, there are companies who have glucose data and say like, this is our advantage. I, I believe that these models are generalizing and as, as we feed them more multimodality, they're going to generalize even better to the point where out of the box, they're going to be able to do a lot of these things that people historically have thought of as their kind of secret sauce. So that maybe took a different turn than what you're expecting, but I think it's a fascinating area. And we're seeing it in medicine and I'd love to see where people are seeing it in other, other disciplines as well.

• 49:48Nathan Labenz?

  Yeah, that's a great one. The, I mean, surprises have not stopped just yet, which shouldn't be, you know, not too crazy because we're only on, I've started counting time from GPT-4 release. So we're on GPT-4, four weeks and one day in the new calendar. So there's, you know, still, I think probably quite a bit in the vast surface area of these things that will be, you know, is yet to be explored and will continue to surprise us for a while. I guess I'd love to hear kind of how some of this, you're starting to touch on some of the business questions, right? Like moats, where did I come from? Does anybody have them? I'll get your sense for kind of where you're headed there.

• 50:28Nathan Labenz?

  It sounds like you're partnered with OpenAI to at least some degree where you had a preview of GPT-4. Do you expect that like? OpenAI is kind of, you know, to borrow a phrase, all you need for the foreseeable future? Do you think that that will turn into like becoming a Foundry customer? Foundry being their, you know, as yet unconfirmed, but I think credibly leaked enterprise offering with robust fine tuning that's coming soon. Do you think like at some point you create your own models and, you know, go a totally different route?

• 51:10Nathan Labenz?

  Maybe it's all of the above, but what do you think is kind of the future of how? You and Kury will use language models in over the next couple of years?

• 51:19Speaker 00

  Yeah, so my general critique of everybody in this space right now is that everybody's trying to believe the things that are convenient. And I think the first like inconvenient truth right now is that OpenAI is way better than everybody else. And it's not particularly close. I played around with Google's models. I played around with other models. I won't name other companies because I don't care about insulting Google as much, but they're not, they're not anywhere close. And that doesn't mean that can't change. And I really, I think everyone should hope that it changes, that there's good competitive forces. But as it stands, like if you're talking about pure intellectual horsepower and capability,

• 51:59Speaker 00

  you are sacrificing pretty much no matter what, unless you're using OpenAI. I think it's really important for people to be specific about their problem statement and the problem that they're solving. I would argue for some of the content creation use cases, like something like a Jasper, that they, the performance may be at a point where it's sufficient enough that you don't really need the latest and greatest model. And from a cost benefit trade off, it's probably. Not worth it. And in that case, it may make sense to train your own model or rely on open source or other models and combined models, you know, and I, we, we don't have a problem like that. So we can't speak to it at length.

• 52:37Speaker 00

  But that's my sort of like high level understanding. For something like us, like performance is absolutely critical in medicine. And so we definitely need to use GPT four. And we need to build on it in pretty robust ways to be able to get the performance we want. And that's everything from, like I mentioned, safety and rigor and, and sort of unit testing to good prompt engineering, to guardrails, to all sorts of like algorithmic improvements. I'd say everything from like, you know, there's people working on, how do you increase, how do you increase the context window or the memory of the language model to, you know, papers like what we're publishing, which is like, how do you use age, utilize agents to kind of get better reasoning?

• 53:21Speaker 00

  I think those are all areas of kind of open, open research for us where we continue to kind of push the envelope. But in many, many cases, we're sort of relying on open AI as

• 53:36Speaker 00

  it's kind of the base model. I think we are also really interested in this topic called goal oriented medicine for us, which is like much like, you know, like auto GPT, like often in medicine, you have a goal, a patient wants to lose weight, they want to control their diabetes, et cetera. It's an open problem to figure out, can you direct a language model with that goal to then interact with the patient proactively in some cases to say, how do we work together to accomplish this thing over time? So for where I stand, like the lowest common denominator still is from the language model perspective continues to be open AI.

• 54:15Speaker 00

  Everything else is sort of like reasoning, safety, improvements, memory. There's so much work to be done to build an advantage on, on the non-core language model stuff. I think the idea that you're going to train a, a language model for a specific use case. Depending on the use case can be anywhere from. Correct, but insignificant to completely delusional. I would say companies that are thinking about like, hey, we have a moat. I just brought up the CGM example. I think that's a great example of how these models are starting to generalize in ways that like your data probably isn't that valuable. And most importantly, like, you know, you know this Nathan, these things are incredible few-shot learners.

• 54:58Speaker 00

  And so you can give them 10 or 100, or worst case, 1,000 examples of of a certain cognitive task and they tend to generalize incredibly well to that task. And so the idea that like, I have a million data points of thing X and therefore that's going to prevent other people from doing it. I don't know that that's a particularly robust viewpoint when if I can just have somebody, you know, whether it's scale or otherwise manually label a thousand of these things. And I can, I can get, you know, a 90, 95% of your performance. It really is going to depend on you needing 99% performance for that to be a substantial advantage. And I'd argue for many, many use cases, that's not the case.

• 55:42Speaker 00

  And it'll end up coming down to UX and your distribution and what have you.

• 55:46Nathan Labenz?

  I appreciate the candor. There's a lot of inconvenient truths, I'd say, right now in the AI space and a lot of denial going around in a lot of different directions. So I think the dose of realism is always welcome.

• 56:02Speaker 00

  The one, one other thing I'll say is I would not underestimate for your use case, like how much generalization really matters. So domain-specific models are another thing that I think are a little bit overblown. Unless you're worried about cost or latency. Again, GPT-4 is probably going to beat your domain-specific model. And, you know, I brought up the example of like lactose intolerance, but we had another patient case where it's like the patient went to Burning Man and their lungs started hurting and It's like, how do you actually make a hypothesis about what's going on with this patient? If you're a, imagine if you're trained only on medical records or like biomedical data. You don't have any idea what Burning Man is. And it turned out the patient got dust in their lungs, right? And so that's what caused the issue. But unless you understand Burning Man, Las Vegas desert, Las Vegas desert, like dust in the lungs.

• 56:47Speaker 00

  Or maybe like, there's also a rare fungal infection in the desert. Like, unless you know that kind of world knowledge, you're trading off performance there. So domain specific models are another thing that, um, I think a little bit overstated in terms of their potential. And I'd love for them to pick up, but I think unless it's like data that the, the models like never can or will see, like a completely left field thing. I suspect your, your generalized models are going to be able to beat you or at least meet you, uh, on performance.

• 57:17Nathan Labenz?

  Life is big. The world is big. The world is messy. There's just, yeah. And especially for something like such a complex system like our own bodies. I mean, they, yeah, the clues that are just kind of sprinkled into conversation that can be so meaningful.

• 57:33Nathan Labenz?

  It is, I, I think what you're saying does make a lot of sense. Like it's, it is very hard to imagine how you could be able to interact with a patient. You might be able to make, you know, you might be able to score well in the USMLE, uh, with a domain specific model perhaps. But it does seem like in those real interactions where like the context matters so much and these little hints, these clues, there is a lot of value to all of that. You know, you might, you might even call it a world model. Um, I don't want to get, uh, into trouble for using that term, uh, out of order. But it does seem like there's something very powerful there. So let's reel it back into present, right? We've, we've outlined your big picture vision. It makes a ton of sense to me. It's incredible to realize that for the most part, it sounds like.

• 58:19Nathan Labenz?

  The core tech that we have today is able to support that vision and that there's like some refinement, some engineering, some integration, savvy usage, guardrails, unit tests, all that stuff that kind of still needs to be proved and vetted out. But if it kind of boils down to the question of like, do we have the core discovery necessary to realize that vision? It sounds like the answer is basically yes. And again, complicate that for me if you think that's wrong, but.

• 58:48Speaker 00

  I would just say like, it's a very complex problem, but I agree with you, like the raw natural language understanding toolset and toolkit. Yeah, I think it's there. There's a lot of complexity in the problem that needs to be solved for.

• 58:59Nathan Labenz?

  So tell us like where you are today, like if I, which I have done, by the way, you know, go to Curae and sign up and I become a patient, you know, my own very fortunate privilege is that I honestly didn't have enough medical needs to like really get too deep into how it can help me. So, you know, thank good fortune, providence, whatever for that. But I'd love to understand kind of how much of this vision already exists? And then how are you thinking about getting there? And this is, you know, a topic that Eric is definitely super interested in as well. Like, how does the introduction of this technology begin to play with the social, regulatory, governmental, legal systems? I mean, medicine touches everything, or maybe medicine, everything touches medicine.

• 59:43Nathan Labenz?

  So where are you today? And kind of how do you navigate a path through the thicket of current structure to get to that future vision?

• 59:51Speaker 00

  Yeah, so we have a direct to consumer product. You can go online, you can try it out. It's curihealth.com, C U R A I. You know, patients can come online, they download our app, they can match with the doctor, they get ongoing access to care. It's $14.99 a month. But a lot of our focus now as a business is more on our enterprise customers. And that's health plans and provider systems that we're working with. And, you know, the concept there that I basically say to folks, at this point, the cat's out of the bag. And over the next three years, like, most patients are going to start with ChatGPT for medical advice. I genuinely believe that. We saw it with Google, like, everybody starts with Google for their medical information and advice. And we're going to see that exacerbated with ChatGPT.

• 1:00:33Speaker 00

  And I think generally, this represents an opportunity for existing health systems and and health plans to say like, let's get on on this trend and in front of it, instead of being reactive, where we were 20 years ago, where people started coming in with their printouts from Google and saying like, I think in this case, it's even more dangerous because you're going to have a lot of scenarios where people are going to self-serve on ChatGPT and never go to the doctor. And that information may or may not be correct. You know, it's really hard to guarantee reliability. I think there's other implications in terms of the business of these institutions where like, you know, for a for an health insurance company, if ChatGPT says to go to a neurologist,

• 1:01:18Speaker 00

  that's an expensive thing. And so we really want to make sure that you really needed to go to the neurologist before the patient goes and self-books or, you know, self-assigns themselves to a neurologist. We're spending a lot of our time and effort kind of scaling up our partnerships with folks who say, we want to create kind of our own consumer-centric, AI-centric version of accessing care that has humans in the loop, that has doctors who can provide oversight and and actually close the loop in terms of providing convenience for the consumer. So if the consumer needs medication, or they need a lab test, or what have you, you know, we see that as a key role we can play is that we can actually deliver medicine in this virtual format, instead of just giving you information.

• 1:02:00Nathan Labenz?

  Does that mean today, like, if I have, say, the app of one of your health system partners, like, do you have this deployed where I can go talk to GPT-4 and kind of have that whole interaction? And then that gets kind of kicked off at some point to like, okay, now you're going to talk to the human doctor that's going to review all that information? Like, is that all live today?

• 1:02:21Speaker 00

  Yeah, so our health system, our first health system partners are going live this year. And that is exactly the conception. Like, you know, we work, we don't work with them, so I can say their name is a fake example. But like, you know, Stanford Healthcare, which is right here, you download the Stanford Healthcare app, you go on their website, there's a button that says get care now or talk to a physician. And you click on it, and really, you get GPT-4 first. And and I shouldn't say GPT-4 because it's really like this set of models and this this system we've built on top of of these large language models and. And then we sort of, we built the system such that it can appropriately triage you to the right kind of provider, depending on what you need.

• 1:03:05Speaker 00

  Um, at the right moment, if you're having a medical emergency and then the doctor can jump in and sort of give you guidance or close the loop. So it's that concept of we kind of give the patient eighty percent of their care. And then the last twenty percent is coming from, uh, from the clinician, especially in terms of the active decision of what to do. And we've built a bunch of tooling on the backend that sort of speeds up the clinicians providing them with automation and decision support. So putting together notes for them, automating the follow-up process of checking in with the patient after a visit. Automating, um, sort of like the, uh, putting together a care plan for them as well. And those are some of the problems we've worked on.

• 1:03:43Speaker 00

  We're in an interesting state right now, Nathan, where like, we're actually with the public launch of G P T four have been releasing like aging out old models and, and putting new ones in. So right now, if you actually download the app today, you probably, a nice certain five percent of patients won't get much interaction with the language model. Because we're doing a slow rollout. Otherwise, you know, presumably, presumably, you know, three months from now, if you're listening to this, this is recorded at the beginning of April, like, uh, you know, ninety-five, a hundred percent of people. We'll be receiving that kind of interaction directly with the language model.

• 1:04:15Nathan Labenz?

  Yeah, again, it is amazing, just the timelines are so short. It only came out a month ago, right? So it's like, and I, I always kind of remind people because the, of course, the hype has, the hype has also come up very quickly. But in a pre-G P T four world, I think it was still reasonable, if not necessarily the right conclusion. To say, well, I don't know, I tried G P T three and it was like pretty dumb still, kind of. And I, you're telling me this is going to change the world. And now like we're in a moment where it's like, yeah, no, here's the real deal. It really is going to change, uh, the world, but you know, it's only been available and still, you know, like A P I access waitlisted, all that kind of stuff. Uh, you know, for not even a full month yet.

• 1:04:59Nathan Labenz?

  So the timelines are just, you know, insane. So that's good to know. So you've got the five percent deployed in kind of your direct model. And then, you know, you're working your way up on that. What are you kind of hearing from the establishment? Like, what are the regulatory barriers that you think you're gonna have to deal with? Like, how big of a problem is HIPAA for you? Every time I feel like I do anything with a doctor, it always falls down on my, you know, I can't get the information where out wherever I want it. It's always a pain. I'm sure that's a challenge for you, but like, how big of a challenge is that? And what do you think are kind of the the most interesting or difficult parts of that overall challenge?

• 1:05:33Speaker 00

  Yeah, I mean, one of the challenges with using GPT out of the box is that it's not HIPAA compliant. And so we've had to, you like, we've had to be intelligent about where we can utilize it, where we can utilize other models, like where we can have to build our own stuff. Yes, it is a challenge. You know, I know that Microsoft and others are working on this. And I think long term, they all know that this that healthcare is a high value use case. And so I'm sure this problem will get solved. But for now, it continues to be a little bit of an obstacle. From the establishment perspective, what's amazing is like, how much chat GPT seems to have changed everything. I never thought I would have like CEOs of health systems and large health plans, some of whom we work with that are like, tell us about your work with chat GPT.

• 1:06:19Speaker 00

  And like, how is this gonna change our business? And I think part of it is that it made our business, like, we've seen a pretty massive acceleration in our business just because it has really made our business digestible. Like, I used to have to explain like AI and what it can do and what it can't do. And now people just like assume that everything can do everything because of chat GPT. It's so abundantly evident that it's hard to argue against. I mean, you see, you definitely see some more thoughtful critiques in the medical establishment that are like, here's where there's errors and there's solutionations and things. And like, you know, when I say like, you have to, you know, like, GPT four is the most powerful model, like, you still have to solve for all of those challenges. And things like explainability are really important when you're just, you know, deploying to doctors.

• 1:07:04Speaker 00

  So how do you combine GPT four with other models? So you can get interpretability and explainability. These are the types of questions we're getting from the establishment. But I'm not really seeing anybody say like, hey, these models can't do it. And that's remarkable because.

• 1:07:22Speaker 00

  Two years ago, ninety, a year ago, ninety-five percent of the establishment was like, these models can't do it. And that all seemed to change like in a period of three to six months.

• 1:07:33Speaker 02

  Are there countries that you think are best suited to take advantage of AI in medicine?

• 1:07:39Speaker 00

  I think certainly there will be a leapfrog effect. I don't know if you folks saw, there was that study that came, or that survey data that came out that was like, who's more, most optimistic? And it was basically like a direct correlation between like optimism on AI and, and the wealth level of the country. And like basically poorer countries are just like abundantly optimistic about this stuff. If you think about a country like India, you know, there's a billion people, a billion plus people, and there's just no ability to service their patients. One of the practical challenges in these developing countries is that the way that medicine is practiced and even coming down to things like the drug supply chain are not necessarily embedded in these models and are

• 1:08:22Speaker 00

  actually like quite nuanced and different. So like, you know, you can ask your favorite large language model, like what would you prescribe this patient? And it will generally give you a answer that is like acceptable to the Western world, and especially it's like highly indexed on the US. But when you go to India, it turns out like they really only have like a very limited drug supply chain. And so your answer becomes irrelevant. You can try and get them, ask the model like, what should I prescribe if I'm in India? And sometimes it can do okay, but other times it doesn't really have the knowledge of like what that entails. And then there's other challenges like basic help literacy and

• 1:08:59Speaker 00

  and just like ability to articulate what's going on with you as a patient. Those are all practical challenges that will get solved. And I think the regulatory environment will be a lot more favorable in those countries. But I mean, we operate primarily in the US and I would say we're really optimistic about the environment in the US. Right now, we operate sort of really under tight doctor's supervision. And so our model is, you know, it creates an opportunity for us because we have this sort of like complex, it's like the difference between like brewing a Starbucks coffee and building an entire Starbucks. Like right now, we have to build the entire Starbucks. In India, you can just brew the coffee and hand it to everybody. And so it's a little

• 1:09:42Speaker 00

  more complex, but building the Starbucks is higher value. And I think longer term allows you to kind of optimize everything, like be able to collect our own data, do our own human feedback, do our own fine tuning, like, and those are all things we're working on that allow us to kind of gain pretty significant advantages. So I am still bullish on the US because the establishment and the regulatory environment, I think they haven't done the thing that some of the other, these other countries are starting to do and come out and say like, you know, absolutely no. I think the mentality continues to be like, let's see and let's be careful and let's, let's have safety. But, you know, we're open to this stuff. You know, China, I think is this different thing. I have to imagine the Chinese government is kind of like, in six months, everybody's going to have a

• 1:10:27Speaker 00

  large language model on their hand and there's going to be no doctors. I mean, I'm only being partially facetious there. It's a little bit alarming, but it's, you know, I think they'll do really well with it long term. I think short term, there'll be some serious damage done if they do something like that.

• 1:10:42Nathan Labenz?

  Yeah, you mentioned China and kind of the, I don't know their medical system at all, but you know, just kind of the ability to make executive decisions on something like this. I was wondering if you also had a point of view around systems like, say, the UK with, you know, the National Health Service or Canada, where there is this kind of much more centralized decision-making structure. Do you think those, those countries maybe could be like super early adopters or super late adopters, just kind of depending on maybe some very idiosyncratic factors?

• 1:11:15Speaker 00

  Yeah, I have to imagine that's true. We've looked a little bit at the UK. I haven't looked at Canada. My general concern is that

• 1:11:25Speaker 00

  There are such large bureaucracies that it's hard to move. The one good thing about the US is like, if you want to get distribution in the US, like yes, eventually you have to work with Medicare and Medicaid and you have to work with United Health Group and some of these super large players and Anthem. But there's a lot of places where you can start and get some evidence and prove out the model and that's a lot harder in Britain where everything is under the flag of the NHS. Now, there is a private healthcare system that has cropped up kind of in response to some of the shortages and stuff and the need is really acute. I mean, pretty much everything you read about the NHS says it's crumbling from the insides. So they need to figure out how to make it economically sustainable and they need to

• 1:12:06Speaker 00

  figure out, you know, there's no question in my mind that like large language models can probably solve a lot of the challenges that they're dealing with. But governments have have to move slow and carefully and most large nationalized health systems are are sort of like careful by nature. They might set up some innovation pockets and stuff, but it's not like they, you know, this is like the core of what they do. And so I'm less optimistic that like this is going to break through in Britain. Britain does have some interesting examples on the mental health side. There's a company called Limbic that's doing a bunch of a mental health triage using kind of large language models and that's a cool model. And so I think there's some opportunity.

• 1:12:51Speaker 00

  The challenge is, is like, as you move from areas where there's like a clear shortage and like mental health is one of these areas where like, nobody's ever going to have the access that they need. And so you just kind of need to find a solution. The problem is when you start to move more into core medicine and it's like, you kind of get into this territory fighting and, you know, elbow jostling kind of thing that I worry that it's going to be harder. But I'd love to be proven wrong.

• 1:13:19Nathan Labenz?

  What do you think is the right way to regulate these, this kind of new paradigm? Like I've, I've heard a little bit around the FDA is maybe going to think about a device regulatory paradigm. Does that make sense to you? Or how would you think about that?

• 1:13:37Speaker 00

  I, I think very clearly the right way is so long as there's doctor oversight, these things should be regulated as decision support. And the doctor is ultimately responsible for making the right decision. And I think longer term, um, these things need to be, you know, if you want one of these things to operate autonomously, it needs to be done much like any other medical device. Like there's a, there's a company called IDXDR. Yeah. Um, and they are like a diabetic retinopathy screening that has like full scale approval to take AI images of like retinal images and then diagnose the patient and come up with a care plan. If I recall, and um, I think it's a good case study where like,

• 1:14:22Speaker 00

  you should be able to show the FDA evidence that you can handle certain kinds of clinical cases effectively. And they get approval to do them autonomously. But there should be a high, there should be a high, you know, bar for evidence because I do think it's dangerous right now. If like in its current state, given what we're, we see like these models are incredibly powerful. But I think releasing them to the average consumer and just letting them practice medicine would be a mistake. So I think right now the way to do it is to start with doctor supervision and then graduate to, hey, we can handle certain kinds of clinical cases autonomously when we get evidence. And then we can kind of go from there. Right now, we don't really know what the ceiling of performance is. Like how close to perfect can these models get?

• 1:15:07Speaker 00

  What does it even mean to be perfect in sort of like a probabilistically uncertain environment? Where we don't really have perfect knowledge of the human body. Those are all questions. And, and the interesting thing is today, the way that's measured in humans, is we measure it based off of, um, what's called standard of care. So basically like if you get sued for malpractice, it's like, what would the average doctor have done? And the issue here is that these models are incredibly good at doing the average thing. And so right now, they probably already meet that bar of like, what would the average doctor have done? Um, and so we, we're gonna have to figure out some like ethical questions about like, is that acceptable? Um, to me, it should be, it should be the same standard.

• 1:15:51Speaker 00

  Because we don't have, like, unlike in the autonomous vehicle world, there's no like perfect driving, right? There's, um, here's, there's, there's, there's so much ambiguity. That I do think we have to compare it to sort of like what's standard of care and what, what's average? And so I think long-term, that's gonna be the question. When you, when you think about regulating these things, it's like, what is sufficient? And I suspect there's gonna be a bunch of lobbying in all directions about like where this ends up. But from, from, from my vantage point, these things have a lot of potential to improve access to care. And if they can replicate what, you know, your median physician is doing, that, that is something that the FDA and others should take seriously and say, hey, we can,

• 1:16:34Speaker 00

  we can totally invert the sort of supply and demand curves of, of medicine in this country.

• 1:16:39Nathan Labenz?

  How do you think about competing against AI-first systems? That just do their best and let the patient decide what to, what to do from there? We'll have a couple of them coming on.

• 1:16:48Speaker 00

  The, the main thing, Eric, is that those systems can't actually like provide the utility to the patient. And the utility is in like actually giving them prescription, modifying their medication, giving them a lab test, interpreting those results. Those are all things that need to be done by a doctor in this country, at least from a regulatory perspective today. So while I expect a lot of people would just go to chat GPT and get a guide, you know, a recommendation. And it says like rest and ice for the next week, great. But if it says, you know, start taking, you know, resus statin, you can't do that for the patient. Um, and so I, you know, ultimately, I think that's, you know, the advantage comes in like completing the job to be done for the patient. Um, and then if you can stick with them over time and do the job, like I said, medicine evolves.

• 1:17:33Speaker 00

  It's not, you're not solving one problem at one moment in time.

• 1:17:37Nathan Labenz?

  You mentioned there's gonna be this like regulatory battle. I'm honestly surprised by how slow that has been to ramp up. And we also kind of talked a little bit about like the possible leapfrogging. I think one really interesting claim that like is starting to maybe be credible now, but.

• 1:17:56Nathan Labenz?

  I don't know, it's like close, right, maybe right on the border is that there's sort of maybe an obligation to deploy these kinds of systems. And you could say like, yes, certainly in the US, like you can't get a prescription filled, you know, without a human doctor's signature, whatever, a lot of places where that's not so true, you know, a lot of places where you can buy anything you want at a pharmacy. Arguably, like, if they're good enough and the alternative is so weak, do you see an argument for just kind of saying, hey, this stuff really ought to be deployed, even if it's still imperfect, even if we haven't figured out all of the, you know, the guardrails, even if we maybe haven't got to a level of safety that would like pass an FDA review, but just internationally, right, the billions of people that don't have the standard that

• 1:18:41Nathan Labenz?

  we have, I could see a pretty compelling case for, we should put that out there now and accept that there will be some downsides and some harms, but like ultimately the benefit maybe dramatically outweighs that.

• 1:18:56Nathan Labenz?

  What's your take on that argument?

• 1:18:58Speaker 00

  I certainly think in parts of the developing world where there's basically zero access to care, the answer is absolutely. There's a guy named Raj Bhajabi, this project called Last Mile Health, and if I recall, what they're doing is they basically go and train like high school graduates in how to do community-based primary care. So you're talking about in parts of Africa where like you have people who are nowhere close to a doctor who are being trained on like, they basically say these are like the 10 most common things you run into here, it's malaria, it's whatever, and we're going to teach you how to practice medicine for these 10 things and like, you don't have to be particularly trained at all.

• 1:19:39Speaker 00

  And, and so in this way, we can expand access to care pretty massively. And, you know, there was a story about Bloomberg training high school graduates to do C-sections, I think as well. I think Bloomberg had worked on that at one point. And it's just an example of like, you're talking about parts of the world where access to care is so poor that these kinds of systems are an absolute must. I do think it goes back to my issue that the real problem to be solved is like, how do you adjust to like the cultural and, and sort of like biomedical? Norms of that region, that is one thing that will need to be solved to do this. But yeah, I think there is a moral imperative to get these things to a lot of people ASAP.

• 1:20:19Nathan Labenz?

  I think that's a great concluding note. I can just give you three kind of real quick rapid fire questions. And you can, you know, give me as as brief of answers as you like on these. First, any AI products that you use beyond like the obvious chat GPT that you would recommend that the audience try out?

• 1:20:39Speaker 00

  Oh, man. I've been using a couple of like, I've been playing around with a couple of these like sheets based things for for spreadsheets, but I'm not like advocating for any of them super strongly. I am increasingly the the ones that I've been playing around with. Like I've been playing around with some of these Y C companies doing like, uh, what I'll call like RPA, like kind of Xavier clones using AI. I think those are super cool. There's one called layup that I liked. I don't have a good answer. I just like, I'm more like playing than I have something in my habits right now.

• 1:21:10Nathan Labenz?

  Honestly, this did not start as a trick question, but you are very, you're in very good company with that answer. And one of the biggest takeaways I've had from this series of conversations has been how there just aren't that many applications that are adding that much value to the core model right now. I do think integration, as you said, with sheets, like that'll make a ton of sense. It'll be way more convenient, you know, when it's in the sheet directly than to like flip over to chat GPT. But

• 1:21:38Speaker 00

  I think the sad thing is I'm trying to learn how to use these things too. Like I have to teach myself. Like I, I know there's some way to look at what I do in a day and be like, this can be GPT eyes and this can't. But the only thing I can do is like, put it into GPT right now. Like, that's all I do. And everything else is like, I'm trying to learn these new products.

• 1:21:59Nathan Labenz?

  You're in good company. We're all learning in real time here together. All right. Second, quick hitter. You hypothetical situation, especially in your take on it, given your medicine lens here. Let's imagine a future world where a million people have the Neuralink implant implanted.

• 1:22:19Nathan Labenz?

  And now it's general availability. If you get one, you have thought to text or thought to UI control. Essentially, you can use your devices and transmit information to your devices straight from your thoughts. Would you be interested in getting one?

• 1:22:38Speaker 00

  I have long said that like the product that I want is the thing that records my thoughts while I'm falling asleep because that's when I have all my best thoughts. There's a particular term, it's like a pedagogic state that your brain gets into. I forget the exact word, but you can actually like, you can coerce your brain to be in this state, which is a separate thing. But

• 1:22:58Speaker 00

  the direct answer to your question is, I don't know. And I think at a certain point, we're start, like, it's very clear to me that in the next 10, 15, 20 years, whenever it happens, we're reaching a point as society where like, life is more about understanding what's like, it will be about one understanding, like, understanding what spiritual fulfillment means to you. And it's not going to be about like optimizing your performance or blah, blah, blah. Or maybe that will be what it is. But like, there's going to be so much infrastructure that cognitively can do so much for us that like, you have to decide, like, what do I want my experience to life in life to be? Do I want to live in VR or the metaverse? Do I want to like, just like go out

• 1:23:40Speaker 00

  in nature and hike? Do I want to like, you know, have some neural implant? Like, I just don't know. And I, I think the sad thing about being entrepreneurs, I probably don't have time to think about those things, even though I'm working directionally on some of them. So my, my, my sad answer is like, I kind of need to know more about like spiritually, what I want out of the next 70 years of my life, be, or maybe it's 170, I don't know with this longevity stuff, before I can say like, yes, I need the implant now. On its surface, like, I want anything that enhances my life. But I think what we're finding is these things.

• 1:24:14Speaker 00

  They have such like, you know, even with our phones, they have these like non like very unpredictable of second order effects on how we live. And you know, it's kind of, it's a little scary in that regard, but I'm also super excited about it.

• 1:24:28Nathan Labenz?

  Well, you kind of anticipate the last question there, which is just zooming out as much as possible. You're just starting to do that. What are your biggest hopes for and fears for society at large as we begin to feel the impacts of AI over the rest of this decade?

• 1:24:45Speaker 00

  Yeah, I'm a pretty staunch capitalist, but I do believe that like we are going to have to figure out how to redistribute wealth or at least redistribute prosperity. It's not even clear to me that like classical economics are going to totally hold up the way that we they have historically. I do think that there's going to be an abundance of wealth created and and I think I'm super optimistic, but

• 1:25:11Speaker 00

  you know, I think this goes wrong if like you turn into like this dystopian, like a few people control the world kind of thing. Or if like we try and stop these things because you won't stop it. People like China and Russia will build these things and you know, I think the world gets dystopian maybe in a different way.

• 1:25:33Speaker 00

  And so for me, you know, if I look out 20, 25 years, like I think the big question for society are like, how are we going to get people to shift from a scarcity mindset to an abundance mindset as we build it? And if we can do that, then we can build a world where people are really happy. And then I think there's a separate question of like, how do we as humans finally shift to a world where like people don't have to work? They don't have to do stuff just to get by. I remember listening to Bill Gates talk at one point, he said like, If the purpose of a human being is to be a hamburger chef, like that's a pretty depressing existence. And so like, you know, a lot of people worried about work displacement and stuff.

• 1:26:15Speaker 00

  And I think there's good reasons because we need everybody to share an economic prosperity. But I don't think anybody aspires, I mean SpongeBob aside, to be like a fast food chef. And so like, I think it's a net good thing. The problem is we don't have answers for what it means and what the implications are for how people should spend their time. So until we figure those things out, and if we don't, we're sort of in trouble. And so that's, that's kind of like what I'm hoping for the next 25 years. People realize there's so much good that's going to come. There's a version of the world in 25 years where like, people could live forever. There's like, energy is free, intelligence is free, like, there's just like, abundance created everywhere. And like, the problem that we're worried about solving

• 1:27:00Speaker 00

  long term is like, how do we then like, expand and colonize more and more places and expand our presence in the universe? And that's like a very exciting version of the world. And people are free to play music and games and socialize and do all the other things that give them joy and meaning in life. But like, to get there is going to require a lot of like, societal restructuring. So personally, I'm really optimistic. And I think compared to the average person in Silicon Valley, I do believe that like, we have to work with existing governments and I want to call like, the public side of the world to make this thing happen. Otherwise, we're just going to build these, these things in isolation. They're going to be more destructive than anything. Which is why I kind of have the perspective I do

• 1:27:43Speaker 00

  on regulation, which is like, yes, I think if we, if we run, let AI run wild in medicine, it's going to be a problem. Otherwise, we're really gonna, we could really benefit from this stuff. We collaborate. I just talked for way too long. I rambled a lot today, but guys, this was awesome.

• 1:27:59Nathan Labenz?

  We love it. Neil Kosa, thank you for being part of the cognitive revolution.