Peter Lee on the future of health and medicine

What if every doctor and every patient had an artificial intelligence copilot to help with medical decisions? Peter Lee joins the show to offer his expertise as co-author of The AI Revolution in Medicine: GPT-4 and Beyond and Corporate Vice President of Research and Incubations at Microsoft. Peter is working at the cutting edge of how to safely incorporate AI into medicine, training healthcare providers on the strengths and limitations of technology like GPT-4 in a medical setting. Reid, Aria, and Peter discuss how to regulate AI while encouraging innovation. Plus, Peter provides a fascinating live demonstration of GPT-4’s ability to listen to a patient’s symptoms and provide a second opinion, illustrating AI’s potential to bridge healthcare gaps and revolutionize care. Read the transcript of this episode here. Read Peter’s book here. For more info on the podcast and transcripts of all of the episodes, visit www.possible.fm/podcast. Topics: 03:47 - Hellos and intros 04:16 - Mapping Peter’s career 07:00 - How patients can use AI in healthcare 08:54 - Peter’s personal experience using AI in healthcare 11:35 - Demo of GPT-4 as a physician’s copilot 15:30 - How best to educate about GPT-4’s capabilities 19:33 - Demo of GPT-4’s “critical thinking” skills 26:57 - How to safely incentivize innovation 30:37 - The importance of building effective technology around AI 31:47 - Exploring “hallucinations” in AI, and other limitations 35:28 - How to best use partnerships to foster innovation 38:21 - Example of new technology using AI in healthcare 40:37 - Rapidfire questions The award-winning Possible podcast is back with a new season that sketches out the brightest version of the future—and what it will take to get there. Most of all, it asks: what if, in the future, everything breaks humanity's way? Tune in for grounded and speculative takes on how technology—and, in particular, AI—is inspiring change and transforming the future. This season, hosts Reid Hoffman and Aria Finger are speaking with a new set of ambitious builders and deep thinkers about everything from art to geopolitics, and from healthcare to education. These conversations also showcase another kind of guest: AI. Whether it's Inflection’s Pi, OpenAI’s ChatGPT, or other AI tools, each episode will use AI to enhance and advance our discussion. Possible is produced by Wonder Media Network and hosted by Reid Hoffman and Aria Finger. Our showrunner is Shaun Young. Possible is produced by Edie Allard, Sara Schleede, and Paloma Moreno Jimenez. Jenny Kaplan is our executive producer and editor. Special thanks to Katie Sanders, Surya Yalamanchili, Saida Sapieva, Ian Alas, Greg Beato, Ben Relles, Katie Zoller, Katy Halliday, Rhoades Clark, and Little Monster Media Company.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Speaker 00

  At some point, it just makes sense that of course you would never practice medicine without washing your hands first. And we will definitively get to a point, probably sooner than we think, where of course you would never ever dream of practicing medicine without the assistance of AI.

• 0:30Reid Hoffman?

  Hi, I'm Reid Hoffman.

• 0:31Aria Finger?

  And I'm Aria Finger.

• 0:33Reid Hoffman?

  We want to know what happens if in the future, everything breaks humanity's way.

• 0:38Aria Finger?

  We're speaking with visionaries in many fields, from art to geopolitics and from healthcare to education.

• 0:46Reid Hoffman?

  These conversations showcase another kind of guest. Whether it's Inflection's Pi or OpenAI's ChatGPT, each episode, we use AI to enhance and advance our discussion.

• 0:58Aria Finger?

  In each episode, we seek out the brightest version of the future and learn what it'll take to get there.

• 1:05Reid Hoffman?

  This is possible.

• 1:13Reid Hoffman?

  Last episode, we spoke with Siddhartha Mukherjee about the future of medicine, disease, cutting-edge therapies, and some philosophy too.

• 1:22Aria Finger?

  It was a fascinating chat and the topic of healthcare is so wide-ranging. We just had to double down. Today, we are diving deeper into how AI might revolutionize health and medicine in the future.

• 1:37Reid Hoffman?

  GPT-4 is already being used by doctors to great effect and it has so much potential to bridge gaps in care for patients as well. That's especially true for people who may not have easy or consistent access to medical care. Wouldn't we say that some support in these cases, of course, with the right guidance and guardrails, is better than no support at all?

• 2:00Aria Finger?

  But as we realize the potential of a tool like GPT-4, it needs to be done carefully. So how do we look at a technology like GPT-4 with all the possibilities there and use it correctly? Our guest today is the one to ask.

• 2:17Reid Hoffman?

  Peter Lee is a renowned computer scientist, currently serving as the Corporate Vice President of Research and Incubations at Microsoft. He co-authored the book, "The AI Revolution in Medicine: GPT-4 and Beyond." He also serves on the Board of Directors for the Allen Institute for Artificial Intelligence, the Brotman-Beatty Institute for Precision Medicine, and the Kaiser Permanente Bernard J. Python School of Medicine.

• 2:42Aria Finger?

  So we sat down and talked to Peter Lee about the future of medicine. But what I think was so great in talking to Peter today is that we actually don't even need to look into the future. We can talk about right now. Peter is talking about how he is using GPT-4 in his own personal medical issues with his family, with himself, but also how doctors, again, today, are using GPT-4 and AI to make them better and smarter. And I think the other thing that Peter does a great job at illuminating for people is that we all need to understand how to use this tool. You're not going to use the calculator for spell check. I think a lot of times people say, "Oh, well, GPT-4 can't say exactly to me

• 3:26Aria Finger?

  what the first paragraph of that book is." Well, no, because that's not how you use it. And so when the medical establishment can learn better how to use it, there's going to be more positive outcomes for everyone.

• 3:41Aria Finger?

  And so here's our conversation with Peter Lee.

• 3:47Reid Hoffman?

  We'd like to trace your professional bio in a somewhat atypical way. You said that there's a number of times in your career you've seen moonshot research jump to the consumers' pockets. Can you recount the story of your career in those instances from where you count your early work to today?

• 4:06Speaker 00

  My career and the actual technical contributions I've made in my career have been more marked by what happens in the background, in the foundations. If I think about the major moments, it's always been a matter of me being asked to do things, sometimes against my will, and me being frankly too spineless to say no. And luckily, it's been smart people who've asked me to do things. So after the 2008 election, my friend Tom Collier was appointed the Science and Technology Policy Director for the Obama White House, and he leaned on all of his friends. I happened to be one of his friends to write two-page position papers, and so he asked me to write two pages on what to do with DARPA.

• 4:50Speaker 00

  Of course, as soon as everyone took office, I was asked to go abandon my professorship at Carnegie Mellon and serve at DARPA. That ended up being a transformational, serious growth experience for me. That first week, I had to meet with the Secretary of Defense, and I had no idea what to tell him. I was driving down from Pittsburgh down to DC, and I got nabbed for speeding. And then I thought, with these trips, I'm going to lose my license. And so I downloaded Trapster on my phone, and then I realized, I know what I'm going to talk about to the Secretary of Defense. I'm going to talk about how crowdsourcing and network effects and machine learning

• 5:35Speaker 00

  can really make a difference. And it's been a series of accidents like that. I was never medically trained. And then I was on a special assignment, working on sort of an internal incubator for Satya Nadella and Harry Shum at Microsoft. And then in 2016, I was supposed to come back to research. And then Satya Nadella says, no, I need you to work on healthcare. I thought I was being punished for something. I thought, what do I know? And furthermore, there are 12 other corporate vice presidents at Microsoft that are doing their own thing in healthcare. They're not going to listen to Peter Lee. But ultimately, you sort of overcome these things. You get immersed and you start understanding,

• 6:20Speaker 00

  wow, there's some really important stuff to do here. And you get a little bit lucky. I just had the good fortune that really smart people saw things in the future and asked me to climb on board. And whether I was smart enough to say yes or too spineless to say no, I ended up being in the good spot.

• 6:41Aria Finger?

  Well, I think that's a lesson we all learn over and over again, that so many people who are doing amazing things, it's not as if you plotted out and this is how I will get to the top of Microsoft. Like you said, you just were working with great people doing incredible work. And so that brings us to today, to healthcare and AI. And so your relationship with AI health and medicine hasn't just been academic. And I understand that you've used GPT-4 in your own life to better understand the medical options available to you and your family. How do you use GPT-4 as a tool in that role? Less in the medical establishment, but more in your own life?

• 7:18Speaker 00

  You can receive, you get a physical exam and then maybe a week later in email, you get a PDF file with all of your blood test results. And I don't know about you, but it's gobbledygook. Like, I'm even an elected member of the National Academy of Medicine and I can't decode these things. And you don't feel good about calling your doctor to waste his or her time with this. It's empowering to just ask GPT-4. I do this through Bing, but you can also use ChatGPT. Look at this and tell me, explain this to me. Is there anything I should be concerned about? It looks like my LDL is a little out of range. What does that mean? And so on.

• 8:00Speaker 00

  Also, you get these, if you have health insurance, you get this weird thing called the explanation of benefits notice. I always felt bad that I could not make heads or tails of these things, but I've since learned that even executive, CCB executives at insurance companies, they don't understand these things either. And again, it's just incredibly powerful and empowering to ask.

• 8:25Speaker 00

  Bing or GPT-4, look at this, explain this to me, do I owe money? Why am I getting this notice? You know, what, what happened? And having the AI, having GPT-4, you know, look at those CPT codes in those explanation benefits notices and tell you, oh, someone in your family, you know, had this lab test done, and this much was covered by insurance, this much is being covered by your provider, then you don't owe money. Things like that are empowering. The thing that I've been less public about, about four months ago, my father passed away after a very long illness. He was ill for about 18 months. And it was his time to go and he went very peacefully.

• 9:09Speaker 00

  But, but it was a struggle for over a year and a half. And part of the struggle is that my two sisters and I were all trying to provide for his care, support his care, but we all lived hundreds of miles away. You know, what happens is you have a primary care physician, you have two specialists, many months, lots of lab tests, lots of other information you collect. And the pressures on family members, on siblings, get to the point where relationships start to fray. And the typical flare up point is, um, you know, we finally get that golden 15 minute phone call appointment with specialist Dr. K.

• 9:51Speaker 00

  And an argument erupts between me and my two sisters on how to best use that 15 minutes. What I've learned is this is just such a common experience that people are going through.

• 10:04Speaker 00

  And so to just give all of the lab test results, all the other notes, to explain in writing the current situation, and then say to GPT-4, we have a 15 minute phone call with Dr. K. What are the best three things to ask and talk about? And it, it just brings the temperature down. Makes us feel more certain and empowered in the conversation with Dr. K. It actually helps Dr. K because she doesn't feel like we're wasting her time and being random. And, you know, it preserves the measure of family harmony in things. And so, you know,

• 10:45Speaker 00

  when you think about that, I can do that because, you know, I have access to technology and I'm a techie and I'm comfortable with using these things. Do I expect the normal consumer to just use ChatGPT in that way, even though ChatGPT is pretty easy to use? Maybe. But what I really hope is that there will be some great companies and ventures that will start to make customized experiences powered by technologies like GPT-4, you know, that can really help. You know, we have this silver tsunami going on in North America, in Europe, in Asia. This is just, I think, going to be a normal problem that everyone is going to have to cope with.

• 11:30Aria Finger?

  Now, I would love to test out GPT-4 and how it can be used in your field. Can you give us a demo about how it can be used as a copilot for doctors?

• 11:41Speaker 00

  Yeah. So the thing that's been discovered is that GPT-4, without any kind of specialization, is extremely good at answering these types of what are called curbside consultation questions. Curbside consults are the kinds of questions that doctors ask each other. And this happens all the time, multiple times a day, particularly in general practice. Let's try one. Hi, GPT-4. I'm a doctor. I have a patient with some chest pain, elevated heart rate, but normal EKG. I've just measured and he has elevated troponin. What else should I be looking for?

• 12:23Speaker 03

  Hey, Peter. Although I should emphasize that I'm not a healthcare professional, elevated troponin levels with chest pain and an elevated heart rate could potentially point to a cardiac event like a heart attack. In this situation, you might want to consider additional tests like coronary angiography to check for arterial blockages, or an echocardiogram to evaluate heart function. Now, you're probably aware that other factors like kidney issues or muscle damage can also raise troponin levels. So having a complete clinical picture is crucial. Definitely consult with specialists to corroborate your findings. What's your current game plan for diagnosis and treatment?

• 13:05Speaker 00

  Well, I'll definitely order an angiogram. Please let me know the urgency of that. Also, one thing I forgot to mention before is I did listen to his lungs and I do hear some evidence of RALs.

• 13:18Speaker 03

  Given the elevated troponin, chest pain, and elevated heart rate, I'd say the angiogram should be conducted as soon as possible to rule out or confirm coronary artery disease. Time is of the essence in potential cardiac events. The presence of RALs in the lungs could suggest fluid accumulation, possibly indicative of heart failure or another condition like pneumonia. In light of this, you might also consider a BNP test to assess heart failure, or even a chest x-ray to get a better idea of what's going on in the lungs. So, all things considered, I'd say you've got multiple reasons to act urgently here. How are you planning to proceed?

• 13:57Speaker 00

  Thanks. In fact, I think we can get that x-ray ordered at the same time as the angiogram, and I think that will save a big step in the patient's treatment, so thank you. So that's a simple type of example, and these are things that doctors consult with each other all the time, but the access to that kind of information is oftentimes not present. You know, when a doctor gets the initial presentation of a patient and some labs, initial lab test results, a doctor will develop what's called a differential diagnosis. Differential diagnosis is the list of possible causes for the patient's maladies with probabilities assigned to each one.

• 14:42Speaker 00

  A great use of GPT-4 is to give all that information to GPT-4 and say, "Take a second look at this. Is there anything I've missed? Anything else I should consider?" We find that GPT-4 is almost superhuman at being able to critique the work that people do and be that second set of eyes.

• 15:03Reid Hoffman?

  Yeah, thousand percent. One of the funny things is when I was running to getting impromptu published, you know, one of the things we talked about was whether or not we should put a medicine chapter in impromptu. Because I knew your AI revolution in medicine was coming and so much better than what I would do. I was like, well, let's not have a medical, let's not have a medicine chapter. There's a much better book coming on this. So my team and I enjoyed the book, obviously. So you use a fictional story to start. So what's your take on how fiction can be used to help or harm our understanding of the future AI? And how would that affect, you know, consumers in the home and the silver tsunami and kids and families and mothers?

• 15:47Reid Hoffman?

  What's your, what's your take on that? I thought the use of fiction was great.

• 15:51Speaker 00

  You know, it, maybe the story about why we wrote a book to begin with. This was, I think it was in August of 2022, when I was first exposed to GPT-4 and the assignment to understand the possible implications to healthcare started. In November, when ChatGPT was released and it sort of blew up, we started to get some emails. I remember one email that we got about three or four days later was from a friend who runs a health clinic. And he said, oh, this ChatGPT stuff is great.

• 16:32Speaker 00

  We're using it now to write after-visit summary emails to our patients. And so I thought, wow, that's really cool, but wow, that's really concerning also. By November, we were starting to understand the tremendous possibilities, incredible capabilities of GPT-4. But also, we're starting to understand that there are some issues like hallucination. If you think of a computer as a machine that does perfect calculation and perfect memory recall, ChatGPT and GPT-4 are not computers. There's something different. There is some kind of reasoning engine. And the immediate worry, especially if you work at a company that's very careful about

• 17:16Speaker 00

  these things like Microsoft, is that, oh, if doctors are going to start assuming that this thing is a computer. Those assumptions are going to get violated in ways that could be harmful to people. And so the question was, how do we best educate the medical community about what this technology is? After asking and consulting with a lot of people, we learned that the large part of the core of the medical community still reads books. So then you have to decide, well, what are you going to try to teach people in that book? And there are several key lessons, but one key lesson is that this is probably unstoppable

• 17:58Speaker 00

  because it's going to be a technology that's in every person's pocket. How do you tell that lesson? Since it was such a new technology, we felt that the only way to do it was to make a fictional account that was realistic, you know, something that would actually happen in an emergency department. And you might have an inexperienced medical resident who just reaches into her pocket to get a second opinion about something. Oh, and by the way, also becomes totally dependent on ChatGPT even to monitor her own health and her own daily schedule. And it was that sort of idea. The other issue is that my writing style and

• 18:39Speaker 00

  Zach's are hopelessly academic. So we were co-writing this book, but it would be dry beyond belief. And so Zach had a colleague, Carrie Goldberg, who is a much, much better writer. She joined the project in order to turn it into a kind of a page turner. Carrie, being a writer, also injected the idea of a fictional account and it ended up being a really fun, fun thing to do.

• 19:06Aria Finger?

  I love it. And it's also just so interesting with the different strengths that you bring together to create this book. And, you know, in the book, you make these sort of three major points. It's like, of course, GPT-4 is a game changer for medicine and health, that it poses risks if we don't begin wide scale testing and understand its limits, and that also we should be working as quickly as possible to ensure broad access to the technology, which we have been talking about a lot. We need to get this in everyone's hands as soon as possible. So we're going to get to all of these points, but I want to start with the actual technology first. You compare it to Wilbur from Charlotte's Web. Can you tell us a little bit more about that comparison?

• 19:45Speaker 00

  So I've been on the road a lot for the last several months talking to leaders of healthcare organizations. So those are leaders of hospitals and health systems, of health insurance companies, people at the National Academy of Medicine and major medical schools. And, you know, one of the first things, which I know Ari, you and Reid have also run into is just how to explain what the heck is this technology anyway? And so one of the things I do, which has turned out to be effective, but is a little silly, is I ask people, have they read the book Charlotte's Web?

• 20:25Speaker 00

  And if you ask a roomful of people that, at least in the United States, everyone raises their hand. And so I say, okay, I'm going to ask you three questions for you to prove to me that you're not lying. And so the first question is a simple question, which is, can you briefly describe one of the main characters of Charlotte's Web? And so everyone can do that. You just mentioned Wilbur. There's also Charlotte, of course, the spider, this Fern, the girl who saves Wilbur from getting slaughtered. So I make the point that some of you haven't read the book in more than 20, 30, 40 years. And when you read it, you distilled into the neural circuitry

• 21:08Speaker 00

  of the flesh in your skull, those facts about the book. And if you think about what your brain is doing, it's almost unbelievable that that can happen. And now you can start to give a lesson about GPT-4 and say, well, GPT-4 has read Charlotte's Web also. And GPT-4, in reading Charlotte's Web, has distilled into its neural circuitry facts about Charlotte's Web. And so let's test it. So I'm going to use GPT-4 on my phone here and ask this question. Hi, GPT-4. Have you read the book Charlotte's Web? And if so, can you describe one of the main characters? And please be very brief, since we're on a podcast.

• 21:50Speaker 03

  Hey Peter. Yep, I'm familiar with Charlotte's Web. One of the main characters is Charlotte, a wise and caring spider who helps save a pig named Wilbur from being slaughtered. She's the brains behind the "some pig" message in her web. How's the podcast going?

• 22:05Speaker 00

  All right, so now, maybe it's not so impressive that a computer can remember that. So let's ask a second question, which requires much more intelligence. And that is the question, what do you think the book Charlotte's Web tries to teach us about the value and nature of friendship? And that's a really hard question, because it's not answered by any words in that book. You have to read between the lines, you have to make connections to personal experience, to social context. And so what about GPT-4? Let's try it. So what do you think that book tries to teach us about the nature and value of friendship?

• 22:43Speaker 03

  The book emphasizes that friendship can be transformative. It's not just about companionship, it's about sacrifice, loyalty, and making each other's lives better. Charlotte literally spins webs to save Wilbur, and in doing so, teaches us the depth a friendship can reach. What's your take on it?

• 23:01Speaker 00

  All right, so now, I tell people, you need to be skeptical of all this. For sure, thousands, maybe millions of children in school have written essays about the meaning of Charlotte's Web. They've probably posted them on the internet, and during GPT-4's training, GPT-4 probably read those essays, and is just simply regurgitating those ideas from schoolchildren, from teachers, from other writers. So, of course, it's not anything special. Those are not original thoughts. That's a skepticism. What is really going on here? What the heck? Are these original thoughts? Is it really thinking, or what? And one of the ways to test that is to ask a question that has never been asked before, and to assess the originality of the responses.

• 23:44Speaker 00

  So, thinking about this, I looked at your bio, and I know you're connected with, you led an organization called DoSomething.org. And so let's see if GPT-4 can make connections with that. So, GPT-4, I'm on a podcast with Reid Hoffman and Aria Finger, and Arya was formerly leading an organization called DoSomething.org. Do you see connections between the lessons about friendship in Charlotte's Web and the goals and mission of DoSomething.org?

• 24:15Speaker 03

  Absolutely, there's a connecting thread here. DoesSomething.org is all about empowering young people to take action, often for the betterment of their communities. Similarly, the friendship between Charlotte and Wilbur brings about positive change, saving Wilbur's life and inspiring the community around them. Both emphasize the idea that individuals, even if they're just a spider or a young person, can make significant impacts through their actions and relationships. How do Reid and Aria view this connection?

• 24:44Speaker 00

  Or Aria specifically?

• 24:46Aria Finger?

  I don't, I don't think there's many essays on the web about the connection between DoesSomething.org and Charlotte's web, so, so Peter, I think you got us.

• 24:53Speaker 00

  Yeah, and now your skepticism has to start getting shaken and you know there's something deeper in that. Now, the last question that I ask people is this, please recite chapter one word for word.

• 25:05Reid Hoffman?

  Yes, no chance.

• 25:06Speaker 00

  And I haven't found a single human being, I'm sure they exist out there, but I haven't found anyone that's able to do that. And that's because the human brain, as amazing as it is, does have some significant limitations. One of its limitations is in rote memorization. And oddly, today's GPT-4 has analogous limitations. And I won't do it here, but if you ask GPT-4 to recite chapter one, today, I think it'll demure and say, well, I shouldn't do that for copyrighted material, so no, I won't do it. And so this is where I get to tell a leader of a healthcare organization. So you see what we have here is a brand new tool.

• 25:47Speaker 00

  It's not a computer in the sense of being perfect rote memorization machine or a perfect calculation machine. It's a reasoning machine of the type we haven't seen before. And if you, as a doctor, tried to use this thing under the assumptions that it's a computer, you can get into trouble. At the same time, it's incredible, the capabilities that it has. And so the very first lesson that I tried to give to clinicians, to doctors, nurses, and leaders, is based on this story to start you down the path of understanding just how new a tool this is. It's a new tool that we have to figure out how to use, for which we don't yet have the

• 26:32Speaker 00

  user manual.

• 26:33Reid Hoffman?

  One of the challenges usually in regulation is that it tends to mammothly restrict

• 26:40Reid Hoffman?

  The speed and delivery of certain kinds of innovation. And in some areas of medicine, that's really important. In some areas, that's restrictive. If we could kind of like wave a wand and kind of get the right balance on regulation innovation here. What do you think the the right kind of intelligent way of kind of regulation, shaping incentives, getting innovation, protecting patient safety? What's like, what would be your your recommendation of a path forward there?

• 27:13Speaker 00

  Roughly speaking, people today, regulators, really don't know what to do. There are some very significant efforts. The National Academy of Medicine has a major effort to define a code of conduct for the use of AI in medicine. But the thing that's interesting about this kind of technology is it doesn't follow the pattern of any medical technology today. So the standard framework of a software medical device doesn't really hold up here. And on top of that, the whole world understands that this is incredibly empowering. It's also a matter of national competitiveness.

• 27:55Speaker 00

  So there is no regulator anywhere in the world that wants to impose regulations that might stand in the way or impede overall competitiveness of their of their nation. And so as we think about these things, my own conclusion is that the medical community itself really needs to assertively take control of the questions of whether, when and how this technology should be adopted and define the foundations on which future regulations would be built. And for that to happen, it's so important for us as a technology community to do everything we can to educate and get the medical world up to speed. What's going to happen, I I'm certain, like today, when we think about disclosures,

• 28:40Speaker 00

  there is a major question, you know, should be a disclosure to patients if a doctor uses AI to assist, let's say, in a diagnosis or developing a treatment option. I'm certain in the very small number of years, the disclosure will work the other way. A patient will demand to know. If a doctor doesn't use AI to assist and double check something and will demand to want to know why. And so there's a difference between what makes sense today in 2023 and what will make sense in the future. And when I say future, the distant future in AI is like 2025. Okay.

• 29:20Speaker 00

  And so that to me means we probably, just to be practical, have to think about some interim code of conduct or interim set of guidelines or regulations that have a very set timeline. The analogy that I give sometimes to people is it's like hand washing. At some point, it just makes sense that of course you would never practice medicine without washing your hands first. And we will definitively get to a point, probably sooner than we think, where of course you would never ever dream of practicing medicine without the assistance of AI.

• 30:00Speaker 00

  We do need something because there is a fundamental lack of understanding of the capability of these systems and also their failure modes. I think there is emerging regulatory requirements that there should be humans in the loop, you know, to finalize or make final decisions, that the humans are still accountable if those decisions are wrong or negligent in any way. And that there would be the development of codes of conduct and guidelines, you know, by organizations like the National Academies, the American Medical Association and so on. One more thing that I've used with doctors and regulators is this analogy.

• 30:42Speaker 00

  It's a little bit of a silly analogy, but to imagine we've just invented copper wire and we've discovered that copper wire can efficiently carry electricity from point A to point B. And we just know in our guts, this is going to change everything for the better. We can't even imagine, you know, carrying digital signals or winding it around pieces of metal and making electric motors. We just know, just even just carrying electricity is going to be big. But we haven't yet invented the light bulb. So we don't know why. And meanwhile, people are getting electrocuted left and right and crazy people are putting their tongues on this thing and so on. But the point here is that it's up to the medical community to invent those light bulbs.

• 31:25Speaker 00

  It's up to us as technologists to invent the light bulbs, platform providers to do that. And as we do that, we'll start to understand the importance of this more broadly. And the need for electrical codes and other regulations, the ideas of insulation. There's all sorts of things that get developed. And so we're just in that early stage.

• 31:47Reid Hoffman?

  I think we should probably address one of the, you know, natural questions like of a skeptic. We're listening to our conversation. It's a, you know, look, I've heard about this AI stuff doing hallucinations. What does that mean? Like if I'm, if I am gonna, even if I don't have access to a doctor, what happens if I get a hallucination? And what does that mean for like bioethics and, you know, kind of, you know, I think you've talked about the nine steps of enlightenment as it comes to AI and medicine. What's a way that we navigate that? And we, cause obviously there's both improvements in the technology, there's humans, there's regulation. Like what are, what are our steps forward? And how do we think about that ethically? How do we think about hallucinations?

• 32:31Speaker 00

  Yeah, I know I listened to your conversation that you had on this podcast with Ethan Mollick. And I like the way that Ethan describes G P D four as the right mindset is to view it as your intern, your personal intern. And I think that's really apt because I use the term reasoning engine, but who the heck knows what that means? Thinking about having your own intern is just so much more relatable. And it's this thing where, you know, if you're going to have your personal intern and you're going to have a conversation about your medical issues, you're going to have to make your own assessments about the correctness and veracity of these things.

• 33:15Speaker 00

  You're going to have to probe, you might have to ask for further consultations and so on. And yet having your personal intern is an undeniably useful thing. It's something that really amplifies what you're able to do in life. And I think that that sort of mindset. That kind of understanding is really fundamental. One last thing to say, companies like OpenAI have been exceptionally cautious about the safety and responsible AI issues of this. And so for the bulk of GPT-4's public life, it's been put in a glass box, not allowed to touch anything, not allowed even to read the internet. It's just, you know, it's only what in the last six months that, you know, it's been

• 33:59Speaker 00

  given permission to do things like use the Bing search engine, or, you know, use your camera to see images in the world, or very recently with advanced data analytics, been given the permission to write small amounts of Python code and execute it as it sees fit. Those kinds of things relate to the AI concept of grounding, of allowing the AI to use tools to ground its answers in some reality. And I think in the very near future, what I expect is that these grounding techniques

• 34:37Speaker 00

  and this ability of the permission for AI systems to use tools will dramatically reduce the bad hallucinations that we see out of these systems while preserving the good hallucinations. And by good hallucinations, I mean things like informed guesses and the ability to imagine. It's just like giving your intern access to a library, to encyclopedias, to the internet, and to a calculator. You want to do those things, and those are capabilities that are only now coming online.

• 35:11Aria Finger?

  You know, you put a lot of onus on us in getting to this better future. You know, you say that there are a lot of doctors aren't using GPT-4 right now, a lot of consumers aren't using it. It's us to, it's up to us to sort of build the light bulbs that is going to create this better future. And so, you know, Microsoft is working with Mercy to figure out the when, the where, and how AI can be used in medicine. Can you talk a little bit more about the relationship specifically and like how you view partnerships, such as with Mercy or Epic, in the future of medicine to help us get to that sort of better future?

• 35:47Speaker 00

  Yeah. Again, going back to this copper wire analogy, okay, so we have this copper wire, what do you do with it? You know, what do you do with the fact that you can carry electricity around? I've felt it is actual healthcare systems like Mercy or healthcare technology providers like Epic. So Mercy is a very large and very progressive health system in the Southeastern US. But interestingly, they have very significant in-house engineering capability. They have been very scrappy and very smart in migrating all of their data to the cloud and building a lot of tools and so on, and they've shown a lot of capacity there.

• 36:28Speaker 00

  So I think they have the ability to invent some light bulbs and toaster ovens and telephones and things like that. Epic is the largest provider of electronic health record systems. Of course, as a health tech platform provider, they also are basically a big engineering house. They build software. That software is used all day long by millions of doctors and nurses. It's also, you know, their systems are the subject of a lot of ire because doctors and nurses don't like spending all their time staring at an Epic screen as opposed to, you know, dealing with patients. And so there's tremendous motivation to invent those light bulbs,

• 37:13Speaker 00

  both at Mercy and at Epic. Mercy is facing a crisis of a huge nursing shortage. I think in the US today, there's estimated over 50,000 nurses short right now in the US today, and that's supposed to get potentially into the millions within five years at the current trajectory. And so just finding ways to attract more young people into the profession, also making people more productive is extremely, extremely important. And so working with these organizations that have their own engineering capacity, that can work and collaborate and co-innovate with us

• 37:56Speaker 00

  in Microsoft Research and in other parts of Microsoft, I've found that to be the most productive way forward. We also acquired a couple of years ago, a health tech company of our own, Nuance Communications. And Nuance makes the most popular medical dictation system. It's the way that doctors can talk to a computer in order to enter their clinical documentation. All of these companies, Mercy, Nuance, Epic, and a whole lot of others are right now doing some incredibly great work to integrate GPT-4 in new ways into their existing products. Epic, for example, they have a product called Inbox.

• 38:40Speaker 00

  It's basically an email system set up so that a doctor can send notes to patients. But it has some intelligence to take information from recent encounters between the doctors and patients and all the clinical histories and so on. Doctors complain about Inbox all the time because they say it's just overwhelming. And it's a lot of work to kind of rummage through five, 10, 15, sometimes upwards of 30 or more documents spread all over the Epic system in order to synthesize a useful, friendly note to a patient.

• 39:19Speaker 00

  GPT-4 is now being used in Inbox to read all of those things and propose a draft note to send to a patient. A doctor can then look at that, edit it, and send it off. Judy Faulkner, the CEO of Epic, reported that in early studies, patients are preferring the notes generated by GPT-4 and have explicitly said they feel more human than the notes written by the doctors. What they're finding is that a doctor is too busy to add those personal touches. Like, congratulations on just becoming a grandparent. Vast wishes on your wedding next month in France. Those extra little personal touches, GPT-4 just has the tireless ability to extract those things

• 40:04Speaker 00

  out of the conversations between doctor and patient, connect the dots between that and the clinical history of the patient, and it makes a real difference. And it dramatically reduces by multiple hours a day the clerical burden on doctors and nurses. So these kinds of things are just emerging right now. I just get incredibly charged up and excited about this because...

• 40:29Speaker 00

  These are the first light bulbs that are coming out.

• 40:33Reid Hoffman?

  That are coming on from the copper wire to extend the metaphor.

• 40:43Reid Hoffman?

  Should we go to rapid fire? All right. Is there a movie, song, or book that fills you with optimism for the future?

• 40:52Speaker 00

  There's a song that's been on my mind that my son introduced me to. My son, over a year ago, when we were really delving into the capabilities of GPT-4, one of the things that my son had suggested to me, a song lyric for the song Aquamarine. So the lyric is by Black Thought and the production of the music is by Danger Mouse. So Black Thought is the emcee for Roots. And the lyrics, I gave it to GPT-4. I had GPT-4 doing the analysis and the analysis is incredible. And so that's in chapter three of my book. But since then, that sucked me into Black Thought and his poetry.

• 41:35Speaker 00

  And the poetry of Aquamarine is particularly incredible and is inspiring in this way because it's about transformation. And so one of the things I invite people to do is to download the lyrics for Aquamarine and have your own conversation with GPT-4 about it and ask questions. And I think I'm certain you'll be blown away. But not only by GPT-4, but you'll be equally blown away by the lyrical genius of Black Thought.

• 42:08Aria Finger?

  By the lyrics, I love that, Peter. What is a question that you wish people would ask you more often?

• 42:15Speaker 00

  I think the answer that I love getting into a conversation about that. So maybe it's the one I wish people would ask is, in our conversation and what we're doing right now, how much of what we're doing right now is more than just next word prediction? And I've just started to wonder, I still feel in my guts that what we're doing when we're conversing like this is something very magnificent and complex. That we're doing all sorts of incredible computation in our brains. But maybe 99% of it is just, is just next word prediction.

• 42:52Aria Finger?

  I mean, if next word prediction can get you GPT-4, it's it's pretty miraculous. So I think that's that's a really interesting question to ask.

• 42:59Reid Hoffman?

  Yeah, I think we're actually, I'm with you. We're doing a lot more than next word prediction, but it's still amazing what next word prediction gives you. Right. So where do you see progress or momentum outside of your industry that inspires you?

• 43:12Speaker 00

  The obvious thing is in has to do with energy and our insatiable appetite for energy and the environment. And it's hard not to be swept away by the optimism that more and more of us are feeling about the making fusion energy more real. But maybe a more interesting answer, just to stick with the medical domain, there's been this long term dream, sometimes it's referred to as real world evidence, RWE. But what it really amounts to is, wouldn't it be great if every single healthcare experience that people have, you know, at home, in life,

• 43:57Speaker 00

  in encounters with their doctors, if all of that somehow contributed to the advancement, directly contributed to the advancement of new medical knowledge? The example that came out of COVID was around the end of 2020, you know, when the COVID pandemic was really, really putting people into severe respiratory distress. Doctors around the world were accidentally discovering that sometimes COVID patients in respiratory distress could avoid intubation if they kept the patient prone, you know, had them on their stomachs. And there were a few of those doctors that were starting to share that information on

• 44:37Speaker 00

  social media, like Twitter and Facebook. But it wasn't until the end of 2021 that a multi-institution, multinational clinical trial verified that that was true. Then it got officially integrated into the practice of medicine. And of course, by the end of 2021, it was almost too late because the virus had mutated to the point where it wasn't putting as many people into respiratory distress as earlier. In the future with AI, we ought to be able to do this routinely, that every single clinical encounter and every single medical experience should be synthesizable into a clinically sound and validated piece of medical knowledge and

• 45:22Speaker 00

  practice. I think we actually are very close to having the tools that we need to make that a reality.

• 45:29Aria Finger?

  I love that. So Peter, can you leave us with a final thought on what you think is possible to achieve if everything breaks humanity's way in the next 15 years? And what's our first step to get in that direction?

• 45:42Speaker 00

  In the space that we inhabit as technologists, what we see all around us are just incredible possibilities. I've never in my life been in a context with so much optimism about what can happen in technology. And so I know it seems dire, like climate catastrophe seems dire. I actually think that the combination of new rapid advances in our ability to use AI to model complex physical systems, fusion energy becoming a reality, I think we're going to avert climate catastrophe.

• 46:25Speaker 00

  And as a result of that, have a foundation to drive all of our technologies that is going to make much, much bigger part of the world, more prosperous.

• 46:36Aria Finger?

  Well, Peter, we love talking to technologists like you because you are helping to build that future. So thank you so much.

• 46:43Reid Hoffman?

  Yeah, Peter, awesome as always.

• 46:45Speaker 00

  Thank you for that.

• 46:50Reid Hoffman?

  Possible is produced by Wondra Media Network, hosted by Ari Finger and me, Reid Hoffman. Our showrunner is Sean Young. Possible is produced by Edie Allard, Sarah Shleid, and Paloma Moreno Jimenez. Jenny Kaplan is our executive producer and editor.

• 47:06Aria Finger?

  Special thanks to Katie Sanders, Surya Yalamanchili, Saida Sapieva, Ian Alice, Greg Beato, and Ben Relis. We'd also like to thank Katie Zoller, Katie Halliday, Rhodes Clark, and Little Monster Media Company.