Latent Space: The AI Engineer Podcast
Hosted by Alessio Fanelli
Watch the full show on YouTube: https://www.youtube.com/watch?v=ujjyNjQ5Cfo
5,746 words · 3 speakers
Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
Hi, everyone. Welcome back to another latest space lightning pod. This is Alessio, partner and CTO at Decibel, and I'm joined by my co-host Wix, founder of Small AI.
Hello. And today we are joined in the remote studio with Brendan Fortuna of Ambiance. Welcome.
Hey, thanks so much for having me. Huge, huge fan of the pod.
So let's talk about Ambiance. You know, I think you are actually our first guest covering or working in the healthcare sector. There's a lot of doctor, medicine, AI scribe things out there. What's the origin and what's your specific pitch for Ambiance?
Yeah, sure, sure. So I'll tell you maybe just a quick TLDR and maybe get into the workflow and share some of the problem for folks who are less familiar with healthcare. So the TLDR is basically, you know, Ambiance is a healthcare company. We're building an AI assistant for doctors. We help doctors take notes. We help them complete, you know, administrative tasks. And the end result is we save them a bunch of time and they can spend that extra time like caring for patients, right? Our customers are actually like big health systems. So like Cleveland Clinic and UCSF, Ardent, Sean Muir. These are the kind of customers we sell to. But the end users are actually the clinicians themselves, doctors, nurse practitioners. The basic workflow with Ambiance is we have a mobile app. The doctors will take it into the room with the patient. You may have seen this, you know, if you're a patient.
We'll then listen to that audio, we'll transcribe it into text. And then we'll, we'll call language models that are fine tuned for medicine to generate all different types of unstructured and structured documentation. And then we'll automatically write it back into the electronic health record. That's the system of record where doctors do a lot of their typing and note taking. And the end result is like we'll save doctors, you know, up to two hours a day. We make them tremendously happier and we help the health, health systems as well see more patients.
Every time someone mentions EHR, I have to mention like it's a, it's a complicated system in the US, right? Like that's basically the default system that is owned by one company. Is that, is that how it works? Like you have to submit data to it and like everyone works off of that system?
Yeah, so there's actually many electronic health records, at least 20 or so. But there's a few big ones that our customers, you know, the big health system drawn Epic is obviously like the Goliath in the room. Over 50% could share Cerner. That's another really big one owned by Oracle, maybe about 27%. Then there's Athena and Medtech, you know, Ambience, we integrate with these EHRs. And they have open APIs, they have some private APIs, but we try to make it seamless for clinicians.
And you previously were working in self-driving.
Of course, right. Yeah, so I joined, I joined Ambience like three and a half years ago to join the founders, Mike and Akil, my close friends. Prior to this, I was at Cruise, I was working on self-driving cars and computer vision and LiDAR. I co-founded our machine learning platform team there and then took some of that knowledge here to Ambience.
Yeah, yeah. Part of me thinks that this, the sort of LLM-centric problem is very different from the machine learning problem that you had at Cruise. But maybe it isn't because what's different, what's the same?
Machine learning is still very frustrating. It's always been frustrating. That's a great question. I think like over the last, you know, four years, there's been huge transformation in the industry and how machine learning has been practiced. You know, before you might focus at Cruise on labeling massive data sets, millions of images, right? Or LiDAR and fine-tuning models that are like a lot smaller. These days, the models are massive. They generalize extremely well. You can use techniques like prompting to kind of few-shot them, even fine-tuning itself is very sample efficient. So it's definitely a lot faster to iterate. It's a lot faster to get into production. That's actually really, really exciting. The thing that I'll say that they have in common is it's still a data engine.
It's still very data-driven, you know, machine learning where what data do you have? Like, what's the distribution? How do you collect really good annotations? How do you evaluate success? Some of these fundamentals haven't changed.
That makes sense. And talking about data being frustrating at healthcare, it's like one of the number one places where that happens.
That's right.
Can we just walk through very quickly the different pieces of like the ambient experience, so to speak? Like where people start using it, you mentioned it, doctors kind of bring it in the room. We can talk about some of the work you've done on reinforcement fine-tuning as well.
Yeah, sure, sure. Absolutely. So the Ambience, the initial use case, as I described, it's called Ambience scribing. That's where the name Ambience came from. You know, that's, we listen to audio and we take notes. But that's actually just like, I would say, 5% of like the total value that we can kind of offer, you know, once you have the audio of a conversation, right? And once you have access to the EHR and you can read and write to it. Ambience is rapidly proliferating an actual AI platform. So why stop at just notes? Why not place, help place orders? Or why not place, you know, ICD 10 codes? Or why not help, you know, maybe call patients on the phone and ask them if they've taken their medication? So Ambience is slowly building out all these generative AI use cases in healthcare, you know, given the transcripts and the access to the EHR.
In terms of the stack, I think it really depends on the use case. Ambience internally, we use prompting, we use chaining, we'll use RAG, we use fine tuning, it will use SFT and RFT.
Yeah, so you mentioned RFT specifically. I think it just went GA on OpenAI maybe a couple of weeks ago.
Mm hmm.
Obviously, everybody's talking about reinforcement learning, but maybe most companies don't really know how to do it. Can you maybe talk about the RFT journey at Ambience? And like, when did you think, when did you decide it was like a good fit? And then how did you spin up the whole infrastructure to get the right data for it?
Yeah, this is a great question. Maybe just, you know, stepping back, just a quick, quick summary of RFT. So RFT is, it's a RL-based method for fine tuning language models and, and teaching them how to think and how to reason, you know, before answering questions. And it's phenomenal for very objective tasks, you know, STEM, math, science, coding are kind of the flagship use cases. And it's the same technique that's used to train these state of the art reasoning models like O3, you know, R1, Claude 4. But it's now, you know, very recently available to all developers and startups. Our foray into RFT has primarily been through the OpenAI kind of platform. So we're using those self-service, you know, APIs.
The basic core concept, right, of the RFT, which I think makes it so special, is unlike supervised learning, where you create these big labeled data sets of thousands of examples, you know, drafted by humans, and you kind of train the model to imitate them, right, and try to get the correct answer. With RFT, you kind of replace those labels with these graders, these programmable graders, these scoring functions, right? They'll output a score of zero to one, and the model learns to kind of hill climb and try to maximize these scores using RL, right? So with the API, it starts with a greater, right? And there's different types. You could try like string match, you could try regx, you could do a fuzzy match. Often you could do a unit test, run some Python code, or even use an LLM grader, or even combine a bunch of graders together into a massive greater, you know, kitchen sink. I've seen some crazy stuff.
But I think the actual like, you know, technique behind the scene, and we don't know exactly how OpenAI does it. But I think like, from our perspective at a high level, the models, each epoch, they're generating four or 64 different candidate trajectories for every example in your dataset. The grader scores them, zero to one. And then the model kind of learns like, what tokens did I generate that helped improve that score? And then reinforce those. Or like, what tokens actually hurt me? And then, you know, don't, don't send those tokens as much. And then through trial and error over thousands and millions of, of, you know, kind of iterations or trajectories, kind of figures out how to maximize the score. Right? So Ambiance, like, we found this, this is attractive to us for many reasons. In medicine, there's a lot of tasks that are fairly objective in nature. And we'll talk about maybe ICD. Also, there's a lot of reasoning.
There's a lot of medical reasoning that's kind of difficult and complex for just, you know, some of the off the shelf pre-shamed or SFT models. And I also think like, in machine learning, we're often optimizing these proxy metrics. Right? Like loss. In the real world, nobody cares about loss. So as a machine, as a machine learning engineer, you're, you're always looking for ways and techniques to kind of optimize for the end objective. Like, what do you really want? Like in basketball, like, it doesn't matter your form. What matters is you get points. Right? And I think that's what RFT brings to the table. It's like, it gives you a way to optimize for the end objective. And Ambiance cares about that in healthcare. And the second, I think, is like, it's, it's tremendously sample efficient. Right? Each example sort of blooms into dozens of labels, right? And trajectories, you can squeeze like X more signal, right?
Out of, out of a dataset. I think those two things are like tremendously profound. I'll pause there, but yeah, there's some cool stuff.
Something worries me about data efficiency. We all want it until it works against us. So sometimes it's possible to learn too much from one sample. You know what I mean?
I saw a paper on that recently. It's like.
Is this RLVR thing?
Yeah.
That, that one isn't more elicitation and like Quinn, uh, being Quinn. But like, did you have examples where it went wrong? And like, what would you be advised there? You know, like, that, that can help others try to do the same thing.
Yeah, I couldn't. I think the interesting discussion is maybe, I mean, there's many things to talk about. The reward hacking, I think, is a thing. I think.
In the physical exam use case, we can kind of talk about that, but we used an LLM grader for the first time. And whenever you're using like an LLM grader, the task is like a little bit more prose or a little longer form generation, you could be very vulnerable to this. The models are super clever. They're incentivized to win, but they'll cheat and they'll do weird things. So yeah, maybe I'll be able to find like, define what is a physical exam and like how kind of the reward hacking that we saw. So in medicine, in the room with a patient, the doctor will do this thing called a physical exam. They'll look at your elbows, they'll test your reflexes, they'll listen to your lungs. At the end of the visit, they go back to their desk and they fill out this like structured form of for each body system, what did they observe? You know, is the heart, you know, good or not? So we framed this as an RFT task. We had the model generate the structured data of a bunch of findings,
right? And we used an LLM grader to kind of measure the accuracy. We were optimizing for like precision and recall, you know, how many of the findings did it get? Did it hallucinate any findings? And we kind of saw like two things. The first was the model started to inflate the number of findings to kind of gain precision. So it's like, I'm going to generate a bunch of the same thing redundantly and I'm going to get great precision, right? And that was one thing. And so we had to kind of like update the grader to kind of constrain the model. And the second thing, there was like this tone degradation. So the model started using these layman terms like grandpa has a, has an unhappy heart. We would get crucified if we put that into a medical note, right? So what we did is in the grader, you know, in addition to just the content and like the semantic accuracy of what it's saying, we also started to add style and we kind of weighed them like 75,
25. And over time, you can kind of harness and, and, and get the reward hacking to go away.
When, why were the, some of the evals, especially, yeah, you mentioned the ICD encoding, which is from understanding how the insurance coding works. How did you eval whether or not the model was good?
Yeah, it's a great question. So this is, this is the one, the ICD 10, it started as a blog post, right? On, on opening eyes website. And then it got picked up by the news media like CNBC and Inc. And, and my grandpa started texting me about it. He's like, oh, like, you know, I saw you in the news, like, when is AI going to replace doctors? Right? And I was like, grandpa's like, AI is not going to replace doctors. So the task was ICD 10 coding is essentially
After the visit with the patient, the doctor has to go back to the EMR and they have to select these things called ICD-10 codes. They're essentially like an international system of like diseases and conditions that have these normalized codes, right? So you'll say in the visit, the doctor will say like, oh, you have left ear pain, right? But the doctor later needs to go map it to this code, which is normalized. There's about 70,000 of them and they look like H65.195. Acuteitis media, comma recurrent, comma left, right? Like that's, doctors didn't go to medical school to do that and they're not that great at it. So what we did is we recruited about 18 physicians out in the wild. We created this like eval set annotated by four expert annotators each, you know, to kind of make sure that the quality is extremely high.
And the clinicians using F1 score were scoring like, let's say around 40%, right? On the F1 score, which was surprisingly low, lower than we thought. We were able to use RFT to kind of hill climb and get that, you know, get a small model O3 mini up to around like 57%. There's still plenty of hill to climb, but we just thought that was like a really interesting result and just early signs that this could be framed, you know, as an RFT task.
Okay. So like what else do you do, right? Like, so I think there's, there's a broad surface area of products and I think there's a, there's an unlimited amounts of demand for help. Like what, what are you hearing from doctors? Like what do they need?
Yeah, I think when you first get into a health system and into healthcare, like you just learn how many problems are kind of unsolved because there hasn't been as much like startup innovation in that space. It's, it's very difficult to get in. There's a lot of like HIPAA requirements and regulations and health systems are traditionally very conservative buyers. And then you have the EMRs, which aren't making life easier. So now that we're getting in, like our clinicians and our health system, you know, partners are telling us about all these different use cases, right? So like, for example, you know, we have, have like early prototypes of like a patient facing agent. One of the problems after the doctor visit is patient, they, they don't take their medication. They don't even go pick it up. They don't go to the labs and this creates like hundreds of thousands of hours of phone calls and inbox messaging, right? And, and it's very time consuming, very expensive.
So Ambience, you know, is like, what, why don't we like use one of these very powerful voice agents to do this kind of more administrative task and just report back to the nurse, right? That's an example. There's another really interesting one, actually, is in certain specialties like oncology or cardiology, before they go into the visit with the patient, they could often spend like 10 minutes or up 30 minutes or 60 minutes looking at the chart. They're going to look at labs and images and other data about the past visit. So Ambiance, actually, we have a product where we call it pre-charting. We'll create this summary of like, here's what happened in the past notes for those visits to save them a bunch of time. You can even create a visit agenda for them so they know like what questions to ask the patient and then make the visit more productive. Those are just two, but there's like dozens and dozens of use cases.
Yeah, this patient side reminds me of the times that I've gone to the doctor and got medicine and never followed the plan. That's right. Exactly. And it's, yeah, it's my fault, but like I could get some help.
I saw in the prep notes that we had, there was one worst story around burning 25K on a one grader.
For exams.
For exams. So I would love to chat through that.
It's definitely relevant to like startups and if you're just toying around. It gets into the cost thing. So it really depends on the dataset size. It depends on what type of grader that you're using, right? String match would obviously do like less efficient, but with like SFT, let's say you're using the OpenAI to do some supervised fine turning, you'll probably have like maybe a few thousand examples. The job takes a few hours. It costs you like a hundred bucks, right? With RFT, maybe you have like 100 examples. The job takes a few days and it costs thousands of dollars. One way to burn money really fast is to use a very, very expensive grader. So I think one of our first experiments we ran is like only 100 examples, right? We quickly burned like immediately 20K just on the grader alone. So just like a watch out for builders out there.
What's your observability stack for LLMs? Like, are you tracking the spend, but on, when you're like setting up an experiment, what does that, the harness, look like, so to speak?
Yeah, great question. So I think we use like a variety of techniques. One of the cool tools that we use internally is BrainTrust. I think they're doing some incredible work over there, building like a tool for domain experts. They do give some built-in observability. They let you deploy on-premise. It's a fantastic technology. Highly recommend folks checking that out. But there's a lot of like areas where that stops, where you kind of have to build some custom tools, right? It could be something like data mining tools or automated ring release tools, right? To release safely and some, you know, maybe automated monitoring tools. And we've had to build a lot of stuff in-house as well.
We had Anker from BrainTrust on the podcast and he's speaking at the conference next week. What is, what are you looking for in eval tooling? Is there something that is really, really good for you that you want to shout out? And maybe what's missing?
Really good question.
Wow, chain of thought there.
I'm just reasoning. Yeah, so I think it really depends on, it depends on the task. You know, if you have a very objective task and it's, it runs as a unit test or a string match, like a script is actually quite good and engineers are very good at automating scripts. I haven't seen, you know, traditional issues on that side. But if you're going to have like domain experts who are non-programmers, if they're going to review, that's actually kind of challenging. It's not obvious how to do that. There's ergonomics is a big issue, like they need to see the outputs side by side or multiple side by side. I think BrainTrust is making tremendous progress there, right? And even allowing you to kind of incorporate some automation. But I still think it's very early days, you know, in the ergonomics, you know, the full IDE for a domain expert, like early days.
Yeah, I'm curious what, whether the domain experts really use these kinds of tools? Or do they need something else? It's, it's something that everyone wants, but I haven't talked to enough domain experts to know.
Yeah, I think this is, this is like the, the billion dollar question, right? I'd love to hear what you all think. Do we need machine learning engineers anymore? Or like, can you just train with domain experts alone? I think the answer is both. I think, I think the answer is both. I think the domain experts, like in our case clinicians, they're really good at like debugging model outputs, meeting with users, distilling that feedback into something actionable, maybe annotating or doing evals. But they don't necessarily have like, you know, the right intuitions about what techniques to even try in the first place. And I think that's where machine learning engineers and researchers come in.
And they kind of guide, guide the domain expert teams. They see something manual, they do a 10x faster. That's kind of how I see the interplay of these teams working.
Yeah, I think like there's the sort of test environments where you sort of, let's say you hire domain experts, clinicians to be labelers for you, like a scale AI or whatever. But like also that's pretty artificial. And I think you want like demonstrated behavior from just day to day work. Like that's kind of where like there's that kind of like a co-pilot like experience, right? Like you don't take the, just the raw thumbs up and down, like you, you actually see like how much of that was useful and stick around. Yeah. I don't, I don't know how to uh, design it beyond that, but I am looking for like, okay, like, no, you don't need any of that, you know, like here's the shortcut.
Yeah. I don't think, I don't think there's any silver bullet. I think it's a kind of like automation, but still like humans reviewing outputs fundamental. You know, fundamental, the sniff test, the vibe check, like huge part.
Uh, segments on hallucination, I don't think we've like dealt with it specifically. Obviously, I think a number one question that people have on medical use cases of L L Ms is what do you do with hallucinations?
Yeah, I think this is interesting. I have, I have some takes here. So I think like base models out of the box, they actually, they really still struggle in healthcare in many ways. One of the issues is this like robustness problem. So the models are like geniuses, right? But they can still like erratically fail a non-trivial percentage of the time. And, and like fumbling agents, it doesn't work in healthcare. This is a patient safety, critical environment, right? I think one of the problems is like the benchmarks, they always report like the best event. So they report like, you know, what's the best metric they got out of 64 attempts? But in healthcare, we're more interested in like, what's the worst event? Right? Um, I think like, you know, Karan and team, uh, you know, at Health Bench at Open R doing like wonderful work here. They're doing some great work at Stanford, but we really need to start to think about that.
If you plot out the worst event for a lot of benchmarks, you're going to be like really surprised. Like as an example, another thing we see a lot in the base models, and this is why they require like a lot of fine tuning. Patient, you know, they start to make these medical inferences. They're so smart, but they start to like infer things that the doctor didn't actually explicitly say. You know, for instance, a patient will say like, I'm feeling sad and stressed out, difficult to sleeping. Right. You know, the base model is going to be like, patient reports increased anxiety and depression. But wait a second, but those are diagnoses, right? The doctor didn't actually confirm those things. Models are going to jump to these conclusions in unsafe ways. I think the last thing I'll say about this is I actually think clinical, real world clinical data is out of distribution. I think as much as the models generalize, if you have no access to that data, it's really hard to learn.
I think that the reasons are maybe twofold. The first is a lot of realistic clinical data is inside these walled gardens. All right, it's inside Epic and Cerner, and they're locked down for patient privacy reasons. But this means there's not a lot of realistic clinical data on the internet, right? To feed into the models. The second I'd say is actually like, you know, there's a lot of tribal knowledge in medicine. So the, the, the base models, they've went to medical school, they've memorized the textbooks. But there's also this thing called residency. It's like two, four, eight years where you get all this hands on experience, right? Working with, you know, attendings and there's a lot of learning that happens there. That's encoded in the EHR, but not actually encoded on the internet.
Yeah, that actually is the same analogy that people use for the coding models. We did an episode with the OpenAI Codex team and they were like, yeah, you, you went to college and you have a, you have basic knowledge of software. But here we are giving them like the first few months, maybe the first year of being on the job training.
Yeah, exactly.
Exactly. It's so huge.
Do you feel like things like Health Bench that OpenAI announced are steps in the right direction to maybe fix some of this? Like a, you know, model evaluation lacking? Or do you think like the same issue?
So I haven't read Health Bench, but like also squeeze in an explanation of what it does.
Yeah, so, so it's a new open source data set released by OpenAI. You know, Cron and a lot of amazing researchers and participants kind of built this data set. I think it has a few thousand examples and these are like very realistic kind of healthcare tasks that they've put together and, and given out to the community. Which I think is phenomenal. And I think like they're thinking about it the right way. They're thinking about it like these academic data sets that we've been kind of saturating for a long time are no longer. Are useful. Models are gonna ace medical exams and they're trying to figure out how can we create these real realistic like clinical cases? And I think that's like the first step and there's still a big hill to climb and that's great to see.
Yeah, I mean broadly on data sharing, I don't know if you've come across Tanish Abraham, he used to work on Medarc at with Stability AI. And now he's working on Sofans, which is kind of the spin out version of it. And he actually, we actually had a really interesting conversation around like, sometimes we, we don't mind like anonymized, anonymized data sharing if we can
train this more natively, but we just need more data. They just like, this is obviously the single most privacy sensitive issue for data on planet earth. At the same time, we also need a lot of data, man. Like, what are you gonna do?
Yeah, this is, this is the billion dollar question. I can also share some more anecdotes, like once you get that data, I didn't tell you it was high quality data. In fact, like it's actually
Yeah, patients lie.
Tremendously messy.
Patients lie, like watch house.
Yeah, it's tremendously messy, um, you know, and, and I think that creates like new problems when you even start to get that data. And I think the, the reasoning models are tremendous when you give them really accurate context, but imagine giving them like messy kind of context and they have to have taste, they have to have clinical tastes and that's something we're still learning about as an industry.
What is clinical taste? Just judgment of what's right, what's wrong?
Yeah, exactly. Like what source do you trust? It's what source do you trust? Similar to like, you know, if you ask, you know, O three to go on the internet and find the right travel kind of destinations. It's just gonna kind of get like tricked by whatever has the best SEO. And then in healthcare, it's the same, like, you know, what is an example of a EHR kind of data element? That actually is messy. Like you shouldn't trust this one. You should actually trust these other, you know, paragraphs over here. So that's what I mean by like clinical taste and intuition.
Have you had, so sometimes reasoning on wrong premises can lead you to wrong results, obviously. And I wonder if you've had conversations with OpenAI on, are there different kinds of reasoning? Or is reasoning just a generally, is it just IQ or is there like seven different types of IQ? And maybe OpenAI has spiked on math and coding IQ, but not so much on the medical IQ. Like how, how distinct are they or how correlated are they?
That's a really good question. I think, I don't know how it works underneath the hood. I do know that when you kind of plot out base model performance on some medical tasks like ICD-10 coding between like, you know, pre-verse generations and new reasoning generations, there's actually not like a big leap. I think like, what we're recognizing is, look, if you don't actually train on the distribution of data in these verticals, you're actually not going to see gains. So I don't know what patterns are shared. I'm sure there's reasoning patterns in mental models, but I also think there are probably mental models, right? In healthcare specifically, that these models need to learn before they can be effective.
Yeah, thinking about patterns, also one idea is that we're gonna talk about in the prep is like, study of Devon for clinical AI research, and like moving more towards in job assistance to like more long-running experimental things. Yeah, can you share more about that?
Yeah, I think the thought here is that our engineering team is like seeing tremendous productivity gains, right? With coding assistant and coding agents, why can't we do the same thing for clinical research? I think there's definitely a world where you have a voice agent, talk with an end user, distill that feedback into, let's say, like a prompt or maybe a PRD for what we should be doing. And then you can have the agent like, go mine the data, go look at the edits, distill that analysis, you know, run evals, even do annotation. I feel like this is fully possible, and it's something I'm excited about. I'm not saying we built it yet.
Great. Yeah, any parting thoughts, call to actions? You know, you got the audience listening to you.
Super happy to be here. I appreciate you both. I love what you're doing. Huge fan of the pod. Yeah, thank you for the time.
Thank you, Brendan.
Absolutely hiring. Hiring engineers, clinician researchers, machine learning engineers, machine learning researchers, you betcha.
Maybe let's focus on a specific role and why it's what you're really looking for that is hard. What's the skill that you want that you cannot hire for?
This is a good question. I do think there's machine learning research talent, the best in the world. That's definitely hard to get, but super valuable as we start to push some of these frontier use cases. That's number one. I also think there's like a really interesting skill set in like a clinician researcher archetype. So these are the folks with the domain knowledge, but how can you find the ones that also have like an experimentalist mindset? That have like a startup work operating system? That's actually surprisingly hard. And I think that's, you know, maybe one of the gold kind of unicorn kind of archetypes. That's really important to us.
So like a, say like a Venn diagram of startup person, also somewhat domain expert, somewhat ML person, right?
Exactly.
Yeah. All right. Well, I think like being specific about that intersection helps you hire more because you're more clear about like, you know, this is what we're doing. So I wanted to elicit that out with you. But thank you for all the important work you're doing. I'm sure you're saving lives. Not all of us can say that. So. Thank you, brother.
Cool. Thank you both. Keep up the great work.
Page updated October 24, 2026 · transcript and summary are machine-generated
