How this PM streamlines 60k-page FDA submissions and saves millions with Claude, Streamlit, and clever AI workflows | Prerna Kaul

Prerna Kaul is a product and platform leader who has spent over 14 years turning machine-learning research into consumer and B2B products at Amazon Alexa, AGI, Moderna, and now Panasonic Well. In today’s episode, she explains how she’s using AI to slash some of the most time-consuming, expensive tasks in life sciences—from generating 60,000-page FDA submissions to crafting communication frameworks that help product managers navigate complex stakeholder dynamics. Her innovations are saving millions of dollars and helping lifesaving treatments reach the market faster. What you’ll learn: How Prerna built an AI system that automates the creation of 60,000-page regulatory documents for the FDA—reducing a process that took 4 to 6 months and 20 specialists to just minutesA step-by-step system for detecting and redacting PHI (protected health information) in clinical trial data using ClaudeHow to build user-friendly interfaces for non-technical colleagues using Streamlit to democratize AI toolsHow to use Claude’s prompt generator to create powerful communication frameworks that help PMs navigate complex stakeholder situationsWhy transparency about AI costs is crucial for gaining organizational buy-in and tracking ROIA practical framework for approaching AI safety and ethics in highly regulated industries— Brought to you by: CodeRabbit https://www.coderabbit.ai/—Cut code review time and bugs in half. Instantly: https://lovable.dev/ Lovable https://lovable.dev/—Build apps by simply chatting with AI: https://lovable.dev/ — Where to find Prerna Kaul: LinkedIn: https://www.linkedin.com/in/prernakkaul/ — Where to find Claire Vo: ChatPRD: https://www.chatprd.ai/ Website: https://clairevo.com/ LinkedIn: https://www.linkedin.com/in/clairevo/ X: https://x.com/clairevo — In this episode, we cover: (00:00) Introduction to Prerna (03:01) The FDA submission challenge: 60,000 pages, months of work, millions in costs (05:20) Getting started in Claude: from prompt to production-ready prototype (10:13) How Claude selected the right models for medical entity recognition (12:04) Using Streamlit to create accessible UIs for non-technical users (16:04) Detecting and redacting PHI in unstructured clinical notes (18:44) Generating the Common Technical Document (CTD) for FDA submission (21:54) Tracking and displaying AI operation costs for stakeholder buy-in (24:38) Real-world impact on vaccine development timelines and costs (26:12) Creating an AI communication coach for product managers (30:22) Training Claude on classic literature and persuasion techniques (31:53) Analyzing a complex stakeholder scenario with multiple competing priorities (34:40) Getting personalized communication strategies inspired by tech leaders (35:40) Summarizing strategic approaches (38:26) Conclusion and final thoughts — Tools referenced: • Claude: https://claude.ai/ • Streamlit: https://streamlit.io/ • Anthropic Console: https://console.anthropic.com/ • Claude Sonnet 4: https://www.anthropic.com/claude/sonnet — Other references: • Claude project chat (AI Product Management Stakeholder Challenges): https://claude.ai/share/caba4ab0-b28a-480c-8633-71920b12999e • XML: ⁠https://www.w3.org/XML/⁠ https://www.w3.org/XML/ • Python: ⁠https://www.python.org/⁠ https://www.python.org/ • RegEx: ⁠https://regex101.com/ https://regex101.com/ • Moderna: https://www.modernatx.com/ • FDA: https://www.fda.gov/ • Project Gutenberg: https://www.gutenberg.org/ • FDA Biologics License Application: https://www.fda.gov/vaccines-blood-biologics/development-approval-process-cber/biologics-license-applications-bla-process-cber • Protected health information (PHI): https://www.hhs.gov/hipaa/for-professionals/privacy/laws-regulations/index.html — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Claire Vo?

  So you are working in the life sciences on really health impacting vaccines and treatments. And with all that amazing investment in scientific work, also comes a lot of paperwork.

• 0:15Speaker 01

  We had to develop a nearly 60,000 page document, would have taken about four to six months of effort, and nearly 20 specialists, not to mention the millions of dollars spent.

• 0:26Claire Vo?

  Where did you start?

• 0:27Speaker 01

  I gave Claude the problem statement, the pitch and the demo. The thinking I had in mind is that Claude is a software engineer and I'm talking to them and trying to tell them why it matters, like any good PM would, and trying to tell them what end product we want to produce as a result. And not only did it fully understand what I was asking it to develop, it created a little setup instructions markdown file for me. And it gives me all of the setup instructions and all of the tasks that it does and its capabilities, which is super handy.

• 0:57Claire Vo?

  Were you really able to shift the cost there? Did you see an impact here?

• 1:01Speaker 01

  We did, we did. And any kind of cost savings you can generate have a direct impact to the bottom line. And in addition, if you're saving on time, you're bringing life-saving vaccines in the hands of people who actually need it.

• 1:18Claire Vo?

  Welcome back to How I AI. I'm Claire Voe, product leader and AI obsessive, here on a mission to help you build better with these new tools. Today, we have Prerna Kall, who's taken years of machine learning experience and helped develop some of the products you know and rely on every day at companies like Amazon Alexa, Moderna, and Panasonic. She's going to show us how she uses AI to accelerate drug discovery and vaccine approval, something that takes tens of thousands of pages of documents, dozens of people, and months and months and months of work. She'll also show us how you can use Jane Austen, Dale Carnegie, and some AI to manage your stakeholders just a little bit easier. Let's get to it.

• 1:59Claire Vo?

  This episode is brought to you by CodeRabbit, the AI code review platform, transforming how engineering teams ship faster with AI without sacrificing code quality.

• 2:10Claire Vo?

  Quality code reviews are critical, but time consuming. CodeRabbit acts as your AI co-pilot, providing instant code review comments and potential impacts of every pull request, beyond just flagging issues. CodeRabbit provides one-click fix suggestions and lets you define custom code quality rules using AST Grep patterns, catching subtle issues that traditional static analysis tools might miss. CodeRabbit brings AI-powered code reviews directly into VS Code, Cursor and Windsor. CodeRabbit has so far reviewed more than 10 million PRs, been installed on 1 million repositories, and has been used by 70,000 open source projects.

• 2:51Claire Vo?

  Get CodeRabbit free for an entire year at coderabbit.ai and use the code HOWIAI. Prerna, it's so nice to have you. I'm really excited about your workflows.

• 3:05Speaker 01

  Thank you so much for having me. Very excited to be here.

• 3:08Claire Vo?

  So you are working in the life sciences on really health impacting vaccines and treatments. And with all that amazing investment in scientific work also comes a lot of paperwork, if I am correct. And so in addition to doing all this amazing research, you also have to document and submit a lot of stuff to the FDA and other regulating industries. And I am sure that is a total pain. So tell me a little bit about the problem that led you to building some of these AI workflows.

• 3:45Speaker 01

  Well, yeah. Firstly, thank you so much for having me. And you're totally right. With life sciences, I think the challenge across all parts of the organization is a lot of paperwork, a lot of regulation. And going back and forth with these regulated industries could take months of time, not to mention all of the people resources involved and delays in getting, in some cases, life-saving drugs and vaccines to market that people need. For example, when I was working at Moderna, we were developing the INT cancer vaccine. Which is necessity for folks with skin cancer. It was sort of a new niche concept that had come out. And we had to develop a nearly sixty thousand page document,

• 4:30Speaker 01

  which is known as the, it's, it's a technical jargon, but biological license agreement, which took around four to six, it would have taken about four to six months of effort and nearly twenty specialists, not to mention the millions of dollars spent across the organization and getting this document together and providing it to regulated authorities across the world. Very labor intensive, very time consuming. And that's where I kind of came in. I was a Gen AI product manager there and really came in to think through, how can we reimagine this using generative AI? This was back in twenty twenty three where I came up with the solution. So a couple of years back.

• 5:10Speaker 01

  And I kind of evolved it over time to kind of specialize in certain areas as well. I'd love to show what we came up with.

• 5:19Claire Vo?

  So it sounds like you created a really high impact tool and sixty thousand pages is pretty, pretty intense. What I love about this workflow is you're going to show us actually how you use Gen AI to build this Gen AI solution. So like all good product people, you started with requirements. Can you just show us your process from you've, you've identified this problem, which is we got to create this massive document. And it's going to take millions and millions of dollars and tons of people to actually having this production thing that, that your team and colleagues could use. So where did you start?

• 5:51Speaker 01

  Like every good PM, started with the requirements. And I actually didn't, hadn't fully formed all of the ideas initially because I think that with, with the complexity in the domain knowledge required, it took a few iterations too. But I'd love to showcase sort of where we started with very simple pitch and then go through how we actually evolved the idea as well.

• 6:16Claire Vo?

  Great.

• 6:17Speaker 01

  So I, I gave Claude the problem statement, the pitch and a demo. The thinking I had in mind is that Claude is a software engineer and I'm talking to them and trying to tell them why it matters, like any good PM would, and trying to tell them what end sort of product we want to produce as a result. So I've included statements such as, you know, this is Stefan Barcel's, who is but NSCU's passion project, he's very excited about it. Here is the cost savings you can get. Here is the, here is why it matters and the impact. And then also spoken about the demo that I want to showcase and how we could sort of iterate and build on it. And Claude came back to me with, not only did it fully understand what I was asking

• 7:01Speaker 01

  it to develop, it created like a little setup instructions, mark down file for me. And it gives me all of the setup instructions and all of the tasks that it does and its capabilities, which is super handy. Not only did it do that, it also gave me all of the code and all of the different aspects to the, to the code as well. So I kind of started here and started thinking through what, what do I really need to do to make this a production ready system? And there were two things that were extremely important in the process. Uh, first is because the BLA is a structured document, it is XML based, it has certain formats.

• 7:40Speaker 01

  I needed Claude to successfully, in a very strict and structured way, take data and structure it into a certain format. So that was a very strict requirement and that's not super easy to get with some of these models as well. The second was that because it is clinical data, it's highly regulated and being able to detect PHI, redacted information was extremely important so that it could correctly redact that and then not have that included in the actual BLA that we would be sending to any authority because it's patient data essentially. So the reason why we chose Anthropic Claude is because of the reputation around safety and alignment with human values as well.

• 8:25Speaker 01

  And we formed this partnership with Claude and really focused on some of the PHI and the safety aspects to make sure that, not, not just from a compliance standpoint, but also as we think about alignment, we were taking care of those use cases. So that this is what it came up with, did a really good job. And I was able to get some code running as well. And I'd love to show you what it came up with.

• 8:50Claire Vo?

  If we can go back to the requirements really quickly, there's just a couple of things I want to call out for folks that I think is really interesting. If you go to the step by step instructions, what I love about it is if you go to the top, it not only gives you sort of how to technically write the code, but something I wanted to call out at the bottom is it actually gives you a pitch narrative to how to run your demo. So what I think is so interesting about this specific prompt is you gave it kind of pretty loose requirements. It's only a couple of paragraphs up top and it gave you a whole set of things. I'm sure many of which you weren't anticipating. I wouldn't have anticipated getting a demo script from Claude when I'm asking it to build Python. But but it gave you that. I thought that was really interesting.

• 9:34Claire Vo?

  And then if you scroll down in the left side, something else I'd like to call out for folks that I think is really interesting is it gave you these kind of like component names of different technical pieces of of the product. So like coming up with mock data, doing PHI reduction, all that kind of stuff. You know, you as a software engineer and a product person, how long do you think it would have taken you to come up with this full set of just this first one shot prompt requirements before?

• 10:07Speaker 01

  At least a couple of days, at least about a week to really think through everything.

• 10:12Claire Vo?

  And then in addition to knowing you wanted to work with the anthropic models because of safety and alignment, were there other libraries or other models that you knew you wanted to use? Did it pick the right ones? Like, how close to the technical implementation did it get?

• 10:26Speaker 01

  In particular, with the with the redaction piece, it would have taken a couple of data scientists to figure out the right solution and work with us. Also, data scientists as well as software engineers to work with us and figure out what's the right solve for this? How do we implement it? How do we test it to get a certain synthetic data? So I think where it really excelled is in picking the right models. And I have some amount of machine learning knowledge, but it would have taken me a couple of days of study to figure out, okay, this is the right model for medical named entity recognition. And this is how you go about it. But it did such a great job with very little information upfront on the domain itself. It was able to absolutely pick the right models and give me the code and then implement it

• 11:11Speaker 01

  and show me the confidence score across that as well.

• 11:14Claire Vo?

  Yeah, and if we look at the code, I just have to laugh because, you know, many, many years ago, I think I have lost hours and hours and days and days of my life to writing Regex.

• 11:28Claire Vo?

  Just, I, I used to spend all my time writing Regex. Terrible, that didn't work. It was just awful. And so, one of the kind of miracles of AI I see is it reduces this toil in technical tools you know you need to use that have documented patterns, but they're just so hard to research and get perfectly right out of, out of the gate. So this was my favorite part of seeing, seeing your code up is seeing all that code just pre-populated for you.

• 11:56Speaker 01

  Yeah, that's such a common and shared pinpoint. Regex has been a, yeah, it's a pain of any software engineer's existence.

• 12:03Claire Vo?

  Yeah, so, okay, so it created this Python script for you and you said you scrolled through it and it looked, you know, pretty correct. And then you decided to deploy this on Streamlit. Can you tell us a little bit about that choice, why you picked that?

• 12:18Speaker 01

  Yeah, so I love using AI to solve for some, and automating my own work and solve for my own sort of day-to-day problems, either be related to work or something I'm trying to work on in my side projects so personally. But I think the real unlock, at least I've found, is that if I can create something of value for other people, and then especially as they are non-technical folks, making a UI and making something generally available and hosting it locally has proven to be a really, really big unlock. Both career-wise and also, you know, working with different colleagues, they find that to be really valuable because they might have something in mind and I have all these skills and I can, I can sort of offer that.

• 13:02Speaker 01

  So that's actually something I've been doing in my current organization as well, where I recently created a Google Adon to convert PRDs into Jira format. And that was a lot of fun and, you know, the product managers especially were very excited about it because they don't necessarily like having to write requirements twice, one in a PRD and then also in Jira. And we, we found a good way to kind of circumvent that and do it in a matter of minutes. So I did that and then the Streamlit idea I also did in a couple of my previous roles and my current role as well to give non-technical users access to all of these tools.

• 13:41Claire Vo?

  Yeah, and just for folks that don't know, Streamlit is a really easy way to put a pretty simple UI on front of like a Python kind of file and set of APIs.

• 13:51Speaker 01

  Exactly.

• 13:52Claire Vo?

  So Claude created your PRD, it gave you code, you were pretty happy with the code. How do you get from us looking at this code right now to a product that your team could actually use to create these documents?

• 14:04Speaker 01

  I'm gonna run for you the actual Streamlit app. So I'll take you to terminal. I essentially run it from the command line and it opens up a web app. It takes me here. It has preloaded an API key from Claude, which I provided in the code. So I've hidden some of those properties. And I'm gonna generate synthetic data for us and I'll talk through, and I'll go through the step as well because it takes a bit of time. And I'll talk through what it's done. So as I mentioned, there's this format called a Common Technical Document as part of the BLE. There are several modules, about five or six different modules. And it has a structure around it. So there is, there is a study report that has a tabular listing.

• 14:48Speaker 01

  It's supposed to have very specific points across the data from the clinical trials and then different literature references as well.

• 14:57Claire Vo?

  Did Claude know about this format kind of from just general knowledge or did you have to provide the format to it?

• 15:05Speaker 01

  So it searched up the common technical document format for me, and then it figured out the right module to apply to my use case.

• 15:15Claire Vo?

  Great, that's awesome.

• 15:16Speaker 01

  Yeah, it's doing pretty well. So I'm running here what is, I'm going through the different, we have different participants here, and there are some clinical notes per participant. This is all clinical trial data, very, very sort of common that you would see in any trial that runs. And by the way, the trials itself, they could run in the order of billions. So redoing a clinical trial is very, very expensive. And so as much as possible, we want to give very accurate information to the FDA so they don't ask us to redo anything. And what I've done here is I've de-identified some of this patient data to remove any sort of names, date of birth,

• 16:00Speaker 01

  any information that could be associated with the person.

• 16:03Claire Vo?

  And one of the challenges I'd love to call out for folks that maybe aren't seeing this is you generated some fake data so you could validate it works. And it's not only this tabular data that has patient ID, ethnicity, whether they were in the control or otherwise, but you also have these free-form clinical notes where there may be embedded PHI written in by a clinician or a study administrator. So it's not as simple as saying just find all the date of birth columns and redact them. You actually have to go through this unstructured data and find potential sources of PHI. Is that part of the challenge?

• 16:39Speaker 01

  That's exactly it. It's a complex problem usually. You know, machine learning doesn't always do a perfect job there, but Claude immediately identified the right model to do this.

• 16:52Claire Vo?

  Great. So, okay, so you have this clinical data preview, and then you're going to show us how you de-identify it so you're not sharing that with folks that don't have access to that health data.

• 17:01Speaker 01

  Exactly. So with this synthetic data, I clicked on this button to detect and redact PHI information, and it scanned through all of the rows, and it found for me all of the PHI. So here's what it came up with. So seeing in the clinical notes, it found that there was some dates, names associated. There was some birth date information as well. And the remaining it was able to keep sort of in a DIA identified way so that I can include it in my reports.

• 17:34Claire Vo?

  Great. And then how would this have been done in the past? I think you said there are software engineerings or ML, you know, engineers that would have built these models to redact the data. Was there any human in the loop at any point in this as well? Like, how was this done before?

• 17:49Speaker 01

  For the past couple of years, we've been, most pharma companies have been using machine learning for this. But these projects, they end up, because it is PHI, there's very high bar. They end up taking months of effort to get right. And especially when clinical trial data might not look the same for every drug. As a company like Moderna was trying to scale, we were trying to launch nearly 50 drugs, bring 50 drugs to market in clinical trials and then 15 to market in the next three to five years.

• 18:26Speaker 01

  That problem was exacerbated. So thinking about coming up with new machine learning models to solve different problems across each of these vaccines. It's a matter of scale and time, really, and we really needed to find a better way to do this.

• 18:42Claire Vo?

  Great. Okay, so we've got redacted health data. You have all your great, you know, billions of patient data points. What's next in this, in this flow?

• 18:53Speaker 01

  So the next thing is to generate this common technical document. And I'll show you a small preview, just for the purpose of this demo. So what it did is it took all of that and it gave me a summary of the different data. It was able to give me a synopsis, which is what the FDA looks for. The study participants, the counts, the age, all of the statistical data points, as well as some of the specific medical terminology that they're looking for. So it kind of summarized everything. It gave me this. And for for context for folks, uh, this is typically done by a medical writer and their their whole, I mean, I've worked very closely with them.

• 19:35Speaker 01

  It's so time consuming because they not only they they can do a pretty good job of generating a like a first draft summary, but they have to go back and forth with their stakeholders for weeks sometimes on a single module to come up with the right set of criteria and getting everyone to align, all of their stakeholders to align is is very, very time consuming. Uh, twenty specialists, as I mentioned, are working on it at any given point in time. And as part of what uh some of these uh the chat GPTs of the world and uh Anthropic can provide, there are also common projects and common environments where all stakeholders can kind of collaborate and work together on these summaries,

• 20:17Speaker 01

  which is so much easier and so much better than working, let's say, in um in a sort of a shared document or and just providing comments.

• 20:25Claire Vo?

  Yep, great. Okay, so it generated from that data this very, I mean, it looks official to me. I'm not I'm not part of the FDA. But, you know, it's a summary of statistics, it's summary of the methodologies, it's summary of the impact. Um, it's pretty long and detailed. It looks like you can download it both to a TXT form, like a doc form, as well as this, as you mentioned, proprietary XML format that's required. Um, and I'm presuming the Claude code just gave you two buttons to to make that happen.

• 20:56Speaker 01

  That's exactly it. And I know it's a little bit proprietary, but I cannot tell you how excited I was to see this button.

• 21:04Claire Vo?

  This episode is brought to you by Lovable. If you've ever had an idea for an app but didn't know where to start, Lovable is for you. Lovable lets you build working apps and websites by simply chatting with AI. Then you can customize it, add automations and deploy it to a live domain. It's perfect for marketers spinning up tools, product managers prototyping new ideas, or founders launching their next business. Unlike no-code tools, Lovable isn't about static pages. It builds full apps with real functionality, and it's fast. What used to take weeks, months, or even years, you can now do over the weekend. So if you've been sitting on an idea, now's the time to bring it to life. Get started for free at lovable.dev.

• 21:49Claire Vo?

  That's lovable.dev. Okay, so if we could just take a halfway point of the podcast, the two things you were most excited about and I was most excited about. I was excited about the RegX, you were excited about the XML. This is a very nerdy and very specific HowIAI podcast. Okay, so you've generated the doc and then I love this last part. So let's talk about what's this last piece of the product.

• 22:14Speaker 01

  What's super interesting is I have worked in a couple of different organizations where as we are trying to scale different AI use cases, cost comes to be a factor because not just token cost, but also acquiring licenses for these models and then running it and as the number of consumers or the users of the actual use case scale, the dollar cost also scales proportionally. So one of the ways in which we could potentially continue to launch and continue to build products around this is by actually being transparent about the cost associated with the operations. And you know, generally, one of the stakeholder arguments

• 22:58Speaker 01

  that have previously received pushback is this is going to be high cost. Can we find a simpler way that's not AI-based? And how I've tried to solve this is actually being transparent about the cost associated with it, how long it would take. And as you can see, this trace and cost analysis is super useful because it says cost per operation, it gives a duration against it, and then it actually breaks down by operation, what was the token cost and the total cost. So extremely useful. You can continue to track and monitor this in production as well.

• 23:32Claire Vo?

  Yeah, so then you can say, for example, you know, PHI redaction per patient costs 15 cents. And you can decide as an organization, is that more or less expensive than alternative models? That's, I mean, I actually haven't heard anybody give this. The suggestion, so I want everybody to hear it, which is if you're getting internal resistance to cost. In fact, I feel this even as a even as a solo founder who's vibe coding things and I see Claude Code give me like, this costs you $5.13. And I send that to people, they're like, that's so much money. I'm like, it's $5. And so I love this idea of bringing true transparency. And I'm sure these numbers are much smaller than your production numbers on your,

• 24:16Claire Vo?

  you know, million person studies, but it does bring a true sense of ROI and investment here that I think can get you through internal hurdles.

• 24:25Speaker 01

  Yeah, it really has helped. And I really love StreamNote for this because it's tracking everything by per operation. So we can even plug and play very easily.

• 24:36Claire Vo?

  This is awesome. And so you built this and are getting it in the hands of folks that need to submit this. And what do you think the actual, you know, you were telling me millions of dollars, months and months and months, tons of people. Were you really able to shift the cost there? Did you see an impact here?

• 24:56Speaker 01

  We did, we did. And we actually applied some of this thinking to our cancer vaccine, the INT vaccine that's coming out and our RSV, COVID RSV combined vaccine as well. Both are past their clinical trials. And I think we're looking at commercialization there. So we found that cost savings was definitely one thing, but the stakeholders themselves were super engaged in the process and they built a system that scales pretty well as the company scales as well. So that was the most useful thing is we're saving on repeated work where if one thinks about, I'll just give some numbers out there. So

• 25:38Speaker 01

  billions of dollars in costs per trial, three clinical trials per vaccine, want to bring 15 vaccines to market. That's, that's probably in the order of a couple of hundred billion. I mean, that's, that's a pretty significant number. And usually, most companies don't have that sort of cash. And any kind of cost savings you can generate have a direct impact to the bottom line. And in addition, if you're saving on time, you're bringing life-saving vaccines in the hands of people who actually need it. So I was really motivated to kind of, kind of help here.

• 26:11Claire Vo?

  Well, this is such a great use case. I love that you showed going from Claude to Streamlit app and then from Streamlit app to real impact on something that I think, you know, as a mom, RSV and COVID vaccines. I thank you. So the sooner you can bring those to market, the happier I'm going to be. So this is, this is amazing. I know you have one more workflow for us on the product management side and you teased this like PRD to Jira thing. So I know you're thinking a lot about just product management generally and AI. So you want to show us your next workflow?

• 26:42Speaker 01

  Absolutely. So the, the problem statement here is that with, with product managers, we, in our day-to-day roles, end up working with a lot of different stakeholders who have various priorities as when it comes to the product. And I personally, I really want to make sure my stakeholders have heard that we're working very, very collaboratively. And while we do that, at the same time, I want to make sure we're making the right decisions and also kind of bringing everyone along and moving things forward. And with influence and communication, which is a super important skill as a PM, it's, it's, it's

• 27:22Speaker 01

  sort of custom where it's, it's hard to say which communication framework to apply when and what type of communication would land best with a certain stakeholder because it's so personal and you really have to empathize and understand folks. So very important skill to work on. I've been starting to lean in on Claude a little bit and training it to kind of give me ideas and use it as sort of a brainstorming buddy in the process. And this has actually really helped me save time in meetings, in ahead of meetings in preparation. So I'm going to go through a prompt that I created. I want to show you this really,

• 28:05Speaker 01

  really cool thing that Claude developed. It's called a prompt generator. So if you go to the console, console.anthropic.com, you'd be able to create a net new prompts or generate prompts as well. And I created one for influence and communication where essentially I was, I didn't create this, by the way, generated it for me. I gave it some instructions and then it gave me very, very precise information on what, what the instructions are. And then it gave me structured values that I can include as part of the prompt. And how it would break down the problem, how it would think and reason through it, and then what outputs it would give me in return as well.

• 28:47Claire Vo?

  So a couple of things I see here is, one, you use this tool that Claude has called Prompt Generator. It generated the prompt. For those that aren't looking at the screen, the other thing I want, it's XML is the name of the game today. So it has these kind of like code block sections for the situation, your knowledge base, how you do strategic thinking, like all good prompts. It starts with the "you are a" blank and gives it a role. And then if we scroll down a little bit, you can see that it also lets you put in different knowledge and then it tells you to provide your response in a very specific format that has a combination of analysis, your communication strategy, play by play,

• 29:29Claire Vo?

  and then like an anticipated question. So this is a very structured prompt. Have you started to reuse this sort of like kind of XML style prompt? Or, you know, have you played with different things? What's your approach? Or do you just use what Claude gives you?

• 29:46Speaker 01

  Yeah, so I usually start with generating a prompt. And then you see this button here, "Use Claude to Optimize Your Prompt"? This is extremely useful because what it does, you can give it feedback and in real time, it will improve your prompt and then add it back in here. So now let's put this prompt in the actual Claude project. So here we go into projects, and this is our influence and communication coach. I'm going to update this. You can see here that you can set project instructions. So I'm going to update this prompt here into Claude and save these instructions. And I've also given it a project knowledge.

• 30:31Speaker 01

  So I've actually trained it on free books from Project Gutenberg that consists of all sorts of things from literature like Dale Carnegie and Tactics on Persuasion and how to win friends and influence people. So all of these cool books that are all free and in the public domain.

• 30:50Claire Vo?

  So I was not expecting Jane Austen to be in your Product Manager Strategic Communication Claude Anthropic project, but I am excited to see it. So you not only gave it's prompt, but you kind of loaded it up with classic Classic literature, classic writing on influence, persuasion, communication. Very, very interesting.

• 31:14Speaker 01

  Yeah, Jane Austen is a favorite of mine. I'm not sure how many folks would use that for persuasion, but I found it very enlightening. So now I'm going to walk you through a prompt that I have been working through. And this is just a test case, but we can try something live. How about, how about we try that, Claire?

• 31:33Claire Vo?

  Let's do that.

• 31:35Speaker 01

  Amazing. So do you have a situation in mind or I can ask it to generate a couple of situations?

• 31:41Claire Vo?

  Oh, let's ask it to generate a couple of situations. I'm curious what it will come up with.

• 31:46Speaker 01

  So I'm using Claude Sonnet four, which is one of the reasoning models. And as a reminder, the, the goal of this particular project is to essentially take a curve ball problem at work and then come up with sort of a structured strategy. So here, what it's done, it's made the assumption that I'm Alex Kim. I'm an AI product manager working at a healthcare AI startup. And these are all of my stakeholders, very, very, you know, sort of experts in the domain and have very strong opinions about various things. And it has, it's broken down a problem statement that I'm trying to solve where two weeks before

• 32:27Speaker 01

  a major presentation, these are all of the concerns that my stakeholders mentioned. So it's actually created that for me. And it has also described the core challenge here, navigating ethics, compliance, competitive pressures, stakeholder expectations. And it's also kind of personalized, which is cool.

• 32:48Claire Vo?

  So for people that aren't seeing what it generated, which is a very realistic, unfortunately, a challenge, which is it generated this scenario, which we're just going to pretend is a scenario we're facing at work, which is I'm a PM and my head of clinical data just identified a bunch of issues, both privacy and accuracy data, two weeks before we have a major presentation with a huge, huge prospect. And then it lists out everyone's different opinions. And I think PMs can really empathize with maybe not this particular situation, but a situation like this. So you've created an AI project that says, give me a sticky situation and I will help you navigate how to navigate the situation, given all this, all the stakeholders.

• 33:33Claire Vo?

  And so. It's listed to the stakeholders. It's given you the core challenge. So what do we do? What do we get next?

• 33:40Speaker 01

  All right. So I'm going to ask it to use this and proceed next steps.

• 33:46Claire Vo?

  And then this is what it's going to use, your strategy framework that you came up with in the prompt to sort of break down the problem and give you an approach you might be able to use.

• 33:57Speaker 01

  Exactly. So it's now sort of analyzing based on the project information that I'd shared with it. And as we can see, it's picked out the most relevant sections from the knowledge as well. So I had given it around 10, 15 different books and documents, but it found the top two and then picked out the right sections from those. So all of this, it was able to search that information and point to that and analyze the whole workflow. So it gave me sort of a situation analysis, which I found particularly helpful because it's sort of validating that it understands the core challenge and the organizational context and some of the cultural factors as well. And here's my favorite part of the prompt,

• 34:42Speaker 01

  where it actually goes in and it looks at some of the influential tech leaders that we have all heard and know of. And it sort of inspires the user to say, OK, this is how Satya Nadella or Andy Jassy or Tim Cook or Jensen Huang would actually approach the problem.

• 34:59Claire Vo?

  This is amazing. And so you're giving really real life examples or quotes that someone can say, OK, if I were to approach it like this other person, which I see, I hear is a very common AI prompting technique, act like this person or act like you work at this place, can really bring a set of unstated principles to a problem that I think is really effective. OK, and so it's come up with document examples, tech leader examples, communication principles. We all love how to make friends and influence people. So some Dale Carnegie principles. And then it's giving you some strategies you might be able to take here.

• 35:38Speaker 01

  Yeah, so out of all of that vast array of information that I gave it, it picked out the three strategies it would apply. So it it talks about the ethics of it, which I personally agree with quite a bit. So it's it's talked about positioning as privacy first as a competitive advantage. And then it's it's talked about compliance and then now it's gonna give me sort of a collaboration strategy as well. So after analyzing the situation, it kind of summarized for me. Here are the three things that you should do in terms of strategically approaching the situation. First, make sure you're acknowledging and validating each stakeholder's expertise.

• 36:19Speaker 01

  Second, collaborate to discover an approach that demonstrates responsible AI leadership. And then talk about the ethics aspect of it. So it's really kind of honed in there. So it's it's very helpful as, of of course, like a good PM, a product leader in the space would have this info. But it's it's helpful to see that as a sort of a thought partner as you're brainstorming over this as well. And then it's actually broken down my pre-work for me on a day-to-day basis. So what are the individual conversations I can have with each of my stakeholders? And what question to focus on and ask in that meeting? And then it's given me a minute-by-minute structure for the final leadership meeting as well.

• 37:05Speaker 01

  And and here is my favorite part. Not only the play-by-play in the meeting, it has given me backpocket questions that could be thrown as curveballs that I should probably have an answer prepared for.

• 37:17Claire Vo?

  My mind is blown right now because I can just think of so many PMs that stress out so much when they're asked to take on a big problem that has executive, kind of like, leadership eyes on it. And so many just don't know how to approach those meetings. Like, who do I talk to? Who do I not need to talk to? What do I show up in this meeting and do? And how do I show up to this meeting really, really well-prepared, even when it's a challenging problem? And you've created this kind of like project that just says, here's my problem, here are all the people. And it's given you a schedule and an agenda to follow in these conversations,

• 37:59Claire Vo?

  which is super helpful and would help PMs not only save a lot of time and prep for these meetings, but actually show up just more prepared and more confident.

• 38:08Speaker 01

  I agree completely. It has been extremely helpful. And yeah, I'm actually also even happy to share this publicly, but it's helpful to folks.

• 38:16Claire Vo?

  People are definitely going to want it. So post in the comments if you would like this publicly shared so you too can prepare for your meeting where all your stakeholders disagree. Well, this has been so amazing. I love that you both showed us building something very technical that solved a really complex and expensive problem, but also something really practical that brings a lot of like heart and experience and strategy to a human problem that feels like it doesn't have a structured solution to it. But in a lot of times, it does have a structured solution to it. So we're going to hit you up with some lightning round questions and then I will get you back to whatever you are building next, or your really tough meetings that you're very prepared for. So I think the

• 38:58Claire Vo?

  first thing I would say is so many of our listeners are in software and technology, but what you're showing is that the life sciences, biotech, hard sciences have so much to benefit from embracing AI. So what would you say to your peers in life sciences who have maybe been doing something a very specific way for a long time about how they can adopt AI and how it can transform the impact that it can have on the world?

• 39:27Speaker 01

  The thing that I have, I've been, I've been doing quite a bit with my stakeholders and I have been encouraging folks as well, my peers to work on is figuring out where they spend majority of their time and which things, which tasks do they actually want to focus on and which tasks do they actually want to delegate. So what are energy sort of maximizing tasks and what are actually a drain on their energy and time? And now you have access to AI and you can take a plot project and create one for any of the use cases and tasks that you don't really want to spend time on. So I think the, the

• 40:10Speaker 01

  best thing about this is it's not just for individuals, but You can make GPTs and projects available to all of your stakeholders. And all of them can benefit from it and really enjoy the perks of your great thinking. So I would encourage people in life sciences as well as any, any folks who are interested in to play around with these tools and not be afraid of actually trying to optimize their own work because they might find it's a common thing for all of their peers in, in the process.

• 40:40Claire Vo?

  So, speaking of being a little bit afraid, I mean, you work in a highly regulated industry with, you know, patient data, you have to have the highest ethics, you really, the burden of accuracy is so important, the burden of privacy is so important. How do you think about safety, privacy, alignment? It's not a problem that I think a lot of the kind of like vibe coders or PMs in software companies really have to grapple with in their day to day, but you do. And I'm just curious how you approach it, what's your framework, where does it stack rank in your priorities?

• 41:16Speaker 01

  Absolutely. I think with most regulated industries, I've worked in finance before and in healthcare. I also currently work with family wellness where there are teenagers and kids involved. Compliance and privacy comes first, but it's, it's a bit more than privacy and compliance. It's the, the ethics overall that one must prioritize. And as I, as that relates to LLMs, I think looking at the benchmark data of which LLMs are performing higher on ethics and what edge cases have been observed in terms of safety is sort of the priority, priority zero of a product leader in the process.

• 41:57Speaker 01

  Because of course, technical, technical leaders, engineers can make decisions on which models to use when, but I think the, the product leaders can provide that lens of, hey, this model has shown for these kind of populations has shown not so promising results in terms of ethics and alignment with human values. How might we shift the process and how might we avoid using some solutions that might not benefit the end consumer? The other aspect to this is also, in particular with safety alignment, it's important to identify how you would evaluate your solutions, both offline as you're developing it, but also in production.

• 42:41Speaker 01

  So the online performance monitoring of it. And that's also an area where product leaders are particularly helpful because they can provide both and share like, here, here are my like 100 things I want to know that the model does well on when it comes to safety. If it's not doing well, let's do a rollback. Let's find a quick solution. And oftentimes, as you know, in tech, things move extremely fast. We're also rolling very quickly and we're operating one click faster than usually we do. So I think just like taking that strategic view is extremely important. And those are some practical ways that I would suggest perhaps folks who are using LLMs could apply.

• 43:23Claire Vo?

  Well, you're setting the bar and I'm really glad that you are one of the product leaders out there that are building these kinds of products. So it's clear you take it seriously. Okay, last question, have to ask everybody. When AI is not listening, when it gives you terrible strategic communication advice? What's your tactic? How do you get the LLM to listen?

• 43:45Speaker 01

  Many emojis. It does really well with emojis. If you, and it also seems to adjust when you, when you put in emotion. I literally tell Claude, Claude, I'm sad, I have a bug. And it immediately fixes everything.

• 44:04Claire Vo?

  I do love the reasoning models that expose their reasoning because I just love reading like in the sidebar of cursor like, user is extremely frustrated and disappointed in my response. I understand why they're so sad. So, okay, emojis, emotions and XML. That's going to be the tagline of this episode. Prerna, it was so great to talk to you. Where can we find you? How can we be helpful?

• 44:29Speaker 01

  You can find me on LinkedIn and I'm happy to connect with folks. I'm also happy to share some of the tools I've developed and how you can help me just stay in touch and continue to engage and talk about all the cool things you're building.

• 44:41Claire Vo?

  Amazing. Well, thank you so much.

• 44:43Speaker 01

  Thank you.

• 44:45Claire Vo?

  Thanks so much for watching. If you enjoyed this show, please like and subscribe here on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.