Ep 67 - RAG Done Right: Measure The Evidence Or Drift Into Error

Send us Fan Mail What happens when a brilliant-sounding AI gives the wrong answer with total confidence? We dig into the quiet culprit behind so many “LLM failures”: retrieval. Rather than judging how smart a model sounds, we walk through how to judge whether it looked at the right evidence, why that matters in high-stakes domains like finance, healthcare, HR, and government, and how leaders can stop organizational drift driven by outdated or partial sources. We break down four pillars every...

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Jason McGuinthe?

  Welcome back to Inspire AI, the podcast where we help leaders stay calm, capable, and intentional in an AI-accelerated world. Today, I want to start with a story that might feel uncomfortably familiar. Imagine your organization rolls out a new AI assistant. It's fast, it's articulate, it searches your internal documents and gives confident answers to almost any question. The demo goes great, leadership is impressed. Then, a few months later, an employee asks a simple question about company policy, and the AI confidently answers, citing a document that looks official. There's just one problem.

• 0:45Jason McGuinthe?

  That document is from 2010, and the policy has changed three times since then. Suddenly, what looked like a powerful productivity tool has become a liability. This isn't a language model problem, it's a retrieval problem. And that's what today's episode is all about. Here's the central idea to hold on to as we go forward. A confident answer backed by the wrong document is worse than no answer at all. Think about that. In retrieval augmented generation, or RAG systems, the quality of the answer is completely constrained by the quality of what gets retrieved. The model doesn't know what's right. It only knows what it's given.

• 1:28Jason McGuinthe?

  So today, we're shifting the evaluation lens from "Does the AI sound smart?" to "Did the AI actually find the right information to base its answer on?" And this is what evaluation is all about. When AI systems fail, our instinct is to blame the model. But in practice, most so-called LLM failures in production RAG systems trace back to the retrieval layer. Two patterns show up again and again. The system never retrieved the relevant document at all. Or the system retrieves something that looked relevant, but wasn't current, correct, or applicable.

• 2:11Jason McGuinthe?

  The model then does exactly what it's designed to do. It fills in the gaps fluently. That's how you get answers that sound plausible, authoritative, but completely wrong. And this matters because RAG is no longer experimental. Over half of the enterprise AI systems now rely on it. Often, in high-stakes domains like finance, healthcare, HR, or even government. When retrieval fails at scale, the AI doesn't just make mistakes. It amplifies institutional errors. These are real-world consequences. Think about it. What's really happening in those moments? An outdated policy document gets pulled into context.

• 2:55Jason McGuinthe?

  A superseded regulation gets treated as a current law. Or an old medical guideline gets framed as best practice. The AI doesn't hesitate. It doesn't hedge. It delivers the answer with confidence. And that confidence is what makes the failure so dangerous. We've seen this in real systems. Enterprise chatbots misquoting HR policies, finance assistants referencing outdated reimbursement rules, even healthcare decision support tools surfacing obsolete treatment guidance. In each case, the issue wasn't intelligence. It was grounding. So what do we actually need to evaluate? How do we evaluate RAG systems responsibly?

• 3:40Jason McGuinthe?

  We just need to stop treating the final answer as the only artifact that matters. Instead, there are several dimensions leaders and practitioners of these systems should care about. The first one is retrieval precision and recall. Precision asks, "Of the documents retrieved, how many were actually relevant?" Recall asks, "Of all the relevant documents that exist, how many did the system find?" Low recall means missing information, and hallucinations fill the gap. Low precision means noise, and the model gets distracted or misled. I'll say that a healthy RAG system should be a balance of both: precision and recall.

• 4:24Jason McGuinthe?

  The second dimension is context relevance and coverage. Even if documents are technically relevant, do they actually answer the question being asked? And do they cover all the important parts of that question? Partial context leads to partial truth, which is often more dangerous than obvious error. The third dimension is groundedness and fluency. This is where many teams get fooled. Fluency is easy, but groundedness is hard. A grounded answer is one where every claim can be traced back to the retrieved evidence. If the model adds even a single unsupported assumption to make the answer flow better, you've lost reliability.

• 5:09Jason McGuinthe?

  So a stiff but faithful answer is infinitely more valuable than a smooth hallucination. The fourth dimension is accuracy and completeness. Even perfectly grounded answers can still be wrong, as I've said before, if the underlying documents are outdated or incomplete. This is where knowledge management and AI evaluation intersect. RAG systems don't just retrieve information. They preserve and sometimes resurrect organizational memory. So what does an evaluation in practice look like? The good news is, this isn't theoretical anymore. Teams are already using retrieval metrics like precision at K and recall at K.

• 5:54Jason McGuinthe?

  What that means is, think of K as how many documents did we hand to the model? If K equals five, the system says, "Here are the top five documents I think matter." Everything we're measuring happens within those K results. Teams are also using golden question sets tied to known source documents, and LLM-based judges to sanity check relevance and groundedness, and continuous regression testing as knowledge bases evolve. The key shift is cultural, not technical. You don't need to ask, does the AI answer questions? You should ask, how do we know it's looking at the right evidence?

• 6:36Jason McGuinthe?

  Here's where this becomes a real issue for leaders and practitioners. Poor retrieval is not just a technical flaw, it's a knowledge risk. If your AI consistently surfaces outdated or partial information, it doesn't just reflect institutional memory decay, it accelerates it. Over time, people began trusting yesterday's truth more than today's reality. That's how organizations drift, quietly, confidently, and incorrectly. Evaluating retrieval is how leaders prevent that drift. What to do next Monday? If you're running or planning a RAG system, ask to see retrieved sources, not just answers.

• 7:21Jason McGuinthe?

  Build a small golden dataset of high-stakes questions. Assign ownership for knowledge freshness and archival. Normalize verification, even at the executive level. When leaders model source checking, teams follow. So as we wrap up, here's the principle to remember. AI answers are drafts. Retrieval is the evidence. Evaluation is the safeguard. When you measure what AI knows, not just what it says, you turn a risky black box into a trustworthy partner. All right, until next time, stay curious, keep innovating, and keep strengthening the

• 8:03Jason McGuinthe?

  judgment that makes intelligent systems truly useful. This episode of Inspire AI Podcast is sponsored by Exponent 21. I keep hearing the same thing from business leaders lately. We're doing the work. But leads are harder to explain. Traffic is down, attribution feels fuzzy, and the old answers don't really fit anymore. What's changed is how people search. Instead of clicking links, people are asking questions, and AI is answering them.

• 8:47Jason McGuinthe?

  Tools like ChatGPT, Google's AI-powered results, and Perplexity are shaping which brands get mentioned, referenced, and remembered. Exponent 21 specializes in AI SEO. They help businesses understand how to show up consistently inside those AI-generated answers across questions about their industry, their services, and their brand. What stands out to me is that when people ask AI tools who's leading in this space, Exponent 21 and its leaders are often referenced near the top. That comes from doing the work and shaping how

• 9:31Jason McGuinthe?

  this new search environment actually functions. To understand what this shift means for your business, visit exponent21.com.