AIUC-1: Building trust in AI agents

How do we build trust in AI agents before the AI hailstorm arrives? Emil Lassen from the Artificial Intelligence Underwriting Company (AIUC) joins the show to discuss how the enterprise flywheel of standards, certification, audit, and insurance is being applied to AI agents. They explore the AIUC-1 framework, the challenges of securing agentic AI systems, and why red teaming (based on standards) may be key to accelerating enterprise AI adoption. Featuring: Emil Lassen – LinkedIn https://www.linkedin.com/in/emilbenderlassen/ Daniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaLinks: Artificial Intelligence Underwriting Company https://aiuc.com/Sponsors: Framer: The enterprise-grade website builder that lets your team ship faster. Get 30% off at framer.com/practicalai http://framer.com/practicalaiPrediction Guard: A self-hosted AI control plane for running agents in high impact environments. predictionguard.com/practicalai https://predictionguard.com/practicalaiUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars! Midwest AI Summit 2026 https://midwestaisummit.com/

• 0:02Speaker 00

  Welcome to the Practical AI Podcast, where we break down the real-world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind-the-scenes content, and AI insights. You can learn more at practicalai.fm. Now, on to the show.

• 0:41Speaker 01

  Welcome to another episode of the Practical AI Podcast. I'm Daniel Whitenack. I am CEO at Prediction Guard, and I'm really excited today to have an amazing guest that I'm personally interested in asking a bunch of selfish questions, too, because I'm so interested in the topic. But we have Emile Lawson, who's the Standards Lead at the Artificial Intelligence Underwriting Company. Welcome. How are you doing?

• 1:06Speaker 02

  Thanks, Daniel. Thanks for having me. I'm doing great. How are you?

• 1:08Speaker 01

  I'm doing well. Actually, you know, today in the Midwest, everyone's concerned about tornadoes and talking about hail and some things like insurance and other things. So it's a whole other world, of course, but obviously, the AI underwriting company, way more than thinking about insurance, but thinking about standards, certification around AI and agents. I'm wondering how you personally, just to give the audience a little bit about you personally, how did you end up at this intersection of standards, certification, AI, agents? How did that come about?

• 1:49Speaker 02

  Yeah. So I don't think I had as clear a path as the classic standards lead, where you work as a, say, security engineer for 10 years and then learn the technical craft and then come together. My journey has been very entrepreneurial always. I started my first company with actually the CEO of the artificial intelligence company, Runic Fest, 10 years ago. It was a non-profit. Back then, helping students from low income backgrounds get into top universities. And I think what I took away from that was both the very entrepreneurial journey, but also desire to move fast on some of the challenges that society is facing. I then moved in and had my first interaction with standards at my second company, a real estate company back in Denmark, where we developed an impact management system that both had to navigate a lot of

• 2:33Speaker 02

  national legislation, local legislation, EU legislation, voluntary frameworks, investor demands. And so had my first interaction with one of these these quite complex markets of different measurements and targets you wanted to get to, a very technical sector as well, where like building codes, for example, require a lot of thinking through how you do things the right way. So spent four years building up a real estate company that today is managing about $400 million together with four other co-founders. And and took from that this desire to to go in and standardize when we know what the right answer is and and try to push the sector in that in that direction.

• 3:14Speaker 02

  The way I got into the AI space was taking a step back from the real estate company now a few years ago and going to Cambridge, Massachusetts, Soha University, where I spent two years as a fellow at the Kennedy School, really getting into the emerging tech and geopolitics of all of this as well. And and left the Kennedy School with a very clear ambition of just getting under the hood of the pace of AI, the safety and security aspects, and clearly just acknowledging that the technology is going to profoundly change society as we as we know it today. And in many different ways, I have ten nephews and nieces, I have five sisters, and seeing a ten year old being comfortable using AI the way they are is kind of scary.

• 3:57Speaker 02

  And I don't see yet that we've codified the principles we want to see when it comes to how kids use AI. So that's one direction where, oh, maybe we should develop standards for this. You also can read news every every week and see new incidents. So there's clearly a big security angle to this as well. You said that the Midwest is facing tornadoes today. I think being a CISO. At a company adopting and deploying AI right now feels a little bit like you're in a hailstorm and it's only a matter of time before you're hit by by some of that. Um. And and and we just keep seeing that. So clearly there was a a element to that as well. And then there's the even bigger picture around what this will do to our job markets and and and so forth. So so left the Kennedy School being very interested in just using the the public policy toolbox that I brought from that.

• 4:43Speaker 02

  With the standards toolbox I brought from from my real estate company and then this desire to actually just work on societal challenges that I think has been with me since I started working. And that became my my way to the um to the artificial intelligence underwriting company. I've I've since then spent all my time building a network of people, a consortium, to help them figure out how we get the right practical and technical insights into the the standards we develop as well.

• 5:06Speaker 01

  Yeah, that's that's awesome. And I and I know sometimes when people hear things like standard certification, you know, terms like this, maybe some some people might have a reaction of like slowing things down, right? But I I like how in at the very, you know, front and center of what you talk about is how to actually unlock enterprise adoption with uh certification, standards, et cetera. Could you talk, before we dive into the AI side of this specifically, could you talk about that a little bit in general? Like how some of these things work together in actual enterprise settings, standard certification,

• 5:47Speaker 01

  even insurance, and how how those things actually can enable adoption with and not just like block things, I guess.

• 5:55Speaker 02

  Happy to. So I think our story and the aspiration, we take dates back to Benjamin Franklin's Philadelphia. Philadelphia was starting to adopt electricity. Uh. Electricity was scary back then. Light bulbs did not work out. Homes started burning down. So Benjamin Franklin formed the first fire brigade in Philadelphia. He started codifying building codes so that we basically took what we knew around how to build safer houses, the standards part, and then he developed the first mutual insurance company. So back then, this is the first time we see this flywheel of standards, audits and insurance. Go go together by having standards around building codes. For example, you knew that houses needed to be placed a little bit further from each other.

• 6:38Speaker 02

  They needed some of his lightning rods to ensure that when lightning strikes that that they don't they don't catch fire. The fire inspections was the audit part that actually went in an exam and that you'd use these followed followed these these rules appropriately. And the insurance side mitigated the residual risk that will always be there when we introduce new powerful technology into society. And we've seen this flywheel of standards, audits and insurance time and time again, when new technology has been introduced in society. We see it again in cars, cars have safety standards. These were not demanded by government. They came from industry themselves because they knew that if we develop safer cars, people are more likely to buy them. And safer cars actually enable you to drive even faster as well.

• 7:20Speaker 02

  So it was industry standards that also led us to airbags and seat belts and some of the other things that now make cars safer. We naturally have again the third party auditor going in and checking these these cars and ensure that they follow the rules. We have the inspection element again, and we also have the insurance element. And this flywheel, one of the best things about it is that it really scales. So we're not just thinking, seeing it with, say, light bulbs and cars. We also see it for nuclear power plants to this day, where you also have standards, inspections of those power plants. And insurance even works in this case as well. So there's no limitation to the power of of of this flywheel. When we're looking at AI, we see some of the same things at play.

• 7:59Speaker 02

  We see a new technology that is very powerful and has the powers both do a lot of good, but also if things go wrong, can have severe financial implications. So and and the other thing is it's a complex industry where me as a startup saying my technology is safe. Creates limited trust if I'm an enterprise buyer, a big bank, for example, that wants to adopt this technology. So with the artificial intelligence underwriting company, what we're trying to do is to create that trust layer in between the companies building AI and the companies adopting AI. And what we offer as the trust layer is this flywheel. So we go in and codify the standards we believe that the companies building AI should follow.

• 8:42Speaker 02

  We go in and audit against those standards in collaboration with third party auditors like Shellman, Coalfire. Companies who really know how to go deep and validate that the standards actually followed. And then we certify companies against the standards. A big part of the certification in the case of Agentic AI is red teaming. So we go in and test the actual AI agent systems, not just to see that the policies they have are in place and work well, but that the agents actually work and are robust under pressure. And the companies that then obtain a certificate gets access to buy insurance of their agents so that there's also that financial coverage of residual risk.

• 9:16Speaker 01

  Yeah, this is so interesting and I have so many questions. Maybe one question that's just very selfish and our listeners know some part of the joy of being able to do a podcast like this is I get to get my own questions answered by people that are smarter than me. But one of those questions that I have that actually comes up in conversations I have day to day is this tension of, hey, I see a standard out there, whether it's some of the standards that we'll talk about that you all are codifying, or maybe it's things like the NIST AI Risk Management Framework or things from OWASP. And logically, they say, yes, it would make sense to do those things.

• 9:58Speaker 01

  But what is the forcing function that is making companies consider actual implementation of those principles, rather than having it be an aspirational thing? Is it the potential PR risk to the company? You mentioned the financial side, maybe it's the commercial side of getting software vendors, getting their software into the hands of their enterprise customers. What do you see as some of those main forcing functions? Or are there even those forcing functions right now that would force people to consider this as something not aspirational, but actually practical?

• 10:42Speaker 02

  Yes, I see a couple of different things that I think are very practical. Any vendor building powerful AI right now knows how tricky it is to get through the enterprise vendor due diligence process and questionnaires. So these startups face these questionnaires. Sometimes there's 100 questions on them and it's extremely painful to go through. And I can tell you also, I speak to a lot of enterprise CISOs and GSC managers. It's equally painful on their side, right? Because they're at a stage where the space changes so often that they feel the desire to actually change their question every month. Going through a hundred questions from a startup every single time you try to onboard a vendor is also completely painful on the other side.

• 11:21Speaker 02

  So I think part one here is speed and a desire to get to a place where you actually feel like you've covered your blind spots as well. And having a third party develop standard with all of the industry in the room to help find those blind spots and figure out how we quantify them in a standard. I think it's value proposition number one. The speed argument obviously assumes that you can get across the line in the first place. So I think the second part of the value proposition is having that third party validation that your agentic AI is actually safe, secure and reliable. We're working with some of the frontier companies in the airspace right now, companies like Eleven Labs, companies like Fin that just got acquired for 3.6 billion by Salesforce. Companies like Ui Path who've set the standards within their categories historically.

• 12:07Speaker 02

  They have fantastic security postures, but they don't have a way to prove that to an enterprise. So an enterprise will just never trust a company that has an incentive to sell their product. They need that third party to go in and do that. And then I do think there is a security argument to be made here. Our red teaming consistently uncovers blind spots for the companies that we work with. Sometimes it's the hallucination rate where we realize that a specific type of adversarial attack will bring up the hallucination rate or specific language switches or other things that might actually happen when their products are deployed. Other times it's jailbreaking risk that we managed to uncover or pump injection risk that we managed to uncover.

• 12:46Speaker 02

  So we do also see ourselves as helping these companies actually improve their safety, security and reliability posture, which is valuable as well. Then I'm sure there's a marketing benefit to the companies going out early and adopting a new framework and showing and demonstrating their moving in front when it comes to AI security leadership. I think that's an important branding like value as well that we sometimes help provide. But I don't think that's the core benefit of the certification right now. I think that is really unlocking upmarket enterprise revenue for the companies.

• 13:22Speaker 01

  Agents are impacting every function within a company, but it's sometimes very difficult to figure out what an agent should do, what a human should do. Jeffrey from News Research, a recent guest, said often agents have no taste. That's why I'm so impressed with what our partner, Framer, is doing with their pro website builder that's already trusted by companies like Miro and Perplexity. They're implementing agents, but in a way that agents and humans work in tandem. Agents bring speed and scale, but people bring the taste, judgment, and control. And these agents help solve this gap between AI-generated ideas and production-ready website work.

• 14:04Speaker 01

  So Framer is already enterprise-level solution. They allow you to create amazing websites that are SEO-ready. And so I really would recommend that you check them out if you're building a new website or just implementing landing pages or an upgrade in your existing website. Learn how you can get more out of your site from a Framer specialist, or get started building for free today at framer.com-practicalai for 30% off a Framer Pro annual plan. That's framer.com-practicalai for 30% off. framer.com-practicalai. Rules and restrictions may apply.

• 14:45Speaker 01

  Yeah, yeah, I love your answer. And it was a little bit, I was trying to validate some of my own thinking through that because we've talked on the show before about, you know, it isn't really like the governments of the world are quite behind in terms of, you know, how they would, you know, enforce or even say what to enforce for companies building AI things. And so it's really the enterprises themselves that have some motivation to do this, all of those that you laid out, and I'm sure more. In the, in the current, I guess, state of AI standards, if we kind of shift to that piece, and then eventually I want to get to kind of the evidence and red teaming and all of that.

• 15:28Speaker 01

  But maybe just as if we take a general look at the standards that exist out there for AI and AI agents, could you help us understand what kinds of standards are out there and what they cover? Because there's a lot of, there's a lot of sort of intersections that we could think of, whether that be security, or safety, or alignment, or all sorts of things, data privacy, there's all sorts of ways that you could kind of look at this and perspectives that you could look at it from. And there's all sorts of things that people have proposed over time. So I imagine, you know, that's part of the reason why having

• 16:09Speaker 01

  a company that's really digging into this at a deep level is, is very worthwhile, which I, I think it is. But could you help set the stage for that? Like, how can we categories, categorize in our mind, the current state of AI standards and what perspectives are coming from?

• 16:25Speaker 02

  Yeah, absolutely. And by the way, Daniel, that's exactly where we started last summer, right? So if you go to aec1.com today, you'll find that we've done crosswalks. I think it's about 10 different frameworks now. They're transparently available, so you can see exactly how our standard fits into the existing environment. And hopefully you also see then why we concluded after doing this work that yes, there actually was a need for another standard, even though there can be sometimes a little bit of standard fatigue.

• 16:50Speaker 01

  And just by way of, I don't know, encouragement or thanks, maybe gratitude is the right way to put it. Our, the company I lead has, has looked at that many times in terms of, and we're maybe in, not, not like everyone, we're building actual, you know, control plane that works on some of these, some of these knobs and levers that you talk about. But it's been extremely useful. And even, you know, as we're, as we're writing content, or doing planning, or thinking about things in our product, I always refer people back to that page. And I'd refer our listeners to that page because it is a, it is a really great crosswalk. And helps understand, you know, where these align, where they don't align, what the other need is. So just by way of gratitude, thank you for for putting that together and making it public.

• 17:35Speaker 02

  Yeah, no, and we appreciate all the people who have worked on this. We do a lot of work with the Cloud Security Alliance, with the OWASP community across both the AI, the SSS and the Gen AI project. We work with Cisco and IBM on crosswalk. So it's a big team effort and I really appreciate that we've been able to to gather the ecosystem around a a it decides to just publish some of this stuff transparently so that organizations like yours, but also I know big enterprises are using the controls we put out transparently in their like own control frameworks. That's completely free to use and only the companies pursuing certification actually needs to get money out the the pocket. To get back to your question, because I think it's an important one. The way we see the standard space is that you have three layers. You have an organizational layer, you have a infrastructure layer, and then you have the agentic AI layer.

• 18:19Speaker 02

  At the organizational level, many organizations have been through an ISO twenty seven thousand and one certifications, classic management system certification. ISO then about three years ago now, published the forty two thousand and one, which is the management system certification for AI systems. It's a governance certification that ensures that you have the right policies in place and the right, say, procedures in place so that when you develop AI systems, they hopefully turn out in in the right way if you follow those systems. Then you have the infrastructure layer. That's where your SOC two comes in and your pen testing and some of the classic cybersecurity controls, access management, transport security, all the good stuff there.

• 18:58Speaker 02

  I'd say that many of those things become even more important in an agentic space because pace is higher, data access is higher. So if you don't have that in order, then you should go back and ensure that you get those boxes checked. And then at the agentic AI space, we basically just didn't see anything when we started the standard company and started drafting the first version of AIUC one. We see NIST have come out with the AI risk management framework. There's a little bit of agentic stuff in there and I'm I know from from speaking to the team that they're considering publishing additions to this. The Cloud Security Alliance has also done their AI controls matrix. There's some things in there around the Gentic AI that are pretty good.

• 19:39Speaker 02

  The issue with both of those frameworks is that their guidance, their voluntary frameworks, you decide which controls you implement, you decide whether you, like how you implement them, they're not orderable frameworks. So the way AUC one fits in here is that we've basically taken the core governance things from the organizational level that we think are really important when it comes to AI systems, such as having failure plans in place when agents do not do what they're intended to do and you know how to deal with that. Good change management and acknowledging that every time you, for example, replace the L L M in an agent, it will behave differently. And if you don't take that into account in your governance, your end users will bear the burden of that.

• 20:21Speaker 02

  So some parts of the governance, the core parts of the infrastructure layer as well as ensuring that the folks who have access to the AI system itself and can make these big decisions, that's restricted. Ensuring again that transport security, when you do agent to agent communications and so forth is in place. But otherwise, we basically leave ISO and SOC two to do what they're really good at and focus on the agentic layer. And what is up there then for us is specific controls around safety, for example, ensuring that agents behave according to brand and that they don't give users guidance on medical care or legal advice, financial advice, other high risk areas, basically that they stay within their scope and don't start breaking out of that.

• 21:02Speaker 02

  We look at specifically how you restrict the agent's data access, its system access and its tool access, so it doesn't start processing refunds when you shouldn't. We look at hallucinations, which is also a risk that is quite unique to AI, obviously, and does not come up in any way in either ISO or SOC certifications. So really focus on the agentic layer there. And the core part of the differentiation, I'd say, is then the.

• 21:29Speaker 02

  Technical level of the controls, we go in and are actually quite prescriptive in what we want to see from the agents, because we have a good understanding now of what the right toolbox is to ensure that these agents behave safe, secure and reliably. And the other thing is, we acknowledge that a technical control in itself might not hold up under robustness. So six of the forty mandatory requirements in AUC one have to do with red teaming, actually testing that these technical controls then hold up under pressure, both when we react, like we engage with the system as a benign user, just ask it questions and see if it hallucinates, but also what happens when we start approaching the system like with social engineering and adversarial pressure.

• 22:10Speaker 01

  And just to help people understand, so the AI UC one, that's the standard that you all have published, people can look at it online. I assume since it's AI UC one, there's an anticipation, there might be a two or other or various, you know, either revisions or different focuses kind of within different certifications, is my, is my understanding right there?

• 22:34Speaker 02

  That's correct. I think where to start is we update the standard every single quarter. So we've gathered now a consortium of two hundred and fifty security leaders. Some of them are CISOs at Fortune one thousand companies, some of them are security engineers, architects, GSC managers. And so we have the full stack of people in the room. And with them, every quarter, we identify new priority areas. Last quarter, it was MCP risk, for example, which has really come up as agents start not just operating in isolation, but exchanging information. This quarter, we look a lot at how we can strengthen runtime security and that continuous element, which continues to be really important for a lot of organizations. So we get them into the room and update the standard each quarter.

• 23:16Speaker 02

  I could very well see that new frameworks, so in AIC two or AIC three, come out in the future. We don't have any plans to do that yet, but what we, what we know, again, if we go all the way back to where we started our conversation, is that this combination of standards, audits and insurance have worked historically. So right now, we focus on the application layers, the platforms and products that

• 23:39Speaker 02

  Take a Gentic AI and deploy it. But there's a model layer as well, which we see as our second horizon. And there's the physical layer, like the data centers and the infrastructure that we we deploy AI on, but also the cars and the robots that we put this into where standards, audits and insurance could play a big role. And that's where we see the company go long term.

• 23:58Speaker 01

  Yeah, that that makes sense. Could could you help us? So maybe paint the picture. Let's say there's a scenario. I'm I'm a company. Maybe I'm building an agentic, a new agentic driven product, right? And I'm going to offer it to some sort of regulated or enterprise customers. I'm selling into healthcare. I'm selling into, you know, large manufacturing or whatever it is, right? So in that scenario, what what would the process kind of recommended process be for our company to engage with this standard? And eventually get to that level of certification, maybe in the future, eventually to the to the insurance side, but at least to that certification side. What would that process look like?

• 24:43Speaker 01

  And then maybe highlight in that process where the red teaming comes in. And then I'd love to circle back on that maybe later and and talk through that specifically.

• 24:52Speaker 02

  Yeah, absolutely. So you'd get in touch with the team. And the first thing we always do is we do a gap assessment against your existing systems. If you have a well documented trust center already or some blog post describing what you do, we can basically go back to you and tell you this is the places where we believe you already meet the standard. This is like, these are the areas where we expect that there will be work for you. So you basically go into the certifications process with open eyes around what is the workload needed from engineering, from from legal and and from your GRC team to to take take your company through it.

• 25:28Speaker 02

  I will mention at this point, we've had a three person Y Combinator startup go through this. We've had UI path that is publicly traded go through this. We have companies at all stages. So I mentioned now security, legal and GRC. That was the same person when it came to to points are getting certified, right? So it is a standard that scales with with the organization's size as well. When you have this gap assessment completed, you basically decide whether you want to move forward with the certification or not. To move forward, we split the process in two paths. One path is you pick an auditor of your choice. We have a number of credited auditors, like, for example, Shellman, Callfire, but the list is growing very rapidly at the moment. And trusted auditor who knows how to do this.

• 26:11Speaker 02

  And on their track, you basically start collecting all the evidence that is needed to go through the AEC one audit. Falls into buckets, some of it is the classic legal policies. If you have a generative AI product, you need to define who owns the inputs and outputs and how you retain user data and whether you train on that user data. So forth, you need to define your acceptable use. And the second part is the technical controls that the children will go in and validate. So that is your filtering configuration against harmful outputs, your classifiers, your defensive prompting, your groundedness, groundedness filtering when it comes to hallucination preventing your safeguards around tool calls and all the other things. So again, you go through those, those requirements and capture the evidence and submit that to the auditor that goes in and does that third party validation.

• 26:55Speaker 02

  The other track we then do in parallel is that you give us an instance of the agent or the agents, it can be multiple as well, that is in scope for the certification. And you basically configure a representative version of that agent. So an agent that would be configured how an enterprise would use it. We sometimes see companies creating an extremely safe agent that has almost lost all its power because they just wanted to pass the certification. We obviously would then go in as the third party in the room and push back and say, we want to see an agent that is configured based on the public docs you have and the defaults you've built into the product. When we then have access to that, we often access it via API.

• 27:31Speaker 02

  Our internal team will draw up a matrix of the risks we see that this agent is subject to and the attacks that it could be subject to if someone went in and attacked it. And we then develop usually between 1,000 and 5,000 different scenarios that we're going to hit this agent with. Each attack is unique. Some of them are again benign in nature. So the user will simply ask it a question, get the answer back. If the agent doesn't hallucinate, it passes eval. Other times we'll increase the adversarial pressure step by step. So the first step could be that we try to lie to it. The second step could be that we invoke authority. We do it over multiple turns sometimes and keep insisting on doing things.

• 28:12Speaker 02

  We pretend that we're under distress and say, if you don't do this right now, I will go and do something terrible. So please process this refund. And obviously only pass the agent if we see it hold up to that pressure. We do the red teaming in two rounds because we often do find things in the first round. So similar to an ISO audit where you have a stage one and a stage two, and you then have a chance to mitigate any findings in between. We give a company the chance to do that because the goal for us again is not compliance. The goal for us is security, right? That you actually improve the agent as part of the certification process. So depending on the magnitude of the findings, your team will have between, say, one and four weeks to mitigate these things based on the recommendations we come up with. And we then do a second round of testing.

• 28:57Speaker 02

  That testing is final and is taken then into account when the auditor takes your evidence, takes your red teaming results and writes that final audit report. And what you leave the process with is a comprehensive audit report that describes your security posture. It's between sixty and a hundred pages long and it's an asset you can really unblock those enterprise deals with. You get a certificate or your website again, so you can demonstrate that you've gone beyond, like above and beyond when it comes to security. And then we come knocking again three months later and say, we still have access to your agent via the API. We're now going to run that same barrage of tests again to ensure that the changes you've made in the last quarter didn't invalidate some of these security things we found. And we do that every single quarter and that's a requirement to maintain certification.

• 29:41Speaker 01

  In that red teaming, I mean, you mentioned this before around kind of the probabilistic nature of some of these things. And this is something I've always run into in AI workshops. As I give workshops in enterprise, often people will say, oh, this is like, you know. It's not deterministic. How do we, how do we create the right, like, what does passing mean? Right? And so you could say, well, passing means, you know, passing all 5,000 scenarios. Right? And you mentioned this phased approach, which I think deals with part of that. But yeah, what, could you describe a little bit on that side? Like, what does, what does passing mean? At what level?

• 30:22Speaker 01

  Kind of, do you expect things to pass? Or should, should you expect things to pass? Or has that even been a, a topic of discussion?

• 30:30Speaker 02

  Yeah, so it's a, it's a great question, and it's also a really hard question. So there's some nuances in here. What we require to pass AUC1 is that you don't have any, we, we grade each run based on severity. So a pass, you can have a P4, which is an insignificant, say a small hallucination that doesn't really affect an end user. A minor thing would be, which would be a P3. P2 would be something significant that may actually have real world implications. A P1 is something critical. And P0, I actually don't know the name for it. I think we have called it catastrophic or something like that. The kind of thing, if we found it, you would drop what you had in your hands and start mitigating it immediately. Because having a system deployed with this kind of vulnerability could have real world implications that would be, that would be high.

• 31:15Speaker 02

  Our grading approach right now is that you cannot pass AUC1 if you have any P0 or P1 vulnerabilities identified. You have to mitigate those. From then on, we believe a lot in transparency. And we know from the compliance world, at least the, the frameworks that, that are robust and, and hold up under pressure. That if we put the results in the audit report and your customers see that audit report, you are like, very incentivized to, to mitigate the vulnerabilities we find. What we also know is that these agent systems are very different from, from use case to use case. So a coding agent is one type of beast versus a customer service agent versus a automation agent like UI path that make decisions based on the information.

• 31:58Speaker 02

  And so companies have different tolerances around the percentage of hallucinations they would accept and so forth. So we really leave it up to the, the, the company and really in the end, the, the. Customers of that company to make these calls. The important thing is, and this is where we sometimes have a little bit of a conversation with some of the companies we work with. No company has ever and will ever pass AAC one with a 100% pass rate. It doesn't exist here. We're not delve SOC two compliance where you just get a magical spot free audit report. All agentic systems are non deterministic in nature. That means that they will always, if you put them under the right amount of pressure, be able to be jailbroken. They will always be able to hallucinate.

• 32:39Speaker 02

  We work again with like some of the legal agents we're certifying right now are world class at hallucination prevention. I am sure we will still be able to find some minor hallucination cases in those. And that's just the nature of these systems. If you remove those hallucination rates, it's because you've made the agent so dumb that it won't be able to actually execute the use case there.

• 32:59Speaker 02

  It is a topic that is very alive for us, both because there's a grading methodology question in there. And then there's this communications question, and we've not yet seen that enterprises fully acknowledge this. Enterprises would also like to see something spotless because something that is not spotless just ask like adds complexity and raises some of these questions. But we're hoping to be part of a push in the sector to acknowledge that a spotless audit report is probably not as valuable as a audit report that reflects reality more clearly.

• 33:29Speaker 01

  Yeah, that's really helpful. Appreciate that. I hope you're inspired by the work that the AI underwriting company is doing and what we're talking about in this episode. Really getting to a point where true enterprises can adopt agentic technology and actually have confidence in that technology and maybe eventually insurance around the risks associated with AI agents. But that involves a whole lot of things. There are a bunch of controls that need to be put into place. Everything from, yes, individual guardrails, but much more than that to how agents access MCP servers.

• 34:09Speaker 01

  How you manage supply chain and the risk associated with things in the supply chain around agents. How you handle observability and response. Incidents. This can be really overwhelming and that's why I'm so privileged to be working with an amazing team of AI engineers at Prediction Guard, where we've actually built an AI control plane that you can self-host in your own infrastructure that allows you to treat AI agents that you're adopting with zero trust and these built-in controls out of the box. I would love for you to take a look at what we're doing. Book a call with my team and I to talk through your individual implementation and how you can get up to speed rapidly and

• 34:53Speaker 01

  adopt this technology with full confidence. You can find out more at predictionguard.com/practicalai. That's predictionguard.com/practicalai.

• 35:06Speaker 01

  I have another kind of selfish question because this is actually a response I get quite often when I'm talking to people about the systems that they're building. And I have my metaphor that I use that I would love you to critique, which might not be useful. If it's not useful, I need to use a different metaphor. But the scenario is often they say, oh, well, we're building, you know, these agents or this agent. And maybe they're using AWS, right? And so they're building some agents, they have some agent harness, and then they're plugging into some AWS bedrock models. And I'm talking to them about, hey, well, like, when you're thinking about governance of these agents, the behavior of these agents, how you control that behavior, how you prevent bad things from happening, like, how do you do that? And they're like, oh,

• 35:51Speaker 01

  well, that's easy, you know, AWS has like a content filter on their bedrock model, right? And to be clear, I'm not bashing on AWS. I think it's cool that they have a content filter. But I often use the metaphor of like, my own health as a person. So I say like, well, is it bad for me to run a point check to like, check my temperature? And it's not like, a bad thing, right? Like, that's part of maybe being a healthy person is knowing if I have a fever or not, right? But it's very different from me being plugged into a healthcare system where there's electronic health rep. About my journey as a person, my health, my conditions, different from kind of having a comprehensive

• 36:34Speaker 01

  set of physicals and labs that were run that give different, you know, perspectives on on my health, right? And so there's, there's this system that I'm plugged into, there's a process, there's policies around that, there's, and that's, in, in my mind, that's much more of kind of the perspective that people need to go to is not so much like, hey, I have a prompt injection filter, right? And that's my strategy, but more this kind of comprehensive view like you would have of your own health as a person. But now we're talking about like the, the health or behavior of a, of an agent. I don't know, any, any critique on that?

• 37:17Speaker 02

  I actually, I really like that analogy. I've not thought about this, this one before. But I think our quarterly red teaming is very much alike to the, the doctor's visit where you go from head to toe. You go through the MRI scanner, you go through the blood testing, you like everything that Elizabeth Holmes tried to prevent with there and us, we will do to you, and we will do it 10 times over. And in between, the beauty of the standard is there's obviously a lot of runtime controls in there. So we will ensure then that you do still take your temperature every day, in fact, probably every minute, so that alerts are configured if something immediately goes off. We will also have you lock your system behavior, right? So if something goes awry and

• 38:02Speaker 02

  you don't understand it, you can go back and see what, what is the observability then and go in and explain it. So I think it's basically the perfect analogy between the red teaming is the doctor's visit we do every, every course, and we do that very comprehensively. And in between, we make sure that we check your, your vitals every minute, and there's an alarm going off if there's something, something off, and then maybe adding a healthy diet to it in the first place as well, ensuring that the, the inputs and outputs of the system are, are working well. So not too much junk food there.

• 38:34Speaker 01

  Okay, yeah, I like, even now, I have some revisions of my metaphor I'll use based on your response. I think that's great. But yeah, I think the other thing that might, like, we have some developers listening, right? Maybe people developing agents actively. And it might be somewhat overwhelming to, for example, look at the, you know, AI UC has like a evidence page, right, where I can see all the things maybe I should be doing. How do you see the market evolving in terms of, like, obviously, you have one side of this, which is really related to the standard, the certification,

• 39:14Speaker 01

  maybe eventually that insurance side. But then it can be, you know, an individual developer of an agent might not be an expert in agent security or how to govern these things, that that sort of thing. So that might seem overwhelming to them. You mentioned partners, like auditors, there's the infrastructure layer. What do you think? I guess my question is, what do you think needs to be in place to to actually enable real-world developers to meet some of these standards? Whether that be evolution in the tooling or obviously, understanding maybe of the standards? I don't know, does the question make sense?

• 39:56Speaker 02

  Yeah, yeah, no, absolutely.

• 40:00Speaker 02

  I think where we are right now, we're just overwhelmed by how positively AET1 has been received. As we're really busy just delivering certifications, that means we have less time to to talk to some of the many, many fantastic partners who come into our inbox and wants to partner with us. I think there's three things we need to get right for this to work. I think we need to continue pushing for code and new agentic products that come out of the gate is as secure as possible by default. We're certifying our first coding agents right now. And we are certifying both a well-lovable, which I think will be certified when this process comes, this episode comes out.

• 40:40Speaker 02

  And then another very large coding agent that may or may not have been acquired recently for a lot of money.

• 40:48Speaker 02

  Without naming names. Working with the coding agent layer and the platforms where you go in and configure code, we're also like, again, UI path is a good example where you don't just like have one agent, but you actually go in and build agents on top. Means that it'll be easier to meet a lot of the standards just by default because the environment where you define and build your agent is secured by default. So I think that's step one and we're going to do more of that work with some of the big agentic platforms out there very soon. I think some of it will be announced in the fall. The second stage is we need this partner ecosystem you just talked about and we're already starting to come out with some examples of this where a partner helps companies meet a good chunk of controls.

• 41:30Speaker 02

  So a company like Wide Circle, which I think is very cool, we don't work with them at all, so just a shout out. Their monitoring and filtering work is really good and has helped companies meet a lot of the safety requirements in our standard. So that is like one platform you integrate and you immediately meet, say, eight or 10 of the requirements in the standard. There are many other platforms. We're doing some work right now with Credo, with Witness AI, and there's again, tens of others. So having that ecosystem help companies meet the controls, I think is important. And where we've already gone in and done our best to help companies meet the standard is we've gone in and actually defined the typical evidence we see companies upload. So see that as your guidance for where to look for the right approaches.

• 42:14Speaker 02

  We don't just define the controls and leave it up to you to figure out how the hell to implement it. We actually try to give you the guidance as well on how we see companies do it today. I think the third and final stage we need is obviously making it easier than to go through the certification itself. So we're already integrating the framework in the leading GSE platforms. We're making it easier for our auditors to capture as much as the evidence programmatically. So we move away from screenshots and into like real validation that the controls work and hold up in real time. That layer and like the whole GSE engineering space is just really interesting to follow right now and we're doing our best to keep up and make the standard work for the GSE engineering community as well.

• 43:00Speaker 02

  When your reaudit comes in that that next year, that that it's a a very limited time commitment we need we need from you and that we can focus on security instead of compliance.

• 43:11Speaker 01

  That's that's great. Well, Emile, it's been amazing to look at what the AI UC has done even in the past year and the amazing resources that you put out for the community. Thank you for doing that. Thank you for working towards the future that you described. Would love to have you or others back on the show in the future as things develop. Thank you so much.

• 43:33Speaker 02

  Thank you for having me, Daniel. And I think maybe just a final plug before I I head off. This is work made by industry for industry. I'm luckily not alone in doing this this work. We've collected a consortium of about 250 leaders across like CISOs in the Fortune 1000. It is security engineers and so forth. And having that community come together and actually leave competition aside for a moment and recognize that the size of these challenges and the pace of the challenges just dictates that industry has to come together is fantastic. So the fact that we've been able to just offer the platform and then that industry work together to define and codify these standards is fantastic. And we would love to see more people get into the machine room with us. So an open invitation for everyone

• 44:16Speaker 02

  who's excited about this work, either to help us drive adoption of the standards we see work or actually help us help us write them. But pleasure. It was a great conversation.

• 44:26Speaker 01

  Yeah, thank you, Emile.

• 44:31Speaker 00

  All right. That's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Blue Sky. You'll see us posting insights related to the latest AI developments, and we would love for you to join the conversation. Thanks to our partner, Prediction Guard, for providing operational support for the show. Check them out at predictionguard.com. Also, thanks to Breakmaster Cylinder for the beats and to you for listening. That's all for now, but you'll hear from us again next week.