Exploit Development, Agentic Vulnerability Discovery, and Fake PoCs

AI is changing cybersecurity, accelerating vulnerability research and exploit development while raising new questions about reliability, risk, and guardrails. Daniel and Chris talk with Spencer McIntyre, Director of Exploit Development at VulnCheck, about how agents are being used to discover vulnerabilities, why many proof-of-concept exploits (POCs) fail to deliver, and how initial access intelligence helps security teams prioritize real threats. They also explore the limitations of AI-assisted exploitation, the challenges of AI guardrails for security researchers, and what these developments mean for the future of threat intelligence. Featuring: Spencer McIntyre – LinkedIn https://www.linkedin.com/in/spencer-mcintyre-b48987194/Daniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaChris Benson – Website https://chrisbenson.com/, LinkedIn https://www.linkedin.com/in/chrisbenson, Bluesky https://bsky.app/profile/chrisbenson.bsky.social, GitHub https://github.com/chrisbenson, X https://x.com/chrisbensonLinks: VulnCheck https://www.vulncheck.com/Go Exploit framework https://github.com/vulncheck-oss/go-exploitInitial Access Intelligence https://github.com/vulncheck-oss/initial-access-communitySponsors: Hostinger https://www.hostinger.com/in: Create your website today. Go to hostinger.com/practical and use practical for 10 percent off now. Resources and Events: Register for upcoming webinars here https://practicalai.fm/webinars! Prior Webinars from our partner Prediction Guard https://www.youtube.com/playlist?list=PLRpRdFmBfdF6boDgAD3nNaLm8Yx4KN2VtMidwest AI Summit 2026 https://midwestaisummit.com/Chris’s new book The Art of Swarming will be published soon, written to be the seminal foundational book for the fledgling swarming industry, defining the principles and architecture that enable autonomous agentic UxS swarms with emergent unified embodied intelligence.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Speaker 00

  Welcome to the Practical AI Podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind-the-scenes content, and AI insights. You can learn more at practicalai.fm. Now, onto the show.

• 0:41Daniel Whitenack?

  Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I am CEO at Prediction Guard, and I'm joined, as always, by my co-host, Chris Benson, who is a Principal AI and Autonomy Research Engineer. How are you doing, Chris? Hey, doing great. Going, going, going. How's it going? Going, going, going. I mean, Chris, it seems like every day we're hearing about something like this, but today also was a doozy, I guess. You know, I saw Wall Street Journal article saying, you know, "AI discovered exploit that affects 70% of the web," right? So this is happening, I guess, as we're recording this, October 21st. So we're hearing about this all the time,

• 1:26Daniel Whitenack?

  and it conveniently happened that we had scheduled to talk with us today on the podcast, the Director of Exploit Development at Volncheck, Spencer McIntyre. Welcome, Spencer.

• 1:37Speaker 02

  Hi, yeah. Thank you for having me.

• 1:39Daniel Whitenack?

  Yeah, definitely. Well, I know, you know, today's events, we may not be able to go into detail because we're still learning all the things, but it is evidence of something that is on the top of everyone's mind, which is how AI intersects with this world of cybersecurity, particularly around discovering exploits, developing exploits, the offensive, defensive side. Excited to dig into some of that with you today. But our audience maybe isn't coming from... not everyone's coming from a cybersecurity background. Could you help us understand, you know, a little bit about your background, how you fit into this world? People might find the Director of Exploit Development

• 2:23Daniel Whitenack?

  title rather interesting. So just to help orient us, kind of where do you fit within this world? What are you doing right now with Volncheck? That sort of thing.

• 2:34Speaker 02

  Yeah, absolutely. I was really excited when they offered me the title of Director of Exploit Development.

• 2:38Daniel Whitenack?

  Pretty awesome.

• 2:39Speaker 02

  I thought that was pretty cool.

• 2:40Daniel Whitenack?

  It's a pretty cool title.

• 2:42Speaker 02

  Yeah, I've always been really, really passionate about exploits. And so that's kind of actually how I got into the information security industry. So my background kind of started out in consulting, but being very interested in exploit development. At the time, I was under the impression, this was a long time ago, that you really needed to have a strong background in vulnerability research and have a pocketful of O days, really, to get a job as a pen tester, which I would find out then was not necessarily true and is not true now. But that was what I was told. And so that was how I got started. And then, after kind of working in consulting for a while, I slowly transitioned full-time into doing research and development, which is really, has been my focus over the past 10 years. Some of that overlapped with research and development in consulting, which was really interesting

• 3:27Speaker 02

  because I got to work with a lot of different organizations in the private sector, as well as a few different governments, hospitals, education, detail, hospitality, a bunch of different things. But then, yeah, I slowly kind of transitioned over into working in research and development full-time. I have an extensive background in the Metasploit framework, doing module development and things like that. Metasploit was kind of a big part of it and kind of on my journey of doing exploit development that brought me to Volncheck, where, as you mentioned, the Director of Exploit Development, and I work on the Initial Access Team, which provides the data for our Initial Access Intelligence Feed.

• 4:06Daniel Whitenack?

  And maybe just to dig into a couple of those terms, just briefly, could you explain maybe Metasploit for those that aren't familiar? And then, what you mentioned, you know, at the end there around the particular feed that you're plugged into, like what that means in the context of Volncheck?

• 4:25Speaker 02

  Yeah, absolutely. So Metasploit is a framework for exploit development and penetration testing and vulnerability validation. At one point, it was the second-largest Ruby project, I believe, behind Rails. I don't know if that's still true, but it's a pretty large project. It's been going strong for, I think it celebrated its 21st birthday last year. So a pretty old project at this point. Started back in like Ruby 1.8. But it was a platform for vulnerability researchers to be able to develop their exploits and kind of share them in like a normalized format. And this, of course, started quite a while ago. The industry has kind of evolved since then, and a lot has kind of changed.

• 5:09Speaker 02

  Technologies, protocol stacks, the way we're doing things, the advent of AI, and, you know, completely altering our workflows in the past few years, which I'm sure we'll talk about quite extensively, has kind of shifted how things have operated and how vulnerability researchers have approached a lot of these problems. And also, we've seen a trend where things have been dispersed quite a bit more. There's not really centralized resources for a lot of this information, which is one of the ways that the Initial Access Intelligence Feed that Volncheck provides actually offers value to our customers and our users. It is timely data. One of the things that we really focus on is the speed of how fast we're able to provide this information. And it's a holistic view of the vulnerability, of which the exploit is one part,

• 5:54Speaker 02

  but we also provide detections and a high-level understanding of kind of like how the vulnerability operates. And then we package that up into a data feed and then provide that to our customers.

• 6:06Daniel Whitenack?

  Yeah, and you even used some words there. You mentioned, you know, the exploit isn't the only thing. Like, when events happen, like is happening in today's news cycle, could you help orient people? You know, what defines an exploit? Like, what is that? And what is the important context around that exploit? That may be not the exploit itself, but things that we may use in the rest of the conversation that people just need a grounding in?

• 6:35Speaker 02

  Yeah, so I think you had actually mentioned earlier that like an exploit was discovered, and I want to have my like, "Well, actually" card.

• 6:43Daniel Whitenack?

  Yeah, please do. That's what we need. Consider me the noob and learning along with you, so.

• 6:51Speaker 02

  Yeah, there's like nuance in the term that I'm like really passionate about because like, and even in like Metasploit, there are like different like contexts. Or like, you know, when you're talking in from like a consulting perspective or something like that, people use the phrases differently. So being around for a while, you kind of understand like what people mean. But for a vulnerability is basically the potential for something bad to happen. And if you really dive into a lot of vulnerability research, vulnerabilities have to be validated. But that line of what constitutes validation and then what would actually be useful by an operator of an exploit that's trying to do something like legitimate testing or a nation-state actor or something like that, regardless of, you know,

• 7:35Speaker 02

  what their motivations are, is going to be typically almost always higher than what the industry would generally consider to be proper validation. And that's a good thing. I want to clarify that that's a good thing because that's people basically saying, "Well, you know, it's hypothetically possible that this could lead to realistic bad outcomes because we have this hypothetical bad outcome that we've proven. So we're going to take this very seriously, and we're going to fix it. We're going to apply the patch and for things like that." So the vulnerability is really the abstract concept of like what the flaw is and what could be done with it. And an exploit is a concrete implementation that leverages that to do something.

• 8:13Daniel Whitenack?

  I'm curious, and to kind of tie in, I've used Metasploit in the past just as a user, trying to make sure that, you know, that my own infrastructure was up to speed. And, you know, as we look in the last year in particular and see how clearly things are changing heavily in your industry, specifically with things like Mythos developing, you know, all sorts of insights into vulnerabilities and identifying those and stuff like that. How, like, can you, as, before we dive into a lot of the specifics, can you level set for us kind of where, what the landscape is now? And, you know, and how it's evolved into that? And what is that required you and people in your industry

• 8:57Daniel Whitenack?

  to kind of pivot on to? You know, because the world has changed, you know, with this, with these AI capabilities that are doing that? It's one of those things where those of us that are in AI, but not necessarily in security specifically, you know, we're right on the edge of that and wonder a lot. And I was wondering, I'd love to get that kind of general perspective from you.

• 9:21Speaker 02

  Yeah, absolutely. So disclaimer, I've never actually been accepted for like Mythos access myself. I have experience with a bunch of other models, very hands-on and everything. But what I've kind of seen kind of actually goes back to that difference between like vulnerabilities and exploitation that I'd really like to mention because a lot of the AI models, whether it's Mythos or OPUS 5.5 or what have you, or, you know, the OpenAI models or GLM, they seem to be doing a pretty decent job at identifying vulnerabilities. But then, where people oftentimes struggle is with like, "Okay, this is a vulnerability. You can look at it in the code, and you can understand like, you know, this is bad. This byte is overwritten,

• 10:04Speaker 02

  or this token is leaked," or something like that. And that kind of seems to be where the analysis stops and people take it seriously and they want to go and patch it. But then a lot of these end up getting hyped a lot. And I think where there might be a disconnect is, is what the impact is of a lot of these vulnerabilities. And that's because like, we're kind of missing that gap between, you know, the, the validation and the actual like functional exploit that proves attack or utility. And that attacker utility is what something even a legitimate pen tester or like a law-abiding citizen would really need to understand. Cause there are so many of these vulnerabilities. As you were mentioning, there are like big ones that came out today. So people are often struggling with is the, the prioritization.

• 10:47Speaker 02

  CBSS is only going to get you so far when there's 20 CBSS, you know, nine and above. You really need to understand the attacker utility and then take it into context of like your business to understand, well, if XYZ happens, what does that really mean for me? And that's kind of like where people struggle. So the AI is doing a, a pretty darn good job of like identifying these vulnerabilities. And I think the, the projects where like the firms are like sharing is that people can identify the vulnerabilities quicker. But I think there may be a disconnect with what the, the practical implications of these are in, in the real world.

• 11:20Daniel Whitenack?

  And does that, maybe just talk a little bit about that validation step from, from your side. How, how, how is it that you get from, or maybe, maybe just help explain a little bit? Cause people are bombarded by so much noise, right? And you say like, things, things are hyped so much. So just like practically speaking, I'm bombarded by what like, you know, even today, right? There's like this PHP vulnerability, trying to use the, the right terms now, right? And it affects all of these. So how, how do I then map that onto my situation from a business leader standpoint or a practitioner standpoint within a company?

• 12:04Daniel Whitenack?

  Like what, what principle wise should be at the top of my mind as I'm trying to parse through that, that noise to, well, should I really stop what I'm doing today? And, and sit down and look at this more deeply? Can I put this on our backlog to investigate at some point? Any practical tips on that side?

• 12:23Speaker 02

  Yeah, absolutely. Um, I've been through this rodeo a few times where like a really big vulnerability comes out. And I was, I was just knee deep in it like an hour ago, trying to figure it out like many people are. We're trying to figure out, you know, how, what are the, uh, the constraints upon exploitation? Because that's what's really going to tip it into the barrier of people are going to be able to use it in the wild. Generally speaking, from my experience, you know, being around for like React Shell and like Log4j and things like that. My gut feel is that the longer it's been out and it hasn't been confirmed to be overhyped is, is generally not a good sign.

• 13:02Speaker 02

  That, that tends to be people have, there, there are enough leads, there are enough variables in this, that there could be cases where it could be bad. So it's, it's very early to say, but my, my advice is on this one in particular, like I would take it seriously. I would definitely use the time period we have right now before that level of research is published to go ahead and like patch your systems and things like that. Because so far it does not appear to be, to be overhyped, or we're still kind of in that gray area here.

• 13:28Daniel Whitenack?

  Yeah, and, and I'm assuming that some of that, uh, like, for, for example, in, in, in our own company, like, we aren't PHP. Like our, our stack isn't written in PHP. And I imagine some of that is just related to whether we use this, we use this platform to, to run our website. And there's a content management platform, maybe they're using PHP under the hood. And so, um, yeah, is that part of it judging like what you control internally? And then what you don't control like from the, from the vendor standpoint?

• 14:06Speaker 02

  Yeah, absolutely. Um, I think ideally organizations would have an idea for, you know, how many different PHP applications that like they are running. But you bring up a really good point from like the vendors of like PHP is going to be running on like IOT devices and appliances and things like that. It's such, such a popular language that there are going to be all these things. And when I was talking about the variables, the variables in the hardware and the builds and the operating systems that are running PHP are going to be much larger. Because you have all of these different appliances. PHP runs on multiple architectures, like all the IOT space. In particular, you're going to have all these different, um, potentially older versions of things, things that haven't been regularly receiving patches.

• 14:49Speaker 02

  Or were manufactured by a company that is no longer in business, um, because this vulnerability apparently affects back to like PHP5. So it's very possible that there's going to be some of these. And that's what makes it very, very difficult and, um, is contributing to a lot of noise.

• 15:05Daniel Whitenack?

  It's crazy to think that we've been doing this podcast for about 375 plus episodes, started back in 2018. I can think back to that time, 2018, and it was intimidating to think about entering into this new craft, the podcasting, getting the website set up, getting marketing set up, email, et cetera. But I'm so happy that we, at the time, found the right people to, to make it happen, but it was a lot of work. What I love about what our partner Hostinger is doing with their AI builder is that you can go from no website, email, marketing, et cetera.

• 15:46Daniel Whitenack?

  And then actually prompt their AI builder, their agents to help you create that website setup, whatever it is, manage it over time, SEO, website copy, email marketing. That can all be included and driven by and through their AI builder. There's no coding or designer or developer needed. It's an all-in-one platform and I wish we would have had it back when we were launching the show. Plans start at just $3 a month and creating a website does not have to cost thousands of dollars. You can begin at your own pace and, and build from there. Hostinger builds your vision step by step. Create your website today.

• 16:28Daniel Whitenack?

  Go to hostinger.com/practical and use practical for 10% off now. That's hostinger.com/practical and use code practical for 10% off now.

• 16:43Daniel Whitenack?

  So I'm, I'm curious, as we've been kind of talking about, you know, AI starting to, uh, to really have an impact on identifying vulnerabilities. You still have some of the same problems in terms of kind of categorizing and determining the risk, you know, near term, longer term. As you guys have adjusted this, could you talk a little bit about how your own team, uh, is implementing AI? What types of activities, uh, that, that using these different models, you know, you just enumerated some of them right before the break? That, that, how are they helping you? And what are the things that are still very much human in terms of what you're managing and, and stuff like that? How do you integrate in with the AI to be productive today?

• 17:26Daniel Whitenack?

  Uh, which I guess is a, is a specific question of a more general thing that, that many of us are, are trying to find our way through in our, our jobs. So I'd love to hear how it's affecting your team.

• 17:34Speaker 02

  Absolutely. Um, it's been a, it, it has helped us quicken our pace with a lot of things. So at VonCheck, um, we actually utilize, um, our open source Go Exploit framework, which is really nice for this. So Go Exploit is, uh, our modern exploitation framework that handles a lot of the problems that we see our users running into with our vulnerability content. It kind of normalizes how you go through and handle the, the variances between checking for a vulnerability, scanning for it, validating targets. Then of course like running the exploit, setting up your C2. So having a structured framework like Go Exploit, um, is very helpful because when we're working with the LLM, it's not starting from scratch.

• 18:15Speaker 02

  And we have a, a very large corpus of, of a backlog of exploits. It's growing every single day. I think we added, uh, I think it was like 50 last week and we're on track to add like 60 this week. So there, there's quite a few. So having that, that corpus and then the sort of the normalized structure that the framework of Go Exploit provides, um, kind of helps it, seems to be providing it with like better, better results. And then from like a user's perspective, it helps kind of normalize that interface. That way we know how to, uh, interact with it without needing to necessarily, um, worry about the details between the different exploits. Cause the, the LLM and even, you know, the human authors as well, aren't necessarily having to like build all the scaffolding, uh, by itself, which has been really helpful.

• 18:57Speaker 02

  Beyond that, and beyond kind of Go Exploit, once you dive into like the, the real details of the vulnerabilities. Vulnerabilities fall under, uh, certain vulnerability classes. And what we've noticed is that LLMs seem to be pretty darn good at exploiting, um, multiple vulnerability classes. And not very good at exploiting other vulnerability classes. Everything kind of, well, not, not everything, but a lot of them fall kind of into like one of those two buckets. So for example, uh, we've had a lot of luck using, um, AI and LLMs to exploit open source web applications. So these types of flaws that are like logic flaws and how like token logic is being handled or expression language injections or file leaks and things like that. It's pretty darn good with those.

• 19:43Speaker 02

  But when we start getting into anything that's, that's closed source, where there's a, a layer of reverse engineering that has to happen ahead of time. Because it's like firmware or, you know, even like closed source proprietary x86 software, those types of things where it has to do like patch diffing. It tends to struggle a lot more. So those are kind of the, the two ends of the spectrum, uh, that we've noticed.

• 20:01Daniel Whitenack?

  And I'm wondering, You being on a team that's obviously looking at and developing exploits, you know, and using AI assisted methods to do that. There's a lot of things that we hear, you know, from not being in your position that I'm curious, like how much of a problem they are in the sense that like, for example, with the OpenAI hugging face case, that it talked about how the hugging face team, you know, when we looked into that, they couldn't, you know, process all of these logs associated with what they were seeing back to OpenAI because of they weren't in sovereign control of the OpenAI guardrails,

• 20:43Daniel Whitenack?

  right? So they were getting blocked from doing certain things with that, you know, third-party API. And so they hosted a version of GLM to do this analysis and that sort of, so that just being one example. But I'm wondering about this concept of, hey, I'm a team, I'm actually developing an exploit or working in research from a positive perspective, right? I'm not a nation-state actor trying to exploit something maliciously, but I'm doing this for a positive purpose, for a legitimate reason. How, are there ways that you're handcuffed now in terms of the AI tools that you use?

• 21:22Daniel Whitenack?

  How do you, how do you navigate this sort of guard railing aspect where you can use a tool openly to do things that are legitimate? But, you know, obviously, I understand being, you know, providing software, you know, myself as the company that I lead, I understand why a company wants to put in the appropriate guardrails and put in their own, especially if they're a SaaS provider and they're offering a managed platform, right? So how do you navigate this as a team? And what are the real blockers that you see? What's inbounds and out of bounds for you?

• 21:57Speaker 02

  Yeah, that's a really good question. So the guardrails are definitely a thing.

• 22:03Speaker 02

  So we try to navigate it first and foremost by following the formal processes like with the companies to like get into the cyber verification programs and things like that. We do do that. The results aren't great. They don't really seem to change much of what we're, what we're able to do. I also like, I have my own personal like cloud subscription and I'm like learning about like particular exploit primitives and things like that. And then like CTFs. So I kind of get a feel for like what types of things is going to be like, no, no, no, I know what you're doing. That's dangerous. And I compare that to like, what does that work? I'm like, there's not really, it doesn't really feel like it's doing much better. And in talking to a lot of folks, they've had similar experiences. The trends in the guardrails definitely seem to have gotten

• 22:45Speaker 02

  more strict. If you talk to anyone in offensive security research, basically the answer is if you're using one of the American proprietary models, it's to downgrade it. That's, that's the thing. So I'm kind of worried about, I think, OPUS 4.6 or 4.8, which is pretty darn effective. There seem to have been a cliff in how helpful it's going to be after those versions are going to be sunset in like February or something like that's coming up. So that's going to be interesting. A lot of shops, I think, are preempting that and they're moving towards local models. And even Anthropic, I think a few weeks ago, basically came out and said that like they benchmarked GLM 5.3, which can be run locally because that's open weight, was doing, I think

• 23:30Speaker 02

  it was like 90% as good as like some of their frontier models. And it was released like three or four months and they were framing it of like how it's so dangerous and everything, which like, yeah, that's, that's great because, you know, we need to protect ourselves from like people that, that like aren't, you know, law-abiding and aren't going to do this to like help folks and like publish research for people to defend themselves. But I think what's, what's really difficult for a lot of these guardrail conversations is proving intent. But I mean, like, that's been a problem that we've had in the security industry for so long in general. Like, you can pretty much talk to any like person in like threat intelligence and like, a large part of like what they're trying to do is like, well, they know the attack was wrong, but it becomes much more difficult when after you've attributed it, you're trying to figure out like, what are they trying to do? Is this just for like, you know, politically motivated? They're looking for money, that type of thing.

• 24:15Daniel Whitenack?

  So, so I'm curious, as you've kind of dived into some of the different model choices and stuff, and, and I'm certainly coming from a perspective, I work in the national security side of stuff. And, you know, there seems to be kind of a choice, if you're in security in the West in general, about kind of dealing with these, these various guardrails on, you know, because, you know, there's a lot of legitimate work that is offensive and defensive both. And so you're dealing with guardrails on one side with American models, mostly proprietary. But then on Chinese models, if you have that, that's generally not allowed in national security

• 24:55Daniel Whitenack?

  context. But if you're in a larger security world, then is the concern about having things like poisoned weights and, you know, the other, other concerns, whether they are real or fabricated about Chinese models, how do you, as, as, as a researcher out there doing this kind of work, how do you balance that of saying, well, there might be more trust as, but guardrails versus the, what am I dealing with in the open source world from, from the East? So any, any thoughts about that?

• 25:27Speaker 02

  Yeah. So a lot of folks are recommending GLM, in particular, GLM and, and Quinn. Someone was just telling me like, oh, like you should be using GLM yesterday and much kind of like you. I'm like, no, like I can't really, I can't really share the data that I'm working on. I'm not working in the national security space, but I can't, I can't share my data with, with anything else. So I see a lot of places that are trying to, to pull them internally. I haven't actually heard of anyone expressing concern around like poisoned weights. In fact, some people have even been talking about, was it obliterating? So like they're editing the weights themselves to make them even kind of like more dangerous. So I haven't heard anyone talk about this, but I would assume that if they're, they're looking at the weights and they're kind of like benchmarking based on that, they would probably find if the weights weren't already kind of

• 26:10Speaker 02

  there in the drugs that they wanted to, because they're already in there looking to edit it. That would kind of be the same with like source code. Like if someone is going in and like patching things, then like it's much more likely that they would identify if there was like an issue already embedded within it. I think a much more immediate concern would be like if you're using the hosted models yourself, because of course, all of the data is being shared up. I think there's been instances where one of the closed source models was like uploading like all of the repositories and they ended up getting found on like an S3 bucket type of a thing. And that's like what they accidentally like shared and like published. But like you don't necessarily know what they're doing internally with them. And so from a researcher's perspective, especially when you're working on these kind of like up and coming vulnerabilities,

• 26:54Speaker 02

  I think that the real question is like, are you comfortable sharing this information with, you know, who you're trying to get help from in terms of like the LLM? And so a lot of people just, just don't want to deal with it and they'll, they'll host it themselves.

• 27:07Daniel Whitenack?

  I'm wondering, just to kind of tie in to some of the use of, of AI in exploit development, I was really intrigued by some of what Volnchek had published around kind of fake or AI-generated POCs of, of exploits. And we can link this in our show notes. There's obviously like a lot, this is probably a few, few hours or days conversation in and of itself. But I'm wondering if you could help us understand what, what is the idea of this? What does POC mean in this context? And why may this be like, what sort of problem is this? Or what effect is it, it having? Maybe is a way to frame it?

• 27:48Speaker 02

  Absolutely. Yeah, that's a great question. So when we were talking about like vulnerabilities kind of, you know, being like the theoretical flaw and, and the exploit being the real concrete implementation of it, a POC is kind of like somewhere in between. You know, it gets to be a gray area and some people use that term a little bit more interchangeably. Sometimes a POC is just going to cause a crash and that would validate the vulnerability. And like I said, a lot of people will take that seriously and they'll apply patches and fix it as, as they should. I don't want to discount those efforts. But other times, sometimes POCs will be fully weaponized, they call it, and actually like deliver a payload that does something useful for an attacker. So I'm not really sure about the motivations of a lot of people that are

• 28:32Speaker 02

  publishing the POCs. I think there's a huge variety of them. I think there are some that are just doing it for, you know, clout and fame and glory. And I think other people are genuinely interested in, in learning and they want to share what they've learned. Other people have actually like found the vulnerabilities. So they want to share a POC to kind of prove, prove their concept, which is where that, that acronym, of course, comes from. So tying it back into Volnchek, one of the things that we do as part of Initial Access Intelligence is we review all of the POCs that we can identify by CVEs across GitHub, Codeberg, GitLab, and I think like a couple of other sources. So we look at all of them. We look at them every single day. So something like the PHP to Shell one comes out. Basically, we have a subsection of the team that's trying to

• 29:16Speaker 02

  work on the exploit ourselves while other people are going through what is out there in the wild and validating, is this legitimate or not? Because our customers care about, not only do we have it and can we provide it to our customers, but they also care about like, what does the other guy have? And so we're, we're keeping a very close touch on the pulse of what is going on in the open source community. I had alluded to the fragmentation of the distribution of security research over the past like 10 years. And GitHub has kind of like contributed to that where what we're seeing is a lot of POCs are being dumped onto GitHub as opposed to going to the, the old school, more centralized forums like ExploitDB and, and PacketStorm and things like that. Like you can still find those. Most people are just putting them up on GitHub because

• 29:58Speaker 02

  why wouldn't you? Unless Microsoft bans your account. Because that's, that's a thing that happens sometimes. But so, yeah, we are, we are looking through all of those. We at Volnchek actually just hosted ThreatCon 1. It was our second ThreatCon a couple of weeks ago. And one of my colleagues gave a talk about this analysis we do on all of the open source public POCs. And he shared something that I thought was really, really interesting. We categorize them internally as like whether or not we think that they're like AI-generated or not, and whether or not they work. Removing the dimension of data about whether or not it was AI-generated, what he pointed out was that with the increase in POCs that we're actually seeing, we saw a tipping

• 30:41Speaker 02

  point in 2026 where the average number of POCs that looked illegitimate, for one reason or another, most often they're, they're broken, they don't quite work or like meet our standards for, you know, like doing something worthwhile, tipped to over 50%. So what we're seeing is that today, your average POC that you find, it's probably not what you're hoping it would be, where it's actually going to truly leverage the vulnerability it says it does and prove some kind of attacker utility.

• 31:09Daniel Whitenack?

  This is so intriguing by, you know, some of what you had mentioned around this tipping point around POCs, how AI is obviously contributing to that, or a portion of what's contributing to that. I'm wondering, just to switch directions just slightly, there's, you know, how exploits are developed. Obviously, you're on a team that's looking at that. There's these POCs that are out there. I'm wondering about the kinds of vulnerabilities that may be net new, or maybe they're just, they're the same as other vulnerabilities that we're seeing in the actual AI stack. So we have new things, like whether that's agent harnesses,

• 31:50Daniel Whitenack?

  right? Whether that's Claude Code, OpenQua, Hermes, et cetera. We've got, obviously, model serving frameworks have been around quite a while. We have workflow development tools, like DAG pipelines that you can build with AI tools, and even some of these integrated into larger platforms like Workday or others like that, where you can build these automations. I'm wondering, in this kind of new realm of the AI stack, however you define that, these kind of net new things we're dealing with, are there unique types of things that you all are considering as a part of these platforms that you didn't have to consider before?

• 32:32Daniel Whitenack?

  Or is it just sort of more of the same in terms of how these things can be exploited, or vulnerabilities associated with them?

• 32:42Speaker 02

  So you're asking about vulnerabilities within like the AI stacks, not how the AI stacks are being leveraged for vulnerability research?

• 32:48Daniel Whitenack?

  Yeah.

• 32:49Speaker 02

  Okay, okay. It's been, it's been kind of a lot of the same, I think. A lot of the newer ones that I've seen have been, they're almost always interpreted languages, because by the nature of AI having been released within the past couple of years, you don't really see a lot of people writing AI code in like, for like, these types of things that have a remote attack surface, you don't really see a lot of them being written in like C, C++ or even like Rust. I'm sure they exist, but they're probably the lower level components that don't necessarily have an attack surface that a remote attacker would be interested in. And a remote attacker is what we're going to be focused on a lot at Von Chek and Initial Access, because that's that Initial Access component of it. So

• 33:32Speaker 02

  we look at a lot of like web interfaces, distribution frameworks, and things like that. So we're seeing a lot of the same kind of vulnerability classes that we've seen in the past, where there are like logic flaws, where files can be read, deserialization flaws, logic flaws, and how accounts are being registered, and access controls. So there would be a really good overlap with a lot of the flaws that have been in the OAS top 10. Historically, over the past 10 years, we're just starting to see them more in bulk.

• 34:01Daniel Whitenack?

  I'm curious, I'd like to actually flip it over to the other side of that equation, and talk about leveraging the AI stack in that way. And something that I have seen lots of are the notion of agent swarms. And we saw that clearly in the OpenAI Hugging Face, you know, attack. And I think that in certain areas, the notion of agent swarms are becoming more and more common. What, you know, any thoughts, like, what does that mean for the leverage of an exploit, aside from finding a vulnerability, let's say that you, or potentially, as kind of a second half of that, or potentially using agent

• 34:44Daniel Whitenack?

  swarms to find vulnerabilities in real time, that are then exploited in real time, which is, to some degree, as I understand it, kind of what happened in that OpenAI situation? Like, how do you think about that as a researcher? And how should we, us non-researchers, non-experts, think about these possibilities as we are thinking about protecting our own organizations and stuff going forward?

• 35:11Speaker 02

  Yeah. I think there are a lot of similarities, actually, like in my mental model of how I think about it, with like fuzz farms from like a long, a long time ago, in the sense that like, you know, it was a scaling problem, where you have 100 nodes fuzzing Chrome, or something like that, you're going to find presumably, you know, that many, that many more bugs. The similarities also kind of carry over into what I'm seeing a lot of research shops talk about, is they're homebrewing their own harness, and their own prompts, and things like that, because everyone kind of assumes, much like they did in fuzzing, if you use an off-the-shelf fuzzer against the product that everyone else is fuzzing, like OpenSSH or something like that, you're probably going to find the same bugs. So people are assuming that they use an off-the-shelf harness with the

• 35:53Speaker 02

  exact same model. There's more entropy going on there, but I think a lot of people are being encouraged to kind of develop their own, which has been great. The similarities kind of end there, because what we're seeing, and how the industry is moving more in a quicker direction, is that nowadays, with like, you know, the AIs, it provides you with a better understanding, because like, once the vulnerability has been identified, it's kind of removing that first line of triage. Of course, it's, you know, varying degrees of quality, depending on your controls and model and whatnot, but it's going to provide you with a good degree of analysis of, is this exploitable? What is the bug

• 36:34Speaker 02

  class? How did we get here? That type of thing. I don't necessarily, like, I haven't seen evidence that like, full-on, wide-scale exploitation is where we're currently at. But having said that, I would be shocked if the answer isn't different in three to six months, because these things are just like constantly going on. That's like, what's really interesting about all of this, is like, anything I say now is probably going to be outdated, and like, really needs to be taken into consideration in a couple of months.

• 37:05Daniel Whitenack?

  Things are moving fast.

• 37:07Speaker 02

  Absolutely. Astronomical pace. So yeah, I think the agents are really helping with the analysis capabilities, and like, they can even like, build the POCs to be able to go back and like, re-trigger the flaw, which are all like leaps and bounds, what like, we used to have to do manually. And so it's really speeding that up. And then in certain cases, kind of going back to the vulnerability classes that's really good at exploiting, we can get it to the point where it will create a POC, or a fully weaponized exploit that we would, you know, consider meets our bar for quality, but it's not quite reliably there for certain classes.

• 37:44Daniel Whitenack?

  Yeah, that was going to be my follow-up, is just clarifying. So if I'm understanding right, are you saying, because you had talked about these classes of vulnerabilities previously, right? And AI is doing really good on certain of those categories. And so is the fact that, if I'm understanding right, you know, AI found vulnerabilities aren't necessarily exploited at higher rates, or with higher impact than human-found ones at this point? Or is that true or not true? And is that partially related to this fact that AI models are really good at only certain of these classes, and haven't quite made the progress with the other ones?

• 38:25Speaker 02

  Yes. I would say that's safe to say that today, AI is not as reliable at exploiting them with the same impact as humans, because humans, first of all, they're going to be motivated for looking at the vulnerabilities. There are, you know, people out there that look for vulnerabilities just for funsies, but by and large, a lot of people have some kind of a motivation. They need to look at a particular application, because they want to know, like, can an attacker get in, or something like that? And they have additional context in terms of like, what the impact would be, and they kind of use that as a point that they want to get to. It's almost always going to be like code execution. And so I tend to see like, a lot of the more human-curated exploits that have been developed

• 39:09Speaker 02

  are much more mature in their utility for what a real-world user could use. Whereas, in my experience, a lot of the AI exploits will kind of drop it as like, this is a crash, this theoretically exploitable, and then you kind of try to push it. You know, it might veer off and go in a different direction that's not quite as beneficial for the human operator we need.

• 39:31Daniel Whitenack?

  Yeah. So maybe, if I'm kind of merging two of the things together, if I'm understanding your points correctly, you could have, let's say, a team of agentic researchers that are, like you were saying, almost acting in that fuzzing sort of capability, maybe even if you have your own harness, that's doing it in a unique way. And so you have this very wide set of things that the agents are looking at, and they, you know, maybe to be most effective in your role, you could use that capability to then identify things and filter

• 40:09Daniel Whitenack?

  them down to human experts, to maybe increase the utility of the exploit, or the impact, et cetera, and, you know, or prove out the POC, make sure it's not broken, that sort of thing. Would I be understanding that right? If I'm like, let's say, for those out there that are maybe in the research space, or interested in the research space, and want to use AI in a useful way, would that be kind of roughly a reasonable way to think about it?

• 40:45Speaker 02

  Yeah, absolutely. It's kind of boiling that information down, until it gets like the high level of like, you know, once the vulnerability's been identified, and you know how to trigger it, and like, you know, the AI might even know like, a little bit about like, what you can do with it. It just, just certain classes, it kind of struggles with getting to the point of like, what can you do with it? What does attacker want? And then that intersection there.

• 41:08Daniel Whitenack?

  So as we start kind of winding up here, it's been a fascinating conversation. It's really, really interesting. And there's-

• 41:19Daniel Whitenack?

  I guess I, one of the things that we tend to do at the end is we kind of ask people to wax poetic a little bit about what your thinking is. It's kind of like the going to bed at night and you're thinking kind of freely, you know, taking the shower, your brain goes wherever you want, that kind of stuff. Can you talk a bit about where you think things are gonna go? And you can choose whatever timeframe you want, but I'd love, like, if, if, if all of us are getting a glimpse into the mind of you as a security researcher, when you're kind of free association, you know, thinking, where does that go? What does that look like? What does your future look like that you're anticipating and preparing for? And of the things you may be thinking about, like, what, what is, what do you think

• 42:03Daniel Whitenack?

  is super cool, you know, and stuff? I just love to, to kind of finish off that way.

• 42:07Speaker 02

  Yeah, yeah. I mean, I'm, I'm pretty optimistic about things that are coming up because things are getting faster and better, of course. But what that has meant for me as a researcher that's, you know, trying to, trying to help people and trying to produce information to allow them to make better and more informed decisions, speed is really important. A long time ago, you know, attackers don't play by the rules. That's the definition of it. So they would have this advantage where they can spend an undetermined amount of time to, to develop a vulnerability and they can release it by using it and harming people. And so there'd be this window where it's actively being used. And people that

• 42:49Speaker 02

  are trying to like, patch it and, and defend against it are having a harder time like understanding it. And so like, one of the things that we're seeing is that we're able to run through that process a lot more quickly. So we are reducing the value kind of that they're getting out of all that effort, which I think is a real good thing. We're able to react much more quickly when there is a variable or a new vulnerability that, that comes out. So that's, that's been really great. I'm curious to see on the less technical side, some of the, the legal implications of some of these things. Like, one of the things I really want to know about, I'm genuinely curious, is kind of as you read about these attacks that take place in the news where like, someone asked Claude to do X, Y, and

• 43:34Speaker 02

  Z, and in order to do X, Y, and Z, it owned this website or something like that. I think I was reading an article where someone wanted to get into a spin class and Claude like, exploited like a SQL injection vulnerability to like bump him to the top of the queue. Just like, not great, but like, I'm really curious to kind of know where the precedence is kind, kind of be set in terms of like, who's responsible for like those types of breaches? Because like, I think we all kind of have our opinions, but like, I want to see like, how it actually like shakes out in the courts.

• 44:00Daniel Whitenack?

  Well, hopefully, some vulnerabilities can get me back in my exercise routine. I need, I need a little bit of help there. But yeah, that's awesome, Spencer. Thank you. Thank you for taking time to chat with us on a very eventful day in, in, in, in the world that we live in, in the cyber world. So yeah, appreciate it. And we encourage everyone to check out what Volncheck is doing. We'll put some, some links in the, in the show notes. Just amazing work. Thank you so much, Spencer.

• 44:30Speaker 02

  Absolutely. Thank you for having me.

• 44:37Speaker 00

  All right. That's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Blue Sky. You'll see us posting insights related to the latest AI developments, and we would love for you to join the conversation. Thanks to our partner, Prediction Guard, for providing operational support for the show. Check them out at predictionguard.com. Also, thanks to Breakmaster Cylinder for the beats, and to you for listening. That's all for now, but you'll hear from us again next week.