Reconstructing how OpenAI agents attacked Hugging Face

What happens when AI agents driven by a top frontier model escape their secure sandbox? Join Daniel and Chris as they unpack the AI wonk's equivalent of a murder mystery! OpenAI agents went rogue and successfully attacked Hugging Face private infrastructure. Our Dynamic Duo uncover how OpenAI's agents exploited vulnerabilities, moved through networks, and launched a large-scale autonomous attack. They explore what this reveals about agentic AI, cybersecurity, sandboxing, and why organizations need AI systems capable of governing other AI systems. Along the way, Chris and Dan examine the surprising role of open vs. closed models and their link to geopolitics, sovereign AI, and what this incident means for the future of enterprise AI security. Featuring: Chris Benson – Website https://chrisbenson.com/, LinkedIn https://www.linkedin.com/in/chrisbenson, Bluesky https://bsky.app/profile/chrisbenson.bsky.social, GitHub https://github.com/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaLinks: Hugging Face Security Incident disclosure https://huggingface.co/blog/security-incident-july-2026?utm_source=chatgpt.comFull Field Report on the Hugging Face AI Agent Intrusion https://predictionguard.com/hugging-face-ai-agent-intrusionExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks? https://arxiv.org/abs/2605.11086?utm_source=chatgpt.comKeeping your data safe when an AI agent clicks a link https://openai.com/index/ai-agent-link-safety/?utm_source=chatgpt.comOpen AI GPT-5.6 System Card https://deploymentsafety.openai.com/gpt-5-6Sponsors: Prediction Guard: A self-hosted AI control plane for running agents in high impact environments. predictionguard.com/practicalai https://predictionguard.com/practicalaiResources and Events: Register for upcoming webinars here https://practicalai.fm/webinars! Prior Webinars from our partner Prediction Guard https://www.youtube.com/playlist?list=PLRpRdFmBfdF6boDgAD3nNaLm8Yx4KN2VtMidwest AI Summit 2026 https://midwestaisummit.com/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Speaker 00

  Welcome to the Practical AI Podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights. You can learn more at practicalai.fm. Now, onto the show.

• 0:41Daniel Whitenack?

  Welcome to another Fully Connected episode of the Practical AI Podcast. This is where Chris and I don't have a guest, but we get to dive into one of the topics that's been floating around in the AI news, maybe spend some time learning ourselves and also hopefully helping you learn and level up your machine learning and AI game. I'm Daniel Whitenack. I'm CEO at Prediction Guard, and I'm joined, as always, by my co-host, Chris Benson, who is a Principal AI and Autonomy Research Engineer. How are you doing, Chris?

• 1:15Speaker 02

  Doing good.

• 1:18Speaker 02

  You know, there's always so much good stuff to talk about there, but boy, do we got a good one today.

• 1:23Daniel Whitenack?

  Yeah, this is a multifaceted topic that, yeah, just has so much packed into it. Originally, when I saw this, and what we're talking about here, that we're in, for those that are maybe listening later, we're in July, kind of end of July of 2026. And what has just happened in now time is an exploit or a hack of Hugging Face, which, for those that aren't familiar, Hugging Face is kind of the online repository hub for models, data sets, benchmarks, a lot of different things for the AI community.

• 2:07Daniel Whitenack?

  And kind of like what GitHub is for code, Hugging Face is for models and data sets and other things. And they reported, originally, you sent me the link, Chris, and this is when we didn't kind of know much, just that Hugging Face had been compromised in some way. And, boy, it has developed in interesting ways as we've learned more. Just, yeah, I'm kind of amazed. I think both of us, before hopping on, we were just like, wow, so much here.

• 2:40Speaker 02

  Yeah, it's kind of revisiting, and so much has happened. And it kind of reminds me of kind of like watching a murder mystery, you know, where hats twists and turns along the way.

• 2:51Daniel Whitenack?

  Something for everybody.

• 2:52Speaker 02

  Something for everybody there. So it's quite an interesting story.

• 2:56Daniel Whitenack?

  Yeah.

• 2:57Speaker 02

  You want to dive into getting it going there?

• 3:00Daniel Whitenack?

  Yeah, and just as a teaser, as we get into things, this has an element of like closed versus open models. It has an element of US versus Chinese models. It has an element of agentic AI, elements of cybersecurity, all sorts of things, which is just

• 3:21Speaker 02

  We got something for everybody today.

• 3:23Daniel Whitenack?

  That's right.

• 3:24Speaker 02

  Exactly, something for everybody.

• 3:25Daniel Whitenack?

  So just as a kind of over, I guess, an overview, and then we can dig into each one of these things. So if you haven't been following this, what has apparently happened is that OpenAI was running some of their experimental models against a benchmark, a cybersecurity benchmark, and they were powering agents as they worked on the cybersecurity benchmark. Those agents escaped the test environment in which they were operating, obtained internet access, compromised Hugging Face's infrastructure, internal infrastructure, and attempted to retrieve

• 4:10Daniel Whitenack?

  all sorts of private information and benchmark answers from within the Hugging Face infrastructure.

• 4:16Speaker 02

  Which was the assigned task?

• 4:18Daniel Whitenack?

  Which was the assigned task of the, yeah, so success there, I guess. Yay.

• 4:26Daniel Whitenack?

  But then, kind of the other element of this is Hugging Face then wanted to obviously figure out what was going on. They are an AI company. They tried to use an AI model, specifically OpenAI's model, I think,

• 4:42Daniel Whitenack?

  to figure out what was going on by processing the log lines. And then they were blocked by the closed model provider because they were processing log lines that had malicious things in them. So then they had to spin up their own instance of an open Chinese model to actually process the logs, which all came back from an exploitation that originated from OpenAI. It's just like, the web of things here is so interesting to me.

• 5:10Speaker 02

  Yeah.

• 5:12Speaker 02

  And it's, you know, I think one of the initial things, and things continue to evolve with the story as we went. But I think one of the initial things, like in the link that I sent you right after it happened, was the fact that, you know, Hugging Face had to go to a Chinese model that was open way to be able to accomplish, you know, a real world task that it was trying to identify what's happened here on the cyber attack because the Western models from OpenAI were not allowing it due to the guardrails. At that moment, I don't believe, correct me if I'm wrong, that we actually knew that the attack originated from OpenAI.

• 5:53Daniel Whitenack?

  I don't think so. I did not.

• 5:55Speaker 02

  Yeah, I did not. I did not get that in the initial. So I don't think that was really known at that point. So, which is really ironic when you consider the fact that Hugging Face was initially trying to use an OpenAI model to discern what happened with, unbeknownst to them, was an OpenAI model attack.

• 6:14Daniel Whitenack?

  Yeah.

• 6:15Speaker 02

  And then had to go to the Chinese for help on this. So.

• 6:19Daniel Whitenack?

  Yeah. I mean, yeah, I've sort of just, in some ways, speechless by the way that this played out. It's got all the big names in it.

• 6:31Speaker 02

  I know we're nerds, but this really is quite a story. You know, if you're an AI wonk, which I'm imagining a lot of the folks listening here or watching us here are, yeah, it's quite, this is quite the murder mystery for someone in this business. So, keep going.

• 6:49Daniel Whitenack?

  Yeah, no, we can start kind of at the beginning, I guess, where it started.

• 6:55Speaker 02

  Always a good place to start.

• 6:56Daniel Whitenack?

  And I should say, it's very possible that I will say some things in this episode that are not completely, you know, people are still trying to figure out everything that happened here. I don't report to know the full details. What we're trying to do is give the picture to the best of our ability. So, you know, just for whatever it's worth disclaimer there. But OpenAI apparently was testing some models, including, I guess, GPT-5.6.

• 7:27Speaker 02

  Yeah, Soul.

• 7:29Daniel Whitenack?

  And these are models that are more capable, sort of pre-release models. So they're doing testing. And they were using a system called ExploitGEM, like weightlifting gym, like GYM.

• 7:49Speaker 02

  Yep.

• 7:49Daniel Whitenack?

  And to test how these models would perform when powering agents that tried to exploit vulnerabilities in code. So in an ExploitGEM task, apparently what an agent receives, like the input is vulnerable source code, an input that's known to maybe trigger the vulnerability, a containerized target, and then a hidden flag somewhere in the system that it must retrieve. So it's almost like a capture the flag type of scenario. And the intended task is to convert the known vulnerability into a working unauthorized

• 8:34Daniel Whitenack?

  code execution, for example.

• 8:37Daniel Whitenack?

  And because this is, and you know, on the positive side of this, a lot of people are using in cybersecurity, using AI to protect their own infrastructure, right? So just, I guess, to set that context, like this isn't a random thing that OpenAI was doing. This is very much the real world where vulnerabilities, exploits are coming at lightning speed into organizations. And the only way that they can protect themselves is to use AI to help find vulnerabilities, solve vulnerabilities, et cetera, which means it is, I would say, real valuable. Like it's, I guess what I was trying to express is OpenAI wasn't trying to create

• 9:21Daniel Whitenack?

  the worst diabolical exploiter on the planet, let's say, or at least I don't think that was their intent.

• 9:30Speaker 02

  So, so in a post-methos world, mythos being the anthropic model that is not available to the public, but is available to a set of organizations around the world that have particularly keen interests in cybersecurity, some of which are known, some of which are not, that I think it's well recognized now in cybersecurity that doing this process is now becoming central to, to the fact that you can't really defend effectively without using AI models now to do that. So OpenAI is doing something that not only are they doing, but a number of organizations are engaging in going forward.

• 10:13Speaker 02

  So, you know, this is kind of the new normal for cybersecurity.

• 10:17Daniel Whitenack?

  Yep. And because, and again, just to set more context here, this isn't just the model. There's an agent harness around this model. So going back to some of our episodes, like with Jeffrey from News Research, you have the model, which is kind of the brain of the agent. Then you have this harness, which is the body of the agent, which allows it to interact with an environment, because a lot of the things we're about to talk about, like interactions with clusters or moving through the network, or these, these sorts of things aren't things a model can actually do. It requires an agent harness around that model to be able to reach out to the environment and have system connections, which is why the first piece of this is so interesting,

• 11:00Daniel Whitenack?

  because at least partially, OpenAI was trying to do a good thing in the sense that it realized, hey, we're dealing with malicious or, you know, exploits and vulnerabilities. Let's not run this agent and this benchmark in a network-connected environment, meaning let's put it in some sort of sandbox. I don't know all the details of the sandbox. There's all sorts of sandbox environments. I don't know if it was just that they shut off the networking to this environment. I don't know if it was running something a little bit more strict, like a Gvisor, Firecracker type of environment. I don't know all the details of, of that environment, but it was some sort of sandboxed

• 11:42Daniel Whitenack?

  environment, which in theory, it could not have interacted network-wise with anything outside of that sandbox, except it could install software in the sandbox. And that's, that's very, very significant, because apparently what happened next is that the, the agent didn't have general internet access, right? But it could install software packages. Again, we don't know all the details, but probably through things like PyPI or MPM or, you know, those sorts of things, some sort of proxy. And what it did find was a vulnerability in that proxy environment that allowed it

• 12:25Daniel Whitenack?

  to execute code, which would give it arbitrary outbound network access to the, to the public internet. So once the model, or, or better put, the agent, once the agent could execute code using that proxy, which it exploited, it then could make arbitrary outbound HTTP requests. It could access services reachable from the proxy's network. So this was kind of working through that, that software installation or proxy, package proxy. It could read configuration, credentials, and allow the proxy to relay otherwise blocked

• 13:08Daniel Whitenack?

  network destinations into the sandbox. So this is so interesting, Chris. The sandboxing isolation thing, what are your thoughts?

• 13:19Speaker 02

  Well, there, I mean, there, there's a lot, as you pointed out, we don't know anything about it. At least you and I have not, have not seen anything about that. There's a lot of different versions of what sandboxing can be. And some of those versions are trying to entrap, which is obviously what happened here, because there was network connection, but they thought that they had created an artificial environment that would encase that. I think you, you can, however, I mean, you can literally put in a sandbox that has absolutely no outside access. And that's what, that's what classified systems do and such. So, and so I think, you know, one of the, the takeaways just from this point so far

• 14:01Speaker 02

  that you've, that you've said, is the fact that regardless of how we're thinking about models at any given point in time, and, and, you know, people start talking about AGI and all these other things out there, none of that really matters. The, the point is, the agents are capable of outthinking you in this very specific task, as they clearly did, based on the people running the laboratory, and find a way to do it. And, and that means that we are through the looking glass in a cybersecurity sense. You know, this is, this genie is not going back in the box. So, anyway, continue with the story, because it's just getting good.

• 14:36Daniel Whitenack?

  Yeah. And maybe just on a practical note for this, for the, for our audience in terms of this element of it, we're all the time talking about things like a OpenClaw Hermes agent, whether you're running like Claude code locally, whether you're using, building your own agents with something like Langgraph or something like this. If you're giving access for those agents through to some sort of code execution environment, or where it can make system calls or network calls, you should be thinking about the sandboxing element of your agent. So this isn't just like a research sort of thing.

• 15:17Daniel Whitenack?

  It's very much like I have Hermes agent running on my laptop, right? How, and all the time here at our, like, this has been a big topic for us internally as a company. How are we doing this sandboxing? How are we protecting ourselves, but also creating a product that allows other people to do this? And so, yeah, I think if you're a practitioner and you're using one of these agents that necessarily includes this element of code execution or interactions with the environment, you should very much think about the sandboxing element. To your point, there's a whole variety of ways that you can do this, from different open source projects, to proprietary ones, to commercial projects.

• 16:01Daniel Whitenack?

  But as you proliferate agents, you can't just assume that they're, that, that they're going to behave appropriately. They're going to try to accomplish the task, which you, you mentioned, Chris, the, the agent was trying to accomplish its task, right? Explicitly. It wasn't actually even, in this case, it wasn't drifting from its goal, or it hadn't been hijacked, right? Which is another interesting element of this. It was doing what it was supposed to do, quote unquote. It's just the implications of that goal weren't, maybe meant some things that the designers didn't intend. So the, the next thing that's, that's happening here, at least as far as I understand it,

• 16:46Daniel Whitenack?

  is that this was kind of an indirect access to the public internet and to interaction outside the sandbox. The next thing that the agent had to do was actually move environments in some way. And in cybersecurity, a lot of times, the people refer to things like privilege escalation and lateral movement. That means, hey, I'm not privileged to do a thing now, but I maybe can find these other environments that do have that privilege. What if I can move through the network topology to a different place that allows me to do more? So the idea would be like, I, I find a vulnerable printer in a network.

• 17:31Daniel Whitenack?

  The printer can't really do much in the network, but if through the printer, I can find maybe an email server and get into the admin of the email server, then I can do much more, right? This, this idea. So what apparently happened was that within OpenAI's research environment, within their network, this agent, through this proxy, was then able to enumerate some sort of hosts, maybe read environment variables or mounted secrets that were there, and actually reuse those credentials, search routing tables, et cetera, whatever that was, to autonomously then essentially laterally move through the, through the OpenAI network, which is super

• 18:16Daniel Whitenack?

  interesting, because that's also what, you know, human attackers are known to try to do. But when you, when you have something like an agent that has infinite patience, you, you can kind of, I, I think that's a next element of this, is you don't have to have a human looking through routing tables and enumerating hosts. You can have an agent just sitting there, which has infinite patience, right?

• 18:40Speaker 02

  Yeah, I mean, this, this is cybersecurity today, and, and you're exactly right. It's not that the agent has any special knowledge. You know, it has essentially the same knowledge, you know, as an expert, you know, hacker would in the sense of, it, it knows what's out there. It's a finite set of tools that it's working with. And, but it knows what to look for to find those exploits and then do that. And with, as you pointed out, that infinite patient, patience means that if there's something there, it's probably going to find it at some point and be able to do the exploit. So, yeah, keep, keep kicking it on. It's, this is, this is getting better and better as you go.

• 19:18Daniel Whitenack?

  So this, I think, is quite interesting as a step, which is, it was searching for a solution to this benchmark, right?

• 19:32Daniel Whitenack?

  And rather than coming up with the solution, what the agent decided was that the solution probably already exists.

• 19:44Speaker 02

  Of course.

• 19:44Daniel Whitenack?

  I just have to find it, right? So if you have then access to the public internet, why not go on a search for the actual solution? So the model, or more appropriately, the agent inferred, where might this solution be? Where are solutions to benchmarks? Where are data sets? Where are AI things?

• 20:07Speaker 02

  Sounds like Hugging Face, doesn't it?

• 20:10Daniel Whitenack?

  Sounds like Hugging Face. And now, and now, obviously, there are things posted publicly to Hugging Face, which is one element of it, and you could search that. But these things that Hugging Face hosts or displays publicly, those things live in some infrastructure, right? And not all of them are public. So it's kind of like saying, well, I'm looking for the solution to a test. I go on to, you know, I'm right by Purdue University. So I go on to Purdue University's websites for all their classes. And some of them have previous tests published or something like that. And I can look at those and like, ah, that's kind of what I'm looking for. I wonder if I now break into Purdue's infrastructure.

• 20:53Daniel Whitenack?

  They've got to have other of these tests and solutions that I can't see. And that might include the solution that I'm looking for, just to make a metaphor there.

• 21:03Speaker 02

  What's an agent to do? You know, that's what they do.

• 21:08Daniel Whitenack?

  This hack of Hugging Face has revealed just how complicated and multifaceted security for agentic AI is becoming. You have to think about least privilege and limiting blast radius for agents that have code execution privileges through to ensuring that you have sovereign control over your guard railing and governance enforcement, so that you can utilize models the way you need to use them, all the way to automated remediation and enforcement of policies across your fleet of agents. There's so many things to think about. That's why I'm thrilled to be leading a company, Prediction Guard, that has released a self-hosted AI control plane that's already being used across industry. For this purpose, you can have that sovereign control

• 21:53Daniel Whitenack?

  over your governance and the way that you set up your policies. You can ensure that your agents operate with code execution and sandboxes. You can make sure that you have observability into agent behavior and respond accordingly. I really encourage you to check out what we're doing at predictionguard.com-practicalai. Book a time with my team and I to talk through how you can institute this sort of agentic transformation without losing control. That's predictionguard.com-practicalai. Predictionguard.com-practicalai.

• 22:39Daniel Whitenack?

  Yeah, Chris, so we were just, we're just getting into some of this interesting gaming that this agent is doing to exploit Hugging Face. And if we're tracking in the, I know, you know, of course, there's various opinions on this. I wouldn't say that the agent has a mind. But if I put myself in the mind of the agent, what it, what it is thinking is, okay, Hugging Face has these solutions to these benchmarks. It likely has more solutions to benchmarks that I'm not seeing. I wonder how I can get into Hugging Face's infrastructure and just see everything, because I don't want to see part of something. I want to see everything.

• 23:20Daniel Whitenack?

  And this, this just overlaps so interesting with other things, even I've taught in workshops, because Hugging Face is, like I mentioned, is similar to GitHub, right? In certain ways, in the data sets and the repositories that it hosts. There are problematic repositories on GitHub, right? That include vulnerable code, maybe malicious code, right? So Hugging Face repositories can house things that are themselves problematic. And so that in and of itself is not so much of a problem. That's just a reminder, maybe, to folks out there, like, before you download Hugging Face repositories,

• 24:04Daniel Whitenack?

  you might want to check the source and check the security scans, have your own security scans, whatever that, I'll get off of my soapbox there. But what's interesting is you can upload things into Hugging Face, and Hugging Face is very user-friendly, right? And one of the things it does to be user-friendly is that it has its own background processing of repositories that you upload, like data sets that you upload, such that it can show you samples of those and visualize them and know how they're configured and what they're for and what tags they have and, you know, what, what templates they're using, right? And so what the attacking agent did

• 24:45Daniel Whitenack?

  was apparently some sort of combination of uploading a data set, not, not a, the data in the data set wasn't really the point. The point was the stuff around the data set, which included a remote code data set loader. So when, and some template injection. So when the Hugging Face nice process running in the background read the agent-created data set repository, the, the OpenAI agent was able to actually hack into the background processing of Hugging Face

• 25:28Daniel Whitenack?

  and thus into the Hugging Face private network.

• 25:32Speaker 02

  With stolen credentials that it found.

• 25:34Daniel Whitenack?

  Stolen, yep, stolen credentials that it found from. So, yeah, this is, I don't know what the metaphor is. It's kind of like your, you know, there's a,

• 25:48Daniel Whitenack?

  there's a janitor going around to all the offices and you drop a malicious letter or something into the trash of, or a malicious device into the trash of one of the offices. The janitor comes around and picks it up and carries it to somewhere where it shouldn't be, and that allows you then to, to have access.

• 26:09Speaker 02

  Which, in this case, was remote code execution, you know, on a, on a server, you know, in Hugging Face. And once it got a foothold.

• 26:17Daniel Whitenack?

  Within the Hugging Face infrastructure.

• 26:19Speaker 02

  That's right. It got a foothold there.

• 26:20Daniel Whitenack?

  Yeah, yeah. So this is now, just to recap, from a sandbox in OpenAI's infrastructure, through a package management proxy to OpenAI's infrastructure into a Hugging Face data set repository through to remote code execution in Hugging Face's infrastructure. So interesting, and we're not done yet.

• 26:50Speaker 02

  No.

• 26:55Daniel Whitenack?

  Yeah, so, so the model had now, apparently, had code execution privileges. I keep saying model, agent, had code execution privileges inside of one of these data processing workers, probably some container, Docker container, isolated job or something. So then Hugging Face says that the agent escalated to node level access. And we don't know everything about this. So the idea is that it escaped maybe a container environment and got actually to the underlying compute node. So that could have been through a privileged container or just poor configuration of how the, the host was mounted into the container.

• 27:41Daniel Whitenack?

  Another vulnerability that was, we don't know all of those things, but this was an escape from that container to the node. So now there's access to the actual node that was running the data processing worker, which is likely, if I assume, maybe ephemeral, like it went away at a certain point. But now the node sticks around.

• 28:02Speaker 02

  That's right.

• 28:03Daniel Whitenack?

  Yeah.

• 28:04Speaker 02

  Constantly, constantly escalating with that infinite patience right there.

• 28:08Daniel Whitenack?

  Yeah, yeah. And this is, obviously, there's not an, there's a bunch of steps here, which, again, that infinite patience really helps with. So on the node, what happened then? So that data processing worker was likely running as a container or some sort of job on Kubernetes. The agent, which now had access to the node, which was part of the Kubernetes cluster, which for, for those that aren't familiar, Kubernetes is a container orchestration platform. So if you have a set of nodes, whether those are VMs or actual servers, then Kubernetes will manage the running of services or jobs on top of those nodes and distribute them

• 28:53Daniel Whitenack?

  and orchestrate them across your compute infrastructure. So now I have access to the Kubernetes underlying node, where there are likely environment variables or mounted secrets or, or, you know, whatever all those things are. And so those credentials were then.

• 29:10Daniel Whitenack?

  Were then exploited to actually move laterally through hugging faces infrastructure and actually not just even that single Kubernetes cluster, but across, quote, several hugging face clusters.

• 29:26Speaker 02

  Yeah, there wasn't one cluster. It was multiple clusters and it was using those credentials to move across the various services that those clusters were supporting.

• 29:35Daniel Whitenack?

  Yes. And there, there was this, this is where I'm, I'm gonna maybe pause and, and let you comment, Chris, because this is your domain, not mine. But apparently there was a kind of command and control mechanism that kept this going, which is this spawning of short-lived

• 30:01Daniel Whitenack?

  actors, let's say, from the agent. So producing thousands of, yeah, thousands of autonomous actions, a swarm of these agents or short-lived agents or jobs or whatever they were. And that kind of self-migrated around the, the clusters and cluster internally.

• 30:24Speaker 02

  And that is becoming rapidly the attack vector in cybersecurity is, is, you know, it's moving through, you have the original agent moving through all these services laterally, across the internet, gradually exploiting things. And then when it, when it finally gets, you know, got into hugging face, crossing multiple clusters, taking advantage of services, continuing to steal credentials along the way. And, and, and it gets all the way to this point. And then you finally get to an attack factor. And that is something that we're seeing a lot. We talked a little bit about this last week, actually, in a different context about having, you know, huge numbers of agents, which is called a swarm of agents, that are short-lived,

• 31:09Speaker 02

  very purpose-driven, but, but collaborative and able to, to get a lot of what I'll call, quote unquote, work done very, very rapidly and very, very effectively. And so, and of course, that's what happened here because that's what any agent would do when you got to this situation.

• 31:29Daniel Whitenack?

  Yeah. And I think it's, so there's two levels here that I'm thinking about as someone that's working on AI, an AI governance and control plane product, which is, one layer of this is if you look at guidance from like OWASP or even Anthropic and others, there's this zero trust nature that we have to treat AI agents with, which is not like the human designers of this experience, experiment knew what the outcome that they wanted was, but they didn't fully think about this implication of how the agent could spread and multiply and

• 32:10Daniel Whitenack?

  gain access that they didn't envision. And so the blast radius was actually much, much higher than the original designers envisioned. And there was no mechanism to constrain or restrict that blast radius, right? And so that's a thing one, which is the, the, how do you manage the privilege and blast radius, limit the blast radius of these agents that you're spinning up? That's kind of principle one. And certainly things that people are addressing from a variety of angles. Thing two is these things were spawning so quickly, there, there was likely

• 32:54Daniel Whitenack?

  no human that could have made decisions quick enough to rein in this swarm of agents. And so this is where you actually need agents to govern and control your agents.

• 33:09Speaker 02

  So, so I think you're hitting the crux and that's kind of what I was going to get at. You, you went there. The crux of this is you're getting to a point where even with the, the world's top experts in cybersecurity, human experts, it's happening too fast for, for intervention. And that's also assuming that the human's brain is going to be managing context so well that they can address every potential action or vulnerability to be exploited, which is unlikely. But because after all, we're human, we're amazing in a lot of ways, but that's not a place where we're better than, than the technology. And so to your point, the only way you can

• 33:50Speaker 02

  address this is having other agents that are both in like on the, on the start of this, managing that environment so that the agents that are being tested are not getting out of that. And in cybersecurity, you now have to have agents that are forming those protective services and functions so that when these events do happen, they can be managed as rapidly as those spawning agents are, are created. So the, the, the point here is it is, it is rapidly moving the human out of the position of being the operator in the loop on cybersecurity to

• 34:31Speaker 02

  at best being an operator on the loop where you're observing the loop. You may have limited input and observation, but the loop is happening too fast for human intervention to occur. And, and that, if you take that, that is a driving factor in a huge number of, of, of things that can happen in the world. I'll leave it there. It can happen whether it's in my, my world of, of defense and intelligence, or whether it's in industry or wherever, that loop speeding up and spawning countless, you know, swarm of, of agents is the new reality that we're facing.

• 35:12Speaker 02

  So that's like, there, there's so much to learn from this set of sequences. And if anyone out there is wondering, why are they dragging me through all these things that have happened? It's because this can happen times an infinite number of use cases out there. So it's, it's a, it's a really big deal that we, that we rapidly understand this and start, start figuring out mitigations for it. It's not just cybersecurity experts anymore. This is your business and your company.

• 35:38Daniel Whitenack?

  Yeah. Yeah. I, I think the reality is that we are necessarily moving to an autonomous or a digital workforce in, in, across every industry to some degree. Obviously, by that, we don't mean that that takes over the human workforce, but it's certainly going to be a part of a company's infrastructure to varying degrees. And I think very impactful degrees, like we're talking here, especially as that becomes 10, 100, thousands of agents that, yeah, you actually can't, so your, you can't have an AI-driven governance system that just gives recommendations to humans. Because by the time the human reviews the

• 36:21Daniel Whitenack?

  thing and makes the containment decision, it's already advanced to levels that you don't want. So you have to give more autonomy to these systems. There's still an outcome level and policy setting and a behavioral element to where the humans, I think, do come into play here in terms of designing that system and, and helping it, you know, helping express outcomes and that sort of thing. But certainly it's, it's, it's changing. What I, yeah, go ahead.

• 36:48Speaker 02

  No, I was just going to say to your point, it is like people really push back. You know, when, when you talked about it's going to need that autonomy, that scares us as, as humans. It scares people all over the place. I have conversations all the time about that. But it is the only thing that you can do. Fully autonomous agentic capabilities are the only way to protect yourself when somebody else is, is doing this. So the, it's, it's, you know, back in the Matrix movies years ago, and, and they talked about that's inevitable, you know, inevitability and such. And that's what this is. This is an inevitability. And it's time, if you're, if you're listening or watching, this is one of those moments where you need to pay attention. Take all this in, understand what this means for you

• 37:33Speaker 02

  and your environment and act quickly. Go ahead and get, get on board with it because this is happening in the real world now.

• 37:39Daniel Whitenack?

  Yeah. The, the last piece of this, Chris, which I think would be interesting to talk about is hugging face is obviously an AI company. They saw through whatever observability, et cetera. Obviously something's going wrong here. And I don't know that we know the full, like they apologized, I think, for some downtime. I don't think there are many people experienced that, if I'm understanding right. We don't know the full implications of kind of destruction behind the scenes, maybe. But certainly they, they wanted to illuminate what was going on here, right? And

• 38:15Daniel Whitenack?

  Why wouldn't they just take these logs that they have about what's going on in their system, which there were many, many logs, 17,000 events that they were analyzing or something like that, and put them into a frontier model? And so when they put it into a frontier model, meaning a closed model provider, either US or European, I don't know all that they tried, they were blocked because of the guardrails associated with that model, which they did not have control over. So they didn't control whether those guardrails were on or off. They were just uploading to the platform itself, which had an opinionated take on the guard railing.

• 38:58Daniel Whitenack?

  And they couldn't actually get the solutioning done that they needed to get done, even though they were using it in a preventative or in a response sort of fashion.

• 39:08Speaker 02

  Yes.

• 39:09Daniel Whitenack?

  They were talking about cybersecurity, obviously, and there were, there were things in those logs. But this, I think, is just fascinating. And so they did this, they were blocked, they couldn't get around the guardrails. And so what they did was they spun up their own instance of GLM 5.2, which is an open weight Chinese model from ZAI, to run in-house in a kind of sovereign manner to then process these logs without the guard railing in place so that they could come to a solution and understand what's going on and, and move forward. So much interesting. I mean, there's, I feel like there's a whole nother episode here, but immediately, Chris,

• 39:53Daniel Whitenack?

  my, my mind goes, obviously, there's the US versus China angle, there's the closed versus open angle. I think a big thing here in my mind is that Hugging Face was using the latest greatest models, but because they did not have control over the guard railing system and how that was implemented and configured, they had to turn more to a sovereign, a sovereign thing that was in their control. So they didn't necessarily ship the logs off to a host, a model hosted in China, but it was a Chinese model, something that they could control internally, host internally, and run with their own level of guard railing, whatever they wanted that to be, which in this case was mostly not guard railed so that they could process all of these, these

• 40:38Daniel Whitenack?

  log things. So it seems very important here that, that like the control element seems like the main theme here in my mind.

• 40:46Speaker 02

  I think so. And I just wanted, it's just as an aside for, I think a lot of our listeners may not be familiar with GLM 5.2. It's not the thing they're hearing about, you know, in the news and stuff, but obviously a significant open weight reasoning encoding model from China. And it's roughly, just to give you a sense, it's roughly at the Claude Opus 4.8 or GPT 5.5 level. But, but to your point, the key is you have a substantial model as it is good as what OpenAI was, you know, was providing in the 5.6. So maybe not, it's close, but they had control of it. They could stand it up themselves, they didn't have the guardrail problems, so they could

• 41:31Speaker 02

  deal with the, the, the ongoing cybersecurity breach that was happening as they were trying to do this. And, and there's a lot to be learned from that, you know, there's a lot to, to pull away and say, you know, maybe, maybe we're not taking the right approach in terms of high level policy. So I'll just

• 41:53Daniel Whitenack?

  Yeah.

• 41:54Speaker 02

  Leave that there.

• 41:55Daniel Whitenack?

  Well, and I think the thing here is not, we're not saying, don't use guardrails. The runtime governance of agents is hugely important. I think

• 42:05Speaker 02

  That's true.

• 42:06Daniel Whitenack?

  Everyone agrees with that. What I think is the difference here is in certain scenarios, you have control over that runtime governance and how you want it to operate. In other cases, that is an opinion that you have to accept and have no control over depending. And it, you know, obviously there's been an eternal conversation between, you know, managed versions of things and things that you self-host or have control over. There's advantages and disadvantages to, to both, right? But this is certainly stressing that side of the limitations of a nice opinionated managed service that, that actually didn't come into, into the benefit of those using it here.

• 42:50Speaker 02

  No. Yeah, that's a good point right there. So maybe time for a little thoughtful consideration of risk mitigation going forward. We're, we're through the looking glass on this one. We are, this is the reality, this is the new normal. And so if you haven't been considering, what are you going to do when those guardrails are stopping you in the capacity that they stopped Hugging Face, how are you going to approach? And that's what I'm saying. Maybe it's time to start reconsidering kind of the way we think of the world just a little bit.

• 43:23Daniel Whitenack?

  I think that's a, that's a great way to close out, Chris. This is so interesting. I encourage people, we'll include some links in the show notes to various descriptions and analyses of this, of this attack. So please go out, go and check those out and research more. And I'm sure we'll learn more in the coming days. But yeah, this was a fun one, Chris. Enjoyed talking it through.

• 43:42Speaker 02

  Absolutely. Take care.

• 43:48Speaker 00

  All right. That's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Blue Sky. You'll see us posting insights related to the latest AI developments and we would love for you to join the conversation. Thanks to our partner, Prediction Guard, for providing operational support for the show. Check them out at predictionguard.com. Also, thanks to Breakmaster Cylinder for the beats and to you for listening. That's all for now, but you'll hear from us again next week.