Building on open source AI? Hugging Face’s policy chief on the costs, risks, and trade-offs

Open source models are becoming more mainstream as enterprises, governments and startups alike rely on the promise of lower costs, more control, and more flexibility. But as increasingly capable models become more freely available, how do companies capture value? And with Chinese open models gaining ground, can American players keep up? On this episode of TechCrunch's Equity podcast, Rebecca Bellan caught up with Irene Solaiman, Chief Policy Officer and SVP of Strategy at Hugging Face, about what startups need to know before building on open models, from choosing the right tools and infrastructure to navigating the trade-offs shaping the future of open AI. Listen to the full episode to hear about: Why startups, enterprises, and governments are switching to open AI models to cut costs, and when it makes less sense to use the most powerful models. Who makes money when the models are free to use, and whether the companies building them can capture the value they create. Why the U. S. may need deep-pocketed players like Nvidia to build an open frontier model that can compete with China’s, and what’s at stake if Chinese models become the global default. How recent attacks on open AI platforms are raising a difficult question: how do you keep an open ecosystem secure without closing it off? Subscribe to Equity on YouTube, Apple Podcasts, Overcast, Spotify and all the casts. You also can follow Equity on X and Threads, at @EquityPod. Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Speaker 00

  Life is all about making decisions. Sometimes you make questionable ones, like buying a new TV that's too big for the wall because the electronics store employee was a little too good at their job. Then sometimes you make great ones, like protecting your home and auto with Progressive. You get customized coverage options so you can be sure you have the right coverage. Get your quote today at progressive.com. Progressive Casualty Insurance Company affiliates and other insurers not available in all states.

• 0:30Rebecca Bellan?

  Hello and welcome back to Equity Tech Crunch's flagship podcast about the business of startups. I'm Rebecca Bellan and this is the episode where we bring on industry experts to help us explore a trend in the tech world and dive deep. Open source models are becoming more mainstream as enterprises, governments, and startups alike rely on the promise of lower costs, more control, and more flexibility. But as increasingly capable models become more freely available, how do companies actually capture value? And with Chinese models gaining ground, can American players keep up? Today, we're joined by Irene Solaiman, Chief Policy Officer and SVP of Strategy at Hugging Face. Irene has spent her career researching how AI models are released, evaluated, and potentially misused. We're going to talk about what startups need to understand before building on open models,

• 1:13Rebecca Bellan?

  from the economics and global competition to the cybersecurity threats that could put openness itself at risk.

• 1:24Rebecca Bellan?

  Irene, welcome to the show.

• 1:26Speaker 02

  Thanks, Rebecca. I'm so glad to be here.

• 1:28Rebecca Bellan?

  Yeah, I'm so glad to talk to you again after we just recently had you at Disrupt a few weeks ago. You were onstage talking to us about evals, and I'm really excited to talk to you because you can just riff, as we both know from our many conversations we've had, you can just riff on so many things when it comes to open source. So just to give our listeners a bit of background, you joined, and stop me if I get anything wrong, you joined Hugging Face as a researcher in April 2022. You became the Policy Director, Head of Global Policy, Chief Policy Officer, and SVP of Strategy in February 2025. Quite a litany of titles there. So in that role, you're running strategy, geopolitics, research, and policy. You founded Eval Eval, an open source research community focused on AI evaluations.

• 2:11Rebecca Bellan?

  And before that, you were at OpenAI, working on bias, social impact, public policy. So stop me, did I get anything wrong?

• 2:18Speaker 02

  You know me better than I know myself, Rebecca.

• 2:20Rebecca Bellan?

  Oh, my gosh, yes, okay. So, one thing I wanted to kick off the conversation with, when we talk about open models, you know, we had Clem DeLong, your CEO, on in July. He talked about companies moving away from expensive AI services and toward open models that they can run themselves as a way to cut costs. We've also got investors pouring billions into training, you know, closed models. But if these models themselves can be commoditized and more freely available, how, who's making money? How are we making money here? Where are the threats? And where are the opportunities?

• 2:55Speaker 02

  Well, first, on the switches, the conversation you had with Clem, it's funny how prescient you've continuously been. I realized it was before the incident that we endured in July. And when we're seeing already a trend of movement towards open models, it's only expedited since then. So as of, I believe, today, Europe time, I'm not sure if you saw Revolut, the kind of neo-bank, for our American listeners, that was founded in London and operates largely in Europe, announced that they are switching to open weight models and using only cloud and codecs for exceptions. We are seeing this overall trend, particularly since July, of more established and startup

• 3:40Speaker 02

  companies announcing their shift to open weights with the explicit purpose of cutting cost. To your question about what does this mean for the model developers, there are a few ways that people earn money, and it is harder for the developers if everything's hosted on-prem, but preferable from an enterprise perspective, both for the cost but also for the data privacy and data residency, especially if you want that in a certain jurisdiction. But particularly for making money, a lot of that is going to be inference costs. It could be a sort of cloud lock-in if folks are using inference via, for example, Alibaba

• 4:22Speaker 02

  Cloud, which is the parent of Quen, Alibaba's excellent flagship model. Or it's a sort of revenue sharing system, which is the kind of boring area of licenses that we can get into if our readers, if our listeners don't get too snoozy.

• 4:38Rebecca Bellan?

  Yeah, okay, so revenue, like for the, for the open model providers, like revenue sharing built into the model license? Or, I know this is something that you've talked about, kind of wanting to spearhead it at Hugging Face.

• 4:48Speaker 02

  Yes, so licenses are generally drafted in legalese, which is why they don't get a lot of active readers. And there are a couple of standard ones that you see most commonly with open weights, MIT and Apache 2.0, they're quite permissive, as opposed to the latest iteration of flagship open weight models that are using custom that require some level of revenue sharing if you make some threshold of revenue.

• 5:19Rebecca Bellan?

  So, so we're talking revenue sharing, there's, there's potential for cloud providers to get a piece of this pie. Maybe the companies that are, you know, building the apps off these open weight models, are we, like, could we possibly be living in a world where open models are creating enormous value for the AI industry while the companies actually developing them are struggling to capture much of it?

• 5:39Speaker 02

  It's hard to say without seeing the explicit financials, and not everything is a public company, so it'll be quite an interesting next few months. The level of value from the deployers and from the developers, I think, also depends on the makeup of the, the business organization. So for Alibaba Cloud, they're just such an enormous cloud company, it could be considered a sort of feature to bring folks into Alibaba Cloud, and Quen is used really worldwide. So it's a, they're both, their smaller model, the 27B, which is Apache 2.0 and rentable on a laptop, and their larger model, which is Frontier Level, the latest is Quen 3.8

• 6:22Speaker 02

  Max, which is open weight, can be used on-prem, or it's easier to use inference. So a lot of folks in those kinds of scenarios do make a substantial amount, but we're not able to view those exact metrics.

• 6:37Rebecca Bellan?

  Right, so, okay, so some of the bigger companies, specifically, we're talking China here, like Alibaba, they're offering open source models, but they're also offering the cloud and the compute, right? So they're able to make, offer this as kind of a package deal. Whereas if we look at maybe the American ecosystem, that's not necessarily the case, right? A lot of the open model providers are just open model providers. They don't have compute that they can, I mean, maybe they have deals with NeoClouds that they're trying to like sell a package deal with, but they're not the ones getting revenue from the compute. So I kind of, I kind of worry for them a little bit more in this, in this case.

• 7:12Speaker 02

  It's a tricky decision on if you're releasing an open model that doesn't quite reach or beat Frontier Level. A lot of folks love using Frontier, I mean, they're great products, they're easy. Then how do you get those kinds of users interested in deploying? And this is where these customized licenses come in for the US model providers on the open weight side. Some of the more visible ones who have done a great job are, I mean, Meta historically has been an open weight champion, and RC, Reflection, have been releasing with permissive licenses, but they also do enterprise contracts. So you'll see with Reflection, they did a enterprise agreement with South Korea and

• 7:57Speaker 02

  have a partnership to build a sovereign AI cloud. So there are other ways to do these kinds of custom solutions.

• 8:03Rebecca Bellan?

  Okay, so, so almost like a white glove service, well, like, you know, you want to build your own sovereign or enterprise solution, and you want to use open source models, well, we'll build it for you, kind of thing.

• 8:13Speaker 02

  Yeah, it's for how much interest there is in AI, there are very few people that really know how to customize this relatively.

• 8:19Rebecca Bellan?

  Yeah, I mean, we're seeing that with, you know, the whole, I mean, it's kind of side note, but with the whole FDE trend that we're seeing, everyone's, you know, this is becoming the hottest job in AI is a deployed engineer, because there's so few people that actually, you know, know enough about business on the one side, and then how to, you know, work with AI tools on the other in order to build anything that's bespoke for enterprises or governments.

• 8:40Speaker 02

  Exactly.

• 8:41Rebecca Bellan?

  You know, a lot of our listeners are founders. So why should a founder care? Like, what do they need to check before trying to build a product on an open model?

• 8:51Speaker 02

  Before building, I mean, this is a question of what are their preferences for the task? And this is where we get into routing. We don't have great model routers to pick and choose the right model for the task. Being able to create a stack where you have a potentially lower cost, but effective model for certain tasks. Let's just say like, I'm thinking a lot about cybersecurity. So some level of cyber monitoring for giving summaries for low stakes infrastructure. You don't probably need something that is the most premier frontier model, but you will need to understand what is the capability of the model for the tasks that I want?

• 9:35Speaker 02

  Where do I put it throughout my infrastructure? And do I have the capacity on hand to run this model? Or do I need to rely on inference, whether that be an open weight, which could be cheaper than a frontier or a frontier model itself?

• 9:51Rebecca Bellan?

  Yeah, okay, that's interesting. Like, this is something I keep hearing that not every company needs the most powerful model for every task, right? So what kinds of work are open models, you know, some of the maybe smaller ones, are they increasingly good enough to handle? Like where, and then, you know, the second question for that is where do the more advanced models still have a meaningful edge?

• 10:11Speaker 02

  That, that, that word is so important, that phrase, good enough. I think this really comes down to a trade off for how much you also want to spend on token maxing. I shudder to think about token cost without open weight models. And this is not a reflection of the frontier labs. It's just incentives in, in the marketplace. For what folks want to use open models for, what they're capable for, they can do quite a lot, if not the same agentic task. I rely entirely on almost exclusively GLM 5.3 these days. Sometimes the KIMI K3 using a Hermes agent for almost everything. And I think this is the trade off for it might need extra prompting.

• 10:55Speaker 02

  It doesn't give me refusals in the way that a frontier model would. But I think this really depends where I, I would say I use it for everything.

• 11:04Rebecca Bellan?

  Yeah. And, you know, it's not just about like, you know, there's a good enough model, but so much of it increasingly is the harness, right? It's, it's the, it's the structure, the infrastructure that's built around the, the model itself. So I mentioned, you've mentioned to me before that there are like less capable models with a better harness can actually outperform a frontier model. So how do you think that this is affecting, you know, overall performance?

• 11:26Speaker 02

  Spot on. On these more specialized startups or tasks, one thing that we had discussed before, thinking about cybersecurity a lot, there are harnesses that can zoom in to specific cyber defense tasks and leverage models. And it will still use a generally powerful model that can address vulnerability identities, for example. These kinds of harnesses, we're just so many, are, are popping up more and more, are custom made. This can be a revenue stream in and of itself of folks who are able to design harnesses that our economy needs right now.

• 12:06Rebecca Bellan?

  Yeah. And we're seeing more and more of that. And it's, it's like, you know, being a journalist, my inbox is filled with startups trying to like, ride the latest wave. So it was, you know, at first it was just like app companies built on AI. And then it was agent companies, those are still pouring in. And now it's like harness companies. I guess agent is a, like the harness is the agent, or, you know, they're, they're related in a lot of ways. You know, at Disrupt, you and I talked a lot about model evaluations. There's a lot of work that can be done to make sure model evaluations are, are being done in the right way. But there's not, one of the things that appears to be missing is that there's no good evaluation for harnesses and tooling.

• 12:43Speaker 02

  Yeah.

• 12:43Rebecca Bellan?

  What do you think about that?

• 12:44Speaker 02

  Yes, yeah. I loved, I love generally talking with you, but especially in evaluations, it's a nerdy area that has gotten attention for how important it is for deployment, but is an area of science. We just have a lot more science around natural language processing, the basis field for LLMs. We don't have a lot of background for harnesses as they're deployed now is relatively novel. We don't have explicit evaluation suites for harnesses in ways that are standardized pre-deployment of harnesses. And the way that, I think a big barrier for this is just how darn costly they are. Agentic workloads, if anybody who's used them, just cost a lot of money for all of these tools.

• 13:28Speaker 02

  And so if you're running evaluation, it'll easily go up into the millions.

• 13:32Rebecca Bellan?

  Yeah, yeah. So how can a, like a startup manage this, right? How can they know whether a cheaper model and its surrounding tools are, are good enough for its product or cheap enough for its product if there's no reliable way to measure, you know, the, the spend and ROI and just like what you're getting?

• 13:49Speaker 02

  It's going to be a trade off for the near term and long term. In the near term, it might be easier just to run inference, to call a frontier lab. If you have like high intensity tasks, if you don't have a lot of AI experts on hand, totally understandable to use what's easy. And a lot of the time that's going to be closed. If you want something that you want to scale up a bit, you know, you'll use it longer term. And you can, this is why a lot of folks are buying GPUs and not renting. You can set it up on prem, you can optimize that model over time. Without renting out the cost of compute going up, your long-term costs will likely plateau or decrease. Desirable depending on the, the business that you're running.

• 14:31Speaker 02

  And for the, the choices that you're making for high intensity coding tasks, the latest open weight models, KIMI K3, Deep Sea V4.1 Flash, and the Qwen 3.8 Max are optimized for coding tasks. So you can look at these types of benchmarks, you can look at the types of coding tasks you're interested in. Or if again, you have these skills, you can fine tune that model. There are a lot, on Hugging Face, a lot of data sets for fine tuning to optimize for certain tasks.

• 15:05Rebecca Bellan?

  You mentioned, just want to quickly dig into this. You mentioned that, you know, a lot of people are buying their own GPUs these days, right?

• 15:10Speaker 02

  Yeah.

• 15:11Rebecca Bellan?

  I'm kind of curious about that. Like, do you have any tips for, because you know, where all we hear about is GPU scarcity and it's, you know, like, let's say I'm a startup founder. I need to mess around, I need to do a lot of experimentation, I'm using open models. I want to have my own, you know, GPU situation going on. Am I buying, like, how many GPUs would I need to buy? And is it okay to, you know, not get the latest, greatest Nvidia chip? Because you probably can't anyway.

• 15:37Speaker 02

  I mean, yeah, the scarcity is not just to your point, the, the availability, but the availability of what? And a lot of, there is a hunger for the latest Nvidia chip because a lot of the newest models run easiest on them. In terms of what you can get your hands on, there are just a lot more libraries and resources, including on Hugging Face, and generally this is the beauty of the open source ecosystem, that will help you adapt your model to that chip. So we see these previous iterations of GPUs, so like the Nvidia Blackwell. A lot of folks use for, for frontier models. And this is, we're going to get a little bit technical here. It's also a question of the latency for what you're deploying.

• 16:22Speaker 02

  And this is the, the level of speed of response. For our incident over the summer, we deployed a Nvidia Quantize. So this is NVFP4, it's a four bit quantized model. It's not going to have that same level of sort of horsepower that a fully high speed kind of inference will give you. But it'll get the job done and it'll cost significantly less than what we estimate, what we estimate the agent swarm cost.

• 16:50Speaker 00

  Progressive's Accident Response can contact emergency services if you're in a major accident. The only thing worse than someone you love being in an accident is not being able to help them. So Progressive is here to help when you can't. So get peace of mind knowing your loved ones are protected with Progressive's Accident Response. Learn more at progressive.com. Progressive Casualty Insurance Company and Affiliates. Limited availability in California. Driver is responsible for emergency and towing charges not covered by the policy. Coverage subject to policy, terms and conditions.

• 17:20Rebecca Bellan?

  And, and for the listener, if you don't know what Irene is referring to with the incident, the incident is obviously when OpenAI's Agent Swarm accidentally kind of hacked into Hugging Face. And Hugging Face was able to defend itself using some, some open models rather than the closed models, which I think threw up some guardrails, which is something that we're, we'll, we'll actually talk about that again, because I want to talk to you about the open code attack. But, and then another disclaimer, Nvidia has acquired or is in the process of acquiring Hugging Face as well. So just, you know, note to that. Another thing about Nvidia, and I'm going to Come to it in a roundabout way. I want to talk a little bit about China and competition for open model dominance. You know, the best open models, you've listed several of them.

• 18:04Rebecca Bellan?

  They're all Chinese at this point. What happens if Chinese models become the default for AI startups around the world and actually not just AI startups, like sovereign AI for governments? And, you know, we're seeing Rio de Janeiro and Japan sovereign models are built on Quin. So what is that? What happens? Is that a problem?

• 18:25Speaker 02

  It's a, it's always a dance when we're talking about open models and with the geopolitics. The geopolitics have become much more central, even the last few months. It's a question of problem for whom. So people love open models, no matter where they come from. I've had to scale up very quickly with our head of APAC. It's really just me and Adina working with Open Frontier Labs, these Chinese labs who are so talented, who work really hard. And it's incredible to feel the ethos of belief in open weights. It would be even more exceptional to have a thriving global ecosystem and to have open frontier US models.

• 19:05Speaker 02

  We are working closely with, with the Muse team, with Reflection, with RC. It's really hard to train an open frontier model, but there's a reason that governments are preferring models developed in the US, for example. There's a reason that GPT OSS still has a large amount of downloads on Hugging Face. There is a hunger for US models. How do we stoke that? I think is a big, a big question around the narrative. There's been a demonization of open source, of them being, these models being more dangerous, which if you look at things for the, the explanations and disclaimers, Rebecca, the last few months of cyber attacks, the most prominent ones have been frontier models, have been closed weight models. And

• 19:50Speaker 02

  we're just balancing with what do we need to make sure that folks get access and have the tools in hand? And a lot of that, a lot of the time, it's open weight.

• 19:59Rebecca Bellan?

  You said that there's like, you know, there's a hunger for GPT OSS. There's a hunger for American open models. What? So, and a lot of people, a lot of companies, governments want to build on American models rather than Chinese. What, what is the main reason? Is it just like the geopolitics of it all?

• 20:13Speaker 02

  So for, also for context, GPT OSS was released by OpenAI around August 2025. That's, that's my bad for not explaining that.

• 20:22Rebecca Bellan?

  Sometimes I'm like, we're all nerds here, right?

• 20:24Speaker 02

  We know.

• 20:25Rebecca Bellan?

  We all know what we're talking about, right?

• 20:27Speaker 02

  But there have just been so many, I measured time and latest frontier generation.

• 20:32Rebecca Bellan?

  I can't believe that was 2025.

• 20:33Speaker 02

  I know. August 2025 was GPT OSS.

• 20:38Rebecca Bellan?

  When's the next open model? Come on now.

• 20:39Speaker 02

  We're also, you know, itching over here.

• 20:42Rebecca Bellan?

  Yeah.

• 20:43Speaker 02

  In, in AI time, that's a really long time for a model to be out there. There, there are much better open weight models since. So the fact that folks are still hungering for this shows us that sometimes it is a hesitancy to deploy Chinese models because of the geopolitics or, or some misunderstanding. I think some of this misunderstanding is the projection that folks are using APIs for Chinese models or models developed in China and then their data would be sent to mainland. We are very purposeful about none of that data being sent to mainland by inference. The partnerships that you see with Chinese based labs and AWS,

• 21:23Speaker 02

  for example, show you that there are ways to deploy these models without having your own GPUs, but based in the US. Some of the, the reasons that folks are using it outside, using chat, US and not Chinese based models, developed models, could be around this, this misperception, avoiding scrutiny. But it could also just be a preference for some of the values if you don't have those kinds of technical depths to be able to, to fine tune. There are some differences and there's some thoughts on what refusals you get from models developed in China, for example.

• 22:03Rebecca Bellan?

  Yeah. I mean, the, the basic one that I know about, and I think this is probably more of a function of their API than, than actually the model. But I'm not sure. You tell me like, you know, earlier, early on in the whole LLM discussion, you know, everyone was testing DeepSeek or not even DeepSeek at this point, probably. It was probably too early for DeepSeek, but they were testing some of the Chinese models, like asking about Tiananmen Square and then they wouldn't answer, right? So I'm not sure. Is that kind of what you mean? Or is there other, other refusals?

• 22:28Speaker 02

  Yeah, that's the classic one. And the part of the reason that you've not seen in-depth analyses is that cultural evaluations are so fraught. We've done a few ourselves and to be able to do it justice would be PhD level of work. But just from, like, this is where we've moved somewhat to vibes versus emails, to a combination of both, is there, there's, there's these kind of, it's like Claude speak, there's different ways that different models speak. The kind of cultures that you're looking for to be emulated, you can, you can fine tune and you can totally adapt and make that your own. But that is relatively intensive. Take some GPUs, take some talent to make work.

• 23:10Rebecca Bellan?

  Yeah. Okay. So, so GPT OSS, are there other open models from the US that are, you know, being downloaded or, or used or really popular on Hugging Face at the moment?

• 23:19Speaker 02

  We are seeing so much hunger for not just the Neutron, but the general family coming out of Nvidia and open weight side. So the Neutron family is the, the more of the language model side. What is generally slept on in the open source community are bio models and science and robotics.

• 23:41Rebecca Bellan?

  You guys just came out with a bio model, Hugging Face.

• 23:44Speaker 02

  We did.

• 23:45Rebecca Bellan?

  You came out with your own. That's really cool. Carbon X. And it's quite a small model. You can run it on one GPU.

• 23:49Speaker 02

  There you go. Yeah. And it's, if you do run it partially because of the one GPU, it is just orders of magnitude less costly than comparable models. And that goes a really long way for everyday biologists.

• 24:01Rebecca Bellan?

  Yeah. I mean, you, you brought it back to Nvidia for me. I promised I was going to bring it back to Nvidia, but you did it for me with Neutron. So I, I guess, you know, one of the, the things we kind of talked about earlier is like, who has the resources to back up a lot of the, you know, if you're going to be at the frontier of open models, like you need money. Like this is expensive to train these models, right? And of course there's, you know, distillation helps, which a lot of people give that a dirty word, but it is a very common practice in, in AI training. But like, does America need an Nvidia like behemoth to actually solve this and not only compete with China on open source, but to compete with OpenAI and Anthropic here in the States?

• 24:40Speaker 02

  I mean, this is a question of what does the overall ecosystem need? And this is, we are undergoing the regulatory process of joining Nvidia, which I'm very excited about. And as this is a thought process that I've had independently, but it's appropriate to give the disclaimer. I think we would thrive most in both open frontier and open local. So this is where you see on Hugging Face, the sweet spots of what people engage with, with both likes and with downloads. We look at many metrics are around like 20 to 40 billion parameters and with especially quantized or really souped up Mac, latest MacBook, DGX Spark, AMD Halo, you can run these locally or the kind of one to three

• 25:22Speaker 02

  trillion, what we call open frontier. For an ecosystem, we, we need the, the smaller open weights that are very permissive, but we probably need the resourcing that something like an Nvidia has to create a frontier American model that folks can explore, do more evaluations on, deploy locally, be a good alternative to our closed frontier.

• 25:52Rebecca Bellan?

  You know, Nvidia has been a big proponent of, of open models recently. I mean, my personal theory is because, you know, the more people are, are using open models, the more GPUs get bought, like just individually, or people need to use Neo clouds. And, you know, it's coming at a time when a lot of the labs are, if not actively doing it, threatening to build their own AI chips. So it's an interesting space, but definitely Jensen Wang's, CEO of Nvidia, his, his embrace of open source has been, you know, parroted by a lot of other CEOs. We're coming to a point where open models, the use of them is not as, you know, stigma anymore, right? Like everyone's kind of admitting to it. It's becoming more mainstream.

• 26:34Rebecca Bellan?

  But at the same time, it's still facing threats, right? And so you and I spoke a little bit about the open code attack. So I'm not sure. I hadn't actually heard of it by the time we talked because our algorithms are simply different.

• 26:46Speaker 02

  Yeah.

• 26:46Rebecca Bellan?

  But, and I feel bad about this because TechCrunch did cover it. But, you know, it was, it was around disrupt time. So it was busy for me. But anyway, you know, mid-October, open code was the target of a hack. So open code is an AI coding agent as part of the open model ecosystem. This was roughly a 48-hour agentic attack targeting users, third-party API keys. It compromised some accounts. There was some financial damage. So, you know, we've seen attacks now targeting Hugging Face and open code. Granted, they were slightly different in motivation. One was human motivated, the open code versus Agent Swarm motivated. But still, these platforms are, you know, potentially becoming bigger targets. Do

• 27:27Rebecca Bellan?

  you worry that the, that security pressures could force more closeness, closeness, closed, closeness in the open model ecosystem?

• 27:37Speaker 02

  This is why we need journalism, Rebecca. We all exist in the algorithmic prisons of our own making, and you are by far not the only person in our field to not have heard of this hack, especially it happened right before the PHP vulnerability exposure and just Black October took another level. The open code attack, we don't yet have public information on who the attacker was. We have some ideas of the motivation. So it was around 48 hours of intense agentic attacks that were seeking third-party API keys. This is meaningfully different from the Hugging Face incident over the summer that we endured, where kudos

• 28:17Speaker 02

  to our infrastructure team were able to recover. And it seems that open code was able to recover. But we did not have compromises of user data, of financial data. And this is, there was financial damage here.

• 28:31Rebecca Bellan?

  Yeah.

• 28:32Speaker 02

  The things that I worry about are what makes an open community targeted is also what makes it special. We are necessarily permissive. We want the average hobbyist in their basement, who isn't, doesn't have an institutional affiliation to be able to contribute. I mean, they are part of who made the internet exist. I mean, this is like examples of Wikipedia, just this, how few people are keeping Wikipedia alive. That's the beauty of open contribution. And we also have, so there's a permissiveness and the wealth of information and data we have. So for our Hugging Face, we have these really interesting evaluation data sets. Open code has these juicy third-party API keys. The combination of that makes us prime targets.

• 29:16Rebecca Bellan?

  Yeah, I mean, it's, it's, there's a, there's a defense gap there, right? So where do you draw the line between securing the open ecosystem and undermining the openness that makes it valuable?

• 29:28Speaker 02

  This is what we're, we're noodling on, on what is that trade-off?

• 29:32Rebecca Bellan?

  What, what kind of conversations are you guys having at Hugging Face? Like, anything that you can share? Like, what, what do you, like, what's, what are you noodling on?

• 29:38Speaker 02

  I mean, the conversations have gotten to a point where a lot of folks are, we are responding to our community. We are just seeing so much interest in obliterated models. This is, it's can of worms, really dicey topic. And they, they do instill fear in a lot of folks.

• 29:55Rebecca Bellan?

  Can you just briefly explain to our listeners what an obliterated model is?

• 29:59Speaker 02

  Yeah, it sounds like such a funny word. Obliterated models are models that have had all of their guardrails removed. And already, some of these open frontier models are pretty permissive in their, in their refusals and guardrails. But you, it can pretty easily just kind of fine tune that out and take those off and re-upload them on Hugging Face. This is not a violation of our Terms of Service. And part of why we are keeping them up is we have so much feedback from cyber defenders and researchers on how this is their only option for either research or defense.

• 30:33Rebecca Bellan?

  Yes. Yeah, I've, I've also come around on obliterated models a little bit after talking to cyber people, because, you know, I wrote about this. There's a company called obliteration.ai, I think, I think that's what it's called. And, and they basically just like host or will provide you with the ability to obliterate any model. So you just, there's no guardrails at all. And you can just imagine how horrible this is, right? Like, you don't want to make it easier for attackers. But at the same time, this makes it easier for defenders. And I guess, you know, the argument is that the attackers are going to obliterate model, they're going to do this anyway. It's important that the defenders have, you know, know how they're going to attack them, right? Essentially, and are able to, it's really good for, for red teaming as well, you know, really helpful for, you know, when you're

• 31:14Rebecca Bellan?

  trying to do the, the most insane version of a hack that you're trying to test your, your systems on to have an obliterated model. So, yeah, it's, it's, it's, it's tough. But then if open model infrastructure becomes, you know, more costly and risky to maintain, who, who, I wonder, would pay for the security? Like, would that potentially give large, well-funded platforms an advantage over the smaller open source projects?

• 31:40Speaker 02

  I mean, there, there's going to be just an advantage in the level of product support that you get from a closed weight lab. Most of these, outside of the, the really well-resourced like the AliBabas, you know, we're still 200 people while we're undergoing this acquisition. So we're not going to be able to provide that level of handholding. But for what folks need for, for cyber defense at a time where there's a lot of appropriate, I'm, I'm quite nervous about the state of the internet. We, we need to have some level of, this is a trade-off. We need to have access to the folks who know what they're doing, who are getting refusals from the most frontier

• 32:23Speaker 02

  models. There's a dichotomy, unfortunately, between cybersecurity and AI safety folks, and both have well and good intentions. But a lot of security folks love open source because of the privacy, because of those on-prem considerations, because of what they can do with it, and really have this ideological preference for open weights. In a way that is creating a new kind of open versus closed conversation. But the urgency, from our perspective and from where I'm sitting at Hugging Face, is that we just need to equip defenders yesterday. And if that means giving folks a more permissive model, it seems that's the balance that we'll have to make.

• 33:05Rebecca Bellan?

  Is there anything that could convince you otherwise? Like, is there any type of attack that might happen that would make you think, actually, like, I don't want, like, attackers to have access to, you know, powerful obliterated open models?

• 33:21Speaker 02

  I mean, this is the visibility of who's doing the attacks and who's doing the defense. I wish that we had better ways. This is what we've asked for policy-wise, is better incident reporting and standards. We're so glad that we were able to be as explicit as possible. But it was quite difficult to figure out how to report it and to whom. So as folks, both on the, the frontier side, but also who are doing defense, who are victims of attacks, are undergoing this, I would ask that they be explicit as possible in sharing openly what they're undergoing, if they have an understanding of what model was being used. To your explicit question of what would change my mind, I think it is a clear, clear data that the risks of open weights being leveraged and attacks outweighs the possibility

• 34:06Speaker 02

  for defense. And a lot of this is compute constrained. I don't think there is enough compute available to a non-nation state actor to do the level of large scale at this point, which could change in the near future. But in our level of urgency, the, the need for, for defense, for even if you've seen these European grocery stores, it might not be the highest priority for critical infrastructure, but for businesses to defend hundreds of thousands, if not millions in damages, seems important.

• 34:39Rebecca Bellan?

  Would you support some kind of a, a policy that restricted, I mean, if not restricting access to models, restricting access to compute?

• 34:46Speaker 02

  I mean, I think this is a, the kind of, we're playing catch-up here. Token efficiency and compute efficiency have made leaps and bounds in the last few years, particularly since DeepSeek R1. I don't think that's the right metric. The general guidance that we need is around what is the appropriate level of access control? Because I fear that we're going to start putting some level of access control on open weight models, and then nobody has access to anything.

• 35:12Rebecca Bellan?

  Yeah. Okay. You know, a lot of startups are building on open models, open source infra. We touched on this a bit last week when we had Wiz on. A decent amount of them don't have dedicated security teams. I just want to get your quick take on this before we close out. How can small startups protect themselves now? Like, what are they underestimating, perhaps, about the current threat landscape?

• 35:29Speaker 02

  It's understandable to, to not already be prepared with security teams, even having some level of familiarity with open resources. My team's working on putting together a plan in the next few weeks. I wish we had it out sooner, but in the meantime, there are really great cybersecurity startups, like Semgrap that have open, openly available documentation. So scale up on the resources available to you. This is the beauty of, of an open internet. And it's okay, it's okay to lean on what's easy for you, but it's on us and the open community to lower the barrier for folks to use cyber defense tools that they can rely on.

• 36:11Rebecca Bellan?

  Okay, great. Well, hey, I think with that, we are out of time. Where can our listeners find you, connect with your work online if they want to, you know, drop in, say hi, slide in your DMs?

• 36:19Speaker 02

  Ah, I'm not, I'm not as good with my DMs as I maybe was pre-chat GPT. You can, you can find us on, make a Hugging Face account. We're, we're all over Hugging Face. I'm most probably available by email, [email protected]. And keep following us on all your favorite social media platforms, at Hugging Face.

• 36:42Rebecca Bellan?

  Okay. Thank you again, Irene, for joining us on the show. It was great to have you. To our listeners, you can find me on Twitter and LinkedIn, and you can find Equity on Threads and Twitter at EquityPod. Talk to you next time. Equity is hosted by TechCrunch senior reporters and produced by Teresa Locan-Solo with editing by Kel. Subscribe on YouTube or wherever you get your podcasts and find out what's next at techcrunch.com/events. Thanks so much for listening and we'll talk to you next time.