What's really open about open-weight AI?

Everyone in AI is talking about open-weight models right now. Some see them as the ticket to faster, more efficient development; others see them as an economic threat; they're either a security risk or the only security solution, depending who you ask. The Verge's Robert Hart explains how open-weight models work, why they're suddenly at the center of discussions about AI safety, and whether the only way to stop a bad guy with AI is a good guy with AI. Further reading: The next Xbox could play every Xbox game ever made Microsoft is bringing Xbox 360 games to PC Apple briefly yanked Telegram from the App Store over CSAM violations These trackable batteries could help you locate a missing camera OpenAI’s rogue AI agent didn’t stop at hacking Hugging Face We’re running out of reasons to ignore AI safety Anthropic says Claude accidentally hacked real companies too China’s Alibaba takes another swipe at America’s AI supremacy OpenAI’s biggest threat may just be open AI Nvidia, Microsoft launch open AI security alliance — without OpenAI, Google, or Anthropic Subscribe to The Verge for unlimited access to theverge.com, subscriber-exclusive newsletters, and our ad-free podcast feed. We love hearing from you! Email your questions and thoughts to [email protected] or call us at 866-VERGE11. Learn more about your ad choices. Visit podcastchoices.com/adchoices

• 0:02Speaker 00

  Hello and welcome to The Vergecast, a flagship podcast of Kimmy K3. I'm your friend David Pierce, and today on the show, we're going to talk about AI models. We've been talking about AI models a lot on the show for the last few weeks in the wake of the OpenAI hack of hugging face and all of these new conversations about AI safety and AI deployment and open weight models and the race with China and how we think about and operate AI in general. It all feels like it's coming to a head in some way right now. So The Verge's Robert Hart is going to come on, and he's going to explain to us what's actually going on here. There's a lot of new vocabulary with these models and a lot of new questions about how they work and how they're made. He's going to make sense of all of it for us. I personally am very excited to have somebody finally make sense of all of it for me. We're going to get to that in just a second, but first, here's everything else happening on The Verge today.

• 0:48Speaker 01

  I'm Jake Kesternakis, and this is 90 Seconds on The Verge for August 4, 2026. Microsoft just started bringing original Xbox games to the PC the other week, and now it's planning to bring Xbox 360 games as well. My colleague Tom Warren has the scoop on a memo that Microsoft sent around to developers asking them to opt into the new program. Microsoft will handle all the emulation hurdles, even customer support. All developers have to do is approve their games for sale and set a price. Microsoft's argument is, why not do it? It's free cash. The rollout is supposed to begin next year. Next up, Apple abruptly pulled Telegram from the App Store last night before restoring it less than an hour later. Apple told various news outlets that it pulled the app due to the presence of CSAM. Then it restored the app after Telegram removed the content and banned the person who posted

• 1:33Speaker 01

  it. The whole incident is very odd, and it's actually the second time it's happened. Apple pulled Telegram in 2018 for the same reason, and again restored it within hours. After this latest incident, Telegram seems downright mad. Telegram spokesperson Remy Vaughn told us that Apple was wrong to pull the app down. Finally, my favorite gadget of the day, my colleague Andrew Lyshevsky spotted some new camera batteries from Falcam that have built-in support for Apple's Find My Network to help you track down missing gear. I love this. This seems so much more convenient than attaching an air tag to every camera you own. It's only available for Canon and Sony right now, but Nikon and Fuji are set to be in the works. They're a little pricey, though, at up to $70 apiece. You can read more at theverge.com. That's 90 Seconds of The Verge for August 4, 2026.

• 2:19Speaker 02

  When it comes to your business, you want cards and tools you can trust. Amex Corporate helps put you in control. Set custom spend parameters to help fit your business needs and quickly issue unlimited corporate cash back cards, all backed by best-in-class fraud protection and the powerful backing of American Express. Get started at go.amex slash Amex Corporate. Terms apply.

• 2:46Speaker 03

  Interrupted sleep, headaches, constant fatigue. For a lot of women, these aren't three separate issues. In perimenopause and menopause, they're often the same hormonal story, and you don't have to quietly push through. Midi can help, because your symptoms have answers. Visit joinmidi.com with code VOX right now to book your first visit today. That's joinmidi.com, code VOX to book your first visit. Joinmidi.com, code VOX. Insurance coverage varies. Check with your plan for coverage.

• 3:22Speaker 00

  All right, let's talk AI models. Joining me now, Verge reporter Robert Hart. Welcome to the show. First time on The Vergecast.

• 3:27Speaker 04

  Yeah, long time listener, first time joining, so.

• 3:30Speaker 00

  Very happy to have you here. And I've brought you here to do maybe the wonkiest thing we have done on this show in some time. We're going to do a little bit of analysis, a little bit of explainer, but we're in a really interesting and complicated moment with AI that is about technology, it's about politics, it's about feelings, and I want to just piece through it. You've done a really good job covering this on the site, and I want to just walk through some of what's going on right now. But I think we need to start with some fairly straightforward explanatory stuff, because I think a lot of very smart people are getting this stuff wrong, and I want to just talk through it a little bit. So just from a very basic level, help me understand what an open weight model is and why it matters as a concept right now.

• 4:14Speaker 04

  Yeah, so I mean, to get even more annoyingly wonky, it's perhaps important to start with what it's not. And I think a lot here is caught up in kind of open source, and that's what we know traditionally from software, where it's pretty free, it's distributed, you can change it, you can make money from it, as long as you also share it freely.

• 4:35Speaker 04

  Now, open weight models are not quite that open. And so wonky part number two is they are only really open on a thing called weights, which are kind of those numerical parameters, the sort of knobs and buttons that an AI has during training that kind of helps it tick, process information, all of that. And that's the only bit that's really made open here. So you can download it, you can build on that, you can use it, but it is not the same as, say, an open source where you can kind of reconstitute it from scratch.

• 5:04Speaker 00

  Right. Have you have you come up with a good metaphor for how to describe weights? It's a really, it's a really messy but really important term, and I think it's really hard to visualize. Like, open source, I actually think, would be very easy to understand, right? You would see all the training data, you would see how they trained it, you would see what the output looked like, you'd be able to sort of peel the thing apart and understand every piece that went into it. Like you said, open weight models are very deliberately not that. They show this one thing, and I have I have not come up with a great way to explain it to people who are not sort of deep in the weeds of AI technology. Have you have you come up with one or heard one that you like?

• 5:37Speaker 04

  Not really, but on the spot, I guess. I mean, I almost imagine, you know, and you kind of have a piece of wood and you run an electric current through it, and it makes that sort of forked pattern.

• 5:50Speaker 04

  I almost imagine the weights as the kind of resultant image that might come off, in that it kind of is something that is the result of something. You couldn't make it from scratch without replicating everything precisely, including the wood, but also the electric, even the weather. But it's still quite useful in terms of you can sort of see the paths, you can trace it, and it's not a perfect one by any means, but it's kind of a, I think, an idea of kind of what, yeah, what weights might be in this sense, in that it's the product of something, but it's can still be really useful if you can, like, build or modify or change.

• 6:25Speaker 00

  I like that. Like, you couldn't, you couldn't make it yourself without knowing exactly what went into it, but even just knowing what it looks like, you can now go and redraw the thing, which ends up being really important. Which brings me to the next thing I just want to sort of quickly explain before we get into this here, which is, who cares? What can I do if I'm a developer or a government or somebody paying for one of these models with an open weights model that I can't do with a closed weights model like we've seen from Anthropic and OpenAI? Like, what is the actual sort of user advantage of an open weights model?

• 7:01Speaker 04

  Quite a lot, really. I mean, it's a lot more flexible, you have a lot more freedom to do largely whatever you want with it. You're not reliant on sending them your data, which is a big thing for a lot of different companies. You could, presuming you have the infrastructure, you could run it yourself, you can tweak it with your own data, so you've kind of got a bit more of a personalized model. And I think important for a lot of the debates we're seeing now as well is that they're a lot less monitorable, and it's a lot harder for providers, say Anthropic, OpenAI, in these cases, to put in the safety rails that might stop it from doing something that you would want it to do.

• 7:41Speaker 00

  Okay. So maybe in that sense, the comparison to open source actually is useful in that the it doesn't function the same way, but your ability to take it and modify it and use it in your own server array, that is very much the same as if it were pure open source software. You don't get to understand the inner workings of the model in the same way, but you get to use it and adapt it in much the same way. Is that is that a reasonably close comparison there?

• 8:09Speaker 04

  Yeah, I mean, there are a lot more restrictions that they might put on it. So some of the ones that are coming out now, they'll have licenses that you have to pay them over a certain threshold, but yeah, it's a lot more flexible in that way.

• 8:20Speaker 00

  This is probably an overgeneralization, and I want you to correct the extent to which this is an overgeneralization, but it seems like China has embraced the idea of open and weight models in a big way, and the US, particularly the sort of frontier labs that we talk about all the time, Anthropic, OpenAI, Google, have not. A, is that a fair characterization? And B, why do you think it's broken down like that?

• 8:42Speaker 04

  To again get annoyingly wonky, it is a bit more complicated than that, but by and large, it is a reasonable characterization. The most prominent US models, kind of as you said, Google, OpenAI, Anthropic, their frontier models are all proprietary, they're all closed systems. And in China, by and large, a lot of them are open weight. There are exceptions, obviously. So Alibaba in China, for example, until its most recent release, a few earlier this year in its frontier scale were proprietary. It kept them closed and it evidently changed its mind this week. And in the US, there is a big, there is still a big ecosystem of open weight players. I mean, Meta is the most obvious that comes to mind,

• 9:27Speaker 04

  who've been making strides recently, but also Google. I mean, it's not the top tier Gemini ones, but their general models are quite popular. And there's sort of a huge spectrum in between that.

• 9:39Speaker 00

  If you grant me the huge generalization, because I think at some point, at the absolute frontier, I think that the debate we're having right now is between, you know, companies like Moonshot and like you said Alibaba in China, and the three major labs here in the US. Is it purely sort of a political difference that is why China would want its models to be open and the US would want them to be closed? Or are we talking about like communism versus capitalism here? Like, what, what, what do you make of the sort of philosophical difference between those things?

• 10:10Speaker 04

  I wouldn't go as far to say it is quite a deeply entrenched divide like that. But I think it is quite tempting to see it like that. I think partly it is business pragmatism in China. It is a way with all the restrictions that have been kind of put on them by, by the US. So they've not had access to the top tier chips, for example. It is harder for them to innovate at the frontier. This is potentially a way of them doing so, or of at least kind of working towards that. But also from a business strategy point of view, there's a huge sort of array of advantages with going open. I mean, it's cheaper by and large for developers to run. It's a much lower barrier to entry.

• 10:52Speaker 04

  There are also the elements we said with privacy or kind of running things on your own system. You can really grow that ecosystem rapidly. And it is kind of almost a projection of soft power if your stuff is becoming almost the default. And the Chinese models are being very widely used globally. American companies, Western companies would be very hesitant to send all of their data to China or to, to, to service hosted in China. If it's open, you can get around that. And so it's also quite a nice gateway for them to stay active in these markets.

• 11:28Speaker 00

  Yeah, that makes sense. So all of this is, is, has been sort of burbling in the background for a long time and has really come to the forefront of the whole AI discussion over the last couple of weeks. And it's happening right next to, I think we've talked about a bunch on this show, the, the OpenAI Hugging Face hack. Now Anthropic is coming out saying, oh, look, we just noticed we've also been hacking everybody. There is a real AI safety thing happening right now. And open weight models have become a big part of this discussion, kind of on both sides, in a strange way. Everybody has feelings about open weight models, no matter how they feel about AI safety.

• 12:09Speaker 00

  Why has open weight become sort of a core part of the, how do we make sure we're doing AI safely discussion? There are so many things this could be about, but it feels like the industry has decided that the debate we're going to have right now is about open weights. Why?

• 12:26Speaker 04

  Yeah, well, I mean, one, and I'll gloss over this just due to the complexity, but there is the whole spectrum of the race with China in the States or the West in general, which means something a bit different to quite literally anyone who mutters it, but that's.

• 12:43Speaker 00

  We have to beat China at everything. It's just a fact of life in America right now for reasons I continue to largely not understand. But yeah, granting that premise, what else is going on here?

• 12:51Speaker 04

  Yeah, so there's that, which obviously is, is one reason. So the, the open weight models they're releasing, let's go. But then the, the safety part as well. So the main opposition, or one of the oppositions to the open models is that, it's, it's very difficult to both monitor how they're used, which OpenAI and Anthropic, they know how those are being used. It's hard to implement details like guardrails to, to stop it by hacking or helping you build a bioweapon, the two kind of main concerns that the, that the frontier labs say. The fear has always been that releasing something very capable openly puts that in the hands of anyone.

• 13:32Speaker 00

  Okay.

• 13:33Speaker 04

  The weird part of this then is OpenAI, it turned out one of its agents hacked Hugging Face, as we know. The difficulty with closed models is, is these safety rails as well. Hugging Face said in their report, oh, well, we couldn't actually use US frontier models. These safety rails activated.

• 13:51Speaker 04

  And then they said they turned to one of the leading Chinese providers, ZAI, for their model to help defend itself against one of the closed models that was attacking it, which kind of flips a lot of that on its head. And so this debate has now really become, I mean, it is a, by definition, a dual use technology. It can be used to hack, it can also be used to defend against hackers.

• 14:12Speaker 04

  And so that's kind of been at the core of this now is that, for a long time, the fear has been it would be used as a weapon. And funnily enough, the most high profile case recently is it's been used as a tool of defending.

• 14:24Speaker 00

  Wait, so hold on. So let me, let me make sure I understand what happened here, because I actually think that, that's really interesting. I hadn't quite thought about it that way. So OpenAI, closed model, very, very deliberately hard for anyone else to understand,

• 14:39Speaker 00

  attacks Hugging Face. Hugging Face says, we, we need a tool to stop this. We're, we're going to deploy our agents to stop this agent. Runs into some safety restriction in the model that they're trying to use, that another closed model that is like, no, I won't, I won't escape this sandbox and go try to fight this thing. So then they're able to use an open weight model because they're able to just essentially remove those guardrails or they don't exist in the first place because it is open and adaptable in that way. Am I understanding that right?

• 15:11Speaker 04

  Yeah, in a very kind of, yeah, in a broad sense, that's.

• 15:14Speaker 00

  What a weird system.

• 15:17Speaker 04

  Yeah. And so that's, I think, why it has been so, as an issue, it has so rapidly bubbled to the surface. A thing that's been simmering for a long time is because all of these tensions that were there suddenly became very real and very tangible in a way that ties together, as I said, all those fears with China, but also open versus closed. And then also the risks of these closed models of what they are actually capable of doing.

• 15:47Speaker 00

  This strikes me as the sort of thing people are going to pretty quickly get, like borderline religious views about that are going to be very hard to change. Because what you just described is two completely reasonable and totally mutually exclusive theories, right? That one says, this technology is too powerful. We can't put it in the hands of everybody or the bad people will use it and things will go horribly wrong. And we're going to be stuck in this arms race, cat and mouse game for forever. The other side says, actually, the only, that's already happening. And the only way to stop it is to put this technology in the hands of everybody. There is no single overlap anywhere between those two things. And it feels like as these models get better and more capable,

• 16:31Speaker 00

  both sides are going to feel more right about their stance on these things. Like, is there a way to reconcile in the middle of this fight?

• 16:39Speaker 04

  Well, I think we'll see, but I think we have to. I mean, I think even the opponents have kind of, I mean, the one big one that has been to watch has been a lot of these kind of open letters you've seen from like the US tech industry, for example. The notable holdouts, at least at the start, were the big three, were Google, OpenAI, Anthropic. To my knowledge, it's only Anthropic that's a continued holdout on that. I might be wrong. They might have signed since. But Daria Amade is also published a quite lengthy blog post explaining the reasoning and says that we're not against open models, but we cannot only have open models. And I think that's probably

• 17:19Speaker 04

  an area that we will, we will end up in, I think. And it's also not to say, I think a lot of this is focused on, like I said, it was more complicated with, there are open models in the US. China could also very easily close off some of its frontier models.

• 17:36Speaker 04

  And I think probably as they get more and more advanced and as sort of that ecosystem develops, I think there'll probably be a mix.

• 17:42Speaker 00

  Yeah, because you, you have more economic incentive to close your models, right? You can make more, if you, if you can say and demonstrate, our model is the best, which I think Anthropic relatively successfully has been able to do over and over for the last year or so.

• 18:00Speaker 00

  Every possible upside you want comes from closing off that model, right? You can make it more expensive. You can limit who gets to access it. You get to be the, the arbiter of good and bad in that model. Like, if you are the best, there are lots of good reasons to be closed that almost no one other than the very best model seems to have. And this is where we get wrapped up again in the everything is a race against China. If, if in fact, you know, Kimi is now a better model, like demonstrably better model, and all of a sudden, everybody wants it. Suddenly you flip from China wants to have openness and to sort of infiltrate the US and make a lot of headway into businesses and start to capture some of these use cases that people can't afford from the greatest models. All of a sudden you say, well, we have the best model. We're going to close it off and make a ton of money from it.

• 18:45Speaker 00

  That does feel very possible to me at this moment.

• 18:48Speaker 04

  Yeah, I mean.

• 18:51Speaker 04

  It feels largely what's been happening already is the kind of end of that. I mean, and also there is a safety element. I mean, Anthropic was explicitly founded. Basically, it was unhappy with what OpenAI were doing.

• 19:03Speaker 00

  And I think the argument against Anthropic in many ways at this point is that Anthropic believes that it is the only one who should be trusted, right? Like, this is what the government has been saying about Anthropic for some time in the US is that this stance that Dario Amade is the only one who gets to decide what we do with AI and what we don't do with AI is ridiculous. And again, like, you're free to think whoever you want to be in charge should be in charge. But at some point, either no one is in charge and we just like let chaos reign because that is the thing that will solve this, or someone has to be in charge. And I feel like Anthropic has been the one most loudly being like, it's fine, the answer's us. We've got it. And that makes a lot of people really angry.

• 19:43Speaker 04

  Yeah, I mean, absolutely. It's, I think their response to the hugging face incident is actually quite telling. I mean, they, I mean, to me, it felt quite petty. That's maybe a controversial view, but it's like, hey, our models can hack things too. And then you look at kind of the details of it. And I mean, their blog detailing this, it quite literally ends in a four bullet point list as to why what happened with them was better than what happened with OpenAI, which, cool. I mean, we're all adults here. Great. It just felt very juvenile, especially when, I mean, OpenAI was, it hacked its way out. To me, Anthropic's was the equivalent. They kind of left the door open.

• 20:27Speaker 04

  Like, it was a very, like, they're saying we're the good guys and their behavior doesn't seem to meet that bar time and again. But they're also, though, I should say, them and OpenAI are doing more than a lot of other actors in this field as well.

• 20:41Speaker 00

  Yeah.

• 20:41Speaker 04

  It just doesn't necessarily meet the expectations they set for themselves.

• 20:48Speaker 03

  Interrupted sleep, headaches, constant fatigue. For a lot of women, these aren't three separate issues. In perimenopause and menopause, they're often the same hormonal story. And you don't have to quietly push through. Midi can help because your symptoms have answers. Visit joinmidi.com with code VOX right now to book your first visit today. That's joinmidi.com, code VOX to book your first visit. Joinmidi.com, code VOX. Insurance coverage varies. Check with your plan for coverage.

• 21:24Speaker 02

  Growing teams need a corporate card built for more, something that doesn't sacrifice speed for control. Introducing the American Express Corporate Cash Back Card. Get 1.5% cash back on eligible business purchases, advanced spending controls, and real-time visibility, plus intelligent expense management. Get started at go.amex slash amexcorporate. Terms and cap apply.

• 21:55Speaker 05

  This episode is brought to you by Michelob Ultra. Good times mean competitions between you and your friends, like seeing who can throw some pigskin the farthest or rush the field the fastest. Winner gets an ultra. Because it's not just a superior-like beer, it's also giving fans superior access to their favorite sporting events. Superior is worth playing for. Visit michelobultra.com/superioraccess to enter. Enjoy responsibly. Copyright 2026 Anheuser-Busch InBev Michelob Ultra Beer, St. Louis, Missouri.

• 22:23Speaker 06

  Are your ad campaigns lighting up the dashboard? But not the pipeline? That's bullspend, and marketers are calling it out in. Dashboard Confessions.

• 22:32Speaker 07

  My boss asked for results, so I opened my dashboard for the only positive sounding metric I had: impressions.

• 22:40Speaker 06

  Cut the bullspend. See revenue, not just reach. LinkedIn delivers the highest return on ad spend of major ad networks. Advertise on LinkedIn. Spend $250 on your first campaign and get a $250 credit. Go to LinkedIn.com/campaign. Terms and conditions apply.

• 22:56Speaker 00

  It feels like right now is going to be, or at least should be, an inflection point in a lot of these conversations, right? You have Jensen Huang, the CEO of Nvidia, writing the open letter about OpenWeights. You have, like you said, the, this new OpenWeights Alliance. Everybody is getting in fights about China and whether it is winning and whether it's not and whether it matters. We are still very much in the throes of this, what do we make of the OpenAI hugging face hack? You wrote last week that it's time to basically stop ignoring AI safety questions and we have to start doing real things about this. Sam Altman is out there being like, maybe we need to pause. Everybody's begging for regulation. It feels like either this is going to be a very noisy moment that everybody just forgets

• 23:39Speaker 00

  because somebody will launch a new model and we'll all move on with our lives, or there's going to be some sort of big structural change in how we talk about AI, like, as an industry and ultimately as a society.

• 23:55Speaker 00

  After writing the piece that it's time to stop ignoring AI safety, what's your read of sort of the temperature of this conversation right now?

• 24:03Speaker 04

  It's hard to know how to think, I think. Like, I think for a lot of people in this space, none of this is surprising at all. Like, I mean, I said for mine, like, it's disappointing from my point of view that something like such a basic error might happen, but it's not surprising. We know that these tools can do this. And I think what it does do is it's, and this is something that has been warned about, at least in theory, for at least a decade, I'd say, if not a lot longer. The problem is, I think, it's, it was lacking a really tangible example, which obviously, as we know, in policy, politics, anything, that is needed. And I think this is maybe a galvanizing kind of thing. It's kind of forming a crucible of some sorts. We've seen some of the industry come together.

• 24:48Speaker 04

  We've seen rumblings at the White House. I'll leave my thoughts aside on what a voluntary code might achieve, but, but then there's sort of other elements as well. Like, I mean, I think it was just yesterday, a group of attorneys general were kind of pressuring OpenAI to preserve evidence on this. So it's possible that there might be some renewed push for actually meaningful regulation here in the US. I think it has galvanized the industry at least in that I think it's spooked people who are working, like, inside these companies.

• 25:21Speaker 00

  Interesting. That's actually exactly what I was about to ask you about because I think the nihilist and also maybe most rational take at this particular moment is to assume that no regulation is coming and that every government will just sort of twist themselves in knots and not make a lot of progress, particularly in the United States. But this also seems like the sort of thing that would make most industries take a really hard look at themselves and say, okay, something has gone awry here. And, and whether it's the people at the top of the industry or the people inside of these companies, look around and say, okay, we actually have a big problem here. And if somebody else isn't going to force us to fix this, we actually have to fix it anyway because it will eventually be bad for business, it will be bad for the world.

• 26:04Speaker 00

  Like, if you sign up to use OpenAI models, we might hack your systems is like not a good business pitch. And so I wonder, is, are, is this what you're hearing? People are starting to think like, oh, we, we need to do something even if no one's going to make us do it?

• 26:19Speaker 04

  I mean, yeah, I mean, not even, you don't even need to sign up. They'll just hack anyone if they go.

• 26:22Speaker 00

  That's true. They'll just do it anyway.

• 26:24Speaker 04

  They don't even need to be a customer. Anyone is game at this point. And the fact that they don't know as well, I think, is what has perhaps caused more alarm. Like, it was only upon review that, say, Anthropic were like, oh, this happened in April, three times.

• 26:38Speaker 00

  The funniest meme I've seen on the internet recently is just a picture of Mark Zuckerberg on the phone and the caption is just him screaming into the phone, "Go find something illegal we did."

• 26:49Speaker 00

  Really enjoyed that, but sorry, keep going.

• 26:51Speaker 04

  Yeah, no, I mean, but I think that is, I think for a lot, it has, I mean, a lot of these people do care quite deeply about what they do and they think that what they're doing is quite impactful, possibly in like a very major way. And a lot of them are concerned about the broad safety impact of this. And from people I've spoken to, that it kind of ranges from, some people seem very despondent. They're like, wow, we're not doing anything now. We're not going to do anything for the next red line or the one after that. Let's just hope at some point we get our act together before it's too late. And so there's that kind of really diabet, but I think others, it's more.

• 27:27Speaker 04

  Okay, well, maybe now is the time to push because evidently we've not been living up to the bar we've set for ourselves. I mean, the thing to remember with all of these hacks, they were quite nice as far as they go. Like, as far as I'm aware, no one died, no huge amount of money was lost, no one was hurt. It could have been a lot worse as far as like a rogue hack goes.

• 27:48Speaker 00

  Yeah, in a funny way, it's, one question I was gonna ask you as you were talking is like, is it, is it in some ways a shame that this is such a deeply unsexy example? Like, it's, it's a company no one has ever heard of that does something no one understands. It was, it was about a relatively low stakes that like, every single thing that happened is scary, but it was all in service of something so mundane that I think it was, it's much easier to write off than if this had happened, you know, to something people have much more real sort of visceral feelings about, even though the actual operation of the thing could have been exactly the same.

• 28:27Speaker 04

  Yeah, I mean, I think the fear for a lot of people in this space might be that like, what does it take to wake up for this? Because as I said, this is something people have been warning about for a very long time. And even though politicians have sort of, you know, here and there paid attention, even with legislation and whatever, like, nothing has really concretely happened that has stopped any of this behavior. And I think the fear for a lot embedded within that ecosystem would be like, well, yeah, what does it take? Are we looking like hacking a hospital? Are we looking at some Chernobyl-type incident? Like, at what point is it gonna be enough that we can kind of sit up, pay attention, do something about it? And then also, it's not so bad that we cannot then contain it?

• 29:12Speaker 00

  Yeah.

• 29:12Speaker 04

  So yeah, I think there's a lot of kind of unease, I think is how I'd describe it, is opinions vary, but unease is the sentiment I get.

• 29:21Speaker 00

  So you, you mentioned your feelings about a voluntary code, but we should mention, potentially, as people are listening to this or watching this, there's a meeting happening at the White House with some of the big AI companies to talk about the way that models get reviewed, to talk about AI safety. And one of the things that they're being asked to do reportedly is essentially voluntary submission for a review of models. I would say, I also have my own suspicions about how real a thing this is, but what do you make in general of this idea of this kind of government review of, of models before they're made available to the public? Is this a possible short solution to this?

• 30:01Speaker 04

  I mean, it depends at what level. So I think that's the thing that the transparency issue that's kind of been really shown by this is that some of the models involved, both with Anthropic and OpenAI, were not public. They were research prototypes or being tested. So it isn't like, at what point are we then kind of interjecting the government into this or, or some form of auditor?

• 30:21Speaker 00

  Right.

• 30:22Speaker 04

  But like, it does show that, yeah, this can happen really early on in that lifecycle. And so unless there is basically sort of a glass house-type transparency, which these companies will obviously bristle at, how do you really enforce that? I mean, the, the alternative is, well, we take their word for it, which, again, I am skeptical naturally on, but.

• 30:43Speaker 00

  Yeah, I mean, it's such an interesting point because I think just in the OpenAI hugging face example, not only was it a research model that wasn't ready to be shipped yet, it was also supposed to be just in testing, right? This was an experiment. So it's not like it was in some kind of early rollout to people. It was supposedly sandboxed, right? Like, the whole idea was that this thing was, was as protected and walled off as it could be. And I've, I'm sure you've seen this too, a lot of researchers are out there now being like, have you ever heard of air gapping? People like, do you, and it's, but it's like, okay, what we're actually discovering is that a big part of the problem is that all of this stuff is available to these models

• 31:25Speaker 00

  and is, is possible for these models from incredibly early on in their development. So at what point do we even consider a model finished enough to be reviewed? Strikes me as, I think you're right, like, maybe a more or less impossible question. And there's no chance these companies are gonna be like, sure, we'd love to have a government team sitting in our offices all day just to make sure we're doing a good job. Like, that ends up being a total non-starter here.

• 31:52Speaker 04

  Yeah, absolutely. And I mean, for me, the kind of part where it comes to, at what point do we consider this model, I suppose, coming under this? When you consider testing it? Like, feels a very clear answer. Like, if it's good enough to be tested and you cannot guarantee its containment, then, I mean, also, build better sandboxes. I really don't understand. That's why it just feels so disappointingly basic, like, air gap things.

• 32:16Speaker 00

  Yeah.

• 32:16Speaker 04

  This doesn't, this feels like negligence sometimes more than, more than a mistake, but.

• 32:22Speaker 00

  Yeah, agreed. So, all right, last thing, and I'm gonna let you go. Do you, do you feel the temperature of this continuing to rise as we go right now? We're what? We're sort of two weeks into the, the panic about the OpenAI hugging face thing. We've been talking about this for some time now. You and I have been at this a while. This feels like the sort of time everyone will find something else shiny to talk about and move on. Do you sense that happening or are we, are we still deep in this?

• 32:52Speaker 04

  On one sense, I do, I do feel it kind of slipping away a little bit. On the other hand, I also feel it merging with everything else that's happening, which I suppose is why I've spoken so loosely about the safety issues, about OpenWeights, about China, because I feel these are now all fast becoming very much the same discussion. Because if you start talking about a slowdown, okay, cool, what about China? What would be the natural question for that? And so I also think there's other elements in the safety sphere that are emerging as well. So say there was that, the, the sort of employees calling for, what was it called? Pacing frontier development or something? A peculiar phrase. And that was largely premised around the idea of self-improvement rather than these other developments.

• 33:36Speaker 04

  So I think it will all kind of fold into one. As for whether that will actually do anything?

• 33:43Speaker 00

  My sense is that's a bad sign, that actually, in fact, one of the problems we've had with AI for a long time is that every conversation has been about everything and we have done a bad job peeling apart all the things that are AI. And so in fact, if now we're going to shove all this stuff back together into one, what do we do about AI discussion? That's only going to make all of this more complicated for everybody. Not to end on a real bummer of a note, but that is, that is where it feels like this might be headed. If, if we try to make this one robust AI safety discussion instead of actually peeling apart, like, this is why I think OpenWeights is interesting. I think we should have a big, complicated discussion about how open most models should be and at what, in what way and available to whom.

• 34:27Speaker 00

  And that should be completely separate from questions about China. And the minute all of this got tied up in China, it struck me as infinitely harder to actually do something about.

• 34:37Speaker 04

  Yeah. It is a bummer, as an end note. Regulation is tough. And yeah, I guess what all of this has illustrated is that, like, self-regulation is, as with many industries, woefully inadequate. And at what thing do we need to happen for someone with power to stop that, to actually intervene?

• 35:00Speaker 00

  It's a good question. We will, we will keep coming back to it. But for now, Robert, thank you. Good to have you on the show. Come back sometime.

• 35:05Speaker 04

  Yeah, it's been great.

• 35:07Speaker 00

  All right, good to see you. All right, that's it for the show. Thank you to Robert for being here and thank you, as always, for watching and listening. If you have thoughts, questions, feelings, feedback, if you have a really good metaphor for OpenWeights models, I liked Robert's of the, the fire through wood, but there's bound to be better ways to explain this than just talking about weights and models. So if you have a great metaphor, I want to hear it. Send us an email, [email protected], call the hotline 866-VERGE11. We love hearing from you about all that and everything else. And as a reminder, the best thing you can do to support everything we're up to here is to subscribe to The Verge, verge.com/subscribe. It gets you all of our podcasts ad-free, including this one. It gets you all of our exclusive newsletters. It gets you all of our coverage.

• 35:48Speaker 00

  Robert has been doing a really terrific job of covering the US AI, OpenAI hugging face stuff. Go read like the last three weeks of his stories and you'll be immediately smarter on all of this. Theverge.com/subscribe. Thank you in advance. The Vergecast is a Verge production and part of the Vox Media Podcast Network. This episode is produced by Josh Kahas, Eric Gomez, Brandon Kiefer, and Travis Starshaw. We'll see you tomorrow. Rock and roll.

• 36:14Speaker 03

  Interrupted sleep, headaches, constant fatigue. For a lot of women, these aren't three separate issues. In perimenopause and menopause, they're often the same hormonal story. And you don't have to quietly push through. Midi can help. Because your symptoms have answers. Visit joinmidi.com with code VOX right now to book your first visit today. That's joinmidi.com, code VOX to book your first visit. Joinmidi.com, code VOX. Insurance coverage varies. Check with your plan for coverage.

• 36:50Speaker 08

  Fall savings are happening now at The Home Depot. Save up to $500 on select appliances. Plus, enjoy free delivery when you spend $998 or more. From prepping to cooking to cleaning up, Frigidaire has you covered. The custom Flex Temp drawer switches from fridge to freezer. With more than 15 cooking modes, including stone baked pizza mode, you can enjoy restaurant-quality pizza in as little as two minutes. Then clean up with the Frigidaire dishwasher, featuring the industry's fastest normal wash cycle. Shop fall savings now at The Home Depot. Off and available through October 21st. U.S. only. See store online for details.