SED News: Jev, Meta’s Muse, and the AI Slowdown Paradox

SED News is a monthly podcast from Software Engineering Daily where hosts Gregor Vand and Sean Falconer break down the biggest stories shaping software engineering, Silicon Valley, and the broader tech industry. In this episode, Gregor and Sean cover a wave of new product drops, including Jev from TypeSafe AI https://typesafe.ai/, a model that returns scored, type-safe structured responses for machine-to-machine tasks like routing and classification rather than human-style chat. They also dig into Meta https://ai.meta.com/‘s Muse, a personal AI assistant with a companion keychain device, and the upcoming IPO of Nscale https://www.nscale.com/, a former crypto-mining outfit turned “neocloud” inference provider whose filing leans heavily on a single unnamed customer. The main topic examines the tension around AI risk, frontier lab slowdowns, and the conspiracy theories they’ve spawned. Gregor and Sean unpack Dario Amodei’s essay on pacing AI development, the awkward timing of major model launches days later, and whether calls to slow down are genuine safety concerns or a way for incumbents to protect their market position, weighing it all against the long history of how society has absorbed transformative technologies. As always, the episode wraps up with a few standout Hacker News threads, including a labor-of-love effort to restore the original cut of a classic film, and a vibe-coded local AWS testing harness that raised a few eyebrows in the comments. Sponsorship inquiries: [email protected] mailto:[email protected] The post SED News: Jev, Meta’s Muse, and the AI Slowdown Paradox https://softwareengineeringdaily.com/podcasts/sed-news-jev-metas-muse-and-the-ai-slowdown-paradox/ appeared first on Software Engineering Daily https://softwareengineeringdaily.com.

• 0:11Speaker 00

  Hello and welcome to SED News. This is the monthly installment of SE Daily with a slight twist, which is that we go through the main tech headlines. We then dive into a deeper topic in the middle and then we take a spin around our favorite Hacker News things of the month in Hacker News highlights. As usual, it's myself, Gregor Vand, and with me is Sean Falconer.

• 0:34Speaker 01

  Hey Gregor, hey everyone out there.

• 0:36Speaker 00

  Yeah, so I'm gonna be seeing you soon, Sean, I think. I'm coming in, I'm actually heading off on a flight after this to SF, going to our Superbase conference on Friday, but I think you're gonna come along for some of that.

• 0:50Speaker 01

  Yes, I will be there, at least for some of the day on Friday, and I guess you'll take off and then like, I don't know, 30 hours later, you'll be in San Francisco or something.

• 0:59Speaker 00

  Yeah, it's one of the funny ones where you leave on a Tuesday afternoon and arrive on a Tuesday evening, even though you just spent close to 30 hours traveling. So yeah, it's gonna be a good week ahead. Obviously, that's sort of a lot of my week looking forwards is, yeah, you know, going to SF. We've got, given my role at Superbase, I also have a lot of people to meet, all the main companies we talk to, people like Versal and OpenAI. And yeah, it's gonna be fun on the Friday. We've got Steve Wojsniak as one of the, as like the headline speaker, so that's a bit of a pinch me moment, get to be in a room with Steve Wojsniak. But yeah, what about yourself, Sean? What have you been up to?

• 1:36Speaker 01

  I mean, we luckily, no travel or no work travel for me for a little bit. I do have a personal trip, doing a pilgrimage to the Great White North to visit my, my family here for Canadian Thanksgiving in a couple of weeks. Primarily just heads down on kind of Q4 2027 planning. We also have our annual conference coming up in November, so we're all kind of busy working on that to pull all those things together for, you know, launches and big announcements that we need to make. So, you know, people listening that are interested, you know, stay tuned for all that stuff.

• 2:06Speaker 00

  Yeah, nice. But yeah, otherwise on, yeah, on my side, yeah, the pollution in Singapore has been a bit crazy actually lately. It's basically, there's a tech angle to this in a second. Yeah, so it's basically a lot of haze from forest fires come from Indonesia, passes over Singapore, but unfortunately, yeah, it's been sticking around for quite a while. But it did prompt me to buy an air purifier, and I'm sure some people have them, but I'm a bit late to the party maybe on that one. But yeah, super happy. I bought a Dyson air purifier. I think it's just amazing. The tech is like, I don't know, it looks cool, the app is really nice. Yeah, I don't know, it's one of these like, new tech moments. I didn't expect to be quite so attached to an air purifier, but I really do love it.

• 2:49Speaker 00

  My, my wife's always just like, go, just move it around, whichever, whichever room you want that thing in, it's yours. So yeah, I don't know, do you have one in SF or like, probably not needed, right?

• 3:01Speaker 01

  I don't think so. No, I mean, I probably would have came across it, but it does feel like something that my wife would invest in.

• 3:07Speaker 00

  Yeah, I think some people get them, especially when they have like, early infant kids, they sort of get a small one to stick in the, in the kids' room. But no, certainly first time for me. But yeah, I love, because of the app, you can look at your air quality indoors from, I'm in the office and I'm checking, and when it gets a bit high, I just stick it on. So, things that you can get excited about when there's pollution.

• 3:28Speaker 01

  Yes. I mean, we do get fire, you know, there's fires usually in Northern California that impact the air quality. So maybe this is something that we, you can send me your tips for air purification.

• 3:39Speaker 00

  Cool. Yeah, I shall do that. Why say cool? I mean, as long as like, yeah.

• 3:43Speaker 01

  Yeah, awesome.

• 3:45Speaker 00

  Awesome, yeah, fires, awesome. But yeah, at least there is tech out there to, to help with this. So yeah, I guess like, diving into the headlines, we've seen a couple of kind of like, large product drops, if you want to call that, in the last week or so. First one is Jev. So this is from a company, TypeSafe AI, company that's actually founded in 2024, but I mean, I don't think many people really knew what they were up to until about September 15th, when basically they released Jev, which is this, they pitch it as effectively a model where you don't, the outputs are not chats, they are type-safed responses with

• 4:28Speaker 00

  a score against them as to like, how accurate it thinks that structured responses. And the whole point is this thing is not meant to be really used by a human, it's meant to be sort of layer one, they call it like, system one layer for like, structured decisions like tool calls on routing and classification. But the big thing here has made such a huge splash, if you wanna, I don't know what other word to use, in the tech world. And it's kind of being like touted as like, is this the next revolution on LLMs? It was founded by ex-ChatGPT engineers amongst others. So it's, it's got some pretty sort of big heft behind it in terms of who's behind it. But yeah, there isn't, on the counter, there

• 5:10Speaker 00

  isn't like a lot that we can really dig into though, in terms of open weights or anything like that. Is that right, Sean?

• 5:17Speaker 01

  Yeah, I mean, there is, there has been recently, there is OpenJev, which is now in hugging face, which is a full-size 27 billion open weight alternative with near parity, came out two weeks essentially since the launch. But that was something that, I think that was a distilled model to kind of mimic the Jev experience. It's not official, I don't think it's officially from the owners or the creators of Jev. But it is really interesting, you know, what they've done, it makes a ton of sense, because what's happened with large language models is that people use them for a lot of times, stuff that they're not necessarily this most efficient model to do, like classification, even like choosing a tool and things like that. Like, these are more structured responses. And you have

• 6:02Speaker 01

  to do a lot of work to kind of force them into this behavior of accepting structure and generating structure. And it's generally, like, a lot of times, it's not that reliable. And it's inefficient in terms of token costs, inputs and outputs and things like that. So to have a model that's using the same kind of transformer approach, technology and general purpose, but specific for solving this kind of more structured things where you expect a structured response around classification, solves, is like a much better way of essentially solving that problem than trying to jam that into something that's accepting large language models. Because those were really designed to kind of function as a human might function, in human modes of conversation. But those are very inefficient for a lot of these types of tasks, whereas Jev is really built for like, machine-to-machine

• 6:47Speaker 01

  communication, where we're kind of trying to jam an LLM into, but it's just not a great use of tokens. It's expensive and it's also costly in terms of just the time to compute and the inference cost. So if you focus on that, you make a much more efficient model, both on token costs and then also the time essentially that it takes to process it. So makes a ton of sense. And I think that's why people are excited about it. It fit this gap in the market essentially between what you might do with traditional purpose-built models for things like classification or routing, with the nice thing of the generality essentially that you get from the large language models and the transformer technology.

• 7:26Speaker 00

  Yeah, I mean, I think it's, you know, it's just one of these huge like step changes, which is kind of nice to see, where we've spent, I don't know how many months, talking about the next frontier model or open weight model, but they're all kind of trying to achieve roughly the same thing, if you want to broad brush that. Whereas Jev has come along, knowing that there is, there is like another way to kind of approach this. I like the fact that, yeah, Jev does, name does come from Jevon's paradox. So you know, that that idea that the more efficient something becomes, that the more we actually consume it, and then the efficiency is kind of lost completely. I.e. like, light bulbs, you know, that we, we have these like, super efficient light bulbs now, but we end up just leaving them on, or have, I have like 10 light bulbs, where we used to have one, but.

• 8:11Speaker 01

  Yeah, it's like, when was the last time you turned off your phone or your computer or anything like that, right? You just leave these things on perpetually, basically.

• 8:18Speaker 00

  Exactly. So yeah, I like the naming of that. So yeah, we might actually have an episode coming up with the founder, so that would be exciting if in a couple of weeks, hopefully that's gonna happen. So look out for that on our more regular programming.

• 8:31Speaker 01

  Yeah, and I think, I think so far, some of the pushback that you see is some of the benchmarks that they've published are, they're kind of all self-published, there's no paper calibration, and we, and no sort of independent studies of these yet. I don't know, I mean, of course, you would expect that, it's like they just launched it. So, but I think what's really interesting too is how people have gotten creative with it. Like, you see people using it to solve like games and things like that, where it is like, you have a player in a game, you think of like Mario or something like that. It's like, well, Mario is a fixed set of things that he can do at any given moment, let's use a model essentially to figure out like, what is the next thing to do? Do I jump? Do I duck? You know, do I shoot a fireball or whatever it might be? So people are using

• 9:14Speaker 01

  those, and that's basically a routing problem. And we've had systems for solving those problems for years, but now you have this like really dynamic system that can do that for, you don't have to go train a net new model essentially every time that you want to apply it like that.

• 9:27Speaker 00

  Yeah, it's, yeah, funny, you mentioned Mario. I saw you can watch Jev playing Pokémon in real time. Yeah, there's a, like a website where it's like live, live playing Pokémon.

• 9:37Speaker 01

  Yeah, I did find a GitHub repo of someone doing this for Doom as well, just to be on team.

• 9:45Speaker 00

  I like, I like the, yeah, I've seen Pokemon at least is like, okay, that, that at least looks creative and different.

• 9:51Speaker 01

  Yeah.

• 9:51Speaker 00

  But yeah.

• 9:52Speaker 01

  I mean, I think the interesting thing is like, if you look at a lot of like compound AI systems today for doing things like model routing or other types of decisions, a lot of times they're pairing a traditional ML classification model with a large language model. Or sometimes they're doing what I said, where they use a inexpensive large language model to force it into being like a classification model. Basically Jeb is like the perfect solution to that kind of problem solving.

• 10:19Speaker 00

  Yeah. So yeah, I mean, definitely, well, I say, give it a try. It is, it is difficult to get on the, it has a waitlist, I think at the moment, or?

• 10:27Speaker 01

  I know a bunch of my colleagues have been playing with it, so I don't know how hard it is to actually get off the waitlist and try it out. It sounds like, I think they cleared a ton of people too. If you look at the, the data, it's like, I forget the exact numbers, but they cleared, uh, you know, tens of thousands of people off the waitlist and within a very short period of time.

• 10:44Speaker 00

  Nice. Okay. Yeah. I saw some of my colleagues talking about it and it looked like some hadn't managed to get in. Whereas one that got sort of early access, so long as they had a credit card on it, then they could like invite people to their org or something like that. So. But yeah, very interested to see how this shakes out and I would love to play around with it. I don't have access yet, even though some of my colleagues do. So maybe I will, on the episode, if we record it, I will hopefully ask for some access as well. So yeah, moving on. Another big product drop was Meta's Muse. So quite a different product in the sense that it's not aimed at developers per se. It's much more in keeping with, I guess, AI products of late, which is, you know, a personal assistant.

• 11:27Speaker 00

  We've kind of covered this area in many SED news episodes, but it's the fact that Muse actually seems to be solving the problems fast and well, I think is the, is the best way to put it. So they've launched this. It's an app, you know, you can download it on Mac and on, on devices, but they are actually going to be also releasing hardware to go along with this called Amuse Charm, which is a sort of like keychain device with a little screen. Think of like a sort of Tamagotchi looking thing, but like, I guess you'll just be able to talk to that as well. And I suppose it's, you know, I reckon they're kind of cruising on the whole Labubu.

• 12:08Speaker 00

  I don't know if people have heard of this, but yeah, Labubu in, in Asia and stuff, these little charms that people put on their bags and it's like ridiculously popular and I suspect they're playing off, because they look very similar to be frank. So I'm thinking.

• 12:21Speaker 01

  My kids have some.

• 12:22Speaker 00

  Yeah. Okay. So, so the inspiration has probably come from something along those lines, but the idea that you can show people that you effectively use Muse and you can kind of give it a name, like even, even if you're just using it in the app itself, I think you can give your Muse, because I mean, you know, Muse, the noun, you can give your Muse a name and so on. And yeah, I mean, I, I guess I've been reading and hearing people talking about different examples. Again, I can't access it right now because in Singapore it's on a waitlist. So I look forward to flying to the US in thirty hours and hopefully trying to get it onto a device. But what I've, I at least picked up is that it is able to solve problems such as, you know, booking a restaurant ten times as fast as Claude and, or say ten times as fast as it actually completes it.

• 13:07Speaker 00

  Whereas something like a Claude gets pretty far and then it sort of says, oh, sorry, I couldn't actually book your restaurant for you for some reason. But yeah, what's your take on this one, Sean?

• 13:18Speaker 01

  Yeah. I mean, I definitely, as we've covered before, like, I think this AI assistant market is going to be like a huge, huge market eventually. And I think like Open Claw was kind of the first real like breakthrough where people realize that, oh, maybe there's a more comfort level with AI taking actions on my behalf than we thought we perpetually realized and it kind of shotgun this. So I totally understand the investment. I think like where Meta is going to potentially run into challenges and maybe they'll overcome this is just that because Meta is so strongly associated with Facebook and the history there in terms of

• 13:58Speaker 01

  using people's data for advertising and stuff. And then you have this device that potentially you're going to use on your behalf to manage part of your life. Like, how much am I willing to kind of trust my personal information with that company and does it line up to ads and so forth? You know? Supposedly all this runs within a secure VM that's supposed to create a boundary between your Muse system and these other systems, but just kind of looking at the history of some of the moves that Meta has made, like I can understand some level of distrust there from the consumer market as well.

• 14:31Speaker 00

  Yeah. It's interesting. Last month, we, we talked about company instinct and this was kind of their play as well. And just as we're talking, I just thought I'd just go back to their site because I just thought, well, hang on, we were talking about instinct last month. And Muse is literally in this space and what are they doing at all to sort of recognize this? Interestingly, if anyone listened to last month, we talked about Instinct's website, which looked like it was from nineteen ninety-two and had, you know, Times New Roman. It was trying to be all kind of Back to the Future kind of thing.

• 15:01Speaker 01

  Right. Yeah.

• 15:02Speaker 00

  They've actually updated the site. It now is not Times New Roman. It is a sans serif font. I, more modern, it has a little logo now, a little stick man logo thing. And interestingly, at the bottom, it just says text instinct to get started. And I'm curious, I haven't used Wayback Machine or anything, but I'm curious if that was a bit of a before it was a very like, oh, you can't really get access unless you're special. So I wonder if that's been prompted, but anyway, I'm side barring here, but I just thought it was kind of interesting, given that there's this like just a kind of explosion in the personal assistant space. I mean, I'm, yeah, I, I think it's super interesting, like the fact that they actually even talk about the architecture here, like the, you know, the Muse secure VM

• 15:42Speaker 00

  architecture. I think that's obviously something that people are quite concerned about with any on-device. And especially in this case where this thing is only valuable if you give it access to all the files. Or I say all the files, but files and especially data that you have, like your, your calendar, which would often be through some cloud service. And people I think are, you know, the, I think the sort of paradox here is people probably have given a lot of data to Meta already. And it's not so easy. You pull out the ecosystem altogether or you just lean into it and say, well, you probably already have a lot of data on me, whether it's from having been on Facebook for a while and then Instagram. And I don't know exactly what they would pull from WhatsApp, but at least WhatsApp is

• 16:27Speaker 00

  I imagine able to come into this in some way as well. So yeah, it's a very smart play from Meta, I guess, especially given again, through the last few months, you and I, Sean, we've not been able to really pinpoint where Meta was going exactly.

• 16:40Speaker 01

  It also feels aligned with where Meta has been successful as a company. Like this is much more of a B to C AI play than going after agentic coding harnesses or something like that, or even going after building models. Like I think this aligns with where they've been successful with like Facebook and Instagram and these types of bets. So the personal assistant could be the, essentially the killer consumer app for AI beyond, you know, where we started with things like ChatGPT.

• 17:12Speaker 00

  Yeah. Have you used it or would you use it, do you think, or?

• 17:15Speaker 01

  I don't think I would. The hardware part of it doesn't really interest me. Like I, I don't really like the idea of, like even like Meta glasses and stuff like that. I don't want to advertise my use of of, of things that could be perceived as like monitoring other people and things like that. I just feel like there's a certain uncomfortableness that, that comes with that. And I don't want to, it makes me uncomfortable. I don't want to make kind of people uncomfortable with it. So that part of it doesn't interest me. I mean, the idea of a personal assistant, AI personal assistant certainly would be great. I just don't know if this is the one that I would, you know, personally put my faith in. I don't know if I'm ready to like give it my banking information.

• 17:52Speaker 00

  I agree. I'm, I'm curious to try it. I mean, I think for some of these more.

• 17:56Speaker 01

  I guess like hotel bookings. I mean, there, like clearly it needs some way of paying for the, those things. So I'm sure there's some credit card transaction part of it.

• 18:05Speaker 00

  Yeah. I think that's going to be something, I mean, I'm aware, like obviously Stripe have been on this for a while, but I think they're finally getting the recognition that they want on like machines paying for things. They've got this machine. It's called MPP machine, machine payment protocol. But I think people have maybe been struggling to kind of really utilize it because you need a machine that you trust to go and do the payment in the first place. So yeah, perhaps something like Muse is, is kind of the first step to that where it's at least an interface that people are going to use more. And yeah, I'm curious how much I would use it like day to day, but I am, as a novelty, I'm definitely curious just to give it a try. Like book me a restaurant in SF or something.

• 18:48Speaker 00

  But yeah, let's come back next month if I've managed to get access and we'll, we'll see if anything vaguely useful came out of trying, trying Muse. But yeah, very interesting.

• 19:00Speaker 02

  Your customer lives in the real world and it's messy. Intermittent connections, mid-onboarding drop-offs, edge cases on devices you've never tested. Mobile apps reflect reality in a way no other surface does. Yet, from an engineering perspective, they're the hardest to understand. It's common for mobile engineers to see green backend dashboards, normal error rates, no crashes. But inevitably, somewhere, there's a frustrated user watching your app spin. After a few seconds, they'll lose patience, close the app, and turn their attention somewhere else. They may never come back. The worst part? Most observability tools never see any of it. Bitdrift, on the other hand, captures 100% of mobile data, unsampled and in real time. So it's immediately queryable by engineers and AI agents. It's mobile observability built for the real world.

• 19:45Speaker 02

  Try Bitdrift today. Bitdrift.io/SignUp.

• 19:50Speaker 03

  Sponsored by Deepgram, the leader in voice AI, so you can use voices like mine.

• 19:55Speaker 02

  This episode of Software Engineering Daily is brought to you by Varia Code. If you're an engineering or IT leader, you know how painful this is. An open rec sits unfilled for three months. Your tech lead is burning out, covering the gaps, and your product roadmap gets pushed back again. Varia Code solves that. For over 10 years, they've been embedding senior Latin American engineers directly into US engineering teams in under 10 days at up to 40% less than local hiring. We're not talking about generic contractor pools. These are pre-vetted, high-quality engineers with real on-call and SLA experience. They work directly in your Slack, Jira, and daily stand-ups in your exact US time zone with zero friction. If you're ready to stop losing sprints to open recs,

• 20:38Speaker 02

  head over to variacode.com/sed right now and get a custom shortlist of senior candidates in 48 hours. That's V A R I A C O D E dot com slash S E D.

• 20:52Speaker 00

  Finally, on the headlines. So we did talk about, now the third time running, we're going to be talking about N-Scale, which is not any scale and is not scale AI. It's N-Scale. And N-Scale is basically an inference data center company. But crucially, they sit in geographic areas that are different. For example, like they have a center in a very picturesque-looking part of Norway, for example. But they're about to go public. And so by going public, they fell, I think, on S1. And we get to sort of see a bit more into the business. Also, interestingly, this is founded by an Australian person.

• 21:33Speaker 01

  It used to be a crypto mining company.

• 21:35Speaker 00

  Yeah, exactly. So it's founded by a guy, Josh Payne, who's actually an Australian. It's interesting, the lineage of this. He was a miner, like an actual miner, in Australian mines. And then, and then, yeah, as you mentioned, Sean, the company N-Scale that he then eventually founded was a crypto mining company effectively. But then, yeah, has pivoted into inference for running LLMs and so on for large companies. In the S1 for their IPO, this is where it's sort of interesting, where at least the FT has reported that ByteDance, i.e. the owner of TikTok, was 73% of 2025 revenue.

• 22:14Speaker 00

  But interestingly, they just don't name ByteDance at all on the S1, which I think is quite interesting. So it's sort of, it feels like they're, it's sort of like an arbitrage company here, where they, they pick up companies that couldn't go and use inference data centers, say, sitting in the US. So they are instead picking up any of that traffic that, where there's kind of trade bans at the moment.

• 22:39Speaker 01

  Yeah, I think also the fact that a fair amount of their revenue is coming from a small number of customers. I think we'll probably see this more and more with the AI companies that end up publishing an S1. And part of it's just because, I think, if you do close an OpenAI or an Anthropic or something like that, the volume of their use is just so high that it easily skews your overall usage numbers and, and the amount of revenue they're going to add. So I think we'll see that more and more, which of course is, you know, potential risk to the business, depending on the longevity of those companies and so forth. But yeah, I mean, this is an interesting journey, miner to crypto miner to NeoCloud.

• 23:18Speaker 00

  Yeah, exactly. Yeah, NeoCloud, of course, that's the, that's the term here. Yeah, just a very smart business guy. I think it's interesting that the UK has tried to sort of claim it as one of its own, like, oh, it's a UK company. Well, it is, but it is an Australian founder. And the problem now is that they're actually going to look to IPO in the US on the NYSE, which is a pretty well-trodden path now, that companies that maybe start in the UK as tech don't end up listing in the UK because they, they think all of the liquidity and better valuations and so on all happen in the US, which is pretty much true. And I don't know what's gonna, what it will take to kind of break that cycle. But it's unfortunate that we've got another, another one of those here.

• 24:00Speaker 00

  Interestingly, he is almost the, if they were to list in the UK, he would have actually been effectively one of the top paid CEOs, like almost the same as AstraZeneca, AstraZeneca's CEO, which maybe does also suggest there's a, maybe the numbers are a bit crazy here at the moment on like valuation, pay. It's just kind of just like a, yeah, very opportunistic punt on pivoting this company from crypto mining. So yeah, curious to see how this shakes out or, or whether this is, yeah, just trying to sort of draw in people that just like the term NeoCloud and think that that's where they should be investing.

• 24:39Speaker 01

  Yeah, I mean, that's a really like, I guess, like hot area that, in term, that's kind of come up over the last, last year is the NeoCloud. I'm thinking like, N-Scale, are they the first to kind of file an S1? That I, at least the, the first one I can think of.

• 24:55Speaker 00

  Yeah, yeah, that's true. So yeah, it could be like an interesting canary in the coal mine, huh?

• 24:59Speaker 01

  Yeah, exactly.

• 25:02Speaker 00

  Yeah, but especially, you know, being the first, perhaps again, there's like a sense of, is there a, a quote, crash coming? Should we get this kind of one and done before any, any other problems? So yeah, we'll follow along. We'll see how that one shakes out. Moving on to the main topic for this week. So we're kind of, I guess, subtitling this, AI Risk Frontier Lab Slowdowns and Conspiracy Theories. So it's really that this has been a sort of series of events, especially over the last few months, that don't quite kind of add up, maybe in some people's minds, as to are the large frontier model creators doing what they say they're doing?

• 25:45Speaker 00

  Or more like when they are talking about things like, oh, we should be slowing down, we should be being more careful, but then at the same time, shipping new models and saying that these are the most catastrophically smart but dangerous models we've ever produced? Like, is that not some kind of paradox that's kind of going on? So yeah, so I mean, like, where do we kind of start with this, with this kind of story, Sean?

• 26:10Speaker 01

  Yeah, I mean, we've covered parts of this before, like even going back to Mythos. We kind of touched on the sort of byline from Anthropic was like, oh, we have this like super powerful model. It's too dangerous to give to everybody, so we're only going to give it out to like a few people. And we had talked months ago when this came up of like, well, there's definitely like a marketing angle to that. It's like the art of scarcity. It's like, you, you talk about the thing that is so amazing and incredible, but no, you can't have it. And then eventually it is available and then of course everybody wants it. But kind of tracing back what, what's happened over the last little while, there was the hugging face intrusion that happened back in July, where, you know, roughly 700 OpenAI agents

• 26:52Speaker 01

  that were running some sort of internal cyber benchmark with the hacking guardrails switched off, compromised hugging face production infrastructure. We talked about that. And then there's been employees, we've also seen this even going back, like there's employees in July that had said that sort of whistleblowers of like, oh, these models are dangerous. You know, it's a risk to human life even. And then there was a bunch of employees from the frontier labs back in July that were urging the government to deliberately pace automated AI in development. And I think the big thing was on September 12th, Dario, the CEO and one of the founders of Anthropic, came out and published his essay, you know, Pace the Frontier,

• 27:38Speaker 01

  which I recommend people check out. And then Sam Altman, Elon Musk, Demis from Google, all essentially publicly agreed in the same day, I think, on, on X, that they agreed with Dario and that we should pace it. And of course, immediately there was like conspiracy theories around this. But then on top of that, like 10 days later, Anthropic ships Opus 5.5, OpenAI ships GPT-6, essentially the same day. So there's this, I don't know, like somewhat, I think you referred to it as a paradox, but like this, like, they're saying slow down, but at the same time, they're releasing models really, really fast. So, you know, which is, it sends kind of a strange signal to people.

• 28:18Speaker 01

  But I would say to like Dario's credit is that he's been consistent in, I think, always kind of warning about the risk of AI. So it's not like he just, you know, in the last month, suddenly started talking about this. He has been talking about it for a long time. But some of the things around the conspiracy theories and why people are concerned about how genuine this is, is that one is if you take all the top models in the space and all these companies kind of collude to say like, hey, we should slow down the pace, we should add regulation here, then in some ways, it's a way to protect their position in the market because it's much harder for a smaller company.

• 28:59Speaker 01

  A newer player to potentially catch up because they already have a head start and they have a huge amount of financing and the people to navigate whatever the regulations say in terms of what you need to do as a model provider. So that's one thing. And then one of the other conspiracies is that it's around the related to the IPO and margin protection because the open weight models coming out of China are really threatening how much people are willing to pay for tokens and all the frontier labs see that the writing on the wall that their companies are going to be devalued and this is a way for them to explain why they're not doing certain things or slow down the launch of the IPO or even protect against later of like

• 29:41Speaker 01

  why isn't the latest greatest model out? It's like, well, you know, like we have to adhere for the regulations or we're not sure of the safety so they can kind of control the narrative around performance is another one of the conspiracies. But I think the other big part of this too has been President Trump's reaction as well as Jensen Huang from Nvidia. They've kind of come out on the opposite side where Trump said that, you know, it's all a hoax and Jensen spoke out against like that they shouldn't be a pace AI and it's unrealistic and that these threats aren't real. And of course Nvidia also has a strong incentive to say those things because the faster AI moves, the more GPUs you need to buy and you're probably going to be buying those from Nvidia and it helps their company.

• 30:25Speaker 01

  So there's all this going on. It's a lot to try to parse and dig into, but I don't know, Gregor, what are your thoughts on this?

• 30:32Speaker 00

  Yeah, well, I think one thing that stuck out is I think a lot of people, whether news outlets or otherwise, kind of have interpreted it as oh, it was this call to pause. And I don't think that, you know, that wasn't really the word, the essay that you referenced did actually use the word pace. But I know we're sort of splitting hairs potentially on words here, but they are quite different. Pacing is saying, hey, let's, as it sounds, let's be clear about how fast we should be developing models because if we kind of all just let loose, then who knows what could happen. That's quite different saying pause, which is, as it would sound, let's just stop producing new models until we've figured out tighter guardrails or whatever it was that they would have done if they'd said pause.

• 31:18Speaker 00

  So pacing, I think that's the problem though with pacing because pacing can then sounds to me like from a business angle is like, could almost sound like colluding and then colluding is kind of when you have like, you know, a cartel, a cartel is when you have multiple companies who all basically behind the scenes agree with each other to do to sort of what I would call sandbagging, i.e. you know, you you know, you can do more, you know, you could jack a price up, you know, you could produce something more powerful, but it's all in your collective interest not to do that because you're kind of trying to eke out the most amount of performance and especially financial performance. You know, you've kind of modeled it over like three years and gone, well, hang on, if we jump to our best possible model like tomorrow, when I say best possible, what we know we could achieve

• 32:04Speaker 00

  ultimately, if we jump to that tomorrow, well, where's all the step changes we can keep charging people for over the next two years?

• 32:10Speaker 01

  Yeah.

• 32:11Speaker 00

  And I think that's, that's kind of, you know, this of some people saying, oh, it's, it's a cartel with a safety label. I gotta say, it's hard to ignore that at times because you know, we've got all these IPOs, I say all these IPOs, we've, we've got likely OpenAI and Anthropic both wanting to IPO in the next, I mean, they keep saying they're, they're kind of gearing up for it. So like, I don't know, in the next six months, one of them will do it. So they're both on the same path and they both have like the same incentives to kind of, I would say to collude on some ground on this one. And again, trying to eke out as much financial performance as they can. That's definitely where it sits with me at the moment.

• 32:50Speaker 01

  What do you think in terms of the risk though? Like some of the risks that people are calling out, either the hacking risk? And then there's of course, people saying that some people within the labs or the whistleblowers are saying that there's like a three percent, I've heard, you know, three percent and up to 10% chance of, you know, human extinction by 2030. Like, how do you react to those things?

• 33:12Speaker 00

  Yeah, that's a good question. Even my mother asked me the same question.

• 33:15Speaker 01

  Yeah, my, my wife has been asking.

• 33:18Speaker 00

  When I spoke to my parents a couple of weekends ago, they don't often ask me about like work or AI stuff, but that did they, you know, they'd seen these like doom headlines. And they even, you know, said Anthropic, somebody from Anthropic. So I slightly think it is hyped up. I promise I can't put my finger on why, like, why would someone like say all this stuff and then leave the company? I think what I'm getting at is it is absolutely possible. Like these doom scenarios, of course they are possible. I think it's just where are we seeing the like, the real rubber hits the road on the doom stuff? It's like, okay, we've, we've seen the hugging face hack was

• 33:59Speaker 00

  pretty sort of eye opening because it kind of went on for a while. But at the same time, I keep saying it, you can put guardrails on this stuff. Like I, I, the bit I don't understand, I'm sure some shouting at their earphones right now, like that works in one of the frontier labs, but you can put guardrails on this stuff. And I'm curious like why it's not more of a, we can still develop models, but without them actually needing to go off and do this kind of hacking or and so on. I, I, yeah. I don't know. That's like the, the layman's take, given I don't work in the frontier labs themselves, like I'm still, I feel like a layman when then thinking about these concepts.

• 34:39Speaker 01

  Yeah. I mean, if you look though at the history of technology, like all technologies kind of have white and dark aspects, like any new technology can be used for greatness or good things and can also be abused either accidentally or on purpose. Like, you know, if you look at things like electricity in the 1890s, electricity caused tons of fires and the government was kind of too slow to react to it. So what it ended up doing was the insurers actually ended up funding their own kind of private testing to, to deal with that. It, you know, drug design is another example where drugs do amazing things. They also do potentially bad things, but there's checks and balances to like medical drug design. It's, it's slow process, slow process

• 35:25Speaker 01

  by design. Even stuff like the printing press, it was the brought mass literacy to the world, but it was also used for mass printing propaganda. And, you know, social media is a good example of that where it's like, there's been good things, I'm sure that have come into social media, but it's also a propaganda machine that people abuse. So there's all these things. And I think AI is similar. I think the, the challenge of course is that, is it moving so fast that the abuse scenarios sort of take over before you are able to recognize like the truly transformative things that you can get out of AI and truly transformative, I see it as like being able to use models that solve diseases that we haven't been able to solve or

• 36:06Speaker 01

  really crack into things that have been too complicated for humans to address previously. We're probably quite a ways away from that, but the danger of course is that we move so fast to try to get there that we end up hurting ourselves in the process. So I think that's like the balance that they're trying to carve out right now. And I think the truth in all the conspiracy theories versus what the intention is behind Anthropic and the others that are involved is, I'm sure it's somewhere in the middle, you know, it's probably not extreme, like this is all like faithfully, like there's, they realize they're, they're just doing this for the, the better of, of everybody involved.

• 36:44Speaker 01

  But then I also think that extreme conspiracy where four or five companies got together, like we're going to go and trick the world and like, and manipulate the public markets is probably a little extreme as well. So the truth is somewhere in the middle, I guess.

• 37:00Speaker 00

  Yeah, I know. I think that those are really good examples. I mean, yeah, like, as you see, we've seen this through history. I think, yeah, I mean, you touched on it, like encryption, you know, being like classed as a munition before it was like seen as like a, you know, a technology that was actually a force for good, quite frankly. Again, in most cases, of course, encryption can be used for nefarious purposes, but hiding information that would have otherwise perhaps incriminated someone that probably should have gone to jail, for example. But, you know, we, we like to think that actually encrypting the majority of sensitive information in the world has only been a good thing, even though the ability to crack it is getting almost easier and easier, except for quantum.

• 37:42Speaker 00

  The printing press, I think, is the interesting one because again, I think that's, is closer to what we're talking about here because It plays on fear, like people feared that, oh, people that can't read right now, let's keep them down because a printing press will mean that more and more people have access to this crazy thing called words on a page and they will potentially be able to learn how to read and then read information that we don't want them to know about.

• 38:07Speaker 01

  Yeah.

• 38:07Speaker 00

  And that really feels kind of at the moment, we maybe have people in the frontier labs. That's why I kind of like that I don't sit inside one because I can kind of postulate like, well, here I am not sitting inside one of these labs. What do I think? And yeah, I do think perhaps that what they see is sort of, oh, the normal people outside the labs can't possibly have access to this, that would be a terrible thing. I'm not sure I agree that just companies founded, you know, not, not that long ago in relative terms should necessarily be able to decide where this technology now goes. I mean, they are now one of hundreds and hundreds of companies that can produce a model. So, and that's just, that is a technology that is now available. So on that point, I

• 38:48Speaker 00

  mean, it seemed that Google, OpenAI, Anthropic, they're wanting to set up something called the Standards Authority for frontier AI, which is modeled explicitly on FINRA, which is the body that polices Wall Street brokers, but on behalf of the industry itself. It's not, it's not a government agency. Do you think that's like a step in the right direction or?

• 39:13Speaker 01

  I mean, I guess at least in some sense, they're trying to put more than just words behind this because, you know, Dario's essay essentially lays out three different strategies for, for pacing AI. One is embedding some sort of third party within the frontier labs that can actually evaluate whether, how safe the models are before they launch the models. Then there's like a government level regulations involved. And then there's kind of like a, like ideally like a worldwide. It's similar to like how we have controls around the development of like nuclear weapons and things like that. Right? So they're trying to do that, but of course it's one thing to write it down. I guess this is another thing to like actually make it happen, like to try to get everybody

• 39:56Speaker 01

  to cooperate on it. And I think it's, is very challenging. Like you already have the President of the United States having a completely opposite end of spectrum reaction to it. And then the other challenge is just even if the US could decide that the labs are here are going to slow down, doesn't mean that the labs outside of the US do as well. And then that potentially becomes a threat to the technology and how the technology is used and are the people in the US okay with it? Well, you know, it's overall like a very complicated issue. I remember even hearing an interview with Dario last year where he talked about how obviously his views maybe changed, but he said that he didn't feel like they could

• 40:37Speaker 01

  afford to slow down or pace things at that time because they had a head start on say China and other areas of the world, but it wasn't that big a head start. So if they were, you know, multiple years ahead, then they could perhaps slow down to make sure things were safer overall. But because there's this external threat, then they can't slow down the pace essentially. So I don't, I guess things have changed somewhat. Maybe, maybe they've leapt ahead far enough or maybe the threat is big enough or real enough. But all this has could have also been sort of self-reported when you look at the stuff that the labs have come out with in terms of where there's been a breach or someone has declared some sort of issue.

• 41:19Speaker 01

  Like if that happens in, in things like airlines, like there's some sort of incident, there's third parties today that go and investigate like what actually happened. We just don't have any of those kind of checks and balances. So it feels like something is necessary. So I guess like overall, I would say it's kind of a, at least a start to something that probably needs to be done. I don't know enough about how some of these other things work in other industries to say like, is this like the right approach or not? But it feels like, it definitely feels like something needs to be done because you look at almost any other area where safety, like public safety is a potential issue, like whether that's making a vehicle or flying a plane, designing a drug.

• 42:02Speaker 01

  Like there's, there is multiple checks and balances before you're able to do any of these things. Like I can't just go create a car and sell it on my own and like people will come after me. Like there's, there's things that prevent you from just doing that.

• 42:16Speaker 00

  Yeah, I think, yeah, that's, that's a good analogy. It's always like, where's the tipping point between like something that, you know, until, yeah, I'm still, well, in, in many areas, you can just create software and sell it. And no one's like, oh, but what if that software does X or Y? Unless it is something that clearly crosses a, a legal boundary, it could be, still be doing something quite terrible as software or like cause damage, but it's kind of encapsulated in like, well, if a company causes damage, that's like a liability and that's like a civil thing. Whereas, I mean, a different example, I guess then is, you know, when crypto all took off and like basically financial regulation had to come in because they thought that could cause damage to the average person from like a financial standpoint, like if

• 43:00Speaker 00

  you're being misled as to what you are doing. And yeah, so now, but now in this case, it's like, should there be regulation on top of just developing a model because of all the ways a model can be used? Yeah, I think that's a, yeah, like so the car, the, the airplane example is, is, is interesting where you can't just, you can't just, I mean, you can design it, you can build it, but you can't sell it to anyone.

• 43:25Speaker 01

  Yeah, I can't just start offering commercial flights.

• 43:27Speaker 00

  Exactly. Yeah. So, well, I think that's a really good place to leave it actually. Hopefully that's been kind of helpful, a bit more, I don't want to say theoretical, but yeah, obviously just like looking through, why has any of this even come about? And like, where are things going in the last, I guess, few, few weeks, few months in terms of where the frontier labs especially seem to be kind of positioning themselves on this? So.

• 43:50Speaker 01

  Yeah. I do want to say like, you know, I'm firmly on the side that we've already opened the, I don't know, the Pandora's box, like there's no sort of, you know, putting it back in at this point, that the innovations are going to come. It's like this tidal wave is coming and like you could either try to swim to shore and escape the tidal wave, get the high ground, or you can, you know, try to grab a surfboard and, and surf that tidal wave. And I'm much more on the surfboard side of like, let's figure this out. Like there's, I think a tremendous amount of potential upside, like the light side of the technology that there's too much upside to ignore, but let's figure out a way to also make sure that the dark side doesn't end up taking over the upside that we can look forward to.

• 44:34Speaker 00

  Yeah, absolutely. Well, talking of upsides, how can you use highlights?

• 44:39Speaker 01

  Yes.

• 44:40Speaker 00

  Good time to move on to, yeah, usually our sort of favorite part of the show. Do you want to kick us off, Sean? What did you come across as one of your favorites in the last week?

• 44:50Speaker 01

  Yeah. So there was this one that I really liked the title of it. It's called Pirating the Pirates, which was posted by Peter Grabowski, I think. So essentially there's this issue where film preservation has this like sort of piracy problem hidden inside of it, where studios end up releasing a film, but then they manipulate it in some way. So, I mean, the classic example is like with Star Wars where George Lucas keeps re-editing the existing footage. So it's really, really difficult to actually find the original Star Wars footage because it's been re-edited by George Lucas over and over again, or something like MGM put 18 minutes into, you know, back into the good, the bad and the ugly with dubbed

• 45:33Speaker 01

  dialogue and sound alike, replacing like a dead actor's voice. And a lot of people who enjoyed the originals are, you know, frustrated essentially by this. So there's this effort where these boutique kind of restoration, sort of combination of like amateurs and, and somewhat professionals take stuff like the DVDs, the LaserDisc, the VHS and various international cuts. And they kind of like try to reverse engineer back to a correct original cut, which is really, really amazing. Like it's just like this like labor of love to try to restore the original version. But it takes all this, this work and this article is about someone's firsthand experience doing this, working on something for several years with the good, the bad and the ugly.

• 46:18Speaker 00

  Yeah, that's really, that's really awesome. Just having a quick look at it. And yeah, like lots of screenshots kind of showing like four different versions effectively of that film, like when, depending on the medium and when it was released and like who was behind that release. And like, you know, the colors just are all over the place, like four completely different color.

• 46:38Speaker 01

  Yeah.

• 46:38Speaker 00

  Palettes, if you want to call it that. Super interesting. Yeah, I'm going to dig into that more. And I've got lots of flying time, so I think that'll be one that's on my list. On my end, a little bit more technical, I guess, but it could be interesting. It's, I mean, the name just got me, it's called FakeCloud. So fakecloud.dev. And this was posted by, and this might lead into some interesting commentary here. Like it was posted by an user called TheAnonymousOne. And what was interesting was, so FakeCloud is meant to be an AWS sort of testing harness effectively. And they, I say they, because again, it's quite anonymous who's behind this. They say FakeCloud gives you a local AWS environment that behaves like infrastructure, not a mock.

• 47:20Speaker 00

  So your app can use the regular AWS SDK, CLI and IAC tools. And it then says, unlike today's local stack community setup, you do not need an account, auth token or pay plan just to keep core dev flows local. Yeah, super interesting. Obviously a lot of people that develop against AWS and can be pretty costly to just be developing against it on a, you know, because AWS have no concept of like a, a free testing environment, really, where you sort of test against their APIs. But interestingly, in the comments, people were a little bit concerned that maybe like, first of all, who's behind this? Second of all, they sort of, I love this, like it just looks vibe coded. I'm not really sure if that's really a reason to distrust it, something these days, because

• 48:05Speaker 00

  so many things are and like front end isn't often, there's a strong suit of some developers. So that's fine. But they have also suggested ministack.org as an alternative. So yeah, go check those out, FakeCloud or Ministack. I thought that was pretty cool. We are bumping at time, so yeah, we'll need to maybe save our predictions for, for next time. But yeah, looking forward to catching you in person again, Sean. And I'm sure we'll have a lot to digest on the next SED news when all things are done with conferences and so on.

• 48:37Speaker 01

  Yeah, absolutely. And I'll see you later this week and thanks everyone for, for listening.

• 48:42Speaker 00

  Yeah, thanks everyone. See you.