Why a Nation Can't Outsource Its Frontier AI - Alistair Pullen (Cosine AI)

This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlst Britain's most capable coding model can't be exported, and that ban is the whole reason Cosine set out to build one from scratch. Alistair Pullen, CEO and co-founder of Cosine, sits down with Tim Scarfe to explain how a frontier system he calls Fable, locked behind US export controls, became the founding case for a UK sovereign model trained on the Isambard supercomputer in Bristol. The bet underneath it is economic. Pullen argues that an inference company, rather than a training-first lab, doesn't need billions to compete: millions, a national compute allocation, and a consortium feedback loop can be enough. From there it gets into the machinery, why open-weight models still trail the frontier on size, active parameters and data, the mixture-of-experts versus dense trade-off and why active params dominate how a model actually feels, and the edge that real coding trajectories confer. The back half is about making agents trustworthy. Pullen makes the case for beating "slop" by rewarding the process instead of the final answer, reframes code review as runtime proof (spin the bug up in a VM and force the agent to actually exploit it), and walks through Swarm, Cosine's system running hundreds of sub-agents in one shot. It ends on why memory is still an unsolved hack, how synthetic graders let you run RL on tasks with no built-in test, and why Pullen reads US export controls as an accidental gift, with a supply-chain sting in the tail. --- TIMESTAMPS: 00:00:00 The sovereign mandate and the Fable ban 00:04:02 Millions vs billions: the inference-company model 00:07:19 The consortium feedback loop 00:07:40 Why open models lag the frontier 00:14:59 MoE vs dense, and why active params matter 00:16:29 Trajectories: the process-data advantage 00:19:48 Beating slop: reward the process, not the answer 00:26:06 Reusable abstractions and the epistemic wall 00:29:56 Code review becomes runtime proof 00:37:32 Do agentic harnesses still matter? 00:40:35 Swarm: orchestrating hundreds of sub-agents 00:45:14 Why memory is still unsolved 00:48:25 Synthetic data and graders for RL 00:53:09 The US export gift and supply-chain risk --- REFERENCES: organization: [00:01:15] Cosine https://cosine.sh [00:04:14] Mistral AI https://mistral.ai [00:05:50] Anthropic https://www.anthropic.com [00:07:42] Cohere https://cohere.com [00:08:36] DeepSeek https://www.deepseek.com tool: [00:02:52] Isambard-AI https://isambard.ac.uk [00:05:56] Colossus (xAI) https://en.wikipedia.org/wiki/Colossus_(supercomputer) [00:07:52] GLM (Z.ai) https://z.ai [00:11:52] NVIDIA B300 https://www.nvidia.com/en-us/data-center/dgx-b300/ [00:15:37] gpt-oss-120b https://huggingface.co/openai/gpt-oss-120b [00:15:52] Devstral 2 https://mistral.ai/news/devstral [00:16:01] Llama 70b https://www.llama.com [00:17:05] Claude Code https://www.anthropic.com/claude-code [00:26:23] ARC-AGI (Francois Chollet) https://arcprize.org [00:40:38] Swarm (Cosine) https://cosine.sh [00:40:50] OpenAI Codex https://github.com/openai/codex [00:41:16] Lumen Outpost (Cosine) https://cosine.sh [00:41:18] Kimi K2 (Moonshot) https://huggingface.co/moonshotai/Kimi-K2-Instruct [00:49:55] SWE-bench https://www.swebench.com [00:52:40] SystemVerilog https://en.wikipedia.org/wiki/SystemVerilog person: [00:23:40] Andrej Karpathy https://karpathy.ai paper: [00:27:10] GRPO (DeepSeekMath) https://arxiv.org/abs/2402.03300 [00:27:13] GSPO https://arxiv.org/abs/2507.18071 Incompressible Knowledge Probes, Bojie Li https://arxiv.org/pdf/2604.24827 Estimating the Size of Claude Opus 4.5/4.6 https://unexcitedneurons.substack.com/p/estimating-the-size-of-claude-opus --- ReScript: https://app.rescript.info/session/5852d2b884c4ce4b?share=10b9799160845bb11779f8ac6cd3124f

• 0:04Speaker 00

  Hey, Bose here. When your next track starts, listen for the decisions that the artist makes. The vocal pushed right up front, the bass that waits half a beat, that sample from a 70s underground gem. Every choice deliberate. With Bose, you'll hear it all. Your music deserves Bose.

• 0:31Speaker 01

  This episode is brought to you by Michelob Ultra. Good times mean competitions between you and your friends, like seeing who can throw some pigskin the farthest or rush the field the fastest. Winner gets an Ultra. Because it's not just a superior light beer, it's also giving fans superior access to their favorite sporting events. Superior is worth playing for. Visit michelobultra.com/superioraccess to enter. Enjoy responsibly. Copyright 2026 Anheuser Busch InBev Michelob Ultra Beer, St. Louis, Missouri.

• 1:00Speaker 02

  I was horrified, as many were, when Fable suddenly got banned. We have obtained the mandate to build the UK's first sort of sovereign LLM. How can you do in millions what they are doing with billions? There isn't a huge amount of room for error or a huge amount of wiggle room. It was not something that was on my bingo card in January, numbers like 10 trillion being knocked around. We were compressing most of the internet at that point. Really funny, opt a cloud delete your production database. It'll say, oh, you're right to point that out. Agentic harnesses are getting less important over time. A model can probably do with bash-only, basically, any task these days. Also, thank you, Donald Trump. Yeah, I mean, for the first time, I feel like a second-class citizen because they

• 1:44Speaker 02

  are going faster than I am, and I really hate that. Me, that boils my blood more than anyone else, and we are going to do everything we can to pull this off. We have no choice but to make it happen. Alistair, it's great to meet you, mate. We are here in London, where it's customary to say, hello, geezer. Hello, geezer.

• 2:01Speaker 03

  So where in London are we?

• 2:03Speaker 02

  We are in Hoxton right now. So we're in Shoreditch. We're about half a mile away from where Cosine started, my apartment, which was in Hoxton Square, just over there. So we haven't come very far, but we have expanded a fair bit since then.

• 2:15Speaker 03

  And what is Cosine?

• 2:16Speaker 02

  Cosine is a frontier lab based here in the UK. Prior to about three months ago, we built best-in-class coding agents specifically for highly regulated and high-side environments. So think things like financial services, insurance, defence and so on. But more recently, we have obtained the mandate to build the UK's first sort of sovereign LLM, which is a much more ambitious vision and something which is on a scale much larger than we've done before, but it's very exciting to be working on it.

• 2:48Speaker 03

  So tell me about that sovereign AI piece. I should say, by the way, so I read your, you know, the article about you in The Telegraph.

• 2:54Speaker 02

  Marvelous.

• 2:54Speaker 03

  And I was horrified, as many were, when Fable suddenly got banned.

• 3:00Speaker 02

  Yes.

• 3:01Speaker 03

  Because of this export control. And now everyone suddenly is thinking about sovereign AI. So tell me the story.

• 3:07Speaker 02

  So, well, it ties into, it ties into a bunch of different things. It ties into like the backstory of Cosine and one of the reasons we're fortunately placed to be able to do sovereign AI is that we have a lot of expertise around model training, model building, all of the infra, algorithms, data, people, that you need to do that kind of thing. And we've been doing that for some time. So we already had all of those things in the organisation. And then probably, I'd say nine, 10 weeks ago, we were inducted into the government's sovereign AI unit, or backed by them, I should say. That is something that they have put out to, you know, increase the number of sovereign AI initiative, you know, companies that are, that are being built in the UK.

• 3:52Speaker 02

  And what that looks like in practice for us is an allocation of compute on the Ismpad supercomputer cluster at Bristol. And that has allowed us to, well, honestly, it's one of the things that's unlocked our ability to even like have the ambition to do something like this, right? Fundamentally, one of the biggest blockers for Like a startup of our size or smaller, to be honest, on being able to approach work like this is a huge part of it is the compute. Like if you raised, you know, 50 to 100 million bucks, like a good chunk of that would go on compute on doing a project like this and to have allocation come from the Sol AI unit is is huge because it genuinely does enable it. And we still use

• 4:33Speaker 02

  some private compute on the side, but fundamentally, all of it will be done on Isambard, which is which is super cool. And and to be honest with you, it was some not something that was on my bingo cards in January, at the beginning of the year when we started out, we I mean, we still do our, our, I'd say, our conventional business of like coding agents and the models that we already have built. But we've been able to take that vision and really take it to the extreme in a way that we wouldn't have been able to otherwise.

• 5:02Speaker 03

  So I'm not I'm not being funny, but the million dollar question is, well, you actually

• 5:06Speaker 02

  I'll give you more than that. I'll give you more than that. Yeah.

• 5:08Speaker 03

  So, you know, folks over there in the US, they've got probably on the order of hundreds of billions. You've got Mistral, which is on the order of, you know, let's say 14.

• 5:17Speaker 02

  Single to double digit billions. Yeah.

• 5:18Speaker 03

  Something like that. How can you do in millions what they are doing with billions?

• 5:22Speaker 02

  Yeah, no, it's a very fair question and one that I probably get more than anything else. We at Cosine are not an inference company, and that sort of ties into the kinds of deployments and the way that we sell our product. So for your viewers, I should probably give a bit of background. Given the fact that we predominantly deploy into highly secure, high sided environments, most of the time, nay, nearly all of the time, we are not hosting the model ourselves. A customer isn't hitting, you know, cosine slash, you know, API slash V1 and then hitting a chat completions endpoint or something like that from us. They are either taking the model weights that we give to them, deploying them on their own GPUs. We have a lot of that. That is like the most air gap,

• 6:06Speaker 02

  the most secure deployment we do. Or they are renting GPUs in some hyperscaler cloud that they're already a part of. So whether it be like Azure or AWS or whatever, and then they'll run the model there. What that means in practice for Cosine is that we license the technology that we build. We don't actually make a margin on tokens or anything like that. And all of this ties into your question, meaning we don't have to spend a lot of the money that the Americans are having to spend on data centers for inference purposes. Now, that's not to say that you don't also need a huge amount of compute for training, obviously you do. And a huge amount of the infrastructure they have in the US will also be used for training. But I think that one of the biggest

• 6:51Speaker 02

  reasons you've seen people like Anthropic struggle recently, and the reason they've signed the deals they have with like the Colossus cluster and so on, is because of inference and not because of training. So you do need significantly less resource if you're not going to do the inference bit. And we're fortunate in that the way that we sell the product, it means we don't really have to. And on the other side of that, we are taking some interesting research approaches in terms of like how you pull something like this off. And we can talk more about that in a minute, I'm sure, in terms of how we are architecting the model, how we're training it, some algorithmic stuff. All of that's to say that we, we do have a credible shot at pulling off the full run, including like the continued pre-training, the mid-training, the post-training, all of those bits.

• 7:34Speaker 02

  But to be completely transparent with you, there isn't a huge amount of room for error or a huge amount of wiggle room. There are obvious places where we have had to make trade-off decisions. You know, that includes, that extends like the scope of RL. I'd always like to do larger RL runs, more generations, more, you know, inference time compute during the RL process to get more variety. We can't do as much of that as we would like to if we had like 10 times more compute, for instance, right? So there are trade-offs, but fundamentally, I think. Given the way that we scoped the project, and we can get into more detail, I'm sure. But given the way we've scoped the project, I think it is viable in that very narrow scope.

• 8:19Speaker 02

  We have some of the largest companies in the UK all feeding use cases and their desires for what they want the model to be able to do directly into us. So that we can train a model that's really good for them. I think that is potentially a feedback loop that hasn't really been explored as much in the space. Obviously, you don't want to say bad things about some of these companies, but yeah, you know, the models from Mistral, from Coherent and so on, they're not competitive. Even the Chinese models, arguably, they've only really started getting competitive the day before yesterday. So, you know. Or so, yeah. GLM 5.2, so the vibes are good, although on the ARC challenge, it didn't do very well, but maybe that was just a red herring. I can talk about that in a minute, but yes. Yeah, very cool.

• 9:00Speaker 02

  But, you know, like, so is it because, because you were almost implying, oh, it's because they weren't trying to make it better. Like, what, how can we make models that are as good as those frontier models? Crudely, I think there are a couple of key things, well, maybe three key things. I think one is architecture slash raw model size. I think the second is active parameter count and the third is data. I believe, and correct me if I'm wrong, I believe the largest model that Mistral have made to date is the 675B Mistral 3 Large, right? sparse M O E, very, very similar to DeepSeek's architecture, if not the same, I think.

• 9:40Speaker 02

  As a result, like fundamentally, that that model exists in the way that it does because it fits a use case that they have seen. It probably fits a GPU deployment profile that they have seen in the enterprises they're trying to sell to in France or Europe. And as a result, pragmatically, they're like, right, this is probably the biggest that we can get away with, given what what certain companies have access to. As a result, you're obviously going to cap out how far you can go in terms of model performance. So. There was a very interesting analysis done in a blog post, which I can't remember the name of, but I can send you post hoc because I just found it very interesting of a breakdown of the probable sizes of models like Sonnet and Opus.

• 10:25Speaker 03

  Yeah, I saw that.

• 10:26Speaker 02

  Did you see that? It was so cool the way that it was done, right? Through Vertex and figuring out, okay, given what we know about open weight models and latency times and stuff like that.

• 10:36Speaker 03

  The one I saw was it was they came up with a bunch of questions and they could infer based on the general knowledge.

• 10:43Speaker 02

  I've seen that one as well.

• 10:44Speaker 03

  Yeah, but that was a little bit sketchy, wasn't it?

• 10:46Speaker 02

  I've seen a more empirical one, I'll send it to you because that blog post alone played a large part in like our decisions architecturally we took for the sovereign model. One of the things that was very clear from that is that the likelihood is, and like again, no one really knows outside of the outside of Anthropic, that something like Sonnet is in the, I believe, like 1.3 to 1.5 trillion total params, M O E, and has probably 100 plus billion active, right? And then an Opus is probably in the 1.5 to 1.8 range and probably has 150 to 180 billion active, depending on the D type we're talking about, whether it's FPA or FP4, whatever.

• 11:27Speaker 03

  What about Fable?

• 11:28Speaker 02

  That wasn't up for long enough for me to try to work because I basically, what I wanted to do is give Fable that blog post and then point it and then point and be like, right, do the analysis on yourself and tell me how big you are. Never got that far though, because I don't think it was up for long enough.

• 11:42Speaker 03

  It was, it's a ridiculous uplift though, isn't it?

• 11:44Speaker 02

  It's, I saw, I'm sure you saw on X, like numbers like 10 trillion being knocked around. I don't know if that's true or not. I have no idea. I think it's obviously bigger than the other ones and obviously has way more active parameters. But I couldn't speculate, I genuinely don't know. But I, yeah, I think that the net of that of that article was that, you know, Opus was in the region of 1.5 to 1.8 with around 150 billion active. Obviously, there's a lot of algorithmic and data work that goes into it, but I think if you don't at least match that architecture. Then you're you're already going to struggle to to sort of reach that ceiling.

• 12:25Speaker 02

  And I think an example of this, and it's way more nuanced than than this fairly basic argument I'm going to make. But if you look at like a DeepSeek V4 Pro, right, 1.6 trillion total. Can't remember the exact number, but it's going to be in the region of 30 to 50 billion active. Yeah. Right. And and my view is that the reason for these architectural decisions is largely for inference ability of the model. Like, like, sure, great that you have those top line 1.6 trillion parameters. But like, if no one can run it because they need two nodes of B300s just to be able to like fit it into memory and actually, you know, run it at decent TPS, then like, how many people can actually take advantage of that? I think that is one of the reasons that whether it be like Chinese open source or European or American open source

• 13:08Speaker 02

  has has not reached closed source performance is like, there is that pragmatic question of, okay, well, the labs have a huge number of GPUs. And they have enough inbound demand to make sure those GPUs are utilized to a level where they're not that worried about having them up. Whereas, you know, the the the open source community doesn't really have the same the same argument. If you're running it yourself, if you're gonna, if you're gonna run a model of that scale on your own hardware and it's not really being utilized that much by your organization, you're going to worry about how much money you're spending on just those GPUs being idle. Yeah. Um, so I I I think that on that first point, architecturally, I think that overall parameter count is obviously important. I think active parameter count is also incredibly important. Um, and then the last bit is data.

• 13:53Speaker 02

  Um, I think that I think that obviously the labs have some element of edge in data, both because they are able to procure so much from the brokers who sell it. Um, and also because um, they are able to, they have internal data functions which are very mature at this point. I'm not saying the other labs don't have that, but they're definitely not at the same scale, both in terms of spend or maturity, particularly, I I I think, I think to an extent, um, and this is definitely not true, but I think to an extent like the pre-training corporate. Is there going to be that much difference once you're in like the thirty trillion token range?

• 14:34Speaker 02

  Okay, you kind of all have roughly the same stuff. We we're compressing most of the internet at that point. Mid-training, similar story. I think post-training is like has been and remains one of the most interesting areas for these labs. And I think that is having having worked with some of the labs on post-training data, because obviously we're very good at coding. It's been interesting to see like how they've been procuring even the formats that they've been using for like the raw data and the different use cases that they're interested about when they're putting, you know, requests out for we want this, we want that, we want the other things. And I think that is one of the key areas they're differentiating, having really good post-training data and also just being able to run RL at like ridiculous scale.

• 15:10Speaker 03

  Quick pause. Agents are getting smarter every day, but even the smartest agents get stuck without the right context and the right tools. That is where Notion comes in. With the recent launch of custom agents, Notion became the collaborative AI workspace where teams and agents work side by side. And now their new development platform is turning that workspace into infrastructure developers can build on. Now, this is exactly how I run MLST. The whole show lives in Notion, my guests, the publishing calendar, the commercial side, everything is in there. But what's changed is that it's now agentic. I just talk to my agent, it can be Claude or any agentic harness.

• 15:51Speaker 03

  And then it then talks to Notion via the MCP or the CLI and it's just done. And then I can access it on my phone. It's an absolute game changer. We'll get to the RL bit because I know you've got an opinion that, you know, RL training is really, really important. But I mean, there's a few things on what you just said. I mean, first of all, the the MOE thing, is it just a trade-off for inference speed? Or do you think there's actually some beneficial advantage in terms of like factorization, having an MOE?

• 16:15Speaker 02

  I think at least like my opinion is if we if we could had trillion parameter dense models, we would.

• 16:20Speaker 03

  It would just be strictly better.

• 16:21Speaker 02

  It would just be better. Yeah, I think but I think like again, that that is taking it to the logical extreme of of. Could you, what would you need to inference that? What would you need to actually run a parameter, you know, model of that, of that size? I think what like an interesting, an interesting example of this is, um, and, and it's not really a fair fight. But if you take at small scale, something like a GPT OSS 120B, right? That is an MOE, I believe it's five billion active. I can't remember, it was a while ago, but it was something like that. And then you take a like Devstraw 2 123B, which is, I believe, like a similar architecture to Lama 70B. So it's fully dense. I don't know if you've used them back to back before. Devstraw feels so much better than the GPT OSS model, right?

• 17:05Speaker 02

  And that, they are architecturally quite, quite different, particularly in like the attention mechanism. But fundamentally, I think that a huge part of that just comes from the fact that one, you have 120 billion active parameters per token. The other, you have five and, and it is a very, we, we've deployed both to customers like last year. And the difference was night and day in terms of how it felt and, and, and what they got out of the coding agent when they were running it.

• 17:29Speaker 03

  Oh, interesting. Yeah. Cause on, on the other stuff, so you know, you were talking about like the data, the pre-training, the algorithmic stuff. I mean, maybe the algorithmic stuff is kind of converged only because we now have this basin of attraction where there's kernel optimizers and entire ecosystems around this. And like, maybe it's kind of converged, but the data thing is interesting, right? Because like, there must be an insane amount of engineering and you know, with L L M's, it's a little bit like, what's the magic word? Like if you, if you frame the question in the right way, it, it has that representational friction and it does interesting things. So I'm, I'm guessing Anthropic do a whole bunch of like, you know, data, data curation and, and pruning. But another thing that Anthropic do is they have Claude code, they have the ecosystem and.

• 18:05Speaker 02

  Have trajectories coming in all day, every day. Yeah.

• 18:07Speaker 03

  Exactly. And those trajectories, because you know, you've got this big thing, which is that it's not about where you end up. It's about how you got there. And, and, and I think software engineering is, is the, it's not about running code. It's actually like just creating. Yeah. Creating mental abstractions, doing experiments, refining those abstractions, sharing them with the team. So, so, so this, this process, like so iteratively, I'm running code, I'm testing things, I'm refining my abstractions. And like Anthropic have access to so much of that data. How much of an advantage is that?

• 18:36Speaker 02

  Obviously, it's a huge advantage. I don't know what their terms of service say. I'm not saying that they train on all that data. I don't know whether they do or not. Certainly, one of the interesting things that having trajectories means. And something that, like, obviously internally, we, you know, collect our own trajectories of our own use of cosine. Obviously, we don't have anyone else's because of the way we deploy it, but for cosine employees using cosine at least. We do still get a fair number of trajectories, nothing like the order of magnitude that Anthropic get, but it, the most useful thing for trajectories for us is

• 19:15Speaker 02

  seeing how users prompt models. So like, there, there, there's one thing when you're putting together RL data sets and you can come up with these like beautifully formed like problem statements that get fed into the model that are very well structured and very well formed and they're very clear about what the expected outcome is. The reality is that users don't prompt models in that way at all. They're like, F you, it doesn't work, why the hell, like, are you dumb? Like, why have you done this, that and the other thing? And, and the reality is, is that's the distribution that the model's going to be exposed to in the real world is you need training data that looks like that. And I think that I would speculate there's a lot of, a lot of the alpha that Anthropic get out of their trajectories they get back is obviously,

• 19:53Speaker 02

  obviously they're helpful. I don't think that they canonically know, it's not like there is a grader distributed to that trajectory, like they don't, it's not like an RL thing where you know that this trajectory was good, like absolutely. I'm sure you can do some judging stuff and you can see how the users responded to be like, okay, well, at this point, the user was probably satisfied that the job was done. But it's also fundamentally seeing like, what is the user saying during the conversation? And how are they saying it is incredibly useful and it's, and, and, and even, even at our scale, we have.

• 20:27Speaker 02

  There's an argument to be made about how representative it is for like the entire development ecosystem, that's a different question, but we do have like an idea of of generally speaking across across months and months and months of usage, like how do engineers interact with these products and how do they respond and all these things? And that's very useful to make things more realistic at training time.

• 20:48Speaker 03

  Yeah, 100%. Yeah, because I mean, you've spoken about SLOP online, it's one of my pet topics as well. So, you know, vibe coding works in the sense that quite often it does the thing that you tell it to do and, and, you know, the test will pass, but actually you're building a spaghetti monster and you're throwing, you know, more bad spaghetti after good spaghetti. And instead of doing a one line fix that it's supposed to do, it'll give you like 200 extra lines and, you know, there's the whole understanding debt thing, we can talk about that. But, you know, so what we actually want to happen, and neural networks do do this to a certain extent, they learn statistical invariances that represent some kind of abstract structure and that helps them to generalize. And obviously, what we want them to do is to learn problems in the abstract so that they can generalize to new novel problems that they've never seen before. So, you know, the idea is we

• 21:31Speaker 03

  capture the thought process and then we capture that generalization if we do it well enough. That's the rough idea. I mean, how do you think about SLOP?

• 21:38Speaker 02

  So, SLOP is also one of my pet hates. When you are, obviously, any of your viewers who I'm sure use Claude Code or Open Code or whatever agentic coding harness they like, Cid Nauseam is like both a combination of model vibe and also SLOP problems. And you hit the nail on the head, I think, in your question, which is when you said that, okay, the test pass, right? Okay. Is it, is it technically functionally correct? Sure, right? But to what cost is the big, is the big question. And I think fundamentally, I mean, like when you think about how these models are trained to do software engineering, and this is the realization that we sort of had when we, when we built Outpost, and I'm sure you've, you've seen the blogs about how we did it.

• 22:25Speaker 02

  Within that realm, normally, and again, we don't have that much insight over how this works in the big labs, but normally, when you're training a model to be better at software engineering, you have some kind of software engineering problem, you have a problem statement, you have some kind of test, often a unit test, but not always, depending on the task type, that needs, that is either failing in the early, in the prior state and passing in the after state. And then, you know, you give the agent the problem, it goes and does its trajectory, its rollout, and then at the end, you run the unit test, and if it passes, then, okay, you got it right, great, you get a reward, and then the weights update. Obviously, the problem with that is, is a few fold,

• 23:10Speaker 02

  but fundamentally, like, it could have come up with the most insane way of doing something, whether it be in terms of like commands that it ran, that could have been unsafe in terms of code, that's absolute crap compared to what it should have actually done. And all of those things get reinforced, whether you like it or not, when you give that reward based on purely correctness. So, there are a number of things that we have done and are continuing to do in the RL process to try to ameliorate this. And in terms of slop specifically, there are a couple of key things. One is that, like, correctness gates everything else. So like, if you get the problem wrong, regardless of whether

• 23:51Speaker 02

  you did it in an elegant way, like, you don't get rewarded. But beyond that, we do have other rewards that target the exact things that we've been talking about. And also, in some cases, but not all, we do have reference implementations for these things. Like, if you are using a pull request from like a permissively licensed open source repo as like some seed data, you do have the original patch, the human made, and you can actually do some level of like, okay, let's compare what the agent wrote, let's compare what the human wrote. And does this seem reasonable to be, you know, 500 lines longer? Probably not. Maybe we shouldn't give the full reward for this. So, there's stuff you can do on the pure reward level, but we're also doing stuff on an

• 24:35Speaker 02

  Algorithmic level, which has been looked at more and more in the space, which is to do with credit attribution in trajectories. One of the big problems with RL as it stands, I think, is that, and I'm by far not the first person to say this thing. Andre Karpstein said this like over a year ago, but fundamentally, this notion of you have a rollout of maybe 256,000 tokens, right? In some extreme cases, and that culminated in like a one or a zero, depending on what the model did. And what we're saying at the moment in many cases is, okay, all of those tokens are equally weighted in getting us to that answer.

• 25:11Speaker 03

  Yeah, yeah.

• 25:12Speaker 02

  Which, when you think about it, is insane, because that's clearly not true. And in so many cases, the, there will be small decisions, or like important decisions in a trajectory that were sort of, you know, forks in the road, that would have either, that could have resulted actually a bad outcome. Where the model decided to go down the right one and ended there. And what we and others in the space, I guess, are trying to do right now is, okay, if you can find out what those ranges of like, I guess, high entropy tokens or places where you know a decision was made, finding that is half the problem.

• 25:53Speaker 02

  And then figuring out, once you know that this is an important thing that happened, determining whether it was good or bad relative to the final outcome is a different story.

• 26:01Speaker 03

  Yeah.

• 26:01Speaker 02

  But if you can do that, A, your RL gets significantly more efficient, because you're not relying. I mean, the analogy I always come up with is like, okay, say you were doing your English A level and you'd written, you were practicing and you'd written like an essay for your teacher. And you'd written a 2,500 word essay on a question and the teacher just gives you, right, okay, that's a B. Thank you so much. And you're like, hmm, I don't, what made it a B? He's like, I'm not going to tell you what made it a B, it was a B. And then what you're going to have to do is you're going to have to write hundreds of essays and you'll get an A on some, you'll get a B on others, you'll get a C on others, and eventually you are going to be like, okay, so when I do this, I tend to get an A more. So I think this is probably a good thing to reinforce, but it would be far easier.

• 26:45Speaker 02

  If the teacher just send, you know, circle the sentence and be like, this is rubbish. You don't say this, right? And that is fundamentally the principle we're trying to bring into like RL across the board. Because you get so much more performance, you get more out of the flops that you have. And also you're teaching the model to learn the things that are actually important and not just the filler, right?

• 27:06Speaker 03

  I know. I mean, the, the great thing about machine learning is that it just generalizes low down the abstraction mountain. So from very superficial statistical generalizations. The bad thing about machine learning is it generalizes.

• 27:17Speaker 02

  Exactly, yes.

• 27:18Speaker 03

  So yeah, I completely agree with you. You know, we have a huge problem with benchmarks in machine learning. So, you know, we, we are obsessed with pass at one, pass at five accuracy. And we don't seem to care about reliability, about, you know, consistency, about security, about abstraction forming. That's clearly the most important thing. I mean, Francois Chollet, he, he did the ARC challenge and unfortunately they were brute forcible. And now, now he's got this new version, which is so difficult to brute force. You have to form abstractions to get any kind of good performance on it. But so, so you're saying there's a new form of RL, perhaps different from the deep sea type, type of RL. Which is like, rather than just being rewarded for getting the right answer, you're actually forcing it to form reusable abstractions and go higher up the mountain.

• 27:55Speaker 02

  Yes. In short, you've explained that far better than I did. But yes, that is, that is, that is essentially what we're trying to get to. One of the nice things is that I think that essentially what we're talking about here is credit attribution within a, within a trajectory. That ports quite nicely to a bunch of different RL algorithms that are in vogue at the moment. You can use it with a GRPO, you can use it with, you know, GSPO and all the different flavors of the algorithm. But fundamentally, having a rigorous and importantly unopinionated way of pointing at ranges of work that an agent has done and being like, this is good, this is bad and so on.

• 28:36Speaker 02

  And for the advantage that's being calculated across those different trajectories that you've made. For that advantage to be attributed to those ranges disproportionately to the rest of the tokens in the trajectory. Means that your weight updates will be more targeted at. Making those characteristics either appear more frequently or less frequently.

• 29:00Speaker 03

  Do we still have an epistemic problem? Because you know, the one problem with machine learning is it doesn't really have like the notion of true and false. So yeah, we can do feedback from code execution feedback. We can do a whole bunch of abstract lenses on actual processes that engineers are doing. But don't we still have this gap, though, that we don't really know whether it was correct or not?

• 29:18Speaker 02

  It's one of the hard, yes, it's one of the hardest things, particularly, I mean, one of the, obviously, one of the reasons that coding has taken off as a use case is because you have verifiable rewards in some guys. And I think one of the things that, I mean, I've just said is that we are trying to bring some of the more like fluffy taste related things and make them verifiable. But on a, on a more floating scale, I think for, obviously for things like law, for things like other non verifiable domains. That is way, way harder. That is way harder. And I think that's one of the reasons, one of the key reasons that we haven't seen like the same revolution in other industries as we've seen in coding.

• 30:00Speaker 02

  Or maths or physics, because you can't just statically like compile some law and see whether you get, you know, a one or a zero. And I don't know, I don't know what the answer to that is. I really don't. Maybe, maybe there are new ways of codifying those domains. Or maybe you just bring everything in distribution, which I think is what's happening these days. I think if you just make everything in distribution, you target every use case. And you, and you have some level of, whether it be human judge or good L L M judge that has been either trained or well prompted by a human, you get close.

• 30:38Speaker 02

  But I still think that essentially, in many cases, you need to enumerate a lot of these problem sets at train time, make sure the model is good at them because otherwise you'll. The dream of generalisation across the board, I think, has already been shown to not really exist that much.

• 30:56Speaker 03

  I know, but we're in such an interesting time because every new model comes out and you know, Fable was so much better, but we want to have systems that do more with less, which is what you're saying. So we want them to acquire these abstractions. And it's a really, really weird situation, right? Because do you think we'll ever get to a point where we can remove the human from the loop? Because right now, I think the basis of AI psychosis is that you have very, very talented humans and they know how to ask the question and they have taste and they go in the right direction. And there's this virtuous co-creation cycle. It's very, very good. So, so you know, we're now in the realm of, it's not vibe coding, it's agentic engineering, it's very exciting. But do you think it could ever be done without humans?

• 31:37Speaker 02

  Yes, I think it, I think it can be done without humans. I don't think we're anywhere near there yet.

• 31:43Speaker 03

  What would that look, do you mean in well specified? You know, I'm not talking about style transfer. You know, like there was that anthropic, they built a C compiler. That's a well specified problem.

• 31:52Speaker 02

  Yeah.

• 31:53Speaker 03

  I give you like a novel application and I can only vaguely specify it.

• 31:57Speaker 02

  Hmm.

• 31:58Speaker 03

  We're going to need a human for a long time, aren't we?

• 32:03Speaker 02

  If you want it to be good and maintainable and actually in the style of something that a senior engineer would write, for now, yes, you definitely need a human there. Probably post hoc to be like, okay, here's the mountain of stuff that you need to change and all the design decisions that like in your chain of thought, you thought were good, but actually weren't because of, you know, real reasons. But I do think that, I do think that we will get there. I think it's going to be through a combination of model improvements, creating RL environments and problems that really look like the kind of things you're talking about. And also harness engineering. I think a combination of all those three things.

• 32:48Speaker 03

  Yes, and we'll get to harness engineering, but okay, but in the meantime, we've got the spaghetti monster mitigation strategy. And and I think one of the big problems is code review, right?

• 32:58Speaker 02

  Yes.

• 32:58Speaker 03

  Because, you know, like the the the AI psychosis is manifested as understanding that. So increasingly, I become less aware of what's going on, and that's actually really bad for me maintaining my competence and for me being able to evolve the software going forward.

• 33:11Speaker 02

  So ask the right questions. Yeah.

• 33:13Speaker 03

  Absolutely. Exactly. So so how can we do this? Because now we are generating ridiculous amounts of code, right? Is is is this a case of let's use more AI to do the review? Or do we still need humans in the review?

• 33:28Speaker 02

  I think that for I I think we need more runtime validation of what AI is producing. And what that looks like is I I I think code review will evolve somewhat. I think it will be more like proof that the thing that it says it's doing is actually doing that thing. Obviously, like AI reading Git Diffs is is is not not useful. I think it can catch things, and I've seen it catch things in the past. But one analogy, and and it's not actually for code review, but one analogy that I can I can tell you that's really good. When it comes to, say, our cybersecurity scanning products that we have, which I guess is analogous to kind of review.

• 34:11Speaker 02

  Because like it's reading basically a whole code base using a swarm. But one of the key things that we saw with that was that it would go through that process. And it would pick up like so many things across large code bases, like this could be a problem, this could be a problem, this could be a problem. And I think one of the things that we see in that and in AI code review is like, sure, like if you just look at that code in isolation and that function definition, it can look quite dodgy. But in reality, the code path is never hit. Or the there is like this other function that's called first that mutates this variable that then means that that doesn't do what you think it does. Or there's an environment variable that happens at runtime that means that this doesn't happen. All of these things that, you know, it's just static analysis, you can't do it. And one of the best ways that we mitigated that problem in that product was that we had what we call exploit validation,

• 34:56Speaker 02

  which is literally like, okay, the agents, the swarm comes up with its list of things. And before it makes it to you, every single one, so we spin up the application in a virtual machine, in a, in a, in a runtime, right? And in as close to production way as possible. And we tell the agent, right, well, you've seen the source code. So if it's vulnerable, you should, you should be able to figure out how to get through it, right? You should be able to like craft your, your horrible zip file to exploit this thing that you think exists. And if you can't, then you just take it out the list because it's clearly a false positive. And what we have been working on is applying the same logic, but to just PRs, right? And we're not necessarily looking for like cyber vulnerabilities, but we're looking for, okay,

• 35:39Speaker 02

  you have allegedly built out this feature, you have this new screen that has this table in it or this form in it that does this. And when you click on this button, it should result in a new entry in the DB and it should show up in the, all the stuff you'd expect, right? And instead of like, obviously you can look at the diff and for the most part, you can get a lot of mileage out of that. But also just like, show me it's doing that, prove that it's done that in some reasonable way before, before it even makes it to me. Because otherwise you end up in this situation, I was just talking to a customer earlier today, where they were saying, when they first started adopting like agentic coding tools,

• 36:17Speaker 02

  they were in this spot where either they would end up with this enormous backlog of code review, right? And, and, and no one likes code review, let's face it, no one actually enjoys that.

• 36:28Speaker 02

  Or you get people being like, after a minute, it looks good to me, merge, thank you so much. And you just get this YOLO merging into like your main branch and that's bad as well. So I think. There is going to be way less cognitive burden if in however you're doing your review, you can see the code, but you can also see canonical proof that at least on the happy path, the thing that it's saying it's doing is actually happening. And the other half, I think, at least in the present day, to this is also really comprehensive end-to-end testing of everything you build. And that is something that really sucks to have to build out, but once you have it, it saves you from so many problems, because I'm sure,

• 37:09Speaker 02

  you know, you've seen, I've seen in like personal projects and so on, you vibe code for an afternoon, you built 10 new features, then all of a sudden, the other five you had before stopped working. Why has this happened? It's like, oh, well, okay, the abstraction you had, I've just messed with it. And now it doesn't work for that thing. So yeah, it's a combination of like defensive stuff, like the end-to-end stuff, and also just proactively lifting mental burden from people by being like, look, here's like either a screen recording or some screenshots or whatever of me showing you that this is what I think it is.

• 37:38Speaker 03

  I know, I know. It's really funny. After Claude deletes your production database, it'll say, oh, you're right to point that out. I'm so sorry for doing that. There's nothing you can do about it, by the way. But yeah. I mean, this is another alignment problem though, right? Because, you know, if we are accumulating understanding debt, the functional descriptions themselves are going to suffer from that because we don't understand the functional description anymore. And it's not just functional descriptions, you know, like there are intents, there are, you know, there's behavior, there's all of these different levels of describing a system, user stories and stuff like that. And unfortunately, these are different views of the blind elephant, right? They don't necessarily have friction with reality. And you see the problem here, like we're just, we're just losing touch with what it's supposed to be doing. In many

• 38:21Speaker 03

  cases, actually, Claude and the models, they're writing the functional tests and then they're kind of hacking their own. Oh, it didn't pass. Okay, I'll just change the test. Now it passes. Great. Here we go. You see it all the time. I know. So, you know, it's a very difficult problem, but one that we need to fix. And, you know, for me, I think a lot of it is to do with scoping and constraints, you know, to at least cut down the size of problem. But we should, we should move on. What are your thoughts on agentic engineering? So you guys have got an agentic harness. If I understand correctly, a couple of years ago, you actually forced everyone to start using that because you really want to optimize the hell out of it. And you were talking about the RL piece. So maybe there's some co-evolution with the agentic harness and the RL. What's important in this agentic harness?

• 39:02Speaker 02

  I think that so broadly, I think that agentic harnesses are getting less important over time.

• 39:09Speaker 03

  Oh, interesting. Why?

• 39:10Speaker 02

  Um, the models are just getting so good.

• 39:12Speaker 03

  Oh.

• 39:13Speaker 02

  Um, I think, I think that, I think that you can get, the proof point is that like a model can probably do with bash only basically any task these days. More slowly and with more tokens, but it can probably still do it. Um, that's not to say that agentic harnesses aren't important, but I think like over time, where's the value coming from? It's coming from the model and not from the harness. Um, I think in the way that we built ours and, and we, we have been building agentic harnesses for a very long time. Like the first agentic model that we had was a fine tuned GPT four turbo model that we trained in like January of 2024. Um, and the coding agent harnesses didn't exist at that point. Claude code didn't exist. None of this stuff existed.

• 39:58Speaker 02

  Um, and we had to build one out and we, we did that symbiotically with the design of the model, which is something we still do today. Because you do get way more performance out of like tightly coupling the two fundamentally. Harness engineering is still something we care a lot about. These days, we actually care more about efficiency than anything else. In a world where token costs and tokenomics, which is a word I heard for the first time today, awful word. Um, tokenomics is becoming increasingly important to particularly enterprises who we sell to. We want our harness to use as few tokens as possible, full stop. That's what we're trying to do.

• 40:31Speaker 03

  Yeah, we'll get to that in a second because if, if the models can do epistemic quantification, you could in principle allocate a budget of tokens to get certain things done. But, but even before we get there, I mean, I want to push back on, on the harness engineering because one thing I have found is subagents are a game changer.

• 40:46Speaker 02

  Yes, absolutely.

• 40:46Speaker 03

  And, and that is because a lot of problems are too complicated for an LLM to do in a single pass. So I'm sure you've, you've had a similar experience. As, as a problem becomes more specified, as you reduce the ambiguity, the entropy goes down, the models get better and the models are better when they have less, you know, when they have less rot in their context. So, so what happens is like the, the engineers after, after doing a bunch of, um, you know, engineering. What they find is that they kind of decompose problems into agentic subtasks. And then they have a fresh agent and the agent has a clear specification and

• 41:18Speaker 02

  Only do this one thing, yeah.

• 41:19Speaker 03

  Exactly, but you know, so what you're doing logically as an engineer is you're factorizing a problem into smaller subproblems. You're getting agents to orchestrate and you're not rotting the context in the main one. I mean, how do you see that evolving over time? Because now it's quite a manual process, but do you think, I mean, you've got this swarm thing.

• 41:34Speaker 02

  Oh, thank you for bringing that up because that was exactly what I was going to want. So yeah, swarm to me is subagent orchestration taken to the logical extreme. We're kind of lucky to be able to do it because I think that obviously Claude Code has, I believe workflows and Codex has subagents as well. A swarm is what it sounds like on the tin. A swarm is genuinely like a real, a lot of, a lot of subagents running at the same time in a hierarchical way. And because Cosine isn't trying to serve hundreds of millions of people a day, we are able to serve a swarm-like feature. Whereas I think if Anthropic had a swarm for like Opus, I think they would, even Colossus one would run out of, run out of tokens.

• 42:14Speaker 02

  Swarm does exactly what you've just outlined automatically, basically. So there is, there is a video on my, on my Twitter and also my LinkedIn of me taking our Lumen outpost model, which is post-trained Kimi K 2.6. So definitely not an Opus or a Mythos or anything like that. And I asked it, okay, I want you to build me a mechanical watch compiler was the use case that I came up with. I am Swiss by birth, so I have a reason to do this. And I basically asked, and actually someone did this exact same task with Fable. I don't know if you saw it on Twitter when Fable was out, but essentially like, I want you to build me a SDK in Python for me to be able to like specify mechanical watches in code, because I don't know horology, but I want to be able to do it anyway.

• 42:58Speaker 02

  I want it to be physically congruent. I want you to use some kind of physics engine. And then I also want like a 3D viewer to be able to like see the thing running. And it all needs to be possible in real life. You can't have things like intersecting which other wouldn't be possible and so on. And that is something that out of the box Kimi cannot do. There's no way, not even close, like it would be terrible. In fact, even Gemini 3.5 and Opus and 5.5 can't really do it. But as soon as you put them in a swarm, and for what swarm looks like for Cosign is you have one orchestrator at the very top. It breaks down a problem into subproblems for

• 43:38Speaker 02

  Basically, product managers or whatever you want to call them, we call them sub planners. But they own verticals of this. So like, within that task, you would have had a sub planner to do the SDK. You would have had a sub planner to do the 3D viewer, you'd had a sub planner to write the documentation and so on. And then those sub planners could then delegate to workers and then they have like a flat layer of as many workers as they like. And for that problem, we use 253 subagents, which I think is more than you tend to see in a Claude code session and so on. I think, I think you'd probably hit your usage limit pretty quickly that way. But when you do that, it is possible and you can do that entire project in one shot.

• 44:20Speaker 02

  And, and, and I am contradicting myself quite badly because I've just said harnesses don't matter, but in that respect, they do obviously matter.

• 44:26Speaker 03

  Oh, indeed. I'm very excited about that. But, you know, it raises the question that when, first of all, when, when you start to have loads and loads of agents, you have more understanding debt and less interactivity.

• 44:35Speaker 02

  Oh, yes.

• 44:36Speaker 03

  You know, for me, the, the, the lack of interactivity is part and parcel of the understanding debt. And, and sometimes you, you want to interject, you want to say, oh, you know, you've gone slightly wrong there. I want to change what this agent's doing. And what many folks have found when they build these agent systems is that the agents interfere with each other. They kind of overwrite its own.

• 44:52Speaker 02

  Yeah, yeah, yeah.

• 44:53Speaker 03

  They go into deadlock. How, how are you dealing with all that?

• 44:55Speaker 02

  So it's a hard problem and we experience all those things. In terms of, you can, so one of the key things that we did is we made you, we gave the ability to interject to like an agent on the lowest level. So say you had a worker that was like two levels down from the top one, you can actually like talk to that one, which is an important thing. And with regard to like other problems in terms of like treading on each other's toes, you can put write locks on files so that you, only one agent can, can edit it at a time. You also can provide context to agents when they are using, say like an agent is reading a file that it just read. We have stuff in the harness which says, okay, another agent has just edited this file.

• 45:39Speaker 02

  So don't be surprised if you see it slightly differently as to how you saw it last time. All these things help. They are not like a panacea, but they certainly help. And they, they, they make sure the agent is less surprised when they're like, oh, where did that come from? That wasn't in my last edit. But fundamentally, it is, it sort of comes back to the point I made earlier where, okay, yes, you will run this thing and it will provide you a huge amount of value very quickly, but you are still going to have to comb through it afterwards and be like, actually, the reality is, I don't like this abstraction you've done. I don't like the way you've done this. There is going to have to be some sweeping afterwards, I think.

• 46:14Speaker 03

  Yeah. What are your thoughts on memory?

• 46:17Speaker 02

  Very hard to get right.

• 46:18Speaker 03

  Okay, tell me more.

• 46:19Speaker 02

  Very hard. So.

• 46:21Speaker 03

  Well, it's a similar thing actually to the RL, because memory is not about the destination, it's about how you got there.

• 46:26Speaker 02

  Yes. Memory is very hard to get right. We've tried a bunch of different approaches.

• 46:34Speaker 02

  Fundamentally, I think every approach to memory that exists right now is a bit of a hack, right? It's like a tool, and it's, it's in many cases like a vector DB or like an embedded version of like some tidbit of knowledge, but they're very hard for the agents to know when to query. It's also fundamentally quite hard for the agent to know whether something was useful enough to write to memory. And it's, it's, it's also difficult to, to keep these things up to date. We've had many situations where an agent's been doing a trajectory. And when it's been doing something that was genuinely the right thing to do, it's like

• 47:19Speaker 02

  used its memory and it's like the memory's been old and out of date. And then the agent's like, oh, well, the memory says you should do it this way. And it changes its tack. And there you're like an engine engineer, you're like, no, please don't do that. There are things that we're looking at internally with regard to like continual, continual learning and stuff like that to try to avoid memory being a tool and for it to just something that for it to be something that's just in the latent space with the model.

• 47:45Speaker 02

  That is also very hard. But I think it is a more intuitive and elegant solution to it just being a tool. It's also a tool that's very hard to get right during RL because it is a huge surface area of foot gunnery in terms of reward hacking, in terms of leakage, in terms of. Being able to query something from the future that it shouldn't have access to yet, despite all the guardrails that you can put in place. It's just hard.

• 48:10Speaker 03

  Yeah, exactly. And in a sense, this is another area for AI psychosis, because I've written a memory C line. And I would almost argue that now you don't even need, you know, vector databases and so on. You can just have an inverted index, you know, just a sequelae. Because the models are so good at asking in different directions, you know. So, yeah, there is a huge problem. It needs to know to retrieve. That's a big one. But it actually works, but it only works for me because it creates this fractionated spaghetti mess again. So there's another spaghetti mess in the memory C line. But it works really, really well, but it doesn't work very well at the organizational level because my spaghetti monster doesn't play with, you know, with John's spaghetti monster.

• 48:46Speaker 02

  Yes.

• 48:46Speaker 03

  So if we could solve that problem, and you're drawing an interesting picture as well of how we can actually optimize the different layers of the sandwich together.

• 48:53Speaker 02

  Yeah, I think that as soon as someone gets it right, you'll just know when you're using it immediately. Like, I haven't seen a single implementation, whether it be in Claude code, to be honest, whether it be ours or ChatGPTs or any of them, where I'm truly like, oh no, this isn't just a hack. This isn't just, this isn't just rag. This is like, it's the one remnant of rag that still exists really, in the more traditional sense. And I am certain that there is a better way out there somewhere.

• 49:25Speaker 03

  Oh, definitely. But I think another thing you've said is specialization or generalization. I mean, for me, a lot of agentic engineering is like emergent specialization. So it's like, let's take a big intelligence to crystallize a small intelligence to do the particular thing we're doing. But as we're nearly out of time, final question is synthetic data generation. So what are you guys doing about that?

• 49:43Speaker 02

  Tons.

• 49:46Speaker 02

  I don't know how much I can get through in five minutes, but I'll do my best. So there are a number of areas that we have done synthetic data generation in, in the past. We're doing a lot more of it in the more forward-looking sense for like the sovereign model. Because the sovereign model can't just be good at software engineering. It has to be useful across the board. And that means that we have to get good at that synthetic data generation, particularly in the RL realm, in things that aren't just coding.

• 50:14Speaker 02

  The bread and butter, though, is coding. And the way that we have done this in the past is with a very cool and sophisticated pipeline that we have built out over the course of like a year and a half now. But that's One of the big cold start problems in coding RL, particularly if you're using like open source repositories as a sort of seed data, and even if you're using closed source, it actually doesn't matter, is that nearly all of the PRs or commits or whatever you want to refer to them as, don't have a built-in grader, right? The ones that do are often bug fixes. Like the classic example, I guess, if you're nerdy enough to be in the space, is like

• 50:59Speaker 02

  a Sweetbench-style problem. You have a Git issue, you have a PR that fixed it, and then because it's open source, you have some sort of regression test that was added, and then that's your seed data. The real world doesn't look like that, unfortunately, and software engineering in the broad doesn't look like that. Meaning that to do RL well, you still need to fundamentally be able to do tasks that aren't bug fixes, which is the vast majority of what engineers do. And you still need to be able to tell whether the agent got it right or not, broadly speaking. And what we do at Cosine is we take like real work that was done.

• 51:40Speaker 02

  We're not magicking up made-up problems for the model to solve, because fundamentally, if the model can come up with a problem, it can probably solve it. But we take real problems that were solved, whether it be feature work, refactoring, whatever it is. And what we're synthesizing is we're synthesizing ground truths or graders or ways of measuring whether that thing has been done. And it is a bit of a minefield, because particularly within RL, obviously it's not supervised, so fundamentally, we need to be able to test these things in a way that isn't too tightly coupled to the original implementation. There are many ways to skin a cat, as we know.

• 52:21Speaker 02

  And what that looks like in practice is you need an implementation-agnostic enough way of testing it that's still rigorous enough to check functional correctness. It is a very fine line to tread, and we have done a lot of work. Around it, we have a long pipeline built on it. We have custom post train models that live inside that pipeline that essentially have gotten good because we had to do a lot of manual labeling in places and stuff like that. But what it has allowed us to do is is have a autonomous pipeline where and this is particularly important for enterprises. We can point at essentially any programming languages, any type of task, any stack, anything like that.

• 53:07Speaker 02

  And say, okay, I want RL data for this problem set and we can get a like a good chunk of it. And that is why for like our outpost model, when we were coming up with the the languages that we wanted to get the model good at, things like Java, Fortran, C++ I could go on. We use that pipeline to to gather the RL data for these things. And in many cases, and in some programming languages, there aren't even test suites. That's where it gets really hard. That's where you have to get a bit a bit inventive in terms of like measuring whether the agent has gotten something right or not like. I think it, yeah, it's it's it's things like Verilog and Cisverilog where you have to actually run, I think what's called a synthesizer in your environment to be able to tell whether like this chip actually works or not.

• 53:50Speaker 03

  An EDA.

• 53:51Speaker 02

  Yeah, exactly. Fortunately, I don't run this pipeline. A chap called Ben does, and he knows far more about that than I do. But yeah, that is broadly within software engineering, how we've gotten very good at it. And it is something that we are generalizing out into other use cases that aren't just software engineering as well.

• 54:09Speaker 03

  Very cool. So in in closing, you might argue that the US government have handed you a commercial advantage here because now it's so it's more important than ever to build sovereign AI. You guys have this model coming out, you know, towards the the end of this year. And so I do you think you're going to be able to do it? But also, are you still at risk from a supply chain point of view? Because, you know, like so so much hardware is controlled by America. How's how's this going to pan out for you guys?

• 54:38Speaker 02

  Naively, I think that if the hardware is already in the UK, I don't know how much there is they can do about that. Like the all of the all of the infrastructure the model is going to be trained on already exists and is up in the UK because we're doing it very soon, right? In fact, like experimentation already is happening upstairs. In terms of Is it, has it been a bit of a gift what's happened recently? Given our positioning, absolutely, yes, without a doubt. It has been probably the busiest I have ever been since founding the company in terms of people coming to us saying, okay, we now realise what you're doing is really important. How can we be involved? Like, the consortium of companies that you read out is growing by the day.

• 55:18Speaker 02

  And the involvement and the urgency importantly from those companies, from government, from just like citizens as well, has gone through the roof. And I feel very fortunate and to an extent lucky that obviously like we were well positioned to take advantage of this early. Also, like, we did put ourselves in that position, but also thank you, Donald Trump.

• 55:37Speaker 03

  How did we not see this coming though? Because it was such an oh shit moment for so many people.

• 55:41Speaker 02

  We did though. I think a lot of people, I mean, we certainly did, but it was always, it was always, it was always fobbed off as like, oh sure, okay. I suppose that could happen. But like, I personally didn't expect it to happen as soon as it did. Like, I am also freshly surprised by like the 5.6 news that I'm sure you've seen as well, where that's going to be rolled out. And there is even for me like a huge part maybe like, oh man, that really sucks. I wanted to try that model and I don't know whether I'll be able to now. And that might just be the existence for us now, unless like we and others do work to get that level of performance out in some other way. And that's like our job now, I guess.

• 56:19Speaker 03

  Yeah, I mean, for the first time, I feel like a second class citizen because those, those folks over there in America, they've got better AI than I do.

• 56:25Speaker 02

  Yes.

• 56:26Speaker 03

  They are going faster than I am and I really hate that.

• 56:29Speaker 02

  And, and, and, and believe me, that boils my blood more than anyone else. And we are going to do everything we can to pull this off. You mentioned like, how are you going to do this? Like, we are just going to, we're just going to make it happen. We have no choice but to make it happen.

• 56:43Speaker 03

  Please do.

• 56:44Speaker 02

  Yes, we are going to do everything we can to make it happen.

• 56:46Speaker 03

  On behalf of everyone in the UK, please do.

• 56:49Speaker 02

  We'll do our best, thank you.

• 56:50Speaker 03

  So it's been a, been a pleasure. Thank you so much.

• 56:51Speaker 02

  Thank you very much for having me.

• 56:56Speaker 04

  Finding a doctor is hard enough. Finding the right doctor, even harder. Someone who takes your insurance, has an available appointment when you need one, and feels like someone you can actually open up to. That's where Zocdoc comes in. Download the Zocdoc app to search and compare doctors by specialty, insurance, and availability. Read real patient reviews, find the right fit for you, and book instantly. Because getting seen by the right doctor matters. Zocdoc: Ready to be seen? Download the Zocdoc app today.

• 57:26Speaker 05

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com. Not authorized by any candidate or candidate's committee.