AI:AM: Was Trump-Xi Anything? What Counts as Utopia? + AWS GPUs Cost 3X & AI Diagnoses Rare Diseases

Nathan Labenz and Prakash Narayanan examine rapid shifts in technology, diplomacy, and medicine with guests Jeremie and Edouard Harris, Steve Hou, Joel Borgen, and Daniel McKinnon. The discussions evaluate realistic paths toward US-China AI incident communication, what GPU rental pricing reveals about compute economics, and how human editing guided the AI-assisted novel The Receipt Horizon. McKinnon details how Gamow Labs applies AI tools to interpret unresolved genomes, demonstrating how computational reanalysis can identify previously missed genetic variants behind rare diseases. Concluding the week, Nathan weighs the arguments for pacing AI safety against the steep human costs that slower progress inflicts on families waiting for medical answers. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/ai-am-was-trump-xi-anything-what-counts-as-utopia-aws-gpus-cost-3x-ai-diagnoses-rare-diseases/ Sponsors: Athena: Athena matches you with a dedicated, top 1% executive assistant to handle your inbox, calendar, and daily workflows so you can save an average of 15 hours a week. Get matched with your EA today at https://athena.com/cognitive Parallel: Parallel provides enterprise-grade web search APIs for AI agents, offering the optimal balance of quality, speed, and cost. Get started for free at https://parallel.ai/tcr Deepgram Flux TTS: Deepgram Flux TTS is a streaming text-to-speech model built for voice agents with natural tone, context awareness, and interruption handling. Try it free until September 12 at https://deepgram.com/keep-talking OutSystems: OutSystems is the leading agentic systems platform, helping enterprises build, modernize, and operate mission-critical applications at the speed of AI. Learn more and start owning your agentic future at https://outsystems.com/tcr Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr CHAPTERS: (00:00) Weekly episode highlights (03:44) US China AI diplomacy (08:12) Verifying compute and compliance (Part 1) (12:46) Sponsors: Athena | Parallel (15:37) Verifying compute and compliance (Part 2) (19:57) Reading diplomatic green flags (Part 1) (27:46) Sponsors: Deepgram Flux TTS | OutSystems | Claude (31:44) Reading diplomatic green flags (Part 2) (31:46) The economics of compute (34:06) Hyperscaler cloud pricing premiums (42:02) OpenAI Dev Day takeaways (47:30) Cowriting novels with AI (52:02) Human judgment in art (58:10) AI diagnosing rare diseases (01:04:01) Testing uncertain genetic variants (01:12:20) Model harnesses and evals (01:22:14) Costs of pacing progress (01:24:09) Episode Outro (01:27:19) Outro PRODUCED BY: https://aipodcast.ing

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Nathan Labenz?

  Real things are starting to happen. This week on AI in the AM. On AI risk and verification, we heard from Jeremy and Ed Harris of Gladstone AI. They remain hard-boiled realists about US-China cooperation, but this week I heard a slight thaw. Here, Ed considers what reciprocal transparency could offer the two countries.

• 0:21Speaker 01

  Certain kinds of transparency can be stabilizing. So the kind of transparency that goes like, hey, we're giving you enough vision into what we're doing to see that we are not doing the thing you fear most. That sort of thing is potentially useful and additionally may not actually be that costly to us to do, depending on how we implement it, simply because the Chinese are already all up on our systems. So really, we're not giving anything away that they don't necessarily have already in many cases, potentially.

• 0:51Nathan Labenz?

  Steve Ho, head of research at Silicone Data, which builds GPU price indexes. We asked why renting apparently identical chips costs so much more at the big cloud providers.

• 1:01Speaker 02

  So in the case of hyperscalers, indeed, you are observing correctly. They charge regularly, consistently, at least two to three times, sometimes more compared to a typical, a new cloud. The reason has to do with a long legacy of, you know, whether it's other type of products being offered on their platform, software analytics, safety, compliance, the fact that they already have this long-established relationship with enterprise users that have been on board for a long time, that don't, that have a certain stickiness for moving. It is being sold as a very much of a differentiated product.

• 1:34Nathan Labenz?

  Prakash, my co-host on AI in the AM. We spent part of the week on OpenAI Dev Day. Here, what changes for building software when the models get faster at the same level of intelligence?

• 1:45Speaker 03

  With ultra-fast, you can, as you type, you can interact. It's an interactive kind of build, software build out, actively building games. I think, I think that's really the future. I think the speed, the latency. Latency at same intelligence is probably something that is going to be very important, especially as you clear these hurdles of capability. Like, the thing can build a game, but can the thing build a game with you in the moment while keeping you in flow?

• 2:14Nathan Labenz?

  Joel Borgen co-wrote his novel with AI models. The text has a few AI tics, but the book is legitimately good. Here, what he supplies as architecture and why the pros still needs him.

• 2:25Speaker 04

  I mean, I started the project knowing more or less what I wanted to have made. I sketched it out myself, especially the first half or so of the book was pretty well set before engaging the models. And then the models are good at certain things. They're getting better at everything. But as far as just prose writing itself, even if you tell it exactly what you want and you have a plan for a chapter, you often get something that's unreadable unless you know how to scaffold it and then edit it yourself.

• 2:59Nathan Labenz?

  Daniel McKinnon founded Gamow Labs, which uses AI to interpret genomes. He reports new diagnoses in children whose cases had gone unresolved. Here, what he found when he looked closely at a model's work.

• 3:10Speaker 05

  And you'll see things like, there's one particular case where Grok 4.6, which at that point was state-of-the-art on our benchmark in Grok build, where it missed one because it just renamed the gene. It just like was saying like, oh, you know, N R F two or whatever is responsible. And then it just changed the name of the gene to something totally different. I'd never seen that before. And I was just like, this is dumb. And our harness and our tools prevent the agents from doing dumb things.

• 3:42Nathan Labenz?

  Welcome to the AI in the AM weekly highlights with these introductions spoken in my cloned voice. Please tell us what worked and what did not. Your feedback helps us make the next one better. Part one, a slight thaw. Jeremy and Ed Harris have been interviewing diplomats who negotiated with China. We discussed the Trump Xi talks and the possibility of a channel for communicating about AI incidents. Ed starts with what he makes of that contact.

• 4:13Speaker 01

  Talking is always better than not talking. So that's, that's a positive. And my understanding from at least the beginnings of the Trump Xi conversation and the stuff leading up to that is that one of the things that may be positive that came out of that relationship was, you know, the development of this, I don't know if you'd call it a red phone, but of at least some kind of theoretical line between the two governments on AI incidents and AI risks. The, what, the, the report that we came out with, which is really just a long newsletter, is informed by speaking to about a dozen State Department diplomats who have dealt with China from the negotiating table and who have kind of seen how these things kind

• 4:58Speaker 01

  of develop in practice. And, and one of the issues that they do see, among many others, is we have tried the red phone thing before in the context of nuclear. And by and large, they, they don't always answer. And particularly in, in critical phases, it's often exercised as a point of leverage, saying, we're going to take away this, this phone line and not answer, rather than as something that is like this, this collaborative unified project that's makes everyone safer. Again, this is not to say if the CCP does structurally take AI seriously, which there is, you know, decent reason to think that they may, we could have a whole different and much more positive level of engagement. All that we're recommending, based on the experience of these folks, is being realistic

• 5:40Speaker 01

  about it and having a backup plan.

• 5:44Nathan Labenz?

  I asked Jeremy, after a serious AI incident, what could the United States ask China to stop doing? And what could be verified with the capabilities that exist today?

• 5:58Speaker 01

  Everything that, that comes after this sentence obviously has not made contact with the intelligence community from a red teaming standpoint. So the true answer is we can't know deeply. I couldn't give you an answer to a level of detail where it would be like, okay, you know, that's executable. One easy thing, if I'm going to caricature, data centers put off a hell of an energy footprint. You know, the thermals on those are really, really bright. Data centers are huge. They haven't by and large yet been built to be hidden and it takes a long time to build data centers. Now this will change. AI 2027 talks about the timelines for this. We think it's quite plausible that, that the timelines could be a lot shorter for hiding data centers, just based on conversations with folks in the industry. But like, whichever way you slice it, like, you're going to have an initial conversation

• 6:43Speaker 01

  where to first order for the 80/20 that you really need then, it's like, so help me God, if I see a cluster and that cluster is yay big, you know, that's like the kind of conversation that you're looking at. And how you quantify that is a matter of, you know, the sort of national technical means that the US currently has.

• 6:58Speaker 03

  So, so wait, are you saying having a cluster above a certain size would be a red line?

• 7:05Speaker 01

  So I'm saying, yeah, initial. So if you're, if you're just in that panic moment, right? You're like, we have to do something. And you ask yourself, what is possible? Like, what is possible to do with the existing assets and infrastructure that we have today? Nothing else. Then you do, you do get into a space, not necessarily where there exists a cluster of this size, because you're not, you can't ask them to tear down the cluster, but we have to see the heat signatures from this, like, go away. So if there is a running cluster above a certain size. And that is, to be clear, a tremendously expensive ask in either direction. The, the depreciation on GPUs is, is the major part of the OpEx cost.

• 7:46Speaker 03

  So, so you're saying an incident happens first?

• 7:48Speaker 01

  Yeah.

• 7:49Speaker 03

  And the response to that incident, the mitigation for that incident is, hey, can you turn off this big data center?

• 7:55Speaker 01

  Yeah, like turn off any cluster above a certain size. That's one, one possibility. Because this is. As of right now, to be clear, like the framing is basically as of right now, that's where we're at. And so in some sense, we're going to get into a, potentially a circular loop here, where you can see how big of an ask that is. That is an insane ask.

• 8:10Nathan Labenz?

  So if I kind of sketch out the logic from beginning to end here, it's like, we don't have that great of a relationship. AI capabilities continue to progress at a fast pace. We expect something crazy to happen. When something crazy enough happens, we're going to find ourselves by default in a spot where we have to ask for some outlandish, super high cost move, like shut down all your big data centers, because we don't have any other mechanisms in place that allow for a lower ask, a better trust but verify type of environment, because we haven't made those investments now.

• 8:55Nathan Labenz?

  That leads me to the question of what should we be doing now to A, ideally not end up in that situation, or B, if we do end up in that situation, have better options available to ask for, aside from shut it all down, which is obviously gonna be tough.

• 9:12Speaker 06

  Develop, basically, it's like, you know, you absolutely nailed it. It's develop better verification and develop better offensive options to ensure compliance in the event that verification is verification returns, no, they're doing it. The better verification stuff you can do, the faster, the less you have to rely on absurdly expensive things like this gigawatt of energy radiation shouldn't be visible from space. If we have techniques like this that are vetted by the intelligence community, even if they are, even if they are just like 50% better than this, even if it's like you can, you have to shut down half your data, I'm making something up here, you have to shut down half your data center and we can like kind of sufficiently verify the other

• 9:56Speaker 06

  half or something equivalent to that, you are saving billions of dollars right off the bat. And so the ability for this industry to like continue to make large amounts of money is actually going to be gated for that period of time by these little verification technologies. And there's already this community of little verification startups that's working on, on this technology. And that's why this is so important. And of course, I will also say, the, the offense side of things is critically necessary. If you don't have those offensive options, you cannot assure compliance, you cannot assure compliance. You can verify and you can monitor the situation and you can say, oh, no, but fundamentally, your, your hands are tied. You don't have the tools to actually do anything about it. So both of those things are super important.

• 10:39Speaker 06

  There's this kind of bottleneck that I think a lot of these verification companies haven't really necessarily priced in. And, and like, so this is kind of where a lot of our, our current work is focused. So imagine what happens when company A goes, I have the thing. This thing is gonna work, right? And, and it's a moment of crisis and, and Trump is casting about, or POTUS, whoever it is at the time, is casting about for options to alleviate this, like the trillion dollar bottleneck. And, and then they actually go, okay, the intelligence community has to now vet this because they're not gonna just like start using it, obviously, right? So how long did it take similar technologies in the past to get used, to get vetted and to become what's known as national technical means, NTMs, right? And the answer is, is years, depending on the technology, but very often years, we

• 11:22Speaker 06

  have to do it, China has to do it, we have to handshake on doing it. And even if you remove money as an obstacle and it's no object, there's just certain things that take serial time to do. And so, so a lot of what we've been doing is focused on saying, okay, treaty, or not treaty, let's say AI agreement verification or compute verification company X, you probably should be talking to IC element Y about this because in a moment of crisis, you A, want as much pre-vetted as possible and B, anybody who's involved in assessing a potential national technical means had better have on speed dial, they better have the signal, the phone number, the email, whatever, of like the founders of like all the verification companies that they plan to use or they may end up having to use.

• 12:04Speaker 06

  You want to cut down on all those, those, those barriers, the boring bureaucratic hurdles that nobody ever thinks about because they're boring and bureaucratic. But these are the things that we're just like trying to shadow right now so that when game time happens, things move more quickly. And today, those companies that potentially are pursuing research trajectories or agendas, in some cases like multi, multi, like your $10 million plus research agendas, that are just oriented in a way that unfortunately, like that appropriately placed person in the intelligence community would look at and be like, that's kind of a non-starter. We want them to get that feedback as soon as possible so they can reorient on things that do have a chance of working.

• 12:41Nathan Labenz?

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 12:46Nathan Labenz?

  Today's episode is brought to you by Athena, the executive assistant company on a mission to improve how people work and live. If you want to increase your impact, you have to free up your time. And that's what Athena does best. They match you with a dedicated, full-time, top 1% executive assistant who can take over your inbox, calendar, travel, and everything else that's quietly eating up your week. Athena is SOC 2, Type 2 certified, so you can rest easy knowing that your sensitive data is in good hands. And as a former AI advisor to the company, I can personally vouch for how much they've invested in AI tools and training.

• 13:31Nathan Labenz?

  In fact, one of the very best AI users I've ever met is an Athena client who delegated the exploration of AI tools and the development of AI workflows to his EA. Athena clients report saving an average of 15 hours a week and the average client refers more than two friends a year. That, to me, checks out. I was an Athena client while running my startup and to this day, I continue to refer friends. Go to athena.com/cognitive right now and get matched with your EA. That's athena.com/cognitive. Give yourself back a few hours this week.

• 14:12Nathan Labenz?

  Go to athena.com/cognitive and see who they'd pair you with this month. Today's episode is sponsored by Parallel, where agents find answers. Most engineers today closely follow new model releases, but don't pay nearly as much attention to their agent's most important tool, web search. If you're like me, your agents are often conducting hundreds of searches per day. And while the unit cost is small, over time, this does start to add up. Before starting to use Parallel, I calculated that my agent's search bill would total roughly $500 this year. The good news is that I just had my agents conduct a systematic test and I found that

• 14:52Nathan Labenz?

  Parallel's fast mode, which costs just $1 per 1,000 queries, worked just as well as my previous default provider. And switching to it will save me roughly 80% of my search bill going forward. Parallel's infrastructure is enterprise grade and their suite of APIs offers a range of Pareto optimal options that allow you to choose the right balance of quality, cost, and speed for your needs. So whether you're building voice agents that need 200 millisecond latency or long horizon agents that need deep research, adding Parallel to your agent's toolkit is a no-brainer. Get started for free at parallel.ai/tcr. That's parallel.ai/tcr.

• 15:38Nathan Labenz?

  Still with Jeremy and Ed on verification, I turn to work that could start among the AI companies themselves.

• 15:45Nathan Labenz?

  How important do you think it is that we remain on the same fundamental tech tree or AI paradigm across US and Chinese AI development? My sense is that like, we're in, in some ways, a very fortunate position right now because we're basically building the same tech in the same way and we're sort of encountering the same surprises along the way. So that's one thing. And then the other question is, I feel like if there's anything good to be found in the OpenAI anthropic adversarial dynamic, it would be that maybe they can be a testbed

• 16:28Nathan Labenz?

  for techniques that might later scale to a US-China dynamic. And so if I was the president, I would say, you two have to figure out a way to police each other. And maybe we expand that circle to a few other frontier companies, but like you two, you're the ones setting off all these alarm bells. I need you guys in a room, whatever technology you need to develop, whatever access you need to give one another, it's on you to figure out a way that you can trust and verify one another. And then maybe we can kind of scale that up to a trans-Pacific dynamic that could work similarly.

• 17:02Speaker 06

  I think that's, that's not, that's not insane. Obviously, there are big differences between, you know, what anthropic and OpenAI respectively have on each other, although poaching of personnel does a decent job of mirroring the kind of access that China clearly has the frontier labs anyway. But there

• 17:18Speaker 01

  There's also a basis of trust, I think, between two fundamentally US companies with fairly close to similar values and blah blah versus like fairly radically different. Not to say that this is like totally amiss or whatever, but like there, there are gonna be some differences as well, some similarities, but I think the similarities might, might be worth mining. Yep.

• 17:37Speaker 06

  Yeah. And it, you know, it's also the case that like, so you're talking about the idea of the importance of the stacks being aligned. The hardware lottery does a lot of really good things in this space. I think the most crucial thing is the US and China clearly don't trust each other in terms of the motives that bring, bring each respective side to the table, right? So when China sees the US come to the table and raise issues like slowing down AI or per safety guardrails, whatever, the interpretation that we've heard consistently from people involved in like the track two, track 1.5 kinds of dialogues and Nathan, I mean, you, you've been kind of in that ecosystem or touched it as well, is, you know, the Chinese kind of view it as an attempt to curtail their own development because they see themselves

• 18:22Speaker 06

  as being behind and they're justified therefore in doing things that even wouldn't be appropriate for America to do in their eyes, just to catch up because they're in second place. Being in second place does also mean, being in second place with respect to scale does also mean that you don't see the warning shots with the same resolution. However, they seem to also be more public than at least I would have expected. And so the spillover is something that's nice that China can just verify themselves directly. So that, that might be a mitigator if you see the same kinds of failure modes emerging from whole brain emulation or any, if some completely wacky other branch of the tech tree were to become dominant in China. So I do think it's good. I think we kind of get there by default. It's hard to imagine alternatives that like really shake things up at this point.

• 19:06Speaker 06

  Quantum machine learning, if you wait long enough, I just don't think that's going to be relevant on the timescales that matter, that could radically reshape algorithms. But what we're seeing right now is an industry that's more or less doubling down on transformer MOEs with some bells and whistles and some, some variations here and there, but everything's kind of a transformer and, and that's, that's what seems to ship. So yeah, I expect that we will have that, the benefit of that. It also comes with the benefit of being able to share safety technology as the US did with Russia during the height of the Cold War at times. And so in principle, that does mean that we can work on each other's safety stacks and that might be the source of some trust building measures, though that term also is problematic for China and they've also pushed back on attempts to do that sort of thing in the past.

• 19:50Speaker 06

  So, I mean, it's a, I think it's, it's a good thing. I don't know how far it goes.

• 19:57Nathan Labenz?

  With the Harris brothers, the discussion returned from verification technology to the diplomatic evidence for cooperation.

• 20:04Nathan Labenz?

  I want to not be naive, but I do want to notice and give appropriate weight to positive signals as I see them somewhat developing. I think Xi's speech at the WAIC was like, pretty friendly, pretty conciliatory, like, gave credit to the US for inventing AI. You know, he certainly didn't call for an international arms race. He warned against overstretched the national security concept. We can dismiss that as just nice talk, probably should have at least some weight on that possibility. But what are the like, meaningful things that you are watching for the decision points that will update your thinking on?

• 20:45Nathan Labenz?

  Are they inclined to at least try to control AI for their own narrow self-interest? Are they inclined to meaningfully cooperate? Or are they inclined to seek some sort of domination as, as we often project that we are interested in doing?

• 21:02Speaker 01

  So in terms of what signs to watch out for, I think anything that looks like a positive sign is at least a positive sign to some degree. So a conciliatory speech is good. It's good. At least it's not a hostile speech, right? It could be worse. Everything is kind of tempered by the fact that like rhetoric is, is, is often used for strategic purposes in this way and to kind of shape the battle space in terms of the narrative and so forth. The, the point is not like, these are not positive signs. It's just like, we have to weigh the evidence in the context of the credibility that has or has not been established by this entity over the past span of time.

• 21:45Speaker 01

  In terms of what are maybe slightly more unfakeable signals that they could give off? That, that you would make me go like, oh, whoa, okay, this is, this looks legit. You can imagine maybe, I don't know, something like DeepSeek and like Zifu coming to some sort of pacing agreement because they, uh, some crazy thing happened over there, their equivalent of the hugging face incident. Um, you can imagine, uh, the, the cyberspace commission that, that has, or the commission that has jurisdiction over the, whether models can be released and like, are they, you know, properly ideological and stuff, actually putting the brakes on something because, um, there was some misalignment thing and they're actually, uh, they, they are finding that

• 22:27Speaker 01

  a model that was properly ideological in testing suddenly is, is not being ideological in the wild or something like this. I would say indications where, that, that seem genuine, that, that they are starting to be on the receiving end of these incidents at the level, the same level of detail that our own frontier labs. Like, I think that begins to, to make us, all of us as humans go like, oh, you know, maybe the, the thing we should be concerned about is the giant, the giant like Shaga thing that we don't understand and that we're growing in the labs, uh, at accelerated pace. Like, that, that would make me feel a little safer.

• 23:03Speaker 06

  Well, maybe procedurally too, like, so if, if you take, we have a list of, uh, I think it's like five different historical traps that we've seen in US and diplomacy. The, these are essentially like a, a brief catalog of the ways in which China behaves when they're full of shit. At least like by, by the assessment of, of a lot of the diplomats we spoke, I should be clear, actually, there, there was a, a dissenting diplomat who felt that some of these things were much more sincere, including the use of the, the language, the, you know, the objections over language, arms control, this and that. And that itself is the epistemic problem that we talked about earlier, but basically I would say, take each of those red flags and then flip them over and you get the corresponding green flag. And so if you, if you don't see arbitrary, uh, concerns raised about language that seems

• 23:48Speaker 06

  random, that's a green flag. If you see engagement, this is actually really important. If you see engagement from empowered people, actually arguably as we did, right, with Xi though, again, you gotta calibrate everything with like, we've seen this before in other contexts, but it's certainly not a red flag. But in terms of like concrete commitments, that's the sort of thing you look for. Empowered people, the lack of sort of capricious arbitrary objections, things that actually look like they're making progress qualitatively, is actually a surprisingly good sign because you can contrast it directly with how things have gone in the past, which is not very good. I mean, the contrasting point is, is actually that low that it can be genuine signal.

• 24:26Nathan Labenz?

  What are the things that we can do that are not so costly to us, but are still credible signals to them, that we are not going to try to use AI to gain a decisive strategic advantage and ultimately make them an offer they can't refuse? Uh, if indeed that is not what we're going to do, which I'm a little worried we might actually be about to try to do that. If we were on the path of trying to seek a Pax Robotica where we can all benefit from the abundance that AI, especially in its Chinese manufactured embodied form might provide for us, what would be the steps that you would prioritize next on our side?

• 25:10Speaker 01

  Well, there may be some stuff we can do that's not functionally really even that costly. Uh, generally, as, as I think, uh, Jared, you guys maybe as well highlighted earlier, certain kinds of transparency can be stabilizing. So the, the kind of transparency that goes like, hey, we're giving you enough vision into what we're doing to see that we are not doing the thing you fear most. That, that sort of thing is, is potentially useful. And additionally, may not actually be that costly to us to do, depending on how we implement it, simply because the Chinese are already all up on our systems. So really, we're not giving anything away that they don't necessarily have already in, in many cases, potentially. So maybe some kind of, you know, level of visibility into here's what we're doing,

• 25:55Speaker 01

  here's what the frontier labs are doing and so forth. Um, the problem is like, that's kind of not necessarily something you want to be doing unilaterally. That would be something that, that would come as part of like, trust building measure, dare I say, between two powers in the wake of a, a moment like this. But stuff, I guess goodwill type stuff we can do now certainly would be more interactions between the verification communities in the United States and in China. And this kind of thing is already happening, actually. There's some, there's some quite good and positive interactions between those communities. So yeah, the, the kind of linkages that you get at the level of academic to academic, startup to startup, all kind of trying to solve for the same mission, these are very, very positive things.

• 26:37Speaker 01

  There are also like the, the, um, that it's true that right at the political levels, the two countries have started separating out and, and, and even at the level of kind of big companies and stuff like that, that, um, you see the, the Chinese kind of steel designs and turn of the, the classic like Chinese, like spinoff story and all this stuff. But there still are like real genuine linkages between the two countries where especially on the academic side and a number of other levels, there's like, there's a bunch of sincere people just talking to a bunch of sincere people about like, yeah, this is a problem and this sucks. Yeah, I agree. Like, let's try to solve it. So the more of those linkages there are, uh, the better. Like, the more people there are on, on both sides of the ocean who have the ability to

• 27:18Speaker 01

  talk to their own kind of domestic leadership and say, look, I've spoken to them, like, they're not, they're not evil, they're just trying to do this or that. To whatever extent that's true, that kind of moderates the more extreme tendencies on both sides. It's very, very hard to do that completely because the, the, the actions of these countries are also constrained in a number of ways, but like, it really does help, I think.

• 27:41Speaker 07

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 27:45Speaker 08

  I've got a text.

• 27:48Speaker 09

  The summer might be almost over, but a new bombshell has entered the villa. Meet the voices of Deepgram Flux TTS. I'm Drew, low key casual.

• 27:57Speaker 10

  I'm Alexis, an upbeat morning person. I'm Haley, always cheerful.

• 28:02Speaker 09

  Flux TTS has some real personalities that are ready to speak. You can interrupt, pause and keep talking without losing the plot.

• 28:09Speaker 08

  Fancy a chat? Come try all the voices free through September 12th at deepgram.com/keeptalking. Terms apply.

• 28:17Nathan Labenz?

  This episode of The Cognitive Revolution is brought to you by Outsystems, the leading agentic systems platform. Outsystems is helping their customers build, modernize and operate enterprise systems, starting from any coding tool in a governed agentic engineering model. It's time to innovate at the speed of AI without compromising quality or control, which is why thousands of enterprises worldwide trust Outsystems for their mission critical apps. And we're not talking about theory, we're talking about massive operational wins. Take for example, a leading global retailer that was struggling to keep up with their rapid expansion. They used Outsystems to build AI agents that automated their invoicing and translation

• 29:00Nathan Labenz?

  processes, shifting 95% of their daily store operations to a new digital app, saving thousands of hours of manual work. Or look at a major global logistics giant that was drowning in technical debt. Their engineers were spending their weeks manually firefighting by reviewing platform logs. Using Outsystems, they built an AI log review agent in just five days. And that one agent now saves them over 1,600 engineering hours every single year, letting their team focus on real innovation instead of reactive maintenance.

• 29:35Nathan Labenz?

  Even in modern digital banking, companies are using this platform to standardize and automate complex financial workflows, delivering secure, personalized experiences at a pace their competitors can't match.

• 29:50Nathan Labenz?

  Outsystems gives you the flexibility to deploy how you want and the governance to do it safely. Stop chasing the hype and start owning your agentic future. Learn more at outsystems.com/tcr. That's outsystems.com/tcr. Today's episode is brought to you by Anthropic. By now, you know my story. Claude drafts my intro essays and I rewrite them. Not because the drafts are bad, but so I can stand behind everything I publish. Well, I have an important update. Claude Fable 5 is the first model to have me rethinking my rule.

• 30:30Nathan Labenz?

  Today, I now think co-authorship, not sole ownership, should often be the goal. Where the model excels, rewriting its work can be more about vanity or a misplaced sense of duty than integrity. I feel it most in songwriting. I'm no lyricist, but I'm good with a song concept and Fable writes some amazing verses. I give it feedback on its misses and I push it to aim for higher inspiration, add layers of meaning, optimize syllable density, and above all, write a hit song. These days, I get compliments on just about every song we write together. Claude is the AI for problem solvers.

• 31:11Nathan Labenz?

  It's the collaborator that understands your entire workflow and thinks with you, not for you. Whether you're debugging code at midnight, building a financial model, or strategizing your next business move, Claude extends your thinking to tackle the problems that matter. For problems worth solving, get started with Claude at claude.ai/tcr. That's claude.ai/tcr. And check out Claude Pro, which includes access to all of the features mentioned in today's episode. Once more, that's claude.ai/tcr. Part two, the economy of intelligence. Steve Ho leads research at Silicon Data, which builds price indexes for rented GPU capacity and model tokens.

• 31:55Nathan Labenz?

  Turning compute into a measurable market means deciding which prices can be compared. Steve starts with the inputs to the GPU rental index.

• 32:06Speaker 02

  And we use a combination of both quota prices and transaction prices in our calculation, making those distinction clear, right? So this is a large, you know, sort of, so to speak, a normalization process, using machine learning to help us make the contracts comparable to the extent that you're looking at a single chip, let's say, or H100 being rented from different parts of the world. So one question that jumps to mind right away is that, for example, in the case of an apartment rental index, you would never consider to use just an offer price. Someone who lists a number on the front of a building, say, you can rent an apartment for $2,000 a month. You wouldn't expect you to just trust that number, you can walk in. But we do use quota prices. Why is that?

• 32:48Speaker 02

  The reason is because unlike in an apartment, you know, you cannot click an API, I'm pressing on API and just get hold of the apartment. You have to walk in and talk to somebody and negotiate and that apartment may not be available. In this case, very often, right, with API executable, you know, GPU rental, you can actually get a hold of the GPU, just same way you can buy something on Amazon, right? That being said, we also have a transaction, so we're comparing them. So that if someone who has three nodes of GPU that rented out, say, all three are 275, I will not presume that I can go back to the same merchant to run another one to I275. It could very well be the case that the next one is not available anymore.

• 33:30Speaker 02

  Or if they rent out two out of three, if you're 275, the next one will not necessarily be 275 again either. It could be $4 or $6 or $1, depending on what everyone else is quoting. The person would be crazy if everybody is quoting at $4 and they continue to run at 225. You would think either they're going to change price or there's something wrong with that price, right? So this is the reason why I gave you a long answer again. But what we do is that we want to provide as broad a set of a coverage of the market as possible and normalize everything so that we're capturing the market as it is, as faithfully as possible.

• 34:07Nathan Labenz?

  One big difference I noticed in prices, just browsing the Silicon Data website, is between the Neo clouds and the hyperscalers. These are not small differences. These are like multiple differences in prices, like seemingly 2 to 4X. That's a, that's a pretty big delta. Why does that delta exist? Is there no way to arbitrage it? And does that imply that like GPU hours are being wasted? Or that when I buy from a hyperscaler, I'm sort of preempting their internal work? And they're using everything that's not sold at runtime? Give me a little peek behind the.

• 34:49Speaker 02

  So you, so you can buy, forget about, again, like I like to use analogies. I gave you analogies in apartment, you know, a single bedroom apartment. I'm not going to use apartment again, although I can. You can, by the way, let's say, imagine we're doing a single patty burger index, right? You can buy a burger from a burger stand off of the street of New York, or you can walk into a high-end steakhouse and order a burger. Believe you me, like those two burgers are going to cost very different, right? They're both burgers that, you know, and what we try to do is we try to, as much as possible, find the marginal price for a unit of compute or burger in this case, that is sort of have the same relative, the same feature. I wouldn't want to compare the price of a three, you know, a three-piece, you know, a three patty burger with a single patty burger, right?

• 35:34Speaker 02

  But once I make those adjustments, there are some adjustments I cannot reasonably make because it's capturing a different type of premium from product bundling or product differentiation. So in the case of hyperscalers, indeed, you are observing correctly. They charge regularly, consistently, at least two to three times, sometimes more, compared to a typical Neo cloud. The reason has to do with a long legacy of, you know, whether it's other type of products being offered on their platform, software analytics, safety, compliance, the fact that they already have this long established relationship with enterprise users that have been on board for a long time, that don't, that have a certain stickiness for moving. It is being sold as a very much of a differentiated product. So going to that single patty burger index analogy I gave you,

• 36:20Speaker 02

  I see people who sell single patty burger with fries and milkshake. I can try to strip out the prices of those two elements and isolate what I think would be the price of a single patty burger from that merchant. But if they, let's say, they blended up the fries and the milkshake and inject it into the, into the patty and say, this is a brand new product and charge two times, three times the price. I can very easily, you know, sort of strip it out, right? At which point I say, okay, I'm raising my hands, okay, you guys are a little bit different of a piece and let me put you in a different category, right? And that's how we have so far handled it. We, you know, we believe that the way this, this way, you're getting to a more, a much purer form of a single unit of compute, right? You know, that is actually getting you closer towards this idea of fungibility

• 37:03Speaker 02

  to the extent that things are direct substitutes to each other.

• 37:07Speaker 03

  The same chip can perform differently at different places. And I think according to your site, you have some software that runs on the chip.

• 37:14Speaker 02

  Silicon Mark, yeah.

• 37:15Speaker 03

  Yeah, that you benchmark the chips with. So I guess every single price in your index has been benchmarked by this, by the system?

• 37:25Speaker 02

  No, so we have a physical benchmarking service called Silicon Mark that actually visits individual GPU at the EUID level to try to say, to assess the GPU health, the performance, you know, the throughput, you know, T flops and so on. At a moment, that physical benchmarking and physical spec performance does not enter into our pricing, right? We do not see a strong relationship, at least at the moment, given the nature of the market, a relationship between how the GPU is specifically performing, even though we do see that in the cross-section, you can actually have a bit of a variance, right? In from GPU to GPU, this is not, I think, surprising or new to anyone who's, you know, sort of in this space, the GPU lottery, right? But over time, if you have a cluster that probably averages out and law of large numbers kicks in.

• 38:10Speaker 02

  So we don't use that. We have six features, including geolocation, you know, sort of CPU memory and various things, and term and provider, but the physical spec is not one of them at this, at this very moment. Eventually, when we head towards a scenario where we potentially could have physical delivery, because inference, let's say, based on my, you know, sort of thesis that could make compute more interchangeable, that could enter into our pricing scheme, but at the moment, it does not.

• 38:40Nathan Labenz?

  Another price that I noticed, having moved on the website, that caught my attention is the proprietary LLM index.

• 38:50Speaker 02

  Yeah.

• 38:50Nathan Labenz?

  There, you break down just token pricing.

• 38:54Speaker 02

  And close, yeah, and now overall, yeah.

• 38:56Nathan Labenz?

  And so the proprietary one, you have up 11.5% over the last seven days. And I guess I'm wondering, how are you measuring that? Because it seems like, you know, quality adjusted, everybody would say prices are coming down, right? Even like dramatically so. And like the retail, you know, posted API price hasn't changed, right? Except when they kind of introduce new models. So what are you measuring on a day-by-day basis that allows you to say what the proprietary cost is doing at such a fine grain of resolution?

• 39:31Speaker 02

  Yeah, so first of all, I want to give a little bit of context for what our token indices mean, because they've been, I think, repeatedly, I think, misunderstood. I think partly has to do with the unfortunate naming. We called it the expenditure index, and then people maybe thought it either means price or total volume, when it does actually mean either. It's actually an expenditure-weighted price index that's normalized to a million. And so that can show trends based on usage mix, which I'll come to in a second. When you point out that proprietary LLM model, which is sort of the frontier model, having recently bounced a little bit higher, should put it in the context that since like maybe

• 40:11Speaker 02

  late June through like basically, you know, middle of this month, it has been on a very sharp downward trend of going from some four dollars to what, you know, one point six or something, that's more than fifty percent drop, right? During this time, we have seen a lot of not just frontier, you know, leading labs cutting prices on their, you know, sort of newer variants by releasing cheaper variants of powerful models, new generation models, but also we've seen other proprietary models coming out like Meta and Grok. Don't forget, these are proprietary AI models as well, right? And they've been very aggressive on the price front.

• 40:52Speaker 02

  So more recently, I think that the bounce could come from a variety of sources, right? You know, basically, if people decided to say, okay, I actually quite like, you know, the more expensive powerful model from Anthropic, and they use more of it, that could actually drive up the expenditure-weighted index, price index. Did the analogy like to give people a little bit, like, forget all these are again LLMs, forget this, imagine these are like cars, right? You know, if you have like, you know, Mercedes have like sort of cheap variants and expensive variants, depending on which cars are being sold more and people are liking more, that could affect the average price of the cars sold, right? It's in the same thing here, right? In the market for tokens, there are two things happening.

• 41:36Speaker 02

  You know, token model prices are changing, but usage behavior is also evolving. And to the extent that we observe volume from a handful of these public inference platforms, early platforms that allow you to look at how people use different type of models, I think this most recent balance of the frontier proprietary LLM index, I don't know, I haven't looked into the details, but suspect has more to do with usage mix than anything else.

• 42:03Nathan Labenz?

  From measuring compute to using it, the next conversation is just the two hosts discussing OpenAI Dev Day. I start with Dots, the personal agents OpenAI presented, and who might want a system they do not have to maintain themselves?

• 42:18Nathan Labenz?

  It was funny, I thought when they led off with the whole Dots thing, I was like, well, I think I have all of this, and I'm pretty sure I'm going to continue to prefer my version that I've gradually evolved over the last nine full months now. So that was definitely, it didn't, that didn't really feel to me like a developer product. I thought that that was a little bit muddled, because it very much felt to me like, that's a consumer product, right? Like, that's for ChatGPT users to use, and it wasn't entirely clear how that would be used by developers, if at all. I haven't been down every last, you know, breakout session video, so there's possibly

• 43:03Nathan Labenz?

  some more that I missed, but certainly at the keynote level, it just felt like that's a product that they are offering on a first-party basis to their users, and I do think it will be really useful for people. I guess I would say, my guess is that people are really going to love these dots, but certainly for like my mom, on the other hand, you know, I would say, go for it, just use that, you know, it's probably pretty easy, you don't have to worry about taking on all this stuff yourself, managing your own database on your computer, troubleshooting when things go wrong, even though the models are getting so good at that on their own. I do think this higher level and more polished abstraction will probably be really good

• 43:44Nathan Labenz?

  for a lot of people who don't care to learn a bunch of new tricks and just want to have this thing that they can delegate to.

• 43:54Nathan Labenz?

  Prakash then turned to the models OpenAI presented at Dev Day. These are his first impressions of their capability and speed.

• 44:02Speaker 03

  They had GPT six point one Soul and GPT six Astra Ultra Fast. GPT six point one Soul is about, I think, the same price, or slightly lower price than I think GPT six Soul, but it's basically Astra for the price of Soul, which is what they're calling it. It seems to be a very competent model. I've used it, it's better, it's definitely better than GPT six Soul. GPT six Soul was only released like, what, two weeks ago, two, three weeks ago, so cadence, cadence of releases is stepping up.

• 44:36Speaker 03

  GPT six Astra Ultra Fast, so now you have, they used to have a fast mode, which is two times speed, Ultra Fast is an eight times speed, and they demoed how Ultra Fast works. With Ultra Fast, you can, as you type, you can interact, it's an interactive kind of build, software build out, interactively building games. I think that's really the future. I think the speed, the latency, latency at same intelligence is probably something that is going to be very important, especially as you clear these hurdles of capability. Like, the thing can build a game, but can the thing build a game with you in the moment while keeping you in flow? I think that's, I think that's what's coming up next.

• 45:18Nathan Labenz?

  Still in our Dev Day conversation, I turned to Sign In with ChatGPT, which lets eligible users bring their plan to participating apps. My example comes from Waymark, the video company I co-founded, and the cost of letting a new customer try an AI product.

• 45:34Nathan Labenz?

  So the classic, try one, try seven days, whatever, those are tried and true tactics that have become very difficult in the context of, oh, but I got to spend a certain amount on tokens for every new user to give them that decent experience, especially if you have something that's like, involves a decent amount of setup or profile processing. With my company, Waymark, we're not by any means like the most token-hungry business, but the first thing that we do when you sign up is we make a big profile of your small business so that we can then use that profile as an input to make video content later. And that profile creation process, it's come down in price, I don't know exactly what it is today.

• 46:17Nathan Labenz?

  At one point, it was like a dollar per customer, and it was like, okay, well, this does start to become material when you think about the all-in cost of customer acquisition, and can we make this flywheel work, and how fast do we get paid back, and all that sort of stuff. But I think this is a, this is a great value add to your ChatGPT subscription, that you can now take it around, and of course, developers will need to implement this, but it shouldn't take too long to tell your coding agents to implement this. So that's great for ChatGPT, great for users, great for the SaaS companies. I think this is a huge win, and I do think it's like pretty pro ecosystem.

• 46:57Nathan Labenz?

  It seems to me that they, this is like one way in which they can actually follow through on the promise of like not trying to eat the world and instead trying to like empower people to build cool stuff.

• 47:12Nathan Labenz?

  Part three, the craft of co-writing. Joel Borgen is the author of The Receipt Horizon, a novel set in a world shaped by advanced AI, which he wrote in collaboration with AI models. We discussed what he supplied as the author and how the collaboration changed the work.

• 47:31Speaker 04

  I may have written the book eventually, but the activation energy that is required to get started, we have, you know, kids and a part-time job, was limiting. I mean, I started it well over a year ago, and the tools at the time were not as developed as they are now. There were a lot of limitations, especially around context length, and you have the thread where you're trying to work on a chapter or part of the book, and you're constantly having earlier context drop out and needing to manage that clearly. I think it's getting easier and easier to make use of the tools, to do something collaborative and still kind of your vision. I mean, I started the project knowing more or less what I wanted to have made.

• 48:13Speaker 04

  I sketched it out myself, especially the first half or so of the book was pretty well set before engaging the models. And then the models are good at certain things, they're getting better at everything, but as far as just prose writing itself, even if you tell it exactly what you want, and you have a plan for a chapter, you often get something that's unreadable unless you know how to scaffold it and then edit it yourself. So I've been working on a follow-up book, actually, and the process has changed quite a bit as, as the models have become more powerful. So the book is definitely a lot better for having done it alongside the AI, but I'm cognizant

• 48:54Speaker 04

  of the fact that, you know, AI writing will be controversial for a lot of readers, I don't want to engage with it, and I think the process made the novel a lot better, and it also had weaknesses as well that I didn't entirely like account for.

• 49:11Nathan Labenz?

  I had asked Joel about building a positive vision of the future into a story that still needs conflict, and about the scaffolding he gives the models.

• 49:20Speaker 04

  Utopias often want to become dystopias in narrative form because you need to have conflict and friction and it's kind of dull just to have everything work out and everybody's happy, of course. So the fact that this was set, you know, after a big AI war and humans are sort of presumably locked into what they're able to do, you see both incredible flourishing in terms of the technologies that's available and the options people have and also some like really major downsides. So I do think so much of what, you know, you've talked about and I agree is trying to paint like a positive vision for the future and that is

• 50:01Speaker 04

  difficult to do in like narrative form. So I didn't want any anything in the book or any character to be like a mouthpiece for me. It's more

• 50:11Speaker 04

  a lot of the stuff that I want to see created in the world is a part of the story and a lot of I present, you know, trying to steel man the people that have would have different opinions about that as well. So as far as the scaffolding, I often will will write out kind of a large scale architecture for the story and a lot of beats and then go back and forth with both the major frontier models. I mean, I've primarily used whatever the flagship Claude model is and then the ChatGPT Pro model from like five on because that was the most that was kind of the workhorse. It did the most kind of detailed work. Claude used to be a lot lazier as far as like how much it would follow up on. And I'm doing it mostly through the chat interface, which is probably

• 50:55Speaker 04

  has its advantages for me and also probably limitations. But, you know, context length, like I said earlier, was a huge unlock as that got longer. You could just, you could give the novel the entire book because at one point in drafting the first one and I would, you know, every time the new model come out, you you'd use that to to try to make it better and up your game and allow it to do more. But as far as the the actual scaffolding for the story, you kind of go in chunks often. And I'll I'll draft chapter by chapter. And then, you know, when I was done with it, I I had the advice to to hire professionals for copy editing and art and layout. And I did that and I learned a lot from doing it. It was very expensive and time consuming and kind of slow and

• 51:40Speaker 04

  there were definitely frustrations involved. But I've tried to kind of extract what I've learned from that and what made the book work better and I've, you know, I fed it to ChatGPT five or six now Pro and developed my own workflows for like doing the same process as the second book. So I think that'll be an interesting experience to see how all that works.

• 52:03Nathan Labenz?

  With Joel, we moved from drafting to judgment, where he still wants a human author making the choices and how he edits the prose.

• 52:11Speaker 04

  But as we move into a world when where the the AI systems can do a lot of what we do, a lot of what we thought was valuable, without was like a human contribution, you think about the abilities it has now, like the reason why I found the book valuable to write is that it was a lot of me in it. It took a lot of effort on my part still. I know you can get, I think I've heard a podcast episode where there was a somebody writing books on Amazon in the romance area and they were mostly automated and AI written. And I think there were something like a couple hundred per year. So they were playing kind of a scale game, you know, a few of them would make a few dollars and overall it was profitable.

• 52:53Speaker 04

  And you can certainly do that and the models are getting so much better that you can probably have customized artwork of the same quality or better than the book that I created over many, many, many months on demand. And then what happens to the art at that point? Thinking of it as an art consumer, if there was a new, a new movie by Imran Bergman or Stanley Kubrick or something in movies that I've spent many hours thinking about and consuming and are deeply moving to me, like what would that world look like where there's just a huge plethora of art on demand? And it's pretty much custom just to you. You know, we already have a shared dropping off of a cultural space, you know, people are more fragmented in what they consume and there's less

• 53:38Speaker 04

  unification, I guess, across the across the cultural landscape. So

• 53:45Speaker 04

  like, is that art still valuable at that point if it's if it's not something that we're sharing with other humans? I don't know the answer to that, but I think we're in a kind of a sweet spot now where in order to get, you know, a work that you're proud of, working with AI, it still requires a great deal of human judgment and work. And I don't know how long that will be the case, given kind of where things are headed. I could envision taking some of the

• 54:15Speaker 04

  the scaffolding I've, I've, I had the, I had my, my book edited, I edited it down about 14% or so, mostly manually and through AI helping me deciding what to cut. Often you can take a now a chapter that's so-so and actually just take away some of the the stuff that AI does particularly badly. The text, the over explanation and improve it in that regard. So I think you could, you could architect something now that that could create a pretty decent book with having the right kind of, we're kind of feedback loops.

• 54:50Speaker 04

  A lot of the the stuff that I've developed is to to take my own preferences where I want it to be be able to give the entire novel to a model and have it come back with like an analysis and a list of like, these are things you should consider. Because doing it manually slowly is extremely time consuming, as you can imagine. So bringing things to your attention like an assistant and saying like, here's something you could consider and then you can decide for yourself kind of point by point, I find is a fairly satisfying way to work.

• 55:22Nathan Labenz?

  Joel is also a musician. To describe a capability he had watched arrive, he turned from prose to musical notation and a viola clef test he had kept trying on successive models.

• 55:35Speaker 04

  I've had this held out test for well over a year now, maybe two years even. Very simple, a simple like high definition capture of five notes on the viola and I give it to every give it to the reasoning models and like, okay, what is the clef? What are the notes? What are the the note values and so forth, the time signature? And not a single one of them got it right until Astra OpenAI put in some, presumably put in some actual musical scores. And so it went from basically not being able to do anything to now I gave it a a complex, you know, 20th century score that's almost painful to look at, given how complex it is. I mean, this

• 56:20Speaker 04

  you know, double sharps and accidentals and ties everywhere. It's, it's like difficult and it did an almost perfect job turning it into a machine readable form via Codex. So the score that like you just basically had a PDF of before, now you can turn it into something that you can manipulate and evaluate. And so it went from kind of zero to 99 overnight. And I think that is an interesting way to turn some of the the scores that aren't machine readable into something that you could use to train a system. So I would love to see Suno move in the direction of

• 56:54Speaker 04

  adding a lot of classical stuff to it because I think in addition to it making interest in classical music and classical music will have it requires a different

• 57:07Speaker 04

  a different level of understanding of the form because it's much larger scale often, you know, like a three hour long opera that has some internal structure or even a 20 or 30 minute single movement or something has a lot more going on than a pop song. So I would love to see see that happen. Again, it does tie into the same

• 57:27Speaker 04

  element of if you're making just kind of private art, just mainly for yourself to listen to. I still think it's valuable, but again, like, like with writing a book, I would want to

• 57:39Speaker 04

  do it in tandem with the model, this hypothetical model in the future and create something that's, you know, a large part of my effort and taste goes into it as well. I don't know how long that that interregnum will last where you have a role for humans and AI to create something together or reuse it as a tool, but it still requires a lot of you and your judgment. But I look forward to experimenting with that once the the models improve.

• 58:11Nathan Labenz?

  Part four, the diagnoses still waiting.

• 58:16Nathan Labenz?

  Daniel McKinnon founded Gamow Labs to use AI agents to interpret genomes and revisit difficult unresolved cases. His son Owen died from a rare lung disease. A human specialist later found the genetic deletion the original sequencing analysis had missed. Daniel subsequently built an AI pipeline that rediscovered it while his family was seeking answers during another pregnancy. That experience helped lead him into this work.

• 58:45Speaker 05

  And it really leapt out at me last summer now when, I mean, we blessedly have a healthy 13 month old right now, but because of our history, we were being monitored very, very carefully. And there was just something a little bit questionable in the 16 week anatomy scan that our own maternal fetal medicine doctor, who's an absolute wonderful person, said, normally I wouldn't even flag this, but because it's you guys, we need to look carefully. And we did a whole genome of the fetus at that point. It came back negative. This is actually the third negative whole genome I've seen in between losing Owen and

• 59:26Speaker 05

  having our son Warren. We actually lost a second pregnancy due to genetic reasons very late. So, you know, we've had basically five years of heartbreak before bringing Warren into this world. And all of it kind of genetic mystery related. And so just as a patient, I've started to learn, or a family of patient, I started to learn a lot about like the failings of this system. And it was that moment where I said, I'm heartbroken, but I'm mad and I'm going to do something and I have tools to do something. Fast forward to last summer, you know, I got this result back. I said, this is unacceptable. This is the number one thing that I want in my life. I really wanted to have a family. I wanted to understand

• 1:00:08Speaker 05

  what would happen. I, you know, I didn't believe these labs. And I basically called all the labs and I said, give me my raw data, which you can get through our HIPAA rights here. And I, you know, vibe coded my own interpretation pipeline, which, I mean, now vibe coding is like crazy, right? Like right now, this would be so easy. You could just say Claude code, you'll make me this thing. But back then, I mean, it was still like, kind of like autocomplete original codex. Like it did actually require quite a bit of work on my end. And I just did it to see if there's any comfort that we could get around the pregnancy. And I was just shocked that it also outperformed on these other cases. And, you know, most clearly is it diagnosed Owen, when, you know, one of the

• 1:00:52Speaker 05

  best, or, you know, some might even say the best prenatal sequencing lab did not. And I knew at that moment that this is something I had to contribute to. I didn't know it would be a company.

• 1:01:05Nathan Labenz?

  The deletion in Owen was 91 kilobases long and removed an enhancer, a region that helps regulate a gene. The specialist had already identified it before Daniel recovered it with AI. We asked Daniel how the original analysis missed that enhancer.

• 1:01:22Speaker 05

  How you miss it is this enhancer is a mega base, 1 million bases upstream from the gene. And what Brady did, what many other labs do, which is a very reasonable assessment, is they have a filter that for structural variants, which tend to be quite messy in what's called next gen sequencing, which is, you know, how sequencing is done today, where you have basically roughly a billion 150 base pair fragments and you need to piece them into this clinical puzzle. They said, we're going to have a filter and anything more than one kilo base up or downstream of the gene, we're not going to consider. And this is a totally reasonable trade off if you have humans looking at all of this

• 1:02:04Speaker 05

  stuff. But my moment was, and this is before codex, before Claude code, was can you put the O3 model, which was the model I used at that point, kind of the first like real agentic model. There was O1, but O3 was really a first breakthrough. Can you put it in some kind of loop and just have it keep looking? And it basically looped and the first loop was like, is there anything wrong with the coding elements, which are kind of obvious things? And it's using these kind of bioinformatics tools. This is very crude compared to what we had today. But then it misses something and it goes through another loop and another loop and it just keeps working. And when you are a clinical lab, whether you're a profit, nonprofit, like whatever your structure, ultimately, you got to move, you know, you

• 1:02:47Speaker 05

  have to spend some amount of time on each case. And if you don't come to a conclusion, you know, you say this is non diagnostic. And this is very common. Most whole genomes, even with infants suspected type genetic disorders come back non diagnostic. And I really think it's one of these like meat space problems is if we can export these problems onto a machine intelligence that can work nonstop and in parallel, then we will be able to see many more kids, treat many more kids and do much more kind of interesting analysis on top of the basic things that human are just pressed for time. And I don't want to claim I've like reinvented this. Like people are trying to build software to accelerate genomic interpretation. People have been doing this for a long time. But I think

• 1:03:32Speaker 05

  what I probably relatively uniquely identified early was that this is a really great task for agentic AI. And from a, you know, improving on it, it's also, it's long time horizon, it's agentic, it's verifiable, and it's unsaturated. So I suspect that this will be a task like math, where we can just generate these like, you know, Olympiad level problems and just keep hill climbing on that until this problem is like basically solved.

• 1:04:02Nathan Labenz?

  Daniel McKinnon then turned to the unresolved cases his team is analyzing now. Sometimes the remaining obstacle is a variant whose effect is unknown. We asked where the new biological evidence would come from.

• 1:04:15Speaker 05

  And on the backside, once we've done the interpretation, the majority of clinical cases we see right now, and to be clear, we are only seeing hard cases. So this isn't like, you know, most generic labs. We are seeing cases that need reanalysis that did not previously have an answer. The majority of them end up in what's called a VUS state. So it's a variant of uncertain significance. And these are scored according to like a very standard rubric developed by the American College of Medical Genetics or ACMG. And you need to get six points or more to be bumped into the likely pathogenic category. And that unlocks a lot of treatment options, insurance options, like just let's say it's

• 1:04:57Speaker 05

  like good to be either benign or likely pathogenic. It's very bad to be in this kind of intermediate stage. And if you are in this intermediate stage in a VUS, you can do two things to get a diagnosis. One is you just wait and you can get more points if more patients emerge who have a similar phenotype or a similar condition as you and the same genetic variant. So this is often what happens when like you read in the news, you're like, oh, this kid has like had epilepsy for 10 years. It finally got a diagnosis. And if they're lucky, oh, and since we know that's a diagnosis, you know, we worked with pharmaceutical company and there's like some off label use of a drug and it actually can like help their condition. That's typically because they just waited until somebody else

• 1:05:40Speaker 05

  had the condition, but this is not scalable and takes a long time. Another fork is you can convince a university lab to kind of care about this problem. And what they'll do is they'll do what's called a functional study. So they'll make a cell line that's like emblematic of your particular condition. So for example, for alveolar capillary dysplasia that commonly use cell lines called IMR 90s, it's a fetal lung cell line. And this is very, very important because this particular gene is only expressed from week 16 to week 20 of development. Like you can't just put them in a cancer cell line. And then you can go in and you can do something like you can edit the genome or you can insert a plasmid or you can do a number of different functional studies to say, okay, in this

• 1:06:24Speaker 05

  physiologically relevant cell line, if I have this genetic mutation, is this gene broken in some way?

• 1:06:33Nathan Labenz?

  Daniel's team had just taken over an existing biology lab to test uncertain variants. He described the experiments starting that day and the feedback loop he hopes to build between real biology and the models making predictions.

• 1:06:47Speaker 05

  So we basically bought what was remaining of Arpegio, hired four people on the team and very quickly pivoted to creating this basically like R L data from real world with biology experiments. And we're three weeks in, we're running our first experiments actually today. And I'm really looking forward to being able to close that feedback loop. And I think a lot of clinical genetics labs are as well. This service is not commercially available. We can not only sell kind of like a lookup table version of this, like, oh, I have a patient with this variant. Can you help me get a couple more ACMG points to get this up to likely path? Because you can get two to four points. Remember, you only need six points. So two to four points is a lot of points for a functional study. But

• 1:07:30Speaker 05

  also it lets the machines learn. Right now, the machines aren't learning. Like you get to a VUS, it's like, that's the end. Now it's like, you get to a VUS, you do the biology study, you say, oh, this region of the genome is actually quite important for this particular disease. We're going to do that. We're going to check these predictions and then we can improve these predictions over time.

• 1:07:52Nathan Labenz?

  We had been discussing earlier genetic testing companies, including council. Daniel returned to the gap between sequencing a genome and interpreting it. The Scorsese sites are from Rare Bench, a benchmark his company built to prioritize genetic variants.

• 1:08:07Speaker 05

  I was like, oh, my God, we sequenced the genome 25 years ago. And you know, Bill Clinton and Tony Blair got up on stage and said, this is going to revolutionize human medicine. And I look back and there's this like graveyard of genomics companies. Like, you know, you mentioned council, which I actually would not say is a graveyard. I think they were like, you know, modestly successful. But like, why is all of healthcare not based on precision medicine? Like, everyone in this space is like, it should be, and there are examples. It's just too challenging. The interpretation is too challenging. So I actually look back and I see, and you know, we kind of talked about council ahead of time, is I think that what was missing prior to now was really that interpretation layer. It's very, very complex.

• 1:08:51Speaker 05

  Like, you know, genome sequencing has gotten what I would call cheap enough, maybe five years ago. I mean, below 1,000 a genome, maybe 500 a genome. There are high throughput labs that are doing this for less than 100 genome. You see announcements on Twitter saying, oh, I can do a genome for less than $100. There's many people doing this right now. Like, the sequencing is not the problem. It is, given your problem, what insights can you derive from that? And that was only possible as of like last summer. Like, I'd say 03 is the first example of that. And it's not just the variant interpretation, right? It's explaining to the provider why it's important. It's ranking variants in a nice way. It's explaining, you know, how can you treat this person? It is ranking different patients for ASO eligibility.

• 1:09:35Speaker 05

  There are many, many, many other things beyond just scoring the variants. Although I will say, we benchmark a variant annotation in Rare Bench and other benchmarks we have. And the best performing, like, I'd say, traditional machine learning based approach in terms of variant ranking is called Lyrical. I think it scores something like 10% on our benchmark. And right now, OPUS 5.5 in Claude Code, just a vanilla thing, scores something like 50%. So there's, these traditional tools are also getting blown out of the water by, you know, this newer approach. But then they can do much more. And you basically need to make a very, very cheap, easy thing that people who are not

• 1:10:16Speaker 05

  surrounded by fancy clinical geneticists, genetic counselors, you know, top-tier hospitals can use. And that's the problem that we're trying to solve.

• 1:10:28Nathan Labenz?

  Daniel distinguished screening a healthy person for possible future problems from investigating an illness that is already present. His team focuses on diagnostics and reanalysis of cases that have gone unresolved.

• 1:10:42Speaker 05

  There's already product market fit for sequencing. Every baby in top NICUs is getting sequenced. Every baby in lower-tier NICUs would get sequenced if they had the resources. And the problem is much better scoped. It's baby has pulmonary hypertension, figure out why. Not, is there something obscure that could be wrong with this baby now or in the future? And that is where we are working with, you know, various hospitals, various families today. This is, there's two forks of this work. One is what I call online. And this is like, case comes in, physicians do not get good results from the traditional labs. Patient consents for reanalysis, they send the data to us, we reanalyze it.

• 1:11:25Speaker 05

  And sometimes, but not always, we find, you know, additional things that can help them make decisions around either pregnancy or a newborn. The second fork is bulk-scale reanalysis. So this is rare disease centers that have thousands of genomes. And they say, there's probably kids that we can diagnose or even treat in that database, but we have a handful of genetic counselors, a handful of bioinformaticians, and we just don't have the resources. And because it's a machine that does most of the work here, you know, we can get a set of 80 or 100 or 200 or whatever, and turn them around in a week. And that's not, this is kind of six months or a year's work that they really can't prioritize. So those are the two forks right now.

• 1:12:10Speaker 05

  And I mean, we're four months in, we're pre-revenue, we're really not thinking about what is the, like, exactly appropriate business model here? We're just trying to diagnose more sick kids.

• 1:12:20Nathan Labenz?

  For those cases, Daniel described what his team adds around the underlying models: the tools, the routing between models, and the evaluations of the whole system.

• 1:12:31Speaker 05

  And so then, you know, stepping one step up is like, what do we do? I mean, we're basically a harness, a tools and an evals company. And I think that you'll see this a lot in vertical AI in general. You know, I've worked in these frontier labs. I started working on LMS, actually the first LM project at Meta, which was OPT175. And unless you are somebody who is very famous with very deep pockets, I don't want to bet against the Neo Labs by any stretch, I think it's possible, you know, you're not going to compete at the model layer. And the intelligence is progressing so fast. You know, we ensemble models, for sure. And that helps with both cost and it helps with performance. If you look at the confusion matrix of Rare Bench, we also published this, you'll see

• 1:13:16Speaker 05

  that the cases kind of cluster and like, Claude is good at this, and Astra is good at this, and even Gemini, right? Gemini is not on the frontier, but they're kind of good at this. And you can kind of ensemble these things together and get better performance. So I'd say that's like one thing we do. And ensembleing, there's like very dumb ways of doing this, but like, there's also smart ways of doing this. And I don't have to go into all the details here, but like, I would say figuring out, you know, unique routing and intelligence per model is an important edge that I think we and many others are doing. And then I think another key thing that people kind of forget about, and this is what I worked on both at Meta and at Google, is evals, is unless you have very structured ways to measure the performance of the system, it's not just the model at this point, it's the whole system, you don't know it's a hill climb.

• 1:13:59Speaker 05

  So we do like pretty robust like trace analysis of how models perform with and without our harnesses. And it's, I mean, this sounds stupid, but like, not that many people actually look that closely at data. Like, this is a meme on Twitter, but it's very true. And you'll see things like, there's one particular case where Grok 4.6, which at that point was state-of-the-art on our benchmark in Grok build, where it missed one because it just renamed the gene. It just like was saying like, oh, you know, N R F 2 or whatever is responsible. And then it just changed the name of the gene to something totally different. I'd never seen that before. And I was just like, this is dumb. And our harness and our tools prevent the agents from doing dumb things.

• 1:14:44Speaker 05

  I think there is a world where if I worked at OpenAI or Anthropic and had access to infinite compute, and I could do millions of rollouts, that the agents would, with enough compute, kind of like all converge on like some answer and like maybe mitigate all of this work, especially because they can write their own tools these days and everything. But really, it's like consistently and repeatedly and, you know, honestly, affordably getting to the right answer. I mean, for $10 a case, like, who cares? Or even $100 a case? But if you would need to do a million rollouts and all of a sudden this is, you know, $1,000 a case or $10,000 a case, it doesn't make sense. So we're doing a lot more research around like how those, you know, cost curves work and how everything works.

• 1:15:26Speaker 05

  But ultimately, our job is to take the smartest intelligences in the world and mix them together and give them access to the best tools and then get answers for our patients. And that's like the core of what we do. And be able to measure if it's working. So I'd say that's like the core of what we do.

• 1:15:43Speaker 03

  So would you say, like, when you talk about vertical AI, you are looking at benchmarks where you combine an intelligence with the tools that you have? How do you evaluate the strength of the tools on their own, you know, excluding, because you're going to upgrade model families over time, right? How do you, how do you measure the performance of what you're building, that layer in between in particular?

• 1:16:09Speaker 05

  Yeah, that's a good question. And I don't, I honestly don't have like a great answer to that question in that we are co-designing the tools with the models. And we have seen cases where, I mean, this is one, like, again, it's a very dumb example. And if you're getting deep into this and you have a specific eval, you will start to see this. Like, if you're not deep, it's very easy to be like, oh, I just had Claude Code like one-shot this video game. Was it good? Like, I don't really know. Like, it seems amazing. And I don't want to communicate that these models aren't amazing. They absolutely are. But with one tool, it was around a specific type of literature search. And the model, I think this was Astra, actually. Actually, here's a good example for Astra after this.

• 1:16:54Speaker 05

  But Astra said, oh, yeah, okay, these are all the papers you tagged for me to read. And then it got, the answer was in the paper and it didn't, it missed it. And then we're like looking through the trace and we're like looking through the context window. And it's like, I don't think you actually read these papers. Like, there's lots of just like little things like that, and it's little paper cuts. And that's what these tools do is it's like, oh, you missed a case here because of this, you missed a case here because of this. And you kind of put it on these better rails. And like, actually, Astra is a good example. The first time we ran the eval for Astra, it just stopped because some of the tools we had, it just like wanted to think about too much. It's almost one of these things, like you see this on X where people are like, oh, I asked the model to do some research for me and I came back and 30 minutes later, it's

• 1:17:36Speaker 05

  like looking at flights from, you know, Dubai to Calcutta on, you know, Emirates airline. Like, why are you looking? And they just kind of do stuff like this. So I think that, you know, we try to co-evolve the tools with the models and we try, you know, we have like, you know, coming back to something that many people might find boring, but I find fascinating, is, you know, evaluating a model is very hard. And we need to just have very robust evals that can catch these things early. And in that case, we caught it early and we updated how Astra called some of the tools and it works again.

• 1:18:09Nathan Labenz?

  Prakash asked Daniel what he most wants the Frontier Labs to improve.

• 1:18:14Speaker 05

  Well, I want them to hill climb my task. Right? Everyone wins when these models get really, really good at clinical genetics is probably we have less to do at the harness layer, which, to be honest, I'm like both bullish and bearish vertical AI in that I think that there's like lots of value in kind of owning a customer relationship. I think open evidence has shown this, but there's also like that layer is getting thinner. When I first started this, like vanilla O3 would not do this task. In fact, O3 scores 0% on my benchmark. I re-benchmarked it just for fun. So it was like, oh, I needed to do all this work and scaffolding to get these things to work. And now like, you know, there's like your 20, 30 percentage points or something.

• 1:18:54Speaker 05

  I mean, it's definitely like shrinking. So, but that said, like my, my goal is to build, you know, this like great AI native clinical diagnostics company, and I want the best person to do the work.

• 1:19:08Nathan Labenz?

  Prakash also asked how Daniel hopes to make those biology experiments cheap enough to run at scale.

• 1:19:14Speaker 05

  But it's, it's really an AI and robotics thing is you have an arm there that is doing 384 experiments at a time in, you know, a bigger plate. You have a cartridge over there with thousands of plates lined up. You have AI that's assisting with primer design and experimental design. And actually, this is one of these things where you say like, oh, you know, can you use AI for this stuff? You have to get special permission, but you can. And then, you know, at very high throughput, you can design, order the primers, design the experiments and run the experiments. And then it comes to analysis as well, is you end up with like very large scale data sets that you need AI to poke through. So it's not, I don't have some like perfect explanation. Like there's just this one invention, but it's a consequence of all those things.

• 1:19:59Nathan Labenz?

  Daniel also described the response to his diagnostic work from people at the frontier model companies. Here he reports new diagnoses in other children whose cases had been unresolved.

• 1:20:11Nathan Labenz?

  What was the process of talking to frontier model companies about your bio use case experience like?

• 1:20:21Speaker 05

  It was really positive for me because it's extremely obvious why I'm doing this. And I've never met anyone who is like, why are you doing this? Every single person I've met at any of these labs has been like, I want to help. Whether wanting to help as a person translates into like, I can pivot this trillion dollar company to work on your problem is like a different thing. And I understand that I've worked at these trillion dollar companies too. But it's been incredibly supportive and I would be pretty surprised if at least the top frontier labs did not spend some amount of time on this problem.

• 1:21:03Speaker 05

  Because again, it's meaningful and it's good. And, you know, even selfishly for them, there's a really negative AI narrative swirling around right now. And you hear things like in Dario's recent tweet where he's like, oh, we're going to cure cancer in five years. We're diagnosing kids today. We have many examples of kids who are undiagnosed, who we have diagnosed at a small company four months in. And that's a great story and it should be told by them, even if, you know, even if it's just for selfish reasons like, hey, people of the United States of America who are not happy with data center build outs or uncomfortable with AI taking jobs, here's a very concrete way where we are helping people today. And then I also think it comes from the individuals is like people find

• 1:21:46Speaker 05

  meaning in their work. And I think that there are a lot of people, especially like mid-career in tech who are like, I've been doing some kind of ads ranking, data munching, whatever for my whole career. And what you're telling me is that you have a very concrete way where you can help kids and I want to do that. And so I think that there's like the combination of like the business reasons and the personal reasons that are driving people to this task.

• 1:22:14Nathan Labenz?

  After that discussion with Daniel, I came back to what slower AI progress could mean for families still waiting for answers. The 50% figure I refer to is a score on Rare Bench, the company's variant prioritization benchmark, rather than the share of patients diagnosed.

• 1:22:33Nathan Labenz?

  Intellectual honesty demands recognition that this is probably still one of the things that we will be trading off when we pace the frontier in the near term, to whatever extent that actually happens. And I still think that's probably a good idea and probably worth it. I don't think the frontier model companies are at risk of like not being able to grow business, not being able to grow revenue, becoming surpassed if they pace their frontier efforts. But it is important to keep in mind that there are very real problems in the world that AI is like climbing that hill on now, but has not finished climbing

• 1:23:14Nathan Labenz?

  on. And this is one, you know, obviously that everybody would love to see solved sooner rather than later. And it's important for me to stay honest with myself and everybody else that like, that is a, you know, very real cost when you're talking individual people and their families. And it's that 50% number that Claude gets to today, you know, that's like, obviously leaves half of the questions unanswered. And those questions are extremely meaningful to people. So I, I do think we shouldn't take that lightly, even if it is a, a trade off that on balance, I think probably we should be willing to

• 1:23:56Nathan Labenz?

  make some compromises on. It's, it's a, it's a costly compromise.

• 1:24:02Nathan Labenz?

  Thank you for listening. Please tell us what worked and what did not. See you in the morning.

• 1:27:18Nathan Labenz?

  If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries, either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts which is now part of A16Z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at ai podcast

• 1:28:01Nathan Labenz?

  dot I N G. And thank you to everyone who listens for being part of the Cognitive Revolution.