How Anthropic Uses Claude Fable 5 With Mike Krieger

Mike Krieger built one of the most consequential consumer apps of the last two decades as the cofounder of Instagram. He is now at the frontier of AI-native product development as head of Anthropic Labs, the team responsible for figuring out what the most capable AI models can do in the hands of real builders. When Krieger first got access to Fable 5 months before its public release, it was exciting and disorienting. “I feel like a total newbie again,” he remembers telling his team. The way he’d been thinking about productivity, strategy, and time management was out of date. The model had outpaced his workflows. Dan Shipper talked with Krieger for AI & I about what it looks like to build with a model as capable as Fable 5, including the new rhythms, challenges, and possibilities it reveals. If you found this episode interesting, please like, subscribe, comment, and share! To hear more from Dan Shipper: Subscribe to Every: https://every.to/subscribe Follow him on X: https://twitter.com/danshipper Get started with Braintrust at https://www.braintrust.dev/ Timestamps: 0:03 Introduction 1:48 How Fable completely reshaped Mike's workflow 4:48 When to use Sonnet versus Fable 10:06 What the media tracker Mike built over a weekend reveals about agent-native architecture 15:00 The cost to build has collapsed 19:03 Is software engineering over? 21:48 How Anthropic's engineering teams work today 38:39 The mechanics of verification 44:39 What people should use the model to build 47:24 Dynamic workflows Links to resources mentioned in the episode: Mike Krieger on X: https://x.com/mikeyk Anthropic Labs: https://www.anthropic.com Claude Code: https://claude.ai/code Every: https://every.to

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:04Dan Shipper?

  Mike, welcome to the show.

• 0:05Speaker 01

  Great to be here, Dan. Good to see you.

• 0:07Dan Shipper?

  So for people who don't know you, you're the head of Anthropic Labs and you're the co-founder of Instagram. And today, what I want to talk to you about is Fable 5. So Fable 5 is dropping tomorrow, recording this the day before. It's going to come out after it drops. But what I really wanted to do is bring you on the show to tell me about what it's like to use this model beyond the first day. I think when a model this powerful drops, it's so useful to have someone who's using it day in and day out to tell you, this is where it's powerful, this is what it actually changes, this is what it doesn't change. So that you're, you kind of like, don't, you kind of don't get the same AI psychosis type thing. You can actually think about, okay, like, this is how it fits into my life.

• 0:43Speaker 01

  Yeah, absolutely. And it's also just been interesting. You know, we've had some, you know, models in this, you know, mythos class leading up to the Fable release, you know, for a couple of months now. And I think it's very exciting to see how people will build with this externally. But I think you're also right that day one impressions, I think it really comes from getting to use this over a couple of weeks. I think we've seen that even with previous models, like the December into January usage. This was Opus 4.5 or Opus 4.6 was really important because people spend extended time on the model and then figure it out. Oh, actually, I wasn't pushing it hard enough. I gotta go further. I gotta rethink what's even possible with this generation.

• 1:21Dan Shipper?

  Totally. I mean, I don't know. I feel like there are people internally at Every who have been using it, who have been like, oh, my God, I think I kind of need a new set of skills to use this model. And I think you can especially see this with people who are maybe more non-technical internally and who are more on the knowledge work side of things where they're like, I don't even know what I would use this for. And the people who are orchestrating agents are like, holy shit, I feel like there's so many new things I need to learn. So I'm curious for you, tell us about the difference between your impression when you first tried it and now.

• 1:47Speaker 01

  Yeah, I think that your point on adapting workflows is a really good one. Quite literally workflows, I'll talk about that in a second, but also just in terms of like, how do I like, think about usage of the model? Because, you know, first, the timing was interesting because it kind of coincided with me transitioning from CPO into labs and going really back into builder mode. And I think it was about a month and a half. Two months into that, that we first had, you know, one of these models available internally. And I I sat there and I and I was like, I feel like a total newbie again because I feel like the way that I am prompting or even thinking about decomposing a task is really out of date now with this model like.

• 2:28Speaker 01

  It's no longer and it's even thinking about the time horizon or the sort of like interactivity model, I think, has to evolve as well. Like going from, I think, early on would be like, I have an idea for this feature. Can we start by do like, absolutely not, right? To great, like, let me express more of the intent. And then just being, you know, I remember like, you know, you know, Roger April be like, wow, on the one shot, it's already incredibly impressive, but then it also understands the intent around how we're going to evolve this and understands like the global context as well. So I think that's been a really interesting evolution till now where, you know, I was funny. I was talking to somebody this morning where, you know, I think about doing work.

• 3:11Speaker 01

  I had a flight and I was like, okay, I can do most of this work remotely. And I don't even worry that like the Wi Fi is going to drop out because I know that if I set up the right, you know, context instructions like flash loop, you know, I'll see it, it'll see it through. And and I think my last two months have been full of a lot of times where I will, you know, wish Claude a good night, set it up on like a pretty complex task or something of this like model class and wake up to, you know, actually, it's usually done by like two in the morning and I guess it just foots its thumbs for the next four hours. But like, really impressive ability to like complete the swing, get itself out of the situation where it's like, okay. All right, well, Mike asked me to do this complex task overnight. I got stuck because this remote service went down.

• 3:56Speaker 01

  I'm going to write a like scaffolded like backend for it for now. So, you know, I'll document that, you know, go all the way through. I have a like good mental model of like how far that's going to get me. And then when it comes back online, I'll fix it. I'll keep track of that fact. It's just like. It is, I think, the most impressive thing for me is like you're just be able to like delegate that kind of level of task and just trust that the right thing will happen by the end. And of course, like they'll review the result and there's still like a whole verification thing that we should, we can and should talk about because I think that's an important part of still completing the, the swing there. But it's really forced me to rethink, like, what does being productive with one of these models look like? And it is much more like, we've talked for a while about, you know, like, what is it like when these models are more of like a companion or a coworker?

• 4:41Speaker 01

  And it really feels like now it's like a teammate that I can delegate like a lot of work to.

• 4:46Dan Shipper?

  And what is your, what is your day to day flow like right now? Because one of the things I noticed is if you, if you just give it a big task and you monologue into it and you just like let it go for a few hours overnight, it's like the most impressive model that I've ever tried. But, you know, it's so slow and it's so expensive that you, I, I feel like I don't want to use it for day to day tasks. So what is your actual flow like in terms of how you use it day to day? And where does it slot in versus other models?

• 5:10Speaker 01

  Yeah, I've ended up having a lot more architectural planning conversations upfront with it as well. So that's been like another interesting change where I think this is an era that I think all models need to continue to improve. And I'm really grateful for the Instagram experience of having to like start, you know, from our initial version that was like duct taped on a server in LA to like being able to scale it and eventually integrate it with like all of like the Facebook infrastructure. Because you kind of develop a sense of what, what infra abstractions and complexity are appropriate for each stage of it. And I still don't always go back and forth with Fable where it'll be like, oh, this is a good, you know, implementation. Like, well, I do plan on shipping this like fairly soon. Like, I think we should probably think about more than one server and like kind of like

• 5:55Speaker 01

  that back and forth is important. But like a lot of that sort of planning and often I actually ask it, it's kind of a, the thing I've realized is Fable can like be so sort of.

• 6:09Speaker 01

  Sort of complete in its thinking in terms of how much you are sort of planning with it. Often just saying, can you just like make an HTML page like that represents what we just talked about? So I can share it with the team is actually valuable or even just a markdown document. But I like having diagrams. So that's been an interesting like use of like, let's plan with it. Let's think it through and then let's have some sort of document that we can align the team on. Because and this is a dynamic I've seen in labs and just teams beyond Anthropic, which is you can build a lot very quickly. And forcing more of that early alignment, even if you do an initial prototype and then back it out into more of a sort of plan architecture that works too. I think is really, really, really key.

• 6:49Speaker 01

  And it actually ends up being the place for like the human to human interaction still stays very, you know, very much part of the process. And then from then on, I think, you know, either overnight or during the, the, the day, like having it execute on those chunks of tasks is really important. And it just means having a lot more concurrent sessions than I did before, because I often will think, all right, there's, there's these few pieces of work. I go back and forth between liking having one like very long running Claude code session and really asking it to do everything in background. Sort of fork sub agents so the main thread stays responsive. And then other times just embracing like, I'm just gonna, it's one of those days where I'm gonna have like five or six tabs like tackle like long, comprehensive work.

• 7:29Speaker 01

  But I do think that there is something to this like long horizon and like, uh, don't, you know, don't worry, I'm, I'm on it. It's gonna take me a while and like more of like this back and forth. And that, that modality, I think is something that we'll have to figure out in our products as well. I think you want to preserve both and they, they interact with each other in interesting ways. And like my preference is usually, I always like having at least one Claude that is high context, but also very, very fast response. And like its instinct is, right, I'm gonna answer you and I'll kick something off if I need to. And if not, I'm just gonna, you know, hang tight and, and, and wait for the next kind of loop. I do think you're right that for the, I'm just trying to fix this, you know. Interaction question or something that's like very fine detailed. Like Fable will go off and think very hard about those things.

• 8:14Speaker 01

  And I think Fable is the first model where I've actually played more with the effort levels for that reason. Where I've been like, okay, this is, I just needed to like tweak some UI. I'm like, you're gonna fall but like no, put it to medium or something and see how that plays out. Didn't find myself doing that as much with Opus maybe because the range felt less like wide. Where it really can feel quite wide with Fable.

• 8:34Dan Shipper?

  What about like a quick question? Like you're, you're on the go. Like, are you asking Fable, you know, random questions as they, as they come to you? Cause it feels like you're using a rocket launcher to kill a mosquito or something? Or are you flipping back and forth?

• 8:47Speaker 01

  It's so funny you ask that because I had been and you know, you're like, it's thinking. It's thinking really hard about it. Then since last week, I was like, no, I was asking it something that like true. I felt embarrassed actually asking Fable about it. It was something like, probably something NBA finals related. And I was like, okay, I switched my iOS app to, to Sonnet. I was like, oh yeah, I use this all the time for fast question. It's like counter of magnitude, like in feeling of like, and it's actually not even the, the sort of like tokens per second. It's actually probably more around how much thinking goes into the answer. And sometimes like the answer does not need to be fully thought through. So yeah, I, I am, I'm thinking myself through it.

• 9:26Speaker 01

  And I think this is a good product question for us too, which is, um, you know, in general, you don't want people to have to be thinking so much about these choices. So ideally what we can sort of coalesce around in the longer run is sort of, you know, maybe like some more bucketable use cases that are really grokable to people. Or maybe it varies by surface where it's actually probably unlikely that most of the time with the iOS app, I'm doing Fable type tasks. And you know, having a sticky model selection per surface might be the way to do that. And we'll have to sort of explore what that means from a product perspective. But I've for sure, I've had the feeling of like, this, this is not a stable worthy question. I am, I should ask Sonnet this.

• 10:02Dan Shipper?

  Can you show us something that you've built with it?

• 10:05Speaker 01

  Yeah, so one of the things that we were, we did this go around is we encouraged personal sort of like account usage for us, like especially on the weekends, which was really fun because, you know, we have, you know, you can imagine like a lot of Anthropic specific, you know, tooling, et cetera, but it was really good to sort of step back and be like, I'm just like, you know, pure Claude code, let's like work on something over the weekend.

• 10:28Dan Shipper?

  And you're in, you're in the terminal app or you're in the desktop app?

• 10:31Speaker 01

  That's a great question. I'm mostly still in the terminal app. It's interesting watching my wife, who's like not a professional engineer and more of a UX designer, PM, like really fall in love with Claude code via the desktop app. And I think it's like sort of simplified some of the abstractions for her in that way. But for this one, I was still, is it Ghosty or Ghostty? Ghosty and, and the terminal app. But let me show you, I am, this is one of those like, everybody has some bespoke need around this. Like I wanted a sort of media tracker experience. And I was like, you know, I'm playing games, like I'm watching TV shows, I get all these recommendations. And I just wanted to build something like that was personal to me and like sort of fit some of the use cases that

• 11:14Speaker 01

  I, that I had. And like, I like the two biggest criteria that I started with was like, one, like really easy to add things. And so like you can talk to Claude, Claude does the Gentic search over everything and then puts the right things in. And then also proactively like, you know, there's a new season or a new like sequel to a game that I could go off and, and research those things. Most of the year I was like, you know, Fable one-shot, which was, which was already impressive. But then the, the thread I've been pulling out a lot in labs this year is how do you sort of bring the software team, which is Claude these days, closer to the software itself. And so this was like maybe, you know, Saturday morning, I had a full weekend with, with kid stuff. So a lot of this was sort of kickoff work, go do, you know, go for a hike with the kids, come back, you know, continue to do the work.

• 11:59Speaker 01

  Sometimes check in on the work on the hike. I probably shouldn't, but you know, it was like nice to like pop into the remote mode and see what was going on there. You know, uh, try not to do that too much. Um, but I had this idea around, hey, like, uh, could you, could we like do a spike on, I say spike a lot with, with these models. They're like, can we do a spike on like, what if you could actually modify the software from within itself, which is, you know, uh, and it was, I built both. It was like a React native version and then this version, which is just the one version. Um, so I already had like a chat type thing where you could sort of ask Claude to, you know, add things by URL, which is like, you know, I want every software to have this where I should never have to like navigate a menu to do anything ever again.

• 12:37Speaker 01

  And this is like, in many ways, Dan, like the, I was trying to distill the like agent native architectures to like, it's like fullest degree, which is like, also have the agent be able to modify the app. So like maybe like phase one of agentated architecture, like every single thing in, in, in this product is, uh, you know, accessible from the agent and, and, and like has tool calls, et cetera. That's like, you know, hopefully becoming cable sticks, although sadly not, uh, in a lot of software. And it's great because I was like, what's that like, somebody had recommended, there's a Brazilian, there's like a show about radioactive stuff in Go. And I did not remember what it was called and Claude was able to figure it out. It was like so much better than being like trying to figure that out intuitively. But then the next step I was interested in is like, what would it mean to actually be able to modify the software from itself on the go?

• 13:17Speaker 01

  And so if you long press this little chat thing, so what it actually, what I built, Claude built, um, was a way where it used, uh, uses our managed agents, uh, to basically take on like edit requests. And then you can preview them and they use like the Rosell life preview thing here. Um, this like this whole like feature was also one shot, which was really cool. Um, and I just added to it over time. Um, but you know, it was like, it actually does like a little diff if you wanted to, you can go into the managed agent conversation and see like what it did, although I almost never do because again, it's like, especially don't particularly care on like the code quality of like, or the like long-term maintainability of this software. You can see that it had a session in here, um, too, but it's been really fun.

• 13:58Speaker 01

  So I'll be using it on the go and say like, you know, I had a feature request the other day, like, um, oh, like the floating action button was too low on native iOS, but it was okay on, on there. Like, can you go up and do it? It did it. Um, it was really fun, uh, with some of the like expo tooling now and actually like live reloaded on my phone, which was also like a really cool, uh, kind of, kind of feeling. But it was just like.

• 14:21Speaker 01

  You know, does this thing need to be like a, you know, production level thing that's going to go to millions? No, but it felt really good to have something where I felt like it didn't have to stop at just the weekend and I could keep working on it just by using it and having this like kind of end to end close thing. So I felt like this was a good manifestation of both like Fable's building ability, but also like, I think a lot of, both of I've been thinking, both you and I have been thinking about like, how does Claude embed itself and like into software beyond just even the usage side of things?

• 14:49Dan Shipper?

  This is really cool and I want, I want people to understand like, so this has been built, you could build something like this, maybe not the self-modifying part, but you could build something like this for like 10 years or 20 years or something like that. But the, the cost to build has like gotten dramatically lower. So think about how much it would have cost to do this in the Instagram days versus now. Like, can you help us understand like how that has changed?

• 15:16Speaker 01

  Yeah, I think, and I think about this a lot when I think back to that, that time as well. Because, you know, I, I thought of myself as a very productive programmer in the early Instagram days. You know, I was like really into mobile development and we had like a good clarity of, of things. And I think the, the gap from idea to fully realized version of like some complete product, like you were still looking at, you know, four-ish days of kind of my all-nighters, which was like my natural state is up till four, you know, sleep until noon. It's not, it's simple to family life, so I've had to shift. But that was like my, my building thing. But yeah, I call it, you know, Instagram V1, which, you know, probably had more features than, than this thing did, but not by an order of magnitude, was like five days of all-nighters,

• 15:57Speaker 01

  me working on like the sort of front end and backend and Kevin working on the initial filters to get that, that out. And, and this was also like, you know, like built on already, you know, many years that I've been working on, on, on iOS pieces as well. And then the iteration, you know, I think a lot about what we were gated on after that launch, when things went well was we had all these ideas for where to take it, but we were just trying to keep the site up or we were just trying to like add the one incremental feature and, you know, hashtags take a week to build. But then there's like all the things that you want to continue doing on it as well. And so I think it's both that shortening of time, like there's still the time required for the idea and the the concept and the iteration. And then the other piece, which is the

• 16:41Speaker 01

  you can then iterate on what you have. And I think a really, I think really fun, but also like very, you know, sort of in the flow kind of way. And then, you know, if now this is me as a sort of professional software engineer, sort of startup founder. Beyond that, if you had that idea, you know, and I saw multiple people go through this, like, is like, well, I'll try to find maybe consultancy that will take this on. But like now there's like, it's a really lossy process of like what I wanted. You know? Yeah.

• 17:11Speaker 01

  So you're never gonna go raise money for it. And I think that the thing that I think is like the most exciting part about these models getting not just more autonomous, but again, closing that gap between intent and execution is what I've seen it do to people's ability to build or not, like builders and

• 17:33Speaker 01

  the trajectory of these models has been, you know, if something Fable, you know, of this general mythos class is like in that class of models and eventually, you know, models of, you know, that are cheaper and more accessible to to other folks become available too. And like as that process happens, like I just think it is just opening up so many, like I got a thing the other day. I get very excited about the stuff if you can't tell from somebody internally and we had built them an internal tool that kind of combined Fable and like access to some internal MCPs. And she said, like, it is the first time in my life, and she works in recruiting. And she's like, the first time in life where like I feel like the thing that's in my head and the thing that exists in the world is now like they're right next to each other. Like I can just do it. And

• 18:18Speaker 01

  it was like very like a meaningful moment to her because prior to that, like I remember these days, these days were five years ago or four years ago where that person, if they wanted a tool, would have to either make do or try to get an internal tools engineer that probably was overloaded with 50 other requirements. But instead now they are just having the time of their lives building. And I think that is, I think that's caused for a lot of hope because I don't think that human capacity for creativity and what's possible is enormous. And I think at our best, we are basically expanding the number of people who can then see that through to something that feels real.

• 18:52Dan Shipper?

  I totally agree, but I do think that there's a question in the back of my mind, and I think it's probably going to be in the back of the minds of some of the people listening. So I want to ask you, given everything you just said, is software engineering over?

• 19:04Speaker 01

  Yeah, I think software engineering is different. It is dramatically changed. And as I probably would have defined it if you had asked me around the Instagram time, like, what is software engineering? I'd probably say like, alright, like thinking through the hard problems and like thinking about an architecture and then like spending a lot of time in, you know, like Textmate, I don't know what that can be, but like, you know, like text editor, you're going to edit this and, or Xcode, you know.

• 19:29Dan Shipper?

  Watching rail casts, you know.

• 19:31Speaker 01

  Yeah, exactly, right, exactly. And understanding the intricacies of Django's, like, ORM layer and then like fixing bugs after you deploy it. Like, so much of that is radically different and collapsing into other parts of like product management. And I think that sort of like PM and split, I think I see it even in our teams, has become much more diffuse. That's radically changed, but I think the overall, like, maybe zoom out from software engineering and think about like software production or, you know, software development, but not in like just the pure developer case. I think that is like alive and well and essential still. So I think that that is the moment that I feel like we are in.

• 20:14Speaker 01

  I think Fable is another step on the direction of, and I'm not going to call it the final step, of course, a lot will still happen, but like I think a pretty significant step in terms of like. The trust, at least I end up placing the model in terms of its capacity to see things through and even, you know, architect things reasonably is quite high. So that part feels like it is, it is not ever going to be done, but it is pretty, pretty done, right? Like it's gone really far. But I think that the overall sort of craft of the what needs you have, like what are you putting out? Like, is it actually good? I think still a very human endeavor. But I also sort of can see that that is not a transition that is a sort of pain free in a way.

• 20:59Speaker 01

  Like, I think there are plenty of people who love the craft of like actually putting, and I used to love stuff like, I solved that problem so elegantly, you would dream about code. And if you're at the experience of like, you would dream about the thing that you're working on. They like, uh, wake up in the morning, like, I figured out how to solve this thing really elegantly. And, and that for sure has, has, has passed. And I think that there's, you know, there, there's, there is a, a feeling of loss, I think, in some of the like better engineers that I talked to, as well as the feeling of, oh my God, but I can do insane amounts of work now at the same time. So we're holding both ideas in our heads at once, I guess.

• 21:31Dan Shipper?

  Which I think is the most important part of this. Like, it's normal to feel sadness for that kind of thing and excitement. But I'm curious, let's just take the thesis of software engineering is alive and well. What does that actually look like inside of Anthropic?

• 21:47Speaker 01

  Yeah, I think there's, there's a few thesis. I think there's still the, the crafting of, well, I gotta take it off from like the full software development cycle. Or like maybe what I see on a day to day, maybe I'll do a little bit of both. But I think there's still a lot of, uh, you know, we all got together. We, we talked about the next way we want to, you know, evolve cowork. And now we've kind of broken it down into areas of ownership. I think that ends up still being quite important because there is still context that you hold as a person that is sort of beyond Claude, right? Like, what is the actual intent of this product? How's it going? What do we need to know about the sort of other products that are coming down the pipeline? That are going to be integrated in some interesting way? So I think that aspect is really important still.

• 22:31Speaker 01

  And so, you know, though we have many Clods to each human. And each human, at least the way we've been working on topic, still kind of has, you know, we call them DRIs, like directly responsible individuals, still has like a DRI ship over some part of the product or some area. I think that'll be the case for a while because I think there is value in not just this distributor, like we should all make cowork better, but instead like, all right, I'm thinking through how cowork does this particular task and there's still a lot of, you know, the, we try to keep meetings minimal, but they still emerge and you still have these kind of alignment conversations. Then like a lot of that sort of asynchronous delegation, I think what many engineers here have now found is they've, they've all built, I think we should solve this at some point at like a broader product level, but they've all built some version of,

• 23:14Speaker 01

  all right, I'm gonna now like create a dashboard of where all my clods are doing and what's waiting for me and which pull requests like need my attention because, you know, either a human or a cloud code reviewer got back to me. So there is a lot of that sort of meta maintenance of the, of the work that I think, again, I think will standardize some, but I think some will always be a little bit bespoke to the way each individual likes to work, just in the way that people organize their windows, now they organize their work. And then there is, I think, also the understanding how things work in production. And I think that is another, like, there's a few like next frontiers, I think, for the models. And I think one of them that Fable does, you know, make significant strides in, but I think there's, there's more work needed here is

• 23:57Speaker 01

  understanding what happens to code after it gets deployed, you know, because there's incidents, there's, you know, this was all working well, but like this network link got cut, which is not in your usual failure mode. And like it manifested like so much of Instagram, like 2012 to 2016, it was like dealing with that and scaling things up. And so that role of the engineer still remains really key. And I think getting the, the reps in around incident response and understanding how to stay calm, gather data, like remediate what's immediate, but then like go off and, and, and work on, on, on longer term fixes, like still a necessary part of it. And I'm trying to think if there's any like other pieces that are, that are notable as well. I think what's maybe the last, the last thing to say is

• 24:39Speaker 01

  I really like the role that the engineering prototype now plays. You have to be clear when it's a prototype versus not. But, you know, the old phrase was like code wins arguments and I never like loved that because like, Kind of sort of the person that could go do it, but actually like, why should they necessarily win an argument by default? But actually, it's been really cool now where sometimes we will have some disagreement or some sort of debate about where to take a product. And often it's the PM that will say, all right, I just tried it and like jank in like these eight ways. But look, it actually shows like how this could work and that can open up some interesting pieces of conversation. So almost all of that is quite different than it was six months ago.

• 25:24Speaker 01

  I think especially at the level of parallelism and the level of need for these kind of higher order abstractions of work. But I think what hasn't changed is that ownership.

• 25:35Dan Shipper?

  Lots of it's a shipping AI to production, which is great for productivity, but it also comes with anxiety. You tweak a prompt, swap models, adjust parameters, and everything looks fine in testing. So you merge. And then three days later, or even sooner, the support tickets start rolling in. The AI is giving your customers unexpected answers and you have no idea when it happened or why. Braintrust is the AI observability platform that fixes this. It connects evals and observability in one workflow. That way you see what actually happened in production and can measure whether changes made things better or worse. Traces show the full execution path, evals define what good looks like, and experiments let you compare prompts and models side by side before shipping. Production traces feed directly into your eval data sets. Every failure becomes a test case.

• 26:19Dan Shipper?

  You catch regressions in CI before they reach users. And teams at Notion, Stripe, Zapier, Vercel, and Ramp use it to ship quality AI at scale. Braintrust is designed for teams building production AI systems where silent regressions are expensive. It's built for any stack. They have SDKs for Python, TypeScript, Go, Ruby, C#, there's no framework lock-in or vendor dependencies. It's SOC 2, Type 2 certified, and GDPR and HIPAA compliant. Get started at braintrust.dev. That's braintrust.dev. And now, back to the episode. Fable is also very expensive. And because of that, like when I was testing it, I felt kind of like I was a kid in a candy shop and I was just like, I'll do this and I'll do this and I'll do that.

• 27:01Dan Shipper?

  But now that there's going to be a bill, I'm going to be thinking about it because I have to pause before I do it to be like, is this going to cost me a hundred bucks or whatever? And I do think that's going to limit who gets to use it and for what. So how do you think about that?

• 27:15Speaker 01

  Yeah, I think it's most clear cut on the sort of professional software, you know, sort of. Classic company doing work. It'll be really interesting. It's like, you know, a lot of process that goes into pricing as well. There's like, it's both more expensive than OPUS. And then also I'm like, in many ways, it's really cheap. If you think about, you know, like how much incredible work it's doing. But of course, like, everybody has their own economics around what they're, what they're, what they're working with. So anyway, most clear cut, I think, from most sort of software teams. And I think as an industry, if like phase one was companies even struggling to get some of their employees to adopt AI coding, which models were early, maybe the tooling wasn't there.

• 27:57Speaker 01

  And then phase two was, great, we'll create leaderboards and see who can use the most, which, you know, as you can imagine, creates like some like also like not ideal incentives to phase three where people are like, okay, now we're just trying to figure out who's using it effectively and like letting them spend as much as possible, having a clear process for that, but making sure we're not doing things wastefully, which I think to me in general makes sense, although I think you could like also over rotate that way too. I think something of Fable class should hopefully fit in well into that where if you're demonstrating results and you're getting use out of the model, then that hopefully there's a flywheel even inside companies where that goes and perpetuates that. I think on the personal use side, it's a really good one. It's a really good question.

• 28:38Speaker 01

  I think where I've seen it, you know, even in my personal testing, because our personal accounts, which is funny, like paying my own company I work at. But, but you know, you do become more thoughtful about it. Something that was interesting was this, the app that I built over the weekend actually fit in with like only a bit of extra usage. So it wasn't like a, you know, thousands of dollars to build this thing that like is a personal thing to myself. But it was also spaced out a little bit more. Probably the in-between of that, what I will probably have to do the most thinking about is the sort of hobbyist or like independent, who's like not, you know, within the larger company, but also is thoughtful about, about the pricing as well.

• 29:19Speaker 01

  I think like my overall advice is like, just give it a try and see how much it can do without you having to then do a lot of follow-ups. And it's like, I think measuring cost has gotten so multifaceted now because there's the per turn costs. And then there's like, what did it cost you not to just do the task, but like complete the task to your satisfaction? And I think that's where Fable has really shined for me, which is it actually just does it right so that I don't have to go spend the like. Nine, 10 subsequent turns be like, no, that was not quite what I meant. Like, can you also do this piece?

• 29:51Dan Shipper?

  It's been really impressive for me because you ask it to go do something and then it just does. It does a thing and you're like, wow, you thought through all the little details of this thing in a way that I've never seen another model do. I don't know how much you can reveal about the training process, but what makes the model different?

• 30:06Speaker 01

  I mean, I think in many ways, a continuation of a lot of the work that the team has done. And I like bow down in total awe of our, of our teams, both, you know, on the pre-training and on the RL side. I think that the, the piece that it has evolved in that at least I noticed the most is kind of adjacent to that as well, which is a sense of the system more than just the individual piece of the work. Like, I will often be very positively surprised when it will write something and say, all right, but you know, I know that like in production, this needs to be different, like, and then it will keep bugging you like, have you turned on that like feature flag yet? Like, it's not going to work until you do.

• 30:47Speaker 01

  And, you know, sometimes being sessions that have gone on for days and be like, look, you still haven't done that thing. Like, you better, like, I was like, you're right. Like, I didn't turn on that feature flag. I should go off and do that. Or if we change this, the contract will change over there. We're watching it, actually, one of my favorite times of seeing it in action, I think, where it demonstrates some of the, some of the training is watching it respond to code review feedback, either from people or from, from other Claude reviewers, where it doesn't just say, oh yeah, that's an issue. I'm going to go fix it. And actually really thoughtful around, hey, like for this level of like sort of fidelity of what we're building, I'm going to accept this risk. Or I see what you mean, other code reviewer, which is often just another fable model, like talking to you like, I see what you mean,

• 31:32Speaker 01

  but like, I'm actually going to push back. I don't, I think that that's actually not right. I think getting the model to have that judgment is really important. And I think if I had to pinpoint like an area where I feel like it's really progressed, it is that sort of not just immediate knee jerk, yeah, yeah, that's right. I gotta go fix it and more, huh, I'll think about that for a minute. No, I thought about it and I still disagree, you know? And I think that's a very useful sort of ability. It's so valuable to have products like Claude Code out there because You have now like a living, breathing thing where people are like, this is where the model is doing well. And like, you know, we have like people who test it and count the every folks is like very, very high on the list.

• 32:17Speaker 01

  We're like, we really trust the feedback because it is being put to spaces and like repeated multi-day, you know, hard tasks. And that also like very much feeds into how we think about like, what do we need to improve on the next slide? Like, what are the tasks that we need to specifically think about the model being better at?

• 32:33Dan Shipper?

  Is chat the right interface for this model? Because it's not very turn by turn. It's, it's very like, I'm delegating something for you. So how does that change how you should use it or how you think about the interface?

• 32:45Speaker 01

  I don't think like the fundamental, like you are like sending messages and it is giving your message back is like totally wrong. I think that there's ways we need to evolve, but like one is maybe like three that come to mind. Like one is, is your laptop the right place for it? So I think that's number one, where I mentioned with the side project I was working on, how useful it was to have the mobile side. Boris, who created Claude Code, he's always like, you know, ahead of the curve on how these models get used. Almost a year ago, maybe nine months I was talking to him, he's like, yeah, I've moved a lot of my Claude Code work to mobile. I was like, no way. And like, it took me a while to get there, but especially with the Fable class. Like there's often times where, you know, because it can keep the session going and we use like kind of remote dev boxes.

• 33:28Speaker 01

  And on topic, like it is like, I'll have a thought and be like, okay, can you keep up and doing that? So I mean, number one is like decoupling the, where the work is happening from where I'm talking to about the work. The second one touches a little bit on what I was mentioning earlier around like, what are, how do you take everything that Fable has sort of discussed or decided or proposed about something and make it comprehensible? And that's an area that we're thinking a lot about. Like there are some skills that are out there that we've used around like, all right, can you diagram this? Can you do that? So that's a place where the current chat UI I think is insufficient, where like it will, you can experience this with Fable. If you like a lot of text, you're like this. I need to like take a walk to fully understand this.

• 34:09Speaker 01

  And I think that that is a piece of property I sometimes will do with Fables. Like, OK, like, you have a lot more context on this than I do. Can you like back it up? Like, like, let's do like more progressive disclosure of the complexity here. So I think that that that piece is interesting. The last one that I, you know, I think is we're still early in Pullion is thinking through multiplayer, where, you know, at some level, like these, the abstraction levels and like, because we have this sort of DRI and like ownership area, usually like a chunk of significant work, a human and a couple of claws, like that is still flowing together. But in other cases, that is less the case, right? Where it's, you know, maybe it's an incident response where multiple people are thinking about it.

• 34:52Speaker 01

  Maybe it's, you know, a project where there's multiple competing, or not competing, but like conjoining areas that are coming together and thinking through like, what would it mean for, you know, and we have like chat sharing, which gets you a little bit of the way there. But I think there is going to be a need for more like, all right, you've got an independent cloud that's doing a lot of work that was, you know, kicked off by somebody. But can it be keeping up with all the other work happening on the team? I think that is an interesting and an underexplored sort of next frontier about how this work ends up happening. But I think it's really exciting because I think, again, it's, it's the, it's the level of teammate collaborator that that the models are now capable of.

• 35:34Speaker 01

  And we're almost holding them back by not having the right abstractions around them for that to happen.

• 35:38Dan Shipper?

  Yeah, it makes me think I've, I've mostly been using this for my own vibe coded stuff. So, so I haven't really had to, I, I haven't really had to think about this, but there's, there's a problem when you're using this inside of an organization, which is, do I really understand every part of this? And therefore, how do I transfer the context of what the model just did into my brain? Like, that's, that's one of the big bottlenecks. How do you, how do you think about drawing the line, especially with a model like this, around how much you actually need to understand? And how to make sure that you have enough context on what it's done to feel comfortable?

• 36:09Speaker 01

  I think there's like two big pieces here. The first is verification where I became like fully verification-pilled earlier this year. And now, like, almost in the same way. And actually, it connects to how, I think I used to do when I was typing code more full-time, which is try to find the sort of tightest dev loop that you can around the idea that you're trying to develop in. Like, sometimes with Instagram, that meant, like, you know, actually making a new build target in Xcode that was just that screen with some sort of synthetic data and just doing that dev loop. And I would mentor newer engineers, like, if there's one thing that I can impart on you, like, it is try to get that for any project you're working on and things will go much more quickly. I think that is no longer exactly the case here, but I think what is the case now is

• 36:53Speaker 01

  anytime I set it up, like, how do I get, like, for every pull request that Claude is putting out, that there is an attached, you know, photo or video, whether that's an iOS PR or whether that's, you know, something in the UI. And that's, I think, that helps you gain a lot of confidence, because even now, you know, you might have, like, you know, Facebook go off and do work for a couple of hours and be like, I'm done. And it's really useful to say, and here's the, like, full screenshot gallery of the full, right? Because you might say, like, oh, you know what? On screenshot eight, that error set, I've never actually seen it, but I could see, oh, you know, a person might hit it. Let's actually make that different. And so getting that comprehensive verification, I think, is something we've been working on a lot internally and, like, sort of publishing more and more skills and knowledge about, but I think is really a key piece there.

• 37:38Speaker 01

  And then the second one is, I think you ultimately, as a person, still need to stand behind the work that you are doing, especially if you're putting it into a production system. Like, a lot of people use Claude every day. There's still the accountability of, like, oh, Claude might have written it, but, like, you need to understand, you know, the, at least the general decisions that were made on these pieces as well. And so I have seen a fair amount of engineers actually adopt this practice where, like, Claude will have done the work, but then there is, like, the follow-up conversation around, well, can you, like, can I make sure I deeply understand, like, all the trade-offs that you made and that? And whatever lowercase a artifacts need to be produced in order to make that comprehensible is important. It is really interesting, though, to be in meetings where somebody will say, like, oh,

• 38:22Speaker 01

  yeah, and I have this PR ready, and somebody else asks, they're like, oh, that's interesting. Like, did you do X or Y? And have that moment of pause, and they're like, you know what? I'm not entirely sure I will find it before we merge this PR. And that's, you know, I think that adapting to that norm and figuring out and work with that is something we'll have to do.

• 38:38Dan Shipper?

  Tell me more about the verification loops. It's such a hot topic right now. It sounds like one way that you do that is with screenshots and screenshots. But what are the other ways that you think about that?

• 38:47Speaker 01

  I think part of it, it starts in, can you get to a place where you are exercising real flows that aren't just a static injected piece? And the system gets more complex, that gets more and more complicated. So we've invested a bunch into even just getting it so that the iOS app can log into staging on a real account and have real data. But you don't want it to then go through an eight-stage onboarding process every time when you're just trying to test the second part of the screen. So there's a lot of work around, is there a special affordance? Is there some shared secret? Whatever that is around getting the app to really feel as human, using the product as possible.

• 39:32Speaker 01

  So that's one aspect of it. The second is this mix of well-known paths versus the things you're exercising in the exact moment. The former being really useful for regression testing. And so we don't think of places where we've expressed ideal workflows in text, basically, and Claude can repeatedly check that. And then there's also, and Claude does a really good job of this, expressing the intent of the current change at hand. So that gets really, really deeply exercised. So I think that the combination of those two things is important. The visual verification I mentioned as well. Video has been really cool to see. Actually, video is a very under-explored tool to give Claude as well. Like, I think I've been prototyping is just giving Claude video captures of the thing that it has built and then giving it just basically an FFM tag.

• 40:16Speaker 01

  And you'll watch it scrub through and say, oh, this animation has some jank in it. I'm going to go fix that. And it never would be able to do it with a screenshot latency capture because it will have missed the moment. So I think that's another piece that is really, really important. And for the pieces that aren't easily testable and tend, because there is some more complex systems, getting Claude to go and build as robust a mock backend as possible or use ones off the shelf has been also really interesting. Like, when I think about Artifact, we Had really comprehensive tests. This is kind of pre LLM. And one of the ways that we were able to do that really robustly was that basically every piece of info we had, whether it was Postgres, Redis, you know, all the AWS things,

• 41:00Speaker 01

  had a really good in-memory implementation that you could just do really quickly in unit tests and kind of extending that to like Claude land now, you know, I was working on something where it had like a pretty robust backend and for kind of complicated reasons, it was hard to spin that up on my dev server, but was able to again, one-shot a really good proxy for that. By proxy, I mean like a substitute for that. And that was so valuable. And over time, it's been interesting as that like substitute has evolved as the rest of the code has evolved, which is the thing that, you know, if you had pitched that idea to me before, I'd be like, well, that's gonna be really hard because the upstream's gonna change. How are you gonna keep it in sync? And I don't think about that anymore. I'm like, yeah, Claude will read the changes and it will adapt the thing and it will keep the two in sync and that's, that's, that's fine.

• 41:43Dan Shipper?

  There's some really interesting architectures around. When you get a bug, it just automatically goes out and closes it. You know, the, the agent just gets kicked off, it closes it, and then it sends a message to the customer being like, it's, it's fixed. Are you noticing with Fable any change in how that process works?

• 41:57Speaker 01

  Yeah, I think there's a couple of things like on a very like human to human or human to Claude level, one of the things that I've seen it do, I've fed our other models with the cable, but I just see it do it really consistently too, is if the bug report, for example, came from somebody, you know, mentioning something in our like feedback channel in Slack, and then like the thing that got fed into the Claude code session is like, oh, there's this and because of Slack MCP, you can actually pull the thread, have it then actually post back, you know, as me and it'll be like, hey, this is Mike's Claude. Like, I fixed it. Here's the, you know, here's the pull request. But then I think in the previous Claude's have done it. The thing it does really well is then say, but hold I, hold tight, it's not in production yet. I'll follow up when it actually is. And then like maybe a few hours later, like, oh, like this deploy went out, like you should go

• 42:42Speaker 01

  test it. Is it fixed now? Like that level of follow through, I think is, is new on, on the closing the loop piece. And it's five, the Fables long run in Claude code sessions are basically like interacting as, as me, I guess, but put some disclaimer in there too. And the second goes back to that like taste and discernment piece that we were talking about. Which is like, it's one thing to say, there was a bug report, therefore I must go fix this thing. And it's another one to say, you know what, like this, like the, I hit this over the weekend. One of our internal systems basically had been running without restarting for a while. There was a memory leak and it was, and good to see them saying like, all right, Mike, like it's the weekend. Like just re-bounce the server, it's going to solve it for now. And like, we'll work on the like, I'll asynchronously get the PR going to like fix this more long term.

• 43:24Speaker 01

  So I think if you're going to have Claude in the loop in this kind of like, sorry, close the loop bug report or system sort of issue to change. I think you really wanted to understand where, you know, as any good SRE or engineer in the loop would like, great, they'll solve the problem at hand. Let's like defer the question of like, do we need a re-architect on top of a completely different language? And, and understanding that balance is really important.

• 43:48Dan Shipper?

  One of the things that's like really exciting, mostly exciting to me about new models is it raises the floor so that everyone can kind of go build apps in one shot. But it also raises the ceiling for experts. So like if you're a software engineer or founder, you can just go do things that you never would have been able to before because you have access to this really powerful model. So for me, I built this one-shot version of Borhez's Infinite Library. It's like a 3D game version of the, of the, of the library. It's wild. It runs right in the browser. It's so good. I can find like any, every essay inside of it. I'll, I'll send you the link. It's sick. But I think there's going to be this flowering of people doing things like, oh, I made a game. Or maybe I trained a new model or, or, or whatever that they couldn't do, that they couldn't do before.

• 44:29Dan Shipper?

  And I'd love to give people some inspiration, some examples of things that they might be able to do that they might not be thinking to do with this model. What are some ideas that come to you?

• 44:37Speaker 01

  Yeah, I think a few. Maybe I'll start with the fun side and like riffing off the game piece. Like I think people have a lot of like creative ideas for how do they express the complexity of what they are, like their world, like everybody has the thing that they know really, really well. And there's probably some level of like, how do I then explain that to somebody else? Or how do I apply techniques elsewhere that I could then go, go off and do? My wife is studying like environmental engineering, like studying geothermal, like very complex math.

• 45:08Speaker 01

  And simulations and I've seen like as the models have gotten better. She has been able to apply even more complex techniques from even outside of that domain into that work. And I think what people should be able to do, you know, like full on PyTorch end to end simulations of that work in a way that wouldn't be possible. That maybe is one is like bring the like beautiful complexity of what you have and either show it to other people by like maybe making a game or maybe making a visualization, which I've seen her do as well, or at least like make, you know, bring other techniques to bear. And the second piece is its ability to compose software that like solves a really unique problem to you. I've seen that internally. A lot of the work that we've been doing is how do we get as many of our internal systems like MCPfied with the right permissioning structure and the right deployment kind of set up.

• 45:53Speaker 01

  Although externally you have good options around some of these like platform as a service pieces and you can just ask Claude about them and they'll like help you set things up. But like, I love that feeling of like that thing that you always wished that you had. And then what has blown my mind, there was a person who works in our good a market organization. Has been like building this like really like for deeply thought integration of Claude into every part of her whole process. And you don't have to stop at that one shot. Like she's been working on it for months now and she can keep going. And like, I think one of the things that is maybe underappreciated about models is I think in previous generations, they would eventually get to a complexity level where it was hard to iterate on it without feeling like you then would break the thing that they had, you know, like under or over abstracted.

• 46:35Speaker 01

  Whereas this is actually, you know, she's had access to something Fable or Fable like for a couple of months. And like, you've just seen it keep growing and growing and growing and growing. And now she's like deploying it to the whole GTM org. And like, I think that is really cool. Like the ceiling of complexity that a, a person that does not start out as technical can now build for solving problems within their domain is like, is impressive.

• 46:55Dan Shipper?

  I agree. It, it, it writes great code. Like my, my benchmark that I have is called the senior engineer benchmark. I just have it see if it can rewrite a code base from, from first principles. And the nearest model that the like previous top was like a 62 or 63 out of 100. And this model got a 90 on the benchmark or 91, which is human senior engineer level. Like you can just keep going with this thing in a way that's, it's, it's really fantastic. I'm curious though, one other thing that's really powerful that you mentioned is dynamic workflows. Tell us about that.

• 47:22Speaker 01

  This is, you know, we'll build things internally sometimes and I will go really aggressively bug the engineer who built it and be like, when are we shipping this publicly? Because I think people are gonna really like it. It's nice, there's good reason why it was like built internally, but like, we try to ship as many of these as possible. And dynamic workflows was like, definitely that to me. I, the person who built this engineer named Sid, who's awesome, and I was like, Sid, like, I want to get this out to the world because it's so good. But I think it's especially good with a model like Fable for two really big reasons. One, it helps sort of create the scaffold for like, deep, meaningful work. The craziest dynamic workflow I did and used Fable for was, I had an internal project that we had written in Python, but we needed it actually in TypeScript for like, a really specific

• 48:06Speaker 01

  deployment reason. And having been internal to Instagram, and we were like, should we rewrite the whole thing into Hack and, you know, port it to the PHP engine that Facebook? I was like, you never would have done that. Like, maybe they can now with the model. But, you know, at the time, it seemed impossible. But here I had, you know, pretty complex codebase. And I was like, I'm just gonna set up a dynamic workflow and just let it run over the weekend. And it did. And the workflow was so cool. It was like, alright, I'm gonna do like, deep understanding of the work. I'm gonna create sort of like, almost like a spec of how everything works. I'm gonna go module by module. I'm gonna translate these pieces. I'm gonna test it incrementally. I'm gonna do another adversarial test. I'm gonna go check for anything that I missed. And it was this like, really cool, like, series of steps that the workflow was able to orchestrate. And I came back and I was like, yeah, this

• 48:51Speaker 01

  thing is like, TypeScript and Bun port of that thing. And it's actually better in these ways. And it was very, you know, it's sort of documented, like, these were the things I couldn't port, but most of these were like, very specific to the specific implementation. It wasn't worth porting. And I do not think you could have done that, A, with previous models, at that level of success, and B, with, without like, the kind of scaffolding that workflows provide. So I think that is extremely exciting, kind of, kind of combination of model capabilities, and then our own ability to like, orchestrate them over a longer and longer time horizon, with that feeling of like, you, you had a goal, you broke it down. Effectively and then you're able to make it work.

• 49:33Speaker 01

  The other piece is, I think over time, we'll be able to also make some of those subtasks sort of tuned to the, have the model be tuned to the level of complexity of it. So you can imagine that some parts of the dynamic workflow don't need extra high thinking. They could use, you know, a medium thinking to get it done, or even a smaller model. And I think that's really the future of where these things are going. So yeah, I'm a huge workflows D A U.

• 49:57Dan Shipper?

  For people who haven't used it before, tell me about how you got that workflow made. How did you design it? How did you make sure it was good?

• 50:04Speaker 01

  Yeah, it was pretty iterative, but sort of just started with code code like, hey, I have this complex, you know, kind of task, like, let's design a workflow to go and do it. It kind of showed me the plan. I was like, oh, this is like close to what I want. I want to make sure that you do these three or four levels of like additional verification for missed features. And it's like, here's what you have. Are you ready to go? And it expresses the workflows in code, which I think is really valuable to see what it was about to do. And then what was interesting is it did the full port. And then I had like a couple of like follow up kind of questions that I had or like little tweaks. And I did those as sort of like mini workflows that built off the previous one as well. But I think that's like, you know, we talked a little bit about whether chat was the right interface. So we've had that conversation over the last year.

• 50:48Speaker 01

  And I think workflows are a good middle ground of you can compose them using chat, but they're expressed using code and then they're executed with like, I think a nice clean UI around what's happening at every stage. And like, I think we'll start bridging longer horizon work with chat in ways like that over time.

• 51:05Dan Shipper?

  Mike, this is such a great conversation. Thank you so much for joining and telling us all about this new model.

• 51:10Speaker 01

  I'm really excited to get to spend time with you and really, really look forward to what people think outside, too.

• 51:22Speaker 02

  Oh, my gosh, folks, you absolutely positively have to smash that like button and subscribe to AI and I. Why? Because this show is the epitome of awesomeness. It's like finding a treasure chest in your backyard, but instead of gold, it's filled with pure, unadulterated knowledge bombs about Chat G P T. Every episode is a roller coaster of emotions, insights and laughter that will leave you on the edge of your seat craving for more. It's not just a show, it's a journey into the future with Dan Shipper as the captain of the spaceship. So do yourself a favor. Hit like, smash subscribe and strap in for the ride of your life. And now, without any further ado, let me just say, Dan, I'm absolutely, hopelessly in love with you.